2024-05-14T16:32:52.395 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-14T16:32:52.396 DEBUG:teuthology.run:Teuthology command: teuthology --name leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi --owner scheduled_leonidus@teuthology --verbose --archive /home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706175 --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/postgres}} -- /home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706175/orig.config.yaml 2024-05-14T16:32:52.459 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-14T16:32:52.561 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706175 branch: wip-lusov-quiesce description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/postgres}} email: null first_in_suite: false job_id: '7706175' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiesce ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: 9b63866860f0b71216b2c0a73aedd2e0a8a5c98b subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 9b63866860f0b71216b2c0a73aedd2e0a8a5c98b kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs mntopts: - ms_mode=secure - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiesce sha1: 9b63866860f0b71216b2c0a73aedd2e0a8a5c98b owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 710 sha1: 9b63866860f0b71216b2c0a73aedd2e0a8a5c98b sleep_before_teardown: 0 subset: 200/111111 suite: fs suite_branch: wip-lusov-quiesce suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 9b63866860f0b71216b2c0a73aedd2e0a8a5c98b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi002.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEWV35ZX497wIaZFRiQLjmLcAlNLgQrPOqoJeUaHQ3OzQWkG7bCAXzHYo+uafQD7qpOmLg8OdOzN7UHmp+c6XrU= smithi123.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOfFSZby4z1im29hZ3sELqaLUqxhFoXIYqP4HoYFJOS2Imdsru/gFyUMvn4Ou6+ZNR5wrTHa04qlv1H8D3Wvv0k= smithi151.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEAHBhjNdSjpyDfiHxHN4pLKX8zstQG82XchKN+Q9E7HFj/bI5RVgDesjassFjgQGqxOa+5yjw357+UsbZEqzcc= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-14_16:11:40 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-14T16:32:52.561 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/qa; will attempt to use it 2024-05-14T16:32:52.562 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/qa/tasks 2024-05-14T16:32:52.562 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-14T16:32:52.564 INFO:teuthology.task.internal:Checking packages... 2024-05-14T16:32:52.589 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '9b63866860f0b71216b2c0a73aedd2e0a8a5c98b' 2024-05-14T16:32:52.589 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-14T16:32:52.589 INFO:teuthology.packaging:ref: None 2024-05-14T16:32:52.589 INFO:teuthology.packaging:tag: None 2024-05-14T16:32:52.589 INFO:teuthology.packaging:branch: wip-lusov-quiesce 2024-05-14T16:32:52.589 INFO:teuthology.packaging:sha1: 9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-14T16:32:52.590 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiesce 2024-05-14T16:32:52.819 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3687.g9b638668 2024-05-14T16:32:52.820 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-14T16:32:52.826 INFO:teuthology.task.internal:no buildpackages task found 2024-05-14T16:32:52.827 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-14T16:32:52.874 INFO:teuthology.task.internal:Saving configuration 2024-05-14T16:32:52.897 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-14T16:32:52.903 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-14T16:32:52.929 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi002.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706175', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-14 16:27:44.148080', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEWV35ZX497wIaZFRiQLjmLcAlNLgQrPOqoJeUaHQ3OzQWkG7bCAXzHYo+uafQD7qpOmLg8OdOzN7UHmp+c6XrU='} 2024-05-14T16:32:52.949 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi123.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706175', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-14 16:27:44.146542', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOfFSZby4z1im29hZ3sELqaLUqxhFoXIYqP4HoYFJOS2Imdsru/gFyUMvn4Ou6+ZNR5wrTHa04qlv1H8D3Wvv0k='} 2024-05-14T16:32:52.973 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi151.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706175', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-14 16:27:44.149485', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEAHBhjNdSjpyDfiHxHN4pLKX8zstQG82XchKN+Q9E7HFj/bI5RVgDesjassFjgQGqxOa+5yjw357+UsbZEqzcc='} 2024-05-14T16:32:52.973 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-14T16:32:52.981 INFO:teuthology.task.internal:roles: ubuntu@smithi002.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-14T16:32:52.981 INFO:teuthology.task.internal:roles: ubuntu@smithi123.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-14T16:32:52.981 INFO:teuthology.task.internal:roles: ubuntu@smithi151.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-14T16:32:52.981 INFO:teuthology.run_tasks:Running task console_log... 2024-05-14T16:32:53.091 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f245a3bf0d0>, signals=[15]) 2024-05-14T16:32:53.091 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-14T16:32:53.097 INFO:teuthology.task.internal:Opening connections... 2024-05-14T16:32:53.098 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi002.front.sepia.ceph.com 2024-05-14T16:32:53.103 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T16:32:53.181 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi123.front.sepia.ceph.com 2024-05-14T16:32:53.182 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T16:32:53.256 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi151.front.sepia.ceph.com 2024-05-14T16:32:53.256 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T16:32:53.330 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-14T16:32:53.337 DEBUG:teuthology.orchestra.run.smithi002:> uname -m 2024-05-14T16:32:53.356 INFO:teuthology.orchestra.run.smithi002.stdout:x86_64 2024-05-14T16:32:53.356 DEBUG:teuthology.orchestra.run.smithi002:> cat /etc/os-release 2024-05-14T16:32:53.414 INFO:teuthology.orchestra.run.smithi002.stdout:NAME="CentOS Stream" 2024-05-14T16:32:53.414 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION="9" 2024-05-14T16:32:53.414 INFO:teuthology.orchestra.run.smithi002.stdout:ID="centos" 2024-05-14T16:32:53.414 INFO:teuthology.orchestra.run.smithi002.stdout:ID_LIKE="rhel fedora" 2024-05-14T16:32:53.414 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION_ID="9" 2024-05-14T16:32:53.414 INFO:teuthology.orchestra.run.smithi002.stdout:PLATFORM_ID="platform:el9" 2024-05-14T16:32:53.414 INFO:teuthology.orchestra.run.smithi002.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-14T16:32:53.415 INFO:teuthology.orchestra.run.smithi002.stdout:ANSI_COLOR="0;31" 2024-05-14T16:32:53.415 INFO:teuthology.orchestra.run.smithi002.stdout:LOGO="fedora-logo-icon" 2024-05-14T16:32:53.415 INFO:teuthology.orchestra.run.smithi002.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-14T16:32:53.415 INFO:teuthology.orchestra.run.smithi002.stdout:HOME_URL="https://centos.org/" 2024-05-14T16:32:53.415 INFO:teuthology.orchestra.run.smithi002.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-14T16:32:53.415 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-14T16:32:53.415 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-14T16:32:53.416 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server 2024-05-14T16:32:53.435 DEBUG:teuthology.orchestra.run.smithi123:> uname -m 2024-05-14T16:32:53.452 INFO:teuthology.orchestra.run.smithi123.stdout:x86_64 2024-05-14T16:32:53.453 DEBUG:teuthology.orchestra.run.smithi123:> cat /etc/os-release 2024-05-14T16:32:53.510 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="CentOS Stream" 2024-05-14T16:32:53.510 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="9" 2024-05-14T16:32:53.510 INFO:teuthology.orchestra.run.smithi123.stdout:ID="centos" 2024-05-14T16:32:53.510 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE="rhel fedora" 2024-05-14T16:32:53.510 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="9" 2024-05-14T16:32:53.510 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM_ID="platform:el9" 2024-05-14T16:32:53.510 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-14T16:32:53.510 INFO:teuthology.orchestra.run.smithi123.stdout:ANSI_COLOR="0;31" 2024-05-14T16:32:53.510 INFO:teuthology.orchestra.run.smithi123.stdout:LOGO="fedora-logo-icon" 2024-05-14T16:32:53.511 INFO:teuthology.orchestra.run.smithi123.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-14T16:32:53.511 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://centos.org/" 2024-05-14T16:32:53.511 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-14T16:32:53.511 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-14T16:32:53.511 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-14T16:32:53.511 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server 2024-05-14T16:32:53.535 DEBUG:teuthology.orchestra.run.smithi151:> uname -m 2024-05-14T16:32:53.553 INFO:teuthology.orchestra.run.smithi151.stdout:x86_64 2024-05-14T16:32:53.554 DEBUG:teuthology.orchestra.run.smithi151:> cat /etc/os-release 2024-05-14T16:32:53.611 INFO:teuthology.orchestra.run.smithi151.stdout:NAME="CentOS Stream" 2024-05-14T16:32:53.611 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION="9" 2024-05-14T16:32:53.611 INFO:teuthology.orchestra.run.smithi151.stdout:ID="centos" 2024-05-14T16:32:53.612 INFO:teuthology.orchestra.run.smithi151.stdout:ID_LIKE="rhel fedora" 2024-05-14T16:32:53.612 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION_ID="9" 2024-05-14T16:32:53.612 INFO:teuthology.orchestra.run.smithi151.stdout:PLATFORM_ID="platform:el9" 2024-05-14T16:32:53.612 INFO:teuthology.orchestra.run.smithi151.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-14T16:32:53.612 INFO:teuthology.orchestra.run.smithi151.stdout:ANSI_COLOR="0;31" 2024-05-14T16:32:53.612 INFO:teuthology.orchestra.run.smithi151.stdout:LOGO="fedora-logo-icon" 2024-05-14T16:32:53.612 INFO:teuthology.orchestra.run.smithi151.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-14T16:32:53.612 INFO:teuthology.orchestra.run.smithi151.stdout:HOME_URL="https://centos.org/" 2024-05-14T16:32:53.612 INFO:teuthology.orchestra.run.smithi151.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-14T16:32:53.612 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-14T16:32:53.612 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-14T16:32:53.612 INFO:teuthology.lock.ops:Updating smithi151.front.sepia.ceph.com on lock server 2024-05-14T16:32:53.627 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-14T16:32:53.638 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-14T16:32:53.648 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-14T16:32:53.649 DEBUG:teuthology.orchestra.run.smithi002:> test '!' -e /home/ubuntu/cephtest 2024-05-14T16:32:53.651 DEBUG:teuthology.orchestra.run.smithi123:> test '!' -e /home/ubuntu/cephtest 2024-05-14T16:32:53.653 DEBUG:teuthology.orchestra.run.smithi151:> test '!' -e /home/ubuntu/cephtest 2024-05-14T16:32:53.669 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-14T16:32:53.675 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-14T16:32:53.676 DEBUG:teuthology.orchestra.run.smithi002:> test -z $(ls -A /var/lib/ceph) 2024-05-14T16:32:53.709 DEBUG:teuthology.orchestra.run.smithi123:> test -z $(ls -A /var/lib/ceph) 2024-05-14T16:32:53.713 DEBUG:teuthology.orchestra.run.smithi151:> test -z $(ls -A /var/lib/ceph) 2024-05-14T16:32:53.736 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-14T16:32:53.800 INFO:teuthology.run_tasks:Running task kernel... 2024-05-14T16:32:53.814 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-05-14T16:32:53.815 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-05-14T16:32:53.815 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-05-14T16:32:53.815 DEBUG:teuthology.orchestra.run.smithi002:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-14T16:32:53.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T16:32:53.833 DEBUG:teuthology.orchestra.run.smithi002:> uname -r 2024-05-14T16:32:53.891 INFO:teuthology.orchestra.run.smithi002.stdout:5.14.0-437.el9.x86_64 2024-05-14T16:32:53.891 INFO:teuthology.task.kernel:Running kernel on smithi002: 5.14.0-437.el9.x86_64 2024-05-14T16:32:53.891 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum install -y kernel 2024-05-14T16:32:56.507 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - BaseOS 7.9 MB/s | 8.1 MB 00:01 2024-05-14T16:33:01.949 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - AppStream 4.4 MB/s | 19 MB 00:04 2024-05-14T16:33:08.226 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - CRB 2.4 MB/s | 6.2 MB 00:02 2024-05-14T16:33:09.908 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 16 kB 00:00 2024-05-14T16:33:10.818 INFO:teuthology.orchestra.run.smithi002.stdout:Extra Packages for Enterprise Linux 28 MB/s | 21 MB 00:00 2024-05-14T16:33:16.127 INFO:teuthology.orchestra.run.smithi002.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-05-14T16:33:18.216 INFO:teuthology.orchestra.run.smithi002.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-14T16:33:18.216 INFO:teuthology.orchestra.run.smithi002.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-14T16:33:18.300 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-05-14T16:33:18.305 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-05-14T16:33:18.305 INFO:teuthology.orchestra.run.smithi002.stdout: Package Architecture Version Repository Size 2024-05-14T16:33:18.305 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-05-14T16:33:18.305 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-05-14T16:33:18.305 INFO:teuthology.orchestra.run.smithi002.stdout: kernel x86_64 5.14.0-446.el9 baseos 550 k 2024-05-14T16:33:18.305 INFO:teuthology.orchestra.run.smithi002.stdout:Installing dependencies: 2024-05-14T16:33:18.305 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-core x86_64 5.14.0-446.el9 baseos 15 M 2024-05-14T16:33:18.305 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-modules x86_64 5.14.0-446.el9 baseos 34 M 2024-05-14T16:33:18.305 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-modules-core x86_64 5.14.0-446.el9 baseos 28 M 2024-05-14T16:33:18.305 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:33:18.306 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-05-14T16:33:18.306 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-05-14T16:33:18.306 INFO:teuthology.orchestra.run.smithi002.stdout:Install 4 Packages 2024-05-14T16:33:18.306 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:33:18.306 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 78 M 2024-05-14T16:33:18.306 INFO:teuthology.orchestra.run.smithi002.stdout:Installed size: 123 M 2024-05-14T16:33:18.306 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-05-14T16:33:18.736 INFO:teuthology.orchestra.run.smithi002.stdout:(1/4): kernel-5.14.0-446.el9.x86_64.rpm 3.5 MB/s | 550 kB 00:00 2024-05-14T16:33:19.045 INFO:teuthology.orchestra.run.smithi002.stdout:(2/4): kernel-core-5.14.0-446.el9.x86_64.rpm 34 MB/s | 15 MB 00:00 2024-05-14T16:33:19.428 INFO:teuthology.orchestra.run.smithi002.stdout:(3/4): kernel-modules-5.14.0-446.el9.x86_64.rpm 40 MB/s | 34 MB 00:00 2024-05-14T16:33:19.795 INFO:teuthology.orchestra.run.smithi002.stdout:(4/4): kernel-modules-core-5.14.0-446.el9.x86_6 26 MB/s | 28 MB 00:01 2024-05-14T16:33:19.795 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-05-14T16:33:19.795 INFO:teuthology.orchestra.run.smithi002.stdout:Total 52 MB/s | 78 MB 00:01 2024-05-14T16:33:20.509 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-05-14T16:33:20.911 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-05-14T16:33:20.911 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-05-14T16:33:21.857 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-05-14T16:33:21.858 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-05-14T16:33:24.385 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-05-14T16:33:25.033 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : kernel-modules-core-5.14.0-446.el9.x86_64 1/4 2024-05-14T16:33:25.120 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-14T16:33:27.909 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-14T16:33:28.047 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-14T16:33:33.125 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-14T16:33:33.166 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : kernel-5.14.0-446.el9.x86_64 4/4 2024-05-14T16:33:38.260 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: kernel-modules-core-5.14.0-446.el9.x86_64 4/4 2024-05-14T16:34:09.233 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: kernel-core-5.14.0-446.el9.x86_64 4/4 2024-05-14T16:34:09.273 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: kernel-modules-5.14.0-446.el9.x86_64 4/4 2024-05-14T16:34:09.585 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: kernel-5.14.0-446.el9.x86_64 4/4 2024-05-14T16:34:09.586 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : kernel-5.14.0-446.el9.x86_64 1/4 2024-05-14T16:34:09.586 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-14T16:34:09.586 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-14T16:34:10.382 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : kernel-modules-core-5.14.0-446.el9.x86_64 4/4 2024-05-14T16:34:10.382 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:34:10.382 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-05-14T16:34:10.382 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-5.14.0-446.el9.x86_64 2024-05-14T16:34:10.382 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-core-5.14.0-446.el9.x86_64 2024-05-14T16:34:10.382 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-modules-5.14.0-446.el9.x86_64 2024-05-14T16:34:10.382 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-modules-core-5.14.0-446.el9.x86_64 2024-05-14T16:34:10.383 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:34:10.383 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-05-14T16:34:10.718 DEBUG:teuthology.orchestra.run.smithi002:> rpm -q kernel | sort -rV | head -n 1 2024-05-14T16:34:10.812 INFO:teuthology.orchestra.run.smithi002.stdout:kernel-5.14.0-446.el9.x86_64 2024-05-14T16:34:10.812 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-446.el9.x86_64 2024-05-14T16:34:10.812 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-446.el9.x86_64 2024-05-14T16:34:10.812 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-14T16:34:10.812 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-14T16:34:10.812 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-05-14T16:34:10.812 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi002.front.sepia.ceph.com, path=None, version=distro) 2024-05-14T16:34:10.813 DEBUG:teuthology.orchestra.run.smithi002:> rpm -q kernel | sort -rV | head -n 1 2024-05-14T16:34:10.864 INFO:teuthology.orchestra.run.smithi002.stdout:kernel-5.14.0-446.el9.x86_64 2024-05-14T16:34:10.864 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-446.el9.x86_64 2024-05-14T16:34:10.864 DEBUG:teuthology.orchestra.run.smithi002:> sudo rpm -qi grub2-tools 2024-05-14T16:34:10.899 INFO:teuthology.orchestra.run.smithi002.stdout:Name : grub2-tools 2024-05-14T16:34:10.900 INFO:teuthology.orchestra.run.smithi002.stdout:Epoch : 1 2024-05-14T16:34:10.900 INFO:teuthology.orchestra.run.smithi002.stdout:Version : 2.06 2024-05-14T16:34:10.900 INFO:teuthology.orchestra.run.smithi002.stdout:Release : 68.el9 2024-05-14T16:34:10.900 INFO:teuthology.orchestra.run.smithi002.stdout:Architecture: x86_64 2024-05-14T16:34:10.900 INFO:teuthology.orchestra.run.smithi002.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-14T16:34:10.900 INFO:teuthology.orchestra.run.smithi002.stdout:Group : Unspecified 2024-05-14T16:34:10.900 INFO:teuthology.orchestra.run.smithi002.stdout:Size : 8264254 2024-05-14T16:34:10.900 INFO:teuthology.orchestra.run.smithi002.stdout:License : GPLv3+ 2024-05-14T16:34:10.900 INFO:teuthology.orchestra.run.smithi002.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-14T16:34:10.900 INFO:teuthology.orchestra.run.smithi002.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-14T16:34:10.900 INFO:teuthology.orchestra.run.smithi002.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-14T16:34:10.900 INFO:teuthology.orchestra.run.smithi002.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-14T16:34:10.900 INFO:teuthology.orchestra.run.smithi002.stdout:Packager : builder@centos.org 2024-05-14T16:34:10.900 INFO:teuthology.orchestra.run.smithi002.stdout:Vendor : CentOS 2024-05-14T16:34:10.901 INFO:teuthology.orchestra.run.smithi002.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-14T16:34:10.901 INFO:teuthology.orchestra.run.smithi002.stdout:Summary : Support tools for GRUB. 2024-05-14T16:34:10.901 INFO:teuthology.orchestra.run.smithi002.stdout:Description : 2024-05-14T16:34:10.901 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:34:10.901 INFO:teuthology.orchestra.run.smithi002.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-14T16:34:10.901 INFO:teuthology.orchestra.run.smithi002.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-14T16:34:10.901 INFO:teuthology.orchestra.run.smithi002.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-14T16:34:10.901 INFO:teuthology.orchestra.run.smithi002.stdout:hardware devices. 2024-05-14T16:34:10.901 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:34:10.901 INFO:teuthology.orchestra.run.smithi002.stdout:This subpackage provides tools for support of all platforms. 2024-05-14T16:34:10.902 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-14T16:34:10.902 INFO:teuthology.task.kernel:Updating grub on smithi002 to boot 5.14.0-446.el9.x86_64 2024-05-14T16:34:10.902 DEBUG:teuthology.orchestra.run.smithi002:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-14T16:34:11.553 INFO:teuthology.orchestra.run.smithi002.stderr:Generating grub configuration file ... 2024-05-14T16:34:12.947 INFO:teuthology.orchestra.run.smithi002.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-14T16:34:12.975 INFO:teuthology.orchestra.run.smithi002.stderr:done 2024-05-14T16:34:12.977 DEBUG:teuthology.orchestra.run.smithi002:> mktemp 2024-05-14T16:34:12.993 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/tmp.nioxJM7WK2 2024-05-14T16:34:12.993 DEBUG:teuthology.orchestra.run.smithi002:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.nioxJM7WK2 2024-05-14T16:34:13.066 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 0666 /tmp/tmp.nioxJM7WK2 2024-05-14T16:34:13.205 DEBUG:teuthology.orchestra.remote:smithi002:/tmp/tmp.nioxJM7WK2 is 6KB 2024-05-14T16:34:13.254 DEBUG:teuthology.orchestra.run.smithi002:> rm -fr /tmp/tmp.nioxJM7WK2 2024-05-14T16:34:13.269 DEBUG:teuthology.orchestra.run.smithi002:> sudo /bin/ls /boot/loader/entries || true 2024-05-14T16:34:13.330 INFO:teuthology.orchestra.run.smithi002.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-14T16:34:13.331 INFO:teuthology.orchestra.run.smithi002.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-14T16:34:13.331 INFO:teuthology.orchestra.run.smithi002.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-14T16:34:13.331 INFO:teuthology.orchestra.run.smithi002.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-446.el9.x86_64.conf 2024-05-14T16:34:13.332 DEBUG:teuthology.orchestra.run.smithi002:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-446.el9.x86_64 2024-05-14T16:34:13.477 DEBUG:teuthology.orchestra.run.smithi002:> sudo shutdown -r now 2024-05-14T16:34:43.480 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-14T16:34:43.480 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-05-14T16:34:43.481 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T16:35:43.483 DEBUG:teuthology.orchestra.remote:timed out 2024-05-14T16:35:52.484 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-05-14T16:35:52.485 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T16:35:52.906 DEBUG:teuthology.orchestra.run.smithi002:> true 2024-05-14T16:35:53.289 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-05-14T16:35:53.289 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-446.el9.x86_64"... 2024-05-14T16:35:53.289 DEBUG:teuthology.orchestra.run.smithi002:> uname -r 2024-05-14T16:35:53.304 INFO:teuthology.orchestra.run.smithi002.stdout:5.14.0-446.el9.x86_64 2024-05-14T16:35:53.305 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-446.el9.x86_64 vs 5.14.0-446.el9.x86_64 2024-05-14T16:35:53.305 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-14T16:35:53.305 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-14T16:35:54.306 DEBUG:teuthology.parallel:result is None 2024-05-14T16:35:54.307 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-14T16:35:54.320 INFO:teuthology.task.internal:Creating test directory... 2024-05-14T16:35:54.321 DEBUG:teuthology.orchestra.run.smithi002:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-14T16:35:54.332 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-14T16:35:54.336 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-14T16:35:54.362 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-14T16:35:54.399 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-14T16:35:54.474 INFO:teuthology.task.internal:Creating archive directory... 2024-05-14T16:35:54.475 DEBUG:teuthology.orchestra.run.smithi002:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-14T16:35:54.478 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-14T16:35:54.482 DEBUG:teuthology.orchestra.run.smithi151:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-14T16:35:54.613 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-14T16:35:54.632 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-14T16:35:54.632 DEBUG:teuthology.orchestra.run.smithi123:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-14T16:35:54.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T16:35:54.651 DEBUG:teuthology.orchestra.run.smithi151:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-14T16:35:54.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T16:35:54.671 DEBUG:teuthology.orchestra.run.smithi002:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-14T16:35:54.676 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-14T16:35:54.696 DEBUG:teuthology.orchestra.run.smithi151:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-14T16:35:54.728 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-14T16:35:54.738 INFO:teuthology.orchestra.run.smithi002.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-14T16:35:54.755 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-14T16:35:54.764 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-14T16:35:54.785 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-14T16:35:54.785 INFO:teuthology.orchestra.run.smithi002.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-14T16:35:54.787 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-14T16:35:54.796 INFO:teuthology.task.internal:Configuring sudo... 2024-05-14T16:35:54.796 DEBUG:teuthology.orchestra.run.smithi002:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-14T16:35:54.799 DEBUG:teuthology.orchestra.run.smithi123:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-14T16:35:54.810 DEBUG:teuthology.orchestra.run.smithi151:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-14T16:35:54.855 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-14T16:35:54.868 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-14T16:35:54.868 DEBUG:teuthology.orchestra.run.smithi002:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-14T16:35:54.872 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-14T16:35:54.877 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-14T16:35:54.911 DEBUG:teuthology.orchestra.run.smithi002:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-14T16:35:54.969 DEBUG:teuthology.orchestra.run.smithi002:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-14T16:35:55.018 DEBUG:teuthology.orchestra.run.smithi002:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-14T16:35:55.107 DEBUG:teuthology.orchestra.run.smithi002:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-14T16:35:55.149 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T16:35:55.150 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-14T16:35:55.216 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-14T16:35:55.263 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-14T16:35:55.340 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-14T16:35:55.382 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-14T16:35:55.424 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-05-14T16:35:55.424 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-14T16:35:55.488 DEBUG:teuthology.orchestra.run.smithi151:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-14T16:35:55.532 DEBUG:teuthology.orchestra.run.smithi151:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-14T16:35:55.581 DEBUG:teuthology.orchestra.run.smithi151:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-14T16:35:55.663 DEBUG:teuthology.orchestra.run.smithi151:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-14T16:35:55.705 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-14T16:35:55.705 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-14T16:35:55.771 DEBUG:teuthology.orchestra.run.smithi002:> sudo service rsyslog restart 2024-05-14T16:35:55.773 DEBUG:teuthology.orchestra.run.smithi123:> sudo service rsyslog restart 2024-05-14T16:35:55.775 DEBUG:teuthology.orchestra.run.smithi151:> sudo service rsyslog restart 2024-05-14T16:35:55.806 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-14T16:35:55.853 INFO:teuthology.orchestra.run.smithi151.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-14T16:35:55.870 INFO:teuthology.orchestra.run.smithi002.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-14T16:35:56.195 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-14T16:35:56.203 INFO:teuthology.task.internal:Starting timer... 2024-05-14T16:35:56.203 INFO:teuthology.run_tasks:Running task pcp... 2024-05-14T16:35:56.251 INFO:teuthology.run_tasks:Running task selinux... 2024-05-14T16:35:56.274 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-14T16:35:56.275 DEBUG:teuthology.orchestra.run.smithi002:> sudo service auditd rotate 2024-05-14T16:35:56.370 INFO:teuthology.orchestra.run.smithi002.stdout:Rotating logs: 2024-05-14T16:35:56.373 DEBUG:teuthology.orchestra.run.smithi123:> sudo service auditd rotate 2024-05-14T16:35:56.416 INFO:teuthology.orchestra.run.smithi123.stdout:Rotating logs: 2024-05-14T16:35:56.417 DEBUG:teuthology.orchestra.run.smithi151:> sudo service auditd rotate 2024-05-14T16:35:56.477 INFO:teuthology.orchestra.run.smithi151.stdout:Rotating logs: 2024-05-14T16:35:56.479 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-14T16:35:56.479 DEBUG:teuthology.orchestra.run.smithi002:> /usr/sbin/getenforce 2024-05-14T16:35:56.513 INFO:teuthology.orchestra.run.smithi002.stdout:Permissive 2024-05-14T16:35:56.514 DEBUG:teuthology.orchestra.run.smithi123:> /usr/sbin/getenforce 2024-05-14T16:35:56.539 INFO:teuthology.orchestra.run.smithi123.stdout:Permissive 2024-05-14T16:35:56.540 DEBUG:teuthology.orchestra.run.smithi151:> /usr/sbin/getenforce 2024-05-14T16:35:56.557 INFO:teuthology.orchestra.run.smithi151.stdout:Permissive 2024-05-14T16:35:56.557 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi002.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi123.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi151.front.sepia.ceph.com': 'permissive'} 2024-05-14T16:35:56.557 DEBUG:teuthology.orchestra.run.smithi002:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-14T16:35:56.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T16:35:56.610 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-14T16:35:56.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T16:35:56.636 DEBUG:teuthology.orchestra.run.smithi151:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-14T16:35:56.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T16:35:56.662 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-14T16:35:56.662 DEBUG:teuthology.orchestra.run.smithi002:> sudo /usr/sbin/setenforce permissive 2024-05-14T16:35:56.921 DEBUG:teuthology.orchestra.run.smithi123:> sudo /usr/sbin/setenforce permissive 2024-05-14T16:35:56.946 DEBUG:teuthology.orchestra.run.smithi151:> sudo /usr/sbin/setenforce permissive 2024-05-14T16:35:56.970 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-14T16:35:56.979 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-14T16:35:56.985 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-14T16:35:57.132 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-14T16:35:57.146 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-14T16:35:57.148 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi002.front.sepia.ceph.com,smithi123.front.sepia.ceph.com,smithi151.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-14T16:40:28.514 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi002.front.sepia.ceph.com'), Remote(name='ubuntu@smithi123.front.sepia.ceph.com'), Remote(name='ubuntu@smithi151.front.sepia.ceph.com')] 2024-05-14T16:40:28.515 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-05-14T16:40:28.516 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T16:40:28.605 DEBUG:teuthology.orchestra.run.smithi002:> true 2024-05-14T16:40:28.694 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-05-14T16:40:28.695 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-05-14T16:40:28.695 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T16:40:28.769 DEBUG:teuthology.orchestra.run.smithi123:> true 2024-05-14T16:40:28.853 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-05-14T16:40:28.853 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-05-14T16:40:28.854 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-14T16:40:28.971 DEBUG:teuthology.orchestra.run.smithi151:> true 2024-05-14T16:40:29.059 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-05-14T16:40:29.059 INFO:teuthology.run_tasks:Running task clock... 2024-05-14T16:40:29.075 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-14T16:40:29.075 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-14T16:40:29.076 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-14T16:40:29.078 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-14T16:40:29.078 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-14T16:40:29.080 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-14T16:40:29.081 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-14T16:40:29.107 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-14T16:40:29.108 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-14T16:40:29.120 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-14T16:40:29.124 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-14T16:40:29.129 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-14T16:40:29.143 INFO:teuthology.orchestra.run.smithi123.stderr:sudo: ntpd: command not found 2024-05-14T16:40:29.143 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-14T16:40:29.150 INFO:teuthology.orchestra.run.smithi002.stderr:sudo: ntpd: command not found 2024-05-14T16:40:29.153 INFO:teuthology.orchestra.run.smithi123.stdout:506 Cannot talk to daemon 2024-05-14T16:40:29.162 INFO:teuthology.orchestra.run.smithi002.stdout:506 Cannot talk to daemon 2024-05-14T16:40:29.166 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-14T16:40:29.167 INFO:teuthology.orchestra.run.smithi151.stderr:sudo: ntpd: command not found 2024-05-14T16:40:29.177 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-14T16:40:29.178 INFO:teuthology.orchestra.run.smithi151.stdout:506 Cannot talk to daemon 2024-05-14T16:40:29.179 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-14T16:40:29.191 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-14T16:40:29.192 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-14T16:40:29.204 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-14T16:40:29.233 INFO:teuthology.orchestra.run.smithi123.stderr:bash: line 1: ntpq: command not found 2024-05-14T16:40:29.236 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-14T16:40:29.236 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2024-05-14T16:40:29.236 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T16:40:29.236 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T16:40:29.236 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T16:40:29.236 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T16:40:29.247 INFO:teuthology.orchestra.run.smithi002.stderr:bash: line 1: ntpq: command not found 2024-05-14T16:40:29.251 INFO:teuthology.orchestra.run.smithi002.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-14T16:40:29.251 INFO:teuthology.orchestra.run.smithi002.stdout:=============================================================================== 2024-05-14T16:40:29.251 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T16:40:29.251 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T16:40:29.252 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T16:40:29.252 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T16:40:29.258 INFO:teuthology.orchestra.run.smithi151.stderr:bash: line 1: ntpq: command not found 2024-05-14T16:40:29.261 INFO:teuthology.orchestra.run.smithi151.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-14T16:40:29.261 INFO:teuthology.orchestra.run.smithi151.stdout:=============================================================================== 2024-05-14T16:40:29.262 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T16:40:29.262 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T16:40:29.262 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T16:40:29.262 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T16:40:29.263 INFO:teuthology.run_tasks:Running task install... 2024-05-14T16:40:29.271 DEBUG:teuthology.task.install:project ceph 2024-05-14T16:40:29.271 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '9b63866860f0b71216b2c0a73aedd2e0a8a5c98b'}} 2024-05-14T16:40:29.271 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '9b63866860f0b71216b2c0a73aedd2e0a8a5c98b'} 2024-05-14T16:40:29.271 INFO:teuthology.task.install:Using flavor: default 2024-05-14T16:40:29.277 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-14T16:40:29.277 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-14T16:40:29.277 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9b63866860f0b71216b2c0a73aedd2e0a8a5c98b', 'tag': None, 'wait_for_package': False} 2024-05-14T16:40:29.278 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-14T16:40:29.280 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9b63866860f0b71216b2c0a73aedd2e0a8a5c98b', 'tag': None, 'wait_for_package': False} 2024-05-14T16:40:29.280 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-14T16:40:29.281 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9b63866860f0b71216b2c0a73aedd2e0a8a5c98b', 'tag': None, 'wait_for_package': False} 2024-05-14T16:40:29.282 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-14T16:40:29.424 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/ 2024-05-14T16:40:29.424 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3687.g9b638668 2024-05-14T16:40:29.428 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/ 2024-05-14T16:40:29.429 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3687.g9b638668 2024-05-14T16:40:29.499 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/ 2024-05-14T16:40:29.499 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3687.g9b638668 2024-05-14T16:40:29.719 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-14T16:40:29.720 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-05-14T16:40:29.720 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-14T16:40:29.744 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-14T16:40:29.744 DEBUG:teuthology.orchestra.run.smithi123:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-14T16:40:29.786 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-14T16:40:29.786 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T16:40:29.786 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-14T16:40:29.810 DEBUG:teuthology.orchestra.run.smithi123:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-14T16:40:29.816 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-14T16:40:29.817 DEBUG:teuthology.orchestra.run.smithi002:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-14T16:40:29.874 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-14T16:40:29.874 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-14T16:40:29.874 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-14T16:40:29.883 DEBUG:teuthology.orchestra.run.smithi123:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-14T16:40:29.887 DEBUG:teuthology.orchestra.run.smithi002:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-14T16:40:29.902 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-14T16:40:29.902 DEBUG:teuthology.orchestra.run.smithi151:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-14T16:40:29.947 INFO:teuthology.orchestra.run.smithi123.stdout:check_obsoletes = 1 2024-05-14T16:40:29.949 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum clean all 2024-05-14T16:40:29.968 DEBUG:teuthology.orchestra.run.smithi002:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-14T16:40:29.969 DEBUG:teuthology.orchestra.run.smithi151:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-14T16:40:30.036 INFO:teuthology.orchestra.run.smithi002.stdout:check_obsoletes = 1 2024-05-14T16:40:30.038 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum clean all 2024-05-14T16:40:30.047 DEBUG:teuthology.orchestra.run.smithi151:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-14T16:40:30.110 INFO:teuthology.orchestra.run.smithi151.stdout:check_obsoletes = 1 2024-05-14T16:40:30.111 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum clean all 2024-05-14T16:40:30.177 INFO:teuthology.orchestra.run.smithi123.stdout:57 files removed 2024-05-14T16:40:30.204 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-14T16:40:30.288 INFO:teuthology.orchestra.run.smithi002.stdout:57 files removed 2024-05-14T16:40:30.316 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-14T16:40:30.342 INFO:teuthology.orchestra.run.smithi151.stdout:57 files removed 2024-05-14T16:40:30.369 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-14T16:40:31.225 INFO:teuthology.orchestra.run.smithi123.stdout:ceph packages for x86_64 110 kB/s | 84 kB 00:00 2024-05-14T16:40:31.285 INFO:teuthology.orchestra.run.smithi002.stdout:ceph packages for x86_64 123 kB/s | 84 kB 00:00 2024-05-14T16:40:31.353 INFO:teuthology.orchestra.run.smithi151.stdout:ceph packages for x86_64 117 kB/s | 84 kB 00:00 2024-05-14T16:40:31.958 INFO:teuthology.orchestra.run.smithi123.stdout:ceph noarch packages 17 kB/s | 12 kB 00:00 2024-05-14T16:40:32.044 INFO:teuthology.orchestra.run.smithi002.stdout:ceph noarch packages 17 kB/s | 12 kB 00:00 2024-05-14T16:40:32.046 INFO:teuthology.orchestra.run.smithi151.stdout:ceph noarch packages 18 kB/s | 12 kB 00:00 2024-05-14T16:40:32.618 INFO:teuthology.orchestra.run.smithi123.stdout:ceph source packages 3.0 kB/s | 1.9 kB 00:00 2024-05-14T16:40:32.714 INFO:teuthology.orchestra.run.smithi151.stdout:ceph source packages 3.0 kB/s | 1.9 kB 00:00 2024-05-14T16:40:32.736 INFO:teuthology.orchestra.run.smithi002.stdout:ceph source packages 2.9 kB/s | 1.9 kB 00:00 2024-05-14T16:40:33.444 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.1 MB 00:00 2024-05-14T16:40:33.959 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - BaseOS 6.2 MB/s | 8.1 MB 00:01 2024-05-14T16:40:34.297 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - BaseOS 5.2 MB/s | 8.1 MB 00:01 2024-05-14T16:40:37.681 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - AppStream 8.4 MB/s | 19 MB 00:02 2024-05-14T16:40:37.869 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - AppStream 6.8 MB/s | 19 MB 00:02 2024-05-14T16:40:38.362 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - AppStream 5.1 MB/s | 19 MB 00:03 2024-05-14T16:40:41.757 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-05-14T16:40:41.819 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - CRB 18 MB/s | 6.2 MB 00:00 2024-05-14T16:40:43.291 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 16 kB 00:00 2024-05-14T16:40:43.304 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - CRB 5.5 MB/s | 6.2 MB 00:01 2024-05-14T16:40:43.511 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-05-14T16:40:44.641 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:00 2024-05-14T16:40:45.030 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 16 kB 00:00 2024-05-14T16:40:45.998 INFO:teuthology.orchestra.run.smithi002.stdout:Extra Packages for Enterprise Linux 26 MB/s | 21 MB 00:00 2024-05-14T16:40:46.634 INFO:teuthology.orchestra.run.smithi151.stdout:Extra Packages for Enterprise Linux 6.6 MB/s | 21 MB 00:03 2024-05-14T16:40:49.771 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 21 kB/s | 1.7 kB 00:00 2024-05-14T16:40:51.448 INFO:teuthology.orchestra.run.smithi002.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-05-14T16:40:51.968 INFO:teuthology.orchestra.run.smithi151.stdout:lab-extras 45 kB/s | 1.7 kB 00:00 2024-05-14T16:40:52.022 INFO:teuthology.orchestra.run.smithi123.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-14T16:40:52.023 INFO:teuthology.orchestra.run.smithi123.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-14T16:40:52.034 INFO:teuthology.orchestra.run.smithi123.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-14T16:40:52.035 INFO:teuthology.orchestra.run.smithi123.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-14T16:40:52.036 INFO:teuthology.orchestra.run.smithi123.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-14T16:40:52.037 INFO:teuthology.orchestra.run.smithi123.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-14T16:40:52.038 INFO:teuthology.orchestra.run.smithi123.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-14T16:40:52.041 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-14T16:40:52.042 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-14T16:40:52.103 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-05-14T16:40:52.124 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-05-14T16:40:52.124 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-05-14T16:40:52.124 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-05-14T16:40:52.124 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-05-14T16:40:52.124 INFO:teuthology.orchestra.run.smithi123.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-14T16:40:52.124 INFO:teuthology.orchestra.run.smithi123.stdout: ceph x86_64 2:19.0.0-3687.g9b638668.el9 ceph 6.6 k 2024-05-14T16:40:52.124 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base x86_64 2:19.0.0-3687.g9b638668.el9 ceph 5.4 M 2024-05-14T16:40:52.124 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse x86_64 2:19.0.0-3687.g9b638668.el9 ceph 891 k 2024-05-14T16:40:52.125 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3687.g9b638668.el9 ceph 145 k 2024-05-14T16:40:52.125 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr x86_64 2:19.0.0-3687.g9b638668.el9 ceph 1.5 M 2024-05-14T16:40:52.125 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 143 k 2024-05-14T16:40:52.125 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 3.5 M 2024-05-14T16:40:52.125 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 7.4 M 2024-05-14T16:40:52.125 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 49 k 2024-05-14T16:40:52.125 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw x86_64 2:19.0.0-3687.g9b638668.el9 ceph 11 M 2024-05-14T16:40:52.125 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test x86_64 2:19.0.0-3687.g9b638668.el9 ceph 49 M 2024-05-14T16:40:52.125 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 273 k 2024-05-14T16:40:52.125 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 763 k 2024-05-14T16:40:52.125 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror x86_64 2:19.0.0-3687.g9b638668.el9 ceph 225 k 2024-05-14T16:40:52.125 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-shell x86_64 2:19.0.0-3687.g9b638668.el9 ceph 27 k 2024-05-14T16:40:52.125 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 18 k 2024-05-14T16:40:52.126 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-14T16:40:52.126 INFO:teuthology.orchestra.run.smithi123.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-14T16:40:52.126 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-14T16:40:52.126 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel x86_64 2:19.0.0-3687.g9b638668.el9 ceph 32 k 2024-05-14T16:40:52.126 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 748 k 2024-05-14T16:40:52.126 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel x86_64 2:19.0.0-3687.g9b638668.el9 ceph 127 k 2024-05-14T16:40:52.126 INFO:teuthology.orchestra.run.smithi123.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-14T16:40:52.126 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-14T16:40:52.126 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-14T16:40:52.126 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-14T16:40:52.126 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-14T16:40:52.127 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-14T16:40:52.127 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs x86_64 2:19.0.0-3687.g9b638668.el9 ceph 161 k 2024-05-14T16:40:52.127 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados x86_64 2:19.0.0-3687.g9b638668.el9 ceph 321 k 2024-05-14T16:40:52.127 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd x86_64 2:19.0.0-3687.g9b638668.el9 ceph 299 k 2024-05-14T16:40:52.127 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw x86_64 2:19.0.0-3687.g9b638668.el9 ceph 100 k 2024-05-14T16:40:52.127 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse x86_64 2:19.0.0-3687.g9b638668.el9 ceph 85 k 2024-05-14T16:40:52.127 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror x86_64 2:19.0.0-3687.g9b638668.el9 ceph 3.1 M 2024-05-14T16:40:52.127 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd x86_64 2:19.0.0-3687.g9b638668.el9 ceph 170 k 2024-05-14T16:40:52.127 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-14T16:40:52.127 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrading: 2024-05-14T16:40:52.127 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-14T16:40:52.127 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-14T16:40:52.127 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-14T16:40:52.128 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-14T16:40:52.128 INFO:teuthology.orchestra.run.smithi123.stdout: librados2 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 3.4 M 2024-05-14T16:40:52.128 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 3.1 M 2024-05-14T16:40:52.128 INFO:teuthology.orchestra.run.smithi123.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-14T16:40:52.128 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-14T16:40:52.128 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-05-14T16:40:52.128 INFO:teuthology.orchestra.run.smithi123.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-14T16:40:52.128 INFO:teuthology.orchestra.run.smithi123.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-14T16:40:52.128 INFO:teuthology.orchestra.run.smithi123.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-14T16:40:52.128 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common x86_64 2:19.0.0-3687.g9b638668.el9 ceph 21 M 2024-05-14T16:40:52.128 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 34 k 2024-05-14T16:40:52.129 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds x86_64 2:19.0.0-3687.g9b638668.el9 ceph 2.4 M 2024-05-14T16:40:52.129 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 274 k 2024-05-14T16:40:52.129 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon x86_64 2:19.0.0-3687.g9b638668.el9 ceph 4.7 M 2024-05-14T16:40:52.129 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd x86_64 2:19.0.0-3687.g9b638668.el9 ceph 17 M 2024-05-14T16:40:52.129 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 16 k 2024-05-14T16:40:52.129 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux x86_64 2:19.0.0-3687.g9b638668.el9 ceph 25 k 2024-05-14T16:40:52.129 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-14T16:40:52.129 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-14T16:40:52.129 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-14T16:40:52.129 INFO:teuthology.orchestra.run.smithi123.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-14T16:40:52.129 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-14T16:40:52.129 INFO:teuthology.orchestra.run.smithi123.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-14T16:40:52.129 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-14T16:40:52.130 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-14T16:40:52.130 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite x86_64 2:19.0.0-3687.g9b638668.el9 ceph 163 k 2024-05-14T16:40:52.130 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-14T16:40:52.130 INFO:teuthology.orchestra.run.smithi123.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-14T16:40:52.130 INFO:teuthology.orchestra.run.smithi123.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-14T16:40:52.130 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-14T16:40:52.130 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-14T16:40:52.130 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 501 k 2024-05-14T16:40:52.130 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-14T16:40:52.130 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 5.4 M 2024-05-14T16:40:52.130 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-14T16:40:52.130 INFO:teuthology.orchestra.run.smithi123.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-14T16:40:52.130 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-14T16:40:52.131 INFO:teuthology.orchestra.run.smithi123.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-14T16:40:52.131 INFO:teuthology.orchestra.run.smithi123.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-14T16:40:52.131 INFO:teuthology.orchestra.run.smithi123.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-14T16:40:52.131 INFO:teuthology.orchestra.run.smithi123.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-14T16:40:52.131 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-14T16:40:52.131 INFO:teuthology.orchestra.run.smithi123.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-14T16:40:52.131 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-14T16:40:52.131 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-14T16:40:52.131 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-14T16:40:52.131 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-14T16:40:52.131 INFO:teuthology.orchestra.run.smithi123.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-14T16:40:52.131 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-14T16:40:52.131 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-14T16:40:52.131 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-14T16:40:52.132 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse x86_64 2:19.0.0-3687.g9b638668.el9 ceph 45 k 2024-05-14T16:40:52.132 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common x86_64 2:19.0.0-3687.g9b638668.el9 ceph 136 k 2024-05-14T16:40:52.132 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-14T16:40:52.132 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-14T16:40:52.132 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-14T16:40:52.132 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-14T16:40:52.132 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-14T16:40:52.132 INFO:teuthology.orchestra.run.smithi123.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-14T16:40:52.132 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-14T16:40:52.132 INFO:teuthology.orchestra.run.smithi123.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-14T16:40:52.133 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-14T16:40:52.133 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-14T16:40:52.133 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-14T16:40:52.133 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-14T16:40:52.133 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-14T16:40:52.133 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-14T16:40:52.133 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-14T16:40:52.133 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-14T16:40:52.133 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-14T16:40:52.133 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-14T16:40:52.133 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-14T16:40:52.133 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-14T16:40:52.133 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-14T16:40:52.133 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-14T16:40:52.134 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-14T16:40:52.134 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-14T16:40:52.134 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-14T16:40:52.134 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-14T16:40:52.134 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-14T16:40:52.134 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-14T16:40:52.134 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-14T16:40:52.134 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-14T16:40:52.134 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-14T16:40:52.134 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-14T16:40:52.134 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-14T16:40:52.134 INFO:teuthology.orchestra.run.smithi123.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-14T16:40:52.134 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-14T16:40:52.135 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-14T16:40:52.135 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-14T16:40:52.135 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-14T16:40:52.135 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-14T16:40:52.135 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-14T16:40:52.135 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-14T16:40:52.135 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-14T16:40:52.135 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-14T16:40:52.135 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-14T16:40:52.135 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-14T16:40:52.135 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-14T16:40:52.135 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-14T16:40:52.135 INFO:teuthology.orchestra.run.smithi123.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-14T16:40:52.135 INFO:teuthology.orchestra.run.smithi123.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-14T16:40:52.136 INFO:teuthology.orchestra.run.smithi123.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-14T16:40:52.136 INFO:teuthology.orchestra.run.smithi123.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-14T16:40:52.136 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-14T16:40:52.136 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-14T16:40:52.136 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-14T16:40:52.136 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-14T16:40:52.136 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-14T16:40:52.136 INFO:teuthology.orchestra.run.smithi123.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-14T16:40:52.136 INFO:teuthology.orchestra.run.smithi123.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-14T16:40:52.136 INFO:teuthology.orchestra.run.smithi123.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-14T16:40:52.136 INFO:teuthology.orchestra.run.smithi123.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-14T16:40:52.136 INFO:teuthology.orchestra.run.smithi123.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-14T16:40:52.136 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-14T16:40:52.136 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-14T16:40:52.137 INFO:teuthology.orchestra.run.smithi123.stdout:Installing weak dependencies: 2024-05-14T16:40:52.137 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-14T16:40:52.137 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-14T16:40:52.137 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-05-14T16:40:52.137 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-05-14T16:40:52.137 INFO:teuthology.orchestra.run.smithi123.stdout:Install 148 Packages 2024-05-14T16:40:52.137 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrade 8 Packages 2024-05-14T16:40:52.137 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-14T16:40:52.138 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 226 M 2024-05-14T16:40:52.138 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-05-14T16:40:52.940 INFO:teuthology.orchestra.run.smithi123.stdout:(1/156): ceph-19.0.0-3687.g9b638668.el9.x86_64. 39 kB/s | 6.6 kB 00:00 2024-05-14T16:40:53.440 INFO:teuthology.orchestra.run.smithi123.stdout:(2/156): ceph-fuse-19.0.0-3687.g9b638668.el9.x8 1.7 MB/s | 891 kB 00:00 2024-05-14T16:40:53.506 INFO:teuthology.orchestra.run.smithi123.stdout:(3/156): ceph-immutable-object-cache-19.0.0-368 2.1 MB/s | 145 kB 00:00 2024-05-14T16:40:53.815 INFO:teuthology.orchestra.run.smithi123.stdout:(4/156): ceph-mds-19.0.0-3687.g9b638668.el9.x86 7.7 MB/s | 2.4 MB 00:00 2024-05-14T16:40:53.875 INFO:teuthology.orchestra.run.smithi002.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-14T16:40:53.875 INFO:teuthology.orchestra.run.smithi002.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-14T16:40:53.887 INFO:teuthology.orchestra.run.smithi002.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-14T16:40:53.887 INFO:teuthology.orchestra.run.smithi002.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-14T16:40:53.888 INFO:teuthology.orchestra.run.smithi002.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-14T16:40:53.889 INFO:teuthology.orchestra.run.smithi002.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-14T16:40:53.891 INFO:teuthology.orchestra.run.smithi002.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-14T16:40:53.893 INFO:teuthology.orchestra.run.smithi002.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-14T16:40:53.894 INFO:teuthology.orchestra.run.smithi002.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-14T16:40:53.958 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-05-14T16:40:53.979 INFO:teuthology.orchestra.run.smithi002.stdout:======================================================================================= 2024-05-14T16:40:53.980 INFO:teuthology.orchestra.run.smithi002.stdout: Package Arch Version Repository Size 2024-05-14T16:40:53.980 INFO:teuthology.orchestra.run.smithi002.stdout:======================================================================================= 2024-05-14T16:40:53.980 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-05-14T16:40:53.980 INFO:teuthology.orchestra.run.smithi002.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-14T16:40:53.980 INFO:teuthology.orchestra.run.smithi002.stdout: ceph x86_64 2:19.0.0-3687.g9b638668.el9 ceph 6.6 k 2024-05-14T16:40:53.980 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-base x86_64 2:19.0.0-3687.g9b638668.el9 ceph 5.4 M 2024-05-14T16:40:53.980 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-fuse x86_64 2:19.0.0-3687.g9b638668.el9 ceph 891 k 2024-05-14T16:40:53.980 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3687.g9b638668.el9 ceph 145 k 2024-05-14T16:40:53.980 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr x86_64 2:19.0.0-3687.g9b638668.el9 ceph 1.5 M 2024-05-14T16:40:53.980 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 143 k 2024-05-14T16:40:53.980 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 3.5 M 2024-05-14T16:40:53.980 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 7.4 M 2024-05-14T16:40:53.980 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-rook noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 49 k 2024-05-14T16:40:53.981 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-radosgw x86_64 2:19.0.0-3687.g9b638668.el9 ceph 11 M 2024-05-14T16:40:53.981 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-test x86_64 2:19.0.0-3687.g9b638668.el9 ceph 49 M 2024-05-14T16:40:53.981 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-volume noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 273 k 2024-05-14T16:40:53.981 INFO:teuthology.orchestra.run.smithi002.stdout: cephadm noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 763 k 2024-05-14T16:40:53.981 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-mirror x86_64 2:19.0.0-3687.g9b638668.el9 ceph 225 k 2024-05-14T16:40:53.981 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-shell x86_64 2:19.0.0-3687.g9b638668.el9 ceph 27 k 2024-05-14T16:40:53.981 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-top noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 18 k 2024-05-14T16:40:53.981 INFO:teuthology.orchestra.run.smithi002.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-14T16:40:53.981 INFO:teuthology.orchestra.run.smithi002.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-14T16:40:53.981 INFO:teuthology.orchestra.run.smithi002.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-14T16:40:53.981 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs-devel x86_64 2:19.0.0-3687.g9b638668.el9 ceph 32 k 2024-05-14T16:40:53.981 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs2 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 748 k 2024-05-14T16:40:53.982 INFO:teuthology.orchestra.run.smithi002.stdout: librados-devel x86_64 2:19.0.0-3687.g9b638668.el9 ceph 127 k 2024-05-14T16:40:53.982 INFO:teuthology.orchestra.run.smithi002.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-14T16:40:53.982 INFO:teuthology.orchestra.run.smithi002.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-14T16:40:53.982 INFO:teuthology.orchestra.run.smithi002.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-14T16:40:53.982 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-14T16:40:53.982 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-14T16:40:53.982 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-14T16:40:53.982 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cephfs x86_64 2:19.0.0-3687.g9b638668.el9 ceph 161 k 2024-05-14T16:40:53.982 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rados x86_64 2:19.0.0-3687.g9b638668.el9 ceph 321 k 2024-05-14T16:40:53.982 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rbd x86_64 2:19.0.0-3687.g9b638668.el9 ceph 299 k 2024-05-14T16:40:53.982 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rgw x86_64 2:19.0.0-3687.g9b638668.el9 ceph 100 k 2024-05-14T16:40:53.982 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-fuse x86_64 2:19.0.0-3687.g9b638668.el9 ceph 85 k 2024-05-14T16:40:53.982 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-mirror x86_64 2:19.0.0-3687.g9b638668.el9 ceph 3.1 M 2024-05-14T16:40:53.983 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-nbd x86_64 2:19.0.0-3687.g9b638668.el9 ceph 170 k 2024-05-14T16:40:53.983 INFO:teuthology.orchestra.run.smithi002.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-14T16:40:53.983 INFO:teuthology.orchestra.run.smithi002.stdout:Upgrading: 2024-05-14T16:40:53.983 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-14T16:40:53.983 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-14T16:40:53.983 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-14T16:40:53.983 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-14T16:40:53.983 INFO:teuthology.orchestra.run.smithi002.stdout: librados2 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 3.4 M 2024-05-14T16:40:53.983 INFO:teuthology.orchestra.run.smithi002.stdout: librbd1 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 3.1 M 2024-05-14T16:40:53.983 INFO:teuthology.orchestra.run.smithi002.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-14T16:40:53.983 INFO:teuthology.orchestra.run.smithi002.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-14T16:40:53.983 INFO:teuthology.orchestra.run.smithi002.stdout:Installing dependencies: 2024-05-14T16:40:53.983 INFO:teuthology.orchestra.run.smithi002.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-14T16:40:53.983 INFO:teuthology.orchestra.run.smithi002.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-14T16:40:53.984 INFO:teuthology.orchestra.run.smithi002.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-14T16:40:53.984 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-common x86_64 2:19.0.0-3687.g9b638668.el9 ceph 21 M 2024-05-14T16:40:53.984 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 34 k 2024-05-14T16:40:53.984 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mds x86_64 2:19.0.0-3687.g9b638668.el9 ceph 2.4 M 2024-05-14T16:40:53.984 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 274 k 2024-05-14T16:40:53.984 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mon x86_64 2:19.0.0-3687.g9b638668.el9 ceph 4.7 M 2024-05-14T16:40:53.984 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-osd x86_64 2:19.0.0-3687.g9b638668.el9 ceph 17 M 2024-05-14T16:40:53.984 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 16 k 2024-05-14T16:40:53.984 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-selinux x86_64 2:19.0.0-3687.g9b638668.el9 ceph 25 k 2024-05-14T16:40:53.984 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-14T16:40:53.984 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-14T16:40:53.984 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-14T16:40:53.984 INFO:teuthology.orchestra.run.smithi002.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-14T16:40:53.985 INFO:teuthology.orchestra.run.smithi002.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-14T16:40:53.985 INFO:teuthology.orchestra.run.smithi002.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-14T16:40:53.985 INFO:teuthology.orchestra.run.smithi002.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-14T16:40:53.985 INFO:teuthology.orchestra.run.smithi002.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-14T16:40:53.985 INFO:teuthology.orchestra.run.smithi002.stdout: libcephsqlite x86_64 2:19.0.0-3687.g9b638668.el9 ceph 163 k 2024-05-14T16:40:53.985 INFO:teuthology.orchestra.run.smithi002.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-14T16:40:53.985 INFO:teuthology.orchestra.run.smithi002.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-14T16:40:53.985 INFO:teuthology.orchestra.run.smithi002.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-14T16:40:53.985 INFO:teuthology.orchestra.run.smithi002.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-14T16:40:53.985 INFO:teuthology.orchestra.run.smithi002.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-14T16:40:53.985 INFO:teuthology.orchestra.run.smithi002.stdout: libradosstriper1 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 501 k 2024-05-14T16:40:53.985 INFO:teuthology.orchestra.run.smithi002.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-14T16:40:53.985 INFO:teuthology.orchestra.run.smithi002.stdout: librgw2 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 5.4 M 2024-05-14T16:40:53.985 INFO:teuthology.orchestra.run.smithi002.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-14T16:40:53.986 INFO:teuthology.orchestra.run.smithi002.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-14T16:40:53.986 INFO:teuthology.orchestra.run.smithi002.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-14T16:40:53.986 INFO:teuthology.orchestra.run.smithi002.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-14T16:40:53.986 INFO:teuthology.orchestra.run.smithi002.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-14T16:40:53.986 INFO:teuthology.orchestra.run.smithi002.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-14T16:40:53.986 INFO:teuthology.orchestra.run.smithi002.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-14T16:40:53.986 INFO:teuthology.orchestra.run.smithi002.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-14T16:40:53.986 INFO:teuthology.orchestra.run.smithi002.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-14T16:40:53.986 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-14T16:40:53.986 INFO:teuthology.orchestra.run.smithi002.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-14T16:40:53.986 INFO:teuthology.orchestra.run.smithi002.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-14T16:40:53.986 INFO:teuthology.orchestra.run.smithi002.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-14T16:40:53.987 INFO:teuthology.orchestra.run.smithi002.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-14T16:40:53.987 INFO:teuthology.orchestra.run.smithi002.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-14T16:40:53.987 INFO:teuthology.orchestra.run.smithi002.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-14T16:40:53.987 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-14T16:40:53.987 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-argparse x86_64 2:19.0.0-3687.g9b638668.el9 ceph 45 k 2024-05-14T16:40:53.987 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-common x86_64 2:19.0.0-3687.g9b638668.el9 ceph 136 k 2024-05-14T16:40:53.987 INFO:teuthology.orchestra.run.smithi002.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-14T16:40:53.987 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-14T16:40:53.987 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-14T16:40:53.987 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-14T16:40:53.987 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-14T16:40:53.987 INFO:teuthology.orchestra.run.smithi002.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-14T16:40:53.987 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-14T16:40:53.988 INFO:teuthology.orchestra.run.smithi002.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-14T16:40:53.988 INFO:teuthology.orchestra.run.smithi002.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-14T16:40:53.988 INFO:teuthology.orchestra.run.smithi002.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-14T16:40:53.988 INFO:teuthology.orchestra.run.smithi002.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-14T16:40:53.988 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-14T16:40:53.988 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-14T16:40:53.988 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-14T16:40:53.988 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-14T16:40:53.988 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-14T16:40:53.988 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-14T16:40:53.988 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-14T16:40:53.988 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-14T16:40:53.988 INFO:teuthology.orchestra.run.smithi002.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-14T16:40:53.988 INFO:teuthology.orchestra.run.smithi002.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-14T16:40:53.989 INFO:teuthology.orchestra.run.smithi002.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-14T16:40:53.989 INFO:teuthology.orchestra.run.smithi002.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-14T16:40:53.989 INFO:teuthology.orchestra.run.smithi002.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-14T16:40:53.989 INFO:teuthology.orchestra.run.smithi002.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-14T16:40:53.989 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-14T16:40:53.989 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-14T16:40:53.989 INFO:teuthology.orchestra.run.smithi002.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-14T16:40:53.989 INFO:teuthology.orchestra.run.smithi002.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-14T16:40:53.989 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-14T16:40:53.989 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-14T16:40:53.989 INFO:teuthology.orchestra.run.smithi002.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-14T16:40:53.989 INFO:teuthology.orchestra.run.smithi002.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-14T16:40:53.989 INFO:teuthology.orchestra.run.smithi002.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-14T16:40:53.990 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-14T16:40:53.990 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-14T16:40:53.990 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-14T16:40:53.990 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-14T16:40:53.990 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-14T16:40:53.990 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-14T16:40:53.990 INFO:teuthology.orchestra.run.smithi002.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-14T16:40:53.990 INFO:teuthology.orchestra.run.smithi002.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-14T16:40:53.990 INFO:teuthology.orchestra.run.smithi002.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-14T16:40:53.990 INFO:teuthology.orchestra.run.smithi002.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-14T16:40:53.990 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-14T16:40:53.990 INFO:teuthology.orchestra.run.smithi002.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-14T16:40:53.990 INFO:teuthology.orchestra.run.smithi002.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-14T16:40:53.991 INFO:teuthology.orchestra.run.smithi002.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-14T16:40:53.991 INFO:teuthology.orchestra.run.smithi002.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-14T16:40:53.991 INFO:teuthology.orchestra.run.smithi002.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-14T16:40:53.991 INFO:teuthology.orchestra.run.smithi002.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-14T16:40:53.991 INFO:teuthology.orchestra.run.smithi002.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-14T16:40:53.991 INFO:teuthology.orchestra.run.smithi002.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-14T16:40:53.991 INFO:teuthology.orchestra.run.smithi002.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-14T16:40:53.991 INFO:teuthology.orchestra.run.smithi002.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-14T16:40:53.991 INFO:teuthology.orchestra.run.smithi002.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-14T16:40:53.991 INFO:teuthology.orchestra.run.smithi002.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-14T16:40:53.991 INFO:teuthology.orchestra.run.smithi002.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-14T16:40:53.991 INFO:teuthology.orchestra.run.smithi002.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-14T16:40:53.991 INFO:teuthology.orchestra.run.smithi002.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-14T16:40:53.991 INFO:teuthology.orchestra.run.smithi002.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-14T16:40:53.992 INFO:teuthology.orchestra.run.smithi002.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-14T16:40:53.992 INFO:teuthology.orchestra.run.smithi002.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-14T16:40:53.992 INFO:teuthology.orchestra.run.smithi002.stdout:Installing weak dependencies: 2024-05-14T16:40:53.992 INFO:teuthology.orchestra.run.smithi002.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-14T16:40:53.992 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:40:53.992 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-05-14T16:40:53.992 INFO:teuthology.orchestra.run.smithi002.stdout:======================================================================================= 2024-05-14T16:40:53.992 INFO:teuthology.orchestra.run.smithi002.stdout:Install 148 Packages 2024-05-14T16:40:53.992 INFO:teuthology.orchestra.run.smithi002.stdout:Upgrade 8 Packages 2024-05-14T16:40:53.992 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:40:53.993 INFO:teuthology.orchestra.run.smithi123.stdout:(5/156): ceph-base-19.0.0-3687.g9b638668.el9.x8 4.4 MB/s | 5.4 MB 00:01 2024-05-14T16:40:53.993 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 226 M 2024-05-14T16:40:53.993 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-05-14T16:40:54.066 INFO:teuthology.orchestra.run.smithi123.stdout:(6/156): ceph-mgr-19.0.0-3687.g9b638668.el9.x86 6.1 MB/s | 1.5 MB 00:00 2024-05-14T16:40:54.252 INFO:teuthology.orchestra.run.smithi151.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-14T16:40:54.252 INFO:teuthology.orchestra.run.smithi151.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-14T16:40:54.263 INFO:teuthology.orchestra.run.smithi151.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-14T16:40:54.264 INFO:teuthology.orchestra.run.smithi151.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-14T16:40:54.264 INFO:teuthology.orchestra.run.smithi151.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-14T16:40:54.265 INFO:teuthology.orchestra.run.smithi151.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-14T16:40:54.267 INFO:teuthology.orchestra.run.smithi151.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-14T16:40:54.269 INFO:teuthology.orchestra.run.smithi151.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-14T16:40:54.270 INFO:teuthology.orchestra.run.smithi151.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-14T16:40:54.332 INFO:teuthology.orchestra.run.smithi151.stdout:Dependencies resolved. 2024-05-14T16:40:54.357 INFO:teuthology.orchestra.run.smithi151.stdout:======================================================================================= 2024-05-14T16:40:54.357 INFO:teuthology.orchestra.run.smithi151.stdout: Package Arch Version Repository Size 2024-05-14T16:40:54.357 INFO:teuthology.orchestra.run.smithi151.stdout:======================================================================================= 2024-05-14T16:40:54.357 INFO:teuthology.orchestra.run.smithi151.stdout:Installing: 2024-05-14T16:40:54.357 INFO:teuthology.orchestra.run.smithi151.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-14T16:40:54.357 INFO:teuthology.orchestra.run.smithi151.stdout: ceph x86_64 2:19.0.0-3687.g9b638668.el9 ceph 6.6 k 2024-05-14T16:40:54.358 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-base x86_64 2:19.0.0-3687.g9b638668.el9 ceph 5.4 M 2024-05-14T16:40:54.358 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-fuse x86_64 2:19.0.0-3687.g9b638668.el9 ceph 891 k 2024-05-14T16:40:54.358 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3687.g9b638668.el9 ceph 145 k 2024-05-14T16:40:54.358 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr x86_64 2:19.0.0-3687.g9b638668.el9 ceph 1.5 M 2024-05-14T16:40:54.358 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 143 k 2024-05-14T16:40:54.358 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 3.5 M 2024-05-14T16:40:54.358 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 7.4 M 2024-05-14T16:40:54.358 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-rook noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 49 k 2024-05-14T16:40:54.358 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-radosgw x86_64 2:19.0.0-3687.g9b638668.el9 ceph 11 M 2024-05-14T16:40:54.358 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-test x86_64 2:19.0.0-3687.g9b638668.el9 ceph 49 M 2024-05-14T16:40:54.358 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-volume noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 273 k 2024-05-14T16:40:54.358 INFO:teuthology.orchestra.run.smithi151.stdout: cephadm noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 763 k 2024-05-14T16:40:54.358 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-mirror x86_64 2:19.0.0-3687.g9b638668.el9 ceph 225 k 2024-05-14T16:40:54.359 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-shell x86_64 2:19.0.0-3687.g9b638668.el9 ceph 27 k 2024-05-14T16:40:54.359 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-top noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 18 k 2024-05-14T16:40:54.359 INFO:teuthology.orchestra.run.smithi151.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-14T16:40:54.359 INFO:teuthology.orchestra.run.smithi151.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-14T16:40:54.359 INFO:teuthology.orchestra.run.smithi151.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-14T16:40:54.359 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs-devel x86_64 2:19.0.0-3687.g9b638668.el9 ceph 32 k 2024-05-14T16:40:54.359 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs2 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 748 k 2024-05-14T16:40:54.359 INFO:teuthology.orchestra.run.smithi151.stdout: librados-devel x86_64 2:19.0.0-3687.g9b638668.el9 ceph 127 k 2024-05-14T16:40:54.359 INFO:teuthology.orchestra.run.smithi151.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-14T16:40:54.359 INFO:teuthology.orchestra.run.smithi151.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-14T16:40:54.359 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-14T16:40:54.359 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-14T16:40:54.359 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-14T16:40:54.359 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-14T16:40:54.360 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cephfs x86_64 2:19.0.0-3687.g9b638668.el9 ceph 161 k 2024-05-14T16:40:54.360 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rados x86_64 2:19.0.0-3687.g9b638668.el9 ceph 321 k 2024-05-14T16:40:54.360 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rbd x86_64 2:19.0.0-3687.g9b638668.el9 ceph 299 k 2024-05-14T16:40:54.360 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rgw x86_64 2:19.0.0-3687.g9b638668.el9 ceph 100 k 2024-05-14T16:40:54.360 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-fuse x86_64 2:19.0.0-3687.g9b638668.el9 ceph 85 k 2024-05-14T16:40:54.360 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-mirror x86_64 2:19.0.0-3687.g9b638668.el9 ceph 3.1 M 2024-05-14T16:40:54.360 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-nbd x86_64 2:19.0.0-3687.g9b638668.el9 ceph 170 k 2024-05-14T16:40:54.360 INFO:teuthology.orchestra.run.smithi151.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-14T16:40:54.360 INFO:teuthology.orchestra.run.smithi151.stdout:Upgrading: 2024-05-14T16:40:54.360 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-14T16:40:54.360 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-14T16:40:54.360 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-14T16:40:54.360 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-14T16:40:54.361 INFO:teuthology.orchestra.run.smithi151.stdout: librados2 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 3.4 M 2024-05-14T16:40:54.361 INFO:teuthology.orchestra.run.smithi151.stdout: librbd1 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 3.1 M 2024-05-14T16:40:54.361 INFO:teuthology.orchestra.run.smithi151.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-14T16:40:54.361 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-14T16:40:54.361 INFO:teuthology.orchestra.run.smithi151.stdout:Installing dependencies: 2024-05-14T16:40:54.361 INFO:teuthology.orchestra.run.smithi151.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-14T16:40:54.361 INFO:teuthology.orchestra.run.smithi151.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-14T16:40:54.361 INFO:teuthology.orchestra.run.smithi151.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-14T16:40:54.361 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-common x86_64 2:19.0.0-3687.g9b638668.el9 ceph 21 M 2024-05-14T16:40:54.361 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 34 k 2024-05-14T16:40:54.361 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mds x86_64 2:19.0.0-3687.g9b638668.el9 ceph 2.4 M 2024-05-14T16:40:54.361 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 274 k 2024-05-14T16:40:54.362 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mon x86_64 2:19.0.0-3687.g9b638668.el9 ceph 4.7 M 2024-05-14T16:40:54.362 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-osd x86_64 2:19.0.0-3687.g9b638668.el9 ceph 17 M 2024-05-14T16:40:54.362 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 16 k 2024-05-14T16:40:54.362 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-selinux x86_64 2:19.0.0-3687.g9b638668.el9 ceph 25 k 2024-05-14T16:40:54.362 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-14T16:40:54.362 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-14T16:40:54.362 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-14T16:40:54.362 INFO:teuthology.orchestra.run.smithi151.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-14T16:40:54.362 INFO:teuthology.orchestra.run.smithi151.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-14T16:40:54.362 INFO:teuthology.orchestra.run.smithi151.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-14T16:40:54.362 INFO:teuthology.orchestra.run.smithi151.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-14T16:40:54.362 INFO:teuthology.orchestra.run.smithi151.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-14T16:40:54.362 INFO:teuthology.orchestra.run.smithi151.stdout: libcephsqlite x86_64 2:19.0.0-3687.g9b638668.el9 ceph 163 k 2024-05-14T16:40:54.363 INFO:teuthology.orchestra.run.smithi151.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-14T16:40:54.363 INFO:teuthology.orchestra.run.smithi151.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-14T16:40:54.363 INFO:teuthology.orchestra.run.smithi151.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-14T16:40:54.363 INFO:teuthology.orchestra.run.smithi151.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-14T16:40:54.363 INFO:teuthology.orchestra.run.smithi151.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-14T16:40:54.363 INFO:teuthology.orchestra.run.smithi151.stdout: libradosstriper1 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 501 k 2024-05-14T16:40:54.363 INFO:teuthology.orchestra.run.smithi151.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-14T16:40:54.363 INFO:teuthology.orchestra.run.smithi151.stdout: librgw2 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 5.4 M 2024-05-14T16:40:54.363 INFO:teuthology.orchestra.run.smithi151.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-14T16:40:54.363 INFO:teuthology.orchestra.run.smithi151.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-14T16:40:54.363 INFO:teuthology.orchestra.run.smithi151.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-14T16:40:54.363 INFO:teuthology.orchestra.run.smithi151.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-14T16:40:54.363 INFO:teuthology.orchestra.run.smithi151.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-14T16:40:54.363 INFO:teuthology.orchestra.run.smithi151.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-14T16:40:54.364 INFO:teuthology.orchestra.run.smithi151.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-14T16:40:54.364 INFO:teuthology.orchestra.run.smithi151.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-14T16:40:54.364 INFO:teuthology.orchestra.run.smithi151.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-14T16:40:54.364 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-14T16:40:54.364 INFO:teuthology.orchestra.run.smithi151.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-14T16:40:54.364 INFO:teuthology.orchestra.run.smithi151.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-14T16:40:54.364 INFO:teuthology.orchestra.run.smithi151.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-14T16:40:54.364 INFO:teuthology.orchestra.run.smithi151.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-14T16:40:54.365 INFO:teuthology.orchestra.run.smithi151.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-14T16:40:54.365 INFO:teuthology.orchestra.run.smithi151.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-14T16:40:54.365 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-14T16:40:54.365 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-argparse x86_64 2:19.0.0-3687.g9b638668.el9 ceph 45 k 2024-05-14T16:40:54.365 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-common x86_64 2:19.0.0-3687.g9b638668.el9 ceph 136 k 2024-05-14T16:40:54.365 INFO:teuthology.orchestra.run.smithi151.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-14T16:40:54.365 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-14T16:40:54.365 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-14T16:40:54.365 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-14T16:40:54.365 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-14T16:40:54.365 INFO:teuthology.orchestra.run.smithi151.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-14T16:40:54.365 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-14T16:40:54.365 INFO:teuthology.orchestra.run.smithi151.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-14T16:40:54.366 INFO:teuthology.orchestra.run.smithi151.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-14T16:40:54.366 INFO:teuthology.orchestra.run.smithi151.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-14T16:40:54.366 INFO:teuthology.orchestra.run.smithi151.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-14T16:40:54.366 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-14T16:40:54.366 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-14T16:40:54.366 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-14T16:40:54.366 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-14T16:40:54.366 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-14T16:40:54.366 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-14T16:40:54.366 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-14T16:40:54.366 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-14T16:40:54.366 INFO:teuthology.orchestra.run.smithi151.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-14T16:40:54.367 INFO:teuthology.orchestra.run.smithi151.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-14T16:40:54.367 INFO:teuthology.orchestra.run.smithi151.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-14T16:40:54.367 INFO:teuthology.orchestra.run.smithi151.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-14T16:40:54.367 INFO:teuthology.orchestra.run.smithi151.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-14T16:40:54.367 INFO:teuthology.orchestra.run.smithi151.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-14T16:40:54.367 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-14T16:40:54.367 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-14T16:40:54.367 INFO:teuthology.orchestra.run.smithi151.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-14T16:40:54.367 INFO:teuthology.orchestra.run.smithi151.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-14T16:40:54.367 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-14T16:40:54.367 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-14T16:40:54.367 INFO:teuthology.orchestra.run.smithi151.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-14T16:40:54.367 INFO:teuthology.orchestra.run.smithi151.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-14T16:40:54.368 INFO:teuthology.orchestra.run.smithi151.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-14T16:40:54.368 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-14T16:40:54.368 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-14T16:40:54.368 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-14T16:40:54.368 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-14T16:40:54.368 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-14T16:40:54.368 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-14T16:40:54.368 INFO:teuthology.orchestra.run.smithi151.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-14T16:40:54.369 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-14T16:40:54.369 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-14T16:40:54.369 INFO:teuthology.orchestra.run.smithi151.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-14T16:40:54.369 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-14T16:40:54.369 INFO:teuthology.orchestra.run.smithi151.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-14T16:40:54.369 INFO:teuthology.orchestra.run.smithi151.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-14T16:40:54.369 INFO:teuthology.orchestra.run.smithi151.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-14T16:40:54.369 INFO:teuthology.orchestra.run.smithi151.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-14T16:40:54.369 INFO:teuthology.orchestra.run.smithi151.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-14T16:40:54.369 INFO:teuthology.orchestra.run.smithi151.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-14T16:40:54.369 INFO:teuthology.orchestra.run.smithi151.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-14T16:40:54.369 INFO:teuthology.orchestra.run.smithi151.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-14T16:40:54.370 INFO:teuthology.orchestra.run.smithi151.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-14T16:40:54.370 INFO:teuthology.orchestra.run.smithi151.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-14T16:40:54.370 INFO:teuthology.orchestra.run.smithi151.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-14T16:40:54.370 INFO:teuthology.orchestra.run.smithi151.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-14T16:40:54.370 INFO:teuthology.orchestra.run.smithi151.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-14T16:40:54.370 INFO:teuthology.orchestra.run.smithi151.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-14T16:40:54.370 INFO:teuthology.orchestra.run.smithi151.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-14T16:40:54.370 INFO:teuthology.orchestra.run.smithi151.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-14T16:40:54.370 INFO:teuthology.orchestra.run.smithi151.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-14T16:40:54.370 INFO:teuthology.orchestra.run.smithi151.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-14T16:40:54.370 INFO:teuthology.orchestra.run.smithi151.stdout:Installing weak dependencies: 2024-05-14T16:40:54.370 INFO:teuthology.orchestra.run.smithi151.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-14T16:40:54.370 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-14T16:40:54.370 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction Summary 2024-05-14T16:40:54.371 INFO:teuthology.orchestra.run.smithi151.stdout:======================================================================================= 2024-05-14T16:40:54.371 INFO:teuthology.orchestra.run.smithi151.stdout:Install 148 Packages 2024-05-14T16:40:54.371 INFO:teuthology.orchestra.run.smithi151.stdout:Upgrade 8 Packages 2024-05-14T16:40:54.371 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-14T16:40:54.371 INFO:teuthology.orchestra.run.smithi151.stdout:Total download size: 226 M 2024-05-14T16:40:54.372 INFO:teuthology.orchestra.run.smithi151.stdout:Downloading Packages: 2024-05-14T16:40:54.624 INFO:teuthology.orchestra.run.smithi002.stdout:(1/156): ceph-19.0.0-3687.g9b638668.el9.x86_64. 19 kB/s | 6.6 kB 00:00 2024-05-14T16:40:54.649 INFO:teuthology.orchestra.run.smithi123.stdout:(7/156): ceph-mon-19.0.0-3687.g9b638668.el9.x86 7.1 MB/s | 4.7 MB 00:00 2024-05-14T16:40:55.202 INFO:teuthology.orchestra.run.smithi151.stdout:(1/156): ceph-19.0.0-3687.g9b638668.el9.x86_64. 22 kB/s | 6.6 kB 00:00 2024-05-14T16:40:55.416 INFO:teuthology.orchestra.run.smithi123.stdout:(8/156): ceph-common-19.0.0-3687.g9b638668.el9. 8.0 MB/s | 21 MB 00:02 2024-05-14T16:40:55.500 INFO:teuthology.orchestra.run.smithi123.stdout:(9/156): ceph-selinux-19.0.0-3687.g9b638668.el9 302 kB/s | 25 kB 00:00 2024-05-14T16:40:55.686 INFO:teuthology.orchestra.run.smithi151.stdout:(2/156): ceph-fuse-19.0.0-3687.g9b638668.el9.x8 1.8 MB/s | 891 kB 00:00 2024-05-14T16:40:55.769 INFO:teuthology.orchestra.run.smithi151.stdout:(3/156): ceph-immutable-object-cache-19.0.0-368 1.7 MB/s | 145 kB 00:00 2024-05-14T16:40:55.791 INFO:teuthology.orchestra.run.smithi002.stdout:(2/156): ceph-fuse-19.0.0-3687.g9b638668.el9.x8 764 kB/s | 891 kB 00:01 2024-05-14T16:40:55.875 INFO:teuthology.orchestra.run.smithi123.stdout:(10/156): ceph-osd-19.0.0-3687.g9b638668.el9.x8 9.3 MB/s | 17 MB 00:01 2024-05-14T16:40:55.924 INFO:teuthology.orchestra.run.smithi002.stdout:(3/156): ceph-immutable-object-cache-19.0.0-368 1.1 MB/s | 145 kB 00:00 2024-05-14T16:40:56.077 INFO:teuthology.orchestra.run.smithi123.stdout:(11/156): cephfs-mirror-19.0.0-3687.g9b638668.e 1.1 MB/s | 225 kB 00:00 2024-05-14T16:40:56.159 INFO:teuthology.orchestra.run.smithi123.stdout:(12/156): cephfs-shell-19.0.0-3687.g9b638668.el 331 kB/s | 27 kB 00:00 2024-05-14T16:40:56.284 INFO:teuthology.orchestra.run.smithi123.stdout:(13/156): ceph-radosgw-19.0.0-3687.g9b638668.el 6.5 MB/s | 11 MB 00:01 2024-05-14T16:40:56.301 INFO:teuthology.orchestra.run.smithi123.stdout:(14/156): libcephfs-devel-19.0.0-3687.g9b638668 223 kB/s | 32 kB 00:00 2024-05-14T16:40:56.411 INFO:teuthology.orchestra.run.smithi151.stdout:(4/156): ceph-mds-19.0.0-3687.g9b638668.el9.x86 3.7 MB/s | 2.4 MB 00:00 2024-05-14T16:40:56.426 INFO:teuthology.orchestra.run.smithi123.stdout:(15/156): libcephsqlite-19.0.0-3687.g9b638668.e 1.3 MB/s | 163 kB 00:00 2024-05-14T16:40:56.476 INFO:teuthology.orchestra.run.smithi123.stdout:(16/156): libcephfs2-19.0.0-3687.g9b638668.el9. 3.8 MB/s | 748 kB 00:00 2024-05-14T16:40:56.543 INFO:teuthology.orchestra.run.smithi123.stdout:(17/156): librados-devel-19.0.0-3687.g9b638668. 1.1 MB/s | 127 kB 00:00 2024-05-14T16:40:56.545 INFO:teuthology.orchestra.run.smithi151.stdout:(5/156): ceph-base-19.0.0-3687.g9b638668.el9.x8 3.3 MB/s | 5.4 MB 00:01 2024-05-14T16:40:56.627 INFO:teuthology.orchestra.run.smithi123.stdout:(18/156): libradosstriper1-19.0.0-3687.g9b63866 3.3 MB/s | 501 kB 00:00 2024-05-14T16:40:56.727 INFO:teuthology.orchestra.run.smithi123.stdout:(19/156): python3-ceph-argparse-19.0.0-3687.g9b 451 kB/s | 45 kB 00:00 2024-05-14T16:40:56.762 INFO:teuthology.orchestra.run.smithi151.stdout:(6/156): ceph-mgr-19.0.0-3687.g9b638668.el9.x86 4.3 MB/s | 1.5 MB 00:00 2024-05-14T16:40:56.827 INFO:teuthology.orchestra.run.smithi123.stdout:(20/156): python3-ceph-common-19.0.0-3687.g9b63 1.3 MB/s | 136 kB 00:00 2024-05-14T16:40:56.944 INFO:teuthology.orchestra.run.smithi123.stdout:(21/156): python3-cephfs-19.0.0-3687.g9b638668. 1.4 MB/s | 161 kB 00:00 2024-05-14T16:40:57.070 INFO:teuthology.orchestra.run.smithi123.stdout:(22/156): python3-rados-19.0.0-3687.g9b638668.e 2.5 MB/s | 321 kB 00:00 2024-05-14T16:40:57.187 INFO:teuthology.orchestra.run.smithi123.stdout:(23/156): python3-rbd-19.0.0-3687.g9b638668.el9 2.5 MB/s | 299 kB 00:00 2024-05-14T16:40:57.288 INFO:teuthology.orchestra.run.smithi123.stdout:(24/156): python3-rgw-19.0.0-3687.g9b638668.el9 997 kB/s | 100 kB 00:00 2024-05-14T16:40:57.387 INFO:teuthology.orchestra.run.smithi123.stdout:(25/156): rbd-fuse-19.0.0-3687.g9b638668.el9.x8 855 kB/s | 85 kB 00:00 2024-05-14T16:40:57.420 INFO:teuthology.orchestra.run.smithi151.stdout:(7/156): ceph-mon-19.0.0-3687.g9b638668.el9.x86 5.4 MB/s | 4.7 MB 00:00 2024-05-14T16:40:58.094 INFO:teuthology.orchestra.run.smithi002.stdout:(4/156): ceph-mds-19.0.0-3687.g9b638668.el9.x86 1.1 MB/s | 2.4 MB 00:02 2024-05-14T16:40:58.154 INFO:teuthology.orchestra.run.smithi123.stdout:(26/156): rbd-mirror-19.0.0-3687.g9b638668.el9. 4.0 MB/s | 3.1 MB 00:00 2024-05-14T16:40:58.255 INFO:teuthology.orchestra.run.smithi123.stdout:(27/156): rbd-nbd-19.0.0-3687.g9b638668.el9.x86 1.7 MB/s | 170 kB 00:00 2024-05-14T16:40:58.347 INFO:teuthology.orchestra.run.smithi123.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3687.g 373 kB/s | 34 kB 00:00 2024-05-14T16:40:58.361 INFO:teuthology.orchestra.run.smithi002.stdout:(5/156): ceph-base-19.0.0-3687.g9b638668.el9.x8 1.3 MB/s | 5.4 MB 00:04 2024-05-14T16:40:58.447 INFO:teuthology.orchestra.run.smithi123.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3687.g9b63866 1.4 MB/s | 143 kB 00:00 2024-05-14T16:40:58.747 INFO:teuthology.orchestra.run.smithi123.stdout:(30/156): librgw2-19.0.0-3687.g9b638668.el9.x86 2.4 MB/s | 5.4 MB 00:02 2024-05-14T16:40:59.344 INFO:teuthology.orchestra.run.smithi002.stdout:(6/156): ceph-mgr-19.0.0-3687.g9b638668.el9.x86 1.2 MB/s | 1.5 MB 00:01 2024-05-14T16:40:59.389 INFO:teuthology.orchestra.run.smithi123.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3687.g9b638 3.8 MB/s | 3.5 MB 00:00 2024-05-14T16:40:59.506 INFO:teuthology.orchestra.run.smithi123.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3687.g9b 2.3 MB/s | 274 kB 00:00 2024-05-14T16:40:59.615 INFO:teuthology.orchestra.run.smithi123.stdout:(33/156): ceph-mgr-rook-19.0.0-3687.g9b638668.e 456 kB/s | 49 kB 00:00 2024-05-14T16:40:59.725 INFO:teuthology.orchestra.run.smithi123.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3687.g9 152 kB/s | 16 kB 00:00 2024-05-14T16:40:59.840 INFO:teuthology.orchestra.run.smithi123.stdout:(35/156): ceph-volume-19.0.0-3687.g9b638668.el9 2.3 MB/s | 273 kB 00:00 2024-05-14T16:41:00.079 INFO:teuthology.orchestra.run.smithi151.stdout:(8/156): ceph-radosgw-19.0.0-3687.g9b638668.el9 4.0 MB/s | 11 MB 00:02 2024-05-14T16:41:00.107 INFO:teuthology.orchestra.run.smithi123.stdout:(36/156): cephadm-19.0.0-3687.g9b638668.el9.noa 2.8 MB/s | 763 kB 00:00 2024-05-14T16:41:00.154 INFO:teuthology.orchestra.run.smithi151.stdout:(9/156): ceph-selinux-19.0.0-3687.g9b638668.el9 336 kB/s | 25 kB 00:00 2024-05-14T16:41:00.191 INFO:teuthology.orchestra.run.smithi123.stdout:(37/156): cephfs-top-19.0.0-3687.g9b638668.el9. 219 kB/s | 18 kB 00:00 2024-05-14T16:41:00.253 INFO:teuthology.orchestra.run.smithi002.stdout:(7/156): ceph-common-19.0.0-3687.g9b638668.el9. 3.5 MB/s | 21 MB 00:05 2024-05-14T16:41:00.341 INFO:teuthology.orchestra.run.smithi123.stdout:(38/156): fuse-2.9.9-15.el9.x86_64.rpm 536 kB/s | 80 kB 00:00 2024-05-14T16:41:00.541 INFO:teuthology.orchestra.run.smithi123.stdout:(39/156): libgfortran-11.4.1-3.el9.x86_64.rpm 3.9 MB/s | 803 kB 00:00 2024-05-14T16:41:00.600 INFO:teuthology.orchestra.run.smithi123.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2024-05-14T16:41:00.675 INFO:teuthology.orchestra.run.smithi123.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.3 MB/s | 253 kB 00:00 2024-05-14T16:41:00.729 INFO:teuthology.orchestra.run.smithi151.stdout:(10/156): ceph-common-19.0.0-3687.g9b638668.el9 3.6 MB/s | 21 MB 00:05 2024-05-14T16:41:00.859 INFO:teuthology.orchestra.run.smithi123.stdout:(42/156): python3-cryptography-36.0.1-4.el9.x86 6.7 MB/s | 1.2 MB 00:00 2024-05-14T16:41:00.930 INFO:teuthology.orchestra.run.smithi151.stdout:(11/156): cephfs-mirror-19.0.0-3687.g9b638668.e 1.1 MB/s | 225 kB 00:00 2024-05-14T16:41:00.951 INFO:teuthology.orchestra.run.smithi123.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-05-14T16:41:01.026 INFO:teuthology.orchestra.run.smithi123.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-05-14T16:41:01.037 INFO:teuthology.orchestra.run.smithi002.stdout:(8/156): ceph-mon-19.0.0-3687.g9b638668.el9.x86 1.8 MB/s | 4.7 MB 00:02 2024-05-14T16:41:01.095 INFO:teuthology.orchestra.run.smithi002.stdout:(9/156): ceph-selinux-19.0.0-3687.g9b638668.el9 434 kB/s | 25 kB 00:00 2024-05-14T16:41:01.101 INFO:teuthology.orchestra.run.smithi123.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-05-14T16:41:01.146 INFO:teuthology.orchestra.run.smithi151.stdout:(12/156): ceph-osd-19.0.0-3687.g9b638668.el9.x8 3.8 MB/s | 17 MB 00:04 2024-05-14T16:41:01.172 INFO:teuthology.orchestra.run.smithi151.stdout:(13/156): cephfs-shell-19.0.0-3687.g9b638668.el 112 kB/s | 27 kB 00:00 2024-05-14T16:41:01.185 INFO:teuthology.orchestra.run.smithi123.stdout:(46/156): ceph-mgr-diskprediction-local-19.0.0- 3.0 MB/s | 7.4 MB 00:02 2024-05-14T16:41:01.205 INFO:teuthology.orchestra.run.smithi151.stdout:(14/156): libcephfs-devel-19.0.0-3687.g9b638668 542 kB/s | 32 kB 00:00 2024-05-14T16:41:01.210 INFO:teuthology.orchestra.run.smithi123.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-05-14T16:41:01.272 INFO:teuthology.orchestra.run.smithi151.stdout:(15/156): libcephsqlite-19.0.0-3687.g9b638668.e 2.4 MB/s | 163 kB 00:00 2024-05-14T16:41:01.435 INFO:teuthology.orchestra.run.smithi151.stdout:(16/156): librados-devel-19.0.0-3687.g9b638668. 781 kB/s | 127 kB 00:00 2024-05-14T16:41:01.444 INFO:teuthology.orchestra.run.smithi123.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 2.6 MB/s | 689 kB 00:00 2024-05-14T16:41:01.473 INFO:teuthology.orchestra.run.smithi151.stdout:(17/156): libcephfs2-19.0.0-3687.g9b638668.el9. 2.4 MB/s | 748 kB 00:00 2024-05-14T16:41:01.477 INFO:teuthology.orchestra.run.smithi123.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 3.5 MB/s | 943 kB 00:00 2024-05-14T16:41:01.503 INFO:teuthology.orchestra.run.smithi123.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-05-14T16:41:01.528 INFO:teuthology.orchestra.run.smithi123.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 429 kB/s | 21 kB 00:00 2024-05-14T16:41:01.562 INFO:teuthology.orchestra.run.smithi123.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-05-14T16:41:01.573 INFO:teuthology.orchestra.run.smithi151.stdout:(18/156): libradosstriper1-19.0.0-3687.g9b63866 3.5 MB/s | 501 kB 00:00 2024-05-14T16:41:01.587 INFO:teuthology.orchestra.run.smithi123.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 559 kB/s | 33 kB 00:00 2024-05-14T16:41:01.637 INFO:teuthology.orchestra.run.smithi123.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-05-14T16:41:01.657 INFO:teuthology.orchestra.run.smithi151.stdout:(19/156): python3-ceph-argparse-19.0.0-3687.g9b 542 kB/s | 45 kB 00:00 2024-05-14T16:41:01.688 INFO:teuthology.orchestra.run.smithi123.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-05-14T16:41:01.738 INFO:teuthology.orchestra.run.smithi123.stdout:(56/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 17 MB/s | 3.0 MB 00:00 2024-05-14T16:41:01.740 INFO:teuthology.orchestra.run.smithi151.stdout:(20/156): python3-ceph-common-19.0.0-3687.g9b63 1.6 MB/s | 136 kB 00:00 2024-05-14T16:41:01.763 INFO:teuthology.orchestra.run.smithi123.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-05-14T16:41:01.798 INFO:teuthology.orchestra.run.smithi123.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 771 kB/s | 45 kB 00:00 2024-05-14T16:41:01.824 INFO:teuthology.orchestra.run.smithi151.stdout:(21/156): python3-cephfs-19.0.0-3687.g9b638668. 1.9 MB/s | 161 kB 00:00 2024-05-14T16:41:01.831 INFO:teuthology.orchestra.run.smithi123.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 9.6 MB/s | 662 kB 00:00 2024-05-14T16:41:01.873 INFO:teuthology.orchestra.run.smithi123.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 582 kB 00:00 2024-05-14T16:41:01.898 INFO:teuthology.orchestra.run.smithi123.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 381 kB/s | 25 kB 00:00 2024-05-14T16:41:01.916 INFO:teuthology.orchestra.run.smithi151.stdout:(22/156): python3-rados-19.0.0-3687.g9b638668.e 3.4 MB/s | 321 kB 00:00 2024-05-14T16:41:01.923 INFO:teuthology.orchestra.run.smithi123.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-05-14T16:41:01.929 INFO:teuthology.orchestra.run.smithi002.stdout:(10/156): ceph-radosgw-19.0.0-3687.g9b638668.el 6.3 MB/s | 11 MB 00:01 2024-05-14T16:41:01.948 INFO:teuthology.orchestra.run.smithi123.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-05-14T16:41:01.974 INFO:teuthology.orchestra.run.smithi123.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-05-14T16:41:01.999 INFO:teuthology.orchestra.run.smithi123.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 701 kB/s | 35 kB 00:00 2024-05-14T16:41:02.017 INFO:teuthology.orchestra.run.smithi151.stdout:(23/156): python3-rbd-19.0.0-3687.g9b638668.el9 2.9 MB/s | 299 kB 00:00 2024-05-14T16:41:02.020 INFO:teuthology.orchestra.run.smithi002.stdout:(11/156): cephfs-mirror-19.0.0-3687.g9b638668.e 2.4 MB/s | 225 kB 00:00 2024-05-14T16:41:02.075 INFO:teuthology.orchestra.run.smithi151.stdout:(24/156): python3-rgw-19.0.0-3687.g9b638668.el9 1.7 MB/s | 100 kB 00:00 2024-05-14T16:41:02.129 INFO:teuthology.orchestra.run.smithi002.stdout:(12/156): cephfs-shell-19.0.0-3687.g9b638668.el 250 kB/s | 27 kB 00:00 2024-05-14T16:41:02.134 INFO:teuthology.orchestra.run.smithi151.stdout:(25/156): rbd-fuse-19.0.0-3687.g9b638668.el9.x8 1.4 MB/s | 85 kB 00:00 2024-05-14T16:41:02.158 INFO:teuthology.orchestra.run.smithi123.stdout:(66/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 28 MB/s | 4.4 MB 00:00 2024-05-14T16:41:02.204 INFO:teuthology.orchestra.run.smithi002.stdout:(13/156): libcephfs-devel-19.0.0-3687.g9b638668 423 kB/s | 32 kB 00:00 2024-05-14T16:41:02.241 INFO:teuthology.orchestra.run.smithi123.stdout:(67/156): openblas-openmp-0.3.21-2.el9.x86_64.r 18 MB/s | 4.7 MB 00:00 2024-05-14T16:41:02.283 INFO:teuthology.orchestra.run.smithi123.stdout:(68/156): postgresql-13.14-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-05-14T16:41:02.317 INFO:teuthology.orchestra.run.smithi123.stdout:(69/156): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-05-14T16:41:02.342 INFO:teuthology.orchestra.run.smithi123.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-05-14T16:41:02.392 INFO:teuthology.orchestra.run.smithi151.stdout:(26/156): librgw2-19.0.0-3687.g9b638668.el9.x86 5.8 MB/s | 5.4 MB 00:00 2024-05-14T16:41:02.396 INFO:teuthology.orchestra.run.smithi002.stdout:(14/156): libcephfs2-19.0.0-3687.g9b638668.el9. 3.8 MB/s | 748 kB 00:00 2024-05-14T16:41:02.417 INFO:teuthology.orchestra.run.smithi123.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-05-14T16:41:02.476 INFO:teuthology.orchestra.run.smithi123.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-05-14T16:41:02.476 INFO:teuthology.orchestra.run.smithi151.stdout:(27/156): rbd-nbd-19.0.0-3687.g9b638668.el9.x86 2.0 MB/s | 170 kB 00:00 2024-05-14T16:41:02.505 INFO:teuthology.orchestra.run.smithi002.stdout:(15/156): libcephsqlite-19.0.0-3687.g9b638668.e 1.5 MB/s | 163 kB 00:00 2024-05-14T16:41:02.576 INFO:teuthology.orchestra.run.smithi151.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3687.g 342 kB/s | 34 kB 00:00 2024-05-14T16:41:02.630 INFO:teuthology.orchestra.run.smithi002.stdout:(16/156): librados-devel-19.0.0-3687.g9b638668. 1.0 MB/s | 127 kB 00:00 2024-05-14T16:41:02.668 INFO:teuthology.orchestra.run.smithi151.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3687.g9b63866 1.5 MB/s | 143 kB 00:00 2024-05-14T16:41:02.701 INFO:teuthology.orchestra.run.smithi123.stdout:(73/156): python3-babel-2.9.1-2.el9.noarch.rpm 27 MB/s | 6.0 MB 00:00 2024-05-14T16:41:02.727 INFO:teuthology.orchestra.run.smithi151.stdout:(30/156): rbd-mirror-19.0.0-3687.g9b638668.el9. 5.2 MB/s | 3.1 MB 00:00 2024-05-14T16:41:02.768 INFO:teuthology.orchestra.run.smithi123.stdout:(74/156): postgresql-server-13.14-1.el9.x86_64. 13 MB/s | 5.8 MB 00:00 2024-05-14T16:41:02.772 INFO:teuthology.orchestra.run.smithi002.stdout:(17/156): libradosstriper1-19.0.0-3687.g9b63866 3.5 MB/s | 501 kB 00:00 2024-05-14T16:41:02.793 INFO:teuthology.orchestra.run.smithi123.stdout:(75/156): python3-devel-3.9.18-3.el9.x86_64.rpm 2.6 MB/s | 244 kB 00:00 2024-05-14T16:41:02.819 INFO:teuthology.orchestra.run.smithi123.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-05-14T16:41:02.836 INFO:teuthology.orchestra.run.smithi123.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 611 kB/s | 26 kB 00:00 2024-05-14T16:41:02.861 INFO:teuthology.orchestra.run.smithi123.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 460 kB/s | 19 kB 00:00 2024-05-14T16:41:02.886 INFO:teuthology.orchestra.run.smithi123.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-05-14T16:41:02.911 INFO:teuthology.orchestra.run.smithi123.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 690 kB/s | 35 kB 00:00 2024-05-14T16:41:02.987 INFO:teuthology.orchestra.run.smithi123.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-05-14T16:41:03.062 INFO:teuthology.orchestra.run.smithi123.stdout:(82/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 29 MB/s | 5.1 MB 00:00 2024-05-14T16:41:03.087 INFO:teuthology.orchestra.run.smithi123.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-05-14T16:41:03.113 INFO:teuthology.orchestra.run.smithi123.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-14T16:41:03.138 INFO:teuthology.orchestra.run.smithi123.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 839 kB/s | 42 kB 00:00 2024-05-14T16:41:03.172 INFO:teuthology.orchestra.run.smithi123.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-05-14T16:41:03.206 INFO:teuthology.orchestra.run.smithi123.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-05-14T16:41:03.231 INFO:teuthology.orchestra.run.smithi123.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.6 MB/s | 279 kB 00:00 2024-05-14T16:41:03.256 INFO:teuthology.orchestra.run.smithi123.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-14T16:41:03.281 INFO:teuthology.orchestra.run.smithi123.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-14T16:41:03.332 INFO:teuthology.orchestra.run.smithi123.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 834 kB/s | 42 kB 00:00 2024-05-14T16:41:03.361 INFO:teuthology.orchestra.run.smithi151.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3687.g9b638 5.1 MB/s | 3.5 MB 00:00 2024-05-14T16:41:03.382 INFO:teuthology.orchestra.run.smithi123.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 881 kB/s | 44 kB 00:00 2024-05-14T16:41:03.432 INFO:teuthology.orchestra.run.smithi123.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-05-14T16:41:03.461 INFO:teuthology.orchestra.run.smithi151.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3687.g9b 2.7 MB/s | 274 kB 00:00 2024-05-14T16:41:03.482 INFO:teuthology.orchestra.run.smithi123.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 718 kB/s | 36 kB 00:00 2024-05-14T16:41:03.533 INFO:teuthology.orchestra.run.smithi123.stdout:(95/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-05-14T16:41:03.561 INFO:teuthology.orchestra.run.smithi151.stdout:(33/156): ceph-mgr-rook-19.0.0-3687.g9b638668.e 494 kB/s | 49 kB 00:00 2024-05-14T16:41:03.592 INFO:teuthology.orchestra.run.smithi123.stdout:(96/156): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 305 kB 00:00 2024-05-14T16:41:03.642 INFO:teuthology.orchestra.run.smithi123.stdout:(97/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-05-14T16:41:03.678 INFO:teuthology.orchestra.run.smithi151.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3687.g9 141 kB/s | 16 kB 00:00 2024-05-14T16:41:03.681 INFO:teuthology.orchestra.run.smithi002.stdout:(18/156): librgw2-19.0.0-3687.g9b638668.el9.x86 5.9 MB/s | 5.4 MB 00:00 2024-05-14T16:41:03.692 INFO:teuthology.orchestra.run.smithi123.stdout:(98/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-05-14T16:41:03.742 INFO:teuthology.orchestra.run.smithi123.stdout:(99/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-14T16:41:03.753 INFO:teuthology.orchestra.run.smithi151.stdout:(35/156): ceph-volume-19.0.0-3687.g9b638668.el9 3.6 MB/s | 273 kB 00:00 2024-05-14T16:41:03.764 INFO:teuthology.orchestra.run.smithi002.stdout:(19/156): python3-ceph-argparse-19.0.0-3687.g9b 542 kB/s | 45 kB 00:00 2024-05-14T16:41:03.793 INFO:teuthology.orchestra.run.smithi123.stdout:(100/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-05-14T16:41:03.857 INFO:teuthology.orchestra.run.smithi002.stdout:(20/156): python3-ceph-common-19.0.0-3687.g9b63 1.4 MB/s | 136 kB 00:00 2024-05-14T16:41:03.920 INFO:teuthology.orchestra.run.smithi151.stdout:(36/156): cephadm-19.0.0-3687.g9b638668.el9.noa 4.5 MB/s | 763 kB 00:00 2024-05-14T16:41:03.940 INFO:teuthology.orchestra.run.smithi002.stdout:(21/156): python3-cephfs-19.0.0-3687.g9b638668. 1.9 MB/s | 161 kB 00:00 2024-05-14T16:41:03.995 INFO:teuthology.orchestra.run.smithi151.stdout:(37/156): cephfs-top-19.0.0-3687.g9b638668.el9. 243 kB/s | 18 kB 00:00 2024-05-14T16:41:04.001 INFO:teuthology.orchestra.run.smithi123.stdout:(101/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-05-14T16:41:04.029 INFO:teuthology.orchestra.run.smithi123.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 95 kB/s | 22 kB 00:00 2024-05-14T16:41:04.032 INFO:teuthology.orchestra.run.smithi002.stdout:(22/156): python3-rados-19.0.0-3687.g9b638668.e 3.4 MB/s | 321 kB 00:00 2024-05-14T16:41:04.124 INFO:teuthology.orchestra.run.smithi002.stdout:(23/156): python3-rbd-19.0.0-3687.g9b638668.el9 3.2 MB/s | 299 kB 00:00 2024-05-14T16:41:04.168 INFO:teuthology.orchestra.run.smithi123.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 5.1 MB/s | 863 kB 00:00 2024-05-14T16:41:04.199 INFO:teuthology.orchestra.run.smithi002.stdout:(24/156): python3-rgw-19.0.0-3687.g9b638668.el9 1.3 MB/s | 100 kB 00:00 2024-05-14T16:41:04.212 INFO:teuthology.orchestra.run.smithi151.stdout:(38/156): ceph-mgr-diskprediction-local-19.0.0- 5.0 MB/s | 7.4 MB 00:01 2024-05-14T16:41:04.266 INFO:teuthology.orchestra.run.smithi002.stdout:(25/156): rbd-fuse-19.0.0-3687.g9b638668.el9.x8 1.3 MB/s | 85 kB 00:00 2024-05-14T16:41:04.421 INFO:teuthology.orchestra.run.smithi151.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 189 kB/s | 80 kB 00:00 2024-05-14T16:41:04.688 INFO:teuthology.orchestra.run.smithi151.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 733 kB/s | 195 kB 00:00 2024-05-14T16:41:04.717 INFO:teuthology.orchestra.run.smithi002.stdout:(26/156): rbd-mirror-19.0.0-3687.g9b638668.el9. 6.9 MB/s | 3.1 MB 00:00 2024-05-14T16:41:04.792 INFO:teuthology.orchestra.run.smithi002.stdout:(27/156): rbd-nbd-19.0.0-3687.g9b638668.el9.x86 2.2 MB/s | 170 kB 00:00 2024-05-14T16:41:04.796 INFO:teuthology.orchestra.run.smithi151.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2024-05-14T16:41:04.867 INFO:teuthology.orchestra.run.smithi002.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3687.g 456 kB/s | 34 kB 00:00 2024-05-14T16:41:04.943 INFO:teuthology.orchestra.run.smithi002.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3687.g9b63866 1.9 MB/s | 143 kB 00:00 2024-05-14T16:41:04.989 INFO:teuthology.orchestra.run.smithi151.stdout:(42/156): libgfortran-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 803 kB 00:00 2024-05-14T16:41:05.027 INFO:teuthology.orchestra.run.smithi123.stdout:(104/156): gperftools-libs-2.9.1-2.el9.x86_64.r 360 kB/s | 309 kB 00:00 2024-05-14T16:41:05.114 INFO:teuthology.orchestra.run.smithi151.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 3.9 MB/s | 1.2 MB 00:00 2024-05-14T16:41:05.136 INFO:teuthology.orchestra.run.smithi123.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 494 kB/s | 548 kB 00:01 2024-05-14T16:41:05.139 INFO:teuthology.orchestra.run.smithi151.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 709 kB/s | 106 kB 00:00 2024-05-14T16:41:05.160 INFO:teuthology.orchestra.run.smithi123.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 146 kB/s | 19 kB 00:00 2024-05-14T16:41:05.223 INFO:teuthology.orchestra.run.smithi151.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-05-14T16:41:05.248 INFO:teuthology.orchestra.run.smithi151.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-05-14T16:41:05.269 INFO:teuthology.orchestra.run.smithi123.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 240 kB/s | 26 kB 00:00 2024-05-14T16:41:05.332 INFO:teuthology.orchestra.run.smithi151.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-05-14T16:41:05.334 INFO:teuthology.orchestra.run.smithi002.stdout:(30/156): ceph-osd-19.0.0-3687.g9b638668.el9.x8 2.8 MB/s | 17 MB 00:05 2024-05-14T16:41:05.443 INFO:teuthology.orchestra.run.smithi002.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3687.g9b638 7.1 MB/s | 3.5 MB 00:00 2024-05-14T16:41:05.448 INFO:teuthology.orchestra.run.smithi151.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 689 kB 00:00 2024-05-14T16:41:05.499 INFO:teuthology.orchestra.run.smithi151.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-05-14T16:41:05.510 INFO:teuthology.orchestra.run.smithi002.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3687.g9b 4.0 MB/s | 274 kB 00:00 2024-05-14T16:41:05.541 INFO:teuthology.orchestra.run.smithi151.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 4.4 MB/s | 943 kB 00:00 2024-05-14T16:41:05.566 INFO:teuthology.orchestra.run.smithi151.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 321 kB/s | 21 kB 00:00 2024-05-14T16:41:05.577 INFO:teuthology.orchestra.run.smithi002.stdout:(33/156): ceph-mgr-rook-19.0.0-3687.g9b638668.e 741 kB/s | 49 kB 00:00 2024-05-14T16:41:05.592 INFO:teuthology.orchestra.run.smithi151.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-05-14T16:41:05.617 INFO:teuthology.orchestra.run.smithi151.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 651 kB/s | 33 kB 00:00 2024-05-14T16:41:05.644 INFO:teuthology.orchestra.run.smithi002.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3687.g9 247 kB/s | 16 kB 00:00 2024-05-14T16:41:05.667 INFO:teuthology.orchestra.run.smithi151.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 366 kB/s | 18 kB 00:00 2024-05-14T16:41:05.678 INFO:teuthology.orchestra.run.smithi123.stdout:(108/156): ceph-test-19.0.0-3687.g9b638668.el9. 4.8 MB/s | 49 MB 00:10 2024-05-14T16:41:05.703 INFO:teuthology.orchestra.run.smithi123.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 112 kB/s | 49 kB 00:00 2024-05-14T16:41:05.719 INFO:teuthology.orchestra.run.smithi002.stdout:(35/156): ceph-volume-19.0.0-3687.g9b638668.el9 3.6 MB/s | 273 kB 00:00 2024-05-14T16:41:05.734 INFO:teuthology.orchestra.run.smithi151.stdout:(55/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 21 MB/s | 3.0 MB 00:00 2024-05-14T16:41:05.751 INFO:teuthology.orchestra.run.smithi151.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 133 kB/s | 11 kB 00:00 2024-05-14T16:41:05.785 INFO:teuthology.orchestra.run.smithi151.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-05-14T16:41:05.802 INFO:teuthology.orchestra.run.smithi151.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 902 kB/s | 45 kB 00:00 2024-05-14T16:41:05.836 INFO:teuthology.orchestra.run.smithi151.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-05-14T16:41:05.877 INFO:teuthology.orchestra.run.smithi151.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 582 kB 00:00 2024-05-14T16:41:05.886 INFO:teuthology.orchestra.run.smithi002.stdout:(36/156): cephadm-19.0.0-3687.g9b638668.el9.noa 4.5 MB/s | 763 kB 00:00 2024-05-14T16:41:05.903 INFO:teuthology.orchestra.run.smithi151.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 381 kB/s | 25 kB 00:00 2024-05-14T16:41:05.928 INFO:teuthology.orchestra.run.smithi151.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-05-14T16:41:05.945 INFO:teuthology.orchestra.run.smithi002.stdout:(37/156): cephfs-top-19.0.0-3687.g9b638668.el9. 312 kB/s | 18 kB 00:00 2024-05-14T16:41:05.970 INFO:teuthology.orchestra.run.smithi151.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-05-14T16:41:05.970 INFO:teuthology.orchestra.run.smithi123.stdout:(110/156): luarocks-3.9.2-1.el9.noarch.rpm 566 kB/s | 151 kB 00:00 2024-05-14T16:41:06.062 INFO:teuthology.orchestra.run.smithi151.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-05-14T16:41:06.087 INFO:teuthology.orchestra.run.smithi002.stdout:(38/156): fuse-2.9.9-15.el9.x86_64.rpm 569 kB/s | 80 kB 00:00 2024-05-14T16:41:06.087 INFO:teuthology.orchestra.run.smithi151.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 301 kB/s | 35 kB 00:00 2024-05-14T16:41:06.237 INFO:teuthology.orchestra.run.smithi123.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 121 kB/s | 67 kB 00:00 2024-05-14T16:41:06.263 INFO:teuthology.orchestra.run.smithi151.stdout:(66/156): openblas-openmp-0.3.21-2.el9.x86_64.r 24 MB/s | 4.7 MB 00:00 2024-05-14T16:41:06.330 INFO:teuthology.orchestra.run.smithi151.stdout:(67/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-05-14T16:41:06.345 INFO:teuthology.orchestra.run.smithi002.stdout:(39/156): libgfortran-11.4.1-3.el9.x86_64.rpm 3.0 MB/s | 803 kB 00:00 2024-05-14T16:41:06.372 INFO:teuthology.orchestra.run.smithi151.stdout:(68/156): postgresql-13.14-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-05-14T16:41:06.413 INFO:teuthology.orchestra.run.smithi151.stdout:(69/156): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 911 kB 00:00 2024-05-14T16:41:06.437 INFO:teuthology.orchestra.run.smithi002.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2024-05-14T16:41:06.439 INFO:teuthology.orchestra.run.smithi151.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-05-14T16:41:06.521 INFO:teuthology.orchestra.run.smithi002.stdout:(41/156): ceph-mgr-diskprediction-local-19.0.0- 6.2 MB/s | 7.4 MB 00:01 2024-05-14T16:41:06.556 INFO:teuthology.orchestra.run.smithi151.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 8.6 MB/s | 1.0 MB 00:00 2024-05-14T16:41:06.588 INFO:teuthology.orchestra.run.smithi002.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-05-14T16:41:06.638 INFO:teuthology.orchestra.run.smithi002.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-05-14T16:41:06.664 INFO:teuthology.orchestra.run.smithi151.stdout:(72/156): postgresql-server-13.14-1.el9.x86_64. 23 MB/s | 5.8 MB 00:00 2024-05-14T16:41:06.690 INFO:teuthology.orchestra.run.smithi151.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 655 kB/s | 87 kB 00:00 2024-05-14T16:41:06.722 INFO:teuthology.orchestra.run.smithi002.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-05-14T16:41:06.814 INFO:teuthology.orchestra.run.smithi002.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-05-14T16:41:06.898 INFO:teuthology.orchestra.run.smithi151.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 1.1 MB/s | 244 kB 00:00 2024-05-14T16:41:06.906 INFO:teuthology.orchestra.run.smithi002.stdout:(46/156): python3-urllib3-1.26.5-5.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-05-14T16:41:06.929 INFO:teuthology.orchestra.run.smithi123.stdout:(112/156): python3-asyncssh-2.13.2-5.el9.noarch 792 kB/s | 548 kB 00:00 2024-05-14T16:41:06.989 INFO:teuthology.orchestra.run.smithi002.stdout:(47/156): python3-cryptography-36.0.1-4.el9.x86 2.6 MB/s | 1.2 MB 00:00 2024-05-14T16:41:07.029 INFO:teuthology.orchestra.run.smithi123.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 435 kB/s | 43 kB 00:00 2024-05-14T16:41:07.065 INFO:teuthology.orchestra.run.smithi151.stdout:(75/156): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-05-14T16:41:07.071 INFO:teuthology.orchestra.run.smithi123.stdout:(114/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 762 kB/s | 839 kB 00:01 2024-05-14T16:41:07.093 INFO:teuthology.orchestra.run.smithi151.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 1.3 MB/s | 248 kB 00:00 2024-05-14T16:41:07.116 INFO:teuthology.orchestra.run.smithi151.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 510 kB/s | 26 kB 00:00 2024-05-14T16:41:07.150 INFO:teuthology.orchestra.run.smithi123.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 349 kB/s | 32 kB 00:00 2024-05-14T16:41:07.150 INFO:teuthology.orchestra.run.smithi151.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-05-14T16:41:07.164 INFO:teuthology.orchestra.run.smithi123.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 163 kB/s | 15 kB 00:00 2024-05-14T16:41:07.166 INFO:teuthology.orchestra.run.smithi151.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-05-14T16:41:07.192 INFO:teuthology.orchestra.run.smithi151.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-05-14T16:41:07.242 INFO:teuthology.orchestra.run.smithi151.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-05-14T16:41:07.257 INFO:teuthology.orchestra.run.smithi002.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 1.9 MB/s | 689 kB 00:00 2024-05-14T16:41:07.323 INFO:teuthology.orchestra.run.smithi002.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 2.8 MB/s | 943 kB 00:00 2024-05-14T16:41:07.349 INFO:teuthology.orchestra.run.smithi002.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 107 kB 00:00 2024-05-14T16:41:07.368 INFO:teuthology.orchestra.run.smithi151.stdout:(82/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.7 MB/s | 222 kB 00:00 2024-05-14T16:41:07.383 INFO:teuthology.orchestra.run.smithi002.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 366 kB/s | 21 kB 00:00 2024-05-14T16:41:07.406 INFO:teuthology.orchestra.run.smithi123.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 606 kB/s | 172 kB 00:00 2024-05-14T16:41:07.426 INFO:teuthology.orchestra.run.smithi151.stdout:(83/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-05-14T16:41:07.441 INFO:teuthology.orchestra.run.smithi002.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 313 kB 00:00 2024-05-14T16:41:07.466 INFO:teuthology.orchestra.run.smithi002.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 391 kB/s | 33 kB 00:00 2024-05-14T16:41:07.501 INFO:teuthology.orchestra.run.smithi151.stdout:(84/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-05-14T16:41:07.525 INFO:teuthology.orchestra.run.smithi002.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 313 kB/s | 18 kB 00:00 2024-05-14T16:41:07.527 INFO:teuthology.orchestra.run.smithi151.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 421 kB/s | 42 kB 00:00 2024-05-14T16:41:07.560 INFO:teuthology.orchestra.run.smithi151.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-05-14T16:41:07.583 INFO:teuthology.orchestra.run.smithi002.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 191 kB/s | 11 kB 00:00 2024-05-14T16:41:07.586 INFO:teuthology.orchestra.run.smithi151.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-05-14T16:41:07.642 INFO:teuthology.orchestra.run.smithi002.stdout:(56/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-05-14T16:41:07.644 INFO:teuthology.orchestra.run.smithi151.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.3 MB/s | 279 kB 00:00 2024-05-14T16:41:07.661 INFO:teuthology.orchestra.run.smithi151.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 678 kB/s | 51 kB 00:00 2024-05-14T16:41:07.678 INFO:teuthology.orchestra.run.smithi151.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-05-14T16:41:07.689 INFO:teuthology.orchestra.run.smithi123.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 683 kB/s | 358 kB 00:00 2024-05-14T16:41:07.701 INFO:teuthology.orchestra.run.smithi002.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 780 kB/s | 45 kB 00:00 2024-05-14T16:41:07.779 INFO:teuthology.orchestra.run.smithi151.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 416 kB/s | 42 kB 00:00 2024-05-14T16:41:07.789 INFO:teuthology.orchestra.run.smithi123.stdout:(119/156): python3-colorama-0.4.4-12.el9.noarch 352 kB/s | 35 kB 00:00 2024-05-14T16:41:07.846 INFO:teuthology.orchestra.run.smithi151.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 663 kB/s | 44 kB 00:00 2024-05-14T16:41:07.851 INFO:teuthology.orchestra.run.smithi002.stdout:(58/156): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-05-14T16:41:07.890 INFO:teuthology.orchestra.run.smithi123.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 626 kB/s | 303 kB 00:00 2024-05-14T16:41:07.901 INFO:teuthology.orchestra.run.smithi002.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.5 MB/s | 3.0 MB 00:00 2024-05-14T16:41:07.946 INFO:teuthology.orchestra.run.smithi151.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.2 MB/s | 221 kB 00:00 2024-05-14T16:41:07.968 INFO:teuthology.orchestra.run.smithi002.stdout:(60/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 383 kB/s | 25 kB 00:00 2024-05-14T16:41:07.994 INFO:teuthology.orchestra.run.smithi002.stdout:(61/156): libtool-2.4.6-45.el9.x86_64.rpm 4.0 MB/s | 582 kB 00:00 2024-05-14T16:41:08.027 INFO:teuthology.orchestra.run.smithi002.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 812 kB/s | 47 kB 00:00 2024-05-14T16:41:08.061 INFO:teuthology.orchestra.run.smithi002.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-05-14T16:41:08.086 INFO:teuthology.orchestra.run.smithi002.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-05-14T16:41:08.111 INFO:teuthology.orchestra.run.smithi002.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 700 kB/s | 35 kB 00:00 2024-05-14T16:41:08.157 INFO:teuthology.orchestra.run.smithi123.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 560 kB/s | 206 kB 00:00 2024-05-14T16:41:08.438 INFO:teuthology.orchestra.run.smithi151.stdout:(94/156): ceph-test-19.0.0-3687.g9b638668.el9.x 5.9 MB/s | 49 MB 00:08 2024-05-14T16:41:08.463 INFO:teuthology.orchestra.run.smithi151.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 69 kB/s | 36 kB 00:00 2024-05-14T16:41:08.465 INFO:teuthology.orchestra.run.smithi123.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 469 kB/s | 144 kB 00:00 2024-05-14T16:41:08.513 INFO:teuthology.orchestra.run.smithi151.stdout:(96/156): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 305 kB 00:00 2024-05-14T16:41:08.555 INFO:teuthology.orchestra.run.smithi151.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 406 kB/s | 47 kB 00:00 2024-05-14T16:41:08.562 INFO:teuthology.orchestra.run.smithi002.stdout:(66/156): openblas-openmp-0.3.21-2.el9.x86_64.r 10 MB/s | 4.7 MB 00:00 2024-05-14T16:41:08.581 INFO:teuthology.orchestra.run.smithi151.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 866 kB/s | 58 kB 00:00 2024-05-14T16:41:08.582 INFO:teuthology.orchestra.run.smithi123.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 92 kB/s | 11 kB 00:00 2024-05-14T16:41:08.629 INFO:teuthology.orchestra.run.smithi002.stdout:(67/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 8.6 MB/s | 4.4 MB 00:00 2024-05-14T16:41:08.708 INFO:teuthology.orchestra.run.smithi123.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 141 kB/s | 18 kB 00:00 2024-05-14T16:41:08.765 INFO:teuthology.orchestra.run.smithi151.stdout:(99/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 15 MB/s | 16 MB 00:01 2024-05-14T16:41:08.798 INFO:teuthology.orchestra.run.smithi151.stdout:(100/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 387 kB/s | 94 kB 00:00 2024-05-14T16:41:08.816 INFO:teuthology.orchestra.run.smithi123.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 214 kB/s | 23 kB 00:00 2024-05-14T16:41:08.823 INFO:teuthology.orchestra.run.smithi151.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 263 kB/s | 64 kB 00:00 2024-05-14T16:41:08.848 INFO:teuthology.orchestra.run.smithi151.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 536 kB/s | 45 kB 00:00 2024-05-14T16:41:08.925 INFO:teuthology.orchestra.run.smithi123.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 179 kB/s | 19 kB 00:00 2024-05-14T16:41:09.007 INFO:teuthology.orchestra.run.smithi151.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 122 kB/s | 22 kB 00:00 2024-05-14T16:41:09.033 INFO:teuthology.orchestra.run.smithi123.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 182 kB/s | 20 kB 00:00 2024-05-14T16:41:09.071 INFO:teuthology.orchestra.run.smithi002.stdout:(68/156): ceph-test-19.0.0-3687.g9b638668.el9.x 6.2 MB/s | 49 MB 00:07 2024-05-14T16:41:09.112 INFO:teuthology.orchestra.run.smithi002.stdout:(69/156): postgresql-contrib-13.14-1.el9.x86_64 1.8 MB/s | 911 kB 00:00 2024-05-14T16:41:09.155 INFO:teuthology.orchestra.run.smithi002.stdout:(70/156): postgresql-13.14-1.el9.x86_64.rpm 2.7 MB/s | 1.6 MB 00:00 2024-05-14T16:41:09.185 INFO:teuthology.orchestra.run.smithi151.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.6 MB/s | 548 kB 00:00 2024-05-14T16:41:09.243 INFO:teuthology.orchestra.run.smithi151.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 335 kB/s | 19 kB 00:00 2024-05-14T16:41:09.280 INFO:teuthology.orchestra.run.smithi002.stdout:(71/156): postgresql-private-libs-13.14-1.el9.x 639 kB/s | 133 kB 00:00 2024-05-14T16:41:09.313 INFO:teuthology.orchestra.run.smithi002.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 6.4 MB/s | 1.0 MB 00:00 2024-05-14T16:41:09.330 INFO:teuthology.orchestra.run.smithi002.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-05-14T16:41:09.352 INFO:teuthology.orchestra.run.smithi151.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 897 kB/s | 309 kB 00:00 2024-05-14T16:41:09.419 INFO:teuthology.orchestra.run.smithi151.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 390 kB/s | 26 kB 00:00 2024-05-14T16:41:09.430 INFO:teuthology.orchestra.run.smithi002.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 2.4 MB/s | 244 kB 00:00 2024-05-14T16:41:09.477 INFO:teuthology.orchestra.run.smithi151.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 837 kB/s | 49 kB 00:00 2024-05-14T16:41:09.539 INFO:teuthology.orchestra.run.smithi002.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-05-14T16:41:09.544 INFO:teuthology.orchestra.run.smithi151.stdout:(109/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-05-14T16:41:09.606 INFO:teuthology.orchestra.run.smithi002.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 385 kB/s | 26 kB 00:00 2024-05-14T16:41:09.661 INFO:teuthology.orchestra.run.smithi151.stdout:(110/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.0 MB/s | 863 kB 00:00 2024-05-14T16:41:09.682 INFO:teuthology.orchestra.run.smithi002.stdout:(77/156): postgresql-server-13.14-1.el9.x86_64. 10 MB/s | 5.8 MB 00:00 2024-05-14T16:41:09.686 INFO:teuthology.orchestra.run.smithi151.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-05-14T16:41:09.707 INFO:teuthology.orchestra.run.smithi002.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 190 kB/s | 19 kB 00:00 2024-05-14T16:41:09.749 INFO:teuthology.orchestra.run.smithi002.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-05-14T16:41:09.783 INFO:teuthology.orchestra.run.smithi002.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 461 kB/s | 35 kB 00:00 2024-05-14T16:41:09.858 INFO:teuthology.orchestra.run.smithi002.stdout:(81/156): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-05-14T16:41:09.911 INFO:teuthology.orchestra.run.smithi151.stdout:(112/156): libarrow-9.0.0-11.el9.x86_64.rpm 6.6 MB/s | 4.4 MB 00:00 2024-05-14T16:41:09.925 INFO:teuthology.orchestra.run.smithi002.stdout:(82/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-05-14T16:41:09.937 INFO:teuthology.orchestra.run.smithi151.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 2.1 MB/s | 548 kB 00:00 2024-05-14T16:41:09.950 INFO:teuthology.orchestra.run.smithi002.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 250 kB 00:00 2024-05-14T16:41:09.992 INFO:teuthology.orchestra.run.smithi002.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-05-14T16:41:10.004 INFO:teuthology.orchestra.run.smithi151.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 473 kB/s | 43 kB 00:00 2024-05-14T16:41:10.008 INFO:teuthology.orchestra.run.smithi002.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 724 kB/s | 42 kB 00:00 2024-05-14T16:41:10.029 INFO:teuthology.orchestra.run.smithi151.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 350 kB/s | 32 kB 00:00 2024-05-14T16:41:10.051 INFO:teuthology.orchestra.run.smithi002.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-05-14T16:41:10.063 INFO:teuthology.orchestra.run.smithi151.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 256 kB/s | 15 kB 00:00 2024-05-14T16:41:10.076 INFO:teuthology.orchestra.run.smithi002.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-05-14T16:41:10.096 INFO:teuthology.orchestra.run.smithi151.stdout:(117/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 1.9 MB/s | 839 kB 00:00 2024-05-14T16:41:10.138 INFO:teuthology.orchestra.run.smithi151.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-05-14T16:41:10.145 INFO:teuthology.orchestra.run.smithi002.stdout:(88/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-05-14T16:41:10.163 INFO:teuthology.orchestra.run.smithi151.stdout:(119/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-05-14T16:41:10.168 INFO:teuthology.orchestra.run.smithi002.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 555 kB/s | 51 kB 00:00 2024-05-14T16:41:10.189 INFO:teuthology.orchestra.run.smithi151.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-05-14T16:41:10.202 INFO:teuthology.orchestra.run.smithi002.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.8 MB/s | 279 kB 00:00 2024-05-14T16:41:10.214 INFO:teuthology.orchestra.run.smithi151.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 468 kB/s | 35 kB 00:00 2024-05-14T16:41:10.227 INFO:teuthology.orchestra.run.smithi002.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 640 kB/s | 54 kB 00:00 2024-05-14T16:41:10.261 INFO:teuthology.orchestra.run.smithi002.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 714 kB/s | 42 kB 00:00 2024-05-14T16:41:10.281 INFO:teuthology.orchestra.run.smithi151.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 1.7 MB/s | 206 kB 00:00 2024-05-14T16:41:10.294 INFO:teuthology.orchestra.run.smithi002.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 660 kB/s | 44 kB 00:00 2024-05-14T16:41:10.306 INFO:teuthology.orchestra.run.smithi151.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-05-14T16:41:10.320 INFO:teuthology.orchestra.run.smithi002.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-05-14T16:41:10.353 INFO:teuthology.orchestra.run.smithi002.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 614 kB/s | 36 kB 00:00 2024-05-14T16:41:10.365 INFO:teuthology.orchestra.run.smithi151.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 184 kB/s | 11 kB 00:00 2024-05-14T16:41:10.387 INFO:teuthology.orchestra.run.smithi002.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 709 kB/s | 47 kB 00:00 2024-05-14T16:41:10.390 INFO:teuthology.orchestra.run.smithi151.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 212 kB/s | 18 kB 00:00 2024-05-14T16:41:10.417 INFO:teuthology.orchestra.run.smithi123.stdout:(128/156): python3-kubernetes-26.1.0-2.el9.noar 756 kB/s | 1.0 MB 00:01 2024-05-14T16:41:10.420 INFO:teuthology.orchestra.run.smithi002.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 4.5 MB/s | 305 kB 00:00 2024-05-14T16:41:10.432 INFO:teuthology.orchestra.run.smithi151.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 347 kB/s | 23 kB 00:00 2024-05-14T16:41:10.446 INFO:teuthology.orchestra.run.smithi002.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 987 kB/s | 58 kB 00:00 2024-05-14T16:41:10.457 INFO:teuthology.orchestra.run.smithi151.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 291 kB/s | 19 kB 00:00 2024-05-14T16:41:10.479 INFO:teuthology.orchestra.run.smithi002.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-05-14T16:41:10.492 INFO:teuthology.orchestra.run.smithi123.stdout:(129/156): python3-grpcio-1.46.7-10.el9.x86_64. 802 kB/s | 2.0 MB 00:02 2024-05-14T16:41:10.499 INFO:teuthology.orchestra.run.smithi151.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 294 kB/s | 20 kB 00:00 2024-05-14T16:41:10.504 INFO:teuthology.orchestra.run.smithi002.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-05-14T16:41:10.541 INFO:teuthology.orchestra.run.smithi151.stdout:(129/156): python3-grpcio-1.46.7-10.el9.x86_64. 5.8 MB/s | 2.0 MB 00:00 2024-05-14T16:41:10.542 INFO:teuthology.orchestra.run.smithi123.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 370 kB/s | 46 kB 00:00 2024-05-14T16:41:10.547 INFO:teuthology.orchestra.run.smithi002.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 669 kB/s | 45 kB 00:00 2024-05-14T16:41:10.566 INFO:teuthology.orchestra.run.smithi151.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 695 kB/s | 46 kB 00:00 2024-05-14T16:41:10.600 INFO:teuthology.orchestra.run.smithi151.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-05-14T16:41:10.625 INFO:teuthology.orchestra.run.smithi151.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 985 kB/s | 58 kB 00:00 2024-05-14T16:41:10.651 INFO:teuthology.orchestra.run.smithi123.stdout:(131/156): libarrow-9.0.0-11.el9.x86_64.rpm 819 kB/s | 4.4 MB 00:05 2024-05-14T16:41:10.659 INFO:teuthology.orchestra.run.smithi151.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-05-14T16:41:10.693 INFO:teuthology.orchestra.run.smithi151.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 245 kB/s | 16 kB 00:00 2024-05-14T16:41:10.726 INFO:teuthology.orchestra.run.smithi151.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-05-14T16:41:10.734 INFO:teuthology.orchestra.run.smithi123.stdout:(132/156): python3-more-itertools-8.12.0-2.el9. 325 kB/s | 79 kB 00:00 2024-05-14T16:41:10.768 INFO:teuthology.orchestra.run.smithi123.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 256 kB/s | 58 kB 00:00 2024-05-14T16:41:10.776 INFO:teuthology.orchestra.run.smithi151.stdout:(136/156): python3-kubernetes-26.1.0-2.el9.noar 3.2 MB/s | 1.0 MB 00:00 2024-05-14T16:41:10.797 INFO:teuthology.orchestra.run.smithi002.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 76 kB/s | 22 kB 00:00 2024-05-14T16:41:10.801 INFO:teuthology.orchestra.run.smithi151.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 259 kB/s | 28 kB 00:00 2024-05-14T16:41:10.827 INFO:teuthology.orchestra.run.smithi151.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 306 kB/s | 31 kB 00:00 2024-05-14T16:41:10.860 INFO:teuthology.orchestra.run.smithi123.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 131 kB/s | 16 kB 00:00 2024-05-14T16:41:10.919 INFO:teuthology.orchestra.run.smithi151.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 390 kB/s | 36 kB 00:00 2024-05-14T16:41:10.994 INFO:teuthology.orchestra.run.smithi151.stdout:(140/156): python3-typing-extensions-4.4.0-2.el 679 kB/s | 51 kB 00:00 2024-05-14T16:41:11.061 INFO:teuthology.orchestra.run.smithi123.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 141 kB/s | 28 kB 00:00 2024-05-14T16:41:11.078 INFO:teuthology.orchestra.run.smithi151.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 684 kB/s | 188 kB 00:00 2024-05-14T16:41:11.119 INFO:teuthology.orchestra.run.smithi123.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 256 kB/s | 90 kB 00:00 2024-05-14T16:41:11.178 INFO:teuthology.orchestra.run.smithi151.stdout:(142/156): python3-webob-1.8.7-6.el9.noarch.rpm 1.2 MB/s | 230 kB 00:00 2024-05-14T16:41:11.186 INFO:teuthology.orchestra.run.smithi123.stdout:(137/156): python3-pecan-1.4.2-3.el9.noarch.rpm 509 kB/s | 272 kB 00:00 2024-05-14T16:41:11.220 INFO:teuthology.orchestra.run.smithi123.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 193 kB/s | 31 kB 00:00 2024-05-14T16:41:11.253 INFO:teuthology.orchestra.run.smithi151.stdout:(143/156): python3-rsa-4.9-2.el9.noarch.rpm 131 kB/s | 59 kB 00:00 2024-05-14T16:41:11.264 INFO:teuthology.orchestra.run.smithi002.stdout:(103/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-05-14T16:41:11.278 INFO:teuthology.orchestra.run.smithi151.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 896 kB/s | 90 kB 00:00 2024-05-14T16:41:11.320 INFO:teuthology.orchestra.run.smithi151.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-05-14T16:41:11.336 INFO:teuthology.orchestra.run.smithi123.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 307 kB/s | 36 kB 00:00 2024-05-14T16:41:11.345 INFO:teuthology.orchestra.run.smithi151.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 217 kB/s | 20 kB 00:00 2024-05-14T16:41:11.371 INFO:teuthology.orchestra.run.smithi151.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-05-14T16:41:11.387 INFO:teuthology.orchestra.run.smithi123.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 295 kB/s | 59 kB 00:00 2024-05-14T16:41:11.389 INFO:teuthology.orchestra.run.smithi002.stdout:(104/156): gperftools-libs-2.9.1-2.el9.x86_64.r 2.4 MB/s | 309 kB 00:00 2024-05-14T16:41:11.423 INFO:teuthology.orchestra.run.smithi002.stdout:(105/156): protobuf-compiler-3.14.0-13.el9.x86_ 986 kB/s | 863 kB 00:00 2024-05-14T16:41:11.460 INFO:teuthology.orchestra.run.smithi002.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 274 kB/s | 19 kB 00:00 2024-05-14T16:41:11.462 INFO:teuthology.orchestra.run.smithi123.stdout:(141/156): python3-typing-extensions-4.4.0-2.el 407 kB/s | 51 kB 00:00 2024-05-14T16:41:11.515 INFO:teuthology.orchestra.run.smithi002.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 477 kB/s | 26 kB 00:00 2024-05-14T16:41:11.554 INFO:teuthology.orchestra.run.smithi123.stdout:(142/156): python3-routes-2.5.1-5.el9.noarch.rp 434 kB/s | 188 kB 00:00 2024-05-14T16:41:11.557 INFO:teuthology.orchestra.run.smithi002.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2024-05-14T16:41:11.563 INFO:teuthology.orchestra.run.smithi151.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 6.5 MB/s | 1.6 MB 00:00 2024-05-14T16:41:11.616 INFO:teuthology.orchestra.run.smithi002.stdout:(109/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-05-14T16:41:11.671 INFO:teuthology.orchestra.run.smithi123.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 429 kB/s | 90 kB 00:00 2024-05-14T16:41:11.674 INFO:teuthology.orchestra.run.smithi002.stdout:(110/156): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-05-14T16:41:11.755 INFO:teuthology.orchestra.run.smithi151.stdout:(149/156): librados2-19.0.0-3687.g9b638668.el9. 8.3 MB/s | 3.4 MB 00:00 2024-05-14T16:41:11.780 INFO:teuthology.orchestra.run.smithi123.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 184 kB/s | 20 kB 00:00 2024-05-14T16:41:11.805 INFO:teuthology.orchestra.run.smithi123.stdout:(145/156): python3-webob-1.8.7-6.el9.noarch.rpm 550 kB/s | 230 kB 00:00 2024-05-14T16:41:11.866 INFO:teuthology.orchestra.run.smithi002.stdout:(111/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.3 MB/s | 839 kB 00:00 2024-05-14T16:41:12.000 INFO:teuthology.orchestra.run.smithi002.stdout:(112/156): python3-asyncssh-2.13.2-5.el9.noarch 4.0 MB/s | 548 kB 00:00 2024-05-14T16:41:12.050 INFO:teuthology.orchestra.run.smithi002.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 869 kB/s | 43 kB 00:00 2024-05-14T16:41:12.072 INFO:teuthology.orchestra.run.smithi151.stdout:(150/156): librbd1-19.0.0-3687.g9b638668.el9.x8 4.5 MB/s | 3.1 MB 00:00 2024-05-14T16:41:12.080 INFO:teuthology.orchestra.run.smithi123.stdout:(146/156): re2-20211101-3.el9.x86_64.rpm 638 kB/s | 192 kB 00:00 2024-05-14T16:41:12.092 INFO:teuthology.orchestra.run.smithi002.stdout:(114/156): python3-cachetools-4.2.4-1.el9.noarc 771 kB/s | 32 kB 00:00 2024-05-14T16:41:12.134 INFO:teuthology.orchestra.run.smithi002.stdout:(115/156): python3-certifi-2021.10.8-2.el9.noar 361 kB/s | 15 kB 00:00 2024-05-14T16:41:12.184 INFO:teuthology.orchestra.run.smithi002.stdout:(116/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-05-14T16:41:12.222 INFO:teuthology.orchestra.run.smithi123.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 639 kB/s | 427 kB 00:00 2024-05-14T16:41:12.251 INFO:teuthology.orchestra.run.smithi002.stdout:(117/156): python3-cherrypy-18.6.1-2.el9.noarch 5.3 MB/s | 358 kB 00:00 2024-05-14T16:41:12.327 INFO:teuthology.orchestra.run.smithi002.stdout:(118/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-05-14T16:41:12.377 INFO:teuthology.orchestra.run.smithi002.stdout:(119/156): python3-colorama-0.4.4-12.el9.noarch 705 kB/s | 35 kB 00:00 2024-05-14T16:41:12.430 INFO:teuthology.orchestra.run.smithi151.stdout:(151/156): NetworkManager-team-1.47.90-1.el9.x8 116 kB/s | 41 kB 00:00 2024-05-14T16:41:12.469 INFO:teuthology.orchestra.run.smithi002.stdout:(120/156): libarrow-9.0.0-11.el9.x86_64.rpm 4.2 MB/s | 4.4 MB 00:01 2024-05-14T16:41:12.480 INFO:teuthology.orchestra.run.smithi151.stdout:(152/156): NetworkManager-1.47.90-1.el9.x86_64. 2.5 MB/s | 2.3 MB 00:00 2024-05-14T16:41:12.494 INFO:teuthology.orchestra.run.smithi002.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 1.7 MB/s | 206 kB 00:00 2024-05-14T16:41:12.536 INFO:teuthology.orchestra.run.smithi002.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 3.4 MB/s | 144 kB 00:00 2024-05-14T16:41:12.564 INFO:teuthology.orchestra.run.smithi123.stdout:(148/156): librados2-19.0.0-3687.g9b638668.el9. 7.0 MB/s | 3.4 MB 00:00 2024-05-14T16:41:12.570 INFO:teuthology.orchestra.run.smithi002.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 323 kB/s | 11 kB 00:00 2024-05-14T16:41:12.604 INFO:teuthology.orchestra.run.smithi002.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 529 kB/s | 18 kB 00:00 2024-05-14T16:41:12.605 INFO:teuthology.orchestra.run.smithi151.stdout:(153/156): openssl-3.2.1-1.el9.x86_64.rpm 11 MB/s | 1.3 MB 00:00 2024-05-14T16:41:12.646 INFO:teuthology.orchestra.run.smithi002.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 554 kB/s | 23 kB 00:00 2024-05-14T16:41:12.647 INFO:teuthology.orchestra.run.smithi151.stdout:(154/156): NetworkManager-libnm-1.47.90-1.el9.x 2.0 MB/s | 1.8 MB 00:00 2024-05-14T16:41:12.688 INFO:teuthology.orchestra.run.smithi002.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 467 kB/s | 19 kB 00:00 2024-05-14T16:41:12.746 INFO:teuthology.orchestra.run.smithi002.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 337 kB/s | 20 kB 00:00 2024-05-14T16:41:12.788 INFO:teuthology.orchestra.run.smithi002.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 6.4 MB/s | 2.0 MB 00:00 2024-05-14T16:41:12.789 INFO:teuthology.orchestra.run.smithi151.stdout:(155/156): NetworkManager-tui-1.47.90-1.el9.x86 690 kB/s | 248 kB 00:00 2024-05-14T16:41:12.830 INFO:teuthology.orchestra.run.smithi002.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2024-05-14T16:41:12.865 INFO:teuthology.orchestra.run.smithi123.stdout:(149/156): librbd1-19.0.0-3687.g9b638668.el9.x8 4.9 MB/s | 3.1 MB 00:00 2024-05-14T16:41:12.880 INFO:teuthology.orchestra.run.smithi002.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-05-14T16:41:13.039 INFO:teuthology.orchestra.run.smithi002.stdout:(131/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 5.7 MB/s | 16 MB 00:02 2024-05-14T16:41:13.064 INFO:teuthology.orchestra.run.smithi002.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 314 kB/s | 58 kB 00:00 2024-05-14T16:41:13.106 INFO:teuthology.orchestra.run.smithi123.stdout:(150/156): NetworkManager-1.47.90-1.el9.x86_64. 4.3 MB/s | 2.3 MB 00:00 2024-05-14T16:41:13.107 INFO:teuthology.orchestra.run.smithi002.stdout:(133/156): python3-kubernetes-26.1.0-2.el9.noar 2.8 MB/s | 1.0 MB 00:00 2024-05-14T16:41:13.132 INFO:teuthology.orchestra.run.smithi002.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 244 kB/s | 16 kB 00:00 2024-05-14T16:41:13.157 INFO:teuthology.orchestra.run.smithi123.stdout:(151/156): NetworkManager-team-1.47.90-1.el9.x8 832 kB/s | 41 kB 00:00 2024-05-14T16:41:13.157 INFO:teuthology.orchestra.run.smithi002.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-05-14T16:41:13.182 INFO:teuthology.orchestra.run.smithi002.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-05-14T16:41:13.208 INFO:teuthology.orchestra.run.smithi002.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-05-14T16:41:13.215 INFO:teuthology.orchestra.run.smithi123.stdout:(152/156): NetworkManager-tui-1.47.90-1.el9.x86 4.1 MB/s | 248 kB 00:00 2024-05-14T16:41:13.233 INFO:teuthology.orchestra.run.smithi002.stdout:(138/156): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-05-14T16:41:13.258 INFO:teuthology.orchestra.run.smithi002.stdout:(139/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.2 MB/s | 272 kB 00:00 2024-05-14T16:41:13.284 INFO:teuthology.orchestra.run.smithi002.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 782 kB/s | 59 kB 00:00 2024-05-14T16:41:13.309 INFO:teuthology.orchestra.run.smithi002.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 474 kB/s | 36 kB 00:00 2024-05-14T16:41:13.331 INFO:teuthology.orchestra.run.smithi151.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 3.3 MB/s | 2.4 MB 00:00 2024-05-14T16:41:13.334 INFO:teuthology.orchestra.run.smithi002.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 675 kB/s | 51 kB 00:00 2024-05-14T16:41:13.337 INFO:teuthology.orchestra.run.smithi151.stdout:-------------------------------------------------------------------------------- 2024-05-14T16:41:13.338 INFO:teuthology.orchestra.run.smithi151.stdout:Total 12 MB/s | 226 MB 00:18 2024-05-14T16:41:13.351 INFO:teuthology.orchestra.run.smithi002.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-05-14T16:41:13.385 INFO:teuthology.orchestra.run.smithi002.stdout:(144/156): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-05-14T16:41:13.399 INFO:teuthology.orchestra.run.smithi123.stdout:(153/156): openssl-3.2.1-1.el9.x86_64.rpm 7.4 MB/s | 1.3 MB 00:00 2024-05-14T16:41:13.410 INFO:teuthology.orchestra.run.smithi002.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 341 kB/s | 20 kB 00:00 2024-05-14T16:41:13.435 INFO:teuthology.orchestra.run.smithi002.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-05-14T16:41:13.460 INFO:teuthology.orchestra.run.smithi002.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-05-14T16:41:13.657 INFO:teuthology.orchestra.run.smithi123.stdout:(154/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 9.3 MB/s | 2.4 MB 00:00 2024-05-14T16:41:13.669 INFO:teuthology.orchestra.run.smithi002.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 6.1 MB/s | 1.6 MB 00:00 2024-05-14T16:41:13.724 INFO:teuthology.orchestra.run.smithi123.stdout:(155/156): NetworkManager-libnm-1.47.90-1.el9.x 2.1 MB/s | 1.8 MB 00:00 2024-05-14T16:41:13.803 INFO:teuthology.orchestra.run.smithi002.stdout:(149/156): librbd1-19.0.0-3687.g9b638668.el9.x8 9.2 MB/s | 3.1 MB 00:00 2024-05-14T16:41:13.891 INFO:teuthology.orchestra.run.smithi123.stdout:(156/156): thrift-0.15.0-2.el9.x86_64.rpm 776 kB/s | 1.6 MB 00:02 2024-05-14T16:41:13.898 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-05-14T16:41:13.898 INFO:teuthology.orchestra.run.smithi123.stdout:Total 10 MB/s | 226 MB 00:21 2024-05-14T16:41:13.928 INFO:teuthology.orchestra.run.smithi002.stdout:(150/156): librados2-19.0.0-3687.g9b638668.el9. 6.9 MB/s | 3.4 MB 00:00 2024-05-14T16:41:14.053 INFO:teuthology.orchestra.run.smithi002.stdout:(151/156): NetworkManager-team-1.47.90-1.el9.x8 333 kB/s | 41 kB 00:00 2024-05-14T16:41:14.170 INFO:teuthology.orchestra.run.smithi002.stdout:(152/156): NetworkManager-tui-1.47.90-1.el9.x86 2.1 MB/s | 248 kB 00:00 2024-05-14T16:41:14.220 INFO:teuthology.orchestra.run.smithi002.stdout:(153/156): NetworkManager-1.47.90-1.el9.x86_64. 4.2 MB/s | 2.3 MB 00:00 2024-05-14T16:41:14.412 INFO:teuthology.orchestra.run.smithi002.stdout:(154/156): NetworkManager-libnm-1.47.90-1.el9.x 3.0 MB/s | 1.8 MB 00:00 2024-05-14T16:41:14.512 INFO:teuthology.orchestra.run.smithi002.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 8.2 MB/s | 2.4 MB 00:00 2024-05-14T16:41:14.523 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction check 2024-05-14T16:41:14.571 INFO:teuthology.orchestra.run.smithi002.stdout:(156/156): openssl-3.2.1-1.el9.x86_64.rpm 3.4 MB/s | 1.3 MB 00:00 2024-05-14T16:41:14.573 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-05-14T16:41:14.574 INFO:teuthology.orchestra.run.smithi002.stdout:Total 11 MB/s | 226 MB 00:20 2024-05-14T16:41:14.675 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction check succeeded. 2024-05-14T16:41:14.675 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction test 2024-05-14T16:41:15.098 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-05-14T16:41:15.246 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-05-14T16:41:15.247 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-05-14T16:41:15.766 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-05-14T16:41:15.931 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-05-14T16:41:15.931 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-05-14T16:41:16.485 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction test succeeded. 2024-05-14T16:41:16.486 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction 2024-05-14T16:41:17.066 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-05-14T16:41:17.066 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-05-14T16:41:18.211 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-05-14T16:41:18.212 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-05-14T16:41:22.609 INFO:teuthology.orchestra.run.smithi151.stdout: Preparing : 1/1 2024-05-14T16:41:22.669 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-14T16:41:22.724 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-14T16:41:22.786 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-14T16:41:22.819 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-14T16:41:22.845 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-14T16:41:22.888 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-14T16:41:23.256 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-14T16:41:23.298 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-05-14T16:41:23.329 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : librados2-2:19.0.0-3687.g9b638668.el9.x86_64 8/164 2024-05-14T16:41:23.395 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-14T16:41:23.433 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-14T16:41:23.459 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librados2-2:19.0.0-3687.g9b638668.el9.x86_64 8/164 2024-05-14T16:41:23.480 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 9/164 2024-05-14T16:41:23.486 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-14T16:41:23.521 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-14T16:41:23.547 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-14T16:41:23.571 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 9/164 2024-05-14T16:41:23.589 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-14T16:41:23.601 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rados-2:19.0.0-3687.g9b638668.el9.x86_64 10/164 2024-05-14T16:41:23.747 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-14T16:41:23.798 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-14T16:41:23.827 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 13/164 2024-05-14T16:41:23.909 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-14T16:41:23.915 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 13/164 2024-05-14T16:41:23.952 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86 14/164 2024-05-14T16:41:24.014 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librados2-2:19.0.0-3687.g9b638668.el9.x86_64 8/164 2024-05-14T16:41:24.015 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86 14/164 2024-05-14T16:41:24.037 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-14T16:41:24.062 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-14T16:41:24.135 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:19.0.0-3687.g9b638668.el9.x86_64 8/164 2024-05-14T16:41:24.165 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 9/164 2024-05-14T16:41:24.245 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-14T16:41:24.256 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 9/164 2024-05-14T16:41:24.286 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rados-2:19.0.0-3687.g9b638668.el9.x86_64 10/164 2024-05-14T16:41:24.371 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-14T16:41:24.437 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-14T16:41:24.492 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-14T16:41:24.534 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-14T16:41:24.545 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 13/164 2024-05-14T16:41:24.586 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-14T16:41:24.647 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 13/164 2024-05-14T16:41:24.671 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86 14/164 2024-05-14T16:41:24.679 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-05-14T16:41:24.683 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T16:41:24.725 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86 14/164 2024-05-14T16:41:24.747 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-14T16:41:24.754 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T16:41:24.759 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-14T16:41:24.766 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-14T16:41:24.802 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-14T16:41:24.869 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-14T16:41:24.910 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-14T16:41:24.935 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-14T16:41:24.947 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-14T16:41:24.952 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T16:41:24.987 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-14T16:41:25.071 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-14T16:41:25.074 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-14T16:41:25.100 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-14T16:41:25.135 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-14T16:41:25.209 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-14T16:41:25.244 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-14T16:41:25.259 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-14T16:41:25.303 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-14T16:41:25.304 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-14T16:41:25.331 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-14T16:41:25.335 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-14T16:41:25.363 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-14T16:41:25.407 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T16:41:25.432 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-14T16:41:25.435 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : librados2-2:19.0.0-3687.g9b638668.el9.x86_64 8/164 2024-05-14T16:41:25.485 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-14T16:41:25.493 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T16:41:25.516 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-14T16:41:25.551 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-14T16:41:25.560 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librados2-2:19.0.0-3687.g9b638668.el9.x86_64 8/164 2024-05-14T16:41:25.586 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 9/164 2024-05-14T16:41:25.628 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-14T16:41:25.662 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-ceph-common-2:19.0.0-3687.g9b638668.el9. 32/164 2024-05-14T16:41:25.662 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 9/164 2024-05-14T16:41:25.691 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rados-2:19.0.0-3687.g9b638668.el9.x86_64 10/164 2024-05-14T16:41:25.692 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T16:41:25.705 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-ceph-argparse-2:19.0.0-3687.g9b638668.el 33/164 2024-05-14T16:41:25.734 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cephfs-2:19.0.0-3687.g9b638668.el9.x86_6 34/164 2024-05-14T16:41:25.805 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-14T16:41:25.808 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-14T16:41:25.843 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-14T16:41:25.848 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-14T16:41:25.869 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-14T16:41:25.878 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-14T16:41:25.923 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-14T16:41:25.931 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-14T16:41:25.955 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-14T16:41:25.958 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 13/164 2024-05-14T16:41:26.003 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-14T16:41:26.047 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-14T16:41:26.063 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 13/164 2024-05-14T16:41:26.065 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-14T16:41:26.079 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-14T16:41:26.092 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86 14/164 2024-05-14T16:41:26.115 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-14T16:41:26.117 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-14T16:41:26.155 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86 14/164 2024-05-14T16:41:26.176 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-14T16:41:26.184 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-14T16:41:26.199 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-14T16:41:26.238 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-14T16:41:26.268 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-14T16:41:26.304 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-14T16:41:26.379 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-14T16:41:26.388 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-14T16:41:26.439 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-common-2:19.0.0-3687.g9b638668.el9. 32/164 2024-05-14T16:41:26.482 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-argparse-2:19.0.0-3687.g9b638668.el 33/164 2024-05-14T16:41:26.511 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cephfs-2:19.0.0-3687.g9b638668.el9.x86_6 34/164 2024-05-14T16:41:26.547 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-14T16:41:26.582 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-14T16:41:26.626 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-14T16:41:26.636 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-14T16:41:26.682 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-14T16:41:26.701 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-14T16:41:26.721 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-14T16:41:26.751 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-14T16:41:26.752 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-14T16:41:26.793 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-14T16:41:26.831 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librados-devel-2:19.0.0-3687.g9b638668.el9.x86_6 44/164 2024-05-14T16:41:26.835 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-14T16:41:26.873 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-14T16:41:26.874 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T16:41:26.944 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T16:41:27.070 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-14T16:41:27.155 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-14T16:41:27.192 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-14T16:41:27.284 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-14T16:41:27.315 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-14T16:41:27.350 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-14T16:41:27.401 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-14T16:41:27.425 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-14T16:41:27.466 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-14T16:41:27.492 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-14T16:41:27.510 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-14T16:41:27.542 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-14T16:41:27.549 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-14T16:41:27.569 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-14T16:41:27.620 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-14T16:41:27.641 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-14T16:41:27.652 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-14T16:41:27.670 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-14T16:41:27.689 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-14T16:41:27.692 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-14T16:41:27.722 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-14T16:41:27.735 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-14T16:41:27.757 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-14T16:41:27.758 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-14T16:41:27.771 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-14T16:41:27.787 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-14T16:41:27.814 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-14T16:41:27.829 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-14T16:41:27.836 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-14T16:41:27.856 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-14T16:41:27.867 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librados-devel-2:19.0.0-3687.g9b638668.el9.x86_6 44/164 2024-05-14T16:41:27.868 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-ceph-common-2:19.0.0-3687.g9b638668.el9. 32/164 2024-05-14T16:41:27.909 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-14T16:41:27.917 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-ceph-argparse-2:19.0.0-3687.g9b638668.el 33/164 2024-05-14T16:41:27.940 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cephfs-2:19.0.0-3687.g9b638668.el9.x86_6 34/164 2024-05-14T16:41:27.992 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-14T16:41:28.044 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-14T16:41:28.068 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-14T16:41:28.097 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-14T16:41:28.122 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-14T16:41:28.129 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-14T16:41:28.159 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-14T16:41:28.175 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-14T16:41:28.201 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-14T16:41:28.215 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-14T16:41:28.278 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-14T16:41:28.294 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-14T16:41:28.318 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-14T16:41:28.325 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-14T16:41:28.351 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-14T16:41:28.357 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-14T16:41:28.375 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-14T16:41:28.382 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-14T16:41:28.382 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-14T16:41:28.407 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-14T16:41:28.410 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-14T16:41:28.457 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-14T16:41:28.481 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-14T16:41:28.492 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-14T16:41:28.536 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-14T16:41:28.578 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-14T16:41:28.610 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-14T16:41:28.639 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-14T16:41:28.667 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-14T16:41:28.701 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-14T16:41:28.715 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-14T16:41:28.740 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-14T16:41:28.758 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-14T16:41:28.772 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-14T16:41:28.788 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-14T16:41:28.801 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-14T16:41:28.833 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-14T16:41:28.836 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-14T16:41:28.869 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-14T16:41:28.910 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-14T16:41:28.925 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-14T16:41:28.927 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-14T16:41:28.945 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-14T16:41:28.963 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-14T16:41:29.003 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-14T16:41:29.013 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-14T16:41:29.022 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-14T16:41:29.029 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-14T16:41:29.049 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-14T16:41:29.053 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-14T16:41:29.075 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-14T16:41:29.093 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-14T16:41:29.107 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-14T16:41:29.128 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-14T16:41:29.130 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librados-devel-2:19.0.0-3687.g9b638668.el9.x86_6 44/164 2024-05-14T16:41:29.155 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-14T16:41:29.172 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-14T16:41:29.253 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-14T16:41:29.255 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-14T16:41:29.286 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-14T16:41:29.300 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-14T16:41:29.313 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-14T16:41:29.330 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-14T16:41:29.352 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-14T16:41:29.387 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-14T16:41:29.400 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-14T16:41:29.418 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-14T16:41:29.448 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-14T16:41:29.483 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-14T16:41:29.494 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-14T16:41:29.516 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-14T16:41:29.556 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-14T16:41:29.566 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-14T16:41:29.600 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-14T16:41:29.630 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-14T16:41:29.676 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-14T16:41:29.687 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-14T16:41:29.704 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-14T16:41:29.739 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-14T16:41:29.751 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-14T16:41:29.770 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-14T16:41:29.786 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-14T16:41:29.818 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-14T16:41:29.830 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-14T16:41:29.872 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-14T16:41:29.916 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-14T16:41:29.972 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-14T16:41:30.018 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-14T16:41:30.072 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-14T16:41:30.105 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-14T16:41:30.115 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-14T16:41:30.161 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-14T16:41:30.191 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-14T16:41:30.204 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-14T16:41:30.232 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-14T16:41:30.263 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-14T16:41:30.331 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-14T16:41:30.351 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-14T16:41:30.366 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-14T16:41:30.391 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-14T16:41:30.406 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-14T16:41:30.438 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-14T16:41:30.463 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-14T16:41:30.559 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-14T16:41:30.568 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-14T16:41:30.597 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-14T16:41:30.674 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-14T16:41:30.688 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-14T16:41:30.702 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-14T16:41:30.721 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-14T16:41:30.752 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-14T16:41:30.757 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-14T16:41:30.786 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-14T16:41:30.792 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-14T16:41:30.831 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-14T16:41:30.871 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-14T16:41:30.887 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-14T16:41:30.900 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-14T16:41:30.935 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-14T16:41:30.953 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-14T16:41:30.968 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-14T16:41:31.010 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-14T16:41:31.041 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-14T16:41:31.044 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-14T16:41:31.080 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-14T16:41:31.134 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-14T16:41:31.184 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-14T16:41:31.222 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-14T16:41:31.258 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-14T16:41:31.271 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-14T16:41:31.286 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-14T16:41:31.316 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-14T16:41:31.364 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-14T16:41:31.380 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-14T16:41:31.433 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-14T16:41:31.436 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-14T16:41:32.013 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-14T16:41:32.118 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-14T16:41:32.190 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-14T16:41:32.238 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-14T16:41:32.272 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-14T16:41:32.373 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-14T16:41:32.431 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-14T16:41:32.519 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-14T16:41:32.549 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-14T16:41:32.653 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-14T16:41:32.733 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-14T16:41:32.733 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-14T16:41:32.761 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-14T16:41:32.763 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-14T16:41:32.794 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-14T16:41:32.843 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-14T16:41:33.106 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-14T16:41:33.140 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 101/164 2024-05-14T16:41:33.222 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 101/164 2024-05-14T16:41:33.281 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rbd-2:19.0.0-3687.g9b638668.el9.x86_64 102/164 2024-05-14T16:41:33.323 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-14T16:41:33.348 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-14T16:41:33.439 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-14T16:41:33.467 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-14T16:41:33.500 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-14T16:41:33.525 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-14T16:41:33.639 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-14T16:41:33.801 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-14T16:41:33.811 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-14T16:41:33.835 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 105/164 2024-05-14T16:41:33.854 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 101/164 2024-05-14T16:41:33.897 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 105/164 2024-05-14T16:41:33.928 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rgw-2:19.0.0-3687.g9b638668.el9.x86_64 106/164 2024-05-14T16:41:33.945 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 101/164 2024-05-14T16:41:33.976 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rbd-2:19.0.0-3687.g9b638668.el9.x86_64 102/164 2024-05-14T16:41:34.013 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-14T16:41:34.494 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-14T16:41:34.524 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 105/164 2024-05-14T16:41:34.595 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 105/164 2024-05-14T16:41:34.617 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rgw-2:19.0.0-3687.g9b638668.el9.x86_64 106/164 2024-05-14T16:41:34.964 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-14T16:41:35.076 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-14T16:41:35.105 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-14T16:41:35.138 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-14T16:41:35.466 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-14T16:41:35.509 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 101/164 2024-05-14T16:41:35.600 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 101/164 2024-05-14T16:41:35.630 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rbd-2:19.0.0-3687.g9b638668.el9.x86_64 102/164 2024-05-14T16:41:35.667 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-14T16:41:36.172 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-14T16:41:36.203 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 105/164 2024-05-14T16:41:36.205 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 107/164 2024-05-14T16:41:36.258 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 107/164 2024-05-14T16:41:36.283 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 105/164 2024-05-14T16:41:36.313 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rgw-2:19.0.0-3687.g9b638668.el9.x86_64 106/164 2024-05-14T16:41:36.801 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 107/164 2024-05-14T16:41:36.836 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 108/164 2024-05-14T16:41:36.927 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 108/164 2024-05-14T16:41:36.928 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-14T16:41:36.928 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-14T16:41:36.928 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-14T16:41:36.954 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 109/164 2024-05-14T16:41:36.964 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 107/164 2024-05-14T16:41:37.013 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 107/164 2024-05-14T16:41:37.558 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 107/164 2024-05-14T16:41:37.592 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 108/164 2024-05-14T16:41:37.683 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 108/164 2024-05-14T16:41:37.683 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-14T16:41:37.683 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-14T16:41:37.683 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-14T16:41:37.701 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 109/164 2024-05-14T16:41:38.719 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 107/164 2024-05-14T16:41:38.751 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 107/164 2024-05-14T16:41:39.331 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 107/164 2024-05-14T16:41:39.379 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 108/164 2024-05-14T16:41:39.472 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 108/164 2024-05-14T16:41:39.472 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-14T16:41:39.472 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-14T16:41:39.472 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:41:39.506 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 109/164 2024-05-14T16:41:48.551 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 109/164 2024-05-14T16:41:48.551 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /sys 2024-05-14T16:41:48.551 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /proc 2024-05-14T16:41:48.551 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /mnt 2024-05-14T16:41:48.551 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /var/tmp 2024-05-14T16:41:48.552 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /home 2024-05-14T16:41:48.552 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /root 2024-05-14T16:41:48.552 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /tmp 2024-05-14T16:41:48.552 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-14T16:41:48.741 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 110/164 2024-05-14T16:41:48.757 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 110/164 2024-05-14T16:41:48.757 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:41:48.757 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-14T16:41:48.757 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-14T16:41:49.437 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 109/164 2024-05-14T16:41:49.437 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /sys 2024-05-14T16:41:49.437 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /proc 2024-05-14T16:41:49.438 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /mnt 2024-05-14T16:41:49.438 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /var/tmp 2024-05-14T16:41:49.438 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /home 2024-05-14T16:41:49.438 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /root 2024-05-14T16:41:49.438 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /tmp 2024-05-14T16:41:49.438 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-14T16:41:49.607 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 110/164 2024-05-14T16:41:49.624 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 110/164 2024-05-14T16:41:49.624 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:41:49.624 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-14T16:41:49.624 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-14T16:41:50.244 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 111/164 2024-05-14T16:41:50.270 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 111/164 2024-05-14T16:41:50.270 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:41:50.270 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-14T16:41:50.270 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-14T16:41:50.270 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-14T16:41:50.270 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-14T16:41:50.495 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 112/164 2024-05-14T16:41:50.518 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 112/164 2024-05-14T16:41:50.518 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:41:50.518 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-14T16:41:50.518 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-14T16:41:50.518 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-14T16:41:50.518 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-14T16:41:50.954 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 113/164 2024-05-14T16:41:50.976 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 113/164 2024-05-14T16:41:50.976 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:41:50.976 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-14T16:41:50.976 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-14T16:41:50.976 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-14T16:41:50.976 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-14T16:41:51.084 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-14T16:41:51.135 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 111/164 2024-05-14T16:41:51.140 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-14T16:41:51.161 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 111/164 2024-05-14T16:41:51.161 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:41:51.161 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-14T16:41:51.161 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-14T16:41:51.161 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-14T16:41:51.161 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-14T16:41:51.196 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-14T16:41:51.385 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 112/164 2024-05-14T16:41:51.390 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-14T16:41:51.408 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 112/164 2024-05-14T16:41:51.408 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:41:51.408 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-14T16:41:51.408 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-14T16:41:51.408 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-14T16:41:51.408 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-14T16:41:51.430 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-14T16:41:51.500 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-14T16:41:51.540 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 109/164 2024-05-14T16:41:51.540 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /sys 2024-05-14T16:41:51.541 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /proc 2024-05-14T16:41:51.541 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /mnt 2024-05-14T16:41:51.541 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /var/tmp 2024-05-14T16:41:51.541 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /home 2024-05-14T16:41:51.541 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /root 2024-05-14T16:41:51.541 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /tmp 2024-05-14T16:41:51.541 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:41:51.589 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-14T16:41:51.710 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 110/164 2024-05-14T16:41:51.727 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 110/164 2024-05-14T16:41:51.728 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:41:51.728 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-14T16:41:51.728 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:41:51.728 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-14T16:41:51.764 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-14T16:41:51.805 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-14T16:41:51.832 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-14T16:41:51.861 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 113/164 2024-05-14T16:41:51.884 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 113/164 2024-05-14T16:41:51.884 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:41:51.884 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-14T16:41:51.885 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-14T16:41:51.885 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-14T16:41:51.885 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-14T16:41:51.991 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-14T16:41:52.048 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-14T16:41:52.103 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-14T16:41:52.236 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: cephadm-2:19.0.0-3687.g9b638668.el9.noarch 125/164 2024-05-14T16:41:52.272 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephadm-2:19.0.0-3687.g9b638668.el9.noarch 125/164 2024-05-14T16:41:52.295 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-14T16:41:52.306 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3687.g9b638668.e 126/164 2024-05-14T16:41:52.348 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-14T16:41:52.368 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3687.g9b638668. 127/164 2024-05-14T16:41:52.406 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noa 128/164 2024-05-14T16:41:52.418 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-14T16:41:52.507 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-14T16:41:52.637 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-14T16:41:52.673 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-14T16:41:52.737 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-14T16:41:52.767 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-14T16:41:53.218 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephadm-2:19.0.0-3687.g9b638668.el9.noarch 125/164 2024-05-14T16:41:53.248 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephadm-2:19.0.0-3687.g9b638668.el9.noarch 125/164 2024-05-14T16:41:53.263 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 111/164 2024-05-14T16:41:53.280 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3687.g9b638668.e 126/164 2024-05-14T16:41:53.291 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 111/164 2024-05-14T16:41:53.292 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:41:53.292 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-14T16:41:53.292 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-14T16:41:53.292 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-14T16:41:53.292 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:41:53.341 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3687.g9b638668. 127/164 2024-05-14T16:41:53.372 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noa 128/164 2024-05-14T16:41:53.449 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noa 128/164 2024-05-14T16:41:53.483 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.n 129/164 2024-05-14T16:41:53.531 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 112/164 2024-05-14T16:41:53.555 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 112/164 2024-05-14T16:41:53.555 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:41:53.555 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-14T16:41:53.555 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-14T16:41:53.555 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-14T16:41:53.555 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:41:54.023 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 113/164 2024-05-14T16:41:54.048 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 113/164 2024-05-14T16:41:54.048 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:41:54.048 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-14T16:41:54.048 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-14T16:41:54.048 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-14T16:41:54.048 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:41:54.154 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-14T16:41:54.212 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-14T16:41:54.281 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-14T16:41:54.410 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noa 128/164 2024-05-14T16:41:54.449 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.n 129/164 2024-05-14T16:41:54.465 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.n 129/164 2024-05-14T16:41:54.477 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-14T16:41:54.484 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b6 130/164 2024-05-14T16:41:54.541 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-14T16:41:54.595 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b6 130/164 2024-05-14T16:41:54.613 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-14T16:41:54.711 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-14T16:41:54.763 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3687.g9b638668.el 131/164 2024-05-14T16:41:54.795 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 132/164 2024-05-14T16:41:54.818 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 132/164 2024-05-14T16:41:54.818 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:41:54.818 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-14T16:41:54.818 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-14T16:41:54.818 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-14T16:41:54.818 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-14T16:41:54.852 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-14T16:41:54.862 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 133/164 2024-05-14T16:41:54.874 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 133/164 2024-05-14T16:41:54.899 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-14T16:41:54.931 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-14T16:41:54.958 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-14T16:41:54.972 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-2:19.0.0-3687.g9b638668.el9.x86_64 134/164 2024-05-14T16:41:55.019 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 135/164 2024-05-14T16:41:55.371 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: cephadm-2:19.0.0-3687.g9b638668.el9.noarch 125/164 2024-05-14T16:41:55.398 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : cephadm-2:19.0.0-3687.g9b638668.el9.noarch 125/164 2024-05-14T16:41:55.422 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.n 129/164 2024-05-14T16:41:55.431 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3687.g9b638668.e 126/164 2024-05-14T16:41:55.449 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b6 130/164 2024-05-14T16:41:55.494 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3687.g9b638668. 127/164 2024-05-14T16:41:55.521 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noa 128/164 2024-05-14T16:41:55.561 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b6 130/164 2024-05-14T16:41:55.728 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3687.g9b638668.el 131/164 2024-05-14T16:41:55.759 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 132/164 2024-05-14T16:41:55.783 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 132/164 2024-05-14T16:41:55.783 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:41:55.783 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-14T16:41:55.783 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-14T16:41:55.783 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-14T16:41:55.783 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-14T16:41:55.826 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 133/164 2024-05-14T16:41:55.837 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 133/164 2024-05-14T16:41:55.935 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-2:19.0.0-3687.g9b638668.el9.x86_64 134/164 2024-05-14T16:41:55.948 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-14T16:41:55.974 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 137/164 2024-05-14T16:41:55.982 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 135/164 2024-05-14T16:41:55.997 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 137/164 2024-05-14T16:41:55.997 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:41:55.997 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-14T16:41:55.998 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-14T16:41:55.998 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-14T16:41:55.998 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-14T16:41:56.050 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3687.g9b638 138/164 2024-05-14T16:41:56.073 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3687.g9b638 138/164 2024-05-14T16:41:56.073 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:41:56.073 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-14T16:41:56.073 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-14T16:41:56.117 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 139/164 2024-05-14T16:41:56.140 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 139/164 2024-05-14T16:41:56.140 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:41:56.140 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-14T16:41:56.140 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-14T16:41:56.140 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-14T16:41:56.140 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-14T16:41:56.443 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 140/164 2024-05-14T16:41:56.466 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 140/164 2024-05-14T16:41:56.466 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:41:56.467 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-14T16:41:56.467 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-14T16:41:56.467 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-14T16:41:56.467 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-14T16:41:56.613 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noa 128/164 2024-05-14T16:41:56.648 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.n 129/164 2024-05-14T16:41:56.905 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-14T16:41:56.937 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 137/164 2024-05-14T16:41:56.960 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 137/164 2024-05-14T16:41:56.960 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:41:56.961 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-14T16:41:56.961 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-14T16:41:56.961 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-14T16:41:56.961 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-14T16:41:57.004 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3687.g9b638 138/164 2024-05-14T16:41:57.026 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3687.g9b638 138/164 2024-05-14T16:41:57.026 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:41:57.026 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-14T16:41:57.027 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-14T16:41:57.080 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 139/164 2024-05-14T16:41:57.102 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 139/164 2024-05-14T16:41:57.103 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:41:57.103 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-14T16:41:57.103 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-14T16:41:57.103 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-14T16:41:57.103 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-14T16:41:57.406 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 140/164 2024-05-14T16:41:57.429 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 140/164 2024-05-14T16:41:57.429 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:41:57.429 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-14T16:41:57.429 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-14T16:41:57.429 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-14T16:41:57.429 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-14T16:41:57.660 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.n 129/164 2024-05-14T16:41:57.682 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b6 130/164 2024-05-14T16:41:57.798 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b6 130/164 2024-05-14T16:41:57.967 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3687.g9b638668.el 131/164 2024-05-14T16:41:58.001 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 132/164 2024-05-14T16:41:58.026 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 132/164 2024-05-14T16:41:58.026 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:41:58.026 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-14T16:41:58.026 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-14T16:41:58.026 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-14T16:41:58.026 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:41:58.076 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 133/164 2024-05-14T16:41:58.088 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 133/164 2024-05-14T16:41:58.188 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-2:19.0.0-3687.g9b638668.el9.x86_64 134/164 2024-05-14T16:41:58.266 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 135/164 2024-05-14T16:41:59.253 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-14T16:41:59.279 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 137/164 2024-05-14T16:41:59.303 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 137/164 2024-05-14T16:41:59.304 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:41:59.304 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-14T16:41:59.304 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-14T16:41:59.304 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-14T16:41:59.304 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:41:59.346 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3687.g9b638 138/164 2024-05-14T16:41:59.370 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3687.g9b638 138/164 2024-05-14T16:41:59.370 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:41:59.370 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-14T16:41:59.370 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:41:59.421 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 139/164 2024-05-14T16:41:59.446 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 139/164 2024-05-14T16:41:59.446 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:41:59.446 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-14T16:41:59.446 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-14T16:41:59.446 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-14T16:41:59.446 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:41:59.764 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 140/164 2024-05-14T16:41:59.788 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 140/164 2024-05-14T16:41:59.789 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-14T16:41:59.789 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-14T16:41:59.789 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-14T16:41:59.789 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-14T16:41:59.789 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:42:01.082 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-test-2:19.0.0-3687.g9b638668.el9.x86_64 141/164 2024-05-14T16:42:01.124 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : rbd-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 142/164 2024-05-14T16:42:01.164 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : rbd-nbd-2:19.0.0-3687.g9b638668.el9.x86_64 143/164 2024-05-14T16:42:01.197 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-14T16:42:01.224 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-14T16:42:01.410 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephfs-shell-2:19.0.0-3687.g9b638668.el9.x86_64 146/164 2024-05-14T16:42:01.463 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-14T16:42:01.494 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libcephfs-devel-2:19.0.0-3687.g9b638668.el9.x86_ 148/164 2024-05-14T16:42:01.523 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 149/164 2024-05-14T16:42:01.570 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 150/164 2024-05-14T16:42:02.025 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-test-2:19.0.0-3687.g9b638668.el9.x86_64 141/164 2024-05-14T16:42:02.066 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 142/164 2024-05-14T16:42:02.108 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-nbd-2:19.0.0-3687.g9b638668.el9.x86_64 143/164 2024-05-14T16:42:02.140 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-14T16:42:02.158 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-14T16:42:02.178 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T16:42:02.220 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T16:42:02.239 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T16:42:02.341 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-shell-2:19.0.0-3687.g9b638668.el9.x86_64 146/164 2024-05-14T16:42:02.388 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-14T16:42:02.418 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs-devel-2:19.0.0-3687.g9b638668.el9.x86_ 148/164 2024-05-14T16:42:02.447 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 149/164 2024-05-14T16:42:02.511 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 150/164 2024-05-14T16:42:03.074 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephfs-top-2:19.0.0-3687.g9b638668.el9.noarch 152/164 2024-05-14T16:42:03.118 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T16:42:03.153 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-14T16:42:03.166 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T16:42:03.186 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T16:42:03.202 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-14T16:42:03.293 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-14T16:42:03.332 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-14T16:42:03.358 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-14T16:42:03.358 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-14T16:42:03.451 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-14T16:42:03.476 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-14T16:42:03.476 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-14T16:42:03.560 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-14T16:42:03.585 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-14T16:42:03.591 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T16:42:03.591 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T16:42:03.771 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T16:42:03.771 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-14T16:42:03.855 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-14T16:42:03.920 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T16:42:04.019 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-top-2:19.0.0-3687.g9b638668.el9.noarch 152/164 2024-05-14T16:42:04.112 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-14T16:42:04.153 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-14T16:42:04.235 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-14T16:42:04.265 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-14T16:42:04.291 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-14T16:42:04.291 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-14T16:42:04.375 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-14T16:42:04.400 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-14T16:42:04.400 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-14T16:42:04.484 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-14T16:42:04.509 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-14T16:42:04.515 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T16:42:04.515 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T16:42:04.668 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-test-2:19.0.0-3687.g9b638668.el9.x86_64 141/164 2024-05-14T16:42:04.702 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T16:42:04.703 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-14T16:42:04.716 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : rbd-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 142/164 2024-05-14T16:42:04.758 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : rbd-nbd-2:19.0.0-3687.g9b638668.el9.x86_64 143/164 2024-05-14T16:42:04.770 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-14T16:42:04.789 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-14T16:42:04.817 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-14T16:42:04.832 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T16:42:05.012 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : cephfs-shell-2:19.0.0-3687.g9b638668.el9.x86_64 146/164 2024-05-14T16:42:05.063 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-14T16:42:05.093 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libcephfs-devel-2:19.0.0-3687.g9b638668.el9.x86_ 148/164 2024-05-14T16:42:05.123 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 149/164 2024-05-14T16:42:05.160 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 150/164 2024-05-14T16:42:05.812 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T16:42:05.844 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T16:42:05.866 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-14T16:42:06.759 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : cephfs-top-2:19.0.0-3687.g9b638668.el9.noarch 152/164 2024-05-14T16:42:06.857 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-14T16:42:06.911 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-14T16:42:07.001 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-14T16:42:07.035 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-14T16:42:07.061 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-14T16:42:07.061 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-14T16:42:07.162 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-14T16:42:07.204 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-14T16:42:07.204 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-14T16:42:07.288 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-14T16:42:07.313 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-14T16:42:07.318 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T16:42:07.319 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T16:42:07.506 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-14T16:42:07.507 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-14T16:42:07.591 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-14T16:42:07.658 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T16:42:09.957 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T16:42:09.957 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-2:19.0.0-3687.g9b638668.el9.x86_64 1/164 2024-05-14T16:42:09.957 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 2/164 2024-05-14T16:42:09.957 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 3/164 2024-05-14T16:42:09.957 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 4/164 2024-05-14T16:42:09.957 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3687.g9b638 5/164 2024-05-14T16:42:09.958 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 6/164 2024-05-14T16:42:09.958 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 7/164 2024-05-14T16:42:09.958 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 8/164 2024-05-14T16:42:09.958 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 9/164 2024-05-14T16:42:09.958 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 10/164 2024-05-14T16:42:09.958 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 11/164 2024-05-14T16:42:09.958 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-test-2:19.0.0-3687.g9b638668.el9.x86_64 12/164 2024-05-14T16:42:09.958 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 13/164 2024-05-14T16:42:09.958 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephfs-shell-2:19.0.0-3687.g9b638668.el9.x86_64 14/164 2024-05-14T16:42:09.958 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libcephfs-devel-2:19.0.0-3687.g9b638668.el9.x86_ 15/164 2024-05-14T16:42:09.958 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 16/164 2024-05-14T16:42:09.959 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 17/164 2024-05-14T16:42:09.959 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librados-devel-2:19.0.0-3687.g9b638668.el9.x86_6 18/164 2024-05-14T16:42:09.959 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86 19/164 2024-05-14T16:42:09.959 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 20/164 2024-05-14T16:42:09.959 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3687.g9b638668.el 21/164 2024-05-14T16:42:09.959 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-ceph-common-2:19.0.0-3687.g9b638668.el9. 22/164 2024-05-14T16:42:09.959 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cephfs-2:19.0.0-3687.g9b638668.el9.x86_6 23/164 2024-05-14T16:42:09.959 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rados-2:19.0.0-3687.g9b638668.el9.x86_64 24/164 2024-05-14T16:42:09.960 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rbd-2:19.0.0-3687.g9b638668.el9.x86_64 25/164 2024-05-14T16:42:09.960 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rgw-2:19.0.0-3687.g9b638668.el9.x86_64 26/164 2024-05-14T16:42:09.960 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : rbd-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 27/164 2024-05-14T16:42:09.960 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 28/164 2024-05-14T16:42:09.960 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : rbd-nbd-2:19.0.0-3687.g9b638668.el9.x86_64 29/164 2024-05-14T16:42:09.960 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3687.g9b638668. 30/164 2024-05-14T16:42:09.960 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noa 31/164 2024-05-14T16:42:09.960 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.n 32/164 2024-05-14T16:42:09.960 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b6 33/164 2024-05-14T16:42:09.960 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3687.g9b638668.el 34/164 2024-05-14T16:42:09.960 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 35/164 2024-05-14T16:42:09.960 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3687.g9b638668.e 36/164 2024-05-14T16:42:09.961 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 37/164 2024-05-14T16:42:09.961 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephadm-2:19.0.0-3687.g9b638668.el9.noarch 38/164 2024-05-14T16:42:09.961 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephfs-top-2:19.0.0-3687.g9b638668.el9.noarch 39/164 2024-05-14T16:42:09.961 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-14T16:42:09.961 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-14T16:42:09.961 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-14T16:42:09.961 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-14T16:42:09.961 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-14T16:42:09.961 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-14T16:42:09.961 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-14T16:42:09.961 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-14T16:42:09.962 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-14T16:42:09.962 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-14T16:42:09.962 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-14T16:42:09.962 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-14T16:42:09.962 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-14T16:42:09.962 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-14T16:42:09.962 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-14T16:42:09.962 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-14T16:42:09.962 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-14T16:42:09.962 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-14T16:42:09.963 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-14T16:42:09.963 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-14T16:42:09.963 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-14T16:42:09.963 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-14T16:42:09.963 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-14T16:42:09.963 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-14T16:42:09.963 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-14T16:42:09.963 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-14T16:42:09.963 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-14T16:42:09.963 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-14T16:42:09.963 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-14T16:42:09.963 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-14T16:42:09.963 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-14T16:42:09.964 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-14T16:42:09.964 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-14T16:42:09.964 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-14T16:42:09.964 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-14T16:42:09.964 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-14T16:42:09.964 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-14T16:42:09.964 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-14T16:42:09.964 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-14T16:42:09.964 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-14T16:42:09.964 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-14T16:42:09.964 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-14T16:42:09.965 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-14T16:42:09.965 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-14T16:42:09.965 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-14T16:42:09.965 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-14T16:42:09.965 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-14T16:42:09.965 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-14T16:42:09.965 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-14T16:42:09.965 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-14T16:42:09.965 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-14T16:42:09.965 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-14T16:42:09.965 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-14T16:42:09.965 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-14T16:42:09.966 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-14T16:42:09.966 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-14T16:42:09.966 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-14T16:42:09.966 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-14T16:42:09.966 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-14T16:42:09.966 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-14T16:42:09.966 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-14T16:42:09.966 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-14T16:42:09.966 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-14T16:42:09.966 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-14T16:42:09.966 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-14T16:42:09.966 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-14T16:42:09.967 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-14T16:42:09.967 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-14T16:42:09.967 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-14T16:42:09.967 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-14T16:42:09.967 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-14T16:42:09.967 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-14T16:42:09.967 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-14T16:42:09.967 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-14T16:42:09.967 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-14T16:42:09.967 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-14T16:42:09.967 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-14T16:42:09.967 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-14T16:42:09.968 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-14T16:42:09.968 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-14T16:42:09.968 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-14T16:42:09.968 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-14T16:42:09.968 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-14T16:42:09.968 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-14T16:42:09.968 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-14T16:42:09.968 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-14T16:42:09.968 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-14T16:42:09.968 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-14T16:42:09.968 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-14T16:42:09.969 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-14T16:42:09.969 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-14T16:42:09.969 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-14T16:42:09.969 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-14T16:42:09.969 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-14T16:42:09.969 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-14T16:42:09.969 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-14T16:42:09.969 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-14T16:42:09.969 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-14T16:42:09.969 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-14T16:42:09.970 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-14T16:42:09.970 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-14T16:42:09.970 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-14T16:42:09.970 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-14T16:42:09.970 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-14T16:42:09.970 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-14T16:42:09.970 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-14T16:42:09.970 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-14T16:42:09.970 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-14T16:42:09.971 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-14T16:42:09.971 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librados2-2:19.0.0-3687.g9b638668.el9.x86_64 149/164 2024-05-14T16:42:09.971 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-14T16:42:09.971 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 151/164 2024-05-14T16:42:09.971 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-14T16:42:09.971 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/164 2024-05-14T16:42:09.971 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-14T16:42:09.971 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/164 2024-05-14T16:42:09.971 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-14T16:42:09.971 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/164 2024-05-14T16:42:09.971 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-14T16:42:09.971 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/164 2024-05-14T16:42:09.972 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-14T16:42:09.972 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-14T16:42:09.972 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-14T16:42:09.972 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-14T16:42:10.130 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T16:42:10.131 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-2:19.0.0-3687.g9b638668.el9.x86_64 1/164 2024-05-14T16:42:10.131 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 2/164 2024-05-14T16:42:10.131 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 3/164 2024-05-14T16:42:10.131 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 4/164 2024-05-14T16:42:10.131 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3687.g9b638 5/164 2024-05-14T16:42:10.131 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 6/164 2024-05-14T16:42:10.131 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 7/164 2024-05-14T16:42:10.131 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 8/164 2024-05-14T16:42:10.131 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 9/164 2024-05-14T16:42:10.131 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 10/164 2024-05-14T16:42:10.131 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 11/164 2024-05-14T16:42:10.132 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-test-2:19.0.0-3687.g9b638668.el9.x86_64 12/164 2024-05-14T16:42:10.132 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 13/164 2024-05-14T16:42:10.132 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-shell-2:19.0.0-3687.g9b638668.el9.x86_64 14/164 2024-05-14T16:42:10.132 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs-devel-2:19.0.0-3687.g9b638668.el9.x86_ 15/164 2024-05-14T16:42:10.132 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 16/164 2024-05-14T16:42:10.132 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 17/164 2024-05-14T16:42:10.132 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados-devel-2:19.0.0-3687.g9b638668.el9.x86_6 18/164 2024-05-14T16:42:10.132 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86 19/164 2024-05-14T16:42:10.132 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 20/164 2024-05-14T16:42:10.132 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3687.g9b638668.el 21/164 2024-05-14T16:42:10.132 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-common-2:19.0.0-3687.g9b638668.el9. 22/164 2024-05-14T16:42:10.132 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cephfs-2:19.0.0-3687.g9b638668.el9.x86_6 23/164 2024-05-14T16:42:10.133 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rados-2:19.0.0-3687.g9b638668.el9.x86_64 24/164 2024-05-14T16:42:10.133 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rbd-2:19.0.0-3687.g9b638668.el9.x86_64 25/164 2024-05-14T16:42:10.133 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rgw-2:19.0.0-3687.g9b638668.el9.x86_64 26/164 2024-05-14T16:42:10.133 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 27/164 2024-05-14T16:42:10.133 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 28/164 2024-05-14T16:42:10.133 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-nbd-2:19.0.0-3687.g9b638668.el9.x86_64 29/164 2024-05-14T16:42:10.133 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3687.g9b638668. 30/164 2024-05-14T16:42:10.133 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noa 31/164 2024-05-14T16:42:10.133 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.n 32/164 2024-05-14T16:42:10.133 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b6 33/164 2024-05-14T16:42:10.133 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3687.g9b638668.el 34/164 2024-05-14T16:42:10.133 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 35/164 2024-05-14T16:42:10.134 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3687.g9b638668.e 36/164 2024-05-14T16:42:10.134 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 37/164 2024-05-14T16:42:10.134 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephadm-2:19.0.0-3687.g9b638668.el9.noarch 38/164 2024-05-14T16:42:10.134 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-top-2:19.0.0-3687.g9b638668.el9.noarch 39/164 2024-05-14T16:42:10.134 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-14T16:42:10.134 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-14T16:42:10.134 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-14T16:42:10.134 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-14T16:42:10.134 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-14T16:42:10.134 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-14T16:42:10.135 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-14T16:42:10.135 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-14T16:42:10.135 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-14T16:42:10.135 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-14T16:42:10.135 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-14T16:42:10.135 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-14T16:42:10.135 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-14T16:42:10.135 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-14T16:42:10.135 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-14T16:42:10.135 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-14T16:42:10.135 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-14T16:42:10.136 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-14T16:42:10.136 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-14T16:42:10.136 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-14T16:42:10.136 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-14T16:42:10.136 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-14T16:42:10.136 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-14T16:42:10.136 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-14T16:42:10.136 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-14T16:42:10.136 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-14T16:42:10.137 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-14T16:42:10.137 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-14T16:42:10.137 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-14T16:42:10.137 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-14T16:42:10.137 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-14T16:42:10.137 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-14T16:42:10.137 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-14T16:42:10.137 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-14T16:42:10.137 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-14T16:42:10.137 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-14T16:42:10.137 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-14T16:42:10.137 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-14T16:42:10.137 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-14T16:42:10.138 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-14T16:42:10.138 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-14T16:42:10.138 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-14T16:42:10.138 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-14T16:42:10.138 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-14T16:42:10.138 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-14T16:42:10.138 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-14T16:42:10.138 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-14T16:42:10.138 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-14T16:42:10.138 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-14T16:42:10.138 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-14T16:42:10.139 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-14T16:42:10.139 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-14T16:42:10.139 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-14T16:42:10.139 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-14T16:42:10.139 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-14T16:42:10.139 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-14T16:42:10.139 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-14T16:42:10.139 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-14T16:42:10.139 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-14T16:42:10.139 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-14T16:42:10.139 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-14T16:42:10.139 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-14T16:42:10.139 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-14T16:42:10.140 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-14T16:42:10.140 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-14T16:42:10.140 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-14T16:42:10.140 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-14T16:42:10.140 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-14T16:42:10.140 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-14T16:42:10.140 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-14T16:42:10.140 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-14T16:42:10.140 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-14T16:42:10.140 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-14T16:42:10.140 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-14T16:42:10.141 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-14T16:42:10.141 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-14T16:42:10.141 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-14T16:42:10.141 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-14T16:42:10.141 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-14T16:42:10.141 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-14T16:42:10.141 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-14T16:42:10.141 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-14T16:42:10.141 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-14T16:42:10.141 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-14T16:42:10.141 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-14T16:42:10.141 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-14T16:42:10.141 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-14T16:42:10.142 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-14T16:42:10.142 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-14T16:42:10.142 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-14T16:42:10.142 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-14T16:42:10.142 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-14T16:42:10.142 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-14T16:42:10.142 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-14T16:42:10.142 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-14T16:42:10.142 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-14T16:42:10.142 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-14T16:42:10.142 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-14T16:42:10.143 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-14T16:42:10.143 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-14T16:42:10.143 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-14T16:42:10.143 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-14T16:42:10.143 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-14T16:42:10.143 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-14T16:42:10.143 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-14T16:42:10.143 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-14T16:42:10.143 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-14T16:42:10.143 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-14T16:42:10.143 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-14T16:42:10.143 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:19.0.0-3687.g9b638668.el9.x86_64 149/164 2024-05-14T16:42:10.143 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-14T16:42:10.143 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 151/164 2024-05-14T16:42:10.144 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-14T16:42:10.144 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/164 2024-05-14T16:42:10.144 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-14T16:42:10.144 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/164 2024-05-14T16:42:10.144 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-14T16:42:10.144 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/164 2024-05-14T16:42:10.144 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-14T16:42:10.144 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/164 2024-05-14T16:42:10.144 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-14T16:42:10.144 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-14T16:42:10.144 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-14T16:42:10.144 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-14T16:42:12.667 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T16:42:12.667 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-14T16:42:12.667 INFO:teuthology.orchestra.run.smithi151.stdout:Upgraded: 2024-05-14T16:42:12.667 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-14T16:42:12.667 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-14T16:42:12.667 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-14T16:42:12.667 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-14T16:42:12.667 INFO:teuthology.orchestra.run.smithi151.stdout: librados2-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.667 INFO:teuthology.orchestra.run.smithi151.stdout: librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.668 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-14T16:42:12.668 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-14T16:42:12.668 INFO:teuthology.orchestra.run.smithi151.stdout:Installed: 2024-05-14T16:42:12.668 INFO:teuthology.orchestra.run.smithi151.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-14T16:42:12.668 INFO:teuthology.orchestra.run.smithi151.stdout: automake-1.16.2-8.el9.noarch 2024-05-14T16:42:12.668 INFO:teuthology.orchestra.run.smithi151.stdout: bison-3.7.4-5.el9.x86_64 2024-05-14T16:42:12.668 INFO:teuthology.orchestra.run.smithi151.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-14T16:42:12.668 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.668 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.668 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.668 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.668 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-grafana-dashboards-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:42:12.669 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-immutable-object-cache-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.669 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.669 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.669 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:42:12.669 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:42:12.669 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:42:12.669 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-modules-core-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:42:12.669 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:42:12.669 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.669 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.669 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-prometheus-alerts-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:42:12.669 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.669 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.670 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-test-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.670 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:42:12.670 INFO:teuthology.orchestra.run.smithi151.stdout: cephadm-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:42:12.670 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.670 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-shell-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.670 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-top-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:42:12.670 INFO:teuthology.orchestra.run.smithi151.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-14T16:42:12.670 INFO:teuthology.orchestra.run.smithi151.stdout: flex-2.6.4-9.el9.x86_64 2024-05-14T16:42:12.670 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-14T16:42:12.670 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-14T16:42:12.670 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-14T16:42:12.670 INFO:teuthology.orchestra.run.smithi151.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-14T16:42:12.670 INFO:teuthology.orchestra.run.smithi151.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-14T16:42:12.670 INFO:teuthology.orchestra.run.smithi151.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-14T16:42:12.671 INFO:teuthology.orchestra.run.smithi151.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-14T16:42:12.671 INFO:teuthology.orchestra.run.smithi151.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-14T16:42:12.671 INFO:teuthology.orchestra.run.smithi151.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-14T16:42:12.671 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs-devel-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.671 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.671 INFO:teuthology.orchestra.run.smithi151.stdout: libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.671 INFO:teuthology.orchestra.run.smithi151.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-14T16:42:12.671 INFO:teuthology.orchestra.run.smithi151.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-14T16:42:12.671 INFO:teuthology.orchestra.run.smithi151.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-14T16:42:12.671 INFO:teuthology.orchestra.run.smithi151.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-14T16:42:12.671 INFO:teuthology.orchestra.run.smithi151.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-14T16:42:12.671 INFO:teuthology.orchestra.run.smithi151.stdout: librados-devel-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.671 INFO:teuthology.orchestra.run.smithi151.stdout: libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.671 INFO:teuthology.orchestra.run.smithi151.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-14T16:42:12.672 INFO:teuthology.orchestra.run.smithi151.stdout: librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.672 INFO:teuthology.orchestra.run.smithi151.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-14T16:42:12.672 INFO:teuthology.orchestra.run.smithi151.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-14T16:42:12.672 INFO:teuthology.orchestra.run.smithi151.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-14T16:42:12.672 INFO:teuthology.orchestra.run.smithi151.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-14T16:42:12.672 INFO:teuthology.orchestra.run.smithi151.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-14T16:42:12.672 INFO:teuthology.orchestra.run.smithi151.stdout: lua-5.4.4-4.el9.x86_64 2024-05-14T16:42:12.672 INFO:teuthology.orchestra.run.smithi151.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-14T16:42:12.672 INFO:teuthology.orchestra.run.smithi151.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-14T16:42:12.672 INFO:teuthology.orchestra.run.smithi151.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-14T16:42:12.672 INFO:teuthology.orchestra.run.smithi151.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-14T16:42:12.672 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-14T16:42:12.672 INFO:teuthology.orchestra.run.smithi151.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-14T16:42:12.672 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-13.14-1.el9.x86_64 2024-05-14T16:42:12.672 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-14T16:42:12.673 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-14T16:42:12.673 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-14T16:42:12.673 INFO:teuthology.orchestra.run.smithi151.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-14T16:42:12.673 INFO:teuthology.orchestra.run.smithi151.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-14T16:42:12.673 INFO:teuthology.orchestra.run.smithi151.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-14T16:42:12.673 INFO:teuthology.orchestra.run.smithi151.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-14T16:42:12.673 INFO:teuthology.orchestra.run.smithi151.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-14T16:42:12.673 INFO:teuthology.orchestra.run.smithi151.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-14T16:42:12.673 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-14T16:42:12.673 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-argparse-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.673 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.673 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cephfs-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.673 INFO:teuthology.orchestra.run.smithi151.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-14T16:42:12.673 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-14T16:42:12.673 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-14T16:42:12.674 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-14T16:42:12.674 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-14T16:42:12.674 INFO:teuthology.orchestra.run.smithi151.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-14T16:42:12.674 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-14T16:42:12.674 INFO:teuthology.orchestra.run.smithi151.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-14T16:42:12.674 INFO:teuthology.orchestra.run.smithi151.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-14T16:42:12.674 INFO:teuthology.orchestra.run.smithi151.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-14T16:42:12.674 INFO:teuthology.orchestra.run.smithi151.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-14T16:42:12.674 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-14T16:42:12.674 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-14T16:42:12.674 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-14T16:42:12.674 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-14T16:42:12.674 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-14T16:42:12.675 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-14T16:42:12.675 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-14T16:42:12.675 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-14T16:42:12.675 INFO:teuthology.orchestra.run.smithi151.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-14T16:42:12.675 INFO:teuthology.orchestra.run.smithi151.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-14T16:42:12.675 INFO:teuthology.orchestra.run.smithi151.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-14T16:42:12.675 INFO:teuthology.orchestra.run.smithi151.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-14T16:42:12.675 INFO:teuthology.orchestra.run.smithi151.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-14T16:42:12.675 INFO:teuthology.orchestra.run.smithi151.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-14T16:42:12.675 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-14T16:42:12.675 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-14T16:42:12.675 INFO:teuthology.orchestra.run.smithi151.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-14T16:42:12.675 INFO:teuthology.orchestra.run.smithi151.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-14T16:42:12.676 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-14T16:42:12.676 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ply-3.11-14.el9.noarch 2024-05-14T16:42:12.676 INFO:teuthology.orchestra.run.smithi151.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-14T16:42:12.676 INFO:teuthology.orchestra.run.smithi151.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-14T16:42:12.676 INFO:teuthology.orchestra.run.smithi151.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-14T16:42:12.676 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-14T16:42:12.676 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-14T16:42:12.676 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-14T16:42:12.676 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-14T16:42:12.676 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-14T16:42:12.676 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-14T16:42:12.676 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rados-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.676 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rbd-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.676 INFO:teuthology.orchestra.run.smithi151.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-14T16:42:12.676 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-14T16:42:12.676 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-14T16:42:12.677 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rgw-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.677 INFO:teuthology.orchestra.run.smithi151.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-14T16:42:12.677 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-14T16:42:12.677 INFO:teuthology.orchestra.run.smithi151.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-14T16:42:12.677 INFO:teuthology.orchestra.run.smithi151.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-14T16:42:12.677 INFO:teuthology.orchestra.run.smithi151.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-14T16:42:12.677 INFO:teuthology.orchestra.run.smithi151.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-14T16:42:12.677 INFO:teuthology.orchestra.run.smithi151.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-14T16:42:12.677 INFO:teuthology.orchestra.run.smithi151.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-14T16:42:12.677 INFO:teuthology.orchestra.run.smithi151.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-14T16:42:12.677 INFO:teuthology.orchestra.run.smithi151.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-14T16:42:12.677 INFO:teuthology.orchestra.run.smithi151.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-14T16:42:12.677 INFO:teuthology.orchestra.run.smithi151.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-14T16:42:12.678 INFO:teuthology.orchestra.run.smithi151.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-14T16:42:12.678 INFO:teuthology.orchestra.run.smithi151.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-14T16:42:12.678 INFO:teuthology.orchestra.run.smithi151.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-14T16:42:12.678 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.678 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.678 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-nbd-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.678 INFO:teuthology.orchestra.run.smithi151.stdout: re2-1:20211101-3.el9.x86_64 2024-05-14T16:42:12.678 INFO:teuthology.orchestra.run.smithi151.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-14T16:42:12.678 INFO:teuthology.orchestra.run.smithi151.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-14T16:42:12.678 INFO:teuthology.orchestra.run.smithi151.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-14T16:42:12.678 INFO:teuthology.orchestra.run.smithi151.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-14T16:42:12.678 INFO:teuthology.orchestra.run.smithi151.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-14T16:42:12.679 INFO:teuthology.orchestra.run.smithi151.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-14T16:42:12.679 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-14T16:42:12.679 INFO:teuthology.orchestra.run.smithi151.stdout:Complete! 2024-05-14T16:42:12.792 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T16:42:12.792 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-14T16:42:12.793 INFO:teuthology.orchestra.run.smithi123.stdout:Upgraded: 2024-05-14T16:42:12.793 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-14T16:42:12.793 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-14T16:42:12.793 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-14T16:42:12.793 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-14T16:42:12.793 INFO:teuthology.orchestra.run.smithi123.stdout: librados2-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.793 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.793 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-14T16:42:12.793 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-14T16:42:12.793 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-05-14T16:42:12.793 INFO:teuthology.orchestra.run.smithi123.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-14T16:42:12.793 INFO:teuthology.orchestra.run.smithi123.stdout: automake-1.16.2-8.el9.noarch 2024-05-14T16:42:12.793 INFO:teuthology.orchestra.run.smithi123.stdout: bison-3.7.4-5.el9.x86_64 2024-05-14T16:42:12.794 INFO:teuthology.orchestra.run.smithi123.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-14T16:42:12.794 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.794 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.794 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.794 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.794 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:42:12.794 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.794 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.794 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.794 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:42:12.794 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:42:12.795 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:42:12.795 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:42:12.795 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:42:12.795 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.795 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.795 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:42:12.795 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.795 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.795 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.795 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:42:12.795 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:42:12.795 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.795 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-shell-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.796 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:42:12.796 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-14T16:42:12.796 INFO:teuthology.orchestra.run.smithi123.stdout: flex-2.6.4-9.el9.x86_64 2024-05-14T16:42:12.796 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-14T16:42:12.796 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-14T16:42:12.796 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-14T16:42:12.796 INFO:teuthology.orchestra.run.smithi123.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-14T16:42:12.796 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-14T16:42:12.796 INFO:teuthology.orchestra.run.smithi123.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-14T16:42:12.796 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-14T16:42:12.796 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-14T16:42:12.796 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-14T16:42:12.796 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.797 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.797 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.797 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-14T16:42:12.797 INFO:teuthology.orchestra.run.smithi123.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-14T16:42:12.797 INFO:teuthology.orchestra.run.smithi123.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-14T16:42:12.797 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-14T16:42:12.797 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-14T16:42:12.797 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.797 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.797 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-14T16:42:12.797 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.797 INFO:teuthology.orchestra.run.smithi123.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-14T16:42:12.797 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-14T16:42:12.797 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-14T16:42:12.798 INFO:teuthology.orchestra.run.smithi123.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-14T16:42:12.798 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-14T16:42:12.798 INFO:teuthology.orchestra.run.smithi123.stdout: lua-5.4.4-4.el9.x86_64 2024-05-14T16:42:12.798 INFO:teuthology.orchestra.run.smithi123.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-14T16:42:12.798 INFO:teuthology.orchestra.run.smithi123.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-14T16:42:12.798 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-14T16:42:12.798 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-14T16:42:12.798 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-14T16:42:12.798 INFO:teuthology.orchestra.run.smithi123.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-14T16:42:12.798 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-13.14-1.el9.x86_64 2024-05-14T16:42:12.798 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-14T16:42:12.798 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-14T16:42:12.798 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-14T16:42:12.799 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-14T16:42:12.799 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-14T16:42:12.799 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-14T16:42:12.799 INFO:teuthology.orchestra.run.smithi123.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-14T16:42:12.799 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-14T16:42:12.799 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-14T16:42:12.799 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-14T16:42:12.799 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.799 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.799 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.799 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-14T16:42:12.799 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-14T16:42:12.799 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-14T16:42:12.799 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-14T16:42:12.800 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-14T16:42:12.800 INFO:teuthology.orchestra.run.smithi123.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-14T16:42:12.800 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-14T16:42:12.800 INFO:teuthology.orchestra.run.smithi123.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-14T16:42:12.800 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-14T16:42:12.800 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-14T16:42:12.800 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-14T16:42:12.800 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-14T16:42:12.800 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-14T16:42:12.800 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-14T16:42:12.800 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-14T16:42:12.800 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-14T16:42:12.800 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-14T16:42:12.800 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-14T16:42:12.801 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-14T16:42:12.801 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-14T16:42:12.801 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-14T16:42:12.801 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-14T16:42:12.801 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-14T16:42:12.801 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-14T16:42:12.801 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-14T16:42:12.801 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-14T16:42:12.801 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-14T16:42:12.801 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-14T16:42:12.802 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-14T16:42:12.802 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-14T16:42:12.802 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ply-3.11-14.el9.noarch 2024-05-14T16:42:12.802 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-14T16:42:12.802 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-14T16:42:12.802 INFO:teuthology.orchestra.run.smithi123.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-14T16:42:12.802 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-14T16:42:12.802 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-14T16:42:12.802 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-14T16:42:12.802 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-14T16:42:12.802 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-14T16:42:12.802 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-14T16:42:12.802 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.803 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.803 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-14T16:42:12.803 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-14T16:42:12.803 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-14T16:42:12.803 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.803 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-14T16:42:12.803 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-14T16:42:12.803 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-14T16:42:12.803 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-14T16:42:12.803 INFO:teuthology.orchestra.run.smithi123.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-14T16:42:12.803 INFO:teuthology.orchestra.run.smithi123.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-14T16:42:12.803 INFO:teuthology.orchestra.run.smithi123.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-14T16:42:12.804 INFO:teuthology.orchestra.run.smithi123.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-14T16:42:12.804 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-14T16:42:12.804 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-14T16:42:12.804 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-14T16:42:12.804 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-14T16:42:12.804 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-14T16:42:12.804 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-14T16:42:12.804 INFO:teuthology.orchestra.run.smithi123.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-14T16:42:12.804 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.804 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.804 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:12.804 INFO:teuthology.orchestra.run.smithi123.stdout: re2-1:20211101-3.el9.x86_64 2024-05-14T16:42:12.804 INFO:teuthology.orchestra.run.smithi123.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-14T16:42:12.804 INFO:teuthology.orchestra.run.smithi123.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-14T16:42:12.805 INFO:teuthology.orchestra.run.smithi123.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-14T16:42:12.805 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-14T16:42:12.805 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-14T16:42:12.805 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-14T16:42:12.805 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-05-14T16:42:12.805 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-05-14T16:42:13.082 DEBUG:teuthology.parallel:result is None 2024-05-14T16:42:13.150 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T16:42:13.150 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-2:19.0.0-3687.g9b638668.el9.x86_64 1/164 2024-05-14T16:42:13.150 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 2/164 2024-05-14T16:42:13.150 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 3/164 2024-05-14T16:42:13.150 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 4/164 2024-05-14T16:42:13.150 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3687.g9b638 5/164 2024-05-14T16:42:13.150 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 6/164 2024-05-14T16:42:13.150 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 7/164 2024-05-14T16:42:13.150 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 8/164 2024-05-14T16:42:13.150 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 9/164 2024-05-14T16:42:13.151 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 10/164 2024-05-14T16:42:13.151 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 11/164 2024-05-14T16:42:13.151 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-test-2:19.0.0-3687.g9b638668.el9.x86_64 12/164 2024-05-14T16:42:13.151 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 13/164 2024-05-14T16:42:13.151 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : cephfs-shell-2:19.0.0-3687.g9b638668.el9.x86_64 14/164 2024-05-14T16:42:13.151 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libcephfs-devel-2:19.0.0-3687.g9b638668.el9.x86_ 15/164 2024-05-14T16:42:13.151 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 16/164 2024-05-14T16:42:13.151 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 17/164 2024-05-14T16:42:13.151 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librados-devel-2:19.0.0-3687.g9b638668.el9.x86_6 18/164 2024-05-14T16:42:13.151 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86 19/164 2024-05-14T16:42:13.151 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 20/164 2024-05-14T16:42:13.151 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3687.g9b638668.el 21/164 2024-05-14T16:42:13.151 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-ceph-common-2:19.0.0-3687.g9b638668.el9. 22/164 2024-05-14T16:42:13.151 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cephfs-2:19.0.0-3687.g9b638668.el9.x86_6 23/164 2024-05-14T16:42:13.151 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rados-2:19.0.0-3687.g9b638668.el9.x86_64 24/164 2024-05-14T16:42:13.152 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rbd-2:19.0.0-3687.g9b638668.el9.x86_64 25/164 2024-05-14T16:42:13.152 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rgw-2:19.0.0-3687.g9b638668.el9.x86_64 26/164 2024-05-14T16:42:13.152 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : rbd-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 27/164 2024-05-14T16:42:13.152 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 28/164 2024-05-14T16:42:13.152 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : rbd-nbd-2:19.0.0-3687.g9b638668.el9.x86_64 29/164 2024-05-14T16:42:13.152 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3687.g9b638668. 30/164 2024-05-14T16:42:13.152 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noa 31/164 2024-05-14T16:42:13.152 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.n 32/164 2024-05-14T16:42:13.152 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b6 33/164 2024-05-14T16:42:13.152 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3687.g9b638668.el 34/164 2024-05-14T16:42:13.152 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 35/164 2024-05-14T16:42:13.152 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3687.g9b638668.e 36/164 2024-05-14T16:42:13.152 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 37/164 2024-05-14T16:42:13.153 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : cephadm-2:19.0.0-3687.g9b638668.el9.noarch 38/164 2024-05-14T16:42:13.153 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : cephfs-top-2:19.0.0-3687.g9b638668.el9.noarch 39/164 2024-05-14T16:42:13.153 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-14T16:42:13.153 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-14T16:42:13.153 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-14T16:42:13.153 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-14T16:42:13.153 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-14T16:42:13.153 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-14T16:42:13.153 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-14T16:42:13.153 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-14T16:42:13.153 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-14T16:42:13.153 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-14T16:42:13.153 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-14T16:42:13.153 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-14T16:42:13.154 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-14T16:42:13.154 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-14T16:42:13.154 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-14T16:42:13.154 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-14T16:42:13.154 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-14T16:42:13.154 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-14T16:42:13.154 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-14T16:42:13.154 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-14T16:42:13.154 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-14T16:42:13.154 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-14T16:42:13.154 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-14T16:42:13.154 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-14T16:42:13.154 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-14T16:42:13.154 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-14T16:42:13.155 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-14T16:42:13.155 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-14T16:42:13.155 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-14T16:42:13.155 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-14T16:42:13.155 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-14T16:42:13.155 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-14T16:42:13.155 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-14T16:42:13.155 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-14T16:42:13.155 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-14T16:42:13.155 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-14T16:42:13.155 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-14T16:42:13.155 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-14T16:42:13.155 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-14T16:42:13.155 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-14T16:42:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-14T16:42:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-14T16:42:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-14T16:42:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-14T16:42:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-14T16:42:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-14T16:42:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-14T16:42:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-14T16:42:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-14T16:42:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-14T16:42:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-14T16:42:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-14T16:42:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-14T16:42:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-14T16:42:13.156 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-14T16:42:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-14T16:42:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-14T16:42:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-14T16:42:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-14T16:42:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-14T16:42:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-14T16:42:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-14T16:42:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-14T16:42:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-14T16:42:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-14T16:42:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-14T16:42:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-14T16:42:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-14T16:42:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-14T16:42:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-14T16:42:13.157 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-14T16:42:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-14T16:42:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-14T16:42:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-14T16:42:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-14T16:42:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-14T16:42:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-14T16:42:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-14T16:42:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-14T16:42:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-14T16:42:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-14T16:42:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-14T16:42:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-14T16:42:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-14T16:42:13.158 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-14T16:42:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-14T16:42:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-14T16:42:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-14T16:42:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-14T16:42:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-14T16:42:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-14T16:42:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-14T16:42:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-14T16:42:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-14T16:42:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-14T16:42:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-14T16:42:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-14T16:42:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-14T16:42:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-14T16:42:13.159 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-14T16:42:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-14T16:42:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-14T16:42:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-14T16:42:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-14T16:42:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-14T16:42:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-14T16:42:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-14T16:42:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-14T16:42:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-14T16:42:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librados2-2:19.0.0-3687.g9b638668.el9.x86_64 149/164 2024-05-14T16:42:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-14T16:42:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 151/164 2024-05-14T16:42:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-14T16:42:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/164 2024-05-14T16:42:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-14T16:42:13.160 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/164 2024-05-14T16:42:13.161 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-14T16:42:13.161 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/164 2024-05-14T16:42:13.161 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-14T16:42:13.161 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/164 2024-05-14T16:42:13.161 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-14T16:42:13.161 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-14T16:42:13.161 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-14T16:42:13.161 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-14T16:42:13.184 DEBUG:teuthology.parallel:result is None 2024-05-14T16:42:15.839 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-14T16:42:15.839 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:42:15.839 INFO:teuthology.orchestra.run.smithi002.stdout:Upgraded: 2024-05-14T16:42:15.839 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-14T16:42:15.839 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-14T16:42:15.839 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-14T16:42:15.839 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-14T16:42:15.839 INFO:teuthology.orchestra.run.smithi002.stdout: librados2-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:15.839 INFO:teuthology.orchestra.run.smithi002.stdout: librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:15.839 INFO:teuthology.orchestra.run.smithi002.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-14T16:42:15.839 INFO:teuthology.orchestra.run.smithi002.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-14T16:42:15.840 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-05-14T16:42:15.840 INFO:teuthology.orchestra.run.smithi002.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-14T16:42:15.840 INFO:teuthology.orchestra.run.smithi002.stdout: automake-1.16.2-8.el9.noarch 2024-05-14T16:42:15.840 INFO:teuthology.orchestra.run.smithi002.stdout: bison-3.7.4-5.el9.x86_64 2024-05-14T16:42:15.840 INFO:teuthology.orchestra.run.smithi002.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-14T16:42:15.840 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:15.840 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:15.840 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:15.840 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:15.840 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-grafana-dashboards-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:42:15.840 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-immutable-object-cache-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:15.840 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:15.840 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:15.840 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:42:15.841 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:42:15.841 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:42:15.841 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-modules-core-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:42:15.841 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:42:15.841 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:15.841 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:15.841 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-prometheus-alerts-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:42:15.841 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:15.841 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:15.841 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-test-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:15.841 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:42:15.841 INFO:teuthology.orchestra.run.smithi002.stdout: cephadm-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:42:15.841 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:15.842 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-shell-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:15.842 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-top-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-14T16:42:15.842 INFO:teuthology.orchestra.run.smithi002.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-14T16:42:15.842 INFO:teuthology.orchestra.run.smithi002.stdout: flex-2.6.4-9.el9.x86_64 2024-05-14T16:42:15.842 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-14T16:42:15.842 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-14T16:42:15.842 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-14T16:42:15.842 INFO:teuthology.orchestra.run.smithi002.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-14T16:42:15.842 INFO:teuthology.orchestra.run.smithi002.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-14T16:42:15.842 INFO:teuthology.orchestra.run.smithi002.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-14T16:42:15.842 INFO:teuthology.orchestra.run.smithi002.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-14T16:42:15.842 INFO:teuthology.orchestra.run.smithi002.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-14T16:42:15.842 INFO:teuthology.orchestra.run.smithi002.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-14T16:42:15.842 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs-devel-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:15.843 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:15.843 INFO:teuthology.orchestra.run.smithi002.stdout: libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:15.843 INFO:teuthology.orchestra.run.smithi002.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-14T16:42:15.843 INFO:teuthology.orchestra.run.smithi002.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-14T16:42:15.843 INFO:teuthology.orchestra.run.smithi002.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-14T16:42:15.843 INFO:teuthology.orchestra.run.smithi002.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-14T16:42:15.843 INFO:teuthology.orchestra.run.smithi002.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-14T16:42:15.843 INFO:teuthology.orchestra.run.smithi002.stdout: librados-devel-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:15.843 INFO:teuthology.orchestra.run.smithi002.stdout: libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:15.843 INFO:teuthology.orchestra.run.smithi002.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-14T16:42:15.843 INFO:teuthology.orchestra.run.smithi002.stdout: librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:15.843 INFO:teuthology.orchestra.run.smithi002.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-14T16:42:15.843 INFO:teuthology.orchestra.run.smithi002.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-14T16:42:15.843 INFO:teuthology.orchestra.run.smithi002.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-14T16:42:15.844 INFO:teuthology.orchestra.run.smithi002.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-14T16:42:15.844 INFO:teuthology.orchestra.run.smithi002.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-14T16:42:15.844 INFO:teuthology.orchestra.run.smithi002.stdout: lua-5.4.4-4.el9.x86_64 2024-05-14T16:42:15.844 INFO:teuthology.orchestra.run.smithi002.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-14T16:42:15.844 INFO:teuthology.orchestra.run.smithi002.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-14T16:42:15.844 INFO:teuthology.orchestra.run.smithi002.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-14T16:42:15.844 INFO:teuthology.orchestra.run.smithi002.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-14T16:42:15.844 INFO:teuthology.orchestra.run.smithi002.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-14T16:42:15.844 INFO:teuthology.orchestra.run.smithi002.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-14T16:42:15.844 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-13.14-1.el9.x86_64 2024-05-14T16:42:15.844 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-14T16:42:15.844 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-14T16:42:15.844 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-14T16:42:15.844 INFO:teuthology.orchestra.run.smithi002.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-14T16:42:15.844 INFO:teuthology.orchestra.run.smithi002.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-14T16:42:15.845 INFO:teuthology.orchestra.run.smithi002.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-14T16:42:15.845 INFO:teuthology.orchestra.run.smithi002.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-14T16:42:15.845 INFO:teuthology.orchestra.run.smithi002.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-14T16:42:15.845 INFO:teuthology.orchestra.run.smithi002.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-14T16:42:15.845 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-14T16:42:15.845 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-argparse-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:15.845 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:15.845 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cephfs-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:15.845 INFO:teuthology.orchestra.run.smithi002.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-14T16:42:15.845 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-14T16:42:15.845 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-14T16:42:15.845 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-14T16:42:15.845 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-14T16:42:15.845 INFO:teuthology.orchestra.run.smithi002.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-14T16:42:15.845 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-14T16:42:15.846 INFO:teuthology.orchestra.run.smithi002.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-14T16:42:15.846 INFO:teuthology.orchestra.run.smithi002.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-14T16:42:15.846 INFO:teuthology.orchestra.run.smithi002.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-14T16:42:15.846 INFO:teuthology.orchestra.run.smithi002.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-14T16:42:15.846 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-14T16:42:15.846 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-14T16:42:15.846 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-14T16:42:15.846 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-14T16:42:15.846 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-14T16:42:15.846 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-14T16:42:15.846 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-14T16:42:15.846 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-14T16:42:15.846 INFO:teuthology.orchestra.run.smithi002.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-14T16:42:15.846 INFO:teuthology.orchestra.run.smithi002.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-14T16:42:15.847 INFO:teuthology.orchestra.run.smithi002.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-14T16:42:15.847 INFO:teuthology.orchestra.run.smithi002.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-14T16:42:15.847 INFO:teuthology.orchestra.run.smithi002.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-14T16:42:15.847 INFO:teuthology.orchestra.run.smithi002.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-14T16:42:15.847 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-14T16:42:15.847 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-14T16:42:15.847 INFO:teuthology.orchestra.run.smithi002.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-14T16:42:15.847 INFO:teuthology.orchestra.run.smithi002.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-14T16:42:15.847 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-14T16:42:15.847 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ply-3.11-14.el9.noarch 2024-05-14T16:42:15.847 INFO:teuthology.orchestra.run.smithi002.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-14T16:42:15.847 INFO:teuthology.orchestra.run.smithi002.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-14T16:42:15.847 INFO:teuthology.orchestra.run.smithi002.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-14T16:42:15.847 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-14T16:42:15.848 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-14T16:42:15.848 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-14T16:42:15.848 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-14T16:42:15.848 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-14T16:42:15.848 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-14T16:42:15.848 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rados-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:15.848 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rbd-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:15.848 INFO:teuthology.orchestra.run.smithi002.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-14T16:42:15.848 INFO:teuthology.orchestra.run.smithi002.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-14T16:42:15.848 INFO:teuthology.orchestra.run.smithi002.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-14T16:42:15.848 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rgw-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:15.848 INFO:teuthology.orchestra.run.smithi002.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-14T16:42:15.848 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-14T16:42:15.848 INFO:teuthology.orchestra.run.smithi002.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-14T16:42:15.848 INFO:teuthology.orchestra.run.smithi002.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-14T16:42:15.849 INFO:teuthology.orchestra.run.smithi002.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-14T16:42:15.849 INFO:teuthology.orchestra.run.smithi002.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-14T16:42:15.849 INFO:teuthology.orchestra.run.smithi002.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-14T16:42:15.849 INFO:teuthology.orchestra.run.smithi002.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-14T16:42:15.849 INFO:teuthology.orchestra.run.smithi002.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-14T16:42:15.849 INFO:teuthology.orchestra.run.smithi002.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-14T16:42:15.849 INFO:teuthology.orchestra.run.smithi002.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-14T16:42:15.849 INFO:teuthology.orchestra.run.smithi002.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-14T16:42:15.849 INFO:teuthology.orchestra.run.smithi002.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-14T16:42:15.849 INFO:teuthology.orchestra.run.smithi002.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-14T16:42:15.849 INFO:teuthology.orchestra.run.smithi002.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-14T16:42:15.849 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:15.849 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:15.849 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-nbd-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-14T16:42:15.849 INFO:teuthology.orchestra.run.smithi002.stdout: re2-1:20211101-3.el9.x86_64 2024-05-14T16:42:15.850 INFO:teuthology.orchestra.run.smithi002.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-14T16:42:15.850 INFO:teuthology.orchestra.run.smithi002.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-14T16:42:15.850 INFO:teuthology.orchestra.run.smithi002.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-14T16:42:15.850 INFO:teuthology.orchestra.run.smithi002.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-14T16:42:15.850 INFO:teuthology.orchestra.run.smithi002.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-14T16:42:15.850 INFO:teuthology.orchestra.run.smithi002.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-14T16:42:15.850 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:42:15.850 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-05-14T16:42:16.269 DEBUG:teuthology.parallel:result is None 2024-05-14T16:42:16.269 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-14T16:42:16.417 DEBUG:teuthology.orchestra.run.smithi002:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-14T16:42:16.440 INFO:teuthology.orchestra.run.smithi002.stdout:19.0.0-3687.g9b638668.el9 2024-05-14T16:42:16.440 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3687.g9b638668.el9 2024-05-14T16:42:16.440 INFO:teuthology.task.install:The correct ceph version 19.0.0-3687.g9b638668 is installed. 2024-05-14T16:42:16.442 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-14T16:42:16.688 DEBUG:teuthology.orchestra.run.smithi123:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-14T16:42:16.712 INFO:teuthology.orchestra.run.smithi123.stdout:19.0.0-3687.g9b638668.el9 2024-05-14T16:42:16.713 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3687.g9b638668.el9 2024-05-14T16:42:16.713 INFO:teuthology.task.install:The correct ceph version 19.0.0-3687.g9b638668 is installed. 2024-05-14T16:42:16.715 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-14T16:42:16.987 DEBUG:teuthology.orchestra.run.smithi151:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-14T16:42:17.013 INFO:teuthology.orchestra.run.smithi151.stdout:19.0.0-3687.g9b638668.el9 2024-05-14T16:42:17.013 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3687.g9b638668.el9 2024-05-14T16:42:17.013 INFO:teuthology.task.install:The correct ceph version 19.0.0-3687.g9b638668 is installed. 2024-05-14T16:42:17.015 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-14T16:42:17.015 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T16:42:17.016 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-14T16:42:17.049 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-05-14T16:42:17.049 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-14T16:42:17.081 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-14T16:42:17.081 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-14T16:42:17.111 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-14T16:42:17.111 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T16:42:17.111 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/usr/bin/daemon-helper 2024-05-14T16:42:17.142 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-14T16:42:17.211 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-05-14T16:42:17.211 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/daemon-helper 2024-05-14T16:42:17.240 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-14T16:42:17.307 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-14T16:42:17.307 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/daemon-helper 2024-05-14T16:42:17.335 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-14T16:42:17.401 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-14T16:42:17.401 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T16:42:17.401 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-14T16:42:17.431 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-14T16:42:17.501 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-05-14T16:42:17.501 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-14T16:42:17.529 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-14T16:42:17.597 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-14T16:42:17.597 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-14T16:42:17.622 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-14T16:42:17.689 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-14T16:42:17.689 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T16:42:17.689 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/usr/bin/stdin-killer 2024-05-14T16:42:17.720 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-14T16:42:17.789 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-05-14T16:42:17.790 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/stdin-killer 2024-05-14T16:42:17.817 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-14T16:42:17.884 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-14T16:42:17.884 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/stdin-killer 2024-05-14T16:42:17.912 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-14T16:42:17.979 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-14T16:42:18.096 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '9b63866860f0b71216b2c0a73aedd2e0a8a5c98b', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-05-14T16:42:18.096 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-14T16:42:18.097 INFO:tasks.cephadm:Cluster fsid is ed40717a-1210-11ef-bc98-c7b262605968 2024-05-14T16:42:18.097 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-14T16:42:18.097 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.2', 'mon.b': '172.21.15.123', 'mon.c': '172.21.15.151'} 2024-05-14T16:42:18.098 INFO:tasks.cephadm:First mon is mon.a on smithi002 2024-05-14T16:42:18.098 INFO:tasks.cephadm:First mgr is x 2024-05-14T16:42:18.098 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-14T16:42:18.098 DEBUG:teuthology.orchestra.run.smithi002:> sudo hostname $(hostname -s) 2024-05-14T16:42:18.130 DEBUG:teuthology.orchestra.run.smithi123:> sudo hostname $(hostname -s) 2024-05-14T16:42:18.159 DEBUG:teuthology.orchestra.run.smithi151:> sudo hostname $(hostname -s) 2024-05-14T16:42:18.187 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-14T16:42:18.188 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-14T16:42:18.335 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '9b63866860f0b71216b2c0a73aedd2e0a8a5c98b', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79371/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3687-g9b638668', 'node_name': '172.21.5.34+adami04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3687.g9b638668'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-14 15:14:57.349842', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiesce', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-05-14T16:42:18.477 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-lusov-quiesce, sha1 9b63866860f0b71216b2c0a73aedd2e0a8a5c98b from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-14T16:42:18.478 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/x86_64/flavors/default/cephadm 2024-05-14T16:42:18.478 INFO:tasks.cephadm:Downloading cephadm from url: https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/x86_64/flavors/default/cephadm 2024-05-14T16:42:18.479 DEBUG:teuthology.orchestra.run.smithi002:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-14T16:42:19.387 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-r--r--. 1 ubuntu ubuntu 782726 May 14 16:42 /home/ubuntu/cephtest/cephadm 2024-05-14T16:42:19.388 DEBUG:teuthology.orchestra.run.smithi123:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-14T16:42:20.929 INFO:teuthology.orchestra.run.smithi123.stdout:-rw-r--r--. 1 ubuntu ubuntu 782726 May 14 16:42 /home/ubuntu/cephtest/cephadm 2024-05-14T16:42:20.929 DEBUG:teuthology.orchestra.run.smithi151:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-14T16:42:21.587 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r--. 1 ubuntu ubuntu 782726 May 14 16:42 /home/ubuntu/cephtest/cephadm 2024-05-14T16:42:21.587 DEBUG:teuthology.orchestra.run.smithi002:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-14T16:42:21.607 DEBUG:teuthology.orchestra.run.smithi123:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-14T16:42:21.626 DEBUG:teuthology.orchestra.run.smithi151:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-14T16:42:21.660 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b on all hosts... 2024-05-14T16:42:21.661 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b pull 2024-05-14T16:42:21.663 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b pull 2024-05-14T16:42:21.668 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b pull 2024-05-14T16:42:22.510 INFO:teuthology.orchestra.run.smithi002.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b... 2024-05-14T16:42:22.605 INFO:teuthology.orchestra.run.smithi151.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b... 2024-05-14T16:42:22.615 INFO:teuthology.orchestra.run.smithi123.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b... 2024-05-14T16:42:50.538 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-05-14T16:42:50.538 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_version": "ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev)", 2024-05-14T16:42:50.538 INFO:teuthology.orchestra.run.smithi123.stdout: "image_id": "c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a", 2024-05-14T16:42:50.538 INFO:teuthology.orchestra.run.smithi123.stdout: "repo_digests": [ 2024-05-14T16:42:50.538 INFO:teuthology.orchestra.run.smithi123.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d" 2024-05-14T16:42:50.538 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-05-14T16:42:50.539 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-05-14T16:42:57.336 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T16:42:57.336 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_version": "ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev)", 2024-05-14T16:42:57.336 INFO:teuthology.orchestra.run.smithi002.stdout: "image_id": "c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a", 2024-05-14T16:42:57.336 INFO:teuthology.orchestra.run.smithi002.stdout: "repo_digests": [ 2024-05-14T16:42:57.336 INFO:teuthology.orchestra.run.smithi002.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d" 2024-05-14T16:42:57.336 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-05-14T16:42:57.336 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T16:43:00.296 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-14T16:43:00.296 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_version": "ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev)", 2024-05-14T16:43:00.296 INFO:teuthology.orchestra.run.smithi151.stdout: "image_id": "c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a", 2024-05-14T16:43:00.297 INFO:teuthology.orchestra.run.smithi151.stdout: "repo_digests": [ 2024-05-14T16:43:00.297 INFO:teuthology.orchestra.run.smithi151.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d" 2024-05-14T16:43:00.297 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-05-14T16:43:00.297 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-14T16:43:00.321 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /etc/ceph 2024-05-14T16:43:00.355 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /etc/ceph 2024-05-14T16:43:00.388 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /etc/ceph 2024-05-14T16:43:00.415 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 777 /etc/ceph 2024-05-14T16:43:00.445 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 777 /etc/ceph 2024-05-14T16:43:00.473 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 777 /etc/ceph 2024-05-14T16:43:00.497 INFO:tasks.cephadm:Writing seed config... 2024-05-14T16:43:00.498 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-14T16:43:00.498 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-14T16:43:00.498 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-14T16:43:00.498 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-14T16:43:00.498 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-14T16:43:00.498 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-14T16:43:00.498 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-14T16:43:00.498 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-14T16:43:00.499 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-14T16:43:00.499 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-14T16:43:00.499 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-14T16:43:00.499 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-14T16:43:00.499 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-14T16:43:00.499 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-14T16:43:00.499 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-14T16:43:00.499 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-14T16:43:00.499 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-14T16:43:00.499 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-14T16:43:00.499 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-14T16:43:00.499 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-14T16:43:00.499 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-14T16:43:00.499 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-05-14T16:43:00.499 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-14T16:43:00.499 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-14T16:43:00.499 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-14T16:43:00.500 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-14T16:43:00.500 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-14T16:43:00.500 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-14T16:43:00.500 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-14T16:43:00.500 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-14T16:43:00.500 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-14T16:43:00.500 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-05-14T16:43:00.500 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-14T16:43:00.500 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-14T16:43:00.500 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-14T16:43:00.500 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-14T16:43:00.500 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-14T16:43:00.500 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-14T16:43:00.500 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-14T16:43:00.500 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-14T16:43:00.500 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-14T16:43:00.501 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-14T16:43:00.501 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-14T16:43:00.501 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-14T16:43:00.501 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-14T16:43:00.501 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-05-14T16:43:00.501 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T16:43:00.501 DEBUG:teuthology.orchestra.run.smithi002:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-14T16:43:00.521 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = ed40717a-1210-11ef-bc98-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-14T16:43:00.521 DEBUG:teuthology.orchestra.run.smithi002:mon.a> sudo journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@mon.a.service 2024-05-14T16:43:00.563 DEBUG:teuthology.orchestra.run.smithi002:mgr.x> sudo journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@mgr.x.service 2024-05-14T16:43:00.607 INFO:tasks.cephadm:Bootstrapping... 2024-05-14T16:43:00.607 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b -v bootstrap --fsid ed40717a-1210-11ef-bc98-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.2 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-14T16:43:00.810 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-05-14T16:43:00.810 INFO:teuthology.orchestra.run.smithi002.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b', '-v', 'bootstrap', '--fsid', 'ed40717a-1210-11ef-bc98-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.2', '--skip-admin-label'] 2024-05-14T16:43:00.811 INFO:teuthology.orchestra.run.smithi002.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-14T16:43:00.811 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying podman|docker is present... 2024-05-14T16:43:00.836 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stdout 5.0.2 2024-05-14T16:43:00.837 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying lvm2 is present... 2024-05-14T16:43:00.837 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying time synchronization is in place... 2024-05-14T16:43:00.845 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-14T16:43:00.845 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-14T16:43:00.851 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-14T16:43:00.851 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2024-05-14T16:43:00.858 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout enabled 2024-05-14T16:43:00.865 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout active 2024-05-14T16:43:00.865 INFO:teuthology.orchestra.run.smithi002.stdout:Unit chronyd.service is enabled and running 2024-05-14T16:43:00.865 INFO:teuthology.orchestra.run.smithi002.stdout:Repeating the final host check... 2024-05-14T16:43:00.889 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stdout 5.0.2 2024-05-14T16:43:00.889 INFO:teuthology.orchestra.run.smithi002.stdout:podman (/bin/podman) version 5.0.2 is present 2024-05-14T16:43:00.890 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl is present 2024-05-14T16:43:00.890 INFO:teuthology.orchestra.run.smithi002.stdout:lvcreate is present 2024-05-14T16:43:00.896 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-14T16:43:00.896 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-14T16:43:00.903 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-14T16:43:00.903 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2024-05-14T16:43:00.909 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout enabled 2024-05-14T16:43:00.916 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout active 2024-05-14T16:43:00.916 INFO:teuthology.orchestra.run.smithi002.stdout:Unit chronyd.service is enabled and running 2024-05-14T16:43:00.916 INFO:teuthology.orchestra.run.smithi002.stdout:Host looks OK 2024-05-14T16:43:00.916 INFO:teuthology.orchestra.run.smithi002.stdout:Cluster fsid: ed40717a-1210-11ef-bc98-c7b262605968 2024-05-14T16:43:00.916 INFO:teuthology.orchestra.run.smithi002.stdout:Acquiring lock 140618366521552 on /run/cephadm/ed40717a-1210-11ef-bc98-c7b262605968.lock 2024-05-14T16:43:00.916 INFO:teuthology.orchestra.run.smithi002.stdout:Lock 140618366521552 acquired on /run/cephadm/ed40717a-1210-11ef-bc98-c7b262605968.lock 2024-05-14T16:43:00.917 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying IP 172.21.15.2 port 3300 ... 2024-05-14T16:43:00.917 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying IP 172.21.15.2 port 6789 ... 2024-05-14T16:43:00.917 INFO:teuthology.orchestra.run.smithi002.stdout:Base mon IP(s) is [172.21.15.2:3300, 172.21.15.2:6789], mon addrv is [v2:172.21.15.2:3300,v1:172.21.15.2:6789] 2024-05-14T16:43:00.921 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.2 metric 100 2024-05-14T16:43:00.921 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.2 metric 100 2024-05-14T16:43:00.923 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-14T16:43:00.923 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-05-14T16:43:00.923 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-05-14T16:43:00.926 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-14T16:43:00.926 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-14T16:43:00.926 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-14T16:43:00.926 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-05-14T16:43:00.926 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1920/64 scope link noprefixroute 2024-05-14T16:43:00.926 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-14T16:43:00.927 INFO:teuthology.orchestra.run.smithi002.stdout:Mon IP `172.21.15.2` is in CIDR network `172.21.0.0/20` 2024-05-14T16:43:00.927 INFO:teuthology.orchestra.run.smithi002.stdout:Mon IP `172.21.15.2` is in CIDR network `172.21.0.0/20` 2024-05-14T16:43:00.927 INFO:teuthology.orchestra.run.smithi002.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-14T16:43:00.927 INFO:teuthology.orchestra.run.smithi002.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-14T16:43:00.928 INFO:teuthology.orchestra.run.smithi002.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b... 2024-05-14T16:43:01.396 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stdout c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a 2024-05-14T16:43:01.396 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b... 2024-05-14T16:43:01.396 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Getting image source signatures 2024-05-14T16:43:01.396 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Copying blob sha256:959e8422c821254139eba7e305bc0e5ccc9179a9c74388bbc27d9cf0e33e5891 2024-05-14T16:43:01.396 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Copying blob sha256:d0e808cd455827ca3108db1b0125dcee9ffc148daa1d7dde963f550bc831d501 2024-05-14T16:43:01.396 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Copying config sha256:c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a 2024-05-14T16:43:01.396 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-14T16:43:02.401 INFO:teuthology.orchestra.run.smithi002.stdout:ceph: stdout ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev) 2024-05-14T16:43:02.401 INFO:teuthology.orchestra.run.smithi002.stdout:Ceph version: ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev) 2024-05-14T16:43:02.401 INFO:teuthology.orchestra.run.smithi002.stdout:Extracting ceph user uid/gid from container image... 2024-05-14T16:43:03.390 INFO:teuthology.orchestra.run.smithi002.stdout:stat: stdout 167 167 2024-05-14T16:43:03.390 INFO:teuthology.orchestra.run.smithi002.stdout:Creating initial keys... 2024-05-14T16:43:04.279 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph-authtool: stdout AQCXlENm6enEKhAA57p1iLNiYd5fPwM67djiYQ== 2024-05-14T16:43:05.244 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph-authtool: stdout AQCYlENmzNhDIhAAnbPQfcF9xnEPDZQN58JFrQ== 2024-05-14T16:43:06.216 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph-authtool: stdout AQCZlENmsbIQIRAAJ7nIsckRlPn+a/PzWJERFw== 2024-05-14T16:43:06.217 INFO:teuthology.orchestra.run.smithi002.stdout:Creating initial monmap... 2024-05-14T16:43:07.147 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-14T16:43:07.148 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-14T16:43:07.148 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to ed40717a-1210-11ef-bc98-c7b262605968 2024-05-14T16:43:07.148 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-14T16:43:07.148 INFO:teuthology.orchestra.run.smithi002.stdout:monmaptool for a [v2:172.21.15.2:3300,v1:172.21.15.2:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-14T16:43:07.148 INFO:teuthology.orchestra.run.smithi002.stdout:setting min_mon_release = quincy 2024-05-14T16:43:07.148 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: set fsid to ed40717a-1210-11ef-bc98-c7b262605968 2024-05-14T16:43:07.148 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-14T16:43:07.148 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:43:07.148 INFO:teuthology.orchestra.run.smithi002.stdout:Creating mon... 2024-05-14T16:43:07.977 INFO:teuthology.orchestra.run.smithi002.stdout:create mon.a on 2024-05-14T16:43:08.187 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-14T16:43:08.341 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-14T16:43:08.519 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-ed40717a-1210-11ef-bc98-c7b262605968.target → /etc/systemd/system/ceph-ed40717a-1210-11ef-bc98-c7b262605968.target. 2024-05-14T16:43:08.519 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-ed40717a-1210-11ef-bc98-c7b262605968.target → /etc/systemd/system/ceph-ed40717a-1210-11ef-bc98-c7b262605968.target. 2024-05-14T16:43:08.762 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-ed40717a-1210-11ef-bc98-c7b262605968@mon.a 2024-05-14T16:43:08.762 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Failed to reset failed state of unit ceph-ed40717a-1210-11ef-bc98-c7b262605968@mon.a.service: Unit ceph-ed40717a-1210-11ef-bc98-c7b262605968@mon.a.service not loaded. 2024-05-14T16:43:08.926 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-ed40717a-1210-11ef-bc98-c7b262605968.target.wants/ceph-ed40717a-1210-11ef-bc98-c7b262605968@mon.a.service → /etc/systemd/system/ceph-ed40717a-1210-11ef-bc98-c7b262605968@.service. 2024-05-14T16:43:09.460 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-14T16:43:09.460 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout disabled 2024-05-14T16:43:09.467 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-14T16:43:09.467 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2024-05-14T16:43:09.467 INFO:teuthology.orchestra.run.smithi002.stdout:firewalld.service is not enabled 2024-05-14T16:43:09.467 INFO:teuthology.orchestra.run.smithi002.stdout:Not possible to enable service . firewalld.service is not available 2024-05-14T16:43:09.468 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mon to start... 2024-05-14T16:43:09.468 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mon... 2024-05-14T16:43:10.502 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:10 smithi002 ceph-mon[31168]: from='client.? 172.21.15.2:0/2087463414' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-14T16:43:10.505 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout cluster: 2024-05-14T16:43:10.512 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout id: ed40717a-1210-11ef-bc98-c7b262605968 2024-05-14T16:43:10.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-14T16:43:10.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-05-14T16:43:10.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout services: 2024-05-14T16:43:10.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.551363s) 2024-05-14T16:43:10.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-14T16:43:10.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-14T16:43:10.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-05-14T16:43:10.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout data: 2024-05-14T16:43:10.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-14T16:43:10.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-14T16:43:10.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-14T16:43:10.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout pgs: 2024-05-14T16:43:10.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-05-14T16:43:10.513 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.884+0000 7f41cb63f700 1 Processor -- start 2024-05-14T16:43:10.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.884+0000 7f41cb63f700 1 -- start start 2024-05-14T16:43:10.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.884+0000 7f41cb63f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41c4107de0 0x7f41c4108200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:10.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.885+0000 7f41cb63f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f41c41087d0 con 0x7f41c4107de0 2024-05-14T16:43:10.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.885+0000 7f41c93db700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41c4107de0 0x7f41c4108200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:10.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.885+0000 7f41c93db700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41c4107de0 0x7f41c4108200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40428/0 (socket says 172.21.15.2:40428) 2024-05-14T16:43:10.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.885+0000 7f41c93db700 1 -- 172.21.15.2:0/291679446 learned_addr learned my addr 172.21.15.2:0/291679446 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:43:10.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.885+0000 7f41c93db700 1 -- 172.21.15.2:0/291679446 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c4108fe0 con 0x7f41c4107de0 2024-05-14T16:43:10.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.885+0000 7f41c93db700 1 --2- 172.21.15.2:0/291679446 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41c4107de0 0x7f41c4108200 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f41c0009e80 tx=0x7f41c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eec0e975bb471b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:10.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.886+0000 7f41bbfff700 1 -- 172.21.15.2:0/291679446 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41c0017070 con 0x7f41c4107de0 2024-05-14T16:43:10.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.886+0000 7f41bbfff700 1 -- 172.21.15.2:0/291679446 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41c0003a20 con 0x7f41c4107de0 2024-05-14T16:43:10.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.886+0000 7f41bbfff700 1 -- 172.21.15.2:0/291679446 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41c0003d20 con 0x7f41c4107de0 2024-05-14T16:43:10.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.886+0000 7f41cb63f700 1 -- 172.21.15.2:0/291679446 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41c4107de0 msgr2=0x7f41c4108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:10.514 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.886+0000 7f41cb63f700 1 --2- 172.21.15.2:0/291679446 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41c4107de0 0x7f41c4108200 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f41c0009e80 tx=0x7f41c000b330 comp rx=0 tx=0).stop 2024-05-14T16:43:10.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.886+0000 7f41cb63f700 1 -- 172.21.15.2:0/291679446 shutdown_connections 2024-05-14T16:43:10.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.886+0000 7f41cb63f700 1 --2- 172.21.15.2:0/291679446 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41c4107de0 0x7f41c4108200 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:10.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.886+0000 7f41cb63f700 1 -- 172.21.15.2:0/291679446 >> 172.21.15.2:0/291679446 conn(0x7f41c41033a0 msgr2=0x7f41c41057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:43:10.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.886+0000 7f41cb63f700 1 -- 172.21.15.2:0/291679446 shutdown_connections 2024-05-14T16:43:10.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.886+0000 7f41cb63f700 1 -- 172.21.15.2:0/291679446 wait complete. 2024-05-14T16:43:10.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.887+0000 7f41cb63f700 1 Processor -- start 2024-05-14T16:43:10.515 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.887+0000 7f41cb63f700 1 -- start start 2024-05-14T16:43:10.516 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.887+0000 7f41cb63f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41c4107de0 0x7f41c419c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:10.516 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.887+0000 7f41cb63f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f41c4109450 con 0x7f41c4107de0 2024-05-14T16:43:10.516 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.887+0000 7f41c93db700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41c4107de0 0x7f41c419c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:10.516 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.887+0000 7f41c93db700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41c4107de0 0x7f41c419c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40432/0 (socket says 172.21.15.2:40432) 2024-05-14T16:43:10.516 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.887+0000 7f41c93db700 1 -- 172.21.15.2:0/2087463414 learned_addr learned my addr 172.21.15.2:0/2087463414 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:43:10.516 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.888+0000 7f41c93db700 1 -- 172.21.15.2:0/2087463414 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c419cb90 con 0x7f41c4107de0 2024-05-14T16:43:10.516 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.888+0000 7f41c93db700 1 --2- 172.21.15.2:0/2087463414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41c4107de0 0x7f41c419c650 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f41c0004000 tx=0x7f41c0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:10.516 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.888+0000 7f41ba7fc700 1 -- 172.21.15.2:0/2087463414 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41c0022020 con 0x7f41c4107de0 2024-05-14T16:43:10.516 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.888+0000 7f41ba7fc700 1 -- 172.21.15.2:0/2087463414 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41c0018040 con 0x7f41c4107de0 2024-05-14T16:43:10.516 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.888+0000 7f41cb63f700 1 -- 172.21.15.2:0/2087463414 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41c419ce20 con 0x7f41c4107de0 2024-05-14T16:43:10.516 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.888+0000 7f41ba7fc700 1 -- 172.21.15.2:0/2087463414 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41c0017070 con 0x7f41c4107de0 2024-05-14T16:43:10.516 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.888+0000 7f41cb63f700 1 -- 172.21.15.2:0/2087463414 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41c419fb10 con 0x7f41c4107de0 2024-05-14T16:43:10.517 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.888+0000 7f41ba7fc700 1 -- 172.21.15.2:0/2087463414 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f41c0025020 con 0x7f41c4107de0 2024-05-14T16:43:10.517 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.888+0000 7f41ba7fc700 1 -- 172.21.15.2:0/2087463414 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f41c000f050 con 0x7f41c4107de0 2024-05-14T16:43:10.517 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.889+0000 7f41cb63f700 1 -- 172.21.15.2:0/2087463414 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41c4064ba0 con 0x7f41c4107de0 2024-05-14T16:43:10.517 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.891+0000 7f41ba7fc700 1 -- 172.21.15.2:0/2087463414 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f41c002c030 con 0x7f41c4107de0 2024-05-14T16:43:10.517 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.964+0000 7f41cb63f700 1 -- 172.21.15.2:0/2087463414 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f41c402e720 con 0x7f41c4107de0 2024-05-14T16:43:10.517 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.965+0000 7f41ba7fc700 1 -- 172.21.15.2:0/2087463414 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f41c001e480 con 0x7f41c4107de0 2024-05-14T16:43:10.517 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.966+0000 7f41cb63f700 1 -- 172.21.15.2:0/2087463414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41c4107de0 msgr2=0x7f41c419c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:10.517 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.966+0000 7f41cb63f700 1 --2- 172.21.15.2:0/2087463414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41c4107de0 0x7f41c419c650 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f41c0004000 tx=0x7f41c0004ad0 comp rx=0 tx=0).stop 2024-05-14T16:43:10.517 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.966+0000 7f41cb63f700 1 -- 172.21.15.2:0/2087463414 shutdown_connections 2024-05-14T16:43:10.517 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.966+0000 7f41cb63f700 1 --2- 172.21.15.2:0/2087463414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41c4107de0 0x7f41c419c650 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:10.517 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.966+0000 7f41cb63f700 1 -- 172.21.15.2:0/2087463414 >> 172.21.15.2:0/2087463414 conn(0x7f41c41033a0 msgr2=0x7f41c4104010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:43:10.517 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.966+0000 7f41cb63f700 1 -- 172.21.15.2:0/2087463414 shutdown_connections 2024-05-14T16:43:10.517 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:09.966+0000 7f41cb63f700 1 -- 172.21.15.2:0/2087463414 wait complete. 2024-05-14T16:43:10.518 INFO:teuthology.orchestra.run.smithi002.stdout:mon is available 2024-05-14T16:43:10.518 INFO:teuthology.orchestra.run.smithi002.stdout:Assimilating anything we can from ceph.conf... 2024-05-14T16:43:11.593 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-05-14T16:43:11.593 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [global] 2024-05-14T16:43:11.593 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout fsid = ed40717a-1210-11ef-bc98-c7b262605968 2024-05-14T16:43:11.593 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-14T16:43:11.593 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.2:3300,v1:172.21.15.2:6789] 2024-05-14T16:43:11.593 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-14T16:43:11.593 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-14T16:43:11.593 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-14T16:43:11.593 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-14T16:43:11.593 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-05-14T16:43:11.593 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-14T16:43:11.594 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-14T16:43:11.594 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-05-14T16:43:11.594 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [osd] 2024-05-14T16:43:11.594 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-14T16:43:11.594 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-14T16:43:11.594 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-14T16:43:11.594 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-14T16:43:11.594 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-14T16:43:11.594 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:10.941+0000 7f0e4f3c8700 1 Processor -- start 2024-05-14T16:43:11.594 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:10.941+0000 7f0e4f3c8700 1 -- start start 2024-05-14T16:43:11.594 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:10.942+0000 7f0e4f3c8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0e48108e60 0x7f0e48109280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:11.594 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:10.942+0000 7f0e4f3c8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0e481097c0 con 0x7f0e48108e60 2024-05-14T16:43:11.594 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:10.942+0000 7f0e4d164700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0e48108e60 0x7f0e48109280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:11.594 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:10.942+0000 7f0e4d164700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0e48108e60 0x7f0e48109280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40434/0 (socket says 172.21.15.2:40434) 2024-05-14T16:43:11.595 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:10.942+0000 7f0e4d164700 1 -- 172.21.15.2:0/1642755817 learned_addr learned my addr 172.21.15.2:0/1642755817 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:43:11.595 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:10.942+0000 7f0e4d164700 1 -- 172.21.15.2:0/1642755817 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e4810a050 con 0x7f0e48108e60 2024-05-14T16:43:11.595 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:10.942+0000 7f0e4d164700 1 --2- 172.21.15.2:0/1642755817 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0e48108e60 0x7f0e48109280 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f0e440098a0 tx=0x7f0e44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74114625e3e57849 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:11.595 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:10.942+0000 7f0e3ffff700 1 -- 172.21.15.2:0/1642755817 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e44004160 con 0x7f0e48108e60 2024-05-14T16:43:11.595 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:10.942+0000 7f0e3ffff700 1 -- 172.21.15.2:0/1642755817 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e44004300 con 0x7f0e48108e60 2024-05-14T16:43:11.595 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:10.942+0000 7f0e3ffff700 1 -- 172.21.15.2:0/1642755817 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e440036a0 con 0x7f0e48108e60 2024-05-14T16:43:11.595 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:10.943+0000 7f0e4f3c8700 1 -- 172.21.15.2:0/1642755817 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0e48108e60 msgr2=0x7f0e48109280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:11.595 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:10.943+0000 7f0e4f3c8700 1 --2- 172.21.15.2:0/1642755817 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0e48108e60 0x7f0e48109280 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f0e440098a0 tx=0x7f0e44009bb0 comp rx=0 tx=0).stop 2024-05-14T16:43:11.595 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:10.943+0000 7f0e4f3c8700 1 -- 172.21.15.2:0/1642755817 shutdown_connections 2024-05-14T16:43:11.595 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:10.943+0000 7f0e4f3c8700 1 --2- 172.21.15.2:0/1642755817 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0e48108e60 0x7f0e48109280 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:11.595 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:10.943+0000 7f0e4f3c8700 1 -- 172.21.15.2:0/1642755817 >> 172.21.15.2:0/1642755817 conn(0x7f0e48104450 msgr2=0x7f0e48106890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:43:11.595 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:10.943+0000 7f0e4f3c8700 1 -- 172.21.15.2:0/1642755817 shutdown_connections 2024-05-14T16:43:11.595 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:10.943+0000 7f0e4f3c8700 1 -- 172.21.15.2:0/1642755817 wait complete. 2024-05-14T16:43:11.596 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:10.944+0000 7f0e4f3c8700 1 Processor -- start 2024-05-14T16:43:11.596 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:10.944+0000 7f0e4f3c8700 1 -- start start 2024-05-14T16:43:11.596 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:10.944+0000 7f0e4f3c8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0e48108e60 0x7f0e4807f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:11.596 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:10.944+0000 7f0e4f3c8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4810a520 con 0x7f0e48108e60 2024-05-14T16:43:11.596 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:10.944+0000 7f0e4d164700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0e48108e60 0x7f0e4807f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:11.596 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:10.944+0000 7f0e4d164700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0e48108e60 0x7f0e4807f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40438/0 (socket says 172.21.15.2:40438) 2024-05-14T16:43:11.596 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:10.944+0000 7f0e4d164700 1 -- 172.21.15.2:0/1747140163 learned_addr learned my addr 172.21.15.2:0/1747140163 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:43:11.596 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:10.944+0000 7f0e4d164700 1 -- 172.21.15.2:0/1747140163 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e4807f8c0 con 0x7f0e48108e60 2024-05-14T16:43:11.596 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:10.944+0000 7f0e4d164700 1 --2- 172.21.15.2:0/1747140163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0e48108e60 0x7f0e4807f380 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0e44000c00 tx=0x7f0e44004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:11.596 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:10.944+0000 7f0e3e7fc700 1 -- 172.21.15.2:0/1747140163 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e44003db0 con 0x7f0e48108e60 2024-05-14T16:43:11.596 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:10.945+0000 7f0e3e7fc700 1 -- 172.21.15.2:0/1747140163 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e44011630 con 0x7f0e48108e60 2024-05-14T16:43:11.596 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:10.945+0000 7f0e4f3c8700 1 -- 172.21.15.2:0/1747140163 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e4807dab0 con 0x7f0e48108e60 2024-05-14T16:43:11.596 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:10.945+0000 7f0e3e7fc700 1 -- 172.21.15.2:0/1747140163 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e44011950 con 0x7f0e48108e60 2024-05-14T16:43:11.596 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:10.945+0000 7f0e4f3c8700 1 -- 172.21.15.2:0/1747140163 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e4807dff0 con 0x7f0e48108e60 2024-05-14T16:43:11.597 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:10.945+0000 7f0e3e7fc700 1 -- 172.21.15.2:0/1747140163 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0e4401a460 con 0x7f0e48108e60 2024-05-14T16:43:11.597 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:10.945+0000 7f0e3e7fc700 1 -- 172.21.15.2:0/1747140163 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0e4401b780 con 0x7f0e48108e60 2024-05-14T16:43:11.597 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:10.946+0000 7f0e4f3c8700 1 -- 172.21.15.2:0/1747140163 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e48064ba0 con 0x7f0e48108e60 2024-05-14T16:43:11.597 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:10.948+0000 7f0e3e7fc700 1 -- 172.21.15.2:0/1747140163 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f0e4401b980 con 0x7f0e48108e60 2024-05-14T16:43:11.597 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:11.020+0000 7f0e4f3c8700 1 -- 172.21.15.2:0/1747140163 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f0e48195c20 con 0x7f0e48108e60 2024-05-14T16:43:11.597 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:11.023+0000 7f0e3e7fc700 1 -- 172.21.15.2:0/1747140163 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+463 (secure 0 0 0) 0x7f0e4401bb60 con 0x7f0e48108e60 2024-05-14T16:43:11.597 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:11.024+0000 7f0e3e7fc700 1 -- 172.21.15.2:0/1747140163 <== mon.0 v2:172.21.15.2:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0e4401ad70 con 0x7f0e48108e60 2024-05-14T16:43:11.597 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:11.024+0000 7f0e4f3c8700 1 -- 172.21.15.2:0/1747140163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0e48108e60 msgr2=0x7f0e4807f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:11.597 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:11.024+0000 7f0e4f3c8700 1 --2- 172.21.15.2:0/1747140163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0e48108e60 0x7f0e4807f380 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0e44000c00 tx=0x7f0e44004750 comp rx=0 tx=0).stop 2024-05-14T16:43:11.597 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:11.024+0000 7f0e4f3c8700 1 -- 172.21.15.2:0/1747140163 shutdown_connections 2024-05-14T16:43:11.597 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:11.024+0000 7f0e4f3c8700 1 --2- 172.21.15.2:0/1747140163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0e48108e60 0x7f0e4807f380 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:11.597 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:11.025+0000 7f0e4f3c8700 1 -- 172.21.15.2:0/1747140163 >> 172.21.15.2:0/1747140163 conn(0x7f0e48104450 msgr2=0x7f0e481931f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:43:11.597 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:11.025+0000 7f0e4f3c8700 1 -- 172.21.15.2:0/1747140163 shutdown_connections 2024-05-14T16:43:11.597 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:11.025+0000 7f0e4f3c8700 1 -- 172.21.15.2:0/1747140163 wait complete. 2024-05-14T16:43:11.597 INFO:teuthology.orchestra.run.smithi002.stdout:Generating new minimal ceph.conf... 2024-05-14T16:43:11.693 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:11 smithi002 ceph-mon[31168]: from='client.? 172.21.15.2:0/1747140163' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-14T16:43:11.693 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:11 smithi002 ceph-mon[31168]: from='client.? 172.21.15.2:0/1747140163' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-14T16:43:12.630 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.018+0000 7f6441e09700 1 Processor -- start 2024-05-14T16:43:12.630 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.018+0000 7f6441e09700 1 -- start start 2024-05-14T16:43:12.630 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.018+0000 7f6441e09700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f643c108eb0 0x7f643c1092d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:12.630 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.018+0000 7f6441e09700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f643c109810 con 0x7f643c108eb0 2024-05-14T16:43:12.631 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.018+0000 7f643b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f643c108eb0 0x7f643c1092d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:12.631 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.018+0000 7f643b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f643c108eb0 0x7f643c1092d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40444/0 (socket says 172.21.15.2:40444) 2024-05-14T16:43:12.631 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.018+0000 7f643b7fe700 1 -- 172.21.15.2:0/987432229 learned_addr learned my addr 172.21.15.2:0/987432229 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:43:12.631 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.019+0000 7f643b7fe700 1 -- 172.21.15.2:0/987432229 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f643c10a050 con 0x7f643c108eb0 2024-05-14T16:43:12.631 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.019+0000 7f643b7fe700 1 --2- 172.21.15.2:0/987432229 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f643c108eb0 0x7f643c1092d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f642c0098a0 tx=0x7f642c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=696ea8f89a75b5b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:12.631 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.019+0000 7f643a7fc700 1 -- 172.21.15.2:0/987432229 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f642c004160 con 0x7f643c108eb0 2024-05-14T16:43:12.631 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.019+0000 7f643a7fc700 1 -- 172.21.15.2:0/987432229 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f642c004300 con 0x7f643c108eb0 2024-05-14T16:43:12.631 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.019+0000 7f643a7fc700 1 -- 172.21.15.2:0/987432229 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f642c011af0 con 0x7f643c108eb0 2024-05-14T16:43:12.631 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.020+0000 7f6441e09700 1 -- 172.21.15.2:0/987432229 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f643c108eb0 msgr2=0x7f643c1092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:12.631 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.020+0000 7f6441e09700 1 --2- 172.21.15.2:0/987432229 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f643c108eb0 0x7f643c1092d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f642c0098a0 tx=0x7f642c009bb0 comp rx=0 tx=0).stop 2024-05-14T16:43:12.631 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.020+0000 7f6441e09700 1 -- 172.21.15.2:0/987432229 shutdown_connections 2024-05-14T16:43:12.631 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.020+0000 7f6441e09700 1 --2- 172.21.15.2:0/987432229 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f643c108eb0 0x7f643c1092d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:12.631 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.020+0000 7f6441e09700 1 -- 172.21.15.2:0/987432229 >> 172.21.15.2:0/987432229 conn(0x7f643c104500 msgr2=0x7f643c106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:43:12.632 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.020+0000 7f6441e09700 1 -- 172.21.15.2:0/987432229 shutdown_connections 2024-05-14T16:43:12.632 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.020+0000 7f6441e09700 1 -- 172.21.15.2:0/987432229 wait complete. 2024-05-14T16:43:12.632 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.020+0000 7f6441e09700 1 Processor -- start 2024-05-14T16:43:12.632 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.021+0000 7f6441e09700 1 -- start start 2024-05-14T16:43:12.632 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.021+0000 7f6441e09700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f643c108eb0 0x7f643c19ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:12.632 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.021+0000 7f6441e09700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f643c10d310 con 0x7f643c108eb0 2024-05-14T16:43:12.632 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.021+0000 7f643b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f643c108eb0 0x7f643c19ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:12.632 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.021+0000 7f643b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f643c108eb0 0x7f643c19ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40448/0 (socket says 172.21.15.2:40448) 2024-05-14T16:43:12.632 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.021+0000 7f643b7fe700 1 -- 172.21.15.2:0/2159666115 learned_addr learned my addr 172.21.15.2:0/2159666115 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:43:12.632 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.021+0000 7f643b7fe700 1 -- 172.21.15.2:0/2159666115 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f643c19d3b0 con 0x7f643c108eb0 2024-05-14T16:43:12.632 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.021+0000 7f643b7fe700 1 --2- 172.21.15.2:0/2159666115 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f643c108eb0 0x7f643c19ce70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f642c0117b0 tx=0x7f642c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:12.632 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.021+0000 7f6438ff9700 1 -- 172.21.15.2:0/2159666115 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f642c003af0 con 0x7f643c108eb0 2024-05-14T16:43:12.633 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.021+0000 7f6438ff9700 1 -- 172.21.15.2:0/2159666115 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f642c020070 con 0x7f643c108eb0 2024-05-14T16:43:12.633 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.021+0000 7f6441e09700 1 -- 172.21.15.2:0/2159666115 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f643c19d640 con 0x7f643c108eb0 2024-05-14T16:43:12.633 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.022+0000 7f6438ff9700 1 -- 172.21.15.2:0/2159666115 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f642c019630 con 0x7f643c108eb0 2024-05-14T16:43:12.633 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.022+0000 7f6441e09700 1 -- 172.21.15.2:0/2159666115 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f643c1a0330 con 0x7f643c108eb0 2024-05-14T16:43:12.633 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.022+0000 7f6438ff9700 1 -- 172.21.15.2:0/2159666115 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f642c019c30 con 0x7f643c108eb0 2024-05-14T16:43:12.633 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.022+0000 7f6438ff9700 1 -- 172.21.15.2:0/2159666115 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f642c00b720 con 0x7f643c108eb0 2024-05-14T16:43:12.633 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.023+0000 7f6441e09700 1 -- 172.21.15.2:0/2159666115 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f643c064ba0 con 0x7f643c108eb0 2024-05-14T16:43:12.633 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.025+0000 7f6438ff9700 1 -- 172.21.15.2:0/2159666115 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f642c0042c0 con 0x7f643c108eb0 2024-05-14T16:43:12.633 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.096+0000 7f6441e09700 1 -- 172.21.15.2:0/2159666115 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f643c1a0880 con 0x7f643c108eb0 2024-05-14T16:43:12.633 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.096+0000 7f6438ff9700 1 -- 172.21.15.2:0/2159666115 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+173 (secure 0 0 0) 0x7f642c01a9c0 con 0x7f643c108eb0 2024-05-14T16:43:12.633 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.097+0000 7f6441e09700 1 -- 172.21.15.2:0/2159666115 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f643c108eb0 msgr2=0x7f643c19ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:12.633 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.097+0000 7f6441e09700 1 --2- 172.21.15.2:0/2159666115 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f643c108eb0 0x7f643c19ce70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f642c0117b0 tx=0x7f642c011b70 comp rx=0 tx=0).stop 2024-05-14T16:43:12.634 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.097+0000 7f6441e09700 1 -- 172.21.15.2:0/2159666115 shutdown_connections 2024-05-14T16:43:12.634 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.097+0000 7f6441e09700 1 --2- 172.21.15.2:0/2159666115 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f643c108eb0 0x7f643c19ce70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:12.634 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.097+0000 7f6441e09700 1 -- 172.21.15.2:0/2159666115 >> 172.21.15.2:0/2159666115 conn(0x7f643c104500 msgr2=0x7f643c105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:43:12.634 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.097+0000 7f6441e09700 1 -- 172.21.15.2:0/2159666115 shutdown_connections 2024-05-14T16:43:12.634 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:12.097+0000 7f6441e09700 1 -- 172.21.15.2:0/2159666115 wait complete. 2024-05-14T16:43:12.634 INFO:teuthology.orchestra.run.smithi002.stdout:Restarting the monitor... 2024-05-14T16:43:12.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:12 smithi002 ceph-mon[31168]: from='client.? 172.21.15.2:0/2159666115' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:43:12.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:12 smithi002 systemd[1]: Stopping Ceph mon.a for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T16:43:13.202 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:12 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mon-a[31164]: 2024-05-14T16:43:12.910+0000 7f198fc9e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T16:43:13.202 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:12 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mon-a[31164]: 2024-05-14T16:43:12.910+0000 7f198fc9e700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-14T16:43:13.202 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:12 smithi002 podman[31414]: 2024-05-14 16:43:12.941463165 +0000 UTC m=+0.144441044 container died b4afb4f3e1aa1b0c97ce1f3b5032728d4bdb9838858b01cc3c3db1f19bb9bcc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mon-a, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux ) 2024-05-14T16:43:13.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:13 smithi002 podman[31414]: 2024-05-14 16:43:13.293401559 +0000 UTC m=+0.496379432 container cleanup b4afb4f3e1aa1b0c97ce1f3b5032728d4bdb9838858b01cc3c3db1f19bb9bcc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T16:43:13.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:13 smithi002 bash[31414]: ceph-ed40717a-1210-11ef-bc98-c7b262605968-mon-a 2024-05-14T16:43:13.940 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:13 smithi002 podman[31428]: 2024-05-14 16:43:13.578137545 +0000 UTC m=+0.639609809 container remove b4afb4f3e1aa1b0c97ce1f3b5032728d4bdb9838858b01cc3c3db1f19bb9bcc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T16:43:13.940 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:13 smithi002 systemd[1]: ceph-ed40717a-1210-11ef-bc98-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-14T16:43:13.941 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:13 smithi002 systemd[1]: Stopped Ceph mon.a for ed40717a-1210-11ef-bc98-c7b262605968. 2024-05-14T16:43:13.941 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:13 smithi002 systemd[1]: Starting Ceph mon.a for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T16:43:14.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:13 smithi002 podman[31519]: 2024-05-14 16:43:13.843758903 +0000 UTC m=+0.018002612 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-14T16:43:15.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:14 smithi002 podman[31519]: 2024-05-14 16:43:14.938213645 +0000 UTC m=+1.112457346 container create c3b1d2255195fe582cbf3457a4fd31d39a25416c7ba12683e39026624e273831 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mon-a, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5) 2024-05-14T16:43:17.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 podman[31519]: 2024-05-14 16:43:16.825032492 +0000 UTC m=+2.999276199 container init c3b1d2255195fe582cbf3457a4fd31d39a25416c7ba12683e39026624e273831 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mon-a, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-14T16:43:17.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 podman[31519]: 2024-05-14 16:43:16.829429554 +0000 UTC m=+3.003673263 container start c3b1d2255195fe582cbf3457a4fd31d39a25416c7ba12683e39026624e273831 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux ) 2024-05-14T16:43:17.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: set uid:gid to 167:167 (ceph:ceph) 2024-05-14T16:43:17.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev), process ceph-mon, pid 2 2024-05-14T16:43:17.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: pidfile_write: ignore empty --pid-file 2024-05-14T16:43:17.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: load: jerasure load: lrc 2024-05-14T16:43:17.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: RocksDB version: 7.9.2 2024-05-14T16:43:17.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Git sha 0 2024-05-14T16:43:17.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Compile date 2024-05-14 14:11:45 2024-05-14T16:43:17.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: DB SUMMARY 2024-05-14T16:43:17.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: DB Session ID: EAIGXTIKRGMZ1GSYA0AT 2024-05-14T16:43:17.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: CURRENT file: CURRENT 2024-05-14T16:43:17.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: IDENTITY file: IDENTITY 2024-05-14T16:43:17.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-14T16:43:17.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-14T16:43:17.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104506 ; 2024-05-14T16:43:17.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.error_if_exists: 0 2024-05-14T16:43:17.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.create_if_missing: 0 2024-05-14T16:43:17.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.paranoid_checks: 1 2024-05-14T16:43:17.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-14T16:43:17.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-14T16:43:17.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-14T16:43:17.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.env: 0x55f5b7242cc0 2024-05-14T16:43:17.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.fs: PosixFileSystem 2024-05-14T16:43:17.247 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.info_log: 0x55f5b7c3bda0 2024-05-14T16:43:17.247 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.max_file_opening_threads: 16 2024-05-14T16:43:17.247 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.statistics: (nil) 2024-05-14T16:43:17.247 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.use_fsync: 0 2024-05-14T16:43:17.247 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.max_log_file_size: 0 2024-05-14T16:43:17.247 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-14T16:43:17.247 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-14T16:43:17.247 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.keep_log_file_num: 1000 2024-05-14T16:43:17.247 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.recycle_log_file_num: 0 2024-05-14T16:43:17.247 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.allow_fallocate: 1 2024-05-14T16:43:17.247 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.allow_mmap_reads: 0 2024-05-14T16:43:17.247 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.allow_mmap_writes: 0 2024-05-14T16:43:17.247 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.use_direct_reads: 0 2024-05-14T16:43:17.247 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-14T16:43:17.247 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.create_missing_column_families: 0 2024-05-14T16:43:17.248 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.db_log_dir: 2024-05-14T16:43:17.248 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.wal_dir: 2024-05-14T16:43:17.248 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-14T16:43:17.248 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-14T16:43:17.248 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-14T16:43:17.248 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-14T16:43:17.248 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-14T16:43:17.248 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-14T16:43:17.248 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.advise_random_on_open: 1 2024-05-14T16:43:17.248 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.db_write_buffer_size: 0 2024-05-14T16:43:17.248 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.write_buffer_manager: 0x55f5b7c0d0e0 2024-05-14T16:43:17.248 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-14T16:43:17.248 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-14T16:43:17.248 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-14T16:43:17.249 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.rate_limiter: (nil) 2024-05-14T16:43:17.249 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-14T16:43:17.249 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.wal_recovery_mode: 2 2024-05-14T16:43:17.249 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.enable_thread_tracking: 0 2024-05-14T16:43:17.249 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.enable_pipelined_write: 0 2024-05-14T16:43:17.249 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.unordered_write: 0 2024-05-14T16:43:17.249 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-14T16:43:17.249 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-14T16:43:17.249 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-14T16:43:17.249 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-14T16:43:17.249 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.row_cache: None 2024-05-14T16:43:17.249 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.wal_filter: None 2024-05-14T16:43:17.249 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-14T16:43:17.251 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.allow_ingest_behind: 0 2024-05-14T16:43:17.251 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.two_write_queues: 0 2024-05-14T16:43:17.251 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.manual_wal_flush: 0 2024-05-14T16:43:17.251 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.wal_compression: 0 2024-05-14T16:43:17.251 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.atomic_flush: 0 2024-05-14T16:43:17.251 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-14T16:43:17.251 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-14T16:43:17.251 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-14T16:43:17.251 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.log_readahead_size: 0 2024-05-14T16:43:17.251 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-14T16:43:17.251 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.best_efforts_recovery: 0 2024-05-14T16:43:17.251 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-14T16:43:17.252 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-14T16:43:17.252 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.allow_data_in_errors: 0 2024-05-14T16:43:17.252 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.db_host_id: __hostname__ 2024-05-14T16:43:17.252 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-14T16:43:17.252 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.max_background_jobs: 2 2024-05-14T16:43:17.252 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.max_background_compactions: -1 2024-05-14T16:43:17.252 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.max_subcompactions: 1 2024-05-14T16:43:17.252 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-14T16:43:17.252 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-14T16:43:17.252 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-14T16:43:17.252 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.max_total_wal_size: 0 2024-05-14T16:43:17.252 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-14T16:43:17.252 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-14T16:43:17.252 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-14T16:43:17.253 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-14T16:43:17.253 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.max_open_files: -1 2024-05-14T16:43:17.253 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.bytes_per_sync: 0 2024-05-14T16:43:17.253 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-14T16:43:17.253 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-14T16:43:17.253 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.compaction_readahead_size: 0 2024-05-14T16:43:17.253 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.max_background_flushes: -1 2024-05-14T16:43:17.253 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Compression algorithms supported: 2024-05-14T16:43:17.253 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-14T16:43:17.253 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: kZSTD supported: 0 2024-05-14T16:43:17.253 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: kXpressCompression supported: 0 2024-05-14T16:43:17.253 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: kLZ4HCCompression supported: 1 2024-05-14T16:43:17.253 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: kZlibCompression supported: 1 2024-05-14T16:43:17.253 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: kSnappyCompression supported: 1 2024-05-14T16:43:17.253 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: kLZ4Compression supported: 1 2024-05-14T16:43:17.254 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: kBZip2Compression supported: 0 2024-05-14T16:43:17.254 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-14T16:43:17.254 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-14T16:43:17.254 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-14T16:43:17.254 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-14T16:43:17.254 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-14T16:43:17.254 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.merge_operator: 2024-05-14T16:43:17.254 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.compaction_filter: None 2024-05-14T16:43:17.254 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.compaction_filter_factory: None 2024-05-14T16:43:17.254 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.sst_partitioner_factory: None 2024-05-14T16:43:17.254 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-14T16:43:17.254 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-14T16:43:17.254 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f5b7c3bf60) 2024-05-14T16:43:17.255 INFO:journalctl@ceph.mon.a.smithi002.stdout: cache_index_and_filter_blocks: 1 2024-05-14T16:43:17.255 INFO:journalctl@ceph.mon.a.smithi002.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-14T16:43:17.255 INFO:journalctl@ceph.mon.a.smithi002.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-14T16:43:17.255 INFO:journalctl@ceph.mon.a.smithi002.stdout: pin_top_level_index_and_filter: 1 2024-05-14T16:43:17.255 INFO:journalctl@ceph.mon.a.smithi002.stdout: index_type: 0 2024-05-14T16:43:17.255 INFO:journalctl@ceph.mon.a.smithi002.stdout: data_block_index_type: 0 2024-05-14T16:43:17.255 INFO:journalctl@ceph.mon.a.smithi002.stdout: index_shortening: 1 2024-05-14T16:43:17.255 INFO:journalctl@ceph.mon.a.smithi002.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-14T16:43:17.255 INFO:journalctl@ceph.mon.a.smithi002.stdout: checksum: 4 2024-05-14T16:43:17.255 INFO:journalctl@ceph.mon.a.smithi002.stdout: no_block_cache: 0 2024-05-14T16:43:17.255 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_cache: 0x55f5b7c3f1f0 2024-05-14T16:43:17.255 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_cache_name: BinnedLRUCache 2024-05-14T16:43:17.255 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_cache_options: 2024-05-14T16:43:17.255 INFO:journalctl@ceph.mon.a.smithi002.stdout: capacity : 536870912 2024-05-14T16:43:17.255 INFO:journalctl@ceph.mon.a.smithi002.stdout: num_shard_bits : 4 2024-05-14T16:43:17.256 INFO:journalctl@ceph.mon.a.smithi002.stdout: strict_capacity_limit : 0 2024-05-14T16:43:17.256 INFO:journalctl@ceph.mon.a.smithi002.stdout: high_pri_pool_ratio: 0.000 2024-05-14T16:43:17.256 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_cache_compressed: (nil) 2024-05-14T16:43:17.256 INFO:journalctl@ceph.mon.a.smithi002.stdout: persistent_cache: (nil) 2024-05-14T16:43:17.256 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_size: 4096 2024-05-14T16:43:17.256 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_size_deviation: 10 2024-05-14T16:43:17.256 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_restart_interval: 16 2024-05-14T16:43:17.256 INFO:journalctl@ceph.mon.a.smithi002.stdout: index_block_restart_interval: 1 2024-05-14T16:43:17.256 INFO:journalctl@ceph.mon.a.smithi002.stdout: metadata_block_size: 4096 2024-05-14T16:43:17.256 INFO:journalctl@ceph.mon.a.smithi002.stdout: partition_filters: 0 2024-05-14T16:43:17.256 INFO:journalctl@ceph.mon.a.smithi002.stdout: use_delta_encoding: 1 2024-05-14T16:43:17.256 INFO:journalctl@ceph.mon.a.smithi002.stdout: filter_policy: bloomfilter 2024-05-14T16:43:17.256 INFO:journalctl@ceph.mon.a.smithi002.stdout: whole_key_filtering: 1 2024-05-14T16:43:17.256 INFO:journalctl@ceph.mon.a.smithi002.stdout: verify_compression: 0 2024-05-14T16:43:17.256 INFO:journalctl@ceph.mon.a.smithi002.stdout: read_amp_bytes_per_bit: 0 2024-05-14T16:43:17.257 INFO:journalctl@ceph.mon.a.smithi002.stdout: format_version: 5 2024-05-14T16:43:17.257 INFO:journalctl@ceph.mon.a.smithi002.stdout: enable_index_compression: 1 2024-05-14T16:43:17.257 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_align: 0 2024-05-14T16:43:17.257 INFO:journalctl@ceph.mon.a.smithi002.stdout: max_auto_readahead_size: 262144 2024-05-14T16:43:17.257 INFO:journalctl@ceph.mon.a.smithi002.stdout: prepopulate_block_cache: 0 2024-05-14T16:43:17.257 INFO:journalctl@ceph.mon.a.smithi002.stdout: initial_auto_readahead_size: 8192 2024-05-14T16:43:17.257 INFO:journalctl@ceph.mon.a.smithi002.stdout: num_file_reads_for_auto_readahead: 2 2024-05-14T16:43:17.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.write_buffer_size: 33554432 2024-05-14T16:43:17.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.max_write_buffer_number: 2 2024-05-14T16:43:17.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.compression: NoCompression 2024-05-14T16:43:17.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.bottommost_compression: Disabled 2024-05-14T16:43:17.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.prefix_extractor: nullptr 2024-05-14T16:43:17.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-14T16:43:17.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.num_levels: 7 2024-05-14T16:43:17.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-14T16:43:17.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-14T16:43:17.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-14T16:43:17.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-14T16:43:17.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-14T16:43:17.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-14T16:43:17.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-14T16:43:17.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-14T16:43:17.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-14T16:43:17.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-14T16:43:17.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-14T16:43:17.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-14T16:43:17.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-14T16:43:17.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.compression_opts.level: 32767 2024-05-14T16:43:17.259 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.compression_opts.strategy: 0 2024-05-14T16:43:17.259 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-14T16:43:17.259 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-14T16:43:17.259 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-14T16:43:17.259 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-14T16:43:17.259 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.compression_opts.enabled: false 2024-05-14T16:43:17.259 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-14T16:43:17.259 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-14T16:43:17.259 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-14T16:43:17.259 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-14T16:43:17.259 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.target_file_size_base: 67108864 2024-05-14T16:43:17.259 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-14T16:43:17.259 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-14T16:43:17.259 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-14T16:43:17.259 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-14T16:43:17.259 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-14T16:43:17.260 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-14T16:43:17.260 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-14T16:43:17.260 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-14T16:43:17.260 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-14T16:43:17.260 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-14T16:43:17.260 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-14T16:43:17.260 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-14T16:43:17.260 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-14T16:43:17.260 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-14T16:43:17.260 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.arena_block_size: 1048576 2024-05-14T16:43:17.260 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-14T16:43:17.260 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-14T16:43:17.260 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.disable_auto_compactions: 0 2024-05-14T16:43:17.260 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-14T16:43:17.260 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-14T16:43:17.261 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-14T16:43:17.261 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-14T16:43:17.261 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-14T16:43:17.261 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-14T16:43:17.261 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-14T16:43:17.261 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-14T16:43:17.261 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-14T16:43:17.261 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-14T16:43:17.261 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-14T16:43:17.261 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.inplace_update_support: 0 2024-05-14T16:43:17.261 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-14T16:43:17.261 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-14T16:43:17.261 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-14T16:43:17.261 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-14T16:43:17.262 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.bloom_locality: 0 2024-05-14T16:43:17.262 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.max_successive_merges: 0 2024-05-14T16:43:17.262 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-14T16:43:17.262 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.paranoid_file_checks: 0 2024-05-14T16:43:17.262 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.force_consistency_checks: 1 2024-05-14T16:43:17.262 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.report_bg_io_stats: 0 2024-05-14T16:43:17.262 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.ttl: 2592000 2024-05-14T16:43:17.262 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-14T16:43:17.262 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-14T16:43:17.262 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-14T16:43:17.262 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.enable_blob_files: false 2024-05-14T16:43:17.262 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.min_blob_size: 0 2024-05-14T16:43:17.262 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.blob_file_size: 268435456 2024-05-14T16:43:17.263 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-14T16:43:17.263 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-14T16:43:17.263 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-14T16:43:17.263 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-14T16:43:17.263 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-14T16:43:17.263 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.blob_file_starting_level: 0 2024-05-14T16:43:17.263 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-14T16:43:17.263 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-14T16:43:17.263 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-14T16:43:17.263 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 489549bf-5f45-49ae-8862-ddc8f99b2efd 2024-05-14T16:43:17.263 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715704996864990, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-14T16:43:17.263 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-14T16:43:17.263 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715704996866256, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99733, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 97615, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14028, "raw_average_key_size": 49, "raw_value_size": 89973, "raw_average_value_size": 317, "num_data_blocks": 12, "num_entries": 283, "num_filter_entries": 283, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715704996, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "489549bf-5f45-49ae-8862-ddc8f99b2efd", "db_session_id": "EAIGXTIKRGMZ1GSYA0AT", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-14T16:43:17.263 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715704996866361, "job": 1, "event": "recovery_finished"} 2024-05-14T16:43:17.264 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-14T16:43:17.264 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-14T16:43:17.264 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f5b7d5c000 2024-05-14T16:43:17.264 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: DB pointer 0x55f5b7d48000 2024-05-14T16:43:17.264 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-14T16:43:17.264 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-14T16:43:17.264 INFO:journalctl@ceph.mon.a.smithi002.stdout: ** DB Stats ** 2024-05-14T16:43:17.264 INFO:journalctl@ceph.mon.a.smithi002.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-14T16:43:17.264 INFO:journalctl@ceph.mon.a.smithi002.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-14T16:43:17.264 INFO:journalctl@ceph.mon.a.smithi002.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-14T16:43:17.264 INFO:journalctl@ceph.mon.a.smithi002.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-14T16:43:17.264 INFO:journalctl@ceph.mon.a.smithi002.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-14T16:43:17.264 INFO:journalctl@ceph.mon.a.smithi002.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-14T16:43:17.264 INFO:journalctl@ceph.mon.a.smithi002.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-14T16:43:17.264 INFO:journalctl@ceph.mon.a.smithi002.stdout: 2024-05-14T16:43:17.265 INFO:journalctl@ceph.mon.a.smithi002.stdout: ** Compaction Stats [default] ** 2024-05-14T16:43:17.265 INFO:journalctl@ceph.mon.a.smithi002.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-14T16:43:17.265 INFO:journalctl@ceph.mon.a.smithi002.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-14T16:43:17.265 INFO:journalctl@ceph.mon.a.smithi002.stdout: L0 2/0 99.26 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 117.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-14T16:43:17.265 INFO:journalctl@ceph.mon.a.smithi002.stdout: Sum 2/0 99.26 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 117.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-14T16:43:17.265 INFO:journalctl@ceph.mon.a.smithi002.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 117.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-14T16:43:17.265 INFO:journalctl@ceph.mon.a.smithi002.stdout: 2024-05-14T16:43:17.265 INFO:journalctl@ceph.mon.a.smithi002.stdout: ** Compaction Stats [default] ** 2024-05-14T16:43:17.265 INFO:journalctl@ceph.mon.a.smithi002.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-14T16:43:17.265 INFO:journalctl@ceph.mon.a.smithi002.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-14T16:43:17.265 INFO:journalctl@ceph.mon.a.smithi002.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 117.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-14T16:43:17.265 INFO:journalctl@ceph.mon.a.smithi002.stdout: 2024-05-14T16:43:17.265 INFO:journalctl@ceph.mon.a.smithi002.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-14T16:43:17.265 INFO:journalctl@ceph.mon.a.smithi002.stdout: 2024-05-14T16:43:17.266 INFO:journalctl@ceph.mon.a.smithi002.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-14T16:43:17.266 INFO:journalctl@ceph.mon.a.smithi002.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-14T16:43:17.266 INFO:journalctl@ceph.mon.a.smithi002.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-14T16:43:17.266 INFO:journalctl@ceph.mon.a.smithi002.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-14T16:43:17.266 INFO:journalctl@ceph.mon.a.smithi002.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-14T16:43:17.266 INFO:journalctl@ceph.mon.a.smithi002.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-14T16:43:17.266 INFO:journalctl@ceph.mon.a.smithi002.stdout: Cumulative compaction: 0.00 GB write, 24.43 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-14T16:43:17.266 INFO:journalctl@ceph.mon.a.smithi002.stdout: Interval compaction: 0.00 GB write, 24.43 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-14T16:43:17.266 INFO:journalctl@ceph.mon.a.smithi002.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-14T16:43:17.266 INFO:journalctl@ceph.mon.a.smithi002.stdout: Block cache BinnedLRUCache@0x55f5b7c3f1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.6e-05 secs_since: 0 2024-05-14T16:43:17.266 INFO:journalctl@ceph.mon.a.smithi002.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-14T16:43:17.266 INFO:journalctl@ceph.mon.a.smithi002.stdout: 2024-05-14T16:43:17.266 INFO:journalctl@ceph.mon.a.smithi002.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-14T16:43:17.267 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: starting mon.a rank 0 at public addrs [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] at bind addrs [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid ed40717a-1210-11ef-bc98-c7b262605968 2024-05-14T16:43:17.267 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: mon.a@-1(???) e1 preinit fsid ed40717a-1210-11ef-bc98-c7b262605968 2024-05-14T16:43:17.267 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: mon.a@-1(???).mds e1 new map 2024-05-14T16:43:17.267 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: mon.a@-1(???).mds e1 print_map 2024-05-14T16:43:17.267 INFO:journalctl@ceph.mon.a.smithi002.stdout: e1 2024-05-14T16:43:17.267 INFO:journalctl@ceph.mon.a.smithi002.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-14T16:43:17.267 INFO:journalctl@ceph.mon.a.smithi002.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-14T16:43:17.267 INFO:journalctl@ceph.mon.a.smithi002.stdout: legacy client fscid: -1 2024-05-14T16:43:17.267 INFO:journalctl@ceph.mon.a.smithi002.stdout: 2024-05-14T16:43:17.267 INFO:journalctl@ceph.mon.a.smithi002.stdout: No filesystems configured 2024-05-14T16:43:17.267 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-14T16:43:17.267 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-14T16:43:17.267 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-14T16:43:17.267 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-14T16:43:17.268 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-14T16:43:17.268 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-14T16:43:17.268 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: monmap epoch 1 2024-05-14T16:43:17.268 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: fsid ed40717a-1210-11ef-bc98-c7b262605968 2024-05-14T16:43:17.268 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: last_changed 2024-05-14T16:43:06.575008+0000 2024-05-14T16:43:17.268 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: created 2024-05-14T16:43:06.575008+0000 2024-05-14T16:43:17.268 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: min_mon_release 19 (squid) 2024-05-14T16:43:17.268 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: election_strategy: 1 2024-05-14T16:43:17.268 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: 0: [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon.a 2024-05-14T16:43:17.268 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: fsmap 2024-05-14T16:43:17.268 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: osdmap e1: 0 total, 0 up, 0 in 2024-05-14T16:43:17.268 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:16 smithi002 ceph-mon[31538]: mgrmap e1: no daemons active 2024-05-14T16:43:17.538 INFO:teuthology.orchestra.run.smithi002.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-14T16:43:17.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:17 smithi002 bash[31519]: c3b1d2255195fe582cbf3457a4fd31d39a25416c7ba12683e39026624e273831 2024-05-14T16:43:17.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:17 smithi002 systemd[1]: Started Ceph mon.a for ed40717a-1210-11ef-bc98-c7b262605968. 2024-05-14T16:43:21.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:21 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/251805592' entity='client.admin' 2024-05-14T16:43:23.692 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.447+0000 7f4778f0e700 1 Processor -- start 2024-05-14T16:43:23.692 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.447+0000 7f4778f0e700 1 -- start start 2024-05-14T16:43:23.692 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.447+0000 7f4778f0e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47741099b0 0x7f477410bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:23.692 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.447+0000 7f4778f0e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f477407cdd0 con 0x7f47741099b0 2024-05-14T16:43:23.692 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.448+0000 7f477259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47741099b0 0x7f477410bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:23.693 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.448+0000 7f477259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47741099b0 0x7f477410bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55634/0 (socket says 172.21.15.2:55634) 2024-05-14T16:43:23.693 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.448+0000 7f477259c700 1 -- 172.21.15.2:0/606125039 learned_addr learned my addr 172.21.15.2:0/606125039 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:43:23.693 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.448+0000 7f477259c700 1 -- 172.21.15.2:0/606125039 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f477410c2e0 con 0x7f47741099b0 2024-05-14T16:43:23.693 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.448+0000 7f477259c700 1 --2- 172.21.15.2:0/606125039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47741099b0 0x7f477410bda0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f47640098a0 tx=0x7f4764009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d859cb4b7cffd4bc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:23.693 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.449+0000 7f4771d9b700 1 -- 172.21.15.2:0/606125039 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4764004160 con 0x7f47741099b0 2024-05-14T16:43:23.693 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.449+0000 7f4771d9b700 1 -- 172.21.15.2:0/606125039 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4764004300 con 0x7f47741099b0 2024-05-14T16:43:23.693 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.449+0000 7f4771d9b700 1 -- 172.21.15.2:0/606125039 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4764011af0 con 0x7f47741099b0 2024-05-14T16:43:23.693 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.449+0000 7f4778f0e700 1 -- 172.21.15.2:0/606125039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47741099b0 msgr2=0x7f477410bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:23.693 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.449+0000 7f4778f0e700 1 --2- 172.21.15.2:0/606125039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47741099b0 0x7f477410bda0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f47640098a0 tx=0x7f4764009bb0 comp rx=0 tx=0).stop 2024-05-14T16:43:23.693 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.449+0000 7f4778f0e700 1 -- 172.21.15.2:0/606125039 shutdown_connections 2024-05-14T16:43:23.693 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.449+0000 7f4778f0e700 1 --2- 172.21.15.2:0/606125039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47741099b0 0x7f477410bda0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:23.693 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.449+0000 7f4778f0e700 1 -- 172.21.15.2:0/606125039 >> 172.21.15.2:0/606125039 conn(0x7f47741033a0 msgr2=0x7f47741057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:43:23.693 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.449+0000 7f4778f0e700 1 -- 172.21.15.2:0/606125039 shutdown_connections 2024-05-14T16:43:23.694 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.449+0000 7f4778f0e700 1 -- 172.21.15.2:0/606125039 wait complete. 2024-05-14T16:43:23.694 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.450+0000 7f4778f0e700 1 Processor -- start 2024-05-14T16:43:23.694 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.450+0000 7f4778f0e700 1 -- start start 2024-05-14T16:43:23.694 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.451+0000 7f4778f0e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47741099b0 0x7f477419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:23.694 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.451+0000 7f4778f0e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f477410d250 con 0x7f47741099b0 2024-05-14T16:43:23.694 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.451+0000 7f477259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47741099b0 0x7f477419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:23.694 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.451+0000 7f477259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47741099b0 0x7f477419ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55636/0 (socket says 172.21.15.2:55636) 2024-05-14T16:43:23.694 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.451+0000 7f477259c700 1 -- 172.21.15.2:0/251805592 learned_addr learned my addr 172.21.15.2:0/251805592 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:43:23.694 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.451+0000 7f477259c700 1 -- 172.21.15.2:0/251805592 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f477419d3c0 con 0x7f47741099b0 2024-05-14T16:43:23.694 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.451+0000 7f477259c700 1 --2- 172.21.15.2:0/251805592 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47741099b0 0x7f477419ce80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4764011ac0 tx=0x7f4764011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:23.694 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.451+0000 7f476bfff700 1 -- 172.21.15.2:0/251805592 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4764003970 con 0x7f47741099b0 2024-05-14T16:43:23.695 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.451+0000 7f4778f0e700 1 -- 172.21.15.2:0/251805592 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f477419d650 con 0x7f47741099b0 2024-05-14T16:43:23.695 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.451+0000 7f476bfff700 1 -- 172.21.15.2:0/251805592 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4764003b10 con 0x7f47741099b0 2024-05-14T16:43:23.695 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.452+0000 7f476bfff700 1 -- 172.21.15.2:0/251805592 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4764024610 con 0x7f47741099b0 2024-05-14T16:43:23.695 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.452+0000 7f4778f0e700 1 -- 172.21.15.2:0/251805592 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47741a0340 con 0x7f47741099b0 2024-05-14T16:43:23.695 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.452+0000 7f476bfff700 1 -- 172.21.15.2:0/251805592 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4764024c20 con 0x7f47741099b0 2024-05-14T16:43:23.695 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.452+0000 7f476bfff700 1 -- 172.21.15.2:0/251805592 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4764019700 con 0x7f47741099b0 2024-05-14T16:43:23.695 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.452+0000 7f4778f0e700 1 -- 172.21.15.2:0/251805592 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4774064ba0 con 0x7f47741099b0 2024-05-14T16:43:23.695 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.455+0000 7f476bfff700 1 -- 172.21.15.2:0/251805592 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f4764043ec0 con 0x7f47741099b0 2024-05-14T16:43:23.695 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.527+0000 7f4778f0e700 1 -- 172.21.15.2:0/251805592 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f47741a0510 con 0x7f47741099b0 2024-05-14T16:43:23.695 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.529+0000 7f476bfff700 1 -- 172.21.15.2:0/251805592 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f4764032030 con 0x7f47741099b0 2024-05-14T16:43:23.695 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.530+0000 7f4778f0e700 1 -- 172.21.15.2:0/251805592 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47741099b0 msgr2=0x7f477419ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:23.695 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.530+0000 7f4778f0e700 1 --2- 172.21.15.2:0/251805592 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47741099b0 0x7f477419ce80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4764011ac0 tx=0x7f4764011ba0 comp rx=0 tx=0).stop 2024-05-14T16:43:23.695 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.530+0000 7f4778f0e700 1 -- 172.21.15.2:0/251805592 shutdown_connections 2024-05-14T16:43:23.696 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.530+0000 7f4778f0e700 1 --2- 172.21.15.2:0/251805592 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47741099b0 0x7f477419ce80 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:23.696 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.530+0000 7f4778f0e700 1 -- 172.21.15.2:0/251805592 >> 172.21.15.2:0/251805592 conn(0x7f47741033a0 msgr2=0x7f4774103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:43:23.696 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.530+0000 7f4778f0e700 1 -- 172.21.15.2:0/251805592 shutdown_connections 2024-05-14T16:43:23.696 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:20.530+0000 7f4778f0e700 1 -- 172.21.15.2:0/251805592 wait complete. 2024-05-14T16:43:23.696 INFO:teuthology.orchestra.run.smithi002.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-14T16:43:23.716 INFO:teuthology.orchestra.run.smithi002.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-14T16:43:23.716 INFO:teuthology.orchestra.run.smithi002.stdout:Creating mgr... 2024-05-14T16:43:23.716 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-14T16:43:23.717 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-14T16:43:23.985 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-ed40717a-1210-11ef-bc98-c7b262605968@mgr.x 2024-05-14T16:43:23.985 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Failed to reset failed state of unit ceph-ed40717a-1210-11ef-bc98-c7b262605968@mgr.x.service: Unit ceph-ed40717a-1210-11ef-bc98-c7b262605968@mgr.x.service not loaded. 2024-05-14T16:43:24.141 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-ed40717a-1210-11ef-bc98-c7b262605968.target.wants/ceph-ed40717a-1210-11ef-bc98-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-ed40717a-1210-11ef-bc98-c7b262605968@.service. 2024-05-14T16:43:24.643 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-14T16:43:24.643 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout disabled 2024-05-14T16:43:24.654 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-14T16:43:24.654 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2024-05-14T16:43:24.654 INFO:teuthology.orchestra.run.smithi002.stdout:firewalld.service is not enabled 2024-05-14T16:43:24.654 INFO:teuthology.orchestra.run.smithi002.stdout:Not possible to enable service . firewalld.service is not available 2024-05-14T16:43:24.660 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-14T16:43:24.661 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout disabled 2024-05-14T16:43:24.668 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-14T16:43:24.668 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2024-05-14T16:43:24.669 INFO:teuthology.orchestra.run.smithi002.stdout:firewalld.service is not enabled 2024-05-14T16:43:24.669 INFO:teuthology.orchestra.run.smithi002.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-14T16:43:24.669 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mgr to start... 2024-05-14T16:43:24.669 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mgr... 2024-05-14T16:43:25.440 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:25 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/1769261714' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-14T16:43:25.440 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:25 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:25.349+0000 7f960d480200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-14T16:43:25.440 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:25 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:25.440+0000 7f960d480200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-14T16:43:25.841 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-05-14T16:43:25.841 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2024-05-14T16:43:25.841 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsid": "ed40717a-1210-11ef-bc98-c7b262605968", 2024-05-14T16:43:25.841 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "health": { 2024-05-14T16:43:25.842 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-14T16:43:25.842 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-14T16:43:25.842 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-14T16:43:25.842 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:43:25.842 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-14T16:43:25.842 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-14T16:43:25.842 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 0 2024-05-14T16:43:25.842 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-05-14T16:43:25.842 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-14T16:43:25.842 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "a" 2024-05-14T16:43:25.842 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-05-14T16:43:25.842 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-05-14T16:43:25.842 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-14T16:43:25.843 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T16:43:25.843 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-14T16:43:25.843 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-14T16:43:25.843 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:43:25.843 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-14T16:43:25.843 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T16:43:25.843 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-14T16:43:25.843 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-14T16:43:25.843 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-14T16:43:25.843 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-14T16:43:25.843 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-14T16:43:25.843 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-14T16:43:25.843 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:43:25.849 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-14T16:43:25.849 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-14T16:43:25.849 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-14T16:43:25.849 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-14T16:43:25.849 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-14T16:43:25.849 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-14T16:43:25.849 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-14T16:43:25.849 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-14T16:43:25.849 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-14T16:43:25.849 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:43:25.849 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-14T16:43:25.849 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T16:43:25.850 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-14T16:43:25.850 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-14T16:43:25.850 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:43:25.850 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-14T16:43:25.850 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-14T16:43:25.850 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-14T16:43:25.850 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-14T16:43:25.850 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-14T16:43:25.850 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-14T16:43:25.850 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "restful" 2024-05-14T16:43:25.850 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-05-14T16:43:25.850 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-14T16:43:25.850 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:43:25.851 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-14T16:43:25.851 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T16:43:25.851 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modified": "2024-05-14T16:43:09.415788+0000", 2024-05-14T16:43:25.851 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-14T16:43:25.851 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:43:25.851 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-14T16:43:25.851 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2024-05-14T16:43:25.851 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.127+0000 7efdc4909700 1 Processor -- start 2024-05-14T16:43:25.851 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.127+0000 7efdc4909700 1 -- start start 2024-05-14T16:43:25.851 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.128+0000 7efdc4909700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdc01099a0 0x7efdc010bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:25.851 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.128+0000 7efdc4909700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7efdc007d630 con 0x7efdc01099a0 2024-05-14T16:43:25.851 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.128+0000 7efdbe59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdc01099a0 0x7efdc010bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:25.851 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.128+0000 7efdbe59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdc01099a0 0x7efdc010bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38770/0 (socket says 172.21.15.2:38770) 2024-05-14T16:43:25.851 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.128+0000 7efdbe59c700 1 -- 172.21.15.2:0/3616687433 learned_addr learned my addr 172.21.15.2:0/3616687433 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:43:25.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.128+0000 7efdbe59c700 1 -- 172.21.15.2:0/3616687433 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc010c2d0 con 0x7efdc01099a0 2024-05-14T16:43:25.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.128+0000 7efdbe59c700 1 --2- 172.21.15.2:0/3616687433 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdc01099a0 0x7efdc010bd90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7efdb00098a0 tx=0x7efdb0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62e66a877f1fcf5c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:25.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.129+0000 7efdbd59a700 1 -- 172.21.15.2:0/3616687433 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efdb0004160 con 0x7efdc01099a0 2024-05-14T16:43:25.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.129+0000 7efdbd59a700 1 -- 172.21.15.2:0/3616687433 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7efdb0004300 con 0x7efdc01099a0 2024-05-14T16:43:25.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.129+0000 7efdbd59a700 1 -- 172.21.15.2:0/3616687433 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efdb0011af0 con 0x7efdc01099a0 2024-05-14T16:43:25.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.130+0000 7efdc4909700 1 -- 172.21.15.2:0/3616687433 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdc01099a0 msgr2=0x7efdc010bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:25.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.130+0000 7efdc4909700 1 --2- 172.21.15.2:0/3616687433 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdc01099a0 0x7efdc010bd90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7efdb00098a0 tx=0x7efdb0009bb0 comp rx=0 tx=0).stop 2024-05-14T16:43:25.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.130+0000 7efdc4909700 1 -- 172.21.15.2:0/3616687433 shutdown_connections 2024-05-14T16:43:25.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.130+0000 7efdc4909700 1 --2- 172.21.15.2:0/3616687433 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdc01099a0 0x7efdc010bd90 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:25.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.130+0000 7efdc4909700 1 -- 172.21.15.2:0/3616687433 >> 172.21.15.2:0/3616687433 conn(0x7efdc01033b0 msgr2=0x7efdc01057d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:43:25.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.130+0000 7efdc4909700 1 -- 172.21.15.2:0/3616687433 shutdown_connections 2024-05-14T16:43:25.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.130+0000 7efdc4909700 1 -- 172.21.15.2:0/3616687433 wait complete. 2024-05-14T16:43:25.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.131+0000 7efdc4909700 1 Processor -- start 2024-05-14T16:43:25.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.131+0000 7efdc4909700 1 -- start start 2024-05-14T16:43:25.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.131+0000 7efdc4909700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdc01099a0 0x7efdc019cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:25.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.131+0000 7efdc4909700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7efdc010d230 con 0x7efdc01099a0 2024-05-14T16:43:25.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.131+0000 7efdbe59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdc01099a0 0x7efdc019cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:25.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.131+0000 7efdbe59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdc01099a0 0x7efdc019cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38774/0 (socket says 172.21.15.2:38774) 2024-05-14T16:43:25.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.131+0000 7efdbe59c700 1 -- 172.21.15.2:0/1769261714 learned_addr learned my addr 172.21.15.2:0/1769261714 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:43:25.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.132+0000 7efdbe59c700 1 -- 172.21.15.2:0/1769261714 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc019d330 con 0x7efdc01099a0 2024-05-14T16:43:25.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.132+0000 7efdbe59c700 1 --2- 172.21.15.2:0/1769261714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdc01099a0 0x7efdc019cdf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efdb0009fb0 tx=0x7efdb0011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:25.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.132+0000 7efdaf7fe700 1 -- 172.21.15.2:0/1769261714 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efdb00036a0 con 0x7efdc01099a0 2024-05-14T16:43:25.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.132+0000 7efdc4909700 1 -- 172.21.15.2:0/1769261714 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdc019d5c0 con 0x7efdc01099a0 2024-05-14T16:43:25.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.132+0000 7efdaf7fe700 1 -- 172.21.15.2:0/1769261714 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7efdb001a430 con 0x7efdc01099a0 2024-05-14T16:43:25.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.132+0000 7efdaf7fe700 1 -- 172.21.15.2:0/1769261714 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efdb0019550 con 0x7efdc01099a0 2024-05-14T16:43:25.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.132+0000 7efdc4909700 1 -- 172.21.15.2:0/1769261714 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdc01a02b0 con 0x7efdc01099a0 2024-05-14T16:43:25.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.132+0000 7efdaf7fe700 1 -- 172.21.15.2:0/1769261714 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efdb0019be0 con 0x7efdc01099a0 2024-05-14T16:43:25.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.132+0000 7efdaf7fe700 1 -- 172.21.15.2:0/1769261714 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7efdb0024730 con 0x7efdc01099a0 2024-05-14T16:43:25.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.133+0000 7efdc4909700 1 -- 172.21.15.2:0/1769261714 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdc0064ba0 con 0x7efdc01099a0 2024-05-14T16:43:25.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.137+0000 7efdaf7fe700 1 -- 172.21.15.2:0/1769261714 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7efdb0043ec0 con 0x7efdc01099a0 2024-05-14T16:43:25.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.210+0000 7efdc4909700 1 -- 172.21.15.2:0/1769261714 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7efdc01a0800 con 0x7efdc01099a0 2024-05-14T16:43:25.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.211+0000 7efdaf7fe700 1 -- 172.21.15.2:0/1769261714 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7efdb0032030 con 0x7efdc01099a0 2024-05-14T16:43:25.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.212+0000 7efdc4909700 1 -- 172.21.15.2:0/1769261714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdc01099a0 msgr2=0x7efdc019cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:25.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.212+0000 7efdc4909700 1 --2- 172.21.15.2:0/1769261714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdc01099a0 0x7efdc019cdf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efdb0009fb0 tx=0x7efdb0011940 comp rx=0 tx=0).stop 2024-05-14T16:43:25.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.212+0000 7efdc4909700 1 -- 172.21.15.2:0/1769261714 shutdown_connections 2024-05-14T16:43:25.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.212+0000 7efdc4909700 1 --2- 172.21.15.2:0/1769261714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdc01099a0 0x7efdc019cdf0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:25.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.212+0000 7efdc4909700 1 -- 172.21.15.2:0/1769261714 >> 172.21.15.2:0/1769261714 conn(0x7efdc01033b0 msgr2=0x7efdc0103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:43:25.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.212+0000 7efdc4909700 1 -- 172.21.15.2:0/1769261714 shutdown_connections 2024-05-14T16:43:25.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:25.212+0000 7efdc4909700 1 -- 172.21.15.2:0/1769261714 wait complete. 2024-05-14T16:43:25.855 INFO:teuthology.orchestra.run.smithi002.stdout:mgr not available, waiting (1/15)... 2024-05-14T16:43:26.994 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:26 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:26.718+0000 7f960d480200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-14T16:43:26.994 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:26 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:26.895+0000 7f960d480200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-14T16:43:27.374 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:27 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:27.072+0000 7f960d480200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-14T16:43:27.374 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:27 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:27.374+0000 7f960d480200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-14T16:43:27.691 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:27 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:27.470+0000 7f960d480200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-14T16:43:27.691 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:27 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:27.691+0000 7f960d480200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-14T16:43:27.975 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:27 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:27.896+0000 7f960d480200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:43:28.546 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:28 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/560354866' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-14T16:43:28.994 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:28 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:28.542+0000 7f960d480200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-14T16:43:28.994 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:28 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:28.777+0000 7f960d480200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-14T16:43:29.494 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:29 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:29.224+0000 7f960d480200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-14T16:43:29.494 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:29 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:29.324+0000 7f960d480200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-14T16:43:29.994 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:29 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:29.711+0000 7f960d480200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-14T16:43:30.442 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:30 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:30.086+0000 7f960d480200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-14T16:43:30.443 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:30 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:30.183+0000 7f960d480200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-14T16:43:30.508 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-05-14T16:43:30.524 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2024-05-14T16:43:30.524 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsid": "ed40717a-1210-11ef-bc98-c7b262605968", 2024-05-14T16:43:30.524 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "health": { 2024-05-14T16:43:30.525 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-14T16:43:30.525 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-14T16:43:30.525 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-14T16:43:30.525 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:43:30.525 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-14T16:43:30.525 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-14T16:43:30.525 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 0 2024-05-14T16:43:30.526 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-05-14T16:43:30.526 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-14T16:43:30.526 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "a" 2024-05-14T16:43:30.526 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-05-14T16:43:30.526 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-05-14T16:43:30.526 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-14T16:43:30.526 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T16:43:30.527 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-14T16:43:30.527 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-14T16:43:30.527 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:43:30.527 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-14T16:43:30.527 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T16:43:30.527 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-14T16:43:30.528 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-14T16:43:30.528 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-14T16:43:30.528 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-14T16:43:30.528 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-14T16:43:30.528 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-14T16:43:30.529 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:43:30.529 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-14T16:43:30.529 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-14T16:43:30.529 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-14T16:43:30.529 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-14T16:43:30.529 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-14T16:43:30.529 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-14T16:43:30.529 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-14T16:43:30.530 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-14T16:43:30.530 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-14T16:43:30.530 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:43:30.530 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-14T16:43:30.530 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T16:43:30.530 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-14T16:43:30.530 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-14T16:43:30.530 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:43:30.531 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-14T16:43:30.531 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-14T16:43:30.531 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-14T16:43:30.531 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-14T16:43:30.531 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-14T16:43:30.531 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-14T16:43:30.531 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "restful" 2024-05-14T16:43:30.531 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-05-14T16:43:30.531 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-14T16:43:30.532 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:43:30.532 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-14T16:43:30.532 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T16:43:30.532 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modified": "2024-05-14T16:43:09.415788+0000", 2024-05-14T16:43:30.532 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-14T16:43:30.532 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:43:30.532 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-14T16:43:30.532 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2024-05-14T16:43:30.533 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.265+0000 7faca447f700 1 Processor -- start 2024-05-14T16:43:30.533 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.265+0000 7faca447f700 1 -- start start 2024-05-14T16:43:30.533 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.266+0000 7faca447f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac9c1077e0 0x7fac9c107c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:30.533 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.266+0000 7faca447f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fac9c07cd90 con 0x7fac9c1077e0 2024-05-14T16:43:30.533 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.266+0000 7faca221b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac9c1077e0 0x7fac9c107c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:30.533 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.266+0000 7faca221b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac9c1077e0 0x7fac9c107c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38788/0 (socket says 172.21.15.2:38788) 2024-05-14T16:43:30.534 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.266+0000 7faca221b700 1 -- 172.21.15.2:0/608871273 learned_addr learned my addr 172.21.15.2:0/608871273 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:43:30.534 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.266+0000 7faca221b700 1 -- 172.21.15.2:0/608871273 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9c108140 con 0x7fac9c1077e0 2024-05-14T16:43:30.534 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.266+0000 7faca221b700 1 --2- 172.21.15.2:0/608871273 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac9c1077e0 0x7fac9c107c00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fac980098a0 tx=0x7fac98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a2da83e0e71c37 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:30.534 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.267+0000 7faca1219700 1 -- 172.21.15.2:0/608871273 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac98004160 con 0x7fac9c1077e0 2024-05-14T16:43:30.534 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.267+0000 7faca1219700 1 -- 172.21.15.2:0/608871273 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fac98004300 con 0x7fac9c1077e0 2024-05-14T16:43:30.534 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.267+0000 7faca1219700 1 -- 172.21.15.2:0/608871273 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac98011af0 con 0x7fac9c1077e0 2024-05-14T16:43:30.534 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.267+0000 7faca447f700 1 -- 172.21.15.2:0/608871273 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac9c1077e0 msgr2=0x7fac9c107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:30.534 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.267+0000 7faca447f700 1 --2- 172.21.15.2:0/608871273 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac9c1077e0 0x7fac9c107c00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fac980098a0 tx=0x7fac98009bb0 comp rx=0 tx=0).stop 2024-05-14T16:43:30.534 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.267+0000 7faca447f700 1 -- 172.21.15.2:0/608871273 shutdown_connections 2024-05-14T16:43:30.534 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.268+0000 7faca447f700 1 --2- 172.21.15.2:0/608871273 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac9c1077e0 0x7fac9c107c00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:30.534 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.268+0000 7faca447f700 1 -- 172.21.15.2:0/608871273 >> 172.21.15.2:0/608871273 conn(0x7fac9c1033a0 msgr2=0x7fac9c1057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:43:30.534 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.268+0000 7faca447f700 1 -- 172.21.15.2:0/608871273 shutdown_connections 2024-05-14T16:43:30.534 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.268+0000 7faca447f700 1 -- 172.21.15.2:0/608871273 wait complete. 2024-05-14T16:43:30.535 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.268+0000 7faca447f700 1 Processor -- start 2024-05-14T16:43:30.535 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.268+0000 7faca447f700 1 -- start start 2024-05-14T16:43:30.535 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.268+0000 7faca447f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac9c1077e0 0x7fac9c19f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:30.535 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.268+0000 7faca447f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fac9c109170 con 0x7fac9c1077e0 2024-05-14T16:43:30.535 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.269+0000 7faca221b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac9c1077e0 0x7fac9c19f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:30.535 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.269+0000 7faca221b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac9c1077e0 0x7fac9c19f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38790/0 (socket says 172.21.15.2:38790) 2024-05-14T16:43:30.535 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.269+0000 7faca221b700 1 -- 172.21.15.2:0/560354866 learned_addr learned my addr 172.21.15.2:0/560354866 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:43:30.535 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.269+0000 7faca221b700 1 -- 172.21.15.2:0/560354866 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9c19f5a0 con 0x7fac9c1077e0 2024-05-14T16:43:30.535 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.269+0000 7faca221b700 1 --2- 172.21.15.2:0/560354866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac9c1077e0 0x7fac9c19f060 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fac98011ac0 tx=0x7fac98011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:30.535 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.269+0000 7fac937fe700 1 -- 172.21.15.2:0/560354866 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac98003970 con 0x7fac9c1077e0 2024-05-14T16:43:30.535 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.269+0000 7fac937fe700 1 -- 172.21.15.2:0/560354866 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fac98003b10 con 0x7fac9c1077e0 2024-05-14T16:43:30.535 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.269+0000 7faca447f700 1 -- 172.21.15.2:0/560354866 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac9c19f830 con 0x7fac9c1077e0 2024-05-14T16:43:30.536 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.269+0000 7fac937fe700 1 -- 172.21.15.2:0/560354866 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac98024610 con 0x7fac9c1077e0 2024-05-14T16:43:30.536 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.269+0000 7faca447f700 1 -- 172.21.15.2:0/560354866 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac9c1a2520 con 0x7fac9c1077e0 2024-05-14T16:43:30.536 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.269+0000 7fac937fe700 1 -- 172.21.15.2:0/560354866 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fac98024c20 con 0x7fac9c1077e0 2024-05-14T16:43:30.536 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.269+0000 7fac937fe700 1 -- 172.21.15.2:0/560354866 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fac98019700 con 0x7fac9c1077e0 2024-05-14T16:43:30.536 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.270+0000 7faca447f700 1 -- 172.21.15.2:0/560354866 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac9c064ba0 con 0x7fac9c1077e0 2024-05-14T16:43:30.536 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.272+0000 7fac937fe700 1 -- 172.21.15.2:0/560354866 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fac9801a9e0 con 0x7fac9c1077e0 2024-05-14T16:43:30.536 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.348+0000 7faca447f700 1 -- 172.21.15.2:0/560354866 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fac9c02e770 con 0x7fac9c1077e0 2024-05-14T16:43:30.536 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.349+0000 7fac937fe700 1 -- 172.21.15.2:0/560354866 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7fac9801a430 con 0x7fac9c1077e0 2024-05-14T16:43:30.536 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.350+0000 7faca447f700 1 -- 172.21.15.2:0/560354866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac9c1077e0 msgr2=0x7fac9c19f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:30.536 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.350+0000 7faca447f700 1 --2- 172.21.15.2:0/560354866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac9c1077e0 0x7fac9c19f060 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fac98011ac0 tx=0x7fac98011ba0 comp rx=0 tx=0).stop 2024-05-14T16:43:30.536 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.350+0000 7faca447f700 1 -- 172.21.15.2:0/560354866 shutdown_connections 2024-05-14T16:43:30.536 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.350+0000 7faca447f700 1 --2- 172.21.15.2:0/560354866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac9c1077e0 0x7fac9c19f060 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:30.536 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.350+0000 7faca447f700 1 -- 172.21.15.2:0/560354866 >> 172.21.15.2:0/560354866 conn(0x7fac9c1033a0 msgr2=0x7fac9c195c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:43:30.537 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.350+0000 7faca447f700 1 -- 172.21.15.2:0/560354866 shutdown_connections 2024-05-14T16:43:30.537 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:28.350+0000 7faca447f700 1 -- 172.21.15.2:0/560354866 wait complete. 2024-05-14T16:43:30.537 INFO:teuthology.orchestra.run.smithi002.stdout:mgr not available, waiting (2/15)... 2024-05-14T16:43:30.744 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:30 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:30.442+0000 7f960d480200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-14T16:43:30.744 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:30 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:30.585+0000 7f960d480200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-14T16:43:30.744 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:30 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:30.686+0000 7f960d480200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-14T16:43:31.494 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:31 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:31.026+0000 7f960d480200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-14T16:43:31.494 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:31 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:31.123+0000 7f960d480200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-14T16:43:32.050 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:31 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:31.763+0000 7f960d480200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-14T16:43:32.050 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:31 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:31.997+0000 7f960d480200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-14T16:43:32.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:32 smithi002 ceph-mon[31538]: Activating manager daemon x 2024-05-14T16:43:32.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:32 smithi002 ceph-mon[31538]: mgrmap e2: x(active, starting, since 0.00317979s) 2024-05-14T16:43:32.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:32 smithi002 ceph-mon[31538]: from='mgr.14100 172.21.15.2:0/4266145764' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-14T16:43:32.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:32 smithi002 ceph-mon[31538]: from='mgr.14100 172.21.15.2:0/4266145764' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-14T16:43:32.495 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:32 smithi002 ceph-mon[31538]: from='mgr.14100 172.21.15.2:0/4266145764' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-14T16:43:32.495 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:32 smithi002 ceph-mon[31538]: from='mgr.14100 172.21.15.2:0/4266145764' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:43:32.495 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:32 smithi002 ceph-mon[31538]: from='mgr.14100 172.21.15.2:0/4266145764' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-14T16:43:32.495 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:32 smithi002 ceph-mon[31538]: Manager daemon x is now available 2024-05-14T16:43:32.495 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:32 smithi002 ceph-mon[31538]: from='mgr.14100 172.21.15.2:0/4266145764' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-14T16:43:32.495 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:32 smithi002 ceph-mon[31538]: from='mgr.14100 172.21.15.2:0/4266145764' entity='mgr.x' 2024-05-14T16:43:32.495 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:32 smithi002 ceph-mon[31538]: from='mgr.14100 172.21.15.2:0/4266145764' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-14T16:43:32.495 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:32 smithi002 ceph-mon[31538]: from='mgr.14100 172.21.15.2:0/4266145764' entity='mgr.x' 2024-05-14T16:43:32.495 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:32 smithi002 ceph-mon[31538]: from='mgr.14100 172.21.15.2:0/4266145764' entity='mgr.x' 2024-05-14T16:43:33.593 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-05-14T16:43:33.594 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2024-05-14T16:43:33.594 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsid": "ed40717a-1210-11ef-bc98-c7b262605968", 2024-05-14T16:43:33.594 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "health": { 2024-05-14T16:43:33.594 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-14T16:43:33.594 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-14T16:43:33.594 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-14T16:43:33.595 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:43:33.595 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-14T16:43:33.595 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-14T16:43:33.595 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 0 2024-05-14T16:43:33.595 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-05-14T16:43:33.595 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-14T16:43:33.595 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "a" 2024-05-14T16:43:33.595 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-05-14T16:43:33.595 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_age": 16, 2024-05-14T16:43:33.595 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-14T16:43:33.595 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T16:43:33.595 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-14T16:43:33.595 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-14T16:43:33.596 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:43:33.596 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-14T16:43:33.596 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T16:43:33.596 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-14T16:43:33.596 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-14T16:43:33.596 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-14T16:43:33.606 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-14T16:43:33.607 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-14T16:43:33.607 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-14T16:43:33.607 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:43:33.607 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-14T16:43:33.607 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-14T16:43:33.607 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-14T16:43:33.607 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-14T16:43:33.607 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-14T16:43:33.607 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-14T16:43:33.607 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-14T16:43:33.607 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-14T16:43:33.607 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-14T16:43:33.607 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:43:33.608 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-14T16:43:33.608 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T16:43:33.608 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-14T16:43:33.608 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-14T16:43:33.608 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:43:33.608 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-14T16:43:33.608 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-14T16:43:33.608 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-14T16:43:33.608 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-14T16:43:33.608 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-14T16:43:33.608 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-14T16:43:33.608 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "restful" 2024-05-14T16:43:33.608 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2024-05-14T16:43:33.608 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-14T16:43:33.608 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:43:33.609 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-14T16:43:33.609 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-14T16:43:33.609 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modified": "2024-05-14T16:43:09.415788+0000", 2024-05-14T16:43:33.609 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-14T16:43:33.609 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2024-05-14T16:43:33.609 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-14T16:43:33.609 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2024-05-14T16:43:33.609 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:32.937+0000 7f54f4cc8700 1 Processor -- start 2024-05-14T16:43:33.609 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:32.937+0000 7f54f4cc8700 1 -- start start 2024-05-14T16:43:33.609 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:32.937+0000 7f54f4cc8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f54f0108eb0 0x7f54f01092d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:33.609 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:32.937+0000 7f54f4cc8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f54f0109810 con 0x7f54f0108eb0 2024-05-14T16:43:33.609 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:32.937+0000 7f54ee59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f54f0108eb0 0x7f54f01092d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:33.609 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:32.937+0000 7f54ee59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f54f0108eb0 0x7f54f01092d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38866/0 (socket says 172.21.15.2:38866) 2024-05-14T16:43:33.609 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:32.937+0000 7f54ee59c700 1 -- 172.21.15.2:0/1566739341 learned_addr learned my addr 172.21.15.2:0/1566739341 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:43:33.610 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:32.938+0000 7f54ee59c700 1 -- 172.21.15.2:0/1566739341 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f010a050 con 0x7f54f0108eb0 2024-05-14T16:43:33.610 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:32.938+0000 7f54ee59c700 1 --2- 172.21.15.2:0/1566739341 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f54f0108eb0 0x7f54f01092d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f54e00098a0 tx=0x7f54e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7686076058864537 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:33.610 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:32.938+0000 7f54ed59a700 1 -- 172.21.15.2:0/1566739341 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54e0004160 con 0x7f54f0108eb0 2024-05-14T16:43:33.610 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:32.938+0000 7f54ed59a700 1 -- 172.21.15.2:0/1566739341 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f54e0004300 con 0x7f54f0108eb0 2024-05-14T16:43:33.610 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:32.938+0000 7f54ed59a700 1 -- 172.21.15.2:0/1566739341 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54e0011af0 con 0x7f54f0108eb0 2024-05-14T16:43:33.610 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:32.939+0000 7f54f4cc8700 1 -- 172.21.15.2:0/1566739341 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f54f0108eb0 msgr2=0x7f54f01092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:33.610 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:32.939+0000 7f54f4cc8700 1 --2- 172.21.15.2:0/1566739341 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f54f0108eb0 0x7f54f01092d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f54e00098a0 tx=0x7f54e0009bb0 comp rx=0 tx=0).stop 2024-05-14T16:43:33.610 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:32.939+0000 7f54f4cc8700 1 -- 172.21.15.2:0/1566739341 shutdown_connections 2024-05-14T16:43:33.611 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:32.939+0000 7f54f4cc8700 1 --2- 172.21.15.2:0/1566739341 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f54f0108eb0 0x7f54f01092d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:33.611 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:32.939+0000 7f54f4cc8700 1 -- 172.21.15.2:0/1566739341 >> 172.21.15.2:0/1566739341 conn(0x7f54f0104500 msgr2=0x7f54f0106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:43:33.611 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:32.939+0000 7f54f4cc8700 1 -- 172.21.15.2:0/1566739341 shutdown_connections 2024-05-14T16:43:33.611 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:32.939+0000 7f54f4cc8700 1 -- 172.21.15.2:0/1566739341 wait complete. 2024-05-14T16:43:33.611 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:32.940+0000 7f54f4cc8700 1 Processor -- start 2024-05-14T16:43:33.611 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:32.940+0000 7f54f4cc8700 1 -- start start 2024-05-14T16:43:33.611 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:32.940+0000 7f54f4cc8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f54f0108eb0 0x7f54f007f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:33.612 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:32.940+0000 7f54f4cc8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f54f010d310 con 0x7f54f0108eb0 2024-05-14T16:43:33.612 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:32.940+0000 7f54ee59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f54f0108eb0 0x7f54f007f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:33.612 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:32.940+0000 7f54ee59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f54f0108eb0 0x7f54f007f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38868/0 (socket says 172.21.15.2:38868) 2024-05-14T16:43:33.612 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:32.940+0000 7f54ee59c700 1 -- 172.21.15.2:0/1546535880 learned_addr learned my addr 172.21.15.2:0/1546535880 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:43:33.612 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:32.940+0000 7f54ee59c700 1 -- 172.21.15.2:0/1546535880 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f007f8c0 con 0x7f54f0108eb0 2024-05-14T16:43:33.612 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:32.940+0000 7f54ee59c700 1 --2- 172.21.15.2:0/1546535880 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f54f0108eb0 0x7f54f007f380 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f54e0009fb0 tx=0x7f54e0011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:33.612 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:32.941+0000 7f54df7fe700 1 -- 172.21.15.2:0/1546535880 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54e00036a0 con 0x7f54f0108eb0 2024-05-14T16:43:33.612 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:32.941+0000 7f54df7fe700 1 -- 172.21.15.2:0/1546535880 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f54e001a430 con 0x7f54f0108eb0 2024-05-14T16:43:33.612 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:32.941+0000 7f54f4cc8700 1 -- 172.21.15.2:0/1546535880 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54f007dab0 con 0x7f54f0108eb0 2024-05-14T16:43:33.612 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:32.941+0000 7f54df7fe700 1 -- 172.21.15.2:0/1546535880 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54e0019550 con 0x7f54f0108eb0 2024-05-14T16:43:33.612 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:32.941+0000 7f54f4cc8700 1 -- 172.21.15.2:0/1546535880 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54f007dff0 con 0x7f54f0108eb0 2024-05-14T16:43:33.613 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:32.941+0000 7f54df7fe700 1 -- 172.21.15.2:0/1546535880 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 2) ==== 49978+0+0 (secure 0 0 0) 0x7f54e0019800 con 0x7f54f0108eb0 2024-05-14T16:43:33.613 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:32.942+0000 7f54f4cc8700 1 -- 172.21.15.2:0/1546535880 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54f0064ba0 con 0x7f54f0108eb0 2024-05-14T16:43:33.613 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:32.942+0000 7f54df7fe700 1 -- 172.21.15.2:0/1546535880 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f54e0053400 con 0x7f54f0108eb0 2024-05-14T16:43:33.613 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:32.944+0000 7f54df7fe700 1 -- 172.21.15.2:0/1546535880 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f54e0017e90 con 0x7f54f0108eb0 2024-05-14T16:43:33.613 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:33.002+0000 7f54df7fe700 1 -- 172.21.15.2:0/1546535880 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 3) ==== 50105+0+0 (secure 0 0 0) 0x7f54e0018350 con 0x7f54f0108eb0 2024-05-14T16:43:33.613 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:33.003+0000 7f54df7fe700 1 --2- 172.21.15.2:0/1546535880 >> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] conn(0x7f54d803eff0 0x7f54d80413e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:33.613 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:33.003+0000 7f54edd9b700 1 --2- 172.21.15.2:0/1546535880 >> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] conn(0x7f54d803eff0 0x7f54d80413e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:33.613 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:33.004+0000 7f54edd9b700 1 --2- 172.21.15.2:0/1546535880 >> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] conn(0x7f54d803eff0 0x7f54d80413e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f54e4006fd0 tx=0x7f54e4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:33.613 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:33.021+0000 7f54f4cc8700 1 -- 172.21.15.2:0/1546535880 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f54f007e270 con 0x7f54f0108eb0 2024-05-14T16:43:33.613 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:33.021+0000 7f54df7fe700 1 -- 172.21.15.2:0/1546535880 <== mon.0 v2:172.21.15.2:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f54e0019e20 con 0x7f54f0108eb0 2024-05-14T16:43:33.613 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:33.022+0000 7f54f4cc8700 1 -- 172.21.15.2:0/1546535880 >> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] conn(0x7f54d803eff0 msgr2=0x7f54d80413e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:33.613 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:33.022+0000 7f54f4cc8700 1 --2- 172.21.15.2:0/1546535880 >> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] conn(0x7f54d803eff0 0x7f54d80413e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f54e4006fd0 tx=0x7f54e4006e40 comp rx=0 tx=0).stop 2024-05-14T16:43:33.613 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:33.022+0000 7f54f4cc8700 1 -- 172.21.15.2:0/1546535880 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f54f0108eb0 msgr2=0x7f54f007f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:33.613 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:33.022+0000 7f54f4cc8700 1 --2- 172.21.15.2:0/1546535880 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f54f0108eb0 0x7f54f007f380 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f54e0009fb0 tx=0x7f54e0011940 comp rx=0 tx=0).stop 2024-05-14T16:43:33.614 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:33.022+0000 7f54f4cc8700 1 -- 172.21.15.2:0/1546535880 shutdown_connections 2024-05-14T16:43:33.614 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:33.022+0000 7f54f4cc8700 1 --2- 172.21.15.2:0/1546535880 >> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] conn(0x7f54d803eff0 0x7f54d80413e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:33.614 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:33.022+0000 7f54f4cc8700 1 --2- 172.21.15.2:0/1546535880 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f54f0108eb0 0x7f54f007f380 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:33.614 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:33.022+0000 7f54f4cc8700 1 -- 172.21.15.2:0/1546535880 >> 172.21.15.2:0/1546535880 conn(0x7f54f0104500 msgr2=0x7f54f0105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:43:33.614 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:33.022+0000 7f54f4cc8700 1 -- 172.21.15.2:0/1546535880 shutdown_connections 2024-05-14T16:43:33.614 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:33.022+0000 7f54f4cc8700 1 -- 172.21.15.2:0/1546535880 wait complete. 2024-05-14T16:43:33.614 INFO:teuthology.orchestra.run.smithi002.stdout:mgr is available 2024-05-14T16:43:34.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:34 smithi002 ceph-mon[31538]: mgrmap e3: x(active, since 1.00708s) 2024-05-14T16:43:34.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:34 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/1546535880' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-14T16:43:34.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-05-14T16:43:34.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [global] 2024-05-14T16:43:34.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout fsid = ed40717a-1210-11ef-bc98-c7b262605968 2024-05-14T16:43:34.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-14T16:43:34.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-14T16:43:34.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-14T16:43:34.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-14T16:43:34.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-14T16:43:34.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-05-14T16:43:34.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-14T16:43:34.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-14T16:43:34.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2024-05-14T16:43:34.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [osd] 2024-05-14T16:43:34.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-14T16:43:34.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-14T16:43:34.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-14T16:43:34.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-14T16:43:34.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-14T16:43:34.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.029+0000 7fd59154b700 1 Processor -- start 2024-05-14T16:43:34.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.029+0000 7fd59154b700 1 -- start start 2024-05-14T16:43:34.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.029+0000 7fd59154b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd58c108e60 0x7fd58c109280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:34.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.030+0000 7fd59154b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd58c1097c0 con 0x7fd58c108e60 2024-05-14T16:43:34.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.030+0000 7fd58affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd58c108e60 0x7fd58c109280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:34.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.030+0000 7fd58affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd58c108e60 0x7fd58c109280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47984/0 (socket says 172.21.15.2:47984) 2024-05-14T16:43:34.872 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.030+0000 7fd58affd700 1 -- 172.21.15.2:0/359751093 learned_addr learned my addr 172.21.15.2:0/359751093 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:43:34.872 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.030+0000 7fd58affd700 1 -- 172.21.15.2:0/359751093 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd58c10a050 con 0x7fd58c108e60 2024-05-14T16:43:34.872 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.030+0000 7fd58affd700 1 --2- 172.21.15.2:0/359751093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd58c108e60 0x7fd58c109280 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd57c0098a0 tx=0x7fd57c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a34684a04842eaeb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:34.872 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.030+0000 7fd58a7fc700 1 -- 172.21.15.2:0/359751093 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd57c004160 con 0x7fd58c108e60 2024-05-14T16:43:34.872 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.030+0000 7fd58a7fc700 1 -- 172.21.15.2:0/359751093 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd57c004300 con 0x7fd58c108e60 2024-05-14T16:43:34.872 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.030+0000 7fd58a7fc700 1 -- 172.21.15.2:0/359751093 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd57c011af0 con 0x7fd58c108e60 2024-05-14T16:43:34.872 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.031+0000 7fd59154b700 1 -- 172.21.15.2:0/359751093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd58c108e60 msgr2=0x7fd58c109280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:34.872 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.031+0000 7fd59154b700 1 --2- 172.21.15.2:0/359751093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd58c108e60 0x7fd58c109280 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd57c0098a0 tx=0x7fd57c009bb0 comp rx=0 tx=0).stop 2024-05-14T16:43:34.872 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.031+0000 7fd59154b700 1 -- 172.21.15.2:0/359751093 shutdown_connections 2024-05-14T16:43:34.872 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.031+0000 7fd59154b700 1 --2- 172.21.15.2:0/359751093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd58c108e60 0x7fd58c109280 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:34.872 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.031+0000 7fd59154b700 1 -- 172.21.15.2:0/359751093 >> 172.21.15.2:0/359751093 conn(0x7fd58c104450 msgr2=0x7fd58c106890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:43:34.872 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.031+0000 7fd59154b700 1 -- 172.21.15.2:0/359751093 shutdown_connections 2024-05-14T16:43:34.872 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.031+0000 7fd59154b700 1 -- 172.21.15.2:0/359751093 wait complete. 2024-05-14T16:43:34.873 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.032+0000 7fd59154b700 1 Processor -- start 2024-05-14T16:43:34.873 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.032+0000 7fd59154b700 1 -- start start 2024-05-14T16:43:34.873 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.032+0000 7fd59154b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd58c108e60 0x7fd58c19ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:34.873 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.032+0000 7fd59154b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd58c10d310 con 0x7fd58c108e60 2024-05-14T16:43:34.873 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.032+0000 7fd58affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd58c108e60 0x7fd58c19ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:34.873 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.032+0000 7fd58affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd58c108e60 0x7fd58c19ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48000/0 (socket says 172.21.15.2:48000) 2024-05-14T16:43:34.873 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.032+0000 7fd58affd700 1 -- 172.21.15.2:0/2201320463 learned_addr learned my addr 172.21.15.2:0/2201320463 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:43:34.873 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.032+0000 7fd58affd700 1 -- 172.21.15.2:0/2201320463 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd58c19d3d0 con 0x7fd58c108e60 2024-05-14T16:43:34.873 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.032+0000 7fd58affd700 1 --2- 172.21.15.2:0/2201320463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd58c108e60 0x7fd58c19ce90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd57c0117b0 tx=0x7fd57c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:34.873 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.033+0000 7fd588ff9700 1 -- 172.21.15.2:0/2201320463 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd57c003af0 con 0x7fd58c108e60 2024-05-14T16:43:34.873 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.033+0000 7fd588ff9700 1 -- 172.21.15.2:0/2201320463 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd57c020070 con 0x7fd58c108e60 2024-05-14T16:43:34.873 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.033+0000 7fd59154b700 1 -- 172.21.15.2:0/2201320463 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd58c19d660 con 0x7fd58c108e60 2024-05-14T16:43:34.873 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.033+0000 7fd588ff9700 1 -- 172.21.15.2:0/2201320463 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd57c019630 con 0x7fd58c108e60 2024-05-14T16:43:34.874 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.033+0000 7fd59154b700 1 -- 172.21.15.2:0/2201320463 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd58c1a0350 con 0x7fd58c108e60 2024-05-14T16:43:34.874 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.034+0000 7fd588ff9700 1 -- 172.21.15.2:0/2201320463 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7fd57c0198f0 con 0x7fd58c108e60 2024-05-14T16:43:34.874 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.034+0000 7fd588ff9700 1 --2- 172.21.15.2:0/2201320463 >> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] conn(0x7fd57803e420 0x7fd5780408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:34.874 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.034+0000 7fd59154b700 1 -- 172.21.15.2:0/2201320463 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd58c064ba0 con 0x7fd58c108e60 2024-05-14T16:43:34.874 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.034+0000 7fd588ff9700 1 -- 172.21.15.2:0/2201320463 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd57c053aa0 con 0x7fd58c108e60 2024-05-14T16:43:34.874 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.034+0000 7fd583fff700 1 --2- 172.21.15.2:0/2201320463 >> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] conn(0x7fd57803e420 0x7fd5780408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:34.874 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.034+0000 7fd583fff700 1 --2- 172.21.15.2:0/2201320463 >> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] conn(0x7fd57803e420 0x7fd5780408e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd574006fd0 tx=0x7fd574006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:34.874 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.040+0000 7fd588ff9700 1 -- 172.21.15.2:0/2201320463 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd57c0042a0 con 0x7fd58c108e60 2024-05-14T16:43:34.874 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.252+0000 7fd59154b700 1 -- 172.21.15.2:0/2201320463 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fd58c1a0520 con 0x7fd58c108e60 2024-05-14T16:43:34.874 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.253+0000 7fd588ff9700 1 -- 172.21.15.2:0/2201320463 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7fd57c02bde0 con 0x7fd58c108e60 2024-05-14T16:43:34.874 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.256+0000 7fd59154b700 1 -- 172.21.15.2:0/2201320463 >> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] conn(0x7fd57803e420 msgr2=0x7fd5780408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:34.875 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.256+0000 7fd59154b700 1 --2- 172.21.15.2:0/2201320463 >> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] conn(0x7fd57803e420 0x7fd5780408e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd574006fd0 tx=0x7fd574006e40 comp rx=0 tx=0).stop 2024-05-14T16:43:34.875 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.256+0000 7fd59154b700 1 -- 172.21.15.2:0/2201320463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd58c108e60 msgr2=0x7fd58c19ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:34.875 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.256+0000 7fd59154b700 1 --2- 172.21.15.2:0/2201320463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd58c108e60 0x7fd58c19ce90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd57c0117b0 tx=0x7fd57c011b70 comp rx=0 tx=0).stop 2024-05-14T16:43:34.875 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.256+0000 7fd59154b700 1 -- 172.21.15.2:0/2201320463 shutdown_connections 2024-05-14T16:43:34.875 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.256+0000 7fd59154b700 1 --2- 172.21.15.2:0/2201320463 >> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] conn(0x7fd57803e420 0x7fd5780408e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:34.875 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.256+0000 7fd59154b700 1 --2- 172.21.15.2:0/2201320463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd58c108e60 0x7fd58c19ce90 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:34.875 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.256+0000 7fd59154b700 1 -- 172.21.15.2:0/2201320463 >> 172.21.15.2:0/2201320463 conn(0x7fd58c104450 msgr2=0x7fd58c104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:43:34.875 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.256+0000 7fd59154b700 1 -- 172.21.15.2:0/2201320463 shutdown_connections 2024-05-14T16:43:34.875 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:34.256+0000 7fd59154b700 1 -- 172.21.15.2:0/2201320463 wait complete. 2024-05-14T16:43:34.875 INFO:teuthology.orchestra.run.smithi002.stdout:Enabling cephadm module... 2024-05-14T16:43:35.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:35 smithi002 ceph-mon[31538]: mgrmap e4: x(active, since 2s) 2024-05-14T16:43:35.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:35 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/2201320463' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-14T16:43:36.291 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:36 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/1331012563' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-14T16:43:36.291 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:36 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: ignoring --setuser ceph since I am not root 2024-05-14T16:43:36.291 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:36 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: ignoring --setgroup ceph since I am not root 2024-05-14T16:43:36.291 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:36 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:36.192+0000 7fb88f025200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-14T16:43:36.608 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:36 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:36.385+0000 7fb88f025200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:43:36.609 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:36 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:36.491+0000 7fb88f025200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-14T16:43:36.616 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.254+0000 7f9b4c840700 1 Processor -- start 2024-05-14T16:43:36.616 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.254+0000 7f9b4c840700 1 -- start start 2024-05-14T16:43:36.616 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.254+0000 7f9b4c840700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b481077a0 0x7f9b48107bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:36.616 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.254+0000 7f9b4c840700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4807d610 con 0x7f9b481077a0 2024-05-14T16:43:36.616 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.255+0000 7f9b4659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b481077a0 0x7f9b48107bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:36.616 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.255+0000 7f9b4659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b481077a0 0x7f9b48107bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48016/0 (socket says 172.21.15.2:48016) 2024-05-14T16:43:36.616 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.255+0000 7f9b4659c700 1 -- 172.21.15.2:0/1472094091 learned_addr learned my addr 172.21.15.2:0/1472094091 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:43:36.616 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.255+0000 7f9b4659c700 1 -- 172.21.15.2:0/1472094091 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b48108100 con 0x7f9b481077a0 2024-05-14T16:43:36.617 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.255+0000 7f9b4659c700 1 --2- 172.21.15.2:0/1472094091 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b481077a0 0x7f9b48107bc0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9b38009e80 tx=0x7f9b3800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff7b96c997e6cc97 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:36.617 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.255+0000 7f9b4559a700 1 -- 172.21.15.2:0/1472094091 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b38017070 con 0x7f9b481077a0 2024-05-14T16:43:36.617 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.255+0000 7f9b4559a700 1 -- 172.21.15.2:0/1472094091 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9b3800b8c0 con 0x7f9b481077a0 2024-05-14T16:43:36.617 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.255+0000 7f9b4559a700 1 -- 172.21.15.2:0/1472094091 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b38004490 con 0x7f9b481077a0 2024-05-14T16:43:36.617 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.256+0000 7f9b4c840700 1 -- 172.21.15.2:0/1472094091 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b481077a0 msgr2=0x7f9b48107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:36.617 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.256+0000 7f9b4c840700 1 --2- 172.21.15.2:0/1472094091 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b481077a0 0x7f9b48107bc0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9b38009e80 tx=0x7f9b3800b330 comp rx=0 tx=0).stop 2024-05-14T16:43:36.617 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.256+0000 7f9b4c840700 1 -- 172.21.15.2:0/1472094091 shutdown_connections 2024-05-14T16:43:36.617 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.256+0000 7f9b4c840700 1 --2- 172.21.15.2:0/1472094091 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b481077a0 0x7f9b48107bc0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:36.617 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.256+0000 7f9b4c840700 1 -- 172.21.15.2:0/1472094091 >> 172.21.15.2:0/1472094091 conn(0x7f9b481033e0 msgr2=0x7f9b48105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:43:36.617 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.256+0000 7f9b4c840700 1 -- 172.21.15.2:0/1472094091 shutdown_connections 2024-05-14T16:43:36.617 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.256+0000 7f9b4c840700 1 -- 172.21.15.2:0/1472094091 wait complete. 2024-05-14T16:43:36.617 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.257+0000 7f9b4c840700 1 Processor -- start 2024-05-14T16:43:36.617 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.257+0000 7f9b4c840700 1 -- start start 2024-05-14T16:43:36.617 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.257+0000 7f9b4c840700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b481077a0 0x7f9b4819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:36.618 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.257+0000 7f9b4c840700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9b48109130 con 0x7f9b481077a0 2024-05-14T16:43:36.618 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.257+0000 7f9b4659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b481077a0 0x7f9b4819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:36.618 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.257+0000 7f9b4659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b481077a0 0x7f9b4819efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48030/0 (socket says 172.21.15.2:48030) 2024-05-14T16:43:36.618 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.257+0000 7f9b4659c700 1 -- 172.21.15.2:0/1331012563 learned_addr learned my addr 172.21.15.2:0/1331012563 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:43:36.618 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.257+0000 7f9b4659c700 1 -- 172.21.15.2:0/1331012563 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b4819f4f0 con 0x7f9b481077a0 2024-05-14T16:43:36.618 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.258+0000 7f9b4659c700 1 --2- 172.21.15.2:0/1331012563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b481077a0 0x7f9b4819efb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9b3800b7b0 tx=0x7f9b3800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:36.618 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.258+0000 7f9b3f7fe700 1 -- 172.21.15.2:0/1331012563 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b38020020 con 0x7f9b481077a0 2024-05-14T16:43:36.618 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.258+0000 7f9b3f7fe700 1 -- 172.21.15.2:0/1331012563 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9b38012430 con 0x7f9b481077a0 2024-05-14T16:43:36.618 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.258+0000 7f9b4c840700 1 -- 172.21.15.2:0/1331012563 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b4819f780 con 0x7f9b481077a0 2024-05-14T16:43:36.618 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.258+0000 7f9b3f7fe700 1 -- 172.21.15.2:0/1331012563 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b38017040 con 0x7f9b481077a0 2024-05-14T16:43:36.618 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.258+0000 7f9b4c840700 1 -- 172.21.15.2:0/1331012563 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b481a2470 con 0x7f9b481077a0 2024-05-14T16:43:36.618 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.259+0000 7f9b3f7fe700 1 -- 172.21.15.2:0/1331012563 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f9b38023020 con 0x7f9b481077a0 2024-05-14T16:43:36.618 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.259+0000 7f9b4c840700 1 -- 172.21.15.2:0/1331012563 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b48064ba0 con 0x7f9b481077a0 2024-05-14T16:43:36.618 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.259+0000 7f9b3f7fe700 1 --2- 172.21.15.2:0/1331012563 >> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] conn(0x7f9b3403e3d0 0x7f9b34040890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:36.619 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.259+0000 7f9b3f7fe700 1 -- 172.21.15.2:0/1331012563 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9b3800f050 con 0x7f9b481077a0 2024-05-14T16:43:36.619 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.259+0000 7f9b45d9b700 1 --2- 172.21.15.2:0/1331012563 >> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] conn(0x7f9b3403e3d0 0x7f9b34040890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:36.619 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.260+0000 7f9b45d9b700 1 --2- 172.21.15.2:0/1331012563 >> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] conn(0x7f9b3403e3d0 0x7f9b34040890 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9b30006fd0 tx=0x7f9b30006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:36.619 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.264+0000 7f9b3f7fe700 1 -- 172.21.15.2:0/1331012563 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9b3802a020 con 0x7f9b481077a0 2024-05-14T16:43:36.619 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:35.544+0000 7f9b4c840700 1 -- 172.21.15.2:0/1331012563 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f9b4802e8a0 con 0x7f9b481077a0 2024-05-14T16:43:36.619 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:36.007+0000 7f9b3f7fe700 1 -- 172.21.15.2:0/1331012563 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f9b3801b9d0 con 0x7f9b481077a0 2024-05-14T16:43:36.619 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:36.009+0000 7f9b3f7fe700 1 -- 172.21.15.2:0/1331012563 <== mon.0 v2:172.21.15.2:3300/0 8 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f9b3801bc90 con 0x7f9b481077a0 2024-05-14T16:43:36.619 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:36.012+0000 7f9b4c840700 1 -- 172.21.15.2:0/1331012563 >> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] conn(0x7f9b3403e3d0 msgr2=0x7f9b34040890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:36.619 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:36.012+0000 7f9b4c840700 1 --2- 172.21.15.2:0/1331012563 >> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] conn(0x7f9b3403e3d0 0x7f9b34040890 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9b30006fd0 tx=0x7f9b30006e40 comp rx=0 tx=0).stop 2024-05-14T16:43:36.619 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:36.012+0000 7f9b4c840700 1 -- 172.21.15.2:0/1331012563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b481077a0 msgr2=0x7f9b4819efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:36.619 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:36.012+0000 7f9b4c840700 1 --2- 172.21.15.2:0/1331012563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b481077a0 0x7f9b4819efb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9b3800b7b0 tx=0x7f9b3800be20 comp rx=0 tx=0).stop 2024-05-14T16:43:36.619 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:36.012+0000 7f9b4c840700 1 -- 172.21.15.2:0/1331012563 shutdown_connections 2024-05-14T16:43:36.619 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:36.012+0000 7f9b4c840700 1 --2- 172.21.15.2:0/1331012563 >> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] conn(0x7f9b3403e3d0 0x7f9b34040890 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:36.620 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:36.012+0000 7f9b4c840700 1 --2- 172.21.15.2:0/1331012563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b481077a0 0x7f9b4819efb0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:36.620 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:36.012+0000 7f9b4c840700 1 -- 172.21.15.2:0/1331012563 >> 172.21.15.2:0/1331012563 conn(0x7f9b481033e0 msgr2=0x7f9b48195c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:43:36.620 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:36.012+0000 7f9b4c840700 1 -- 172.21.15.2:0/1331012563 shutdown_connections 2024-05-14T16:43:36.620 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:36.012+0000 7f9b4c840700 1 -- 172.21.15.2:0/1331012563 wait complete. 2024-05-14T16:43:36.860 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:36 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:36.817+0000 7fb88f025200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-14T16:43:37.244 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:36 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:36.909+0000 7fb88f025200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-14T16:43:37.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:37 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/1331012563' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-14T16:43:37.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:37 smithi002 ceph-mon[31538]: mgrmap e5: x(active, since 4s) 2024-05-14T16:43:37.990 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2024-05-14T16:43:37.990 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-14T16:43:37.990 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-14T16:43:37.990 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-14T16:43:37.990 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-14T16:43:37.990 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2024-05-14T16:43:37.990 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.073+0000 7f126f0dd700 1 Processor -- start 2024-05-14T16:43:37.990 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.073+0000 7f126f0dd700 1 -- start start 2024-05-14T16:43:37.990 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.073+0000 7f126f0dd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1268109a20 0x7f126810be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:37.990 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.073+0000 7f126f0dd700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f126807d5b0 con 0x7f1268109a20 2024-05-14T16:43:37.990 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.073+0000 7f126ce79700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1268109a20 0x7f126810be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:37.991 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.074+0000 7f126ce79700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1268109a20 0x7f126810be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48054/0 (socket says 172.21.15.2:48054) 2024-05-14T16:43:37.991 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.074+0000 7f126ce79700 1 -- 172.21.15.2:0/104581261 learned_addr learned my addr 172.21.15.2:0/104581261 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:43:37.991 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.074+0000 7f126ce79700 1 -- 172.21.15.2:0/104581261 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f126810c350 con 0x7f1268109a20 2024-05-14T16:43:37.991 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.074+0000 7f126ce79700 1 --2- 172.21.15.2:0/104581261 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1268109a20 0x7f126810be10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f125c0098a0 tx=0x7f125c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3342d20003036da server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:37.991 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.074+0000 7f12677fe700 1 -- 172.21.15.2:0/104581261 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f125c004160 con 0x7f1268109a20 2024-05-14T16:43:37.991 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.074+0000 7f12677fe700 1 -- 172.21.15.2:0/104581261 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f125c004300 con 0x7f1268109a20 2024-05-14T16:43:37.991 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.074+0000 7f12677fe700 1 -- 172.21.15.2:0/104581261 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f125c011af0 con 0x7f1268109a20 2024-05-14T16:43:37.991 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.075+0000 7f126f0dd700 1 -- 172.21.15.2:0/104581261 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1268109a20 msgr2=0x7f126810be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:37.991 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.075+0000 7f126f0dd700 1 --2- 172.21.15.2:0/104581261 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1268109a20 0x7f126810be10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f125c0098a0 tx=0x7f125c009bb0 comp rx=0 tx=0).stop 2024-05-14T16:43:37.991 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.075+0000 7f126f0dd700 1 -- 172.21.15.2:0/104581261 shutdown_connections 2024-05-14T16:43:37.991 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.075+0000 7f126f0dd700 1 --2- 172.21.15.2:0/104581261 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1268109a20 0x7f126810be10 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:37.991 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.075+0000 7f126f0dd700 1 -- 172.21.15.2:0/104581261 >> 172.21.15.2:0/104581261 conn(0x7f12681033b0 msgr2=0x7f12681057d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:43:37.991 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.075+0000 7f126f0dd700 1 -- 172.21.15.2:0/104581261 shutdown_connections 2024-05-14T16:43:37.991 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.075+0000 7f126f0dd700 1 -- 172.21.15.2:0/104581261 wait complete. 2024-05-14T16:43:37.991 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.076+0000 7f126f0dd700 1 Processor -- start 2024-05-14T16:43:37.992 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.076+0000 7f126f0dd700 1 -- start start 2024-05-14T16:43:37.992 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.076+0000 7f126f0dd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1268109a20 0x7f12681092a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:37.992 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.076+0000 7f126f0dd700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f126810d250 con 0x7f1268109a20 2024-05-14T16:43:37.992 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.076+0000 7f126ce79700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1268109a20 0x7f12681092a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:37.992 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.076+0000 7f126ce79700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1268109a20 0x7f12681092a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48066/0 (socket says 172.21.15.2:48066) 2024-05-14T16:43:37.992 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.076+0000 7f126ce79700 1 -- 172.21.15.2:0/1250000733 learned_addr learned my addr 172.21.15.2:0/1250000733 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:43:37.992 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.076+0000 7f126ce79700 1 -- 172.21.15.2:0/1250000733 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12681097e0 con 0x7f1268109a20 2024-05-14T16:43:37.992 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.077+0000 7f126ce79700 1 --2- 172.21.15.2:0/1250000733 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1268109a20 0x7f12681092a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f125c011ac0 tx=0x7f125c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:37.992 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.077+0000 7f1265ffb700 1 -- 172.21.15.2:0/1250000733 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f125c003970 con 0x7f1268109a20 2024-05-14T16:43:37.992 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.077+0000 7f1265ffb700 1 -- 172.21.15.2:0/1250000733 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f125c003b10 con 0x7f1268109a20 2024-05-14T16:43:37.992 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.077+0000 7f1265ffb700 1 -- 172.21.15.2:0/1250000733 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f125c024610 con 0x7f1268109a20 2024-05-14T16:43:37.992 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.077+0000 7f126f0dd700 1 -- 172.21.15.2:0/1250000733 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12681079d0 con 0x7f1268109a20 2024-05-14T16:43:37.992 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.077+0000 7f126f0dd700 1 -- 172.21.15.2:0/1250000733 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1268107f10 con 0x7f1268109a20 2024-05-14T16:43:37.992 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.078+0000 7f1265ffb700 1 -- 172.21.15.2:0/1250000733 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f125c01a430 con 0x7f1268109a20 2024-05-14T16:43:37.993 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.078+0000 7f126f0dd700 1 -- 172.21.15.2:0/1250000733 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1268064ba0 con 0x7f1268109a20 2024-05-14T16:43:37.993 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.078+0000 7f1265ffb700 1 --2- 172.21.15.2:0/1250000733 >> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] conn(0x7f125803e4c0 0x7f1258040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:37.993 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.078+0000 7f1265ffb700 1 -- 172.21.15.2:0/1250000733 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f125c053810 con 0x7f1268109a20 2024-05-14T16:43:37.993 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.078+0000 7f1267fff700 1 -- 172.21.15.2:0/1250000733 >> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] conn(0x7f125803e4c0 msgr2=0x7f1258040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/2574015037 2024-05-14T16:43:37.993 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.078+0000 7f1267fff700 1 --2- 172.21.15.2:0/1250000733 >> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] conn(0x7f125803e4c0 0x7f1258040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T16:43:37.993 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.084+0000 7f1265ffb700 1 -- 172.21.15.2:0/1250000733 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f125c00b3e0 con 0x7f1268109a20 2024-05-14T16:43:37.993 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.279+0000 7f1267fff700 1 -- 172.21.15.2:0/1250000733 >> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] conn(0x7f125803e4c0 msgr2=0x7f1258040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/2574015037 2024-05-14T16:43:37.993 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.279+0000 7f1267fff700 1 --2- 172.21.15.2:0/1250000733 >> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] conn(0x7f125803e4c0 0x7f1258040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-14T16:43:37.993 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.405+0000 7f126f0dd700 1 -- 172.21.15.2:0/1250000733 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f1268108840 con 0x7f1268109a20 2024-05-14T16:43:37.993 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.406+0000 7f1265ffb700 1 -- 172.21.15.2:0/1250000733 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f125c053230 con 0x7f1268109a20 2024-05-14T16:43:37.993 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.408+0000 7f126f0dd700 1 -- 172.21.15.2:0/1250000733 >> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] conn(0x7f125803e4c0 msgr2=0x7f1258040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T16:43:37.993 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.408+0000 7f126f0dd700 1 --2- 172.21.15.2:0/1250000733 >> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] conn(0x7f125803e4c0 0x7f1258040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:37.993 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.408+0000 7f126f0dd700 1 -- 172.21.15.2:0/1250000733 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1268109a20 msgr2=0x7f12681092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:37.993 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.408+0000 7f126f0dd700 1 --2- 172.21.15.2:0/1250000733 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1268109a20 0x7f12681092a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f125c011ac0 tx=0x7f125c011ba0 comp rx=0 tx=0).stop 2024-05-14T16:43:37.994 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.408+0000 7f126f0dd700 1 -- 172.21.15.2:0/1250000733 shutdown_connections 2024-05-14T16:43:37.994 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.408+0000 7f126f0dd700 1 --2- 172.21.15.2:0/1250000733 >> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] conn(0x7f125803e4c0 0x7f1258040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:37.994 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.408+0000 7f126f0dd700 1 --2- 172.21.15.2:0/1250000733 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1268109a20 0x7f12681092a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:37.994 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.408+0000 7f126f0dd700 1 -- 172.21.15.2:0/1250000733 >> 172.21.15.2:0/1250000733 conn(0x7f12681033b0 msgr2=0x7f1268103e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:43:37.994 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.408+0000 7f126f0dd700 1 -- 172.21.15.2:0/1250000733 shutdown_connections 2024-05-14T16:43:37.994 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:37.408+0000 7f126f0dd700 1 -- 172.21.15.2:0/1250000733 wait complete. 2024-05-14T16:43:37.994 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for the mgr to restart... 2024-05-14T16:43:37.994 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mgr epoch 5... 2024-05-14T16:43:38.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:38 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/1250000733' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-14T16:43:38.244 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:38 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:38.177+0000 7fb88f025200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-14T16:43:38.622 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:38 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:38.359+0000 7fb88f025200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-14T16:43:38.622 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:38 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:38.538+0000 7fb88f025200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-14T16:43:38.937 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:38 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:38.839+0000 7fb88f025200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-14T16:43:39.244 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:38 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:38.935+0000 7fb88f025200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-14T16:43:39.244 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:39 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:39.147+0000 7fb88f025200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-14T16:43:39.744 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:39 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:39.348+0000 7fb88f025200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:43:40.206 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:39 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:39.977+0000 7fb88f025200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-14T16:43:40.994 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:40 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:40.206+0000 7fb88f025200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-14T16:43:40.994 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:40 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:40.646+0000 7fb88f025200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-14T16:43:40.994 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:40 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:40.746+0000 7fb88f025200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-14T16:43:41.425 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:41 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:41.129+0000 7fb88f025200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-14T16:43:41.744 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:41 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:41.502+0000 7fb88f025200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-14T16:43:41.744 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:41 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:41.599+0000 7fb88f025200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-14T16:43:42.244 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:41 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:41.853+0000 7fb88f025200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-14T16:43:42.244 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:41 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:41.993+0000 7fb88f025200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-14T16:43:42.244 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:42 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:42.093+0000 7fb88f025200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-14T16:43:42.546 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:42 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:42.449+0000 7fb88f025200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-14T16:43:42.546 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:42 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:42.546+0000 7fb88f025200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-14T16:43:43.457 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:43 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:43.178+0000 7fb88f025200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-14T16:43:43.458 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:43 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:43:43.405+0000 7fb88f025200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-14T16:43:43.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:43 smithi002 ceph-mon[31538]: Active manager daemon x restarted 2024-05-14T16:43:43.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:43 smithi002 ceph-mon[31538]: Activating manager daemon x 2024-05-14T16:43:43.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:43 smithi002 ceph-mon[31538]: osdmap e2: 0 total, 0 up, 0 in 2024-05-14T16:43:43.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:43 smithi002 ceph-mon[31538]: mgrmap e6: x(active, starting, since 0.00341845s) 2024-05-14T16:43:43.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:43 smithi002 ceph-mon[31538]: from='mgr.14118 172.21.15.2:0/1997519028' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:43:43.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:43 smithi002 ceph-mon[31538]: from='mgr.14118 172.21.15.2:0/1997519028' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-14T16:43:43.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:43 smithi002 ceph-mon[31538]: from='mgr.14118 172.21.15.2:0/1997519028' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-14T16:43:43.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:43 smithi002 ceph-mon[31538]: from='mgr.14118 172.21.15.2:0/1997519028' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-14T16:43:43.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:43 smithi002 ceph-mon[31538]: from='mgr.14118 172.21.15.2:0/1997519028' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-14T16:43:43.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:43 smithi002 ceph-mon[31538]: Manager daemon x is now available 2024-05-14T16:43:43.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:43 smithi002 ceph-mon[31538]: from='mgr.14118 172.21.15.2:0/1997519028' entity='mgr.x' 2024-05-14T16:43:43.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:43 smithi002 ceph-mon[31538]: from='mgr.14118 172.21.15.2:0/1997519028' entity='mgr.x' 2024-05-14T16:43:43.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:43 smithi002 ceph-mon[31538]: from='mgr.14118 172.21.15.2:0/1997519028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:43:43.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:43 smithi002 ceph-mon[31538]: from='mgr.14118 172.21.15.2:0/1997519028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:43:43.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:43 smithi002 ceph-mon[31538]: from='mgr.14118 172.21.15.2:0/1997519028' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-14T16:43:44.725 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:44 smithi002 ceph-mon[31538]: Found migration_current of "None". Setting to last migration. 2024-05-14T16:43:44.725 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:44 smithi002 ceph-mon[31538]: from='mgr.14118 172.21.15.2:0/1997519028' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-14T16:43:44.725 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:44 smithi002 ceph-mon[31538]: from='mgr.14118 172.21.15.2:0/1997519028' entity='mgr.x' 2024-05-14T16:43:44.725 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:44 smithi002 ceph-mon[31538]: from='mgr.14118 172.21.15.2:0/1997519028' entity='mgr.x' 2024-05-14T16:43:44.725 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:44 smithi002 ceph-mon[31538]: mgrmap e7: x(active, since 1.00609s) 2024-05-14T16:43:44.948 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2024-05-14T16:43:44.948 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-14T16:43:44.948 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-14T16:43:44.948 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2024-05-14T16:43:44.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.414+0000 7f1f5b7c0700 1 Processor -- start 2024-05-14T16:43:44.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.414+0000 7f1f5b7c0700 1 -- start start 2024-05-14T16:43:44.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.414+0000 7f1f5b7c0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f54109960 0x7f1f5410bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:44.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.415+0000 7f1f5b7c0700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5407d670 con 0x7f1f54109960 2024-05-14T16:43:44.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.415+0000 7f1f5955c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f54109960 0x7f1f5410bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:44.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.415+0000 7f1f5955c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f54109960 0x7f1f5410bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48080/0 (socket says 172.21.15.2:48080) 2024-05-14T16:43:44.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.415+0000 7f1f5955c700 1 -- 172.21.15.2:0/3356670909 learned_addr learned my addr 172.21.15.2:0/3356670909 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:43:44.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.415+0000 7f1f5955c700 1 -- 172.21.15.2:0/3356670909 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f5410c290 con 0x7f1f54109960 2024-05-14T16:43:44.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.415+0000 7f1f5955c700 1 --2- 172.21.15.2:0/3356670909 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f54109960 0x7f1f5410bd50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1f500098a0 tx=0x7f1f50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=923b93d7c1e2a5fb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:44.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.415+0000 7f1f47fff700 1 -- 172.21.15.2:0/3356670909 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f50004160 con 0x7f1f54109960 2024-05-14T16:43:44.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.416+0000 7f1f47fff700 1 -- 172.21.15.2:0/3356670909 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1f50004300 con 0x7f1f54109960 2024-05-14T16:43:44.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.416+0000 7f1f47fff700 1 -- 172.21.15.2:0/3356670909 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f50011af0 con 0x7f1f54109960 2024-05-14T16:43:44.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.416+0000 7f1f5b7c0700 1 -- 172.21.15.2:0/3356670909 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f54109960 msgr2=0x7f1f5410bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:44.950 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.416+0000 7f1f5b7c0700 1 --2- 172.21.15.2:0/3356670909 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f54109960 0x7f1f5410bd50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1f500098a0 tx=0x7f1f50009bb0 comp rx=0 tx=0).stop 2024-05-14T16:43:44.950 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.416+0000 7f1f5b7c0700 1 -- 172.21.15.2:0/3356670909 shutdown_connections 2024-05-14T16:43:44.950 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.416+0000 7f1f5b7c0700 1 --2- 172.21.15.2:0/3356670909 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f54109960 0x7f1f5410bd50 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:44.950 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.416+0000 7f1f5b7c0700 1 -- 172.21.15.2:0/3356670909 >> 172.21.15.2:0/3356670909 conn(0x7f1f541033b0 msgr2=0x7f1f541057d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:43:44.950 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.416+0000 7f1f5b7c0700 1 -- 172.21.15.2:0/3356670909 shutdown_connections 2024-05-14T16:43:44.950 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.417+0000 7f1f5b7c0700 1 -- 172.21.15.2:0/3356670909 wait complete. 2024-05-14T16:43:44.950 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.417+0000 7f1f5b7c0700 1 Processor -- start 2024-05-14T16:43:44.950 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.417+0000 7f1f5b7c0700 1 -- start start 2024-05-14T16:43:44.950 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.417+0000 7f1f5b7c0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f54109960 0x7f1f5419cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:44.950 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.417+0000 7f1f5b7c0700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5410d250 con 0x7f1f54109960 2024-05-14T16:43:44.950 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.417+0000 7f1f5955c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f54109960 0x7f1f5419cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:44.950 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.418+0000 7f1f5955c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f54109960 0x7f1f5419cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48084/0 (socket says 172.21.15.2:48084) 2024-05-14T16:43:44.950 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.418+0000 7f1f5955c700 1 -- 172.21.15.2:0/2451539073 learned_addr learned my addr 172.21.15.2:0/2451539073 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:43:44.951 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.418+0000 7f1f5955c700 1 -- 172.21.15.2:0/2451539073 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f5419d330 con 0x7f1f54109960 2024-05-14T16:43:44.951 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.418+0000 7f1f5955c700 1 --2- 172.21.15.2:0/2451539073 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f54109960 0x7f1f5419cdf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1f50011ac0 tx=0x7f1f50011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:44.951 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.418+0000 7f1f467fc700 1 -- 172.21.15.2:0/2451539073 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f50003970 con 0x7f1f54109960 2024-05-14T16:43:44.951 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.418+0000 7f1f467fc700 1 -- 172.21.15.2:0/2451539073 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1f50003b10 con 0x7f1f54109960 2024-05-14T16:43:44.951 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.418+0000 7f1f5b7c0700 1 -- 172.21.15.2:0/2451539073 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f5419d5c0 con 0x7f1f54109960 2024-05-14T16:43:44.951 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.418+0000 7f1f467fc700 1 -- 172.21.15.2:0/2451539073 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f50024610 con 0x7f1f54109960 2024-05-14T16:43:44.951 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.418+0000 7f1f5b7c0700 1 -- 172.21.15.2:0/2451539073 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f541a02b0 con 0x7f1f54109960 2024-05-14T16:43:44.951 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.419+0000 7f1f467fc700 1 -- 172.21.15.2:0/2451539073 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f1f50024890 con 0x7f1f54109960 2024-05-14T16:43:44.951 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.420+0000 7f1f467fc700 1 --2- 172.21.15.2:0/2451539073 >> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] conn(0x7f1f4003e470 0x7f1f40040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:44.951 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.420+0000 7f1f467fc700 1 -- 172.21.15.2:0/2451539073 --> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1f40041040 con 0x7f1f4003e470 2024-05-14T16:43:44.951 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.420+0000 7f1f467fc700 1 -- 172.21.15.2:0/2451539073 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1f50053860 con 0x7f1f54109960 2024-05-14T16:43:44.951 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.420+0000 7f1f58d5b700 1 -- 172.21.15.2:0/2451539073 >> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] conn(0x7f1f4003e470 msgr2=0x7f1f40040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/2574015037 2024-05-14T16:43:44.951 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.420+0000 7f1f58d5b700 1 --2- 172.21.15.2:0/2451539073 >> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] conn(0x7f1f4003e470 0x7f1f40040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T16:43:44.952 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.621+0000 7f1f58d5b700 1 -- 172.21.15.2:0/2451539073 >> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] conn(0x7f1f4003e470 msgr2=0x7f1f40040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/2574015037 2024-05-14T16:43:44.952 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:38.621+0000 7f1f58d5b700 1 --2- 172.21.15.2:0/2451539073 >> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] conn(0x7f1f4003e470 0x7f1f40040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-14T16:43:44.952 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:39.022+0000 7f1f58d5b700 1 -- 172.21.15.2:0/2451539073 >> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] conn(0x7f1f4003e470 msgr2=0x7f1f40040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/2574015037 2024-05-14T16:43:44.952 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:39.022+0000 7f1f58d5b700 1 --2- 172.21.15.2:0/2451539073 >> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] conn(0x7f1f4003e470 0x7f1f40040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-14T16:43:44.952 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:39.824+0000 7f1f58d5b700 1 -- 172.21.15.2:0/2451539073 >> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] conn(0x7f1f4003e470 msgr2=0x7f1f40040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/2574015037 2024-05-14T16:43:44.952 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:39.824+0000 7f1f58d5b700 1 --2- 172.21.15.2:0/2451539073 >> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] conn(0x7f1f4003e470 0x7f1f40040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-14T16:43:44.952 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:41.425+0000 7f1f58d5b700 1 -- 172.21.15.2:0/2451539073 >> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] conn(0x7f1f4003e470 msgr2=0x7f1f40040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/2574015037 2024-05-14T16:43:44.952 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:41.425+0000 7f1f58d5b700 1 --2- 172.21.15.2:0/2451539073 >> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] conn(0x7f1f4003e470 0x7f1f40040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-14T16:43:44.952 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:43.408+0000 7f1f467fc700 1 -- 172.21.15.2:0/2451539073 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7f1f5002d490 con 0x7f1f54109960 2024-05-14T16:43:44.952 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:43.408+0000 7f1f467fc700 1 -- 172.21.15.2:0/2451539073 >> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] conn(0x7f1f4003e470 msgr2=0x7f1f40040930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T16:43:44.952 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:43.408+0000 7f1f467fc700 1 --2- 172.21.15.2:0/2451539073 >> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] conn(0x7f1f4003e470 0x7f1f40040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:44.952 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:44.410+0000 7f1f467fc700 1 -- 172.21.15.2:0/2451539073 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f1f50055820 con 0x7f1f54109960 2024-05-14T16:43:44.952 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:44.410+0000 7f1f467fc700 1 --2- 172.21.15.2:0/2451539073 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f1f40041d60 0x7f1f40044150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:44.952 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:44.410+0000 7f1f467fc700 1 -- 172.21.15.2:0/2451539073 --> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1f40041040 con 0x7f1f40041d60 2024-05-14T16:43:44.953 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:44.410+0000 7f1f58d5b700 1 --2- 172.21.15.2:0/2451539073 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f1f40041d60 0x7f1f40044150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:44.953 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:44.411+0000 7f1f58d5b700 1 --2- 172.21.15.2:0/2451539073 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f1f40041d60 0x7f1f40044150 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1f48003a10 tx=0x7f1f48009290 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:44.953 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:44.412+0000 7f1f467fc700 1 -- 172.21.15.2:0/2451539073 <== mgr.14118 v2:172.21.15.2:6800/3582988344 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f1f40041040 con 0x7f1f40041d60 2024-05-14T16:43:44.953 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:44.420+0000 7f1f3ffff700 1 -- 172.21.15.2:0/2451539073 --> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f1f5406a650 con 0x7f1f40041d60 2024-05-14T16:43:44.953 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:44.420+0000 7f1f467fc700 1 -- 172.21.15.2:0/2451539073 <== mgr.14118 v2:172.21.15.2:6800/3582988344 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f1f5406a650 con 0x7f1f40041d60 2024-05-14T16:43:44.953 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:44.421+0000 7f1f3ffff700 1 -- 172.21.15.2:0/2451539073 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f1f40041d60 msgr2=0x7f1f40044150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:44.953 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:44.421+0000 7f1f3ffff700 1 --2- 172.21.15.2:0/2451539073 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f1f40041d60 0x7f1f40044150 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1f48003a10 tx=0x7f1f48009290 comp rx=0 tx=0).stop 2024-05-14T16:43:44.953 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:44.421+0000 7f1f3ffff700 1 -- 172.21.15.2:0/2451539073 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f54109960 msgr2=0x7f1f5419cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:44.953 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:44.421+0000 7f1f3ffff700 1 --2- 172.21.15.2:0/2451539073 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f54109960 0x7f1f5419cdf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1f50011ac0 tx=0x7f1f50011ba0 comp rx=0 tx=0).stop 2024-05-14T16:43:44.953 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:44.421+0000 7f1f3ffff700 1 -- 172.21.15.2:0/2451539073 shutdown_connections 2024-05-14T16:43:44.953 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:44.421+0000 7f1f3ffff700 1 --2- 172.21.15.2:0/2451539073 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f54109960 0x7f1f5419cdf0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:44.953 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:44.421+0000 7f1f3ffff700 1 --2- 172.21.15.2:0/2451539073 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f1f40041d60 0x7f1f40044150 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:44.954 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:44.421+0000 7f1f3ffff700 1 --2- 172.21.15.2:0/2451539073 >> [v2:172.21.15.2:6800/2574015037,v1:172.21.15.2:6801/2574015037] conn(0x7f1f4003e470 0x7f1f40040930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:44.954 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:44.421+0000 7f1f3ffff700 1 -- 172.21.15.2:0/2451539073 >> 172.21.15.2:0/2451539073 conn(0x7f1f541033b0 msgr2=0x7f1f54103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:43:44.954 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:44.421+0000 7f1f3ffff700 1 -- 172.21.15.2:0/2451539073 shutdown_connections 2024-05-14T16:43:44.954 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:44.421+0000 7f1f3ffff700 1 -- 172.21.15.2:0/2451539073 wait complete. 2024-05-14T16:43:44.954 INFO:teuthology.orchestra.run.smithi002.stdout:mgr epoch 5 is available 2024-05-14T16:43:44.954 INFO:teuthology.orchestra.run.smithi002.stdout:Setting orchestrator backend to cephadm... 2024-05-14T16:43:45.665 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:45 smithi002 ceph-mon[31538]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-14T16:43:45.665 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:45 smithi002 ceph-mon[31538]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-14T16:43:45.665 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:45 smithi002 ceph-mon[31538]: [14/May/2024:16:43:45] ENGINE Bus STARTING 2024-05-14T16:43:45.665 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:45 smithi002 ceph-mon[31538]: from='mgr.14118 172.21.15.2:0/1997519028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:43:46.197 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.403+0000 7f6b6df93700 1 Processor -- start 2024-05-14T16:43:46.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.404+0000 7f6b6df93700 1 -- start start 2024-05-14T16:43:46.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.404+0000 7f6b6df93700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b68108ed0 0x7f6b681092f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:46.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.404+0000 7f6b6df93700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6b68109830 con 0x7f6b68108ed0 2024-05-14T16:43:46.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.404+0000 7f6b677fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b68108ed0 0x7f6b681092f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:46.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.404+0000 7f6b677fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b68108ed0 0x7f6b681092f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36052/0 (socket says 172.21.15.2:36052) 2024-05-14T16:43:46.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.404+0000 7f6b677fe700 1 -- 172.21.15.2:0/937453892 learned_addr learned my addr 172.21.15.2:0/937453892 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:43:46.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.404+0000 7f6b677fe700 1 -- 172.21.15.2:0/937453892 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b68109fe0 con 0x7f6b68108ed0 2024-05-14T16:43:46.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.404+0000 7f6b677fe700 1 --2- 172.21.15.2:0/937453892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b68108ed0 0x7f6b681092f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6b580098a0 tx=0x7f6b58009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb8d4751826ef26 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:46.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.405+0000 7f6b667fc700 1 -- 172.21.15.2:0/937453892 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b58004160 con 0x7f6b68108ed0 2024-05-14T16:43:46.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.405+0000 7f6b667fc700 1 -- 172.21.15.2:0/937453892 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6b58004300 con 0x7f6b68108ed0 2024-05-14T16:43:46.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.405+0000 7f6b667fc700 1 -- 172.21.15.2:0/937453892 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b58011af0 con 0x7f6b68108ed0 2024-05-14T16:43:46.198 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.405+0000 7f6b6df93700 1 -- 172.21.15.2:0/937453892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b68108ed0 msgr2=0x7f6b681092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:46.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.405+0000 7f6b6df93700 1 --2- 172.21.15.2:0/937453892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b68108ed0 0x7f6b681092f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6b580098a0 tx=0x7f6b58009bb0 comp rx=0 tx=0).stop 2024-05-14T16:43:46.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.405+0000 7f6b6df93700 1 -- 172.21.15.2:0/937453892 shutdown_connections 2024-05-14T16:43:46.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.405+0000 7f6b6df93700 1 --2- 172.21.15.2:0/937453892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b68108ed0 0x7f6b681092f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:46.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.405+0000 7f6b6df93700 1 -- 172.21.15.2:0/937453892 >> 172.21.15.2:0/937453892 conn(0x7f6b681044b0 msgr2=0x7f6b681068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:43:46.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.405+0000 7f6b6df93700 1 -- 172.21.15.2:0/937453892 shutdown_connections 2024-05-14T16:43:46.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.405+0000 7f6b6df93700 1 -- 172.21.15.2:0/937453892 wait complete. 2024-05-14T16:43:46.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.406+0000 7f6b6df93700 1 Processor -- start 2024-05-14T16:43:46.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.406+0000 7f6b6df93700 1 -- start start 2024-05-14T16:43:46.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.406+0000 7f6b6df93700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b68108ed0 0x7f6b6819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:46.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.406+0000 7f6b6df93700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6810d420 con 0x7f6b68108ed0 2024-05-14T16:43:46.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.407+0000 7f6b677fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b68108ed0 0x7f6b6819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:46.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.407+0000 7f6b677fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b68108ed0 0x7f6b6819ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36058/0 (socket says 172.21.15.2:36058) 2024-05-14T16:43:46.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.407+0000 7f6b677fe700 1 -- 172.21.15.2:0/707856295 learned_addr learned my addr 172.21.15.2:0/707856295 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:43:46.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.407+0000 7f6b677fe700 1 -- 172.21.15.2:0/707856295 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b6819d370 con 0x7f6b68108ed0 2024-05-14T16:43:46.199 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.407+0000 7f6b677fe700 1 --2- 172.21.15.2:0/707856295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b68108ed0 0x7f6b6819ce30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6b58009fb0 tx=0x7f6b58011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:46.200 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.407+0000 7f6b64ff9700 1 -- 172.21.15.2:0/707856295 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b580036a0 con 0x7f6b68108ed0 2024-05-14T16:43:46.200 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.407+0000 7f6b64ff9700 1 -- 172.21.15.2:0/707856295 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6b5801a430 con 0x7f6b68108ed0 2024-05-14T16:43:46.200 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.407+0000 7f6b6df93700 1 -- 172.21.15.2:0/707856295 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b6819d600 con 0x7f6b68108ed0 2024-05-14T16:43:46.200 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.407+0000 7f6b64ff9700 1 -- 172.21.15.2:0/707856295 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b58019550 con 0x7f6b68108ed0 2024-05-14T16:43:46.200 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.407+0000 7f6b6df93700 1 -- 172.21.15.2:0/707856295 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b681a02f0 con 0x7f6b68108ed0 2024-05-14T16:43:46.200 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.408+0000 7f6b64ff9700 1 -- 172.21.15.2:0/707856295 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f6b58019800 con 0x7f6b68108ed0 2024-05-14T16:43:46.200 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.408+0000 7f6b6df93700 1 -- 172.21.15.2:0/707856295 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b68064ba0 con 0x7f6b68108ed0 2024-05-14T16:43:46.200 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.408+0000 7f6b64ff9700 1 --2- 172.21.15.2:0/707856295 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f6b5003e340 0x7f6b50040800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:46.200 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.409+0000 7f6b64ff9700 1 -- 172.21.15.2:0/707856295 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6b58053790 con 0x7f6b68108ed0 2024-05-14T16:43:46.200 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.409+0000 7f6b66ffd700 1 --2- 172.21.15.2:0/707856295 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f6b5003e340 0x7f6b50040800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:46.200 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.409+0000 7f6b66ffd700 1 --2- 172.21.15.2:0/707856295 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f6b5003e340 0x7f6b50040800 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c006fd0 tx=0x7f6b5c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:46.200 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.414+0000 7f6b64ff9700 1 -- 172.21.15.2:0/707856295 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6b58018d00 con 0x7f6b68108ed0 2024-05-14T16:43:46.200 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.460+0000 7f6b64ff9700 1 -- 172.21.15.2:0/707856295 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f6b5802d440 con 0x7f6b68108ed0 2024-05-14T16:43:46.200 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.644+0000 7f6b6df93700 1 -- 172.21.15.2:0/707856295 --> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f6b681056d0 con 0x7f6b5003e340 2024-05-14T16:43:46.201 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.649+0000 7f6b64ff9700 1 -- 172.21.15.2:0/707856295 <== mgr.14118 v2:172.21.15.2:6800/3582988344 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6b681056d0 con 0x7f6b5003e340 2024-05-14T16:43:46.201 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.651+0000 7f6b6df93700 1 -- 172.21.15.2:0/707856295 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f6b5003e340 msgr2=0x7f6b50040800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:46.201 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.652+0000 7f6b6df93700 1 --2- 172.21.15.2:0/707856295 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f6b5003e340 0x7f6b50040800 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c006fd0 tx=0x7f6b5c006e40 comp rx=0 tx=0).stop 2024-05-14T16:43:46.201 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.652+0000 7f6b6df93700 1 -- 172.21.15.2:0/707856295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b68108ed0 msgr2=0x7f6b6819ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:46.201 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.652+0000 7f6b6df93700 1 --2- 172.21.15.2:0/707856295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b68108ed0 0x7f6b6819ce30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6b58009fb0 tx=0x7f6b58011940 comp rx=0 tx=0).stop 2024-05-14T16:43:46.201 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.652+0000 7f6b6df93700 1 -- 172.21.15.2:0/707856295 shutdown_connections 2024-05-14T16:43:46.201 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.652+0000 7f6b6df93700 1 --2- 172.21.15.2:0/707856295 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f6b5003e340 0x7f6b50040800 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:46.201 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.652+0000 7f6b6df93700 1 --2- 172.21.15.2:0/707856295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6b68108ed0 0x7f6b6819ce30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:46.201 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.652+0000 7f6b6df93700 1 -- 172.21.15.2:0/707856295 >> 172.21.15.2:0/707856295 conn(0x7f6b681044b0 msgr2=0x7f6b68104fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:43:46.201 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.652+0000 7f6b6df93700 1 -- 172.21.15.2:0/707856295 shutdown_connections 2024-05-14T16:43:46.201 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:45.652+0000 7f6b6df93700 1 -- 172.21.15.2:0/707856295 wait complete. 2024-05-14T16:43:46.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:46 smithi002 ceph-mon[31538]: [14/May/2024:16:43:45] ENGINE Serving on https://172.21.15.2:7150 2024-05-14T16:43:46.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:46 smithi002 ceph-mon[31538]: [14/May/2024:16:43:45] ENGINE Serving on http://172.21.15.2:8765 2024-05-14T16:43:46.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:46 smithi002 ceph-mon[31538]: [14/May/2024:16:43:45] ENGINE Bus STARTED 2024-05-14T16:43:46.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:46 smithi002 ceph-mon[31538]: mgrmap e8: x(active, since 2s) 2024-05-14T16:43:46.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:46 smithi002 ceph-mon[31538]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:43:46.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:46 smithi002 ceph-mon[31538]: from='mgr.14118 172.21.15.2:0/1997519028' entity='mgr.x' 2024-05-14T16:43:46.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:46 smithi002 ceph-mon[31538]: from='mgr.14118 172.21.15.2:0/1997519028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:43:47.403 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-14T16:43:47.403 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.636+0000 7fb0d23d7700 1 Processor -- start 2024-05-14T16:43:47.403 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.637+0000 7fb0d23d7700 1 -- start start 2024-05-14T16:43:47.403 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.637+0000 7fb0d23d7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0cc108e60 0x7fb0cc109280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:47.403 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.637+0000 7fb0d23d7700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb0cc1097c0 con 0x7fb0cc108e60 2024-05-14T16:43:47.403 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.637+0000 7fb0cbfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0cc108e60 0x7fb0cc109280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:47.403 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.637+0000 7fb0cbfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0cc108e60 0x7fb0cc109280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36072/0 (socket says 172.21.15.2:36072) 2024-05-14T16:43:47.403 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.637+0000 7fb0cbfff700 1 -- 172.21.15.2:0/3611550994 learned_addr learned my addr 172.21.15.2:0/3611550994 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:43:47.403 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.637+0000 7fb0cbfff700 1 -- 172.21.15.2:0/3611550994 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0cc10a050 con 0x7fb0cc108e60 2024-05-14T16:43:47.404 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.637+0000 7fb0cbfff700 1 --2- 172.21.15.2:0/3611550994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0cc108e60 0x7fb0cc109280 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc0098a0 tx=0x7fb0bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a5cb0ed8d571708 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:47.404 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.638+0000 7fb0cb7fe700 1 -- 172.21.15.2:0/3611550994 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0bc004160 con 0x7fb0cc108e60 2024-05-14T16:43:47.404 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.638+0000 7fb0cb7fe700 1 -- 172.21.15.2:0/3611550994 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb0bc004300 con 0x7fb0cc108e60 2024-05-14T16:43:47.404 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.638+0000 7fb0cb7fe700 1 -- 172.21.15.2:0/3611550994 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0bc011af0 con 0x7fb0cc108e60 2024-05-14T16:43:47.404 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.638+0000 7fb0d23d7700 1 -- 172.21.15.2:0/3611550994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0cc108e60 msgr2=0x7fb0cc109280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:47.404 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.638+0000 7fb0d23d7700 1 --2- 172.21.15.2:0/3611550994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0cc108e60 0x7fb0cc109280 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc0098a0 tx=0x7fb0bc009bb0 comp rx=0 tx=0).stop 2024-05-14T16:43:47.404 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.638+0000 7fb0d23d7700 1 -- 172.21.15.2:0/3611550994 shutdown_connections 2024-05-14T16:43:47.404 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.638+0000 7fb0d23d7700 1 --2- 172.21.15.2:0/3611550994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0cc108e60 0x7fb0cc109280 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:47.404 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.638+0000 7fb0d23d7700 1 -- 172.21.15.2:0/3611550994 >> 172.21.15.2:0/3611550994 conn(0x7fb0cc104450 msgr2=0x7fb0cc106890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:43:47.404 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.638+0000 7fb0d23d7700 1 -- 172.21.15.2:0/3611550994 shutdown_connections 2024-05-14T16:43:47.404 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.638+0000 7fb0d23d7700 1 -- 172.21.15.2:0/3611550994 wait complete. 2024-05-14T16:43:47.404 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.639+0000 7fb0d23d7700 1 Processor -- start 2024-05-14T16:43:47.404 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.639+0000 7fb0d23d7700 1 -- start start 2024-05-14T16:43:47.404 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.639+0000 7fb0d23d7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0cc108e60 0x7fb0cc19ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:47.405 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.639+0000 7fb0d23d7700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb0cc10d310 con 0x7fb0cc108e60 2024-05-14T16:43:47.405 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.639+0000 7fb0cbfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0cc108e60 0x7fb0cc19ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:47.405 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.639+0000 7fb0cbfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0cc108e60 0x7fb0cc19ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36088/0 (socket says 172.21.15.2:36088) 2024-05-14T16:43:47.405 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.639+0000 7fb0cbfff700 1 -- 172.21.15.2:0/251648741 learned_addr learned my addr 172.21.15.2:0/251648741 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:43:47.405 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.640+0000 7fb0cbfff700 1 -- 172.21.15.2:0/251648741 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0cc19d3d0 con 0x7fb0cc108e60 2024-05-14T16:43:47.405 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.640+0000 7fb0cbfff700 1 --2- 172.21.15.2:0/251648741 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0cc108e60 0x7fb0cc19ce90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc0117b0 tx=0x7fb0bc011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:47.405 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.640+0000 7fb0c9ffb700 1 -- 172.21.15.2:0/251648741 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0bc003af0 con 0x7fb0cc108e60 2024-05-14T16:43:47.405 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.640+0000 7fb0c9ffb700 1 -- 172.21.15.2:0/251648741 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb0bc020070 con 0x7fb0cc108e60 2024-05-14T16:43:47.405 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.640+0000 7fb0d23d7700 1 -- 172.21.15.2:0/251648741 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0cc19d660 con 0x7fb0cc108e60 2024-05-14T16:43:47.405 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.640+0000 7fb0c9ffb700 1 -- 172.21.15.2:0/251648741 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0bc019630 con 0x7fb0cc108e60 2024-05-14T16:43:47.405 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.640+0000 7fb0d23d7700 1 -- 172.21.15.2:0/251648741 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0cc1a0350 con 0x7fb0cc108e60 2024-05-14T16:43:47.405 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.641+0000 7fb0d23d7700 1 -- 172.21.15.2:0/251648741 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0cc06a650 con 0x7fb0cc108e60 2024-05-14T16:43:47.405 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.641+0000 7fb0c9ffb700 1 -- 172.21.15.2:0/251648741 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fb0bc01a430 con 0x7fb0cc108e60 2024-05-14T16:43:47.406 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.641+0000 7fb0c9ffb700 1 --2- 172.21.15.2:0/251648741 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7fb0b803e470 0x7fb0b8040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:47.406 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.641+0000 7fb0c9ffb700 1 -- 172.21.15.2:0/251648741 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb0bc053ae0 con 0x7fb0cc108e60 2024-05-14T16:43:47.406 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.642+0000 7fb0c3fff700 1 --2- 172.21.15.2:0/251648741 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7fb0b803e470 0x7fb0b8040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:47.406 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.642+0000 7fb0c3fff700 1 --2- 172.21.15.2:0/251648741 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7fb0b803e470 0x7fb0b8040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4006fd0 tx=0x7fb0b4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:47.406 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.646+0000 7fb0c9ffb700 1 -- 172.21.15.2:0/251648741 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb0bc02b260 con 0x7fb0cc108e60 2024-05-14T16:43:47.406 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.865+0000 7fb0d23d7700 1 -- 172.21.15.2:0/251648741 --> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fb0cc105740 con 0x7fb0b803e470 2024-05-14T16:43:47.406 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.866+0000 7fb0c9ffb700 1 -- 172.21.15.2:0/251648741 <== mgr.14118 v2:172.21.15.2:6800/3582988344 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fb0cc105740 con 0x7fb0b803e470 2024-05-14T16:43:47.406 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.868+0000 7fb0d23d7700 1 -- 172.21.15.2:0/251648741 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7fb0b803e470 msgr2=0x7fb0b8040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:47.406 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.868+0000 7fb0d23d7700 1 --2- 172.21.15.2:0/251648741 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7fb0b803e470 0x7fb0b8040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4006fd0 tx=0x7fb0b4006e40 comp rx=0 tx=0).stop 2024-05-14T16:43:47.406 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.868+0000 7fb0d23d7700 1 -- 172.21.15.2:0/251648741 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0cc108e60 msgr2=0x7fb0cc19ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:47.406 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.868+0000 7fb0d23d7700 1 --2- 172.21.15.2:0/251648741 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0cc108e60 0x7fb0cc19ce90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc0117b0 tx=0x7fb0bc011b70 comp rx=0 tx=0).stop 2024-05-14T16:43:47.406 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.868+0000 7fb0d23d7700 1 -- 172.21.15.2:0/251648741 shutdown_connections 2024-05-14T16:43:47.407 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.868+0000 7fb0d23d7700 1 --2- 172.21.15.2:0/251648741 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7fb0b803e470 0x7fb0b8040930 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:47.407 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.868+0000 7fb0d23d7700 1 --2- 172.21.15.2:0/251648741 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0cc108e60 0x7fb0cc19ce90 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:47.407 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.868+0000 7fb0d23d7700 1 -- 172.21.15.2:0/251648741 >> 172.21.15.2:0/251648741 conn(0x7fb0cc104450 msgr2=0x7fb0cc105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:43:47.407 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.868+0000 7fb0d23d7700 1 -- 172.21.15.2:0/251648741 shutdown_connections 2024-05-14T16:43:47.407 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:46.868+0000 7fb0d23d7700 1 -- 172.21.15.2:0/251648741 wait complete. 2024-05-14T16:43:47.407 INFO:teuthology.orchestra.run.smithi002.stdout:Generating ssh key... 2024-05-14T16:43:47.724 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:47 smithi002 ceph-mon[31538]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:43:48.850 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:48 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: Generating public/private rsa key pair. 2024-05-14T16:43:48.851 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:48 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: Your identification has been saved in /tmp/tmpigpss1__/key. 2024-05-14T16:43:48.851 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:48 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: Your public key has been saved in /tmp/tmpigpss1__/key.pub. 2024-05-14T16:43:48.851 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:48 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: The key fingerprint is: 2024-05-14T16:43:48.851 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:48 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: SHA256:SSvq4XgmK2PrzLRP4ZS47Wh7CL7dleC5T9IJAJJzWeY ceph-ed40717a-1210-11ef-bc98-c7b262605968 2024-05-14T16:43:48.851 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:48 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: The key's randomart image is: 2024-05-14T16:43:48.851 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:48 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: +---[RSA 3072]----+ 2024-05-14T16:43:48.851 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:48 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: |.o oo | 2024-05-14T16:43:48.851 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:48 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: |+ +o | 2024-05-14T16:43:48.851 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:48 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: | o .E . | 2024-05-14T16:43:48.851 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:48 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: | . o . o | 2024-05-14T16:43:48.851 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:48 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: | . + o. S | 2024-05-14T16:43:48.851 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:48 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: |. = o.=.o | 2024-05-14T16:43:48.852 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:48 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: |oo.+o+ * | 2024-05-14T16:43:48.852 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:48 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: |*=B=+.= | 2024-05-14T16:43:48.852 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:48 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: |+%OOoo.. | 2024-05-14T16:43:48.852 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:43:48 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: +----[SHA256]-----+ 2024-05-14T16:43:49.152 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:47.985+0000 7fb04fff2700 1 Processor -- start 2024-05-14T16:43:49.152 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:47.985+0000 7fb04fff2700 1 -- start start 2024-05-14T16:43:49.152 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:47.986+0000 7fb04fff2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb048107de0 0x7fb048108200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:49.152 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:47.986+0000 7fb04fff2700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb0481087d0 con 0x7fb048107de0 2024-05-14T16:43:49.152 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:47.986+0000 7fb04dd8e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb048107de0 0x7fb048108200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:49.152 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:47.986+0000 7fb04dd8e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb048107de0 0x7fb048108200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36100/0 (socket says 172.21.15.2:36100) 2024-05-14T16:43:49.153 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:47.986+0000 7fb04dd8e700 1 -- 172.21.15.2:0/433050660 learned_addr learned my addr 172.21.15.2:0/433050660 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:43:49.153 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:47.986+0000 7fb04dd8e700 1 -- 172.21.15.2:0/433050660 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb048108fe0 con 0x7fb048107de0 2024-05-14T16:43:49.153 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:47.986+0000 7fb04dd8e700 1 --2- 172.21.15.2:0/433050660 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb048107de0 0x7fb048108200 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb0440098a0 tx=0x7fb044009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6fdac8b34a28f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:49.153 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:47.986+0000 7fb04cd8c700 1 -- 172.21.15.2:0/433050660 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb044004160 con 0x7fb048107de0 2024-05-14T16:43:49.153 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:47.987+0000 7fb04cd8c700 1 -- 172.21.15.2:0/433050660 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb044004300 con 0x7fb048107de0 2024-05-14T16:43:49.153 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:47.987+0000 7fb04cd8c700 1 -- 172.21.15.2:0/433050660 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb044011af0 con 0x7fb048107de0 2024-05-14T16:43:49.153 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:47.987+0000 7fb04fff2700 1 -- 172.21.15.2:0/433050660 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb048107de0 msgr2=0x7fb048108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:49.153 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:47.987+0000 7fb04fff2700 1 --2- 172.21.15.2:0/433050660 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb048107de0 0x7fb048108200 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb0440098a0 tx=0x7fb044009bb0 comp rx=0 tx=0).stop 2024-05-14T16:43:49.153 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:47.987+0000 7fb04fff2700 1 -- 172.21.15.2:0/433050660 shutdown_connections 2024-05-14T16:43:49.153 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:47.987+0000 7fb04fff2700 1 --2- 172.21.15.2:0/433050660 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb048107de0 0x7fb048108200 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:49.153 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:47.987+0000 7fb04fff2700 1 -- 172.21.15.2:0/433050660 >> 172.21.15.2:0/433050660 conn(0x7fb0481033a0 msgr2=0x7fb0481057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:43:49.153 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:47.987+0000 7fb04fff2700 1 -- 172.21.15.2:0/433050660 shutdown_connections 2024-05-14T16:43:49.154 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:47.987+0000 7fb04fff2700 1 -- 172.21.15.2:0/433050660 wait complete. 2024-05-14T16:43:49.154 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:47.988+0000 7fb04fff2700 1 Processor -- start 2024-05-14T16:43:49.154 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:47.988+0000 7fb04fff2700 1 -- start start 2024-05-14T16:43:49.154 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:47.988+0000 7fb04fff2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb048107de0 0x7fb048192f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:49.154 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:47.988+0000 7fb04fff2700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb04810c390 con 0x7fb048107de0 2024-05-14T16:43:49.154 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:47.988+0000 7fb04dd8e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb048107de0 0x7fb048192f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:49.154 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:47.988+0000 7fb04dd8e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb048107de0 0x7fb048192f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36102/0 (socket says 172.21.15.2:36102) 2024-05-14T16:43:49.154 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:47.988+0000 7fb04dd8e700 1 -- 172.21.15.2:0/401224112 learned_addr learned my addr 172.21.15.2:0/401224112 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:43:49.154 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:47.989+0000 7fb04dd8e700 1 -- 172.21.15.2:0/401224112 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb048193490 con 0x7fb048107de0 2024-05-14T16:43:49.154 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:47.989+0000 7fb04dd8e700 1 --2- 172.21.15.2:0/401224112 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb048107de0 0x7fb048192f50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb0440117b0 tx=0x7fb044011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:49.155 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:47.989+0000 7fb03effd700 1 -- 172.21.15.2:0/401224112 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb044003af0 con 0x7fb048107de0 2024-05-14T16:43:49.155 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:47.989+0000 7fb03effd700 1 -- 172.21.15.2:0/401224112 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb044020070 con 0x7fb048107de0 2024-05-14T16:43:49.155 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:47.989+0000 7fb04fff2700 1 -- 172.21.15.2:0/401224112 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb048191680 con 0x7fb048107de0 2024-05-14T16:43:49.155 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:47.989+0000 7fb03effd700 1 -- 172.21.15.2:0/401224112 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb044019630 con 0x7fb048107de0 2024-05-14T16:43:49.155 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:47.989+0000 7fb04fff2700 1 -- 172.21.15.2:0/401224112 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb048191bc0 con 0x7fb048107de0 2024-05-14T16:43:49.155 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:47.990+0000 7fb03effd700 1 -- 172.21.15.2:0/401224112 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fb04401a9c0 con 0x7fb048107de0 2024-05-14T16:43:49.155 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:47.990+0000 7fb04fff2700 1 -- 172.21.15.2:0/401224112 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb048064ba0 con 0x7fb048107de0 2024-05-14T16:43:49.155 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:47.990+0000 7fb03effd700 1 --2- 172.21.15.2:0/401224112 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7fb03403e470 0x7fb034040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:49.155 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:47.990+0000 7fb03effd700 1 -- 172.21.15.2:0/401224112 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb044053ae0 con 0x7fb048107de0 2024-05-14T16:43:49.155 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:47.990+0000 7fb04d58d700 1 --2- 172.21.15.2:0/401224112 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7fb03403e470 0x7fb034040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:49.155 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:47.991+0000 7fb04d58d700 1 --2- 172.21.15.2:0/401224112 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7fb03403e470 0x7fb034040930 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb038006fd0 tx=0x7fb038006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:49.156 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:47.995+0000 7fb03effd700 1 -- 172.21.15.2:0/401224112 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb0440172b0 con 0x7fb048107de0 2024-05-14T16:43:49.156 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:48.215+0000 7fb04fff2700 1 -- 172.21.15.2:0/401224112 --> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fb048196d60 con 0x7fb03403e470 2024-05-14T16:43:49.156 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:48.546+0000 7fb03effd700 1 -- 172.21.15.2:0/401224112 <== mgr.14118 v2:172.21.15.2:6800/3582988344 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb048196d60 con 0x7fb03403e470 2024-05-14T16:43:49.156 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:48.548+0000 7fb04fff2700 1 -- 172.21.15.2:0/401224112 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7fb03403e470 msgr2=0x7fb034040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:49.156 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:48.548+0000 7fb04fff2700 1 --2- 172.21.15.2:0/401224112 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7fb03403e470 0x7fb034040930 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb038006fd0 tx=0x7fb038006e40 comp rx=0 tx=0).stop 2024-05-14T16:43:49.156 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:48.548+0000 7fb04fff2700 1 -- 172.21.15.2:0/401224112 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb048107de0 msgr2=0x7fb048192f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:49.156 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:48.548+0000 7fb04fff2700 1 --2- 172.21.15.2:0/401224112 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb048107de0 0x7fb048192f50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb0440117b0 tx=0x7fb044011b70 comp rx=0 tx=0).stop 2024-05-14T16:43:49.156 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:48.549+0000 7fb04fff2700 1 -- 172.21.15.2:0/401224112 shutdown_connections 2024-05-14T16:43:49.156 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:48.549+0000 7fb04fff2700 1 --2- 172.21.15.2:0/401224112 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7fb03403e470 0x7fb034040930 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:49.156 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:48.549+0000 7fb04fff2700 1 --2- 172.21.15.2:0/401224112 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb048107de0 0x7fb048192f50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:49.157 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:48.549+0000 7fb04fff2700 1 -- 172.21.15.2:0/401224112 >> 172.21.15.2:0/401224112 conn(0x7fb0481033a0 msgr2=0x7fb048104270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:43:49.157 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:48.549+0000 7fb04fff2700 1 -- 172.21.15.2:0/401224112 shutdown_connections 2024-05-14T16:43:49.157 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:48.549+0000 7fb04fff2700 1 -- 172.21.15.2:0/401224112 wait complete. 2024-05-14T16:43:49.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:49 smithi002 ceph-mon[31538]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:43:49.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:49 smithi002 ceph-mon[31538]: Generating ssh key... 2024-05-14T16:43:49.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:49 smithi002 ceph-mon[31538]: from='mgr.14118 172.21.15.2:0/1997519028' entity='mgr.x' 2024-05-14T16:43:49.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:49 smithi002 ceph-mon[31538]: from='mgr.14118 172.21.15.2:0/1997519028' entity='mgr.x' 2024-05-14T16:43:50.407 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCkup2e5TEAzUYYjWhK4e7NNumGHuwF2oCFL60Gfoo978sGM8XvjjTTNuT5etFgmr6VR95GKw2dADmE8orO+i3TTWaVXfvZVtu2/QZNi16B5nog286LdDWZnXt6qjYifDmnHrprGST6xrw+i0Cp5KmQ6+X5bYiQqRWdVrd8O1m+UWpat9WoF3oLj9YfCfklA1aD1kcsWdZDZVj8NcbVQBoHDZV+0ji5ljkWv6poAfvw0Z/tUnxKKBxxldsQbhSQltLKJ2uxKby+9hQJG9J6H9ofPUfkCk/3teiSzHystFKVegeMknYc8WOzVdBLoqOc8at6sO1OMRhRYfmn1KvFues54085hL742cr0zBEV9ygTX7y/vFnJwZRYRV4s+iA2M/mPLI/Ck2nxWqbLcnDqsc1uGtGGt0EBAdcaxTcnrYIhKHcD3zulwOK3daxJMut13h0O45yKWHXMuea5TWky2YKzUAJXUia6zq3X5l3iYqAv3w05WV0gmqO/lrf2r3OeRXc= ceph-ed40717a-1210-11ef-bc98-c7b262605968 2024-05-14T16:43:50.407 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.592+0000 7f42ce16c700 1 Processor -- start 2024-05-14T16:43:50.407 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.593+0000 7f42ce16c700 1 -- start start 2024-05-14T16:43:50.407 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.593+0000 7f42ce16c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42c8108eb0 0x7f42c81092d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:50.407 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.593+0000 7f42ce16c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f42c8109810 con 0x7f42c8108eb0 2024-05-14T16:43:50.407 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.593+0000 7f42c77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42c8108eb0 0x7f42c81092d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:50.408 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.593+0000 7f42c77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42c8108eb0 0x7f42c81092d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36108/0 (socket says 172.21.15.2:36108) 2024-05-14T16:43:50.408 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.593+0000 7f42c77fe700 1 -- 172.21.15.2:0/3463179936 learned_addr learned my addr 172.21.15.2:0/3463179936 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:43:50.408 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.593+0000 7f42c77fe700 1 -- 172.21.15.2:0/3463179936 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c810a050 con 0x7f42c8108eb0 2024-05-14T16:43:50.408 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.593+0000 7f42c77fe700 1 --2- 172.21.15.2:0/3463179936 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42c8108eb0 0x7f42c81092d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f42b00098a0 tx=0x7f42b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b1dea54a5909fa2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:50.408 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.594+0000 7f42c67fc700 1 -- 172.21.15.2:0/3463179936 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42b0004160 con 0x7f42c8108eb0 2024-05-14T16:43:50.408 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.594+0000 7f42c67fc700 1 -- 172.21.15.2:0/3463179936 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f42b0004300 con 0x7f42c8108eb0 2024-05-14T16:43:50.408 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.594+0000 7f42c67fc700 1 -- 172.21.15.2:0/3463179936 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42b0011af0 con 0x7f42c8108eb0 2024-05-14T16:43:50.408 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.594+0000 7f42ce16c700 1 -- 172.21.15.2:0/3463179936 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42c8108eb0 msgr2=0x7f42c81092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:50.408 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.594+0000 7f42ce16c700 1 --2- 172.21.15.2:0/3463179936 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42c8108eb0 0x7f42c81092d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f42b00098a0 tx=0x7f42b0009bb0 comp rx=0 tx=0).stop 2024-05-14T16:43:50.408 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.594+0000 7f42ce16c700 1 -- 172.21.15.2:0/3463179936 shutdown_connections 2024-05-14T16:43:50.408 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.594+0000 7f42ce16c700 1 --2- 172.21.15.2:0/3463179936 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42c8108eb0 0x7f42c81092d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:50.408 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.594+0000 7f42ce16c700 1 -- 172.21.15.2:0/3463179936 >> 172.21.15.2:0/3463179936 conn(0x7f42c8104500 msgr2=0x7f42c8106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:43:50.408 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.594+0000 7f42ce16c700 1 -- 172.21.15.2:0/3463179936 shutdown_connections 2024-05-14T16:43:50.408 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.594+0000 7f42ce16c700 1 -- 172.21.15.2:0/3463179936 wait complete. 2024-05-14T16:43:50.409 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.595+0000 7f42ce16c700 1 Processor -- start 2024-05-14T16:43:50.409 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.595+0000 7f42ce16c700 1 -- start start 2024-05-14T16:43:50.409 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.595+0000 7f42ce16c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42c8108eb0 0x7f42c807f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:50.409 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.595+0000 7f42ce16c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f42c810d310 con 0x7f42c8108eb0 2024-05-14T16:43:50.409 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.596+0000 7f42c77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42c8108eb0 0x7f42c807f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:50.409 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.596+0000 7f42c77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42c8108eb0 0x7f42c807f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36122/0 (socket says 172.21.15.2:36122) 2024-05-14T16:43:50.409 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.596+0000 7f42c77fe700 1 -- 172.21.15.2:0/2252701786 learned_addr learned my addr 172.21.15.2:0/2252701786 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:43:50.409 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.596+0000 7f42c77fe700 1 -- 172.21.15.2:0/2252701786 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c807f8c0 con 0x7f42c8108eb0 2024-05-14T16:43:50.409 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.596+0000 7f42c77fe700 1 --2- 172.21.15.2:0/2252701786 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42c8108eb0 0x7f42c807f380 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f42b0009fb0 tx=0x7f42b0011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:50.409 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.596+0000 7f42c4ff9700 1 -- 172.21.15.2:0/2252701786 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42b00036a0 con 0x7f42c8108eb0 2024-05-14T16:43:50.409 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.596+0000 7f42c4ff9700 1 -- 172.21.15.2:0/2252701786 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f42b001a430 con 0x7f42c8108eb0 2024-05-14T16:43:50.409 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.596+0000 7f42c4ff9700 1 -- 172.21.15.2:0/2252701786 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42b0019550 con 0x7f42c8108eb0 2024-05-14T16:43:50.409 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.596+0000 7f42ce16c700 1 -- 172.21.15.2:0/2252701786 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42c807dab0 con 0x7f42c8108eb0 2024-05-14T16:43:50.409 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.596+0000 7f42ce16c700 1 -- 172.21.15.2:0/2252701786 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42c807dff0 con 0x7f42c8108eb0 2024-05-14T16:43:50.410 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.597+0000 7f42c4ff9700 1 -- 172.21.15.2:0/2252701786 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f42b001a8e0 con 0x7f42c8108eb0 2024-05-14T16:43:50.410 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.597+0000 7f42ce16c700 1 -- 172.21.15.2:0/2252701786 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42c8064ba0 con 0x7f42c8108eb0 2024-05-14T16:43:50.410 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.598+0000 7f42c4ff9700 1 --2- 172.21.15.2:0/2252701786 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f42b403e470 0x7f42b4040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:50.410 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.598+0000 7f42c4ff9700 1 -- 172.21.15.2:0/2252701786 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f42b0053810 con 0x7f42c8108eb0 2024-05-14T16:43:50.410 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.598+0000 7f42c6ffd700 1 --2- 172.21.15.2:0/2252701786 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f42b403e470 0x7f42b4040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:50.410 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.598+0000 7f42c6ffd700 1 --2- 172.21.15.2:0/2252701786 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f42b403e470 0x7f42b4040930 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f42b8006fd0 tx=0x7f42b8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:50.410 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.603+0000 7f42c4ff9700 1 -- 172.21.15.2:0/2252701786 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f42b000b3e0 con 0x7f42c8108eb0 2024-05-14T16:43:50.410 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.821+0000 7f42ce16c700 1 -- 172.21.15.2:0/2252701786 --> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f42c81057a0 con 0x7f42b403e470 2024-05-14T16:43:50.410 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.822+0000 7f42c4ff9700 1 -- 172.21.15.2:0/2252701786 <== mgr.14118 v2:172.21.15.2:6800/3582988344 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f42c81057a0 con 0x7f42b403e470 2024-05-14T16:43:50.410 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.824+0000 7f42ce16c700 1 -- 172.21.15.2:0/2252701786 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f42b403e470 msgr2=0x7f42b4040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:50.410 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.824+0000 7f42ce16c700 1 --2- 172.21.15.2:0/2252701786 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f42b403e470 0x7f42b4040930 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f42b8006fd0 tx=0x7f42b8006e40 comp rx=0 tx=0).stop 2024-05-14T16:43:50.410 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.824+0000 7f42ce16c700 1 -- 172.21.15.2:0/2252701786 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42c8108eb0 msgr2=0x7f42c807f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:50.410 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.824+0000 7f42ce16c700 1 --2- 172.21.15.2:0/2252701786 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42c8108eb0 0x7f42c807f380 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f42b0009fb0 tx=0x7f42b0011940 comp rx=0 tx=0).stop 2024-05-14T16:43:50.410 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.824+0000 7f42ce16c700 1 -- 172.21.15.2:0/2252701786 shutdown_connections 2024-05-14T16:43:50.411 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.824+0000 7f42ce16c700 1 --2- 172.21.15.2:0/2252701786 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f42b403e470 0x7f42b4040930 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:50.411 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.824+0000 7f42ce16c700 1 --2- 172.21.15.2:0/2252701786 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42c8108eb0 0x7f42c807f380 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:50.411 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.824+0000 7f42ce16c700 1 -- 172.21.15.2:0/2252701786 >> 172.21.15.2:0/2252701786 conn(0x7f42c8104500 msgr2=0x7f42c8105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:43:50.411 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.824+0000 7f42ce16c700 1 -- 172.21.15.2:0/2252701786 shutdown_connections 2024-05-14T16:43:50.411 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:49.824+0000 7f42ce16c700 1 -- 172.21.15.2:0/2252701786 wait complete. 2024-05-14T16:43:50.411 INFO:teuthology.orchestra.run.smithi002.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-14T16:43:50.411 INFO:teuthology.orchestra.run.smithi002.stdout:Adding key to root@localhost authorized_keys... 2024-05-14T16:43:50.411 INFO:teuthology.orchestra.run.smithi002.stdout:Adding host smithi002... 2024-05-14T16:43:50.651 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:50 smithi002 ceph-mon[31538]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:43:51.704 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:51 smithi002 ceph-mon[31538]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi002", "addr": "172.21.15.2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:43:52.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:52 smithi002 ceph-mon[31538]: Deploying cephadm binary to smithi002 2024-05-14T16:43:55.972 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout Added host 'smithi002' with addr '172.21.15.2' 2024-05-14T16:43:55.973 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:50.830+0000 7f5c105f5700 1 Processor -- start 2024-05-14T16:43:55.973 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:50.830+0000 7f5c105f5700 1 -- start start 2024-05-14T16:43:55.973 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:50.830+0000 7f5c105f5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c08108eb0 0x7f5c081092d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:55.973 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:50.830+0000 7f5c105f5700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5c08109810 con 0x7f5c08108eb0 2024-05-14T16:43:55.973 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:50.830+0000 7f5c0e391700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c08108eb0 0x7f5c081092d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:55.973 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:50.830+0000 7f5c0e391700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c08108eb0 0x7f5c081092d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36136/0 (socket says 172.21.15.2:36136) 2024-05-14T16:43:55.973 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:50.830+0000 7f5c0e391700 1 -- 172.21.15.2:0/3081871031 learned_addr learned my addr 172.21.15.2:0/3081871031 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:43:55.973 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:50.831+0000 7f5c0e391700 1 -- 172.21.15.2:0/3081871031 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c0810a050 con 0x7f5c08108eb0 2024-05-14T16:43:55.973 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:50.831+0000 7f5c0e391700 1 --2- 172.21.15.2:0/3081871031 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c08108eb0 0x7f5c081092d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5c040098a0 tx=0x7f5c04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b61fabb6da090fe7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:55.973 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:50.831+0000 7f5c0d38f700 1 -- 172.21.15.2:0/3081871031 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c04004160 con 0x7f5c08108eb0 2024-05-14T16:43:55.973 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:50.831+0000 7f5c0d38f700 1 -- 172.21.15.2:0/3081871031 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5c04004300 con 0x7f5c08108eb0 2024-05-14T16:43:55.973 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:50.831+0000 7f5c0d38f700 1 -- 172.21.15.2:0/3081871031 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c04011af0 con 0x7f5c08108eb0 2024-05-14T16:43:55.973 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:50.831+0000 7f5c105f5700 1 -- 172.21.15.2:0/3081871031 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c08108eb0 msgr2=0x7f5c081092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:55.974 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:50.831+0000 7f5c105f5700 1 --2- 172.21.15.2:0/3081871031 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c08108eb0 0x7f5c081092d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5c040098a0 tx=0x7f5c04009bb0 comp rx=0 tx=0).stop 2024-05-14T16:43:55.974 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:50.832+0000 7f5c105f5700 1 -- 172.21.15.2:0/3081871031 shutdown_connections 2024-05-14T16:43:55.974 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:50.832+0000 7f5c105f5700 1 --2- 172.21.15.2:0/3081871031 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c08108eb0 0x7f5c081092d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:55.974 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:50.832+0000 7f5c105f5700 1 -- 172.21.15.2:0/3081871031 >> 172.21.15.2:0/3081871031 conn(0x7f5c08104500 msgr2=0x7f5c08106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:43:55.974 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:50.832+0000 7f5c105f5700 1 -- 172.21.15.2:0/3081871031 shutdown_connections 2024-05-14T16:43:55.974 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:50.832+0000 7f5c105f5700 1 -- 172.21.15.2:0/3081871031 wait complete. 2024-05-14T16:43:55.974 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:50.832+0000 7f5c105f5700 1 Processor -- start 2024-05-14T16:43:55.974 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:50.832+0000 7f5c105f5700 1 -- start start 2024-05-14T16:43:55.974 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:50.833+0000 7f5c105f5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c08108eb0 0x7f5c0819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:55.974 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:50.833+0000 7f5c105f5700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0810d310 con 0x7f5c08108eb0 2024-05-14T16:43:55.974 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:50.833+0000 7f5c0e391700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c08108eb0 0x7f5c0819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:55.974 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:50.833+0000 7f5c0e391700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c08108eb0 0x7f5c0819ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36138/0 (socket says 172.21.15.2:36138) 2024-05-14T16:43:55.974 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:50.833+0000 7f5c0e391700 1 -- 172.21.15.2:0/162088356 learned_addr learned my addr 172.21.15.2:0/162088356 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:43:55.975 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:50.833+0000 7f5c0e391700 1 -- 172.21.15.2:0/162088356 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c0819d3b0 con 0x7f5c08108eb0 2024-05-14T16:43:55.975 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:50.833+0000 7f5c0e391700 1 --2- 172.21.15.2:0/162088356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c08108eb0 0x7f5c0819ce70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5c040117b0 tx=0x7f5c04011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:55.975 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:50.833+0000 7f5bff7fe700 1 -- 172.21.15.2:0/162088356 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c04003af0 con 0x7f5c08108eb0 2024-05-14T16:43:55.975 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:50.833+0000 7f5bff7fe700 1 -- 172.21.15.2:0/162088356 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5c04020070 con 0x7f5c08108eb0 2024-05-14T16:43:55.975 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:50.833+0000 7f5c105f5700 1 -- 172.21.15.2:0/162088356 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c0819d640 con 0x7f5c08108eb0 2024-05-14T16:43:55.975 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:50.833+0000 7f5bff7fe700 1 -- 172.21.15.2:0/162088356 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c04019630 con 0x7f5c08108eb0 2024-05-14T16:43:55.975 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:50.833+0000 7f5c105f5700 1 -- 172.21.15.2:0/162088356 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c081a0330 con 0x7f5c08108eb0 2024-05-14T16:43:55.975 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:50.834+0000 7f5bff7fe700 1 -- 172.21.15.2:0/162088356 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f5c0401a9c0 con 0x7f5c08108eb0 2024-05-14T16:43:55.975 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:50.834+0000 7f5c105f5700 1 -- 172.21.15.2:0/162088356 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c08064ba0 con 0x7f5c08108eb0 2024-05-14T16:43:55.975 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:50.835+0000 7f5bff7fe700 1 --2- 172.21.15.2:0/162088356 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f5bf403e4c0 0x7f5bf4040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:55.975 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:50.835+0000 7f5bff7fe700 1 -- 172.21.15.2:0/162088356 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5c04053ac0 con 0x7f5c08108eb0 2024-05-14T16:43:55.975 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:50.835+0000 7f5c0db90700 1 --2- 172.21.15.2:0/162088356 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f5bf403e4c0 0x7f5bf4040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:55.975 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:50.835+0000 7f5c0db90700 1 --2- 172.21.15.2:0/162088356 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f5bf403e4c0 0x7f5bf4040980 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8006fd0 tx=0x7f5bf8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:55.976 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:50.840+0000 7f5bff7fe700 1 -- 172.21.15.2:0/162088356 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5c040172b0 con 0x7f5c08108eb0 2024-05-14T16:43:55.976 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:51.063+0000 7f5c105f5700 1 -- 172.21.15.2:0/162088356 --> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi002", "addr": "172.21.15.2", "target": ["mon-mgr", ""]}) -- 0x7f5c081057a0 con 0x7f5bf403e4c0 2024-05-14T16:43:55.976 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:55.364+0000 7f5bff7fe700 1 -- 172.21.15.2:0/162088356 <== mgr.14118 v2:172.21.15.2:6800/3582988344 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7f5c081057a0 con 0x7f5bf403e4c0 2024-05-14T16:43:55.976 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:55.367+0000 7f5c105f5700 1 -- 172.21.15.2:0/162088356 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f5bf403e4c0 msgr2=0x7f5bf4040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:55.976 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:55.367+0000 7f5c105f5700 1 --2- 172.21.15.2:0/162088356 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f5bf403e4c0 0x7f5bf4040980 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8006fd0 tx=0x7f5bf8006e40 comp rx=0 tx=0).stop 2024-05-14T16:43:55.976 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:55.367+0000 7f5c105f5700 1 -- 172.21.15.2:0/162088356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c08108eb0 msgr2=0x7f5c0819ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:55.976 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:55.367+0000 7f5c105f5700 1 --2- 172.21.15.2:0/162088356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c08108eb0 0x7f5c0819ce70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5c040117b0 tx=0x7f5c04011b70 comp rx=0 tx=0).stop 2024-05-14T16:43:55.976 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:55.367+0000 7f5c105f5700 1 -- 172.21.15.2:0/162088356 shutdown_connections 2024-05-14T16:43:55.976 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:55.367+0000 7f5c105f5700 1 --2- 172.21.15.2:0/162088356 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f5bf403e4c0 0x7f5bf4040980 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:55.976 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:55.367+0000 7f5c105f5700 1 --2- 172.21.15.2:0/162088356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c08108eb0 0x7f5c0819ce70 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:55.976 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:55.367+0000 7f5c105f5700 1 -- 172.21.15.2:0/162088356 >> 172.21.15.2:0/162088356 conn(0x7f5c08104500 msgr2=0x7f5c08105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:43:55.976 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:55.367+0000 7f5c105f5700 1 -- 172.21.15.2:0/162088356 shutdown_connections 2024-05-14T16:43:55.976 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:55.367+0000 7f5c105f5700 1 -- 172.21.15.2:0/162088356 wait complete. 2024-05-14T16:43:55.976 INFO:teuthology.orchestra.run.smithi002.stdout:Deploying unmanaged mon service... 2024-05-14T16:43:56.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:56 smithi002 ceph-mon[31538]: from='mgr.14118 172.21.15.2:0/1997519028' entity='mgr.x' 2024-05-14T16:43:56.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:56 smithi002 ceph-mon[31538]: Added host smithi002 2024-05-14T16:43:56.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:56 smithi002 ceph-mon[31538]: from='mgr.14118 172.21.15.2:0/1997519028' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:43:57.894 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:57 smithi002 ceph-mon[31538]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:43:57.894 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:57 smithi002 ceph-mon[31538]: Saving service mon spec with placement count:5 2024-05-14T16:43:57.894 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:57 smithi002 ceph-mon[31538]: from='mgr.14118 172.21.15.2:0/1997519028' entity='mgr.x' 2024-05-14T16:43:58.455 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-14T16:43:58.455 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.493+0000 7fa783a88700 1 Processor -- start 2024-05-14T16:43:58.455 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.493+0000 7fa783a88700 1 -- start start 2024-05-14T16:43:58.455 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.493+0000 7fa783a88700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa77c07f680 0x7fa77c07db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:58.455 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.493+0000 7fa783a88700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa77c07e100 con 0x7fa77c07f680 2024-05-14T16:43:58.455 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.493+0000 7fa781824700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa77c07f680 0x7fa77c07db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:58.455 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.493+0000 7fa781824700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa77c07f680 0x7fa77c07db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47674/0 (socket says 172.21.15.2:47674) 2024-05-14T16:43:58.455 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.493+0000 7fa781824700 1 -- 172.21.15.2:0/4190341285 learned_addr learned my addr 172.21.15.2:0/4190341285 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:43:58.455 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.494+0000 7fa781824700 1 -- 172.21.15.2:0/4190341285 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa77c07e280 con 0x7fa77c07f680 2024-05-14T16:43:58.456 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.494+0000 7fa781824700 1 --2- 172.21.15.2:0/4190341285 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa77c07f680 0x7fa77c07db30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa7780098a0 tx=0x7fa778009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d340bbb530d18c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:58.456 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.494+0000 7fa780822700 1 -- 172.21.15.2:0/4190341285 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa778004160 con 0x7fa77c07f680 2024-05-14T16:43:58.456 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.494+0000 7fa780822700 1 -- 172.21.15.2:0/4190341285 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa778004300 con 0x7fa77c07f680 2024-05-14T16:43:58.456 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.494+0000 7fa780822700 1 -- 172.21.15.2:0/4190341285 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa778011af0 con 0x7fa77c07f680 2024-05-14T16:43:58.456 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.494+0000 7fa783a88700 1 -- 172.21.15.2:0/4190341285 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa77c07f680 msgr2=0x7fa77c07db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:58.456 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.494+0000 7fa783a88700 1 --2- 172.21.15.2:0/4190341285 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa77c07f680 0x7fa77c07db30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa7780098a0 tx=0x7fa778009bb0 comp rx=0 tx=0).stop 2024-05-14T16:43:58.456 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.494+0000 7fa783a88700 1 -- 172.21.15.2:0/4190341285 shutdown_connections 2024-05-14T16:43:58.456 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.494+0000 7fa783a88700 1 --2- 172.21.15.2:0/4190341285 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa77c07f680 0x7fa77c07db30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:58.456 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.495+0000 7fa783a88700 1 -- 172.21.15.2:0/4190341285 >> 172.21.15.2:0/4190341285 conn(0x7fa77c104500 msgr2=0x7fa77c106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:43:58.456 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.495+0000 7fa783a88700 1 -- 172.21.15.2:0/4190341285 shutdown_connections 2024-05-14T16:43:58.456 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.495+0000 7fa783a88700 1 -- 172.21.15.2:0/4190341285 wait complete. 2024-05-14T16:43:58.456 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.495+0000 7fa783a88700 1 Processor -- start 2024-05-14T16:43:58.456 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.495+0000 7fa783a88700 1 -- start start 2024-05-14T16:43:58.456 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.495+0000 7fa783a88700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa77c07f680 0x7fa77c19f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:58.457 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.495+0000 7fa783a88700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa77c10f430 con 0x7fa77c07f680 2024-05-14T16:43:58.457 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.495+0000 7fa781824700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa77c07f680 0x7fa77c19f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:58.457 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.496+0000 7fa781824700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa77c07f680 0x7fa77c19f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47678/0 (socket says 172.21.15.2:47678) 2024-05-14T16:43:58.457 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.496+0000 7fa781824700 1 -- 172.21.15.2:0/2584542381 learned_addr learned my addr 172.21.15.2:0/2584542381 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:43:58.457 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.496+0000 7fa781824700 1 -- 172.21.15.2:0/2584542381 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa77c19f580 con 0x7fa77c07f680 2024-05-14T16:43:58.457 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.496+0000 7fa781824700 1 --2- 172.21.15.2:0/2584542381 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa77c07f680 0x7fa77c19f040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa778009fb0 tx=0x7fa778011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:58.457 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.496+0000 7fa772ffd700 1 -- 172.21.15.2:0/2584542381 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7780036a0 con 0x7fa77c07f680 2024-05-14T16:43:58.457 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.496+0000 7fa772ffd700 1 -- 172.21.15.2:0/2584542381 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa77801a430 con 0x7fa77c07f680 2024-05-14T16:43:58.457 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.496+0000 7fa772ffd700 1 -- 172.21.15.2:0/2584542381 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa778019550 con 0x7fa77c07f680 2024-05-14T16:43:58.457 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.496+0000 7fa783a88700 1 -- 172.21.15.2:0/2584542381 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa77c19f810 con 0x7fa77c07f680 2024-05-14T16:43:58.457 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.496+0000 7fa783a88700 1 -- 172.21.15.2:0/2584542381 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa77c1a2500 con 0x7fa77c07f680 2024-05-14T16:43:58.457 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.497+0000 7fa772ffd700 1 -- 172.21.15.2:0/2584542381 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fa778019800 con 0x7fa77c07f680 2024-05-14T16:43:58.457 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.497+0000 7fa783a88700 1 -- 172.21.15.2:0/2584542381 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa77c064ba0 con 0x7fa77c07f680 2024-05-14T16:43:58.457 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.497+0000 7fa772ffd700 1 --2- 172.21.15.2:0/2584542381 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7fa76803e470 0x7fa768040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:43:58.457 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.497+0000 7fa772ffd700 1 -- 172.21.15.2:0/2584542381 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa778053840 con 0x7fa77c07f680 2024-05-14T16:43:58.458 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.497+0000 7fa781023700 1 --2- 172.21.15.2:0/2584542381 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7fa76803e470 0x7fa768040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:43:58.458 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.498+0000 7fa781023700 1 --2- 172.21.15.2:0/2584542381 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7fa76803e470 0x7fa768040930 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa76c006fd0 tx=0x7fa76c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:43:58.458 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.502+0000 7fa772ffd700 1 -- 172.21.15.2:0/2584542381 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa778023e00 con 0x7fa77c07f680 2024-05-14T16:43:58.458 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.745+0000 7fa783a88700 1 -- 172.21.15.2:0/2584542381 --> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fa77c105770 con 0x7fa76803e470 2024-05-14T16:43:58.458 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.750+0000 7fa772ffd700 1 -- 172.21.15.2:0/2584542381 <== mgr.14118 v2:172.21.15.2:6800/3582988344 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fa77c105770 con 0x7fa76803e470 2024-05-14T16:43:58.458 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.752+0000 7fa783a88700 1 -- 172.21.15.2:0/2584542381 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7fa76803e470 msgr2=0x7fa768040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:58.458 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.752+0000 7fa783a88700 1 --2- 172.21.15.2:0/2584542381 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7fa76803e470 0x7fa768040930 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa76c006fd0 tx=0x7fa76c006e40 comp rx=0 tx=0).stop 2024-05-14T16:43:58.458 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.752+0000 7fa783a88700 1 -- 172.21.15.2:0/2584542381 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa77c07f680 msgr2=0x7fa77c19f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:43:58.458 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.752+0000 7fa783a88700 1 --2- 172.21.15.2:0/2584542381 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa77c07f680 0x7fa77c19f040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa778009fb0 tx=0x7fa778011940 comp rx=0 tx=0).stop 2024-05-14T16:43:58.458 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.752+0000 7fa783a88700 1 -- 172.21.15.2:0/2584542381 shutdown_connections 2024-05-14T16:43:58.459 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.752+0000 7fa783a88700 1 --2- 172.21.15.2:0/2584542381 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7fa76803e470 0x7fa768040930 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:58.459 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.752+0000 7fa783a88700 1 --2- 172.21.15.2:0/2584542381 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa77c07f680 0x7fa77c19f040 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:43:58.459 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.752+0000 7fa783a88700 1 -- 172.21.15.2:0/2584542381 >> 172.21.15.2:0/2584542381 conn(0x7fa77c104500 msgr2=0x7fa77c105060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:43:58.459 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.752+0000 7fa783a88700 1 -- 172.21.15.2:0/2584542381 shutdown_connections 2024-05-14T16:43:58.459 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:56.752+0000 7fa783a88700 1 -- 172.21.15.2:0/2584542381 wait complete. 2024-05-14T16:43:58.459 INFO:teuthology.orchestra.run.smithi002.stdout:Deploying unmanaged mgr service... 2024-05-14T16:43:58.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:58 smithi002 ceph-mon[31538]: from='mgr.14118 172.21.15.2:0/1997519028' entity='mgr.x' 2024-05-14T16:43:58.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:43:58 smithi002 ceph-mon[31538]: from='mgr.14118 172.21.15.2:0/1997519028' entity='mgr.x' 2024-05-14T16:44:00.332 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:00 smithi002 ceph-mon[31538]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:44:00.332 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:00 smithi002 ceph-mon[31538]: Saving service mgr spec with placement count:2 2024-05-14T16:44:00.332 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:00 smithi002 ceph-mon[31538]: from='mgr.14118 172.21.15.2:0/1997519028' entity='mgr.x' 2024-05-14T16:44:00.332 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:00 smithi002 ceph-mon[31538]: from='mgr.14118 172.21.15.2:0/1997519028' entity='mgr.x' 2024-05-14T16:44:00.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-14T16:44:00.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:58.923+0000 7f7c5642d700 1 Processor -- start 2024-05-14T16:44:00.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:58.923+0000 7f7c5642d700 1 -- start start 2024-05-14T16:44:00.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:58.923+0000 7f7c5642d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c50107790 0x7f7c50107bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:00.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:58.923+0000 7f7c5642d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5007d630 con 0x7f7c50107790 2024-05-14T16:44:00.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:58.923+0000 7f7c4ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c50107790 0x7f7c50107bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:00.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:58.923+0000 7f7c4ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c50107790 0x7f7c50107bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47682/0 (socket says 172.21.15.2:47682) 2024-05-14T16:44:00.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:58.923+0000 7f7c4ffff700 1 -- 172.21.15.2:0/3106871189 learned_addr learned my addr 172.21.15.2:0/3106871189 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:44:00.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:58.924+0000 7f7c4ffff700 1 -- 172.21.15.2:0/3106871189 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c501080f0 con 0x7f7c50107790 2024-05-14T16:44:00.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:58.924+0000 7f7c4ffff700 1 --2- 172.21.15.2:0/3106871189 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c50107790 0x7f7c50107bb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7c400098a0 tx=0x7f7c40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7310b41ccf00a5a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:00.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:58.924+0000 7f7c4f7fe700 1 -- 172.21.15.2:0/3106871189 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c40004160 con 0x7f7c50107790 2024-05-14T16:44:00.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:58.924+0000 7f7c4f7fe700 1 -- 172.21.15.2:0/3106871189 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c40004300 con 0x7f7c50107790 2024-05-14T16:44:00.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:58.924+0000 7f7c4f7fe700 1 -- 172.21.15.2:0/3106871189 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c40011a20 con 0x7f7c50107790 2024-05-14T16:44:00.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:58.925+0000 7f7c5642d700 1 -- 172.21.15.2:0/3106871189 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c50107790 msgr2=0x7f7c50107bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:00.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:58.925+0000 7f7c5642d700 1 --2- 172.21.15.2:0/3106871189 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c50107790 0x7f7c50107bb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7c400098a0 tx=0x7f7c40009bb0 comp rx=0 tx=0).stop 2024-05-14T16:44:00.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:58.925+0000 7f7c5642d700 1 -- 172.21.15.2:0/3106871189 shutdown_connections 2024-05-14T16:44:00.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:58.925+0000 7f7c5642d700 1 --2- 172.21.15.2:0/3106871189 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c50107790 0x7f7c50107bb0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:00.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:58.925+0000 7f7c5642d700 1 -- 172.21.15.2:0/3106871189 >> 172.21.15.2:0/3106871189 conn(0x7f7c501033b0 msgr2=0x7f7c501057d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:00.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:58.925+0000 7f7c5642d700 1 -- 172.21.15.2:0/3106871189 shutdown_connections 2024-05-14T16:44:00.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:58.925+0000 7f7c5642d700 1 -- 172.21.15.2:0/3106871189 wait complete. 2024-05-14T16:44:00.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:58.925+0000 7f7c5642d700 1 Processor -- start 2024-05-14T16:44:00.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:58.925+0000 7f7c5642d700 1 -- start start 2024-05-14T16:44:00.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:58.926+0000 7f7c5642d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c50107790 0x7f7c5019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:00.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:58.926+0000 7f7c5642d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7c50109160 con 0x7f7c50107790 2024-05-14T16:44:00.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:58.926+0000 7f7c4ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c50107790 0x7f7c5019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:00.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:58.926+0000 7f7c4ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c50107790 0x7f7c5019ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47690/0 (socket says 172.21.15.2:47690) 2024-05-14T16:44:00.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:58.926+0000 7f7c4ffff700 1 -- 172.21.15.2:0/1488476365 learned_addr learned my addr 172.21.15.2:0/1488476365 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:44:00.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:58.926+0000 7f7c4ffff700 1 -- 172.21.15.2:0/1488476365 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c5019f4d0 con 0x7f7c50107790 2024-05-14T16:44:00.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:58.926+0000 7f7c4ffff700 1 --2- 172.21.15.2:0/1488476365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c50107790 0x7f7c5019ef90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7c400116e0 tx=0x7f7c40011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:00.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:58.926+0000 7f7c4dffb700 1 -- 172.21.15.2:0/1488476365 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c40003b20 con 0x7f7c50107790 2024-05-14T16:44:00.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:58.926+0000 7f7c4dffb700 1 -- 172.21.15.2:0/1488476365 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c4001b8f0 con 0x7f7c50107790 2024-05-14T16:44:00.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:58.926+0000 7f7c5642d700 1 -- 172.21.15.2:0/1488476365 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c5019f760 con 0x7f7c50107790 2024-05-14T16:44:00.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:58.926+0000 7f7c4dffb700 1 -- 172.21.15.2:0/1488476365 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c400238c0 con 0x7f7c50107790 2024-05-14T16:44:00.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:58.926+0000 7f7c5642d700 1 -- 172.21.15.2:0/1488476365 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c501a2450 con 0x7f7c50107790 2024-05-14T16:44:00.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:58.927+0000 7f7c5642d700 1 -- 172.21.15.2:0/1488476365 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c50064ba0 con 0x7f7c50107790 2024-05-14T16:44:00.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:58.928+0000 7f7c4dffb700 1 -- 172.21.15.2:0/1488476365 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f7c40023a80 con 0x7f7c50107790 2024-05-14T16:44:00.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:58.928+0000 7f7c4dffb700 1 --2- 172.21.15.2:0/1488476365 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f7c3c03e470 0x7f7c3c040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:00.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:58.928+0000 7f7c4dffb700 1 -- 172.21.15.2:0/1488476365 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7c40054090 con 0x7f7c50107790 2024-05-14T16:44:00.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:58.928+0000 7f7c47fff700 1 --2- 172.21.15.2:0/1488476365 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f7c3c03e470 0x7f7c3c040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:00.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:58.929+0000 7f7c47fff700 1 --2- 172.21.15.2:0/1488476365 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f7c3c03e470 0x7f7c3c040930 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7c38006fd0 tx=0x7f7c38006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:00.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:58.933+0000 7f7c4dffb700 1 -- 172.21.15.2:0/1488476365 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7c4001f030 con 0x7f7c50107790 2024-05-14T16:44:00.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:59.196+0000 7f7c5642d700 1 -- 172.21.15.2:0/1488476365 --> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f7c5002e860 con 0x7f7c3c03e470 2024-05-14T16:44:00.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:59.201+0000 7f7c4dffb700 1 -- 172.21.15.2:0/1488476365 <== mgr.14118 v2:172.21.15.2:6800/3582988344 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f7c5002e860 con 0x7f7c3c03e470 2024-05-14T16:44:00.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:59.203+0000 7f7c5642d700 1 -- 172.21.15.2:0/1488476365 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f7c3c03e470 msgr2=0x7f7c3c040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:00.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:59.203+0000 7f7c5642d700 1 --2- 172.21.15.2:0/1488476365 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f7c3c03e470 0x7f7c3c040930 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7c38006fd0 tx=0x7f7c38006e40 comp rx=0 tx=0).stop 2024-05-14T16:44:00.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:59.203+0000 7f7c5642d700 1 -- 172.21.15.2:0/1488476365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c50107790 msgr2=0x7f7c5019ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:00.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:59.203+0000 7f7c5642d700 1 --2- 172.21.15.2:0/1488476365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c50107790 0x7f7c5019ef90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7c400116e0 tx=0x7f7c40011aa0 comp rx=0 tx=0).stop 2024-05-14T16:44:00.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:59.203+0000 7f7c5642d700 1 -- 172.21.15.2:0/1488476365 shutdown_connections 2024-05-14T16:44:00.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:59.203+0000 7f7c5642d700 1 --2- 172.21.15.2:0/1488476365 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f7c3c03e470 0x7f7c3c040930 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:00.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:59.203+0000 7f7c5642d700 1 --2- 172.21.15.2:0/1488476365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c50107790 0x7f7c5019ef90 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:00.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:59.203+0000 7f7c5642d700 1 -- 172.21.15.2:0/1488476365 >> 172.21.15.2:0/1488476365 conn(0x7f7c501033b0 msgr2=0x7f7c50195c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:00.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:59.203+0000 7f7c5642d700 1 -- 172.21.15.2:0/1488476365 shutdown_connections 2024-05-14T16:44:00.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:43:59.203+0000 7f7c5642d700 1 -- 172.21.15.2:0/1488476365 wait complete. 2024-05-14T16:44:01.649 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:00.797+0000 7f9a0d951700 1 Processor -- start 2024-05-14T16:44:01.649 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:00.797+0000 7f9a0d951700 1 -- start start 2024-05-14T16:44:01.649 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:00.797+0000 7f9a0d951700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a081093b0 0x7f9a081097d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:01.650 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:00.797+0000 7f9a0d951700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9a08109da0 con 0x7f9a081093b0 2024-05-14T16:44:01.650 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:00.797+0000 7f9a06ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a081093b0 0x7f9a081097d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:01.650 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:00.797+0000 7f9a06ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a081093b0 0x7f9a081097d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47706/0 (socket says 172.21.15.2:47706) 2024-05-14T16:44:01.650 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:00.797+0000 7f9a06ffd700 1 -- 172.21.15.2:0/2858183008 learned_addr learned my addr 172.21.15.2:0/2858183008 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:44:01.650 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:00.798+0000 7f9a06ffd700 1 -- 172.21.15.2:0/2858183008 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a08108010 con 0x7f9a081093b0 2024-05-14T16:44:01.650 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:00.798+0000 7f9a06ffd700 1 --2- 172.21.15.2:0/2858183008 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a081093b0 0x7f9a081097d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f99f80098a0 tx=0x7f99f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fa185014bc2a638 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:01.650 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:00.798+0000 7f9a05ffb700 1 -- 172.21.15.2:0/2858183008 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99f8004160 con 0x7f9a081093b0 2024-05-14T16:44:01.650 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:00.798+0000 7f9a05ffb700 1 -- 172.21.15.2:0/2858183008 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99f8004300 con 0x7f9a081093b0 2024-05-14T16:44:01.650 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:00.798+0000 7f9a05ffb700 1 -- 172.21.15.2:0/2858183008 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99f8011a20 con 0x7f9a081093b0 2024-05-14T16:44:01.650 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:00.799+0000 7f9a0d951700 1 -- 172.21.15.2:0/2858183008 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a081093b0 msgr2=0x7f9a081097d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:01.651 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:00.799+0000 7f9a0d951700 1 --2- 172.21.15.2:0/2858183008 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a081093b0 0x7f9a081097d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f99f80098a0 tx=0x7f99f8009bb0 comp rx=0 tx=0).stop 2024-05-14T16:44:01.651 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:00.799+0000 7f9a0d951700 1 -- 172.21.15.2:0/2858183008 shutdown_connections 2024-05-14T16:44:01.651 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:00.799+0000 7f9a0d951700 1 --2- 172.21.15.2:0/2858183008 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a081093b0 0x7f9a081097d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:01.651 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:00.799+0000 7f9a0d951700 1 -- 172.21.15.2:0/2858183008 >> 172.21.15.2:0/2858183008 conn(0x7f9a08102b20 msgr2=0x7f9a08104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:01.651 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:00.799+0000 7f9a0d951700 1 -- 172.21.15.2:0/2858183008 shutdown_connections 2024-05-14T16:44:01.651 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:00.799+0000 7f9a0d951700 1 -- 172.21.15.2:0/2858183008 wait complete. 2024-05-14T16:44:01.651 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:00.800+0000 7f9a0d951700 1 Processor -- start 2024-05-14T16:44:01.651 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:00.800+0000 7f9a0d951700 1 -- start start 2024-05-14T16:44:01.651 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:00.800+0000 7f9a0d951700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a081093b0 0x7f9a0819f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:01.651 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:00.800+0000 7f9a0d951700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9a08106f00 con 0x7f9a081093b0 2024-05-14T16:44:01.651 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:00.800+0000 7f9a06ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a081093b0 0x7f9a0819f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:01.652 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:00.800+0000 7f9a06ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a081093b0 0x7f9a0819f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47718/0 (socket says 172.21.15.2:47718) 2024-05-14T16:44:01.652 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:00.800+0000 7f9a06ffd700 1 -- 172.21.15.2:0/2926960034 learned_addr learned my addr 172.21.15.2:0/2926960034 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:44:01.652 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:00.800+0000 7f9a06ffd700 1 -- 172.21.15.2:0/2926960034 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a0819f5a0 con 0x7f9a081093b0 2024-05-14T16:44:01.652 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:00.801+0000 7f9a06ffd700 1 --2- 172.21.15.2:0/2926960034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a081093b0 0x7f9a0819f060 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f99f8000c00 tx=0x7f99f8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:01.652 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:00.801+0000 7f9a0c94f700 1 -- 172.21.15.2:0/2926960034 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99f8003aa0 con 0x7f9a081093b0 2024-05-14T16:44:01.652 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:00.801+0000 7f9a0c94f700 1 -- 172.21.15.2:0/2926960034 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99f801b8f0 con 0x7f9a081093b0 2024-05-14T16:44:01.652 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:00.801+0000 7f9a0c94f700 1 -- 172.21.15.2:0/2926960034 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99f8023890 con 0x7f9a081093b0 2024-05-14T16:44:01.652 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:00.801+0000 7f9a0d951700 1 -- 172.21.15.2:0/2926960034 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a0819f830 con 0x7f9a081093b0 2024-05-14T16:44:01.652 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:00.801+0000 7f9a0d951700 1 -- 172.21.15.2:0/2926960034 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a081a2520 con 0x7f9a081093b0 2024-05-14T16:44:01.652 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:00.802+0000 7f9a0c94f700 1 -- 172.21.15.2:0/2926960034 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f99f802c3f0 con 0x7f9a081093b0 2024-05-14T16:44:01.652 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:00.802+0000 7f9a0d951700 1 -- 172.21.15.2:0/2926960034 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a08064ba0 con 0x7f9a081093b0 2024-05-14T16:44:01.652 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:00.802+0000 7f9a0c94f700 1 --2- 172.21.15.2:0/2926960034 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f99f003e470 0x7f99f0040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:01.653 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:00.802+0000 7f9a0c94f700 1 -- 172.21.15.2:0/2926960034 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f99f8053dc0 con 0x7f9a081093b0 2024-05-14T16:44:01.653 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:00.803+0000 7f9a067fc700 1 --2- 172.21.15.2:0/2926960034 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f99f003e470 0x7f99f0040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:01.653 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:00.803+0000 7f9a067fc700 1 --2- 172.21.15.2:0/2926960034 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f99f003e470 0x7f99f0040930 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f99fc006fd0 tx=0x7f99fc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:01.653 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:00.808+0000 7f9a0c94f700 1 -- 172.21.15.2:0/2926960034 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f99f801f030 con 0x7f9a081093b0 2024-05-14T16:44:01.653 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:01.025+0000 7f9a0d951700 1 -- 172.21.15.2:0/2926960034 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f9a081a2b10 con 0x7f9a081093b0 2024-05-14T16:44:01.653 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:01.028+0000 7f9a0c94f700 1 -- 172.21.15.2:0/2926960034 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f99f8017e90 con 0x7f9a081093b0 2024-05-14T16:44:01.653 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:01.032+0000 7f9a0d951700 1 -- 172.21.15.2:0/2926960034 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f99f003e470 msgr2=0x7f99f0040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:01.653 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:01.032+0000 7f9a0d951700 1 --2- 172.21.15.2:0/2926960034 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f99f003e470 0x7f99f0040930 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f99fc006fd0 tx=0x7f99fc006e40 comp rx=0 tx=0).stop 2024-05-14T16:44:01.653 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:01.032+0000 7f9a0d951700 1 -- 172.21.15.2:0/2926960034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a081093b0 msgr2=0x7f9a0819f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:01.653 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:01.032+0000 7f9a0d951700 1 --2- 172.21.15.2:0/2926960034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a081093b0 0x7f9a0819f060 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f99f8000c00 tx=0x7f99f8011aa0 comp rx=0 tx=0).stop 2024-05-14T16:44:01.653 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:01.032+0000 7f9a0d951700 1 -- 172.21.15.2:0/2926960034 shutdown_connections 2024-05-14T16:44:01.653 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:01.032+0000 7f9a0d951700 1 --2- 172.21.15.2:0/2926960034 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f99f003e470 0x7f99f0040930 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:01.654 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:01.032+0000 7f9a0d951700 1 --2- 172.21.15.2:0/2926960034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a081093b0 0x7f9a0819f060 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:01.654 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:01.032+0000 7f9a0d951700 1 -- 172.21.15.2:0/2926960034 >> 172.21.15.2:0/2926960034 conn(0x7f9a08102b20 msgr2=0x7f9a08103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:01.654 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:01.032+0000 7f9a0d951700 1 -- 172.21.15.2:0/2926960034 shutdown_connections 2024-05-14T16:44:01.654 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:01.032+0000 7f9a0d951700 1 -- 172.21.15.2:0/2926960034 wait complete. 2024-05-14T16:44:02.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:02 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/2926960034' entity='client.admin' 2024-05-14T16:44:02.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:02 smithi002 ceph-mon[31538]: from='mgr.14118 172.21.15.2:0/1997519028' entity='mgr.x' 2024-05-14T16:44:03.365 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:03 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/1039474839' entity='client.admin' 2024-05-14T16:44:03.373 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.100+0000 7fd0b9a5f700 1 Processor -- start 2024-05-14T16:44:03.374 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.100+0000 7fd0b9a5f700 1 -- start start 2024-05-14T16:44:03.374 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.100+0000 7fd0b9a5f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b4108e60 0x7fd0b4109280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:03.374 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.100+0000 7fd0b9a5f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b41097c0 con 0x7fd0b4108e60 2024-05-14T16:44:03.374 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.100+0000 7fd0b2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b4108e60 0x7fd0b4109280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:03.374 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.100+0000 7fd0b2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b4108e60 0x7fd0b4109280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47722/0 (socket says 172.21.15.2:47722) 2024-05-14T16:44:03.374 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.100+0000 7fd0b2ffd700 1 -- 172.21.15.2:0/555876524 learned_addr learned my addr 172.21.15.2:0/555876524 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:44:03.374 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.101+0000 7fd0b2ffd700 1 -- 172.21.15.2:0/555876524 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0b410a050 con 0x7fd0b4108e60 2024-05-14T16:44:03.374 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.101+0000 7fd0b2ffd700 1 --2- 172.21.15.2:0/555876524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b4108e60 0x7fd0b4109280 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd0a40098a0 tx=0x7fd0a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a61eee5dd8ed771d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:03.374 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.101+0000 7fd0b27fc700 1 -- 172.21.15.2:0/555876524 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0a4004160 con 0x7fd0b4108e60 2024-05-14T16:44:03.374 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.101+0000 7fd0b27fc700 1 -- 172.21.15.2:0/555876524 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0a4004300 con 0x7fd0b4108e60 2024-05-14T16:44:03.374 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.101+0000 7fd0b27fc700 1 -- 172.21.15.2:0/555876524 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0a4011a20 con 0x7fd0b4108e60 2024-05-14T16:44:03.374 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.102+0000 7fd0b9a5f700 1 -- 172.21.15.2:0/555876524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b4108e60 msgr2=0x7fd0b4109280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:03.374 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.102+0000 7fd0b9a5f700 1 --2- 172.21.15.2:0/555876524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b4108e60 0x7fd0b4109280 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd0a40098a0 tx=0x7fd0a4009bb0 comp rx=0 tx=0).stop 2024-05-14T16:44:03.375 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.102+0000 7fd0b9a5f700 1 -- 172.21.15.2:0/555876524 shutdown_connections 2024-05-14T16:44:03.375 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.102+0000 7fd0b9a5f700 1 --2- 172.21.15.2:0/555876524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b4108e60 0x7fd0b4109280 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:03.375 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.102+0000 7fd0b9a5f700 1 -- 172.21.15.2:0/555876524 >> 172.21.15.2:0/555876524 conn(0x7fd0b4104490 msgr2=0x7fd0b41068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:03.375 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.102+0000 7fd0b9a5f700 1 -- 172.21.15.2:0/555876524 shutdown_connections 2024-05-14T16:44:03.375 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.102+0000 7fd0b9a5f700 1 -- 172.21.15.2:0/555876524 wait complete. 2024-05-14T16:44:03.375 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.103+0000 7fd0b9a5f700 1 Processor -- start 2024-05-14T16:44:03.375 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.103+0000 7fd0b9a5f700 1 -- start start 2024-05-14T16:44:03.375 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.103+0000 7fd0b9a5f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b4108e60 0x7fd0b419ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:03.375 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.103+0000 7fd0b9a5f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b410d310 con 0x7fd0b4108e60 2024-05-14T16:44:03.375 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.103+0000 7fd0b2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b4108e60 0x7fd0b419ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:03.375 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.103+0000 7fd0b2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b4108e60 0x7fd0b419ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47728/0 (socket says 172.21.15.2:47728) 2024-05-14T16:44:03.375 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.103+0000 7fd0b2ffd700 1 -- 172.21.15.2:0/1039474839 learned_addr learned my addr 172.21.15.2:0/1039474839 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:44:03.375 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.103+0000 7fd0b2ffd700 1 -- 172.21.15.2:0/1039474839 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0b419d340 con 0x7fd0b4108e60 2024-05-14T16:44:03.375 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.103+0000 7fd0b2ffd700 1 --2- 172.21.15.2:0/1039474839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b4108e60 0x7fd0b419ce00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4000c00 tx=0x7fd0a4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:03.376 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.104+0000 7fd0b8a5d700 1 -- 172.21.15.2:0/1039474839 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0a4003aa0 con 0x7fd0b4108e60 2024-05-14T16:44:03.376 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.104+0000 7fd0b8a5d700 1 -- 172.21.15.2:0/1039474839 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0a401b8f0 con 0x7fd0b4108e60 2024-05-14T16:44:03.376 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.104+0000 7fd0b9a5f700 1 -- 172.21.15.2:0/1039474839 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0b419d5d0 con 0x7fd0b4108e60 2024-05-14T16:44:03.376 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.104+0000 7fd0b8a5d700 1 -- 172.21.15.2:0/1039474839 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0a4023890 con 0x7fd0b4108e60 2024-05-14T16:44:03.376 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.104+0000 7fd0b9a5f700 1 -- 172.21.15.2:0/1039474839 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0b41a02c0 con 0x7fd0b4108e60 2024-05-14T16:44:03.376 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.105+0000 7fd0b8a5d700 1 -- 172.21.15.2:0/1039474839 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fd0a401b440 con 0x7fd0b4108e60 2024-05-14T16:44:03.376 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.105+0000 7fd0b9a5f700 1 -- 172.21.15.2:0/1039474839 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0b4064ba0 con 0x7fd0b4108e60 2024-05-14T16:44:03.376 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.105+0000 7fd0b8a5d700 1 --2- 172.21.15.2:0/1039474839 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7fd0a003e4c0 0x7fd0a0040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:03.376 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.105+0000 7fd0b8a5d700 1 -- 172.21.15.2:0/1039474839 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd0a4053de0 con 0x7fd0b4108e60 2024-05-14T16:44:03.376 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.105+0000 7fd0abfff700 1 --2- 172.21.15.2:0/1039474839 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7fd0a003e4c0 0x7fd0a0040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:03.376 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.106+0000 7fd0abfff700 1 --2- 172.21.15.2:0/1039474839 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7fd0a003e4c0 0x7fd0a0040980 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd09c006fd0 tx=0x7fd09c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:03.376 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.110+0000 7fd0b8a5d700 1 -- 172.21.15.2:0/1039474839 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd0a401f030 con 0x7fd0b4108e60 2024-05-14T16:44:03.376 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.345+0000 7fd0b9a5f700 1 -- 172.21.15.2:0/1039474839 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fd0b41a0920 con 0x7fd0b4108e60 2024-05-14T16:44:03.376 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.347+0000 7fd0b8a5d700 1 -- 172.21.15.2:0/1039474839 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fd0a400f2e0 con 0x7fd0b4108e60 2024-05-14T16:44:03.377 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.350+0000 7fd0b9a5f700 1 -- 172.21.15.2:0/1039474839 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7fd0a003e4c0 msgr2=0x7fd0a0040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:03.377 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.350+0000 7fd0b9a5f700 1 --2- 172.21.15.2:0/1039474839 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7fd0a003e4c0 0x7fd0a0040980 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd09c006fd0 tx=0x7fd09c006e40 comp rx=0 tx=0).stop 2024-05-14T16:44:03.377 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.350+0000 7fd0b9a5f700 1 -- 172.21.15.2:0/1039474839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b4108e60 msgr2=0x7fd0b419ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:03.377 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.350+0000 7fd0b9a5f700 1 --2- 172.21.15.2:0/1039474839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b4108e60 0x7fd0b419ce00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4000c00 tx=0x7fd0a4011aa0 comp rx=0 tx=0).stop 2024-05-14T16:44:03.377 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.350+0000 7fd0b9a5f700 1 -- 172.21.15.2:0/1039474839 shutdown_connections 2024-05-14T16:44:03.377 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.350+0000 7fd0b9a5f700 1 --2- 172.21.15.2:0/1039474839 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7fd0a003e4c0 0x7fd0a0040980 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:03.377 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.350+0000 7fd0b9a5f700 1 --2- 172.21.15.2:0/1039474839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b4108e60 0x7fd0b419ce00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:03.377 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.350+0000 7fd0b9a5f700 1 -- 172.21.15.2:0/1039474839 >> 172.21.15.2:0/1039474839 conn(0x7fd0b4104490 msgr2=0x7fd0b4104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:03.377 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.350+0000 7fd0b9a5f700 1 -- 172.21.15.2:0/1039474839 shutdown_connections 2024-05-14T16:44:03.377 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:02.350+0000 7fd0b9a5f700 1 -- 172.21.15.2:0/1039474839 wait complete. 2024-05-14T16:44:03.377 INFO:teuthology.orchestra.run.smithi002.stdout:Enabling the dashboard module... 2024-05-14T16:44:04.741 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:04 smithi002 ceph-mon[31538]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:44:04.741 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:04 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/613387869' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-14T16:44:04.741 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:44:04 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: ignoring --setuser ceph since I am not root 2024-05-14T16:44:04.742 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:44:04 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: ignoring --setgroup ceph since I am not root 2024-05-14T16:44:04.742 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:44:04 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:44:04.609+0000 7f37f7f9e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-14T16:44:04.994 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:44:04 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:44:04.807+0000 7f37f7f9e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:44:04.994 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:44:04 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:44:04.915+0000 7f37f7f9e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-14T16:44:05.029 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:03.840+0000 7f41f1796700 1 Processor -- start 2024-05-14T16:44:05.029 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:03.840+0000 7f41f1796700 1 -- start start 2024-05-14T16:44:05.029 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:03.840+0000 7f41f1796700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41ec07f640 0x7f41ec07daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:05.029 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:03.840+0000 7f41f1796700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f41ec07e030 con 0x7f41ec07f640 2024-05-14T16:44:05.029 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:03.840+0000 7f41eaffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41ec07f640 0x7f41ec07daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:05.029 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:03.840+0000 7f41eaffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41ec07f640 0x7f41ec07daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46300/0 (socket says 172.21.15.2:46300) 2024-05-14T16:44:05.029 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:03.840+0000 7f41eaffd700 1 -- 172.21.15.2:0/2544449649 learned_addr learned my addr 172.21.15.2:0/2544449649 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:44:05.029 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:03.841+0000 7f41eaffd700 1 -- 172.21.15.2:0/2544449649 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41ec07e1b0 con 0x7f41ec07f640 2024-05-14T16:44:05.030 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:03.841+0000 7f41eaffd700 1 --2- 172.21.15.2:0/2544449649 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41ec07f640 0x7f41ec07daf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0098a0 tx=0x7f41dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e4929d691a7cb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:05.030 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:03.841+0000 7f41e9ffb700 1 -- 172.21.15.2:0/2544449649 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41dc004160 con 0x7f41ec07f640 2024-05-14T16:44:05.030 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:03.841+0000 7f41e9ffb700 1 -- 172.21.15.2:0/2544449649 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41dc004300 con 0x7f41ec07f640 2024-05-14T16:44:05.030 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:03.841+0000 7f41e9ffb700 1 -- 172.21.15.2:0/2544449649 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41dc011a20 con 0x7f41ec07f640 2024-05-14T16:44:05.030 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:03.842+0000 7f41f1796700 1 -- 172.21.15.2:0/2544449649 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41ec07f640 msgr2=0x7f41ec07daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:05.030 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:03.842+0000 7f41f1796700 1 --2- 172.21.15.2:0/2544449649 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41ec07f640 0x7f41ec07daf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0098a0 tx=0x7f41dc009bb0 comp rx=0 tx=0).stop 2024-05-14T16:44:05.030 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:03.842+0000 7f41f1796700 1 -- 172.21.15.2:0/2544449649 shutdown_connections 2024-05-14T16:44:05.030 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:03.842+0000 7f41f1796700 1 --2- 172.21.15.2:0/2544449649 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41ec07f640 0x7f41ec07daf0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:05.030 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:03.842+0000 7f41f1796700 1 -- 172.21.15.2:0/2544449649 >> 172.21.15.2:0/2544449649 conn(0x7f41ec104480 msgr2=0x7f41ec1068c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:05.030 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:03.842+0000 7f41f1796700 1 -- 172.21.15.2:0/2544449649 shutdown_connections 2024-05-14T16:44:05.030 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:03.842+0000 7f41f1796700 1 -- 172.21.15.2:0/2544449649 wait complete. 2024-05-14T16:44:05.030 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:03.842+0000 7f41f1796700 1 Processor -- start 2024-05-14T16:44:05.030 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:03.843+0000 7f41f1796700 1 -- start start 2024-05-14T16:44:05.030 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:03.843+0000 7f41f1796700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41ec07f640 0x7f41ec19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:05.031 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:03.843+0000 7f41f1796700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f41ec10f440 con 0x7f41ec07f640 2024-05-14T16:44:05.031 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:03.843+0000 7f41eaffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41ec07f640 0x7f41ec19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:05.031 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:03.843+0000 7f41eaffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41ec07f640 0x7f41ec19efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46310/0 (socket says 172.21.15.2:46310) 2024-05-14T16:44:05.031 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:03.843+0000 7f41eaffd700 1 -- 172.21.15.2:0/613387869 learned_addr learned my addr 172.21.15.2:0/613387869 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:44:05.031 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:03.843+0000 7f41eaffd700 1 -- 172.21.15.2:0/613387869 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41ec19f500 con 0x7f41ec07f640 2024-05-14T16:44:05.031 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:03.843+0000 7f41eaffd700 1 --2- 172.21.15.2:0/613387869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41ec07f640 0x7f41ec19efc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0116e0 tx=0x7f41dc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:05.031 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:03.844+0000 7f41d3fff700 1 -- 172.21.15.2:0/613387869 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41dc003b20 con 0x7f41ec07f640 2024-05-14T16:44:05.031 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:03.844+0000 7f41d3fff700 1 -- 172.21.15.2:0/613387869 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41dc01b8f0 con 0x7f41ec07f640 2024-05-14T16:44:05.031 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:03.844+0000 7f41f1796700 1 -- 172.21.15.2:0/613387869 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41ec19f790 con 0x7f41ec07f640 2024-05-14T16:44:05.031 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:03.844+0000 7f41d3fff700 1 -- 172.21.15.2:0/613387869 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41dc0238c0 con 0x7f41ec07f640 2024-05-14T16:44:05.031 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:03.844+0000 7f41f1796700 1 -- 172.21.15.2:0/613387869 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41ec1a2480 con 0x7f41ec07f640 2024-05-14T16:44:05.031 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:03.845+0000 7f41d3fff700 1 -- 172.21.15.2:0/613387869 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f41dc023b70 con 0x7f41ec07f640 2024-05-14T16:44:05.031 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:03.845+0000 7f41f1796700 1 -- 172.21.15.2:0/613387869 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41ec064ba0 con 0x7f41ec07f640 2024-05-14T16:44:05.031 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:03.845+0000 7f41d3fff700 1 --2- 172.21.15.2:0/613387869 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f41d403e420 0x7f41d40408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:05.032 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:03.845+0000 7f41d3fff700 1 -- 172.21.15.2:0/613387869 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f41dc053fe0 con 0x7f41ec07f640 2024-05-14T16:44:05.032 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:03.845+0000 7f41ea7fc700 1 --2- 172.21.15.2:0/613387869 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f41d403e420 0x7f41d40408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:05.032 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:03.846+0000 7f41ea7fc700 1 --2- 172.21.15.2:0/613387869 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f41d403e420 0x7f41d40408e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f41e0006fd0 tx=0x7f41e0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:05.032 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:03.850+0000 7f41d3fff700 1 -- 172.21.15.2:0/613387869 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f41dc01f030 con 0x7f41ec07f640 2024-05-14T16:44:05.032 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:04.126+0000 7f41f1796700 1 -- 172.21.15.2:0/613387869 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f41ec1a2a30 con 0x7f41ec07f640 2024-05-14T16:44:05.032 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:04.417+0000 7f41d3fff700 1 -- 172.21.15.2:0/613387869 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f41dc00edf0 con 0x7f41ec07f640 2024-05-14T16:44:05.032 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:04.419+0000 7f41d3fff700 1 -- 172.21.15.2:0/613387869 <== mon.0 v2:172.21.15.2:3300/0 8 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f41dc01acb0 con 0x7f41ec07f640 2024-05-14T16:44:05.032 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:04.421+0000 7f41f1796700 1 -- 172.21.15.2:0/613387869 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f41d403e420 msgr2=0x7f41d40408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:05.032 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:04.421+0000 7f41f1796700 1 --2- 172.21.15.2:0/613387869 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f41d403e420 0x7f41d40408e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f41e0006fd0 tx=0x7f41e0006e40 comp rx=0 tx=0).stop 2024-05-14T16:44:05.032 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:04.421+0000 7f41f1796700 1 -- 172.21.15.2:0/613387869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41ec07f640 msgr2=0x7f41ec19efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:05.032 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:04.421+0000 7f41f1796700 1 --2- 172.21.15.2:0/613387869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41ec07f640 0x7f41ec19efc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0116e0 tx=0x7f41dc011aa0 comp rx=0 tx=0).stop 2024-05-14T16:44:05.032 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:04.421+0000 7f41f1796700 1 -- 172.21.15.2:0/613387869 shutdown_connections 2024-05-14T16:44:05.032 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:04.421+0000 7f41f1796700 1 --2- 172.21.15.2:0/613387869 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f41d403e420 0x7f41d40408e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:05.033 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:04.421+0000 7f41f1796700 1 --2- 172.21.15.2:0/613387869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f41ec07f640 0x7f41ec19efc0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:05.033 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:04.421+0000 7f41f1796700 1 -- 172.21.15.2:0/613387869 >> 172.21.15.2:0/613387869 conn(0x7f41ec104480 msgr2=0x7f41ec104fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:05.033 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:04.421+0000 7f41f1796700 1 -- 172.21.15.2:0/613387869 shutdown_connections 2024-05-14T16:44:05.033 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:04.421+0000 7f41f1796700 1 -- 172.21.15.2:0/613387869 wait complete. 2024-05-14T16:44:05.343 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:44:05 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:44:05.251+0000 7f37f7f9e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-14T16:44:05.343 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:44:05 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:44:05.343+0000 7f37f7f9e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-14T16:44:05.680 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:05 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/613387869' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-14T16:44:05.680 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:05 smithi002 ceph-mon[31538]: mgrmap e9: x(active, since 21s) 2024-05-14T16:44:06.339 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2024-05-14T16:44:06.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-14T16:44:06.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-14T16:44:06.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-14T16:44:06.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-14T16:44:06.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2024-05-14T16:44:06.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.474+0000 7f793e8dd700 1 Processor -- start 2024-05-14T16:44:06.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.474+0000 7f793e8dd700 1 -- start start 2024-05-14T16:44:06.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.474+0000 7f793e8dd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7938109900 0x7f793810bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:06.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.474+0000 7f793e8dd700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f793807d6c0 con 0x7f7938109900 2024-05-14T16:44:06.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.474+0000 7f7937fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7938109900 0x7f793810bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:06.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.474+0000 7f7937fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7938109900 0x7f793810bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46328/0 (socket says 172.21.15.2:46328) 2024-05-14T16:44:06.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.474+0000 7f7937fff700 1 -- 172.21.15.2:0/2968175720 learned_addr learned my addr 172.21.15.2:0/2968175720 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:44:06.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.475+0000 7f7937fff700 1 -- 172.21.15.2:0/2968175720 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f793810c230 con 0x7f7938109900 2024-05-14T16:44:06.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.475+0000 7f7937fff700 1 --2- 172.21.15.2:0/2968175720 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7938109900 0x7f793810bcf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f79280098a0 tx=0x7f7928009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88f222067d698165 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:06.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.475+0000 7f7936ffd700 1 -- 172.21.15.2:0/2968175720 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7928004160 con 0x7f7938109900 2024-05-14T16:44:06.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.475+0000 7f7936ffd700 1 -- 172.21.15.2:0/2968175720 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7928004300 con 0x7f7938109900 2024-05-14T16:44:06.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.475+0000 7f7936ffd700 1 -- 172.21.15.2:0/2968175720 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7928011a20 con 0x7f7938109900 2024-05-14T16:44:06.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.476+0000 7f793e8dd700 1 -- 172.21.15.2:0/2968175720 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7938109900 msgr2=0x7f793810bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:06.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.476+0000 7f793e8dd700 1 --2- 172.21.15.2:0/2968175720 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7938109900 0x7f793810bcf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f79280098a0 tx=0x7f7928009bb0 comp rx=0 tx=0).stop 2024-05-14T16:44:06.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.476+0000 7f793e8dd700 1 -- 172.21.15.2:0/2968175720 shutdown_connections 2024-05-14T16:44:06.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.476+0000 7f793e8dd700 1 --2- 172.21.15.2:0/2968175720 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7938109900 0x7f793810bcf0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:06.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.476+0000 7f793e8dd700 1 -- 172.21.15.2:0/2968175720 >> 172.21.15.2:0/2968175720 conn(0x7f7938103330 msgr2=0x7f7938105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:06.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.476+0000 7f793e8dd700 1 -- 172.21.15.2:0/2968175720 shutdown_connections 2024-05-14T16:44:06.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.476+0000 7f793e8dd700 1 -- 172.21.15.2:0/2968175720 wait complete. 2024-05-14T16:44:06.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.476+0000 7f793e8dd700 1 Processor -- start 2024-05-14T16:44:06.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.477+0000 7f793e8dd700 1 -- start start 2024-05-14T16:44:06.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.477+0000 7f793e8dd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7938109900 0x7f7938109180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:06.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.477+0000 7f793e8dd700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f793810d240 con 0x7f7938109900 2024-05-14T16:44:06.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.477+0000 7f7937fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7938109900 0x7f7938109180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:06.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.477+0000 7f7937fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7938109900 0x7f7938109180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46332/0 (socket says 172.21.15.2:46332) 2024-05-14T16:44:06.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.477+0000 7f7937fff700 1 -- 172.21.15.2:0/471702174 learned_addr learned my addr 172.21.15.2:0/471702174 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:44:06.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.477+0000 7f7937fff700 1 -- 172.21.15.2:0/471702174 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79381096c0 con 0x7f7938109900 2024-05-14T16:44:06.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.477+0000 7f7937fff700 1 --2- 172.21.15.2:0/471702174 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7938109900 0x7f7938109180 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f79280116e0 tx=0x7f7928011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:06.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.478+0000 7f79357fa700 1 -- 172.21.15.2:0/471702174 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7928003b20 con 0x7f7938109900 2024-05-14T16:44:06.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.478+0000 7f79357fa700 1 -- 172.21.15.2:0/471702174 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f792801b8f0 con 0x7f7938109900 2024-05-14T16:44:06.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.478+0000 7f793e8dd700 1 -- 172.21.15.2:0/471702174 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79381078b0 con 0x7f7938109900 2024-05-14T16:44:06.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.478+0000 7f793e8dd700 1 -- 172.21.15.2:0/471702174 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7938107df0 con 0x7f7938109900 2024-05-14T16:44:06.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.478+0000 7f79357fa700 1 -- 172.21.15.2:0/471702174 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f79280238c0 con 0x7f7938109900 2024-05-14T16:44:06.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.479+0000 7f79357fa700 1 -- 172.21.15.2:0/471702174 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f7928023b90 con 0x7f7938109900 2024-05-14T16:44:06.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.479+0000 7f79357fa700 1 --2- 172.21.15.2:0/471702174 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f792003e470 0x7f7920040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:06.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.479+0000 7f79377fe700 1 -- 172.21.15.2:0/471702174 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f792003e470 msgr2=0x7f7920040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3582988344 2024-05-14T16:44:06.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.479+0000 7f79357fa700 1 -- 172.21.15.2:0/471702174 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7928054050 con 0x7f7938109900 2024-05-14T16:44:06.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.479+0000 7f79377fe700 1 --2- 172.21.15.2:0/471702174 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f792003e470 0x7f7920040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T16:44:06.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.479+0000 7f793e8dd700 1 -- 172.21.15.2:0/471702174 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7938064ba0 con 0x7f7938109900 2024-05-14T16:44:06.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.484+0000 7f79357fa700 1 -- 172.21.15.2:0/471702174 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f792801f030 con 0x7f7938109900 2024-05-14T16:44:06.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.680+0000 7f79377fe700 1 -- 172.21.15.2:0/471702174 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f792003e470 msgr2=0x7f7920040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3582988344 2024-05-14T16:44:06.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.680+0000 7f79377fe700 1 --2- 172.21.15.2:0/471702174 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f792003e470 0x7f7920040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-14T16:44:06.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.786+0000 7f793e8dd700 1 -- 172.21.15.2:0/471702174 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f7938108470 con 0x7f7938109900 2024-05-14T16:44:06.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.787+0000 7f79357fa700 1 -- 172.21.15.2:0/471702174 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f792801bbc0 con 0x7f7938109900 2024-05-14T16:44:06.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.792+0000 7f793e8dd700 1 -- 172.21.15.2:0/471702174 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f792003e470 msgr2=0x7f7920040930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T16:44:06.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.792+0000 7f793e8dd700 1 --2- 172.21.15.2:0/471702174 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f792003e470 0x7f7920040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:06.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.792+0000 7f793e8dd700 1 -- 172.21.15.2:0/471702174 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7938109900 msgr2=0x7f7938109180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:06.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.792+0000 7f793e8dd700 1 --2- 172.21.15.2:0/471702174 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7938109900 0x7f7938109180 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f79280116e0 tx=0x7f7928011aa0 comp rx=0 tx=0).stop 2024-05-14T16:44:06.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.792+0000 7f793e8dd700 1 -- 172.21.15.2:0/471702174 shutdown_connections 2024-05-14T16:44:06.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.792+0000 7f793e8dd700 1 --2- 172.21.15.2:0/471702174 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7f792003e470 0x7f7920040930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:06.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.792+0000 7f793e8dd700 1 --2- 172.21.15.2:0/471702174 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7938109900 0x7f7938109180 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:06.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.792+0000 7f793e8dd700 1 -- 172.21.15.2:0/471702174 >> 172.21.15.2:0/471702174 conn(0x7f7938103330 msgr2=0x7f7938103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:06.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.792+0000 7f793e8dd700 1 -- 172.21.15.2:0/471702174 shutdown_connections 2024-05-14T16:44:06.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:05.792+0000 7f793e8dd700 1 -- 172.21.15.2:0/471702174 wait complete. 2024-05-14T16:44:06.344 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for the mgr to restart... 2024-05-14T16:44:06.344 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mgr epoch 9... 2024-05-14T16:44:06.742 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:06 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/471702174' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-14T16:44:06.742 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:44:06 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:44:06.640+0000 7f37f7f9e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-14T16:44:06.994 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:44:06 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:44:06.826+0000 7f37f7f9e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-14T16:44:07.312 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:44:07 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:44:07.007+0000 7f37f7f9e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-14T16:44:07.312 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:44:07 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:44:07.312+0000 7f37f7f9e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-14T16:44:07.622 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:44:07 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:44:07.409+0000 7f37f7f9e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-14T16:44:07.622 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:44:07 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:44:07.622+0000 7f37f7f9e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-14T16:44:07.994 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:44:07 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:44:07.826+0000 7f37f7f9e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:44:08.744 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:44:08 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:44:08.487+0000 7f37f7f9e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-14T16:44:08.744 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:44:08 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:44:08.720+0000 7f37f7f9e200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-14T16:44:09.494 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:44:09 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:44:09.172+0000 7f37f7f9e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-14T16:44:09.494 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:44:09 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:44:09.276+0000 7f37f7f9e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-14T16:44:09.994 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:44:09 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:44:09.684+0000 7f37f7f9e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-14T16:44:10.428 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:44:10 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:44:10.064+0000 7f37f7f9e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-14T16:44:10.429 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:44:10 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:44:10.165+0000 7f37f7f9e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-14T16:44:10.744 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:44:10 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:44:10.428+0000 7f37f7f9e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-14T16:44:10.744 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:44:10 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:44:10.571+0000 7f37f7f9e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-14T16:44:10.744 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:44:10 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:44:10.673+0000 7f37f7f9e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-14T16:44:11.494 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:44:11 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:44:11.015+0000 7f37f7f9e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-14T16:44:11.494 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:44:11 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:44:11.115+0000 7f37f7f9e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-14T16:44:12.048 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:44:11 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:44:11.761+0000 7f37f7f9e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-14T16:44:12.048 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:44:11 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:44:11.996+0000 7f37f7f9e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-14T16:44:12.387 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:12 smithi002 ceph-mon[31538]: Active manager daemon x restarted 2024-05-14T16:44:12.387 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:12 smithi002 ceph-mon[31538]: Activating manager daemon x 2024-05-14T16:44:12.388 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:12 smithi002 ceph-mon[31538]: osdmap e3: 0 total, 0 up, 0 in 2024-05-14T16:44:12.388 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:12 smithi002 ceph-mon[31538]: mgrmap e10: x(active, starting, since 0.00353988s) 2024-05-14T16:44:12.388 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:12 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:44:12.388 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:12 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-14T16:44:12.388 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:12 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-14T16:44:12.388 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:12 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-14T16:44:12.388 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:12 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-14T16:44:12.388 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:12 smithi002 ceph-mon[31538]: Manager daemon x is now available 2024-05-14T16:44:12.388 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:12 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:44:13.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:13 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-14T16:44:13.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:13 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-14T16:44:13.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:13 smithi002 ceph-mon[31538]: mgrmap e11: x(active, since 1.0077s) 2024-05-14T16:44:14.117 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2024-05-14T16:44:14.118 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-14T16:44:14.118 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-14T16:44:14.118 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2024-05-14T16:44:14.118 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.746+0000 7fbbb1af5700 1 Processor -- start 2024-05-14T16:44:14.118 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.746+0000 7fbbb1af5700 1 -- start start 2024-05-14T16:44:14.118 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.746+0000 7fbbb1af5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbac109990 0x7fbbac10bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:14.118 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.746+0000 7fbbb1af5700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbbac07cdf0 con 0x7fbbac109990 2024-05-14T16:44:14.118 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.746+0000 7fbbab7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbac109990 0x7fbbac10bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:14.118 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.746+0000 7fbbab7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbac109990 0x7fbbac10bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46338/0 (socket says 172.21.15.2:46338) 2024-05-14T16:44:14.118 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.746+0000 7fbbab7fe700 1 -- 172.21.15.2:0/1192775496 learned_addr learned my addr 172.21.15.2:0/1192775496 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:44:14.119 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.746+0000 7fbbab7fe700 1 -- 172.21.15.2:0/1192775496 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbac10c2c0 con 0x7fbbac109990 2024-05-14T16:44:14.119 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.747+0000 7fbbab7fe700 1 --2- 172.21.15.2:0/1192775496 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbac109990 0x7fbbac10bd80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0098a0 tx=0x7fbb9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd917378da25af54 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:14.119 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.747+0000 7fbbaa7fc700 1 -- 172.21.15.2:0/1192775496 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb9c004160 con 0x7fbbac109990 2024-05-14T16:44:14.119 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.747+0000 7fbbaa7fc700 1 -- 172.21.15.2:0/1192775496 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb9c004300 con 0x7fbbac109990 2024-05-14T16:44:14.119 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.747+0000 7fbbaa7fc700 1 -- 172.21.15.2:0/1192775496 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb9c011a20 con 0x7fbbac109990 2024-05-14T16:44:14.119 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.747+0000 7fbbb1af5700 1 -- 172.21.15.2:0/1192775496 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbac109990 msgr2=0x7fbbac10bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:14.119 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.747+0000 7fbbb1af5700 1 --2- 172.21.15.2:0/1192775496 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbac109990 0x7fbbac10bd80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0098a0 tx=0x7fbb9c009bb0 comp rx=0 tx=0).stop 2024-05-14T16:44:14.119 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.747+0000 7fbbb1af5700 1 -- 172.21.15.2:0/1192775496 shutdown_connections 2024-05-14T16:44:14.119 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.747+0000 7fbbb1af5700 1 --2- 172.21.15.2:0/1192775496 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbac109990 0x7fbbac10bd80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:14.119 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.747+0000 7fbbb1af5700 1 -- 172.21.15.2:0/1192775496 >> 172.21.15.2:0/1192775496 conn(0x7fbbac1033a0 msgr2=0x7fbbac1057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:14.119 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.748+0000 7fbbb1af5700 1 -- 172.21.15.2:0/1192775496 shutdown_connections 2024-05-14T16:44:14.119 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.748+0000 7fbbb1af5700 1 -- 172.21.15.2:0/1192775496 wait complete. 2024-05-14T16:44:14.119 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.748+0000 7fbbb1af5700 1 Processor -- start 2024-05-14T16:44:14.120 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.748+0000 7fbbb1af5700 1 -- start start 2024-05-14T16:44:14.120 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.749+0000 7fbbb1af5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbac109990 0x7fbbac19ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:14.120 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.749+0000 7fbbb1af5700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbbac10d2c0 con 0x7fbbac109990 2024-05-14T16:44:14.120 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.749+0000 7fbbab7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbac109990 0x7fbbac19ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:14.120 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.749+0000 7fbbab7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbac109990 0x7fbbac19ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46340/0 (socket says 172.21.15.2:46340) 2024-05-14T16:44:14.120 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.749+0000 7fbbab7fe700 1 -- 172.21.15.2:0/3018003400 learned_addr learned my addr 172.21.15.2:0/3018003400 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:44:14.120 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.749+0000 7fbbab7fe700 1 -- 172.21.15.2:0/3018003400 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbac19d350 con 0x7fbbac109990 2024-05-14T16:44:14.120 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.749+0000 7fbbab7fe700 1 --2- 172.21.15.2:0/3018003400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbac109990 0x7fbbac19ce10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0116e0 tx=0x7fbb9c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:14.120 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.749+0000 7fbba8ff9700 1 -- 172.21.15.2:0/3018003400 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb9c003b20 con 0x7fbbac109990 2024-05-14T16:44:14.120 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.749+0000 7fbba8ff9700 1 -- 172.21.15.2:0/3018003400 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb9c01b8f0 con 0x7fbbac109990 2024-05-14T16:44:14.120 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.749+0000 7fbbb1af5700 1 -- 172.21.15.2:0/3018003400 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbac19d5e0 con 0x7fbbac109990 2024-05-14T16:44:14.120 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.749+0000 7fbba8ff9700 1 -- 172.21.15.2:0/3018003400 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb9c0238c0 con 0x7fbbac109990 2024-05-14T16:44:14.120 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.750+0000 7fbbb1af5700 1 -- 172.21.15.2:0/3018003400 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbac1a02d0 con 0x7fbbac109990 2024-05-14T16:44:14.121 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.751+0000 7fbba8ff9700 1 -- 172.21.15.2:0/3018003400 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7fbb9c023b70 con 0x7fbbac109990 2024-05-14T16:44:14.121 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.751+0000 7fbba8ff9700 1 --2- 172.21.15.2:0/3018003400 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7fbb9803e470 0x7fbb98040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:14.121 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.751+0000 7fbba8ff9700 1 -- 172.21.15.2:0/3018003400 --> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fbb98041040 con 0x7fbb9803e470 2024-05-14T16:44:14.121 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.751+0000 7fbbaaffd700 1 -- 172.21.15.2:0/3018003400 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7fbb9803e470 msgr2=0x7fbb98040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3582988344 2024-05-14T16:44:14.121 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.751+0000 7fbba8ff9700 1 -- 172.21.15.2:0/3018003400 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbb9c054010 con 0x7fbbac109990 2024-05-14T16:44:14.121 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.751+0000 7fbbaaffd700 1 --2- 172.21.15.2:0/3018003400 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7fbb9803e470 0x7fbb98040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T16:44:14.121 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.952+0000 7fbbaaffd700 1 -- 172.21.15.2:0/3018003400 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7fbb9803e470 msgr2=0x7fbb98040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3582988344 2024-05-14T16:44:14.121 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:06.952+0000 7fbbaaffd700 1 --2- 172.21.15.2:0/3018003400 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7fbb9803e470 0x7fbb98040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-14T16:44:14.121 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:07.353+0000 7fbbaaffd700 1 -- 172.21.15.2:0/3018003400 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7fbb9803e470 msgr2=0x7fbb98040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3582988344 2024-05-14T16:44:14.121 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:07.353+0000 7fbbaaffd700 1 --2- 172.21.15.2:0/3018003400 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7fbb9803e470 0x7fbb98040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-14T16:44:14.121 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:08.154+0000 7fbbaaffd700 1 -- 172.21.15.2:0/3018003400 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7fbb9803e470 msgr2=0x7fbb98040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3582988344 2024-05-14T16:44:14.121 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:08.154+0000 7fbbaaffd700 1 --2- 172.21.15.2:0/3018003400 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7fbb9803e470 0x7fbb98040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-14T16:44:14.121 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:09.755+0000 7fbbaaffd700 1 -- 172.21.15.2:0/3018003400 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7fbb9803e470 msgr2=0x7fbb98040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3582988344 2024-05-14T16:44:14.122 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:09.755+0000 7fbbaaffd700 1 --2- 172.21.15.2:0/3018003400 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7fbb9803e470 0x7fbb98040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-14T16:44:14.122 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:11.999+0000 7fbba8ff9700 1 -- 172.21.15.2:0/3018003400 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7fbb9c01b440 con 0x7fbbac109990 2024-05-14T16:44:14.122 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:11.999+0000 7fbba8ff9700 1 -- 172.21.15.2:0/3018003400 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7fbb9803e470 msgr2=0x7fbb98040930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T16:44:14.122 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:11.999+0000 7fbba8ff9700 1 --2- 172.21.15.2:0/3018003400 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7fbb9803e470 0x7fbb98040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:14.122 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:13.002+0000 7fbba8ff9700 1 -- 172.21.15.2:0/3018003400 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7fbb9c02b520 con 0x7fbbac109990 2024-05-14T16:44:14.122 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:13.002+0000 7fbba8ff9700 1 --2- 172.21.15.2:0/3018003400 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fbb98041ef0 0x7fbb980442e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:14.122 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:13.002+0000 7fbba8ff9700 1 -- 172.21.15.2:0/3018003400 --> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fbb9c018ea0 con 0x7fbb98041ef0 2024-05-14T16:44:14.122 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:13.002+0000 7fbbaaffd700 1 --2- 172.21.15.2:0/3018003400 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fbb98041ef0 0x7fbb980442e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:14.122 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:13.003+0000 7fbbaaffd700 1 --2- 172.21.15.2:0/3018003400 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fbb98041ef0 0x7fbb980442e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbb94003a10 tx=0x7fbb940092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:14.122 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:13.004+0000 7fbba8ff9700 1 -- 172.21.15.2:0/3018003400 <== mgr.14150 v2:172.21.15.2:6800/1498936028 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fbb9c018ea0 con 0x7fbb98041ef0 2024-05-14T16:44:14.122 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:13.012+0000 7fbbb1af5700 1 -- 172.21.15.2:0/3018003400 --> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fbbac06a650 con 0x7fbb98041ef0 2024-05-14T16:44:14.122 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:13.013+0000 7fbba8ff9700 1 -- 172.21.15.2:0/3018003400 <== mgr.14150 v2:172.21.15.2:6800/1498936028 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fbbac06a650 con 0x7fbb98041ef0 2024-05-14T16:44:14.122 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:13.013+0000 7fbbb1af5700 1 -- 172.21.15.2:0/3018003400 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fbb98041ef0 msgr2=0x7fbb980442e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:14.123 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:13.013+0000 7fbbb1af5700 1 --2- 172.21.15.2:0/3018003400 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fbb98041ef0 0x7fbb980442e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbb94003a10 tx=0x7fbb940092b0 comp rx=0 tx=0).stop 2024-05-14T16:44:14.123 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:13.013+0000 7fbbb1af5700 1 -- 172.21.15.2:0/3018003400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbac109990 msgr2=0x7fbbac19ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:14.123 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:13.013+0000 7fbbb1af5700 1 --2- 172.21.15.2:0/3018003400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbac109990 0x7fbbac19ce10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0116e0 tx=0x7fbb9c011aa0 comp rx=0 tx=0).stop 2024-05-14T16:44:14.123 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:13.013+0000 7fbbb1af5700 1 -- 172.21.15.2:0/3018003400 shutdown_connections 2024-05-14T16:44:14.123 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:13.013+0000 7fbbb1af5700 1 --2- 172.21.15.2:0/3018003400 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fbb98041ef0 0x7fbb980442e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:14.123 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:13.013+0000 7fbbb1af5700 1 --2- 172.21.15.2:0/3018003400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbac109990 0x7fbbac19ce10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:14.123 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:13.013+0000 7fbbb1af5700 1 --2- 172.21.15.2:0/3018003400 >> [v2:172.21.15.2:6800/3582988344,v1:172.21.15.2:6801/3582988344] conn(0x7fbb9803e470 0x7fbb98040930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:14.123 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:13.013+0000 7fbbb1af5700 1 -- 172.21.15.2:0/3018003400 >> 172.21.15.2:0/3018003400 conn(0x7fbbac1033a0 msgr2=0x7fbbac103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:14.123 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:13.013+0000 7fbbb1af5700 1 -- 172.21.15.2:0/3018003400 shutdown_connections 2024-05-14T16:44:14.123 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:13.013+0000 7fbbb1af5700 1 -- 172.21.15.2:0/3018003400 wait complete. 2024-05-14T16:44:14.123 INFO:teuthology.orchestra.run.smithi002.stdout:mgr epoch 9 is available 2024-05-14T16:44:14.123 INFO:teuthology.orchestra.run.smithi002.stdout:Generating a dashboard self-signed certificate... 2024-05-14T16:44:14.383 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:14 smithi002 ceph-mon[31538]: [14/May/2024:16:44:13] ENGINE Bus STARTING 2024-05-14T16:44:14.383 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:14 smithi002 ceph-mon[31538]: [14/May/2024:16:44:13] ENGINE Serving on http://172.21.15.2:8765 2024-05-14T16:44:14.383 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:14 smithi002 ceph-mon[31538]: [14/May/2024:16:44:13] ENGINE Serving on https://172.21.15.2:7150 2024-05-14T16:44:14.383 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:14 smithi002 ceph-mon[31538]: [14/May/2024:16:44:13] ENGINE Bus STARTED 2024-05-14T16:44:15.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:15 smithi002 ceph-mon[31538]: mgrmap e12: x(active, since 2s) 2024-05-14T16:44:15.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:15 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:44:15.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:15 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:44:15.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:15 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:44:15.697 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-14T16:44:15.697 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.575+0000 7f2e32545700 1 Processor -- start 2024-05-14T16:44:15.697 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.575+0000 7f2e32545700 1 -- start start 2024-05-14T16:44:15.697 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.575+0000 7f2e32545700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e2c107780 0x7f2e2c107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:15.697 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.575+0000 7f2e32545700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2c07cdf0 con 0x7f2e2c107780 2024-05-14T16:44:15.697 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.575+0000 7f2e2bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e2c107780 0x7f2e2c107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:15.697 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.575+0000 7f2e2bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e2c107780 0x7f2e2c107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52908/0 (socket says 172.21.15.2:52908) 2024-05-14T16:44:15.697 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.575+0000 7f2e2bfff700 1 -- 172.21.15.2:0/1398959583 learned_addr learned my addr 172.21.15.2:0/1398959583 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:44:15.697 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.575+0000 7f2e2bfff700 1 -- 172.21.15.2:0/1398959583 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e2c1080e0 con 0x7f2e2c107780 2024-05-14T16:44:15.697 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.576+0000 7f2e2bfff700 1 --2- 172.21.15.2:0/1398959583 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e2c107780 0x7f2e2c107ba0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c0098a0 tx=0x7f2e1c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0d3ec8fcf537852 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:15.698 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.576+0000 7f2e2affd700 1 -- 172.21.15.2:0/1398959583 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e1c004160 con 0x7f2e2c107780 2024-05-14T16:44:15.698 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.576+0000 7f2e2affd700 1 -- 172.21.15.2:0/1398959583 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e1c004300 con 0x7f2e2c107780 2024-05-14T16:44:15.698 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.576+0000 7f2e2affd700 1 -- 172.21.15.2:0/1398959583 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e1c011a20 con 0x7f2e2c107780 2024-05-14T16:44:15.698 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.576+0000 7f2e32545700 1 -- 172.21.15.2:0/1398959583 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e2c107780 msgr2=0x7f2e2c107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:15.698 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.576+0000 7f2e32545700 1 --2- 172.21.15.2:0/1398959583 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e2c107780 0x7f2e2c107ba0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c0098a0 tx=0x7f2e1c009bb0 comp rx=0 tx=0).stop 2024-05-14T16:44:15.698 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.576+0000 7f2e32545700 1 -- 172.21.15.2:0/1398959583 shutdown_connections 2024-05-14T16:44:15.698 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.576+0000 7f2e32545700 1 --2- 172.21.15.2:0/1398959583 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e2c107780 0x7f2e2c107ba0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:15.698 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.576+0000 7f2e32545700 1 -- 172.21.15.2:0/1398959583 >> 172.21.15.2:0/1398959583 conn(0x7f2e2c1033a0 msgr2=0x7f2e2c1057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:15.698 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.577+0000 7f2e32545700 1 -- 172.21.15.2:0/1398959583 shutdown_connections 2024-05-14T16:44:15.698 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.577+0000 7f2e32545700 1 -- 172.21.15.2:0/1398959583 wait complete. 2024-05-14T16:44:15.698 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.577+0000 7f2e32545700 1 Processor -- start 2024-05-14T16:44:15.699 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.577+0000 7f2e32545700 1 -- start start 2024-05-14T16:44:15.699 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.578+0000 7f2e32545700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e2c107780 0x7f2e2c19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:15.699 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.578+0000 7f2e32545700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2c109150 con 0x7f2e2c107780 2024-05-14T16:44:15.699 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.578+0000 7f2e2bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e2c107780 0x7f2e2c19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:15.699 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.578+0000 7f2e2bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e2c107780 0x7f2e2c19f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52910/0 (socket says 172.21.15.2:52910) 2024-05-14T16:44:15.700 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.578+0000 7f2e2bfff700 1 -- 172.21.15.2:0/365770746 learned_addr learned my addr 172.21.15.2:0/365770746 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:44:15.700 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.578+0000 7f2e2bfff700 1 -- 172.21.15.2:0/365770746 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e2c19f540 con 0x7f2e2c107780 2024-05-14T16:44:15.700 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.578+0000 7f2e2bfff700 1 --2- 172.21.15.2:0/365770746 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e2c107780 0x7f2e2c19f000 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c0116e0 tx=0x7f2e1c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:15.700 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.578+0000 7f2e297fa700 1 -- 172.21.15.2:0/365770746 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e1c003b20 con 0x7f2e2c107780 2024-05-14T16:44:15.700 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.578+0000 7f2e32545700 1 -- 172.21.15.2:0/365770746 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e2c19f7d0 con 0x7f2e2c107780 2024-05-14T16:44:15.700 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.578+0000 7f2e32545700 1 -- 172.21.15.2:0/365770746 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e2c1a24c0 con 0x7f2e2c107780 2024-05-14T16:44:15.701 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.579+0000 7f2e297fa700 1 -- 172.21.15.2:0/365770746 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e1c01b8f0 con 0x7f2e2c107780 2024-05-14T16:44:15.701 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.579+0000 7f2e297fa700 1 -- 172.21.15.2:0/365770746 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e1c0238c0 con 0x7f2e2c107780 2024-05-14T16:44:15.701 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.579+0000 7f2e297fa700 1 -- 172.21.15.2:0/365770746 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f2e1c023b40 con 0x7f2e2c107780 2024-05-14T16:44:15.701 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.580+0000 7f2e297fa700 1 --2- 172.21.15.2:0/365770746 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f2e1403e4d0 0x7f2e14040990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:15.701 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.580+0000 7f2e297fa700 1 -- 172.21.15.2:0/365770746 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2e1c054f20 con 0x7f2e2c107780 2024-05-14T16:44:15.701 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.580+0000 7f2e32545700 1 -- 172.21.15.2:0/365770746 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e18005360 con 0x7f2e2c107780 2024-05-14T16:44:15.701 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.580+0000 7f2e2b7fe700 1 --2- 172.21.15.2:0/365770746 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f2e1403e4d0 0x7f2e14040990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:15.701 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.580+0000 7f2e2b7fe700 1 --2- 172.21.15.2:0/365770746 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f2e1403e4d0 0x7f2e14040990 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2e20006fd0 tx=0x7f2e20006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:15.702 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.585+0000 7f2e297fa700 1 -- 172.21.15.2:0/365770746 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2e1c01f030 con 0x7f2e2c107780 2024-05-14T16:44:15.702 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.820+0000 7f2e32545700 1 -- 172.21.15.2:0/365770746 --> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f2e18000bf0 con 0x7f2e1403e4d0 2024-05-14T16:44:15.702 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.919+0000 7f2e297fa700 1 -- 172.21.15.2:0/365770746 <== mgr.14150 v2:172.21.15.2:6800/1498936028 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f2e18000bf0 con 0x7f2e1403e4d0 2024-05-14T16:44:15.702 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.921+0000 7f2e32545700 1 -- 172.21.15.2:0/365770746 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f2e1403e4d0 msgr2=0x7f2e14040990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:15.702 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.921+0000 7f2e32545700 1 --2- 172.21.15.2:0/365770746 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f2e1403e4d0 0x7f2e14040990 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2e20006fd0 tx=0x7f2e20006e40 comp rx=0 tx=0).stop 2024-05-14T16:44:15.702 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.921+0000 7f2e32545700 1 -- 172.21.15.2:0/365770746 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e2c107780 msgr2=0x7f2e2c19f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:15.702 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.921+0000 7f2e32545700 1 --2- 172.21.15.2:0/365770746 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e2c107780 0x7f2e2c19f000 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c0116e0 tx=0x7f2e1c011aa0 comp rx=0 tx=0).stop 2024-05-14T16:44:15.702 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.922+0000 7f2e32545700 1 -- 172.21.15.2:0/365770746 shutdown_connections 2024-05-14T16:44:15.702 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.922+0000 7f2e32545700 1 --2- 172.21.15.2:0/365770746 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f2e1403e4d0 0x7f2e14040990 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:15.702 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.922+0000 7f2e32545700 1 --2- 172.21.15.2:0/365770746 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e2c107780 0x7f2e2c19f000 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:15.702 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.922+0000 7f2e32545700 1 -- 172.21.15.2:0/365770746 >> 172.21.15.2:0/365770746 conn(0x7f2e2c1033a0 msgr2=0x7f2e2c195c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:15.702 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.922+0000 7f2e32545700 1 -- 172.21.15.2:0/365770746 shutdown_connections 2024-05-14T16:44:15.702 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:14.922+0000 7f2e32545700 1 -- 172.21.15.2:0/365770746 wait complete. 2024-05-14T16:44:15.702 INFO:teuthology.orchestra.run.smithi002.stdout:Creating initial admin user... 2024-05-14T16:44:16.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:16 smithi002 ceph-mon[31538]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:44:16.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:16 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:44:17.458 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:17 smithi002 ceph-mon[31538]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:44:17.459 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:17 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:44:17.461 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$aZtb.xE843PjEyCTLKgsheKbUdgGeE2PbiSyrpiigemovv/XQNTHu", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715705056, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-14T16:44:17.467 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.175+0000 7f5b2c893700 1 Processor -- start 2024-05-14T16:44:17.467 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.175+0000 7f5b2c893700 1 -- start start 2024-05-14T16:44:17.467 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.176+0000 7f5b2c893700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b28108e60 0x7f5b28109280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:17.467 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.176+0000 7f5b2c893700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5b281097c0 con 0x7f5b28108e60 2024-05-14T16:44:17.467 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.176+0000 7f5b2659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b28108e60 0x7f5b28109280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:17.467 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.176+0000 7f5b2659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b28108e60 0x7f5b28109280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52922/0 (socket says 172.21.15.2:52922) 2024-05-14T16:44:17.467 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.176+0000 7f5b2659c700 1 -- 172.21.15.2:0/2365431694 learned_addr learned my addr 172.21.15.2:0/2365431694 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:44:17.467 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.176+0000 7f5b2659c700 1 -- 172.21.15.2:0/2365431694 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b2810a050 con 0x7f5b28108e60 2024-05-14T16:44:17.467 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.176+0000 7f5b2659c700 1 --2- 172.21.15.2:0/2365431694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b28108e60 0x7f5b28109280 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5b180098a0 tx=0x7f5b18009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f38f6e20cce5b30 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:17.467 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.176+0000 7f5b2559a700 1 -- 172.21.15.2:0/2365431694 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b18004160 con 0x7f5b28108e60 2024-05-14T16:44:17.468 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.177+0000 7f5b2559a700 1 -- 172.21.15.2:0/2365431694 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b18004300 con 0x7f5b28108e60 2024-05-14T16:44:17.468 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.177+0000 7f5b2559a700 1 -- 172.21.15.2:0/2365431694 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b18011a20 con 0x7f5b28108e60 2024-05-14T16:44:17.468 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.177+0000 7f5b2c893700 1 -- 172.21.15.2:0/2365431694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b28108e60 msgr2=0x7f5b28109280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:17.468 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.177+0000 7f5b2c893700 1 --2- 172.21.15.2:0/2365431694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b28108e60 0x7f5b28109280 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5b180098a0 tx=0x7f5b18009bb0 comp rx=0 tx=0).stop 2024-05-14T16:44:17.468 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.177+0000 7f5b2c893700 1 -- 172.21.15.2:0/2365431694 shutdown_connections 2024-05-14T16:44:17.468 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.177+0000 7f5b2c893700 1 --2- 172.21.15.2:0/2365431694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b28108e60 0x7f5b28109280 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:17.468 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.177+0000 7f5b2c893700 1 -- 172.21.15.2:0/2365431694 >> 172.21.15.2:0/2365431694 conn(0x7f5b28104450 msgr2=0x7f5b28106890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:17.468 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.177+0000 7f5b2c893700 1 -- 172.21.15.2:0/2365431694 shutdown_connections 2024-05-14T16:44:17.468 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.177+0000 7f5b2c893700 1 -- 172.21.15.2:0/2365431694 wait complete. 2024-05-14T16:44:17.468 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.178+0000 7f5b2c893700 1 Processor -- start 2024-05-14T16:44:17.468 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.178+0000 7f5b2c893700 1 -- start start 2024-05-14T16:44:17.468 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.178+0000 7f5b2c893700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b28108e60 0x7f5b2807f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:17.469 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.178+0000 7f5b2c893700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5b2810d310 con 0x7f5b28108e60 2024-05-14T16:44:17.469 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.179+0000 7f5b2659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b28108e60 0x7f5b2807f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:17.469 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.179+0000 7f5b2659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b28108e60 0x7f5b2807f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52926/0 (socket says 172.21.15.2:52926) 2024-05-14T16:44:17.469 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.179+0000 7f5b2659c700 1 -- 172.21.15.2:0/1544384399 learned_addr learned my addr 172.21.15.2:0/1544384399 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:44:17.469 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.179+0000 7f5b2659c700 1 -- 172.21.15.2:0/1544384399 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b2807f8c0 con 0x7f5b28108e60 2024-05-14T16:44:17.469 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.179+0000 7f5b2659c700 1 --2- 172.21.15.2:0/1544384399 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b28108e60 0x7f5b2807f380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5b180119f0 tx=0x7f5b18011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:17.469 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.179+0000 7f5b177fe700 1 -- 172.21.15.2:0/1544384399 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b180039e0 con 0x7f5b28108e60 2024-05-14T16:44:17.469 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.179+0000 7f5b177fe700 1 -- 172.21.15.2:0/1544384399 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b1801b8f0 con 0x7f5b28108e60 2024-05-14T16:44:17.469 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.179+0000 7f5b2c893700 1 -- 172.21.15.2:0/1544384399 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b2807dab0 con 0x7f5b28108e60 2024-05-14T16:44:17.469 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.179+0000 7f5b177fe700 1 -- 172.21.15.2:0/1544384399 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b180237a0 con 0x7f5b28108e60 2024-05-14T16:44:17.469 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.179+0000 7f5b2c893700 1 -- 172.21.15.2:0/1544384399 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b2807dff0 con 0x7f5b28108e60 2024-05-14T16:44:17.469 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.180+0000 7f5b177fe700 1 -- 172.21.15.2:0/1544384399 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f5b1802c3f0 con 0x7f5b28108e60 2024-05-14T16:44:17.469 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.181+0000 7f5b177fe700 1 --2- 172.21.15.2:0/1544384399 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f5b1003e4d0 0x7f5b10040990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:17.469 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.181+0000 7f5b177fe700 1 -- 172.21.15.2:0/1544384399 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5b18053f00 con 0x7f5b28108e60 2024-05-14T16:44:17.470 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.181+0000 7f5b2c893700 1 -- 172.21.15.2:0/1544384399 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b28064ba0 con 0x7f5b28108e60 2024-05-14T16:44:17.470 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.181+0000 7f5b25d9b700 1 --2- 172.21.15.2:0/1544384399 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f5b1003e4d0 0x7f5b10040990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:17.470 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.181+0000 7f5b25d9b700 1 --2- 172.21.15.2:0/1544384399 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f5b1003e4d0 0x7f5b10040990 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c006fd0 tx=0x7f5b1c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:17.470 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.186+0000 7f5b177fe700 1 -- 172.21.15.2:0/1544384399 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5b1801f030 con 0x7f5b28108e60 2024-05-14T16:44:17.470 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.424+0000 7f5b2c893700 1 -- 172.21.15.2:0/1544384399 --> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f5b2807e1c0 con 0x7f5b1003e4d0 2024-05-14T16:44:17.470 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.671+0000 7f5b177fe700 1 -- 172.21.15.2:0/1544384399 <== mgr.14150 v2:172.21.15.2:6800/1498936028 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f5b2807e1c0 con 0x7f5b1003e4d0 2024-05-14T16:44:17.470 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.673+0000 7f5b2c893700 1 -- 172.21.15.2:0/1544384399 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f5b1003e4d0 msgr2=0x7f5b10040990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:17.470 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.673+0000 7f5b2c893700 1 --2- 172.21.15.2:0/1544384399 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f5b1003e4d0 0x7f5b10040990 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c006fd0 tx=0x7f5b1c006e40 comp rx=0 tx=0).stop 2024-05-14T16:44:17.470 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.673+0000 7f5b2c893700 1 -- 172.21.15.2:0/1544384399 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b28108e60 msgr2=0x7f5b2807f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:17.470 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.673+0000 7f5b2c893700 1 --2- 172.21.15.2:0/1544384399 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b28108e60 0x7f5b2807f380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5b180119f0 tx=0x7f5b18011ad0 comp rx=0 tx=0).stop 2024-05-14T16:44:17.470 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.674+0000 7f5b2c893700 1 -- 172.21.15.2:0/1544384399 shutdown_connections 2024-05-14T16:44:17.470 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.674+0000 7f5b2c893700 1 --2- 172.21.15.2:0/1544384399 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f5b1003e4d0 0x7f5b10040990 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:17.470 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.674+0000 7f5b2c893700 1 --2- 172.21.15.2:0/1544384399 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b28108e60 0x7f5b2807f380 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:17.471 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.674+0000 7f5b2c893700 1 -- 172.21.15.2:0/1544384399 >> 172.21.15.2:0/1544384399 conn(0x7f5b28104450 msgr2=0x7f5b28104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:17.471 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.674+0000 7f5b2c893700 1 -- 172.21.15.2:0/1544384399 shutdown_connections 2024-05-14T16:44:17.471 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:16.674+0000 7f5b2c893700 1 -- 172.21.15.2:0/1544384399 wait complete. 2024-05-14T16:44:17.471 INFO:teuthology.orchestra.run.smithi002.stdout:Fetching dashboard port number... 2024-05-14T16:44:18.868 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 8443 2024-05-14T16:44:18.868 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:17.916+0000 7fe9c874c700 1 Processor -- start 2024-05-14T16:44:18.868 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:17.916+0000 7fe9c874c700 1 -- start start 2024-05-14T16:44:18.868 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:17.917+0000 7fe9c874c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9c0107d90 0x7fe9c01081b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:18.868 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:17.917+0000 7fe9c874c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c0108780 con 0x7fe9c0107d90 2024-05-14T16:44:18.868 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:17.917+0000 7fe9c64e8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9c0107d90 0x7fe9c01081b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:18.868 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:17.917+0000 7fe9c64e8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9c0107d90 0x7fe9c01081b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52938/0 (socket says 172.21.15.2:52938) 2024-05-14T16:44:18.868 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:17.917+0000 7fe9c64e8700 1 -- 172.21.15.2:0/510373719 learned_addr learned my addr 172.21.15.2:0/510373719 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:44:18.868 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:17.917+0000 7fe9c64e8700 1 -- 172.21.15.2:0/510373719 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9c0108f90 con 0x7fe9c0107d90 2024-05-14T16:44:18.868 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:17.917+0000 7fe9c64e8700 1 --2- 172.21.15.2:0/510373719 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9c0107d90 0x7fe9c01081b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc0098a0 tx=0x7fe9bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=971895e841d3b53d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:18.868 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:17.917+0000 7fe9c54e6700 1 -- 172.21.15.2:0/510373719 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9bc004160 con 0x7fe9c0107d90 2024-05-14T16:44:18.868 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:17.917+0000 7fe9c54e6700 1 -- 172.21.15.2:0/510373719 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9bc004300 con 0x7fe9c0107d90 2024-05-14T16:44:18.869 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:17.918+0000 7fe9c54e6700 1 -- 172.21.15.2:0/510373719 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9bc011a20 con 0x7fe9c0107d90 2024-05-14T16:44:18.869 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:17.918+0000 7fe9c874c700 1 -- 172.21.15.2:0/510373719 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9c0107d90 msgr2=0x7fe9c01081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:18.869 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:17.918+0000 7fe9c874c700 1 --2- 172.21.15.2:0/510373719 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9c0107d90 0x7fe9c01081b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc0098a0 tx=0x7fe9bc009bb0 comp rx=0 tx=0).stop 2024-05-14T16:44:18.869 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:17.918+0000 7fe9c874c700 1 -- 172.21.15.2:0/510373719 shutdown_connections 2024-05-14T16:44:18.869 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:17.918+0000 7fe9c874c700 1 --2- 172.21.15.2:0/510373719 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9c0107d90 0x7fe9c01081b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:18.869 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:17.918+0000 7fe9c874c700 1 -- 172.21.15.2:0/510373719 >> 172.21.15.2:0/510373719 conn(0x7fe9c0103330 msgr2=0x7fe9c0105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:18.869 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:17.918+0000 7fe9c874c700 1 -- 172.21.15.2:0/510373719 shutdown_connections 2024-05-14T16:44:18.869 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:17.918+0000 7fe9c874c700 1 -- 172.21.15.2:0/510373719 wait complete. 2024-05-14T16:44:18.869 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:17.919+0000 7fe9c874c700 1 Processor -- start 2024-05-14T16:44:18.869 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:17.919+0000 7fe9c874c700 1 -- start start 2024-05-14T16:44:18.869 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:17.919+0000 7fe9c874c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9c0107d90 0x7fe9c0192f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:18.869 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:17.919+0000 7fe9c874c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c010c340 con 0x7fe9c0107d90 2024-05-14T16:44:18.869 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:17.919+0000 7fe9c64e8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9c0107d90 0x7fe9c0192f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:18.869 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:17.919+0000 7fe9c64e8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9c0107d90 0x7fe9c0192f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52952/0 (socket says 172.21.15.2:52952) 2024-05-14T16:44:18.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:17.919+0000 7fe9c64e8700 1 -- 172.21.15.2:0/1731726190 learned_addr learned my addr 172.21.15.2:0/1731726190 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:44:18.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:17.919+0000 7fe9c64e8700 1 -- 172.21.15.2:0/1731726190 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9c0193440 con 0x7fe9c0107d90 2024-05-14T16:44:18.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:17.919+0000 7fe9c64e8700 1 --2- 172.21.15.2:0/1731726190 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9c0107d90 0x7fe9c0192f00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc0119f0 tx=0x7fe9bc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:18.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:17.920+0000 7fe9b77fe700 1 -- 172.21.15.2:0/1731726190 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9bc0039e0 con 0x7fe9c0107d90 2024-05-14T16:44:18.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:17.920+0000 7fe9b77fe700 1 -- 172.21.15.2:0/1731726190 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9bc01b8f0 con 0x7fe9c0107d90 2024-05-14T16:44:18.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:17.920+0000 7fe9c874c700 1 -- 172.21.15.2:0/1731726190 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9c0191630 con 0x7fe9c0107d90 2024-05-14T16:44:18.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:17.920+0000 7fe9b77fe700 1 -- 172.21.15.2:0/1731726190 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9bc0237a0 con 0x7fe9c0107d90 2024-05-14T16:44:18.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:17.920+0000 7fe9c874c700 1 -- 172.21.15.2:0/1731726190 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9c0191b70 con 0x7fe9c0107d90 2024-05-14T16:44:18.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:17.921+0000 7fe9b77fe700 1 -- 172.21.15.2:0/1731726190 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7fe9bc01b440 con 0x7fe9c0107d90 2024-05-14T16:44:18.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:17.921+0000 7fe9c874c700 1 -- 172.21.15.2:0/1731726190 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9c0064ba0 con 0x7fe9c0107d90 2024-05-14T16:44:18.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:17.921+0000 7fe9b77fe700 1 --2- 172.21.15.2:0/1731726190 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fe9ac03e4d0 0x7fe9ac040990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:18.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:17.921+0000 7fe9b77fe700 1 -- 172.21.15.2:0/1731726190 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe9bc053ea0 con 0x7fe9c0107d90 2024-05-14T16:44:18.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:17.921+0000 7fe9c5ce7700 1 --2- 172.21.15.2:0/1731726190 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fe9ac03e4d0 0x7fe9ac040990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:18.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:17.922+0000 7fe9c5ce7700 1 --2- 172.21.15.2:0/1731726190 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fe9ac03e4d0 0x7fe9ac040990 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0006fd0 tx=0x7fe9b0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:18.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:17.926+0000 7fe9b77fe700 1 -- 172.21.15.2:0/1731726190 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe9bc01f030 con 0x7fe9c0107d90 2024-05-14T16:44:18.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:18.056+0000 7fe9b77fe700 1 -- 172.21.15.2:0/1731726190 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7fe9bc00b3e0 con 0x7fe9c0107d90 2024-05-14T16:44:18.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:18.145+0000 7fe9c874c700 1 -- 172.21.15.2:0/1731726190 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fe9c0196bf0 con 0x7fe9c0107d90 2024-05-14T16:44:18.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:18.146+0000 7fe9b77fe700 1 -- 172.21.15.2:0/1731726190 <== mon.0 v2:172.21.15.2:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fe9bc021300 con 0x7fe9c0107d90 2024-05-14T16:44:18.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:18.148+0000 7fe9c874c700 1 -- 172.21.15.2:0/1731726190 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fe9ac03e4d0 msgr2=0x7fe9ac040990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:18.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:18.148+0000 7fe9c874c700 1 --2- 172.21.15.2:0/1731726190 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fe9ac03e4d0 0x7fe9ac040990 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0006fd0 tx=0x7fe9b0006e40 comp rx=0 tx=0).stop 2024-05-14T16:44:18.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:18.148+0000 7fe9c874c700 1 -- 172.21.15.2:0/1731726190 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9c0107d90 msgr2=0x7fe9c0192f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:18.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:18.148+0000 7fe9c874c700 1 --2- 172.21.15.2:0/1731726190 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9c0107d90 0x7fe9c0192f00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc0119f0 tx=0x7fe9bc011ad0 comp rx=0 tx=0).stop 2024-05-14T16:44:18.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:18.148+0000 7fe9c874c700 1 -- 172.21.15.2:0/1731726190 shutdown_connections 2024-05-14T16:44:18.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:18.148+0000 7fe9c874c700 1 --2- 172.21.15.2:0/1731726190 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fe9ac03e4d0 0x7fe9ac040990 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:18.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:18.148+0000 7fe9c874c700 1 --2- 172.21.15.2:0/1731726190 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9c0107d90 0x7fe9c0192f00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:18.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:18.148+0000 7fe9c874c700 1 -- 172.21.15.2:0/1731726190 >> 172.21.15.2:0/1731726190 conn(0x7fe9c0103330 msgr2=0x7fe9c01041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:18.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:18.148+0000 7fe9c874c700 1 -- 172.21.15.2:0/1731726190 shutdown_connections 2024-05-14T16:44:18.872 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:18.148+0000 7fe9c874c700 1 -- 172.21.15.2:0/1731726190 wait complete. 2024-05-14T16:44:18.875 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-14T16:44:18.875 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout disabled 2024-05-14T16:44:18.884 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-14T16:44:18.884 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2024-05-14T16:44:18.884 INFO:teuthology.orchestra.run.smithi002.stdout:firewalld.service is not enabled 2024-05-14T16:44:18.884 INFO:teuthology.orchestra.run.smithi002.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-14T16:44:18.886 INFO:teuthology.orchestra.run.smithi002.stdout:Ceph Dashboard is now available at: 2024-05-14T16:44:18.886 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:44:18.886 INFO:teuthology.orchestra.run.smithi002.stdout: URL: https://smithi002.front.sepia.ceph.com:8443/ 2024-05-14T16:44:18.886 INFO:teuthology.orchestra.run.smithi002.stdout: User: admin 2024-05-14T16:44:18.886 INFO:teuthology.orchestra.run.smithi002.stdout: Password: fthmnsxvfk 2024-05-14T16:44:18.886 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:44:18.887 INFO:teuthology.orchestra.run.smithi002.stdout:Saving cluster configuration to /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/config directory 2024-05-14T16:44:18.887 INFO:teuthology.orchestra.run.smithi002.stdout:Enabling autotune for osd_memory_target 2024-05-14T16:44:19.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:19 smithi002 ceph-mon[31538]: mgrmap e13: x(active, since 6s) 2024-05-14T16:44:19.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:19 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/1731726190' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-14T16:44:20.215 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.400+0000 7fcf77555700 1 Processor -- start 2024-05-14T16:44:20.215 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.401+0000 7fcf77555700 1 -- start start 2024-05-14T16:44:20.215 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.401+0000 7fcf77555700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf7007f5a0 0x7fcf7007db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:20.215 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.401+0000 7fcf77555700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7007e050 con 0x7fcf7007f5a0 2024-05-14T16:44:20.215 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.401+0000 7fcf752f1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf7007f5a0 0x7fcf7007db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:20.216 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.401+0000 7fcf752f1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf7007f5a0 0x7fcf7007db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52960/0 (socket says 172.21.15.2:52960) 2024-05-14T16:44:20.216 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.401+0000 7fcf752f1700 1 -- 172.21.15.2:0/2309408512 learned_addr learned my addr 172.21.15.2:0/2309408512 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:44:20.216 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.401+0000 7fcf752f1700 1 -- 172.21.15.2:0/2309408512 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf7007e1d0 con 0x7fcf7007f5a0 2024-05-14T16:44:20.216 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.402+0000 7fcf752f1700 1 --2- 172.21.15.2:0/2309408512 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf7007f5a0 0x7fcf7007db10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c0098a0 tx=0x7fcf6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f20ac5e97a78694 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:20.216 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.402+0000 7fcf67fff700 1 -- 172.21.15.2:0/2309408512 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf6c004160 con 0x7fcf7007f5a0 2024-05-14T16:44:20.216 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.402+0000 7fcf67fff700 1 -- 172.21.15.2:0/2309408512 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf6c004300 con 0x7fcf7007f5a0 2024-05-14T16:44:20.216 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.402+0000 7fcf67fff700 1 -- 172.21.15.2:0/2309408512 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf6c011a20 con 0x7fcf7007f5a0 2024-05-14T16:44:20.216 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.402+0000 7fcf77555700 1 -- 172.21.15.2:0/2309408512 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf7007f5a0 msgr2=0x7fcf7007db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:20.216 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.402+0000 7fcf77555700 1 --2- 172.21.15.2:0/2309408512 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf7007f5a0 0x7fcf7007db10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c0098a0 tx=0x7fcf6c009bb0 comp rx=0 tx=0).stop 2024-05-14T16:44:20.216 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.402+0000 7fcf77555700 1 -- 172.21.15.2:0/2309408512 shutdown_connections 2024-05-14T16:44:20.216 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.402+0000 7fcf77555700 1 --2- 172.21.15.2:0/2309408512 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf7007f5a0 0x7fcf7007db10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:20.216 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.402+0000 7fcf77555700 1 -- 172.21.15.2:0/2309408512 >> 172.21.15.2:0/2309408512 conn(0x7fcf70104520 msgr2=0x7fcf70106940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:20.216 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.403+0000 7fcf77555700 1 -- 172.21.15.2:0/2309408512 shutdown_connections 2024-05-14T16:44:20.216 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.403+0000 7fcf77555700 1 -- 172.21.15.2:0/2309408512 wait complete. 2024-05-14T16:44:20.217 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.403+0000 7fcf77555700 1 Processor -- start 2024-05-14T16:44:20.217 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.403+0000 7fcf77555700 1 -- start start 2024-05-14T16:44:20.217 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.403+0000 7fcf77555700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf7007f5a0 0x7fcf7019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:20.217 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.403+0000 7fcf77555700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7010f350 con 0x7fcf7007f5a0 2024-05-14T16:44:20.217 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.404+0000 7fcf752f1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf7007f5a0 0x7fcf7019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:20.217 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.404+0000 7fcf752f1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf7007f5a0 0x7fcf7019efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52962/0 (socket says 172.21.15.2:52962) 2024-05-14T16:44:20.217 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.404+0000 7fcf752f1700 1 -- 172.21.15.2:0/2716605017 learned_addr learned my addr 172.21.15.2:0/2716605017 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:44:20.217 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.404+0000 7fcf752f1700 1 -- 172.21.15.2:0/2716605017 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf7019f4e0 con 0x7fcf7007f5a0 2024-05-14T16:44:20.217 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.404+0000 7fcf752f1700 1 --2- 172.21.15.2:0/2716605017 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf7007f5a0 0x7fcf7019efa0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c0116e0 tx=0x7fcf6c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:20.217 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.404+0000 7fcf667fc700 1 -- 172.21.15.2:0/2716605017 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf6c003b20 con 0x7fcf7007f5a0 2024-05-14T16:44:20.217 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.404+0000 7fcf667fc700 1 -- 172.21.15.2:0/2716605017 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf6c01b8f0 con 0x7fcf7007f5a0 2024-05-14T16:44:20.217 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.404+0000 7fcf77555700 1 -- 172.21.15.2:0/2716605017 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf7019f770 con 0x7fcf7007f5a0 2024-05-14T16:44:20.217 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.404+0000 7fcf667fc700 1 -- 172.21.15.2:0/2716605017 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf6c0238c0 con 0x7fcf7007f5a0 2024-05-14T16:44:20.218 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.404+0000 7fcf77555700 1 -- 172.21.15.2:0/2716605017 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf701a2460 con 0x7fcf7007f5a0 2024-05-14T16:44:20.218 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.405+0000 7fcf667fc700 1 -- 172.21.15.2:0/2716605017 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7fcf6c023a60 con 0x7fcf7007f5a0 2024-05-14T16:44:20.218 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.405+0000 7fcf77555700 1 -- 172.21.15.2:0/2716605017 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf70064ba0 con 0x7fcf7007f5a0 2024-05-14T16:44:20.218 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.405+0000 7fcf667fc700 1 --2- 172.21.15.2:0/2716605017 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fcf5c03e570 0x7fcf5c040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:20.218 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.405+0000 7fcf667fc700 1 -- 172.21.15.2:0/2716605017 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcf6c054340 con 0x7fcf7007f5a0 2024-05-14T16:44:20.218 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.406+0000 7fcf74af0700 1 --2- 172.21.15.2:0/2716605017 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fcf5c03e570 0x7fcf5c040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:20.218 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.406+0000 7fcf74af0700 1 --2- 172.21.15.2:0/2716605017 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fcf5c03e570 0x7fcf5c040a30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcf60006fd0 tx=0x7fcf60006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:20.218 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.410+0000 7fcf667fc700 1 -- 172.21.15.2:0/2716605017 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcf6c01f030 con 0x7fcf7007f5a0 2024-05-14T16:44:20.218 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.624+0000 7fcf77555700 1 -- 172.21.15.2:0/2716605017 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fcf701a2a80 con 0x7fcf7007f5a0 2024-05-14T16:44:20.218 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.625+0000 7fcf667fc700 1 -- 172.21.15.2:0/2716605017 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fcf6c00b3e0 con 0x7fcf7007f5a0 2024-05-14T16:44:20.219 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.627+0000 7fcf77555700 1 -- 172.21.15.2:0/2716605017 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fcf5c03e570 msgr2=0x7fcf5c040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:20.219 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.627+0000 7fcf77555700 1 --2- 172.21.15.2:0/2716605017 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fcf5c03e570 0x7fcf5c040a30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcf60006fd0 tx=0x7fcf60006e40 comp rx=0 tx=0).stop 2024-05-14T16:44:20.219 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.627+0000 7fcf77555700 1 -- 172.21.15.2:0/2716605017 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf7007f5a0 msgr2=0x7fcf7019efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:20.219 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.627+0000 7fcf77555700 1 --2- 172.21.15.2:0/2716605017 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf7007f5a0 0x7fcf7019efa0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c0116e0 tx=0x7fcf6c011aa0 comp rx=0 tx=0).stop 2024-05-14T16:44:20.219 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.627+0000 7fcf77555700 1 -- 172.21.15.2:0/2716605017 shutdown_connections 2024-05-14T16:44:20.219 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.627+0000 7fcf77555700 1 --2- 172.21.15.2:0/2716605017 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fcf5c03e570 0x7fcf5c040a30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:20.219 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.627+0000 7fcf77555700 1 --2- 172.21.15.2:0/2716605017 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf7007f5a0 0x7fcf7019efa0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:20.219 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.627+0000 7fcf77555700 1 -- 172.21.15.2:0/2716605017 >> 172.21.15.2:0/2716605017 conn(0x7fcf70104520 msgr2=0x7fcf70105080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:20.219 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.627+0000 7fcf77555700 1 -- 172.21.15.2:0/2716605017 shutdown_connections 2024-05-14T16:44:20.219 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:19.627+0000 7fcf77555700 1 -- 172.21.15.2:0/2716605017 wait complete. 2024-05-14T16:44:21.556 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.642+0000 7f27ebeb7700 1 Processor -- start 2024-05-14T16:44:21.556 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.642+0000 7f27ebeb7700 1 -- start start 2024-05-14T16:44:21.556 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.642+0000 7f27ebeb7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27e4108e80 0x7f27e41092a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:21.556 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.642+0000 7f27ebeb7700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f27e41097e0 con 0x7f27e4108e80 2024-05-14T16:44:21.556 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.643+0000 7f27e9c53700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27e4108e80 0x7f27e41092a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:21.556 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.643+0000 7f27e9c53700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27e4108e80 0x7f27e41092a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52974/0 (socket says 172.21.15.2:52974) 2024-05-14T16:44:21.556 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.643+0000 7f27e9c53700 1 -- 172.21.15.2:0/1855216473 learned_addr learned my addr 172.21.15.2:0/1855216473 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:44:21.556 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.643+0000 7f27e9c53700 1 -- 172.21.15.2:0/1855216473 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e4109fe0 con 0x7f27e4108e80 2024-05-14T16:44:21.556 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.643+0000 7f27e9c53700 1 --2- 172.21.15.2:0/1855216473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27e4108e80 0x7f27e41092a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f27d4009e80 tx=0x7f27d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=340fc97fa0ad360e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:21.557 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.643+0000 7f27e8c51700 1 -- 172.21.15.2:0/1855216473 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27d4017070 con 0x7f27e4108e80 2024-05-14T16:44:21.557 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.643+0000 7f27e8c51700 1 -- 172.21.15.2:0/1855216473 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27d400b8c0 con 0x7f27e4108e80 2024-05-14T16:44:21.557 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.643+0000 7f27e8c51700 1 -- 172.21.15.2:0/1855216473 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27d4004490 con 0x7f27e4108e80 2024-05-14T16:44:21.557 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.644+0000 7f27ebeb7700 1 -- 172.21.15.2:0/1855216473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27e4108e80 msgr2=0x7f27e41092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:21.557 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.644+0000 7f27ebeb7700 1 --2- 172.21.15.2:0/1855216473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27e4108e80 0x7f27e41092a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f27d4009e80 tx=0x7f27d400b330 comp rx=0 tx=0).stop 2024-05-14T16:44:21.557 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.644+0000 7f27ebeb7700 1 -- 172.21.15.2:0/1855216473 shutdown_connections 2024-05-14T16:44:21.557 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.644+0000 7f27ebeb7700 1 --2- 172.21.15.2:0/1855216473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27e4108e80 0x7f27e41092a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:21.557 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.644+0000 7f27ebeb7700 1 -- 172.21.15.2:0/1855216473 >> 172.21.15.2:0/1855216473 conn(0x7f27e4104480 msgr2=0x7f27e41068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:21.557 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.644+0000 7f27ebeb7700 1 -- 172.21.15.2:0/1855216473 shutdown_connections 2024-05-14T16:44:21.557 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.644+0000 7f27ebeb7700 1 -- 172.21.15.2:0/1855216473 wait complete. 2024-05-14T16:44:21.557 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.645+0000 7f27ebeb7700 1 Processor -- start 2024-05-14T16:44:21.557 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.645+0000 7f27ebeb7700 1 -- start start 2024-05-14T16:44:21.557 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.645+0000 7f27ebeb7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27e4108e80 0x7f27e407f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:21.558 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.645+0000 7f27ebeb7700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f27e410d420 con 0x7f27e4108e80 2024-05-14T16:44:21.558 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.645+0000 7f27e9c53700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27e4108e80 0x7f27e407f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:21.558 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.645+0000 7f27e9c53700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27e4108e80 0x7f27e407f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52982/0 (socket says 172.21.15.2:52982) 2024-05-14T16:44:21.558 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.645+0000 7f27e9c53700 1 -- 172.21.15.2:0/668914356 learned_addr learned my addr 172.21.15.2:0/668914356 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:44:21.558 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.645+0000 7f27e9c53700 1 -- 172.21.15.2:0/668914356 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e407f8c0 con 0x7f27e4108e80 2024-05-14T16:44:21.558 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.645+0000 7f27e9c53700 1 --2- 172.21.15.2:0/668914356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27e4108e80 0x7f27e407f380 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f27d4015040 tx=0x7f27d400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:21.558 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.646+0000 7f27e2ffd700 1 -- 172.21.15.2:0/668914356 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27d4017040 con 0x7f27e4108e80 2024-05-14T16:44:21.558 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.646+0000 7f27ebeb7700 1 -- 172.21.15.2:0/668914356 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27e407dab0 con 0x7f27e4108e80 2024-05-14T16:44:21.558 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.646+0000 7f27e2ffd700 1 -- 172.21.15.2:0/668914356 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27d4022070 con 0x7f27e4108e80 2024-05-14T16:44:21.558 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.646+0000 7f27e2ffd700 1 -- 172.21.15.2:0/668914356 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27d4012430 con 0x7f27e4108e80 2024-05-14T16:44:21.558 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.646+0000 7f27ebeb7700 1 -- 172.21.15.2:0/668914356 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27e407dff0 con 0x7f27e4108e80 2024-05-14T16:44:21.558 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.647+0000 7f27e2ffd700 1 -- 172.21.15.2:0/668914356 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f27d401d440 con 0x7f27e4108e80 2024-05-14T16:44:21.558 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.647+0000 7f27ebeb7700 1 -- 172.21.15.2:0/668914356 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27e410d5a0 con 0x7f27e4108e80 2024-05-14T16:44:21.558 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.647+0000 7f27e2ffd700 1 --2- 172.21.15.2:0/668914356 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f27cc03e560 0x7f27cc040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:21.559 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.647+0000 7f27e2ffd700 1 -- 172.21.15.2:0/668914356 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f27d400f050 con 0x7f27e4108e80 2024-05-14T16:44:21.559 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.647+0000 7f27e9452700 1 --2- 172.21.15.2:0/668914356 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f27cc03e560 0x7f27cc040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:21.559 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.648+0000 7f27e9452700 1 --2- 172.21.15.2:0/668914356 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f27cc03e560 0x7f27cc040a20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f27d8006fd0 tx=0x7f27d8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:21.559 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.652+0000 7f27e2ffd700 1 -- 172.21.15.2:0/668914356 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f27d401b3f0 con 0x7f27e4108e80 2024-05-14T16:44:21.559 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.983+0000 7f27ebeb7700 1 -- 172.21.15.2:0/668914356 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f27e4064ba0 con 0x7f27e4108e80 2024-05-14T16:44:21.559 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.986+0000 7f27e2ffd700 1 -- 172.21.15.2:0/668914356 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f27d4027d10 con 0x7f27e4108e80 2024-05-14T16:44:21.559 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.988+0000 7f27ebeb7700 1 -- 172.21.15.2:0/668914356 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f27cc03e560 msgr2=0x7f27cc040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:21.559 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.988+0000 7f27ebeb7700 1 --2- 172.21.15.2:0/668914356 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f27cc03e560 0x7f27cc040a20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f27d8006fd0 tx=0x7f27d8006e40 comp rx=0 tx=0).stop 2024-05-14T16:44:21.559 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.988+0000 7f27ebeb7700 1 -- 172.21.15.2:0/668914356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27e4108e80 msgr2=0x7f27e407f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:21.559 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.988+0000 7f27ebeb7700 1 --2- 172.21.15.2:0/668914356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27e4108e80 0x7f27e407f380 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f27d4015040 tx=0x7f27d400bd50 comp rx=0 tx=0).stop 2024-05-14T16:44:21.559 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.988+0000 7f27ebeb7700 1 -- 172.21.15.2:0/668914356 shutdown_connections 2024-05-14T16:44:21.559 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.988+0000 7f27ebeb7700 1 --2- 172.21.15.2:0/668914356 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f27cc03e560 0x7f27cc040a20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:21.559 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.988+0000 7f27ebeb7700 1 --2- 172.21.15.2:0/668914356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27e4108e80 0x7f27e407f380 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:21.560 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.988+0000 7f27ebeb7700 1 -- 172.21.15.2:0/668914356 >> 172.21.15.2:0/668914356 conn(0x7f27e4104480 msgr2=0x7f27e4104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:21.560 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.988+0000 7f27ebeb7700 1 -- 172.21.15.2:0/668914356 shutdown_connections 2024-05-14T16:44:21.560 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2024-05-14T16:44:20.988+0000 7f27ebeb7700 1 -- 172.21.15.2:0/668914356 wait complete. 2024-05-14T16:44:21.560 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-14T16:44:21.560 INFO:teuthology.orchestra.run.smithi002.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-14T16:44:21.560 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:44:21.560 INFO:teuthology.orchestra.run.smithi002.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-14T16:44:21.560 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:44:21.560 INFO:teuthology.orchestra.run.smithi002.stdout:Or, if you are only running a single cluster on this host: 2024-05-14T16:44:21.560 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:44:21.560 INFO:teuthology.orchestra.run.smithi002.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-14T16:44:21.560 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:44:21.560 INFO:teuthology.orchestra.run.smithi002.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-14T16:44:21.560 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:44:21.560 INFO:teuthology.orchestra.run.smithi002.stdout: ceph telemetry on 2024-05-14T16:44:21.561 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:44:21.561 INFO:teuthology.orchestra.run.smithi002.stdout:For more information see: 2024-05-14T16:44:21.561 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:44:21.561 INFO:teuthology.orchestra.run.smithi002.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-14T16:44:21.561 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:44:21.561 INFO:teuthology.orchestra.run.smithi002.stdout:Bootstrap complete. 2024-05-14T16:44:21.601 INFO:tasks.cephadm:Fetching config... 2024-05-14T16:44:21.602 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T16:44:21.602 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-14T16:44:21.620 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-14T16:44:21.620 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T16:44:21.620 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-14T16:44:21.676 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-14T16:44:21.676 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T16:44:21.676 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-14T16:44:21.743 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-14T16:44:21.743 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T16:44:21.744 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-14T16:44:21.801 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-14T16:44:21.801 DEBUG:teuthology.orchestra.run.smithi002:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCkup2e5TEAzUYYjWhK4e7NNumGHuwF2oCFL60Gfoo978sGM8XvjjTTNuT5etFgmr6VR95GKw2dADmE8orO+i3TTWaVXfvZVtu2/QZNi16B5nog286LdDWZnXt6qjYifDmnHrprGST6xrw+i0Cp5KmQ6+X5bYiQqRWdVrd8O1m+UWpat9WoF3oLj9YfCfklA1aD1kcsWdZDZVj8NcbVQBoHDZV+0ji5ljkWv6poAfvw0Z/tUnxKKBxxldsQbhSQltLKJ2uxKby+9hQJG9J6H9ofPUfkCk/3teiSzHystFKVegeMknYc8WOzVdBLoqOc8at6sO1OMRhRYfmn1KvFues54085hL742cr0zBEV9ygTX7y/vFnJwZRYRV4s+iA2M/mPLI/Ck2nxWqbLcnDqsc1uGtGGt0EBAdcaxTcnrYIhKHcD3zulwOK3daxJMut13h0O45yKWHXMuea5TWky2YKzUAJXUia6zq3X5l3iYqAv3w05WV0gmqO/lrf2r3OeRXc= ceph-ed40717a-1210-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-14T16:44:21.876 INFO:teuthology.orchestra.run.smithi002.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCkup2e5TEAzUYYjWhK4e7NNumGHuwF2oCFL60Gfoo978sGM8XvjjTTNuT5etFgmr6VR95GKw2dADmE8orO+i3TTWaVXfvZVtu2/QZNi16B5nog286LdDWZnXt6qjYifDmnHrprGST6xrw+i0Cp5KmQ6+X5bYiQqRWdVrd8O1m+UWpat9WoF3oLj9YfCfklA1aD1kcsWdZDZVj8NcbVQBoHDZV+0ji5ljkWv6poAfvw0Z/tUnxKKBxxldsQbhSQltLKJ2uxKby+9hQJG9J6H9ofPUfkCk/3teiSzHystFKVegeMknYc8WOzVdBLoqOc8at6sO1OMRhRYfmn1KvFues54085hL742cr0zBEV9ygTX7y/vFnJwZRYRV4s+iA2M/mPLI/Ck2nxWqbLcnDqsc1uGtGGt0EBAdcaxTcnrYIhKHcD3zulwOK3daxJMut13h0O45yKWHXMuea5TWky2YKzUAJXUia6zq3X5l3iYqAv3w05WV0gmqO/lrf2r3OeRXc= ceph-ed40717a-1210-11ef-bc98-c7b262605968 2024-05-14T16:44:21.889 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCkup2e5TEAzUYYjWhK4e7NNumGHuwF2oCFL60Gfoo978sGM8XvjjTTNuT5etFgmr6VR95GKw2dADmE8orO+i3TTWaVXfvZVtu2/QZNi16B5nog286LdDWZnXt6qjYifDmnHrprGST6xrw+i0Cp5KmQ6+X5bYiQqRWdVrd8O1m+UWpat9WoF3oLj9YfCfklA1aD1kcsWdZDZVj8NcbVQBoHDZV+0ji5ljkWv6poAfvw0Z/tUnxKKBxxldsQbhSQltLKJ2uxKby+9hQJG9J6H9ofPUfkCk/3teiSzHystFKVegeMknYc8WOzVdBLoqOc8at6sO1OMRhRYfmn1KvFues54085hL742cr0zBEV9ygTX7y/vFnJwZRYRV4s+iA2M/mPLI/Ck2nxWqbLcnDqsc1uGtGGt0EBAdcaxTcnrYIhKHcD3zulwOK3daxJMut13h0O45yKWHXMuea5TWky2YKzUAJXUia6zq3X5l3iYqAv3w05WV0gmqO/lrf2r3OeRXc= ceph-ed40717a-1210-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-14T16:44:21.927 INFO:teuthology.orchestra.run.smithi123.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCkup2e5TEAzUYYjWhK4e7NNumGHuwF2oCFL60Gfoo978sGM8XvjjTTNuT5etFgmr6VR95GKw2dADmE8orO+i3TTWaVXfvZVtu2/QZNi16B5nog286LdDWZnXt6qjYifDmnHrprGST6xrw+i0Cp5KmQ6+X5bYiQqRWdVrd8O1m+UWpat9WoF3oLj9YfCfklA1aD1kcsWdZDZVj8NcbVQBoHDZV+0ji5ljkWv6poAfvw0Z/tUnxKKBxxldsQbhSQltLKJ2uxKby+9hQJG9J6H9ofPUfkCk/3teiSzHystFKVegeMknYc8WOzVdBLoqOc8at6sO1OMRhRYfmn1KvFues54085hL742cr0zBEV9ygTX7y/vFnJwZRYRV4s+iA2M/mPLI/Ck2nxWqbLcnDqsc1uGtGGt0EBAdcaxTcnrYIhKHcD3zulwOK3daxJMut13h0O45yKWHXMuea5TWky2YKzUAJXUia6zq3X5l3iYqAv3w05WV0gmqO/lrf2r3OeRXc= ceph-ed40717a-1210-11ef-bc98-c7b262605968 2024-05-14T16:44:21.939 DEBUG:teuthology.orchestra.run.smithi151:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCkup2e5TEAzUYYjWhK4e7NNumGHuwF2oCFL60Gfoo978sGM8XvjjTTNuT5etFgmr6VR95GKw2dADmE8orO+i3TTWaVXfvZVtu2/QZNi16B5nog286LdDWZnXt6qjYifDmnHrprGST6xrw+i0Cp5KmQ6+X5bYiQqRWdVrd8O1m+UWpat9WoF3oLj9YfCfklA1aD1kcsWdZDZVj8NcbVQBoHDZV+0ji5ljkWv6poAfvw0Z/tUnxKKBxxldsQbhSQltLKJ2uxKby+9hQJG9J6H9ofPUfkCk/3teiSzHystFKVegeMknYc8WOzVdBLoqOc8at6sO1OMRhRYfmn1KvFues54085hL742cr0zBEV9ygTX7y/vFnJwZRYRV4s+iA2M/mPLI/Ck2nxWqbLcnDqsc1uGtGGt0EBAdcaxTcnrYIhKHcD3zulwOK3daxJMut13h0O45yKWHXMuea5TWky2YKzUAJXUia6zq3X5l3iYqAv3w05WV0gmqO/lrf2r3OeRXc= ceph-ed40717a-1210-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-14T16:44:21.981 INFO:teuthology.orchestra.run.smithi151.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCkup2e5TEAzUYYjWhK4e7NNumGHuwF2oCFL60Gfoo978sGM8XvjjTTNuT5etFgmr6VR95GKw2dADmE8orO+i3TTWaVXfvZVtu2/QZNi16B5nog286LdDWZnXt6qjYifDmnHrprGST6xrw+i0Cp5KmQ6+X5bYiQqRWdVrd8O1m+UWpat9WoF3oLj9YfCfklA1aD1kcsWdZDZVj8NcbVQBoHDZV+0ji5ljkWv6poAfvw0Z/tUnxKKBxxldsQbhSQltLKJ2uxKby+9hQJG9J6H9ofPUfkCk/3teiSzHystFKVegeMknYc8WOzVdBLoqOc8at6sO1OMRhRYfmn1KvFues54085hL742cr0zBEV9ygTX7y/vFnJwZRYRV4s+iA2M/mPLI/Ck2nxWqbLcnDqsc1uGtGGt0EBAdcaxTcnrYIhKHcD3zulwOK3daxJMut13h0O45yKWHXMuea5TWky2YKzUAJXUia6zq3X5l3iYqAv3w05WV0gmqO/lrf2r3OeRXc= ceph-ed40717a-1210-11ef-bc98-c7b262605968 2024-05-14T16:44:22.002 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-14T16:44:22.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:21 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/668914356' entity='client.admin' 2024-05-14T16:44:22.250 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:44:24.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:24.830+0000 7eff34558700 1 -- 172.21.15.2:0/1138463818 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff2c102fc0 msgr2=0x7eff2c1033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:24.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:24.830+0000 7eff34558700 1 --2- 172.21.15.2:0/1138463818 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff2c102fc0 0x7eff2c1033e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7eff28009910 tx=0x7eff28009c20 comp rx=0 tx=0).stop 2024-05-14T16:44:24.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:24.830+0000 7eff34558700 1 -- 172.21.15.2:0/1138463818 shutdown_connections 2024-05-14T16:44:24.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:24.830+0000 7eff34558700 1 --2- 172.21.15.2:0/1138463818 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff2c102fc0 0x7eff2c1033e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:24.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:24.830+0000 7eff34558700 1 -- 172.21.15.2:0/1138463818 >> 172.21.15.2:0/1138463818 conn(0x7eff2c0fe560 msgr2=0x7eff2c1009a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:24.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:24.830+0000 7eff34558700 1 -- 172.21.15.2:0/1138463818 shutdown_connections 2024-05-14T16:44:24.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:24.830+0000 7eff34558700 1 -- 172.21.15.2:0/1138463818 wait complete. 2024-05-14T16:44:24.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:24.831+0000 7eff34558700 1 Processor -- start 2024-05-14T16:44:24.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:24.831+0000 7eff34558700 1 -- start start 2024-05-14T16:44:24.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:24.831+0000 7eff34558700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff2c102fc0 0x7eff2c0771b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:24.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:24.831+0000 7eff34558700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c108f00 con 0x7eff2c102fc0 2024-05-14T16:44:24.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:24.831+0000 7eff322f4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff2c102fc0 0x7eff2c0771b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:24.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:24.832+0000 7eff322f4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff2c102fc0 0x7eff2c0771b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36312/0 (socket says 172.21.15.2:36312) 2024-05-14T16:44:24.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:24.832+0000 7eff322f4700 1 -- 172.21.15.2:0/492312902 learned_addr learned my addr 172.21.15.2:0/492312902 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:44:24.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:24.832+0000 7eff322f4700 1 -- 172.21.15.2:0/492312902 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff2c0776f0 con 0x7eff2c102fc0 2024-05-14T16:44:24.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:24.832+0000 7eff322f4700 1 --2- 172.21.15.2:0/492312902 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff2c102fc0 0x7eff2c0771b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7eff28005950 tx=0x7eff28004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:24.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:24.832+0000 7eff237fe700 1 -- 172.21.15.2:0/492312902 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff28026070 con 0x7eff2c102fc0 2024-05-14T16:44:24.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:24.832+0000 7eff237fe700 1 -- 172.21.15.2:0/492312902 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff2800bc00 con 0x7eff2c102fc0 2024-05-14T16:44:24.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:24.832+0000 7eff237fe700 1 -- 172.21.15.2:0/492312902 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff28003a10 con 0x7eff2c102fc0 2024-05-14T16:44:24.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:24.832+0000 7eff34558700 1 -- 172.21.15.2:0/492312902 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff2c075930 con 0x7eff2c102fc0 2024-05-14T16:44:24.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:24.833+0000 7eff34558700 1 -- 172.21.15.2:0/492312902 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff2c075e10 con 0x7eff2c102fc0 2024-05-14T16:44:24.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:24.834+0000 7eff34558700 1 -- 172.21.15.2:0/492312902 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff2c109680 con 0x7eff2c102fc0 2024-05-14T16:44:24.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:24.834+0000 7eff237fe700 1 -- 172.21.15.2:0/492312902 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7eff28029070 con 0x7eff2c102fc0 2024-05-14T16:44:24.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:24.834+0000 7eff237fe700 1 --2- 172.21.15.2:0/492312902 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7eff1803e570 0x7eff18040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:24.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:24.834+0000 7eff237fe700 1 -- 172.21.15.2:0/492312902 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7eff2801e070 con 0x7eff2c102fc0 2024-05-14T16:44:24.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:24.834+0000 7eff31af3700 1 --2- 172.21.15.2:0/492312902 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7eff1803e570 0x7eff18040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:24.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:24.835+0000 7eff31af3700 1 --2- 172.21.15.2:0/492312902 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7eff1803e570 0x7eff18040a30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7eff1c006fd0 tx=0x7eff1c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:24.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:24.839+0000 7eff237fe700 1 -- 172.21.15.2:0/492312902 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7eff28013070 con 0x7eff2c102fc0 2024-05-14T16:44:25.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:25.067+0000 7eff34558700 1 -- 172.21.15.2:0/492312902 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7eff2c191930 con 0x7eff2c102fc0 2024-05-14T16:44:25.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:25.069+0000 7eff237fe700 1 -- 172.21.15.2:0/492312902 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7eff280300c0 con 0x7eff2c102fc0 2024-05-14T16:44:25.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:25.072+0000 7eff34558700 1 -- 172.21.15.2:0/492312902 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7eff1803e570 msgr2=0x7eff18040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:25.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:25.072+0000 7eff34558700 1 --2- 172.21.15.2:0/492312902 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7eff1803e570 0x7eff18040a30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7eff1c006fd0 tx=0x7eff1c006e40 comp rx=0 tx=0).stop 2024-05-14T16:44:25.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:25.072+0000 7eff34558700 1 -- 172.21.15.2:0/492312902 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff2c102fc0 msgr2=0x7eff2c0771b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:25.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:25.072+0000 7eff34558700 1 --2- 172.21.15.2:0/492312902 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff2c102fc0 0x7eff2c0771b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7eff28005950 tx=0x7eff28004fa0 comp rx=0 tx=0).stop 2024-05-14T16:44:25.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:25.073+0000 7eff34558700 1 -- 172.21.15.2:0/492312902 shutdown_connections 2024-05-14T16:44:25.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:25.073+0000 7eff34558700 1 --2- 172.21.15.2:0/492312902 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7eff1803e570 0x7eff18040a30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:25.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:25.073+0000 7eff34558700 1 --2- 172.21.15.2:0/492312902 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff2c102fc0 0x7eff2c0771b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:25.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:25.073+0000 7eff34558700 1 -- 172.21.15.2:0/492312902 >> 172.21.15.2:0/492312902 conn(0x7eff2c0fe560 msgr2=0x7eff2c0ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:25.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:25.073+0000 7eff34558700 1 -- 172.21.15.2:0/492312902 shutdown_connections 2024-05-14T16:44:25.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:25.073+0000 7eff34558700 1 -- 172.21.15.2:0/492312902 wait complete. 2024-05-14T16:44:25.840 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-14T16:44:25.841 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-14T16:44:26.309 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:44:26.433 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:26 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/492312902' entity='client.admin' 2024-05-14T16:44:27.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:27 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:44:27.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:27 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:44:27.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:27 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi002", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:44:27.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:27 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:44:27.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:27 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:44:27.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:27 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:44:27.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:27.761+0000 7f17e4d18700 1 -- 172.21.15.2:0/2201627072 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17e0102fc0 msgr2=0x7f17e01033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:27.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:27.761+0000 7f17e4d18700 1 --2- 172.21.15.2:0/2201627072 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17e0102fc0 0x7f17e01033e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f17d0009910 tx=0x7f17d0009c20 comp rx=0 tx=0).stop 2024-05-14T16:44:27.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:27.761+0000 7f17e4d18700 1 -- 172.21.15.2:0/2201627072 shutdown_connections 2024-05-14T16:44:27.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:27.761+0000 7f17e4d18700 1 --2- 172.21.15.2:0/2201627072 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17e0102fc0 0x7f17e01033e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:27.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:27.761+0000 7f17e4d18700 1 -- 172.21.15.2:0/2201627072 >> 172.21.15.2:0/2201627072 conn(0x7f17e00fe560 msgr2=0x7f17e01009a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:27.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:27.761+0000 7f17e4d18700 1 -- 172.21.15.2:0/2201627072 shutdown_connections 2024-05-14T16:44:27.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:27.761+0000 7f17e4d18700 1 -- 172.21.15.2:0/2201627072 wait complete. 2024-05-14T16:44:27.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:27.762+0000 7f17e4d18700 1 Processor -- start 2024-05-14T16:44:27.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:27.762+0000 7f17e4d18700 1 -- start start 2024-05-14T16:44:27.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:27.762+0000 7f17e4d18700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17e0102fc0 0x7f17e0077200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:27.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:27.762+0000 7f17e4d18700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f17e0108f00 con 0x7f17e0102fc0 2024-05-14T16:44:27.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:27.762+0000 7f17de59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17e0102fc0 0x7f17e0077200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:27.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:27.762+0000 7f17de59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17e0102fc0 0x7f17e0077200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36316/0 (socket says 172.21.15.2:36316) 2024-05-14T16:44:27.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:27.762+0000 7f17de59c700 1 -- 172.21.15.2:0/4230058357 learned_addr learned my addr 172.21.15.2:0/4230058357 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:44:27.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:27.762+0000 7f17de59c700 1 -- 172.21.15.2:0/4230058357 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17e0077740 con 0x7f17e0102fc0 2024-05-14T16:44:27.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:27.763+0000 7f17de59c700 1 --2- 172.21.15.2:0/4230058357 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17e0102fc0 0x7f17e0077200 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f17d0005800 tx=0x7f17d0009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:27.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:27.763+0000 7f17cf7fe700 1 -- 172.21.15.2:0/4230058357 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17d0027070 con 0x7f17e0102fc0 2024-05-14T16:44:27.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:27.763+0000 7f17e4d18700 1 -- 172.21.15.2:0/4230058357 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17e0075930 con 0x7f17e0102fc0 2024-05-14T16:44:27.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:27.763+0000 7f17cf7fe700 1 -- 172.21.15.2:0/4230058357 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17d000be10 con 0x7f17e0102fc0 2024-05-14T16:44:27.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:27.763+0000 7f17cf7fe700 1 -- 172.21.15.2:0/4230058357 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17d000f460 con 0x7f17e0102fc0 2024-05-14T16:44:27.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:27.763+0000 7f17e4d18700 1 -- 172.21.15.2:0/4230058357 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17e0075e70 con 0x7f17e0102fc0 2024-05-14T16:44:27.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:27.764+0000 7f17cf7fe700 1 -- 172.21.15.2:0/4230058357 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f17d002a070 con 0x7f17e0102fc0 2024-05-14T16:44:27.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:27.764+0000 7f17e4d18700 1 -- 172.21.15.2:0/4230058357 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17e0109680 con 0x7f17e0102fc0 2024-05-14T16:44:27.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:27.764+0000 7f17cf7fe700 1 --2- 172.21.15.2:0/4230058357 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f17c803e510 0x7f17c80409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:27.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:27.764+0000 7f17cf7fe700 1 -- 172.21.15.2:0/4230058357 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f17d001e030 con 0x7f17e0102fc0 2024-05-14T16:44:27.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:27.764+0000 7f17ddd9b700 1 --2- 172.21.15.2:0/4230058357 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f17c803e510 0x7f17c80409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:27.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:27.765+0000 7f17ddd9b700 1 --2- 172.21.15.2:0/4230058357 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f17c803e510 0x7f17c80409d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f17d4006fd0 tx=0x7f17d4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:27.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:27.769+0000 7f17cf7fe700 1 -- 172.21.15.2:0/4230058357 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f17d0013070 con 0x7f17e0102fc0 2024-05-14T16:44:27.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:27.988+0000 7f17e4d18700 1 -- 172.21.15.2:0/4230058357 --> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f17e0191a10 con 0x7f17c803e510 2024-05-14T16:44:27.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:27.992+0000 7f17cf7fe700 1 -- 172.21.15.2:0/4230058357 <== mgr.14150 v2:172.21.15.2:6800/1498936028 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f17e0191a10 con 0x7f17c803e510 2024-05-14T16:44:27.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:27.994+0000 7f17e4d18700 1 -- 172.21.15.2:0/4230058357 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f17c803e510 msgr2=0x7f17c80409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:27.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:27.994+0000 7f17e4d18700 1 --2- 172.21.15.2:0/4230058357 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f17c803e510 0x7f17c80409d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f17d4006fd0 tx=0x7f17d4006e40 comp rx=0 tx=0).stop 2024-05-14T16:44:27.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:27.994+0000 7f17e4d18700 1 -- 172.21.15.2:0/4230058357 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17e0102fc0 msgr2=0x7f17e0077200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:27.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:27.994+0000 7f17e4d18700 1 --2- 172.21.15.2:0/4230058357 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17e0102fc0 0x7f17e0077200 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f17d0005800 tx=0x7f17d0009f60 comp rx=0 tx=0).stop 2024-05-14T16:44:27.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:27.994+0000 7f17e4d18700 1 -- 172.21.15.2:0/4230058357 shutdown_connections 2024-05-14T16:44:27.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:27.994+0000 7f17e4d18700 1 --2- 172.21.15.2:0/4230058357 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f17c803e510 0x7f17c80409d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:27.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:27.994+0000 7f17e4d18700 1 --2- 172.21.15.2:0/4230058357 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17e0102fc0 0x7f17e0077200 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:27.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:27.994+0000 7f17e4d18700 1 -- 172.21.15.2:0/4230058357 >> 172.21.15.2:0/4230058357 conn(0x7f17e00fe560 msgr2=0x7f17e00ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:27.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:27.994+0000 7f17e4d18700 1 -- 172.21.15.2:0/4230058357 shutdown_connections 2024-05-14T16:44:27.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:27.994+0000 7f17e4d18700 1 -- 172.21.15.2:0/4230058357 wait complete. 2024-05-14T16:44:28.752 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi123 2024-05-14T16:44:28.753 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-05-14T16:44:28.753 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/etc/ceph/ceph.conf 2024-05-14T16:44:28.772 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-05-14T16:44:28.772 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:44:28.830 INFO:tasks.cephadm:Adding host smithi123 to orchestrator... 2024-05-14T16:44:28.830 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph orch host add smithi123 2024-05-14T16:44:29.076 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:44:29.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:28 smithi002 ceph-mon[31538]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:44:29.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:28 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:44:29.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:28 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:44:29.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:28 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:44:29.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:28 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:44:29.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:28 smithi002 ceph-mon[31538]: Updating smithi002:/etc/ceph/ceph.conf 2024-05-14T16:44:29.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:28 smithi002 ceph-mon[31538]: Updating smithi002:/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:44:30.330 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:30 smithi002 ceph-mon[31538]: Updating smithi002:/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:44:30.331 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:30 smithi002 ceph-mon[31538]: Updating smithi002:/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T16:44:30.331 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:30 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:44:30.331 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:30 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:44:30.331 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:30 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:44:30.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:30.645+0000 7f56b5be4700 1 -- 172.21.15.2:0/2379248637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f56b0104150 msgr2=0x7f56b0104530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:30.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:30.645+0000 7f56b5be4700 1 --2- 172.21.15.2:0/2379248637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f56b0104150 0x7f56b0104530 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5698009910 tx=0x7f5698009c20 comp rx=0 tx=0).stop 2024-05-14T16:44:30.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:30.645+0000 7f56b5be4700 1 -- 172.21.15.2:0/2379248637 shutdown_connections 2024-05-14T16:44:30.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:30.645+0000 7f56b5be4700 1 --2- 172.21.15.2:0/2379248637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f56b0104150 0x7f56b0104530 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:30.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:30.645+0000 7f56b5be4700 1 -- 172.21.15.2:0/2379248637 >> 172.21.15.2:0/2379248637 conn(0x7f56b00fd9a0 msgr2=0x7f56b00ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:30.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:30.645+0000 7f56b5be4700 1 -- 172.21.15.2:0/2379248637 shutdown_connections 2024-05-14T16:44:30.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:30.645+0000 7f56b5be4700 1 -- 172.21.15.2:0/2379248637 wait complete. 2024-05-14T16:44:30.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:30.645+0000 7f56b5be4700 1 Processor -- start 2024-05-14T16:44:30.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:30.646+0000 7f56b5be4700 1 -- start start 2024-05-14T16:44:30.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:30.646+0000 7f56b5be4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f56b0104150 0x7f56b0101db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:30.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:30.646+0000 7f56b5be4700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f56b010d4d0 con 0x7f56b0104150 2024-05-14T16:44:30.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:30.646+0000 7f56af7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f56b0104150 0x7f56b0101db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:30.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:30.646+0000 7f56af7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f56b0104150 0x7f56b0101db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36336/0 (socket says 172.21.15.2:36336) 2024-05-14T16:44:30.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:30.646+0000 7f56af7fe700 1 -- 172.21.15.2:0/3702769740 learned_addr learned my addr 172.21.15.2:0/3702769740 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:44:30.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:30.646+0000 7f56af7fe700 1 -- 172.21.15.2:0/3702769740 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b0103ca0 con 0x7f56b0104150 2024-05-14T16:44:30.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:30.646+0000 7f56af7fe700 1 --2- 172.21.15.2:0/3702769740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f56b0104150 0x7f56b0101db0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f56980056d0 tx=0x7f56980057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:30.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:30.646+0000 7f56acff9700 1 -- 172.21.15.2:0/3702769740 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5698026070 con 0x7f56b0104150 2024-05-14T16:44:30.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:30.647+0000 7f56acff9700 1 -- 172.21.15.2:0/3702769740 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5698004370 con 0x7f56b0104150 2024-05-14T16:44:30.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:30.647+0000 7f56b5be4700 1 -- 172.21.15.2:0/3702769740 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56b01022f0 con 0x7f56b0104150 2024-05-14T16:44:30.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:30.647+0000 7f56acff9700 1 -- 172.21.15.2:0/3702769740 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56980039b0 con 0x7f56b0104150 2024-05-14T16:44:30.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:30.647+0000 7f56b5be4700 1 -- 172.21.15.2:0/3702769740 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56b01027b0 con 0x7f56b0104150 2024-05-14T16:44:30.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:30.647+0000 7f56b5be4700 1 -- 172.21.15.2:0/3702769740 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56b006bc90 con 0x7f56b0104150 2024-05-14T16:44:30.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:30.648+0000 7f56acff9700 1 -- 172.21.15.2:0/3702769740 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f5698029070 con 0x7f56b0104150 2024-05-14T16:44:30.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:30.648+0000 7f56acff9700 1 --2- 172.21.15.2:0/3702769740 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f569c03e510 0x7f569c0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:30.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:30.648+0000 7f56acff9700 1 -- 172.21.15.2:0/3702769740 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f569801e070 con 0x7f56b0104150 2024-05-14T16:44:30.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:30.648+0000 7f56aeffd700 1 --2- 172.21.15.2:0/3702769740 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f569c03e510 0x7f569c0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:30.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:30.648+0000 7f56aeffd700 1 --2- 172.21.15.2:0/3702769740 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f569c03e510 0x7f569c0409d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f56a0006fd0 tx=0x7f56a0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:30.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:30.653+0000 7f56acff9700 1 -- 172.21.15.2:0/3702769740 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5698013070 con 0x7f56b0104150 2024-05-14T16:44:30.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:30.876+0000 7f56b5be4700 1 -- 172.21.15.2:0/3702769740 --> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi123", "target": ["mon-mgr", ""]}) -- 0x7f56b01032e0 con 0x7f569c03e510 2024-05-14T16:44:31.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:31 smithi002 ceph-mon[31538]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi123", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:44:32.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:32 smithi002 ceph-mon[31538]: Deploying cephadm binary to smithi123 2024-05-14T16:44:32.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:32 smithi002 ceph-mon[31538]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:44:35.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:35.223+0000 7f56acff9700 1 -- 172.21.15.2:0/3702769740 <== mgr.14150 v2:172.21.15.2:6800/1498936028 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f56b01032e0 con 0x7f569c03e510 2024-05-14T16:44:35.224 INFO:teuthology.orchestra.run.smithi002.stdout:Added host 'smithi123' with addr '172.21.15.123' 2024-05-14T16:44:35.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:35.226+0000 7f56b5be4700 1 -- 172.21.15.2:0/3702769740 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f569c03e510 msgr2=0x7f569c0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:35.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:35.226+0000 7f56b5be4700 1 --2- 172.21.15.2:0/3702769740 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f569c03e510 0x7f569c0409d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f56a0006fd0 tx=0x7f56a0006e40 comp rx=0 tx=0).stop 2024-05-14T16:44:35.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:35.226+0000 7f56b5be4700 1 -- 172.21.15.2:0/3702769740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f56b0104150 msgr2=0x7f56b0101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:35.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:35.226+0000 7f56b5be4700 1 --2- 172.21.15.2:0/3702769740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f56b0104150 0x7f56b0101db0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f56980056d0 tx=0x7f56980057b0 comp rx=0 tx=0).stop 2024-05-14T16:44:35.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:35.226+0000 7f56b5be4700 1 -- 172.21.15.2:0/3702769740 shutdown_connections 2024-05-14T16:44:35.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:35.226+0000 7f56b5be4700 1 --2- 172.21.15.2:0/3702769740 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f569c03e510 0x7f569c0409d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:35.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:35.226+0000 7f56b5be4700 1 --2- 172.21.15.2:0/3702769740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f56b0104150 0x7f56b0101db0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:35.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:35.226+0000 7f56b5be4700 1 -- 172.21.15.2:0/3702769740 >> 172.21.15.2:0/3702769740 conn(0x7f56b00fd9a0 msgr2=0x7f56b0104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:35.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:35.226+0000 7f56b5be4700 1 -- 172.21.15.2:0/3702769740 shutdown_connections 2024-05-14T16:44:35.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:35.226+0000 7f56b5be4700 1 -- 172.21.15.2:0/3702769740 wait complete. 2024-05-14T16:44:35.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:35 smithi002 ceph-mon[31538]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:44:35.796 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph orch host ls --format=json 2024-05-14T16:44:36.050 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:44:36.488 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:36 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:44:36.488 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:36 smithi002 ceph-mon[31538]: Added host smithi123 2024-05-14T16:44:36.489 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:36 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:44:36.489 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:36 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:44:37.384 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:37 smithi002 ceph-mon[31538]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:44:37.384 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:37 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:44:37.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:37.600+0000 7fbe47354700 1 -- 172.21.15.2:0/3246246731 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe401031e0 msgr2=0x7fbe401035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:37.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:37.600+0000 7fbe47354700 1 --2- 172.21.15.2:0/3246246731 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe401031e0 0x7fbe401035c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c009910 tx=0x7fbe3c009c20 comp rx=0 tx=0).stop 2024-05-14T16:44:37.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:37.600+0000 7fbe47354700 1 -- 172.21.15.2:0/3246246731 shutdown_connections 2024-05-14T16:44:37.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:37.600+0000 7fbe47354700 1 --2- 172.21.15.2:0/3246246731 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe401031e0 0x7fbe401035c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:37.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:37.600+0000 7fbe47354700 1 -- 172.21.15.2:0/3246246731 >> 172.21.15.2:0/3246246731 conn(0x7fbe400fea70 msgr2=0x7fbe40100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:37.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:37.600+0000 7fbe47354700 1 -- 172.21.15.2:0/3246246731 shutdown_connections 2024-05-14T16:44:37.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:37.600+0000 7fbe47354700 1 -- 172.21.15.2:0/3246246731 wait complete. 2024-05-14T16:44:37.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:37.600+0000 7fbe47354700 1 Processor -- start 2024-05-14T16:44:37.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:37.601+0000 7fbe47354700 1 -- start start 2024-05-14T16:44:37.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:37.601+0000 7fbe47354700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe401031e0 0x7fbe4019e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:37.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:37.601+0000 7fbe47354700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4010d460 con 0x7fbe401031e0 2024-05-14T16:44:37.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:37.601+0000 7fbe450f0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe401031e0 0x7fbe4019e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:37.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:37.601+0000 7fbe450f0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe401031e0 0x7fbe4019e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60154/0 (socket says 172.21.15.2:60154) 2024-05-14T16:44:37.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:37.601+0000 7fbe450f0700 1 -- 172.21.15.2:0/1745026108 learned_addr learned my addr 172.21.15.2:0/1745026108 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:44:37.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:37.601+0000 7fbe450f0700 1 -- 172.21.15.2:0/1745026108 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe4019eb80 con 0x7fbe401031e0 2024-05-14T16:44:37.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:37.601+0000 7fbe450f0700 1 --2- 172.21.15.2:0/1745026108 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe401031e0 0x7fbe4019e640 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c0056d0 tx=0x7fbe3c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:37.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:37.601+0000 7fbe367fc700 1 -- 172.21.15.2:0/1745026108 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe3c026070 con 0x7fbe401031e0 2024-05-14T16:44:37.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:37.601+0000 7fbe367fc700 1 -- 172.21.15.2:0/1745026108 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe3c00bba0 con 0x7fbe401031e0 2024-05-14T16:44:37.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:37.601+0000 7fbe47354700 1 -- 172.21.15.2:0/1745026108 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe4019ee70 con 0x7fbe401031e0 2024-05-14T16:44:37.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:37.601+0000 7fbe367fc700 1 -- 172.21.15.2:0/1745026108 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe3c003920 con 0x7fbe401031e0 2024-05-14T16:44:37.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:37.602+0000 7fbe47354700 1 -- 172.21.15.2:0/1745026108 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe401987c0 con 0x7fbe401031e0 2024-05-14T16:44:37.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:37.602+0000 7fbe47354700 1 -- 172.21.15.2:0/1745026108 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe401046c0 con 0x7fbe401031e0 2024-05-14T16:44:37.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:37.602+0000 7fbe367fc700 1 -- 172.21.15.2:0/1745026108 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7fbe3c029070 con 0x7fbe401031e0 2024-05-14T16:44:37.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:37.603+0000 7fbe367fc700 1 --2- 172.21.15.2:0/1745026108 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fbe2c03e570 0x7fbe2c040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:37.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:37.603+0000 7fbe367fc700 1 -- 172.21.15.2:0/1745026108 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbe3c01e070 con 0x7fbe401031e0 2024-05-14T16:44:37.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:37.603+0000 7fbe448ef700 1 --2- 172.21.15.2:0/1745026108 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fbe2c03e570 0x7fbe2c040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:37.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:37.603+0000 7fbe448ef700 1 --2- 172.21.15.2:0/1745026108 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fbe2c03e570 0x7fbe2c040a30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbe30006fd0 tx=0x7fbe30006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:37.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:37.608+0000 7fbe367fc700 1 -- 172.21.15.2:0/1745026108 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbe3c013070 con 0x7fbe401031e0 2024-05-14T16:44:37.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:37.833+0000 7fbe47354700 1 -- 172.21.15.2:0/1745026108 --> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbe40199cf0 con 0x7fbe2c03e570 2024-05-14T16:44:37.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:37.834+0000 7fbe367fc700 1 -- 172.21.15.2:0/1745026108 <== mgr.14150 v2:172.21.15.2:6800/1498936028 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7fbe40199cf0 con 0x7fbe2c03e570 2024-05-14T16:44:37.834 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:44:37.834 INFO:teuthology.orchestra.run.smithi002.stdout:[{"addr": "172.21.15.2", "hostname": "smithi002", "labels": [], "status": ""}, {"addr": "172.21.15.123", "hostname": "smithi123", "labels": [], "status": ""}] 2024-05-14T16:44:37.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:37.836+0000 7fbe47354700 1 -- 172.21.15.2:0/1745026108 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fbe2c03e570 msgr2=0x7fbe2c040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:37.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:37.836+0000 7fbe47354700 1 --2- 172.21.15.2:0/1745026108 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fbe2c03e570 0x7fbe2c040a30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbe30006fd0 tx=0x7fbe30006e40 comp rx=0 tx=0).stop 2024-05-14T16:44:37.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:37.836+0000 7fbe47354700 1 -- 172.21.15.2:0/1745026108 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe401031e0 msgr2=0x7fbe4019e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:37.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:37.836+0000 7fbe47354700 1 --2- 172.21.15.2:0/1745026108 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe401031e0 0x7fbe4019e640 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c0056d0 tx=0x7fbe3c0057b0 comp rx=0 tx=0).stop 2024-05-14T16:44:37.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:37.836+0000 7fbe47354700 1 -- 172.21.15.2:0/1745026108 shutdown_connections 2024-05-14T16:44:37.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:37.836+0000 7fbe47354700 1 --2- 172.21.15.2:0/1745026108 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fbe2c03e570 0x7fbe2c040a30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:37.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:37.836+0000 7fbe47354700 1 --2- 172.21.15.2:0/1745026108 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbe401031e0 0x7fbe4019e640 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:37.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:37.836+0000 7fbe47354700 1 -- 172.21.15.2:0/1745026108 >> 172.21.15.2:0/1745026108 conn(0x7fbe400fea70 msgr2=0x7fbe40104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:37.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:37.836+0000 7fbe47354700 1 -- 172.21.15.2:0/1745026108 shutdown_connections 2024-05-14T16:44:37.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:37.836+0000 7fbe47354700 1 -- 172.21.15.2:0/1745026108 wait complete. 2024-05-14T16:44:38.452 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi151 2024-05-14T16:44:38.452 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-14T16:44:38.452 DEBUG:teuthology.orchestra.run.smithi151:> dd of=/etc/ceph/ceph.conf 2024-05-14T16:44:38.471 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-14T16:44:38.471 DEBUG:teuthology.orchestra.run.smithi151:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:44:38.529 INFO:tasks.cephadm:Adding host smithi151 to orchestrator... 2024-05-14T16:44:38.529 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph orch host add smithi151 2024-05-14T16:44:38.774 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:44:39.184 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:38 smithi002 ceph-mon[31538]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:44:39.184 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:38 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:44:39.184 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:38 smithi002 ceph-mon[31538]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:44:40.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:40.209+0000 7f7f87819700 1 -- 172.21.15.2:0/2261603091 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f801031b0 msgr2=0x7f7f80103590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:40.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:40.209+0000 7f7f87819700 1 --2- 172.21.15.2:0/2261603091 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f801031b0 0x7f7f80103590 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c009910 tx=0x7f7f7c009c20 comp rx=0 tx=0).stop 2024-05-14T16:44:40.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:40.209+0000 7f7f87819700 1 -- 172.21.15.2:0/2261603091 shutdown_connections 2024-05-14T16:44:40.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:40.209+0000 7f7f87819700 1 --2- 172.21.15.2:0/2261603091 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f801031b0 0x7f7f80103590 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:40.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:40.209+0000 7f7f87819700 1 -- 172.21.15.2:0/2261603091 >> 172.21.15.2:0/2261603091 conn(0x7f7f800fea80 msgr2=0x7f7f80100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:40.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:40.209+0000 7f7f87819700 1 -- 172.21.15.2:0/2261603091 shutdown_connections 2024-05-14T16:44:40.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:40.209+0000 7f7f87819700 1 -- 172.21.15.2:0/2261603091 wait complete. 2024-05-14T16:44:40.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:40.210+0000 7f7f87819700 1 Processor -- start 2024-05-14T16:44:40.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:40.210+0000 7f7f87819700 1 -- start start 2024-05-14T16:44:40.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:40.210+0000 7f7f87819700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f801031b0 0x7f7f80075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:40.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:40.210+0000 7f7f87819700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7f8010d430 con 0x7f7f801031b0 2024-05-14T16:44:40.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:40.210+0000 7f7f855b5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f801031b0 0x7f7f80075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:40.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:40.210+0000 7f7f855b5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f801031b0 0x7f7f80075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60178/0 (socket says 172.21.15.2:60178) 2024-05-14T16:44:40.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:40.210+0000 7f7f855b5700 1 -- 172.21.15.2:0/2116392563 learned_addr learned my addr 172.21.15.2:0/2116392563 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:44:40.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:40.210+0000 7f7f855b5700 1 -- 172.21.15.2:0/2116392563 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f80077730 con 0x7f7f801031b0 2024-05-14T16:44:40.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:40.211+0000 7f7f855b5700 1 --2- 172.21.15.2:0/2116392563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f801031b0 0x7f7f80075840 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0056d0 tx=0x7f7f7c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:40.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:40.211+0000 7f7f767fc700 1 -- 172.21.15.2:0/2116392563 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f7c026070 con 0x7f7f801031b0 2024-05-14T16:44:40.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:40.211+0000 7f7f767fc700 1 -- 172.21.15.2:0/2116392563 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f7c00bc00 con 0x7f7f801031b0 2024-05-14T16:44:40.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:40.211+0000 7f7f87819700 1 -- 172.21.15.2:0/2116392563 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f80075de0 con 0x7f7f801031b0 2024-05-14T16:44:40.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:40.211+0000 7f7f767fc700 1 -- 172.21.15.2:0/2116392563 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f7c003920 con 0x7f7f801031b0 2024-05-14T16:44:40.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:40.211+0000 7f7f87819700 1 -- 172.21.15.2:0/2116392563 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f800762a0 con 0x7f7f801031b0 2024-05-14T16:44:40.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:40.212+0000 7f7f767fc700 1 -- 172.21.15.2:0/2116392563 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f7f7c029070 con 0x7f7f801031b0 2024-05-14T16:44:40.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:40.212+0000 7f7f87819700 1 -- 172.21.15.2:0/2116392563 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f80104690 con 0x7f7f801031b0 2024-05-14T16:44:40.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:40.212+0000 7f7f767fc700 1 --2- 172.21.15.2:0/2116392563 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f7f6c03e510 0x7f7f6c0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:40.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:40.212+0000 7f7f767fc700 1 -- 172.21.15.2:0/2116392563 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7f7c01e070 con 0x7f7f801031b0 2024-05-14T16:44:40.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:40.212+0000 7f7f84db4700 1 --2- 172.21.15.2:0/2116392563 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f7f6c03e510 0x7f7f6c0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:40.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:40.212+0000 7f7f84db4700 1 --2- 172.21.15.2:0/2116392563 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f7f6c03e510 0x7f7f6c0409d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7f70006fd0 tx=0x7f7f70006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:40.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:40.217+0000 7f7f767fc700 1 -- 172.21.15.2:0/2116392563 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7f7c013070 con 0x7f7f801031b0 2024-05-14T16:44:40.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:40.435+0000 7f7f87819700 1 -- 172.21.15.2:0/2116392563 --> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi151", "target": ["mon-mgr", ""]}) -- 0x7f7f80076530 con 0x7f7f6c03e510 2024-05-14T16:44:41.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:41 smithi002 ceph-mon[31538]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:44:41.495 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:41 smithi002 ceph-mon[31538]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi151", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:44:42.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:42 smithi002 ceph-mon[31538]: Deploying cephadm binary to smithi151 2024-05-14T16:44:42.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:42 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:44:42.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:42 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:44:42.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:42 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:44:42.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:42 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:44:42.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:42 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi123", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:44:42.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:42 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:44:42.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:42 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:44:42.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:42 smithi002 ceph-mon[31538]: Updating smithi123:/etc/ceph/ceph.conf 2024-05-14T16:44:43.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:43 smithi002 ceph-mon[31538]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:44:44.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:44.661+0000 7f7f767fc700 1 -- 172.21.15.2:0/2116392563 <== mgr.14150 v2:172.21.15.2:6800/1498936028 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f7f80076530 con 0x7f7f6c03e510 2024-05-14T16:44:44.662 INFO:teuthology.orchestra.run.smithi002.stdout:Added host 'smithi151' with addr '172.21.15.151' 2024-05-14T16:44:44.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:44.663+0000 7f7f87819700 1 -- 172.21.15.2:0/2116392563 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f7f6c03e510 msgr2=0x7f7f6c0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:44.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:44.663+0000 7f7f87819700 1 --2- 172.21.15.2:0/2116392563 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f7f6c03e510 0x7f7f6c0409d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7f70006fd0 tx=0x7f7f70006e40 comp rx=0 tx=0).stop 2024-05-14T16:44:44.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:44.663+0000 7f7f87819700 1 -- 172.21.15.2:0/2116392563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f801031b0 msgr2=0x7f7f80075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:44.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:44.663+0000 7f7f87819700 1 --2- 172.21.15.2:0/2116392563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f801031b0 0x7f7f80075840 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0056d0 tx=0x7f7f7c0057b0 comp rx=0 tx=0).stop 2024-05-14T16:44:44.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:44.664+0000 7f7f87819700 1 -- 172.21.15.2:0/2116392563 shutdown_connections 2024-05-14T16:44:44.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:44.664+0000 7f7f87819700 1 --2- 172.21.15.2:0/2116392563 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f7f6c03e510 0x7f7f6c0409d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:44.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:44.664+0000 7f7f87819700 1 --2- 172.21.15.2:0/2116392563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f801031b0 0x7f7f80075840 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:44.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:44.664+0000 7f7f87819700 1 -- 172.21.15.2:0/2116392563 >> 172.21.15.2:0/2116392563 conn(0x7f7f800fea80 msgr2=0x7f7f80104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:44.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:44.664+0000 7f7f87819700 1 -- 172.21.15.2:0/2116392563 shutdown_connections 2024-05-14T16:44:44.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:44.664+0000 7f7f87819700 1 -- 172.21.15.2:0/2116392563 wait complete. 2024-05-14T16:44:45.273 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph orch host ls --format=json 2024-05-14T16:44:45.518 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:44:45.641 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:45 smithi002 ceph-mon[31538]: Updating smithi123:/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:44:45.641 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:45 smithi002 ceph-mon[31538]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:44:45.641 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:45 smithi002 ceph-mon[31538]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:44:45.641 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:45 smithi002 ceph-mon[31538]: Updating smithi123:/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T16:44:45.641 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:45 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:44:45.641 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:45 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:44:45.641 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:45 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:44:45.642 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:45 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:44:45.642 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:45 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:44:45.642 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:45 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:44:46.633 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:46 smithi002 ceph-mon[31538]: Added host smithi151 2024-05-14T16:44:46.633 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:46 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:44:47.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:47.120+0000 7f3ca76c9700 1 -- 172.21.15.2:0/2460153548 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ca01031b0 msgr2=0x7f3ca0103590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:47.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:47.120+0000 7f3ca76c9700 1 --2- 172.21.15.2:0/2460153548 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ca01031b0 0x7f3ca0103590 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c009910 tx=0x7f3c9c009c20 comp rx=0 tx=0).stop 2024-05-14T16:44:47.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:47.120+0000 7f3ca76c9700 1 -- 172.21.15.2:0/2460153548 shutdown_connections 2024-05-14T16:44:47.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:47.120+0000 7f3ca76c9700 1 --2- 172.21.15.2:0/2460153548 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ca01031b0 0x7f3ca0103590 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:47.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:47.120+0000 7f3ca76c9700 1 -- 172.21.15.2:0/2460153548 >> 172.21.15.2:0/2460153548 conn(0x7f3ca00fea80 msgr2=0x7f3ca0100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:47.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:47.120+0000 7f3ca76c9700 1 -- 172.21.15.2:0/2460153548 shutdown_connections 2024-05-14T16:44:47.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:47.120+0000 7f3ca76c9700 1 -- 172.21.15.2:0/2460153548 wait complete. 2024-05-14T16:44:47.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:47.120+0000 7f3ca76c9700 1 Processor -- start 2024-05-14T16:44:47.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:47.121+0000 7f3ca76c9700 1 -- start start 2024-05-14T16:44:47.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:47.121+0000 7f3ca76c9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ca01031b0 0x7f3ca0075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:47.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:47.121+0000 7f3ca76c9700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca010d430 con 0x7f3ca01031b0 2024-05-14T16:44:47.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:47.121+0000 7f3ca5465700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ca01031b0 0x7f3ca0075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:47.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:47.121+0000 7f3ca5465700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ca01031b0 0x7f3ca0075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53182/0 (socket says 172.21.15.2:53182) 2024-05-14T16:44:47.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:47.121+0000 7f3ca5465700 1 -- 172.21.15.2:0/2043790925 learned_addr learned my addr 172.21.15.2:0/2043790925 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:44:47.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:47.121+0000 7f3ca5465700 1 -- 172.21.15.2:0/2043790925 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca0077730 con 0x7f3ca01031b0 2024-05-14T16:44:47.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:47.121+0000 7f3ca5465700 1 --2- 172.21.15.2:0/2043790925 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ca01031b0 0x7f3ca0075840 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c0056d0 tx=0x7f3c9c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:47.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:47.121+0000 7f3c927fc700 1 -- 172.21.15.2:0/2043790925 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c9c026070 con 0x7f3ca01031b0 2024-05-14T16:44:47.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:47.121+0000 7f3ca76c9700 1 -- 172.21.15.2:0/2043790925 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ca0075de0 con 0x7f3ca01031b0 2024-05-14T16:44:47.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:47.122+0000 7f3c927fc700 1 -- 172.21.15.2:0/2043790925 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c9c00bc00 con 0x7f3ca01031b0 2024-05-14T16:44:47.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:47.122+0000 7f3ca76c9700 1 -- 172.21.15.2:0/2043790925 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ca00762a0 con 0x7f3ca01031b0 2024-05-14T16:44:47.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:47.122+0000 7f3c927fc700 1 -- 172.21.15.2:0/2043790925 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c9c003920 con 0x7f3ca01031b0 2024-05-14T16:44:47.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:47.122+0000 7f3c927fc700 1 -- 172.21.15.2:0/2043790925 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f3c9c029070 con 0x7f3ca01031b0 2024-05-14T16:44:47.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:47.122+0000 7f3ca76c9700 1 -- 172.21.15.2:0/2043790925 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ca0104690 con 0x7f3ca01031b0 2024-05-14T16:44:47.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:47.123+0000 7f3c927fc700 1 --2- 172.21.15.2:0/2043790925 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f3c8c03e570 0x7f3c8c040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:47.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:47.123+0000 7f3c927fc700 1 -- 172.21.15.2:0/2043790925 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3c9c01e070 con 0x7f3ca01031b0 2024-05-14T16:44:47.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:47.123+0000 7f3ca4c64700 1 --2- 172.21.15.2:0/2043790925 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f3c8c03e570 0x7f3c8c040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:47.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:47.123+0000 7f3ca4c64700 1 --2- 172.21.15.2:0/2043790925 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f3c8c03e570 0x7f3c8c040a30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3c94006fd0 tx=0x7f3c94006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:47.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:47.128+0000 7f3c927fc700 1 -- 172.21.15.2:0/2043790925 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3c9c013070 con 0x7f3ca01031b0 2024-05-14T16:44:47.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:47.359+0000 7f3ca76c9700 1 -- 172.21.15.2:0/2043790925 --> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3ca0076d70 con 0x7f3c8c03e570 2024-05-14T16:44:47.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:47.360+0000 7f3c927fc700 1 -- 172.21.15.2:0/2043790925 <== mgr.14150 v2:172.21.15.2:6800/1498936028 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f3ca0076d70 con 0x7f3c8c03e570 2024-05-14T16:44:47.360 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:44:47.360 INFO:teuthology.orchestra.run.smithi002.stdout:[{"addr": "172.21.15.2", "hostname": "smithi002", "labels": [], "status": ""}, {"addr": "172.21.15.123", "hostname": "smithi123", "labels": [], "status": ""}, {"addr": "172.21.15.151", "hostname": "smithi151", "labels": [], "status": ""}] 2024-05-14T16:44:47.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:47.363+0000 7f3ca76c9700 1 -- 172.21.15.2:0/2043790925 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f3c8c03e570 msgr2=0x7f3c8c040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:47.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:47.363+0000 7f3ca76c9700 1 --2- 172.21.15.2:0/2043790925 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f3c8c03e570 0x7f3c8c040a30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3c94006fd0 tx=0x7f3c94006e40 comp rx=0 tx=0).stop 2024-05-14T16:44:47.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:47.363+0000 7f3ca76c9700 1 -- 172.21.15.2:0/2043790925 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ca01031b0 msgr2=0x7f3ca0075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:47.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:47.363+0000 7f3ca76c9700 1 --2- 172.21.15.2:0/2043790925 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ca01031b0 0x7f3ca0075840 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c0056d0 tx=0x7f3c9c0057b0 comp rx=0 tx=0).stop 2024-05-14T16:44:47.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:47.363+0000 7f3ca76c9700 1 -- 172.21.15.2:0/2043790925 shutdown_connections 2024-05-14T16:44:47.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:47.363+0000 7f3ca76c9700 1 --2- 172.21.15.2:0/2043790925 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f3c8c03e570 0x7f3c8c040a30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:47.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:47.363+0000 7f3ca76c9700 1 --2- 172.21.15.2:0/2043790925 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ca01031b0 0x7f3ca0075840 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:47.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:47.363+0000 7f3ca76c9700 1 -- 172.21.15.2:0/2043790925 >> 172.21.15.2:0/2043790925 conn(0x7f3ca00fea80 msgr2=0x7f3ca0104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:47.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:47.363+0000 7f3ca76c9700 1 -- 172.21.15.2:0/2043790925 shutdown_connections 2024-05-14T16:44:47.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:47.364+0000 7f3ca76c9700 1 -- 172.21.15.2:0/2043790925 wait complete. 2024-05-14T16:44:47.675 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:47 smithi002 ceph-mon[31538]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:44:47.675 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:47 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:44:47.922 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-14T16:44:47.922 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph osd crush tunables default 2024-05-14T16:44:48.167 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:44:48.570 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:48 smithi002 ceph-mon[31538]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:44:49.562 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:49 smithi002 ceph-mon[31538]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:44:49.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:49.778+0000 7fdb506a2700 1 -- 172.21.15.2:0/633323538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb48104150 msgr2=0x7fdb48104530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:49.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:49.778+0000 7fdb506a2700 1 --2- 172.21.15.2:0/633323538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb48104150 0x7fdb48104530 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdb44009910 tx=0x7fdb44009c20 comp rx=0 tx=0).stop 2024-05-14T16:44:49.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:49.778+0000 7fdb506a2700 1 -- 172.21.15.2:0/633323538 shutdown_connections 2024-05-14T16:44:49.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:49.778+0000 7fdb506a2700 1 --2- 172.21.15.2:0/633323538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb48104150 0x7fdb48104530 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:49.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:49.778+0000 7fdb506a2700 1 -- 172.21.15.2:0/633323538 >> 172.21.15.2:0/633323538 conn(0x7fdb480fd9a0 msgr2=0x7fdb480ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:49.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:49.778+0000 7fdb506a2700 1 -- 172.21.15.2:0/633323538 shutdown_connections 2024-05-14T16:44:49.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:49.778+0000 7fdb506a2700 1 -- 172.21.15.2:0/633323538 wait complete. 2024-05-14T16:44:49.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:49.778+0000 7fdb506a2700 1 Processor -- start 2024-05-14T16:44:49.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:49.779+0000 7fdb506a2700 1 -- start start 2024-05-14T16:44:49.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:49.779+0000 7fdb506a2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb48104150 0x7fdb48101db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:49.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:49.779+0000 7fdb506a2700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4810d4d0 con 0x7fdb48104150 2024-05-14T16:44:49.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:49.779+0000 7fdb4e43e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb48104150 0x7fdb48101db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:49.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:49.779+0000 7fdb4e43e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb48104150 0x7fdb48101db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53194/0 (socket says 172.21.15.2:53194) 2024-05-14T16:44:49.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:49.779+0000 7fdb4e43e700 1 -- 172.21.15.2:0/3066896887 learned_addr learned my addr 172.21.15.2:0/3066896887 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:44:49.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:49.779+0000 7fdb4e43e700 1 -- 172.21.15.2:0/3066896887 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb48103ca0 con 0x7fdb48104150 2024-05-14T16:44:49.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:49.779+0000 7fdb4e43e700 1 --2- 172.21.15.2:0/3066896887 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb48104150 0x7fdb48101db0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdb44006010 tx=0x7fdb44005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:49.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:49.779+0000 7fdb3f7fe700 1 -- 172.21.15.2:0/3066896887 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb44026070 con 0x7fdb48104150 2024-05-14T16:44:49.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:49.779+0000 7fdb3f7fe700 1 -- 172.21.15.2:0/3066896887 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb4400bc00 con 0x7fdb48104150 2024-05-14T16:44:49.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:49.779+0000 7fdb506a2700 1 -- 172.21.15.2:0/3066896887 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb481022f0 con 0x7fdb48104150 2024-05-14T16:44:49.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:49.780+0000 7fdb3f7fe700 1 -- 172.21.15.2:0/3066896887 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb44003910 con 0x7fdb48104150 2024-05-14T16:44:49.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:49.780+0000 7fdb506a2700 1 -- 172.21.15.2:0/3066896887 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb481027b0 con 0x7fdb48104150 2024-05-14T16:44:49.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:49.780+0000 7fdb506a2700 1 -- 172.21.15.2:0/3066896887 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb4806bc90 con 0x7fdb48104150 2024-05-14T16:44:49.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:49.781+0000 7fdb3f7fe700 1 -- 172.21.15.2:0/3066896887 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7fdb44029070 con 0x7fdb48104150 2024-05-14T16:44:49.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:49.781+0000 7fdb3f7fe700 1 --2- 172.21.15.2:0/3066896887 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fdb3403e570 0x7fdb34040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:49.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:49.781+0000 7fdb3f7fe700 1 -- 172.21.15.2:0/3066896887 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdb4401e070 con 0x7fdb48104150 2024-05-14T16:44:49.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:49.781+0000 7fdb4dc3d700 1 --2- 172.21.15.2:0/3066896887 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fdb3403e570 0x7fdb34040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:49.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:49.781+0000 7fdb4dc3d700 1 --2- 172.21.15.2:0/3066896887 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fdb3403e570 0x7fdb34040a30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdb38006fd0 tx=0x7fdb38006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:49.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:49.786+0000 7fdb3f7fe700 1 -- 172.21.15.2:0/3066896887 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdb44013070 con 0x7fdb48104150 2024-05-14T16:44:50.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:50.006+0000 7fdb506a2700 1 -- 172.21.15.2:0/3066896887 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fdb481032e0 con 0x7fdb48104150 2024-05-14T16:44:50.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:50.376+0000 7fdb3f7fe700 1 -- 172.21.15.2:0/3066896887 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fdb4400d340 con 0x7fdb48104150 2024-05-14T16:44:50.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:50.379+0000 7fdb506a2700 1 -- 172.21.15.2:0/3066896887 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fdb3403e570 msgr2=0x7fdb34040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:50.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:50.379+0000 7fdb506a2700 1 --2- 172.21.15.2:0/3066896887 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fdb3403e570 0x7fdb34040a30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdb38006fd0 tx=0x7fdb38006e40 comp rx=0 tx=0).stop 2024-05-14T16:44:50.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:50.379+0000 7fdb506a2700 1 -- 172.21.15.2:0/3066896887 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb48104150 msgr2=0x7fdb48101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:50.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:50.379+0000 7fdb506a2700 1 --2- 172.21.15.2:0/3066896887 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb48104150 0x7fdb48101db0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdb44006010 tx=0x7fdb44005780 comp rx=0 tx=0).stop 2024-05-14T16:44:50.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:50.379+0000 7fdb506a2700 1 -- 172.21.15.2:0/3066896887 shutdown_connections 2024-05-14T16:44:50.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:50.379+0000 7fdb506a2700 1 --2- 172.21.15.2:0/3066896887 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fdb3403e570 0x7fdb34040a30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:50.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:50.379+0000 7fdb506a2700 1 --2- 172.21.15.2:0/3066896887 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb48104150 0x7fdb48101db0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:50.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:50.379+0000 7fdb506a2700 1 -- 172.21.15.2:0/3066896887 >> 172.21.15.2:0/3066896887 conn(0x7fdb480fd9a0 msgr2=0x7fdb48104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:50.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:50.379+0000 7fdb506a2700 1 -- 172.21.15.2:0/3066896887 shutdown_connections 2024-05-14T16:44:50.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:44:50.379+0000 7fdb506a2700 1 -- 172.21.15.2:0/3066896887 wait complete. 2024-05-14T16:44:50.380 INFO:teuthology.orchestra.run.smithi002.stderr:adjusted tunables profile to default 2024-05-14T16:44:50.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:50 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/3066896887' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-14T16:44:50.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:50 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:44:50.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:50 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:44:50.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:50 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:44:50.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:50 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:44:50.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:50 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi151", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:44:50.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:50 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:44:50.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:50 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:44:51.101 INFO:tasks.cephadm:Adding mon.a on smithi002 2024-05-14T16:44:51.101 INFO:tasks.cephadm:Adding mon.b on smithi123 2024-05-14T16:44:51.101 INFO:tasks.cephadm:Adding mon.c on smithi151 2024-05-14T16:44:51.101 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph orch apply mon '3;smithi002:172.21.15.2=a;smithi123:172.21.15.123=b;smithi151:172.21.15.151=c' 2024-05-14T16:44:51.332 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:44:51.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:51 smithi002 ceph-mon[31538]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:44:51.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:51 smithi002 ceph-mon[31538]: Updating smithi151:/etc/ceph/ceph.conf 2024-05-14T16:44:51.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:51 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/3066896887' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-14T16:44:51.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:51 smithi002 ceph-mon[31538]: osdmap e4: 0 total, 0 up, 0 in 2024-05-14T16:44:51.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:51 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:44:51.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:51 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:44:51.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:51 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:44:52.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:52 smithi002 ceph-mon[31538]: Updating smithi151:/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:44:52.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:52 smithi002 ceph-mon[31538]: Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:44:52.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:52 smithi002 ceph-mon[31538]: Updating smithi151:/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T16:44:52.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:52.775+0000 7f689fb28700 1 -- 172.21.15.151:0/2326919343 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68981076c0 msgr2=0x7f6898107aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:52.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:52.775+0000 7f689fb28700 1 --2- 172.21.15.151:0/2326919343 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68981076c0 0x7f6898107aa0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f689000aa20 tx=0x7f689000ad30 comp rx=0 tx=0).stop 2024-05-14T16:44:52.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:52.775+0000 7f689fb28700 1 -- 172.21.15.151:0/2326919343 shutdown_connections 2024-05-14T16:44:52.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:52.775+0000 7f689fb28700 1 --2- 172.21.15.151:0/2326919343 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68981076c0 0x7f6898107aa0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:52.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:52.775+0000 7f689fb28700 1 -- 172.21.15.151:0/2326919343 >> 172.21.15.151:0/2326919343 conn(0x7f6898079470 msgr2=0x7f6898079880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:52.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:52.776+0000 7f689fb28700 1 -- 172.21.15.151:0/2326919343 shutdown_connections 2024-05-14T16:44:52.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:52.776+0000 7f689fb28700 1 -- 172.21.15.151:0/2326919343 wait complete. 2024-05-14T16:44:52.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:52.776+0000 7f689fb28700 1 Processor -- start 2024-05-14T16:44:52.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:52.776+0000 7f689fb28700 1 -- start start 2024-05-14T16:44:52.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:52.776+0000 7f689fb28700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68981076c0 0x7f689819e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:52.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:52.776+0000 7f689fb28700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f689810b260 con 0x7f68981076c0 2024-05-14T16:44:52.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:52.777+0000 7f689d8c4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68981076c0 0x7f689819e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:52.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:52.777+0000 7f689d8c4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68981076c0 0x7f689819e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.151:45020/0 (socket says 172.21.15.151:45020) 2024-05-14T16:44:52.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:52.777+0000 7f689d8c4700 1 -- 172.21.15.151:0/812091113 learned_addr learned my addr 172.21.15.151:0/812091113 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-14T16:44:52.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:52.777+0000 7f689d8c4700 1 -- 172.21.15.151:0/812091113 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f689819eb90 con 0x7f68981076c0 2024-05-14T16:44:52.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:52.777+0000 7f689d8c4700 1 --2- 172.21.15.151:0/812091113 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68981076c0 0x7f689819e650 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6890006010 tx=0x7f6890005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:52.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:52.777+0000 7f6896ffd700 1 -- 172.21.15.151:0/812091113 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6890005400 con 0x7f68981076c0 2024-05-14T16:44:52.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:52.777+0000 7f689fb28700 1 -- 172.21.15.151:0/812091113 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f689819ee20 con 0x7f68981076c0 2024-05-14T16:44:52.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:52.777+0000 7f6896ffd700 1 -- 172.21.15.151:0/812091113 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68900055a0 con 0x7f68981076c0 2024-05-14T16:44:52.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:52.777+0000 7f6896ffd700 1 -- 172.21.15.151:0/812091113 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f689000e7a0 con 0x7f68981076c0 2024-05-14T16:44:52.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:52.778+0000 7f689fb28700 1 -- 172.21.15.151:0/812091113 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6898198720 con 0x7f68981076c0 2024-05-14T16:44:52.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:52.778+0000 7f689fb28700 1 -- 172.21.15.151:0/812091113 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6898102d20 con 0x7f68981076c0 2024-05-14T16:44:52.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:52.778+0000 7f6896ffd700 1 -- 172.21.15.151:0/812091113 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f6890016b40 con 0x7f68981076c0 2024-05-14T16:44:52.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:52.779+0000 7f6896ffd700 1 --2- 172.21.15.151:0/812091113 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f688003e510 0x7f68800409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:52.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:52.779+0000 7f6896ffd700 1 -- 172.21.15.151:0/812091113 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f689001e070 con 0x7f68981076c0 2024-05-14T16:44:52.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:52.779+0000 7f689d0c3700 1 --2- 172.21.15.151:0/812091113 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f688003e510 0x7f68800409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:52.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:52.779+0000 7f689d0c3700 1 --2- 172.21.15.151:0/812091113 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f688003e510 0x7f68800409d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f688c006fd0 tx=0x7f688c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:52.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:52.784+0000 7f6896ffd700 1 -- 172.21.15.151:0/812091113 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6890007df0 con 0x7f68981076c0 2024-05-14T16:44:53.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:53.005+0000 7f689fb28700 1 -- 172.21.15.151:0/812091113 --> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi002:172.21.15.2=a;smithi123:172.21.15.123=b;smithi151:172.21.15.151=c", "target": ["mon-mgr", ""]}) -- 0x7f6898199d10 con 0x7f688003e510 2024-05-14T16:44:53.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:53.012+0000 7f6896ffd700 1 -- 172.21.15.151:0/812091113 <== mgr.14150 v2:172.21.15.2:6800/1498936028 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6898199d10 con 0x7f688003e510 2024-05-14T16:44:53.012 INFO:teuthology.orchestra.run.smithi151.stdout:Scheduled mon update... 2024-05-14T16:44:53.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:53.014+0000 7f689fb28700 1 -- 172.21.15.151:0/812091113 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f688003e510 msgr2=0x7f68800409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:53.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:53.014+0000 7f689fb28700 1 --2- 172.21.15.151:0/812091113 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f688003e510 0x7f68800409d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f688c006fd0 tx=0x7f688c006e40 comp rx=0 tx=0).stop 2024-05-14T16:44:53.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:53.014+0000 7f689fb28700 1 -- 172.21.15.151:0/812091113 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68981076c0 msgr2=0x7f689819e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:53.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:53.014+0000 7f689fb28700 1 --2- 172.21.15.151:0/812091113 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68981076c0 0x7f689819e650 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6890006010 tx=0x7f6890005050 comp rx=0 tx=0).stop 2024-05-14T16:44:53.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:53.014+0000 7f689fb28700 1 -- 172.21.15.151:0/812091113 shutdown_connections 2024-05-14T16:44:53.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:53.014+0000 7f689fb28700 1 --2- 172.21.15.151:0/812091113 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f688003e510 0x7f68800409d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:53.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:53.014+0000 7f689fb28700 1 --2- 172.21.15.151:0/812091113 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68981076c0 0x7f689819e650 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:53.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:53.014+0000 7f689fb28700 1 -- 172.21.15.151:0/812091113 >> 172.21.15.151:0/812091113 conn(0x7f6898079470 msgr2=0x7f68980feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:53.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:53.014+0000 7f689fb28700 1 -- 172.21.15.151:0/812091113 shutdown_connections 2024-05-14T16:44:53.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:53.014+0000 7f689fb28700 1 -- 172.21.15.151:0/812091113 wait complete. 2024-05-14T16:44:53.611 DEBUG:teuthology.orchestra.run.smithi123:mon.b> sudo journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@mon.b.service 2024-05-14T16:44:53.614 DEBUG:teuthology.orchestra.run.smithi151:mon.c> sudo journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@mon.c.service 2024-05-14T16:44:53.616 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-14T16:44:53.616 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-14T16:44:53.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:53 smithi002 ceph-mon[31538]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:44:53.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:53 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:44:53.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:53 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:44:53.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:53 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:44:53.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:53 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:44:53.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:53 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:44:53.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:53 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T16:44:53.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:53 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:44:54.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:54 smithi002 ceph-mon[31538]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi002:172.21.15.2=a;smithi123:172.21.15.123=b;smithi151:172.21.15.151=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:44:54.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:54 smithi002 ceph-mon[31538]: Saving service mon spec with placement smithi002:172.21.15.2=a;smithi123:172.21.15.123=b;smithi151:172.21.15.151=c;count:3 2024-05-14T16:44:54.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:54 smithi002 ceph-mon[31538]: Deploying daemon mon.c on smithi151 2024-05-14T16:44:55.497 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T16:44:55.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:55 smithi002 ceph-mon[31538]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:44:57.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:57.147+0000 7f203b81d700 1 -- 172.21.15.151:0/3345954036 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20341032c0 msgr2=0x7f20341036a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:57.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:57.147+0000 7f203b81d700 1 --2- 172.21.15.151:0/3345954036 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20341032c0 0x7f20341036a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f202c009910 tx=0x7f202c009c20 comp rx=0 tx=0).stop 2024-05-14T16:44:57.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:57.147+0000 7f203b81d700 1 -- 172.21.15.151:0/3345954036 shutdown_connections 2024-05-14T16:44:57.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:57.147+0000 7f203b81d700 1 --2- 172.21.15.151:0/3345954036 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20341032c0 0x7f20341036a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:57.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:57.147+0000 7f203b81d700 1 -- 172.21.15.151:0/3345954036 >> 172.21.15.151:0/3345954036 conn(0x7f20340feb50 msgr2=0x7f2034100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:57.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:57.147+0000 7f203b81d700 1 -- 172.21.15.151:0/3345954036 shutdown_connections 2024-05-14T16:44:57.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:57.147+0000 7f203b81d700 1 -- 172.21.15.151:0/3345954036 wait complete. 2024-05-14T16:44:57.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:57.148+0000 7f203b81d700 1 Processor -- start 2024-05-14T16:44:57.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:57.148+0000 7f203b81d700 1 -- start start 2024-05-14T16:44:57.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:57.148+0000 7f203b81d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20341032c0 0x7f203419e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:57.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:57.148+0000 7f203b81d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f203410d540 con 0x7f20341032c0 2024-05-14T16:44:57.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:57.148+0000 7f20395b9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20341032c0 0x7f203419e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:57.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:57.148+0000 7f20395b9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20341032c0 0x7f203419e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.151:37168/0 (socket says 172.21.15.151:37168) 2024-05-14T16:44:57.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:57.148+0000 7f20395b9700 1 -- 172.21.15.151:0/2383477640 learned_addr learned my addr 172.21.15.151:0/2383477640 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-14T16:44:57.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:57.148+0000 7f20395b9700 1 -- 172.21.15.151:0/2383477640 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f203419ec70 con 0x7f20341032c0 2024-05-14T16:44:57.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:57.149+0000 7f20395b9700 1 --2- 172.21.15.151:0/2383477640 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20341032c0 0x7f203419e730 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f202c0056d0 tx=0x7f202c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:57.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:57.149+0000 7f202a7fc700 1 -- 172.21.15.151:0/2383477640 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f202c026070 con 0x7f20341032c0 2024-05-14T16:44:57.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:57.149+0000 7f203b81d700 1 -- 172.21.15.151:0/2383477640 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f203419ef60 con 0x7f20341032c0 2024-05-14T16:44:57.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:57.149+0000 7f202a7fc700 1 -- 172.21.15.151:0/2383477640 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f202c00bba0 con 0x7f20341032c0 2024-05-14T16:44:57.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:57.149+0000 7f202a7fc700 1 -- 172.21.15.151:0/2383477640 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f202c003920 con 0x7f20341032c0 2024-05-14T16:44:57.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:57.149+0000 7f203b81d700 1 -- 172.21.15.151:0/2383477640 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20341988f0 con 0x7f20341032c0 2024-05-14T16:44:57.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:57.150+0000 7f203b81d700 1 -- 172.21.15.151:0/2383477640 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20341047a0 con 0x7f20341032c0 2024-05-14T16:44:57.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:57.150+0000 7f202a7fc700 1 -- 172.21.15.151:0/2383477640 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f202c029070 con 0x7f20341032c0 2024-05-14T16:44:57.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:57.150+0000 7f202a7fc700 1 --2- 172.21.15.151:0/2383477640 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f202003e510 0x7f20200409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:44:57.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:57.150+0000 7f202a7fc700 1 -- 172.21.15.151:0/2383477640 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f202c01e070 con 0x7f20341032c0 2024-05-14T16:44:57.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:57.150+0000 7f2038db8700 1 --2- 172.21.15.151:0/2383477640 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f202003e510 0x7f20200409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:44:57.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:57.151+0000 7f2038db8700 1 --2- 172.21.15.151:0/2383477640 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f202003e510 0x7f20200409d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2024006fd0 tx=0x7f2024006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:44:57.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:57.156+0000 7f202a7fc700 1 -- 172.21.15.151:0/2383477640 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f202c013070 con 0x7f20341032c0 2024-05-14T16:44:57.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:57.450+0000 7f203b81d700 1 -- 172.21.15.151:0/2383477640 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f2034199e20 con 0x7f20341032c0 2024-05-14T16:44:57.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:57.451+0000 7f202a7fc700 1 -- 172.21.15.151:0/2383477640 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+741 (secure 0 0 0) 0x7f202c030090 con 0x7f20341032c0 2024-05-14T16:44:57.451 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-14T16:44:57.451 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":1,"fsid":"ed40717a-1210-11ef-bc98-c7b262605968","modified":"2024-05-14T16:43:06.575008Z","created":"2024-05-14T16:43:06.575008Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:3300","nonce":0},{"type":"v1","addr":"172.21.15.2:6789","nonce":0}]},"addr":"172.21.15.2:6789/0","public_addr":"172.21.15.2:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-14T16:44:57.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:57.453+0000 7f203b81d700 1 -- 172.21.15.151:0/2383477640 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f202003e510 msgr2=0x7f20200409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:57.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:57.453+0000 7f203b81d700 1 --2- 172.21.15.151:0/2383477640 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f202003e510 0x7f20200409d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2024006fd0 tx=0x7f2024006e40 comp rx=0 tx=0).stop 2024-05-14T16:44:57.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:57.453+0000 7f203b81d700 1 -- 172.21.15.151:0/2383477640 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20341032c0 msgr2=0x7f203419e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:44:57.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:57.453+0000 7f203b81d700 1 --2- 172.21.15.151:0/2383477640 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20341032c0 0x7f203419e730 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f202c0056d0 tx=0x7f202c0057b0 comp rx=0 tx=0).stop 2024-05-14T16:44:57.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:57.453+0000 7f203b81d700 1 -- 172.21.15.151:0/2383477640 shutdown_connections 2024-05-14T16:44:57.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:57.453+0000 7f203b81d700 1 --2- 172.21.15.151:0/2383477640 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f202003e510 0x7f20200409d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:57.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:57.453+0000 7f203b81d700 1 --2- 172.21.15.151:0/2383477640 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20341032c0 0x7f203419e730 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:44:57.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:57.453+0000 7f203b81d700 1 -- 172.21.15.151:0/2383477640 >> 172.21.15.151:0/2383477640 conn(0x7f20340feb50 msgr2=0x7f2034104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:44:57.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:57.453+0000 7f203b81d700 1 -- 172.21.15.151:0/2383477640 shutdown_connections 2024-05-14T16:44:57.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:44:57.453+0000 7f203b81d700 1 -- 172.21.15.151:0/2383477640 wait complete. 2024-05-14T16:44:57.454 INFO:teuthology.orchestra.run.smithi151.stderr:dumped monmap epoch 1 2024-05-14T16:44:57.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:44:57 smithi002 ceph-mon[31538]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:44:59.238 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-14T16:44:59.238 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-14T16:44:59.486 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T16:45:02.995 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:02 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:45:02.995 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:02 smithi002 ceph-mon[31538]: mon.a calling monitor election 2024-05-14T16:45:02.995 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:02 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:45:02.995 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:02 smithi002 ceph-mon[31538]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:02.995 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:02 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:45:02.995 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:02 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:45:02.995 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:02 smithi002 ceph-mon[31538]: mon.c calling monitor election 2024-05-14T16:45:02.995 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:02 smithi002 ceph-mon[31538]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:02.996 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:02 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:45:02.996 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:02 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:45:02.996 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:02 smithi002 ceph-mon[31538]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:02.996 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:02 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:45:02.996 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:02 smithi002 ceph-mon[31538]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-14T16:45:02.996 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:02 smithi002 ceph-mon[31538]: monmap epoch 2 2024-05-14T16:45:02.996 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:02 smithi002 ceph-mon[31538]: fsid ed40717a-1210-11ef-bc98-c7b262605968 2024-05-14T16:45:02.996 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:02 smithi002 ceph-mon[31538]: last_changed 2024-05-14T16:44:57.593167+0000 2024-05-14T16:45:02.996 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:02 smithi002 ceph-mon[31538]: created 2024-05-14T16:43:06.575008+0000 2024-05-14T16:45:02.996 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:02 smithi002 ceph-mon[31538]: min_mon_release 19 (squid) 2024-05-14T16:45:02.996 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:02 smithi002 ceph-mon[31538]: election_strategy: 1 2024-05-14T16:45:02.996 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:02 smithi002 ceph-mon[31538]: 0: [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon.a 2024-05-14T16:45:02.996 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:02 smithi002 ceph-mon[31538]: 1: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.c 2024-05-14T16:45:02.996 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:02 smithi002 ceph-mon[31538]: fsmap 2024-05-14T16:45:02.997 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:02 smithi002 ceph-mon[31538]: osdmap e4: 0 total, 0 up, 0 in 2024-05-14T16:45:02.997 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:02 smithi002 ceph-mon[31538]: mgrmap e13: x(active, since 50s) 2024-05-14T16:45:02.997 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:02 smithi002 ceph-mon[31538]: overall HEALTH_OK 2024-05-14T16:45:02.997 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:02 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:02.997 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:02 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:02.997 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:02 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:02.997 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:02 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T16:45:02.997 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:02 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:03.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:02 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:03.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:03.778+0000 7f45f2fe3700 1 -- 172.21.15.151:0/824047424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45ec104270 msgr2=0x7f45d8005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:03.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:03.778+0000 7f45f2fe3700 1 --2- 172.21.15.151:0/824047424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45ec104270 0x7f45d8005610 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f45e4009bf0 tx=0x7f45e4009f00 comp rx=0 tx=0).stop 2024-05-14T16:45:03.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:03.778+0000 7f45f2fe3700 1 -- 172.21.15.151:0/824047424 shutdown_connections 2024-05-14T16:45:03.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:03.778+0000 7f45f2fe3700 1 --2- 172.21.15.151:0/824047424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45ec104270 0x7f45d8005610 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:03.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:03.778+0000 7f45f2fe3700 1 -- 172.21.15.151:0/824047424 >> 172.21.15.151:0/824047424 conn(0x7f45ec0fda80 msgr2=0x7f45ec0ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:45:03.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:03.778+0000 7f45f2fe3700 1 -- 172.21.15.151:0/824047424 shutdown_connections 2024-05-14T16:45:03.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:03.778+0000 7f45f2fe3700 1 -- 172.21.15.151:0/824047424 wait complete. 2024-05-14T16:45:03.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:03.779+0000 7f45f2fe3700 1 Processor -- start 2024-05-14T16:45:03.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:03.779+0000 7f45f2fe3700 1 -- start start 2024-05-14T16:45:03.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:03.779+0000 7f45f2fe3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45ec104270 0x7f45ec101ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:03.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:03.779+0000 7f45f2fe3700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45ec102410 0x7f45ec102890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:03.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:03.779+0000 7f45f2fe3700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f45ec10af90 con 0x7f45ec104270 2024-05-14T16:45:03.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:03.779+0000 7f45f2fe3700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f45ec10ae10 con 0x7f45ec102410 2024-05-14T16:45:03.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:03.779+0000 7f45ebfff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45ec102410 0x7f45ec102890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:03.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:03.779+0000 7f45f0d7f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45ec104270 0x7f45ec101ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:03.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:03.779+0000 7f45f0d7f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45ec104270 0x7f45ec101ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.151:37232/0 (socket says 172.21.15.151:37232) 2024-05-14T16:45:03.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:03.779+0000 7f45ebfff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45ec102410 0x7f45ec102890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41996/0 (socket says 172.21.15.151:41996) 2024-05-14T16:45:03.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:03.779+0000 7f45f0d7f700 1 -- 172.21.15.151:0/3778398560 learned_addr learned my addr 172.21.15.151:0/3778398560 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-14T16:45:03.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:03.779+0000 7f45ebfff700 1 -- 172.21.15.151:0/3778398560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45ec102410 msgr2=0x7f45ec102890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-14T16:45:03.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:03.779+0000 7f45ebfff700 1 -- 172.21.15.151:0/3778398560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45ec102410 msgr2=0x7f45ec102890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-14T16:45:03.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:03.779+0000 7f45ebfff700 1 --2- 172.21.15.151:0/3778398560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45ec102410 0x7f45ec102890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-14T16:45:03.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:03.779+0000 7f45ebfff700 1 --2- 172.21.15.151:0/3778398560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45ec102410 0x7f45ec102890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T16:45:03.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:03.779+0000 7f45f0d7f700 1 -- 172.21.15.151:0/3778398560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45ec102410 msgr2=0x7f45ec102890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T16:45:03.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:03.779+0000 7f45f0d7f700 1 --2- 172.21.15.151:0/3778398560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45ec102410 0x7f45ec102890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:03.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:03.780+0000 7f45f0d7f700 1 -- 172.21.15.151:0/3778398560 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45ec103df0 con 0x7f45ec104270 2024-05-14T16:45:03.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:03.780+0000 7f45f0d7f700 1 --2- 172.21.15.151:0/3778398560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45ec104270 0x7f45ec101ed0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f45e4009610 tx=0x7f45e40044b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:45:03.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:03.780+0000 7f45e9ffb700 1 -- 172.21.15.151:0/3778398560 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f45e4017470 con 0x7f45ec104270 2024-05-14T16:45:03.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:03.780+0000 7f45e9ffb700 1 -- 172.21.15.151:0/3778398560 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45e4024030 con 0x7f45ec104270 2024-05-14T16:45:03.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:03.780+0000 7f45f2fe3700 1 -- 172.21.15.151:0/3778398560 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45ec1a4d00 con 0x7f45ec104270 2024-05-14T16:45:03.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:03.780+0000 7f45e9ffb700 1 -- 172.21.15.151:0/3778398560 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f45e4020950 con 0x7f45ec104270 2024-05-14T16:45:03.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:03.780+0000 7f45f2fe3700 1 -- 172.21.15.151:0/3778398560 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45ec1a51e0 con 0x7f45ec104270 2024-05-14T16:45:03.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:03.781+0000 7f45e9ffb700 1 -- 172.21.15.151:0/3778398560 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f45e401e070 con 0x7f45ec104270 2024-05-14T16:45:03.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:03.781+0000 7f45f2fe3700 1 -- 172.21.15.151:0/3778398560 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45ec06bfd0 con 0x7f45ec104270 2024-05-14T16:45:03.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:03.781+0000 7f45e9ffb700 1 --2- 172.21.15.151:0/3778398560 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f45cc03e300 0x7f45cc0407c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:03.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:03.782+0000 7f45e9ffb700 1 -- 172.21.15.151:0/3778398560 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f45e401c070 con 0x7f45ec104270 2024-05-14T16:45:03.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:03.782+0000 7f45ebfff700 1 --2- 172.21.15.151:0/3778398560 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f45cc03e300 0x7f45cc0407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:03.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:03.782+0000 7f45ebfff700 1 --2- 172.21.15.151:0/3778398560 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f45cc03e300 0x7f45cc0407c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f45dc009850 tx=0x7f45dc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:45:03.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:03.787+0000 7f45e9ffb700 1 -- 172.21.15.151:0/3778398560 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f45e4014d80 con 0x7f45ec104270 2024-05-14T16:45:03.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:03 smithi002 ceph-mon[31538]: Deploying daemon mon.b on smithi123 2024-05-14T16:45:03.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:03 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:45:04.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:04.074+0000 7f45f2fe3700 1 -- 172.21.15.151:0/3778398560 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f45ec02e720 con 0x7f45ec104270 2024-05-14T16:45:04.074 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:03 smithi151 ceph-mon[36066]: Deploying daemon mon.b on smithi123 2024-05-14T16:45:04.074 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:03 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:45:04.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:04.074+0000 7f45e9ffb700 1 -- 172.21.15.151:0/3778398560 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7f45e401fc20 con 0x7f45ec104270 2024-05-14T16:45:04.075 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-14T16:45:04.075 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":2,"fsid":"ed40717a-1210-11ef-bc98-c7b262605968","modified":"2024-05-14T16:44:57.593167Z","created":"2024-05-14T16:43:06.575008Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:3300","nonce":0},{"type":"v1","addr":"172.21.15.2:6789","nonce":0}]},"addr":"172.21.15.2:6789/0","public_addr":"172.21.15.2:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:3300","nonce":0},{"type":"v1","addr":"172.21.15.151:6789","nonce":0}]},"addr":"172.21.15.151:6789/0","public_addr":"172.21.15.151:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-14T16:45:04.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:04.076+0000 7f45f2fe3700 1 -- 172.21.15.151:0/3778398560 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f45cc03e300 msgr2=0x7f45cc0407c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:04.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:04.076+0000 7f45f2fe3700 1 --2- 172.21.15.151:0/3778398560 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f45cc03e300 0x7f45cc0407c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f45dc009850 tx=0x7f45dc008040 comp rx=0 tx=0).stop 2024-05-14T16:45:04.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:04.076+0000 7f45f2fe3700 1 -- 172.21.15.151:0/3778398560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45ec104270 msgr2=0x7f45ec101ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:04.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:04.076+0000 7f45f2fe3700 1 --2- 172.21.15.151:0/3778398560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45ec104270 0x7f45ec101ed0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f45e4009610 tx=0x7f45e40044b0 comp rx=0 tx=0).stop 2024-05-14T16:45:04.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:04.076+0000 7f45f2fe3700 1 -- 172.21.15.151:0/3778398560 shutdown_connections 2024-05-14T16:45:04.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:04.076+0000 7f45f2fe3700 1 --2- 172.21.15.151:0/3778398560 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f45cc03e300 0x7f45cc0407c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:04.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:04.076+0000 7f45f2fe3700 1 --2- 172.21.15.151:0/3778398560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45ec104270 0x7f45ec101ed0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:04.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:04.076+0000 7f45f2fe3700 1 --2- 172.21.15.151:0/3778398560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45ec102410 0x7f45ec102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:04.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:04.076+0000 7f45f2fe3700 1 -- 172.21.15.151:0/3778398560 >> 172.21.15.151:0/3778398560 conn(0x7f45ec0fda80 msgr2=0x7f45ec0ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:45:04.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:04.077+0000 7f45f2fe3700 1 -- 172.21.15.151:0/3778398560 shutdown_connections 2024-05-14T16:45:04.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:04.077+0000 7f45f2fe3700 1 -- 172.21.15.151:0/3778398560 wait complete. 2024-05-14T16:45:04.078 INFO:teuthology.orchestra.run.smithi151.stderr:dumped monmap epoch 2 2024-05-14T16:45:04.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:04 smithi002 ceph-mon[31538]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:04.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:04 smithi002 ceph-mon[31538]: from='client.? 172.21.15.151:0/3778398560' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-14T16:45:04.994 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:45:04 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:45:04.593+0000 7f37bdc16700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-14T16:45:05.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:04 smithi151 ceph-mon[36066]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:05.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:04 smithi151 ceph-mon[36066]: from='client.? 172.21.15.151:0/3778398560' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-14T16:45:05.631 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-14T16:45:05.632 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-14T16:45:05.874 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T16:45:06.719 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: mon.b@-1(???) e0 preinit fsid ed40717a-1210-11ef-bc98-c7b262605968 2024-05-14T16:45:06.720 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: mon.b@-1(synchronizing).mds e1 new map 2024-05-14T16:45:06.720 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: mon.b@-1(synchronizing).mds e1 print_map 2024-05-14T16:45:06.720 INFO:journalctl@ceph.mon.b.smithi123.stdout: e1 2024-05-14T16:45:06.720 INFO:journalctl@ceph.mon.b.smithi123.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-14T16:45:06.720 INFO:journalctl@ceph.mon.b.smithi123.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-14T16:45:06.720 INFO:journalctl@ceph.mon.b.smithi123.stdout: legacy client fscid: -1 2024-05-14T16:45:06.720 INFO:journalctl@ceph.mon.b.smithi123.stdout: 2024-05-14T16:45:06.720 INFO:journalctl@ceph.mon.b.smithi123.stdout: No filesystems configured 2024-05-14T16:45:07.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-14T16:45:07.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-14T16:45:07.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-14T16:45:07.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-14T16:45:07.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-14T16:45:07.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-14T16:45:07.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-14T16:45:07.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-14T16:45:07.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-14T16:45:07.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-14T16:45:07.162 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:07.162 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi151", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:45:07.162 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: Deploying cephadm binary to smithi151 2024-05-14T16:45:07.162 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:07.162 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:07.162 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:07.162 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:07.162 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi123", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:45:07.162 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:07.162 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:45:07.162 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: Updating smithi123:/etc/ceph/ceph.conf 2024-05-14T16:45:07.162 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:07.162 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: Updating smithi123:/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:45:07.163 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:07.163 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:45:07.163 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: Updating smithi123:/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T16:45:07.163 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:07.163 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:07.163 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:07.163 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:07.163 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:45:07.163 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:07.163 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: Added host smithi151 2024-05-14T16:45:07.163 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:07.163 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:07.163 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:07.164 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:45:07.164 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:07.164 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/3066896887' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-14T16:45:07.164 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:07.164 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:07.164 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:07.164 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:07.164 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi151", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:45:07.164 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:07.164 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:45:07.164 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:07.164 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: Updating smithi151:/etc/ceph/ceph.conf 2024-05-14T16:45:07.164 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/3066896887' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-14T16:45:07.165 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: osdmap e4: 0 total, 0 up, 0 in 2024-05-14T16:45:07.165 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:07.165 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:07.165 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:07.165 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: Updating smithi151:/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:45:07.165 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:45:07.165 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: Updating smithi151:/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T16:45:07.165 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:07.165 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:07.165 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:45:07.165 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:07.165 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:45:07.165 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:07.166 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T16:45:07.166 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:07.166 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi002:172.21.15.2=a;smithi123:172.21.15.123=b;smithi151:172.21.15.151=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:45:07.166 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: Saving service mon spec with placement smithi002:172.21.15.2=a;smithi123:172.21.15.123=b;smithi151:172.21.15.151=c;count:3 2024-05-14T16:45:07.166 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: Deploying daemon mon.c on smithi151 2024-05-14T16:45:07.166 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:07.166 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:07.167 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:45:07.167 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: mon.a calling monitor election 2024-05-14T16:45:07.167 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:45:07.167 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:07.167 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:45:07.167 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:45:07.167 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: mon.c calling monitor election 2024-05-14T16:45:07.167 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:07.167 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:45:07.167 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:45:07.167 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:07.167 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:45:07.167 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-14T16:45:07.167 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: monmap epoch 2 2024-05-14T16:45:07.168 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: fsid ed40717a-1210-11ef-bc98-c7b262605968 2024-05-14T16:45:07.168 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: last_changed 2024-05-14T16:44:57.593167+0000 2024-05-14T16:45:07.168 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: created 2024-05-14T16:43:06.575008+0000 2024-05-14T16:45:07.168 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: min_mon_release 19 (squid) 2024-05-14T16:45:07.168 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: election_strategy: 1 2024-05-14T16:45:07.168 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: 0: [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon.a 2024-05-14T16:45:07.168 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: 1: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.c 2024-05-14T16:45:07.168 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: fsmap 2024-05-14T16:45:07.168 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: osdmap e4: 0 total, 0 up, 0 in 2024-05-14T16:45:07.168 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: mgrmap e13: x(active, since 50s) 2024-05-14T16:45:07.168 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: overall HEALTH_OK 2024-05-14T16:45:07.168 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:07.168 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:07.169 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:07.169 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T16:45:07.169 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:07.169 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: Deploying daemon mon.b on smithi123 2024-05-14T16:45:07.169 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:45:07.169 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:07.169 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: from='client.? 172.21.15.151:0/3778398560' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-14T16:45:07.169 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 ceph-mon[35971]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-14T16:45:07.169 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 bash[35950]: b249d40e5d46299738ac5008d309560d70d0f69cb5fc2f019dc113dc50fed313 2024-05-14T16:45:07.169 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:06 smithi123 systemd[1]: Started Ceph mon.b for ed40717a-1210-11ef-bc98-c7b262605968. 2024-05-14T16:45:11.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.780+0000 7f5421bca700 1 -- 172.21.15.151:0/3060687340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f541c102190 msgr2=0x7f5408005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:11.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.780+0000 7f5421bca700 1 --2- 172.21.15.151:0/3060687340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f541c102190 0x7f5408005610 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f540c005fa0 tx=0x7f540c00ff70 comp rx=0 tx=0).stop 2024-05-14T16:45:11.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.780+0000 7f5421bca700 1 -- 172.21.15.151:0/3060687340 shutdown_connections 2024-05-14T16:45:11.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.780+0000 7f5421bca700 1 --2- 172.21.15.151:0/3060687340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f541c102190 0x7f5408005610 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:11.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.780+0000 7f5421bca700 1 -- 172.21.15.151:0/3060687340 >> 172.21.15.151:0/3060687340 conn(0x7f541c0fda20 msgr2=0x7f541c0ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:45:11.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.780+0000 7f5421bca700 1 -- 172.21.15.151:0/3060687340 shutdown_connections 2024-05-14T16:45:11.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.780+0000 7f5421bca700 1 -- 172.21.15.151:0/3060687340 wait complete. 2024-05-14T16:45:11.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.781+0000 7f5421bca700 1 Processor -- start 2024-05-14T16:45:11.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.781+0000 7f5421bca700 1 -- start start 2024-05-14T16:45:11.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.782+0000 7f5421bca700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f541c102190 0x7f541c19e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:11.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.782+0000 7f5421bca700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f541c19ece0 0x7f541c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:11.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.782+0000 7f5421bca700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f541c198ef0 0x7f541c1993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:11.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.782+0000 7f5421bca700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f541c109e20 con 0x7f541c102190 2024-05-14T16:45:11.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.782+0000 7f5421bca700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f541c109ca0 con 0x7f541c198ef0 2024-05-14T16:45:11.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.782+0000 7f5421bca700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f541c109fa0 con 0x7f541c19ece0 2024-05-14T16:45:11.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.782+0000 7f5412ffd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f541c19ece0 0x7f541c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:11.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.782+0000 7f541b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f541c102190 0x7f541c19e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:11.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.782+0000 7f5412ffd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f541c19ece0 0x7f541c1989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:38224/0 (socket says 172.21.15.151:38224) 2024-05-14T16:45:11.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.782+0000 7f5412ffd700 1 -- 172.21.15.151:0/2984405184 learned_addr learned my addr 172.21.15.151:0/2984405184 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-14T16:45:11.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.782+0000 7f541bfff700 1 --2- 172.21.15.151:0/2984405184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f541c198ef0 0x7f541c1993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:11.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.783+0000 7f5412ffd700 1 -- 172.21.15.151:0/2984405184 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f541c19ece0 msgr2=0x7f541c1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-14T16:45:11.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.783+0000 7f5412ffd700 1 -- 172.21.15.151:0/2984405184 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f541c19ece0 msgr2=0x7f541c1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-14T16:45:11.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.783+0000 7f5412ffd700 1 --2- 172.21.15.151:0/2984405184 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f541c19ece0 0x7f541c1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-14T16:45:11.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.783+0000 7f5412ffd700 1 --2- 172.21.15.151:0/2984405184 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f541c19ece0 0x7f541c1989b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T16:45:11.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.783+0000 7f541bfff700 1 -- 172.21.15.151:0/2984405184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f541c198ef0 msgr2=0x7f541c1993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-05-14T16:45:11.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.783+0000 7f541bfff700 1 -- 172.21.15.151:0/2984405184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f541c198ef0 msgr2=0x7f541c1993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-14T16:45:11.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.783+0000 7f541bfff700 1 --2- 172.21.15.151:0/2984405184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f541c198ef0 0x7f541c1993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-14T16:45:11.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.783+0000 7f541bfff700 1 --2- 172.21.15.151:0/2984405184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f541c198ef0 0x7f541c1993a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T16:45:11.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.783+0000 7f541b7fe700 1 -- 172.21.15.151:0/2984405184 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f541c19ece0 msgr2=0x7f541c1989b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T16:45:11.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.783+0000 7f541b7fe700 1 --2- 172.21.15.151:0/2984405184 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f541c19ece0 0x7f541c1989b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:11.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.783+0000 7f541b7fe700 1 -- 172.21.15.151:0/2984405184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f541c198ef0 msgr2=0x7f541c1993a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T16:45:11.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.783+0000 7f541b7fe700 1 --2- 172.21.15.151:0/2984405184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f541c198ef0 0x7f541c1993a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:11.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.783+0000 7f541b7fe700 1 -- 172.21.15.151:0/2984405184 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f541c1a5670 con 0x7f541c102190 2024-05-14T16:45:11.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.783+0000 7f541b7fe700 1 --2- 172.21.15.151:0/2984405184 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f541c102190 0x7f541c19e7a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f540c01f000 tx=0x7f540c01d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:45:11.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.783+0000 7f54197fa700 1 -- 172.21.15.151:0/2984405184 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540c02b020 con 0x7f541c102190 2024-05-14T16:45:11.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.783+0000 7f54197fa700 1 -- 172.21.15.151:0/2984405184 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f540c01dd40 con 0x7f541c102190 2024-05-14T16:45:11.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.783+0000 7f54197fa700 1 -- 172.21.15.151:0/2984405184 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540c022040 con 0x7f541c102190 2024-05-14T16:45:11.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.783+0000 7f5421bca700 1 -- 172.21.15.151:0/2984405184 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f541c1a5840 con 0x7f541c102190 2024-05-14T16:45:11.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.784+0000 7f5421bca700 1 -- 172.21.15.151:0/2984405184 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f541c1a5d30 con 0x7f541c102190 2024-05-14T16:45:11.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.784+0000 7f54197fa700 1 -- 172.21.15.151:0/2984405184 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f540c02e020 con 0x7f541c102190 2024-05-14T16:45:11.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.785+0000 7f5421bca700 1 -- 172.21.15.151:0/2984405184 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f541c064ba0 con 0x7f541c102190 2024-05-14T16:45:11.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.785+0000 7f54197fa700 1 --2- 172.21.15.151:0/2984405184 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f53fc03e9a0 0x7f53fc040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:11.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.785+0000 7f54197fa700 1 -- 172.21.15.151:0/2984405184 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f540c0553b0 con 0x7f541c102190 2024-05-14T16:45:11.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.785+0000 7f5412ffd700 1 --2- 172.21.15.151:0/2984405184 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f53fc03e9a0 0x7f53fc040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:11.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.786+0000 7f5412ffd700 1 --2- 172.21.15.151:0/2984405184 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f53fc03e9a0 0x7f53fc040e60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5404009180 tx=0x7f5404008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:45:11.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:11.790+0000 7f54197fa700 1 -- 172.21.15.151:0/2984405184 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f540c035020 con 0x7f541c102190 2024-05-14T16:45:12.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:12.092+0000 7f5421bca700 1 -- 172.21.15.151:0/2984405184 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f541c02e560 con 0x7f541c102190 2024-05-14T16:45:12.092 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:11 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:45:12.093 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:11 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:45:12.093 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:11 smithi151 ceph-mon[36066]: mon.a calling monitor election 2024-05-14T16:45:12.093 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:11 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:45:12.093 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:11 smithi151 ceph-mon[36066]: mon.c calling monitor election 2024-05-14T16:45:12.093 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:11 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:45:12.093 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:11 smithi151 ceph-mon[36066]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:12.093 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:11 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:45:12.093 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:11 smithi151 ceph-mon[36066]: mon.b calling monitor election 2024-05-14T16:45:12.093 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:11 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:45:12.093 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:11 smithi151 ceph-mon[36066]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:12.093 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:11 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:45:12.093 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:11 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:45:12.093 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:11 smithi151 ceph-mon[36066]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-14T16:45:12.093 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:11 smithi151 ceph-mon[36066]: monmap epoch 3 2024-05-14T16:45:12.093 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:11 smithi151 ceph-mon[36066]: fsid ed40717a-1210-11ef-bc98-c7b262605968 2024-05-14T16:45:12.094 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:11 smithi151 ceph-mon[36066]: last_changed 2024-05-14T16:45:06.727143+0000 2024-05-14T16:45:12.094 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:11 smithi151 ceph-mon[36066]: created 2024-05-14T16:43:06.575008+0000 2024-05-14T16:45:12.094 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:11 smithi151 ceph-mon[36066]: min_mon_release 19 (squid) 2024-05-14T16:45:12.094 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:11 smithi151 ceph-mon[36066]: election_strategy: 1 2024-05-14T16:45:12.094 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:11 smithi151 ceph-mon[36066]: 0: [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon.a 2024-05-14T16:45:12.094 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:11 smithi151 ceph-mon[36066]: 1: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.c 2024-05-14T16:45:12.094 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:11 smithi151 ceph-mon[36066]: 2: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.b 2024-05-14T16:45:12.094 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:11 smithi151 ceph-mon[36066]: fsmap 2024-05-14T16:45:12.094 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:11 smithi151 ceph-mon[36066]: osdmap e4: 0 total, 0 up, 0 in 2024-05-14T16:45:12.094 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:11 smithi151 ceph-mon[36066]: mgrmap e13: x(active, since 59s) 2024-05-14T16:45:12.094 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:11 smithi151 ceph-mon[36066]: overall HEALTH_OK 2024-05-14T16:45:12.094 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:11 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:12.094 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:11 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:12.095 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:11 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:12.095 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:11 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:12.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:12.093+0000 7f54197fa700 1 -- 172.21.15.151:0/2984405184 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1281 (secure 0 0 0) 0x7f540c026370 con 0x7f541c102190 2024-05-14T16:45:12.095 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-14T16:45:12.095 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":3,"fsid":"ed40717a-1210-11ef-bc98-c7b262605968","modified":"2024-05-14T16:45:06.727143Z","created":"2024-05-14T16:43:06.575008Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:3300","nonce":0},{"type":"v1","addr":"172.21.15.2:6789","nonce":0}]},"addr":"172.21.15.2:6789/0","public_addr":"172.21.15.2:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:3300","nonce":0},{"type":"v1","addr":"172.21.15.151:6789","nonce":0}]},"addr":"172.21.15.151:6789/0","public_addr":"172.21.15.151:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:3300","nonce":0},{"type":"v1","addr":"172.21.15.123:6789","nonce":0}]},"addr":"172.21.15.123:6789/0","public_addr":"172.21.15.123:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-05-14T16:45:12.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:12.095+0000 7f5421bca700 1 -- 172.21.15.151:0/2984405184 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f53fc03e9a0 msgr2=0x7f53fc040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:12.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:12.095+0000 7f5421bca700 1 --2- 172.21.15.151:0/2984405184 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f53fc03e9a0 0x7f53fc040e60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5404009180 tx=0x7f5404008040 comp rx=0 tx=0).stop 2024-05-14T16:45:12.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:12.095+0000 7f5421bca700 1 -- 172.21.15.151:0/2984405184 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f541c102190 msgr2=0x7f541c19e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:12.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:12.095+0000 7f5421bca700 1 --2- 172.21.15.151:0/2984405184 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f541c102190 0x7f541c19e7a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f540c01f000 tx=0x7f540c01d560 comp rx=0 tx=0).stop 2024-05-14T16:45:12.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:12.095+0000 7f5421bca700 1 -- 172.21.15.151:0/2984405184 shutdown_connections 2024-05-14T16:45:12.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:12.095+0000 7f5421bca700 1 --2- 172.21.15.151:0/2984405184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f541c198ef0 0x7f541c1993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:12.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:12.095+0000 7f5421bca700 1 --2- 172.21.15.151:0/2984405184 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f53fc03e9a0 0x7f53fc040e60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:12.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:12.095+0000 7f5421bca700 1 --2- 172.21.15.151:0/2984405184 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f541c102190 0x7f541c19e7a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:12.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:12.095+0000 7f5421bca700 1 --2- 172.21.15.151:0/2984405184 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f541c19ece0 0x7f541c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:12.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:12.095+0000 7f5421bca700 1 -- 172.21.15.151:0/2984405184 >> 172.21.15.151:0/2984405184 conn(0x7f541c0fda20 msgr2=0x7f541c0ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:45:12.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:12.095+0000 7f5421bca700 1 -- 172.21.15.151:0/2984405184 shutdown_connections 2024-05-14T16:45:12.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:12.095+0000 7f5421bca700 1 -- 172.21.15.151:0/2984405184 wait complete. 2024-05-14T16:45:12.097 INFO:teuthology.orchestra.run.smithi151.stderr:dumped monmap epoch 3 2024-05-14T16:45:12.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:11 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:45:12.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:11 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:45:12.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:11 smithi123 ceph-mon[35971]: mon.a calling monitor election 2024-05-14T16:45:12.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:11 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:45:12.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:11 smithi123 ceph-mon[35971]: mon.c calling monitor election 2024-05-14T16:45:12.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:11 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:45:12.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:11 smithi123 ceph-mon[35971]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:12.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:11 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:45:12.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:11 smithi123 ceph-mon[35971]: mon.b calling monitor election 2024-05-14T16:45:12.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:11 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:45:12.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:11 smithi123 ceph-mon[35971]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:12.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:11 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:45:12.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:11 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:45:12.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:11 smithi123 ceph-mon[35971]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-14T16:45:12.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:11 smithi123 ceph-mon[35971]: monmap epoch 3 2024-05-14T16:45:12.162 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:11 smithi123 ceph-mon[35971]: fsid ed40717a-1210-11ef-bc98-c7b262605968 2024-05-14T16:45:12.162 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:11 smithi123 ceph-mon[35971]: last_changed 2024-05-14T16:45:06.727143+0000 2024-05-14T16:45:12.162 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:11 smithi123 ceph-mon[35971]: created 2024-05-14T16:43:06.575008+0000 2024-05-14T16:45:12.162 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:11 smithi123 ceph-mon[35971]: min_mon_release 19 (squid) 2024-05-14T16:45:12.162 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:11 smithi123 ceph-mon[35971]: election_strategy: 1 2024-05-14T16:45:12.162 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:11 smithi123 ceph-mon[35971]: 0: [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon.a 2024-05-14T16:45:12.162 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:11 smithi123 ceph-mon[35971]: 1: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.c 2024-05-14T16:45:12.162 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:11 smithi123 ceph-mon[35971]: 2: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.b 2024-05-14T16:45:12.162 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:11 smithi123 ceph-mon[35971]: fsmap 2024-05-14T16:45:12.162 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:11 smithi123 ceph-mon[35971]: osdmap e4: 0 total, 0 up, 0 in 2024-05-14T16:45:12.162 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:11 smithi123 ceph-mon[35971]: mgrmap e13: x(active, since 59s) 2024-05-14T16:45:12.162 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:11 smithi123 ceph-mon[35971]: overall HEALTH_OK 2024-05-14T16:45:12.162 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:11 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:12.162 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:11 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:12.162 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:11 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:12.163 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:11 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:12.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:11 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:45:12.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:11 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:45:12.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:11 smithi002 ceph-mon[31538]: mon.a calling monitor election 2024-05-14T16:45:12.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:11 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:45:12.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:11 smithi002 ceph-mon[31538]: mon.c calling monitor election 2024-05-14T16:45:12.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:11 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:45:12.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:11 smithi002 ceph-mon[31538]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:12.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:11 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:45:12.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:11 smithi002 ceph-mon[31538]: mon.b calling monitor election 2024-05-14T16:45:12.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:11 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:45:12.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:11 smithi002 ceph-mon[31538]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:12.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:11 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:45:12.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:11 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:45:12.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:11 smithi002 ceph-mon[31538]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-14T16:45:12.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:11 smithi002 ceph-mon[31538]: monmap epoch 3 2024-05-14T16:45:12.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:11 smithi002 ceph-mon[31538]: fsid ed40717a-1210-11ef-bc98-c7b262605968 2024-05-14T16:45:12.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:11 smithi002 ceph-mon[31538]: last_changed 2024-05-14T16:45:06.727143+0000 2024-05-14T16:45:12.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:11 smithi002 ceph-mon[31538]: created 2024-05-14T16:43:06.575008+0000 2024-05-14T16:45:12.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:11 smithi002 ceph-mon[31538]: min_mon_release 19 (squid) 2024-05-14T16:45:12.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:11 smithi002 ceph-mon[31538]: election_strategy: 1 2024-05-14T16:45:12.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:11 smithi002 ceph-mon[31538]: 0: [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon.a 2024-05-14T16:45:12.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:11 smithi002 ceph-mon[31538]: 1: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.c 2024-05-14T16:45:12.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:11 smithi002 ceph-mon[31538]: 2: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.b 2024-05-14T16:45:12.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:11 smithi002 ceph-mon[31538]: fsmap 2024-05-14T16:45:12.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:11 smithi002 ceph-mon[31538]: osdmap e4: 0 total, 0 up, 0 in 2024-05-14T16:45:12.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:11 smithi002 ceph-mon[31538]: mgrmap e13: x(active, since 59s) 2024-05-14T16:45:12.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:11 smithi002 ceph-mon[31538]: overall HEALTH_OK 2024-05-14T16:45:12.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:11 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:12.247 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:11 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:12.247 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:11 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:12.247 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:11 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:12.677 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-14T16:45:12.677 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph config generate-minimal-conf 2024-05-14T16:45:12.935 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:45:12.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:12 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:45:12.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:12 smithi002 ceph-mon[31538]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:12.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:12 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:12.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:12 smithi002 ceph-mon[31538]: from='client.? 172.21.15.151:0/2984405184' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-14T16:45:12.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:12 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:45:13.069 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:12 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:45:13.069 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:12 smithi151 ceph-mon[36066]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:13.069 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:12 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:13.069 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:12 smithi151 ceph-mon[36066]: from='client.? 172.21.15.151:0/2984405184' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-14T16:45:13.069 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:12 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:45:13.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:12 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:45:13.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:12 smithi123 ceph-mon[35971]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:13.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:12 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:13.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:12 smithi123 ceph-mon[35971]: from='client.? 172.21.15.151:0/2984405184' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-14T16:45:13.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:12 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:45:14.333 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:14 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:14.333 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:14 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:14.333 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:14 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:14.333 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:14 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:14.334 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:14 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:14.334 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:14 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:45:14.334 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:14 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:14.334 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:14 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:14.334 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:14 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:14.334 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:14 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:14.334 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:14 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:14.334 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:14 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:14.334 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:14 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:14.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:14 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:14.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:14 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:14.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:14 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:14.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:14 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:14.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:14 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:14.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:14 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:45:14.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:14 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:14.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:14 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:14.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:14 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:14.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:14 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:14.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:14 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:14.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:14 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:14.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:14 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:14.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.584+0000 7f61e8150700 1 -- 172.21.15.2:0/1956273454 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f61e01029a0 msgr2=0x7f61e0102d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:14.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.584+0000 7f61e8150700 1 --2- 172.21.15.2:0/1956273454 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f61e01029a0 0x7f61e0102d80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f61dc009910 tx=0x7f61dc009c20 comp rx=0 tx=0).stop 2024-05-14T16:45:14.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.584+0000 7f61e8150700 1 -- 172.21.15.2:0/1956273454 shutdown_connections 2024-05-14T16:45:14.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.584+0000 7f61e8150700 1 --2- 172.21.15.2:0/1956273454 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f61e01029a0 0x7f61e0102d80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:14.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.584+0000 7f61e8150700 1 -- 172.21.15.2:0/1956273454 >> 172.21.15.2:0/1956273454 conn(0x7f61e00779d0 msgr2=0x7f61e0077de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:45:14.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.584+0000 7f61e8150700 1 -- 172.21.15.2:0/1956273454 shutdown_connections 2024-05-14T16:45:14.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.584+0000 7f61e8150700 1 -- 172.21.15.2:0/1956273454 wait complete. 2024-05-14T16:45:14.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.585+0000 7f61e8150700 1 Processor -- start 2024-05-14T16:45:14.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.585+0000 7f61e8150700 1 -- start start 2024-05-14T16:45:14.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.585+0000 7f61e8150700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61e01029a0 0x7f61e019aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:14.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.585+0000 7f61e8150700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f61e019b420 0x7f61e019f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:14.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.585+0000 7f61e8150700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61e019fcf0 0x7f61e01a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:14.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.585+0000 7f61e8150700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f61e010f680 con 0x7f61e019b420 2024-05-14T16:45:14.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.585+0000 7f61e8150700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f61e010f500 con 0x7f61e019fcf0 2024-05-14T16:45:14.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.585+0000 7f61e8150700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f61e010f800 con 0x7f61e01029a0 2024-05-14T16:45:14.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.585+0000 7f61e56eb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f61e019b420 0x7f61e019f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:14.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.586+0000 7f61e56eb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f61e019b420 0x7f61e019f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59312/0 (socket says 172.21.15.2:59312) 2024-05-14T16:45:14.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.586+0000 7f61e56eb700 1 -- 172.21.15.2:0/476791329 learned_addr learned my addr 172.21.15.2:0/476791329 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:45:14.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.586+0000 7f61e5eec700 1 --2- 172.21.15.2:0/476791329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61e01029a0 0x7f61e019aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:14.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.586+0000 7f61e66ed700 1 --2- 172.21.15.2:0/476791329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61e019fcf0 0x7f61e01a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:14.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.586+0000 7f61e56eb700 1 -- 172.21.15.2:0/476791329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61e01029a0 msgr2=0x7f61e019aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:14.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.586+0000 7f61e56eb700 1 --2- 172.21.15.2:0/476791329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61e01029a0 0x7f61e019aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:14.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.586+0000 7f61e56eb700 1 -- 172.21.15.2:0/476791329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61e019fcf0 msgr2=0x7f61e01a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:14.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.586+0000 7f61e56eb700 1 --2- 172.21.15.2:0/476791329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61e019fcf0 0x7f61e01a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:14.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.586+0000 7f61e56eb700 1 -- 172.21.15.2:0/476791329 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e01a7a60 con 0x7f61e019b420 2024-05-14T16:45:14.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.586+0000 7f61e56eb700 1 --2- 172.21.15.2:0/476791329 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f61e019b420 0x7f61e019f7b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f61d000b780 tx=0x7f61d000ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:45:14.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.586+0000 7f61d6ffd700 1 -- 172.21.15.2:0/476791329 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d0016070 con 0x7f61e019b420 2024-05-14T16:45:14.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.586+0000 7f61d6ffd700 1 -- 172.21.15.2:0/476791329 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61d00036a0 con 0x7f61e019b420 2024-05-14T16:45:14.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.586+0000 7f61d6ffd700 1 -- 172.21.15.2:0/476791329 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d00119c0 con 0x7f61e019b420 2024-05-14T16:45:14.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.587+0000 7f61e8150700 1 -- 172.21.15.2:0/476791329 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e01a7c90 con 0x7f61e019b420 2024-05-14T16:45:14.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.587+0000 7f61e8150700 1 -- 172.21.15.2:0/476791329 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e01a81c0 con 0x7f61e019b420 2024-05-14T16:45:14.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.588+0000 7f61d6ffd700 1 -- 172.21.15.2:0/476791329 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f61d0019070 con 0x7f61e019b420 2024-05-14T16:45:14.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.588+0000 7f61e8150700 1 -- 172.21.15.2:0/476791329 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61e0064ba0 con 0x7f61e019b420 2024-05-14T16:45:14.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.588+0000 7f61d6ffd700 1 --2- 172.21.15.2:0/476791329 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f61cc03e9a0 0x7f61cc040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:14.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.588+0000 7f61d6ffd700 1 -- 172.21.15.2:0/476791329 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f61d0051380 con 0x7f61e019b420 2024-05-14T16:45:14.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.588+0000 7f61e5eec700 1 --2- 172.21.15.2:0/476791329 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f61cc03e9a0 0x7f61cc040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:14.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.589+0000 7f61e5eec700 1 --2- 172.21.15.2:0/476791329 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f61cc03e9a0 0x7f61cc040e60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f61dc006010 tx=0x7f61dc01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:45:14.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.598+0000 7f61d6ffd700 1 -- 172.21.15.2:0/476791329 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f61d0019360 con 0x7f61e019b420 2024-05-14T16:45:14.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:14 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:14.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:14 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:14.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:14 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:14.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:14 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:14.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:14 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:14.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:14 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:45:14.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:14 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:14.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:14 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:14.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:14 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:14.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:14 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:14.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:14 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:14.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:14 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:14.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:14 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:14.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.822+0000 7f61e8150700 1 -- 172.21.15.2:0/476791329 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f61e01a8560 con 0x7f61e019b420 2024-05-14T16:45:14.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.823+0000 7f61d6ffd700 1 -- 172.21.15.2:0/476791329 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f61d0018db0 con 0x7f61e019b420 2024-05-14T16:45:14.823 INFO:teuthology.orchestra.run.smithi002.stdout:# minimal ceph.conf for ed40717a-1210-11ef-bc98-c7b262605968 2024-05-14T16:45:14.823 INFO:teuthology.orchestra.run.smithi002.stdout:[global] 2024-05-14T16:45:14.823 INFO:teuthology.orchestra.run.smithi002.stdout: fsid = ed40717a-1210-11ef-bc98-c7b262605968 2024-05-14T16:45:14.823 INFO:teuthology.orchestra.run.smithi002.stdout: mon_host = [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] 2024-05-14T16:45:14.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.825+0000 7f61e8150700 1 -- 172.21.15.2:0/476791329 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f61cc03e9a0 msgr2=0x7f61cc040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:14.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.825+0000 7f61e8150700 1 --2- 172.21.15.2:0/476791329 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f61cc03e9a0 0x7f61cc040e60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f61dc006010 tx=0x7f61dc01a040 comp rx=0 tx=0).stop 2024-05-14T16:45:14.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.825+0000 7f61e8150700 1 -- 172.21.15.2:0/476791329 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f61e019b420 msgr2=0x7f61e019f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:14.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.825+0000 7f61e8150700 1 --2- 172.21.15.2:0/476791329 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f61e019b420 0x7f61e019f7b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f61d000b780 tx=0x7f61d000ba90 comp rx=0 tx=0).stop 2024-05-14T16:45:14.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.825+0000 7f61e8150700 1 -- 172.21.15.2:0/476791329 shutdown_connections 2024-05-14T16:45:14.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.825+0000 7f61e8150700 1 --2- 172.21.15.2:0/476791329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61e019fcf0 0x7f61e01a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:14.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.825+0000 7f61e8150700 1 --2- 172.21.15.2:0/476791329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61e01029a0 0x7f61e019aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:14.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.825+0000 7f61e8150700 1 --2- 172.21.15.2:0/476791329 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f61cc03e9a0 0x7f61cc040e60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:14.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.825+0000 7f61e8150700 1 --2- 172.21.15.2:0/476791329 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f61e019b420 0x7f61e019f7b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:14.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.825+0000 7f61e8150700 1 -- 172.21.15.2:0/476791329 >> 172.21.15.2:0/476791329 conn(0x7f61e00779d0 msgr2=0x7f61e0100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:45:14.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.825+0000 7f61e8150700 1 -- 172.21.15.2:0/476791329 shutdown_connections 2024-05-14T16:45:14.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:14.825+0000 7f61e8150700 1 -- 172.21.15.2:0/476791329 wait complete. 2024-05-14T16:45:15.451 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:15 smithi002 ceph-mon[31538]: Updating smithi002:/etc/ceph/ceph.conf 2024-05-14T16:45:15.451 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:15 smithi002 ceph-mon[31538]: Updating smithi123:/etc/ceph/ceph.conf 2024-05-14T16:45:15.451 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:15 smithi002 ceph-mon[31538]: Updating smithi151:/etc/ceph/ceph.conf 2024-05-14T16:45:15.451 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:15 smithi002 ceph-mon[31538]: Updating smithi002:/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:45:15.451 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:15 smithi002 ceph-mon[31538]: Updating smithi151:/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:45:15.451 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:15 smithi002 ceph-mon[31538]: Updating smithi123:/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:45:15.451 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:15 smithi002 ceph-mon[31538]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:15.451 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:15 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:15.451 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:15 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:15.451 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:15 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:15.451 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:15 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:15.451 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:15 smithi002 ceph-mon[31538]: Reconfiguring mon.a (unknown last config time)... 2024-05-14T16:45:15.452 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:15 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T16:45:15.452 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:15 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T16:45:15.452 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:15 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:15.452 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:15 smithi002 ceph-mon[31538]: Reconfiguring daemon mon.a on smithi002 2024-05-14T16:45:15.452 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:15 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/476791329' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:15.453 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-14T16:45:15.453 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T16:45:15.453 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/ceph/ceph.conf 2024-05-14T16:45:15.483 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T16:45:15.484 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:45:15.547 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:15 smithi123 ceph-mon[35971]: Updating smithi002:/etc/ceph/ceph.conf 2024-05-14T16:45:15.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:15 smithi123 ceph-mon[35971]: Updating smithi123:/etc/ceph/ceph.conf 2024-05-14T16:45:15.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:15 smithi123 ceph-mon[35971]: Updating smithi151:/etc/ceph/ceph.conf 2024-05-14T16:45:15.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:15 smithi123 ceph-mon[35971]: Updating smithi002:/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:45:15.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:15 smithi123 ceph-mon[35971]: Updating smithi151:/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:45:15.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:15 smithi123 ceph-mon[35971]: Updating smithi123:/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:45:15.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:15 smithi123 ceph-mon[35971]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:15.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:15 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:15.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:15 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:15.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:15 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:15.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:15 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:15.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:15 smithi123 ceph-mon[35971]: Reconfiguring mon.a (unknown last config time)... 2024-05-14T16:45:15.548 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:15 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T16:45:15.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:15 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T16:45:15.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:15 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:15.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:15 smithi123 ceph-mon[35971]: Reconfiguring daemon mon.a on smithi002 2024-05-14T16:45:15.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:15 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/476791329' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:15.550 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-05-14T16:45:15.550 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.conf 2024-05-14T16:45:15.580 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-05-14T16:45:15.580 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:45:15.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:15 smithi151 ceph-mon[36066]: Updating smithi002:/etc/ceph/ceph.conf 2024-05-14T16:45:15.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:15 smithi151 ceph-mon[36066]: Updating smithi123:/etc/ceph/ceph.conf 2024-05-14T16:45:15.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:15 smithi151 ceph-mon[36066]: Updating smithi151:/etc/ceph/ceph.conf 2024-05-14T16:45:15.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:15 smithi151 ceph-mon[36066]: Updating smithi002:/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:45:15.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:15 smithi151 ceph-mon[36066]: Updating smithi151:/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:45:15.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:15 smithi151 ceph-mon[36066]: Updating smithi123:/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:45:15.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:15 smithi151 ceph-mon[36066]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:15.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:15 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:15.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:15 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:15.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:15 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:15.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:15 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:15.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:15 smithi151 ceph-mon[36066]: Reconfiguring mon.a (unknown last config time)... 2024-05-14T16:45:15.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:15 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T16:45:15.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:15 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T16:45:15.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:15 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:15.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:15 smithi151 ceph-mon[36066]: Reconfiguring daemon mon.a on smithi002 2024-05-14T16:45:15.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:15 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/476791329' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:15.646 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-14T16:45:15.647 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/ceph/ceph.conf 2024-05-14T16:45:15.679 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-14T16:45:15.679 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:45:15.746 INFO:tasks.cephadm:Adding mgr.x on smithi002 2024-05-14T16:45:15.747 INFO:tasks.cephadm:Adding mgr.y on smithi123 2024-05-14T16:45:15.747 INFO:tasks.cephadm:Adding mgr.z on smithi151 2024-05-14T16:45:15.747 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph orch apply mgr '3;smithi002=x;smithi123=y;smithi151=z' 2024-05-14T16:45:16.020 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T16:45:17.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.456+0000 7ff17fe18700 1 -- 172.21.15.151:0/3397056862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff178101fe0 msgr2=0x7ff178102400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:17.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.456+0000 7ff17fe18700 1 --2- 172.21.15.151:0/3397056862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff178101fe0 0x7ff178102400 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff174009910 tx=0x7ff174009c20 comp rx=0 tx=0).stop 2024-05-14T16:45:17.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.456+0000 7ff17fe18700 1 -- 172.21.15.151:0/3397056862 shutdown_connections 2024-05-14T16:45:17.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.456+0000 7ff17fe18700 1 --2- 172.21.15.151:0/3397056862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff178101fe0 0x7ff178102400 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:17.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.456+0000 7ff17fe18700 1 -- 172.21.15.151:0/3397056862 >> 172.21.15.151:0/3397056862 conn(0x7ff1780fd580 msgr2=0x7ff1780ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:45:17.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.456+0000 7ff17fe18700 1 -- 172.21.15.151:0/3397056862 shutdown_connections 2024-05-14T16:45:17.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.456+0000 7ff17fe18700 1 -- 172.21.15.151:0/3397056862 wait complete. 2024-05-14T16:45:17.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.456+0000 7ff17fe18700 1 Processor -- start 2024-05-14T16:45:17.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.456+0000 7ff17fe18700 1 -- start start 2024-05-14T16:45:17.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.457+0000 7ff17fe18700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff178101fe0 0x7ff1780ff520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:17.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.457+0000 7ff17fe18700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1780fdb70 0x7ff1780fdff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:17.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.457+0000 7ff17fe18700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1780fe530 0x7ff1781a28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:17.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.457+0000 7ff17fe18700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff178107f20 con 0x7ff1780fdb70 2024-05-14T16:45:17.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.457+0000 7ff17fe18700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff178107da0 con 0x7ff1780fe530 2024-05-14T16:45:17.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.457+0000 7ff17fe18700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff1781080a0 con 0x7ff178101fe0 2024-05-14T16:45:17.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.457+0000 7ff17dbb4700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff178101fe0 0x7ff1780ff520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:17.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.457+0000 7ff17dbb4700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff178101fe0 0x7ff1780ff520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:34790/0 (socket says 172.21.15.151:34790) 2024-05-14T16:45:17.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.457+0000 7ff17dbb4700 1 -- 172.21.15.151:0/2561360862 learned_addr learned my addr 172.21.15.151:0/2561360862 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-14T16:45:17.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.457+0000 7ff17e3b5700 1 --2- 172.21.15.151:0/2561360862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1780fe530 0x7ff1781a28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:17.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.457+0000 7ff17d3b3700 1 --2- 172.21.15.151:0/2561360862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1780fdb70 0x7ff1780fdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:17.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.457+0000 7ff17dbb4700 1 -- 172.21.15.151:0/2561360862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1780fe530 msgr2=0x7ff1781a28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:17.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.457+0000 7ff17dbb4700 1 --2- 172.21.15.151:0/2561360862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1780fe530 0x7ff1781a28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:17.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.457+0000 7ff17dbb4700 1 -- 172.21.15.151:0/2561360862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1780fdb70 msgr2=0x7ff1780fdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:17.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.457+0000 7ff17dbb4700 1 --2- 172.21.15.151:0/2561360862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1780fdb70 0x7ff1780fdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:17.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.457+0000 7ff17dbb4700 1 -- 172.21.15.151:0/2561360862 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1781a2ea0 con 0x7ff178101fe0 2024-05-14T16:45:17.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.457+0000 7ff17e3b5700 1 --2- 172.21.15.151:0/2561360862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1780fe530 0x7ff1781a28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:45:17.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.457+0000 7ff17dbb4700 1 --2- 172.21.15.151:0/2561360862 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff178101fe0 0x7ff1780ff520 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff1740059d0 tx=0x7ff1740050d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:45:17.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.457+0000 7ff16effd700 1 -- 172.21.15.151:0/2561360862 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff174027070 con 0x7ff178101fe0 2024-05-14T16:45:17.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.457+0000 7ff16effd700 1 -- 172.21.15.151:0/2561360862 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff17400be10 con 0x7ff178101fe0 2024-05-14T16:45:17.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.457+0000 7ff16effd700 1 -- 172.21.15.151:0/2561360862 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff174022e20 con 0x7ff178101fe0 2024-05-14T16:45:17.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.457+0000 7ff17fe18700 1 -- 172.21.15.151:0/2561360862 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1781a3130 con 0x7ff178101fe0 2024-05-14T16:45:17.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.457+0000 7ff17fe18700 1 -- 172.21.15.151:0/2561360862 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1781a36e0 con 0x7ff178101fe0 2024-05-14T16:45:17.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.458+0000 7ff16effd700 1 -- 172.21.15.151:0/2561360862 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7ff17402a070 con 0x7ff178101fe0 2024-05-14T16:45:17.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.458+0000 7ff17fe18700 1 -- 172.21.15.151:0/2561360862 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff178064ba0 con 0x7ff178101fe0 2024-05-14T16:45:17.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.458+0000 7ff16effd700 1 --2- 172.21.15.151:0/2561360862 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7ff16403e9a0 0x7ff164040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:17.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.459+0000 7ff16effd700 1 -- 172.21.15.151:0/2561360862 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff174053880 con 0x7ff178101fe0 2024-05-14T16:45:17.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.459+0000 7ff17d3b3700 1 --2- 172.21.15.151:0/2561360862 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7ff16403e9a0 0x7ff164040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:17.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.459+0000 7ff17d3b3700 1 --2- 172.21.15.151:0/2561360862 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7ff16403e9a0 0x7ff164040e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff168009850 tx=0x7ff168008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:45:17.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.464+0000 7ff16effd700 1 -- 172.21.15.151:0/2561360862 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff174013070 con 0x7ff178101fe0 2024-05-14T16:45:17.551 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:17 smithi123 ceph-mon[35971]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:17.551 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:17 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:17.551 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:17 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:17.551 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:17 smithi123 ceph-mon[35971]: Reconfiguring mon.b (monmap changed)... 2024-05-14T16:45:17.551 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:17 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T16:45:17.551 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:17 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T16:45:17.551 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:17 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:17.551 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:17 smithi123 ceph-mon[35971]: Reconfiguring daemon mon.b on smithi123 2024-05-14T16:45:17.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:17 smithi151 ceph-mon[36066]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:17.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:17 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:17.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:17 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:17.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:17 smithi151 ceph-mon[36066]: Reconfiguring mon.b (monmap changed)... 2024-05-14T16:45:17.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:17 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T16:45:17.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:17 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T16:45:17.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:17 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:17.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:17 smithi151 ceph-mon[36066]: Reconfiguring daemon mon.b on smithi123 2024-05-14T16:45:17.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.681+0000 7ff17fe18700 1 -- 172.21.15.151:0/2561360862 --> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi002=x;smithi123=y;smithi151=z", "target": ["mon-mgr", ""]}) -- 0x7ff178063980 con 0x7ff16403e9a0 2024-05-14T16:45:17.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.688+0000 7ff16effd700 1 -- 172.21.15.151:0/2561360862 <== mgr.14150 v2:172.21.15.2:6800/1498936028 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff178063980 con 0x7ff16403e9a0 2024-05-14T16:45:17.689 INFO:teuthology.orchestra.run.smithi151.stdout:Scheduled mgr update... 2024-05-14T16:45:17.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.691+0000 7ff17fe18700 1 -- 172.21.15.151:0/2561360862 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7ff16403e9a0 msgr2=0x7ff164040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:17.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.691+0000 7ff17fe18700 1 --2- 172.21.15.151:0/2561360862 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7ff16403e9a0 0x7ff164040e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff168009850 tx=0x7ff168008040 comp rx=0 tx=0).stop 2024-05-14T16:45:17.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.691+0000 7ff17fe18700 1 -- 172.21.15.151:0/2561360862 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff178101fe0 msgr2=0x7ff1780ff520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:17.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.691+0000 7ff17fe18700 1 --2- 172.21.15.151:0/2561360862 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff178101fe0 0x7ff1780ff520 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff1740059d0 tx=0x7ff1740050d0 comp rx=0 tx=0).stop 2024-05-14T16:45:17.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.691+0000 7ff17fe18700 1 -- 172.21.15.151:0/2561360862 shutdown_connections 2024-05-14T16:45:17.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.691+0000 7ff17fe18700 1 --2- 172.21.15.151:0/2561360862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1780fe530 0x7ff1781a28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:17.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.691+0000 7ff17fe18700 1 --2- 172.21.15.151:0/2561360862 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff178101fe0 0x7ff1780ff520 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:17.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.691+0000 7ff17fe18700 1 --2- 172.21.15.151:0/2561360862 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7ff16403e9a0 0x7ff164040e60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:17.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.691+0000 7ff17fe18700 1 --2- 172.21.15.151:0/2561360862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1780fdb70 0x7ff1780fdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:17.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.691+0000 7ff17fe18700 1 -- 172.21.15.151:0/2561360862 >> 172.21.15.151:0/2561360862 conn(0x7ff1780fd580 msgr2=0x7ff17818f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:45:17.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.691+0000 7ff17fe18700 1 -- 172.21.15.151:0/2561360862 shutdown_connections 2024-05-14T16:45:17.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:45:17.691+0000 7ff17fe18700 1 -- 172.21.15.151:0/2561360862 wait complete. 2024-05-14T16:45:17.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:17 smithi002 ceph-mon[31538]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:17.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:17 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:17.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:17 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:17.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:17 smithi002 ceph-mon[31538]: Reconfiguring mon.b (monmap changed)... 2024-05-14T16:45:17.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:17 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T16:45:17.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:17 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T16:45:17.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:17 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:17.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:17 smithi002 ceph-mon[31538]: Reconfiguring daemon mon.b on smithi123 2024-05-14T16:45:18.261 DEBUG:teuthology.orchestra.run.smithi123:mgr.y> sudo journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@mgr.y.service 2024-05-14T16:45:18.263 DEBUG:teuthology.orchestra.run.smithi151:mgr.z> sudo journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@mgr.z.service 2024-05-14T16:45:18.265 INFO:tasks.cephadm:Deploying OSDs... 2024-05-14T16:45:18.265 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T16:45:18.265 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/scratch_devs of=/dev/stdout 2024-05-14T16:45:18.286 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-14T16:45:18.286 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_1 2024-05-14T16:45:18.346 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-14T16:45:18.346 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T16:45:18.346 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-05-14T16:45:18.346 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T16:45:18.346 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T16:45:18.347 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2024-05-14 16:44:24.736900355 +0000 2024-05-14T16:45:18.347 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2024-05-14 16:39:42.623866122 +0000 2024-05-14T16:45:18.347 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2024-05-14 16:39:42.623866122 +0000 2024-05-14T16:45:18.347 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: 2024-05-14 16:39:42.623866122 +0000 2024-05-14T16:45:18.347 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-14T16:45:18.417 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2024-05-14T16:45:18.417 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2024-05-14T16:45:18.417 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 0.000113145 s, 4.5 MB/s 2024-05-14T16:45:18.419 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-14T16:45:18.479 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_2 2024-05-14T16:45:18.537 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-14T16:45:18.537 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T16:45:18.538 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-05-14T16:45:18.538 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T16:45:18.538 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T16:45:18.538 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2024-05-14 16:44:24.770899605 +0000 2024-05-14T16:45:18.538 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2024-05-14 16:39:42.963858506 +0000 2024-05-14T16:45:18.538 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2024-05-14 16:39:42.963858506 +0000 2024-05-14T16:45:18.538 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: 2024-05-14 16:39:42.963858506 +0000 2024-05-14T16:45:18.538 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-14T16:45:18.604 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2024-05-14T16:45:18.605 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2024-05-14T16:45:18.605 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 0.000104982 s, 4.9 MB/s 2024-05-14T16:45:18.606 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-14T16:45:18.665 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_3 2024-05-14T16:45:18.723 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-14T16:45:18.723 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T16:45:18.723 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-05-14T16:45:18.724 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T16:45:18.724 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T16:45:18.724 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2024-05-14 16:44:24.814898635 +0000 2024-05-14T16:45:18.724 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2024-05-14 16:39:43.308850778 +0000 2024-05-14T16:45:18.724 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2024-05-14 16:39:43.308850778 +0000 2024-05-14T16:45:18.724 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: 2024-05-14 16:39:43.308850778 +0000 2024-05-14T16:45:18.724 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-14T16:45:18.735 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:18 smithi151 ceph-mon[36066]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi002=x;smithi123=y;smithi151=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:45:18.735 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:18 smithi151 ceph-mon[36066]: Saving service mgr spec with placement smithi002=x;smithi123=y;smithi151=z;count:3 2024-05-14T16:45:18.735 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:18 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:18.735 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:18 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:18.735 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:18 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:18.735 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:18 smithi151 ceph-mon[36066]: Reconfiguring mon.c (monmap changed)... 2024-05-14T16:45:18.735 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:18 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T16:45:18.735 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:18 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T16:45:18.735 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:18 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:18.735 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:18 smithi151 ceph-mon[36066]: Reconfiguring daemon mon.c on smithi151 2024-05-14T16:45:18.736 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:18 smithi151 ceph-mon[36066]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:18.789 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2024-05-14T16:45:18.789 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2024-05-14T16:45:18.789 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 0.000110225 s, 4.6 MB/s 2024-05-14T16:45:18.791 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-14T16:45:18.848 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_4 2024-05-14T16:45:18.905 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-14T16:45:18.906 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T16:45:18.906 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-05-14T16:45:18.906 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T16:45:18.906 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T16:45:18.906 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2024-05-14 16:44:24.851897820 +0000 2024-05-14T16:45:18.906 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2024-05-14 16:39:43.645843228 +0000 2024-05-14T16:45:18.906 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2024-05-14 16:39:43.645843228 +0000 2024-05-14T16:45:18.906 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: 2024-05-14 16:39:43.645843228 +0000 2024-05-14T16:45:18.906 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-14T16:45:18.969 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:18 smithi002 ceph-mon[31538]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi002=x;smithi123=y;smithi151=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:45:18.969 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:18 smithi002 ceph-mon[31538]: Saving service mgr spec with placement smithi002=x;smithi123=y;smithi151=z;count:3 2024-05-14T16:45:18.969 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:18 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:18.969 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:18 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:18.969 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:18 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:18.969 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:18 smithi002 ceph-mon[31538]: Reconfiguring mon.c (monmap changed)... 2024-05-14T16:45:18.969 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:18 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T16:45:18.969 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:18 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T16:45:18.969 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:18 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:18.969 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:18 smithi002 ceph-mon[31538]: Reconfiguring daemon mon.c on smithi151 2024-05-14T16:45:18.969 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:18 smithi002 ceph-mon[31538]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:18.971 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2024-05-14T16:45:18.971 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2024-05-14T16:45:18.971 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 0.000108864 s, 4.7 MB/s 2024-05-14T16:45:18.972 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-14T16:45:19.029 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-05-14T16:45:19.029 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/scratch_devs of=/dev/stdout 2024-05-14T16:45:19.048 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-14T16:45:19.048 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_1 2024-05-14T16:45:19.105 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-14T16:45:19.106 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T16:45:19.106 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-05-14T16:45:19.106 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T16:45:19.106 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T16:45:19.106 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-05-14 16:45:06.173658148 +0000 2024-05-14T16:45:19.106 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-05-14 16:39:45.058867793 +0000 2024-05-14T16:45:19.106 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-05-14 16:39:45.058867793 +0000 2024-05-14T16:45:19.106 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-05-14 16:39:45.058867793 +0000 2024-05-14T16:45:19.106 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-14T16:45:19.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:18 smithi123 ceph-mon[35971]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi002=x;smithi123=y;smithi151=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:45:19.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:18 smithi123 ceph-mon[35971]: Saving service mgr spec with placement smithi002=x;smithi123=y;smithi151=z;count:3 2024-05-14T16:45:19.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:18 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:19.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:18 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:19.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:18 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:19.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:18 smithi123 ceph-mon[35971]: Reconfiguring mon.c (monmap changed)... 2024-05-14T16:45:19.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:18 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-14T16:45:19.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:18 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-14T16:45:19.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:18 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:19.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:18 smithi123 ceph-mon[35971]: Reconfiguring daemon mon.c on smithi151 2024-05-14T16:45:19.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:18 smithi123 ceph-mon[35971]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:19.172 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-05-14T16:45:19.172 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-05-14T16:45:19.172 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000286568 s, 1.8 MB/s 2024-05-14T16:45:19.173 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-14T16:45:19.231 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_2 2024-05-14T16:45:19.288 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-14T16:45:19.288 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T16:45:19.288 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-05-14T16:45:19.288 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T16:45:19.288 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T16:45:19.288 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-05-14 16:45:06.174658124 +0000 2024-05-14T16:45:19.288 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-05-14 16:39:45.426858944 +0000 2024-05-14T16:45:19.288 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-05-14 16:39:45.426858944 +0000 2024-05-14T16:45:19.288 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-05-14 16:39:45.426858944 +0000 2024-05-14T16:45:19.289 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-14T16:45:19.354 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-05-14T16:45:19.354 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-05-14T16:45:19.354 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000253858 s, 2.0 MB/s 2024-05-14T16:45:19.355 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-14T16:45:19.413 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_3 2024-05-14T16:45:19.470 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-14T16:45:19.470 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T16:45:19.470 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-14T16:45:19.470 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T16:45:19.470 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T16:45:19.470 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-05-14 16:45:06.174658124 +0000 2024-05-14T16:45:19.470 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-05-14 16:39:45.771850648 +0000 2024-05-14T16:45:19.470 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-05-14 16:39:45.771850648 +0000 2024-05-14T16:45:19.470 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-05-14 16:39:45.771850648 +0000 2024-05-14T16:45:19.471 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-14T16:45:19.536 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-05-14T16:45:19.536 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-05-14T16:45:19.536 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000288204 s, 1.8 MB/s 2024-05-14T16:45:19.537 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-14T16:45:19.594 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_4 2024-05-14T16:45:19.652 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-14T16:45:19.652 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T16:45:19.652 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-05-14T16:45:19.652 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T16:45:19.652 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T16:45:19.652 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-05-14 16:45:06.174658124 +0000 2024-05-14T16:45:19.652 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-05-14 16:39:46.110842496 +0000 2024-05-14T16:45:19.652 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-05-14 16:39:46.110842496 +0000 2024-05-14T16:45:19.652 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-05-14 16:39:46.110842496 +0000 2024-05-14T16:45:19.653 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-14T16:45:19.718 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-05-14T16:45:19.718 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-05-14T16:45:19.718 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000249143 s, 2.1 MB/s 2024-05-14T16:45:19.719 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-14T16:45:19.777 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-14T16:45:19.777 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/scratch_devs of=/dev/stdout 2024-05-14T16:45:19.794 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-14T16:45:19.795 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_1 2024-05-14T16:45:19.851 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-14T16:45:19.851 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T16:45:19.851 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-05-14T16:45:19.851 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T16:45:19.851 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T16:45:19.851 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-05-14 16:44:56.486424628 +0000 2024-05-14T16:45:19.852 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-05-14 16:39:48.675466167 +0000 2024-05-14T16:45:19.852 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-05-14 16:39:48.675466167 +0000 2024-05-14T16:45:19.852 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-05-14 16:39:48.675466167 +0000 2024-05-14T16:45:19.852 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-14T16:45:19.917 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-05-14T16:45:19.917 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-05-14T16:45:19.917 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000258735 s, 2.0 MB/s 2024-05-14T16:45:19.919 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-14T16:45:19.974 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_2 2024-05-14T16:45:20.030 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-14T16:45:20.030 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T16:45:20.030 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-14T16:45:20.030 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T16:45:20.030 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T16:45:20.030 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-05-14 16:44:56.486424628 +0000 2024-05-14T16:45:20.031 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-05-14 16:39:49.015461715 +0000 2024-05-14T16:45:20.031 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-05-14 16:39:49.015461715 +0000 2024-05-14T16:45:20.031 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-05-14 16:39:49.015461715 +0000 2024-05-14T16:45:20.031 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-14T16:45:20.092 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-05-14T16:45:20.092 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-05-14T16:45:20.092 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000217002 s, 2.4 MB/s 2024-05-14T16:45:20.093 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-14T16:45:20.149 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_3 2024-05-14T16:45:20.205 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-14T16:45:20.205 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T16:45:20.205 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-05-14T16:45:20.205 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T16:45:20.205 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T16:45:20.205 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-05-14 16:44:56.486424628 +0000 2024-05-14T16:45:20.205 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-05-14 16:39:49.358457225 +0000 2024-05-14T16:45:20.205 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-05-14 16:39:49.358457225 +0000 2024-05-14T16:45:20.205 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-05-14 16:39:49.358457225 +0000 2024-05-14T16:45:20.205 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-14T16:45:20.267 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-05-14T16:45:20.267 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-05-14T16:45:20.267 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000176222 s, 2.9 MB/s 2024-05-14T16:45:20.268 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-14T16:45:20.324 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_4 2024-05-14T16:45:20.380 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-14T16:45:20.380 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-14T16:45:20.380 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-14T16:45:20.380 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-14T16:45:20.380 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-05-14T16:45:20.380 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-05-14 16:44:56.487424615 +0000 2024-05-14T16:45:20.380 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-05-14 16:39:49.680453009 +0000 2024-05-14T16:45:20.380 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-05-14 16:39:49.680453009 +0000 2024-05-14T16:45:20.380 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-05-14 16:39:49.680453009 +0000 2024-05-14T16:45:20.381 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-14T16:45:20.442 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-05-14T16:45:20.442 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-05-14T16:45:20.442 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000228931 s, 2.2 MB/s 2024-05-14T16:45:20.444 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-14T16:45:20.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:20 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:20.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:20 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:20.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:20 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:45:20.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:20 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:20.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:20 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:45:20.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:20 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:20.495 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:20 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T16:45:20.495 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:20 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-14T16:45:20.495 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:20 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T16:45:20.495 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:20 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:20.499 INFO:tasks.cephadm:Deploying osd.0 on smithi002 with /dev/vg_nvme/lv_4... 2024-05-14T16:45:20.499 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-14T16:45:20.523 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:20 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:20.523 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:20 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:20.523 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:20 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:45:20.524 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:20 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:20.524 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:20 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:45:20.524 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:20 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:20.524 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:20 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T16:45:20.524 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:20 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-14T16:45:20.524 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:20 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T16:45:20.524 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:20 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:20.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:20 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:20.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:20 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:20.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:20 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:45:20.661 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:20 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:20.661 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:20 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:45:20.661 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:20 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:20.661 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:20 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T16:45:20.661 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:20 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-14T16:45:20.661 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:20 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T16:45:20.661 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:20 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:20.750 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:45:21.469 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:21 smithi151 ceph-mon[36066]: Deploying daemon mgr.z on smithi151 2024-05-14T16:45:21.469 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:21 smithi151 ceph-mon[36066]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:21.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:21 smithi002 ceph-mon[31538]: Deploying daemon mgr.z on smithi151 2024-05-14T16:45:21.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:21 smithi002 ceph-mon[31538]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:21.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:21 smithi123 ceph-mon[35971]: Deploying daemon mgr.z on smithi151 2024-05-14T16:45:21.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:21 smithi123 ceph-mon[35971]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:22.104 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:45:21 smithi151 systemd[1]: Starting Ceph mgr.z for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T16:45:22.357 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:45:22 smithi151 podman[37884]: 2024-05-14 16:45:22.105186986 +0000 UTC m=+0.084965213 container create dda976e47705075fdec36aa263fadcbb67939193e22dea33e9bab4c558f85345 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T16:45:22.357 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:45:22 smithi151 podman[37884]: 2024-05-14 16:45:22.037184013 +0000 UTC m=+0.016962244 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T16:45:22.357 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:45:22 smithi151 podman[37884]: 2024-05-14 16:45:22.299414128 +0000 UTC m=+0.279192346 container init dda976e47705075fdec36aa263fadcbb67939193e22dea33e9bab4c558f85345 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True) 2024-05-14T16:45:22.357 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:45:22 smithi151 podman[37884]: 2024-05-14 16:45:22.302277995 +0000 UTC m=+0.282056223 container start dda976e47705075fdec36aa263fadcbb67939193e22dea33e9bab4c558f85345 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-14T16:45:22.357 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:45:22 smithi151 ceph-mgr[37903]: -- 172.21.15.151:0/3682647353 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f379d8e5a0 con 0x55f379d62c00 2024-05-14T16:45:22.641 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:45:22 smithi151 bash[37884]: dda976e47705075fdec36aa263fadcbb67939193e22dea33e9bab4c558f85345 2024-05-14T16:45:22.641 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:45:22 smithi151 systemd[1]: Started Ceph mgr.z for ed40717a-1210-11ef-bc98-c7b262605968. 2024-05-14T16:45:22.641 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:45:22 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:45:22.456+0000 7fa41486b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-14T16:45:22.641 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:45:22 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:45:22.640+0000 7fa41486b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:45:22.690 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-14T16:45:22.690 INFO:teuthology.orchestra.run.smithi002.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-14T16:45:22.690 INFO:teuthology.orchestra.run.smithi002.stderr: stderr: 10+0 records in 2024-05-14T16:45:22.690 INFO:teuthology.orchestra.run.smithi002.stderr:10+0 records out 2024-05-14T16:45:22.690 INFO:teuthology.orchestra.run.smithi002.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00856732 s, 1.2 GB/s 2024-05-14T16:45:22.690 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping successful for: 2024-05-14T16:45:23.052 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:45:22 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:45:22.743+0000 7fa41486b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-14T16:45:23.052 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:45:23 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:45:23.052+0000 7fa41486b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-14T16:45:23.354 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:45:23 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:45:23.137+0000 7fa41486b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-14T16:45:23.370 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi002:vg_nvme/lv_4 2024-05-14T16:45:23.623 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:45:23.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:23 smithi123 ceph-mon[35971]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:23.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:23 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:23.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:23 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:23.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:23 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:23.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:23 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T16:45:23.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:23 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-14T16:45:23.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:23 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T16:45:23.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:23 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:23.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:23 smithi123 ceph-mon[35971]: Deploying daemon mgr.y on smithi123 2024-05-14T16:45:23.725 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:23 smithi151 ceph-mon[36066]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:23.725 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:23 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:23.725 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:23 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:23.726 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:23 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:23.726 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:23 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T16:45:23.726 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:23 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-14T16:45:23.726 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:23 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T16:45:23.726 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:23 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:23.726 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:23 smithi151 ceph-mon[36066]: Deploying daemon mgr.y on smithi123 2024-05-14T16:45:23.736 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:23 smithi002 ceph-mon[31538]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:23.736 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:23 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:23.736 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:23 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:23.736 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:23 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:23.736 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:23 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T16:45:23.736 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:23 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-14T16:45:23.736 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:23 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T16:45:23.736 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:23 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:23.737 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:23 smithi002 ceph-mon[31538]: Deploying daemon mgr.y on smithi123 2024-05-14T16:45:24.659 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:45:24 smithi123 systemd[1]: Starting Ceph mgr.y for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T16:45:24.687 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:45:24 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:45:24.394+0000 7fa41486b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-14T16:45:24.688 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:45:24 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:45:24.569+0000 7fa41486b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-14T16:45:24.910 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:45:24 smithi123 podman[37323]: 2024-05-14 16:45:24.659395961 +0000 UTC m=+0.110909021 container create f4debe38cf600a509fb12f4cc8bfcb6fa0f6a9a2286fea647f146405c9ca482f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux ) 2024-05-14T16:45:24.911 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:45:24 smithi123 podman[37323]: 2024-05-14 16:45:24.565552079 +0000 UTC m=+0.017065155 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T16:45:24.911 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:45:24 smithi123 podman[37323]: 2024-05-14 16:45:24.861683519 +0000 UTC m=+0.313196576 container init f4debe38cf600a509fb12f4cc8bfcb6fa0f6a9a2286fea647f146405c9ca482f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T16:45:24.911 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:45:24 smithi123 podman[37323]: 2024-05-14 16:45:24.864521654 +0000 UTC m=+0.316034718 container start f4debe38cf600a509fb12f4cc8bfcb6fa0f6a9a2286fea647f146405c9ca482f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T16:45:24.911 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:45:24 smithi123 ceph-mgr[37342]: -- 172.21.15.123:0/75585685 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560ce383c5a0 con 0x560ce3811000 2024-05-14T16:45:25.034 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:45:24 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:45:24.745+0000 7fa41486b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-14T16:45:25.034 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:45:25 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:45:25.033+0000 7fa41486b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-14T16:45:25.208 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:45:24 smithi123 bash[37323]: f4debe38cf600a509fb12f4cc8bfcb6fa0f6a9a2286fea647f146405c9ca482f 2024-05-14T16:45:25.209 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:45:24 smithi123 systemd[1]: Started Ceph mgr.y for ed40717a-1210-11ef-bc98-c7b262605968. 2024-05-14T16:45:25.209 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:45:25 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:45:25.018+0000 7f992550c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-14T16:45:25.209 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:45:25 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:45:25.209+0000 7f992550c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:45:25.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.222+0000 7fe976ac1700 1 -- 172.21.15.2:0/3974945583 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9701024e0 msgr2=0x7fe970102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:25.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.222+0000 7fe976ac1700 1 --2- 172.21.15.2:0/3974945583 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9701024e0 0x7fe970102900 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe9580099c0 tx=0x7fe958009cd0 comp rx=0 tx=0).stop 2024-05-14T16:45:25.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.222+0000 7fe976ac1700 1 -- 172.21.15.2:0/3974945583 shutdown_connections 2024-05-14T16:45:25.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.222+0000 7fe976ac1700 1 --2- 172.21.15.2:0/3974945583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe970104080 0x7fe97010a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:25.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.222+0000 7fe976ac1700 1 --2- 172.21.15.2:0/3974945583 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9701024e0 0x7fe970102900 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:25.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.222+0000 7fe976ac1700 1 --2- 172.21.15.2:0/3974945583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9701036e0 0x7fe970103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:25.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.222+0000 7fe976ac1700 1 -- 172.21.15.2:0/3974945583 >> 172.21.15.2:0/3974945583 conn(0x7fe9700fda60 msgr2=0x7fe9700ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:45:25.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.223+0000 7fe976ac1700 1 -- 172.21.15.2:0/3974945583 shutdown_connections 2024-05-14T16:45:25.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.223+0000 7fe976ac1700 1 -- 172.21.15.2:0/3974945583 wait complete. 2024-05-14T16:45:25.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.223+0000 7fe976ac1700 1 Processor -- start 2024-05-14T16:45:25.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.223+0000 7fe976ac1700 1 -- start start 2024-05-14T16:45:25.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.224+0000 7fe976ac1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9701024e0 0x7fe97006c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:25.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.224+0000 7fe976ac1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9701036e0 0x7fe97019e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:25.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.224+0000 7fe976ac1700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe970104080 0x7fe9701a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:25.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.224+0000 7fe976ac1700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe97010dfb0 con 0x7fe9701024e0 2024-05-14T16:45:25.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.224+0000 7fe976ac1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe97010de30 con 0x7fe9701036e0 2024-05-14T16:45:25.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.224+0000 7fe976ac1700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe97010e130 con 0x7fe970104080 2024-05-14T16:45:25.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.224+0000 7fe97485d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9701024e0 0x7fe97006c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:25.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.224+0000 7fe97485d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9701024e0 0x7fe97006c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47062/0 (socket says 172.21.15.2:47062) 2024-05-14T16:45:25.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.224+0000 7fe97485d700 1 -- 172.21.15.2:0/1371771004 learned_addr learned my addr 172.21.15.2:0/1371771004 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:45:25.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.224+0000 7fe96ffff700 1 --2- 172.21.15.2:0/1371771004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9701036e0 0x7fe97019e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:25.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.224+0000 7fe97505e700 1 --2- 172.21.15.2:0/1371771004 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe970104080 0x7fe9701a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:25.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.224+0000 7fe97485d700 1 -- 172.21.15.2:0/1371771004 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe970104080 msgr2=0x7fe9701a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:25.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.224+0000 7fe97485d700 1 --2- 172.21.15.2:0/1371771004 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe970104080 0x7fe9701a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:25.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.224+0000 7fe97485d700 1 -- 172.21.15.2:0/1371771004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9701036e0 msgr2=0x7fe97019e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:25.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.224+0000 7fe97485d700 1 --2- 172.21.15.2:0/1371771004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9701036e0 0x7fe97019e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:25.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.224+0000 7fe97485d700 1 -- 172.21.15.2:0/1371771004 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9701a13b0 con 0x7fe9701024e0 2024-05-14T16:45:25.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.224+0000 7fe97485d700 1 --2- 172.21.15.2:0/1371771004 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9701024e0 0x7fe97006c7f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe95800b580 tx=0x7fe958003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:45:25.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.225+0000 7fe96dffb700 1 -- 172.21.15.2:0/1371771004 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe958009e60 con 0x7fe9701024e0 2024-05-14T16:45:25.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.225+0000 7fe96dffb700 1 -- 172.21.15.2:0/1371771004 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe95800f460 con 0x7fe9701024e0 2024-05-14T16:45:25.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.225+0000 7fe976ac1700 1 -- 172.21.15.2:0/1371771004 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9701a1640 con 0x7fe9701024e0 2024-05-14T16:45:25.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.225+0000 7fe96dffb700 1 -- 172.21.15.2:0/1371771004 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe958017650 con 0x7fe9701024e0 2024-05-14T16:45:25.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.225+0000 7fe976ac1700 1 -- 172.21.15.2:0/1371771004 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9701a1af0 con 0x7fe9701024e0 2024-05-14T16:45:25.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.226+0000 7fe96dffb700 1 -- 172.21.15.2:0/1371771004 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7fe9580052d0 con 0x7fe9701024e0 2024-05-14T16:45:25.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.226+0000 7fe976ac1700 1 -- 172.21.15.2:0/1371771004 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe97006a230 con 0x7fe9701024e0 2024-05-14T16:45:25.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.226+0000 7fe96dffb700 1 --2- 172.21.15.2:0/1371771004 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fe95c03e750 0x7fe95c040c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:25.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.226+0000 7fe96dffb700 1 -- 172.21.15.2:0/1371771004 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe9580536f0 con 0x7fe9701024e0 2024-05-14T16:45:25.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.226+0000 7fe96ffff700 1 --2- 172.21.15.2:0/1371771004 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fe95c03e750 0x7fe95c040c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:25.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.227+0000 7fe96ffff700 1 --2- 172.21.15.2:0/1371771004 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fe95c03e750 0x7fe95c040c10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe97006cd50 tx=0x7fe9600073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:45:25.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.231+0000 7fe96dffb700 1 -- 172.21.15.2:0/1371771004 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe958013070 con 0x7fe9701024e0 2024-05-14T16:45:25.338 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:45:25 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:45:25.126+0000 7fa41486b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-14T16:45:25.338 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:45:25 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:45:25.337+0000 7fa41486b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-14T16:45:25.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:25.454+0000 7fe976ac1700 1 -- 172.21.15.2:0/1371771004 --> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fe970063980 con 0x7fe95c03e750 2024-05-14T16:45:25.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:25 smithi002 ceph-mon[31538]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:25.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:25 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:25.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:25 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:25.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:25 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:25.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:25 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:25.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:25 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:45:25.604 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:45:25 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:45:25.539+0000 7fa41486b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:45:25.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:25 smithi151 ceph-mon[36066]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:25.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:25 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:25.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:25 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:25.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:25 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:25.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:25 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:25.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:25 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:45:25.651 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:45:25 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:45:25.319+0000 7f992550c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-14T16:45:25.652 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:45:25 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:45:25.652+0000 7f992550c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-14T16:45:25.652 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:25 smithi123 ceph-mon[35971]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:25.652 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:25 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:25.652 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:25 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:25.652 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:25 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:25.652 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:25 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:25.652 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:25 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:45:25.910 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:45:25 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:45:25.749+0000 7f992550c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-14T16:45:26.354 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:45:26 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:45:26.177+0000 7fa41486b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-14T16:45:26.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:26 smithi123 ceph-mon[35971]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:45:26.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:26 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:45:26.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:26 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:45:26.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:26 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:26.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:26 smithi002 ceph-mon[31538]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:45:26.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:26 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:45:26.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:26 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:45:26.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:26 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:26.842 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:45:26 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:45:26.399+0000 7fa41486b200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-14T16:45:26.842 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:26 smithi151 ceph-mon[36066]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:45:26.842 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:26 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:45:26.842 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:26 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:45:26.842 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:26 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:27.104 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:45:26 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:45:26.841+0000 7fa41486b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-14T16:45:27.104 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:45:26 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:45:26.938+0000 7fa41486b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-14T16:45:27.341 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:45:27 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:45:27.003+0000 7f992550c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-14T16:45:27.341 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:45:27 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:45:27.172+0000 7f992550c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-14T16:45:27.341 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:45:27 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:45:27.341+0000 7f992550c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-14T16:45:27.604 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:45:27 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:45:27.310+0000 7fa41486b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-14T16:45:27.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:27 smithi151 ceph-mon[36066]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:27.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:27 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:27.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:27 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:27.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:27 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:27.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:27 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:27.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:27 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:27.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:27 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:45:27.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:27 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:27.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:27 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T16:45:27.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:27 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T16:45:27.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:27 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:27.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:27 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:27.627 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:27 smithi123 ceph-mon[35971]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:27.627 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:27 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:27.627 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:27 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:27.627 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:27 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:27.627 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:27 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:27.627 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:27 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:27.627 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:27 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:45:27.627 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:27 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:27.627 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:27 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T16:45:27.628 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:27 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T16:45:27.628 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:27 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:27.628 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:27 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:27.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:27 smithi002 ceph-mon[31538]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:27.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:27 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:27.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:27 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:27.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:27 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:27.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:27 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:27.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:27 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:27.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:27 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:45:27.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:27 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:27.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:27 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-14T16:45:27.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:27 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-14T16:45:27.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:27 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:27.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:27 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:27.910 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:45:27 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:45:27.627+0000 7f992550c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-14T16:45:27.910 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:45:27 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:45:27.718+0000 7f992550c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-14T16:45:28.004 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:45:27 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:45:27.667+0000 7fa41486b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-14T16:45:28.004 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:45:27 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:45:27.759+0000 7fa41486b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-14T16:45:28.354 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:45:28 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:45:28.003+0000 7fa41486b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-14T16:45:28.354 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:45:28 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:45:28.138+0000 7fa41486b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-14T16:45:28.355 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:45:28 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:45:28.234+0000 7fa41486b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-14T16:45:28.410 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:45:27 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:45:27.919+0000 7f992550c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-14T16:45:28.410 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:45:28 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:45:28.109+0000 7f992550c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:45:28.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:28 smithi002 ceph-mon[31538]: Reconfiguring mgr.x (unknown last config time)... 2024-05-14T16:45:28.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:28 smithi002 ceph-mon[31538]: Reconfiguring daemon mgr.x on smithi002 2024-05-14T16:45:28.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:28 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/3659747013' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "153f20ec-5bcf-4786-9ede-2e512a1e3d13"}]: dispatch 2024-05-14T16:45:28.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:28 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/3659747013' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "153f20ec-5bcf-4786-9ede-2e512a1e3d13"}]': finished 2024-05-14T16:45:28.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:28 smithi002 ceph-mon[31538]: osdmap e5: 1 total, 0 up, 1 in 2024-05-14T16:45:28.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:28 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:45:28.854 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:45:28 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:45:28.565+0000 7fa41486b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-14T16:45:28.854 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:45:28 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:45:28.657+0000 7fa41486b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-14T16:45:28.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:28 smithi151 ceph-mon[36066]: Reconfiguring mgr.x (unknown last config time)... 2024-05-14T16:45:28.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:28 smithi151 ceph-mon[36066]: Reconfiguring daemon mgr.x on smithi002 2024-05-14T16:45:28.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:28 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/3659747013' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "153f20ec-5bcf-4786-9ede-2e512a1e3d13"}]: dispatch 2024-05-14T16:45:28.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:28 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/3659747013' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "153f20ec-5bcf-4786-9ede-2e512a1e3d13"}]': finished 2024-05-14T16:45:28.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:28 smithi151 ceph-mon[36066]: osdmap e5: 1 total, 0 up, 1 in 2024-05-14T16:45:28.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:28 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:45:28.910 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:45:28 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:45:28.704+0000 7f992550c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-14T16:45:28.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:28 smithi123 ceph-mon[35971]: Reconfiguring mgr.x (unknown last config time)... 2024-05-14T16:45:28.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:28 smithi123 ceph-mon[35971]: Reconfiguring daemon mgr.x on smithi002 2024-05-14T16:45:28.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:28 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/3659747013' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "153f20ec-5bcf-4786-9ede-2e512a1e3d13"}]: dispatch 2024-05-14T16:45:28.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:28 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/3659747013' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "153f20ec-5bcf-4786-9ede-2e512a1e3d13"}]': finished 2024-05-14T16:45:28.911 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:28 smithi123 ceph-mon[35971]: osdmap e5: 1 total, 0 up, 1 in 2024-05-14T16:45:28.911 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:28 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:45:29.338 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:45:28 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:45:28.923+0000 7f992550c200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-14T16:45:29.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:29 smithi151 ceph-mon[36066]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:29.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:29 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/737834351' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:45:29.605 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:45:29 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:45:29.263+0000 7fa41486b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-14T16:45:29.605 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:45:29 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:45:29.483+0000 7fa41486b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-14T16:45:29.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:29 smithi123 ceph-mon[35971]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:29.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:29 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/737834351' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:45:29.660 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:45:29 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:45:29.339+0000 7f992550c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-14T16:45:29.660 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:45:29 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:45:29.434+0000 7f992550c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-14T16:45:29.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:29 smithi002 ceph-mon[31538]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:29.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:29 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/737834351' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:45:30.154 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:45:29 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:45:29.802+0000 7f992550c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-14T16:45:30.154 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:45:30 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:45:30.154+0000 7f992550c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-14T16:45:30.410 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:45:30 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:45:30.246+0000 7f992550c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-14T16:45:30.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:30.483+0000 7fe96dffb700 1 -- 172.21.15.2:0/1371771004 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 14) ==== 99936+0+0 (secure 0 0 0) 0x7fe9580213c0 con 0x7fe9701024e0 2024-05-14T16:45:30.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:30 smithi002 ceph-mon[31538]: Standby manager daemon z started 2024-05-14T16:45:30.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:30 smithi002 ceph-mon[31538]: from='mgr.? 172.21.15.151:0/1394594548' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-14T16:45:30.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:30 smithi002 ceph-mon[31538]: from='mgr.? 172.21.15.151:0/1394594548' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T16:45:30.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:30 smithi002 ceph-mon[31538]: from='mgr.? 172.21.15.151:0/1394594548' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-14T16:45:30.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:30 smithi002 ceph-mon[31538]: from='mgr.? 172.21.15.151:0/1394594548' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T16:45:30.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:30 smithi151 ceph-mon[36066]: Standby manager daemon z started 2024-05-14T16:45:30.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:30 smithi151 ceph-mon[36066]: from='mgr.? 172.21.15.151:0/1394594548' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-14T16:45:30.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:30 smithi151 ceph-mon[36066]: from='mgr.? 172.21.15.151:0/1394594548' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T16:45:30.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:30 smithi151 ceph-mon[36066]: from='mgr.? 172.21.15.151:0/1394594548' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-14T16:45:30.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:30 smithi151 ceph-mon[36066]: from='mgr.? 172.21.15.151:0/1394594548' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T16:45:30.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:30 smithi123 ceph-mon[35971]: Standby manager daemon z started 2024-05-14T16:45:30.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:30 smithi123 ceph-mon[35971]: from='mgr.? 172.21.15.151:0/1394594548' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-14T16:45:30.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:30 smithi123 ceph-mon[35971]: from='mgr.? 172.21.15.151:0/1394594548' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T16:45:30.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:30 smithi123 ceph-mon[35971]: from='mgr.? 172.21.15.151:0/1394594548' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-14T16:45:30.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:30 smithi123 ceph-mon[35971]: from='mgr.? 172.21.15.151:0/1394594548' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T16:45:30.910 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:45:30 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:45:30.486+0000 7f992550c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-14T16:45:30.911 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:45:30 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:45:30.618+0000 7f992550c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-14T16:45:30.911 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:45:30 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:45:30.712+0000 7f992550c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-14T16:45:31.410 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:45:31 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:45:31.034+0000 7f992550c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-14T16:45:31.410 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:45:31 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:45:31.126+0000 7f992550c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-14T16:45:31.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:31 smithi002 ceph-mon[31538]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:31.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:31 smithi002 ceph-mon[31538]: mgrmap e14: x(active, since 78s), standbys: z 2024-05-14T16:45:31.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:31 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-14T16:45:31.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:31 smithi151 ceph-mon[36066]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:31.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:31 smithi151 ceph-mon[36066]: mgrmap e14: x(active, since 78s), standbys: z 2024-05-14T16:45:31.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:31 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-14T16:45:31.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:31 smithi123 ceph-mon[35971]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:31.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:31 smithi123 ceph-mon[35971]: mgrmap e14: x(active, since 78s), standbys: z 2024-05-14T16:45:31.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:31 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-14T16:45:31.910 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:45:31 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:45:31.724+0000 7f992550c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-14T16:45:32.410 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:45:31 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:45:31.938+0000 7f992550c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-14T16:45:32.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:32.493+0000 7fe96dffb700 1 -- 172.21.15.2:0/1371771004 <== mon.0 v2:172.21.15.2:3300/0 8 ==== mgrmap(e 15) ==== 149595+0+0 (secure 0 0 0) 0x7fe958048980 con 0x7fe9701024e0 2024-05-14T16:45:32.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:32 smithi002 ceph-mon[31538]: Standby manager daemon y started 2024-05-14T16:45:32.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:32 smithi002 ceph-mon[31538]: from='mgr.? 172.21.15.123:0/717528753' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-14T16:45:32.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:32 smithi002 ceph-mon[31538]: from='mgr.? 172.21.15.123:0/717528753' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T16:45:32.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:32 smithi002 ceph-mon[31538]: from='mgr.? 172.21.15.123:0/717528753' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-14T16:45:32.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:32 smithi002 ceph-mon[31538]: from='mgr.? 172.21.15.123:0/717528753' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T16:45:32.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:32 smithi151 ceph-mon[36066]: Standby manager daemon y started 2024-05-14T16:45:32.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:32 smithi151 ceph-mon[36066]: from='mgr.? 172.21.15.123:0/717528753' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-14T16:45:32.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:32 smithi151 ceph-mon[36066]: from='mgr.? 172.21.15.123:0/717528753' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T16:45:32.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:32 smithi151 ceph-mon[36066]: from='mgr.? 172.21.15.123:0/717528753' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-14T16:45:32.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:32 smithi151 ceph-mon[36066]: from='mgr.? 172.21.15.123:0/717528753' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T16:45:32.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:32 smithi123 ceph-mon[35971]: Standby manager daemon y started 2024-05-14T16:45:32.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:32 smithi123 ceph-mon[35971]: from='mgr.? 172.21.15.123:0/717528753' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-14T16:45:32.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:32 smithi123 ceph-mon[35971]: from='mgr.? 172.21.15.123:0/717528753' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T16:45:32.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:32 smithi123 ceph-mon[35971]: from='mgr.? 172.21.15.123:0/717528753' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-14T16:45:32.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:32 smithi123 ceph-mon[35971]: from='mgr.? 172.21.15.123:0/717528753' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T16:45:33.742 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:33 smithi002 ceph-mon[31538]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:33.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:33 smithi002 ceph-mon[31538]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-05-14T16:45:33.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:33 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-14T16:45:33.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:33 smithi151 ceph-mon[36066]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:33.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:33 smithi151 ceph-mon[36066]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-05-14T16:45:33.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:33 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-14T16:45:33.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:33 smithi123 ceph-mon[35971]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:33.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:33 smithi123 ceph-mon[35971]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-05-14T16:45:33.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:33 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-14T16:45:34.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:34 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:34.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:34 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:34.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:34 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:45:34.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:34 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:34.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:34 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:45:34.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:34 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:34.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:34 smithi002 ceph-mon[31538]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:35.106 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:34 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:35.106 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:34 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:35.106 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:34 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:45:35.106 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:34 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:35.106 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:34 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:45:35.106 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:34 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:35.106 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:34 smithi151 ceph-mon[36066]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:35.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:34 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:35.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:34 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:35.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:34 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:45:35.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:34 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:35.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:34 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:45:35.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:34 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:35.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:34 smithi123 ceph-mon[35971]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:37.163 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:37 smithi002 ceph-mon[31538]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:37.163 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:37 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-14T16:45:37.163 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:37 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:37.163 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:37 smithi002 ceph-mon[31538]: Deploying daemon osd.0 on smithi002 2024-05-14T16:45:37.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:37 smithi151 ceph-mon[36066]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:37.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:37 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-14T16:45:37.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:37 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:37.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:37 smithi151 ceph-mon[36066]: Deploying daemon osd.0 on smithi002 2024-05-14T16:45:37.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:37 smithi123 ceph-mon[35971]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:37.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:37 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-14T16:45:37.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:37 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:37.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:37 smithi123 ceph-mon[35971]: Deploying daemon osd.0 on smithi002 2024-05-14T16:45:39.234 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:39 smithi002 ceph-mon[31538]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:39.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:39 smithi151 ceph-mon[36066]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:39.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:39 smithi123 ceph-mon[35971]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:41.247 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:41 smithi002 ceph-mon[31538]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:41.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:41 smithi151 ceph-mon[36066]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:41.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:41 smithi123 ceph-mon[35971]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:42.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:42 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:42.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:42 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:42.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:42 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:45:42.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:42 smithi002 ceph-mon[31538]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:43.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:42 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:43.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:42 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:43.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:42 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:45:43.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:42 smithi151 ceph-mon[36066]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:43.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:42 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:43.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:42 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:43.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:42 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:45:43.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:42 smithi123 ceph-mon[35971]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:45.346 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:45 smithi002 ceph-mon[31538]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:45.347 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:45 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:45.347 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:45 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:45.347 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:45 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:45.347 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:45 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:45:45.347 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:45 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:45.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:45.427+0000 7fe96dffb700 1 -- 172.21.15.2:0/1371771004 <== mgr.14150 v2:172.21.15.2:6800/1498936028 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe970063980 con 0x7fe95c03e750 2024-05-14T16:45:45.428 INFO:teuthology.orchestra.run.smithi002.stdout:Created osd(s) 0 on host 'smithi002' 2024-05-14T16:45:45.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:45.431+0000 7fe976ac1700 1 -- 172.21.15.2:0/1371771004 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fe95c03e750 msgr2=0x7fe95c040c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:45.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:45.431+0000 7fe976ac1700 1 --2- 172.21.15.2:0/1371771004 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fe95c03e750 0x7fe95c040c10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe97006cd50 tx=0x7fe9600073d0 comp rx=0 tx=0).stop 2024-05-14T16:45:45.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:45.431+0000 7fe976ac1700 1 -- 172.21.15.2:0/1371771004 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9701024e0 msgr2=0x7fe97006c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:45.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:45.431+0000 7fe976ac1700 1 --2- 172.21.15.2:0/1371771004 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9701024e0 0x7fe97006c7f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe95800b580 tx=0x7fe958003e60 comp rx=0 tx=0).stop 2024-05-14T16:45:45.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:45.431+0000 7fe976ac1700 1 -- 172.21.15.2:0/1371771004 shutdown_connections 2024-05-14T16:45:45.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:45.431+0000 7fe976ac1700 1 --2- 172.21.15.2:0/1371771004 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe970104080 0x7fe9701a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:45.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:45.431+0000 7fe976ac1700 1 --2- 172.21.15.2:0/1371771004 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fe95c03e750 0x7fe95c040c10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:45.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:45.431+0000 7fe976ac1700 1 --2- 172.21.15.2:0/1371771004 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9701024e0 0x7fe97006c7f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:45.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:45.431+0000 7fe976ac1700 1 --2- 172.21.15.2:0/1371771004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9701036e0 0x7fe97019e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:45.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:45.431+0000 7fe976ac1700 1 -- 172.21.15.2:0/1371771004 >> 172.21.15.2:0/1371771004 conn(0x7fe9700fda60 msgr2=0x7fe970106a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:45:45.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:45.432+0000 7fe976ac1700 1 -- 172.21.15.2:0/1371771004 shutdown_connections 2024-05-14T16:45:45.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:45:45.432+0000 7fe976ac1700 1 -- 172.21.15.2:0/1371771004 wait complete. 2024-05-14T16:45:45.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:45 smithi151 ceph-mon[36066]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:45.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:45 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:45.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:45 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:45.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:45 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:45.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:45 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:45:45.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:45 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:45.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:45 smithi123 ceph-mon[35971]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:45.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:45 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:45.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:45 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:45.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:45 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:45.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:45 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:45:45.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:45 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:46.128 DEBUG:teuthology.orchestra.run.smithi002:osd.0> sudo journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@osd.0.service 2024-05-14T16:45:46.131 INFO:tasks.cephadm:Deploying osd.1 on smithi123 with /dev/vg_nvme/lv_4... 2024-05-14T16:45:46.131 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-14T16:45:46.375 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T16:45:46.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:46 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:46.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:46 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:46.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:46 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:45:46.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:46 smithi123 ceph-mon[35971]: from='osd.0 [v2:172.21.15.2:6802/2654237820,v1:172.21.15.2:6803/2654237820]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-14T16:45:46.670 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:46 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:46.670 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:46 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:46.670 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:46 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:45:46.670 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:46 smithi002 ceph-mon[31538]: from='osd.0 [v2:172.21.15.2:6802/2654237820,v1:172.21.15.2:6803/2654237820]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-14T16:45:46.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:46 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:46.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:46 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:46.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:46 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:45:46.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:46 smithi151 ceph-mon[36066]: from='osd.0 [v2:172.21.15.2:6802/2654237820,v1:172.21.15.2:6803/2654237820]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-14T16:45:47.418 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:47 smithi123 ceph-mon[35971]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:47.418 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:47 smithi123 ceph-mon[35971]: from='osd.0 [v2:172.21.15.2:6802/2654237820,v1:172.21.15.2:6803/2654237820]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-14T16:45:47.418 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:47 smithi123 ceph-mon[35971]: osdmap e6: 1 total, 0 up, 1 in 2024-05-14T16:45:47.418 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:47 smithi123 ceph-mon[35971]: from='osd.0 [v2:172.21.15.2:6802/2654237820,v1:172.21.15.2:6803/2654237820]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-05-14T16:45:47.418 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:47 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:45:47.676 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:47 smithi002 ceph-mon[31538]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:47.676 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:47 smithi002 ceph-mon[31538]: from='osd.0 [v2:172.21.15.2:6802/2654237820,v1:172.21.15.2:6803/2654237820]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-14T16:45:47.676 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:47 smithi002 ceph-mon[31538]: osdmap e6: 1 total, 0 up, 1 in 2024-05-14T16:45:47.676 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:47 smithi002 ceph-mon[31538]: from='osd.0 [v2:172.21.15.2:6802/2654237820,v1:172.21.15.2:6803/2654237820]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-05-14T16:45:47.676 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:47 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:45:47.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:47 smithi151 ceph-mon[36066]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:47.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:47 smithi151 ceph-mon[36066]: from='osd.0 [v2:172.21.15.2:6802/2654237820,v1:172.21.15.2:6803/2654237820]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-14T16:45:47.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:47 smithi151 ceph-mon[36066]: osdmap e6: 1 total, 0 up, 1 in 2024-05-14T16:45:47.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:47 smithi151 ceph-mon[36066]: from='osd.0 [v2:172.21.15.2:6802/2654237820,v1:172.21.15.2:6803/2654237820]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-05-14T16:45:47.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:47 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:45:47.994 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 16:45:47 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-0[46328]: 2024-05-14T16:45:47.849+0000 7f20ec3c6700 -1 osd.0 0 waiting for initial osdmap 2024-05-14T16:45:47.994 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 16:45:47 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-0[46328]: 2024-05-14T16:45:47.853+0000 7f20e39b0700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T16:45:48.233 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-14T16:45:48.233 INFO:teuthology.orchestra.run.smithi123.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-14T16:45:48.233 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10+0 records in 2024-05-14T16:45:48.234 INFO:teuthology.orchestra.run.smithi123.stderr:10+0 records out 2024-05-14T16:45:48.234 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0127872 s, 820 MB/s 2024-05-14T16:45:48.234 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping successful for: 2024-05-14T16:45:48.530 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:48 smithi123 ceph-mon[35971]: from='osd.0 [v2:172.21.15.2:6802/2654237820,v1:172.21.15.2:6803/2654237820]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-05-14T16:45:48.530 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:48 smithi123 ceph-mon[35971]: osdmap e7: 1 total, 0 up, 1 in 2024-05-14T16:45:48.530 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:48 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:45:48.530 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:48 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:45:48.530 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:48 smithi123 ceph-mon[35971]: from='osd.0 [v2:172.21.15.2:6802/2654237820,v1:172.21.15.2:6803/2654237820]' entity='osd.0' 2024-05-14T16:45:48.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:48 smithi002 ceph-mon[31538]: from='osd.0 [v2:172.21.15.2:6802/2654237820,v1:172.21.15.2:6803/2654237820]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-05-14T16:45:48.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:48 smithi002 ceph-mon[31538]: osdmap e7: 1 total, 0 up, 1 in 2024-05-14T16:45:48.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:48 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:45:48.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:48 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:45:48.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:48 smithi002 ceph-mon[31538]: from='osd.0 [v2:172.21.15.2:6802/2654237820,v1:172.21.15.2:6803/2654237820]' entity='osd.0' 2024-05-14T16:45:48.843 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi123:vg_nvme/lv_4 2024-05-14T16:45:48.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:48 smithi151 ceph-mon[36066]: from='osd.0 [v2:172.21.15.2:6802/2654237820,v1:172.21.15.2:6803/2654237820]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-05-14T16:45:48.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:48 smithi151 ceph-mon[36066]: osdmap e7: 1 total, 0 up, 1 in 2024-05-14T16:45:48.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:48 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:45:48.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:48 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:45:48.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:48 smithi151 ceph-mon[36066]: from='osd.0 [v2:172.21.15.2:6802/2654237820,v1:172.21.15.2:6803/2654237820]' entity='osd.0' 2024-05-14T16:45:49.089 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T16:45:49.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:49 smithi002 ceph-mon[31538]: purged_snaps scrub starts 2024-05-14T16:45:49.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:49 smithi002 ceph-mon[31538]: purged_snaps scrub ok 2024-05-14T16:45:49.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:49 smithi002 ceph-mon[31538]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:49.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:49 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:45:49.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:49 smithi002 ceph-mon[31538]: osd.0 [v2:172.21.15.2:6802/2654237820,v1:172.21.15.2:6803/2654237820] boot 2024-05-14T16:45:49.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:49 smithi002 ceph-mon[31538]: osdmap e8: 1 total, 1 up, 1 in 2024-05-14T16:45:49.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:49 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:45:49.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:49 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:49.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:49 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:49.495 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:49 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:45:49.495 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:49 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:49.495 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:49 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:49.495 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:49 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:45:49.495 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:49 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:49.498 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:49 smithi123 ceph-mon[35971]: purged_snaps scrub starts 2024-05-14T16:45:49.498 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:49 smithi123 ceph-mon[35971]: purged_snaps scrub ok 2024-05-14T16:45:49.498 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:49 smithi123 ceph-mon[35971]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:49.498 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:49 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:45:49.498 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:49 smithi123 ceph-mon[35971]: osd.0 [v2:172.21.15.2:6802/2654237820,v1:172.21.15.2:6803/2654237820] boot 2024-05-14T16:45:49.498 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:49 smithi123 ceph-mon[35971]: osdmap e8: 1 total, 1 up, 1 in 2024-05-14T16:45:49.498 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:49 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:45:49.498 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:49 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:49.499 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:49 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:49.499 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:49 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:45:49.499 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:49 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:49.499 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:49 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:49.499 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:49 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:45:49.499 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:49 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:49.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:49 smithi151 ceph-mon[36066]: purged_snaps scrub starts 2024-05-14T16:45:49.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:49 smithi151 ceph-mon[36066]: purged_snaps scrub ok 2024-05-14T16:45:49.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:49 smithi151 ceph-mon[36066]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-14T16:45:49.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:49 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:45:49.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:49 smithi151 ceph-mon[36066]: osd.0 [v2:172.21.15.2:6802/2654237820,v1:172.21.15.2:6803/2654237820] boot 2024-05-14T16:45:49.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:49 smithi151 ceph-mon[36066]: osdmap e8: 1 total, 1 up, 1 in 2024-05-14T16:45:49.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:49 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:45:49.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:49 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:49.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:49 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:49.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:49 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:45:49.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:49 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:49.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:49 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:49.856 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:49 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:45:49.856 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:49 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:45:50.446 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:50 smithi123 ceph-mon[35971]: Detected new or changed devices on smithi002 2024-05-14T16:45:50.446 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:50 smithi123 ceph-mon[35971]: Adjusting osd_memory_target on smithi002 to 17093M 2024-05-14T16:45:50.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.637+0000 7f1c68749700 1 -- 172.21.15.123:0/2763520456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c601069c0 msgr2=0x7f1c60108db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:50.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.637+0000 7f1c68749700 1 --2- 172.21.15.123:0/2763520456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c601069c0 0x7f1c60108db0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1c50009990 tx=0x7f1c50009ca0 comp rx=0 tx=0).stop 2024-05-14T16:45:50.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.638+0000 7f1c68749700 1 -- 172.21.15.123:0/2763520456 shutdown_connections 2024-05-14T16:45:50.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.638+0000 7f1c68749700 1 --2- 172.21.15.123:0/2763520456 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c601092f0 0x7f1c6010b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:50.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.638+0000 7f1c68749700 1 --2- 172.21.15.123:0/2763520456 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c60104090 0x7f1c60106480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:50.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.638+0000 7f1c68749700 1 --2- 172.21.15.123:0/2763520456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c601069c0 0x7f1c60108db0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:50.640 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.638+0000 7f1c68749700 1 -- 172.21.15.123:0/2763520456 >> 172.21.15.123:0/2763520456 conn(0x7f1c600fda40 msgr2=0x7f1c600ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:45:50.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.638+0000 7f1c68749700 1 -- 172.21.15.123:0/2763520456 shutdown_connections 2024-05-14T16:45:50.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.638+0000 7f1c68749700 1 -- 172.21.15.123:0/2763520456 wait complete. 2024-05-14T16:45:50.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.638+0000 7f1c68749700 1 Processor -- start 2024-05-14T16:45:50.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.638+0000 7f1c68749700 1 -- start start 2024-05-14T16:45:50.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.638+0000 7f1c68749700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c60104090 0x7f1c601997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:50.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.638+0000 7f1c68749700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c601069c0 0x7f1c60199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:50.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.638+0000 7f1c68749700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c601092f0 0x7f1c601a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:50.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.638+0000 7f1c68749700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6010f0a0 con 0x7f1c601069c0 2024-05-14T16:45:50.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.638+0000 7f1c68749700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6010ef20 con 0x7f1c601092f0 2024-05-14T16:45:50.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.638+0000 7f1c68749700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6010f220 con 0x7f1c60104090 2024-05-14T16:45:50.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.639+0000 7f1c66ce6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c601092f0 0x7f1c601a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:50.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.639+0000 7f1c66ce6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c601092f0 0x7f1c601a0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44846/0 (socket says 172.21.15.123:44846) 2024-05-14T16:45:50.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.639+0000 7f1c66ce6700 1 -- 172.21.15.123:0/2607829659 learned_addr learned my addr 172.21.15.123:0/2607829659 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-14T16:45:50.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.639+0000 7f1c65ce4700 1 --2- 172.21.15.123:0/2607829659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c601069c0 0x7f1c60199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:50.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.639+0000 7f1c664e5700 1 --2- 172.21.15.123:0/2607829659 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c60104090 0x7f1c601997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:50.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.639+0000 7f1c66ce6700 1 -- 172.21.15.123:0/2607829659 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c60104090 msgr2=0x7f1c601997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:50.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.639+0000 7f1c66ce6700 1 --2- 172.21.15.123:0/2607829659 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c60104090 0x7f1c601997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:50.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.639+0000 7f1c66ce6700 1 -- 172.21.15.123:0/2607829659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c601069c0 msgr2=0x7f1c60199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:45:50.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.639+0000 7f1c66ce6700 1 --2- 172.21.15.123:0/2607829659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c601069c0 0x7f1c60199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:45:50.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.639+0000 7f1c66ce6700 1 -- 172.21.15.123:0/2607829659 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c601a1410 con 0x7f1c601092f0 2024-05-14T16:45:50.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.639+0000 7f1c66ce6700 1 --2- 172.21.15.123:0/2607829659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c601092f0 0x7f1c601a0d70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800b9e0 tx=0x7f1c5800bcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:45:50.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.639+0000 7f1c577fe700 1 -- 172.21.15.123:0/2607829659 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5800d8e0 con 0x7f1c601092f0 2024-05-14T16:45:50.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.639+0000 7f1c68749700 1 -- 172.21.15.123:0/2607829659 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c601a1700 con 0x7f1c601092f0 2024-05-14T16:45:50.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.639+0000 7f1c577fe700 1 -- 172.21.15.123:0/2607829659 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c58015420 con 0x7f1c601092f0 2024-05-14T16:45:50.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.639+0000 7f1c577fe700 1 -- 172.21.15.123:0/2607829659 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c58014610 con 0x7f1c601092f0 2024-05-14T16:45:50.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.639+0000 7f1c68749700 1 -- 172.21.15.123:0/2607829659 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c601a1c70 con 0x7f1c601092f0 2024-05-14T16:45:50.643 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.640+0000 7f1c68749700 1 -- 172.21.15.123:0/2607829659 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c6006a230 con 0x7f1c601092f0 2024-05-14T16:45:50.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.641+0000 7f1c577fe700 1 -- 172.21.15.123:0/2607829659 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 15) ==== 149595+0+0 (secure 0 0 0) 0x7f1c580155d0 con 0x7f1c601092f0 2024-05-14T16:45:50.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.642+0000 7f1c577fe700 1 --2- 172.21.15.123:0/2607829659 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f1c4c0b27f0 0x7f1c4c0b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:45:50.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.642+0000 7f1c577fe700 1 -- 172.21.15.123:0/2607829659 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f1c580e0060 con 0x7f1c601092f0 2024-05-14T16:45:50.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.642+0000 7f1c664e5700 1 --2- 172.21.15.123:0/2607829659 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f1c4c0b27f0 0x7f1c4c0b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:45:50.644 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.643+0000 7f1c664e5700 1 --2- 172.21.15.123:0/2607829659 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f1c4c0b27f0 0x7f1c4c0b4cb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c005950 tx=0x7f1c5c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:45:50.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.647+0000 7f1c577fe700 1 -- 172.21.15.123:0/2607829659 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1c580a97f0 con 0x7f1c601092f0 2024-05-14T16:45:50.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:50 smithi002 ceph-mon[31538]: Detected new or changed devices on smithi002 2024-05-14T16:45:50.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:50 smithi002 ceph-mon[31538]: Adjusting osd_memory_target on smithi002 to 17093M 2024-05-14T16:45:50.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:50 smithi151 ceph-mon[36066]: Detected new or changed devices on smithi002 2024-05-14T16:45:50.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:50 smithi151 ceph-mon[36066]: Adjusting osd_memory_target on smithi002 to 17093M 2024-05-14T16:45:50.858 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:45:50.858+0000 7f1c68749700 1 -- 172.21.15.123:0/2607829659 --> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f1c60063a40 con 0x7f1c4c0b27f0 2024-05-14T16:45:51.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:51 smithi123 ceph-mon[35971]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:45:51.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:51 smithi123 ceph-mon[35971]: osdmap e9: 1 total, 1 up, 1 in 2024-05-14T16:45:51.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:51 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:45:51.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:51 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:45:51.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:51 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:51.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:51 smithi002 ceph-mon[31538]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:45:51.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:51 smithi002 ceph-mon[31538]: osdmap e9: 1 total, 1 up, 1 in 2024-05-14T16:45:51.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:51 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:45:51.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:51 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:45:51.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:51 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:51.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:51 smithi151 ceph-mon[36066]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:45:51.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:51 smithi151 ceph-mon[36066]: osdmap e9: 1 total, 1 up, 1 in 2024-05-14T16:45:51.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:51 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:45:51.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:51 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:45:51.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:51 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:45:52.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:52 smithi123 ceph-mon[35971]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:45:52.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:52 smithi002 ceph-mon[31538]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:45:52.755 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:52 smithi151 ceph-mon[36066]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:45:53.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:53 smithi002 ceph-mon[31538]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:45:53.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:53 smithi151 ceph-mon[36066]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:45:53.869 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:53 smithi123 ceph-mon[35971]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:45:54.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:54 smithi002 ceph-mon[31538]: from='client.? 172.21.15.123:0/523669238' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e95419f-9a9d-4f7a-b55d-ecb54971918a"}]: dispatch 2024-05-14T16:45:54.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:54 smithi002 ceph-mon[31538]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e95419f-9a9d-4f7a-b55d-ecb54971918a"}]: dispatch 2024-05-14T16:45:54.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:54 smithi002 ceph-mon[31538]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e95419f-9a9d-4f7a-b55d-ecb54971918a"}]': finished 2024-05-14T16:45:54.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:54 smithi002 ceph-mon[31538]: osdmap e10: 2 total, 1 up, 2 in 2024-05-14T16:45:54.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:54 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:45:54.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:54 smithi002 ceph-mon[31538]: from='client.? 172.21.15.123:0/578710813' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:45:54.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:54 smithi151 ceph-mon[36066]: from='client.? 172.21.15.123:0/523669238' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e95419f-9a9d-4f7a-b55d-ecb54971918a"}]: dispatch 2024-05-14T16:45:54.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:54 smithi151 ceph-mon[36066]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e95419f-9a9d-4f7a-b55d-ecb54971918a"}]: dispatch 2024-05-14T16:45:54.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:54 smithi151 ceph-mon[36066]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e95419f-9a9d-4f7a-b55d-ecb54971918a"}]': finished 2024-05-14T16:45:54.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:54 smithi151 ceph-mon[36066]: osdmap e10: 2 total, 1 up, 2 in 2024-05-14T16:45:54.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:54 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:45:54.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:54 smithi151 ceph-mon[36066]: from='client.? 172.21.15.123:0/578710813' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:45:54.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:54 smithi123 ceph-mon[35971]: from='client.? 172.21.15.123:0/523669238' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e95419f-9a9d-4f7a-b55d-ecb54971918a"}]: dispatch 2024-05-14T16:45:54.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:54 smithi123 ceph-mon[35971]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e95419f-9a9d-4f7a-b55d-ecb54971918a"}]: dispatch 2024-05-14T16:45:54.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:54 smithi123 ceph-mon[35971]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e95419f-9a9d-4f7a-b55d-ecb54971918a"}]': finished 2024-05-14T16:45:54.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:54 smithi123 ceph-mon[35971]: osdmap e10: 2 total, 1 up, 2 in 2024-05-14T16:45:54.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:54 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:45:54.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:54 smithi123 ceph-mon[35971]: from='client.? 172.21.15.123:0/578710813' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:45:55.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:55 smithi002 ceph-mon[31538]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:45:55.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:55 smithi151 ceph-mon[36066]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:45:55.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:55 smithi123 ceph-mon[35971]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:45:57.740 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:57 smithi123 ceph-mon[35971]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:45:57.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:57 smithi002 ceph-mon[31538]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:45:57.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:57 smithi151 ceph-mon[36066]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:45:59.541 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:45:59 smithi123 ceph-mon[35971]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:45:59.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:45:59 smithi002 ceph-mon[31538]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:45:59.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:45:59 smithi151 ceph-mon[36066]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:46:01.505 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:01 smithi123 ceph-mon[35971]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:46:01.505 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:01 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-14T16:46:01.505 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:01 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:01.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:01 smithi002 ceph-mon[31538]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:46:01.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:01 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-14T16:46:01.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:01 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:01.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:01 smithi151 ceph-mon[36066]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:46:01.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:01 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-14T16:46:01.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:01 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:02.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:02 smithi002 ceph-mon[31538]: Deploying daemon osd.1 on smithi123 2024-05-14T16:46:02.762 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:02 smithi123 ceph-mon[35971]: Deploying daemon osd.1 on smithi123 2024-05-14T16:46:02.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:02 smithi151 ceph-mon[36066]: Deploying daemon osd.1 on smithi123 2024-05-14T16:46:03.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:03 smithi123 ceph-mon[35971]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:46:03.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:03 smithi002 ceph-mon[31538]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:46:03.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:03 smithi151 ceph-mon[36066]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:46:05.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:05 smithi123 ceph-mon[35971]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:46:05.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:05 smithi002 ceph-mon[31538]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:46:05.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:05 smithi151 ceph-mon[36066]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:46:07.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:07.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:07.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:46:07.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:06 smithi123 ceph-mon[35971]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:46:07.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:06 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:07.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:06 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:07.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:06 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:46:07.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:06 smithi002 ceph-mon[31538]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:46:07.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:06 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:07.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:06 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:07.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:06 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:46:07.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:06 smithi151 ceph-mon[36066]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:46:08.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:46:08.878+0000 7f1c577fe700 1 -- 172.21.15.123:0/2607829659 <== mgr.14150 v2:172.21.15.2:6800/1498936028 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1c60063a40 con 0x7f1c4c0b27f0 2024-05-14T16:46:08.878 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 1 on host 'smithi123' 2024-05-14T16:46:08.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:46:08.880+0000 7f1c68749700 1 -- 172.21.15.123:0/2607829659 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f1c4c0b27f0 msgr2=0x7f1c4c0b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:08.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:46:08.880+0000 7f1c68749700 1 --2- 172.21.15.123:0/2607829659 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f1c4c0b27f0 0x7f1c4c0b4cb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c005950 tx=0x7f1c5c00b410 comp rx=0 tx=0).stop 2024-05-14T16:46:08.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:46:08.880+0000 7f1c68749700 1 -- 172.21.15.123:0/2607829659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c601092f0 msgr2=0x7f1c601a0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:08.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:46:08.880+0000 7f1c68749700 1 --2- 172.21.15.123:0/2607829659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c601092f0 0x7f1c601a0d70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800b9e0 tx=0x7f1c5800bcf0 comp rx=0 tx=0).stop 2024-05-14T16:46:08.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:46:08.880+0000 7f1c68749700 1 -- 172.21.15.123:0/2607829659 shutdown_connections 2024-05-14T16:46:08.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:46:08.880+0000 7f1c68749700 1 --2- 172.21.15.123:0/2607829659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c601092f0 0x7f1c601a0d70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:08.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:46:08.880+0000 7f1c68749700 1 --2- 172.21.15.123:0/2607829659 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c60104090 0x7f1c601997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:08.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:46:08.880+0000 7f1c68749700 1 --2- 172.21.15.123:0/2607829659 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f1c4c0b27f0 0x7f1c4c0b4cb0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:08.880 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:46:08.880+0000 7f1c68749700 1 --2- 172.21.15.123:0/2607829659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c601069c0 0x7f1c60199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:08.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:46:08.880+0000 7f1c68749700 1 -- 172.21.15.123:0/2607829659 >> 172.21.15.123:0/2607829659 conn(0x7f1c600fda40 msgr2=0x7f1c60104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:46:08.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:46:08.881+0000 7f1c68749700 1 -- 172.21.15.123:0/2607829659 shutdown_connections 2024-05-14T16:46:08.881 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:46:08.881+0000 7f1c68749700 1 -- 172.21.15.123:0/2607829659 wait complete. 2024-05-14T16:46:09.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:09 smithi123 ceph-mon[35971]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:46:09.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:09 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:09.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:09 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:09.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:09 smithi151 ceph-mon[36066]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:46:09.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:09 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:09.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:09 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:09.486 DEBUG:teuthology.orchestra.run.smithi123:osd.1> sudo journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@osd.1.service 2024-05-14T16:46:09.488 INFO:tasks.cephadm:Deploying osd.2 on smithi151 with /dev/vg_nvme/lv_4... 2024-05-14T16:46:09.489 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-14T16:46:09.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:09 smithi002 ceph-mon[31538]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:46:09.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:09 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:09.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:09 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:09.732 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T16:46:10.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:10 smithi151 ceph-mon[36066]: from='osd.1 [v2:172.21.15.123:6800/2668156992,v1:172.21.15.123:6801/2668156992]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-14T16:46:10.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:10 smithi151 ceph-mon[36066]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-14T16:46:10.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:10 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:10.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:10 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:10.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:10 smithi123 ceph-mon[35971]: from='osd.1 [v2:172.21.15.123:6800/2668156992,v1:172.21.15.123:6801/2668156992]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-14T16:46:10.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:10 smithi123 ceph-mon[35971]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-14T16:46:10.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:10 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:10.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:10 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:10.160 INFO:journalctl@ceph.osd.1.smithi123.stdout:May 14 16:46:09 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-1[40562]: 2024-05-14T16:46:09.871+0000 7f6356cae7c0 -1 osd.1 0 log_to_monitors true 2024-05-14T16:46:10.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:10 smithi002 ceph-mon[31538]: from='osd.1 [v2:172.21.15.123:6800/2668156992,v1:172.21.15.123:6801/2668156992]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-14T16:46:10.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:10 smithi002 ceph-mon[31538]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-14T16:46:10.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:10 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:10.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:10 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:11.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:11 smithi123 ceph-mon[35971]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:46:11.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:11 smithi123 ceph-mon[35971]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-14T16:46:11.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:11 smithi123 ceph-mon[35971]: osdmap e11: 2 total, 1 up, 2 in 2024-05-14T16:46:11.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:11 smithi123 ceph-mon[35971]: from='osd.1 [v2:172.21.15.123:6800/2668156992,v1:172.21.15.123:6801/2668156992]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-14T16:46:11.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:11 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:46:11.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:11 smithi123 ceph-mon[35971]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-14T16:46:11.173 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:11 smithi151 ceph-mon[36066]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:46:11.173 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:11 smithi151 ceph-mon[36066]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-14T16:46:11.174 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:11 smithi151 ceph-mon[36066]: osdmap e11: 2 total, 1 up, 2 in 2024-05-14T16:46:11.174 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:11 smithi151 ceph-mon[36066]: from='osd.1 [v2:172.21.15.123:6800/2668156992,v1:172.21.15.123:6801/2668156992]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-14T16:46:11.174 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:11 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:46:11.174 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:11 smithi151 ceph-mon[36066]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-14T16:46:11.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:11 smithi002 ceph-mon[31538]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:46:11.495 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:11 smithi002 ceph-mon[31538]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-14T16:46:11.495 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:11 smithi002 ceph-mon[31538]: osdmap e11: 2 total, 1 up, 2 in 2024-05-14T16:46:11.495 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:11 smithi002 ceph-mon[31538]: from='osd.1 [v2:172.21.15.123:6800/2668156992,v1:172.21.15.123:6801/2668156992]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-14T16:46:11.495 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:11 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:46:11.495 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:11 smithi002 ceph-mon[31538]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-14T16:46:11.769 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-14T16:46:11.769 INFO:teuthology.orchestra.run.smithi151.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-14T16:46:11.769 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10+0 records in 2024-05-14T16:46:11.769 INFO:teuthology.orchestra.run.smithi151.stderr:10+0 records out 2024-05-14T16:46:11.769 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113993 s, 920 MB/s 2024-05-14T16:46:11.769 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping successful for: 2024-05-14T16:46:11.847 INFO:journalctl@ceph.osd.1.smithi123.stdout:May 14 16:46:11 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-1[40562]: 2024-05-14T16:46:11.514+0000 7f634c312700 -1 osd.1 0 waiting for initial osdmap 2024-05-14T16:46:11.847 INFO:journalctl@ceph.osd.1.smithi123.stdout:May 14 16:46:11 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-1[40562]: 2024-05-14T16:46:11.516+0000 7f63440fd700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T16:46:12.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:12 smithi123 ceph-mon[35971]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-05-14T16:46:12.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:12 smithi123 ceph-mon[35971]: osdmap e12: 2 total, 1 up, 2 in 2024-05-14T16:46:12.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:12 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:46:12.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:12 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:46:12.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:12 smithi123 ceph-mon[35971]: from='osd.1 ' entity='osd.1' 2024-05-14T16:46:12.352 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:12 smithi151 ceph-mon[36066]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-05-14T16:46:12.352 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:12 smithi151 ceph-mon[36066]: osdmap e12: 2 total, 1 up, 2 in 2024-05-14T16:46:12.352 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:12 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:46:12.352 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:12 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:46:12.352 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:12 smithi151 ceph-mon[36066]: from='osd.1 ' entity='osd.1' 2024-05-14T16:46:12.380 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi151:vg_nvme/lv_4 2024-05-14T16:46:12.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:12 smithi002 ceph-mon[31538]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-05-14T16:46:12.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:12 smithi002 ceph-mon[31538]: osdmap e12: 2 total, 1 up, 2 in 2024-05-14T16:46:12.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:12 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:46:12.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:12 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:46:12.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:12 smithi002 ceph-mon[31538]: from='osd.1 ' entity='osd.1' 2024-05-14T16:46:12.626 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T16:46:13.326 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:13 smithi123 ceph-mon[35971]: purged_snaps scrub starts 2024-05-14T16:46:13.326 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:13 smithi123 ceph-mon[35971]: purged_snaps scrub ok 2024-05-14T16:46:13.327 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:13 smithi123 ceph-mon[35971]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:46:13.327 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:13 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:46:13.327 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:13 smithi123 ceph-mon[35971]: osd.1 [v2:172.21.15.123:6800/2668156992,v1:172.21.15.123:6801/2668156992] boot 2024-05-14T16:46:13.327 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:13 smithi123 ceph-mon[35971]: osdmap e13: 2 total, 2 up, 2 in 2024-05-14T16:46:13.327 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:13 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:46:13.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:13 smithi151 ceph-mon[36066]: purged_snaps scrub starts 2024-05-14T16:46:13.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:13 smithi151 ceph-mon[36066]: purged_snaps scrub ok 2024-05-14T16:46:13.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:13 smithi151 ceph-mon[36066]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:46:13.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:13 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:46:13.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:13 smithi151 ceph-mon[36066]: osd.1 [v2:172.21.15.123:6800/2668156992,v1:172.21.15.123:6801/2668156992] boot 2024-05-14T16:46:13.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:13 smithi151 ceph-mon[36066]: osdmap e13: 2 total, 2 up, 2 in 2024-05-14T16:46:13.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:13 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:46:13.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:13 smithi002 ceph-mon[31538]: purged_snaps scrub starts 2024-05-14T16:46:13.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:13 smithi002 ceph-mon[31538]: purged_snaps scrub ok 2024-05-14T16:46:13.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:13 smithi002 ceph-mon[31538]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-14T16:46:13.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:13 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:46:13.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:13 smithi002 ceph-mon[31538]: osd.1 [v2:172.21.15.123:6800/2668156992,v1:172.21.15.123:6801/2668156992] boot 2024-05-14T16:46:13.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:13 smithi002 ceph-mon[31538]: osdmap e13: 2 total, 2 up, 2 in 2024-05-14T16:46:13.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:13 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:46:14.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.232+0000 7f19b177b700 1 -- 172.21.15.151:0/3646905776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f19ac1035a0 msgr2=0x7f19ac1039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:14.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.232+0000 7f19b177b700 1 --2- 172.21.15.151:0/3646905776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f19ac1035a0 0x7f19ac1039c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f199c009990 tx=0x7f199c009ca0 comp rx=0 tx=0).stop 2024-05-14T16:46:14.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.232+0000 7f19b177b700 1 -- 172.21.15.151:0/3646905776 shutdown_connections 2024-05-14T16:46:14.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.232+0000 7f19b177b700 1 --2- 172.21.15.151:0/3646905776 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19ac105140 0x7f19ac10b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:14.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.232+0000 7f19b177b700 1 --2- 172.21.15.151:0/3646905776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f19ac1035a0 0x7f19ac1039c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:14.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.232+0000 7f19b177b700 1 --2- 172.21.15.151:0/3646905776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19ac1047a0 0x7f19ac104c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:14.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.232+0000 7f19b177b700 1 -- 172.21.15.151:0/3646905776 >> 172.21.15.151:0/3646905776 conn(0x7f19ac0feb40 msgr2=0x7f19ac100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:46:14.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.232+0000 7f19b177b700 1 -- 172.21.15.151:0/3646905776 shutdown_connections 2024-05-14T16:46:14.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.232+0000 7f19b177b700 1 -- 172.21.15.151:0/3646905776 wait complete. 2024-05-14T16:46:14.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.233+0000 7f19b177b700 1 Processor -- start 2024-05-14T16:46:14.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.233+0000 7f19b177b700 1 -- start start 2024-05-14T16:46:14.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.233+0000 7f19b177b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f19ac1035a0 0x7f19ac0772e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:14.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.233+0000 7f19b177b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19ac1047a0 0x7f19ac075930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:14.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.233+0000 7f19b177b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19ac105140 0x7f19ac075e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:14.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.233+0000 7f19b177b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f19ac10f110 con 0x7f19ac1047a0 2024-05-14T16:46:14.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.233+0000 7f19b177b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f19ac10ef90 con 0x7f19ac105140 2024-05-14T16:46:14.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.233+0000 7f19b177b700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f19ac10f290 con 0x7f19ac1035a0 2024-05-14T16:46:14.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.233+0000 7f19aaffd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f19ac1035a0 0x7f19ac0772e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:14.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.233+0000 7f19aaffd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f19ac1035a0 0x7f19ac0772e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:34062/0 (socket says 172.21.15.151:34062) 2024-05-14T16:46:14.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.233+0000 7f19aaffd700 1 -- 172.21.15.151:0/1288856168 learned_addr learned my addr 172.21.15.151:0/1288856168 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-14T16:46:14.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.234+0000 7f19ab7fe700 1 --2- 172.21.15.151:0/1288856168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19ac105140 0x7f19ac075e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:14.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.234+0000 7f19aaffd700 1 -- 172.21.15.151:0/1288856168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19ac105140 msgr2=0x7f19ac075e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:14.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.234+0000 7f19aaffd700 1 --2- 172.21.15.151:0/1288856168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19ac105140 0x7f19ac075e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:14.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.234+0000 7f19aa7fc700 1 --2- 172.21.15.151:0/1288856168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19ac1047a0 0x7f19ac075930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:14.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.234+0000 7f19aaffd700 1 -- 172.21.15.151:0/1288856168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19ac1047a0 msgr2=0x7f19ac075930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:14.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.234+0000 7f19aaffd700 1 --2- 172.21.15.151:0/1288856168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19ac1047a0 0x7f19ac075930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:14.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.234+0000 7f19aaffd700 1 -- 172.21.15.151:0/1288856168 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19ac076720 con 0x7f19ac1035a0 2024-05-14T16:46:14.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.234+0000 7f19ab7fe700 1 --2- 172.21.15.151:0/1288856168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19ac105140 0x7f19ac075e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:46:14.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.234+0000 7f19aa7fc700 1 --2- 172.21.15.151:0/1288856168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19ac1047a0 0x7f19ac075930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:46:14.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.234+0000 7f19aaffd700 1 --2- 172.21.15.151:0/1288856168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f19ac1035a0 0x7f19ac0772e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f199c005f50 tx=0x7f199c0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:46:14.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.234+0000 7f19a3fff700 1 -- 172.21.15.151:0/1288856168 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199c027070 con 0x7f19ac1035a0 2024-05-14T16:46:14.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.234+0000 7f19a3fff700 1 -- 172.21.15.151:0/1288856168 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f199c022470 con 0x7f19ac1035a0 2024-05-14T16:46:14.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.234+0000 7f19b177b700 1 -- 172.21.15.151:0/1288856168 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19ac1a3930 con 0x7f19ac1035a0 2024-05-14T16:46:14.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.234+0000 7f19a3fff700 1 -- 172.21.15.151:0/1288856168 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199c00f6c0 con 0x7f19ac1035a0 2024-05-14T16:46:14.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.234+0000 7f19b177b700 1 -- 172.21.15.151:0/1288856168 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19ac1a3e00 con 0x7f19ac1035a0 2024-05-14T16:46:14.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.235+0000 7f19b177b700 1 -- 172.21.15.151:0/1288856168 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19ac06a230 con 0x7f19ac1035a0 2024-05-14T16:46:14.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.236+0000 7f19a3fff700 1 -- 172.21.15.151:0/1288856168 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 15) ==== 149595+0+0 (secure 0 0 0) 0x7f199c02a070 con 0x7f19ac1035a0 2024-05-14T16:46:14.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.237+0000 7f19a3fff700 1 --2- 172.21.15.151:0/1288856168 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f19980b27f0 0x7f19980b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:14.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.237+0000 7f19a3fff700 1 -- 172.21.15.151:0/1288856168 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f199c0e1ee0 con 0x7f19ac1035a0 2024-05-14T16:46:14.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.237+0000 7f19aa7fc700 1 --2- 172.21.15.151:0/1288856168 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f19980b27f0 0x7f19980b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:14.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.238+0000 7f19aa7fc700 1 --2- 172.21.15.151:0/1288856168 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f19980b27f0 0x7f19980b4cb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1994007900 tx=0x7f1994008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:46:14.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.241+0000 7f19a3fff700 1 -- 172.21.15.151:0/1288856168 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f199c013070 con 0x7f19ac1035a0 2024-05-14T16:46:14.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:14.456+0000 7f19b177b700 1 -- 172.21.15.151:0/1288856168 --> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f19ac063a40 con 0x7f19980b27f0 2024-05-14T16:46:14.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:14 smithi151 ceph-mon[36066]: Detected new or changed devices on smithi123 2024-05-14T16:46:14.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:14 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:14.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:14 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:14.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:14 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:46:14.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:14 smithi151 ceph-mon[36066]: Adjusting osd_memory_target on smithi123 to 17088M 2024-05-14T16:46:14.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:14 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:14.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:14 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:14.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:14 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:46:14.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:14 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:14.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:14 smithi151 ceph-mon[36066]: osdmap e14: 2 total, 2 up, 2 in 2024-05-14T16:46:14.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:14 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:46:14.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:14 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:14.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:14 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:46:14.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:14 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:14.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:14 smithi123 ceph-mon[35971]: Detected new or changed devices on smithi123 2024-05-14T16:46:14.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:14 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:14.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:14 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:14.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:14 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:46:14.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:14 smithi123 ceph-mon[35971]: Adjusting osd_memory_target on smithi123 to 17088M 2024-05-14T16:46:14.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:14 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:14.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:14 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:14.661 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:14 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:46:14.661 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:14 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:14.661 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:14 smithi123 ceph-mon[35971]: osdmap e14: 2 total, 2 up, 2 in 2024-05-14T16:46:14.661 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:14 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:46:14.661 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:14 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:14.661 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:14 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:46:14.661 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:14 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:14.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:14 smithi002 ceph-mon[31538]: Detected new or changed devices on smithi123 2024-05-14T16:46:14.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:14 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:14.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:14 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:14.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:14 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:46:14.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:14 smithi002 ceph-mon[31538]: Adjusting osd_memory_target on smithi123 to 17088M 2024-05-14T16:46:14.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:14 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:14.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:14 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:14.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:14 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:46:14.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:14 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:14.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:14 smithi002 ceph-mon[31538]: osdmap e14: 2 total, 2 up, 2 in 2024-05-14T16:46:14.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:14 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:46:14.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:14 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:14.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:14 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:46:14.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:14 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:15.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:15 smithi151 ceph-mon[36066]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:46:15.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:15 smithi151 ceph-mon[36066]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:46:15.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:15 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:46:15.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:15 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:46:15.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:15 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:15.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:15 smithi123 ceph-mon[35971]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:46:15.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:15 smithi123 ceph-mon[35971]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:46:15.661 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:15 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:46:15.661 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:15 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:46:15.661 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:15 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:15.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:15 smithi002 ceph-mon[31538]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:46:15.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:15 smithi002 ceph-mon[31538]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:46:15.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:15 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:46:15.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:15 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:46:15.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:15 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:17.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:17 smithi151 ceph-mon[36066]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:46:17.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:17 smithi151 ceph-mon[36066]: from='client.? 172.21.15.151:0/1788969998' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6ce7ac7-be6c-4549-a356-59cb81b5b492"}]: dispatch 2024-05-14T16:46:17.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:17 smithi151 ceph-mon[36066]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6ce7ac7-be6c-4549-a356-59cb81b5b492"}]: dispatch 2024-05-14T16:46:17.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:17 smithi151 ceph-mon[36066]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6ce7ac7-be6c-4549-a356-59cb81b5b492"}]': finished 2024-05-14T16:46:17.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:17 smithi151 ceph-mon[36066]: osdmap e15: 3 total, 2 up, 3 in 2024-05-14T16:46:17.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:17 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:46:17.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:17 smithi123 ceph-mon[35971]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:46:17.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:17 smithi123 ceph-mon[35971]: from='client.? 172.21.15.151:0/1788969998' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6ce7ac7-be6c-4549-a356-59cb81b5b492"}]: dispatch 2024-05-14T16:46:17.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:17 smithi123 ceph-mon[35971]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6ce7ac7-be6c-4549-a356-59cb81b5b492"}]: dispatch 2024-05-14T16:46:17.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:17 smithi123 ceph-mon[35971]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6ce7ac7-be6c-4549-a356-59cb81b5b492"}]': finished 2024-05-14T16:46:17.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:17 smithi123 ceph-mon[35971]: osdmap e15: 3 total, 2 up, 3 in 2024-05-14T16:46:17.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:17 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:46:17.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:17 smithi002 ceph-mon[31538]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:46:17.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:17 smithi002 ceph-mon[31538]: from='client.? 172.21.15.151:0/1788969998' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6ce7ac7-be6c-4549-a356-59cb81b5b492"}]: dispatch 2024-05-14T16:46:17.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:17 smithi002 ceph-mon[31538]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6ce7ac7-be6c-4549-a356-59cb81b5b492"}]: dispatch 2024-05-14T16:46:17.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:17 smithi002 ceph-mon[31538]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6ce7ac7-be6c-4549-a356-59cb81b5b492"}]': finished 2024-05-14T16:46:17.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:17 smithi002 ceph-mon[31538]: osdmap e15: 3 total, 2 up, 3 in 2024-05-14T16:46:17.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:17 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:46:18.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:18 smithi123 ceph-mon[35971]: from='client.? 172.21.15.151:0/1927562375' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:46:18.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:18 smithi002 ceph-mon[31538]: from='client.? 172.21.15.151:0/1927562375' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:46:18.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:18 smithi151 ceph-mon[36066]: from='client.? 172.21.15.151:0/1927562375' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:46:19.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:19 smithi123 ceph-mon[35971]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:46:19.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:19 smithi002 ceph-mon[31538]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:46:19.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:19 smithi151 ceph-mon[36066]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:46:21.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:21 smithi151 ceph-mon[36066]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:46:21.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:21 smithi123 ceph-mon[35971]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:46:21.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:21 smithi002 ceph-mon[31538]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:46:23.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:23 smithi123 ceph-mon[35971]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:46:23.681 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:23 smithi151 ceph-mon[36066]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:46:23.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:23 smithi002 ceph-mon[31538]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:46:24.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:24 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-14T16:46:24.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:24 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:24.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:24 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-14T16:46:24.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:24 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:24.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:24 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-14T16:46:24.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:24 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:25.542 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:25 smithi151 ceph-mon[36066]: Deploying daemon osd.2 on smithi151 2024-05-14T16:46:25.543 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:25 smithi151 ceph-mon[36066]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:46:25.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:25 smithi123 ceph-mon[35971]: Deploying daemon osd.2 on smithi151 2024-05-14T16:46:25.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:25 smithi123 ceph-mon[35971]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:46:25.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:25 smithi002 ceph-mon[31538]: Deploying daemon osd.2 on smithi151 2024-05-14T16:46:25.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:25 smithi002 ceph-mon[31538]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:46:27.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:27 smithi123 ceph-mon[35971]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:46:27.719 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:27 smithi151 ceph-mon[36066]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:46:27.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:27 smithi002 ceph-mon[31538]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:46:29.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:29 smithi123 ceph-mon[35971]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:46:29.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:29 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:29.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:29 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:29.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:29 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:46:29.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:29 smithi151 ceph-mon[36066]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:46:29.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:29 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:29.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:29 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:29.740 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:29 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:46:29.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:29 smithi002 ceph-mon[31538]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:46:29.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:29 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:29.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:29 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:29.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:29 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:46:31.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:31 smithi151 ceph-mon[36066]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:46:31.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:31 smithi123 ceph-mon[35971]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:46:31.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:31 smithi002 ceph-mon[31538]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:46:32.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:32 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:32.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:32 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:32.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:32 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:32.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:32 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:46:32.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:32 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:32.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:32 smithi002 ceph-mon[31538]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:46:33.082 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:32 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:33.082 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:32 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:33.082 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:32 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:33.082 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:32 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:46:33.082 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:32 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:33.082 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:32 smithi151 ceph-mon[36066]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:46:33.123 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 2 on host 'smithi151' 2024-05-14T16:46:33.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:33.122+0000 7f19a3fff700 1 -- 172.21.15.151:0/1288856168 <== mgr.14150 v2:172.21.15.2:6800/1498936028 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f19ac063a40 con 0x7f19980b27f0 2024-05-14T16:46:33.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:33.125+0000 7f19b177b700 1 -- 172.21.15.151:0/1288856168 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f19980b27f0 msgr2=0x7f19980b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:33.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:33.125+0000 7f19b177b700 1 --2- 172.21.15.151:0/1288856168 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f19980b27f0 0x7f19980b4cb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1994007900 tx=0x7f1994008040 comp rx=0 tx=0).stop 2024-05-14T16:46:33.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:33.125+0000 7f19b177b700 1 -- 172.21.15.151:0/1288856168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f19ac1035a0 msgr2=0x7f19ac0772e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:33.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:33.125+0000 7f19b177b700 1 --2- 172.21.15.151:0/1288856168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f19ac1035a0 0x7f19ac0772e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f199c005f50 tx=0x7f199c0047b0 comp rx=0 tx=0).stop 2024-05-14T16:46:33.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:33.125+0000 7f19b177b700 1 -- 172.21.15.151:0/1288856168 shutdown_connections 2024-05-14T16:46:33.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:33.125+0000 7f19b177b700 1 --2- 172.21.15.151:0/1288856168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19ac105140 0x7f19ac075e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:33.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:33.125+0000 7f19b177b700 1 --2- 172.21.15.151:0/1288856168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f19ac1035a0 0x7f19ac0772e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:33.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:33.125+0000 7f19b177b700 1 --2- 172.21.15.151:0/1288856168 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f19980b27f0 0x7f19980b4cb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:33.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:33.125+0000 7f19b177b700 1 --2- 172.21.15.151:0/1288856168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19ac1047a0 0x7f19ac075930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:33.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:33.125+0000 7f19b177b700 1 -- 172.21.15.151:0/1288856168 >> 172.21.15.151:0/1288856168 conn(0x7f19ac0feb40 msgr2=0x7f19ac107bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:46:33.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:33.125+0000 7f19b177b700 1 -- 172.21.15.151:0/1288856168 shutdown_connections 2024-05-14T16:46:33.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:46:33.125+0000 7f19b177b700 1 -- 172.21.15.151:0/1288856168 wait complete. 2024-05-14T16:46:33.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:32 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:33.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:32 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:33.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:32 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:33.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:32 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:46:33.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:32 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:33.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:32 smithi123 ceph-mon[35971]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:46:33.761 DEBUG:teuthology.orchestra.run.smithi151:osd.2> sudo journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@osd.2.service 2024-05-14T16:46:33.764 INFO:tasks.cephadm:Deploying osd.3 on smithi002 with /dev/vg_nvme/lv_3... 2024-05-14T16:46:33.764 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-14T16:46:34.010 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:46:34.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:34 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:34.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:34 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:34.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:34 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:46:34.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:34 smithi151 ceph-mon[36066]: from='osd.2 [v2:172.21.15.151:6800/2160189636,v1:172.21.15.151:6801/2160189636]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-14T16:46:34.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:34 smithi151 ceph-mon[36066]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-14T16:46:34.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:34 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:34.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:34 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:34.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:34 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:46:34.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:34 smithi123 ceph-mon[35971]: from='osd.2 [v2:172.21.15.151:6800/2160189636,v1:172.21.15.151:6801/2160189636]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-14T16:46:34.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:34 smithi123 ceph-mon[35971]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-14T16:46:34.412 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:34 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:34.412 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:34 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:34.412 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:34 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:46:34.412 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:34 smithi002 ceph-mon[31538]: from='osd.2 [v2:172.21.15.151:6800/2160189636,v1:172.21.15.151:6801/2160189636]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-14T16:46:34.412 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:34 smithi002 ceph-mon[31538]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-14T16:46:35.257 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:35 smithi151 ceph-mon[36066]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:46:35.257 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:35 smithi151 ceph-mon[36066]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-14T16:46:35.257 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:35 smithi151 ceph-mon[36066]: osdmap e16: 3 total, 2 up, 3 in 2024-05-14T16:46:35.257 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:35 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:46:35.257 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:35 smithi151 ceph-mon[36066]: from='osd.2 [v2:172.21.15.151:6800/2160189636,v1:172.21.15.151:6801/2160189636]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-14T16:46:35.257 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:35 smithi151 ceph-mon[36066]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-14T16:46:35.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:35 smithi123 ceph-mon[35971]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:46:35.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:35 smithi123 ceph-mon[35971]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-14T16:46:35.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:35 smithi123 ceph-mon[35971]: osdmap e16: 3 total, 2 up, 3 in 2024-05-14T16:46:35.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:35 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:46:35.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:35 smithi123 ceph-mon[35971]: from='osd.2 [v2:172.21.15.151:6800/2160189636,v1:172.21.15.151:6801/2160189636]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-14T16:46:35.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:35 smithi123 ceph-mon[35971]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-14T16:46:35.466 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:35 smithi002 ceph-mon[31538]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:46:35.466 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:35 smithi002 ceph-mon[31538]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-14T16:46:35.466 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:35 smithi002 ceph-mon[31538]: osdmap e16: 3 total, 2 up, 3 in 2024-05-14T16:46:35.467 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:35 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:46:35.467 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:35 smithi002 ceph-mon[31538]: from='osd.2 [v2:172.21.15.151:6800/2160189636,v1:172.21.15.151:6801/2160189636]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-14T16:46:35.467 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:35 smithi002 ceph-mon[31538]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-14T16:46:35.854 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 14 16:46:35 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-2[41154]: 2024-05-14T16:46:35.563+0000 7fe217825700 -1 osd.2 0 waiting for initial osdmap 2024-05-14T16:46:35.855 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 14 16:46:35 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-2[41154]: 2024-05-14T16:46:35.570+0000 7fe20f610700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T16:46:36.065 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-14T16:46:36.065 INFO:teuthology.orchestra.run.smithi002.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-14T16:46:36.065 INFO:teuthology.orchestra.run.smithi002.stderr: stderr: 10+0 records in 2024-05-14T16:46:36.066 INFO:teuthology.orchestra.run.smithi002.stderr:10+0 records out 2024-05-14T16:46:36.066 INFO:teuthology.orchestra.run.smithi002.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00849004 s, 1.2 GB/s 2024-05-14T16:46:36.066 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping successful for: 2024-05-14T16:46:36.382 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:36 smithi002 ceph-mon[31538]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-05-14T16:46:36.383 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:36 smithi002 ceph-mon[31538]: osdmap e17: 3 total, 2 up, 3 in 2024-05-14T16:46:36.383 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:36 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:46:36.383 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:36 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:46:36.383 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:36 smithi002 ceph-mon[31538]: from='osd.2 ' entity='osd.2' 2024-05-14T16:46:36.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:36 smithi123 ceph-mon[35971]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-05-14T16:46:36.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:36 smithi123 ceph-mon[35971]: osdmap e17: 3 total, 2 up, 3 in 2024-05-14T16:46:36.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:36 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:46:36.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:36 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:46:36.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:36 smithi123 ceph-mon[35971]: from='osd.2 ' entity='osd.2' 2024-05-14T16:46:36.506 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:36 smithi151 ceph-mon[36066]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-05-14T16:46:36.506 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:36 smithi151 ceph-mon[36066]: osdmap e17: 3 total, 2 up, 3 in 2024-05-14T16:46:36.506 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:36 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:46:36.506 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:36 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:46:36.506 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:36 smithi151 ceph-mon[36066]: from='osd.2 ' entity='osd.2' 2024-05-14T16:46:36.742 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi002:vg_nvme/lv_3 2024-05-14T16:46:36.997 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:46:37.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:37 smithi123 ceph-mon[35971]: purged_snaps scrub starts 2024-05-14T16:46:37.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:37 smithi123 ceph-mon[35971]: purged_snaps scrub ok 2024-05-14T16:46:37.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:37 smithi123 ceph-mon[35971]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:46:37.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:37 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:46:37.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:37 smithi123 ceph-mon[35971]: osd.2 [v2:172.21.15.151:6800/2160189636,v1:172.21.15.151:6801/2160189636] boot 2024-05-14T16:46:37.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:37 smithi123 ceph-mon[35971]: osdmap e18: 3 total, 3 up, 3 in 2024-05-14T16:46:37.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:37 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:46:37.412 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:37 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:37.412 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:37 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:37.412 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:37 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:46:37.412 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:37 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:37.412 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:37 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:37.412 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:37 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:46:37.412 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:37 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:37.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:37 smithi002 ceph-mon[31538]: purged_snaps scrub starts 2024-05-14T16:46:37.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:37 smithi002 ceph-mon[31538]: purged_snaps scrub ok 2024-05-14T16:46:37.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:37 smithi002 ceph-mon[31538]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:46:37.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:37 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:46:37.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:37 smithi002 ceph-mon[31538]: osd.2 [v2:172.21.15.151:6800/2160189636,v1:172.21.15.151:6801/2160189636] boot 2024-05-14T16:46:37.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:37 smithi002 ceph-mon[31538]: osdmap e18: 3 total, 3 up, 3 in 2024-05-14T16:46:37.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:37 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:46:37.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:37 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:37.422 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:37 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:37.423 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:37 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:46:37.423 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:37 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:37.423 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:37 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:37.423 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:37 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:46:37.423 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:37 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:37.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:37 smithi151 ceph-mon[36066]: purged_snaps scrub starts 2024-05-14T16:46:37.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:37 smithi151 ceph-mon[36066]: purged_snaps scrub ok 2024-05-14T16:46:37.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:37 smithi151 ceph-mon[36066]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-14T16:46:37.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:37 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:46:37.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:37 smithi151 ceph-mon[36066]: osd.2 [v2:172.21.15.151:6800/2160189636,v1:172.21.15.151:6801/2160189636] boot 2024-05-14T16:46:37.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:37 smithi151 ceph-mon[36066]: osdmap e18: 3 total, 3 up, 3 in 2024-05-14T16:46:37.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:37 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:46:37.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:37 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:37.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:37 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:37.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:37 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:46:37.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:37 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:37.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:37 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:37.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:37 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:46:37.606 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:37 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:38.353 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:38 smithi002 ceph-mon[31538]: Detected new or changed devices on smithi151 2024-05-14T16:46:38.353 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:38 smithi002 ceph-mon[31538]: Adjusting osd_memory_target on smithi151 to 17088M 2024-05-14T16:46:38.353 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:38 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-14T16:46:38.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:38 smithi123 ceph-mon[35971]: Detected new or changed devices on smithi151 2024-05-14T16:46:38.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:38 smithi123 ceph-mon[35971]: Adjusting osd_memory_target on smithi151 to 17088M 2024-05-14T16:46:38.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:38 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-14T16:46:38.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.555+0000 7f31ef8a4700 1 -- 172.21.15.2:0/2978770630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31e8106960 msgr2=0x7f31e8108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:38.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.555+0000 7f31ef8a4700 1 --2- 172.21.15.2:0/2978770630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31e8106960 0x7f31e8108d50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f31d8009990 tx=0x7f31d8009ca0 comp rx=0 tx=0).stop 2024-05-14T16:46:38.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.555+0000 7f31ef8a4700 1 -- 172.21.15.2:0/2978770630 shutdown_connections 2024-05-14T16:46:38.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.555+0000 7f31ef8a4700 1 --2- 172.21.15.2:0/2978770630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31e8109290 0x7f31e810b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:38.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.555+0000 7f31ef8a4700 1 --2- 172.21.15.2:0/2978770630 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f31e8104030 0x7f31e8106420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:38.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.555+0000 7f31ef8a4700 1 --2- 172.21.15.2:0/2978770630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31e8106960 0x7f31e8108d50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:38.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.555+0000 7f31ef8a4700 1 -- 172.21.15.2:0/2978770630 >> 172.21.15.2:0/2978770630 conn(0x7f31e80fda40 msgr2=0x7f31e80ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:46:38.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.555+0000 7f31ef8a4700 1 -- 172.21.15.2:0/2978770630 shutdown_connections 2024-05-14T16:46:38.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.555+0000 7f31ef8a4700 1 -- 172.21.15.2:0/2978770630 wait complete. 2024-05-14T16:46:38.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.556+0000 7f31ef8a4700 1 Processor -- start 2024-05-14T16:46:38.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.556+0000 7f31ef8a4700 1 -- start start 2024-05-14T16:46:38.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.556+0000 7f31ef8a4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31e8104030 0x7f31e81996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:38.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.556+0000 7f31ef8a4700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f31e8106960 0x7f31e8199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:38.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.556+0000 7f31ef8a4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31e8109290 0x7f31e81a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:38.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.556+0000 7f31ef8a4700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f31e810f060 con 0x7f31e8104030 2024-05-14T16:46:38.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.556+0000 7f31ef8a4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f31e810eee0 con 0x7f31e8109290 2024-05-14T16:46:38.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.556+0000 7f31ef8a4700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f31e810f1e0 con 0x7f31e8106960 2024-05-14T16:46:38.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.556+0000 7f31ed640700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31e8104030 0x7f31e81996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:38.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.556+0000 7f31ed640700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31e8104030 0x7f31e81996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37562/0 (socket says 172.21.15.2:37562) 2024-05-14T16:46:38.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.556+0000 7f31ed640700 1 -- 172.21.15.2:0/627435199 learned_addr learned my addr 172.21.15.2:0/627435199 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:46:38.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.556+0000 7f31ede41700 1 --2- 172.21.15.2:0/627435199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31e8109290 0x7f31e81a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:38.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.556+0000 7f31ed640700 1 -- 172.21.15.2:0/627435199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f31e8106960 msgr2=0x7f31e8199c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:38.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.556+0000 7f31ece3f700 1 --2- 172.21.15.2:0/627435199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f31e8106960 0x7f31e8199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:38.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.557+0000 7f31ed640700 1 --2- 172.21.15.2:0/627435199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f31e8106960 0x7f31e8199c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:38.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.557+0000 7f31ed640700 1 -- 172.21.15.2:0/627435199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31e8109290 msgr2=0x7f31e81a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:38.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.557+0000 7f31ed640700 1 --2- 172.21.15.2:0/627435199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31e8109290 0x7f31e81a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:38.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.557+0000 7f31ed640700 1 -- 172.21.15.2:0/627435199 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e81a1330 con 0x7f31e8104030 2024-05-14T16:46:38.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.557+0000 7f31ede41700 1 --2- 172.21.15.2:0/627435199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31e8109290 0x7f31e81a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:46:38.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.557+0000 7f31ed640700 1 --2- 172.21.15.2:0/627435199 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31e8104030 0x7f31e81996d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f31e400c950 tx=0x7f31e400cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:46:38.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.557+0000 7f31de7fc700 1 -- 172.21.15.2:0/627435199 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e4007ac0 con 0x7f31e8104030 2024-05-14T16:46:38.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.557+0000 7f31ef8a4700 1 -- 172.21.15.2:0/627435199 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31e81a1620 con 0x7f31e8104030 2024-05-14T16:46:38.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.557+0000 7f31de7fc700 1 -- 172.21.15.2:0/627435199 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31e4004ce0 con 0x7f31e8104030 2024-05-14T16:46:38.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.557+0000 7f31de7fc700 1 -- 172.21.15.2:0/627435199 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e4005550 con 0x7f31e8104030 2024-05-14T16:46:38.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.557+0000 7f31ef8a4700 1 -- 172.21.15.2:0/627435199 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31e81a1b60 con 0x7f31e8104030 2024-05-14T16:46:38.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.558+0000 7f31ef8a4700 1 -- 172.21.15.2:0/627435199 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31e806a230 con 0x7f31e8104030 2024-05-14T16:46:38.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.559+0000 7f31de7fc700 1 -- 172.21.15.2:0/627435199 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 15) ==== 149595+0+0 (secure 0 0 0) 0x7f31e4007c60 con 0x7f31e8104030 2024-05-14T16:46:38.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.560+0000 7f31de7fc700 1 --2- 172.21.15.2:0/627435199 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f31d40b27f0 0x7f31d40b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:46:38.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.561+0000 7f31ece3f700 1 --2- 172.21.15.2:0/627435199 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f31d40b27f0 0x7f31d40b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:46:38.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.561+0000 7f31de7fc700 1 -- 172.21.15.2:0/627435199 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3226+0+0 (secure 0 0 0) 0x7f31e40e0d30 con 0x7f31e8104030 2024-05-14T16:46:38.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.561+0000 7f31ece3f700 1 --2- 172.21.15.2:0/627435199 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f31d40b27f0 0x7f31d40b4cb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f31d8006010 tx=0x7f31d8003860 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:46:38.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.564+0000 7f31de7fc700 1 -- 172.21.15.2:0/627435199 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f31e4014030 con 0x7f31e8104030 2024-05-14T16:46:38.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:38 smithi151 ceph-mon[36066]: Detected new or changed devices on smithi151 2024-05-14T16:46:38.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:38 smithi151 ceph-mon[36066]: Adjusting osd_memory_target on smithi151 to 17088M 2024-05-14T16:46:38.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:38 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-14T16:46:38.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:38.790+0000 7f31ef8a4700 1 -- 172.21.15.2:0/627435199 --> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f31e8063980 con 0x7f31d40b27f0 2024-05-14T16:46:39.402 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:39 smithi002 ceph-mon[31538]: pgmap v81: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:46:39.402 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:39 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-14T16:46:39.402 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:39 smithi002 ceph-mon[31538]: osdmap e19: 3 total, 3 up, 3 in 2024-05-14T16:46:39.402 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:39 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-14T16:46:39.402 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:39 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:46:39.402 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:39 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:46:39.402 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:39 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:39.402 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 16:46:39 smithi002 sudo[50940]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-14T16:46:39.402 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 16:46:39 smithi002 sudo[50940]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T16:46:39.403 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 16:46:39 smithi002 sudo[50940]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T16:46:39.403 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 16:46:39 smithi002 sudo[50940]: pam_unix(sudo:session): session closed for user root 2024-05-14T16:46:39.403 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 16:46:39 smithi002 sudo[50943]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-14T16:46:39.403 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 16:46:39 smithi002 sudo[50943]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T16:46:39.403 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 16:46:39 smithi002 sudo[50943]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T16:46:39.403 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 16:46:39 smithi002 sudo[50943]: pam_unix(sudo:session): session closed for user root 2024-05-14T16:46:39.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:39 smithi123 ceph-mon[35971]: pgmap v81: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:46:39.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:39 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-14T16:46:39.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:39 smithi123 ceph-mon[35971]: osdmap e19: 3 total, 3 up, 3 in 2024-05-14T16:46:39.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:39 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-14T16:46:39.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:39 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:46:39.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:39 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:46:39.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:39 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:39.410 INFO:journalctl@ceph.osd.1.smithi123.stdout:May 14 16:46:39 smithi123 sudo[44702]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-14T16:46:39.410 INFO:journalctl@ceph.osd.1.smithi123.stdout:May 14 16:46:39 smithi123 sudo[44702]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T16:46:39.410 INFO:journalctl@ceph.osd.1.smithi123.stdout:May 14 16:46:39 smithi123 sudo[44702]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T16:46:39.410 INFO:journalctl@ceph.osd.1.smithi123.stdout:May 14 16:46:39 smithi123 sudo[44702]: pam_unix(sudo:session): session closed for user root 2024-05-14T16:46:39.411 INFO:journalctl@ceph.osd.1.smithi123.stdout:May 14 16:46:39 smithi123 sudo[44705]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-14T16:46:39.411 INFO:journalctl@ceph.osd.1.smithi123.stdout:May 14 16:46:39 smithi123 sudo[44705]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T16:46:39.411 INFO:journalctl@ceph.osd.1.smithi123.stdout:May 14 16:46:39 smithi123 sudo[44705]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T16:46:39.411 INFO:journalctl@ceph.osd.1.smithi123.stdout:May 14 16:46:39 smithi123 sudo[44705]: pam_unix(sudo:session): session closed for user root 2024-05-14T16:46:39.441 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 14 16:46:39 smithi151 sudo[45066]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-14T16:46:39.441 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 14 16:46:39 smithi151 sudo[45066]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T16:46:39.441 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 14 16:46:39 smithi151 sudo[45066]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T16:46:39.441 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 14 16:46:39 smithi151 sudo[45066]: pam_unix(sudo:session): session closed for user root 2024-05-14T16:46:39.441 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 14 16:46:39 smithi151 sudo[45069]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-14T16:46:39.441 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 14 16:46:39 smithi151 sudo[45069]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T16:46:39.441 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 14 16:46:39 smithi151 sudo[45069]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T16:46:39.441 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 14 16:46:39 smithi151 sudo[45069]: pam_unix(sudo:session): session closed for user root 2024-05-14T16:46:39.441 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:39 smithi151 ceph-mon[36066]: pgmap v81: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:46:39.441 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:39 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-14T16:46:39.442 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:39 smithi151 ceph-mon[36066]: osdmap e19: 3 total, 3 up, 3 in 2024-05-14T16:46:39.442 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:39 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-14T16:46:39.442 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:39 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:46:39.442 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:39 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:46:39.442 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:39 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:39.442 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:39 smithi151 sudo[45072]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-14T16:46:39.699 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:39 smithi002 sudo[51002]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-14T16:46:39.699 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:39 smithi002 sudo[51002]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T16:46:39.699 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:39 smithi002 sudo[51002]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T16:46:39.699 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:39 smithi002 sudo[51002]: pam_unix(sudo:session): session closed for user root 2024-05-14T16:46:39.699 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:39 smithi002 sudo[51005]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-14T16:46:39.699 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:39 smithi002 sudo[51005]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T16:46:39.699 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:39 smithi002 sudo[51005]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T16:46:39.699 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:39 smithi002 sudo[51005]: pam_unix(sudo:session): session closed for user root 2024-05-14T16:46:39.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:39 smithi151 sudo[45072]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T16:46:39.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:39 smithi151 sudo[45072]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T16:46:39.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:39 smithi151 sudo[45072]: pam_unix(sudo:session): session closed for user root 2024-05-14T16:46:39.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:39 smithi151 sudo[45075]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-14T16:46:39.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:39 smithi151 sudo[45075]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T16:46:39.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:39 smithi151 sudo[45075]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T16:46:39.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:39 smithi151 sudo[45075]: pam_unix(sudo:session): session closed for user root 2024-05-14T16:46:39.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:39 smithi123 sudo[44708]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-14T16:46:39.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:39 smithi123 sudo[44708]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T16:46:39.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:39 smithi123 sudo[44708]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T16:46:39.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:39 smithi123 sudo[44708]: pam_unix(sudo:session): session closed for user root 2024-05-14T16:46:39.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:39 smithi123 sudo[44711]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-14T16:46:39.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:39 smithi123 sudo[44711]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-14T16:46:39.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:39 smithi123 sudo[44711]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-14T16:46:39.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:39 smithi123 sudo[44711]: pam_unix(sudo:session): session closed for user root 2024-05-14T16:46:40.404 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:40 smithi002 ceph-mon[31538]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:46:40.405 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:40 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-14T16:46:40.405 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:40 smithi002 ceph-mon[31538]: osdmap e20: 3 total, 3 up, 3 in 2024-05-14T16:46:40.405 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:40 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:40.405 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:40 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:40.405 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:40 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:40.405 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:40 smithi002 ceph-mon[31538]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T16:46:40.405 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:40 smithi002 ceph-mon[31538]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T16:46:40.405 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:40 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:46:40.405 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:40 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:46:40.405 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:40 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:46:40.405 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:40 smithi002 ceph-mon[31538]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T16:46:40.405 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:40 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:46:40.405 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:40 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:46:40.405 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:40 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:46:40.406 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:40 smithi002 ceph-mon[31538]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T16:46:40.406 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:40 smithi002 ceph-mon[31538]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T16:46:40.406 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:40 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:46:40.406 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:40 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:46:40.406 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:40 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:46:40.406 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:40 smithi002 ceph-mon[31538]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T16:46:40.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:40.403+0000 7f31de7fc700 1 -- 172.21.15.2:0/627435199 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f31e40a5aa0 con 0x7f31e8104030 2024-05-14T16:46:40.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:40 smithi123 ceph-mon[35971]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:46:40.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:40 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-14T16:46:40.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:40 smithi123 ceph-mon[35971]: osdmap e20: 3 total, 3 up, 3 in 2024-05-14T16:46:40.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:40 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:40.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:40 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:40.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:40 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:40.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:40 smithi123 ceph-mon[35971]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T16:46:40.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:40 smithi123 ceph-mon[35971]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T16:46:40.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:40 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:46:40.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:40 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:46:40.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:40 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:46:40.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:40 smithi123 ceph-mon[35971]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T16:46:40.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:40 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:46:40.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:40 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:46:40.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:40 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:46:40.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:40 smithi123 ceph-mon[35971]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T16:46:40.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:40 smithi123 ceph-mon[35971]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T16:46:40.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:40 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:46:40.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:40 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:46:40.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:40 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:46:40.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:40 smithi123 ceph-mon[35971]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T16:46:40.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:40 smithi151 ceph-mon[36066]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:46:40.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:40 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-14T16:46:40.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:40 smithi151 ceph-mon[36066]: osdmap e20: 3 total, 3 up, 3 in 2024-05-14T16:46:40.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:40 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:40.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:40 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:40.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:40 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:40.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:40 smithi151 ceph-mon[36066]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T16:46:40.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:40 smithi151 ceph-mon[36066]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T16:46:40.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:40 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:46:40.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:40 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:46:40.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:40 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:46:40.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:40 smithi151 ceph-mon[36066]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T16:46:40.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:40 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:46:40.606 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:40 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:46:40.606 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:40 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:46:40.606 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:40 smithi151 ceph-mon[36066]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T16:46:40.606 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:40 smithi151 ceph-mon[36066]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-14T16:46:40.606 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:40 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:46:40.606 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:40 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:46:40.606 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:40 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:46:40.606 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:40 smithi151 ceph-mon[36066]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-14T16:46:41.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:41 smithi002 ceph-mon[31538]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:46:41.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:41 smithi002 ceph-mon[31538]: osdmap e21: 3 total, 3 up, 3 in 2024-05-14T16:46:41.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:41 smithi002 ceph-mon[31538]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-14T16:46:41.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:41 smithi151 ceph-mon[36066]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:46:41.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:41 smithi151 ceph-mon[36066]: osdmap e21: 3 total, 3 up, 3 in 2024-05-14T16:46:41.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:41 smithi151 ceph-mon[36066]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-14T16:46:41.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:41 smithi123 ceph-mon[35971]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:46:41.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:41 smithi123 ceph-mon[35971]: osdmap e21: 3 total, 3 up, 3 in 2024-05-14T16:46:41.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:41 smithi123 ceph-mon[35971]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-14T16:46:42.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:42 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/3290773446' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2fd592bf-c212-411a-afbe-a7c498e5ac93"}]: dispatch 2024-05-14T16:46:42.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:42 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/3290773446' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2fd592bf-c212-411a-afbe-a7c498e5ac93"}]': finished 2024-05-14T16:46:42.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:42 smithi002 ceph-mon[31538]: osdmap e22: 4 total, 3 up, 4 in 2024-05-14T16:46:42.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:42 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:46:42.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:42 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/3290773446' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2fd592bf-c212-411a-afbe-a7c498e5ac93"}]: dispatch 2024-05-14T16:46:42.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:42 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/3290773446' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2fd592bf-c212-411a-afbe-a7c498e5ac93"}]': finished 2024-05-14T16:46:42.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:42 smithi151 ceph-mon[36066]: osdmap e22: 4 total, 3 up, 4 in 2024-05-14T16:46:42.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:42 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:46:42.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:42 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/3290773446' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2fd592bf-c212-411a-afbe-a7c498e5ac93"}]: dispatch 2024-05-14T16:46:42.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:42 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/3290773446' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2fd592bf-c212-411a-afbe-a7c498e5ac93"}]': finished 2024-05-14T16:46:42.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:42 smithi123 ceph-mon[35971]: osdmap e22: 4 total, 3 up, 4 in 2024-05-14T16:46:42.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:42 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:46:43.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:43 smithi002 ceph-mon[31538]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:46:43.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:43 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/2041579046' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:46:43.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:43 smithi151 ceph-mon[36066]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:46:43.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:43 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/2041579046' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:46:43.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:43 smithi123 ceph-mon[35971]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:46:43.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:43 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/2041579046' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:46:45.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:45 smithi002 ceph-mon[31538]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:46:45.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:45 smithi151 ceph-mon[36066]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:46:45.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:45 smithi123 ceph-mon[35971]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:46:47.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:47 smithi002 ceph-mon[31538]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:46:47.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:47 smithi151 ceph-mon[36066]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:46:47.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:47 smithi123 ceph-mon[35971]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:46:49.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:49 smithi002 ceph-mon[31538]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:46:49.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:49 smithi151 ceph-mon[36066]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:46:49.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:49 smithi123 ceph-mon[35971]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:46:50.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:50 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-14T16:46:50.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:50 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:50.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:50 smithi002 ceph-mon[31538]: Deploying daemon osd.3 on smithi002 2024-05-14T16:46:50.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:50 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-14T16:46:50.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:50 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:50.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:50 smithi151 ceph-mon[36066]: Deploying daemon osd.3 on smithi002 2024-05-14T16:46:50.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:50 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-14T16:46:50.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:50 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:50.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:50 smithi123 ceph-mon[35971]: Deploying daemon osd.3 on smithi002 2024-05-14T16:46:51.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:51 smithi002 ceph-mon[31538]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:46:51.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:51 smithi151 ceph-mon[36066]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:46:51.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:51 smithi123 ceph-mon[35971]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:46:53.462 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:53 smithi123 ceph-mon[35971]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:46:53.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:53 smithi002 ceph-mon[31538]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:46:53.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:53 smithi151 ceph-mon[36066]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:46:55.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:55 smithi002 ceph-mon[31538]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:46:55.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:55 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:55.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:55 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:55.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:55 smithi151 ceph-mon[36066]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:46:55.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:55 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:55.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:55 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:55.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:55 smithi123 ceph-mon[35971]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:46:55.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:55 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:55.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:55 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:56.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:56 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:46:56.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:56 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:46:56.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:56 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:46:57.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:57 smithi002 ceph-mon[31538]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:46:57.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:57 smithi151 ceph-mon[36066]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:46:57.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:57 smithi123 ceph-mon[35971]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:46:58.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:58.854+0000 7f31de7fc700 1 -- 172.21.15.2:0/627435199 <== mgr.14150 v2:172.21.15.2:6800/1498936028 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f31e8063980 con 0x7f31d40b27f0 2024-05-14T16:46:58.854 INFO:teuthology.orchestra.run.smithi002.stdout:Created osd(s) 3 on host 'smithi002' 2024-05-14T16:46:58.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:58.856+0000 7f31ef8a4700 1 -- 172.21.15.2:0/627435199 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f31d40b27f0 msgr2=0x7f31d40b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:58.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:58.856+0000 7f31ef8a4700 1 --2- 172.21.15.2:0/627435199 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f31d40b27f0 0x7f31d40b4cb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f31d8006010 tx=0x7f31d8003860 comp rx=0 tx=0).stop 2024-05-14T16:46:58.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:58.856+0000 7f31ef8a4700 1 -- 172.21.15.2:0/627435199 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31e8104030 msgr2=0x7f31e81996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:46:58.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:58.856+0000 7f31ef8a4700 1 --2- 172.21.15.2:0/627435199 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31e8104030 0x7f31e81996d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f31e400c950 tx=0x7f31e400cd10 comp rx=0 tx=0).stop 2024-05-14T16:46:58.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:58.856+0000 7f31ef8a4700 1 -- 172.21.15.2:0/627435199 shutdown_connections 2024-05-14T16:46:58.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:58.856+0000 7f31ef8a4700 1 --2- 172.21.15.2:0/627435199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31e8109290 0x7f31e81a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:58.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:58.856+0000 7f31ef8a4700 1 --2- 172.21.15.2:0/627435199 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f31d40b27f0 0x7f31d40b4cb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:58.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:58.856+0000 7f31ef8a4700 1 --2- 172.21.15.2:0/627435199 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31e8104030 0x7f31e81996d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:58.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:58.856+0000 7f31ef8a4700 1 --2- 172.21.15.2:0/627435199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f31e8106960 0x7f31e8199c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:46:58.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:58.856+0000 7f31ef8a4700 1 -- 172.21.15.2:0/627435199 >> 172.21.15.2:0/627435199 conn(0x7f31e80fda40 msgr2=0x7f31e81048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:46:58.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:58.857+0000 7f31ef8a4700 1 -- 172.21.15.2:0/627435199 shutdown_connections 2024-05-14T16:46:58.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:46:58.857+0000 7f31ef8a4700 1 -- 172.21.15.2:0/627435199 wait complete. 2024-05-14T16:46:59.073 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:59 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:59.074 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:59 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:59.074 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:59 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:59.074 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:59 smithi002 ceph-mon[31538]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:46:59.074 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:59 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:46:59.074 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:59 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:59.074 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:59 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:46:59.074 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:59 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:59.074 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:46:59 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:59.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:59 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:59.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:59 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:59.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:59 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:59.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:59 smithi151 ceph-mon[36066]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:46:59.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:59 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:46:59.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:59 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:59.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:59 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:46:59.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:59 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:59.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:46:59 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:59.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:59 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:59.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:59 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:59.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:59 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:46:59.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:59 smithi123 ceph-mon[35971]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:46:59.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:59 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:46:59.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:59 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:59.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:59 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:46:59.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:59 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:59.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:46:59 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:46:59.626 DEBUG:teuthology.orchestra.run.smithi002:osd.3> sudo journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@osd.3.service 2024-05-14T16:46:59.629 INFO:tasks.cephadm:Deploying osd.4 on smithi123 with /dev/vg_nvme/lv_3... 2024-05-14T16:46:59.630 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-14T16:46:59.892 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T16:47:00.176 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:00 smithi002 ceph-mon[31538]: from='osd.3 [v2:172.21.15.2:6810/3451424073,v1:172.21.15.2:6811/3451424073]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-14T16:47:00.332 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:00 smithi123 ceph-mon[35971]: from='osd.3 [v2:172.21.15.2:6810/3451424073,v1:172.21.15.2:6811/3451424073]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-14T16:47:00.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:00 smithi151 ceph-mon[36066]: from='osd.3 [v2:172.21.15.2:6810/3451424073,v1:172.21.15.2:6811/3451424073]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-14T16:47:01.254 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:01 smithi123 ceph-mon[35971]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:47:01.254 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:01 smithi123 ceph-mon[35971]: from='osd.3 [v2:172.21.15.2:6810/3451424073,v1:172.21.15.2:6811/3451424073]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-14T16:47:01.254 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:01 smithi123 ceph-mon[35971]: osdmap e23: 4 total, 3 up, 4 in 2024-05-14T16:47:01.254 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:01 smithi123 ceph-mon[35971]: from='osd.3 [v2:172.21.15.2:6810/3451424073,v1:172.21.15.2:6811/3451424073]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-05-14T16:47:01.254 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:01 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:47:01.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:01 smithi151 ceph-mon[36066]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:47:01.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:01 smithi151 ceph-mon[36066]: from='osd.3 [v2:172.21.15.2:6810/3451424073,v1:172.21.15.2:6811/3451424073]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-14T16:47:01.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:01 smithi151 ceph-mon[36066]: osdmap e23: 4 total, 3 up, 4 in 2024-05-14T16:47:01.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:01 smithi151 ceph-mon[36066]: from='osd.3 [v2:172.21.15.2:6810/3451424073,v1:172.21.15.2:6811/3451424073]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-05-14T16:47:01.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:01 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:47:01.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:01 smithi002 ceph-mon[31538]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:47:01.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:01 smithi002 ceph-mon[31538]: from='osd.3 [v2:172.21.15.2:6810/3451424073,v1:172.21.15.2:6811/3451424073]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-14T16:47:01.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:01 smithi002 ceph-mon[31538]: osdmap e23: 4 total, 3 up, 4 in 2024-05-14T16:47:01.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:01 smithi002 ceph-mon[31538]: from='osd.3 [v2:172.21.15.2:6810/3451424073,v1:172.21.15.2:6811/3451424073]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-05-14T16:47:01.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:01 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:47:01.866 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-14T16:47:01.867 INFO:teuthology.orchestra.run.smithi123.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-14T16:47:01.867 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10+0 records in 2024-05-14T16:47:01.867 INFO:teuthology.orchestra.run.smithi123.stderr:10+0 records out 2024-05-14T16:47:01.867 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011302 s, 928 MB/s 2024-05-14T16:47:01.867 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping successful for: 2024-05-14T16:47:01.942 INFO:journalctl@ceph.osd.3.smithi002.stdout:May 14 16:47:01 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-3[53111]: 2024-05-14T16:47:01.545+0000 7f6bf8a4f700 -1 osd.3 0 waiting for initial osdmap 2024-05-14T16:47:01.942 INFO:journalctl@ceph.osd.3.smithi002.stdout:May 14 16:47:01 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-3[53111]: 2024-05-14T16:47:01.552+0000 7f6bf083a700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T16:47:02.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:02 smithi123 ceph-mon[35971]: from='osd.3 [v2:172.21.15.2:6810/3451424073,v1:172.21.15.2:6811/3451424073]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-05-14T16:47:02.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:02 smithi123 ceph-mon[35971]: osdmap e24: 4 total, 3 up, 4 in 2024-05-14T16:47:02.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:02 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:47:02.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:02 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:47:02.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:02 smithi123 ceph-mon[35971]: from='osd.3 [v2:172.21.15.2:6810/3451424073,v1:172.21.15.2:6811/3451424073]' entity='osd.3' 2024-05-14T16:47:02.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:02 smithi123 ceph-mon[35971]: osd.3 [v2:172.21.15.2:6810/3451424073,v1:172.21.15.2:6811/3451424073] boot 2024-05-14T16:47:02.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:02 smithi123 ceph-mon[35971]: osdmap e25: 4 total, 4 up, 4 in 2024-05-14T16:47:02.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:02 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:47:02.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:02 smithi002 ceph-mon[31538]: from='osd.3 [v2:172.21.15.2:6810/3451424073,v1:172.21.15.2:6811/3451424073]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-05-14T16:47:02.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:02 smithi002 ceph-mon[31538]: osdmap e24: 4 total, 3 up, 4 in 2024-05-14T16:47:02.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:02 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:47:02.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:02 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:47:02.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:02 smithi002 ceph-mon[31538]: from='osd.3 [v2:172.21.15.2:6810/3451424073,v1:172.21.15.2:6811/3451424073]' entity='osd.3' 2024-05-14T16:47:02.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:02 smithi002 ceph-mon[31538]: osd.3 [v2:172.21.15.2:6810/3451424073,v1:172.21.15.2:6811/3451424073] boot 2024-05-14T16:47:02.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:02 smithi002 ceph-mon[31538]: osdmap e25: 4 total, 4 up, 4 in 2024-05-14T16:47:02.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:02 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:47:02.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:02 smithi151 ceph-mon[36066]: from='osd.3 [v2:172.21.15.2:6810/3451424073,v1:172.21.15.2:6811/3451424073]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-05-14T16:47:02.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:02 smithi151 ceph-mon[36066]: osdmap e24: 4 total, 3 up, 4 in 2024-05-14T16:47:02.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:02 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:47:02.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:02 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:47:02.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:02 smithi151 ceph-mon[36066]: from='osd.3 [v2:172.21.15.2:6810/3451424073,v1:172.21.15.2:6811/3451424073]' entity='osd.3' 2024-05-14T16:47:02.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:02 smithi151 ceph-mon[36066]: osd.3 [v2:172.21.15.2:6810/3451424073,v1:172.21.15.2:6811/3451424073] boot 2024-05-14T16:47:02.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:02 smithi151 ceph-mon[36066]: osdmap e25: 4 total, 4 up, 4 in 2024-05-14T16:47:02.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:02 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:47:02.512 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi123:vg_nvme/lv_3 2024-05-14T16:47:02.754 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T16:47:03.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:03 smithi151 ceph-mon[36066]: purged_snaps scrub starts 2024-05-14T16:47:03.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:03 smithi151 ceph-mon[36066]: purged_snaps scrub ok 2024-05-14T16:47:03.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:03 smithi151 ceph-mon[36066]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:47:03.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:03 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:03.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:03 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:03.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:03 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:47:03.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:03 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:47:03.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:03 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:03.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:03 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:47:03.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:03 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:47:03.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:03 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:03.356 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:03 smithi151 ceph-mon[36066]: osdmap e26: 4 total, 4 up, 4 in 2024-05-14T16:47:03.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:03 smithi123 ceph-mon[35971]: purged_snaps scrub starts 2024-05-14T16:47:03.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:03 smithi123 ceph-mon[35971]: purged_snaps scrub ok 2024-05-14T16:47:03.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:03 smithi123 ceph-mon[35971]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:47:03.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:03 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:03.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:03 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:03.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:03 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:47:03.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:03 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:47:03.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:03 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:03.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:03 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:47:03.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:03 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:47:03.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:03 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:03.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:03 smithi123 ceph-mon[35971]: osdmap e26: 4 total, 4 up, 4 in 2024-05-14T16:47:03.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:03 smithi002 ceph-mon[31538]: purged_snaps scrub starts 2024-05-14T16:47:03.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:03 smithi002 ceph-mon[31538]: purged_snaps scrub ok 2024-05-14T16:47:03.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:03 smithi002 ceph-mon[31538]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-14T16:47:03.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:03 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:03.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:03 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:03.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:03 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:47:03.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:03 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:47:03.495 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:03 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:03.495 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:03 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:47:03.495 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:03 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:47:03.495 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:03 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:03.495 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:03 smithi002 ceph-mon[31538]: osdmap e26: 4 total, 4 up, 4 in 2024-05-14T16:47:04.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:04 smithi151 ceph-mon[36066]: Detected new or changed devices on smithi002 2024-05-14T16:47:04.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:04 smithi151 ceph-mon[36066]: Adjusting osd_memory_target on smithi002 to 8546M 2024-05-14T16:47:04.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:04 smithi151 ceph-mon[36066]: osdmap e27: 4 total, 4 up, 4 in 2024-05-14T16:47:04.386 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:04 smithi123 ceph-mon[35971]: Detected new or changed devices on smithi002 2024-05-14T16:47:04.386 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:04 smithi123 ceph-mon[35971]: Adjusting osd_memory_target on smithi002 to 8546M 2024-05-14T16:47:04.386 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:04 smithi123 ceph-mon[35971]: osdmap e27: 4 total, 4 up, 4 in 2024-05-14T16:47:04.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.389+0000 7fb1c347b700 1 -- 172.21.15.123:0/3262782955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1bc104040 msgr2=0x7fb1bc10a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:04.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.389+0000 7fb1c347b700 1 --2- 172.21.15.123:0/3262782955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1bc104040 0x7fb1bc10a8f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb1b80099c0 tx=0x7fb1b8009cd0 comp rx=0 tx=0).stop 2024-05-14T16:47:04.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.389+0000 7fb1c347b700 1 -- 172.21.15.123:0/3262782955 shutdown_connections 2024-05-14T16:47:04.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.389+0000 7fb1c347b700 1 --2- 172.21.15.123:0/3262782955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1bc104040 0x7fb1bc10a8f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:04.389 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.389+0000 7fb1c347b700 1 --2- 172.21.15.123:0/3262782955 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1bc1024a0 0x7fb1bc1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:04.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.389+0000 7fb1c347b700 1 --2- 172.21.15.123:0/3262782955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1bc1036a0 0x7fb1bc103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:04.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.389+0000 7fb1c347b700 1 -- 172.21.15.123:0/3262782955 >> 172.21.15.123:0/3262782955 conn(0x7fb1bc0fda40 msgr2=0x7fb1bc0ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:04.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.389+0000 7fb1c347b700 1 -- 172.21.15.123:0/3262782955 shutdown_connections 2024-05-14T16:47:04.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.389+0000 7fb1c347b700 1 -- 172.21.15.123:0/3262782955 wait complete. 2024-05-14T16:47:04.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.389+0000 7fb1c347b700 1 Processor -- start 2024-05-14T16:47:04.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.389+0000 7fb1c347b700 1 -- start start 2024-05-14T16:47:04.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.390+0000 7fb1c347b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1bc1024a0 0x7fb1bc1996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:04.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.390+0000 7fb1c347b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1bc1036a0 0x7fb1bc199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:04.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.390+0000 7fb1c347b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1bc104040 0x7fb1bc1a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:04.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.390+0000 7fb1c347b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb1bc10df90 con 0x7fb1bc1024a0 2024-05-14T16:47:04.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.390+0000 7fb1c347b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb1bc10de10 con 0x7fb1bc1036a0 2024-05-14T16:47:04.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.390+0000 7fb1c347b700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb1bc10e110 con 0x7fb1bc104040 2024-05-14T16:47:04.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.390+0000 7fb1c0a16700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1bc1036a0 0x7fb1bc199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:04.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.390+0000 7fb1c0a16700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1bc1036a0 0x7fb1bc199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54480/0 (socket says 172.21.15.123:54480) 2024-05-14T16:47:04.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.390+0000 7fb1c0a16700 1 -- 172.21.15.123:0/3026200410 learned_addr learned my addr 172.21.15.123:0/3026200410 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-14T16:47:04.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.390+0000 7fb1c1217700 1 --2- 172.21.15.123:0/3026200410 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1bc1024a0 0x7fb1bc1996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:04.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.390+0000 7fb1c1a18700 1 --2- 172.21.15.123:0/3026200410 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1bc104040 0x7fb1bc1a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:04.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.390+0000 7fb1c0a16700 1 -- 172.21.15.123:0/3026200410 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1bc104040 msgr2=0x7fb1bc1a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:04.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.390+0000 7fb1c0a16700 1 --2- 172.21.15.123:0/3026200410 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1bc104040 0x7fb1bc1a0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:04.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.390+0000 7fb1c0a16700 1 -- 172.21.15.123:0/3026200410 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1bc1024a0 msgr2=0x7fb1bc1996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:04.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.390+0000 7fb1c0a16700 1 --2- 172.21.15.123:0/3026200410 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1bc1024a0 0x7fb1bc1996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:04.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.390+0000 7fb1c0a16700 1 -- 172.21.15.123:0/3026200410 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1bc1a1350 con 0x7fb1bc1036a0 2024-05-14T16:47:04.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.390+0000 7fb1c0a16700 1 --2- 172.21.15.123:0/3026200410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1bc1036a0 0x7fb1bc199c30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac00b530 tx=0x7fb1ac00b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:04.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.390+0000 7fb1b27fc700 1 -- 172.21.15.123:0/3026200410 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac015020 con 0x7fb1bc1036a0 2024-05-14T16:47:04.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.390+0000 7fb1b27fc700 1 -- 172.21.15.123:0/3026200410 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1ac011ce0 con 0x7fb1bc1036a0 2024-05-14T16:47:04.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.390+0000 7fb1c347b700 1 -- 172.21.15.123:0/3026200410 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1bc1a1640 con 0x7fb1bc1036a0 2024-05-14T16:47:04.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.390+0000 7fb1b27fc700 1 -- 172.21.15.123:0/3026200410 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac00f2e0 con 0x7fb1bc1036a0 2024-05-14T16:47:04.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.390+0000 7fb1c347b700 1 -- 172.21.15.123:0/3026200410 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1bc1a1c50 con 0x7fb1bc1036a0 2024-05-14T16:47:04.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.391+0000 7fb1c347b700 1 -- 172.21.15.123:0/3026200410 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1bc06a230 con 0x7fb1bc1036a0 2024-05-14T16:47:04.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.392+0000 7fb1b27fc700 1 -- 172.21.15.123:0/3026200410 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fb1ac018020 con 0x7fb1bc1036a0 2024-05-14T16:47:04.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.393+0000 7fb1b27fc700 1 --2- 172.21.15.123:0/3026200410 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fb1a80b2900 0x7fb1a80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:04.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.393+0000 7fb1b27fc700 1 -- 172.21.15.123:0/3026200410 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3669+0+0 (secure 0 0 0) 0x7fb1ac0e0d60 con 0x7fb1bc1036a0 2024-05-14T16:47:04.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.393+0000 7fb1c1217700 1 --2- 172.21.15.123:0/3026200410 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fb1a80b2900 0x7fb1a80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:04.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.394+0000 7fb1c1217700 1 --2- 172.21.15.123:0/3026200410 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fb1a80b2900 0x7fb1a80b4dc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc103500 tx=0x7fb1b400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:04.397 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.397+0000 7fb1b27fc700 1 -- 172.21.15.123:0/3026200410 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb1ac01f020 con 0x7fb1bc1036a0 2024-05-14T16:47:04.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:04 smithi002 ceph-mon[31538]: Detected new or changed devices on smithi002 2024-05-14T16:47:04.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:04 smithi002 ceph-mon[31538]: Adjusting osd_memory_target on smithi002 to 8546M 2024-05-14T16:47:04.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:04 smithi002 ceph-mon[31538]: osdmap e27: 4 total, 4 up, 4 in 2024-05-14T16:47:04.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:04.604+0000 7fb1c347b700 1 -- 172.21.15.123:0/3026200410 --> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb1bc0ffc10 con 0x7fb1a80b2900 2024-05-14T16:47:05.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:05 smithi151 ceph-mon[36066]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-05-14T16:47:05.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:05 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:47:05.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:05 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:47:05.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:05 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:47:05.379 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:05 smithi123 ceph-mon[35971]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-05-14T16:47:05.379 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:05 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:47:05.379 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:05 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:47:05.379 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:05 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:47:05.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:05 smithi002 ceph-mon[31538]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-05-14T16:47:05.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:05 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:47:05.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:05 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:47:05.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:05 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:47:06.251 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:06 smithi123 ceph-mon[35971]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:47:06.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:06 smithi151 ceph-mon[36066]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:47:06.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:06 smithi002 ceph-mon[31538]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:47:07.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:07 smithi151 ceph-mon[36066]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-05-14T16:47:07.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:07 smithi123 ceph-mon[35971]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-05-14T16:47:07.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:07 smithi002 ceph-mon[31538]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-05-14T16:47:08.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:08 smithi151 ceph-mon[36066]: from='client.? 172.21.15.123:0/3974871578' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af1905e7-e036-47c6-be59-b085bc41fede"}]: dispatch 2024-05-14T16:47:08.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:08 smithi151 ceph-mon[36066]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af1905e7-e036-47c6-be59-b085bc41fede"}]: dispatch 2024-05-14T16:47:08.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:08 smithi151 ceph-mon[36066]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "af1905e7-e036-47c6-be59-b085bc41fede"}]': finished 2024-05-14T16:47:08.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:08 smithi151 ceph-mon[36066]: osdmap e28: 5 total, 4 up, 5 in 2024-05-14T16:47:08.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:08 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:47:08.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:08 smithi151 ceph-mon[36066]: from='client.? 172.21.15.123:0/3830738136' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:47:08.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:08 smithi123 ceph-mon[35971]: from='client.? 172.21.15.123:0/3974871578' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af1905e7-e036-47c6-be59-b085bc41fede"}]: dispatch 2024-05-14T16:47:08.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:08 smithi123 ceph-mon[35971]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af1905e7-e036-47c6-be59-b085bc41fede"}]: dispatch 2024-05-14T16:47:08.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:08 smithi123 ceph-mon[35971]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "af1905e7-e036-47c6-be59-b085bc41fede"}]': finished 2024-05-14T16:47:08.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:08 smithi123 ceph-mon[35971]: osdmap e28: 5 total, 4 up, 5 in 2024-05-14T16:47:08.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:08 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:47:08.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:08 smithi123 ceph-mon[35971]: from='client.? 172.21.15.123:0/3830738136' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:47:08.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:08 smithi002 ceph-mon[31538]: from='client.? 172.21.15.123:0/3974871578' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af1905e7-e036-47c6-be59-b085bc41fede"}]: dispatch 2024-05-14T16:47:08.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:08 smithi002 ceph-mon[31538]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af1905e7-e036-47c6-be59-b085bc41fede"}]: dispatch 2024-05-14T16:47:08.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:08 smithi002 ceph-mon[31538]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "af1905e7-e036-47c6-be59-b085bc41fede"}]': finished 2024-05-14T16:47:08.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:08 smithi002 ceph-mon[31538]: osdmap e28: 5 total, 4 up, 5 in 2024-05-14T16:47:08.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:08 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:47:08.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:08 smithi002 ceph-mon[31538]: from='client.? 172.21.15.123:0/3830738136' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:47:09.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:09 smithi123 ceph-mon[35971]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail 2024-05-14T16:47:09.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:09 smithi002 ceph-mon[31538]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail 2024-05-14T16:47:09.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:09 smithi151 ceph-mon[36066]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail 2024-05-14T16:47:11.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:11 smithi123 ceph-mon[35971]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-05-14T16:47:11.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:11 smithi002 ceph-mon[31538]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-05-14T16:47:11.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:11 smithi151 ceph-mon[36066]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-05-14T16:47:13.494 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:13 smithi123 ceph-mon[35971]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-05-14T16:47:13.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:13 smithi002 ceph-mon[31538]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-05-14T16:47:13.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:13 smithi151 ceph-mon[36066]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-05-14T16:47:15.649 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:15 smithi123 ceph-mon[35971]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:47:15.650 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:15 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-14T16:47:15.650 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:15 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:47:15.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:15 smithi002 ceph-mon[31538]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:47:15.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:15 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-14T16:47:15.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:15 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:47:15.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:15 smithi151 ceph-mon[36066]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:47:15.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:15 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-14T16:47:15.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:15 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:47:16.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:16 smithi123 ceph-mon[35971]: Deploying daemon osd.4 on smithi123 2024-05-14T16:47:16.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:16 smithi002 ceph-mon[31538]: Deploying daemon osd.4 on smithi123 2024-05-14T16:47:16.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:16 smithi151 ceph-mon[36066]: Deploying daemon osd.4 on smithi123 2024-05-14T16:47:17.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:17 smithi123 ceph-mon[35971]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:47:17.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:17 smithi002 ceph-mon[31538]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:47:17.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:17 smithi151 ceph-mon[36066]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:47:19.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:19 smithi123 ceph-mon[35971]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-05-14T16:47:19.684 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:19 smithi151 ceph-mon[36066]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-05-14T16:47:19.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:19 smithi002 ceph-mon[31538]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-05-14T16:47:21.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:21 smithi123 ceph-mon[35971]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-14T16:47:21.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:21 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:21.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:21 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:21.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:21 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:47:21.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:21 smithi002 ceph-mon[31538]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-14T16:47:21.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:21 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:21.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:21 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:21.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:21 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:47:21.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:21 smithi151 ceph-mon[36066]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-14T16:47:21.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:21 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:21.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:21 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:21.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:21 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:47:23.657 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:23 smithi151 ceph-mon[36066]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail 2024-05-14T16:47:23.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:23 smithi123 ceph-mon[35971]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail 2024-05-14T16:47:23.671 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:23.671+0000 7fb1b27fc700 1 -- 172.21.15.123:0/3026200410 <== mgr.14150 v2:172.21.15.2:6800/1498936028 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb1bc0ffc10 con 0x7fb1a80b2900 2024-05-14T16:47:23.671 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 4 on host 'smithi123' 2024-05-14T16:47:23.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:23.673+0000 7fb1c347b700 1 -- 172.21.15.123:0/3026200410 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fb1a80b2900 msgr2=0x7fb1a80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:23.673 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:23.673+0000 7fb1c347b700 1 --2- 172.21.15.123:0/3026200410 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fb1a80b2900 0x7fb1a80b4dc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc103500 tx=0x7fb1b400b040 comp rx=0 tx=0).stop 2024-05-14T16:47:23.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:23.673+0000 7fb1c347b700 1 -- 172.21.15.123:0/3026200410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1bc1036a0 msgr2=0x7fb1bc199c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:23.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:23.673+0000 7fb1c347b700 1 --2- 172.21.15.123:0/3026200410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1bc1036a0 0x7fb1bc199c30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac00b530 tx=0x7fb1ac00b840 comp rx=0 tx=0).stop 2024-05-14T16:47:23.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:23.673+0000 7fb1c347b700 1 -- 172.21.15.123:0/3026200410 shutdown_connections 2024-05-14T16:47:23.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:23.673+0000 7fb1c347b700 1 --2- 172.21.15.123:0/3026200410 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1bc104040 0x7fb1bc1a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:23.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:23.673+0000 7fb1c347b700 1 --2- 172.21.15.123:0/3026200410 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fb1a80b2900 0x7fb1a80b4dc0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:23.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:23.673+0000 7fb1c347b700 1 --2- 172.21.15.123:0/3026200410 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1bc1024a0 0x7fb1bc1996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:23.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:23.673+0000 7fb1c347b700 1 --2- 172.21.15.123:0/3026200410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1bc1036a0 0x7fb1bc199c30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:23.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:23.673+0000 7fb1c347b700 1 -- 172.21.15.123:0/3026200410 >> 172.21.15.123:0/3026200410 conn(0x7fb1bc0fda40 msgr2=0x7fb1bc0ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:23.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:23.673+0000 7fb1c347b700 1 -- 172.21.15.123:0/3026200410 shutdown_connections 2024-05-14T16:47:23.674 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:47:23.673+0000 7fb1c347b700 1 -- 172.21.15.123:0/3026200410 wait complete. 2024-05-14T16:47:23.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:23 smithi002 ceph-mon[31538]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail 2024-05-14T16:47:24.368 DEBUG:teuthology.orchestra.run.smithi123:osd.4> sudo journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@osd.4.service 2024-05-14T16:47:24.370 INFO:tasks.cephadm:Deploying osd.5 on smithi151 with /dev/vg_nvme/lv_3... 2024-05-14T16:47:24.370 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-14T16:47:24.614 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T16:47:24.736 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:24 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:24.736 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:24 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:24.736 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:24 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:24.736 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:24 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:24.736 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:24 smithi151 ceph-mon[36066]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail 2024-05-14T16:47:24.736 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:24 smithi151 ceph-mon[36066]: from='osd.4 [v2:172.21.15.123:6808/2101935762,v1:172.21.15.123:6809/2101935762]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-14T16:47:24.736 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:24 smithi151 ceph-mon[36066]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-14T16:47:24.816 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:24 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:24.816 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:24 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:24.816 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:24 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:24.816 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:24 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:24.816 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:24 smithi123 ceph-mon[35971]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail 2024-05-14T16:47:24.816 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:24 smithi123 ceph-mon[35971]: from='osd.4 [v2:172.21.15.123:6808/2101935762,v1:172.21.15.123:6809/2101935762]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-14T16:47:24.816 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:24 smithi123 ceph-mon[35971]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-14T16:47:24.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:24 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:24.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:24 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:24.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:24 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:24.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:24 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:24.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:24 smithi002 ceph-mon[31538]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail 2024-05-14T16:47:24.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:24 smithi002 ceph-mon[31538]: from='osd.4 [v2:172.21.15.123:6808/2101935762,v1:172.21.15.123:6809/2101935762]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-14T16:47:24.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:24 smithi002 ceph-mon[31538]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-14T16:47:25.820 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:25 smithi123 ceph-mon[35971]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-14T16:47:25.820 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:25 smithi123 ceph-mon[35971]: osdmap e29: 5 total, 4 up, 5 in 2024-05-14T16:47:25.821 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:25 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:47:25.821 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:25 smithi123 ceph-mon[35971]: from='osd.4 [v2:172.21.15.123:6808/2101935762,v1:172.21.15.123:6809/2101935762]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-14T16:47:25.821 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:25 smithi123 ceph-mon[35971]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-14T16:47:25.976 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:25 smithi151 ceph-mon[36066]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-14T16:47:25.976 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:25 smithi151 ceph-mon[36066]: osdmap e29: 5 total, 4 up, 5 in 2024-05-14T16:47:25.977 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:25 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:47:25.977 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:25 smithi151 ceph-mon[36066]: from='osd.4 [v2:172.21.15.123:6808/2101935762,v1:172.21.15.123:6809/2101935762]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-14T16:47:25.977 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:25 smithi151 ceph-mon[36066]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-14T16:47:25.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:25 smithi002 ceph-mon[31538]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-14T16:47:25.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:25 smithi002 ceph-mon[31538]: osdmap e29: 5 total, 4 up, 5 in 2024-05-14T16:47:25.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:25 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:47:25.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:25 smithi002 ceph-mon[31538]: from='osd.4 [v2:172.21.15.123:6808/2101935762,v1:172.21.15.123:6809/2101935762]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-14T16:47:25.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:25 smithi002 ceph-mon[31538]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-14T16:47:26.536 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-14T16:47:26.536 INFO:teuthology.orchestra.run.smithi151.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-14T16:47:26.536 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10+0 records in 2024-05-14T16:47:26.536 INFO:teuthology.orchestra.run.smithi151.stderr:10+0 records out 2024-05-14T16:47:26.536 INFO:teuthology.orchestra.run.smithi151.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114528 s, 916 MB/s 2024-05-14T16:47:26.536 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping successful for: 2024-05-14T16:47:26.660 INFO:journalctl@ceph.osd.4.smithi123.stdout:May 14 16:47:26 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-4[47500]: 2024-05-14T16:47:26.177+0000 7f191b9a3700 -1 osd.4 0 waiting for initial osdmap 2024-05-14T16:47:26.661 INFO:journalctl@ceph.osd.4.smithi123.stdout:May 14 16:47:26 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-4[47500]: 2024-05-14T16:47:26.185+0000 7f191378e700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T16:47:26.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:26 smithi002 ceph-mon[31538]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-05-14T16:47:26.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:26 smithi002 ceph-mon[31538]: osdmap e30: 5 total, 4 up, 5 in 2024-05-14T16:47:26.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:26 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:47:26.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:26 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:47:26.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:26 smithi002 ceph-mon[31538]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail 2024-05-14T16:47:26.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:26 smithi002 ceph-mon[31538]: from='osd.4 ' entity='osd.4' 2024-05-14T16:47:27.072 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:26 smithi123 ceph-mon[35971]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-05-14T16:47:27.072 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:26 smithi123 ceph-mon[35971]: osdmap e30: 5 total, 4 up, 5 in 2024-05-14T16:47:27.072 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:26 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:47:27.073 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:26 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:47:27.073 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:26 smithi123 ceph-mon[35971]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail 2024-05-14T16:47:27.073 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:26 smithi123 ceph-mon[35971]: from='osd.4 ' entity='osd.4' 2024-05-14T16:47:27.103 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:26 smithi151 ceph-mon[36066]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-05-14T16:47:27.103 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:26 smithi151 ceph-mon[36066]: osdmap e30: 5 total, 4 up, 5 in 2024-05-14T16:47:27.103 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:26 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:47:27.103 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:26 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:47:27.103 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:26 smithi151 ceph-mon[36066]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 131 MiB used, 357 GiB / 358 GiB avail 2024-05-14T16:47:27.103 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:26 smithi151 ceph-mon[36066]: from='osd.4 ' entity='osd.4' 2024-05-14T16:47:27.130 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi151:vg_nvme/lv_3 2024-05-14T16:47:27.373 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T16:47:27.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:27 smithi002 ceph-mon[31538]: purged_snaps scrub starts 2024-05-14T16:47:27.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:27 smithi002 ceph-mon[31538]: purged_snaps scrub ok 2024-05-14T16:47:27.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:27 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:47:27.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:27 smithi002 ceph-mon[31538]: osd.4 [v2:172.21.15.123:6808/2101935762,v1:172.21.15.123:6809/2101935762] boot 2024-05-14T16:47:27.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:27 smithi002 ceph-mon[31538]: osdmap e31: 5 total, 5 up, 5 in 2024-05-14T16:47:27.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:27 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:47:27.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:27 smithi002 ceph-mon[31538]: Detected new or changed devices on smithi123 2024-05-14T16:47:27.995 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:27 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:27.995 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:27 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:27.995 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:27 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:47:27.995 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:27 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:47:27.995 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:27 smithi002 ceph-mon[31538]: Adjusting osd_memory_target on smithi123 to 8544M 2024-05-14T16:47:27.995 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:27 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:27.995 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:27 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:47:27.995 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:27 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:47:27.995 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:27 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:27.995 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:27 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:47:27.995 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:27 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:47:27.995 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:27 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:47:27.995 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:27 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:28.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:27 smithi151 ceph-mon[36066]: purged_snaps scrub starts 2024-05-14T16:47:28.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:27 smithi151 ceph-mon[36066]: purged_snaps scrub ok 2024-05-14T16:47:28.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:27 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:47:28.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:27 smithi151 ceph-mon[36066]: osd.4 [v2:172.21.15.123:6808/2101935762,v1:172.21.15.123:6809/2101935762] boot 2024-05-14T16:47:28.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:27 smithi151 ceph-mon[36066]: osdmap e31: 5 total, 5 up, 5 in 2024-05-14T16:47:28.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:27 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:47:28.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:27 smithi151 ceph-mon[36066]: Detected new or changed devices on smithi123 2024-05-14T16:47:28.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:27 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:28.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:27 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:28.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:27 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:47:28.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:27 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:47:28.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:27 smithi151 ceph-mon[36066]: Adjusting osd_memory_target on smithi123 to 8544M 2024-05-14T16:47:28.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:27 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:28.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:27 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:47:28.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:27 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:47:28.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:27 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:28.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:27 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:47:28.106 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:27 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:47:28.106 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:27 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:47:28.106 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:27 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:28.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:27 smithi123 ceph-mon[35971]: purged_snaps scrub starts 2024-05-14T16:47:28.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:27 smithi123 ceph-mon[35971]: purged_snaps scrub ok 2024-05-14T16:47:28.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:27 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:47:28.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:27 smithi123 ceph-mon[35971]: osd.4 [v2:172.21.15.123:6808/2101935762,v1:172.21.15.123:6809/2101935762] boot 2024-05-14T16:47:28.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:27 smithi123 ceph-mon[35971]: osdmap e31: 5 total, 5 up, 5 in 2024-05-14T16:47:28.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:27 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:47:28.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:27 smithi123 ceph-mon[35971]: Detected new or changed devices on smithi123 2024-05-14T16:47:28.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:27 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:28.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:27 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:28.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:27 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:47:28.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:27 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:47:28.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:27 smithi123 ceph-mon[35971]: Adjusting osd_memory_target on smithi123 to 8544M 2024-05-14T16:47:28.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:27 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:28.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:27 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:47:28.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:27 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:47:28.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:27 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:28.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:27 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:47:28.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:27 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:47:28.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:27 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:47:28.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:27 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:28.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:28.984+0000 7f7613b63700 1 -- 172.21.15.151:0/928872571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f760c1024a0 msgr2=0x7f760c1028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:28.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:28.984+0000 7f7613b63700 1 --2- 172.21.15.151:0/928872571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f760c1024a0 0x7f760c1028c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7608009990 tx=0x7f7608009ca0 comp rx=0 tx=0).stop 2024-05-14T16:47:28.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:28.984+0000 7f7613b63700 1 -- 172.21.15.151:0/928872571 shutdown_connections 2024-05-14T16:47:28.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:28.984+0000 7f7613b63700 1 --2- 172.21.15.151:0/928872571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f760c104040 0x7f760c10a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:28.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:28.984+0000 7f7613b63700 1 --2- 172.21.15.151:0/928872571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f760c1024a0 0x7f760c1028c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:28.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:28.984+0000 7f7613b63700 1 --2- 172.21.15.151:0/928872571 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f760c1036a0 0x7f760c103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:28.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:28.984+0000 7f7613b63700 1 -- 172.21.15.151:0/928872571 >> 172.21.15.151:0/928872571 conn(0x7f760c0fda40 msgr2=0x7f760c0ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:28.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:28.984+0000 7f7613b63700 1 -- 172.21.15.151:0/928872571 shutdown_connections 2024-05-14T16:47:28.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:28.984+0000 7f7613b63700 1 -- 172.21.15.151:0/928872571 wait complete. 2024-05-14T16:47:28.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:28.985+0000 7f7613b63700 1 Processor -- start 2024-05-14T16:47:28.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:28.985+0000 7f7613b63700 1 -- start start 2024-05-14T16:47:28.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:28.985+0000 7f7613b63700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f760c1024a0 0x7f760c06bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:28.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:28.985+0000 7f7613b63700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f760c1036a0 0x7f760c19e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:28.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:28.985+0000 7f7613b63700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f760c104040 0x7f760c1a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:28.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:28.985+0000 7f7613b63700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f760c10df40 con 0x7f760c104040 2024-05-14T16:47:28.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:28.985+0000 7f7613b63700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f760c10ddc0 con 0x7f760c1036a0 2024-05-14T16:47:28.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:28.985+0000 7f7613b63700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f760c10e0c0 con 0x7f760c1024a0 2024-05-14T16:47:28.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:28.985+0000 7f76118ff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f760c1024a0 0x7f760c06bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:28.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:28.985+0000 7f76118ff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f760c1024a0 0x7f760c06bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:40980/0 (socket says 172.21.15.151:40980) 2024-05-14T16:47:28.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:28.985+0000 7f76118ff700 1 -- 172.21.15.151:0/1621246960 learned_addr learned my addr 172.21.15.151:0/1621246960 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-14T16:47:28.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:28.985+0000 7f76110fe700 1 --2- 172.21.15.151:0/1621246960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f760c1036a0 0x7f760c19e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:28.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:28.985+0000 7f7612100700 1 --2- 172.21.15.151:0/1621246960 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f760c104040 0x7f760c1a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:28.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:28.985+0000 7f76118ff700 1 -- 172.21.15.151:0/1621246960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f760c1036a0 msgr2=0x7f760c19e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:28.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:28.985+0000 7f76118ff700 1 --2- 172.21.15.151:0/1621246960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f760c1036a0 0x7f760c19e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:28.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:28.985+0000 7f76118ff700 1 -- 172.21.15.151:0/1621246960 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f760c104040 msgr2=0x7f760c1a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:28.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:28.985+0000 7f76118ff700 1 --2- 172.21.15.151:0/1621246960 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f760c104040 0x7f760c1a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:28.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:28.986+0000 7f76118ff700 1 -- 172.21.15.151:0/1621246960 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f760c1a13d0 con 0x7f760c1024a0 2024-05-14T16:47:28.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:28.986+0000 7f76118ff700 1 --2- 172.21.15.151:0/1621246960 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f760c1024a0 0x7f760c06bda0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f760800b2e0 tx=0x7f76080047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:28.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:28.986+0000 7f7602ffd700 1 -- 172.21.15.151:0/1621246960 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7608027070 con 0x7f760c1024a0 2024-05-14T16:47:28.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:28.986+0000 7f7602ffd700 1 -- 172.21.15.151:0/1621246960 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7608022470 con 0x7f760c1024a0 2024-05-14T16:47:28.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:28.986+0000 7f7613b63700 1 -- 172.21.15.151:0/1621246960 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f760c1a1660 con 0x7f760c1024a0 2024-05-14T16:47:28.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:28.986+0000 7f7602ffd700 1 -- 172.21.15.151:0/1621246960 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7608017650 con 0x7f760c1024a0 2024-05-14T16:47:28.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:28.986+0000 7f7613b63700 1 -- 172.21.15.151:0/1621246960 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f760c1a1b10 con 0x7f760c1024a0 2024-05-14T16:47:28.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:28.987+0000 7f7613b63700 1 -- 172.21.15.151:0/1621246960 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f760c06a230 con 0x7f760c1024a0 2024-05-14T16:47:28.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:28.988+0000 7f7602ffd700 1 -- 172.21.15.151:0/1621246960 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f760802a070 con 0x7f760c1024a0 2024-05-14T16:47:28.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:28.988+0000 7f7602ffd700 1 --2- 172.21.15.151:0/1621246960 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f75f80b2900 0x7f75f80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:28.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:28.989+0000 7f7602ffd700 1 -- 172.21.15.151:0/1621246960 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4101+0+0 (secure 0 0 0) 0x7f76080e1f20 con 0x7f760c1024a0 2024-05-14T16:47:28.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:28.989+0000 7f76110fe700 1 --2- 172.21.15.151:0/1621246960 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f75f80b2900 0x7f75f80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:28.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:28.989+0000 7f76110fe700 1 --2- 172.21.15.151:0/1621246960 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f75f80b2900 0x7f75f80b4dc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f760c06c300 tx=0x7f75fc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:28.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:28.993+0000 7f7602ffd700 1 -- 172.21.15.151:0/1621246960 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7608013070 con 0x7f760c1024a0 2024-05-14T16:47:29.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:29.204+0000 7f7613b63700 1 -- 172.21.15.151:0/1621246960 --> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f760c063980 con 0x7f75f80b2900 2024-05-14T16:47:29.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:29 smithi002 ceph-mon[31538]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:47:29.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:29 smithi002 ceph-mon[31538]: osdmap e32: 5 total, 5 up, 5 in 2024-05-14T16:47:29.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:29 smithi151 ceph-mon[36066]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:47:29.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:29 smithi151 ceph-mon[36066]: osdmap e32: 5 total, 5 up, 5 in 2024-05-14T16:47:29.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:29 smithi123 ceph-mon[35971]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:47:29.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:29 smithi123 ceph-mon[35971]: osdmap e32: 5 total, 5 up, 5 in 2024-05-14T16:47:30.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:30 smithi002 ceph-mon[31538]: from='client.24214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:47:30.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:30 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:47:30.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:30 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:47:30.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:30 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:47:30.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:30 smithi151 ceph-mon[36066]: from='client.24214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:47:30.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:30 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:47:30.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:30 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:47:30.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:30 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:47:30.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:30 smithi123 ceph-mon[35971]: from='client.24214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:47:30.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:30 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:47:30.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:30 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:47:30.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:30 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:47:31.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:31 smithi151 ceph-mon[36066]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:47:31.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:31 smithi002 ceph-mon[31538]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:47:31.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:31 smithi123 ceph-mon[35971]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:47:32.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:32 smithi002 ceph-mon[31538]: from='client.? 172.21.15.151:0/3965799249' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9c166916-af78-4cc2-bd14-d1220b6e7d9a"}]: dispatch 2024-05-14T16:47:32.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:32 smithi002 ceph-mon[31538]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9c166916-af78-4cc2-bd14-d1220b6e7d9a"}]: dispatch 2024-05-14T16:47:32.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:32 smithi002 ceph-mon[31538]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9c166916-af78-4cc2-bd14-d1220b6e7d9a"}]': finished 2024-05-14T16:47:32.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:32 smithi002 ceph-mon[31538]: osdmap e33: 6 total, 5 up, 6 in 2024-05-14T16:47:32.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:32 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:47:32.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:32 smithi151 ceph-mon[36066]: from='client.? 172.21.15.151:0/3965799249' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9c166916-af78-4cc2-bd14-d1220b6e7d9a"}]: dispatch 2024-05-14T16:47:32.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:32 smithi151 ceph-mon[36066]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9c166916-af78-4cc2-bd14-d1220b6e7d9a"}]: dispatch 2024-05-14T16:47:32.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:32 smithi151 ceph-mon[36066]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9c166916-af78-4cc2-bd14-d1220b6e7d9a"}]': finished 2024-05-14T16:47:32.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:32 smithi151 ceph-mon[36066]: osdmap e33: 6 total, 5 up, 6 in 2024-05-14T16:47:32.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:32 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:47:32.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:32 smithi123 ceph-mon[35971]: from='client.? 172.21.15.151:0/3965799249' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9c166916-af78-4cc2-bd14-d1220b6e7d9a"}]: dispatch 2024-05-14T16:47:32.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:32 smithi123 ceph-mon[35971]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9c166916-af78-4cc2-bd14-d1220b6e7d9a"}]: dispatch 2024-05-14T16:47:32.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:32 smithi123 ceph-mon[35971]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9c166916-af78-4cc2-bd14-d1220b6e7d9a"}]': finished 2024-05-14T16:47:32.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:32 smithi123 ceph-mon[35971]: osdmap e33: 6 total, 5 up, 6 in 2024-05-14T16:47:32.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:32 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:47:33.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:33 smithi002 ceph-mon[31538]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:47:33.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:33 smithi002 ceph-mon[31538]: from='client.? 172.21.15.151:0/3803301462' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:47:33.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:33 smithi151 ceph-mon[36066]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:47:33.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:33 smithi151 ceph-mon[36066]: from='client.? 172.21.15.151:0/3803301462' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:47:33.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:33 smithi123 ceph-mon[35971]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:47:33.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:33 smithi123 ceph-mon[35971]: from='client.? 172.21.15.151:0/3803301462' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:47:35.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:35 smithi123 ceph-mon[35971]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:47:35.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:35 smithi002 ceph-mon[31538]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:47:35.600 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:35 smithi151 ceph-mon[36066]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:47:37.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:37 smithi002 ceph-mon[31538]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:47:37.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:37 smithi151 ceph-mon[36066]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:47:37.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:37 smithi123 ceph-mon[35971]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:47:39.284 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:39 smithi151 ceph-mon[36066]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:47:39.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:39 smithi002 ceph-mon[31538]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:47:39.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:39 smithi123 ceph-mon[35971]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:47:40.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:40 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-14T16:47:40.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:40 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:47:40.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:40 smithi002 ceph-mon[31538]: Deploying daemon osd.5 on smithi151 2024-05-14T16:47:40.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:40 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-14T16:47:40.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:40 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:47:40.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:40 smithi151 ceph-mon[36066]: Deploying daemon osd.5 on smithi151 2024-05-14T16:47:40.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:40 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-14T16:47:40.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:40 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:47:40.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:40 smithi123 ceph-mon[35971]: Deploying daemon osd.5 on smithi151 2024-05-14T16:47:41.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:41 smithi002 ceph-mon[31538]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:47:41.575 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:41 smithi151 ceph-mon[36066]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:47:41.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:41 smithi123 ceph-mon[35971]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:47:43.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:43 smithi151 ceph-mon[36066]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:47:43.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:43 smithi002 ceph-mon[31538]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:47:43.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:43 smithi123 ceph-mon[35971]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:47:45.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:45 smithi002 ceph-mon[31538]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:47:45.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:45 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:45.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:45 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:45.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:45 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:47:45.544 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:45 smithi151 ceph-mon[36066]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:47:45.544 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:45 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:45.544 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:45 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:45.545 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:45 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:47:45.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:45 smithi123 ceph-mon[35971]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:47:45.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:45 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:45.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:45 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:45.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:45 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:47:47.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:47 smithi002 ceph-mon[31538]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:47:47.521 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:47 smithi151 ceph-mon[36066]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:47:47.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:47 smithi123 ceph-mon[35971]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:47:48.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:48.676+0000 7f7602ffd700 1 -- 172.21.15.151:0/1621246960 <== mgr.14150 v2:172.21.15.2:6800/1498936028 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f760c063980 con 0x7f75f80b2900 2024-05-14T16:47:48.685 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 5 on host 'smithi151' 2024-05-14T16:47:48.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:48.678+0000 7f7613b63700 1 -- 172.21.15.151:0/1621246960 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f75f80b2900 msgr2=0x7f75f80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:48.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:48.678+0000 7f7613b63700 1 --2- 172.21.15.151:0/1621246960 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f75f80b2900 0x7f75f80b4dc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f760c06c300 tx=0x7f75fc009260 comp rx=0 tx=0).stop 2024-05-14T16:47:48.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:48.678+0000 7f7613b63700 1 -- 172.21.15.151:0/1621246960 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f760c1024a0 msgr2=0x7f760c06bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:48.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:48.678+0000 7f7613b63700 1 --2- 172.21.15.151:0/1621246960 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f760c1024a0 0x7f760c06bda0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f760800b2e0 tx=0x7f76080047b0 comp rx=0 tx=0).stop 2024-05-14T16:47:48.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:48.678+0000 7f7613b63700 1 -- 172.21.15.151:0/1621246960 shutdown_connections 2024-05-14T16:47:48.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:48.678+0000 7f7613b63700 1 --2- 172.21.15.151:0/1621246960 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f75f80b2900 0x7f75f80b4dc0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:48.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:48.678+0000 7f7613b63700 1 --2- 172.21.15.151:0/1621246960 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f760c104040 0x7f760c1a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:48.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:48.678+0000 7f7613b63700 1 --2- 172.21.15.151:0/1621246960 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f760c1024a0 0x7f760c06bda0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:48.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:48.678+0000 7f7613b63700 1 --2- 172.21.15.151:0/1621246960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f760c1036a0 0x7f760c19e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:48.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:48.678+0000 7f7613b63700 1 -- 172.21.15.151:0/1621246960 >> 172.21.15.151:0/1621246960 conn(0x7f760c0fda40 msgr2=0x7f760c106ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:48.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:48.678+0000 7f7613b63700 1 -- 172.21.15.151:0/1621246960 shutdown_connections 2024-05-14T16:47:48.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:47:48.678+0000 7f7613b63700 1 -- 172.21.15.151:0/1621246960 wait complete. 2024-05-14T16:47:48.989 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:48 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:48.989 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:48 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:48.989 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:48 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:47:48.989 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:48 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:47:48.989 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:48 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:48.989 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:48 smithi151 ceph-mon[36066]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:47:48.989 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:48 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:48.989 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:48 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:47:48.989 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:48 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:48.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:48 smithi151 ceph-mon[36066]: from='osd.5 [v2:172.21.15.151:6808/3695961886,v1:172.21.15.151:6809/3695961886]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-14T16:47:48.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:48 smithi151 ceph-mon[36066]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-14T16:47:49.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:48 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:49.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:48 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:49.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:48 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:47:49.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:48 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:47:49.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:48 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:49.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:48 smithi123 ceph-mon[35971]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:47:49.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:48 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:49.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:48 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:47:49.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:48 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:49.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:48 smithi123 ceph-mon[35971]: from='osd.5 [v2:172.21.15.151:6808/3695961886,v1:172.21.15.151:6809/3695961886]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-14T16:47:49.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:48 smithi123 ceph-mon[35971]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-14T16:47:49.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:48 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:49.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:48 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:49.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:48 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:47:49.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:48 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:47:49.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:48 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:49.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:48 smithi002 ceph-mon[31538]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:47:49.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:48 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:49.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:48 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:47:49.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:48 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:49.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:48 smithi002 ceph-mon[31538]: from='osd.5 [v2:172.21.15.151:6808/3695961886,v1:172.21.15.151:6809/3695961886]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-14T16:47:49.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:48 smithi002 ceph-mon[31538]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-14T16:47:49.284 DEBUG:teuthology.orchestra.run.smithi151:osd.5> sudo journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@osd.5.service 2024-05-14T16:47:49.287 INFO:tasks.cephadm:Deploying osd.6 on smithi002 with /dev/vg_nvme/lv_2... 2024-05-14T16:47:49.287 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-14T16:47:49.549 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:47:50.917 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:50 smithi151 ceph-mon[36066]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-14T16:47:50.917 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:50 smithi151 ceph-mon[36066]: osdmap e34: 6 total, 5 up, 6 in 2024-05-14T16:47:50.917 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:50 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:47:50.917 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:50 smithi151 ceph-mon[36066]: from='osd.5 [v2:172.21.15.151:6808/3695961886,v1:172.21.15.151:6809/3695961886]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-14T16:47:50.917 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:50 smithi151 ceph-mon[36066]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-14T16:47:50.917 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:50 smithi151 ceph-mon[36066]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:47:50.973 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:50 smithi002 ceph-mon[31538]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-14T16:47:50.973 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:50 smithi002 ceph-mon[31538]: osdmap e34: 6 total, 5 up, 6 in 2024-05-14T16:47:50.973 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:50 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:47:50.973 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:50 smithi002 ceph-mon[31538]: from='osd.5 [v2:172.21.15.151:6808/3695961886,v1:172.21.15.151:6809/3695961886]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-14T16:47:50.973 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:50 smithi002 ceph-mon[31538]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-14T16:47:50.973 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:50 smithi002 ceph-mon[31538]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:47:51.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:50 smithi123 ceph-mon[35971]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-14T16:47:51.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:50 smithi123 ceph-mon[35971]: osdmap e34: 6 total, 5 up, 6 in 2024-05-14T16:47:51.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:50 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:47:51.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:50 smithi123 ceph-mon[35971]: from='osd.5 [v2:172.21.15.151:6808/3695961886,v1:172.21.15.151:6809/3695961886]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-14T16:47:51.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:50 smithi123 ceph-mon[35971]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-14T16:47:51.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:50 smithi123 ceph-mon[35971]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:47:51.355 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 16:47:51 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5[47942]: 2024-05-14T16:47:51.097+0000 7f6089f1e700 -1 osd.5 0 waiting for initial osdmap 2024-05-14T16:47:51.355 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 16:47:51 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5[47942]: 2024-05-14T16:47:51.106+0000 7f6081d09700 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T16:47:51.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:51 smithi151 ceph-mon[36066]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-05-14T16:47:51.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:51 smithi151 ceph-mon[36066]: osdmap e35: 6 total, 5 up, 6 in 2024-05-14T16:47:51.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:51 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:47:51.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:51 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:47:51.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:51 smithi151 ceph-mon[36066]: from='osd.5 ' entity='osd.5' 2024-05-14T16:47:51.864 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-14T16:47:51.864 INFO:teuthology.orchestra.run.smithi002.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-14T16:47:51.864 INFO:teuthology.orchestra.run.smithi002.stderr: stderr: 10+0 records in 2024-05-14T16:47:51.864 INFO:teuthology.orchestra.run.smithi002.stderr:10+0 records out 2024-05-14T16:47:51.864 INFO:teuthology.orchestra.run.smithi002.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00827767 s, 1.3 GB/s 2024-05-14T16:47:51.864 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping successful for: 2024-05-14T16:47:51.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:51 smithi002 ceph-mon[31538]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-05-14T16:47:51.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:51 smithi002 ceph-mon[31538]: osdmap e35: 6 total, 5 up, 6 in 2024-05-14T16:47:51.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:51 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:47:51.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:51 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:47:51.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:51 smithi002 ceph-mon[31538]: from='osd.5 ' entity='osd.5' 2024-05-14T16:47:52.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:51 smithi123 ceph-mon[35971]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-05-14T16:47:52.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:51 smithi123 ceph-mon[35971]: osdmap e35: 6 total, 5 up, 6 in 2024-05-14T16:47:52.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:51 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:47:52.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:51 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:47:52.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:51 smithi123 ceph-mon[35971]: from='osd.5 ' entity='osd.5' 2024-05-14T16:47:52.530 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi002:vg_nvme/lv_2 2024-05-14T16:47:52.794 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:47:52.837 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:52 smithi002 ceph-mon[31538]: purged_snaps scrub starts 2024-05-14T16:47:52.837 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:52 smithi002 ceph-mon[31538]: purged_snaps scrub ok 2024-05-14T16:47:52.837 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:52 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:47:52.837 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:52 smithi002 ceph-mon[31538]: osd.5 [v2:172.21.15.151:6808/3695961886,v1:172.21.15.151:6809/3695961886] boot 2024-05-14T16:47:52.837 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:52 smithi002 ceph-mon[31538]: osdmap e36: 6 total, 6 up, 6 in 2024-05-14T16:47:52.837 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:52 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:47:52.837 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:52 smithi002 ceph-mon[31538]: osdmap e37: 6 total, 6 up, 6 in 2024-05-14T16:47:52.838 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:52 smithi002 ceph-mon[31538]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:47:52.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:52 smithi151 ceph-mon[36066]: purged_snaps scrub starts 2024-05-14T16:47:52.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:52 smithi151 ceph-mon[36066]: purged_snaps scrub ok 2024-05-14T16:47:52.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:52 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:47:52.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:52 smithi151 ceph-mon[36066]: osd.5 [v2:172.21.15.151:6808/3695961886,v1:172.21.15.151:6809/3695961886] boot 2024-05-14T16:47:52.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:52 smithi151 ceph-mon[36066]: osdmap e36: 6 total, 6 up, 6 in 2024-05-14T16:47:52.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:52 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:47:52.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:52 smithi151 ceph-mon[36066]: osdmap e37: 6 total, 6 up, 6 in 2024-05-14T16:47:52.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:52 smithi151 ceph-mon[36066]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:47:53.061 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:52 smithi123 ceph-mon[35971]: purged_snaps scrub starts 2024-05-14T16:47:53.062 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:52 smithi123 ceph-mon[35971]: purged_snaps scrub ok 2024-05-14T16:47:53.062 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:52 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:47:53.062 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:52 smithi123 ceph-mon[35971]: osd.5 [v2:172.21.15.151:6808/3695961886,v1:172.21.15.151:6809/3695961886] boot 2024-05-14T16:47:53.062 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:52 smithi123 ceph-mon[35971]: osdmap e36: 6 total, 6 up, 6 in 2024-05-14T16:47:53.062 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:52 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:47:53.062 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:52 smithi123 ceph-mon[35971]: osdmap e37: 6 total, 6 up, 6 in 2024-05-14T16:47:53.062 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:52 smithi123 ceph-mon[35971]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-05-14T16:47:53.954 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:53 smithi002 ceph-mon[31538]: Detected new or changed devices on smithi151 2024-05-14T16:47:53.954 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:53 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:53.954 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:53 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:53.954 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:53 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:47:53.954 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:53 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:47:53.954 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:53 smithi002 ceph-mon[31538]: Adjusting osd_memory_target on smithi151 to 8544M 2024-05-14T16:47:53.954 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:53 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:53.954 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:53 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:47:53.954 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:53 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:47:53.955 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:53 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:53.955 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:53 smithi002 ceph-mon[31538]: osdmap e38: 6 total, 6 up, 6 in 2024-05-14T16:47:54.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:53 smithi151 ceph-mon[36066]: Detected new or changed devices on smithi151 2024-05-14T16:47:54.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:53 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:54.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:53 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:54.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:53 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:47:54.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:53 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:47:54.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:53 smithi151 ceph-mon[36066]: Adjusting osd_memory_target on smithi151 to 8544M 2024-05-14T16:47:54.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:53 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:54.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:53 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:47:54.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:53 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:47:54.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:53 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:54.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:53 smithi151 ceph-mon[36066]: osdmap e38: 6 total, 6 up, 6 in 2024-05-14T16:47:54.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:53 smithi123 ceph-mon[35971]: Detected new or changed devices on smithi151 2024-05-14T16:47:54.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:53 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:54.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:53 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:54.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:53 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:47:54.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:53 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:47:54.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:53 smithi123 ceph-mon[35971]: Adjusting osd_memory_target on smithi151 to 8544M 2024-05-14T16:47:54.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:53 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:54.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:53 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:47:54.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:53 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:47:54.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:53 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:47:54.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:53 smithi123 ceph-mon[35971]: osdmap e38: 6 total, 6 up, 6 in 2024-05-14T16:47:54.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.739+0000 7f669a24a700 1 -- 172.21.15.2:0/2278081713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f66941036a0 msgr2=0x7f6694103b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:54.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.739+0000 7f669a24a700 1 --2- 172.21.15.2:0/2278081713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f66941036a0 0x7f6694103b00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f66880099c0 tx=0x7f6688009cd0 comp rx=0 tx=0).stop 2024-05-14T16:47:54.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.739+0000 7f669a24a700 1 -- 172.21.15.2:0/2278081713 shutdown_connections 2024-05-14T16:47:54.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.739+0000 7f669a24a700 1 --2- 172.21.15.2:0/2278081713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6694104040 0x7f669410a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:54.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.739+0000 7f669a24a700 1 --2- 172.21.15.2:0/2278081713 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f66941024a0 0x7f66941028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:54.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.739+0000 7f669a24a700 1 --2- 172.21.15.2:0/2278081713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f66941036a0 0x7f6694103b00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:54.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.739+0000 7f669a24a700 1 -- 172.21.15.2:0/2278081713 >> 172.21.15.2:0/2278081713 conn(0x7f66940fda40 msgr2=0x7f66940ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:47:54.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.739+0000 7f669a24a700 1 -- 172.21.15.2:0/2278081713 shutdown_connections 2024-05-14T16:47:54.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.739+0000 7f669a24a700 1 -- 172.21.15.2:0/2278081713 wait complete. 2024-05-14T16:47:54.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.740+0000 7f669a24a700 1 Processor -- start 2024-05-14T16:47:54.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.740+0000 7f669a24a700 1 -- start start 2024-05-14T16:47:54.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.740+0000 7f669a24a700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f66941024a0 0x7f66941997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:54.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.740+0000 7f669a24a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66941036a0 0x7f6694199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:54.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.740+0000 7f669a24a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6694104040 0x7f66941a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:54.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.740+0000 7f669a24a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f669410e030 con 0x7f6694104040 2024-05-14T16:47:54.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.740+0000 7f669a24a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f669410deb0 con 0x7f66941036a0 2024-05-14T16:47:54.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.740+0000 7f669a24a700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f669410e1b0 con 0x7f66941024a0 2024-05-14T16:47:54.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.740+0000 7f6693fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6694104040 0x7f66941a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:54.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.740+0000 7f6692ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66941036a0 0x7f6694199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:54.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.740+0000 7f6693fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6694104040 0x7f66941a0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47498/0 (socket says 172.21.15.2:47498) 2024-05-14T16:47:54.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.740+0000 7f6693fff700 1 -- 172.21.15.2:0/2654054504 learned_addr learned my addr 172.21.15.2:0/2654054504 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:47:54.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.740+0000 7f66937fe700 1 --2- 172.21.15.2:0/2654054504 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f66941024a0 0x7f66941997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:54.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.740+0000 7f6693fff700 1 -- 172.21.15.2:0/2654054504 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f66941024a0 msgr2=0x7f66941997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:54.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.740+0000 7f6693fff700 1 --2- 172.21.15.2:0/2654054504 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f66941024a0 0x7f66941997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:54.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.740+0000 7f6693fff700 1 -- 172.21.15.2:0/2654054504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66941036a0 msgr2=0x7f6694199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:47:54.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.740+0000 7f6693fff700 1 --2- 172.21.15.2:0/2654054504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66941036a0 0x7f6694199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:47:54.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.740+0000 7f6693fff700 1 -- 172.21.15.2:0/2654054504 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66941a1450 con 0x7f6694104040 2024-05-14T16:47:54.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.741+0000 7f6693fff700 1 --2- 172.21.15.2:0/2654054504 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6694104040 0x7f66941a0db0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f668c00b9a0 tx=0x7f668c00bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:54.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.741+0000 7f6690ff9700 1 -- 172.21.15.2:0/2654054504 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668c00d850 con 0x7f6694104040 2024-05-14T16:47:54.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.741+0000 7f6690ff9700 1 -- 172.21.15.2:0/2654054504 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f668c00e420 con 0x7f6694104040 2024-05-14T16:47:54.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.741+0000 7f669a24a700 1 -- 172.21.15.2:0/2654054504 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66941a1740 con 0x7f6694104040 2024-05-14T16:47:54.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.741+0000 7f6690ff9700 1 -- 172.21.15.2:0/2654054504 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668c017640 con 0x7f6694104040 2024-05-14T16:47:54.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.741+0000 7f669a24a700 1 -- 172.21.15.2:0/2654054504 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66941a1d00 con 0x7f6694104040 2024-05-14T16:47:54.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.742+0000 7f669a24a700 1 -- 172.21.15.2:0/2654054504 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f669406a230 con 0x7f6694104040 2024-05-14T16:47:54.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.743+0000 7f6690ff9700 1 -- 172.21.15.2:0/2654054504 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f668c00d9f0 con 0x7f6694104040 2024-05-14T16:47:54.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.744+0000 7f6690ff9700 1 --2- 172.21.15.2:0/2654054504 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f667c0b2900 0x7f667c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:47:54.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.744+0000 7f66937fe700 1 --2- 172.21.15.2:0/2654054504 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f667c0b2900 0x7f667c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:47:54.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.744+0000 7f6690ff9700 1 -- 172.21.15.2:0/2654054504 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4533+0+0 (secure 0 0 0) 0x7f668c0e0fc0 con 0x7f6694104040 2024-05-14T16:47:54.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.745+0000 7f66937fe700 1 --2- 172.21.15.2:0/2654054504 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f667c0b2900 0x7f667c0b4dc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6694103500 tx=0x7f6684009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:47:54.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.748+0000 7f6690ff9700 1 -- 172.21.15.2:0/2654054504 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f668c013030 con 0x7f6694104040 2024-05-14T16:47:54.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:47:54.972+0000 7f669a24a700 1 -- 172.21.15.2:0/2654054504 --> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f669406b930 con 0x7f667c0b2900 2024-05-14T16:47:54.973 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:54 smithi002 ceph-mon[31538]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-14T16:47:55.032 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:54 smithi151 ceph-mon[36066]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-14T16:47:55.159 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:54 smithi123 ceph-mon[35971]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-14T16:47:55.939 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:55 smithi002 ceph-mon[31538]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:47:55.939 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:55 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:47:55.939 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:55 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:47:55.939 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:55 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:47:55.954 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:55 smithi151 ceph-mon[36066]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:47:55.954 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:55 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:47:55.954 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:55 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:47:55.955 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:55 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:47:56.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:55 smithi123 ceph-mon[35971]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:47:56.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:55 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:47:56.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:55 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:47:56.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:55 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:47:56.986 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:56 smithi002 ceph-mon[31538]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-05-14T16:47:57.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:56 smithi151 ceph-mon[36066]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-05-14T16:47:57.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:56 smithi123 ceph-mon[35971]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-05-14T16:47:58.351 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:58 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/2197781708' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60e9bf66-2338-41e8-9280-398d4e3b322f"}]: dispatch 2024-05-14T16:47:58.352 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:58 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/2197781708' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "60e9bf66-2338-41e8-9280-398d4e3b322f"}]': finished 2024-05-14T16:47:58.352 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:58 smithi002 ceph-mon[31538]: osdmap e39: 7 total, 6 up, 7 in 2024-05-14T16:47:58.352 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:58 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:47:58.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:58 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/2197781708' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60e9bf66-2338-41e8-9280-398d4e3b322f"}]: dispatch 2024-05-14T16:47:58.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:58 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/2197781708' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "60e9bf66-2338-41e8-9280-398d4e3b322f"}]': finished 2024-05-14T16:47:58.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:58 smithi151 ceph-mon[36066]: osdmap e39: 7 total, 6 up, 7 in 2024-05-14T16:47:58.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:58 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:47:58.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:58 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/2197781708' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60e9bf66-2338-41e8-9280-398d4e3b322f"}]: dispatch 2024-05-14T16:47:58.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:58 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/2197781708' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "60e9bf66-2338-41e8-9280-398d4e3b322f"}]': finished 2024-05-14T16:47:58.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:58 smithi123 ceph-mon[35971]: osdmap e39: 7 total, 6 up, 7 in 2024-05-14T16:47:58.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:58 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:47:59.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:59 smithi151 ceph-mon[36066]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-05-14T16:47:59.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:47:59 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/2941273420' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:47:59.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:59 smithi123 ceph-mon[35971]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-05-14T16:47:59.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:47:59 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/2941273420' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:47:59.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:59 smithi002 ceph-mon[31538]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-05-14T16:47:59.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:47:59 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/2941273420' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:48:01.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:01 smithi151 ceph-mon[36066]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T16:48:01.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:01 smithi123 ceph-mon[35971]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T16:48:01.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:01 smithi002 ceph-mon[31538]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T16:48:03.316 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:03 smithi002 ceph-mon[31538]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-14T16:48:03.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:03 smithi151 ceph-mon[36066]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-14T16:48:03.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:03 smithi123 ceph-mon[35971]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-14T16:48:05.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:05 smithi151 ceph-mon[36066]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:48:05.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:05 smithi123 ceph-mon[35971]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:48:05.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:05 smithi002 ceph-mon[31538]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:48:06.243 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:06 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-14T16:48:06.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:06 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:48:06.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:06 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-14T16:48:06.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:06 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:48:06.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-14T16:48:06.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:48:07.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:07 smithi002 ceph-mon[31538]: Deploying daemon osd.6 on smithi002 2024-05-14T16:48:07.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:07 smithi002 ceph-mon[31538]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:48:07.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:07 smithi151 ceph-mon[36066]: Deploying daemon osd.6 on smithi002 2024-05-14T16:48:07.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:07 smithi151 ceph-mon[36066]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:48:07.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:07 smithi123 ceph-mon[35971]: Deploying daemon osd.6 on smithi002 2024-05-14T16:48:07.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:07 smithi123 ceph-mon[35971]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:48:09.174 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:09 smithi002 ceph-mon[31538]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:48:09.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:09 smithi151 ceph-mon[36066]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:48:09.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:09 smithi123 ceph-mon[35971]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:48:11.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:11 smithi151 ceph-mon[36066]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:48:11.408 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:11 smithi002 ceph-mon[31538]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:48:11.409 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:11 smithi123 ceph-mon[35971]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:48:13.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:13 smithi151 ceph-mon[36066]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:48:13.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:13 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:13.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:13 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:13.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:13 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:48:13.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:13 smithi123 ceph-mon[35971]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:48:13.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:13 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:13.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:13 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:13.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:13 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:48:13.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:13 smithi002 ceph-mon[31538]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:48:13.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:13 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:13.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:13 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:13.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:13 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:48:15.243 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:15 smithi002 ceph-mon[31538]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:48:15.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:15 smithi151 ceph-mon[36066]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:48:15.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:15 smithi123 ceph-mon[35971]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:48:16.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:48:16.184+0000 7f6690ff9700 1 -- 172.21.15.2:0/2654054504 <== mgr.14150 v2:172.21.15.2:6800/1498936028 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f669406b930 con 0x7f667c0b2900 2024-05-14T16:48:16.184 INFO:teuthology.orchestra.run.smithi002.stdout:Created osd(s) 6 on host 'smithi002' 2024-05-14T16:48:16.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:48:16.187+0000 7f669a24a700 1 -- 172.21.15.2:0/2654054504 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f667c0b2900 msgr2=0x7f667c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:16.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:48:16.187+0000 7f669a24a700 1 --2- 172.21.15.2:0/2654054504 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f667c0b2900 0x7f667c0b4dc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6694103500 tx=0x7f6684009190 comp rx=0 tx=0).stop 2024-05-14T16:48:16.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:48:16.187+0000 7f669a24a700 1 -- 172.21.15.2:0/2654054504 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6694104040 msgr2=0x7f66941a0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:16.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:48:16.187+0000 7f669a24a700 1 --2- 172.21.15.2:0/2654054504 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6694104040 0x7f66941a0db0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f668c00b9a0 tx=0x7f668c00bcb0 comp rx=0 tx=0).stop 2024-05-14T16:48:16.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:48:16.187+0000 7f669a24a700 1 -- 172.21.15.2:0/2654054504 shutdown_connections 2024-05-14T16:48:16.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:48:16.187+0000 7f669a24a700 1 --2- 172.21.15.2:0/2654054504 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f667c0b2900 0x7f667c0b4dc0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:16.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:48:16.187+0000 7f669a24a700 1 --2- 172.21.15.2:0/2654054504 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6694104040 0x7f66941a0db0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:16.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:48:16.187+0000 7f669a24a700 1 --2- 172.21.15.2:0/2654054504 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f66941024a0 0x7f66941997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:16.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:48:16.187+0000 7f669a24a700 1 --2- 172.21.15.2:0/2654054504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66941036a0 0x7f6694199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:16.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:48:16.187+0000 7f669a24a700 1 -- 172.21.15.2:0/2654054504 >> 172.21.15.2:0/2654054504 conn(0x7f66940fda40 msgr2=0x7f669406b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:16.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:48:16.187+0000 7f669a24a700 1 -- 172.21.15.2:0/2654054504 shutdown_connections 2024-05-14T16:48:16.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:48:16.187+0000 7f669a24a700 1 -- 172.21.15.2:0/2654054504 wait complete. 2024-05-14T16:48:16.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:16 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:16.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:16 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:16.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:16 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:48:16.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:16 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:48:16.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:16 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:16.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:16 smithi002 ceph-mon[31538]: from='osd.6 [v2:172.21.15.2:6818/2711893600,v1:172.21.15.2:6819/2711893600]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-14T16:48:16.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:16 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:16.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:16 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:16.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:16 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:48:16.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:16 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:16.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:16 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:16.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:16 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:48:16.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:16 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:48:16.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:16 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:16.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:16 smithi151 ceph-mon[36066]: from='osd.6 [v2:172.21.15.2:6818/2711893600,v1:172.21.15.2:6819/2711893600]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-14T16:48:16.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:16 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:16.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:16 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:16.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:16 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:48:16.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:16 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:16.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:16 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:16.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:16 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:48:16.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:16 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:48:16.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:16 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:16.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:16 smithi123 ceph-mon[35971]: from='osd.6 [v2:172.21.15.2:6818/2711893600,v1:172.21.15.2:6819/2711893600]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-14T16:48:16.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:16 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:16.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:16 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:16.661 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:16 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:48:16.834 DEBUG:teuthology.orchestra.run.smithi002:osd.6> sudo journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@osd.6.service 2024-05-14T16:48:16.837 INFO:tasks.cephadm:Deploying osd.7 on smithi123 with /dev/vg_nvme/lv_2... 2024-05-14T16:48:16.837 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-14T16:48:17.079 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T16:48:17.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:17 smithi002 ceph-mon[31538]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:48:17.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:17 smithi002 ceph-mon[31538]: from='osd.6 [v2:172.21.15.2:6818/2711893600,v1:172.21.15.2:6819/2711893600]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-14T16:48:17.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:17 smithi002 ceph-mon[31538]: osdmap e40: 7 total, 6 up, 7 in 2024-05-14T16:48:17.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:17 smithi002 ceph-mon[31538]: from='osd.6 [v2:172.21.15.2:6818/2711893600,v1:172.21.15.2:6819/2711893600]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-05-14T16:48:17.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:17 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:48:17.497 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:17 smithi123 ceph-mon[35971]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:48:17.497 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:17 smithi123 ceph-mon[35971]: from='osd.6 [v2:172.21.15.2:6818/2711893600,v1:172.21.15.2:6819/2711893600]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-14T16:48:17.497 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:17 smithi123 ceph-mon[35971]: osdmap e40: 7 total, 6 up, 7 in 2024-05-14T16:48:17.497 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:17 smithi123 ceph-mon[35971]: from='osd.6 [v2:172.21.15.2:6818/2711893600,v1:172.21.15.2:6819/2711893600]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-05-14T16:48:17.497 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:17 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:48:17.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:17 smithi151 ceph-mon[36066]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:48:17.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:17 smithi151 ceph-mon[36066]: from='osd.6 [v2:172.21.15.2:6818/2711893600,v1:172.21.15.2:6819/2711893600]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-14T16:48:17.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:17 smithi151 ceph-mon[36066]: osdmap e40: 7 total, 6 up, 7 in 2024-05-14T16:48:17.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:17 smithi151 ceph-mon[36066]: from='osd.6 [v2:172.21.15.2:6818/2711893600,v1:172.21.15.2:6819/2711893600]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-05-14T16:48:17.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:17 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:48:17.993 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 16:48:17 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T16:48:17.796+0000 7fedefcad700 -1 osd.6 0 waiting for initial osdmap 2024-05-14T16:48:17.993 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 16:48:17 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T16:48:17.807+0000 7fede7a98700 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T16:48:18.530 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:18 smithi002 ceph-mon[31538]: from='osd.6 [v2:172.21.15.2:6818/2711893600,v1:172.21.15.2:6819/2711893600]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-05-14T16:48:18.530 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:18 smithi002 ceph-mon[31538]: osdmap e41: 7 total, 6 up, 7 in 2024-05-14T16:48:18.530 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:18 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:48:18.531 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:18 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:48:18.531 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:18 smithi002 ceph-mon[31538]: from='osd.6 [v2:172.21.15.2:6818/2711893600,v1:172.21.15.2:6819/2711893600]' entity='osd.6' 2024-05-14T16:48:18.536 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:18 smithi123 ceph-mon[35971]: from='osd.6 [v2:172.21.15.2:6818/2711893600,v1:172.21.15.2:6819/2711893600]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-05-14T16:48:18.536 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:18 smithi123 ceph-mon[35971]: osdmap e41: 7 total, 6 up, 7 in 2024-05-14T16:48:18.536 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:18 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:48:18.536 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:18 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:48:18.536 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:18 smithi123 ceph-mon[35971]: from='osd.6 [v2:172.21.15.2:6818/2711893600,v1:172.21.15.2:6819/2711893600]' entity='osd.6' 2024-05-14T16:48:18.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:18 smithi151 ceph-mon[36066]: from='osd.6 [v2:172.21.15.2:6818/2711893600,v1:172.21.15.2:6819/2711893600]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-05-14T16:48:18.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:18 smithi151 ceph-mon[36066]: osdmap e41: 7 total, 6 up, 7 in 2024-05-14T16:48:18.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:18 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:48:18.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:18 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:48:18.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:18 smithi151 ceph-mon[36066]: from='osd.6 [v2:172.21.15.2:6818/2711893600,v1:172.21.15.2:6819/2711893600]' entity='osd.6' 2024-05-14T16:48:19.246 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-14T16:48:19.247 INFO:teuthology.orchestra.run.smithi123.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-14T16:48:19.247 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10+0 records in 2024-05-14T16:48:19.247 INFO:teuthology.orchestra.run.smithi123.stderr:10+0 records out 2024-05-14T16:48:19.247 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115343 s, 909 MB/s 2024-05-14T16:48:19.247 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping successful for: 2024-05-14T16:48:19.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:19 smithi002 ceph-mon[31538]: purged_snaps scrub starts 2024-05-14T16:48:19.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:19 smithi002 ceph-mon[31538]: purged_snaps scrub ok 2024-05-14T16:48:19.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:19 smithi002 ceph-mon[31538]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:48:19.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:19 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:48:19.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:19 smithi002 ceph-mon[31538]: osd.6 [v2:172.21.15.2:6818/2711893600,v1:172.21.15.2:6819/2711893600] boot 2024-05-14T16:48:19.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:19 smithi002 ceph-mon[31538]: osdmap e42: 7 total, 7 up, 7 in 2024-05-14T16:48:19.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:19 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:48:19.553 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:19 smithi123 ceph-mon[35971]: purged_snaps scrub starts 2024-05-14T16:48:19.553 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:19 smithi123 ceph-mon[35971]: purged_snaps scrub ok 2024-05-14T16:48:19.553 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:19 smithi123 ceph-mon[35971]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:48:19.553 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:19 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:48:19.553 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:19 smithi123 ceph-mon[35971]: osd.6 [v2:172.21.15.2:6818/2711893600,v1:172.21.15.2:6819/2711893600] boot 2024-05-14T16:48:19.553 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:19 smithi123 ceph-mon[35971]: osdmap e42: 7 total, 7 up, 7 in 2024-05-14T16:48:19.554 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:19 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:48:19.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:19 smithi151 ceph-mon[36066]: purged_snaps scrub starts 2024-05-14T16:48:19.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:19 smithi151 ceph-mon[36066]: purged_snaps scrub ok 2024-05-14T16:48:19.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:19 smithi151 ceph-mon[36066]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-05-14T16:48:19.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:19 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:48:19.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:19 smithi151 ceph-mon[36066]: osd.6 [v2:172.21.15.2:6818/2711893600,v1:172.21.15.2:6819/2711893600] boot 2024-05-14T16:48:19.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:19 smithi151 ceph-mon[36066]: osdmap e42: 7 total, 7 up, 7 in 2024-05-14T16:48:19.606 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:19 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:48:19.896 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi123:vg_nvme/lv_2 2024-05-14T16:48:20.159 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T16:48:21.106 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:20 smithi151 ceph-mon[36066]: osdmap e43: 7 total, 7 up, 7 in 2024-05-14T16:48:21.107 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:20 smithi151 ceph-mon[36066]: Detected new or changed devices on smithi002 2024-05-14T16:48:21.107 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:20 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:21.107 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:20 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:21.107 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:20 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:48:21.107 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:20 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:48:21.107 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:20 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:48:21.108 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:20 smithi151 ceph-mon[36066]: Adjusting osd_memory_target on smithi002 to 5697M 2024-05-14T16:48:21.108 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:20 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:21.108 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:20 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:48:21.108 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:20 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:48:21.108 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:20 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:21.108 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:20 smithi151 ceph-mon[36066]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:48:21.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:20 smithi123 ceph-mon[35971]: osdmap e43: 7 total, 7 up, 7 in 2024-05-14T16:48:21.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:20 smithi123 ceph-mon[35971]: Detected new or changed devices on smithi002 2024-05-14T16:48:21.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:20 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:21.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:20 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:21.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:20 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:48:21.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:20 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:48:21.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:20 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:48:21.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:20 smithi123 ceph-mon[35971]: Adjusting osd_memory_target on smithi002 to 5697M 2024-05-14T16:48:21.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:20 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:21.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:20 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:48:21.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:20 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:48:21.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:20 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:21.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:20 smithi123 ceph-mon[35971]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:48:21.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:20 smithi002 ceph-mon[31538]: osdmap e43: 7 total, 7 up, 7 in 2024-05-14T16:48:21.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:20 smithi002 ceph-mon[31538]: Detected new or changed devices on smithi002 2024-05-14T16:48:21.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:20 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:21.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:20 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:21.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:20 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:48:21.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:20 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:48:21.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:20 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:48:21.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:20 smithi002 ceph-mon[31538]: Adjusting osd_memory_target on smithi002 to 5697M 2024-05-14T16:48:21.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:20 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:21.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:20 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:48:21.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:20 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:48:21.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:20 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:21.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:20 smithi002 ceph-mon[31538]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:48:21.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.725+0000 7f47345d9700 1 -- 172.21.15.123:0/4246931320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f472c104050 msgr2=0x7f472c106440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:21.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.725+0000 7f47345d9700 1 --2- 172.21.15.123:0/4246931320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f472c104050 0x7f472c106440 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f471c00b0a0 tx=0x7f471c00b3b0 comp rx=0 tx=0).stop 2024-05-14T16:48:21.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.725+0000 7f47345d9700 1 -- 172.21.15.123:0/4246931320 shutdown_connections 2024-05-14T16:48:21.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.725+0000 7f47345d9700 1 --2- 172.21.15.123:0/4246931320 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f472c1092b0 0x7f472c10b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:21.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.725+0000 7f47345d9700 1 --2- 172.21.15.123:0/4246931320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f472c104050 0x7f472c106440 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:21.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.725+0000 7f47345d9700 1 --2- 172.21.15.123:0/4246931320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f472c106980 0x7f472c108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:21.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.725+0000 7f47345d9700 1 -- 172.21.15.123:0/4246931320 >> 172.21.15.123:0/4246931320 conn(0x7f472c0fdaa0 msgr2=0x7f472c0ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:21.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.725+0000 7f47345d9700 1 -- 172.21.15.123:0/4246931320 shutdown_connections 2024-05-14T16:48:21.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.725+0000 7f47345d9700 1 -- 172.21.15.123:0/4246931320 wait complete. 2024-05-14T16:48:21.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.725+0000 7f47345d9700 1 Processor -- start 2024-05-14T16:48:21.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.726+0000 7f47345d9700 1 -- start start 2024-05-14T16:48:21.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.726+0000 7f47345d9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f472c104050 0x7f472c199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:21.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.726+0000 7f47345d9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f472c106980 0x7f472c199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:21.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.726+0000 7f47345d9700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f472c1092b0 0x7f472c1a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:21.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.726+0000 7f47345d9700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f472c10f080 con 0x7f472c106980 2024-05-14T16:48:21.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.726+0000 7f47345d9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f472c10ef00 con 0x7f472c104050 2024-05-14T16:48:21.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.726+0000 7f47345d9700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f472c10f200 con 0x7f472c1092b0 2024-05-14T16:48:21.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.726+0000 7f4732375700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f472c104050 0x7f472c199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:21.728 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.726+0000 7f4732375700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f472c104050 0x7f472c199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:59718/0 (socket says 172.21.15.123:59718) 2024-05-14T16:48:21.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.726+0000 7f4732375700 1 -- 172.21.15.123:0/1379106612 learned_addr learned my addr 172.21.15.123:0/1379106612 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-14T16:48:21.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.726+0000 7f4732375700 1 -- 172.21.15.123:0/1379106612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f472c1092b0 msgr2=0x7f472c1a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:21.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.726+0000 7f4732b76700 1 --2- 172.21.15.123:0/1379106612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f472c1092b0 0x7f472c1a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:21.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.726+0000 7f4731b74700 1 --2- 172.21.15.123:0/1379106612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f472c106980 0x7f472c199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:21.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.726+0000 7f4732375700 1 --2- 172.21.15.123:0/1379106612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f472c1092b0 0x7f472c1a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:21.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.726+0000 7f4732375700 1 -- 172.21.15.123:0/1379106612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f472c106980 msgr2=0x7f472c199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:21.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.726+0000 7f4732375700 1 --2- 172.21.15.123:0/1379106612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f472c106980 0x7f472c199c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:21.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.726+0000 7f4732375700 1 -- 172.21.15.123:0/1379106612 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f472c1a1360 con 0x7f472c104050 2024-05-14T16:48:21.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.726+0000 7f4732b76700 1 --2- 172.21.15.123:0/1379106612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f472c1092b0 0x7f472c1a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:48:21.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.726+0000 7f4732375700 1 --2- 172.21.15.123:0/1379106612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f472c104050 0x7f472c199700 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f471c00b070 tx=0x7f471c004ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:21.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.726+0000 7f472b7fe700 1 -- 172.21.15.123:0/1379106612 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c021070 con 0x7f472c104050 2024-05-14T16:48:21.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.726+0000 7f47345d9700 1 -- 172.21.15.123:0/1379106612 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f472c1a15f0 con 0x7f472c104050 2024-05-14T16:48:21.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.726+0000 7f472b7fe700 1 -- 172.21.15.123:0/1379106612 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f471c004020 con 0x7f472c104050 2024-05-14T16:48:21.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.726+0000 7f472b7fe700 1 -- 172.21.15.123:0/1379106612 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c01d6b0 con 0x7f472c104050 2024-05-14T16:48:21.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.727+0000 7f47345d9700 1 -- 172.21.15.123:0/1379106612 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f472c1a1b80 con 0x7f472c104050 2024-05-14T16:48:21.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.727+0000 7f47345d9700 1 -- 172.21.15.123:0/1379106612 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f472c06a230 con 0x7f472c104050 2024-05-14T16:48:21.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.729+0000 7f472b7fe700 1 -- 172.21.15.123:0/1379106612 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f471c003a00 con 0x7f472c104050 2024-05-14T16:48:21.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.729+0000 7f472b7fe700 1 --2- 172.21.15.123:0/1379106612 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f47140b2880 0x7f47140b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:21.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.729+0000 7f472b7fe700 1 -- 172.21.15.123:0/1379106612 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4965+0+0 (secure 0 0 0) 0x7f471c00f020 con 0x7f472c104050 2024-05-14T16:48:21.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.730+0000 7f4731b74700 1 --2- 172.21.15.123:0/1379106612 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f47140b2880 0x7f47140b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:21.731 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.730+0000 7f4731b74700 1 --2- 172.21.15.123:0/1379106612 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f47140b2880 0x7f47140b4d40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4720005950 tx=0x7f47200058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:21.732 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.733+0000 7f472b7fe700 1 -- 172.21.15.123:0/1379106612 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f471c02a020 con 0x7f472c104050 2024-05-14T16:48:21.948 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:21.949+0000 7f47345d9700 1 -- 172.21.15.123:0/1379106612 --> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f472c063980 con 0x7f47140b2880 2024-05-14T16:48:22.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:21 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:48:22.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:21 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:48:22.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:21 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:48:22.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:21 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:48:22.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:21 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:48:22.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:21 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:48:22.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:21 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:48:22.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:21 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:48:22.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:21 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:48:23.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:22 smithi123 ceph-mon[35971]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:48:23.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:22 smithi123 ceph-mon[35971]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:48:23.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:22 smithi002 ceph-mon[31538]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:48:23.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:22 smithi002 ceph-mon[31538]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:48:23.255 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:22 smithi151 ceph-mon[36066]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:48:23.256 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:22 smithi151 ceph-mon[36066]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:48:25.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:25 smithi123 ceph-mon[35971]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:48:25.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:25 smithi123 ceph-mon[35971]: from='client.? 172.21.15.123:0/1094460837' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c93e07a1-e941-493e-8468-33429df2dff0"}]: dispatch 2024-05-14T16:48:25.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:25 smithi123 ceph-mon[35971]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c93e07a1-e941-493e-8468-33429df2dff0"}]: dispatch 2024-05-14T16:48:25.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:25 smithi123 ceph-mon[35971]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c93e07a1-e941-493e-8468-33429df2dff0"}]': finished 2024-05-14T16:48:25.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:25 smithi123 ceph-mon[35971]: osdmap e44: 8 total, 7 up, 8 in 2024-05-14T16:48:25.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:25 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:48:25.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:25 smithi151 ceph-mon[36066]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:48:25.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:25 smithi151 ceph-mon[36066]: from='client.? 172.21.15.123:0/1094460837' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c93e07a1-e941-493e-8468-33429df2dff0"}]: dispatch 2024-05-14T16:48:25.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:25 smithi151 ceph-mon[36066]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c93e07a1-e941-493e-8468-33429df2dff0"}]: dispatch 2024-05-14T16:48:25.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:25 smithi151 ceph-mon[36066]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c93e07a1-e941-493e-8468-33429df2dff0"}]': finished 2024-05-14T16:48:25.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:25 smithi151 ceph-mon[36066]: osdmap e44: 8 total, 7 up, 8 in 2024-05-14T16:48:25.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:25 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:48:25.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:25 smithi002 ceph-mon[31538]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:48:25.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:25 smithi002 ceph-mon[31538]: from='client.? 172.21.15.123:0/1094460837' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c93e07a1-e941-493e-8468-33429df2dff0"}]: dispatch 2024-05-14T16:48:25.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:25 smithi002 ceph-mon[31538]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c93e07a1-e941-493e-8468-33429df2dff0"}]: dispatch 2024-05-14T16:48:25.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:25 smithi002 ceph-mon[31538]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c93e07a1-e941-493e-8468-33429df2dff0"}]': finished 2024-05-14T16:48:25.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:25 smithi002 ceph-mon[31538]: osdmap e44: 8 total, 7 up, 8 in 2024-05-14T16:48:25.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:25 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:48:26.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:26 smithi123 ceph-mon[35971]: from='client.? 172.21.15.123:0/1655250626' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:48:26.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:26 smithi002 ceph-mon[31538]: from='client.? 172.21.15.123:0/1655250626' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:48:26.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:26 smithi151 ceph-mon[36066]: from='client.? 172.21.15.123:0/1655250626' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:48:27.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:27 smithi123 ceph-mon[35971]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:48:27.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:27 smithi002 ceph-mon[31538]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:48:27.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:27 smithi151 ceph-mon[36066]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:48:29.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:29 smithi123 ceph-mon[35971]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:48:29.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:29 smithi002 ceph-mon[31538]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:48:29.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:29 smithi151 ceph-mon[36066]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:48:31.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:31 smithi123 ceph-mon[35971]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:48:31.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:31 smithi002 ceph-mon[31538]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:48:31.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:31 smithi151 ceph-mon[36066]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:48:33.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:33 smithi123 ceph-mon[35971]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:48:33.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:33 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-14T16:48:33.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:33 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:48:33.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:33 smithi123 ceph-mon[35971]: Deploying daemon osd.7 on smithi123 2024-05-14T16:48:33.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:33 smithi002 ceph-mon[31538]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:48:33.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:33 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-14T16:48:33.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:33 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:48:33.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:33 smithi002 ceph-mon[31538]: Deploying daemon osd.7 on smithi123 2024-05-14T16:48:33.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:33 smithi151 ceph-mon[36066]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:48:33.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:33 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-14T16:48:33.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:33 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:48:33.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:33 smithi151 ceph-mon[36066]: Deploying daemon osd.7 on smithi123 2024-05-14T16:48:35.250 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:35 smithi123 ceph-mon[35971]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:48:35.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:35 smithi151 ceph-mon[36066]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:48:35.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:35 smithi002 ceph-mon[31538]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:48:37.400 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:37 smithi123 ceph-mon[35971]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:48:37.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:37 smithi002 ceph-mon[31538]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:48:37.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:37 smithi151 ceph-mon[36066]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:48:39.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:38 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:39.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:38 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:39.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:38 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:48:39.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:38 smithi002 ceph-mon[31538]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:48:39.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:38 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:39.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:38 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:39.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:38 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:48:39.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:38 smithi151 ceph-mon[36066]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:48:39.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:38 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:39.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:38 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:39.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:38 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:48:39.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:38 smithi123 ceph-mon[35971]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:48:41.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:41 smithi151 ceph-mon[36066]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:48:41.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:41 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:41.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:41 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:41.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:41 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:48:41.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:41 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:48:41.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:41 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:41.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:41 smithi123 ceph-mon[35971]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:48:41.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:41 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:41.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:41 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:41.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:41 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:48:41.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:41 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:48:41.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:41 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:41.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:41 smithi002 ceph-mon[31538]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:48:41.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:41 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:41.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:41 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:41.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:41 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:48:41.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:41 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:48:41.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:41 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:41.868 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:41.867+0000 7f472b7fe700 1 -- 172.21.15.123:0/1379106612 <== mgr.14150 v2:172.21.15.2:6800/1498936028 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f472c063980 con 0x7f47140b2880 2024-05-14T16:48:41.868 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 7 on host 'smithi123' 2024-05-14T16:48:41.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:41.870+0000 7f47345d9700 1 -- 172.21.15.123:0/1379106612 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f47140b2880 msgr2=0x7f47140b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:41.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:41.870+0000 7f47345d9700 1 --2- 172.21.15.123:0/1379106612 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f47140b2880 0x7f47140b4d40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4720005950 tx=0x7f47200058e0 comp rx=0 tx=0).stop 2024-05-14T16:48:41.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:41.870+0000 7f47345d9700 1 -- 172.21.15.123:0/1379106612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f472c104050 msgr2=0x7f472c199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:41.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:41.870+0000 7f47345d9700 1 --2- 172.21.15.123:0/1379106612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f472c104050 0x7f472c199700 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f471c00b070 tx=0x7f471c004ef0 comp rx=0 tx=0).stop 2024-05-14T16:48:41.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:41.870+0000 7f47345d9700 1 -- 172.21.15.123:0/1379106612 shutdown_connections 2024-05-14T16:48:41.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:41.870+0000 7f47345d9700 1 --2- 172.21.15.123:0/1379106612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f472c1092b0 0x7f472c1a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:41.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:41.870+0000 7f47345d9700 1 --2- 172.21.15.123:0/1379106612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f472c104050 0x7f472c199700 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:41.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:41.870+0000 7f47345d9700 1 --2- 172.21.15.123:0/1379106612 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f47140b2880 0x7f47140b4d40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:41.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:41.870+0000 7f47345d9700 1 --2- 172.21.15.123:0/1379106612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f472c106980 0x7f472c199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:41.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:41.870+0000 7f47345d9700 1 -- 172.21.15.123:0/1379106612 >> 172.21.15.123:0/1379106612 conn(0x7f472c0fdaa0 msgr2=0x7f472c104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:41.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:41.870+0000 7f47345d9700 1 -- 172.21.15.123:0/1379106612 shutdown_connections 2024-05-14T16:48:41.871 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:48:41.870+0000 7f47345d9700 1 -- 172.21.15.123:0/1379106612 wait complete. 2024-05-14T16:48:42.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:42 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:48:42.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:42 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:42.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:42 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:42.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:42 smithi151 ceph-mon[36066]: from='osd.7 [v2:172.21.15.123:6816/3634155507,v1:172.21.15.123:6817/3634155507]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-14T16:48:42.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:42 smithi151 ceph-mon[36066]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-14T16:48:42.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:42 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:48:42.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:42 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:42.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:42 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:42.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:42 smithi123 ceph-mon[35971]: from='osd.7 [v2:172.21.15.123:6816/3634155507,v1:172.21.15.123:6817/3634155507]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-14T16:48:42.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:42 smithi123 ceph-mon[35971]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-14T16:48:42.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:42 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:48:42.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:42 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:42.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:42 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:42.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:42 smithi002 ceph-mon[31538]: from='osd.7 [v2:172.21.15.123:6816/3634155507,v1:172.21.15.123:6817/3634155507]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-14T16:48:42.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:42 smithi002 ceph-mon[31538]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-14T16:48:42.563 DEBUG:teuthology.orchestra.run.smithi123:osd.7> sudo journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@osd.7.service 2024-05-14T16:48:42.565 INFO:tasks.cephadm:Deploying osd.8 on smithi151 with /dev/vg_nvme/lv_2... 2024-05-14T16:48:42.565 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-14T16:48:42.811 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T16:48:43.185 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:43 smithi151 ceph-mon[36066]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:48:43.185 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:43 smithi151 ceph-mon[36066]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-14T16:48:43.185 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:43 smithi151 ceph-mon[36066]: osdmap e45: 8 total, 7 up, 8 in 2024-05-14T16:48:43.185 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:43 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:48:43.185 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:43 smithi151 ceph-mon[36066]: from='osd.7 [v2:172.21.15.123:6816/3634155507,v1:172.21.15.123:6817/3634155507]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-14T16:48:43.185 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:43 smithi151 ceph-mon[36066]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-14T16:48:43.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:43 smithi123 ceph-mon[35971]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:48:43.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:43 smithi123 ceph-mon[35971]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-14T16:48:43.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:43 smithi123 ceph-mon[35971]: osdmap e45: 8 total, 7 up, 8 in 2024-05-14T16:48:43.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:43 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:48:43.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:43 smithi123 ceph-mon[35971]: from='osd.7 [v2:172.21.15.123:6816/3634155507,v1:172.21.15.123:6817/3634155507]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-14T16:48:43.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:43 smithi123 ceph-mon[35971]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-14T16:48:43.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:43 smithi002 ceph-mon[31538]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:48:43.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:43 smithi002 ceph-mon[31538]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-14T16:48:43.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:43 smithi002 ceph-mon[31538]: osdmap e45: 8 total, 7 up, 8 in 2024-05-14T16:48:43.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:43 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:48:43.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:43 smithi002 ceph-mon[31538]: from='osd.7 [v2:172.21.15.123:6816/3634155507,v1:172.21.15.123:6817/3634155507]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-14T16:48:43.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:43 smithi002 ceph-mon[31538]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-14T16:48:44.660 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 16:48:44 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T16:48:44.298+0000 7fcff1332700 -1 osd.7 0 waiting for initial osdmap 2024-05-14T16:48:44.660 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 16:48:44 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T16:48:44.307+0000 7fcfe891c700 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T16:48:44.866 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:44 smithi151 ceph-mon[36066]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-05-14T16:48:44.866 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:44 smithi151 ceph-mon[36066]: osdmap e46: 8 total, 7 up, 8 in 2024-05-14T16:48:44.866 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:44 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:48:44.866 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:44 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:48:44.866 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:44 smithi151 ceph-mon[36066]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:48:44.866 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:44 smithi151 ceph-mon[36066]: from='osd.7 ' entity='osd.7' 2024-05-14T16:48:44.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:44 smithi002 ceph-mon[31538]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-05-14T16:48:44.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:44 smithi002 ceph-mon[31538]: osdmap e46: 8 total, 7 up, 8 in 2024-05-14T16:48:44.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:44 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:48:44.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:44 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:48:44.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:44 smithi002 ceph-mon[31538]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:48:44.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:44 smithi002 ceph-mon[31538]: from='osd.7 ' entity='osd.7' 2024-05-14T16:48:45.073 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-14T16:48:45.073 INFO:teuthology.orchestra.run.smithi151.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-14T16:48:45.073 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10+0 records in 2024-05-14T16:48:45.073 INFO:teuthology.orchestra.run.smithi151.stderr:10+0 records out 2024-05-14T16:48:45.073 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0133483 s, 786 MB/s 2024-05-14T16:48:45.073 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping successful for: 2024-05-14T16:48:45.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:44 smithi123 ceph-mon[35971]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-05-14T16:48:45.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:44 smithi123 ceph-mon[35971]: osdmap e46: 8 total, 7 up, 8 in 2024-05-14T16:48:45.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:44 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:48:45.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:44 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:48:45.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:44 smithi123 ceph-mon[35971]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 626 GiB / 626 GiB avail 2024-05-14T16:48:45.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:44 smithi123 ceph-mon[35971]: from='osd.7 ' entity='osd.7' 2024-05-14T16:48:45.683 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi151:vg_nvme/lv_2 2024-05-14T16:48:45.932 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T16:48:46.054 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:45 smithi151 ceph-mon[36066]: purged_snaps scrub starts 2024-05-14T16:48:46.054 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:45 smithi151 ceph-mon[36066]: purged_snaps scrub ok 2024-05-14T16:48:46.054 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:45 smithi151 ceph-mon[36066]: osd.7 [v2:172.21.15.123:6816/3634155507,v1:172.21.15.123:6817/3634155507] boot 2024-05-14T16:48:46.054 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:45 smithi151 ceph-mon[36066]: osdmap e47: 8 total, 8 up, 8 in 2024-05-14T16:48:46.054 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:45 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:48:46.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:45 smithi123 ceph-mon[35971]: purged_snaps scrub starts 2024-05-14T16:48:46.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:45 smithi123 ceph-mon[35971]: purged_snaps scrub ok 2024-05-14T16:48:46.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:45 smithi123 ceph-mon[35971]: osd.7 [v2:172.21.15.123:6816/3634155507,v1:172.21.15.123:6817/3634155507] boot 2024-05-14T16:48:46.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:45 smithi123 ceph-mon[35971]: osdmap e47: 8 total, 8 up, 8 in 2024-05-14T16:48:46.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:45 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:48:46.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:45 smithi002 ceph-mon[31538]: purged_snaps scrub starts 2024-05-14T16:48:46.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:45 smithi002 ceph-mon[31538]: purged_snaps scrub ok 2024-05-14T16:48:46.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:45 smithi002 ceph-mon[31538]: osd.7 [v2:172.21.15.123:6816/3634155507,v1:172.21.15.123:6817/3634155507] boot 2024-05-14T16:48:46.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:45 smithi002 ceph-mon[31538]: osdmap e47: 8 total, 8 up, 8 in 2024-05-14T16:48:46.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:45 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:48:47.022 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:46 smithi151 ceph-mon[36066]: osdmap e48: 8 total, 8 up, 8 in 2024-05-14T16:48:47.022 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:46 smithi151 ceph-mon[36066]: Detected new or changed devices on smithi123 2024-05-14T16:48:47.022 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:46 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:47.022 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:46 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:47.023 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:46 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:48:47.023 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:46 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:48:47.023 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:46 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:48:47.023 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:46 smithi151 ceph-mon[36066]: Adjusting osd_memory_target on smithi123 to 5696M 2024-05-14T16:48:47.023 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:46 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:47.023 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:46 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:48:47.023 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:46 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:48:47.023 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:46 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:47.023 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:46 smithi151 ceph-mon[36066]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:48:47.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:46 smithi123 ceph-mon[35971]: osdmap e48: 8 total, 8 up, 8 in 2024-05-14T16:48:47.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:46 smithi123 ceph-mon[35971]: Detected new or changed devices on smithi123 2024-05-14T16:48:47.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:46 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:47.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:46 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:47.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:46 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:48:47.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:46 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:48:47.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:46 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:48:47.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:46 smithi123 ceph-mon[35971]: Adjusting osd_memory_target on smithi123 to 5696M 2024-05-14T16:48:47.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:46 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:47.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:46 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:48:47.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:46 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:48:47.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:46 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:47.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:46 smithi123 ceph-mon[35971]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:48:47.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:46 smithi002 ceph-mon[31538]: osdmap e48: 8 total, 8 up, 8 in 2024-05-14T16:48:47.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:46 smithi002 ceph-mon[31538]: Detected new or changed devices on smithi123 2024-05-14T16:48:47.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:46 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:47.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:46 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:47.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:46 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:48:47.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:46 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:48:47.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:46 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:48:47.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:46 smithi002 ceph-mon[31538]: Adjusting osd_memory_target on smithi123 to 5696M 2024-05-14T16:48:47.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:46 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:47.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:46 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:48:47.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:46 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:48:47.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:46 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:48:47.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:46 smithi002 ceph-mon[31538]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:48:47.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.591+0000 7f89fe31e700 1 -- 172.21.15.151:0/571076527 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89f8106960 msgr2=0x7f89f8108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:47.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.591+0000 7f89fe31e700 1 --2- 172.21.15.151:0/571076527 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89f8106960 0x7f89f8108d50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f89ec009990 tx=0x7f89ec009ca0 comp rx=0 tx=0).stop 2024-05-14T16:48:47.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.591+0000 7f89fe31e700 1 -- 172.21.15.151:0/571076527 shutdown_connections 2024-05-14T16:48:47.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.591+0000 7f89fe31e700 1 --2- 172.21.15.151:0/571076527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89f8109290 0x7f89f810b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:47.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.591+0000 7f89fe31e700 1 --2- 172.21.15.151:0/571076527 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89f8104030 0x7f89f8106420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:47.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.591+0000 7f89fe31e700 1 --2- 172.21.15.151:0/571076527 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89f8106960 0x7f89f8108d50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:47.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.591+0000 7f89fe31e700 1 -- 172.21.15.151:0/571076527 >> 172.21.15.151:0/571076527 conn(0x7f89f80fda40 msgr2=0x7f89f80ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:48:47.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.591+0000 7f89fe31e700 1 -- 172.21.15.151:0/571076527 shutdown_connections 2024-05-14T16:48:47.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.591+0000 7f89fe31e700 1 -- 172.21.15.151:0/571076527 wait complete. 2024-05-14T16:48:47.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.591+0000 7f89fe31e700 1 Processor -- start 2024-05-14T16:48:47.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.591+0000 7f89fe31e700 1 -- start start 2024-05-14T16:48:47.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.592+0000 7f89fe31e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89f8104030 0x7f89f8103990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:47.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.592+0000 7f89fe31e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89f8106960 0x7f89f8101fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:47.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.592+0000 7f89fe31e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89f8109290 0x7f89f8102520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:47.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.592+0000 7f89fe31e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f89f810f060 con 0x7f89f8106960 2024-05-14T16:48:47.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.592+0000 7f89fe31e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f89f810eee0 con 0x7f89f8109290 2024-05-14T16:48:47.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.592+0000 7f89fe31e700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f89f810f1e0 con 0x7f89f8104030 2024-05-14T16:48:47.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.592+0000 7f89f7fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89f8104030 0x7f89f8103990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:47.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.592+0000 7f89f7fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89f8104030 0x7f89f8103990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37986/0 (socket says 172.21.15.151:37986) 2024-05-14T16:48:47.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.592+0000 7f89f7fff700 1 -- 172.21.15.151:0/1343900363 learned_addr learned my addr 172.21.15.151:0/1343900363 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-14T16:48:47.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.592+0000 7f89fc8bb700 1 --2- 172.21.15.151:0/1343900363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89f8109290 0x7f89f8102520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:47.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.592+0000 7f89f77fe700 1 --2- 172.21.15.151:0/1343900363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89f8106960 0x7f89f8101fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:47.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.592+0000 7f89f7fff700 1 -- 172.21.15.151:0/1343900363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89f8109290 msgr2=0x7f89f8102520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:47.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.592+0000 7f89f7fff700 1 --2- 172.21.15.151:0/1343900363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89f8109290 0x7f89f8102520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:47.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.592+0000 7f89f7fff700 1 -- 172.21.15.151:0/1343900363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89f8106960 msgr2=0x7f89f8101fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:48:47.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.592+0000 7f89f7fff700 1 --2- 172.21.15.151:0/1343900363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89f8106960 0x7f89f8101fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:48:47.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.592+0000 7f89f7fff700 1 -- 172.21.15.151:0/1343900363 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89f8102dd0 con 0x7f89f8104030 2024-05-14T16:48:47.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.592+0000 7f89fc8bb700 1 --2- 172.21.15.151:0/1343900363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89f8109290 0x7f89f8102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:48:47.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.592+0000 7f89f7fff700 1 --2- 172.21.15.151:0/1343900363 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89f8104030 0x7f89f8103990 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f89e800d950 tx=0x7f89e800dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:47.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.592+0000 7f89f57fa700 1 -- 172.21.15.151:0/1343900363 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e8016070 con 0x7f89f8104030 2024-05-14T16:48:47.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.592+0000 7f89f57fa700 1 -- 172.21.15.151:0/1343900363 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89e8010460 con 0x7f89f8104030 2024-05-14T16:48:47.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.592+0000 7f89fe31e700 1 -- 172.21.15.151:0/1343900363 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89f81a3880 con 0x7f89f8104030 2024-05-14T16:48:47.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.592+0000 7f89f57fa700 1 -- 172.21.15.151:0/1343900363 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e800f620 con 0x7f89f8104030 2024-05-14T16:48:47.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.593+0000 7f89fe31e700 1 -- 172.21.15.151:0/1343900363 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89f81a3d90 con 0x7f89f8104030 2024-05-14T16:48:47.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.594+0000 7f89fe31e700 1 -- 172.21.15.151:0/1343900363 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89f806a230 con 0x7f89f8104030 2024-05-14T16:48:47.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.595+0000 7f89f57fa700 1 -- 172.21.15.151:0/1343900363 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f89e8019070 con 0x7f89f8104030 2024-05-14T16:48:47.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.595+0000 7f89f57fa700 1 --2- 172.21.15.151:0/1343900363 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f89e00b2880 0x7f89e00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:48:47.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.596+0000 7f89f57fa700 1 -- 172.21.15.151:0/1343900363 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5397+0+0 (secure 0 0 0) 0x7f89e80e1320 con 0x7f89f8104030 2024-05-14T16:48:47.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.596+0000 7f89f77fe700 1 --2- 172.21.15.151:0/1343900363 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f89e00b2880 0x7f89e00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:48:47.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.596+0000 7f89f77fe700 1 --2- 172.21.15.151:0/1343900363 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f89e00b2880 0x7f89e00b4d40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f89ec006010 tx=0x7f89ec009510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:48:47.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.599+0000 7f89f57fa700 1 -- 172.21.15.151:0/1343900363 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f89e80a9ba0 con 0x7f89f8104030 2024-05-14T16:48:47.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:48:47.810+0000 7f89fe31e700 1 -- 172.21.15.151:0/1343900363 --> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f89f8063980 con 0x7f89e00b2880 2024-05-14T16:48:48.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:47 smithi151 ceph-mon[36066]: osdmap e49: 8 total, 8 up, 8 in 2024-05-14T16:48:48.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:47 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:48:48.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:47 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:48:48.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:47 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:48:48.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:47 smithi123 ceph-mon[35971]: osdmap e49: 8 total, 8 up, 8 in 2024-05-14T16:48:48.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:47 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:48:48.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:47 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:48:48.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:47 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:48:48.243 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:47 smithi002 ceph-mon[31538]: osdmap e49: 8 total, 8 up, 8 in 2024-05-14T16:48:48.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:47 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:48:48.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:47 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:48:48.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:47 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:48:49.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:48 smithi151 ceph-mon[36066]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:48:49.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:48 smithi151 ceph-mon[36066]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:48:49.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:48 smithi123 ceph-mon[35971]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:48:49.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:48 smithi123 ceph-mon[35971]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:48:49.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:48 smithi002 ceph-mon[31538]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:48:49.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:48 smithi002 ceph-mon[31538]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:48:51.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:51 smithi151 ceph-mon[36066]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-14T16:48:51.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:51 smithi151 ceph-mon[36066]: from='client.? 172.21.15.151:0/1028359451' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1155fb58-d614-4f45-91e2-5da3a93e7d9c"}]: dispatch 2024-05-14T16:48:51.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:51 smithi151 ceph-mon[36066]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1155fb58-d614-4f45-91e2-5da3a93e7d9c"}]: dispatch 2024-05-14T16:48:51.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:51 smithi151 ceph-mon[36066]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1155fb58-d614-4f45-91e2-5da3a93e7d9c"}]': finished 2024-05-14T16:48:51.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:51 smithi151 ceph-mon[36066]: osdmap e50: 9 total, 8 up, 9 in 2024-05-14T16:48:51.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:51 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:48:51.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:51 smithi123 ceph-mon[35971]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-14T16:48:51.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:51 smithi123 ceph-mon[35971]: from='client.? 172.21.15.151:0/1028359451' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1155fb58-d614-4f45-91e2-5da3a93e7d9c"}]: dispatch 2024-05-14T16:48:51.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:51 smithi123 ceph-mon[35971]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1155fb58-d614-4f45-91e2-5da3a93e7d9c"}]: dispatch 2024-05-14T16:48:51.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:51 smithi123 ceph-mon[35971]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1155fb58-d614-4f45-91e2-5da3a93e7d9c"}]': finished 2024-05-14T16:48:51.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:51 smithi123 ceph-mon[35971]: osdmap e50: 9 total, 8 up, 9 in 2024-05-14T16:48:51.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:51 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:48:51.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:51 smithi002 ceph-mon[31538]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-14T16:48:51.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:51 smithi002 ceph-mon[31538]: from='client.? 172.21.15.151:0/1028359451' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1155fb58-d614-4f45-91e2-5da3a93e7d9c"}]: dispatch 2024-05-14T16:48:51.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:51 smithi002 ceph-mon[31538]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1155fb58-d614-4f45-91e2-5da3a93e7d9c"}]: dispatch 2024-05-14T16:48:51.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:51 smithi002 ceph-mon[31538]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1155fb58-d614-4f45-91e2-5da3a93e7d9c"}]': finished 2024-05-14T16:48:51.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:51 smithi002 ceph-mon[31538]: osdmap e50: 9 total, 8 up, 9 in 2024-05-14T16:48:51.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:51 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:48:52.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:52 smithi123 ceph-mon[35971]: from='client.? 172.21.15.151:0/3094745672' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:48:52.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:52 smithi002 ceph-mon[31538]: from='client.? 172.21.15.151:0/3094745672' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:48:52.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:52 smithi151 ceph-mon[36066]: from='client.? 172.21.15.151:0/3094745672' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:48:53.409 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:53 smithi123 ceph-mon[35971]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-05-14T16:48:53.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:53 smithi002 ceph-mon[31538]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-05-14T16:48:53.505 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:53 smithi151 ceph-mon[36066]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-05-14T16:48:55.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:55 smithi151 ceph-mon[36066]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T16:48:55.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:55 smithi123 ceph-mon[35971]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T16:48:55.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:55 smithi002 ceph-mon[31538]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T16:48:57.295 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:57 smithi151 ceph-mon[36066]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-14T16:48:57.409 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:57 smithi123 ceph-mon[35971]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-14T16:48:57.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:57 smithi002 ceph-mon[31538]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-14T16:48:58.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:58 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-14T16:48:58.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:58 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:48:58.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:58 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-14T16:48:58.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:58 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:48:58.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:58 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-14T16:48:58.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:58 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:48:59.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:59 smithi123 ceph-mon[35971]: Deploying daemon osd.8 on smithi151 2024-05-14T16:48:59.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:48:59 smithi123 ceph-mon[35971]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:48:59.476 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:59 smithi151 ceph-mon[36066]: Deploying daemon osd.8 on smithi151 2024-05-14T16:48:59.476 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:48:59 smithi151 ceph-mon[36066]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:48:59.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:59 smithi002 ceph-mon[31538]: Deploying daemon osd.8 on smithi151 2024-05-14T16:48:59.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:48:59 smithi002 ceph-mon[31538]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:49:01.309 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:01 smithi151 ceph-mon[36066]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:49:01.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:01 smithi123 ceph-mon[35971]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:49:01.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:01 smithi002 ceph-mon[31538]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:49:03.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:03 smithi151 ceph-mon[36066]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:49:03.409 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:03 smithi123 ceph-mon[35971]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:49:03.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:03 smithi002 ceph-mon[31538]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:49:04.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:04 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:04.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:04 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:04.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:04 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:49:04.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:04 smithi123 ceph-mon[35971]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:49:04.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:04 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:04.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:04 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:04.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:04 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:49:04.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:04 smithi002 ceph-mon[31538]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:49:05.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:04 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:05.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:04 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:05.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:04 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:49:05.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:04 smithi151 ceph-mon[36066]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:49:07.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:49:07.580+0000 7f89f57fa700 1 -- 172.21.15.151:0/1343900363 <== mgr.14150 v2:172.21.15.2:6800/1498936028 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f89f8063980 con 0x7f89e00b2880 2024-05-14T16:49:07.581 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 8 on host 'smithi151' 2024-05-14T16:49:07.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:49:07.583+0000 7f89fe31e700 1 -- 172.21.15.151:0/1343900363 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f89e00b2880 msgr2=0x7f89e00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:07.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:49:07.583+0000 7f89fe31e700 1 --2- 172.21.15.151:0/1343900363 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f89e00b2880 0x7f89e00b4d40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f89ec006010 tx=0x7f89ec009510 comp rx=0 tx=0).stop 2024-05-14T16:49:07.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:49:07.583+0000 7f89fe31e700 1 -- 172.21.15.151:0/1343900363 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89f8104030 msgr2=0x7f89f8103990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:07.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:49:07.583+0000 7f89fe31e700 1 --2- 172.21.15.151:0/1343900363 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89f8104030 0x7f89f8103990 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f89e800d950 tx=0x7f89e800dd10 comp rx=0 tx=0).stop 2024-05-14T16:49:07.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:49:07.583+0000 7f89fe31e700 1 -- 172.21.15.151:0/1343900363 shutdown_connections 2024-05-14T16:49:07.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:49:07.583+0000 7f89fe31e700 1 --2- 172.21.15.151:0/1343900363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89f8109290 0x7f89f8102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:07.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:49:07.583+0000 7f89fe31e700 1 --2- 172.21.15.151:0/1343900363 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89f8104030 0x7f89f8103990 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:07.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:49:07.583+0000 7f89fe31e700 1 --2- 172.21.15.151:0/1343900363 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f89e00b2880 0x7f89e00b4d40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:07.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:49:07.583+0000 7f89fe31e700 1 --2- 172.21.15.151:0/1343900363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89f8106960 0x7f89f8101fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:07.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:49:07.583+0000 7f89fe31e700 1 -- 172.21.15.151:0/1343900363 >> 172.21.15.151:0/1343900363 conn(0x7f89f80fda40 msgr2=0x7f89f81048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:07.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:49:07.583+0000 7f89fe31e700 1 -- 172.21.15.151:0/1343900363 shutdown_connections 2024-05-14T16:49:07.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:49:07.583+0000 7f89fe31e700 1 -- 172.21.15.151:0/1343900363 wait complete. 2024-05-14T16:49:07.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:07 smithi151 ceph-mon[36066]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:49:07.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:07 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:07.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:07 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:07.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:07 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:49:07.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:07 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:49:07.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:07 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:07.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:07 smithi123 ceph-mon[35971]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:49:07.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:07 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:07.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:07 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:07.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:07 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:49:07.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:07 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:49:07.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:07 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:07.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:07 smithi002 ceph-mon[31538]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:49:07.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:07 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:07.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:07 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:07.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:07 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:49:07.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:07 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:49:07.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:07 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:08.245 DEBUG:teuthology.orchestra.run.smithi151:osd.8> sudo journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@osd.8.service 2024-05-14T16:49:08.247 INFO:tasks.cephadm:Deploying osd.9 on smithi002 with /dev/vg_nvme/lv_1... 2024-05-14T16:49:08.247 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-14T16:49:08.508 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:49:08.723 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:08 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:08.724 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:08 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:08.724 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:08 smithi151 ceph-mon[36066]: from='osd.8 [v2:172.21.15.151:6816/4271107010,v1:172.21.15.151:6817/4271107010]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-14T16:49:08.724 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:08 smithi151 ceph-mon[36066]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-14T16:49:08.724 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:08 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:49:08.724 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:08 smithi151 ceph-mon[36066]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:49:08.860 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:08 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:08.860 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:08 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:08.860 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:08 smithi002 ceph-mon[31538]: from='osd.8 [v2:172.21.15.151:6816/4271107010,v1:172.21.15.151:6817/4271107010]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-14T16:49:08.860 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:08 smithi002 ceph-mon[31538]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-14T16:49:08.860 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:08 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:49:08.860 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:08 smithi002 ceph-mon[31538]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:49:08.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:08 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:08.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:08 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:08.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:08 smithi123 ceph-mon[35971]: from='osd.8 [v2:172.21.15.151:6816/4271107010,v1:172.21.15.151:6817/4271107010]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-14T16:49:08.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:08 smithi123 ceph-mon[35971]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-14T16:49:08.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:08 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:49:08.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:08 smithi123 ceph-mon[35971]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:49:09.842 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:09 smithi151 ceph-mon[36066]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-14T16:49:09.842 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:09 smithi151 ceph-mon[36066]: osdmap e51: 9 total, 8 up, 9 in 2024-05-14T16:49:09.842 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:09 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:49:09.842 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:09 smithi151 ceph-mon[36066]: from='osd.8 [v2:172.21.15.151:6816/4271107010,v1:172.21.15.151:6817/4271107010]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-14T16:49:09.842 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:09 smithi151 ceph-mon[36066]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-14T16:49:09.842 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 16:49:09 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T16:49:09.592+0000 7f7fd3655700 -1 osd.8 0 waiting for initial osdmap 2024-05-14T16:49:09.842 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 16:49:09 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T16:49:09.601+0000 7f7fcf448700 -1 osd.8 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T16:49:09.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:09 smithi123 ceph-mon[35971]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-14T16:49:09.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:09 smithi123 ceph-mon[35971]: osdmap e51: 9 total, 8 up, 9 in 2024-05-14T16:49:09.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:09 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:49:09.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:09 smithi123 ceph-mon[35971]: from='osd.8 [v2:172.21.15.151:6816/4271107010,v1:172.21.15.151:6817/4271107010]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-14T16:49:09.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:09 smithi123 ceph-mon[35971]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-14T16:49:09.940 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:09 smithi002 ceph-mon[31538]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-14T16:49:09.941 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:09 smithi002 ceph-mon[31538]: osdmap e51: 9 total, 8 up, 9 in 2024-05-14T16:49:09.941 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:09 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:49:09.941 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:09 smithi002 ceph-mon[31538]: from='osd.8 [v2:172.21.15.151:6816/4271107010,v1:172.21.15.151:6817/4271107010]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-14T16:49:09.941 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:09 smithi002 ceph-mon[31538]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-14T16:49:10.839 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-14T16:49:10.839 INFO:teuthology.orchestra.run.smithi002.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-14T16:49:10.839 INFO:teuthology.orchestra.run.smithi002.stderr: stderr: 10+0 records in 2024-05-14T16:49:10.840 INFO:teuthology.orchestra.run.smithi002.stderr:10+0 records out 2024-05-14T16:49:10.840 INFO:teuthology.orchestra.run.smithi002.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00840182 s, 1.2 GB/s 2024-05-14T16:49:10.840 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping successful for: 2024-05-14T16:49:10.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:10 smithi151 ceph-mon[36066]: purged_snaps scrub starts 2024-05-14T16:49:10.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:10 smithi151 ceph-mon[36066]: purged_snaps scrub ok 2024-05-14T16:49:10.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:10 smithi151 ceph-mon[36066]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-05-14T16:49:10.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:10 smithi151 ceph-mon[36066]: osdmap e52: 9 total, 8 up, 9 in 2024-05-14T16:49:10.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:10 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:49:10.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:10 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:49:10.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:10 smithi151 ceph-mon[36066]: pgmap v191: 1 pgs: 1 peering; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:49:10.863 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:10 smithi002 ceph-mon[31538]: purged_snaps scrub starts 2024-05-14T16:49:10.863 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:10 smithi002 ceph-mon[31538]: purged_snaps scrub ok 2024-05-14T16:49:10.863 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:10 smithi002 ceph-mon[31538]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-05-14T16:49:10.863 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:10 smithi002 ceph-mon[31538]: osdmap e52: 9 total, 8 up, 9 in 2024-05-14T16:49:10.863 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:10 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:49:10.864 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:10 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:49:10.864 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:10 smithi002 ceph-mon[31538]: pgmap v191: 1 pgs: 1 peering; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:49:10.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:10 smithi123 ceph-mon[35971]: purged_snaps scrub starts 2024-05-14T16:49:10.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:10 smithi123 ceph-mon[35971]: purged_snaps scrub ok 2024-05-14T16:49:10.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:10 smithi123 ceph-mon[35971]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-05-14T16:49:10.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:10 smithi123 ceph-mon[35971]: osdmap e52: 9 total, 8 up, 9 in 2024-05-14T16:49:10.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:10 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:49:10.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:10 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:49:10.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:10 smithi123 ceph-mon[35971]: pgmap v191: 1 pgs: 1 peering; 577 KiB data, 260 MiB used, 715 GiB / 715 GiB avail 2024-05-14T16:49:11.571 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi002:vg_nvme/lv_1 2024-05-14T16:49:11.821 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:49:11.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:11 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:49:11.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:11 smithi151 ceph-mon[36066]: osd.8 [v2:172.21.15.151:6816/4271107010,v1:172.21.15.151:6817/4271107010] boot 2024-05-14T16:49:11.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:11 smithi151 ceph-mon[36066]: osdmap e53: 9 total, 9 up, 9 in 2024-05-14T16:49:11.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:11 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:49:11.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:11 smithi151 ceph-mon[36066]: Detected new or changed devices on smithi151 2024-05-14T16:49:11.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:11 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:11.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:11 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:11.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:11 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:11.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:11 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:11.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:11 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:11.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:11 smithi151 ceph-mon[36066]: Adjusting osd_memory_target on smithi151 to 5696M 2024-05-14T16:49:11.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:11 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:11.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:11 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:49:11.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:11 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:49:11.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:11 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:11.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:11 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:49:11.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:11 smithi123 ceph-mon[35971]: osd.8 [v2:172.21.15.151:6816/4271107010,v1:172.21.15.151:6817/4271107010] boot 2024-05-14T16:49:11.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:11 smithi123 ceph-mon[35971]: osdmap e53: 9 total, 9 up, 9 in 2024-05-14T16:49:11.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:11 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:49:11.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:11 smithi123 ceph-mon[35971]: Detected new or changed devices on smithi151 2024-05-14T16:49:11.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:11 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:11.911 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:11 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:11.911 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:11 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:11.911 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:11 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:11.911 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:11 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:11.911 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:11 smithi123 ceph-mon[35971]: Adjusting osd_memory_target on smithi151 to 5696M 2024-05-14T16:49:11.911 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:11 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:11.911 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:11 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:49:11.911 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:11 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:49:11.911 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:11 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:11.944 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:11 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:49:11.944 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:11 smithi002 ceph-mon[31538]: osd.8 [v2:172.21.15.151:6816/4271107010,v1:172.21.15.151:6817/4271107010] boot 2024-05-14T16:49:11.944 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:11 smithi002 ceph-mon[31538]: osdmap e53: 9 total, 9 up, 9 in 2024-05-14T16:49:11.944 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:11 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:49:11.944 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:11 smithi002 ceph-mon[31538]: Detected new or changed devices on smithi151 2024-05-14T16:49:11.944 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:11 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:11.944 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:11 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:11.945 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:11 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:11.945 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:11 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:11.945 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:11 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:11.945 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:11 smithi002 ceph-mon[31538]: Adjusting osd_memory_target on smithi151 to 5696M 2024-05-14T16:49:11.945 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:11 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:11.945 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:11 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:49:11.945 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:11 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:49:11.945 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:11 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:12.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:12 smithi123 ceph-mon[35971]: osdmap e54: 9 total, 9 up, 9 in 2024-05-14T16:49:12.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:12 smithi123 ceph-mon[35971]: pgmap v194: 1 pgs: 1 peering; 577 KiB data, 287 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:49:12.936 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:12 smithi002 ceph-mon[31538]: osdmap e54: 9 total, 9 up, 9 in 2024-05-14T16:49:12.936 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:12 smithi002 ceph-mon[31538]: pgmap v194: 1 pgs: 1 peering; 577 KiB data, 287 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:49:13.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:12 smithi151 ceph-mon[36066]: osdmap e54: 9 total, 9 up, 9 in 2024-05-14T16:49:13.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:12 smithi151 ceph-mon[36066]: pgmap v194: 1 pgs: 1 peering; 577 KiB data, 287 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:49:13.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.484+0000 7fee843f3700 1 -- 172.21.15.2:0/3784646377 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee7c1035c0 msgr2=0x7fee7c1039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:13.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.484+0000 7fee843f3700 1 --2- 172.21.15.2:0/3784646377 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee7c1035c0 0x7fee7c1039e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fee780099c0 tx=0x7fee78009cd0 comp rx=0 tx=0).stop 2024-05-14T16:49:13.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.484+0000 7fee843f3700 1 -- 172.21.15.2:0/3784646377 shutdown_connections 2024-05-14T16:49:13.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.484+0000 7fee843f3700 1 --2- 172.21.15.2:0/3784646377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee7c105160 0x7fee7c10ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:13.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.484+0000 7fee843f3700 1 --2- 172.21.15.2:0/3784646377 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee7c1035c0 0x7fee7c1039e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:13.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.484+0000 7fee843f3700 1 --2- 172.21.15.2:0/3784646377 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee7c1047c0 0x7fee7c104c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:13.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.484+0000 7fee843f3700 1 -- 172.21.15.2:0/3784646377 >> 172.21.15.2:0/3784646377 conn(0x7fee7c0feb60 msgr2=0x7fee7c100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:13.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.484+0000 7fee843f3700 1 -- 172.21.15.2:0/3784646377 shutdown_connections 2024-05-14T16:49:13.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.484+0000 7fee843f3700 1 -- 172.21.15.2:0/3784646377 wait complete. 2024-05-14T16:49:13.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.485+0000 7fee843f3700 1 Processor -- start 2024-05-14T16:49:13.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.485+0000 7fee843f3700 1 -- start start 2024-05-14T16:49:13.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.485+0000 7fee843f3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee7c1035c0 0x7fee7c199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:13.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.485+0000 7fee843f3700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee7c1047c0 0x7fee7c199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:13.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.485+0000 7fee843f3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee7c105160 0x7fee7c1a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:13.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.485+0000 7fee843f3700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fee7c10f050 con 0x7fee7c105160 2024-05-14T16:49:13.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.485+0000 7fee843f3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fee7c10eed0 con 0x7fee7c1035c0 2024-05-14T16:49:13.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.485+0000 7fee843f3700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fee7c10f1d0 con 0x7fee7c1047c0 2024-05-14T16:49:13.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.486+0000 7fee82990700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee7c105160 0x7fee7c1a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:13.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.486+0000 7fee82990700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee7c105160 0x7fee7c1a0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53214/0 (socket says 172.21.15.2:53214) 2024-05-14T16:49:13.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.486+0000 7fee82990700 1 -- 172.21.15.2:0/4289953007 learned_addr learned my addr 172.21.15.2:0/4289953007 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:49:13.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.486+0000 7fee8198e700 1 --2- 172.21.15.2:0/4289953007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee7c1047c0 0x7fee7c199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:13.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.486+0000 7fee8218f700 1 --2- 172.21.15.2:0/4289953007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee7c1035c0 0x7fee7c199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:13.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.486+0000 7fee82990700 1 -- 172.21.15.2:0/4289953007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee7c1047c0 msgr2=0x7fee7c199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:13.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.486+0000 7fee82990700 1 --2- 172.21.15.2:0/4289953007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee7c1047c0 0x7fee7c199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:13.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.486+0000 7fee82990700 1 -- 172.21.15.2:0/4289953007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee7c1035c0 msgr2=0x7fee7c199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:13.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.486+0000 7fee82990700 1 --2- 172.21.15.2:0/4289953007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee7c1035c0 0x7fee7c199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:13.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.486+0000 7fee82990700 1 -- 172.21.15.2:0/4289953007 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee7c1a1380 con 0x7fee7c105160 2024-05-14T16:49:13.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.486+0000 7fee8218f700 1 --2- 172.21.15.2:0/4289953007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee7c1035c0 0x7fee7c199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:49:13.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.486+0000 7fee82990700 1 --2- 172.21.15.2:0/4289953007 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee7c105160 0x7fee7c1a0ce0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fee74007f10 tx=0x7fee7400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:13.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.486+0000 7fee737fe700 1 -- 172.21.15.2:0/4289953007 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee74004bb0 con 0x7fee7c105160 2024-05-14T16:49:13.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.486+0000 7fee843f3700 1 -- 172.21.15.2:0/4289953007 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee7c1a1670 con 0x7fee7c105160 2024-05-14T16:49:13.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.486+0000 7fee737fe700 1 -- 172.21.15.2:0/4289953007 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee74004d50 con 0x7fee7c105160 2024-05-14T16:49:13.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.486+0000 7fee737fe700 1 -- 172.21.15.2:0/4289953007 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee74005a30 con 0x7fee7c105160 2024-05-14T16:49:13.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.486+0000 7fee843f3700 1 -- 172.21.15.2:0/4289953007 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee7c1a1bb0 con 0x7fee7c105160 2024-05-14T16:49:13.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.487+0000 7fee843f3700 1 -- 172.21.15.2:0/4289953007 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee7c06a230 con 0x7fee7c105160 2024-05-14T16:49:13.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.489+0000 7fee737fe700 1 -- 172.21.15.2:0/4289953007 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fee7401a020 con 0x7fee7c105160 2024-05-14T16:49:13.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.490+0000 7fee737fe700 1 --2- 172.21.15.2:0/4289953007 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fee680b2880 0x7fee680b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:13.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.490+0000 7fee8218f700 1 --2- 172.21.15.2:0/4289953007 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fee680b2880 0x7fee680b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:13.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.491+0000 7fee737fe700 1 -- 172.21.15.2:0/4289953007 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5829+0+0 (secure 0 0 0) 0x7fee740e1650 con 0x7fee7c105160 2024-05-14T16:49:13.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.491+0000 7fee8218f700 1 --2- 172.21.15.2:0/4289953007 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fee680b2880 0x7fee680b4d40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fee7800b580 tx=0x7fee78005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:13.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.493+0000 7fee737fe700 1 -- 172.21.15.2:0/4289953007 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fee74010070 con 0x7fee7c105160 2024-05-14T16:49:13.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:13.718+0000 7fee843f3700 1 -- 172.21.15.2:0/4289953007 --> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fee7c063980 con 0x7fee680b2880 2024-05-14T16:49:13.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:13 smithi002 ceph-mon[31538]: osdmap e55: 9 total, 9 up, 9 in 2024-05-14T16:49:13.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:13 smithi123 ceph-mon[35971]: osdmap e55: 9 total, 9 up, 9 in 2024-05-14T16:49:14.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:13 smithi151 ceph-mon[36066]: osdmap e55: 9 total, 9 up, 9 in 2024-05-14T16:49:14.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:14 smithi123 ceph-mon[35971]: from='client.14403 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:49:14.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:14 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:49:14.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:14 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:49:14.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:14 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:49:14.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:14 smithi123 ceph-mon[35971]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 287 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:49:14.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:14 smithi002 ceph-mon[31538]: from='client.14403 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:49:14.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:14 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:49:14.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:14 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:49:14.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:14 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:49:14.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:14 smithi002 ceph-mon[31538]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 287 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:49:15.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:14 smithi151 ceph-mon[36066]: from='client.14403 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:49:15.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:14 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:49:15.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:14 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:49:15.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:14 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:49:15.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:14 smithi151 ceph-mon[36066]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 287 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:49:17.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:17 smithi123 ceph-mon[35971]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-14T16:49:17.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:17 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/1170586500' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "82b34e3b-0017-4ca4-8f86-0993ca78ce7e"}]: dispatch 2024-05-14T16:49:17.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:17 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/1170586500' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "82b34e3b-0017-4ca4-8f86-0993ca78ce7e"}]': finished 2024-05-14T16:49:17.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:17 smithi123 ceph-mon[35971]: osdmap e56: 10 total, 9 up, 10 in 2024-05-14T16:49:17.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:17 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:49:17.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:17 smithi002 ceph-mon[31538]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-14T16:49:17.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:17 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/1170586500' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "82b34e3b-0017-4ca4-8f86-0993ca78ce7e"}]: dispatch 2024-05-14T16:49:17.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:17 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/1170586500' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "82b34e3b-0017-4ca4-8f86-0993ca78ce7e"}]': finished 2024-05-14T16:49:17.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:17 smithi002 ceph-mon[31538]: osdmap e56: 10 total, 9 up, 10 in 2024-05-14T16:49:17.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:17 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:49:17.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:17 smithi151 ceph-mon[36066]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-14T16:49:17.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:17 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/1170586500' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "82b34e3b-0017-4ca4-8f86-0993ca78ce7e"}]: dispatch 2024-05-14T16:49:17.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:17 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/1170586500' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "82b34e3b-0017-4ca4-8f86-0993ca78ce7e"}]': finished 2024-05-14T16:49:17.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:17 smithi151 ceph-mon[36066]: osdmap e56: 10 total, 9 up, 10 in 2024-05-14T16:49:17.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:17 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:49:18.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:18 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/3052482351' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:49:18.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:18 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/3052482351' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:49:18.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:18 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/3052482351' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:49:19.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:19 smithi123 ceph-mon[35971]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 804 GiB / 805 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-05-14T16:49:19.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:19 smithi002 ceph-mon[31538]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 804 GiB / 805 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-05-14T16:49:19.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:19 smithi151 ceph-mon[36066]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 804 GiB / 805 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-05-14T16:49:21.409 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:21 smithi123 ceph-mon[35971]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T16:49:21.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:21 smithi002 ceph-mon[31538]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T16:49:21.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:21 smithi151 ceph-mon[36066]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T16:49:23.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:23 smithi002 ceph-mon[31538]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-14T16:49:23.407 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:23 smithi151 ceph-mon[36066]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-14T16:49:23.409 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:23 smithi123 ceph-mon[35971]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-14T16:49:25.375 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:25 smithi002 ceph-mon[31538]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:49:25.376 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:25 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-14T16:49:25.376 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:25 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:49:25.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:25 smithi123 ceph-mon[35971]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:49:25.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:25 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-14T16:49:25.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:25 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:49:25.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:25 smithi151 ceph-mon[36066]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:49:25.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:25 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-14T16:49:25.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:25 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:49:26.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:26 smithi123 ceph-mon[35971]: Deploying daemon osd.9 on smithi002 2024-05-14T16:49:26.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:26 smithi002 ceph-mon[31538]: Deploying daemon osd.9 on smithi002 2024-05-14T16:49:26.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:26 smithi151 ceph-mon[36066]: Deploying daemon osd.9 on smithi002 2024-05-14T16:49:27.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:27 smithi123 ceph-mon[35971]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:49:27.432 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:27 smithi002 ceph-mon[31538]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:49:27.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:27 smithi151 ceph-mon[36066]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:49:29.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:29 smithi123 ceph-mon[35971]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:49:29.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:29 smithi002 ceph-mon[31538]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:49:29.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:29 smithi151 ceph-mon[36066]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:49:31.242 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:31 smithi002 ceph-mon[31538]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:49:31.242 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:31 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:31.242 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:31 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:31.242 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:31 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:49:31.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:31 smithi123 ceph-mon[35971]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:49:31.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:31 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:31.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:31 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:31.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:31 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:49:31.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:31 smithi151 ceph-mon[36066]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:49:31.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:31 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:31.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:31 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:31.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:31 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:49:33.328 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:33 smithi002 ceph-mon[31538]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:49:33.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:33 smithi123 ceph-mon[35971]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:49:33.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:33 smithi151 ceph-mon[36066]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:49:34.283 INFO:teuthology.orchestra.run.smithi002.stdout:Created osd(s) 9 on host 'smithi002' 2024-05-14T16:49:34.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:34.282+0000 7fee737fe700 1 -- 172.21.15.2:0/4289953007 <== mgr.14150 v2:172.21.15.2:6800/1498936028 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fee7c063980 con 0x7fee680b2880 2024-05-14T16:49:34.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:34.285+0000 7fee843f3700 1 -- 172.21.15.2:0/4289953007 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fee680b2880 msgr2=0x7fee680b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:34.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:34.285+0000 7fee843f3700 1 --2- 172.21.15.2:0/4289953007 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fee680b2880 0x7fee680b4d40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fee7800b580 tx=0x7fee78005fb0 comp rx=0 tx=0).stop 2024-05-14T16:49:34.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:34.285+0000 7fee843f3700 1 -- 172.21.15.2:0/4289953007 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee7c105160 msgr2=0x7fee7c1a0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:34.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:34.285+0000 7fee843f3700 1 --2- 172.21.15.2:0/4289953007 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee7c105160 0x7fee7c1a0ce0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fee74007f10 tx=0x7fee7400d3b0 comp rx=0 tx=0).stop 2024-05-14T16:49:34.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:34.285+0000 7fee843f3700 1 -- 172.21.15.2:0/4289953007 shutdown_connections 2024-05-14T16:49:34.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:34.285+0000 7fee843f3700 1 --2- 172.21.15.2:0/4289953007 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fee680b2880 0x7fee680b4d40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:34.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:34.285+0000 7fee843f3700 1 --2- 172.21.15.2:0/4289953007 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee7c105160 0x7fee7c1a0ce0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:34.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:34.285+0000 7fee843f3700 1 --2- 172.21.15.2:0/4289953007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee7c1035c0 0x7fee7c199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:34.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:34.285+0000 7fee843f3700 1 --2- 172.21.15.2:0/4289953007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee7c1047c0 0x7fee7c199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:34.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:34.285+0000 7fee843f3700 1 -- 172.21.15.2:0/4289953007 >> 172.21.15.2:0/4289953007 conn(0x7fee7c0feb60 msgr2=0x7fee7c107b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:34.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:34.285+0000 7fee843f3700 1 -- 172.21.15.2:0/4289953007 shutdown_connections 2024-05-14T16:49:34.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:49:34.286+0000 7fee843f3700 1 -- 172.21.15.2:0/4289953007 wait complete. 2024-05-14T16:49:34.908 DEBUG:teuthology.orchestra.run.smithi002:osd.9> sudo journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@osd.9.service 2024-05-14T16:49:34.911 INFO:tasks.cephadm:Deploying osd.10 on smithi123 with /dev/vg_nvme/lv_1... 2024-05-14T16:49:34.912 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-14T16:49:35.161 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T16:49:35.538 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:35 smithi123 ceph-mon[35971]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:49:35.538 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:35 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:35.538 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:35 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:35.538 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:35 smithi123 ceph-mon[35971]: from='osd.9 [v2:172.21.15.2:6826/3479431021,v1:172.21.15.2:6827/3479431021]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-14T16:49:35.538 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:35 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:35.538 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:35 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:35.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:35 smithi151 ceph-mon[36066]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:49:35.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:35 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:35.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:35 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:35.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:35 smithi151 ceph-mon[36066]: from='osd.9 [v2:172.21.15.2:6826/3479431021,v1:172.21.15.2:6827/3479431021]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-14T16:49:35.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:35 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:35.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:35 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:35.637 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:35 smithi002 ceph-mon[31538]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:49:35.637 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:35 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:35.637 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:35 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:35.637 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:35 smithi002 ceph-mon[31538]: from='osd.9 [v2:172.21.15.2:6826/3479431021,v1:172.21.15.2:6827/3479431021]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-14T16:49:35.637 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:35 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:35.637 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:35 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:36.589 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:36 smithi123 ceph-mon[35971]: from='osd.9 [v2:172.21.15.2:6826/3479431021,v1:172.21.15.2:6827/3479431021]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-14T16:49:36.589 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:36 smithi123 ceph-mon[35971]: osdmap e57: 10 total, 9 up, 10 in 2024-05-14T16:49:36.589 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:36 smithi123 ceph-mon[35971]: from='osd.9 [v2:172.21.15.2:6826/3479431021,v1:172.21.15.2:6827/3479431021]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-05-14T16:49:36.590 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:36 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:49:36.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:36 smithi151 ceph-mon[36066]: from='osd.9 [v2:172.21.15.2:6826/3479431021,v1:172.21.15.2:6827/3479431021]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-14T16:49:36.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:36 smithi151 ceph-mon[36066]: osdmap e57: 10 total, 9 up, 10 in 2024-05-14T16:49:36.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:36 smithi151 ceph-mon[36066]: from='osd.9 [v2:172.21.15.2:6826/3479431021,v1:172.21.15.2:6827/3479431021]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-05-14T16:49:36.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:36 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:49:36.658 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:36 smithi002 ceph-mon[31538]: from='osd.9 [v2:172.21.15.2:6826/3479431021,v1:172.21.15.2:6827/3479431021]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-14T16:49:36.658 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:36 smithi002 ceph-mon[31538]: osdmap e57: 10 total, 9 up, 10 in 2024-05-14T16:49:36.658 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:36 smithi002 ceph-mon[31538]: from='osd.9 [v2:172.21.15.2:6826/3479431021,v1:172.21.15.2:6827/3479431021]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2024-05-14T16:49:36.658 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:36 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:49:36.659 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 16:49:36 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T16:49:36.289+0000 7fe621458700 -1 osd.9 0 waiting for initial osdmap 2024-05-14T16:49:36.659 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 16:49:36 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T16:49:36.299+0000 7fe61d24b700 -1 osd.9 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T16:49:37.181 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-14T16:49:37.181 INFO:teuthology.orchestra.run.smithi123.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-14T16:49:37.181 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10+0 records in 2024-05-14T16:49:37.181 INFO:teuthology.orchestra.run.smithi123.stderr:10+0 records out 2024-05-14T16:49:37.182 INFO:teuthology.orchestra.run.smithi123.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0115134 s, 911 MB/s 2024-05-14T16:49:37.182 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping successful for: 2024-05-14T16:49:37.488 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:37 smithi123 ceph-mon[35971]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:49:37.488 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:37 smithi123 ceph-mon[35971]: from='osd.9 [v2:172.21.15.2:6826/3479431021,v1:172.21.15.2:6827/3479431021]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-05-14T16:49:37.489 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:37 smithi123 ceph-mon[35971]: osdmap e58: 10 total, 9 up, 10 in 2024-05-14T16:49:37.489 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:37 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:49:37.489 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:37 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:49:37.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:37 smithi002 ceph-mon[31538]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:49:37.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:37 smithi002 ceph-mon[31538]: from='osd.9 [v2:172.21.15.2:6826/3479431021,v1:172.21.15.2:6827/3479431021]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-05-14T16:49:37.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:37 smithi002 ceph-mon[31538]: osdmap e58: 10 total, 9 up, 10 in 2024-05-14T16:49:37.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:37 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:49:37.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:37 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:49:37.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:37 smithi151 ceph-mon[36066]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 804 GiB / 805 GiB avail 2024-05-14T16:49:37.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:37 smithi151 ceph-mon[36066]: from='osd.9 [v2:172.21.15.2:6826/3479431021,v1:172.21.15.2:6827/3479431021]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2024-05-14T16:49:37.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:37 smithi151 ceph-mon[36066]: osdmap e58: 10 total, 9 up, 10 in 2024-05-14T16:49:37.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:37 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:49:37.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:37 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:49:37.896 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi123:vg_nvme/lv_1 2024-05-14T16:49:38.138 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T16:49:38.529 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:38 smithi123 ceph-mon[35971]: purged_snaps scrub starts 2024-05-14T16:49:38.529 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:38 smithi123 ceph-mon[35971]: purged_snaps scrub ok 2024-05-14T16:49:38.529 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:38 smithi123 ceph-mon[35971]: osd.9 [v2:172.21.15.2:6826/3479431021,v1:172.21.15.2:6827/3479431021] boot 2024-05-14T16:49:38.529 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:38 smithi123 ceph-mon[35971]: osdmap e59: 10 total, 10 up, 10 in 2024-05-14T16:49:38.529 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:38 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:49:38.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:38 smithi151 ceph-mon[36066]: purged_snaps scrub starts 2024-05-14T16:49:38.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:38 smithi151 ceph-mon[36066]: purged_snaps scrub ok 2024-05-14T16:49:38.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:38 smithi151 ceph-mon[36066]: osd.9 [v2:172.21.15.2:6826/3479431021,v1:172.21.15.2:6827/3479431021] boot 2024-05-14T16:49:38.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:38 smithi151 ceph-mon[36066]: osdmap e59: 10 total, 10 up, 10 in 2024-05-14T16:49:38.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:38 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:49:38.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:38 smithi002 ceph-mon[31538]: purged_snaps scrub starts 2024-05-14T16:49:38.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:38 smithi002 ceph-mon[31538]: purged_snaps scrub ok 2024-05-14T16:49:38.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:38 smithi002 ceph-mon[31538]: osd.9 [v2:172.21.15.2:6826/3479431021,v1:172.21.15.2:6827/3479431021] boot 2024-05-14T16:49:38.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:38 smithi002 ceph-mon[31538]: osdmap e59: 10 total, 10 up, 10 in 2024-05-14T16:49:38.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:38 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:49:39.587 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:39 smithi123 ceph-mon[35971]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:49:39.587 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:39 smithi123 ceph-mon[35971]: osdmap e60: 10 total, 10 up, 10 in 2024-05-14T16:49:39.587 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:39 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:39.587 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:39 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:39.587 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:39 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:39.588 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:39 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:39.588 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:39 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:39.588 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:39 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:39.588 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:39 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:39.588 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:39 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:49:39.588 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:39 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:49:39.588 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:39 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:39.588 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:39 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:49:39.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:39 smithi151 ceph-mon[36066]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:49:39.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:39 smithi151 ceph-mon[36066]: osdmap e60: 10 total, 10 up, 10 in 2024-05-14T16:49:39.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:39 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:39.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:39 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:39.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:39 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:39.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:39 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:39.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:39 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:39.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:39 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:39.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:39 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:39.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:39 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:49:39.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:39 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:49:39.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:39 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:39.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:39 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:49:39.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:39 smithi002 ceph-mon[31538]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:49:39.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:39 smithi002 ceph-mon[31538]: osdmap e60: 10 total, 10 up, 10 in 2024-05-14T16:49:39.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:39 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:39.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:39 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:39.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:39 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:39.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:39 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:39.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:39 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:39.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:39 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:49:39.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:39 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:39.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:39 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:49:39.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:39 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:49:39.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:39 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:39.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:39 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:49:39.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:49:39.932+0000 7f799ac0a700 1 -- 172.21.15.123:0/28514201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7994109290 msgr2=0x7f799410b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:39.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:49:39.932+0000 7f799ac0a700 1 --2- 172.21.15.123:0/28514201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7994109290 0x7f799410b700 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f798c00b0a0 tx=0x7f798c00b3b0 comp rx=0 tx=0).stop 2024-05-14T16:49:39.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:49:39.932+0000 7f799ac0a700 1 -- 172.21.15.123:0/28514201 shutdown_connections 2024-05-14T16:49:39.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:49:39.932+0000 7f799ac0a700 1 --2- 172.21.15.123:0/28514201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7994109290 0x7f799410b700 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:39.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:49:39.932+0000 7f799ac0a700 1 --2- 172.21.15.123:0/28514201 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7994104030 0x7f7994106420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:39.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:49:39.932+0000 7f799ac0a700 1 --2- 172.21.15.123:0/28514201 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7994106960 0x7f7994108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:39.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:49:39.932+0000 7f799ac0a700 1 -- 172.21.15.123:0/28514201 >> 172.21.15.123:0/28514201 conn(0x7f79940fda40 msgr2=0x7f79940ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:49:39.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:49:39.932+0000 7f799ac0a700 1 -- 172.21.15.123:0/28514201 shutdown_connections 2024-05-14T16:49:39.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:49:39.932+0000 7f799ac0a700 1 -- 172.21.15.123:0/28514201 wait complete. 2024-05-14T16:49:39.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:49:39.933+0000 7f799ac0a700 1 Processor -- start 2024-05-14T16:49:39.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:49:39.933+0000 7f799ac0a700 1 -- start start 2024-05-14T16:49:39.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:49:39.933+0000 7f799ac0a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7994104030 0x7f79941996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:39.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:49:39.933+0000 7f799ac0a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7994106960 0x7f7994199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:39.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:49:39.933+0000 7f799ac0a700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7994109290 0x7f79941a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:39.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:49:39.933+0000 7f799ac0a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f799410f0a0 con 0x7f7994106960 2024-05-14T16:49:39.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:49:39.933+0000 7f799ac0a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f799410ef20 con 0x7f7994104030 2024-05-14T16:49:39.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:49:39.933+0000 7f799ac0a700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f799410f220 con 0x7f7994109290 2024-05-14T16:49:39.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:49:39.933+0000 7f79989a6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7994104030 0x7f79941996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:39.934 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:49:39.933+0000 7f79989a6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7994104030 0x7f79941996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40396/0 (socket says 172.21.15.123:40396) 2024-05-14T16:49:39.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:49:39.933+0000 7f79989a6700 1 -- 172.21.15.123:0/1073971626 learned_addr learned my addr 172.21.15.123:0/1073971626 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-05-14T16:49:39.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:49:39.933+0000 7f79989a6700 1 -- 172.21.15.123:0/1073971626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7994109290 msgr2=0x7f79941a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:39.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:49:39.933+0000 7f79989a6700 1 --2- 172.21.15.123:0/1073971626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7994109290 0x7f79941a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:39.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:49:39.933+0000 7f79989a6700 1 -- 172.21.15.123:0/1073971626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7994106960 msgr2=0x7f7994199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:49:39.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:49:39.933+0000 7f79989a6700 1 --2- 172.21.15.123:0/1073971626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7994106960 0x7f7994199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:49:39.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:49:39.933+0000 7f79989a6700 1 -- 172.21.15.123:0/1073971626 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79941a1350 con 0x7f7994104030 2024-05-14T16:49:39.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:49:39.933+0000 7f79989a6700 1 --2- 172.21.15.123:0/1073971626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7994104030 0x7f79941996f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f799000d980 tx=0x7f799000dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:39.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:49:39.933+0000 7f7989ffb700 1 -- 172.21.15.123:0/1073971626 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7990017070 con 0x7f7994104030 2024-05-14T16:49:39.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:49:39.933+0000 7f7989ffb700 1 -- 172.21.15.123:0/1073971626 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7990004830 con 0x7f7994104030 2024-05-14T16:49:39.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:49:39.934+0000 7f7989ffb700 1 -- 172.21.15.123:0/1073971626 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7990005440 con 0x7f7994104030 2024-05-14T16:49:39.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:49:39.934+0000 7f799ac0a700 1 -- 172.21.15.123:0/1073971626 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79941a1640 con 0x7f7994104030 2024-05-14T16:49:39.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:49:39.934+0000 7f799ac0a700 1 -- 172.21.15.123:0/1073971626 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79941a1bb0 con 0x7f7994104030 2024-05-14T16:49:39.935 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:49:39.935+0000 7f799ac0a700 1 -- 172.21.15.123:0/1073971626 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f799406a230 con 0x7f7994104030 2024-05-14T16:49:39.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:49:39.936+0000 7f7989ffb700 1 -- 172.21.15.123:0/1073971626 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f79900055e0 con 0x7f7994104030 2024-05-14T16:49:39.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:49:39.936+0000 7f7989ffb700 1 --2- 172.21.15.123:0/1073971626 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f797c0b2880 0x7f797c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:49:39.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:49:39.937+0000 7f7989ffb700 1 -- 172.21.15.123:0/1073971626 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6261+0+0 (secure 0 0 0) 0x7f79900e1db0 con 0x7f7994104030 2024-05-14T16:49:39.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:49:39.937+0000 7f798bfff700 1 --2- 172.21.15.123:0/1073971626 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f797c0b2880 0x7f797c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:49:39.936 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:49:39.937+0000 7f798bfff700 1 --2- 172.21.15.123:0/1073971626 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f797c0b2880 0x7f797c0b4d40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f79840097b0 tx=0x7f7984006d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:49:39.939 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:49:39.940+0000 7f7989ffb700 1 -- 172.21.15.123:0/1073971626 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7990020020 con 0x7f7994104030 2024-05-14T16:49:40.151 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:49:40.152+0000 7f799ac0a700 1 -- 172.21.15.123:0/1073971626 --> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f7994063980 con 0x7f797c0b2880 2024-05-14T16:49:40.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:40 smithi123 ceph-mon[35971]: Detected new or changed devices on smithi002 2024-05-14T16:49:40.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:40 smithi123 ceph-mon[35971]: Adjusting osd_memory_target on smithi002 to 4273M 2024-05-14T16:49:40.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:40 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:49:40.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:40 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:49:40.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:40 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:40.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:40 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:49:40.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:40 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:49:40.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:40 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:49:40.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:40 smithi151 ceph-mon[36066]: Detected new or changed devices on smithi002 2024-05-14T16:49:40.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:40 smithi151 ceph-mon[36066]: Adjusting osd_memory_target on smithi002 to 4273M 2024-05-14T16:49:40.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:40 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:49:40.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:40 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:49:40.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:40 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:40.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:40 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:49:40.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:40 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:49:40.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:40 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:49:40.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:40 smithi002 ceph-mon[31538]: Detected new or changed devices on smithi002 2024-05-14T16:49:40.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:40 smithi002 ceph-mon[31538]: Adjusting osd_memory_target on smithi002 to 4273M 2024-05-14T16:49:40.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:40 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:49:40.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:40 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:49:40.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:40 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:40.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:40 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:49:40.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:40 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:49:40.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:40 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:49:41.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:41 smithi123 ceph-mon[35971]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:49:41.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:41 smithi123 ceph-mon[35971]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:49:41.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:41 smithi151 ceph-mon[36066]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:49:41.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:41 smithi151 ceph-mon[36066]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:49:41.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:41 smithi002 ceph-mon[31538]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:49:41.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:41 smithi002 ceph-mon[31538]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:49:43.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:43 smithi151 ceph-mon[36066]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:49:43.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:43 smithi151 ceph-mon[36066]: from='client.? 172.21.15.123:0/2499901524' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32e2ace9-eb5d-4258-a2ac-49b782be2b5c"}]: dispatch 2024-05-14T16:49:43.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:43 smithi151 ceph-mon[36066]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32e2ace9-eb5d-4258-a2ac-49b782be2b5c"}]: dispatch 2024-05-14T16:49:43.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:43 smithi151 ceph-mon[36066]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32e2ace9-eb5d-4258-a2ac-49b782be2b5c"}]': finished 2024-05-14T16:49:43.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:43 smithi151 ceph-mon[36066]: osdmap e61: 11 total, 10 up, 11 in 2024-05-14T16:49:43.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:43 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:49:43.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:43 smithi123 ceph-mon[35971]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:49:43.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:43 smithi123 ceph-mon[35971]: from='client.? 172.21.15.123:0/2499901524' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32e2ace9-eb5d-4258-a2ac-49b782be2b5c"}]: dispatch 2024-05-14T16:49:43.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:43 smithi123 ceph-mon[35971]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32e2ace9-eb5d-4258-a2ac-49b782be2b5c"}]: dispatch 2024-05-14T16:49:43.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:43 smithi123 ceph-mon[35971]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32e2ace9-eb5d-4258-a2ac-49b782be2b5c"}]': finished 2024-05-14T16:49:43.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:43 smithi123 ceph-mon[35971]: osdmap e61: 11 total, 10 up, 11 in 2024-05-14T16:49:43.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:43 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:49:43.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:43 smithi002 ceph-mon[31538]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:49:43.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:43 smithi002 ceph-mon[31538]: from='client.? 172.21.15.123:0/2499901524' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32e2ace9-eb5d-4258-a2ac-49b782be2b5c"}]: dispatch 2024-05-14T16:49:43.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:43 smithi002 ceph-mon[31538]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32e2ace9-eb5d-4258-a2ac-49b782be2b5c"}]: dispatch 2024-05-14T16:49:43.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:43 smithi002 ceph-mon[31538]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32e2ace9-eb5d-4258-a2ac-49b782be2b5c"}]': finished 2024-05-14T16:49:43.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:43 smithi002 ceph-mon[31538]: osdmap e61: 11 total, 10 up, 11 in 2024-05-14T16:49:43.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:43 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:49:44.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:44 smithi151 ceph-mon[36066]: from='client.? 172.21.15.123:0/622453809' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:49:44.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:44 smithi123 ceph-mon[35971]: from='client.? 172.21.15.123:0/622453809' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:49:44.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:44 smithi002 ceph-mon[31538]: from='client.? 172.21.15.123:0/622453809' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:49:45.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:45 smithi151 ceph-mon[36066]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:49:45.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:45 smithi123 ceph-mon[35971]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:49:45.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:45 smithi002 ceph-mon[31538]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:49:47.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:47 smithi123 ceph-mon[35971]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:49:47.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:47 smithi151 ceph-mon[36066]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:49:47.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:47 smithi002 ceph-mon[31538]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:49:49.405 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:49 smithi123 ceph-mon[35971]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:49:49.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:49 smithi151 ceph-mon[36066]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:49:49.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:49 smithi002 ceph-mon[31538]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:49:51.464 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:51 smithi123 ceph-mon[35971]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:49:51.464 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:51 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-14T16:49:51.464 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:51 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:49:51.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:51 smithi151 ceph-mon[36066]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:49:51.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:51 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-14T16:49:51.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:51 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:49:51.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:51 smithi002 ceph-mon[31538]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:49:51.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:51 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-14T16:49:51.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:51 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:49:52.544 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:52 smithi123 ceph-mon[35971]: Deploying daemon osd.10 on smithi123 2024-05-14T16:49:52.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:52 smithi151 ceph-mon[36066]: Deploying daemon osd.10 on smithi123 2024-05-14T16:49:52.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:52 smithi002 ceph-mon[31538]: Deploying daemon osd.10 on smithi123 2024-05-14T16:49:53.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:53 smithi151 ceph-mon[36066]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:49:53.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:53 smithi123 ceph-mon[35971]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:49:53.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:53 smithi002 ceph-mon[31538]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:49:55.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:55 smithi123 ceph-mon[35971]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:49:55.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:55 smithi151 ceph-mon[36066]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:49:55.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:55 smithi002 ceph-mon[31538]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:49:57.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:57 smithi123 ceph-mon[35971]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:49:57.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:57 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:57.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:57 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:49:57.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:57 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:57.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:57 smithi002 ceph-mon[31538]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:49:57.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:57 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:57.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:57 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:49:57.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:57 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:57.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:57 smithi151 ceph-mon[36066]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:49:57.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:57 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:57.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:57 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:49:57.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:57 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:49:59.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:49:59 smithi123 ceph-mon[35971]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:49:59.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:49:59 smithi002 ceph-mon[31538]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:49:59.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:49:59 smithi151 ceph-mon[36066]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:50:00.423 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:50:00.422+0000 7f7989ffb700 1 -- 172.21.15.123:0/1073971626 <== mgr.14150 v2:172.21.15.2:6800/1498936028 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f7994063980 con 0x7f797c0b2880 2024-05-14T16:50:00.424 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 10 on host 'smithi123' 2024-05-14T16:50:00.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:50:00.425+0000 7f799ac0a700 1 -- 172.21.15.123:0/1073971626 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f797c0b2880 msgr2=0x7f797c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:00.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:50:00.425+0000 7f799ac0a700 1 --2- 172.21.15.123:0/1073971626 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f797c0b2880 0x7f797c0b4d40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f79840097b0 tx=0x7f7984006d20 comp rx=0 tx=0).stop 2024-05-14T16:50:00.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:50:00.425+0000 7f799ac0a700 1 -- 172.21.15.123:0/1073971626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7994104030 msgr2=0x7f79941996f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:00.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:50:00.425+0000 7f799ac0a700 1 --2- 172.21.15.123:0/1073971626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7994104030 0x7f79941996f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f799000d980 tx=0x7f799000dd40 comp rx=0 tx=0).stop 2024-05-14T16:50:00.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:50:00.425+0000 7f799ac0a700 1 -- 172.21.15.123:0/1073971626 shutdown_connections 2024-05-14T16:50:00.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:50:00.425+0000 7f799ac0a700 1 --2- 172.21.15.123:0/1073971626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7994109290 0x7f79941a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:00.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:50:00.425+0000 7f799ac0a700 1 --2- 172.21.15.123:0/1073971626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7994104030 0x7f79941996f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:00.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:50:00.425+0000 7f799ac0a700 1 --2- 172.21.15.123:0/1073971626 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f797c0b2880 0x7f797c0b4d40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:00.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:50:00.425+0000 7f799ac0a700 1 --2- 172.21.15.123:0/1073971626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7994106960 0x7f7994199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:00.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:50:00.425+0000 7f799ac0a700 1 -- 172.21.15.123:0/1073971626 >> 172.21.15.123:0/1073971626 conn(0x7f79940fda40 msgr2=0x7f7994104970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:00.426 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:50:00.425+0000 7f799ac0a700 1 -- 172.21.15.123:0/1073971626 shutdown_connections 2024-05-14T16:50:00.427 INFO:teuthology.orchestra.run.smithi123.stderr:2024-05-14T16:50:00.425+0000 7f799ac0a700 1 -- 172.21.15.123:0/1073971626 wait complete. 2024-05-14T16:50:00.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:00 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:00.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:00 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:00.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:00 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:00.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:00 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:50:00.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:00 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:00.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:00 smithi151 ceph-mon[36066]: overall HEALTH_OK 2024-05-14T16:50:00.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:00 smithi151 ceph-mon[36066]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:50:00.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:00 smithi151 ceph-mon[36066]: from='osd.10 [v2:172.21.15.123:6824/4266374435,v1:172.21.15.123:6825/4266374435]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-14T16:50:00.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:00 smithi151 ceph-mon[36066]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-14T16:50:00.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:00 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:00.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:00 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:00.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:00 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:50:00.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:00 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:00.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:00 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:00.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:00 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:00.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:00 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:50:00.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:00 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:00.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:00 smithi123 ceph-mon[35971]: overall HEALTH_OK 2024-05-14T16:50:00.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:00 smithi123 ceph-mon[35971]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:50:00.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:00 smithi123 ceph-mon[35971]: from='osd.10 [v2:172.21.15.123:6824/4266374435,v1:172.21.15.123:6825/4266374435]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-14T16:50:00.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:00 smithi123 ceph-mon[35971]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-14T16:50:00.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:00 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:00.911 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:00 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:00.911 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:00 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:50:00.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:00 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:00.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:00 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:00.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:00 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:00.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:00 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:50:00.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:00 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:00.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:00 smithi002 ceph-mon[31538]: overall HEALTH_OK 2024-05-14T16:50:00.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:00 smithi002 ceph-mon[31538]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:50:00.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:00 smithi002 ceph-mon[31538]: from='osd.10 [v2:172.21.15.123:6824/4266374435,v1:172.21.15.123:6825/4266374435]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-14T16:50:00.995 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:00 smithi002 ceph-mon[31538]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-14T16:50:00.995 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:00 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:00.995 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:00 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:00.995 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:00 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:50:01.106 DEBUG:teuthology.orchestra.run.smithi123:osd.10> sudo journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@osd.10.service 2024-05-14T16:50:01.108 INFO:tasks.cephadm:Deploying osd.11 on smithi151 with /dev/vg_nvme/lv_1... 2024-05-14T16:50:01.108 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-14T16:50:01.357 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T16:50:01.733 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:01 smithi151 ceph-mon[36066]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-14T16:50:01.733 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:01 smithi151 ceph-mon[36066]: osdmap e62: 11 total, 10 up, 11 in 2024-05-14T16:50:01.733 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:01 smithi151 ceph-mon[36066]: from='osd.10 [v2:172.21.15.123:6824/4266374435,v1:172.21.15.123:6825/4266374435]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-14T16:50:01.733 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:01 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:50:01.733 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:01 smithi151 ceph-mon[36066]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-14T16:50:01.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:01 smithi123 ceph-mon[35971]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-14T16:50:01.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:01 smithi123 ceph-mon[35971]: osdmap e62: 11 total, 10 up, 11 in 2024-05-14T16:50:01.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:01 smithi123 ceph-mon[35971]: from='osd.10 [v2:172.21.15.123:6824/4266374435,v1:172.21.15.123:6825/4266374435]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-14T16:50:01.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:01 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:50:01.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:01 smithi123 ceph-mon[35971]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-14T16:50:01.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:01 smithi002 ceph-mon[31538]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-14T16:50:01.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:01 smithi002 ceph-mon[31538]: osdmap e62: 11 total, 10 up, 11 in 2024-05-14T16:50:01.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:01 smithi002 ceph-mon[31538]: from='osd.10 [v2:172.21.15.123:6824/4266374435,v1:172.21.15.123:6825/4266374435]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-14T16:50:01.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:01 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:50:01.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:01 smithi002 ceph-mon[31538]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-05-14T16:50:02.310 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 16:50:02 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T16:50:02.049+0000 7f34fa2f9700 -1 osd.10 0 waiting for initial osdmap 2024-05-14T16:50:02.310 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 16:50:02 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T16:50:02.059+0000 7f34f20e4700 -1 osd.10 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T16:50:02.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:02 smithi151 ceph-mon[36066]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-05-14T16:50:02.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:02 smithi151 ceph-mon[36066]: osdmap e63: 11 total, 10 up, 11 in 2024-05-14T16:50:02.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:02 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:50:02.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:02 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:50:02.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:02 smithi151 ceph-mon[36066]: from='osd.10 ' entity='osd.10' 2024-05-14T16:50:02.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:02 smithi151 ceph-mon[36066]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:50:02.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:02 smithi123 ceph-mon[35971]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-05-14T16:50:02.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:02 smithi123 ceph-mon[35971]: osdmap e63: 11 total, 10 up, 11 in 2024-05-14T16:50:02.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:02 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:50:02.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:02 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:50:02.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:02 smithi123 ceph-mon[35971]: from='osd.10 ' entity='osd.10' 2024-05-14T16:50:02.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:02 smithi123 ceph-mon[35971]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:50:02.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:02 smithi002 ceph-mon[31538]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-05-14T16:50:02.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:02 smithi002 ceph-mon[31538]: osdmap e63: 11 total, 10 up, 11 in 2024-05-14T16:50:02.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:02 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:50:02.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:02 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:50:02.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:02 smithi002 ceph-mon[31538]: from='osd.10 ' entity='osd.10' 2024-05-14T16:50:02.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:02 smithi002 ceph-mon[31538]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 894 GiB / 894 GiB avail 2024-05-14T16:50:03.433 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-14T16:50:03.433 INFO:teuthology.orchestra.run.smithi151.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-14T16:50:03.433 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10+0 records in 2024-05-14T16:50:03.433 INFO:teuthology.orchestra.run.smithi151.stderr:10+0 records out 2024-05-14T16:50:03.433 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114654 s, 915 MB/s 2024-05-14T16:50:03.433 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping successful for: 2024-05-14T16:50:03.631 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:03 smithi151 ceph-mon[36066]: purged_snaps scrub starts 2024-05-14T16:50:03.631 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:03 smithi151 ceph-mon[36066]: purged_snaps scrub ok 2024-05-14T16:50:03.631 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:03 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:50:03.631 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:03 smithi151 ceph-mon[36066]: osd.10 [v2:172.21.15.123:6824/4266374435,v1:172.21.15.123:6825/4266374435] boot 2024-05-14T16:50:03.631 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:03 smithi151 ceph-mon[36066]: osdmap e64: 11 total, 11 up, 11 in 2024-05-14T16:50:03.631 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:03 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:50:03.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:03 smithi123 ceph-mon[35971]: purged_snaps scrub starts 2024-05-14T16:50:03.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:03 smithi123 ceph-mon[35971]: purged_snaps scrub ok 2024-05-14T16:50:03.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:03 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:50:03.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:03 smithi123 ceph-mon[35971]: osd.10 [v2:172.21.15.123:6824/4266374435,v1:172.21.15.123:6825/4266374435] boot 2024-05-14T16:50:03.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:03 smithi123 ceph-mon[35971]: osdmap e64: 11 total, 11 up, 11 in 2024-05-14T16:50:03.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:03 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:50:03.895 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:03 smithi002 ceph-mon[31538]: purged_snaps scrub starts 2024-05-14T16:50:03.895 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:03 smithi002 ceph-mon[31538]: purged_snaps scrub ok 2024-05-14T16:50:03.895 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:03 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:50:03.895 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:03 smithi002 ceph-mon[31538]: osd.10 [v2:172.21.15.123:6824/4266374435,v1:172.21.15.123:6825/4266374435] boot 2024-05-14T16:50:03.895 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:03 smithi002 ceph-mon[31538]: osdmap e64: 11 total, 11 up, 11 in 2024-05-14T16:50:03.896 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:03 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:50:04.169 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi151:vg_nvme/lv_1 2024-05-14T16:50:04.411 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T16:50:05.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:05 smithi151 ceph-mon[36066]: osdmap e65: 11 total, 11 up, 11 in 2024-05-14T16:50:05.240 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:05 smithi151 ceph-mon[36066]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:50:05.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:05 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:05.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:05 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:05.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:05 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:50:05.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:05 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:50:05.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:05 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:50:05.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:05 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:50:05.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:05 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:05.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:05 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:05.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:05 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:50:05.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:05 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:05.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:05 smithi123 ceph-mon[35971]: osdmap e65: 11 total, 11 up, 11 in 2024-05-14T16:50:05.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:05 smithi123 ceph-mon[35971]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:50:05.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:05 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:05.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:05 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:05.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:05 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:50:05.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:05 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:50:05.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:05 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:50:05.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:05 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:50:05.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:05 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:05.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:05 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:05.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:05 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:50:05.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:05 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:05.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:05 smithi002 ceph-mon[31538]: osdmap e65: 11 total, 11 up, 11 in 2024-05-14T16:50:05.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:05 smithi002 ceph-mon[31538]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:50:05.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:05 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:05.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:05 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:05.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:05 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:50:05.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:05 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:50:05.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:05 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:50:05.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:05 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:50:05.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:05 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:05.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:05 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:05.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:05 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:50:05.495 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:05 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:06.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:06.133+0000 7f6ea6d9d700 1 -- 172.21.15.151:0/4083501684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ea01034c0 msgr2=0x7f6ea01038e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:06.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:06.133+0000 7f6ea6d9d700 1 --2- 172.21.15.151:0/4083501684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ea01034c0 0x7f6ea01038e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c009990 tx=0x7f6e9c009ca0 comp rx=0 tx=0).stop 2024-05-14T16:50:06.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:06.133+0000 7f6ea6d9d700 1 -- 172.21.15.151:0/4083501684 shutdown_connections 2024-05-14T16:50:06.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:06.133+0000 7f6ea6d9d700 1 --2- 172.21.15.151:0/4083501684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ea0105060 0x7f6ea010b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:06.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:06.133+0000 7f6ea6d9d700 1 --2- 172.21.15.151:0/4083501684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ea01034c0 0x7f6ea01038e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:06.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:06.133+0000 7f6ea6d9d700 1 --2- 172.21.15.151:0/4083501684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ea01046c0 0x7f6ea0104b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:06.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:06.133+0000 7f6ea6d9d700 1 -- 172.21.15.151:0/4083501684 >> 172.21.15.151:0/4083501684 conn(0x7f6ea00fea80 msgr2=0x7f6ea0100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:06.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:06.134+0000 7f6ea6d9d700 1 -- 172.21.15.151:0/4083501684 shutdown_connections 2024-05-14T16:50:06.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:06.134+0000 7f6ea6d9d700 1 -- 172.21.15.151:0/4083501684 wait complete. 2024-05-14T16:50:06.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:06.134+0000 7f6ea6d9d700 1 Processor -- start 2024-05-14T16:50:06.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:06.134+0000 7f6ea6d9d700 1 -- start start 2024-05-14T16:50:06.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:06.135+0000 7f6ea6d9d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ea01034c0 0x7f6ea0199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:06.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:06.135+0000 7f6ea6d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ea01046c0 0x7f6ea0199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:06.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:06.135+0000 7f6ea6d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ea0105060 0x7f6ea01a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:06.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:06.135+0000 7f6ea6d9d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea010f030 con 0x7f6ea0105060 2024-05-14T16:50:06.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:06.135+0000 7f6ea6d9d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea010eeb0 con 0x7f6ea01046c0 2024-05-14T16:50:06.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:06.135+0000 7f6ea6d9d700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea010f1b0 con 0x7f6ea01034c0 2024-05-14T16:50:06.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:06.135+0000 7f6ea5d9b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ea01034c0 0x7f6ea0199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:06.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:06.135+0000 7f6ea5d9b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ea01034c0 0x7f6ea0199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:34590/0 (socket says 172.21.15.151:34590) 2024-05-14T16:50:06.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:06.135+0000 7f6ea5d9b700 1 -- 172.21.15.151:0/3292297743 learned_addr learned my addr 172.21.15.151:0/3292297743 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-14T16:50:06.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:06.135+0000 7f6ea5d9b700 1 -- 172.21.15.151:0/3292297743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ea01046c0 msgr2=0x7f6ea0199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:06.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:06.135+0000 7f6ea5d9b700 1 --2- 172.21.15.151:0/3292297743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ea01046c0 0x7f6ea0199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:06.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:06.135+0000 7f6ea5d9b700 1 -- 172.21.15.151:0/3292297743 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ea0105060 msgr2=0x7f6ea01a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:06.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:06.135+0000 7f6ea5d9b700 1 --2- 172.21.15.151:0/3292297743 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ea0105060 0x7f6ea01a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:06.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:06.135+0000 7f6ea5d9b700 1 -- 172.21.15.151:0/3292297743 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea01a1310 con 0x7f6ea01034c0 2024-05-14T16:50:06.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:06.135+0000 7f6ea5d9b700 1 --2- 172.21.15.151:0/3292297743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ea01034c0 0x7f6ea0199700 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c0047b0 tx=0x7f6e9c009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:06.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:06.135+0000 7f6e96ffd700 1 -- 172.21.15.151:0/3292297743 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c026070 con 0x7f6ea01034c0 2024-05-14T16:50:06.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:06.135+0000 7f6e96ffd700 1 -- 172.21.15.151:0/3292297743 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e9c0047e0 con 0x7f6ea01034c0 2024-05-14T16:50:06.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:06.135+0000 7f6ea6d9d700 1 -- 172.21.15.151:0/3292297743 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ea01a15a0 con 0x7f6ea01034c0 2024-05-14T16:50:06.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:06.135+0000 7f6e96ffd700 1 -- 172.21.15.151:0/3292297743 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c005270 con 0x7f6ea01034c0 2024-05-14T16:50:06.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:06.135+0000 7f6ea6d9d700 1 -- 172.21.15.151:0/3292297743 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ea01a1a50 con 0x7f6ea01034c0 2024-05-14T16:50:06.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:06.136+0000 7f6ea6d9d700 1 -- 172.21.15.151:0/3292297743 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ea006a230 con 0x7f6ea01034c0 2024-05-14T16:50:06.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:06.138+0000 7f6e96ffd700 1 -- 172.21.15.151:0/3292297743 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f6e9c029070 con 0x7f6ea01034c0 2024-05-14T16:50:06.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:06.139+0000 7f6e96ffd700 1 --2- 172.21.15.151:0/3292297743 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f6e8c0b2900 0x7f6e8c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:06.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:06.139+0000 7f6e96ffd700 1 -- 172.21.15.151:0/3292297743 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6694+0+0 (secure 0 0 0) 0x7f6e9c0e2b10 con 0x7f6ea01034c0 2024-05-14T16:50:06.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:06.139+0000 7f6ea559a700 1 --2- 172.21.15.151:0/3292297743 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f6e8c0b2900 0x7f6e8c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:06.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:06.140+0000 7f6ea559a700 1 --2- 172.21.15.151:0/3292297743 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f6e8c0b2900 0x7f6e8c0b4dc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6e900097b0 tx=0x7f6e90006d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:06.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:06.143+0000 7f6e96ffd700 1 -- 172.21.15.151:0/3292297743 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6e9c030020 con 0x7f6ea01034c0 2024-05-14T16:50:06.159 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:06 smithi123 ceph-mon[35971]: Detected new or changed devices on smithi123 2024-05-14T16:50:06.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:06 smithi123 ceph-mon[35971]: Adjusting osd_memory_target on smithi123 to 4272M 2024-05-14T16:50:06.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:06 smithi123 ceph-mon[35971]: osdmap e66: 11 total, 11 up, 11 in 2024-05-14T16:50:06.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:06.352+0000 7f6ea6d9d700 1 -- 172.21.15.151:0/3292297743 --> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f6ea0063980 con 0x7f6e8c0b2900 2024-05-14T16:50:06.352 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:06 smithi151 ceph-mon[36066]: Detected new or changed devices on smithi123 2024-05-14T16:50:06.352 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:06 smithi151 ceph-mon[36066]: Adjusting osd_memory_target on smithi123 to 4272M 2024-05-14T16:50:06.352 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:06 smithi151 ceph-mon[36066]: osdmap e66: 11 total, 11 up, 11 in 2024-05-14T16:50:06.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:06 smithi002 ceph-mon[31538]: Detected new or changed devices on smithi123 2024-05-14T16:50:06.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:06 smithi002 ceph-mon[31538]: Adjusting osd_memory_target on smithi123 to 4272M 2024-05-14T16:50:06.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:06 smithi002 ceph-mon[31538]: osdmap e66: 11 total, 11 up, 11 in 2024-05-14T16:50:07.253 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:07 smithi151 ceph-mon[36066]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:50:07.253 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:07 smithi151 ceph-mon[36066]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:50:07.254 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:07 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:50:07.254 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:07 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:50:07.254 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:07 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:07.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:07 smithi123 ceph-mon[35971]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:50:07.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:07 smithi123 ceph-mon[35971]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:50:07.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:07 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:50:07.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:07 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:50:07.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:07 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:07.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:07 smithi002 ceph-mon[31538]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:50:07.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:07 smithi002 ceph-mon[31538]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:50:07.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:07 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-14T16:50:07.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:07 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-14T16:50:07.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:07 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:09.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:09 smithi123 ceph-mon[35971]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:50:09.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:09 smithi002 ceph-mon[31538]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:50:09.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:09 smithi151 ceph-mon[36066]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:50:10.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:10 smithi123 ceph-mon[35971]: from='client.? 172.21.15.151:0/983073879' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e3c2ead-1af6-4bda-87cd-12770c81fa67"}]: dispatch 2024-05-14T16:50:10.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:10 smithi123 ceph-mon[35971]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e3c2ead-1af6-4bda-87cd-12770c81fa67"}]: dispatch 2024-05-14T16:50:10.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:10 smithi123 ceph-mon[35971]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6e3c2ead-1af6-4bda-87cd-12770c81fa67"}]': finished 2024-05-14T16:50:10.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:10 smithi123 ceph-mon[35971]: osdmap e67: 12 total, 11 up, 12 in 2024-05-14T16:50:10.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:10 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:50:10.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:10 smithi123 ceph-mon[35971]: from='client.? 172.21.15.151:0/2246027065' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:50:10.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:10 smithi002 ceph-mon[31538]: from='client.? 172.21.15.151:0/983073879' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e3c2ead-1af6-4bda-87cd-12770c81fa67"}]: dispatch 2024-05-14T16:50:10.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:10 smithi002 ceph-mon[31538]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e3c2ead-1af6-4bda-87cd-12770c81fa67"}]: dispatch 2024-05-14T16:50:10.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:10 smithi002 ceph-mon[31538]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6e3c2ead-1af6-4bda-87cd-12770c81fa67"}]': finished 2024-05-14T16:50:10.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:10 smithi002 ceph-mon[31538]: osdmap e67: 12 total, 11 up, 12 in 2024-05-14T16:50:10.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:10 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:50:10.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:10 smithi002 ceph-mon[31538]: from='client.? 172.21.15.151:0/2246027065' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:50:10.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:10 smithi151 ceph-mon[36066]: from='client.? 172.21.15.151:0/983073879' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e3c2ead-1af6-4bda-87cd-12770c81fa67"}]: dispatch 2024-05-14T16:50:10.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:10 smithi151 ceph-mon[36066]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e3c2ead-1af6-4bda-87cd-12770c81fa67"}]: dispatch 2024-05-14T16:50:10.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:10 smithi151 ceph-mon[36066]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6e3c2ead-1af6-4bda-87cd-12770c81fa67"}]': finished 2024-05-14T16:50:10.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:10 smithi151 ceph-mon[36066]: osdmap e67: 12 total, 11 up, 12 in 2024-05-14T16:50:10.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:10 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:50:10.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:10 smithi151 ceph-mon[36066]: from='client.? 172.21.15.151:0/2246027065' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-14T16:50:11.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:11 smithi151 ceph-mon[36066]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-14T16:50:11.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:11 smithi123 ceph-mon[35971]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-14T16:50:11.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:11 smithi002 ceph-mon[31538]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-14T16:50:13.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:13 smithi151 ceph-mon[36066]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T16:50:13.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:13 smithi123 ceph-mon[35971]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T16:50:13.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:13 smithi002 ceph-mon[31538]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-14T16:50:15.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:15 smithi151 ceph-mon[36066]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-14T16:50:15.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:15 smithi123 ceph-mon[35971]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-14T16:50:15.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:15 smithi002 ceph-mon[31538]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-14T16:50:17.570 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:17 smithi151 ceph-mon[36066]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:50:17.571 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:17 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-14T16:50:17.571 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:17 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:17.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:17 smithi123 ceph-mon[35971]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:50:17.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:17 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-14T16:50:17.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:17 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:17.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:17 smithi002 ceph-mon[31538]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:50:17.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:17 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-14T16:50:17.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:17 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:18.598 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:18 smithi151 ceph-mon[36066]: Deploying daemon osd.11 on smithi151 2024-05-14T16:50:18.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:18 smithi123 ceph-mon[35971]: Deploying daemon osd.11 on smithi151 2024-05-14T16:50:18.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:18 smithi002 ceph-mon[31538]: Deploying daemon osd.11 on smithi151 2024-05-14T16:50:19.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:19 smithi151 ceph-mon[36066]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:50:19.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:19 smithi123 ceph-mon[35971]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:50:19.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:19 smithi002 ceph-mon[31538]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-14T16:50:21.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:21 smithi151 ceph-mon[36066]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-05-14T16:50:21.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:21 smithi123 ceph-mon[35971]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-05-14T16:50:21.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:21 smithi002 ceph-mon[31538]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-05-14T16:50:23.549 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:23 smithi151 ceph-mon[36066]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:50:23.549 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:23 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:23.550 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:23 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:23.550 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:23 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:50:23.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:23 smithi123 ceph-mon[35971]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:50:23.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:23 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:23.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:23 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:23.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:23 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:50:23.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:23 smithi002 ceph-mon[31538]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:50:23.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:23 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:23.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:23 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:23.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:23 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:50:25.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:25 smithi151 ceph-mon[36066]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:50:25.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:25 smithi123 ceph-mon[35971]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:50:25.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:25 smithi002 ceph-mon[31538]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:50:26.826 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 11 on host 'smithi151' 2024-05-14T16:50:26.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:26.825+0000 7f6e96ffd700 1 -- 172.21.15.151:0/3292297743 <== mgr.14150 v2:172.21.15.2:6800/1498936028 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f6ea0063980 con 0x7f6e8c0b2900 2024-05-14T16:50:26.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:26.828+0000 7f6ea6d9d700 1 -- 172.21.15.151:0/3292297743 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f6e8c0b2900 msgr2=0x7f6e8c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:26.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:26.828+0000 7f6ea6d9d700 1 --2- 172.21.15.151:0/3292297743 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f6e8c0b2900 0x7f6e8c0b4dc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6e900097b0 tx=0x7f6e90006d20 comp rx=0 tx=0).stop 2024-05-14T16:50:26.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:26.829+0000 7f6ea6d9d700 1 -- 172.21.15.151:0/3292297743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ea01034c0 msgr2=0x7f6ea0199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:26.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:26.829+0000 7f6ea6d9d700 1 --2- 172.21.15.151:0/3292297743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ea01034c0 0x7f6ea0199700 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c0047b0 tx=0x7f6e9c009580 comp rx=0 tx=0).stop 2024-05-14T16:50:26.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:26.829+0000 7f6ea6d9d700 1 -- 172.21.15.151:0/3292297743 shutdown_connections 2024-05-14T16:50:26.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:26.829+0000 7f6ea6d9d700 1 --2- 172.21.15.151:0/3292297743 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f6e8c0b2900 0x7f6e8c0b4dc0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:26.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:26.829+0000 7f6ea6d9d700 1 --2- 172.21.15.151:0/3292297743 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ea0105060 0x7f6ea01a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:26.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:26.829+0000 7f6ea6d9d700 1 --2- 172.21.15.151:0/3292297743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ea01034c0 0x7f6ea0199700 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:26.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:26.829+0000 7f6ea6d9d700 1 --2- 172.21.15.151:0/3292297743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ea01046c0 0x7f6ea0199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:26.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:26.829+0000 7f6ea6d9d700 1 -- 172.21.15.151:0/3292297743 >> 172.21.15.151:0/3292297743 conn(0x7f6ea00fea80 msgr2=0x7f6ea0107ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:26.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:26.829+0000 7f6ea6d9d700 1 -- 172.21.15.151:0/3292297743 shutdown_connections 2024-05-14T16:50:26.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:26.829+0000 7f6ea6d9d700 1 -- 172.21.15.151:0/3292297743 wait complete. 2024-05-14T16:50:27.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:26 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:27.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:26 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:27.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:26 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:27.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:26 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:50:27.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:26 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:27.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:26 smithi151 ceph-mon[36066]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:50:27.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:26 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:27.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:26 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:27.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:26 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:50:27.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:26 smithi151 ceph-mon[36066]: from='osd.11 [v2:172.21.15.151:6824/4129194698,v1:172.21.15.151:6825/4129194698]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-14T16:50:27.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:26 smithi151 ceph-mon[36066]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-14T16:50:27.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:26 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:27.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:26 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:27.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:26 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:27.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:26 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:50:27.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:26 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:27.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:26 smithi002 ceph-mon[31538]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:50:27.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:26 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:27.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:26 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:27.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:26 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:50:27.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:26 smithi002 ceph-mon[31538]: from='osd.11 [v2:172.21.15.151:6824/4129194698,v1:172.21.15.151:6825/4129194698]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-14T16:50:27.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:26 smithi002 ceph-mon[31538]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-14T16:50:27.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:26 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:27.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:26 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:27.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:26 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:27.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:26 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:50:27.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:26 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:27.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:26 smithi123 ceph-mon[35971]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:50:27.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:26 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:27.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:26 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:27.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:26 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:50:27.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:26 smithi123 ceph-mon[35971]: from='osd.11 [v2:172.21.15.151:6824/4129194698,v1:172.21.15.151:6825/4129194698]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-14T16:50:27.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:26 smithi123 ceph-mon[35971]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-14T16:50:27.495 DEBUG:teuthology.orchestra.run.smithi151:osd.11> sudo journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@osd.11.service 2024-05-14T16:50:27.498 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-14T16:50:27.498 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph osd stat -f json 2024-05-14T16:50:27.758 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:50:29.050 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 16:50:28 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T16:50:28.791+0000 7fec99dbd700 -1 osd.11 0 waiting for initial osdmap 2024-05-14T16:50:29.051 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 16:50:28 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T16:50:28.807+0000 7fec95bb0700 -1 osd.11 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-14T16:50:29.051 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:28 smithi151 ceph-mon[36066]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-14T16:50:29.051 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:28 smithi151 ceph-mon[36066]: from='osd.11 [v2:172.21.15.151:6824/4129194698,v1:172.21.15.151:6825/4129194698]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-14T16:50:29.051 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:28 smithi151 ceph-mon[36066]: osdmap e68: 12 total, 11 up, 12 in 2024-05-14T16:50:29.051 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:28 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:50:29.051 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:28 smithi151 ceph-mon[36066]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-14T16:50:29.051 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:28 smithi151 ceph-mon[36066]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:50:29.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:28 smithi123 ceph-mon[35971]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-14T16:50:29.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:28 smithi123 ceph-mon[35971]: from='osd.11 [v2:172.21.15.151:6824/4129194698,v1:172.21.15.151:6825/4129194698]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-14T16:50:29.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:28 smithi123 ceph-mon[35971]: osdmap e68: 12 total, 11 up, 12 in 2024-05-14T16:50:29.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:28 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:50:29.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:28 smithi123 ceph-mon[35971]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-14T16:50:29.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:28 smithi123 ceph-mon[35971]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:50:29.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:28 smithi002 ceph-mon[31538]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-14T16:50:29.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:28 smithi002 ceph-mon[31538]: from='osd.11 [v2:172.21.15.151:6824/4129194698,v1:172.21.15.151:6825/4129194698]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-14T16:50:29.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:28 smithi002 ceph-mon[31538]: osdmap e68: 12 total, 11 up, 12 in 2024-05-14T16:50:29.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:28 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:50:29.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:28 smithi002 ceph-mon[31538]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-14T16:50:29.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:28 smithi002 ceph-mon[31538]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:50:29.839 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:29 smithi002 ceph-mon[31538]: purged_snaps scrub starts 2024-05-14T16:50:29.839 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:29 smithi002 ceph-mon[31538]: purged_snaps scrub ok 2024-05-14T16:50:29.839 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:29 smithi002 ceph-mon[31538]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-05-14T16:50:29.839 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:29 smithi002 ceph-mon[31538]: osdmap e69: 12 total, 11 up, 12 in 2024-05-14T16:50:29.840 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:29 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:50:29.840 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:29 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:50:29.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:29.954+0000 7ff451b9a700 1 -- 172.21.15.2:0/2005847498 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff44c105160 msgr2=0x7ff44c10ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:29.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:29.954+0000 7ff451b9a700 1 --2- 172.21.15.2:0/2005847498 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff44c105160 0x7ff44c10ba10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff44400b0d0 tx=0x7ff44400b3e0 comp rx=0 tx=0).stop 2024-05-14T16:50:29.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:29.954+0000 7ff451b9a700 1 -- 172.21.15.2:0/2005847498 shutdown_connections 2024-05-14T16:50:29.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:29.954+0000 7ff451b9a700 1 --2- 172.21.15.2:0/2005847498 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff44c105160 0x7ff44c10ba10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:29.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:29.954+0000 7ff451b9a700 1 --2- 172.21.15.2:0/2005847498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff44c1035c0 0x7ff44c1039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:29.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:29.954+0000 7ff451b9a700 1 --2- 172.21.15.2:0/2005847498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff44c1047c0 0x7ff44c104c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:29.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:29.954+0000 7ff451b9a700 1 -- 172.21.15.2:0/2005847498 >> 172.21.15.2:0/2005847498 conn(0x7ff44c0feb60 msgr2=0x7ff44c100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:29.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:29.954+0000 7ff451b9a700 1 -- 172.21.15.2:0/2005847498 shutdown_connections 2024-05-14T16:50:29.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:29.954+0000 7ff451b9a700 1 -- 172.21.15.2:0/2005847498 wait complete. 2024-05-14T16:50:29.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:29.955+0000 7ff451b9a700 1 Processor -- start 2024-05-14T16:50:29.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:29.955+0000 7ff451b9a700 1 -- start start 2024-05-14T16:50:29.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:29.955+0000 7ff451b9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff44c1035c0 0x7ff44c19b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:29.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:29.955+0000 7ff451b9a700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff44c1047c0 0x7ff44c19bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:29.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:29.955+0000 7ff451b9a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff44c105160 0x7ff44c1a2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:29.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:29.955+0000 7ff451b9a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff44c10f040 con 0x7ff44c1035c0 2024-05-14T16:50:29.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:29.955+0000 7ff451b9a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff44c10eec0 con 0x7ff44c105160 2024-05-14T16:50:29.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:29.955+0000 7ff451b9a700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff44c10f1c0 con 0x7ff44c1047c0 2024-05-14T16:50:29.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:29.955+0000 7ff44b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff44c1035c0 0x7ff44c19b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:29.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:29.955+0000 7ff44b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff44c1035c0 0x7ff44c19b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46536/0 (socket says 172.21.15.2:46536) 2024-05-14T16:50:29.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:29.955+0000 7ff44b7fe700 1 -- 172.21.15.2:0/2110908152 learned_addr learned my addr 172.21.15.2:0/2110908152 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:50:29.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:29.955+0000 7ff443fff700 1 --2- 172.21.15.2:0/2110908152 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff44c1047c0 0x7ff44c19bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:29.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:29.955+0000 7ff44bfff700 1 --2- 172.21.15.2:0/2110908152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff44c105160 0x7ff44c1a2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:29.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:29.955+0000 7ff44b7fe700 1 -- 172.21.15.2:0/2110908152 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff44c1047c0 msgr2=0x7ff44c19bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:29.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:29.955+0000 7ff44b7fe700 1 --2- 172.21.15.2:0/2110908152 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff44c1047c0 0x7ff44c19bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:29.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:29.955+0000 7ff44b7fe700 1 -- 172.21.15.2:0/2110908152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff44c105160 msgr2=0x7ff44c1a2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:29.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:29.955+0000 7ff44b7fe700 1 --2- 172.21.15.2:0/2110908152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff44c105160 0x7ff44c1a2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:29.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:29.955+0000 7ff44b7fe700 1 -- 172.21.15.2:0/2110908152 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff44c1a3460 con 0x7ff44c1035c0 2024-05-14T16:50:29.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:29.956+0000 7ff44b7fe700 1 --2- 172.21.15.2:0/2110908152 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff44c1035c0 0x7ff44c19b800 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff43c00d9b0 tx=0x7ff43c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:29.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:29.956+0000 7ff4497fa700 1 -- 172.21.15.2:0/2110908152 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43c016070 con 0x7ff44c1035c0 2024-05-14T16:50:29.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:29.956+0000 7ff451b9a700 1 -- 172.21.15.2:0/2110908152 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff44c1a3750 con 0x7ff44c1035c0 2024-05-14T16:50:29.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:29.956+0000 7ff4497fa700 1 -- 172.21.15.2:0/2110908152 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff43c00be00 con 0x7ff44c1035c0 2024-05-14T16:50:29.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:29.956+0000 7ff4497fa700 1 -- 172.21.15.2:0/2110908152 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43c010e30 con 0x7ff44c1035c0 2024-05-14T16:50:29.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:29.956+0000 7ff451b9a700 1 -- 172.21.15.2:0/2110908152 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff44c1a3c90 con 0x7ff44c1035c0 2024-05-14T16:50:29.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:29.957+0000 7ff451b9a700 1 -- 172.21.15.2:0/2110908152 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff44c06a230 con 0x7ff44c1035c0 2024-05-14T16:50:29.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:29.958+0000 7ff4497fa700 1 -- 172.21.15.2:0/2110908152 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7ff43c019070 con 0x7ff44c1035c0 2024-05-14T16:50:29.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:29.959+0000 7ff4497fa700 1 --2- 172.21.15.2:0/2110908152 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7ff4380b2880 0x7ff4380b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:29.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:29.959+0000 7ff443fff700 1 --2- 172.21.15.2:0/2110908152 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7ff4380b2880 0x7ff4380b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:29.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:29.959+0000 7ff4497fa700 1 -- 172.21.15.2:0/2110908152 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7ff43c0e2270 con 0x7ff44c1035c0 2024-05-14T16:50:29.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:29.960+0000 7ff443fff700 1 --2- 172.21.15.2:0/2110908152 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7ff4380b2880 0x7ff4380b4d40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff43400abd0 tx=0x7ff43400a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:29.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:29.963+0000 7ff4497fa700 1 -- 172.21.15.2:0/2110908152 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff43c020030 con 0x7ff44c1035c0 2024-05-14T16:50:30.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:29 smithi151 ceph-mon[36066]: purged_snaps scrub starts 2024-05-14T16:50:30.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:29 smithi151 ceph-mon[36066]: purged_snaps scrub ok 2024-05-14T16:50:30.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:29 smithi151 ceph-mon[36066]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-05-14T16:50:30.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:29 smithi151 ceph-mon[36066]: osdmap e69: 12 total, 11 up, 12 in 2024-05-14T16:50:30.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:29 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:50:30.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:29 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:50:30.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:29 smithi123 ceph-mon[35971]: purged_snaps scrub starts 2024-05-14T16:50:30.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:29 smithi123 ceph-mon[35971]: purged_snaps scrub ok 2024-05-14T16:50:30.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:29 smithi123 ceph-mon[35971]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-05-14T16:50:30.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:29 smithi123 ceph-mon[35971]: osdmap e69: 12 total, 11 up, 12 in 2024-05-14T16:50:30.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:29 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:50:30.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:29 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:50:30.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:30.176+0000 7ff451b9a700 1 -- 172.21.15.2:0/2110908152 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7ff44c1a4160 con 0x7ff44c1035c0 2024-05-14T16:50:30.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:30.176+0000 7ff4497fa700 1 -- 172.21.15.2:0/2110908152 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7ff43c0aa390 con 0x7ff44c1035c0 2024-05-14T16:50:30.176 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:50:30.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:30.179+0000 7ff451b9a700 1 -- 172.21.15.2:0/2110908152 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7ff4380b2880 msgr2=0x7ff4380b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:30.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:30.179+0000 7ff451b9a700 1 --2- 172.21.15.2:0/2110908152 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7ff4380b2880 0x7ff4380b4d40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff43400abd0 tx=0x7ff43400a380 comp rx=0 tx=0).stop 2024-05-14T16:50:30.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:30.179+0000 7ff451b9a700 1 -- 172.21.15.2:0/2110908152 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff44c1035c0 msgr2=0x7ff44c19b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:30.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:30.179+0000 7ff451b9a700 1 --2- 172.21.15.2:0/2110908152 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff44c1035c0 0x7ff44c19b800 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff43c00d9b0 tx=0x7ff43c00dcc0 comp rx=0 tx=0).stop 2024-05-14T16:50:30.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:30.179+0000 7ff451b9a700 1 -- 172.21.15.2:0/2110908152 shutdown_connections 2024-05-14T16:50:30.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:30.179+0000 7ff451b9a700 1 --2- 172.21.15.2:0/2110908152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff44c105160 0x7ff44c1a2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:30.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:30.179+0000 7ff451b9a700 1 --2- 172.21.15.2:0/2110908152 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7ff4380b2880 0x7ff4380b4d40 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:30.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:30.179+0000 7ff451b9a700 1 --2- 172.21.15.2:0/2110908152 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff44c1035c0 0x7ff44c19b800 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:30.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:30.179+0000 7ff451b9a700 1 --2- 172.21.15.2:0/2110908152 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff44c1047c0 0x7ff44c19bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:30.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:30.179+0000 7ff451b9a700 1 -- 172.21.15.2:0/2110908152 >> 172.21.15.2:0/2110908152 conn(0x7ff44c0feb60 msgr2=0x7ff44c075770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:30.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:30.179+0000 7ff451b9a700 1 -- 172.21.15.2:0/2110908152 shutdown_connections 2024-05-14T16:50:30.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:30.179+0000 7ff451b9a700 1 -- 172.21.15.2:0/2110908152 wait complete. 2024-05-14T16:50:30.856 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":70,"num_osds":12,"num_up_osds":12,"osd_up_since":1715705429,"num_in_osds":12,"osd_in_since":1715705409,"num_remapped_pgs":0} 2024-05-14T16:50:30.857 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T16:50:31.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:30 smithi151 ceph-mon[36066]: osd.11 [v2:172.21.15.151:6824/4129194698,v1:172.21.15.151:6825/4129194698] boot 2024-05-14T16:50:31.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:30 smithi151 ceph-mon[36066]: osdmap e70: 12 total, 12 up, 12 in 2024-05-14T16:50:31.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:30 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:50:31.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:30 smithi151 ceph-mon[36066]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:50:31.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:30 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/2110908152' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-14T16:50:31.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:30 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:31.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:30 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:31.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:30 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:50:31.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:30 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:50:31.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:30 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:50:31.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:30 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:50:31.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:30 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:31.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:30 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:31.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:30 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:50:31.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:30 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:31.112 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:50:31.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:30 smithi123 ceph-mon[35971]: osd.11 [v2:172.21.15.151:6824/4129194698,v1:172.21.15.151:6825/4129194698] boot 2024-05-14T16:50:31.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:30 smithi123 ceph-mon[35971]: osdmap e70: 12 total, 12 up, 12 in 2024-05-14T16:50:31.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:30 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:50:31.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:30 smithi123 ceph-mon[35971]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:50:31.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:30 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/2110908152' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-14T16:50:31.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:30 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:31.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:30 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:31.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:30 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:50:31.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:30 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:50:31.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:30 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:50:31.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:30 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:50:31.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:30 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:31.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:30 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:31.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:30 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:50:31.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:30 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:31.235 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:30 smithi002 ceph-mon[31538]: osd.11 [v2:172.21.15.151:6824/4129194698,v1:172.21.15.151:6825/4129194698] boot 2024-05-14T16:50:31.235 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:30 smithi002 ceph-mon[31538]: osdmap e70: 12 total, 12 up, 12 in 2024-05-14T16:50:31.235 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:30 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:50:31.235 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:30 smithi002 ceph-mon[31538]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 983 GiB / 983 GiB avail 2024-05-14T16:50:31.235 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:30 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/2110908152' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-14T16:50:31.236 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:30 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:31.236 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:30 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:31.236 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:30 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:50:31.236 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:30 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:50:31.236 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:30 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:50:31.236 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:30 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:50:31.236 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:30 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:31.236 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:30 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:31.236 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:30 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:50:31.236 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:30 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:32.093 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:31 smithi002 ceph-mon[31538]: Detected new or changed devices on smithi151 2024-05-14T16:50:32.093 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:31 smithi002 ceph-mon[31538]: Adjusting osd_memory_target on smithi151 to 4272M 2024-05-14T16:50:32.093 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:31 smithi002 ceph-mon[31538]: osdmap e71: 12 total, 12 up, 12 in 2024-05-14T16:50:32.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:31 smithi151 ceph-mon[36066]: Detected new or changed devices on smithi151 2024-05-14T16:50:32.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:31 smithi151 ceph-mon[36066]: Adjusting osd_memory_target on smithi151 to 4272M 2024-05-14T16:50:32.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:31 smithi151 ceph-mon[36066]: osdmap e71: 12 total, 12 up, 12 in 2024-05-14T16:50:32.159 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:31 smithi123 ceph-mon[35971]: Detected new or changed devices on smithi151 2024-05-14T16:50:32.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:31 smithi123 ceph-mon[35971]: Adjusting osd_memory_target on smithi151 to 4272M 2024-05-14T16:50:32.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:31 smithi123 ceph-mon[35971]: osdmap e71: 12 total, 12 up, 12 in 2024-05-14T16:50:33.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.015+0000 7fac10922700 1 -- 172.21.15.2:0/120906650 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac0c06c6d0 msgr2=0x7fac0c110400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:33.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.015+0000 7fac10922700 1 --2- 172.21.15.2:0/120906650 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac0c06c6d0 0x7fac0c110400 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fac040099c0 tx=0x7fac04009cd0 comp rx=0 tx=0).stop 2024-05-14T16:50:33.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.015+0000 7fac10922700 1 -- 172.21.15.2:0/120906650 shutdown_connections 2024-05-14T16:50:33.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.015+0000 7fac10922700 1 --2- 172.21.15.2:0/120906650 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac0c06c6d0 0x7fac0c110400 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:33.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.015+0000 7fac10922700 1 --2- 172.21.15.2:0/120906650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac0c109630 0x7fac0c109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:33.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.015+0000 7fac10922700 1 --2- 172.21.15.2:0/120906650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac0c06bd10 0x7fac0c06c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:33.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.015+0000 7fac10922700 1 -- 172.21.15.2:0/120906650 >> 172.21.15.2:0/120906650 conn(0x7fac0c0794e0 msgr2=0x7fac0c0798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:33.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.015+0000 7fac10922700 1 -- 172.21.15.2:0/120906650 shutdown_connections 2024-05-14T16:50:33.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.015+0000 7fac10922700 1 -- 172.21.15.2:0/120906650 wait complete. 2024-05-14T16:50:33.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.016+0000 7fac10922700 1 Processor -- start 2024-05-14T16:50:33.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.016+0000 7fac10922700 1 -- start start 2024-05-14T16:50:33.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.016+0000 7fac10922700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac0c06bd10 0x7fac0c19f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:33.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.016+0000 7fac10922700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac0c06c6d0 0x7fac0c19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:33.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.016+0000 7fac10922700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac0c109630 0x7fac0c1999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:33.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.016+0000 7fac10922700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fac0c113780 con 0x7fac0c06c6d0 2024-05-14T16:50:33.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.016+0000 7fac10922700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fac0c113600 con 0x7fac0c109630 2024-05-14T16:50:33.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.016+0000 7fac10922700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fac0c113900 con 0x7fac0c06bd10 2024-05-14T16:50:33.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.016+0000 7fac09d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac0c06c6d0 0x7fac0c19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:33.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.016+0000 7fac09d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac0c06c6d0 0x7fac0c19fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46556/0 (socket says 172.21.15.2:46556) 2024-05-14T16:50:33.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.016+0000 7fac09d9b700 1 -- 172.21.15.2:0/1540951529 learned_addr learned my addr 172.21.15.2:0/1540951529 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:50:33.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.017+0000 7fac0ad9d700 1 --2- 172.21.15.2:0/1540951529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac0c109630 0x7fac0c1999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:33.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.017+0000 7fac0a59c700 1 --2- 172.21.15.2:0/1540951529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac0c06bd10 0x7fac0c19f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:33.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.017+0000 7fac09d9b700 1 -- 172.21.15.2:0/1540951529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac0c06bd10 msgr2=0x7fac0c19f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:33.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.017+0000 7fac09d9b700 1 --2- 172.21.15.2:0/1540951529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac0c06bd10 0x7fac0c19f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:33.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.017+0000 7fac09d9b700 1 -- 172.21.15.2:0/1540951529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac0c109630 msgr2=0x7fac0c1999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:33.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.017+0000 7fac09d9b700 1 --2- 172.21.15.2:0/1540951529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac0c109630 0x7fac0c1999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:33.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.017+0000 7fac09d9b700 1 -- 172.21.15.2:0/1540951529 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac0c19a120 con 0x7fac0c06c6d0 2024-05-14T16:50:33.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.017+0000 7fac0a59c700 1 --2- 172.21.15.2:0/1540951529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac0c06bd10 0x7fac0c19f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:50:33.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.017+0000 7fac09d9b700 1 --2- 172.21.15.2:0/1540951529 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac0c06c6d0 0x7fac0c19fd40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fac0000d950 tx=0x7fac0000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:33.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.017+0000 7fabfb7fe700 1 -- 172.21.15.2:0/1540951529 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac00016070 con 0x7fac0c06c6d0 2024-05-14T16:50:33.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.017+0000 7fabfb7fe700 1 -- 172.21.15.2:0/1540951529 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac00010460 con 0x7fac0c06c6d0 2024-05-14T16:50:33.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.017+0000 7fabfb7fe700 1 -- 172.21.15.2:0/1540951529 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0000f620 con 0x7fac0c06c6d0 2024-05-14T16:50:33.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.017+0000 7fac10922700 1 -- 172.21.15.2:0/1540951529 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac0c19a410 con 0x7fac0c06c6d0 2024-05-14T16:50:33.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.017+0000 7fac10922700 1 -- 172.21.15.2:0/1540951529 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac0c1a66d0 con 0x7fac0c06c6d0 2024-05-14T16:50:33.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.018+0000 7fac10922700 1 -- 172.21.15.2:0/1540951529 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac0c06a230 con 0x7fac0c06c6d0 2024-05-14T16:50:33.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.019+0000 7fabfb7fe700 1 -- 172.21.15.2:0/1540951529 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fac00019070 con 0x7fac0c06c6d0 2024-05-14T16:50:33.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.020+0000 7fabfb7fe700 1 --2- 172.21.15.2:0/1540951529 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fabf40b2900 0x7fabf40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:33.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.020+0000 7fac0a59c700 1 --2- 172.21.15.2:0/1540951529 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fabf40b2900 0x7fabf40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:33.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.020+0000 7fabfb7fe700 1 -- 172.21.15.2:0/1540951529 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fac000e19c0 con 0x7fac0c06c6d0 2024-05-14T16:50:33.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.021+0000 7fac0a59c700 1 --2- 172.21.15.2:0/1540951529 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fabf40b2900 0x7fabf40b4dc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fac0c19ab00 tx=0x7fabfc009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:33.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.024+0000 7fabfb7fe700 1 -- 172.21.15.2:0/1540951529 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fac000a9ae0 con 0x7fac0c06c6d0 2024-05-14T16:50:33.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:32 smithi151 ceph-mon[36066]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:33.159 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:32 smithi123 ceph-mon[35971]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:33.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.243+0000 7fac10922700 1 -- 172.21.15.2:0/1540951529 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fac0c02e560 con 0x7fac0c06c6d0 2024-05-14T16:50:33.243 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:32 smithi002 ceph-mon[31538]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:33.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.244+0000 7fabfb7fe700 1 -- 172.21.15.2:0/1540951529 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+18436 (secure 0 0 0) 0x7fac000a93e0 con 0x7fac0c06c6d0 2024-05-14T16:50:33.245 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:50:33.245 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":71,"fsid":"ed40717a-1210-11ef-bc98-c7b262605968","created":"2024-05-14T16:43:09.415298+0000","modified":"2024-05-14T16:50:31.627530+0000","last_up_change":"2024-05-14T16:50:29.787654+0000","last_in_change":"2024-05-14T16:50:09.287932+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T16:46:38.090903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"153f20ec-5bcf-4786-9ede-2e512a1e3d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6803","nonce":2654237820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6805","nonce":2654237820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6809","nonce":2654237820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6807","nonce":2654237820}]},"public_addr":"172.21.15.2:6803/2654237820","cluster_addr":"172.21.15.2:6805/2654237820","heartbeat_back_addr":"172.21.15.2:6809/2654237820","heartbeat_front_addr":"172.21.15.2:6807/2654237820","state":["exists","up"]},{"osd":1,"uuid":"0e95419f-9a9d-4f7a-b55d-ecb54971918a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6801","nonce":2668156992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6803","nonce":2668156992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6807","nonce":2668156992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6805","nonce":2668156992}]},"public_addr":"172.21.15.123:6801/2668156992","cluster_addr":"172.21.15.123:6803/2668156992","heartbeat_back_addr":"172.21.15.123:6807/2668156992","heartbeat_front_addr":"172.21.15.123:6805/2668156992","state":["exists","up"]},{"osd":2,"uuid":"b6ce7ac7-be6c-4549-a356-59cb81b5b492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6801","nonce":2160189636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6803","nonce":2160189636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6807","nonce":2160189636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6805","nonce":2160189636}]},"public_addr":"172.21.15.151:6801/2160189636","cluster_addr":"172.21.15.151:6803/2160189636","heartbeat_back_addr":"172.21.15.151:6807/2160189636","heartbeat_front_addr":"172.21.15.151:6805/2160189636","state":["exists","up"]},{"osd":3,"uuid":"2fd592bf-c212-411a-afbe-a7c498e5ac93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6811","nonce":3451424073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6813","nonce":3451424073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6817","nonce":3451424073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6815","nonce":3451424073}]},"public_addr":"172.21.15.2:6811/3451424073","cluster_addr":"172.21.15.2:6813/3451424073","heartbeat_back_addr":"172.21.15.2:6817/3451424073","heartbeat_front_addr":"172.21.15.2:6815/3451424073","state":["exists","up"]},{"osd":4,"uuid":"af1905e7-e036-47c6-be59-b085bc41fede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6809","nonce":2101935762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6811","nonce":2101935762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6815","nonce":2101935762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6813","nonce":2101935762}]},"public_addr":"172.21.15.123:6809/2101935762","cluster_addr":"172.21.15.123:6811/2101935762","heartbeat_back_addr":"172.21.15.123:6815/2101935762","heartbeat_front_addr":"172.21.15.123:6813/2101935762","state":["exists","up"]},{"osd":5,"uuid":"9c166916-af78-4cc2-bd14-d1220b6e7d9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6809","nonce":3695961886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6811","nonce":3695961886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6815","nonce":3695961886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6813","nonce":3695961886}]},"public_addr":"172.21.15.151:6809/3695961886","cluster_addr":"172.21.15.151:6811/3695961886","heartbeat_back_addr":"172.21.15.151:6815/3695961886","heartbeat_front_addr":"172.21.15.151:6813/3695961886","state":["exists","up"]},{"osd":6,"uuid":"60e9bf66-2338-41e8-9280-398d4e3b322f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6819","nonce":2711893600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6821","nonce":2711893600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6825","nonce":2711893600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6823","nonce":2711893600}]},"public_addr":"172.21.15.2:6819/2711893600","cluster_addr":"172.21.15.2:6821/2711893600","heartbeat_back_addr":"172.21.15.2:6825/2711893600","heartbeat_front_addr":"172.21.15.2:6823/2711893600","state":["exists","up"]},{"osd":7,"uuid":"c93e07a1-e941-493e-8468-33429df2dff0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6817","nonce":3634155507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6819","nonce":3634155507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6823","nonce":3634155507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6821","nonce":3634155507}]},"public_addr":"172.21.15.123:6817/3634155507","cluster_addr":"172.21.15.123:6819/3634155507","heartbeat_back_addr":"172.21.15.123:6823/3634155507","heartbeat_front_addr":"172.21.15.123:6821/3634155507","state":["exists","up"]},{"osd":8,"uuid":"1155fb58-d614-4f45-91e2-5da3a93e7d9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6817","nonce":4271107010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6819","nonce":4271107010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6823","nonce":4271107010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6821","nonce":4271107010}]},"public_addr":"172.21.15.151:6817/4271107010","cluster_addr":"172.21.15.151:6819/4271107010","heartbeat_back_addr":"172.21.15.151:6823/4271107010","heartbeat_front_addr":"172.21.15.151:6821/4271107010","state":["exists","up"]},{"osd":9,"uuid":"82b34e3b-0017-4ca4-8f86-0993ca78ce7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6827","nonce":3479431021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6829","nonce":3479431021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6833","nonce":3479431021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6831","nonce":3479431021}]},"public_addr":"172.21.15.2:6827/3479431021","cluster_addr":"172.21.15.2:6829/3479431021","heartbeat_back_addr":"172.21.15.2:6833/3479431021","heartbeat_front_addr":"172.21.15.2:6831/3479431021","state":["exists","up"]},{"osd":10,"uuid":"32e2ace9-eb5d-4258-a2ac-49b782be2b5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6825","nonce":4266374435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6827","nonce":4266374435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6831","nonce":4266374435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6829","nonce":4266374435}]},"public_addr":"172.21.15.123:6825/4266374435","cluster_addr":"172.21.15.123:6827/4266374435","heartbeat_back_addr":"172.21.15.123:6831/4266374435","heartbeat_front_addr":"172.21.15.123:6829/4266374435","state":["exists","up"]},{"osd":11,"uuid":"6e3c2ead-1af6-4bda-87cd-12770c81fa67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6825","nonce":4129194698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6827","nonce":4129194698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6831","nonce":4129194698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6829","nonce":4129194698}]},"public_addr":"172.21.15.151:6825/4129194698","cluster_addr":"172.21.15.151:6827/4129194698","heartbeat_back_addr":"172.21.15.151:6831/4129194698","heartbeat_front_addr":"172.21.15.151:6829/4129194698","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:45:46.540023+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:46:10.845186+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:46:34.283112+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:47:00.072467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:47:25.315644+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:47:49.748688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:48:16.964258+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:48:42.875515+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:49:08.596500+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:49:35.450943+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:50:01.293368+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:50:27.793713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/4267809161":"2024-05-15T16:44:11.998214+0000","172.21.15.2:6801/3582988344":"2024-05-15T16:44:11.998214+0000","172.21.15.2:6800/2574015037":"2024-05-15T16:43:43.407184+0000","172.21.15.2:0/2100732571":"2024-05-15T16:44:11.998214+0000","172.21.15.2:0/2954199324":"2024-05-15T16:43:43.407184+0000","172.21.15.2:6801/2574015037":"2024-05-15T16:43:43.407184+0000","172.21.15.2:0/3171550944":"2024-05-15T16:43:43.407184+0000","172.21.15.2:0/1679761136":"2024-05-15T16:44:11.998214+0000","172.21.15.2:0/1746766591":"2024-05-15T16:43:43.407184+0000","172.21.15.2:6800/3582988344":"2024-05-15T16:44:11.998214+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T16:50:33.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.249+0000 7fac10922700 1 -- 172.21.15.2:0/1540951529 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fabf40b2900 msgr2=0x7fabf40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:33.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.249+0000 7fac10922700 1 --2- 172.21.15.2:0/1540951529 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fabf40b2900 0x7fabf40b4dc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fac0c19ab00 tx=0x7fabfc009040 comp rx=0 tx=0).stop 2024-05-14T16:50:33.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.249+0000 7fac10922700 1 -- 172.21.15.2:0/1540951529 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac0c06c6d0 msgr2=0x7fac0c19fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:33.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.249+0000 7fac10922700 1 --2- 172.21.15.2:0/1540951529 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac0c06c6d0 0x7fac0c19fd40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fac0000d950 tx=0x7fac0000dd10 comp rx=0 tx=0).stop 2024-05-14T16:50:33.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.249+0000 7fac10922700 1 -- 172.21.15.2:0/1540951529 shutdown_connections 2024-05-14T16:50:33.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.249+0000 7fac10922700 1 --2- 172.21.15.2:0/1540951529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac0c109630 0x7fac0c1999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:33.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.249+0000 7fac10922700 1 --2- 172.21.15.2:0/1540951529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac0c06bd10 0x7fac0c19f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:33.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.249+0000 7fac10922700 1 --2- 172.21.15.2:0/1540951529 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fabf40b2900 0x7fabf40b4dc0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:33.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.249+0000 7fac10922700 1 --2- 172.21.15.2:0/1540951529 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac0c06c6d0 0x7fac0c19fd40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:33.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.249+0000 7fac10922700 1 -- 172.21.15.2:0/1540951529 >> 172.21.15.2:0/1540951529 conn(0x7fac0c0794e0 msgr2=0x7fac0c101060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:33.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.249+0000 7fac10922700 1 -- 172.21.15.2:0/1540951529 shutdown_connections 2024-05-14T16:50:33.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:33.249+0000 7fac10922700 1 -- 172.21.15.2:0/1540951529 wait complete. 2024-05-14T16:50:33.986 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:33 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/1540951529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:50:34.014 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-14T16:46:38.090903+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-14T16:50:34.015 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-14T16:50:34.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:33 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/1540951529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:50:34.159 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:33 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/1540951529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:50:34.261 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:50:34.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:34 smithi002 ceph-mon[31538]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:35.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:34 smithi151 ceph-mon[36066]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:35.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:34 smithi123 ceph-mon[35971]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:36.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.177+0000 7f7b54c70700 1 -- 172.21.15.2:0/1303921452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b50103fc0 msgr2=0x7f7b501103b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:36.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.177+0000 7f7b54c70700 1 --2- 172.21.15.2:0/1303921452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b50103fc0 0x7f7b501103b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7b4800b0d0 tx=0x7f7b4800b3e0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.177+0000 7f7b54c70700 1 -- 172.21.15.2:0/1303921452 shutdown_connections 2024-05-14T16:50:36.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.177+0000 7f7b54c70700 1 --2- 172.21.15.2:0/1303921452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b50103fc0 0x7f7b501103b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.177+0000 7f7b54c70700 1 --2- 172.21.15.2:0/1303921452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b501095e0 0x7f7b501099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.177+0000 7f7b54c70700 1 --2- 172.21.15.2:0/1303921452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b50103600 0x7f7b50103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.177+0000 7f7b54c70700 1 -- 172.21.15.2:0/1303921452 >> 172.21.15.2:0/1303921452 conn(0x7f7b5007a700 msgr2=0x7f7b5007ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:36.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.177+0000 7f7b54c70700 1 -- 172.21.15.2:0/1303921452 shutdown_connections 2024-05-14T16:50:36.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.177+0000 7f7b54c70700 1 -- 172.21.15.2:0/1303921452 wait complete. 2024-05-14T16:50:36.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.178+0000 7f7b54c70700 1 Processor -- start 2024-05-14T16:50:36.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.178+0000 7f7b54c70700 1 -- start start 2024-05-14T16:50:36.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.179+0000 7f7b54c70700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b50103600 0x7f7b5019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:36.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.179+0000 7f7b54c70700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b50103fc0 0x7f7b5019fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:36.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.179+0000 7f7b54c70700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b501095e0 0x7f7b50199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:36.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.179+0000 7f7b54c70700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7b50113740 con 0x7f7b501095e0 2024-05-14T16:50:36.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.179+0000 7f7b54c70700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7b501135c0 con 0x7f7b50103fc0 2024-05-14T16:50:36.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.179+0000 7f7b54c70700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7b501138c0 con 0x7f7b50103600 2024-05-14T16:50:36.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.179+0000 7f7b4ed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b501095e0 0x7f7b50199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:36.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.179+0000 7f7b4e59c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b50103600 0x7f7b5019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:36.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.179+0000 7f7b4ed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b501095e0 0x7f7b50199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54182/0 (socket says 172.21.15.2:54182) 2024-05-14T16:50:36.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.179+0000 7f7b4ed9d700 1 -- 172.21.15.2:0/2265882489 learned_addr learned my addr 172.21.15.2:0/2265882489 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:50:36.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.179+0000 7f7b4dd9b700 1 --2- 172.21.15.2:0/2265882489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b50103fc0 0x7f7b5019fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:36.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.179+0000 7f7b4ed9d700 1 -- 172.21.15.2:0/2265882489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b50103600 msgr2=0x7f7b5019f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:36.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.179+0000 7f7b4ed9d700 1 --2- 172.21.15.2:0/2265882489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b50103600 0x7f7b5019f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.179+0000 7f7b4ed9d700 1 -- 172.21.15.2:0/2265882489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b50103fc0 msgr2=0x7f7b5019fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:36.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.179+0000 7f7b4ed9d700 1 --2- 172.21.15.2:0/2265882489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b50103fc0 0x7f7b5019fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.179+0000 7f7b4ed9d700 1 -- 172.21.15.2:0/2265882489 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b5019a0e0 con 0x7f7b501095e0 2024-05-14T16:50:36.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.179+0000 7f7b4dd9b700 1 --2- 172.21.15.2:0/2265882489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b50103fc0 0x7f7b5019fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:50:36.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.179+0000 7f7b4ed9d700 1 --2- 172.21.15.2:0/2265882489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b501095e0 0x7f7b50199980 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7b48009850 tx=0x7f7b48007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:36.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.180+0000 7f7b3f7fe700 1 -- 172.21.15.2:0/2265882489 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4801f070 con 0x7f7b501095e0 2024-05-14T16:50:36.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.180+0000 7f7b54c70700 1 -- 172.21.15.2:0/2265882489 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b5019a370 con 0x7f7b501095e0 2024-05-14T16:50:36.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.180+0000 7f7b3f7fe700 1 -- 172.21.15.2:0/2265882489 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b48004c50 con 0x7f7b501095e0 2024-05-14T16:50:36.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.180+0000 7f7b3f7fe700 1 -- 172.21.15.2:0/2265882489 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4801b7b0 con 0x7f7b501095e0 2024-05-14T16:50:36.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.180+0000 7f7b54c70700 1 -- 172.21.15.2:0/2265882489 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b501a6640 con 0x7f7b501095e0 2024-05-14T16:50:36.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.181+0000 7f7b54c70700 1 -- 172.21.15.2:0/2265882489 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b50104c50 con 0x7f7b501095e0 2024-05-14T16:50:36.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.182+0000 7f7b3f7fe700 1 -- 172.21.15.2:0/2265882489 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f7b48022070 con 0x7f7b501095e0 2024-05-14T16:50:36.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.183+0000 7f7b3f7fe700 1 --2- 172.21.15.2:0/2265882489 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f7b380b2880 0x7f7b380b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:36.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.183+0000 7f7b4e59c700 1 --2- 172.21.15.2:0/2265882489 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f7b380b2880 0x7f7b380b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:36.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.183+0000 7f7b3f7fe700 1 -- 172.21.15.2:0/2265882489 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f7b4800f020 con 0x7f7b501095e0 2024-05-14T16:50:36.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.183+0000 7f7b4e59c700 1 --2- 172.21.15.2:0/2265882489 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f7b380b2880 0x7f7b380b4d40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7b501090a0 tx=0x7f7b4000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:36.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.186+0000 7f7b3f7fe700 1 -- 172.21.15.2:0/2265882489 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7b48029020 con 0x7f7b501095e0 2024-05-14T16:50:36.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.398+0000 7f7b54c70700 1 -- 172.21.15.2:0/2265882489 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7b5002e560 con 0x7f7b501095e0 2024-05-14T16:50:36.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.399+0000 7f7b3f7fe700 1 -- 172.21.15.2:0/2265882489 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7f7b48009cb0 con 0x7f7b501095e0 2024-05-14T16:50:36.399 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2024-05-14T16:50:36.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.401+0000 7f7b54c70700 1 -- 172.21.15.2:0/2265882489 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f7b380b2880 msgr2=0x7f7b380b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:36.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.401+0000 7f7b54c70700 1 --2- 172.21.15.2:0/2265882489 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f7b380b2880 0x7f7b380b4d40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7b501090a0 tx=0x7f7b4000b410 comp rx=0 tx=0).stop 2024-05-14T16:50:36.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.401+0000 7f7b54c70700 1 -- 172.21.15.2:0/2265882489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b501095e0 msgr2=0x7f7b50199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:36.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.401+0000 7f7b54c70700 1 --2- 172.21.15.2:0/2265882489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b501095e0 0x7f7b50199980 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7b48009850 tx=0x7f7b48007a20 comp rx=0 tx=0).stop 2024-05-14T16:50:36.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.401+0000 7f7b54c70700 1 -- 172.21.15.2:0/2265882489 shutdown_connections 2024-05-14T16:50:36.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.401+0000 7f7b54c70700 1 --2- 172.21.15.2:0/2265882489 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f7b380b2880 0x7f7b380b4d40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.401+0000 7f7b54c70700 1 --2- 172.21.15.2:0/2265882489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b501095e0 0x7f7b50199980 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.401+0000 7f7b54c70700 1 --2- 172.21.15.2:0/2265882489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b50103600 0x7f7b5019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.401+0000 7f7b54c70700 1 --2- 172.21.15.2:0/2265882489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b50103fc0 0x7f7b5019fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:36.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.402+0000 7f7b54c70700 1 -- 172.21.15.2:0/2265882489 >> 172.21.15.2:0/2265882489 conn(0x7f7b5007a700 msgr2=0x7f7b50102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:36.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.402+0000 7f7b54c70700 1 -- 172.21.15.2:0/2265882489 shutdown_connections 2024-05-14T16:50:36.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:36.402+0000 7f7b54c70700 1 -- 172.21.15.2:0/2265882489 wait complete. 2024-05-14T16:50:37.139 INFO:tasks.cephadm:Adding mds.a on smithi002 2024-05-14T16:50:37.139 INFO:tasks.cephadm:Adding mds.d on smithi002 2024-05-14T16:50:37.139 INFO:tasks.cephadm:Adding mds.g on smithi002 2024-05-14T16:50:37.139 INFO:tasks.cephadm:Adding mds.j on smithi002 2024-05-14T16:50:37.139 INFO:tasks.cephadm:Adding mds.b on smithi123 2024-05-14T16:50:37.139 INFO:tasks.cephadm:Adding mds.e on smithi123 2024-05-14T16:50:37.139 INFO:tasks.cephadm:Adding mds.h on smithi123 2024-05-14T16:50:37.139 INFO:tasks.cephadm:Adding mds.k on smithi123 2024-05-14T16:50:37.139 INFO:tasks.cephadm:Adding mds.c on smithi151 2024-05-14T16:50:37.139 INFO:tasks.cephadm:Adding mds.f on smithi151 2024-05-14T16:50:37.139 INFO:tasks.cephadm:Adding mds.i on smithi151 2024-05-14T16:50:37.140 INFO:tasks.cephadm:Adding mds.l on smithi151 2024-05-14T16:50:37.140 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph orch apply mds all '12;smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l' 2024-05-14T16:50:37.392 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T16:50:37.409 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:37 smithi123 ceph-mon[35971]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:37.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:37 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/2265882489' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-14T16:50:37.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:37 smithi002 ceph-mon[31538]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:37.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:37 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/2265882489' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-14T16:50:37.514 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:37 smithi151 ceph-mon[36066]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:37.514 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:37 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/2265882489' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-14T16:50:39.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.016+0000 7f716db30700 1 -- 172.21.15.151:0/272944939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71681035a0 msgr2=0x7f71681039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:39.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.016+0000 7f716db30700 1 --2- 172.21.15.151:0/272944939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71681035a0 0x7f71681039c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7158009990 tx=0x7f7158009ca0 comp rx=0 tx=0).stop 2024-05-14T16:50:39.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.016+0000 7f716db30700 1 -- 172.21.15.151:0/272944939 shutdown_connections 2024-05-14T16:50:39.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.016+0000 7f716db30700 1 --2- 172.21.15.151:0/272944939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7168105140 0x7f716810b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:39.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.016+0000 7f716db30700 1 --2- 172.21.15.151:0/272944939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71681035a0 0x7f71681039c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:39.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.016+0000 7f716db30700 1 --2- 172.21.15.151:0/272944939 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71681047a0 0x7f7168104c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:39.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.016+0000 7f716db30700 1 -- 172.21.15.151:0/272944939 >> 172.21.15.151:0/272944939 conn(0x7f71680feb40 msgr2=0x7f7168100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:39.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.017+0000 7f716db30700 1 -- 172.21.15.151:0/272944939 shutdown_connections 2024-05-14T16:50:39.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.017+0000 7f716db30700 1 -- 172.21.15.151:0/272944939 wait complete. 2024-05-14T16:50:39.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.017+0000 7f716db30700 1 Processor -- start 2024-05-14T16:50:39.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.017+0000 7f716db30700 1 -- start start 2024-05-14T16:50:39.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.017+0000 7f716db30700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71681035a0 0x7f7168199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:39.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.017+0000 7f716db30700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71681047a0 0x7f7168199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:39.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.017+0000 7f716db30700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7168105140 0x7f71681a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:39.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.017+0000 7f716db30700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f716810f060 con 0x7f7168105140 2024-05-14T16:50:39.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.017+0000 7f716db30700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f716810eee0 con 0x7f71681035a0 2024-05-14T16:50:39.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.017+0000 7f716db30700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f716810f1e0 con 0x7f71681047a0 2024-05-14T16:50:39.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.018+0000 7f7166ffd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71681047a0 0x7f7168199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:39.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.018+0000 7f7166ffd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71681047a0 0x7f7168199cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36484/0 (socket says 172.21.15.151:36484) 2024-05-14T16:50:39.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.018+0000 7f7166ffd700 1 -- 172.21.15.151:0/2838453507 learned_addr learned my addr 172.21.15.151:0/2838453507 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-14T16:50:39.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.018+0000 7f7167fff700 1 --2- 172.21.15.151:0/2838453507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7168105140 0x7f71681a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:39.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.018+0000 7f71677fe700 1 --2- 172.21.15.151:0/2838453507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71681035a0 0x7f7168199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:39.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.018+0000 7f7166ffd700 1 -- 172.21.15.151:0/2838453507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71681035a0 msgr2=0x7f7168199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:39.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.018+0000 7f7166ffd700 1 --2- 172.21.15.151:0/2838453507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71681035a0 0x7f7168199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:39.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.018+0000 7f7166ffd700 1 -- 172.21.15.151:0/2838453507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7168105140 msgr2=0x7f71681a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:39.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.018+0000 7f7166ffd700 1 --2- 172.21.15.151:0/2838453507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7168105140 0x7f71681a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:39.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.018+0000 7f7166ffd700 1 -- 172.21.15.151:0/2838453507 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71681a13d0 con 0x7f71681047a0 2024-05-14T16:50:39.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.018+0000 7f7166ffd700 1 --2- 172.21.15.151:0/2838453507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71681047a0 0x7f7168199cb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f715c00d980 tx=0x7f715c00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:39.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.018+0000 7f7164ff9700 1 -- 172.21.15.151:0/2838453507 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c016070 con 0x7f71681047a0 2024-05-14T16:50:39.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.018+0000 7f7164ff9700 1 -- 172.21.15.151:0/2838453507 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f715c00de70 con 0x7f71681047a0 2024-05-14T16:50:39.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.018+0000 7f716db30700 1 -- 172.21.15.151:0/2838453507 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71681a16c0 con 0x7f71681047a0 2024-05-14T16:50:39.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.018+0000 7f7164ff9700 1 -- 172.21.15.151:0/2838453507 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c00f3c0 con 0x7f71681047a0 2024-05-14T16:50:39.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.018+0000 7f716db30700 1 -- 172.21.15.151:0/2838453507 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71681a1c00 con 0x7f71681047a0 2024-05-14T16:50:39.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.019+0000 7f716db30700 1 -- 172.21.15.151:0/2838453507 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f716806a230 con 0x7f71681047a0 2024-05-14T16:50:39.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.021+0000 7f7164ff9700 1 -- 172.21.15.151:0/2838453507 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f715c019070 con 0x7f71681047a0 2024-05-14T16:50:39.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.021+0000 7f7164ff9700 1 --2- 172.21.15.151:0/2838453507 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f71500b2880 0x7f71500b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:39.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.021+0000 7f7164ff9700 1 -- 172.21.15.151:0/2838453507 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f715c0e18a0 con 0x7f71681047a0 2024-05-14T16:50:39.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.021+0000 7f71677fe700 1 --2- 172.21.15.151:0/2838453507 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f71500b2880 0x7f71500b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:39.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.022+0000 7f71677fe700 1 --2- 172.21.15.151:0/2838453507 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f71500b2880 0x7f71500b4d40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f71580038a0 tx=0x7f7158005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:39.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.025+0000 7f7164ff9700 1 -- 172.21.15.151:0/2838453507 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f715c0a9a60 con 0x7f71681047a0 2024-05-14T16:50:39.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.238+0000 7f716db30700 1 -- 172.21.15.151:0/2838453507 --> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l", "target": ["mon-mgr", ""]}) -- 0x7f7168063b40 con 0x7f71500b2880 2024-05-14T16:50:39.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.246+0000 7f7164ff9700 1 -- 172.21.15.151:0/2838453507 <== mgr.14150 v2:172.21.15.2:6800/1498936028 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f7168063b40 con 0x7f71500b2880 2024-05-14T16:50:39.246 INFO:teuthology.orchestra.run.smithi151.stdout:Scheduled mds.all update... 2024-05-14T16:50:39.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.249+0000 7f716db30700 1 -- 172.21.15.151:0/2838453507 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f71500b2880 msgr2=0x7f71500b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:39.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.249+0000 7f716db30700 1 --2- 172.21.15.151:0/2838453507 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f71500b2880 0x7f71500b4d40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f71580038a0 tx=0x7f7158005fb0 comp rx=0 tx=0).stop 2024-05-14T16:50:39.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.249+0000 7f716db30700 1 -- 172.21.15.151:0/2838453507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71681047a0 msgr2=0x7f7168199cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:39.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.249+0000 7f716db30700 1 --2- 172.21.15.151:0/2838453507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71681047a0 0x7f7168199cb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f715c00d980 tx=0x7f715c00dc90 comp rx=0 tx=0).stop 2024-05-14T16:50:39.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.249+0000 7f716db30700 1 -- 172.21.15.151:0/2838453507 shutdown_connections 2024-05-14T16:50:39.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.249+0000 7f716db30700 1 --2- 172.21.15.151:0/2838453507 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f71500b2880 0x7f71500b4d40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:39.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.249+0000 7f716db30700 1 --2- 172.21.15.151:0/2838453507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7168105140 0x7f71681a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:39.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.249+0000 7f716db30700 1 --2- 172.21.15.151:0/2838453507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71681035a0 0x7f7168199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:39.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.249+0000 7f716db30700 1 --2- 172.21.15.151:0/2838453507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71681047a0 0x7f7168199cb0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:39.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.249+0000 7f716db30700 1 -- 172.21.15.151:0/2838453507 >> 172.21.15.151:0/2838453507 conn(0x7f71680feb40 msgr2=0x7f7168107bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:39.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.249+0000 7f716db30700 1 -- 172.21.15.151:0/2838453507 shutdown_connections 2024-05-14T16:50:39.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-14T16:50:39.249+0000 7f716db30700 1 -- 172.21.15.151:0/2838453507 wait complete. 2024-05-14T16:50:39.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:39 smithi123 ceph-mon[35971]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:39.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:39 smithi002 ceph-mon[31538]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:39.539 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:39 smithi151 ceph-mon[36066]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:39.996 DEBUG:teuthology.orchestra.run.smithi002:mds.a> sudo journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.a.service 2024-05-14T16:50:39.999 DEBUG:teuthology.orchestra.run.smithi002:mds.d> sudo journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.d.service 2024-05-14T16:50:40.045 DEBUG:teuthology.orchestra.run.smithi002:mds.g> sudo journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.g.service 2024-05-14T16:50:40.091 DEBUG:teuthology.orchestra.run.smithi002:mds.j> sudo journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.j.service 2024-05-14T16:50:40.136 DEBUG:teuthology.orchestra.run.smithi123:mds.b> sudo journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.b.service 2024-05-14T16:50:40.138 DEBUG:teuthology.orchestra.run.smithi123:mds.e> sudo journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.e.service 2024-05-14T16:50:40.182 DEBUG:teuthology.orchestra.run.smithi123:mds.h> sudo journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.h.service 2024-05-14T16:50:40.227 DEBUG:teuthology.orchestra.run.smithi123:mds.k> sudo journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.k.service 2024-05-14T16:50:40.272 DEBUG:teuthology.orchestra.run.smithi151:mds.c> sudo journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.c.service 2024-05-14T16:50:40.276 DEBUG:teuthology.orchestra.run.smithi151:mds.f> sudo journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.f.service 2024-05-14T16:50:40.321 DEBUG:teuthology.orchestra.run.smithi151:mds.i> sudo journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.i.service 2024-05-14T16:50:40.365 DEBUG:teuthology.orchestra.run.smithi151:mds.l> sudo journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.l.service 2024-05-14T16:50:40.407 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-14T16:50:40.408 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:50:40.409 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:40 smithi123 ceph-mon[35971]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:50:40.409 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:40 smithi123 ceph-mon[35971]: Saving service mds.all spec with placement smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l;count:12 2024-05-14T16:50:40.409 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:40 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:40.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:40 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:50:40.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:40 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:40.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:40 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:50:40.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:40 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:40.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:40 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:50:40.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:40 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:50:40.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:40 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:40.433 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:40 smithi002 ceph-mon[31538]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:50:40.433 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:40 smithi002 ceph-mon[31538]: Saving service mds.all spec with placement smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l;count:12 2024-05-14T16:50:40.433 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:40 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:40.433 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:40 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:50:40.433 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:40 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:40.434 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:40 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:50:40.434 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:40 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:40.434 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:40 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:50:40.434 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:40 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:50:40.434 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:40 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:40.513 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:40 smithi151 ceph-mon[36066]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:50:40.514 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:40 smithi151 ceph-mon[36066]: Saving service mds.all spec with placement smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l;count:12 2024-05-14T16:50:40.514 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:40 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:40.514 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:40 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:50:40.514 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:40 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:40.514 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:40 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:50:40.514 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:40 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:40.514 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:40 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:50:40.514 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:40 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:50:40.514 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:40 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:40.661 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:50:41.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:41 smithi002 ceph-mon[31538]: Deploying daemon mds.l on smithi151 2024-05-14T16:50:41.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:41 smithi002 ceph-mon[31538]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:41.558 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:41 smithi151 ceph-mon[36066]: Deploying daemon mds.l on smithi151 2024-05-14T16:50:41.558 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:41 smithi151 ceph-mon[36066]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:41.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:41 smithi123 ceph-mon[35971]: Deploying daemon mds.l on smithi151 2024-05-14T16:50:41.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:41 smithi123 ceph-mon[35971]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:42.104 INFO:journalctl@ceph.mds.l.smithi151.stdout:May 14 16:50:42 smithi151 systemd[1]: Starting Ceph mds.l for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T16:50:42.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.388+0000 7fedde573700 1 -- 172.21.15.2:0/195330711 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedd8102440 msgr2=0x7fedd81028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:42.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.388+0000 7fedde573700 1 --2- 172.21.15.2:0/195330711 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedd8102440 0x7fedd81028c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fedc0009990 tx=0x7fedc0009ca0 comp rx=0 tx=0).stop 2024-05-14T16:50:42.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.388+0000 7fedde573700 1 -- 172.21.15.2:0/195330711 shutdown_connections 2024-05-14T16:50:42.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.388+0000 7fedde573700 1 --2- 172.21.15.2:0/195330711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fedd8102e00 0x7fedd810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:42.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.388+0000 7fedde573700 1 --2- 172.21.15.2:0/195330711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedd8108520 0x7fedd8108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:42.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.388+0000 7fedde573700 1 --2- 172.21.15.2:0/195330711 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedd8102440 0x7fedd81028c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:42.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.388+0000 7fedde573700 1 -- 172.21.15.2:0/195330711 >> 172.21.15.2:0/195330711 conn(0x7fedd80793f0 msgr2=0x7fedd8079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:42.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.388+0000 7fedde573700 1 -- 172.21.15.2:0/195330711 shutdown_connections 2024-05-14T16:50:42.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.389+0000 7fedde573700 1 -- 172.21.15.2:0/195330711 wait complete. 2024-05-14T16:50:42.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.389+0000 7fedde573700 1 Processor -- start 2024-05-14T16:50:42.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.389+0000 7fedde573700 1 -- start start 2024-05-14T16:50:42.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.389+0000 7fedde573700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedd8102440 0x7fedd819f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:42.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.389+0000 7fedde573700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fedd8102e00 0x7fedd819fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:42.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.389+0000 7fedde573700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedd8108520 0x7fedd8199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:42.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.389+0000 7fedde573700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fedd81125f0 con 0x7fedd8102440 2024-05-14T16:50:42.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.389+0000 7fedde573700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fedd8112470 con 0x7fedd8108520 2024-05-14T16:50:42.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.389+0000 7fedde573700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fedd8112770 con 0x7fedd8102e00 2024-05-14T16:50:42.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.390+0000 7fedd7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedd8102440 0x7fedd819f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:42.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.390+0000 7fedd7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedd8102440 0x7fedd819f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54202/0 (socket says 172.21.15.2:54202) 2024-05-14T16:50:42.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.390+0000 7fedd7fff700 1 -- 172.21.15.2:0/2665597181 learned_addr learned my addr 172.21.15.2:0/2665597181 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:50:42.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.390+0000 7fedcffff700 1 --2- 172.21.15.2:0/2665597181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fedd8102e00 0x7fedd819fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:42.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.390+0000 7feddcb10700 1 --2- 172.21.15.2:0/2665597181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedd8108520 0x7fedd8199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:42.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.390+0000 7fedd7fff700 1 -- 172.21.15.2:0/2665597181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fedd8102e00 msgr2=0x7fedd819fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:42.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.390+0000 7fedd7fff700 1 --2- 172.21.15.2:0/2665597181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fedd8102e00 0x7fedd819fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:42.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.390+0000 7fedd7fff700 1 -- 172.21.15.2:0/2665597181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedd8108520 msgr2=0x7fedd8199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:42.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.390+0000 7fedd7fff700 1 --2- 172.21.15.2:0/2665597181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedd8108520 0x7fedd8199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:42.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.390+0000 7fedd7fff700 1 -- 172.21.15.2:0/2665597181 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd819a1b0 con 0x7fedd8102440 2024-05-14T16:50:42.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.390+0000 7fedd7fff700 1 --2- 172.21.15.2:0/2665597181 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedd8102440 0x7fedd819f890 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fedc800d980 tx=0x7fedc800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:42.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.390+0000 7fedd5ffb700 1 -- 172.21.15.2:0/2665597181 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc8016070 con 0x7fedd8102440 2024-05-14T16:50:42.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.390+0000 7fedd5ffb700 1 -- 172.21.15.2:0/2665597181 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedc8004830 con 0x7fedd8102440 2024-05-14T16:50:42.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.390+0000 7fedde573700 1 -- 172.21.15.2:0/2665597181 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedd819a4a0 con 0x7fedd8102440 2024-05-14T16:50:42.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.390+0000 7fedd5ffb700 1 -- 172.21.15.2:0/2665597181 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc80054c0 con 0x7fedd8102440 2024-05-14T16:50:42.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.390+0000 7fedde573700 1 -- 172.21.15.2:0/2665597181 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedd81a6760 con 0x7fedd8102440 2024-05-14T16:50:42.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.391+0000 7fedde573700 1 -- 172.21.15.2:0/2665597181 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedd806a230 con 0x7fedd8102440 2024-05-14T16:50:42.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.393+0000 7fedd5ffb700 1 -- 172.21.15.2:0/2665597181 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fedc8004220 con 0x7fedd8102440 2024-05-14T16:50:42.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.393+0000 7fedd5ffb700 1 --2- 172.21.15.2:0/2665597181 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fedc40b2880 0x7fedc40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:42.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.393+0000 7fedcffff700 1 --2- 172.21.15.2:0/2665597181 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fedc40b2880 0x7fedc40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:42.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.394+0000 7fedd5ffb700 1 -- 172.21.15.2:0/2665597181 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fedc80e2020 con 0x7fedd8102440 2024-05-14T16:50:42.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.394+0000 7fedcffff700 1 --2- 172.21.15.2:0/2665597181 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fedc40b2880 0x7fedc40b4d40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fedc000b2e0 tx=0x7fedc0005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:42.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.397+0000 7fedd5ffb700 1 -- 172.21.15.2:0/2665597181 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fedc801f020 con 0x7fedd8102440 2024-05-14T16:50:42.516 INFO:journalctl@ceph.mds.l.smithi151.stdout:May 14 16:50:42 smithi151 podman[66320]: 2024-05-14 16:50:42.148008234 +0000 UTC m=+0.017310752 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T16:50:42.517 INFO:journalctl@ceph.mds.l.smithi151.stdout:May 14 16:50:42 smithi151 podman[66320]: 2024-05-14 16:50:42.261243524 +0000 UTC m=+0.130546037 container create d132161610f97c2f62f0a968f453d9584dfbbbb42258135fc8fc07b6183a8c78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-l, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-14T16:50:42.517 INFO:journalctl@ceph.mds.l.smithi151.stdout:May 14 16:50:42 smithi151 podman[66320]: 2024-05-14 16:50:42.517885192 +0000 UTC m=+0.387187711 container init d132161610f97c2f62f0a968f453d9584dfbbbb42258135fc8fc07b6183a8c78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-l, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240514) 2024-05-14T16:50:42.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.667+0000 7fedde573700 1 -- 172.21.15.2:0/2665597181 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fedd802e560 con 0x7fedd8102440 2024-05-14T16:50:42.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.668+0000 7fedd5ffb700 1 -- 172.21.15.2:0/2665597181 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fedc80aa1e0 con 0x7fedd8102440 2024-05-14T16:50:42.668 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:50:42.668 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1880336731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1880336731},{"type":"v1","addr":"172.21.15.151:6833","nonce":1880336731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-14T16:50:42.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.670+0000 7fedde573700 1 -- 172.21.15.2:0/2665597181 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fedc40b2880 msgr2=0x7fedc40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:42.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.670+0000 7fedde573700 1 --2- 172.21.15.2:0/2665597181 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fedc40b2880 0x7fedc40b4d40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fedc000b2e0 tx=0x7fedc0005960 comp rx=0 tx=0).stop 2024-05-14T16:50:42.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.670+0000 7fedde573700 1 -- 172.21.15.2:0/2665597181 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedd8102440 msgr2=0x7fedd819f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:42.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.670+0000 7fedde573700 1 --2- 172.21.15.2:0/2665597181 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedd8102440 0x7fedd819f890 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fedc800d980 tx=0x7fedc800dc90 comp rx=0 tx=0).stop 2024-05-14T16:50:42.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.670+0000 7fedde573700 1 -- 172.21.15.2:0/2665597181 shutdown_connections 2024-05-14T16:50:42.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.670+0000 7fedde573700 1 --2- 172.21.15.2:0/2665597181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedd8108520 0x7fedd8199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:42.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.670+0000 7fedde573700 1 --2- 172.21.15.2:0/2665597181 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fedc40b2880 0x7fedc40b4d40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:42.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.670+0000 7fedde573700 1 --2- 172.21.15.2:0/2665597181 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedd8102440 0x7fedd819f890 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:42.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.670+0000 7fedde573700 1 --2- 172.21.15.2:0/2665597181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fedd8102e00 0x7fedd819fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:42.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.670+0000 7fedde573700 1 -- 172.21.15.2:0/2665597181 >> 172.21.15.2:0/2665597181 conn(0x7fedd80793f0 msgr2=0x7fedd806b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:42.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.670+0000 7fedde573700 1 -- 172.21.15.2:0/2665597181 shutdown_connections 2024-05-14T16:50:42.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:42.670+0000 7fedde573700 1 -- 172.21.15.2:0/2665597181 wait complete. 2024-05-14T16:50:42.672 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 2 2024-05-14T16:50:42.854 INFO:journalctl@ceph.mds.l.smithi151.stdout:May 14 16:50:42 smithi151 podman[66320]: 2024-05-14 16:50:42.520523249 +0000 UTC m=+0.389825771 container start d132161610f97c2f62f0a968f453d9584dfbbbb42258135fc8fc07b6183a8c78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-l, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T16:50:42.854 INFO:journalctl@ceph.mds.l.smithi151.stdout:May 14 16:50:42 smithi151 ceph-mds[66339]: -- 172.21.15.151:0/4084461457 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5622c1707a40 con 0x5622c16db000 2024-05-14T16:50:42.854 INFO:journalctl@ceph.mds.l.smithi151.stdout:May 14 16:50:42 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-l[66335]: starting mds.l at 2024-05-14T16:50:42.855 INFO:journalctl@ceph.mds.l.smithi151.stdout:May 14 16:50:42 smithi151 bash[66320]: d132161610f97c2f62f0a968f453d9584dfbbbb42258135fc8fc07b6183a8c78 2024-05-14T16:50:42.855 INFO:journalctl@ceph.mds.l.smithi151.stdout:May 14 16:50:42 smithi151 systemd[1]: Started Ceph mds.l for ed40717a-1210-11ef-bc98-c7b262605968. 2024-05-14T16:50:43.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:43 smithi151 ceph-mon[36066]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:43.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:43 smithi151 ceph-mon[36066]: mds.? [v2:172.21.15.151:6832/1880336731,v1:172.21.15.151:6833/1880336731] up:boot 2024-05-14T16:50:43.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:43 smithi151 ceph-mon[36066]: fsmap 1 up:standby 2024-05-14T16:50:43.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:43 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-14T16:50:43.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:43 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/2665597181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:50:43.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:43 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:43.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:43 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:43.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:43 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:43.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:43 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:50:43.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:43 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:50:43.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:43 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:43.605 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:43 smithi123 ceph-mon[35971]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:43.606 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:43 smithi123 ceph-mon[35971]: mds.? [v2:172.21.15.151:6832/1880336731,v1:172.21.15.151:6833/1880336731] up:boot 2024-05-14T16:50:43.606 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:43 smithi123 ceph-mon[35971]: fsmap 1 up:standby 2024-05-14T16:50:43.606 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:43 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-14T16:50:43.606 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:43 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/2665597181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:50:43.606 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:43 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:43.606 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:43 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:43.606 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:43 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:43.606 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:43 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:50:43.606 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:43 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:50:43.606 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:43 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:43.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:43 smithi002 ceph-mon[31538]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:43.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:43 smithi002 ceph-mon[31538]: mds.? [v2:172.21.15.151:6832/1880336731,v1:172.21.15.151:6833/1880336731] up:boot 2024-05-14T16:50:43.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:43 smithi002 ceph-mon[31538]: fsmap 1 up:standby 2024-05-14T16:50:43.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:43 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-14T16:50:43.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:43 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/2665597181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:50:43.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:43 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:43.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:43 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:43.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:43 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:43.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:43 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:50:43.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:43 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:50:43.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:43 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:44.553 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:44 smithi123 ceph-mon[35971]: Deploying daemon mds.k on smithi123 2024-05-14T16:50:44.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:44 smithi151 ceph-mon[36066]: Deploying daemon mds.k on smithi123 2024-05-14T16:50:44.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:44 smithi002 ceph-mon[31538]: Deploying daemon mds.k on smithi123 2024-05-14T16:50:45.256 INFO:journalctl@ceph.mds.k.smithi123.stdout:May 14 16:50:45 smithi123 podman[65355]: 2024-05-14 16:50:45.16493852 +0000 UTC m=+0.129391091 container create 34721ed81fd7ca6b861c5116a67ecb8ce6f1faaeaff514aba5e52cab59e188c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-k, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-14T16:50:45.438 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:50:45.511 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:45 smithi123 ceph-mon[35971]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:45.511 INFO:journalctl@ceph.mds.k.smithi123.stdout:May 14 16:50:45 smithi123 podman[65355]: 2024-05-14 16:50:45.404233694 +0000 UTC m=+0.368686283 container init 34721ed81fd7ca6b861c5116a67ecb8ce6f1faaeaff514aba5e52cab59e188c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, ceph=True) 2024-05-14T16:50:45.511 INFO:journalctl@ceph.mds.k.smithi123.stdout:May 14 16:50:45 smithi123 podman[65355]: 2024-05-14 16:50:45.407200786 +0000 UTC m=+0.371653373 container start 34721ed81fd7ca6b861c5116a67ecb8ce6f1faaeaff514aba5e52cab59e188c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-k, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, ceph=True) 2024-05-14T16:50:45.511 INFO:journalctl@ceph.mds.k.smithi123.stdout:May 14 16:50:45 smithi123 ceph-mds[65374]: -- 172.21.15.123:0/1698810965 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558f2c25da40 con 0x558f2c230800 2024-05-14T16:50:45.511 INFO:journalctl@ceph.mds.k.smithi123.stdout:May 14 16:50:45 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-k[65370]: starting mds.k at 2024-05-14T16:50:45.511 INFO:journalctl@ceph.mds.k.smithi123.stdout:May 14 16:50:45 smithi123 bash[65355]: 34721ed81fd7ca6b861c5116a67ecb8ce6f1faaeaff514aba5e52cab59e188c9 2024-05-14T16:50:45.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:45 smithi151 ceph-mon[36066]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:45.710 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:50:45.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:45 smithi002 ceph-mon[31538]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:45.909 INFO:journalctl@ceph.mds.k.smithi123.stdout:May 14 16:50:45 smithi123 systemd[1]: Started Ceph mds.k for ed40717a-1210-11ef-bc98-c7b262605968. 2024-05-14T16:50:46.833 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:46 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:46.833 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:46 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:46.833 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:46 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:46.833 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:46 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:50:46.833 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:46 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:50:46.833 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:46 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:46.833 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:46 smithi123 ceph-mon[35971]: Deploying daemon mds.j on smithi002 2024-05-14T16:50:46.833 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:46 smithi123 ceph-mon[35971]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:46.833 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:46 smithi123 ceph-mon[35971]: mds.? [v2:172.21.15.123:6832/2635167,v1:172.21.15.123:6833/2635167] up:boot 2024-05-14T16:50:46.833 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:46 smithi123 ceph-mon[35971]: fsmap 2 up:standby 2024-05-14T16:50:46.834 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:46 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-14T16:50:46.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:46 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:46.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:46 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:46.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:46 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:46.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:46 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:50:46.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:46 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:50:46.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:46 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:46.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:46 smithi151 ceph-mon[36066]: Deploying daemon mds.j on smithi002 2024-05-14T16:50:46.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:46 smithi151 ceph-mon[36066]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:46.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:46 smithi151 ceph-mon[36066]: mds.? [v2:172.21.15.123:6832/2635167,v1:172.21.15.123:6833/2635167] up:boot 2024-05-14T16:50:46.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:46 smithi151 ceph-mon[36066]: fsmap 2 up:standby 2024-05-14T16:50:46.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:46 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-14T16:50:46.954 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:46 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:46.955 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:46 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:46.955 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:46 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:46.955 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:46 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:50:46.955 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:46 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:50:46.955 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:46 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:46.955 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:46 smithi002 ceph-mon[31538]: Deploying daemon mds.j on smithi002 2024-05-14T16:50:46.955 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:46 smithi002 ceph-mon[31538]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:46.955 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:46 smithi002 ceph-mon[31538]: mds.? [v2:172.21.15.123:6832/2635167,v1:172.21.15.123:6833/2635167] up:boot 2024-05-14T16:50:46.955 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:46 smithi002 ceph-mon[31538]: fsmap 2 up:standby 2024-05-14T16:50:46.955 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:46 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-14T16:50:48.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.121+0000 7f8473b3f700 1 -- 172.21.15.2:0/990304089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f846c1041d0 msgr2=0x7f846c1081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:48.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.121+0000 7f8473b3f700 1 --2- 172.21.15.2:0/990304089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f846c1041d0 0x7f846c1081e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f84600099c0 tx=0x7f8460009cd0 comp rx=0 tx=0).stop 2024-05-14T16:50:48.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.121+0000 7f8473b3f700 1 -- 172.21.15.2:0/990304089 shutdown_connections 2024-05-14T16:50:48.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.121+0000 7f8473b3f700 1 --2- 172.21.15.2:0/990304089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f846c108910 0x7f846c110490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:48.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.121+0000 7f8473b3f700 1 --2- 172.21.15.2:0/990304089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f846c103820 0x7f846c103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:48.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.121+0000 7f8473b3f700 1 --2- 172.21.15.2:0/990304089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f846c1041d0 0x7f846c1081e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:48.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.121+0000 7f8473b3f700 1 -- 172.21.15.2:0/990304089 >> 172.21.15.2:0/990304089 conn(0x7f846c07a6b0 msgr2=0x7f846c07aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:48.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.121+0000 7f8473b3f700 1 -- 172.21.15.2:0/990304089 shutdown_connections 2024-05-14T16:50:48.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.121+0000 7f8473b3f700 1 -- 172.21.15.2:0/990304089 wait complete. 2024-05-14T16:50:48.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.122+0000 7f8473b3f700 1 Processor -- start 2024-05-14T16:50:48.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.122+0000 7f8473b3f700 1 -- start start 2024-05-14T16:50:48.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.122+0000 7f8473b3f700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f846c103820 0x7f846c19f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:48.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.122+0000 7f8473b3f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f846c1041d0 0x7f846c19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:48.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.122+0000 7f8473b3f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f846c108910 0x7f846c199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:48.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.122+0000 7f8473b3f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f846c113730 con 0x7f846c108910 2024-05-14T16:50:48.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.122+0000 7f8473b3f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f846c1135b0 con 0x7f846c1041d0 2024-05-14T16:50:48.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.122+0000 7f8473b3f700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f846c1138b0 con 0x7f846c103820 2024-05-14T16:50:48.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.122+0000 7f84720dc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f846c108910 0x7f846c199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:48.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.122+0000 7f84718db700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f846c103820 0x7f846c19f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:48.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.122+0000 7f84720dc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f846c108910 0x7f846c199950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41460/0 (socket says 172.21.15.2:41460) 2024-05-14T16:50:48.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.122+0000 7f84718db700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f846c103820 0x7f846c19f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:59546/0 (socket says 172.21.15.2:59546) 2024-05-14T16:50:48.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.122+0000 7f84720dc700 1 -- 172.21.15.2:0/1516707417 learned_addr learned my addr 172.21.15.2:0/1516707417 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:50:48.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.122+0000 7f84710da700 1 --2- 172.21.15.2:0/1516707417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f846c1041d0 0x7f846c19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:48.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.123+0000 7f84720dc700 1 -- 172.21.15.2:0/1516707417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f846c103820 msgr2=0x7f846c19f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:48.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.123+0000 7f84720dc700 1 --2- 172.21.15.2:0/1516707417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f846c103820 0x7f846c19f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:48.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.123+0000 7f84720dc700 1 -- 172.21.15.2:0/1516707417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f846c1041d0 msgr2=0x7f846c19fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:48.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.123+0000 7f84720dc700 1 --2- 172.21.15.2:0/1516707417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f846c1041d0 0x7f846c19fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:48.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.123+0000 7f84720dc700 1 -- 172.21.15.2:0/1516707417 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f846c19a0b0 con 0x7f846c108910 2024-05-14T16:50:48.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.123+0000 7f84720dc700 1 --2- 172.21.15.2:0/1516707417 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f846c108910 0x7f846c199950 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8464007d70 tx=0x7f846400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:48.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.123+0000 7f845effd700 1 -- 172.21.15.2:0/1516707417 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8464016020 con 0x7f846c108910 2024-05-14T16:50:48.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.123+0000 7f845effd700 1 -- 172.21.15.2:0/1516707417 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f846400ac90 con 0x7f846c108910 2024-05-14T16:50:48.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.123+0000 7f8473b3f700 1 -- 172.21.15.2:0/1516707417 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f846c19a3a0 con 0x7f846c108910 2024-05-14T16:50:48.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.123+0000 7f845effd700 1 -- 172.21.15.2:0/1516707417 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84640115e0 con 0x7f846c108910 2024-05-14T16:50:48.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.123+0000 7f8473b3f700 1 -- 172.21.15.2:0/1516707417 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f846c1a6660 con 0x7f846c108910 2024-05-14T16:50:48.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.124+0000 7f8473b3f700 1 -- 172.21.15.2:0/1516707417 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f846c06a230 con 0x7f846c108910 2024-05-14T16:50:48.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.126+0000 7f845effd700 1 -- 172.21.15.2:0/1516707417 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f8464019020 con 0x7f846c108910 2024-05-14T16:50:48.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.126+0000 7f845effd700 1 --2- 172.21.15.2:0/1516707417 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f84580b2900 0x7f84580b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:48.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.126+0000 7f84718db700 1 --2- 172.21.15.2:0/1516707417 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f84580b2900 0x7f84580b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:48.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.126+0000 7f845effd700 1 -- 172.21.15.2:0/1516707417 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f84640e1c70 con 0x7f846c108910 2024-05-14T16:50:48.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.127+0000 7f84718db700 1 --2- 172.21.15.2:0/1516707417 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f84580b2900 0x7f84580b4dc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f846c19aa90 tx=0x7f846800a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:48.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.131+0000 7f845effd700 1 -- 172.21.15.2:0/1516707417 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8464020020 con 0x7f846c108910 2024-05-14T16:50:48.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.399+0000 7f8473b3f700 1 -- 172.21.15.2:0/1516707417 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f846c02e560 con 0x7f846c108910 2024-05-14T16:50:48.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.400+0000 7f845effd700 1 -- 172.21.15.2:0/1516707417 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2215 (secure 0 0 0) 0x7f84640a9e30 con 0x7f846c108910 2024-05-14T16:50:48.400 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:50:48.400 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1880336731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1880336731},{"type":"v1","addr":"172.21.15.151:6833","nonce":1880336731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2635167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2635167},{"type":"v1","addr":"172.21.15.123:6833","nonce":2635167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-14T16:50:48.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.402+0000 7f8473b3f700 1 -- 172.21.15.2:0/1516707417 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f84580b2900 msgr2=0x7f84580b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:48.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.403+0000 7f8473b3f700 1 --2- 172.21.15.2:0/1516707417 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f84580b2900 0x7f84580b4dc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f846c19aa90 tx=0x7f846800a380 comp rx=0 tx=0).stop 2024-05-14T16:50:48.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.403+0000 7f8473b3f700 1 -- 172.21.15.2:0/1516707417 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f846c108910 msgr2=0x7f846c199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:48.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.403+0000 7f8473b3f700 1 --2- 172.21.15.2:0/1516707417 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f846c108910 0x7f846c199950 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8464007d70 tx=0x7f846400a340 comp rx=0 tx=0).stop 2024-05-14T16:50:48.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.403+0000 7f8473b3f700 1 -- 172.21.15.2:0/1516707417 shutdown_connections 2024-05-14T16:50:48.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.403+0000 7f8473b3f700 1 --2- 172.21.15.2:0/1516707417 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f84580b2900 0x7f84580b4dc0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:48.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.403+0000 7f8473b3f700 1 --2- 172.21.15.2:0/1516707417 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f846c108910 0x7f846c199950 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:48.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.403+0000 7f8473b3f700 1 --2- 172.21.15.2:0/1516707417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f846c103820 0x7f846c19f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:48.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.403+0000 7f8473b3f700 1 --2- 172.21.15.2:0/1516707417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f846c1041d0 0x7f846c19fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:48.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.403+0000 7f8473b3f700 1 -- 172.21.15.2:0/1516707417 >> 172.21.15.2:0/1516707417 conn(0x7f846c07a6b0 msgr2=0x7f846c101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:48.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.403+0000 7f8473b3f700 1 -- 172.21.15.2:0/1516707417 shutdown_connections 2024-05-14T16:50:48.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:48.403+0000 7f8473b3f700 1 -- 172.21.15.2:0/1516707417 wait complete. 2024-05-14T16:50:48.404 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 3 2024-05-14T16:50:49.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:49 smithi002 ceph-mon[31538]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:49.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:49 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/1516707417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:50:49.244 INFO:journalctl@ceph.mds.j.smithi002.stdout:May 14 16:50:49 smithi002 systemd[1]: Starting Ceph mds.j for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T16:50:49.409 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:49 smithi123 ceph-mon[35971]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:49.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:49 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/1516707417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:50:49.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:49 smithi151 ceph-mon[36066]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:49.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:49 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/1516707417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:50:49.696 INFO:journalctl@ceph.mds.j.smithi002.stdout:May 14 16:50:49 smithi002 podman[71614]: 2024-05-14 16:50:49.468381415 +0000 UTC m=+0.017815109 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T16:50:49.696 INFO:journalctl@ceph.mds.j.smithi002.stdout:May 14 16:50:49 smithi002 podman[71614]: 2024-05-14 16:50:49.565742983 +0000 UTC m=+0.115176709 container create 327406706a1bf2e7c10b834c4d697166b01225316dfaecbc7247ed70fdc5c665 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-14T16:50:50.245 INFO:journalctl@ceph.mds.j.smithi002.stdout:May 14 16:50:49 smithi002 podman[71614]: 2024-05-14 16:50:49.826508412 +0000 UTC m=+0.375942098 container init 327406706a1bf2e7c10b834c4d697166b01225316dfaecbc7247ed70fdc5c665 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-j, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-14T16:50:50.245 INFO:journalctl@ceph.mds.j.smithi002.stdout:May 14 16:50:49 smithi002 podman[71614]: 2024-05-14 16:50:49.829354517 +0000 UTC m=+0.378788201 container start 327406706a1bf2e7c10b834c4d697166b01225316dfaecbc7247ed70fdc5c665 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-j, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514) 2024-05-14T16:50:50.245 INFO:journalctl@ceph.mds.j.smithi002.stdout:May 14 16:50:49 smithi002 ceph-mds[71633]: -- 172.21.15.2:0/3221752914 <== mon.0 v2:172.21.15.2:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5629cf3a9a40 con 0x5629cf37cc00 2024-05-14T16:50:50.245 INFO:journalctl@ceph.mds.j.smithi002.stdout:May 14 16:50:49 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-j[71629]: starting mds.j at 2024-05-14T16:50:50.245 INFO:journalctl@ceph.mds.j.smithi002.stdout:May 14 16:50:49 smithi002 bash[71614]: 327406706a1bf2e7c10b834c4d697166b01225316dfaecbc7247ed70fdc5c665 2024-05-14T16:50:50.245 INFO:journalctl@ceph.mds.j.smithi002.stdout:May 14 16:50:49 smithi002 systemd[1]: Started Ceph mds.j for ed40717a-1210-11ef-bc98-c7b262605968. 2024-05-14T16:50:51.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:50 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:51.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:50 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:51.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:50 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:51.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:50 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:50:51.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:50 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:50:51.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:50 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:51.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:50 smithi002 ceph-mon[31538]: Deploying daemon mds.d on smithi002 2024-05-14T16:50:51.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:50 smithi002 ceph-mon[31538]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:51.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:50 smithi002 ceph-mon[31538]: mds.? [v2:172.21.15.2:6834/2275696923,v1:172.21.15.2:6835/2275696923] up:boot 2024-05-14T16:50:51.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:50 smithi002 ceph-mon[31538]: fsmap 3 up:standby 2024-05-14T16:50:51.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:50 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-14T16:50:51.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:50 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:51.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:50 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:51.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:50 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:51.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:50 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:50:51.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:50 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:50:51.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:50 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:51.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:50 smithi151 ceph-mon[36066]: Deploying daemon mds.d on smithi002 2024-05-14T16:50:51.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:50 smithi151 ceph-mon[36066]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:51.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:50 smithi151 ceph-mon[36066]: mds.? [v2:172.21.15.2:6834/2275696923,v1:172.21.15.2:6835/2275696923] up:boot 2024-05-14T16:50:51.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:50 smithi151 ceph-mon[36066]: fsmap 3 up:standby 2024-05-14T16:50:51.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:50 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-14T16:50:51.384 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:50:51.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:50 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:51.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:50 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:51.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:50 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:51.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:50 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:50:51.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:50 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:50:51.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:50 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:51.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:50 smithi123 ceph-mon[35971]: Deploying daemon mds.d on smithi002 2024-05-14T16:50:51.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:50 smithi123 ceph-mon[35971]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:51.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:50 smithi123 ceph-mon[35971]: mds.? [v2:172.21.15.2:6834/2275696923,v1:172.21.15.2:6835/2275696923] up:boot 2024-05-14T16:50:51.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:50 smithi123 ceph-mon[35971]: fsmap 3 up:standby 2024-05-14T16:50:51.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:50 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-14T16:50:51.765 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:50:53.383 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:53 smithi002 ceph-mon[31538]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:53.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:53 smithi123 ceph-mon[35971]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:53.432 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:53 smithi151 ceph-mon[36066]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:54.074 INFO:journalctl@ceph.mds.d.smithi002.stdout:May 14 16:50:53 smithi002 systemd[1]: Started Ceph mds.d for ed40717a-1210-11ef-bc98-c7b262605968. 2024-05-14T16:50:54.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.855+0000 7f24ed665700 1 -- 172.21.15.2:0/1271428751 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24e8108620 msgr2=0x7f24e8108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:54.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.855+0000 7f24ed665700 1 --2- 172.21.15.2:0/1271428751 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24e8108620 0x7f24e8108a00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f24d8009990 tx=0x7f24d8009ca0 comp rx=0 tx=0).stop 2024-05-14T16:50:54.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.855+0000 7f24ed665700 1 -- 172.21.15.2:0/1271428751 shutdown_connections 2024-05-14T16:50:54.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.855+0000 7f24ed665700 1 --2- 172.21.15.2:0/1271428751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24e81028c0 0x7f24e8110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:54.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.855+0000 7f24ed665700 1 --2- 172.21.15.2:0/1271428751 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24e8108620 0x7f24e8108a00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:54.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.855+0000 7f24ed665700 1 --2- 172.21.15.2:0/1271428751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f24e8101f00 0x7f24e8102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:54.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.855+0000 7f24ed665700 1 -- 172.21.15.2:0/1271428751 >> 172.21.15.2:0/1271428751 conn(0x7f24e8077780 msgr2=0x7f24e8077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:54.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.855+0000 7f24ed665700 1 -- 172.21.15.2:0/1271428751 shutdown_connections 2024-05-14T16:50:54.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.855+0000 7f24ed665700 1 -- 172.21.15.2:0/1271428751 wait complete. 2024-05-14T16:50:54.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.855+0000 7f24ed665700 1 Processor -- start 2024-05-14T16:50:54.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.856+0000 7f24ed665700 1 -- start start 2024-05-14T16:50:54.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.856+0000 7f24ed665700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24e8101f00 0x7f24e819f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:54.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.856+0000 7f24ed665700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24e81028c0 0x7f24e819fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:54.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.856+0000 7f24ed665700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f24e8108620 0x7f24e8199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:54.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.856+0000 7f24ed665700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f24e8113770 con 0x7f24e8101f00 2024-05-14T16:50:54.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.856+0000 7f24ed665700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f24e81135f0 con 0x7f24e81028c0 2024-05-14T16:50:54.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.856+0000 7f24ed665700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f24e81138f0 con 0x7f24e8108620 2024-05-14T16:50:54.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.856+0000 7f24e6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24e8101f00 0x7f24e819f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:54.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.856+0000 7f24e6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24e8101f00 0x7f24e819f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36824/0 (socket says 172.21.15.2:36824) 2024-05-14T16:50:54.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.856+0000 7f24e6ffd700 1 -- 172.21.15.2:0/2629624836 learned_addr learned my addr 172.21.15.2:0/2629624836 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:50:54.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.856+0000 7f24e77fe700 1 --2- 172.21.15.2:0/2629624836 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f24e8108620 0x7f24e8199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:54.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.856+0000 7f24e67fc700 1 --2- 172.21.15.2:0/2629624836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24e81028c0 0x7f24e819fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:54.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.856+0000 7f24e6ffd700 1 -- 172.21.15.2:0/2629624836 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f24e8108620 msgr2=0x7f24e8199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:54.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.856+0000 7f24e6ffd700 1 --2- 172.21.15.2:0/2629624836 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f24e8108620 0x7f24e8199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:54.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.856+0000 7f24e6ffd700 1 -- 172.21.15.2:0/2629624836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24e81028c0 msgr2=0x7f24e819fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:54.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.856+0000 7f24e6ffd700 1 --2- 172.21.15.2:0/2629624836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24e81028c0 0x7f24e819fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:54.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.856+0000 7f24e6ffd700 1 -- 172.21.15.2:0/2629624836 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e819a250 con 0x7f24e8101f00 2024-05-14T16:50:54.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.856+0000 7f24e77fe700 1 --2- 172.21.15.2:0/2629624836 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f24e8108620 0x7f24e8199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:50:54.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.856+0000 7f24e67fc700 1 --2- 172.21.15.2:0/2629624836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24e81028c0 0x7f24e819fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:50:54.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.857+0000 7f24e6ffd700 1 --2- 172.21.15.2:0/2629624836 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24e8101f00 0x7f24e819f930 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f24d8000c00 tx=0x7f24d800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:54.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.857+0000 7f24cffff700 1 -- 172.21.15.2:0/2629624836 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d8027070 con 0x7f24e8101f00 2024-05-14T16:50:54.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.857+0000 7f24ed665700 1 -- 172.21.15.2:0/2629624836 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24e819a4e0 con 0x7f24e8101f00 2024-05-14T16:50:54.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.857+0000 7f24cffff700 1 -- 172.21.15.2:0/2629624836 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24d8022470 con 0x7f24e8101f00 2024-05-14T16:50:54.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.857+0000 7f24cffff700 1 -- 172.21.15.2:0/2629624836 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d8017650 con 0x7f24e8101f00 2024-05-14T16:50:54.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.857+0000 7f24ed665700 1 -- 172.21.15.2:0/2629624836 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24e8103010 con 0x7f24e8101f00 2024-05-14T16:50:54.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.858+0000 7f24ed665700 1 -- 172.21.15.2:0/2629624836 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24e806a230 con 0x7f24e8101f00 2024-05-14T16:50:54.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.859+0000 7f24cffff700 1 -- 172.21.15.2:0/2629624836 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f24d802a070 con 0x7f24e8101f00 2024-05-14T16:50:54.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.861+0000 7f24cffff700 1 --2- 172.21.15.2:0/2629624836 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f24d00b2880 0x7f24d00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:54.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.861+0000 7f24cffff700 1 -- 172.21.15.2:0/2629624836 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f24d80e2ad0 con 0x7f24e8101f00 2024-05-14T16:50:54.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.861+0000 7f24e67fc700 1 --2- 172.21.15.2:0/2629624836 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f24d00b2880 0x7f24d00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:54.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.861+0000 7f24e67fc700 1 --2- 172.21.15.2:0/2629624836 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f24d00b2880 0x7f24d00b4d40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f24e8077e30 tx=0x7f24dc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:54.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:53.866+0000 7f24cffff700 1 -- 172.21.15.2:0/2629624836 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f24d8013070 con 0x7f24e8101f00 2024-05-14T16:50:54.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:54.146+0000 7f24ed665700 1 -- 172.21.15.2:0/2629624836 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24e802e560 con 0x7f24e8101f00 2024-05-14T16:50:54.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:54.147+0000 7f24cffff700 1 -- 172.21.15.2:0/2629624836 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3859 (secure 0 0 0) 0x7f24d80aad20 con 0x7f24e8101f00 2024-05-14T16:50:54.147 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:50:54.147 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2275696923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2275696923},{"type":"v1","addr":"172.21.15.2:6835","nonce":2275696923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1880336731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1880336731},{"type":"v1","addr":"172.21.15.151:6833","nonce":1880336731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2415605548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2415605548},{"type":"v1","addr":"172.21.15.2:6837","nonce":2415605548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2635167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2635167},{"type":"v1","addr":"172.21.15.123:6833","nonce":2635167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-14T16:50:54.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:54.149+0000 7f24ed665700 1 -- 172.21.15.2:0/2629624836 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f24d00b2880 msgr2=0x7f24d00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:54.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:54.149+0000 7f24ed665700 1 --2- 172.21.15.2:0/2629624836 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f24d00b2880 0x7f24d00b4d40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f24e8077e30 tx=0x7f24dc009260 comp rx=0 tx=0).stop 2024-05-14T16:50:54.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:54.149+0000 7f24ed665700 1 -- 172.21.15.2:0/2629624836 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24e8101f00 msgr2=0x7f24e819f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:54.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:54.149+0000 7f24ed665700 1 --2- 172.21.15.2:0/2629624836 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24e8101f00 0x7f24e819f930 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f24d8000c00 tx=0x7f24d800bfd0 comp rx=0 tx=0).stop 2024-05-14T16:50:54.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:54.149+0000 7f24ed665700 1 -- 172.21.15.2:0/2629624836 shutdown_connections 2024-05-14T16:50:54.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:54.149+0000 7f24ed665700 1 --2- 172.21.15.2:0/2629624836 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f24e8108620 0x7f24e8199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:54.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:54.149+0000 7f24ed665700 1 --2- 172.21.15.2:0/2629624836 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f24d00b2880 0x7f24d00b4d40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:54.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:54.149+0000 7f24ed665700 1 --2- 172.21.15.2:0/2629624836 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24e8101f00 0x7f24e819f930 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:54.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:54.149+0000 7f24ed665700 1 --2- 172.21.15.2:0/2629624836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24e81028c0 0x7f24e819fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:54.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:54.149+0000 7f24ed665700 1 -- 172.21.15.2:0/2629624836 >> 172.21.15.2:0/2629624836 conn(0x7f24e8077780 msgr2=0x7f24e8100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:54.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:54.149+0000 7f24ed665700 1 -- 172.21.15.2:0/2629624836 shutdown_connections 2024-05-14T16:50:54.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:54.150+0000 7f24ed665700 1 -- 172.21.15.2:0/2629624836 wait complete. 2024-05-14T16:50:54.150 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 5 2024-05-14T16:50:55.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:54 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:55.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:54 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:55.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:54 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:55.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:54 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:50:55.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:54 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:50:55.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:54 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:55.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:54 smithi002 ceph-mon[31538]: Deploying daemon mds.e on smithi123 2024-05-14T16:50:55.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:54 smithi002 ceph-mon[31538]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:55.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:54 smithi002 ceph-mon[31538]: mds.? [v2:172.21.15.2:6836/2415605548,v1:172.21.15.2:6837/2415605548] up:boot 2024-05-14T16:50:55.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:54 smithi002 ceph-mon[31538]: fsmap 4 up:standby 2024-05-14T16:50:55.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:54 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-14T16:50:55.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:54 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/2629624836' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:50:55.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:54 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:55.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:54 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:55.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:54 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:55.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:54 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:50:55.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:54 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:50:55.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:54 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:55.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:54 smithi151 ceph-mon[36066]: Deploying daemon mds.e on smithi123 2024-05-14T16:50:55.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:54 smithi151 ceph-mon[36066]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:55.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:54 smithi151 ceph-mon[36066]: mds.? [v2:172.21.15.2:6836/2415605548,v1:172.21.15.2:6837/2415605548] up:boot 2024-05-14T16:50:55.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:54 smithi151 ceph-mon[36066]: fsmap 4 up:standby 2024-05-14T16:50:55.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:54 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-14T16:50:55.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:54 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/2629624836' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:50:55.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:54 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:55.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:54 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:55.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:54 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:55.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:54 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:50:55.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:54 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:50:55.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:54 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:55.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:54 smithi123 ceph-mon[35971]: Deploying daemon mds.e on smithi123 2024-05-14T16:50:55.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:54 smithi123 ceph-mon[35971]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:55.412 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:54 smithi123 ceph-mon[35971]: mds.? [v2:172.21.15.2:6836/2415605548,v1:172.21.15.2:6837/2415605548] up:boot 2024-05-14T16:50:55.412 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:54 smithi123 ceph-mon[35971]: fsmap 4 up:standby 2024-05-14T16:50:55.412 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:54 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-14T16:50:55.412 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:54 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/2629624836' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:50:56.339 INFO:journalctl@ceph.mds.e.smithi123.stdout:May 14 16:50:56 smithi123 systemd[1]: Starting Ceph mds.e for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T16:50:56.590 INFO:journalctl@ceph.mds.e.smithi123.stdout:May 14 16:50:56 smithi123 podman[65672]: 2024-05-14 16:50:56.459106146 +0000 UTC m=+0.017225576 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T16:50:56.590 INFO:journalctl@ceph.mds.e.smithi123.stdout:May 14 16:50:56 smithi123 podman[65672]: 2024-05-14 16:50:56.591159806 +0000 UTC m=+0.149279279 container create 1b372958c4df314f2107950dbceba89e1e80456f6c423fc4bd4cd85d0e6d1c10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-e, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-14T16:50:56.790 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:50:57.060 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:50:57.130 INFO:journalctl@ceph.mds.e.smithi123.stdout:May 14 16:50:56 smithi123 podman[65672]: 2024-05-14 16:50:56.867021269 +0000 UTC m=+0.425140699 container init 1b372958c4df314f2107950dbceba89e1e80456f6c423fc4bd4cd85d0e6d1c10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True) 2024-05-14T16:50:57.130 INFO:journalctl@ceph.mds.e.smithi123.stdout:May 14 16:50:56 smithi123 podman[65672]: 2024-05-14 16:50:56.86987747 +0000 UTC m=+0.427996909 container start 1b372958c4df314f2107950dbceba89e1e80456f6c423fc4bd4cd85d0e6d1c10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-e, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True) 2024-05-14T16:50:57.130 INFO:journalctl@ceph.mds.e.smithi123.stdout:May 14 16:50:56 smithi123 ceph-mds[65692]: -- 172.21.15.123:0/2841033095 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ec146b5a40 con 0x55ec14688800 2024-05-14T16:50:57.130 INFO:journalctl@ceph.mds.e.smithi123.stdout:May 14 16:50:56 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-e[65688]: starting mds.e at 2024-05-14T16:50:57.130 INFO:journalctl@ceph.mds.e.smithi123.stdout:May 14 16:50:56 smithi123 bash[65672]: 1b372958c4df314f2107950dbceba89e1e80456f6c423fc4bd4cd85d0e6d1c10 2024-05-14T16:50:57.130 INFO:journalctl@ceph.mds.e.smithi123.stdout:May 14 16:50:56 smithi123 systemd[1]: Started Ceph mds.e for ed40717a-1210-11ef-bc98-c7b262605968. 2024-05-14T16:50:57.409 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:57 smithi123 ceph-mon[35971]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:57.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:57 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:57.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:57 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:57.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:57 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:57.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:57 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:50:57.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:57 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:50:57.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:57 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:57.457 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:57 smithi002 ceph-mon[31538]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:57.457 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:57 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:57.457 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:57 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:57.457 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:57 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:57.457 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:57 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:50:57.457 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:57 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:50:57.458 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:57 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:57.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:57 smithi151 ceph-mon[36066]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:57.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:57 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:57.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:57 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:57.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:57 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:50:57.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:57 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:50:57.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:57 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:50:57.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:57 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:50:58.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:58 smithi151 ceph-mon[36066]: Deploying daemon mds.f on smithi151 2024-05-14T16:50:58.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:58 smithi151 ceph-mon[36066]: mds.? [v2:172.21.15.123:6834/1198670013,v1:172.21.15.123:6835/1198670013] up:boot 2024-05-14T16:50:58.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:58 smithi151 ceph-mon[36066]: fsmap 5 up:standby 2024-05-14T16:50:58.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:58 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T16:50:58.408 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:58 smithi002 ceph-mon[31538]: Deploying daemon mds.f on smithi151 2024-05-14T16:50:58.408 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:58 smithi002 ceph-mon[31538]: mds.? [v2:172.21.15.123:6834/1198670013,v1:172.21.15.123:6835/1198670013] up:boot 2024-05-14T16:50:58.408 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:58 smithi002 ceph-mon[31538]: fsmap 5 up:standby 2024-05-14T16:50:58.408 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:58 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T16:50:58.409 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:58 smithi123 ceph-mon[35971]: Deploying daemon mds.f on smithi151 2024-05-14T16:50:58.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:58 smithi123 ceph-mon[35971]: mds.? [v2:172.21.15.123:6834/1198670013,v1:172.21.15.123:6835/1198670013] up:boot 2024-05-14T16:50:58.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:58 smithi123 ceph-mon[35971]: fsmap 5 up:standby 2024-05-14T16:50:58.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:58 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T16:50:58.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.757+0000 7f0088e2f700 1 -- 172.21.15.2:0/2585047708 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00841037f0 msgr2=0x7f0084103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:58.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.757+0000 7f0088e2f700 1 --2- 172.21.15.2:0/2585047708 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00841037f0 0x7f0084103bd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f00740099c0 tx=0x7f0074009cd0 comp rx=0 tx=0).stop 2024-05-14T16:50:58.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.757+0000 7f0088e2f700 1 -- 172.21.15.2:0/2585047708 shutdown_connections 2024-05-14T16:50:58.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.757+0000 7f0088e2f700 1 --2- 172.21.15.2:0/2585047708 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f00841088f0 0x7f0084110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:58.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.757+0000 7f0088e2f700 1 --2- 172.21.15.2:0/2585047708 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00841037f0 0x7f0084103bd0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:58.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.757+0000 7f0088e2f700 1 --2- 172.21.15.2:0/2585047708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00841041a0 0x7f00841081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:58.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.757+0000 7f0088e2f700 1 -- 172.21.15.2:0/2585047708 >> 172.21.15.2:0/2585047708 conn(0x7f008407a6a0 msgr2=0x7f008407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:58.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.757+0000 7f0088e2f700 1 -- 172.21.15.2:0/2585047708 shutdown_connections 2024-05-14T16:50:58.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.757+0000 7f0088e2f700 1 -- 172.21.15.2:0/2585047708 wait complete. 2024-05-14T16:50:58.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.758+0000 7f0088e2f700 1 Processor -- start 2024-05-14T16:50:58.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.758+0000 7f0088e2f700 1 -- start start 2024-05-14T16:50:58.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.758+0000 7f0088e2f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00841037f0 0x7f008419f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:58.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.758+0000 7f0088e2f700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f00841041a0 0x7f008419fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:58.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.758+0000 7f0088e2f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00841088f0 0x7f0084199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:58.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.758+0000 7f0088e2f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f00841136c0 con 0x7f00841037f0 2024-05-14T16:50:58.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.758+0000 7f0088e2f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0084113540 con 0x7f00841088f0 2024-05-14T16:50:58.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.758+0000 7f0088e2f700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0084113840 con 0x7f00841041a0 2024-05-14T16:50:58.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.758+0000 7f008259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00841037f0 0x7f008419f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:58.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.758+0000 7f008259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00841037f0 0x7f008419f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36858/0 (socket says 172.21.15.2:36858) 2024-05-14T16:50:58.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.758+0000 7f008259c700 1 -- 172.21.15.2:0/2203438939 learned_addr learned my addr 172.21.15.2:0/2203438939 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:50:58.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.758+0000 7f0081d9b700 1 --2- 172.21.15.2:0/2203438939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f00841041a0 0x7f008419fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:58.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.758+0000 7f0082d9d700 1 --2- 172.21.15.2:0/2203438939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00841088f0 0x7f0084199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:58.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.758+0000 7f008259c700 1 -- 172.21.15.2:0/2203438939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f00841041a0 msgr2=0x7f008419fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:58.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.758+0000 7f008259c700 1 --2- 172.21.15.2:0/2203438939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f00841041a0 0x7f008419fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:58.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.758+0000 7f008259c700 1 -- 172.21.15.2:0/2203438939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00841088f0 msgr2=0x7f0084199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:58.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.758+0000 7f008259c700 1 --2- 172.21.15.2:0/2203438939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00841088f0 0x7f0084199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:58.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.758+0000 7f008259c700 1 -- 172.21.15.2:0/2203438939 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f008419a160 con 0x7f00841037f0 2024-05-14T16:50:58.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.758+0000 7f0081d9b700 1 --2- 172.21.15.2:0/2203438939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f00841041a0 0x7f008419fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:50:58.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.758+0000 7f0082d9d700 1 --2- 172.21.15.2:0/2203438939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00841088f0 0x7f0084199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:50:58.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.759+0000 7f008259c700 1 --2- 172.21.15.2:0/2203438939 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00841037f0 0x7f008419f840 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f007400b580 tx=0x7f00740047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:58.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.759+0000 7f00737fe700 1 -- 172.21.15.2:0/2203438939 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0074027070 con 0x7f00841037f0 2024-05-14T16:50:58.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.759+0000 7f0088e2f700 1 -- 172.21.15.2:0/2203438939 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f008419a3f0 con 0x7f00841037f0 2024-05-14T16:50:58.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.759+0000 7f00737fe700 1 -- 172.21.15.2:0/2203438939 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0074004020 con 0x7f00841037f0 2024-05-14T16:50:58.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.759+0000 7f0088e2f700 1 -- 172.21.15.2:0/2203438939 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00841a6710 con 0x7f00841037f0 2024-05-14T16:50:58.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.759+0000 7f00737fe700 1 -- 172.21.15.2:0/2203438939 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0074022e30 con 0x7f00841037f0 2024-05-14T16:50:58.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.760+0000 7f0088e2f700 1 -- 172.21.15.2:0/2203438939 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f008406a230 con 0x7f00841037f0 2024-05-14T16:50:58.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.761+0000 7f00737fe700 1 -- 172.21.15.2:0/2203438939 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f0074022590 con 0x7f00841037f0 2024-05-14T16:50:58.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.762+0000 7f00737fe700 1 --2- 172.21.15.2:0/2203438939 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f006c0b2880 0x7f006c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:50:58.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.762+0000 7f0081d9b700 1 --2- 172.21.15.2:0/2203438939 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f006c0b2880 0x7f006c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:50:58.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.762+0000 7f00737fe700 1 -- 172.21.15.2:0/2203438939 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f00740e29a0 con 0x7f00841037f0 2024-05-14T16:50:58.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.762+0000 7f0081d9b700 1 --2- 172.21.15.2:0/2203438939 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f006c0b2880 0x7f006c0b4d40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0078005fd0 tx=0x7f0078005f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:50:58.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:58.767+0000 7f00737fe700 1 -- 172.21.15.2:0/2203438939 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0074013070 con 0x7f00841037f0 2024-05-14T16:50:59.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:59.028+0000 7f0088e2f700 1 -- 172.21.15.2:0/2203438939 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f008402e560 con 0x7f00841037f0 2024-05-14T16:50:59.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:59.029+0000 7f00737fe700 1 -- 172.21.15.2:0/2203438939 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4687 (secure 0 0 0) 0x7f00740aabf0 con 0x7f00841037f0 2024-05-14T16:50:59.028 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:50:59.029 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2275696923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2275696923},{"type":"v1","addr":"172.21.15.2:6835","nonce":2275696923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1880336731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1880336731},{"type":"v1","addr":"172.21.15.151:6833","nonce":1880336731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2415605548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2415605548},{"type":"v1","addr":"172.21.15.2:6837","nonce":2415605548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2635167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2635167},{"type":"v1","addr":"172.21.15.123:6833","nonce":2635167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1198670013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1198670013},{"type":"v1","addr":"172.21.15.123:6835","nonce":1198670013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-14T16:50:59.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:59.031+0000 7f0088e2f700 1 -- 172.21.15.2:0/2203438939 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f006c0b2880 msgr2=0x7f006c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:59.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:59.031+0000 7f0088e2f700 1 --2- 172.21.15.2:0/2203438939 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f006c0b2880 0x7f006c0b4d40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0078005fd0 tx=0x7f0078005f60 comp rx=0 tx=0).stop 2024-05-14T16:50:59.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:59.031+0000 7f0088e2f700 1 -- 172.21.15.2:0/2203438939 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00841037f0 msgr2=0x7f008419f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:50:59.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:59.031+0000 7f0088e2f700 1 --2- 172.21.15.2:0/2203438939 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00841037f0 0x7f008419f840 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f007400b580 tx=0x7f00740047b0 comp rx=0 tx=0).stop 2024-05-14T16:50:59.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:59.031+0000 7f0088e2f700 1 -- 172.21.15.2:0/2203438939 shutdown_connections 2024-05-14T16:50:59.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:59.031+0000 7f0088e2f700 1 --2- 172.21.15.2:0/2203438939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00841088f0 0x7f0084199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:59.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:59.031+0000 7f0088e2f700 1 --2- 172.21.15.2:0/2203438939 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f006c0b2880 0x7f006c0b4d40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:59.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:59.031+0000 7f0088e2f700 1 --2- 172.21.15.2:0/2203438939 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00841037f0 0x7f008419f840 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:59.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:59.031+0000 7f0088e2f700 1 --2- 172.21.15.2:0/2203438939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f00841041a0 0x7f008419fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:50:59.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:59.031+0000 7f0088e2f700 1 -- 172.21.15.2:0/2203438939 >> 172.21.15.2:0/2203438939 conn(0x7f008407a6a0 msgr2=0x7f0084101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:50:59.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:59.031+0000 7f0088e2f700 1 -- 172.21.15.2:0/2203438939 shutdown_connections 2024-05-14T16:50:59.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:50:59.031+0000 7f0088e2f700 1 -- 172.21.15.2:0/2203438939 wait complete. 2024-05-14T16:50:59.032 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 6 2024-05-14T16:50:59.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:59 smithi151 ceph-mon[36066]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:59.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:50:59 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/2203438939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:50:59.355 INFO:journalctl@ceph.mds.f.smithi151.stdout:May 14 16:50:59 smithi151 systemd[1]: Starting Ceph mds.f for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T16:50:59.409 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:59 smithi123 ceph-mon[35971]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:59.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:50:59 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/2203438939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:50:59.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:59 smithi002 ceph-mon[31538]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:50:59.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:50:59 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/2203438939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:50:59.705 INFO:journalctl@ceph.mds.f.smithi151.stdout:May 14 16:50:59 smithi151 podman[66637]: 2024-05-14 16:50:59.356883356 +0000 UTC m=+0.017868352 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T16:50:59.706 INFO:journalctl@ceph.mds.f.smithi151.stdout:May 14 16:50:59 smithi151 podman[66637]: 2024-05-14 16:50:59.47004824 +0000 UTC m=+0.131033233 container create 681def580df539294c1d860b9b42eb84399a1fbcfe94914aed50b3b52a6b1b75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-f, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T16:50:59.706 INFO:journalctl@ceph.mds.f.smithi151.stdout:May 14 16:50:59 smithi151 podman[66637]: 2024-05-14 16:50:59.70682228 +0000 UTC m=+0.367807273 container init 681def580df539294c1d860b9b42eb84399a1fbcfe94914aed50b3b52a6b1b75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-f, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, ceph=True) 2024-05-14T16:51:00.104 INFO:journalctl@ceph.mds.f.smithi151.stdout:May 14 16:50:59 smithi151 podman[66637]: 2024-05-14 16:50:59.709496817 +0000 UTC m=+0.370481815 container start 681def580df539294c1d860b9b42eb84399a1fbcfe94914aed50b3b52a6b1b75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-14T16:51:00.104 INFO:journalctl@ceph.mds.f.smithi151.stdout:May 14 16:50:59 smithi151 ceph-mds[66656]: -- 172.21.15.151:0/1100707894 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564b85463a40 con 0x564b85436800 2024-05-14T16:51:00.104 INFO:journalctl@ceph.mds.f.smithi151.stdout:May 14 16:50:59 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-f[66652]: starting mds.f at 2024-05-14T16:51:00.104 INFO:journalctl@ceph.mds.f.smithi151.stdout:May 14 16:50:59 smithi151 bash[66637]: 681def580df539294c1d860b9b42eb84399a1fbcfe94914aed50b3b52a6b1b75 2024-05-14T16:51:00.105 INFO:journalctl@ceph.mds.f.smithi151.stdout:May 14 16:50:59 smithi151 systemd[1]: Started Ceph mds.f for ed40717a-1210-11ef-bc98-c7b262605968. 2024-05-14T16:51:01.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:00 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:01.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:00 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:01.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:00 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:01.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:00 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:51:01.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:00 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:51:01.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:00 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:51:01.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:00 smithi151 ceph-mon[36066]: Deploying daemon mds.c on smithi151 2024-05-14T16:51:01.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:00 smithi151 ceph-mon[36066]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:01.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:00 smithi151 ceph-mon[36066]: mds.? [v2:172.21.15.151:6834/762044339,v1:172.21.15.151:6835/762044339] up:boot 2024-05-14T16:51:01.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:00 smithi151 ceph-mon[36066]: fsmap 6 up:standby 2024-05-14T16:51:01.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:00 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T16:51:01.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:00 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:01.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:00 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:01.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:00 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:01.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:00 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:51:01.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:00 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:51:01.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:00 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:51:01.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:00 smithi123 ceph-mon[35971]: Deploying daemon mds.c on smithi151 2024-05-14T16:51:01.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:00 smithi123 ceph-mon[35971]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:01.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:00 smithi123 ceph-mon[35971]: mds.? [v2:172.21.15.151:6834/762044339,v1:172.21.15.151:6835/762044339] up:boot 2024-05-14T16:51:01.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:00 smithi123 ceph-mon[35971]: fsmap 6 up:standby 2024-05-14T16:51:01.161 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:00 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T16:51:01.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:00 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:01.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:00 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:01.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:00 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:01.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:00 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:51:01.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:00 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:51:01.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:00 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:51:01.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:00 smithi002 ceph-mon[31538]: Deploying daemon mds.c on smithi151 2024-05-14T16:51:01.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:00 smithi002 ceph-mon[31538]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:01.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:00 smithi002 ceph-mon[31538]: mds.? [v2:172.21.15.151:6834/762044339,v1:172.21.15.151:6835/762044339] up:boot 2024-05-14T16:51:01.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:00 smithi002 ceph-mon[31538]: fsmap 6 up:standby 2024-05-14T16:51:01.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:00 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T16:51:01.867 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:51:02.104 INFO:journalctl@ceph.mds.c.smithi151.stdout:May 14 16:51:02 smithi151 systemd[1]: Starting Ceph mds.c for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T16:51:02.130 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:51:02.497 INFO:journalctl@ceph.mds.c.smithi151.stdout:May 14 16:51:02 smithi151 podman[66940]: 2024-05-14 16:51:02.120679386 +0000 UTC m=+0.017808571 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T16:51:02.497 INFO:journalctl@ceph.mds.c.smithi151.stdout:May 14 16:51:02 smithi151 podman[66940]: 2024-05-14 16:51:02.248825015 +0000 UTC m=+0.145954188 container create 0371af3365c24591bbf7af36555b4b9838b3d0aa9e13dcda1794adb044803d67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-14T16:51:02.497 INFO:journalctl@ceph.mds.c.smithi151.stdout:May 14 16:51:02 smithi151 podman[66940]: 2024-05-14 16:51:02.498215822 +0000 UTC m=+0.395345006 container init 0371af3365c24591bbf7af36555b4b9838b3d0aa9e13dcda1794adb044803d67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T16:51:02.854 INFO:journalctl@ceph.mds.c.smithi151.stdout:May 14 16:51:02 smithi151 podman[66940]: 2024-05-14 16:51:02.50085208 +0000 UTC m=+0.397981266 container start 0371af3365c24591bbf7af36555b4b9838b3d0aa9e13dcda1794adb044803d67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T16:51:02.854 INFO:journalctl@ceph.mds.c.smithi151.stdout:May 14 16:51:02 smithi151 ceph-mds[66959]: -- 172.21.15.151:0/2577070743 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5646876c1a40 con 0x564687694c00 2024-05-14T16:51:02.854 INFO:journalctl@ceph.mds.c.smithi151.stdout:May 14 16:51:02 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-c[66955]: starting mds.c at 2024-05-14T16:51:02.855 INFO:journalctl@ceph.mds.c.smithi151.stdout:May 14 16:51:02 smithi151 bash[66940]: 0371af3365c24591bbf7af36555b4b9838b3d0aa9e13dcda1794adb044803d67 2024-05-14T16:51:02.855 INFO:journalctl@ceph.mds.c.smithi151.stdout:May 14 16:51:02 smithi151 systemd[1]: Started Ceph mds.c for ed40717a-1210-11ef-bc98-c7b262605968. 2024-05-14T16:51:03.409 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:03 smithi123 ceph-mon[35971]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:03.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:03 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:03.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:03 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:03.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:03 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:03.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:03 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:51:03.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:03 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:51:03.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:03 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:51:03.447 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:03 smithi002 ceph-mon[31538]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:03.447 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:03 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:03.447 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:03 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:03.447 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:03 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:03.447 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:03 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:51:03.447 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:03 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:51:03.447 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:03 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:51:03.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:03 smithi151 ceph-mon[36066]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:03.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:03 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:03.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:03 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:03.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:03 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:03.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:03 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:51:03.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:03 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:51:03.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:03 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:51:03.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:03.786+0000 7fdc3894e700 1 -- 172.21.15.2:0/4199431746 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc341088f0 msgr2=0x7fdc34110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:03.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:03.786+0000 7fdc3894e700 1 --2- 172.21.15.2:0/4199431746 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc341088f0 0x7fdc34110470 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c00b0d0 tx=0x7fdc2c00b3e0 comp rx=0 tx=0).stop 2024-05-14T16:51:03.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:03.786+0000 7fdc3894e700 1 -- 172.21.15.2:0/4199431746 shutdown_connections 2024-05-14T16:51:03.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:03.786+0000 7fdc3894e700 1 --2- 172.21.15.2:0/4199431746 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc341088f0 0x7fdc34110470 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:03.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:03.786+0000 7fdc3894e700 1 --2- 172.21.15.2:0/4199431746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc341037f0 0x7fdc34103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:03.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:03.786+0000 7fdc3894e700 1 --2- 172.21.15.2:0/4199431746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc341041a0 0x7fdc341081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:03.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:03.786+0000 7fdc3894e700 1 -- 172.21.15.2:0/4199431746 >> 172.21.15.2:0/4199431746 conn(0x7fdc3407a6a0 msgr2=0x7fdc3407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:03.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:03.787+0000 7fdc3894e700 1 -- 172.21.15.2:0/4199431746 shutdown_connections 2024-05-14T16:51:03.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:03.787+0000 7fdc3894e700 1 -- 172.21.15.2:0/4199431746 wait complete. 2024-05-14T16:51:03.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:03.787+0000 7fdc3894e700 1 Processor -- start 2024-05-14T16:51:03.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:03.787+0000 7fdc3894e700 1 -- start start 2024-05-14T16:51:03.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:03.787+0000 7fdc3894e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc341037f0 0x7fdc3419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:03.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:03.787+0000 7fdc3894e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc341041a0 0x7fdc3419fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:03.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:03.788+0000 7fdc3894e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc341088f0 0x7fdc341998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:03.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:03.788+0000 7fdc3894e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fdc34113710 con 0x7fdc341041a0 2024-05-14T16:51:03.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:03.788+0000 7fdc3894e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdc34113590 con 0x7fdc341037f0 2024-05-14T16:51:03.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:03.788+0000 7fdc3894e700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdc34113890 con 0x7fdc341088f0 2024-05-14T16:51:03.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:03.788+0000 7fdc31d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc341041a0 0x7fdc3419fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:03.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:03.788+0000 7fdc31d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc341041a0 0x7fdc3419fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52064/0 (socket says 172.21.15.2:52064) 2024-05-14T16:51:03.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:03.788+0000 7fdc31d9b700 1 -- 172.21.15.2:0/441447420 learned_addr learned my addr 172.21.15.2:0/441447420 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:51:03.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:03.788+0000 7fdc3259c700 1 --2- 172.21.15.2:0/441447420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc341037f0 0x7fdc3419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:03.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:03.788+0000 7fdc31d9b700 1 -- 172.21.15.2:0/441447420 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc341088f0 msgr2=0x7fdc341998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:03.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:03.788+0000 7fdc32d9d700 1 --2- 172.21.15.2:0/441447420 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc341088f0 0x7fdc341998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:03.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:03.788+0000 7fdc31d9b700 1 --2- 172.21.15.2:0/441447420 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc341088f0 0x7fdc341998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:03.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:03.788+0000 7fdc31d9b700 1 -- 172.21.15.2:0/441447420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc341037f0 msgr2=0x7fdc3419f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:03.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:03.788+0000 7fdc31d9b700 1 --2- 172.21.15.2:0/441447420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc341037f0 0x7fdc3419f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:03.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:03.788+0000 7fdc31d9b700 1 -- 172.21.15.2:0/441447420 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc3419a050 con 0x7fdc341041a0 2024-05-14T16:51:03.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:03.788+0000 7fdc31d9b700 1 --2- 172.21.15.2:0/441447420 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc341041a0 0x7fdc3419fc70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdc2800c9b0 tx=0x7fdc2800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:03.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:03.788+0000 7fdc237fe700 1 -- 172.21.15.2:0/441447420 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc28007b00 con 0x7fdc341041a0 2024-05-14T16:51:03.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:03.788+0000 7fdc237fe700 1 -- 172.21.15.2:0/441447420 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc28007ca0 con 0x7fdc341041a0 2024-05-14T16:51:03.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:03.788+0000 7fdc3894e700 1 -- 172.21.15.2:0/441447420 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc3419a340 con 0x7fdc341041a0 2024-05-14T16:51:03.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:03.788+0000 7fdc237fe700 1 -- 172.21.15.2:0/441447420 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc28018660 con 0x7fdc341041a0 2024-05-14T16:51:03.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:03.788+0000 7fdc3894e700 1 -- 172.21.15.2:0/441447420 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc341a6600 con 0x7fdc341041a0 2024-05-14T16:51:03.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:03.789+0000 7fdc3894e700 1 -- 172.21.15.2:0/441447420 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc3406a230 con 0x7fdc341041a0 2024-05-14T16:51:03.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:03.791+0000 7fdc237fe700 1 -- 172.21.15.2:0/441447420 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fdc2800f450 con 0x7fdc341041a0 2024-05-14T16:51:03.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:03.792+0000 7fdc237fe700 1 --2- 172.21.15.2:0/441447420 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fdc1c0b2880 0x7fdc1c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:03.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:03.792+0000 7fdc3259c700 1 --2- 172.21.15.2:0/441447420 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fdc1c0b2880 0x7fdc1c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:03.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:03.792+0000 7fdc237fe700 1 -- 172.21.15.2:0/441447420 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fdc280e1d80 con 0x7fdc341041a0 2024-05-14T16:51:03.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:03.793+0000 7fdc3259c700 1 --2- 172.21.15.2:0/441447420 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fdc1c0b2880 0x7fdc1c0b4d40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdc3419aa30 tx=0x7fdc240073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:03.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:03.797+0000 7fdc237fe700 1 -- 172.21.15.2:0/441447420 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdc28014030 con 0x7fdc341041a0 2024-05-14T16:51:04.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:04.059+0000 7fdc3894e700 1 -- 172.21.15.2:0/441447420 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc3402e560 con 0x7fdc341041a0 2024-05-14T16:51:04.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:04.060+0000 7fdc237fe700 1 -- 172.21.15.2:0/441447420 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6337 (secure 0 0 0) 0x7fdc280a9ea0 con 0x7fdc341041a0 2024-05-14T16:51:04.060 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:51:04.060 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2275696923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2275696923},{"type":"v1","addr":"172.21.15.2:6835","nonce":2275696923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1880336731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1880336731},{"type":"v1","addr":"172.21.15.151:6833","nonce":1880336731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2415605548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2415605548},{"type":"v1","addr":"172.21.15.2:6837","nonce":2415605548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2635167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2635167},{"type":"v1","addr":"172.21.15.123:6833","nonce":2635167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/762044339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":762044339},{"type":"v1","addr":"172.21.15.151:6835","nonce":762044339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/132043151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":132043151},{"type":"v1","addr":"172.21.15.151:6837","nonce":132043151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1198670013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1198670013},{"type":"v1","addr":"172.21.15.123:6835","nonce":1198670013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-14T16:51:04.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:04.062+0000 7fdc3894e700 1 -- 172.21.15.2:0/441447420 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fdc1c0b2880 msgr2=0x7fdc1c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:04.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:04.062+0000 7fdc3894e700 1 --2- 172.21.15.2:0/441447420 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fdc1c0b2880 0x7fdc1c0b4d40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdc3419aa30 tx=0x7fdc240073d0 comp rx=0 tx=0).stop 2024-05-14T16:51:04.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:04.062+0000 7fdc3894e700 1 -- 172.21.15.2:0/441447420 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc341041a0 msgr2=0x7fdc3419fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:04.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:04.062+0000 7fdc3894e700 1 --2- 172.21.15.2:0/441447420 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc341041a0 0x7fdc3419fc70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdc2800c9b0 tx=0x7fdc2800ccc0 comp rx=0 tx=0).stop 2024-05-14T16:51:04.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:04.063+0000 7fdc3894e700 1 -- 172.21.15.2:0/441447420 shutdown_connections 2024-05-14T16:51:04.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:04.063+0000 7fdc3894e700 1 --2- 172.21.15.2:0/441447420 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc341088f0 0x7fdc341998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:04.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:04.063+0000 7fdc3894e700 1 --2- 172.21.15.2:0/441447420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc341037f0 0x7fdc3419f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:04.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:04.063+0000 7fdc3894e700 1 --2- 172.21.15.2:0/441447420 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fdc1c0b2880 0x7fdc1c0b4d40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:04.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:04.063+0000 7fdc3894e700 1 --2- 172.21.15.2:0/441447420 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc341041a0 0x7fdc3419fc70 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:04.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:04.063+0000 7fdc3894e700 1 -- 172.21.15.2:0/441447420 >> 172.21.15.2:0/441447420 conn(0x7fdc3407a6a0 msgr2=0x7fdc341016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:04.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:04.063+0000 7fdc3894e700 1 -- 172.21.15.2:0/441447420 shutdown_connections 2024-05-14T16:51:04.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:04.063+0000 7fdc3894e700 1 -- 172.21.15.2:0/441447420 wait complete. 2024-05-14T16:51:04.064 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 8 2024-05-14T16:51:04.327 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:04 smithi002 ceph-mon[31538]: Deploying daemon mds.h on smithi123 2024-05-14T16:51:04.327 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:04 smithi002 ceph-mon[31538]: mds.? [v2:172.21.15.151:6836/132043151,v1:172.21.15.151:6837/132043151] up:boot 2024-05-14T16:51:04.327 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:04 smithi002 ceph-mon[31538]: fsmap 7 up:standby 2024-05-14T16:51:04.328 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:04 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T16:51:04.328 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:04 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/441447420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:51:04.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:04 smithi123 ceph-mon[35971]: Deploying daemon mds.h on smithi123 2024-05-14T16:51:04.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:04 smithi123 ceph-mon[35971]: mds.? [v2:172.21.15.151:6836/132043151,v1:172.21.15.151:6837/132043151] up:boot 2024-05-14T16:51:04.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:04 smithi123 ceph-mon[35971]: fsmap 7 up:standby 2024-05-14T16:51:04.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:04 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T16:51:04.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:04 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/441447420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:51:04.433 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:04 smithi151 ceph-mon[36066]: Deploying daemon mds.h on smithi123 2024-05-14T16:51:04.433 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:04 smithi151 ceph-mon[36066]: mds.? [v2:172.21.15.151:6836/132043151,v1:172.21.15.151:6837/132043151] up:boot 2024-05-14T16:51:04.433 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:04 smithi151 ceph-mon[36066]: fsmap 7 up:standby 2024-05-14T16:51:04.433 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:04 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T16:51:04.433 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:04 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/441447420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:51:05.141 INFO:journalctl@ceph.mds.h.smithi123.stdout:May 14 16:51:04 smithi123 systemd[1]: Starting Ceph mds.h for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T16:51:05.409 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:05 smithi123 ceph-mon[35971]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:05.410 INFO:journalctl@ceph.mds.h.smithi123.stdout:May 14 16:51:05 smithi123 podman[65982]: 2024-05-14 16:51:05.059985609 +0000 UTC m=+0.017344897 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T16:51:05.410 INFO:journalctl@ceph.mds.h.smithi123.stdout:May 14 16:51:05 smithi123 podman[65982]: 2024-05-14 16:51:05.161580928 +0000 UTC m=+0.118940214 container create 14bb6f7ef1ce96ef4da657c680e34c1bdad3760480c5e5b4b9f0d8fc6cac1d44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-h, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T16:51:05.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:05 smithi002 ceph-mon[31538]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:05.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:05 smithi151 ceph-mon[36066]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:05.910 INFO:journalctl@ceph.mds.h.smithi123.stdout:May 14 16:51:05 smithi123 podman[65982]: 2024-05-14 16:51:05.43098439 +0000 UTC m=+0.388343668 container init 14bb6f7ef1ce96ef4da657c680e34c1bdad3760480c5e5b4b9f0d8fc6cac1d44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-h, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T16:51:05.910 INFO:journalctl@ceph.mds.h.smithi123.stdout:May 14 16:51:05 smithi123 podman[65982]: 2024-05-14 16:51:05.434222559 +0000 UTC m=+0.391581846 container start 14bb6f7ef1ce96ef4da657c680e34c1bdad3760480c5e5b4b9f0d8fc6cac1d44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-h, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T16:51:05.910 INFO:journalctl@ceph.mds.h.smithi123.stdout:May 14 16:51:05 smithi123 ceph-mds[66001]: -- 172.21.15.123:0/1563466365 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5599c4e01a40 con 0x5599c4dd4c00 2024-05-14T16:51:05.910 INFO:journalctl@ceph.mds.h.smithi123.stdout:May 14 16:51:05 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-h[65997]: starting mds.h at 2024-05-14T16:51:05.910 INFO:journalctl@ceph.mds.h.smithi123.stdout:May 14 16:51:05 smithi123 bash[65982]: 14bb6f7ef1ce96ef4da657c680e34c1bdad3760480c5e5b4b9f0d8fc6cac1d44 2024-05-14T16:51:05.910 INFO:journalctl@ceph.mds.h.smithi123.stdout:May 14 16:51:05 smithi123 systemd[1]: Started Ceph mds.h for ed40717a-1210-11ef-bc98-c7b262605968. 2024-05-14T16:51:06.639 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:51:06.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:06 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:06.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:06 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:06.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:06 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:06.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:06 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:51:06.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:06 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:51:06.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:06 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:51:06.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:06 smithi151 ceph-mon[36066]: Deploying daemon mds.i on smithi151 2024-05-14T16:51:06.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:06 smithi151 ceph-mon[36066]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:06.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:06 smithi151 ceph-mon[36066]: mds.? [v2:172.21.15.123:6836/333426645,v1:172.21.15.123:6837/333426645] up:boot 2024-05-14T16:51:06.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:06 smithi151 ceph-mon[36066]: fsmap 8 up:standby 2024-05-14T16:51:06.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:06 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T16:51:06.906 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:51:06.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:06.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:06.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:06.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:51:06.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:51:06.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:51:06.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:06 smithi123 ceph-mon[35971]: Deploying daemon mds.i on smithi151 2024-05-14T16:51:06.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:06 smithi123 ceph-mon[35971]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:06.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:06 smithi123 ceph-mon[35971]: mds.? [v2:172.21.15.123:6836/333426645,v1:172.21.15.123:6837/333426645] up:boot 2024-05-14T16:51:06.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:06 smithi123 ceph-mon[35971]: fsmap 8 up:standby 2024-05-14T16:51:06.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:06 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T16:51:06.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:06 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:06.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:06 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:06.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:06 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:06.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:06 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:51:06.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:06 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:51:06.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:06 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:51:06.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:06 smithi002 ceph-mon[31538]: Deploying daemon mds.i on smithi151 2024-05-14T16:51:06.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:06 smithi002 ceph-mon[31538]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:06.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:06 smithi002 ceph-mon[31538]: mds.? [v2:172.21.15.123:6836/333426645,v1:172.21.15.123:6837/333426645] up:boot 2024-05-14T16:51:06.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:06 smithi002 ceph-mon[31538]: fsmap 8 up:standby 2024-05-14T16:51:06.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:06 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T16:51:07.842 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 16:51:07 smithi151 systemd[1]: Starting Ceph mds.i for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T16:51:08.104 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 16:51:07 smithi151 podman[67241]: 2024-05-14 16:51:07.745570529 +0000 UTC m=+0.017812378 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T16:51:08.105 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 16:51:07 smithi151 podman[67241]: 2024-05-14 16:51:07.851148019 +0000 UTC m=+0.123389877 container create d2272915ee3cd763c5b443d25b6efff74fe08e614e01b71fbc434cf1bfd0ab95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T16:51:08.105 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 16:51:08 smithi151 podman[67241]: 2024-05-14 16:51:08.079738418 +0000 UTC m=+0.351980254 container init d2272915ee3cd763c5b443d25b6efff74fe08e614e01b71fbc434cf1bfd0ab95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-14T16:51:08.105 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 16:51:08 smithi151 podman[67241]: 2024-05-14 16:51:08.08238767 +0000 UTC m=+0.354629515 container start d2272915ee3cd763c5b443d25b6efff74fe08e614e01b71fbc434cf1bfd0ab95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-14T16:51:08.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.572+0000 7f7c9bcc8700 1 -- 172.21.15.2:0/2642507160 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c94102a80 msgr2=0x7f7c94102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:08.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.572+0000 7f7c9bcc8700 1 --2- 172.21.15.2:0/2642507160 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c94102a80 0x7f7c94102e60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7c900099c0 tx=0x7f7c90009cd0 comp rx=0 tx=0).stop 2024-05-14T16:51:08.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.572+0000 7f7c9bcc8700 1 -- 172.21.15.2:0/2642507160 shutdown_connections 2024-05-14T16:51:08.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.572+0000 7f7c9bcc8700 1 --2- 172.21.15.2:0/2642507160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c941077d0 0x7f7c9410f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:08.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.572+0000 7f7c9bcc8700 1 --2- 172.21.15.2:0/2642507160 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c94102a80 0x7f7c94102e60 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:08.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.572+0000 7f7c9bcc8700 1 --2- 172.21.15.2:0/2642507160 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c94103430 0x7f7c941070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:08.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.572+0000 7f7c9bcc8700 1 -- 172.21.15.2:0/2642507160 >> 172.21.15.2:0/2642507160 conn(0x7f7c940793f0 msgr2=0x7f7c94079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:08.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.572+0000 7f7c9bcc8700 1 -- 172.21.15.2:0/2642507160 shutdown_connections 2024-05-14T16:51:08.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.572+0000 7f7c9bcc8700 1 -- 172.21.15.2:0/2642507160 wait complete. 2024-05-14T16:51:08.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.573+0000 7f7c9bcc8700 1 Processor -- start 2024-05-14T16:51:08.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.573+0000 7f7c9bcc8700 1 -- start start 2024-05-14T16:51:08.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.573+0000 7f7c9bcc8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c94102a80 0x7f7c94104140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:08.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.573+0000 7f7c9bcc8700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c94103430 0x7f7c94104680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:08.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.573+0000 7f7c9bcc8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c941077d0 0x7f7c94104bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:08.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.574+0000 7f7c9bcc8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7c941126b0 con 0x7f7c941077d0 2024-05-14T16:51:08.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.574+0000 7f7c9bcc8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7c94112530 con 0x7f7c94102a80 2024-05-14T16:51:08.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.574+0000 7f7c9bcc8700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7c94112830 con 0x7f7c94103430 2024-05-14T16:51:08.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.574+0000 7f7c9a265700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c941077d0 0x7f7c94104bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:08.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.574+0000 7f7c9a265700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c941077d0 0x7f7c94104bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52094/0 (socket says 172.21.15.2:52094) 2024-05-14T16:51:08.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.574+0000 7f7c9a265700 1 -- 172.21.15.2:0/1648550730 learned_addr learned my addr 172.21.15.2:0/1648550730 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:51:08.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.574+0000 7f7c99263700 1 --2- 172.21.15.2:0/1648550730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c94103430 0x7f7c94104680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:08.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.574+0000 7f7c99a64700 1 --2- 172.21.15.2:0/1648550730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c94102a80 0x7f7c94104140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:08.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.574+0000 7f7c9a265700 1 -- 172.21.15.2:0/1648550730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c94103430 msgr2=0x7f7c94104680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:08.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.574+0000 7f7c9a265700 1 --2- 172.21.15.2:0/1648550730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c94103430 0x7f7c94104680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:08.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.574+0000 7f7c9a265700 1 -- 172.21.15.2:0/1648550730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c94102a80 msgr2=0x7f7c94104140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:08.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.574+0000 7f7c9a265700 1 --2- 172.21.15.2:0/1648550730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c94102a80 0x7f7c94104140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:08.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.574+0000 7f7c9a265700 1 -- 172.21.15.2:0/1648550730 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c941ac660 con 0x7f7c941077d0 2024-05-14T16:51:08.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.574+0000 7f7c9a265700 1 --2- 172.21.15.2:0/1648550730 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c941077d0 0x7f7c94104bc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c007ee0 tx=0x7f7c8c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:08.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.574+0000 7f7c86ffd700 1 -- 172.21.15.2:0/1648550730 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c004bb0 con 0x7f7c941077d0 2024-05-14T16:51:08.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.574+0000 7f7c86ffd700 1 -- 172.21.15.2:0/1648550730 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c8c004d50 con 0x7f7c941077d0 2024-05-14T16:51:08.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.574+0000 7f7c9bcc8700 1 -- 172.21.15.2:0/1648550730 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c941ac8f0 con 0x7f7c941077d0 2024-05-14T16:51:08.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.574+0000 7f7c86ffd700 1 -- 172.21.15.2:0/1648550730 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c005920 con 0x7f7c941077d0 2024-05-14T16:51:08.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.575+0000 7f7c9bcc8700 1 -- 172.21.15.2:0/1648550730 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c941ace80 con 0x7f7c941077d0 2024-05-14T16:51:08.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.576+0000 7f7c9bcc8700 1 -- 172.21.15.2:0/1648550730 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c9406b500 con 0x7f7c941077d0 2024-05-14T16:51:08.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.578+0000 7f7c86ffd700 1 -- 172.21.15.2:0/1648550730 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f7c8c01a020 con 0x7f7c941077d0 2024-05-14T16:51:08.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.579+0000 7f7c86ffd700 1 --2- 172.21.15.2:0/1648550730 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f7c800b2900 0x7f7c800b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:08.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.579+0000 7f7c99a64700 1 --2- 172.21.15.2:0/1648550730 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f7c800b2900 0x7f7c800b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:08.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.579+0000 7f7c86ffd700 1 -- 172.21.15.2:0/1648550730 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f7c8c0e19d0 con 0x7f7c941077d0 2024-05-14T16:51:08.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.579+0000 7f7c99a64700 1 --2- 172.21.15.2:0/1648550730 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f7c800b2900 0x7f7c800b4dc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7c90009990 tx=0x7f7c90005a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:08.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.582+0000 7f7c86ffd700 1 -- 172.21.15.2:0/1648550730 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7c8c010070 con 0x7f7c941077d0 2024-05-14T16:51:08.604 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 16:51:08 smithi151 ceph-mds[67260]: -- 172.21.15.151:0/453057220 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559a6fe49a40 con 0x559a6fe1cc00 2024-05-14T16:51:08.604 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 16:51:08 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: starting mds.i at 2024-05-14T16:51:08.604 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 16:51:08 smithi151 bash[67241]: d2272915ee3cd763c5b443d25b6efff74fe08e614e01b71fbc434cf1bfd0ab95 2024-05-14T16:51:08.604 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 16:51:08 smithi151 systemd[1]: Started Ceph mds.i for ed40717a-1210-11ef-bc98-c7b262605968. 2024-05-14T16:51:08.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.859+0000 7f7c9bcc8700 1 -- 172.21.15.2:0/1648550730 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c9410b420 con 0x7f7c941077d0 2024-05-14T16:51:08.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.859+0000 7f7c86ffd700 1 -- 172.21.15.2:0/1648550730 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+7992 (secure 0 0 0) 0x7f7c8c0a9b90 con 0x7f7c941077d0 2024-05-14T16:51:08.860 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:51:08.860 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2275696923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2275696923},{"type":"v1","addr":"172.21.15.2:6835","nonce":2275696923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1880336731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1880336731},{"type":"v1","addr":"172.21.15.151:6833","nonce":1880336731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2415605548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2415605548},{"type":"v1","addr":"172.21.15.2:6837","nonce":2415605548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2635167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2635167},{"type":"v1","addr":"172.21.15.123:6833","nonce":2635167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/762044339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":762044339},{"type":"v1","addr":"172.21.15.151:6835","nonce":762044339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/132043151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":132043151},{"type":"v1","addr":"172.21.15.151:6837","nonce":132043151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24427,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/4107834375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":4107834375},{"type":"v1","addr":"172.21.15.151:6839","nonce":4107834375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1198670013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1198670013},{"type":"v1","addr":"172.21.15.123:6835","nonce":1198670013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/333426645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":333426645},{"type":"v1","addr":"172.21.15.123:6837","nonce":333426645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-14T16:51:08.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.862+0000 7f7c9bcc8700 1 -- 172.21.15.2:0/1648550730 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f7c800b2900 msgr2=0x7f7c800b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:08.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.862+0000 7f7c9bcc8700 1 --2- 172.21.15.2:0/1648550730 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f7c800b2900 0x7f7c800b4dc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7c90009990 tx=0x7f7c90005a90 comp rx=0 tx=0).stop 2024-05-14T16:51:08.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.862+0000 7f7c9bcc8700 1 -- 172.21.15.2:0/1648550730 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c941077d0 msgr2=0x7f7c94104bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:08.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.862+0000 7f7c9bcc8700 1 --2- 172.21.15.2:0/1648550730 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c941077d0 0x7f7c94104bc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c007ee0 tx=0x7f7c8c00d3b0 comp rx=0 tx=0).stop 2024-05-14T16:51:08.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.862+0000 7f7c9bcc8700 1 -- 172.21.15.2:0/1648550730 shutdown_connections 2024-05-14T16:51:08.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.862+0000 7f7c9bcc8700 1 --2- 172.21.15.2:0/1648550730 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f7c800b2900 0x7f7c800b4dc0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:08.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.862+0000 7f7c9bcc8700 1 --2- 172.21.15.2:0/1648550730 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c941077d0 0x7f7c94104bc0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:08.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.862+0000 7f7c9bcc8700 1 --2- 172.21.15.2:0/1648550730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c94102a80 0x7f7c94104140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:08.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.862+0000 7f7c9bcc8700 1 --2- 172.21.15.2:0/1648550730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c94103430 0x7f7c94104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:08.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.862+0000 7f7c9bcc8700 1 -- 172.21.15.2:0/1648550730 >> 172.21.15.2:0/1648550730 conn(0x7f7c940793f0 msgr2=0x7f7c9406bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:08.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.862+0000 7f7c9bcc8700 1 -- 172.21.15.2:0/1648550730 shutdown_connections 2024-05-14T16:51:08.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:08.862+0000 7f7c9bcc8700 1 -- 172.21.15.2:0/1648550730 wait complete. 2024-05-14T16:51:08.864 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 10 2024-05-14T16:51:09.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:09 smithi002 ceph-mon[31538]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:09.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:09 smithi002 ceph-mon[31538]: mds.? [v2:172.21.15.151:6838/4107834375,v1:172.21.15.151:6839/4107834375] up:boot 2024-05-14T16:51:09.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:09 smithi002 ceph-mon[31538]: fsmap 9 up:standby 2024-05-14T16:51:09.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:09 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T16:51:09.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:09 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:09.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:09 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:09.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:09 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:09.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:09 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:51:09.495 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:09 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:51:09.495 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:09 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:51:09.495 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:09 smithi002 ceph-mon[31538]: Deploying daemon mds.g on smithi002 2024-05-14T16:51:09.495 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:09 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/1648550730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:51:09.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:09 smithi151 ceph-mon[36066]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:09.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:09 smithi151 ceph-mon[36066]: mds.? [v2:172.21.15.151:6838/4107834375,v1:172.21.15.151:6839/4107834375] up:boot 2024-05-14T16:51:09.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:09 smithi151 ceph-mon[36066]: fsmap 9 up:standby 2024-05-14T16:51:09.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:09 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T16:51:09.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:09 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:09.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:09 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:09.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:09 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:09.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:09 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:51:09.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:09 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:51:09.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:09 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:51:09.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:09 smithi151 ceph-mon[36066]: Deploying daemon mds.g on smithi002 2024-05-14T16:51:09.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:09 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/1648550730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:51:09.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:09 smithi123 ceph-mon[35971]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:09.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:09 smithi123 ceph-mon[35971]: mds.? [v2:172.21.15.151:6838/4107834375,v1:172.21.15.151:6839/4107834375] up:boot 2024-05-14T16:51:09.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:09 smithi123 ceph-mon[35971]: fsmap 9 up:standby 2024-05-14T16:51:09.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:09 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T16:51:09.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:09 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:09.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:09 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:09.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:09 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:09.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:09 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:51:09.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:09 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:51:09.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:09 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:51:09.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:09 smithi123 ceph-mon[35971]: Deploying daemon mds.g on smithi002 2024-05-14T16:51:09.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:09 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/1648550730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:51:11.165 INFO:journalctl@ceph.mds.g.smithi002.stdout:May 14 16:51:10 smithi002 systemd[1]: Starting Ceph mds.g for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T16:51:11.472 INFO:journalctl@ceph.mds.g.smithi002.stdout:May 14 16:51:11 smithi002 podman[72861]: 2024-05-14 16:51:11.115837601 +0000 UTC m=+0.017816327 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T16:51:11.472 INFO:journalctl@ceph.mds.g.smithi002.stdout:May 14 16:51:11 smithi002 podman[72861]: 2024-05-14 16:51:11.227961256 +0000 UTC m=+0.129939970 container create 441f07e027cc8a489e6f60d484e12e83b79d855bf7bbadb162a82a1b737c6f7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-g, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514) 2024-05-14T16:51:11.472 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:11 smithi002 ceph-mon[31538]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:11.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:11 smithi151 ceph-mon[36066]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:11.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:11 smithi123 ceph-mon[35971]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:11.744 INFO:journalctl@ceph.mds.g.smithi002.stdout:May 14 16:51:11 smithi002 podman[72861]: 2024-05-14 16:51:11.472663921 +0000 UTC m=+0.374642643 container init 441f07e027cc8a489e6f60d484e12e83b79d855bf7bbadb162a82a1b737c6f7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-g, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-14T16:51:11.744 INFO:journalctl@ceph.mds.g.smithi002.stdout:May 14 16:51:11 smithi002 podman[72861]: 2024-05-14 16:51:11.476098227 +0000 UTC m=+0.378076942 container start 441f07e027cc8a489e6f60d484e12e83b79d855bf7bbadb162a82a1b737c6f7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-g, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-14T16:51:11.744 INFO:journalctl@ceph.mds.g.smithi002.stdout:May 14 16:51:11 smithi002 ceph-mds[72880]: -- 172.21.15.2:0/511544843 <== mon.0 v2:172.21.15.2:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558faffdfa40 con 0x558faffb2c00 2024-05-14T16:51:11.744 INFO:journalctl@ceph.mds.g.smithi002.stdout:May 14 16:51:11 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-g[72876]: starting mds.g at 2024-05-14T16:51:11.744 INFO:journalctl@ceph.mds.g.smithi002.stdout:May 14 16:51:11 smithi002 bash[72861]: 441f07e027cc8a489e6f60d484e12e83b79d855bf7bbadb162a82a1b737c6f7b 2024-05-14T16:51:11.744 INFO:journalctl@ceph.mds.g.smithi002.stdout:May 14 16:51:11 smithi002 systemd[1]: Started Ceph mds.g for ed40717a-1210-11ef-bc98-c7b262605968. 2024-05-14T16:51:12.098 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:51:12.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:12 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:12.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:12 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:12.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:12 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:12.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:12 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:51:12.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:12 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:51:12.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:12 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:51:12.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:12 smithi002 ceph-mon[31538]: Deploying daemon mds.a on smithi002 2024-05-14T16:51:12.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:12 smithi002 ceph-mon[31538]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:12.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:12 smithi002 ceph-mon[31538]: mds.? [v2:172.21.15.2:6838/716718761,v1:172.21.15.2:6839/716718761] up:boot 2024-05-14T16:51:12.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:12 smithi002 ceph-mon[31538]: fsmap 10 up:standby 2024-05-14T16:51:12.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:12 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-14T16:51:12.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:12 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:12.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:12 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:12.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:12 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:12.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:12 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:51:12.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:12 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:51:12.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:12 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:51:12.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:12 smithi123 ceph-mon[35971]: Deploying daemon mds.a on smithi002 2024-05-14T16:51:12.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:12 smithi123 ceph-mon[35971]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:12.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:12 smithi123 ceph-mon[35971]: mds.? [v2:172.21.15.2:6838/716718761,v1:172.21.15.2:6839/716718761] up:boot 2024-05-14T16:51:12.911 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:12 smithi123 ceph-mon[35971]: fsmap 10 up:standby 2024-05-14T16:51:12.911 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:12 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-14T16:51:13.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:12 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:13.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:12 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:13.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:12 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:13.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:12 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:51:13.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:12 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:51:13.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:12 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:51:13.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:12 smithi151 ceph-mon[36066]: Deploying daemon mds.a on smithi002 2024-05-14T16:51:13.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:12 smithi151 ceph-mon[36066]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:13.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:12 smithi151 ceph-mon[36066]: mds.? [v2:172.21.15.2:6838/716718761,v1:172.21.15.2:6839/716718761] up:boot 2024-05-14T16:51:13.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:12 smithi151 ceph-mon[36066]: fsmap 10 up:standby 2024-05-14T16:51:13.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:12 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-14T16:51:13.330 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:51:15.217 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:15 smithi002 ceph-mon[31538]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:15.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.223+0000 7fd5e6a0f700 1 -- 172.21.15.2:0/938908780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5e01037f0 msgr2=0x7fd5e0103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:15.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.223+0000 7fd5e6a0f700 1 --2- 172.21.15.2:0/938908780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5e01037f0 0x7fd5e0103bd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd5d00099c0 tx=0x7fd5d0009cd0 comp rx=0 tx=0).stop 2024-05-14T16:51:15.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.223+0000 7fd5e6a0f700 1 -- 172.21.15.2:0/938908780 shutdown_connections 2024-05-14T16:51:15.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.223+0000 7fd5e6a0f700 1 --2- 172.21.15.2:0/938908780 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5e01088f0 0x7fd5e0110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:15.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.223+0000 7fd5e6a0f700 1 --2- 172.21.15.2:0/938908780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5e01037f0 0x7fd5e0103bd0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:15.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.223+0000 7fd5e6a0f700 1 --2- 172.21.15.2:0/938908780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5e01041a0 0x7fd5e01081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:15.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.223+0000 7fd5e6a0f700 1 -- 172.21.15.2:0/938908780 >> 172.21.15.2:0/938908780 conn(0x7fd5e007a6a0 msgr2=0x7fd5e007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:15.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.223+0000 7fd5e6a0f700 1 -- 172.21.15.2:0/938908780 shutdown_connections 2024-05-14T16:51:15.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.223+0000 7fd5e6a0f700 1 -- 172.21.15.2:0/938908780 wait complete. 2024-05-14T16:51:15.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.224+0000 7fd5e6a0f700 1 Processor -- start 2024-05-14T16:51:15.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.224+0000 7fd5e6a0f700 1 -- start start 2024-05-14T16:51:15.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.224+0000 7fd5e6a0f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5e01037f0 0x7fd5e0075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:15.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.224+0000 7fd5e6a0f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5e01041a0 0x7fd5e0075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:15.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.224+0000 7fd5e6a0f700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5e01088f0 0x7fd5e00762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:15.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.224+0000 7fd5e6a0f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e01136c0 con 0x7fd5e01037f0 2024-05-14T16:51:15.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.224+0000 7fd5e6a0f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e0113540 con 0x7fd5e01041a0 2024-05-14T16:51:15.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.224+0000 7fd5e6a0f700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e0113840 con 0x7fd5e01088f0 2024-05-14T16:51:15.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.224+0000 7fd5dffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5e01037f0 0x7fd5e0075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:15.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.224+0000 7fd5dffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5e01037f0 0x7fd5e0075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37722/0 (socket says 172.21.15.2:37722) 2024-05-14T16:51:15.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.224+0000 7fd5dffff700 1 -- 172.21.15.2:0/4154110232 learned_addr learned my addr 172.21.15.2:0/4154110232 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:51:15.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.224+0000 7fd5df7fe700 1 --2- 172.21.15.2:0/4154110232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5e01041a0 0x7fd5e0075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:15.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.224+0000 7fd5e4fac700 1 --2- 172.21.15.2:0/4154110232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5e01088f0 0x7fd5e00762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:15.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.225+0000 7fd5dffff700 1 -- 172.21.15.2:0/4154110232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5e01088f0 msgr2=0x7fd5e00762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:15.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.225+0000 7fd5dffff700 1 --2- 172.21.15.2:0/4154110232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5e01088f0 0x7fd5e00762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:15.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.225+0000 7fd5dffff700 1 -- 172.21.15.2:0/4154110232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5e01041a0 msgr2=0x7fd5e0075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:15.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.225+0000 7fd5dffff700 1 --2- 172.21.15.2:0/4154110232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5e01041a0 0x7fd5e0075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:15.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.225+0000 7fd5dffff700 1 -- 172.21.15.2:0/4154110232 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e01a8090 con 0x7fd5e01037f0 2024-05-14T16:51:15.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.225+0000 7fd5dffff700 1 --2- 172.21.15.2:0/4154110232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5e01037f0 0x7fd5e0075840 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd5d000b580 tx=0x7fd5d00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:15.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.225+0000 7fd5dd7fa700 1 -- 172.21.15.2:0/4154110232 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0027070 con 0x7fd5e01037f0 2024-05-14T16:51:15.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.225+0000 7fd5e6a0f700 1 -- 172.21.15.2:0/4154110232 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e01a8320 con 0x7fd5e01037f0 2024-05-14T16:51:15.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.225+0000 7fd5dd7fa700 1 -- 172.21.15.2:0/4154110232 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5d0003f20 con 0x7fd5e01037f0 2024-05-14T16:51:15.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.225+0000 7fd5dd7fa700 1 -- 172.21.15.2:0/4154110232 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0022d10 con 0x7fd5e01037f0 2024-05-14T16:51:15.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.225+0000 7fd5e6a0f700 1 -- 172.21.15.2:0/4154110232 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e01a87d0 con 0x7fd5e01037f0 2024-05-14T16:51:15.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.226+0000 7fd5e6a0f700 1 -- 172.21.15.2:0/4154110232 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5e010bb30 con 0x7fd5e01037f0 2024-05-14T16:51:15.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.227+0000 7fd5dd7fa700 1 -- 172.21.15.2:0/4154110232 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fd5d0022470 con 0x7fd5e01037f0 2024-05-14T16:51:15.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.228+0000 7fd5dd7fa700 1 --2- 172.21.15.2:0/4154110232 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fd5c80b2880 0x7fd5c80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:15.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.228+0000 7fd5df7fe700 1 --2- 172.21.15.2:0/4154110232 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fd5c80b2880 0x7fd5c80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:15.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.228+0000 7fd5dd7fa700 1 -- 172.21.15.2:0/4154110232 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fd5d00e2d20 con 0x7fd5e01037f0 2024-05-14T16:51:15.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.229+0000 7fd5df7fe700 1 --2- 172.21.15.2:0/4154110232 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fd5c80b2880 0x7fd5c80b4d40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0103110 tx=0x7fd5d40073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:15.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.232+0000 7fd5dd7fa700 1 -- 172.21.15.2:0/4154110232 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd5d0013070 con 0x7fd5e01037f0 2024-05-14T16:51:15.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:15 smithi123 ceph-mon[35971]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:15.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.494+0000 7fd5e6a0f700 1 -- 172.21.15.2:0/4154110232 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5e006a230 con 0x7fd5e01037f0 2024-05-14T16:51:15.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.495+0000 7fd5dd7fa700 1 -- 172.21.15.2:0/4154110232 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8812 (secure 0 0 0) 0x7fd5d00aaf70 con 0x7fd5e01037f0 2024-05-14T16:51:15.495 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:51:15.495 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2275696923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2275696923},{"type":"v1","addr":"172.21.15.2:6835","nonce":2275696923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/716718761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":716718761},{"type":"v1","addr":"172.21.15.2:6839","nonce":716718761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1880336731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1880336731},{"type":"v1","addr":"172.21.15.151:6833","nonce":1880336731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2415605548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2415605548},{"type":"v1","addr":"172.21.15.2:6837","nonce":2415605548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2635167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2635167},{"type":"v1","addr":"172.21.15.123:6833","nonce":2635167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/762044339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":762044339},{"type":"v1","addr":"172.21.15.151:6835","nonce":762044339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/132043151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":132043151},{"type":"v1","addr":"172.21.15.151:6837","nonce":132043151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24427,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/4107834375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":4107834375},{"type":"v1","addr":"172.21.15.151:6839","nonce":4107834375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1198670013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1198670013},{"type":"v1","addr":"172.21.15.123:6835","nonce":1198670013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/333426645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":333426645},{"type":"v1","addr":"172.21.15.123:6837","nonce":333426645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-14T16:51:15.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.497+0000 7fd5e6a0f700 1 -- 172.21.15.2:0/4154110232 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fd5c80b2880 msgr2=0x7fd5c80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:15.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.497+0000 7fd5e6a0f700 1 --2- 172.21.15.2:0/4154110232 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fd5c80b2880 0x7fd5c80b4d40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0103110 tx=0x7fd5d40073d0 comp rx=0 tx=0).stop 2024-05-14T16:51:15.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.497+0000 7fd5e6a0f700 1 -- 172.21.15.2:0/4154110232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5e01037f0 msgr2=0x7fd5e0075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:15.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.497+0000 7fd5e6a0f700 1 --2- 172.21.15.2:0/4154110232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5e01037f0 0x7fd5e0075840 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd5d000b580 tx=0x7fd5d00047b0 comp rx=0 tx=0).stop 2024-05-14T16:51:15.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.497+0000 7fd5e6a0f700 1 -- 172.21.15.2:0/4154110232 shutdown_connections 2024-05-14T16:51:15.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.497+0000 7fd5e6a0f700 1 --2- 172.21.15.2:0/4154110232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5e01088f0 0x7fd5e00762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:15.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.497+0000 7fd5e6a0f700 1 --2- 172.21.15.2:0/4154110232 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fd5c80b2880 0x7fd5c80b4d40 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:15.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.497+0000 7fd5e6a0f700 1 --2- 172.21.15.2:0/4154110232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5e01037f0 0x7fd5e0075840 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:15.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.497+0000 7fd5e6a0f700 1 --2- 172.21.15.2:0/4154110232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5e01041a0 0x7fd5e0075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:15.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.497+0000 7fd5e6a0f700 1 -- 172.21.15.2:0/4154110232 >> 172.21.15.2:0/4154110232 conn(0x7fd5e007a6a0 msgr2=0x7fd5e0101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:15.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.497+0000 7fd5e6a0f700 1 -- 172.21.15.2:0/4154110232 shutdown_connections 2024-05-14T16:51:15.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:15.498+0000 7fd5e6a0f700 1 -- 172.21.15.2:0/4154110232 wait complete. 2024-05-14T16:51:15.498 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 11 2024-05-14T16:51:15.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:15 smithi151 ceph-mon[36066]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:16.398 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:16 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/4154110232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:51:16.398 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:16 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:16.398 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:16 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:16.398 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:16 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:16.399 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:16 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:51:16.399 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:16 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:51:16.399 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:16 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:51:16.410 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:16 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/4154110232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:51:16.410 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:16 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:16.410 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:16 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:16.410 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:16 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:16.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:16 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:51:16.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:16 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:51:16.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:16 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:51:16.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:16 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/4154110232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:51:16.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:16 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:16.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:16 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:16.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:16 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:16.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:16 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-14T16:51:16.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:16 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-14T16:51:16.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:16 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:51:17.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:17 smithi123 ceph-mon[35971]: Deploying daemon mds.b on smithi123 2024-05-14T16:51:17.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:17 smithi123 ceph-mon[35971]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:17.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:17 smithi123 ceph-mon[35971]: mds.? [v2:172.21.15.2:6840/3644815179,v1:172.21.15.2:6841/3644815179] up:boot 2024-05-14T16:51:17.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:17 smithi123 ceph-mon[35971]: fsmap 11 up:standby 2024-05-14T16:51:17.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:17 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-14T16:51:17.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:17 smithi002 ceph-mon[31538]: Deploying daemon mds.b on smithi123 2024-05-14T16:51:17.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:17 smithi002 ceph-mon[31538]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:17.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:17 smithi002 ceph-mon[31538]: mds.? [v2:172.21.15.2:6840/3644815179,v1:172.21.15.2:6841/3644815179] up:boot 2024-05-14T16:51:17.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:17 smithi002 ceph-mon[31538]: fsmap 11 up:standby 2024-05-14T16:51:17.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:17 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-14T16:51:17.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:17 smithi151 ceph-mon[36066]: Deploying daemon mds.b on smithi123 2024-05-14T16:51:17.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:17 smithi151 ceph-mon[36066]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:17.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:17 smithi151 ceph-mon[36066]: mds.? [v2:172.21.15.2:6840/3644815179,v1:172.21.15.2:6841/3644815179] up:boot 2024-05-14T16:51:17.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:17 smithi151 ceph-mon[36066]: fsmap 11 up:standby 2024-05-14T16:51:17.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:17 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-14T16:51:18.160 INFO:journalctl@ceph.mds.b.smithi123.stdout:May 14 16:51:17 smithi123 systemd[1]: Starting Ceph mds.b for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T16:51:18.412 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:51:18.437 INFO:journalctl@ceph.mds.b.smithi123.stdout:May 14 16:51:18 smithi123 podman[66285]: 2024-05-14 16:51:18.075477797 +0000 UTC m=+0.017437271 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T16:51:18.437 INFO:journalctl@ceph.mds.b.smithi123.stdout:May 14 16:51:18 smithi123 podman[66285]: 2024-05-14 16:51:18.18451999 +0000 UTC m=+0.126479456 container create 0cf46c0bdaab045b58d2076dbcecd97ff2449d44697cf67c2c3c74bd678df44a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-14T16:51:18.437 INFO:journalctl@ceph.mds.b.smithi123.stdout:May 14 16:51:18 smithi123 podman[66285]: 2024-05-14 16:51:18.412132281 +0000 UTC m=+0.354091753 container init 0cf46c0bdaab045b58d2076dbcecd97ff2449d44697cf67c2c3c74bd678df44a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-14T16:51:18.437 INFO:journalctl@ceph.mds.b.smithi123.stdout:May 14 16:51:18 smithi123 podman[66285]: 2024-05-14 16:51:18.414699024 +0000 UTC m=+0.356658505 container start 0cf46c0bdaab045b58d2076dbcecd97ff2449d44697cf67c2c3c74bd678df44a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-14T16:51:18.437 INFO:journalctl@ceph.mds.b.smithi123.stdout:May 14 16:51:18 smithi123 ceph-mds[66304]: -- 172.21.15.123:0/1240616969 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5589c1a2fa40 con 0x5589c1a02c00 2024-05-14T16:51:18.684 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:51:18.909 INFO:journalctl@ceph.mds.b.smithi123.stdout:May 14 16:51:18 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-b[66300]: starting mds.b at 2024-05-14T16:51:18.910 INFO:journalctl@ceph.mds.b.smithi123.stdout:May 14 16:51:18 smithi123 bash[66285]: 0cf46c0bdaab045b58d2076dbcecd97ff2449d44697cf67c2c3c74bd678df44a 2024-05-14T16:51:18.910 INFO:journalctl@ceph.mds.b.smithi123.stdout:May 14 16:51:18 smithi123 systemd[1]: Started Ceph mds.b for ed40717a-1210-11ef-bc98-c7b262605968. 2024-05-14T16:51:19.838 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:19 smithi002 ceph-mon[31538]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:19.838 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:19 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:19.838 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:19 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:19.838 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:19 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:19.838 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:19 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:19.838 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:19 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:19.838 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:19 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:51:19.839 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:19 smithi002 ceph-mon[31538]: mds.? [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] up:boot 2024-05-14T16:51:19.839 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:19 smithi002 ceph-mon[31538]: fsmap 12 up:standby 2024-05-14T16:51:19.839 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:19 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T16:51:19.852 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:19 smithi151 ceph-mon[36066]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:19.852 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:19 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:19.852 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:19 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:19.852 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:19 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:19.852 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:19 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:19.852 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:19 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:19.852 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:19 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:51:19.852 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:19 smithi151 ceph-mon[36066]: mds.? [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] up:boot 2024-05-14T16:51:19.852 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:19 smithi151 ceph-mon[36066]: fsmap 12 up:standby 2024-05-14T16:51:19.852 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:19 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T16:51:19.857 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:19 smithi123 ceph-mon[35971]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:19.857 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:19 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:19.857 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:19 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:19.857 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:19 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:19.858 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:19 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:19.858 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:19 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:19.858 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:19 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:51:19.858 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:19 smithi123 ceph-mon[35971]: mds.? [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] up:boot 2024-05-14T16:51:19.858 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:19 smithi123 ceph-mon[35971]: fsmap 12 up:standby 2024-05-14T16:51:19.858 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:19 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T16:51:20.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.987+0000 7f229683d700 1 -- 172.21.15.2:0/2664415861 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2290101f00 msgr2=0x7f2290102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:20.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.987+0000 7f229683d700 1 --2- 172.21.15.2:0/2664415861 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2290101f00 0x7f2290102380 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2278009990 tx=0x7f2278009ca0 comp rx=0 tx=0).stop 2024-05-14T16:51:20.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.987+0000 7f229683d700 1 -- 172.21.15.2:0/2664415861 shutdown_connections 2024-05-14T16:51:20.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.987+0000 7f229683d700 1 --2- 172.21.15.2:0/2664415861 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22901028c0 0x7f2290110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:20.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.987+0000 7f229683d700 1 --2- 172.21.15.2:0/2664415861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2290108620 0x7f2290108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:20.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.987+0000 7f229683d700 1 --2- 172.21.15.2:0/2664415861 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2290101f00 0x7f2290102380 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:20.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.987+0000 7f229683d700 1 -- 172.21.15.2:0/2664415861 >> 172.21.15.2:0/2664415861 conn(0x7f2290077780 msgr2=0x7f2290077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:20.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.987+0000 7f229683d700 1 -- 172.21.15.2:0/2664415861 shutdown_connections 2024-05-14T16:51:20.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.987+0000 7f229683d700 1 -- 172.21.15.2:0/2664415861 wait complete. 2024-05-14T16:51:20.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.987+0000 7f229683d700 1 Processor -- start 2024-05-14T16:51:20.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.987+0000 7f229683d700 1 -- start start 2024-05-14T16:51:20.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.988+0000 7f229683d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2290101f00 0x7f22901030e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:20.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.988+0000 7f229683d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22901028c0 0x7f2290103620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:20.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.988+0000 7f229683d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2290108620 0x7f2290103b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:20.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.988+0000 7f229683d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2290113770 con 0x7f2290101f00 2024-05-14T16:51:20.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.988+0000 7f229683d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f22901135f0 con 0x7f2290108620 2024-05-14T16:51:20.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.988+0000 7f229683d700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f22901138f0 con 0x7f22901028c0 2024-05-14T16:51:20.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.988+0000 7f228ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2290101f00 0x7f22901030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:20.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.988+0000 7f228ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2290101f00 0x7f22901030e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37750/0 (socket says 172.21.15.2:37750) 2024-05-14T16:51:20.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.988+0000 7f228ffff700 1 -- 172.21.15.2:0/3819046337 learned_addr learned my addr 172.21.15.2:0/3819046337 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:51:20.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.988+0000 7f2287fff700 1 --2- 172.21.15.2:0/3819046337 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22901028c0 0x7f2290103620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:20.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.988+0000 7f2294dda700 1 --2- 172.21.15.2:0/3819046337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2290108620 0x7f2290103b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:20.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.988+0000 7f228ffff700 1 -- 172.21.15.2:0/3819046337 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22901028c0 msgr2=0x7f2290103620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:21.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.988+0000 7f228ffff700 1 --2- 172.21.15.2:0/3819046337 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22901028c0 0x7f2290103620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:21.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.988+0000 7f228ffff700 1 -- 172.21.15.2:0/3819046337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2290108620 msgr2=0x7f2290103b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:21.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.988+0000 7f228ffff700 1 --2- 172.21.15.2:0/3819046337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2290108620 0x7f2290103b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:21.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.988+0000 7f228ffff700 1 -- 172.21.15.2:0/3819046337 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22901a8170 con 0x7f2290101f00 2024-05-14T16:51:21.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.988+0000 7f2287fff700 1 --2- 172.21.15.2:0/3819046337 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22901028c0 0x7f2290103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:51:21.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.988+0000 7f228ffff700 1 --2- 172.21.15.2:0/3819046337 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2290101f00 0x7f22901030e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f228000e970 tx=0x7f228000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:21.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.989+0000 7f228dffb700 1 -- 172.21.15.2:0/3819046337 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2280004220 con 0x7f2290101f00 2024-05-14T16:51:21.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.989+0000 7f229683d700 1 -- 172.21.15.2:0/3819046337 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22901a8460 con 0x7f2290101f00 2024-05-14T16:51:21.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.989+0000 7f228dffb700 1 -- 172.21.15.2:0/3819046337 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22800043c0 con 0x7f2290101f00 2024-05-14T16:51:21.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.989+0000 7f228dffb700 1 -- 172.21.15.2:0/3819046337 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2280003ce0 con 0x7f2290101f00 2024-05-14T16:51:21.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.989+0000 7f229683d700 1 -- 172.21.15.2:0/3819046337 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22901a89c0 con 0x7f2290101f00 2024-05-14T16:51:21.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.990+0000 7f229683d700 1 -- 172.21.15.2:0/3819046337 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f229006a230 con 0x7f2290101f00 2024-05-14T16:51:21.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.991+0000 7f228dffb700 1 -- 172.21.15.2:0/3819046337 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f228000c7e0 con 0x7f2290101f00 2024-05-14T16:51:21.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.992+0000 7f228dffb700 1 --2- 172.21.15.2:0/3819046337 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f227c0b2880 0x7f227c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:21.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.992+0000 7f2287fff700 1 --2- 172.21.15.2:0/3819046337 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f227c0b2880 0x7f227c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:21.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.992+0000 7f228dffb700 1 -- 172.21.15.2:0/3819046337 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f22800e2010 con 0x7f2290101f00 2024-05-14T16:51:21.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.992+0000 7f2287fff700 1 --2- 172.21.15.2:0/3819046337 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f227c0b2880 0x7f227c0b4d40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f227800b2e0 tx=0x7f227801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:21.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:20.995+0000 7f228dffb700 1 -- 172.21.15.2:0/3819046337 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f22800aa260 con 0x7f2290101f00 2024-05-14T16:51:21.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:21.285+0000 7f229683d700 1 -- 172.21.15.2:0/3819046337 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f229002e560 con 0x7f2290101f00 2024-05-14T16:51:21.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:21.285+0000 7f228dffb700 1 -- 172.21.15.2:0/3819046337 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10461 (secure 0 0 0) 0x7f22800a9b60 con 0x7f2290101f00 2024-05-14T16:51:21.285 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:51:21.286 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2275696923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2275696923},{"type":"v1","addr":"172.21.15.2:6835","nonce":2275696923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/716718761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":716718761},{"type":"v1","addr":"172.21.15.2:6839","nonce":716718761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/3644815179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3644815179},{"type":"v1","addr":"172.21.15.2:6841","nonce":3644815179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1880336731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1880336731},{"type":"v1","addr":"172.21.15.151:6833","nonce":1880336731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2415605548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2415605548},{"type":"v1","addr":"172.21.15.2:6837","nonce":2415605548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2635167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2635167},{"type":"v1","addr":"172.21.15.123:6833","nonce":2635167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/762044339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":762044339},{"type":"v1","addr":"172.21.15.151:6835","nonce":762044339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/132043151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":132043151},{"type":"v1","addr":"172.21.15.151:6837","nonce":132043151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24427,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/4107834375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":4107834375},{"type":"v1","addr":"172.21.15.151:6839","nonce":4107834375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24431,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1198670013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1198670013},{"type":"v1","addr":"172.21.15.123:6835","nonce":1198670013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/333426645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":333426645},{"type":"v1","addr":"172.21.15.123:6837","nonce":333426645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24479,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/580275877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":580275877},{"type":"v1","addr":"172.21.15.123:6839","nonce":580275877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-05-14T16:51:21.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:21.288+0000 7f229683d700 1 -- 172.21.15.2:0/3819046337 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f227c0b2880 msgr2=0x7f227c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:21.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:21.288+0000 7f229683d700 1 --2- 172.21.15.2:0/3819046337 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f227c0b2880 0x7f227c0b4d40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f227800b2e0 tx=0x7f227801a040 comp rx=0 tx=0).stop 2024-05-14T16:51:21.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:21.288+0000 7f229683d700 1 -- 172.21.15.2:0/3819046337 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2290101f00 msgr2=0x7f22901030e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:21.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:21.288+0000 7f229683d700 1 --2- 172.21.15.2:0/3819046337 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2290101f00 0x7f22901030e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f228000e970 tx=0x7f228000ec80 comp rx=0 tx=0).stop 2024-05-14T16:51:21.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:21.288+0000 7f229683d700 1 -- 172.21.15.2:0/3819046337 shutdown_connections 2024-05-14T16:51:21.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:21.288+0000 7f229683d700 1 --2- 172.21.15.2:0/3819046337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2290108620 0x7f2290103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:21.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:21.288+0000 7f229683d700 1 --2- 172.21.15.2:0/3819046337 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f227c0b2880 0x7f227c0b4d40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:21.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:21.288+0000 7f229683d700 1 --2- 172.21.15.2:0/3819046337 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2290101f00 0x7f22901030e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:21.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:21.288+0000 7f229683d700 1 --2- 172.21.15.2:0/3819046337 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22901028c0 0x7f2290103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:21.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:21.288+0000 7f229683d700 1 -- 172.21.15.2:0/3819046337 >> 172.21.15.2:0/3819046337 conn(0x7f2290077780 msgr2=0x7f2290100a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:21.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:21.288+0000 7f229683d700 1 -- 172.21.15.2:0/3819046337 shutdown_connections 2024-05-14T16:51:21.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:21.288+0000 7f229683d700 1 -- 172.21.15.2:0/3819046337 wait complete. 2024-05-14T16:51:21.289 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 13 2024-05-14T16:51:21.610 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:21 smithi002 ceph-mon[31538]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:21.611 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:21 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:21.611 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:21 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:21.611 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:21 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:21.611 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:21 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:21.611 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:21 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/3819046337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:51:21.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:21 smithi151 ceph-mon[36066]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:21.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:21 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:21.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:21 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:21.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:21 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:21.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:21 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:21.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:21 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/3819046337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:51:21.884 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-14T16:51:21.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:21 smithi123 ceph-mon[35971]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:21.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:21 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:21.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:21 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:21.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:21 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:21.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:21 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:21.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:21 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/3819046337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:51:22.144 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:51:23.183 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:22 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:23.183 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:22 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:23.183 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:22 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:51:23.183 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:22 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:51:23.183 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:22 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:23.183 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:22 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:23.183 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:22 smithi151 ceph-mon[36066]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:23.184 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:22 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:51:23.202 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:22 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:23.202 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:22 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:23.202 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:22 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:51:23.202 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:22 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:51:23.202 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:22 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:23.203 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:22 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:23.203 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:22 smithi123 ceph-mon[35971]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:23.203 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:22 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:51:23.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:22 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:23.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:22 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:23.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:22 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:51:23.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:22 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:51:23.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:22 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:23.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:22 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:23.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:22 smithi002 ceph-mon[31538]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:23.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:22 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:51:23.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:23.753+0000 7f62c25d1700 1 -- 172.21.15.2:0/1130401322 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62bc06bd10 msgr2=0x7f62bc06c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:23.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:23.753+0000 7f62c25d1700 1 --2- 172.21.15.2:0/1130401322 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62bc06bd10 0x7f62bc06c190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f62a40099c0 tx=0x7f62a4009cd0 comp rx=0 tx=0).stop 2024-05-14T16:51:23.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:23.753+0000 7f62c25d1700 1 -- 172.21.15.2:0/1130401322 shutdown_connections 2024-05-14T16:51:23.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:23.753+0000 7f62c25d1700 1 --2- 172.21.15.2:0/1130401322 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62bc06c6d0 0x7f62bc110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:23.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:23.753+0000 7f62c25d1700 1 --2- 172.21.15.2:0/1130401322 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62bc109630 0x7f62bc109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:23.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:23.753+0000 7f62c25d1700 1 --2- 172.21.15.2:0/1130401322 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62bc06bd10 0x7f62bc06c190 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:23.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:23.753+0000 7f62c25d1700 1 -- 172.21.15.2:0/1130401322 >> 172.21.15.2:0/1130401322 conn(0x7f62bc0794e0 msgr2=0x7f62bc0798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:23.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:23.753+0000 7f62c25d1700 1 -- 172.21.15.2:0/1130401322 shutdown_connections 2024-05-14T16:51:23.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:23.753+0000 7f62c25d1700 1 -- 172.21.15.2:0/1130401322 wait complete. 2024-05-14T16:51:23.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:23.754+0000 7f62c25d1700 1 Processor -- start 2024-05-14T16:51:23.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:23.754+0000 7f62c25d1700 1 -- start start 2024-05-14T16:51:23.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:23.754+0000 7f62c25d1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62bc06bd10 0x7f62bc19f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:23.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:23.754+0000 7f62c25d1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62bc06c6d0 0x7f62bc19fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:23.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:23.754+0000 7f62c25d1700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62bc109630 0x7f62bc199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:23.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:23.754+0000 7f62c25d1700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f62bc113790 con 0x7f62bc06bd10 2024-05-14T16:51:23.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:23.754+0000 7f62c25d1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f62bc113610 con 0x7f62bc06c6d0 2024-05-14T16:51:23.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:23.754+0000 7f62c25d1700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f62bc113910 con 0x7f62bc109630 2024-05-14T16:51:23.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:23.754+0000 7f62bbfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62bc06bd10 0x7f62bc19f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:23.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:23.754+0000 7f62bbfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62bc06bd10 0x7f62bc19f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52134/0 (socket says 172.21.15.2:52134) 2024-05-14T16:51:23.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:23.754+0000 7f62bbfff700 1 -- 172.21.15.2:0/2722469218 learned_addr learned my addr 172.21.15.2:0/2722469218 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:51:23.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:23.754+0000 7f62b3fff700 1 --2- 172.21.15.2:0/2722469218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62bc06c6d0 0x7f62bc19fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:23.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:23.754+0000 7f62c0b6e700 1 --2- 172.21.15.2:0/2722469218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62bc109630 0x7f62bc199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:23.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:23.754+0000 7f62bbfff700 1 -- 172.21.15.2:0/2722469218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62bc109630 msgr2=0x7f62bc199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:23.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:23.754+0000 7f62bbfff700 1 --2- 172.21.15.2:0/2722469218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62bc109630 0x7f62bc199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:23.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:23.754+0000 7f62bbfff700 1 -- 172.21.15.2:0/2722469218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62bc06c6d0 msgr2=0x7f62bc19fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:23.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:23.754+0000 7f62bbfff700 1 --2- 172.21.15.2:0/2722469218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62bc06c6d0 0x7f62bc19fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:23.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:23.754+0000 7f62bbfff700 1 -- 172.21.15.2:0/2722469218 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62bc19a1d0 con 0x7f62bc06bd10 2024-05-14T16:51:23.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:23.755+0000 7f62bbfff700 1 --2- 172.21.15.2:0/2722469218 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62bc06bd10 0x7f62bc19f8b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f62ac00d950 tx=0x7f62ac00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:23.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:23.755+0000 7f62b9ffb700 1 -- 172.21.15.2:0/2722469218 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac016070 con 0x7f62bc06bd10 2024-05-14T16:51:23.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:23.755+0000 7f62b9ffb700 1 -- 172.21.15.2:0/2722469218 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62ac00fd40 con 0x7f62bc06bd10 2024-05-14T16:51:23.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:23.755+0000 7f62c25d1700 1 -- 172.21.15.2:0/2722469218 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62bc19a4c0 con 0x7f62bc06bd10 2024-05-14T16:51:23.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:23.755+0000 7f62b9ffb700 1 -- 172.21.15.2:0/2722469218 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac00e3b0 con 0x7f62bc06bd10 2024-05-14T16:51:23.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:23.755+0000 7f62c25d1700 1 -- 172.21.15.2:0/2722469218 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62bc1a6780 con 0x7f62bc06bd10 2024-05-14T16:51:23.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:23.756+0000 7f62c25d1700 1 -- 172.21.15.2:0/2722469218 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62bc06a230 con 0x7f62bc06bd10 2024-05-14T16:51:23.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:23.757+0000 7f62b9ffb700 1 -- 172.21.15.2:0/2722469218 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f62ac019070 con 0x7f62bc06bd10 2024-05-14T16:51:23.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:23.758+0000 7f62b9ffb700 1 --2- 172.21.15.2:0/2722469218 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f62a80b2880 0x7f62a80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:23.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:23.758+0000 7f62b3fff700 1 --2- 172.21.15.2:0/2722469218 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f62a80b2880 0x7f62a80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:23.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:23.758+0000 7f62b9ffb700 1 -- 172.21.15.2:0/2722469218 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f62ac0e1730 con 0x7f62bc06bd10 2024-05-14T16:51:23.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:23.758+0000 7f62b3fff700 1 --2- 172.21.15.2:0/2722469218 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f62a80b2880 0x7f62a80b4d40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f62a4009990 tx=0x7f62a4009650 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:23.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:23.761+0000 7f62b9ffb700 1 -- 172.21.15.2:0/2722469218 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f62ac0a98f0 con 0x7f62bc06bd10 2024-05-14T16:51:24.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:24.022+0000 7f62c25d1700 1 -- 172.21.15.2:0/2722469218 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f62bc02e560 con 0x7f62bc06bd10 2024-05-14T16:51:24.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:24.022+0000 7f62b9ffb700 1 -- 172.21.15.2:0/2722469218 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v71) ==== 84+0+65 (secure 0 0 0) 0x7f62ac0a91f0 con 0x7f62bc06bd10 2024-05-14T16:51:24.022 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:51:24.022 INFO:teuthology.orchestra.run.smithi002.stdout:[ 2024-05-14T16:51:24.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T16:51:24.022 INFO:teuthology.orchestra.run.smithi002.stdout: "poolnum": 1, 2024-05-14T16:51:24.022 INFO:teuthology.orchestra.run.smithi002.stdout: "poolname": ".mgr" 2024-05-14T16:51:24.023 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:51:24.023 INFO:teuthology.orchestra.run.smithi002.stdout:] 2024-05-14T16:51:24.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:24.024+0000 7f62c25d1700 1 -- 172.21.15.2:0/2722469218 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f62a80b2880 msgr2=0x7f62a80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:24.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:24.024+0000 7f62c25d1700 1 --2- 172.21.15.2:0/2722469218 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f62a80b2880 0x7f62a80b4d40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f62a4009990 tx=0x7f62a4009650 comp rx=0 tx=0).stop 2024-05-14T16:51:24.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:24.024+0000 7f62c25d1700 1 -- 172.21.15.2:0/2722469218 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62bc06bd10 msgr2=0x7f62bc19f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:24.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:24.024+0000 7f62c25d1700 1 --2- 172.21.15.2:0/2722469218 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62bc06bd10 0x7f62bc19f8b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f62ac00d950 tx=0x7f62ac00dc60 comp rx=0 tx=0).stop 2024-05-14T16:51:24.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:24.024+0000 7f62c25d1700 1 -- 172.21.15.2:0/2722469218 shutdown_connections 2024-05-14T16:51:24.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:24.024+0000 7f62c25d1700 1 --2- 172.21.15.2:0/2722469218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62bc109630 0x7f62bc199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:24.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:24.024+0000 7f62c25d1700 1 --2- 172.21.15.2:0/2722469218 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f62a80b2880 0x7f62a80b4d40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:24.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:24.024+0000 7f62c25d1700 1 --2- 172.21.15.2:0/2722469218 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62bc06bd10 0x7f62bc19f8b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:24.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:24.024+0000 7f62c25d1700 1 --2- 172.21.15.2:0/2722469218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62bc06c6d0 0x7f62bc19fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:24.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:24.024+0000 7f62c25d1700 1 -- 172.21.15.2:0/2722469218 >> 172.21.15.2:0/2722469218 conn(0x7f62bc0794e0 msgr2=0x7f62bc1011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:24.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:24.024+0000 7f62c25d1700 1 -- 172.21.15.2:0/2722469218 shutdown_connections 2024-05-14T16:51:24.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:24.024+0000 7f62c25d1700 1 -- 172.21.15.2:0/2722469218 wait complete. 2024-05-14T16:51:25.114 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:24 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:25.114 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:24 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:25.114 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:24 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:25.115 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:24 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:25.115 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:24 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/2722469218' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-14T16:51:25.115 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:24 smithi002 ceph-mon[31538]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:25.146 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-14T16:51:25.146 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-14T16:51:25.159 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:24 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:25.159 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:24 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:25.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:24 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:25.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:24 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:25.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:24 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/2722469218' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-14T16:51:25.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:24 smithi123 ceph-mon[35971]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:25.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:24 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:25.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:24 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:25.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:24 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:25.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:24 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:25.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:24 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/2722469218' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-14T16:51:25.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:24 smithi151 ceph-mon[36066]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:25.420 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:51:27.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.064+0000 7fc5aca28700 1 -- 172.21.15.2:0/1090244677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5a81095a0 msgr2=0x7fc5a8109980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:27.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.064+0000 7fc5aca28700 1 --2- 172.21.15.2:0/1090244677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5a81095a0 0x7fc5a8109980 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc598009990 tx=0x7fc598009ca0 comp rx=0 tx=0).stop 2024-05-14T16:51:27.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.064+0000 7fc5aca28700 1 -- 172.21.15.2:0/1090244677 shutdown_connections 2024-05-14T16:51:27.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.064+0000 7fc5aca28700 1 --2- 172.21.15.2:0/1090244677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5a806c470 0x7fc5a81103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:27.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.064+0000 7fc5aca28700 1 --2- 172.21.15.2:0/1090244677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5a81095a0 0x7fc5a8109980 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:27.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.064+0000 7fc5aca28700 1 --2- 172.21.15.2:0/1090244677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5a806bab0 0x7fc5a806bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:27.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.064+0000 7fc5aca28700 1 -- 172.21.15.2:0/1090244677 >> 172.21.15.2:0/1090244677 conn(0x7fc5a80793b0 msgr2=0x7fc5a80797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:27.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.064+0000 7fc5aca28700 1 -- 172.21.15.2:0/1090244677 shutdown_connections 2024-05-14T16:51:27.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.064+0000 7fc5aca28700 1 -- 172.21.15.2:0/1090244677 wait complete. 2024-05-14T16:51:27.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.065+0000 7fc5aca28700 1 Processor -- start 2024-05-14T16:51:27.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.065+0000 7fc5aca28700 1 -- start start 2024-05-14T16:51:27.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.065+0000 7fc5aca28700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5a806bab0 0x7fc5a8102500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:27.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.065+0000 7fc5aca28700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5a806c470 0x7fc5a8102a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:27.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.065+0000 7fc5aca28700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5a81095a0 0x7fc5a8102f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:27.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.065+0000 7fc5aca28700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a81136d0 con 0x7fc5a806bab0 2024-05-14T16:51:27.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.065+0000 7fc5aca28700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a8113550 con 0x7fc5a806c470 2024-05-14T16:51:27.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.065+0000 7fc5aca28700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a8113850 con 0x7fc5a81095a0 2024-05-14T16:51:27.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.065+0000 7fc5a659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5a806bab0 0x7fc5a8102500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:27.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.065+0000 7fc5a659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5a806bab0 0x7fc5a8102500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52138/0 (socket says 172.21.15.2:52138) 2024-05-14T16:51:27.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.065+0000 7fc5a659c700 1 -- 172.21.15.2:0/2824503980 learned_addr learned my addr 172.21.15.2:0/2824503980 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:51:27.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.065+0000 7fc5a5d9b700 1 --2- 172.21.15.2:0/2824503980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5a806c470 0x7fc5a8102a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:27.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.065+0000 7fc5a6d9d700 1 --2- 172.21.15.2:0/2824503980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5a81095a0 0x7fc5a8102f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:27.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.066+0000 7fc5a659c700 1 -- 172.21.15.2:0/2824503980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5a81095a0 msgr2=0x7fc5a8102f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:27.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.066+0000 7fc5a659c700 1 --2- 172.21.15.2:0/2824503980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5a81095a0 0x7fc5a8102f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:27.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.066+0000 7fc5a659c700 1 -- 172.21.15.2:0/2824503980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5a806c470 msgr2=0x7fc5a8102a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:27.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.066+0000 7fc5a659c700 1 --2- 172.21.15.2:0/2824503980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5a806c470 0x7fc5a8102a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:27.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.066+0000 7fc5a659c700 1 -- 172.21.15.2:0/2824503980 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a81a8130 con 0x7fc5a806bab0 2024-05-14T16:51:27.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.066+0000 7fc5a659c700 1 --2- 172.21.15.2:0/2824503980 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5a806bab0 0x7fc5a8102500 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc598000c00 tx=0x7fc5980047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:27.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.066+0000 7fc5977fe700 1 -- 172.21.15.2:0/2824503980 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc598027070 con 0x7fc5a806bab0 2024-05-14T16:51:27.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.066+0000 7fc5aca28700 1 -- 172.21.15.2:0/2824503980 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5a81a83c0 con 0x7fc5a806bab0 2024-05-14T16:51:27.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.066+0000 7fc5977fe700 1 -- 172.21.15.2:0/2824503980 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc598003f20 con 0x7fc5a806bab0 2024-05-14T16:51:27.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.066+0000 7fc5977fe700 1 -- 172.21.15.2:0/2824503980 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc598022cf0 con 0x7fc5a806bab0 2024-05-14T16:51:27.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.066+0000 7fc5aca28700 1 -- 172.21.15.2:0/2824503980 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5a81a8870 con 0x7fc5a806bab0 2024-05-14T16:51:27.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.067+0000 7fc5aca28700 1 -- 172.21.15.2:0/2824503980 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5a810ba80 con 0x7fc5a806bab0 2024-05-14T16:51:27.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.069+0000 7fc5977fe700 1 -- 172.21.15.2:0/2824503980 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fc59802a070 con 0x7fc5a806bab0 2024-05-14T16:51:27.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.069+0000 7fc5977fe700 1 --2- 172.21.15.2:0/2824503980 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fc5900b2880 0x7fc5900b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:27.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.070+0000 7fc5a5d9b700 1 --2- 172.21.15.2:0/2824503980 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fc5900b2880 0x7fc5900b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:27.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.070+0000 7fc5977fe700 1 -- 172.21.15.2:0/2824503980 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fc5980e2e50 con 0x7fc5a806bab0 2024-05-14T16:51:27.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.070+0000 7fc5a5d9b700 1 --2- 172.21.15.2:0/2824503980 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fc5900b2880 0x7fc5900b4d40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8079a20 tx=0x7fc59c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:27.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.073+0000 7fc5977fe700 1 -- 172.21.15.2:0/2824503980 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc598013070 con 0x7fc5a806bab0 2024-05-14T16:51:27.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:27.292+0000 7fc5aca28700 1 -- 172.21.15.2:0/2824503980 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc5a8103a30 con 0x7fc5a806bab0 2024-05-14T16:51:27.417 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:27 smithi123 ceph-mon[35971]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:27.417 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:27 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:27.417 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:27 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:27.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:27 smithi002 ceph-mon[31538]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:27.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:27 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:27.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:27 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:27.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:27 smithi151 ceph-mon[36066]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:27.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:27 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:27.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:27 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:28.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:28.251+0000 7fc5977fe700 1 -- 172.21.15.2:0/2824503980 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v72) ==== 138+0+0 (secure 0 0 0) 0x7fc5980ab0a0 con 0x7fc5a806bab0 2024-05-14T16:51:28.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:28.255+0000 7fc5aca28700 1 -- 172.21.15.2:0/2824503980 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fc5900b2880 msgr2=0x7fc5900b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:28.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:28.255+0000 7fc5aca28700 1 --2- 172.21.15.2:0/2824503980 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fc5900b2880 0x7fc5900b4d40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8079a20 tx=0x7fc59c00b410 comp rx=0 tx=0).stop 2024-05-14T16:51:28.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:28.255+0000 7fc5aca28700 1 -- 172.21.15.2:0/2824503980 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5a806bab0 msgr2=0x7fc5a8102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:28.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:28.255+0000 7fc5aca28700 1 --2- 172.21.15.2:0/2824503980 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5a806bab0 0x7fc5a8102500 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc598000c00 tx=0x7fc5980047b0 comp rx=0 tx=0).stop 2024-05-14T16:51:28.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:28.255+0000 7fc5aca28700 1 -- 172.21.15.2:0/2824503980 shutdown_connections 2024-05-14T16:51:28.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:28.255+0000 7fc5aca28700 1 --2- 172.21.15.2:0/2824503980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5a81095a0 0x7fc5a8102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:28.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:28.255+0000 7fc5aca28700 1 --2- 172.21.15.2:0/2824503980 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fc5900b2880 0x7fc5900b4d40 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:28.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:28.255+0000 7fc5aca28700 1 --2- 172.21.15.2:0/2824503980 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5a806bab0 0x7fc5a8102500 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:28.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:28.255+0000 7fc5aca28700 1 --2- 172.21.15.2:0/2824503980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5a806c470 0x7fc5a8102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:28.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:28.255+0000 7fc5aca28700 1 -- 172.21.15.2:0/2824503980 >> 172.21.15.2:0/2824503980 conn(0x7fc5a80793b0 msgr2=0x7fc5a8101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:28.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:28.256+0000 7fc5aca28700 1 -- 172.21.15.2:0/2824503980 shutdown_connections 2024-05-14T16:51:28.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:28.256+0000 7fc5aca28700 1 -- 172.21.15.2:0/2824503980 wait complete. 2024-05-14T16:51:28.257 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_metadata' created 2024-05-14T16:51:28.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:28 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:51:28.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:28 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:51:28.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:28 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:28.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:28 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/2824503980' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-14T16:51:28.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:28 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:51:28.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:28 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:51:28.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:28 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:28.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:28 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/2824503980' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-14T16:51:28.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:28 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:51:28.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:28 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:51:28.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:28 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:51:28.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:28 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/2824503980' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-14T16:51:28.821 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-14T16:51:29.076 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:51:29.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:29 smithi002 ceph-mon[31538]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:29.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:29 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/2824503980' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-14T16:51:29.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:29 smithi002 ceph-mon[31538]: osdmap e72: 12 total, 12 up, 12 in 2024-05-14T16:51:29.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:29 smithi151 ceph-mon[36066]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:29.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:29 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/2824503980' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-14T16:51:29.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:29 smithi151 ceph-mon[36066]: osdmap e72: 12 total, 12 up, 12 in 2024-05-14T16:51:29.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:29 smithi123 ceph-mon[35971]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:29.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:29 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/2824503980' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-14T16:51:29.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:29 smithi123 ceph-mon[35971]: osdmap e72: 12 total, 12 up, 12 in 2024-05-14T16:51:30.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:30 smithi151 ceph-mon[36066]: osdmap e73: 12 total, 12 up, 12 in 2024-05-14T16:51:30.633 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:30 smithi002 ceph-mon[31538]: osdmap e73: 12 total, 12 up, 12 in 2024-05-14T16:51:30.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:30 smithi123 ceph-mon[35971]: osdmap e73: 12 total, 12 up, 12 in 2024-05-14T16:51:30.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:30.875+0000 7f37706ca700 1 -- 172.21.15.2:0/1523405113 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f376806bd10 msgr2=0x7f376806c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:30.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:30.875+0000 7f37706ca700 1 --2- 172.21.15.2:0/1523405113 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f376806bd10 0x7f376806c190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f37580099c0 tx=0x7f3758009cd0 comp rx=0 tx=0).stop 2024-05-14T16:51:30.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:30.875+0000 7f37706ca700 1 -- 172.21.15.2:0/1523405113 shutdown_connections 2024-05-14T16:51:30.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:30.875+0000 7f37706ca700 1 --2- 172.21.15.2:0/1523405113 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f376806c6d0 0x7f3768110430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:30.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:30.875+0000 7f37706ca700 1 --2- 172.21.15.2:0/1523405113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3768109660 0x7f3768109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:30.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:30.875+0000 7f37706ca700 1 --2- 172.21.15.2:0/1523405113 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f376806bd10 0x7f376806c190 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:30.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:30.875+0000 7f37706ca700 1 -- 172.21.15.2:0/1523405113 >> 172.21.15.2:0/1523405113 conn(0x7f3768078350 msgr2=0x7f3768078760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:30.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:30.875+0000 7f37706ca700 1 -- 172.21.15.2:0/1523405113 shutdown_connections 2024-05-14T16:51:30.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:30.875+0000 7f37706ca700 1 -- 172.21.15.2:0/1523405113 wait complete. 2024-05-14T16:51:30.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:30.876+0000 7f37706ca700 1 Processor -- start 2024-05-14T16:51:30.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:30.876+0000 7f37706ca700 1 -- start start 2024-05-14T16:51:30.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:30.876+0000 7f37706ca700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f376806bd10 0x7f3768102640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:30.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:30.876+0000 7f37706ca700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f376806c6d0 0x7f3768102b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:30.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:30.876+0000 7f37706ca700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3768109660 0x7f37681030c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:30.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:30.876+0000 7f37706ca700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3768113760 con 0x7f376806bd10 2024-05-14T16:51:30.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:30.876+0000 7f37706ca700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f37681135e0 con 0x7f376806c6d0 2024-05-14T16:51:30.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:30.876+0000 7f37706ca700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f37681138e0 con 0x7f3768109660 2024-05-14T16:51:30.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:30.876+0000 7f376e466700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f376806bd10 0x7f3768102640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:30.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:30.876+0000 7f376e466700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f376806bd10 0x7f3768102640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52160/0 (socket says 172.21.15.2:52160) 2024-05-14T16:51:30.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:30.876+0000 7f376e466700 1 -- 172.21.15.2:0/1997734162 learned_addr learned my addr 172.21.15.2:0/1997734162 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:51:30.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:30.877+0000 7f376ec67700 1 --2- 172.21.15.2:0/1997734162 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3768109660 0x7f37681030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:30.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:30.877+0000 7f376e466700 1 -- 172.21.15.2:0/1997734162 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3768109660 msgr2=0x7f37681030c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:30.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:30.877+0000 7f376e466700 1 --2- 172.21.15.2:0/1997734162 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3768109660 0x7f37681030c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:30.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:30.877+0000 7f376e466700 1 -- 172.21.15.2:0/1997734162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f376806c6d0 msgr2=0x7f3768102b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:30.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:30.877+0000 7f376e466700 1 --2- 172.21.15.2:0/1997734162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f376806c6d0 0x7f3768102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:30.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:30.877+0000 7f376e466700 1 -- 172.21.15.2:0/1997734162 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37681a8180 con 0x7f376806bd10 2024-05-14T16:51:30.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:30.877+0000 7f376e466700 1 --2- 172.21.15.2:0/1997734162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f376806bd10 0x7f3768102640 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f376400d980 tx=0x7f376400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:30.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:30.877+0000 7f375f7fe700 1 -- 172.21.15.2:0/1997734162 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3764016070 con 0x7f376806bd10 2024-05-14T16:51:30.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:30.877+0000 7f37706ca700 1 -- 172.21.15.2:0/1997734162 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37681a8470 con 0x7f376806bd10 2024-05-14T16:51:30.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:30.877+0000 7f375f7fe700 1 -- 172.21.15.2:0/1997734162 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37640044d0 con 0x7f376806bd10 2024-05-14T16:51:30.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:30.877+0000 7f375f7fe700 1 -- 172.21.15.2:0/1997734162 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3764004d00 con 0x7f376806bd10 2024-05-14T16:51:30.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:30.877+0000 7f37706ca700 1 -- 172.21.15.2:0/1997734162 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37681a89b0 con 0x7f376806bd10 2024-05-14T16:51:30.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:30.878+0000 7f37706ca700 1 -- 172.21.15.2:0/1997734162 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f376806a230 con 0x7f376806bd10 2024-05-14T16:51:30.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:30.880+0000 7f375f7fe700 1 -- 172.21.15.2:0/1997734162 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f3764004020 con 0x7f376806bd10 2024-05-14T16:51:30.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:30.881+0000 7f375f7fe700 1 --2- 172.21.15.2:0/1997734162 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f37540b2880 0x7f37540b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:30.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:30.881+0000 7f376dc65700 1 --2- 172.21.15.2:0/1997734162 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f37540b2880 0x7f37540b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:30.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:30.881+0000 7f375f7fe700 1 -- 172.21.15.2:0/1997734162 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7497+0+0 (secure 0 0 0) 0x7f37640e2350 con 0x7f376806bd10 2024-05-14T16:51:30.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:30.882+0000 7f376dc65700 1 --2- 172.21.15.2:0/1997734162 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f37540b2880 0x7f37540b4d40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3758003800 tx=0x7f3758000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:30.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:30.885+0000 7f375f7fe700 1 -- 172.21.15.2:0/1997734162 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f376401f020 con 0x7f376806bd10 2024-05-14T16:51:31.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:31.098+0000 7f37706ca700 1 -- 172.21.15.2:0/1997734162 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f376802e7a0 con 0x7f376806bd10 2024-05-14T16:51:31.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:31.271+0000 7f375f7fe700 1 -- 172.21.15.2:0/1997734162 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v75) ==== 170+0+0 (secure 0 0 0) 0x7f37640aa300 con 0x7f376806bd10 2024-05-14T16:51:31.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:31.273+0000 7f37706ca700 1 -- 172.21.15.2:0/1997734162 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f37540b2880 msgr2=0x7f37540b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:31.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:31.273+0000 7f37706ca700 1 --2- 172.21.15.2:0/1997734162 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f37540b2880 0x7f37540b4d40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3758003800 tx=0x7f3758000bc0 comp rx=0 tx=0).stop 2024-05-14T16:51:31.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:31.273+0000 7f37706ca700 1 -- 172.21.15.2:0/1997734162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f376806bd10 msgr2=0x7f3768102640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:31.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:31.273+0000 7f37706ca700 1 --2- 172.21.15.2:0/1997734162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f376806bd10 0x7f3768102640 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f376400d980 tx=0x7f376400dd40 comp rx=0 tx=0).stop 2024-05-14T16:51:31.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:31.273+0000 7f37706ca700 1 -- 172.21.15.2:0/1997734162 shutdown_connections 2024-05-14T16:51:31.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:31.273+0000 7f37706ca700 1 --2- 172.21.15.2:0/1997734162 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3768109660 0x7f37681030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:31.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:31.273+0000 7f37706ca700 1 --2- 172.21.15.2:0/1997734162 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f37540b2880 0x7f37540b4d40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:31.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:31.273+0000 7f37706ca700 1 --2- 172.21.15.2:0/1997734162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f376806bd10 0x7f3768102640 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:31.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:31.273+0000 7f37706ca700 1 --2- 172.21.15.2:0/1997734162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f376806c6d0 0x7f3768102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:31.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:31.273+0000 7f37706ca700 1 -- 172.21.15.2:0/1997734162 >> 172.21.15.2:0/1997734162 conn(0x7f3768078350 msgr2=0x7f37681013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:31.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:31.274+0000 7f37706ca700 1 -- 172.21.15.2:0/1997734162 shutdown_connections 2024-05-14T16:51:31.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:31.274+0000 7f37706ca700 1 -- 172.21.15.2:0/1997734162 wait complete. 2024-05-14T16:51:31.276 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_data' created 2024-05-14T16:51:31.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:31 smithi151 ceph-mon[36066]: pgmap v282: 33 pgs: 6 active+clean, 1 creating+activating, 3 creating+peering, 23 unknown; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:31.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:31 smithi151 ceph-mon[36066]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-14T16:51:31.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:31 smithi151 ceph-mon[36066]: osdmap e74: 12 total, 12 up, 12 in 2024-05-14T16:51:31.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:31 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/1997734162' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-14T16:51:31.610 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:31 smithi002 ceph-mon[31538]: pgmap v282: 33 pgs: 6 active+clean, 1 creating+activating, 3 creating+peering, 23 unknown; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:31.610 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:31 smithi002 ceph-mon[31538]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-14T16:51:31.610 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:31 smithi002 ceph-mon[31538]: osdmap e74: 12 total, 12 up, 12 in 2024-05-14T16:51:31.610 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:31 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/1997734162' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-14T16:51:31.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:31 smithi123 ceph-mon[35971]: pgmap v282: 33 pgs: 6 active+clean, 1 creating+activating, 3 creating+peering, 23 unknown; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:31.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:31 smithi123 ceph-mon[35971]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-14T16:51:31.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:31 smithi123 ceph-mon[35971]: osdmap e74: 12 total, 12 up, 12 in 2024-05-14T16:51:31.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:31 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/1997734162' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-14T16:51:31.944 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-14T16:51:32.202 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:51:32.564 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:32 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/1997734162' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-14T16:51:32.564 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:32 smithi002 ceph-mon[31538]: osdmap e75: 12 total, 12 up, 12 in 2024-05-14T16:51:32.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:32 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/1997734162' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-14T16:51:32.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:32 smithi151 ceph-mon[36066]: osdmap e75: 12 total, 12 up, 12 in 2024-05-14T16:51:32.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:32 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/1997734162' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-14T16:51:32.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:32 smithi123 ceph-mon[35971]: osdmap e75: 12 total, 12 up, 12 in 2024-05-14T16:51:33.455 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:33 smithi002 ceph-mon[31538]: pgmap v285: 97 pgs: 14 active+clean, 1 creating+activating, 15 creating+peering, 67 unknown; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:33.456 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:33 smithi002 ceph-mon[31538]: osdmap e76: 12 total, 12 up, 12 in 2024-05-14T16:51:33.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:33 smithi151 ceph-mon[36066]: pgmap v285: 97 pgs: 14 active+clean, 1 creating+activating, 15 creating+peering, 67 unknown; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:33.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:33 smithi151 ceph-mon[36066]: osdmap e76: 12 total, 12 up, 12 in 2024-05-14T16:51:33.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:33 smithi123 ceph-mon[35971]: pgmap v285: 97 pgs: 14 active+clean, 1 creating+activating, 15 creating+peering, 67 unknown; 577 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:33.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:33 smithi123 ceph-mon[35971]: osdmap e76: 12 total, 12 up, 12 in 2024-05-14T16:51:33.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:33.832+0000 7fa2503f0700 1 -- 172.21.15.2:0/3975147132 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa248103820 msgr2=0x7fa248103c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:33.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:33.832+0000 7fa2503f0700 1 --2- 172.21.15.2:0/3975147132 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa248103820 0x7fa248103c00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa2440099c0 tx=0x7fa244009cd0 comp rx=0 tx=0).stop 2024-05-14T16:51:33.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:33.832+0000 7fa2503f0700 1 -- 172.21.15.2:0/3975147132 shutdown_connections 2024-05-14T16:51:33.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:33.832+0000 7fa2503f0700 1 --2- 172.21.15.2:0/3975147132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa248108910 0x7fa248110490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:33.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:33.832+0000 7fa2503f0700 1 --2- 172.21.15.2:0/3975147132 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa248103820 0x7fa248103c00 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:33.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:33.832+0000 7fa2503f0700 1 --2- 172.21.15.2:0/3975147132 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2481041d0 0x7fa2481081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:33.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:33.832+0000 7fa2503f0700 1 -- 172.21.15.2:0/3975147132 >> 172.21.15.2:0/3975147132 conn(0x7fa24807a6b0 msgr2=0x7fa24807aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:33.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:33.832+0000 7fa2503f0700 1 -- 172.21.15.2:0/3975147132 shutdown_connections 2024-05-14T16:51:33.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:33.832+0000 7fa2503f0700 1 -- 172.21.15.2:0/3975147132 wait complete. 2024-05-14T16:51:33.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:33.833+0000 7fa2503f0700 1 Processor -- start 2024-05-14T16:51:33.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:33.833+0000 7fa2503f0700 1 -- start start 2024-05-14T16:51:33.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:33.833+0000 7fa2503f0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa248103820 0x7fa24819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:33.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:33.833+0000 7fa2503f0700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2481041d0 0x7fa24819fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:33.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:33.833+0000 7fa2503f0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa248108910 0x7fa2481999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:33.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:33.833+0000 7fa2503f0700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa2481136f0 con 0x7fa248103820 2024-05-14T16:51:33.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:33.833+0000 7fa2503f0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa248113570 con 0x7fa248108910 2024-05-14T16:51:33.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:33.833+0000 7fa2503f0700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa248113870 con 0x7fa2481041d0 2024-05-14T16:51:33.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:33.833+0000 7fa24e18c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa248103820 0x7fa24819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:33.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:33.833+0000 7fa24e18c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa248103820 0x7fa24819f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45154/0 (socket says 172.21.15.2:45154) 2024-05-14T16:51:33.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:33.833+0000 7fa24e18c700 1 -- 172.21.15.2:0/388063851 learned_addr learned my addr 172.21.15.2:0/388063851 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:51:33.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:33.833+0000 7fa24d98b700 1 --2- 172.21.15.2:0/388063851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2481041d0 0x7fa24819fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:33.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:33.833+0000 7fa24e98d700 1 --2- 172.21.15.2:0/388063851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa248108910 0x7fa2481999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:33.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:33.833+0000 7fa24e18c700 1 -- 172.21.15.2:0/388063851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2481041d0 msgr2=0x7fa24819fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:33.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:33.833+0000 7fa24e18c700 1 --2- 172.21.15.2:0/388063851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2481041d0 0x7fa24819fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:33.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:33.833+0000 7fa24e18c700 1 -- 172.21.15.2:0/388063851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa248108910 msgr2=0x7fa2481999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:33.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:33.833+0000 7fa24e18c700 1 --2- 172.21.15.2:0/388063851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa248108910 0x7fa2481999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:33.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:33.833+0000 7fa24e18c700 1 -- 172.21.15.2:0/388063851 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa24819a110 con 0x7fa248103820 2024-05-14T16:51:33.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:33.834+0000 7fa24e18c700 1 --2- 172.21.15.2:0/388063851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa248103820 0x7fa24819f7f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa2440098e0 tx=0x7fa2440057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:33.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:33.834+0000 7fa23f7fe700 1 -- 172.21.15.2:0/388063851 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa244027070 con 0x7fa248103820 2024-05-14T16:51:33.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:33.834+0000 7fa23f7fe700 1 -- 172.21.15.2:0/388063851 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa244003f20 con 0x7fa248103820 2024-05-14T16:51:33.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:33.834+0000 7fa2503f0700 1 -- 172.21.15.2:0/388063851 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa24819a3a0 con 0x7fa248103820 2024-05-14T16:51:33.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:33.834+0000 7fa23f7fe700 1 -- 172.21.15.2:0/388063851 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa244022da0 con 0x7fa248103820 2024-05-14T16:51:33.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:33.834+0000 7fa2503f0700 1 -- 172.21.15.2:0/388063851 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2481a66c0 con 0x7fa248103820 2024-05-14T16:51:33.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:33.835+0000 7fa2503f0700 1 -- 172.21.15.2:0/388063851 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa24806a230 con 0x7fa248103820 2024-05-14T16:51:33.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:33.836+0000 7fa23f7fe700 1 -- 172.21.15.2:0/388063851 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fa24402a070 con 0x7fa248103820 2024-05-14T16:51:33.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:33.837+0000 7fa23f7fe700 1 --2- 172.21.15.2:0/388063851 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fa2340b2900 0x7fa2340b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:33.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:33.837+0000 7fa24d98b700 1 --2- 172.21.15.2:0/388063851 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fa2340b2900 0x7fa2340b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:33.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:33.837+0000 7fa23f7fe700 1 -- 172.21.15.2:0/388063851 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7879+0+0 (secure 0 0 0) 0x7fa2440e3510 con 0x7fa248103820 2024-05-14T16:51:33.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:33.838+0000 7fa24d98b700 1 --2- 172.21.15.2:0/388063851 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fa2340b2900 0x7fa2340b4dc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa248103190 tx=0x7fa23800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:33.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:33.841+0000 7fa23f7fe700 1 -- 172.21.15.2:0/388063851 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa244031020 con 0x7fa248103820 2024-05-14T16:51:34.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:34.105+0000 7fa2503f0700 1 -- 172.21.15.2:0/388063851 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa24802e560 con 0x7fa248103820 2024-05-14T16:51:34.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:34.289+0000 7fa23f7fe700 1 -- 172.21.15.2:0/388063851 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fa244013070 con 0x7fa248103820 2024-05-14T16:51:34.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:34.292+0000 7fa2503f0700 1 -- 172.21.15.2:0/388063851 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fa2340b2900 msgr2=0x7fa2340b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:34.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:34.292+0000 7fa2503f0700 1 --2- 172.21.15.2:0/388063851 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fa2340b2900 0x7fa2340b4dc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa248103190 tx=0x7fa23800b410 comp rx=0 tx=0).stop 2024-05-14T16:51:34.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:34.292+0000 7fa2503f0700 1 -- 172.21.15.2:0/388063851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa248103820 msgr2=0x7fa24819f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:34.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:34.292+0000 7fa2503f0700 1 --2- 172.21.15.2:0/388063851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa248103820 0x7fa24819f7f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa2440098e0 tx=0x7fa2440057c0 comp rx=0 tx=0).stop 2024-05-14T16:51:34.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:34.292+0000 7fa2503f0700 1 -- 172.21.15.2:0/388063851 shutdown_connections 2024-05-14T16:51:34.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:34.292+0000 7fa2503f0700 1 --2- 172.21.15.2:0/388063851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa248108910 0x7fa2481999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:34.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:34.292+0000 7fa2503f0700 1 --2- 172.21.15.2:0/388063851 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fa2340b2900 0x7fa2340b4dc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:34.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:34.292+0000 7fa2503f0700 1 --2- 172.21.15.2:0/388063851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa248103820 0x7fa24819f7f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:34.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:34.292+0000 7fa2503f0700 1 --2- 172.21.15.2:0/388063851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2481041d0 0x7fa24819fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:34.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:34.292+0000 7fa2503f0700 1 -- 172.21.15.2:0/388063851 >> 172.21.15.2:0/388063851 conn(0x7fa24807a6b0 msgr2=0x7fa2481018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:34.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:34.292+0000 7fa2503f0700 1 -- 172.21.15.2:0/388063851 shutdown_connections 2024-05-14T16:51:34.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:34.292+0000 7fa2503f0700 1 -- 172.21.15.2:0/388063851 wait complete. 2024-05-14T16:51:34.293 INFO:teuthology.orchestra.run.smithi002.stderr:new fs with metadata pool 2 and data pool 3 2024-05-14T16:51:34.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:34 smithi002 ceph-mon[31538]: osdmap e77: 12 total, 12 up, 12 in 2024-05-14T16:51:34.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:34 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/388063851' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-14T16:51:34.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:34 smithi002 ceph-mon[31538]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-14T16:51:34.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:34 smithi151 ceph-mon[36066]: osdmap e77: 12 total, 12 up, 12 in 2024-05-14T16:51:34.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:34 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/388063851' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-14T16:51:34.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:34 smithi151 ceph-mon[36066]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-14T16:51:34.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:34 smithi123 ceph-mon[35971]: osdmap e77: 12 total, 12 up, 12 in 2024-05-14T16:51:34.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:34 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/388063851' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-14T16:51:34.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:34 smithi123 ceph-mon[35971]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-14T16:51:35.082 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T16:51:35.338 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:51:35.462 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:35 smithi002 ceph-mon[31538]: pgmap v288: 97 pgs: 51 active+clean, 1 creating+activating, 16 creating+peering, 29 unknown; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:35.462 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:35 smithi002 ceph-mon[31538]: osdmap e78: 12 total, 12 up, 12 in 2024-05-14T16:51:35.462 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:35 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/388063851' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-14T16:51:35.462 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:35 smithi002 ceph-mon[31538]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-14T16:51:35.462 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:35 smithi002 ceph-mon[31538]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-14T16:51:35.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:35 smithi151 ceph-mon[36066]: pgmap v288: 97 pgs: 51 active+clean, 1 creating+activating, 16 creating+peering, 29 unknown; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:35.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:35 smithi151 ceph-mon[36066]: osdmap e78: 12 total, 12 up, 12 in 2024-05-14T16:51:35.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:35 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/388063851' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-14T16:51:35.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:35 smithi151 ceph-mon[36066]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-14T16:51:35.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:35 smithi151 ceph-mon[36066]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-14T16:51:35.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:35 smithi123 ceph-mon[35971]: pgmap v288: 97 pgs: 51 active+clean, 1 creating+activating, 16 creating+peering, 29 unknown; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:35.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:35 smithi123 ceph-mon[35971]: osdmap e78: 12 total, 12 up, 12 in 2024-05-14T16:51:35.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:35 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/388063851' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-14T16:51:35.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:35 smithi123 ceph-mon[35971]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-14T16:51:35.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:35 smithi123 ceph-mon[35971]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-14T16:51:36.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:36 smithi002 ceph-mon[31538]: mds.? [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] up:active 2024-05-14T16:51:36.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:36 smithi002 ceph-mon[31538]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T16:51:36.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:36 smithi002 ceph-mon[31538]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T16:51:36.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:36 smithi151 ceph-mon[36066]: mds.? [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] up:active 2024-05-14T16:51:36.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:36 smithi151 ceph-mon[36066]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T16:51:36.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:36 smithi151 ceph-mon[36066]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T16:51:36.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:36 smithi123 ceph-mon[35971]: mds.? [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] up:active 2024-05-14T16:51:36.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:36 smithi123 ceph-mon[35971]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T16:51:36.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:36 smithi123 ceph-mon[35971]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T16:51:37.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.074+0000 7fbedc3f6700 1 -- 172.21.15.2:0/3478258505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbed41037f0 msgr2=0x7fbed4103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:37.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.074+0000 7fbedc3f6700 1 --2- 172.21.15.2:0/3478258505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbed41037f0 0x7fbed4103bd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbed00099c0 tx=0x7fbed0009cd0 comp rx=0 tx=0).stop 2024-05-14T16:51:37.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.074+0000 7fbedc3f6700 1 -- 172.21.15.2:0/3478258505 shutdown_connections 2024-05-14T16:51:37.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.074+0000 7fbedc3f6700 1 --2- 172.21.15.2:0/3478258505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbed41088f0 0x7fbed4110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:37.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.074+0000 7fbedc3f6700 1 --2- 172.21.15.2:0/3478258505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbed41037f0 0x7fbed4103bd0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:37.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.074+0000 7fbedc3f6700 1 --2- 172.21.15.2:0/3478258505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbed41041a0 0x7fbed41081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:37.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.074+0000 7fbedc3f6700 1 -- 172.21.15.2:0/3478258505 >> 172.21.15.2:0/3478258505 conn(0x7fbed407a6a0 msgr2=0x7fbed407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:37.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.074+0000 7fbedc3f6700 1 -- 172.21.15.2:0/3478258505 shutdown_connections 2024-05-14T16:51:37.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.074+0000 7fbedc3f6700 1 -- 172.21.15.2:0/3478258505 wait complete. 2024-05-14T16:51:37.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.074+0000 7fbedc3f6700 1 Processor -- start 2024-05-14T16:51:37.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.075+0000 7fbedc3f6700 1 -- start start 2024-05-14T16:51:37.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.075+0000 7fbedc3f6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbed41037f0 0x7fbed419f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:37.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.075+0000 7fbedc3f6700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbed41041a0 0x7fbed419fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:37.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.075+0000 7fbedc3f6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbed41088f0 0x7fbed4199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:37.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.075+0000 7fbedc3f6700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbed4113710 con 0x7fbed41088f0 2024-05-14T16:51:37.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.075+0000 7fbedc3f6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbed4113590 con 0x7fbed41037f0 2024-05-14T16:51:37.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.075+0000 7fbedc3f6700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbed4113890 con 0x7fbed41041a0 2024-05-14T16:51:37.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.075+0000 7fbeda993700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbed41088f0 0x7fbed4199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:37.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.075+0000 7fbeda993700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbed41088f0 0x7fbed4199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45170/0 (socket says 172.21.15.2:45170) 2024-05-14T16:51:37.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.075+0000 7fbeda993700 1 -- 172.21.15.2:0/78342453 learned_addr learned my addr 172.21.15.2:0/78342453 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:51:37.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.075+0000 7fbed9991700 1 --2- 172.21.15.2:0/78342453 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbed41041a0 0x7fbed419fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:37.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.075+0000 7fbeda192700 1 --2- 172.21.15.2:0/78342453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbed41037f0 0x7fbed419f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:37.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.075+0000 7fbeda993700 1 -- 172.21.15.2:0/78342453 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbed41041a0 msgr2=0x7fbed419fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:37.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.075+0000 7fbeda993700 1 --2- 172.21.15.2:0/78342453 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbed41041a0 0x7fbed419fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:37.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.075+0000 7fbeda993700 1 -- 172.21.15.2:0/78342453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbed41037f0 msgr2=0x7fbed419f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:37.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.075+0000 7fbeda993700 1 --2- 172.21.15.2:0/78342453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbed41037f0 0x7fbed419f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:37.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.075+0000 7fbeda993700 1 -- 172.21.15.2:0/78342453 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbed419a170 con 0x7fbed41088f0 2024-05-14T16:51:37.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.076+0000 7fbeda993700 1 --2- 172.21.15.2:0/78342453 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbed41088f0 0x7fbed4199a10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbecc007d70 tx=0x7fbecc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:37.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.076+0000 7fbecb7fe700 1 -- 172.21.15.2:0/78342453 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc016020 con 0x7fbed41088f0 2024-05-14T16:51:37.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.076+0000 7fbedc3f6700 1 -- 172.21.15.2:0/78342453 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbed419a460 con 0x7fbed41088f0 2024-05-14T16:51:37.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.076+0000 7fbecb7fe700 1 -- 172.21.15.2:0/78342453 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbecc00ac90 con 0x7fbed41088f0 2024-05-14T16:51:37.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.076+0000 7fbecb7fe700 1 -- 172.21.15.2:0/78342453 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc0115e0 con 0x7fbed41088f0 2024-05-14T16:51:37.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.076+0000 7fbedc3f6700 1 -- 172.21.15.2:0/78342453 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbed41a66d0 con 0x7fbed41088f0 2024-05-14T16:51:37.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.077+0000 7fbedc3f6700 1 -- 172.21.15.2:0/78342453 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbed410bb30 con 0x7fbed41088f0 2024-05-14T16:51:37.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.078+0000 7fbecb7fe700 1 -- 172.21.15.2:0/78342453 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fbecc019020 con 0x7fbed41088f0 2024-05-14T16:51:37.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.079+0000 7fbecb7fe700 1 --2- 172.21.15.2:0/78342453 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fbec00b2880 0x7fbec00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:37.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.080+0000 7fbeda192700 1 --2- 172.21.15.2:0/78342453 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fbec00b2880 0x7fbec00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:37.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.080+0000 7fbecb7fe700 1 -- 172.21.15.2:0/78342453 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7fbecc0e24c0 con 0x7fbed41088f0 2024-05-14T16:51:37.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.080+0000 7fbeda192700 1 --2- 172.21.15.2:0/78342453 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fbec00b2880 0x7fbec00b4d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbed0009990 tx=0x7fbed0009f10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:37.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.083+0000 7fbecb7fe700 1 -- 172.21.15.2:0/78342453 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbecc020020 con 0x7fbed41088f0 2024-05-14T16:51:37.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.305+0000 7fbedc3f6700 1 -- 172.21.15.2:0/78342453 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbed419aef0 con 0x7fbed41088f0 2024-05-14T16:51:37.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.306+0000 7fbecb7fe700 1 -- 172.21.15.2:0/78342453 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22199 (secure 0 0 0) 0x7fbecc0aa310 con 0x7fbed41088f0 2024-05-14T16:51:37.307 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:51:37.307 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":78,"fsid":"ed40717a-1210-11ef-bc98-c7b262605968","created":"2024-05-14T16:43:09.415298+0000","modified":"2024-05-14T16:51:34.106952+0000","last_up_change":"2024-05-14T16:50:29.787654+0000","last_in_change":"2024-05-14T16:50:09.287932+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T16:46:38.090903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T16:51:27.296955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T16:51:31.103019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"153f20ec-5bcf-4786-9ede-2e512a1e3d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6803","nonce":2654237820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6805","nonce":2654237820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6809","nonce":2654237820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6807","nonce":2654237820}]},"public_addr":"172.21.15.2:6803/2654237820","cluster_addr":"172.21.15.2:6805/2654237820","heartbeat_back_addr":"172.21.15.2:6809/2654237820","heartbeat_front_addr":"172.21.15.2:6807/2654237820","state":["exists","up"]},{"osd":1,"uuid":"0e95419f-9a9d-4f7a-b55d-ecb54971918a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6801","nonce":2668156992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6803","nonce":2668156992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6807","nonce":2668156992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6805","nonce":2668156992}]},"public_addr":"172.21.15.123:6801/2668156992","cluster_addr":"172.21.15.123:6803/2668156992","heartbeat_back_addr":"172.21.15.123:6807/2668156992","heartbeat_front_addr":"172.21.15.123:6805/2668156992","state":["exists","up"]},{"osd":2,"uuid":"b6ce7ac7-be6c-4549-a356-59cb81b5b492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6801","nonce":2160189636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6803","nonce":2160189636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6807","nonce":2160189636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6805","nonce":2160189636}]},"public_addr":"172.21.15.151:6801/2160189636","cluster_addr":"172.21.15.151:6803/2160189636","heartbeat_back_addr":"172.21.15.151:6807/2160189636","heartbeat_front_addr":"172.21.15.151:6805/2160189636","state":["exists","up"]},{"osd":3,"uuid":"2fd592bf-c212-411a-afbe-a7c498e5ac93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6811","nonce":3451424073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6813","nonce":3451424073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6817","nonce":3451424073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6815","nonce":3451424073}]},"public_addr":"172.21.15.2:6811/3451424073","cluster_addr":"172.21.15.2:6813/3451424073","heartbeat_back_addr":"172.21.15.2:6817/3451424073","heartbeat_front_addr":"172.21.15.2:6815/3451424073","state":["exists","up"]},{"osd":4,"uuid":"af1905e7-e036-47c6-be59-b085bc41fede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6809","nonce":2101935762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6811","nonce":2101935762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6815","nonce":2101935762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6813","nonce":2101935762}]},"public_addr":"172.21.15.123:6809/2101935762","cluster_addr":"172.21.15.123:6811/2101935762","heartbeat_back_addr":"172.21.15.123:6815/2101935762","heartbeat_front_addr":"172.21.15.123:6813/2101935762","state":["exists","up"]},{"osd":5,"uuid":"9c166916-af78-4cc2-bd14-d1220b6e7d9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6809","nonce":3695961886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6811","nonce":3695961886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6815","nonce":3695961886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6813","nonce":3695961886}]},"public_addr":"172.21.15.151:6809/3695961886","cluster_addr":"172.21.15.151:6811/3695961886","heartbeat_back_addr":"172.21.15.151:6815/3695961886","heartbeat_front_addr":"172.21.15.151:6813/3695961886","state":["exists","up"]},{"osd":6,"uuid":"60e9bf66-2338-41e8-9280-398d4e3b322f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6819","nonce":2711893600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6821","nonce":2711893600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6825","nonce":2711893600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6823","nonce":2711893600}]},"public_addr":"172.21.15.2:6819/2711893600","cluster_addr":"172.21.15.2:6821/2711893600","heartbeat_back_addr":"172.21.15.2:6825/2711893600","heartbeat_front_addr":"172.21.15.2:6823/2711893600","state":["exists","up"]},{"osd":7,"uuid":"c93e07a1-e941-493e-8468-33429df2dff0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6817","nonce":3634155507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6819","nonce":3634155507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6823","nonce":3634155507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6821","nonce":3634155507}]},"public_addr":"172.21.15.123:6817/3634155507","cluster_addr":"172.21.15.123:6819/3634155507","heartbeat_back_addr":"172.21.15.123:6823/3634155507","heartbeat_front_addr":"172.21.15.123:6821/3634155507","state":["exists","up"]},{"osd":8,"uuid":"1155fb58-d614-4f45-91e2-5da3a93e7d9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6817","nonce":4271107010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6819","nonce":4271107010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6823","nonce":4271107010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6821","nonce":4271107010}]},"public_addr":"172.21.15.151:6817/4271107010","cluster_addr":"172.21.15.151:6819/4271107010","heartbeat_back_addr":"172.21.15.151:6823/4271107010","heartbeat_front_addr":"172.21.15.151:6821/4271107010","state":["exists","up"]},{"osd":9,"uuid":"82b34e3b-0017-4ca4-8f86-0993ca78ce7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6827","nonce":3479431021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6829","nonce":3479431021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6833","nonce":3479431021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6831","nonce":3479431021}]},"public_addr":"172.21.15.2:6827/3479431021","cluster_addr":"172.21.15.2:6829/3479431021","heartbeat_back_addr":"172.21.15.2:6833/3479431021","heartbeat_front_addr":"172.21.15.2:6831/3479431021","state":["exists","up"]},{"osd":10,"uuid":"32e2ace9-eb5d-4258-a2ac-49b782be2b5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6825","nonce":4266374435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6827","nonce":4266374435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6831","nonce":4266374435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6829","nonce":4266374435}]},"public_addr":"172.21.15.123:6825/4266374435","cluster_addr":"172.21.15.123:6827/4266374435","heartbeat_back_addr":"172.21.15.123:6831/4266374435","heartbeat_front_addr":"172.21.15.123:6829/4266374435","state":["exists","up"]},{"osd":11,"uuid":"6e3c2ead-1af6-4bda-87cd-12770c81fa67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6825","nonce":4129194698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6827","nonce":4129194698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6831","nonce":4129194698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6829","nonce":4129194698}]},"public_addr":"172.21.15.151:6825/4129194698","cluster_addr":"172.21.15.151:6827/4129194698","heartbeat_back_addr":"172.21.15.151:6831/4129194698","heartbeat_front_addr":"172.21.15.151:6829/4129194698","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:45:46.540023+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:46:10.845186+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:46:34.283112+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:47:00.072467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:47:25.315644+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:47:49.748688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:48:16.964258+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:48:42.875515+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:49:08.596500+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:49:35.450943+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:50:01.293368+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:50:27.793713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/4267809161":"2024-05-15T16:44:11.998214+0000","172.21.15.2:6801/3582988344":"2024-05-15T16:44:11.998214+0000","172.21.15.2:6800/2574015037":"2024-05-15T16:43:43.407184+0000","172.21.15.2:0/2100732571":"2024-05-15T16:44:11.998214+0000","172.21.15.2:0/2954199324":"2024-05-15T16:43:43.407184+0000","172.21.15.2:6801/2574015037":"2024-05-15T16:43:43.407184+0000","172.21.15.2:0/3171550944":"2024-05-15T16:43:43.407184+0000","172.21.15.2:0/1679761136":"2024-05-15T16:44:11.998214+0000","172.21.15.2:0/1746766591":"2024-05-15T16:43:43.407184+0000","172.21.15.2:6800/3582988344":"2024-05-15T16:44:11.998214+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T16:51:37.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.309+0000 7fbedc3f6700 1 -- 172.21.15.2:0/78342453 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fbec00b2880 msgr2=0x7fbec00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:37.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.309+0000 7fbedc3f6700 1 --2- 172.21.15.2:0/78342453 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fbec00b2880 0x7fbec00b4d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbed0009990 tx=0x7fbed0009f10 comp rx=0 tx=0).stop 2024-05-14T16:51:37.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.309+0000 7fbedc3f6700 1 -- 172.21.15.2:0/78342453 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbed41088f0 msgr2=0x7fbed4199a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:37.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.309+0000 7fbedc3f6700 1 --2- 172.21.15.2:0/78342453 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbed41088f0 0x7fbed4199a10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbecc007d70 tx=0x7fbecc00a340 comp rx=0 tx=0).stop 2024-05-14T16:51:37.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.309+0000 7fbedc3f6700 1 -- 172.21.15.2:0/78342453 shutdown_connections 2024-05-14T16:51:37.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.309+0000 7fbedc3f6700 1 --2- 172.21.15.2:0/78342453 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fbec00b2880 0x7fbec00b4d40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:37.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.309+0000 7fbedc3f6700 1 --2- 172.21.15.2:0/78342453 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbed41088f0 0x7fbed4199a10 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:37.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.309+0000 7fbedc3f6700 1 --2- 172.21.15.2:0/78342453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbed41037f0 0x7fbed419f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:37.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.309+0000 7fbedc3f6700 1 --2- 172.21.15.2:0/78342453 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbed41041a0 0x7fbed419fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:37.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.309+0000 7fbedc3f6700 1 -- 172.21.15.2:0/78342453 >> 172.21.15.2:0/78342453 conn(0x7fbed407a6a0 msgr2=0x7fbed4101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:37.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.309+0000 7fbedc3f6700 1 -- 172.21.15.2:0/78342453 shutdown_connections 2024-05-14T16:51:37.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:37.309+0000 7fbedc3f6700 1 -- 172.21.15.2:0/78342453 wait complete. 2024-05-14T16:51:37.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:37 smithi002 ceph-mon[31538]: pgmap v290: 97 pgs: 81 active+clean, 16 creating+peering; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:37.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:37 smithi002 ceph-mon[31538]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-14T16:51:37.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:37 smithi002 ceph-mon[31538]: Cluster is now healthy 2024-05-14T16:51:37.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:37 smithi151 ceph-mon[36066]: pgmap v290: 97 pgs: 81 active+clean, 16 creating+peering; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:37.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:37 smithi151 ceph-mon[36066]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-14T16:51:37.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:37 smithi151 ceph-mon[36066]: Cluster is now healthy 2024-05-14T16:51:37.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:37 smithi123 ceph-mon[35971]: pgmap v290: 97 pgs: 81 active+clean, 16 creating+peering; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:51:37.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:37 smithi123 ceph-mon[35971]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-14T16:51:37.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:37 smithi123 ceph-mon[35971]: Cluster is now healthy 2024-05-14T16:51:38.099 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T16:51:38.357 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:51:38.479 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:38 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/78342453' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:51:38.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:38 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/78342453' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:51:38.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:38 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/78342453' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:51:39.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:39 smithi123 ceph-mon[35971]: pgmap v291: 97 pgs: 97 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-14T16:51:39.410 INFO:journalctl@ceph.mds.b.smithi123.stdout:May 14 16:51:39 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-b[66300]: 2024-05-14T16:51:39.150+0000 7f75be881700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-14T16:51:39.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:39 smithi151 ceph-mon[36066]: pgmap v291: 97 pgs: 97 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-14T16:51:39.715 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:39 smithi002 ceph-mon[31538]: pgmap v291: 97 pgs: 97 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-14T16:51:40.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.125+0000 7fc128369700 1 -- 172.21.15.2:0/1915271569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc12006c6d0 msgr2=0x7fc120110430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:40.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.125+0000 7fc128369700 1 --2- 172.21.15.2:0/1915271569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc12006c6d0 0x7fc120110430 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc1180099c0 tx=0x7fc118009cd0 comp rx=0 tx=0).stop 2024-05-14T16:51:40.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.125+0000 7fc128369700 1 -- 172.21.15.2:0/1915271569 shutdown_connections 2024-05-14T16:51:40.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.125+0000 7fc128369700 1 --2- 172.21.15.2:0/1915271569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc12006c6d0 0x7fc120110430 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:40.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.125+0000 7fc128369700 1 --2- 172.21.15.2:0/1915271569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc120109660 0x7fc120109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:40.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.125+0000 7fc128369700 1 --2- 172.21.15.2:0/1915271569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc12006bd10 0x7fc12006c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:40.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.125+0000 7fc128369700 1 -- 172.21.15.2:0/1915271569 >> 172.21.15.2:0/1915271569 conn(0x7fc120078350 msgr2=0x7fc120078760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:40.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.125+0000 7fc128369700 1 -- 172.21.15.2:0/1915271569 shutdown_connections 2024-05-14T16:51:40.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.125+0000 7fc128369700 1 -- 172.21.15.2:0/1915271569 wait complete. 2024-05-14T16:51:40.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.126+0000 7fc128369700 1 Processor -- start 2024-05-14T16:51:40.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.126+0000 7fc128369700 1 -- start start 2024-05-14T16:51:40.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.126+0000 7fc128369700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc12006bd10 0x7fc12019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:40.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.126+0000 7fc128369700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc12006c6d0 0x7fc12019fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:40.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.126+0000 7fc128369700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc120109660 0x7fc120199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:40.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.126+0000 7fc128369700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc1201137c0 con 0x7fc120109660 2024-05-14T16:51:40.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.126+0000 7fc128369700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc120113640 con 0x7fc12006bd10 2024-05-14T16:51:40.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.126+0000 7fc128369700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc120113940 con 0x7fc12006c6d0 2024-05-14T16:51:40.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.126+0000 7fc126906700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc120109660 0x7fc120199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:40.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.126+0000 7fc126906700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc120109660 0x7fc120199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45182/0 (socket says 172.21.15.2:45182) 2024-05-14T16:51:40.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.126+0000 7fc126906700 1 -- 172.21.15.2:0/921654016 learned_addr learned my addr 172.21.15.2:0/921654016 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:51:40.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.126+0000 7fc125904700 1 --2- 172.21.15.2:0/921654016 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc12006c6d0 0x7fc12019fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:40.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.126+0000 7fc126105700 1 --2- 172.21.15.2:0/921654016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc12006bd10 0x7fc12019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:40.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.127+0000 7fc126906700 1 -- 172.21.15.2:0/921654016 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc12006c6d0 msgr2=0x7fc12019fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:40.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.127+0000 7fc126906700 1 --2- 172.21.15.2:0/921654016 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc12006c6d0 0x7fc12019fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:40.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.127+0000 7fc126906700 1 -- 172.21.15.2:0/921654016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc12006bd10 msgr2=0x7fc12019f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:40.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.127+0000 7fc126906700 1 --2- 172.21.15.2:0/921654016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc12006bd10 0x7fc12019f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:40.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.127+0000 7fc126906700 1 -- 172.21.15.2:0/921654016 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc12019a180 con 0x7fc120109660 2024-05-14T16:51:40.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.127+0000 7fc126105700 1 --2- 172.21.15.2:0/921654016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc12006bd10 0x7fc12019f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:51:40.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.127+0000 7fc126906700 1 --2- 172.21.15.2:0/921654016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc120109660 0x7fc120199a20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc118005910 tx=0x7fc118009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:40.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.127+0000 7fc1177fe700 1 -- 172.21.15.2:0/921654016 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc118027070 con 0x7fc120109660 2024-05-14T16:51:40.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.127+0000 7fc128369700 1 -- 172.21.15.2:0/921654016 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc12019a410 con 0x7fc120109660 2024-05-14T16:51:40.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.127+0000 7fc1177fe700 1 -- 172.21.15.2:0/921654016 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc118003f20 con 0x7fc120109660 2024-05-14T16:51:40.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.127+0000 7fc1177fe700 1 -- 172.21.15.2:0/921654016 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc118022c50 con 0x7fc120109660 2024-05-14T16:51:40.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.127+0000 7fc128369700 1 -- 172.21.15.2:0/921654016 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1201a66e0 con 0x7fc120109660 2024-05-14T16:51:40.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.128+0000 7fc128369700 1 -- 172.21.15.2:0/921654016 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc12010baf0 con 0x7fc120109660 2024-05-14T16:51:40.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.130+0000 7fc1177fe700 1 -- 172.21.15.2:0/921654016 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fc11802a070 con 0x7fc120109660 2024-05-14T16:51:40.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.131+0000 7fc1177fe700 1 --2- 172.21.15.2:0/921654016 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fc10c0b2880 0x7fc10c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:40.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.131+0000 7fc126105700 1 --2- 172.21.15.2:0/921654016 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fc10c0b2880 0x7fc10c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:40.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.131+0000 7fc1177fe700 1 -- 172.21.15.2:0/921654016 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7fc1180e2f30 con 0x7fc120109660 2024-05-14T16:51:40.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.131+0000 7fc126105700 1 --2- 172.21.15.2:0/921654016 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fc10c0b2880 0x7fc10c0b4d40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc120109f20 tx=0x7fc11c009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:40.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.134+0000 7fc1177fe700 1 -- 172.21.15.2:0/921654016 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc118013070 con 0x7fc120109660 2024-05-14T16:51:40.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.358+0000 7fc128369700 1 -- 172.21.15.2:0/921654016 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc12019af00 con 0x7fc120109660 2024-05-14T16:51:40.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.359+0000 7fc1177fe700 1 -- 172.21.15.2:0/921654016 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22199 (secure 0 0 0) 0x7fc1180aae30 con 0x7fc120109660 2024-05-14T16:51:40.359 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:51:40.359 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":78,"fsid":"ed40717a-1210-11ef-bc98-c7b262605968","created":"2024-05-14T16:43:09.415298+0000","modified":"2024-05-14T16:51:34.106952+0000","last_up_change":"2024-05-14T16:50:29.787654+0000","last_in_change":"2024-05-14T16:50:09.287932+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T16:46:38.090903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T16:51:27.296955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T16:51:31.103019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"153f20ec-5bcf-4786-9ede-2e512a1e3d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6803","nonce":2654237820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6805","nonce":2654237820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6809","nonce":2654237820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6807","nonce":2654237820}]},"public_addr":"172.21.15.2:6803/2654237820","cluster_addr":"172.21.15.2:6805/2654237820","heartbeat_back_addr":"172.21.15.2:6809/2654237820","heartbeat_front_addr":"172.21.15.2:6807/2654237820","state":["exists","up"]},{"osd":1,"uuid":"0e95419f-9a9d-4f7a-b55d-ecb54971918a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6801","nonce":2668156992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6803","nonce":2668156992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6807","nonce":2668156992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6805","nonce":2668156992}]},"public_addr":"172.21.15.123:6801/2668156992","cluster_addr":"172.21.15.123:6803/2668156992","heartbeat_back_addr":"172.21.15.123:6807/2668156992","heartbeat_front_addr":"172.21.15.123:6805/2668156992","state":["exists","up"]},{"osd":2,"uuid":"b6ce7ac7-be6c-4549-a356-59cb81b5b492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6801","nonce":2160189636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6803","nonce":2160189636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6807","nonce":2160189636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6805","nonce":2160189636}]},"public_addr":"172.21.15.151:6801/2160189636","cluster_addr":"172.21.15.151:6803/2160189636","heartbeat_back_addr":"172.21.15.151:6807/2160189636","heartbeat_front_addr":"172.21.15.151:6805/2160189636","state":["exists","up"]},{"osd":3,"uuid":"2fd592bf-c212-411a-afbe-a7c498e5ac93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6811","nonce":3451424073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6813","nonce":3451424073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6817","nonce":3451424073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6815","nonce":3451424073}]},"public_addr":"172.21.15.2:6811/3451424073","cluster_addr":"172.21.15.2:6813/3451424073","heartbeat_back_addr":"172.21.15.2:6817/3451424073","heartbeat_front_addr":"172.21.15.2:6815/3451424073","state":["exists","up"]},{"osd":4,"uuid":"af1905e7-e036-47c6-be59-b085bc41fede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6809","nonce":2101935762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6811","nonce":2101935762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6815","nonce":2101935762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6813","nonce":2101935762}]},"public_addr":"172.21.15.123:6809/2101935762","cluster_addr":"172.21.15.123:6811/2101935762","heartbeat_back_addr":"172.21.15.123:6815/2101935762","heartbeat_front_addr":"172.21.15.123:6813/2101935762","state":["exists","up"]},{"osd":5,"uuid":"9c166916-af78-4cc2-bd14-d1220b6e7d9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6809","nonce":3695961886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6811","nonce":3695961886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6815","nonce":3695961886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6813","nonce":3695961886}]},"public_addr":"172.21.15.151:6809/3695961886","cluster_addr":"172.21.15.151:6811/3695961886","heartbeat_back_addr":"172.21.15.151:6815/3695961886","heartbeat_front_addr":"172.21.15.151:6813/3695961886","state":["exists","up"]},{"osd":6,"uuid":"60e9bf66-2338-41e8-9280-398d4e3b322f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6819","nonce":2711893600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6821","nonce":2711893600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6825","nonce":2711893600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6823","nonce":2711893600}]},"public_addr":"172.21.15.2:6819/2711893600","cluster_addr":"172.21.15.2:6821/2711893600","heartbeat_back_addr":"172.21.15.2:6825/2711893600","heartbeat_front_addr":"172.21.15.2:6823/2711893600","state":["exists","up"]},{"osd":7,"uuid":"c93e07a1-e941-493e-8468-33429df2dff0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6817","nonce":3634155507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6819","nonce":3634155507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6823","nonce":3634155507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6821","nonce":3634155507}]},"public_addr":"172.21.15.123:6817/3634155507","cluster_addr":"172.21.15.123:6819/3634155507","heartbeat_back_addr":"172.21.15.123:6823/3634155507","heartbeat_front_addr":"172.21.15.123:6821/3634155507","state":["exists","up"]},{"osd":8,"uuid":"1155fb58-d614-4f45-91e2-5da3a93e7d9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6817","nonce":4271107010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6819","nonce":4271107010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6823","nonce":4271107010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6821","nonce":4271107010}]},"public_addr":"172.21.15.151:6817/4271107010","cluster_addr":"172.21.15.151:6819/4271107010","heartbeat_back_addr":"172.21.15.151:6823/4271107010","heartbeat_front_addr":"172.21.15.151:6821/4271107010","state":["exists","up"]},{"osd":9,"uuid":"82b34e3b-0017-4ca4-8f86-0993ca78ce7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6827","nonce":3479431021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6829","nonce":3479431021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6833","nonce":3479431021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6831","nonce":3479431021}]},"public_addr":"172.21.15.2:6827/3479431021","cluster_addr":"172.21.15.2:6829/3479431021","heartbeat_back_addr":"172.21.15.2:6833/3479431021","heartbeat_front_addr":"172.21.15.2:6831/3479431021","state":["exists","up"]},{"osd":10,"uuid":"32e2ace9-eb5d-4258-a2ac-49b782be2b5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6825","nonce":4266374435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6827","nonce":4266374435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6831","nonce":4266374435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6829","nonce":4266374435}]},"public_addr":"172.21.15.123:6825/4266374435","cluster_addr":"172.21.15.123:6827/4266374435","heartbeat_back_addr":"172.21.15.123:6831/4266374435","heartbeat_front_addr":"172.21.15.123:6829/4266374435","state":["exists","up"]},{"osd":11,"uuid":"6e3c2ead-1af6-4bda-87cd-12770c81fa67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6825","nonce":4129194698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6827","nonce":4129194698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6831","nonce":4129194698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6829","nonce":4129194698}]},"public_addr":"172.21.15.151:6825/4129194698","cluster_addr":"172.21.15.151:6827/4129194698","heartbeat_back_addr":"172.21.15.151:6831/4129194698","heartbeat_front_addr":"172.21.15.151:6829/4129194698","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:45:46.540023+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:46:10.845186+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:46:34.283112+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:47:00.072467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:47:25.315644+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:47:49.748688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:48:16.964258+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:48:42.875515+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:49:08.596500+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:49:35.450943+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:50:01.293368+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:50:27.793713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/4267809161":"2024-05-15T16:44:11.998214+0000","172.21.15.2:6801/3582988344":"2024-05-15T16:44:11.998214+0000","172.21.15.2:6800/2574015037":"2024-05-15T16:43:43.407184+0000","172.21.15.2:0/2100732571":"2024-05-15T16:44:11.998214+0000","172.21.15.2:0/2954199324":"2024-05-15T16:43:43.407184+0000","172.21.15.2:6801/2574015037":"2024-05-15T16:43:43.407184+0000","172.21.15.2:0/3171550944":"2024-05-15T16:43:43.407184+0000","172.21.15.2:0/1679761136":"2024-05-15T16:44:11.998214+0000","172.21.15.2:0/1746766591":"2024-05-15T16:43:43.407184+0000","172.21.15.2:6800/3582988344":"2024-05-15T16:44:11.998214+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T16:51:40.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.362+0000 7fc128369700 1 -- 172.21.15.2:0/921654016 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fc10c0b2880 msgr2=0x7fc10c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:40.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.362+0000 7fc128369700 1 --2- 172.21.15.2:0/921654016 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fc10c0b2880 0x7fc10c0b4d40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc120109f20 tx=0x7fc11c009040 comp rx=0 tx=0).stop 2024-05-14T16:51:40.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.362+0000 7fc128369700 1 -- 172.21.15.2:0/921654016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc120109660 msgr2=0x7fc120199a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:40.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.362+0000 7fc128369700 1 --2- 172.21.15.2:0/921654016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc120109660 0x7fc120199a20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc118005910 tx=0x7fc118009580 comp rx=0 tx=0).stop 2024-05-14T16:51:40.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.362+0000 7fc128369700 1 -- 172.21.15.2:0/921654016 shutdown_connections 2024-05-14T16:51:40.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.362+0000 7fc128369700 1 --2- 172.21.15.2:0/921654016 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fc10c0b2880 0x7fc10c0b4d40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:40.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.362+0000 7fc128369700 1 --2- 172.21.15.2:0/921654016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc120109660 0x7fc120199a20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:40.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.362+0000 7fc128369700 1 --2- 172.21.15.2:0/921654016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc12006bd10 0x7fc12019f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:40.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.362+0000 7fc128369700 1 --2- 172.21.15.2:0/921654016 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc12006c6d0 0x7fc12019fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:40.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.362+0000 7fc128369700 1 -- 172.21.15.2:0/921654016 >> 172.21.15.2:0/921654016 conn(0x7fc120078350 msgr2=0x7fc120101060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:40.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.362+0000 7fc128369700 1 -- 172.21.15.2:0/921654016 shutdown_connections 2024-05-14T16:51:40.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:40.362+0000 7fc128369700 1 -- 172.21.15.2:0/921654016 wait complete. 2024-05-14T16:51:41.035 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-14T16:51:41.287 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:51:41.558 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:41 smithi002 ceph-mon[31538]: pgmap v292: 97 pgs: 97 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-14T16:51:41.558 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:41 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/921654016' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:51:41.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:41 smithi151 ceph-mon[36066]: pgmap v292: 97 pgs: 97 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-14T16:51:41.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:41 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/921654016' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:51:41.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:41 smithi123 ceph-mon[35971]: pgmap v292: 97 pgs: 97 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-14T16:51:41.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:41 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/921654016' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:51:42.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:42.975+0000 7f02cdacc700 1 -- 172.21.15.2:0/3602611964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02c8108920 msgr2=0x7f02c81104a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:42.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:42.975+0000 7f02cdacc700 1 --2- 172.21.15.2:0/3602611964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02c8108920 0x7f02c81104a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f02c000b0d0 tx=0x7f02c000b3e0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:42.975+0000 7f02cdacc700 1 -- 172.21.15.2:0/3602611964 shutdown_connections 2024-05-14T16:51:42.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:42.975+0000 7f02cdacc700 1 --2- 172.21.15.2:0/3602611964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02c8108920 0x7f02c81104a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:42.975+0000 7f02cdacc700 1 --2- 172.21.15.2:0/3602611964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02c8103850 0x7f02c8103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:42.975+0000 7f02cdacc700 1 --2- 172.21.15.2:0/3602611964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02c8104200 0x7f02c81081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:42.975+0000 7f02cdacc700 1 -- 172.21.15.2:0/3602611964 >> 172.21.15.2:0/3602611964 conn(0x7f02c807a5d0 msgr2=0x7f02c807a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:42.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:42.975+0000 7f02cdacc700 1 -- 172.21.15.2:0/3602611964 shutdown_connections 2024-05-14T16:51:42.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:42.975+0000 7f02cdacc700 1 -- 172.21.15.2:0/3602611964 wait complete. 2024-05-14T16:51:42.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:42.976+0000 7f02cdacc700 1 Processor -- start 2024-05-14T16:51:42.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:42.976+0000 7f02cdacc700 1 -- start start 2024-05-14T16:51:42.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:42.976+0000 7f02cdacc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02c8103850 0x7f02c819f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:42.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:42.976+0000 7f02cdacc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02c8104200 0x7f02c819fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:42.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:42.976+0000 7f02cdacc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02c8108920 0x7f02c8199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:42.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:42.976+0000 7f02cdacc700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f02c81137e0 con 0x7f02c8104200 2024-05-14T16:51:42.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:42.976+0000 7f02cdacc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f02c8113660 con 0x7f02c8103850 2024-05-14T16:51:42.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:42.976+0000 7f02cdacc700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f02c8113960 con 0x7f02c8108920 2024-05-14T16:51:42.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:42.976+0000 7f02beffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02c8104200 0x7f02c819fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:42.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:42.977+0000 7f02beffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02c8104200 0x7f02c819fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45196/0 (socket says 172.21.15.2:45196) 2024-05-14T16:51:42.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:42.977+0000 7f02beffd700 1 -- 172.21.15.2:0/3863191167 learned_addr learned my addr 172.21.15.2:0/3863191167 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:51:42.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:42.977+0000 7f02c77fe700 1 --2- 172.21.15.2:0/3863191167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02c8103850 0x7f02c819f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:42.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:42.977+0000 7f02c7fff700 1 --2- 172.21.15.2:0/3863191167 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02c8108920 0x7f02c8199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:42.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:42.977+0000 7f02beffd700 1 -- 172.21.15.2:0/3863191167 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02c8108920 msgr2=0x7f02c8199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:42.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:42.977+0000 7f02beffd700 1 --2- 172.21.15.2:0/3863191167 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02c8108920 0x7f02c8199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:42.977+0000 7f02beffd700 1 -- 172.21.15.2:0/3863191167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02c8103850 msgr2=0x7f02c819f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:42.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:42.977+0000 7f02beffd700 1 --2- 172.21.15.2:0/3863191167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02c8103850 0x7f02c819f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:42.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:42.977+0000 7f02beffd700 1 -- 172.21.15.2:0/3863191167 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c819a240 con 0x7f02c8104200 2024-05-14T16:51:42.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:42.977+0000 7f02beffd700 1 --2- 172.21.15.2:0/3863191167 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02c8104200 0x7f02c819fe10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f02b000b530 tx=0x7f02b000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:42.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:42.977+0000 7f02c57fa700 1 -- 172.21.15.2:0/3863191167 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b0015020 con 0x7f02c8104200 2024-05-14T16:51:42.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:42.977+0000 7f02c57fa700 1 -- 172.21.15.2:0/3863191167 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02b0011ce0 con 0x7f02c8104200 2024-05-14T16:51:42.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:42.977+0000 7f02cdacc700 1 -- 172.21.15.2:0/3863191167 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02c819a530 con 0x7f02c8104200 2024-05-14T16:51:42.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:42.977+0000 7f02c57fa700 1 -- 172.21.15.2:0/3863191167 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b0010e40 con 0x7f02c8104200 2024-05-14T16:51:42.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:42.977+0000 7f02cdacc700 1 -- 172.21.15.2:0/3863191167 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02c81a67a0 con 0x7f02c8104200 2024-05-14T16:51:42.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:42.978+0000 7f02cdacc700 1 -- 172.21.15.2:0/3863191167 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02c806a230 con 0x7f02c8104200 2024-05-14T16:51:42.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:42.980+0000 7f02c57fa700 1 -- 172.21.15.2:0/3863191167 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f02b0018020 con 0x7f02c8104200 2024-05-14T16:51:42.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:42.980+0000 7f02c57fa700 1 --2- 172.21.15.2:0/3863191167 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f02b40b2900 0x7f02b40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:42.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:42.981+0000 7f02c77fe700 1 --2- 172.21.15.2:0/3863191167 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f02b40b2900 0x7f02b40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:42.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:42.981+0000 7f02c57fa700 1 -- 172.21.15.2:0/3863191167 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f02b00e1910 con 0x7f02c8104200 2024-05-14T16:51:42.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:42.981+0000 7f02c77fe700 1 --2- 172.21.15.2:0/3863191167 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f02b40b2900 0x7f02b40b4dc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f02c8108460 tx=0x7f02b8009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:42.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:42.985+0000 7f02c57fa700 1 -- 172.21.15.2:0/3863191167 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f02b001f020 con 0x7f02c8104200 2024-05-14T16:51:43.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:43.248+0000 7f02cdacc700 1 -- 172.21.15.2:0/3863191167 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f02c802e560 con 0x7f02c8104200 2024-05-14T16:51:43.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:43.309+0000 7f02c57fa700 1 -- 172.21.15.2:0/3863191167 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f02b00a9780 con 0x7f02c8104200 2024-05-14T16:51:43.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:43.312+0000 7f02cdacc700 1 -- 172.21.15.2:0/3863191167 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f02b40b2900 msgr2=0x7f02b40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:43.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:43.312+0000 7f02cdacc700 1 --2- 172.21.15.2:0/3863191167 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f02b40b2900 0x7f02b40b4dc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f02c8108460 tx=0x7f02b8009190 comp rx=0 tx=0).stop 2024-05-14T16:51:43.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:43.312+0000 7f02cdacc700 1 -- 172.21.15.2:0/3863191167 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02c8104200 msgr2=0x7f02c819fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:43.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:43.312+0000 7f02cdacc700 1 --2- 172.21.15.2:0/3863191167 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02c8104200 0x7f02c819fe10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f02b000b530 tx=0x7f02b000b840 comp rx=0 tx=0).stop 2024-05-14T16:51:43.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:43.312+0000 7f02cdacc700 1 -- 172.21.15.2:0/3863191167 shutdown_connections 2024-05-14T16:51:43.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:43.312+0000 7f02cdacc700 1 --2- 172.21.15.2:0/3863191167 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02c8108920 0x7f02c8199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:43.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:43.312+0000 7f02cdacc700 1 --2- 172.21.15.2:0/3863191167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02c8103850 0x7f02c819f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:43.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:43.312+0000 7f02cdacc700 1 --2- 172.21.15.2:0/3863191167 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f02b40b2900 0x7f02b40b4dc0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:43.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:43.312+0000 7f02cdacc700 1 --2- 172.21.15.2:0/3863191167 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02c8104200 0x7f02c819fe10 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:43.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:43.312+0000 7f02cdacc700 1 -- 172.21.15.2:0/3863191167 >> 172.21.15.2:0/3863191167 conn(0x7f02c807a5d0 msgr2=0x7f02c81017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:43.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:43.312+0000 7f02cdacc700 1 -- 172.21.15.2:0/3863191167 shutdown_connections 2024-05-14T16:51:43.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:43.312+0000 7f02cdacc700 1 -- 172.21.15.2:0/3863191167 wait complete. 2024-05-14T16:51:43.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:43 smithi151 ceph-mon[36066]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-05-14T16:51:43.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:43 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/3863191167' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-14T16:51:43.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:43 smithi123 ceph-mon[35971]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-05-14T16:51:43.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:43 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/3863191167' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-14T16:51:43.684 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:43 smithi002 ceph-mon[31538]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-05-14T16:51:43.684 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:43 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/3863191167' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-14T16:51:44.071 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-05-14T16:51:44.071 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-05-14T16:51:44.323 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:51:44.572 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:44 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/3863191167' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-14T16:51:44.572 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:44 smithi002 ceph-mon[31538]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T16:51:44.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:44 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/3863191167' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-14T16:51:44.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:44 smithi151 ceph-mon[36066]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T16:51:44.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:44 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/3863191167' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-14T16:51:44.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:44 smithi123 ceph-mon[35971]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-14T16:51:45.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:45 smithi151 ceph-mon[36066]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-14T16:51:45.609 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:45 smithi002 ceph-mon[31538]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-14T16:51:45.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:45 smithi123 ceph-mon[35971]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-14T16:51:46.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.205+0000 7f47dcdab700 1 -- 172.21.15.2:0/4148127328 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47d806bd10 msgr2=0x7f47d806c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:46.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.205+0000 7f47dcdab700 1 --2- 172.21.15.2:0/4148127328 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47d806bd10 0x7f47d806c190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f47cc0099c0 tx=0x7f47cc009cd0 comp rx=0 tx=0).stop 2024-05-14T16:51:46.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.205+0000 7f47dcdab700 1 -- 172.21.15.2:0/4148127328 shutdown_connections 2024-05-14T16:51:46.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.205+0000 7f47dcdab700 1 --2- 172.21.15.2:0/4148127328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47d806c6d0 0x7f47d8110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:46.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.205+0000 7f47dcdab700 1 --2- 172.21.15.2:0/4148127328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47d8109630 0x7f47d8109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:46.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.205+0000 7f47dcdab700 1 --2- 172.21.15.2:0/4148127328 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47d806bd10 0x7f47d806c190 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:46.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.205+0000 7f47dcdab700 1 -- 172.21.15.2:0/4148127328 >> 172.21.15.2:0/4148127328 conn(0x7f47d80794e0 msgr2=0x7f47d80798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:46.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.205+0000 7f47dcdab700 1 -- 172.21.15.2:0/4148127328 shutdown_connections 2024-05-14T16:51:46.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.205+0000 7f47dcdab700 1 -- 172.21.15.2:0/4148127328 wait complete. 2024-05-14T16:51:46.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.206+0000 7f47dcdab700 1 Processor -- start 2024-05-14T16:51:46.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.206+0000 7f47dcdab700 1 -- start start 2024-05-14T16:51:46.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.206+0000 7f47dcdab700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47d806bd10 0x7f47d81024b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:46.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.206+0000 7f47dcdab700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47d806c6d0 0x7f47d81029f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:46.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.206+0000 7f47dcdab700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47d8109630 0x7f47d8102f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:46.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.206+0000 7f47dcdab700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f47d8113790 con 0x7f47d806c6d0 2024-05-14T16:51:46.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.207+0000 7f47dcdab700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f47d8113610 con 0x7f47d806bd10 2024-05-14T16:51:46.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.207+0000 7f47dcdab700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f47d8113910 con 0x7f47d8109630 2024-05-14T16:51:46.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.207+0000 7f47d5d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47d806c6d0 0x7f47d81029f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:46.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.207+0000 7f47d5d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47d806c6d0 0x7f47d81029f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58250/0 (socket says 172.21.15.2:58250) 2024-05-14T16:51:46.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.207+0000 7f47d5d9b700 1 -- 172.21.15.2:0/2436577262 learned_addr learned my addr 172.21.15.2:0/2436577262 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:51:46.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.207+0000 7f47d6d9d700 1 --2- 172.21.15.2:0/2436577262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47d8109630 0x7f47d8102f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:46.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.207+0000 7f47d659c700 1 --2- 172.21.15.2:0/2436577262 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47d806bd10 0x7f47d81024b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:46.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.207+0000 7f47d5d9b700 1 -- 172.21.15.2:0/2436577262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47d8109630 msgr2=0x7f47d8102f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:46.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.207+0000 7f47d5d9b700 1 --2- 172.21.15.2:0/2436577262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47d8109630 0x7f47d8102f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:46.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.207+0000 7f47d5d9b700 1 -- 172.21.15.2:0/2436577262 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47d806bd10 msgr2=0x7f47d81024b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:46.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.207+0000 7f47d5d9b700 1 --2- 172.21.15.2:0/2436577262 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47d806bd10 0x7f47d81024b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:46.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.207+0000 7f47d5d9b700 1 -- 172.21.15.2:0/2436577262 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d81a8110 con 0x7f47d806c6d0 2024-05-14T16:51:46.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.207+0000 7f47d659c700 1 --2- 172.21.15.2:0/2436577262 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47d806bd10 0x7f47d81024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:51:46.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.207+0000 7f47d5d9b700 1 --2- 172.21.15.2:0/2436577262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47d806c6d0 0x7f47d81029f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f47cc006010 tx=0x7f47cc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:46.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.207+0000 7f47c77fe700 1 -- 172.21.15.2:0/2436577262 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc026070 con 0x7f47d806c6d0 2024-05-14T16:51:46.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.207+0000 7f47c77fe700 1 -- 172.21.15.2:0/2436577262 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47cc00f460 con 0x7f47d806c6d0 2024-05-14T16:51:46.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.207+0000 7f47c77fe700 1 -- 172.21.15.2:0/2436577262 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc009e60 con 0x7f47d806c6d0 2024-05-14T16:51:46.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.207+0000 7f47dcdab700 1 -- 172.21.15.2:0/2436577262 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47d81a83a0 con 0x7f47d806c6d0 2024-05-14T16:51:46.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.208+0000 7f47dcdab700 1 -- 172.21.15.2:0/2436577262 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47d81a87a0 con 0x7f47d806c6d0 2024-05-14T16:51:46.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.209+0000 7f47dcdab700 1 -- 172.21.15.2:0/2436577262 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47d810bac0 con 0x7f47d806c6d0 2024-05-14T16:51:46.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.210+0000 7f47c77fe700 1 -- 172.21.15.2:0/2436577262 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f47cc029070 con 0x7f47d806c6d0 2024-05-14T16:51:46.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.211+0000 7f47c77fe700 1 --2- 172.21.15.2:0/2436577262 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f47c00b2880 0x7f47c00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:46.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.211+0000 7f47d659c700 1 --2- 172.21.15.2:0/2436577262 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f47c00b2880 0x7f47c00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:46.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.211+0000 7f47c77fe700 1 -- 172.21.15.2:0/2436577262 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f47cc0e29c0 con 0x7f47d806c6d0 2024-05-14T16:51:46.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.211+0000 7f47d659c700 1 --2- 172.21.15.2:0/2436577262 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f47c00b2880 0x7f47c00b4d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f47c8009c90 tx=0x7f47c8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:46.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.215+0000 7f47c77fe700 1 -- 172.21.15.2:0/2436577262 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f47cc013070 con 0x7f47d806c6d0 2024-05-14T16:51:46.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.478+0000 7f47dcdab700 1 -- 172.21.15.2:0/2436577262 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f47d802e600 con 0x7f47d806c6d0 2024-05-14T16:51:46.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.970+0000 7f47c77fe700 1 -- 172.21.15.2:0/2436577262 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f47cc0aa820 con 0x7f47d806c6d0 2024-05-14T16:51:46.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.973+0000 7f47dcdab700 1 -- 172.21.15.2:0/2436577262 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f47c00b2880 msgr2=0x7f47c00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:46.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.973+0000 7f47dcdab700 1 --2- 172.21.15.2:0/2436577262 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f47c00b2880 0x7f47c00b4d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f47c8009c90 tx=0x7f47c8009260 comp rx=0 tx=0).stop 2024-05-14T16:51:46.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.973+0000 7f47dcdab700 1 -- 172.21.15.2:0/2436577262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47d806c6d0 msgr2=0x7f47d81029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:46.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.973+0000 7f47dcdab700 1 --2- 172.21.15.2:0/2436577262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47d806c6d0 0x7f47d81029f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f47cc006010 tx=0x7f47cc0045e0 comp rx=0 tx=0).stop 2024-05-14T16:51:46.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.973+0000 7f47dcdab700 1 -- 172.21.15.2:0/2436577262 shutdown_connections 2024-05-14T16:51:46.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.973+0000 7f47dcdab700 1 --2- 172.21.15.2:0/2436577262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47d8109630 0x7f47d8102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:46.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.973+0000 7f47dcdab700 1 --2- 172.21.15.2:0/2436577262 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47d806bd10 0x7f47d81024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:46.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.973+0000 7f47dcdab700 1 --2- 172.21.15.2:0/2436577262 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f47c00b2880 0x7f47c00b4d40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:46.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.973+0000 7f47dcdab700 1 --2- 172.21.15.2:0/2436577262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47d806c6d0 0x7f47d81029f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:46.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.973+0000 7f47dcdab700 1 -- 172.21.15.2:0/2436577262 >> 172.21.15.2:0/2436577262 conn(0x7f47d80794e0 msgr2=0x7f47d81010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:46.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.974+0000 7f47dcdab700 1 -- 172.21.15.2:0/2436577262 shutdown_connections 2024-05-14T16:51:46.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:46.974+0000 7f47dcdab700 1 -- 172.21.15.2:0/2436577262 wait complete. 2024-05-14T16:51:47.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:47 smithi151 ceph-mon[36066]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-14T16:51:47.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:47 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/2436577262' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-14T16:51:47.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:47 smithi151 ceph-mon[36066]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-14T16:51:47.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:47 smithi151 ceph-mon[36066]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-14T16:51:47.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:47 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/2436577262' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-14T16:51:47.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:47 smithi151 ceph-mon[36066]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-14T16:51:47.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:47 smithi151 ceph-mon[36066]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-14T16:51:47.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:47 smithi123 ceph-mon[35971]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-14T16:51:47.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:47 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/2436577262' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-14T16:51:47.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:47 smithi123 ceph-mon[35971]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-14T16:51:47.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:47 smithi123 ceph-mon[35971]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-14T16:51:47.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:47 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/2436577262' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-14T16:51:47.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:47 smithi123 ceph-mon[35971]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-14T16:51:47.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:47 smithi123 ceph-mon[35971]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-14T16:51:47.667 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:47 smithi002 ceph-mon[31538]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-14T16:51:47.667 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:47 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/2436577262' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-14T16:51:47.667 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:47 smithi002 ceph-mon[31538]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-14T16:51:47.667 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:47 smithi002 ceph-mon[31538]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-14T16:51:47.667 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:47 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/2436577262' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-14T16:51:47.667 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:47 smithi002 ceph-mon[31538]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-14T16:51:47.667 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:47 smithi002 ceph-mon[31538]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-14T16:51:47.702 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-14T16:51:47.959 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:51:49.225 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:48 smithi002 ceph-mon[31538]: mds.? [v2:172.21.15.123:6836/333426645,v1:172.21.15.123:6837/333426645] up:active 2024-05-14T16:51:49.225 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:48 smithi002 ceph-mon[31538]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-14T16:51:49.225 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:48 smithi002 ceph-mon[31538]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-14T16:51:49.225 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:48 smithi002 ceph-mon[31538]: Cluster is now healthy 2024-05-14T16:51:49.225 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:48 smithi002 ceph-mon[31538]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-14T16:51:49.225 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:48 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-05-14T16:51:49.225 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:48 smithi002 ceph-mon[31538]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-05-14T16:51:49.225 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:48 smithi002 ceph-mon[31538]: pgmap v296: 97 pgs: 97 active+clean; 580 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-05-14T16:51:49.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:48 smithi151 ceph-mon[36066]: mds.? [v2:172.21.15.123:6836/333426645,v1:172.21.15.123:6837/333426645] up:active 2024-05-14T16:51:49.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:48 smithi151 ceph-mon[36066]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-14T16:51:49.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:48 smithi151 ceph-mon[36066]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-14T16:51:49.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:48 smithi151 ceph-mon[36066]: Cluster is now healthy 2024-05-14T16:51:49.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:48 smithi151 ceph-mon[36066]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-14T16:51:49.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:48 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-05-14T16:51:49.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:48 smithi151 ceph-mon[36066]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-05-14T16:51:49.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:48 smithi151 ceph-mon[36066]: pgmap v296: 97 pgs: 97 active+clean; 580 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-05-14T16:51:49.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:48 smithi123 ceph-mon[35971]: mds.? [v2:172.21.15.123:6836/333426645,v1:172.21.15.123:6837/333426645] up:active 2024-05-14T16:51:49.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:48 smithi123 ceph-mon[35971]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-14T16:51:49.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:48 smithi123 ceph-mon[35971]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-14T16:51:49.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:48 smithi123 ceph-mon[35971]: Cluster is now healthy 2024-05-14T16:51:49.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:48 smithi123 ceph-mon[35971]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-14T16:51:49.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:48 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-05-14T16:51:49.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:48 smithi123 ceph-mon[35971]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-05-14T16:51:49.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:48 smithi123 ceph-mon[35971]: pgmap v296: 97 pgs: 97 active+clean; 580 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-05-14T16:51:49.410 INFO:journalctl@ceph.mds.b.smithi123.stdout:May 14 16:51:49 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-b[66300]: 2024-05-14T16:51:49.151+0000 7f75be881700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-14T16:51:49.410 INFO:journalctl@ceph.mds.b.smithi123.stdout:May 14 16:51:49 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-b[66300]: 2024-05-14T16:51:49.151+0000 7f75be881700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-14T16:51:49.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.800+0000 7f0baa215700 1 -- 172.21.15.2:0/609511617 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ba4102470 msgr2=0x7f0ba41028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:49.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.800+0000 7f0baa215700 1 --2- 172.21.15.2:0/609511617 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ba4102470 0x7f0ba41028f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0b98009990 tx=0x7f0b98009ca0 comp rx=0 tx=0).stop 2024-05-14T16:51:49.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.800+0000 7f0baa215700 1 -- 172.21.15.2:0/609511617 shutdown_connections 2024-05-14T16:51:49.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.800+0000 7f0baa215700 1 --2- 172.21.15.2:0/609511617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ba4102e30 0x7f0ba410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:49.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.800+0000 7f0baa215700 1 --2- 172.21.15.2:0/609511617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ba41084b0 0x7f0ba4108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:49.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.800+0000 7f0baa215700 1 --2- 172.21.15.2:0/609511617 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ba4102470 0x7f0ba41028f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:49.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.800+0000 7f0baa215700 1 -- 172.21.15.2:0/609511617 >> 172.21.15.2:0/609511617 conn(0x7f0ba40793f0 msgr2=0x7f0ba4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:49.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.800+0000 7f0baa215700 1 -- 172.21.15.2:0/609511617 shutdown_connections 2024-05-14T16:51:49.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.800+0000 7f0baa215700 1 -- 172.21.15.2:0/609511617 wait complete. 2024-05-14T16:51:49.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.801+0000 7f0baa215700 1 Processor -- start 2024-05-14T16:51:49.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.801+0000 7f0baa215700 1 -- start start 2024-05-14T16:51:49.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.801+0000 7f0baa215700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ba4102470 0x7f0ba419be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:49.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.801+0000 7f0baa215700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ba4102e30 0x7f0ba419c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:49.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.801+0000 7f0baa215700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ba41084b0 0x7f0ba41a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:49.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.801+0000 7f0baa215700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba41125f0 con 0x7f0ba41084b0 2024-05-14T16:51:49.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.801+0000 7f0baa215700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba4112470 con 0x7f0ba4102e30 2024-05-14T16:51:49.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.801+0000 7f0baa215700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba4112770 con 0x7f0ba4102470 2024-05-14T16:51:49.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.802+0000 7f0ba37fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ba4102470 0x7f0ba419be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:49.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.802+0000 7f0ba3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ba41084b0 0x7f0ba41a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:49.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.802+0000 7f0ba37fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ba4102470 0x7f0ba419be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:48034/0 (socket says 172.21.15.2:48034) 2024-05-14T16:51:49.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.802+0000 7f0ba3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ba41084b0 0x7f0ba41a0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58266/0 (socket says 172.21.15.2:58266) 2024-05-14T16:51:49.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.802+0000 7f0ba37fe700 1 -- 172.21.15.2:0/2553030506 learned_addr learned my addr 172.21.15.2:0/2553030506 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:51:49.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.802+0000 7f0ba2ffd700 1 --2- 172.21.15.2:0/2553030506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ba4102e30 0x7f0ba419c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:49.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.802+0000 7f0ba3fff700 1 -- 172.21.15.2:0/2553030506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ba4102470 msgr2=0x7f0ba419be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:49.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.802+0000 7f0ba3fff700 1 --2- 172.21.15.2:0/2553030506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ba4102470 0x7f0ba419be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:49.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.802+0000 7f0ba3fff700 1 -- 172.21.15.2:0/2553030506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ba4102e30 msgr2=0x7f0ba419c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:49.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.802+0000 7f0ba3fff700 1 --2- 172.21.15.2:0/2553030506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ba4102e30 0x7f0ba419c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:49.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.802+0000 7f0ba3fff700 1 -- 172.21.15.2:0/2553030506 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba41a0e50 con 0x7f0ba41084b0 2024-05-14T16:51:49.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.802+0000 7f0ba37fe700 1 --2- 172.21.15.2:0/2553030506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ba4102470 0x7f0ba419be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:51:49.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.802+0000 7f0ba3fff700 1 --2- 172.21.15.2:0/2553030506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ba41084b0 0x7f0ba41a0710 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c00efc0 tx=0x7f0b9c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:49.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.802+0000 7f0ba0ff9700 1 -- 172.21.15.2:0/2553030506 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c0092e0 con 0x7f0ba41084b0 2024-05-14T16:51:49.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.802+0000 7f0baa215700 1 -- 172.21.15.2:0/2553030506 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ba41a10e0 con 0x7f0ba41084b0 2024-05-14T16:51:49.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.802+0000 7f0ba0ff9700 1 -- 172.21.15.2:0/2553030506 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b9c004bb0 con 0x7f0ba41084b0 2024-05-14T16:51:49.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.802+0000 7f0ba0ff9700 1 -- 172.21.15.2:0/2553030506 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c004020 con 0x7f0ba41084b0 2024-05-14T16:51:49.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.802+0000 7f0baa215700 1 -- 172.21.15.2:0/2553030506 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ba41a89c0 con 0x7f0ba41084b0 2024-05-14T16:51:49.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.803+0000 7f0baa215700 1 -- 172.21.15.2:0/2553030506 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ba4103b60 con 0x7f0ba41084b0 2024-05-14T16:51:49.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.805+0000 7f0ba0ff9700 1 -- 172.21.15.2:0/2553030506 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f0b9c01b030 con 0x7f0ba41084b0 2024-05-14T16:51:49.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.805+0000 7f0ba0ff9700 1 --2- 172.21.15.2:0/2553030506 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f0b8c0b2880 0x7f0b8c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:49.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.805+0000 7f0ba37fe700 1 --2- 172.21.15.2:0/2553030506 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f0b8c0b2880 0x7f0b8c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:49.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.805+0000 7f0ba0ff9700 1 -- 172.21.15.2:0/2553030506 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f0b9c0e2500 con 0x7f0ba41084b0 2024-05-14T16:51:49.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.806+0000 7f0ba37fe700 1 --2- 172.21.15.2:0/2553030506 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f0b8c0b2880 0x7f0b8c0b4d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0b94005950 tx=0x7f0b9400b500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:49.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:49.809+0000 7f0ba0ff9700 1 -- 172.21.15.2:0/2553030506 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0b9c022020 con 0x7f0ba41084b0 2024-05-14T16:51:50.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:50.070+0000 7f0baa215700 1 -- 172.21.15.2:0/2553030506 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f0ba419d1d0 con 0x7f0ba41084b0 2024-05-14T16:51:50.070 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:49 smithi002 ceph-mon[31538]: mds.? [v2:172.21.15.123:6834/1198670013,v1:172.21.15.123:6835/1198670013] up:active 2024-05-14T16:51:50.070 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:49 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-05-14T16:51:50.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:49 smithi151 ceph-mon[36066]: mds.? [v2:172.21.15.123:6834/1198670013,v1:172.21.15.123:6835/1198670013] up:active 2024-05-14T16:51:50.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:49 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-05-14T16:51:50.409 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:49 smithi123 ceph-mon[35971]: mds.? [v2:172.21.15.123:6834/1198670013,v1:172.21.15.123:6835/1198670013] up:active 2024-05-14T16:51:50.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:49 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-05-14T16:51:50.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:50.993+0000 7f0ba0ff9700 1 -- 172.21.15.2:0/2553030506 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f0ba419d1d0 con 0x7f0ba41084b0 2024-05-14T16:51:50.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:50.997+0000 7f0baa215700 1 -- 172.21.15.2:0/2553030506 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f0b8c0b2880 msgr2=0x7f0b8c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:50.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:50.997+0000 7f0baa215700 1 --2- 172.21.15.2:0/2553030506 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f0b8c0b2880 0x7f0b8c0b4d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0b94005950 tx=0x7f0b9400b500 comp rx=0 tx=0).stop 2024-05-14T16:51:50.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:50.997+0000 7f0baa215700 1 -- 172.21.15.2:0/2553030506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ba41084b0 msgr2=0x7f0ba41a0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:50.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:50.997+0000 7f0baa215700 1 --2- 172.21.15.2:0/2553030506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ba41084b0 0x7f0ba41a0710 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c00efc0 tx=0x7f0b9c00c3a0 comp rx=0 tx=0).stop 2024-05-14T16:51:50.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:50.997+0000 7f0baa215700 1 -- 172.21.15.2:0/2553030506 shutdown_connections 2024-05-14T16:51:50.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:50.997+0000 7f0baa215700 1 --2- 172.21.15.2:0/2553030506 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f0b8c0b2880 0x7f0b8c0b4d40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:50.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:50.997+0000 7f0baa215700 1 --2- 172.21.15.2:0/2553030506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0ba41084b0 0x7f0ba41a0710 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:50.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:50.997+0000 7f0baa215700 1 --2- 172.21.15.2:0/2553030506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ba4102470 0x7f0ba419be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:50.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:50.997+0000 7f0baa215700 1 --2- 172.21.15.2:0/2553030506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ba4102e30 0x7f0ba419c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:50.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:50.997+0000 7f0baa215700 1 -- 172.21.15.2:0/2553030506 >> 172.21.15.2:0/2553030506 conn(0x7f0ba40793f0 msgr2=0x7f0ba406bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:50.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:50.997+0000 7f0baa215700 1 -- 172.21.15.2:0/2553030506 shutdown_connections 2024-05-14T16:51:50.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:50.997+0000 7f0baa215700 1 -- 172.21.15.2:0/2553030506 wait complete. 2024-05-14T16:51:51.324 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:50 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/2553030506' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-14T16:51:51.324 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:50 smithi002 ceph-mon[31538]: pgmap v297: 97 pgs: 97 active+clean; 580 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 4 op/s 2024-05-14T16:51:51.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:50 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/2553030506' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-14T16:51:51.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:50 smithi151 ceph-mon[36066]: pgmap v297: 97 pgs: 97 active+clean; 580 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 4 op/s 2024-05-14T16:51:51.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:50 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/2553030506' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-14T16:51:51.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:50 smithi123 ceph-mon[35971]: pgmap v297: 97 pgs: 97 active+clean; 580 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 4 op/s 2024-05-14T16:51:51.741 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-14T16:51:52.000 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:51:52.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:51 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/2553030506' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-14T16:51:52.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:51 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-05-14T16:51:52.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:51 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:51:52.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:51 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/2553030506' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-14T16:51:52.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:51 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-05-14T16:51:52.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:51 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:51:52.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:51 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/2553030506' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-14T16:51:52.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:51 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-05-14T16:51:52.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:51 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:51:53.016 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:53 smithi002 ceph-mon[31538]: pgmap v298: 97 pgs: 97 active+clean; 580 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 1.7 KiB/s wr, 5 op/s 2024-05-14T16:51:53.256 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:53 smithi151 ceph-mon[36066]: pgmap v298: 97 pgs: 97 active+clean; 580 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 1.7 KiB/s wr, 5 op/s 2024-05-14T16:51:53.311 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:53 smithi123 ceph-mon[35971]: pgmap v298: 97 pgs: 97 active+clean; 580 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 1.7 KiB/s wr, 5 op/s 2024-05-14T16:51:53.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.599+0000 7f48f03f1700 1 -- 172.21.15.2:0/2192262709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48e8108520 msgr2=0x7f48e8108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:53.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.599+0000 7f48f03f1700 1 --2- 172.21.15.2:0/2192262709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48e8108520 0x7f48e8108900 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f48e400b0a0 tx=0x7f48e400b3b0 comp rx=0 tx=0).stop 2024-05-14T16:51:53.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.599+0000 7f48f03f1700 1 -- 172.21.15.2:0/2192262709 shutdown_connections 2024-05-14T16:51:53.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.599+0000 7f48f03f1700 1 --2- 172.21.15.2:0/2192262709 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48e8102e00 0x7f48e810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:53.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.599+0000 7f48f03f1700 1 --2- 172.21.15.2:0/2192262709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48e8108520 0x7f48e8108900 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:53.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.599+0000 7f48f03f1700 1 --2- 172.21.15.2:0/2192262709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48e8102440 0x7f48e81028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:53.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.599+0000 7f48f03f1700 1 -- 172.21.15.2:0/2192262709 >> 172.21.15.2:0/2192262709 conn(0x7f48e80793f0 msgr2=0x7f48e8079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:53.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.599+0000 7f48f03f1700 1 -- 172.21.15.2:0/2192262709 shutdown_connections 2024-05-14T16:51:53.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.599+0000 7f48f03f1700 1 -- 172.21.15.2:0/2192262709 wait complete. 2024-05-14T16:51:53.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.600+0000 7f48f03f1700 1 Processor -- start 2024-05-14T16:51:53.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.600+0000 7f48f03f1700 1 -- start start 2024-05-14T16:51:53.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.600+0000 7f48f03f1700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48e8102440 0x7f48e819aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:53.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.600+0000 7f48f03f1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48e8102e00 0x7f48e819af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:53.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.600+0000 7f48f03f1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48e8108520 0x7f48e81a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:53.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.601+0000 7f48f03f1700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f48e81125d0 con 0x7f48e8102e00 2024-05-14T16:51:53.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.601+0000 7f48f03f1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f48e8112450 con 0x7f48e8108520 2024-05-14T16:51:53.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.601+0000 7f48f03f1700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f48e8112750 con 0x7f48e8102440 2024-05-14T16:51:53.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.601+0000 7f48ed98c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48e8102e00 0x7f48e819af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:53.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.601+0000 7f48ed98c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48e8102e00 0x7f48e819af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58288/0 (socket says 172.21.15.2:58288) 2024-05-14T16:51:53.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.601+0000 7f48ed98c700 1 -- 172.21.15.2:0/603151102 learned_addr learned my addr 172.21.15.2:0/603151102 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:51:53.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.601+0000 7f48ee98e700 1 --2- 172.21.15.2:0/603151102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48e8108520 0x7f48e81a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:53.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.601+0000 7f48ee18d700 1 --2- 172.21.15.2:0/603151102 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48e8102440 0x7f48e819aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:53.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.601+0000 7f48ed98c700 1 -- 172.21.15.2:0/603151102 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48e8102440 msgr2=0x7f48e819aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:53.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.601+0000 7f48ed98c700 1 --2- 172.21.15.2:0/603151102 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48e8102440 0x7f48e819aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:53.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.601+0000 7f48ed98c700 1 -- 172.21.15.2:0/603151102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48e8108520 msgr2=0x7f48e81a5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:53.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.601+0000 7f48ed98c700 1 --2- 172.21.15.2:0/603151102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48e8108520 0x7f48e81a5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:53.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.601+0000 7f48ed98c700 1 -- 172.21.15.2:0/603151102 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48e81a6010 con 0x7f48e8102e00 2024-05-14T16:51:53.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.601+0000 7f48ed98c700 1 --2- 172.21.15.2:0/603151102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48e8102e00 0x7f48e819af90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f48d800c920 tx=0x7f48d800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:53.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.601+0000 7f48df7fe700 1 -- 172.21.15.2:0/603151102 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d8004a30 con 0x7f48e8102e00 2024-05-14T16:51:53.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.601+0000 7f48df7fe700 1 -- 172.21.15.2:0/603151102 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48d80054d0 con 0x7f48e8102e00 2024-05-14T16:51:53.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.601+0000 7f48f03f1700 1 -- 172.21.15.2:0/603151102 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48e81a6240 con 0x7f48e8102e00 2024-05-14T16:51:53.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.602+0000 7f48df7fe700 1 -- 172.21.15.2:0/603151102 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d8007c50 con 0x7f48e8102e00 2024-05-14T16:51:53.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.602+0000 7f48f03f1700 1 -- 172.21.15.2:0/603151102 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48e81a67d0 con 0x7f48e8102e00 2024-05-14T16:51:53.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.602+0000 7f48f03f1700 1 -- 172.21.15.2:0/603151102 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48e810a9b0 con 0x7f48e8102e00 2024-05-14T16:51:53.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.604+0000 7f48df7fe700 1 -- 172.21.15.2:0/603151102 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f48d8004c60 con 0x7f48e8102e00 2024-05-14T16:51:53.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.605+0000 7f48df7fe700 1 --2- 172.21.15.2:0/603151102 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f48d40b2900 0x7f48d40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:53.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.605+0000 7f48ee18d700 1 --2- 172.21.15.2:0/603151102 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f48d40b2900 0x7f48d40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:53.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.605+0000 7f48df7fe700 1 -- 172.21.15.2:0/603151102 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f48d80e1f20 con 0x7f48e8102e00 2024-05-14T16:51:53.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.605+0000 7f48ee18d700 1 --2- 172.21.15.2:0/603151102 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f48d40b2900 0x7f48d40b4dc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f48e4009e60 tx=0x7f48e400b7b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:53.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.608+0000 7f48df7fe700 1 -- 172.21.15.2:0/603151102 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f48d8014030 con 0x7f48e8102e00 2024-05-14T16:51:53.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:53.876+0000 7f48f03f1700 1 -- 172.21.15.2:0/603151102 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f48e806a230 con 0x7f48e8102e00 2024-05-14T16:51:54.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:54.007+0000 7f48df7fe700 1 -- 172.21.15.2:0/603151102 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v24) ==== 115+0+0 (secure 0 0 0) 0x7f48d80a9d70 con 0x7f48e8102e00 2024-05-14T16:51:54.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:54.009+0000 7f48f03f1700 1 -- 172.21.15.2:0/603151102 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f48d40b2900 msgr2=0x7f48d40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:54.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:54.009+0000 7f48f03f1700 1 --2- 172.21.15.2:0/603151102 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f48d40b2900 0x7f48d40b4dc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f48e4009e60 tx=0x7f48e400b7b0 comp rx=0 tx=0).stop 2024-05-14T16:51:54.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:54.009+0000 7f48f03f1700 1 -- 172.21.15.2:0/603151102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48e8102e00 msgr2=0x7f48e819af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:54.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:54.009+0000 7f48f03f1700 1 --2- 172.21.15.2:0/603151102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48e8102e00 0x7f48e819af90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f48d800c920 tx=0x7f48d800cc30 comp rx=0 tx=0).stop 2024-05-14T16:51:54.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:54.009+0000 7f48f03f1700 1 -- 172.21.15.2:0/603151102 shutdown_connections 2024-05-14T16:51:54.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:54.009+0000 7f48f03f1700 1 --2- 172.21.15.2:0/603151102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48e8108520 0x7f48e81a5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:54.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:54.009+0000 7f48f03f1700 1 --2- 172.21.15.2:0/603151102 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48e8102440 0x7f48e819aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:54.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:54.009+0000 7f48f03f1700 1 --2- 172.21.15.2:0/603151102 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f48d40b2900 0x7f48d40b4dc0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:54.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:54.009+0000 7f48f03f1700 1 --2- 172.21.15.2:0/603151102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48e8102e00 0x7f48e819af90 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:54.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:54.009+0000 7f48f03f1700 1 -- 172.21.15.2:0/603151102 >> 172.21.15.2:0/603151102 conn(0x7f48e80793f0 msgr2=0x7f48e81012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:54.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:54.010+0000 7f48f03f1700 1 -- 172.21.15.2:0/603151102 shutdown_connections 2024-05-14T16:51:54.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:54.010+0000 7f48f03f1700 1 -- 172.21.15.2:0/603151102 wait complete. 2024-05-14T16:51:54.243 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:54 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/603151102' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-14T16:51:54.272 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:54 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/603151102' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-14T16:51:54.409 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:54 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/603151102' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-14T16:51:54.823 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-05-14T16:51:54.823 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-14T16:51:55.079 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:51:55.202 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:55 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/603151102' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-14T16:51:55.203 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:55 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:51:55.203 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:55 smithi002 ceph-mon[31538]: pgmap v299: 97 pgs: 97 active+clean; 582 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-14T16:51:55.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:55 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/603151102' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-14T16:51:55.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:55 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:51:55.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:55 smithi151 ceph-mon[36066]: pgmap v299: 97 pgs: 97 active+clean; 582 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-14T16:51:55.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:55 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/603151102' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-14T16:51:55.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:55 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:51:55.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:55 smithi123 ceph-mon[35971]: pgmap v299: 97 pgs: 97 active+clean; 582 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-14T16:51:56.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.725+0000 7fb20bd24700 1 -- 172.21.15.2:0/1855183509 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2041037f0 msgr2=0x7fb204103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:56.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.725+0000 7fb20bd24700 1 --2- 172.21.15.2:0/1855183509 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2041037f0 0x7fb204103bd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb20000b0d0 tx=0x7fb20000b3e0 comp rx=0 tx=0).stop 2024-05-14T16:51:56.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.725+0000 7fb20bd24700 1 -- 172.21.15.2:0/1855183509 shutdown_connections 2024-05-14T16:51:56.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.725+0000 7fb20bd24700 1 --2- 172.21.15.2:0/1855183509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2041088f0 0x7fb204110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:56.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.725+0000 7fb20bd24700 1 --2- 172.21.15.2:0/1855183509 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2041037f0 0x7fb204103bd0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:56.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.725+0000 7fb20bd24700 1 --2- 172.21.15.2:0/1855183509 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2041041a0 0x7fb2041081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:56.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.725+0000 7fb20bd24700 1 -- 172.21.15.2:0/1855183509 >> 172.21.15.2:0/1855183509 conn(0x7fb20407a6a0 msgr2=0x7fb20407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:56.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.726+0000 7fb20bd24700 1 -- 172.21.15.2:0/1855183509 shutdown_connections 2024-05-14T16:51:56.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.726+0000 7fb20bd24700 1 -- 172.21.15.2:0/1855183509 wait complete. 2024-05-14T16:51:56.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.726+0000 7fb20bd24700 1 Processor -- start 2024-05-14T16:51:56.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.726+0000 7fb20bd24700 1 -- start start 2024-05-14T16:51:56.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.727+0000 7fb20bd24700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2041037f0 0x7fb20419f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:56.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.727+0000 7fb20bd24700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2041041a0 0x7fb20419fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:56.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.727+0000 7fb20bd24700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2041088f0 0x7fb204199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:56.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.727+0000 7fb20bd24700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb2041136e0 con 0x7fb2041088f0 2024-05-14T16:51:56.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.727+0000 7fb20bd24700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb204113560 con 0x7fb2041037f0 2024-05-14T16:51:56.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.727+0000 7fb20bd24700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb204113860 con 0x7fb2041041a0 2024-05-14T16:51:56.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.727+0000 7fb20a2c1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2041088f0 0x7fb204199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:56.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.727+0000 7fb20a2c1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2041088f0 0x7fb204199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34676/0 (socket says 172.21.15.2:34676) 2024-05-14T16:51:56.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.727+0000 7fb20a2c1700 1 -- 172.21.15.2:0/3205772409 learned_addr learned my addr 172.21.15.2:0/3205772409 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:51:56.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.727+0000 7fb2092bf700 1 --2- 172.21.15.2:0/3205772409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2041041a0 0x7fb20419fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:56.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.727+0000 7fb209ac0700 1 --2- 172.21.15.2:0/3205772409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2041037f0 0x7fb20419f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:56.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.727+0000 7fb20a2c1700 1 -- 172.21.15.2:0/3205772409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2041041a0 msgr2=0x7fb20419fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:56.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.727+0000 7fb20a2c1700 1 --2- 172.21.15.2:0/3205772409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2041041a0 0x7fb20419fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:56.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.727+0000 7fb20a2c1700 1 -- 172.21.15.2:0/3205772409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2041037f0 msgr2=0x7fb20419f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:56.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.727+0000 7fb20a2c1700 1 --2- 172.21.15.2:0/3205772409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2041037f0 0x7fb20419f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:56.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.727+0000 7fb20a2c1700 1 -- 172.21.15.2:0/3205772409 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb20419a180 con 0x7fb2041088f0 2024-05-14T16:51:56.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.727+0000 7fb209ac0700 1 --2- 172.21.15.2:0/3205772409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2041037f0 0x7fb20419f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:51:56.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.727+0000 7fb20a2c1700 1 --2- 172.21.15.2:0/3205772409 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2041088f0 0x7fb204199a20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc00c970 tx=0x7fb1fc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:56.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.728+0000 7fb1faffd700 1 -- 172.21.15.2:0/3205772409 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc00ce60 con 0x7fb2041088f0 2024-05-14T16:51:56.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.728+0000 7fb1faffd700 1 -- 172.21.15.2:0/3205772409 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1fc004830 con 0x7fb2041088f0 2024-05-14T16:51:56.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.728+0000 7fb20bd24700 1 -- 172.21.15.2:0/3205772409 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb20419a470 con 0x7fb2041088f0 2024-05-14T16:51:56.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.728+0000 7fb1faffd700 1 -- 172.21.15.2:0/3205772409 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc0054c0 con 0x7fb2041088f0 2024-05-14T16:51:56.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.728+0000 7fb20bd24700 1 -- 172.21.15.2:0/3205772409 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2041a6730 con 0x7fb2041088f0 2024-05-14T16:51:56.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.729+0000 7fb20bd24700 1 -- 172.21.15.2:0/3205772409 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb20410bb30 con 0x7fb2041088f0 2024-05-14T16:51:56.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.730+0000 7fb1faffd700 1 -- 172.21.15.2:0/3205772409 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fb1fc0049e0 con 0x7fb2041088f0 2024-05-14T16:51:56.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.731+0000 7fb1faffd700 1 --2- 172.21.15.2:0/3205772409 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fb1f00b2900 0x7fb1f00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:56.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.731+0000 7fb1faffd700 1 -- 172.21.15.2:0/3205772409 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7fb1fc0e1ee0 con 0x7fb2041088f0 2024-05-14T16:51:56.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.731+0000 7fb209ac0700 1 --2- 172.21.15.2:0/3205772409 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fb1f00b2900 0x7fb1f00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:56.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.731+0000 7fb209ac0700 1 --2- 172.21.15.2:0/3205772409 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fb1f00b2900 0x7fb1f00b4dc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb200009e60 tx=0x7fb20000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:56.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.734+0000 7fb1faffd700 1 -- 172.21.15.2:0/3205772409 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb1fc014030 con 0x7fb2041088f0 2024-05-14T16:51:56.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:56.986+0000 7fb20bd24700 1 -- 172.21.15.2:0/3205772409 --> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fb204063980 con 0x7fb1f00b2900 2024-05-14T16:51:57.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:57.011+0000 7fb1faffd700 1 -- 172.21.15.2:0/3205772409 <== mgr.14150 v2:172.21.15.2:6800/1498936028 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb204063980 con 0x7fb1f00b2900 2024-05-14T16:51:57.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:57.014+0000 7fb20bd24700 1 -- 172.21.15.2:0/3205772409 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fb1f00b2900 msgr2=0x7fb1f00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:57.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:57.014+0000 7fb20bd24700 1 --2- 172.21.15.2:0/3205772409 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fb1f00b2900 0x7fb1f00b4dc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb200009e60 tx=0x7fb20000b040 comp rx=0 tx=0).stop 2024-05-14T16:51:57.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:57.014+0000 7fb20bd24700 1 -- 172.21.15.2:0/3205772409 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2041088f0 msgr2=0x7fb204199a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:57.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:57.014+0000 7fb20bd24700 1 --2- 172.21.15.2:0/3205772409 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2041088f0 0x7fb204199a20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc00c970 tx=0x7fb1fc00cc80 comp rx=0 tx=0).stop 2024-05-14T16:51:57.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:57.014+0000 7fb20bd24700 1 -- 172.21.15.2:0/3205772409 shutdown_connections 2024-05-14T16:51:57.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:57.014+0000 7fb20bd24700 1 --2- 172.21.15.2:0/3205772409 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fb1f00b2900 0x7fb1f00b4dc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:57.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:57.014+0000 7fb20bd24700 1 --2- 172.21.15.2:0/3205772409 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2041088f0 0x7fb204199a20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:57.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:57.014+0000 7fb20bd24700 1 --2- 172.21.15.2:0/3205772409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2041037f0 0x7fb20419f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:57.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:57.014+0000 7fb20bd24700 1 --2- 172.21.15.2:0/3205772409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2041041a0 0x7fb20419fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:57.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:57.014+0000 7fb20bd24700 1 -- 172.21.15.2:0/3205772409 >> 172.21.15.2:0/3205772409 conn(0x7fb20407a6a0 msgr2=0x7fb204109a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:57.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:57.014+0000 7fb20bd24700 1 -- 172.21.15.2:0/3205772409 shutdown_connections 2024-05-14T16:51:57.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:57.014+0000 7fb20bd24700 1 -- 172.21.15.2:0/3205772409 wait complete. 2024-05-14T16:51:57.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:57 smithi002 ceph-mon[31538]: pgmap v300: 97 pgs: 97 active+clean; 582 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-14T16:51:57.244 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:51:56 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:51:56.991+0000 7f3759277700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T16:51:57.244 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:51:56 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:51:56.991+0000 7f3759277700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T16:51:57.244 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:51:56 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:51:56.991+0000 7f3759277700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T16:51:57.244 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:51:56 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:51:56.991+0000 7f3759277700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T16:51:57.244 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:51:56 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:51:56.991+0000 7f3759277700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T16:51:57.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:57 smithi123 ceph-mon[35971]: pgmap v300: 97 pgs: 97 active+clean; 582 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-14T16:51:57.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:57 smithi151 ceph-mon[36066]: pgmap v300: 97 pgs: 97 active+clean; 582 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-14T16:51:57.785 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-14T16:51:58.040 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:51:58.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:58 smithi123 ceph-mon[35971]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:51:58.484 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:58 smithi002 ceph-mon[31538]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:51:58.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:58 smithi151 ceph-mon[36066]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:51:59.448 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:51:59 smithi002 ceph-mon[31538]: pgmap v301: 97 pgs: 97 active+clean; 582 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-05-14T16:51:59.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:51:59 smithi151 ceph-mon[36066]: pgmap v301: 97 pgs: 97 active+clean; 582 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-05-14T16:51:59.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:51:59 smithi123 ceph-mon[35971]: pgmap v301: 97 pgs: 97 active+clean; 582 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-05-14T16:51:59.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:59.802+0000 7fa723d04700 1 -- 172.21.15.2:0/400095134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa71c103f90 msgr2=0x7fa71c110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:59.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:59.802+0000 7fa723d04700 1 --2- 172.21.15.2:0/400095134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa71c103f90 0x7fa71c110470 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa71400b0a0 tx=0x7fa71400b3b0 comp rx=0 tx=0).stop 2024-05-14T16:51:59.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:59.802+0000 7fa723d04700 1 -- 172.21.15.2:0/400095134 shutdown_connections 2024-05-14T16:51:59.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:59.802+0000 7fa723d04700 1 --2- 172.21.15.2:0/400095134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa71c103f90 0x7fa71c110470 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:59.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:59.802+0000 7fa723d04700 1 --2- 172.21.15.2:0/400095134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa71c108500 0x7fa71c1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:59.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:59.802+0000 7fa723d04700 1 --2- 172.21.15.2:0/400095134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa71c1035f0 0x7fa71c103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:59.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:59.802+0000 7fa723d04700 1 -- 172.21.15.2:0/400095134 >> 172.21.15.2:0/400095134 conn(0x7fa71c07a6a0 msgr2=0x7fa71c07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:51:59.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:59.802+0000 7fa723d04700 1 -- 172.21.15.2:0/400095134 shutdown_connections 2024-05-14T16:51:59.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:59.802+0000 7fa723d04700 1 -- 172.21.15.2:0/400095134 wait complete. 2024-05-14T16:51:59.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:59.803+0000 7fa723d04700 1 Processor -- start 2024-05-14T16:51:59.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:59.803+0000 7fa723d04700 1 -- start start 2024-05-14T16:51:59.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:59.803+0000 7fa723d04700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa71c1035f0 0x7fa71c19f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:59.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:59.803+0000 7fa723d04700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa71c103f90 0x7fa71c19fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:59.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:59.803+0000 7fa723d04700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa71c108500 0x7fa71c1999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:59.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:59.803+0000 7fa723d04700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa71c1137e0 con 0x7fa71c108500 2024-05-14T16:51:59.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:59.803+0000 7fa723d04700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa71c113660 con 0x7fa71c1035f0 2024-05-14T16:51:59.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:59.803+0000 7fa723d04700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa71c113960 con 0x7fa71c103f90 2024-05-14T16:51:59.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:59.803+0000 7fa7222a1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa71c108500 0x7fa71c1999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:59.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:59.803+0000 7fa7222a1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa71c108500 0x7fa71c1999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34702/0 (socket says 172.21.15.2:34702) 2024-05-14T16:51:59.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:59.804+0000 7fa7222a1700 1 -- 172.21.15.2:0/1791617901 learned_addr learned my addr 172.21.15.2:0/1791617901 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:51:59.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:59.804+0000 7fa72129f700 1 --2- 172.21.15.2:0/1791617901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa71c103f90 0x7fa71c19fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:59.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:59.804+0000 7fa721aa0700 1 --2- 172.21.15.2:0/1791617901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa71c1035f0 0x7fa71c19f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:59.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:59.804+0000 7fa7222a1700 1 -- 172.21.15.2:0/1791617901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa71c103f90 msgr2=0x7fa71c19fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:59.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:59.804+0000 7fa7222a1700 1 --2- 172.21.15.2:0/1791617901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa71c103f90 0x7fa71c19fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:59.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:59.804+0000 7fa7222a1700 1 -- 172.21.15.2:0/1791617901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa71c1035f0 msgr2=0x7fa71c19f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:51:59.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:59.804+0000 7fa7222a1700 1 --2- 172.21.15.2:0/1791617901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa71c1035f0 0x7fa71c19f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:51:59.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:59.804+0000 7fa7222a1700 1 -- 172.21.15.2:0/1791617901 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa71c19a100 con 0x7fa71c108500 2024-05-14T16:51:59.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:59.804+0000 7fa7222a1700 1 --2- 172.21.15.2:0/1791617901 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa71c108500 0x7fa71c1999a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa71400b830 tx=0x7fa714004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:59.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:59.804+0000 7fa712ffd700 1 -- 172.21.15.2:0/1791617901 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714021070 con 0x7fa71c108500 2024-05-14T16:51:59.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:59.804+0000 7fa712ffd700 1 -- 172.21.15.2:0/1791617901 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa714007e30 con 0x7fa71c108500 2024-05-14T16:51:59.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:59.804+0000 7fa723d04700 1 -- 172.21.15.2:0/1791617901 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa71c19a390 con 0x7fa71c108500 2024-05-14T16:51:59.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:59.804+0000 7fa712ffd700 1 -- 172.21.15.2:0/1791617901 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7140045f0 con 0x7fa71c108500 2024-05-14T16:51:59.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:59.804+0000 7fa723d04700 1 -- 172.21.15.2:0/1791617901 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa71c1a66b0 con 0x7fa71c108500 2024-05-14T16:51:59.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:59.805+0000 7fa723d04700 1 -- 172.21.15.2:0/1791617901 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa71c06a230 con 0x7fa71c108500 2024-05-14T16:51:59.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:59.807+0000 7fa712ffd700 1 -- 172.21.15.2:0/1791617901 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fa714003c00 con 0x7fa71c108500 2024-05-14T16:51:59.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:59.807+0000 7fa712ffd700 1 --2- 172.21.15.2:0/1791617901 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fa7080b2990 0x7fa7080b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:51:59.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:59.807+0000 7fa721aa0700 1 --2- 172.21.15.2:0/1791617901 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fa7080b2990 0x7fa7080b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:51:59.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:59.807+0000 7fa712ffd700 1 -- 172.21.15.2:0/1791617901 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7fa71400f020 con 0x7fa71c108500 2024-05-14T16:51:59.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:59.808+0000 7fa721aa0700 1 --2- 172.21.15.2:0/1791617901 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fa7080b2990 0x7fa7080b4e50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa71c19aae0 tx=0x7fa71800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:51:59.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:51:59.811+0000 7fa712ffd700 1 -- 172.21.15.2:0/1791617901 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa71402a020 con 0x7fa71c108500 2024-05-14T16:52:00.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:00.056+0000 7fa723d04700 1 -- 172.21.15.2:0/1791617901 --> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fa71c063980 con 0x7fa7080b2990 2024-05-14T16:52:00.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:00.088+0000 7fa712ffd700 1 -- 172.21.15.2:0/1791617901 <== mgr.14150 v2:172.21.15.2:6800/1498936028 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa71c063980 con 0x7fa7080b2990 2024-05-14T16:52:00.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:00.090+0000 7fa723d04700 1 -- 172.21.15.2:0/1791617901 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fa7080b2990 msgr2=0x7fa7080b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:00.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:00.090+0000 7fa723d04700 1 --2- 172.21.15.2:0/1791617901 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fa7080b2990 0x7fa7080b4e50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa71c19aae0 tx=0x7fa71800b410 comp rx=0 tx=0).stop 2024-05-14T16:52:00.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:00.090+0000 7fa723d04700 1 -- 172.21.15.2:0/1791617901 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa71c108500 msgr2=0x7fa71c1999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:00.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:00.090+0000 7fa723d04700 1 --2- 172.21.15.2:0/1791617901 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa71c108500 0x7fa71c1999a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa71400b830 tx=0x7fa714004ce0 comp rx=0 tx=0).stop 2024-05-14T16:52:00.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:00.090+0000 7fa723d04700 1 -- 172.21.15.2:0/1791617901 shutdown_connections 2024-05-14T16:52:00.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:00.090+0000 7fa723d04700 1 --2- 172.21.15.2:0/1791617901 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fa7080b2990 0x7fa7080b4e50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:00.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:00.090+0000 7fa723d04700 1 --2- 172.21.15.2:0/1791617901 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa71c108500 0x7fa71c1999a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:00.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:00.090+0000 7fa723d04700 1 --2- 172.21.15.2:0/1791617901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa71c1035f0 0x7fa71c19f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:00.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:00.090+0000 7fa723d04700 1 --2- 172.21.15.2:0/1791617901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa71c103f90 0x7fa71c19fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:00.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:00.090+0000 7fa723d04700 1 -- 172.21.15.2:0/1791617901 >> 172.21.15.2:0/1791617901 conn(0x7fa71c07a6a0 msgr2=0x7fa71c101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:00.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:00.090+0000 7fa723d04700 1 -- 172.21.15.2:0/1791617901 shutdown_connections 2024-05-14T16:52:00.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:00.090+0000 7fa723d04700 1 -- 172.21.15.2:0/1791617901 wait complete. 2024-05-14T16:52:00.466 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:00 smithi002 ceph-mon[31538]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-14T16:52:00.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:00 smithi151 ceph-mon[36066]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-14T16:52:00.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:00 smithi123 ceph-mon[35971]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-14T16:52:00.854 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-14T16:52:01.111 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:52:01.480 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:01 smithi002 ceph-mon[31538]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:52:01.480 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:01 smithi002 ceph-mon[31538]: pgmap v302: 97 pgs: 97 active+clean; 588 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-05-14T16:52:01.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:01 smithi151 ceph-mon[36066]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:52:01.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:01 smithi151 ceph-mon[36066]: pgmap v302: 97 pgs: 97 active+clean; 588 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-05-14T16:52:01.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:01 smithi123 ceph-mon[35971]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:52:01.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:01 smithi123 ceph-mon[35971]: pgmap v302: 97 pgs: 97 active+clean; 588 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-05-14T16:52:02.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:02.995+0000 7f25bddcd700 1 -- 172.21.15.2:0/3382107682 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25b8103fa0 msgr2=0x7f25b8110450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:02.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:02.995+0000 7f25bddcd700 1 --2- 172.21.15.2:0/3382107682 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25b8103fa0 0x7f25b8110450 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f25b00099c0 tx=0x7f25b0009cd0 comp rx=0 tx=0).stop 2024-05-14T16:52:02.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:02.995+0000 7f25bddcd700 1 -- 172.21.15.2:0/3382107682 shutdown_connections 2024-05-14T16:52:02.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:02.995+0000 7f25bddcd700 1 --2- 172.21.15.2:0/3382107682 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25b8103fa0 0x7f25b8110450 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:02.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:02.995+0000 7f25bddcd700 1 --2- 172.21.15.2:0/3382107682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25b8109680 0x7f25b8109a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:02.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:02.995+0000 7f25bddcd700 1 --2- 172.21.15.2:0/3382107682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25b81035e0 0x7f25b8103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:02.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:02.995+0000 7f25bddcd700 1 -- 172.21.15.2:0/3382107682 >> 172.21.15.2:0/3382107682 conn(0x7f25b807a6f0 msgr2=0x7f25b807ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:02.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:02.995+0000 7f25bddcd700 1 -- 172.21.15.2:0/3382107682 shutdown_connections 2024-05-14T16:52:02.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:02.995+0000 7f25bddcd700 1 -- 172.21.15.2:0/3382107682 wait complete. 2024-05-14T16:52:02.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:02.996+0000 7f25bddcd700 1 Processor -- start 2024-05-14T16:52:02.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:02.996+0000 7f25bddcd700 1 -- start start 2024-05-14T16:52:02.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:02.996+0000 7f25bddcd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25b81035e0 0x7f25b819f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:02.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:02.996+0000 7f25bddcd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25b8103fa0 0x7f25b819fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:02.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:02.996+0000 7f25bddcd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25b8109680 0x7f25b81999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:02.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:02.996+0000 7f25bddcd700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f25b81137e0 con 0x7f25b8103fa0 2024-05-14T16:52:02.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:02.996+0000 7f25bddcd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f25b8113660 con 0x7f25b81035e0 2024-05-14T16:52:02.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:02.996+0000 7f25bddcd700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f25b8113960 con 0x7f25b8109680 2024-05-14T16:52:02.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:02.996+0000 7f25b6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25b8103fa0 0x7f25b819fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:02.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:02.996+0000 7f25b6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25b8103fa0 0x7f25b819fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34716/0 (socket says 172.21.15.2:34716) 2024-05-14T16:52:02.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:02.996+0000 7f25b6ffd700 1 -- 172.21.15.2:0/1093446524 learned_addr learned my addr 172.21.15.2:0/1093446524 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:52:02.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:02.996+0000 7f25b77fe700 1 --2- 172.21.15.2:0/1093446524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25b81035e0 0x7f25b819f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:02.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:02.997+0000 7f25b6ffd700 1 -- 172.21.15.2:0/1093446524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25b8109680 msgr2=0x7f25b81999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:02.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:02.997+0000 7f25b7fff700 1 --2- 172.21.15.2:0/1093446524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25b8109680 0x7f25b81999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:02.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:02.997+0000 7f25b6ffd700 1 --2- 172.21.15.2:0/1093446524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25b8109680 0x7f25b81999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:02.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:02.997+0000 7f25b6ffd700 1 -- 172.21.15.2:0/1093446524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25b81035e0 msgr2=0x7f25b819f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:02.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:02.997+0000 7f25b6ffd700 1 --2- 172.21.15.2:0/1093446524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25b81035e0 0x7f25b819f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:02.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:02.997+0000 7f25b6ffd700 1 -- 172.21.15.2:0/1093446524 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b819a120 con 0x7f25b8103fa0 2024-05-14T16:52:02.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:02.997+0000 7f25b7fff700 1 --2- 172.21.15.2:0/1093446524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25b8109680 0x7f25b81999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:52:02.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:02.997+0000 7f25b6ffd700 1 --2- 172.21.15.2:0/1093446524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25b8103fa0 0x7f25b819fd40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f25a000d980 tx=0x7f25a000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:02.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:02.997+0000 7f25b4ff9700 1 -- 172.21.15.2:0/1093446524 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a0017070 con 0x7f25b8103fa0 2024-05-14T16:52:02.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:02.997+0000 7f25bddcd700 1 -- 172.21.15.2:0/1093446524 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25b819a410 con 0x7f25b8103fa0 2024-05-14T16:52:02.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:02.997+0000 7f25b4ff9700 1 -- 172.21.15.2:0/1093446524 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25a0005020 con 0x7f25b8103fa0 2024-05-14T16:52:02.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:02.997+0000 7f25b4ff9700 1 -- 172.21.15.2:0/1093446524 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a0009b50 con 0x7f25b8103fa0 2024-05-14T16:52:02.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:02.997+0000 7f25bddcd700 1 -- 172.21.15.2:0/1093446524 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25b81a66d0 con 0x7f25b8103fa0 2024-05-14T16:52:03.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:02.998+0000 7f25bddcd700 1 -- 172.21.15.2:0/1093446524 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25b8104cf0 con 0x7f25b8103fa0 2024-05-14T16:52:03.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:03.000+0000 7f25b4ff9700 1 -- 172.21.15.2:0/1093446524 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f25a00051d0 con 0x7f25b8103fa0 2024-05-14T16:52:03.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:03.000+0000 7f25b4ff9700 1 --2- 172.21.15.2:0/1093446524 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f25a40b2910 0x7f25a40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:03.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:03.000+0000 7f25b77fe700 1 --2- 172.21.15.2:0/1093446524 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f25a40b2910 0x7f25a40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:03.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:03.000+0000 7f25b4ff9700 1 -- 172.21.15.2:0/1093446524 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f25a00e2360 con 0x7f25b8103fa0 2024-05-14T16:52:03.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:03.001+0000 7f25b77fe700 1 --2- 172.21.15.2:0/1093446524 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f25a40b2910 0x7f25a40b4dd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f25a8009550 tx=0x7f25a80091b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:03.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:03.004+0000 7f25b4ff9700 1 -- 172.21.15.2:0/1093446524 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f25a0020030 con 0x7f25b8103fa0 2024-05-14T16:52:03.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:03.256+0000 7f25bddcd700 1 -- 172.21.15.2:0/1093446524 --> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f25b819aef0 con 0x7f25a40b2910 2024-05-14T16:52:03.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:03.261+0000 7f25b4ff9700 1 -- 172.21.15.2:0/1093446524 <== mgr.14150 v2:172.21.15.2:6800/1498936028 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f25b819aef0 con 0x7f25a40b2910 2024-05-14T16:52:03.261 INFO:teuthology.orchestra.run.smithi002.stdout:/volumes/qa/sv_0/9f51b39a-9a19-43ef-a0fd-4ed365bb2836 2024-05-14T16:52:03.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:03.263+0000 7f25bddcd700 1 -- 172.21.15.2:0/1093446524 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f25a40b2910 msgr2=0x7f25a40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:03.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:03.263+0000 7f25bddcd700 1 --2- 172.21.15.2:0/1093446524 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f25a40b2910 0x7f25a40b4dd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f25a8009550 tx=0x7f25a80091b0 comp rx=0 tx=0).stop 2024-05-14T16:52:03.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:03.263+0000 7f25bddcd700 1 -- 172.21.15.2:0/1093446524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25b8103fa0 msgr2=0x7f25b819fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:03.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:03.263+0000 7f25bddcd700 1 --2- 172.21.15.2:0/1093446524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25b8103fa0 0x7f25b819fd40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f25a000d980 tx=0x7f25a000dc90 comp rx=0 tx=0).stop 2024-05-14T16:52:03.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:03.263+0000 7f25bddcd700 1 -- 172.21.15.2:0/1093446524 shutdown_connections 2024-05-14T16:52:03.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:03.263+0000 7f25bddcd700 1 --2- 172.21.15.2:0/1093446524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25b8109680 0x7f25b81999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:03.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:03.263+0000 7f25bddcd700 1 --2- 172.21.15.2:0/1093446524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25b81035e0 0x7f25b819f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:03.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:03.263+0000 7f25bddcd700 1 --2- 172.21.15.2:0/1093446524 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f25a40b2910 0x7f25a40b4dd0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:03.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:03.263+0000 7f25bddcd700 1 --2- 172.21.15.2:0/1093446524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25b8103fa0 0x7f25b819fd40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:03.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:03.263+0000 7f25bddcd700 1 -- 172.21.15.2:0/1093446524 >> 172.21.15.2:0/1093446524 conn(0x7f25b807a6f0 msgr2=0x7f25b8102260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:03.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:03.264+0000 7f25bddcd700 1 -- 172.21.15.2:0/1093446524 shutdown_connections 2024-05-14T16:52:03.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:03.264+0000 7f25bddcd700 1 -- 172.21.15.2:0/1093446524 wait complete. 2024-05-14T16:52:03.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:03 smithi002 ceph-mon[31538]: pgmap v303: 97 pgs: 97 active+clean; 588 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-05-14T16:52:03.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:03 smithi151 ceph-mon[36066]: pgmap v303: 97 pgs: 97 active+clean; 588 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-05-14T16:52:03.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:03 smithi123 ceph-mon[35971]: pgmap v303: 97 pgs: 97 active+clean; 588 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-05-14T16:52:03.926 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-14T16:52:04.187 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:52:04.457 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:04 smithi002 ceph-mon[31538]: from='client.14682 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:52:04.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:04 smithi151 ceph-mon[36066]: from='client.14682 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:52:04.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:04 smithi123 ceph-mon[35971]: from='client.14682 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:52:05.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:05 smithi151 ceph-mon[36066]: pgmap v304: 97 pgs: 97 active+clean; 608 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.1 KiB/s wr, 10 op/s 2024-05-14T16:52:05.377 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:05 smithi002 ceph-mon[31538]: pgmap v304: 97 pgs: 97 active+clean; 608 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.1 KiB/s wr, 10 op/s 2024-05-14T16:52:05.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:05 smithi123 ceph-mon[35971]: pgmap v304: 97 pgs: 97 active+clean; 608 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.1 KiB/s wr, 10 op/s 2024-05-14T16:52:05.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.962+0000 7faa4bc97700 1 -- 172.21.15.2:0/1850680011 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa44104680 msgr2=0x7faa44101a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:05.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.962+0000 7faa4bc97700 1 --2- 172.21.15.2:0/1850680011 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa44104680 0x7faa44101a90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faa40009990 tx=0x7faa40009ca0 comp rx=0 tx=0).stop 2024-05-14T16:52:05.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.962+0000 7faa4bc97700 1 -- 172.21.15.2:0/1850680011 shutdown_connections 2024-05-14T16:52:05.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.962+0000 7faa4bc97700 1 --2- 172.21.15.2:0/1850680011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa44110210 0x7faa44112600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:05.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.962+0000 7faa4bc97700 1 --2- 172.21.15.2:0/1850680011 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa44104680 0x7faa44101a90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:05.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.962+0000 7faa4bc97700 1 --2- 172.21.15.2:0/1850680011 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa44101fd0 0x7faa44102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:05.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.962+0000 7faa4bc97700 1 -- 172.21.15.2:0/1850680011 >> 172.21.15.2:0/1850680011 conn(0x7faa44078a40 msgr2=0x7faa440ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:05.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.962+0000 7faa4bc97700 1 -- 172.21.15.2:0/1850680011 shutdown_connections 2024-05-14T16:52:05.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.962+0000 7faa4bc97700 1 -- 172.21.15.2:0/1850680011 wait complete. 2024-05-14T16:52:05.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.963+0000 7faa4bc97700 1 Processor -- start 2024-05-14T16:52:05.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.963+0000 7faa4bc97700 1 -- start start 2024-05-14T16:52:05.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.963+0000 7faa4bc97700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa44101fd0 0x7faa4419c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:05.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.963+0000 7faa4bc97700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa44104680 0x7faa4419c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:05.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.963+0000 7faa4bc97700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa44110210 0x7faa441a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:05.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.963+0000 7faa4bc97700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7faa44115b70 con 0x7faa44110210 2024-05-14T16:52:05.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.963+0000 7faa4bc97700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faa441159f0 con 0x7faa44101fd0 2024-05-14T16:52:05.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.963+0000 7faa4bc97700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faa44115cf0 con 0x7faa44104680 2024-05-14T16:52:05.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.963+0000 7faa4a234700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa44110210 0x7faa441a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:05.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.963+0000 7faa4a234700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa44110210 0x7faa441a0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36452/0 (socket says 172.21.15.2:36452) 2024-05-14T16:52:05.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.963+0000 7faa4a234700 1 -- 172.21.15.2:0/3130882351 learned_addr learned my addr 172.21.15.2:0/3130882351 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:52:05.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.964+0000 7faa49a33700 1 --2- 172.21.15.2:0/3130882351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa44101fd0 0x7faa4419c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:05.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.964+0000 7faa49232700 1 --2- 172.21.15.2:0/3130882351 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa44104680 0x7faa4419c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:05.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.964+0000 7faa4a234700 1 -- 172.21.15.2:0/3130882351 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa44104680 msgr2=0x7faa4419c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:05.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.964+0000 7faa4a234700 1 --2- 172.21.15.2:0/3130882351 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa44104680 0x7faa4419c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:05.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.964+0000 7faa4a234700 1 -- 172.21.15.2:0/3130882351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa44101fd0 msgr2=0x7faa4419c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:05.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.964+0000 7faa4a234700 1 --2- 172.21.15.2:0/3130882351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa44101fd0 0x7faa4419c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:05.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.964+0000 7faa4a234700 1 -- 172.21.15.2:0/3130882351 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa441a1050 con 0x7faa44110210 2024-05-14T16:52:05.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.964+0000 7faa49232700 1 --2- 172.21.15.2:0/3130882351 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa44104680 0x7faa4419c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:52:05.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.964+0000 7faa4a234700 1 --2- 172.21.15.2:0/3130882351 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa44110210 0x7faa441a0910 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7faa3c007f10 tx=0x7faa3c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:05.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.964+0000 7faa3affd700 1 -- 172.21.15.2:0/3130882351 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3c00dbd0 con 0x7faa44110210 2024-05-14T16:52:05.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.964+0000 7faa3affd700 1 -- 172.21.15.2:0/3130882351 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa3c00dd70 con 0x7faa44110210 2024-05-14T16:52:05.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.964+0000 7faa4bc97700 1 -- 172.21.15.2:0/3130882351 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa441a1340 con 0x7faa44110210 2024-05-14T16:52:05.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.964+0000 7faa3affd700 1 -- 172.21.15.2:0/3130882351 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3c0044e0 con 0x7faa44110210 2024-05-14T16:52:05.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.964+0000 7faa4bc97700 1 -- 172.21.15.2:0/3130882351 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa441a8bc0 con 0x7faa44110210 2024-05-14T16:52:05.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.965+0000 7faa4bc97700 1 -- 172.21.15.2:0/3130882351 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa4406a230 con 0x7faa44110210 2024-05-14T16:52:05.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.966+0000 7faa3affd700 1 -- 172.21.15.2:0/3130882351 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7faa3c01a030 con 0x7faa44110210 2024-05-14T16:52:05.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.967+0000 7faa3affd700 1 --2- 172.21.15.2:0/3130882351 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7faa300b2910 0x7faa300b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:05.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.967+0000 7faa49a33700 1 --2- 172.21.15.2:0/3130882351 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7faa300b2910 0x7faa300b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:05.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.967+0000 7faa3affd700 1 -- 172.21.15.2:0/3130882351 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7faa3c0e16a0 con 0x7faa44110210 2024-05-14T16:52:05.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.968+0000 7faa49a33700 1 --2- 172.21.15.2:0/3130882351 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7faa300b2910 0x7faa300b4dd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7faa4000b5c0 tx=0x7faa40005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:05.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:05.972+0000 7faa3affd700 1 -- 172.21.15.2:0/3130882351 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faa3c021020 con 0x7faa44110210 2024-05-14T16:52:06.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:06.222+0000 7faa4bc97700 1 -- 172.21.15.2:0/3130882351 --> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7faa44100920 con 0x7faa300b2910 2024-05-14T16:52:06.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:06.252+0000 7faa3affd700 1 -- 172.21.15.2:0/3130882351 <== mgr.14150 v2:172.21.15.2:6800/1498936028 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7faa44100920 con 0x7faa300b2910 2024-05-14T16:52:06.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:06.255+0000 7faa4bc97700 1 -- 172.21.15.2:0/3130882351 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7faa300b2910 msgr2=0x7faa300b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:06.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:06.255+0000 7faa4bc97700 1 --2- 172.21.15.2:0/3130882351 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7faa300b2910 0x7faa300b4dd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7faa4000b5c0 tx=0x7faa40005960 comp rx=0 tx=0).stop 2024-05-14T16:52:06.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:06.255+0000 7faa4bc97700 1 -- 172.21.15.2:0/3130882351 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa44110210 msgr2=0x7faa441a0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:06.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:06.255+0000 7faa4bc97700 1 --2- 172.21.15.2:0/3130882351 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa44110210 0x7faa441a0910 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7faa3c007f10 tx=0x7faa3c00d3b0 comp rx=0 tx=0).stop 2024-05-14T16:52:06.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:06.255+0000 7faa4bc97700 1 -- 172.21.15.2:0/3130882351 shutdown_connections 2024-05-14T16:52:06.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:06.255+0000 7faa4bc97700 1 --2- 172.21.15.2:0/3130882351 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7faa300b2910 0x7faa300b4dd0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:06.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:06.255+0000 7faa4bc97700 1 --2- 172.21.15.2:0/3130882351 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa44110210 0x7faa441a0910 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:06.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:06.255+0000 7faa4bc97700 1 --2- 172.21.15.2:0/3130882351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa44101fd0 0x7faa4419c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:06.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:06.255+0000 7faa4bc97700 1 --2- 172.21.15.2:0/3130882351 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa44104680 0x7faa4419c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:06.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:06.255+0000 7faa4bc97700 1 -- 172.21.15.2:0/3130882351 >> 172.21.15.2:0/3130882351 conn(0x7faa44078a40 msgr2=0x7faa44100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:06.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:06.255+0000 7faa4bc97700 1 -- 172.21.15.2:0/3130882351 shutdown_connections 2024-05-14T16:52:06.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:06.255+0000 7faa4bc97700 1 -- 172.21.15.2:0/3130882351 wait complete. 2024-05-14T16:52:06.872 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-14T16:52:07.122 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:52:07.436 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:07 smithi002 ceph-mon[31538]: pgmap v305: 97 pgs: 97 active+clean; 608 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.8 KiB/s wr, 6 op/s 2024-05-14T16:52:07.436 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:07 smithi002 ceph-mon[31538]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:52:07.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:07 smithi151 ceph-mon[36066]: pgmap v305: 97 pgs: 97 active+clean; 608 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.8 KiB/s wr, 6 op/s 2024-05-14T16:52:07.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:07 smithi151 ceph-mon[36066]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:52:07.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:07 smithi123 ceph-mon[35971]: pgmap v305: 97 pgs: 97 active+clean; 608 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.8 KiB/s wr, 6 op/s 2024-05-14T16:52:07.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:07 smithi123 ceph-mon[35971]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:52:08.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:08.803+0000 7f1fe4dec700 1 -- 172.21.15.2:0/1006932172 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1fe01041a0 msgr2=0x7f1fe01081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:08.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:08.803+0000 7f1fe4dec700 1 --2- 172.21.15.2:0/1006932172 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1fe01041a0 0x7f1fe01081c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1fc80099c0 tx=0x7f1fc8009cd0 comp rx=0 tx=0).stop 2024-05-14T16:52:08.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:08.803+0000 7f1fe4dec700 1 -- 172.21.15.2:0/1006932172 shutdown_connections 2024-05-14T16:52:08.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:08.803+0000 7f1fe4dec700 1 --2- 172.21.15.2:0/1006932172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fe01088f0 0x7f1fe0110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:08.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:08.803+0000 7f1fe4dec700 1 --2- 172.21.15.2:0/1006932172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fe01037f0 0x7f1fe0103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:08.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:08.803+0000 7f1fe4dec700 1 --2- 172.21.15.2:0/1006932172 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1fe01041a0 0x7f1fe01081c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:08.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:08.803+0000 7f1fe4dec700 1 -- 172.21.15.2:0/1006932172 >> 172.21.15.2:0/1006932172 conn(0x7f1fe007a6a0 msgr2=0x7f1fe007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:08.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:08.803+0000 7f1fe4dec700 1 -- 172.21.15.2:0/1006932172 shutdown_connections 2024-05-14T16:52:08.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:08.803+0000 7f1fe4dec700 1 -- 172.21.15.2:0/1006932172 wait complete. 2024-05-14T16:52:08.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:08.804+0000 7f1fe4dec700 1 Processor -- start 2024-05-14T16:52:08.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:08.804+0000 7f1fe4dec700 1 -- start start 2024-05-14T16:52:08.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:08.804+0000 7f1fe4dec700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1fe01037f0 0x7f1fe019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:08.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:08.804+0000 7f1fe4dec700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fe01041a0 0x7f1fe019fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:08.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:08.804+0000 7f1fe4dec700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fe01088f0 0x7f1fe0199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:08.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:08.804+0000 7f1fe4dec700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe01136b0 con 0x7f1fe01037f0 2024-05-14T16:52:08.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:08.804+0000 7f1fe4dec700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe0113530 con 0x7f1fe01041a0 2024-05-14T16:52:08.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:08.804+0000 7f1fe4dec700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe0113830 con 0x7f1fe01088f0 2024-05-14T16:52:08.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:08.804+0000 7f1fde59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1fe01037f0 0x7f1fe019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:08.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:08.804+0000 7f1fde59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1fe01037f0 0x7f1fe019f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36462/0 (socket says 172.21.15.2:36462) 2024-05-14T16:52:08.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:08.804+0000 7f1fde59c700 1 -- 172.21.15.2:0/3214781507 learned_addr learned my addr 172.21.15.2:0/3214781507 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:52:08.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:08.804+0000 7f1fd7fff700 1 --2- 172.21.15.2:0/3214781507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fe01041a0 0x7f1fe019fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:08.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:08.805+0000 7f1fded9d700 1 --2- 172.21.15.2:0/3214781507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fe01088f0 0x7f1fe0199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:08.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:08.805+0000 7f1fde59c700 1 -- 172.21.15.2:0/3214781507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fe01088f0 msgr2=0x7f1fe0199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:08.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:08.805+0000 7f1fde59c700 1 --2- 172.21.15.2:0/3214781507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fe01088f0 0x7f1fe0199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:08.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:08.805+0000 7f1fde59c700 1 -- 172.21.15.2:0/3214781507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fe01041a0 msgr2=0x7f1fe019fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:08.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:08.805+0000 7f1fde59c700 1 --2- 172.21.15.2:0/3214781507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fe01041a0 0x7f1fe019fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:08.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:08.805+0000 7f1fde59c700 1 -- 172.21.15.2:0/3214781507 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe019a090 con 0x7f1fe01037f0 2024-05-14T16:52:08.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:08.805+0000 7f1fde59c700 1 --2- 172.21.15.2:0/3214781507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1fe01037f0 0x7f1fe019f770 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1fd000e9a0 tx=0x7f1fd000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:08.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:08.805+0000 7f1fd77fe700 1 -- 172.21.15.2:0/3214781507 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd000cbd0 con 0x7f1fe01037f0 2024-05-14T16:52:08.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:08.805+0000 7f1fd77fe700 1 -- 172.21.15.2:0/3214781507 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fd000cd70 con 0x7f1fe01037f0 2024-05-14T16:52:08.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:08.805+0000 7f1fe4dec700 1 -- 172.21.15.2:0/3214781507 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe019a380 con 0x7f1fe01037f0 2024-05-14T16:52:08.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:08.805+0000 7f1fd77fe700 1 -- 172.21.15.2:0/3214781507 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0018680 con 0x7f1fe01037f0 2024-05-14T16:52:08.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:08.805+0000 7f1fe4dec700 1 -- 172.21.15.2:0/3214781507 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe01a6640 con 0x7f1fe01037f0 2024-05-14T16:52:08.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:08.806+0000 7f1fe4dec700 1 -- 172.21.15.2:0/3214781507 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fe006a230 con 0x7f1fe01037f0 2024-05-14T16:52:08.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:08.808+0000 7f1fd77fe700 1 -- 172.21.15.2:0/3214781507 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f1fd0010430 con 0x7f1fe01037f0 2024-05-14T16:52:08.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:08.809+0000 7f1fd77fe700 1 --2- 172.21.15.2:0/3214781507 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f1fcc0b2910 0x7f1fcc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:08.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:08.809+0000 7f1fd7fff700 1 --2- 172.21.15.2:0/3214781507 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f1fcc0b2910 0x7f1fcc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:08.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:08.809+0000 7f1fd77fe700 1 -- 172.21.15.2:0/3214781507 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f1fd00e26d0 con 0x7f1fe01037f0 2024-05-14T16:52:08.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:08.809+0000 7f1fd7fff700 1 --2- 172.21.15.2:0/3214781507 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f1fcc0b2910 0x7f1fcc0b4dd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0108190 tx=0x7f1fc8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:08.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:08.813+0000 7f1fd77fe700 1 -- 172.21.15.2:0/3214781507 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1fd00aa440 con 0x7f1fe01037f0 2024-05-14T16:52:09.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:09.063+0000 7f1fe4dec700 1 -- 172.21.15.2:0/3214781507 --> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f1fe019ae60 con 0x7f1fcc0b2910 2024-05-14T16:52:09.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:09.069+0000 7f1fd77fe700 1 -- 172.21.15.2:0/3214781507 <== mgr.14150 v2:172.21.15.2:6800/1498936028 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f1fe019ae60 con 0x7f1fcc0b2910 2024-05-14T16:52:09.068 INFO:teuthology.orchestra.run.smithi002.stdout:/volumes/qa/sv_1/83bdfcd3-fc78-423b-b57a-a691f0db09d1 2024-05-14T16:52:09.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:09.071+0000 7f1fe4dec700 1 -- 172.21.15.2:0/3214781507 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f1fcc0b2910 msgr2=0x7f1fcc0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:09.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:09.071+0000 7f1fe4dec700 1 --2- 172.21.15.2:0/3214781507 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f1fcc0b2910 0x7f1fcc0b4dd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0108190 tx=0x7f1fc8005fb0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:09.071+0000 7f1fe4dec700 1 -- 172.21.15.2:0/3214781507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1fe01037f0 msgr2=0x7f1fe019f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:09.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:09.071+0000 7f1fe4dec700 1 --2- 172.21.15.2:0/3214781507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1fe01037f0 0x7f1fe019f770 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1fd000e9a0 tx=0x7f1fd000ecb0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:09.071+0000 7f1fe4dec700 1 -- 172.21.15.2:0/3214781507 shutdown_connections 2024-05-14T16:52:09.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:09.071+0000 7f1fe4dec700 1 --2- 172.21.15.2:0/3214781507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fe01088f0 0x7f1fe0199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:09.071+0000 7f1fe4dec700 1 --2- 172.21.15.2:0/3214781507 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f1fcc0b2910 0x7f1fcc0b4dd0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:09.071+0000 7f1fe4dec700 1 --2- 172.21.15.2:0/3214781507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1fe01037f0 0x7f1fe019f770 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:09.071+0000 7f1fe4dec700 1 --2- 172.21.15.2:0/3214781507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fe01041a0 0x7f1fe019fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:09.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:09.071+0000 7f1fe4dec700 1 -- 172.21.15.2:0/3214781507 >> 172.21.15.2:0/3214781507 conn(0x7f1fe007a6a0 msgr2=0x7f1fe0101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:09.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:09.071+0000 7f1fe4dec700 1 -- 172.21.15.2:0/3214781507 shutdown_connections 2024-05-14T16:52:09.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:09.071+0000 7f1fe4dec700 1 -- 172.21.15.2:0/3214781507 wait complete. 2024-05-14T16:52:09.447 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:09 smithi002 ceph-mon[31538]: pgmap v306: 97 pgs: 97 active+clean; 608 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-05-14T16:52:09.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:09 smithi151 ceph-mon[36066]: pgmap v306: 97 pgs: 97 active+clean; 608 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-05-14T16:52:09.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:09 smithi123 ceph-mon[35971]: pgmap v306: 97 pgs: 97 active+clean; 608 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-05-14T16:52:09.780 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:52:10.031 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:52:10.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:10 smithi002 ceph-mon[31538]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:52:10.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:10 smithi151 ceph-mon[36066]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:52:10.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:10 smithi123 ceph-mon[35971]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:52:11.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:11 smithi002 ceph-mon[31538]: pgmap v307: 97 pgs: 97 active+clean; 630 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.8 KiB/s wr, 9 op/s 2024-05-14T16:52:11.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:11 smithi151 ceph-mon[36066]: pgmap v307: 97 pgs: 97 active+clean; 630 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.8 KiB/s wr, 9 op/s 2024-05-14T16:52:11.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:11 smithi123 ceph-mon[35971]: pgmap v307: 97 pgs: 97 active+clean; 630 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.8 KiB/s wr, 9 op/s 2024-05-14T16:52:11.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.770+0000 7fea5c51e700 1 -- 172.21.15.2:0/208752588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea541041a0 msgr2=0x7fea541081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:11.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.770+0000 7fea5c51e700 1 --2- 172.21.15.2:0/208752588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea541041a0 0x7fea541081c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fea440099c0 tx=0x7fea44009cd0 comp rx=0 tx=0).stop 2024-05-14T16:52:11.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.770+0000 7fea5c51e700 1 -- 172.21.15.2:0/208752588 shutdown_connections 2024-05-14T16:52:11.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.770+0000 7fea5c51e700 1 --2- 172.21.15.2:0/208752588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea541088f0 0x7fea54110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:11.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.770+0000 7fea5c51e700 1 --2- 172.21.15.2:0/208752588 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea541037f0 0x7fea54103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:11.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.770+0000 7fea5c51e700 1 --2- 172.21.15.2:0/208752588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea541041a0 0x7fea541081c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:11.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.771+0000 7fea5c51e700 1 -- 172.21.15.2:0/208752588 >> 172.21.15.2:0/208752588 conn(0x7fea5407a6a0 msgr2=0x7fea5407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:11.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.771+0000 7fea5c51e700 1 -- 172.21.15.2:0/208752588 shutdown_connections 2024-05-14T16:52:11.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.771+0000 7fea5c51e700 1 -- 172.21.15.2:0/208752588 wait complete. 2024-05-14T16:52:11.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.771+0000 7fea5c51e700 1 Processor -- start 2024-05-14T16:52:11.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.771+0000 7fea5c51e700 1 -- start start 2024-05-14T16:52:11.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.772+0000 7fea5c51e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea541037f0 0x7fea5419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:11.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.772+0000 7fea5c51e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea541041a0 0x7fea5419fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:11.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.772+0000 7fea5c51e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea541088f0 0x7fea54199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:11.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.772+0000 7fea5c51e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fea541136b0 con 0x7fea541041a0 2024-05-14T16:52:11.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.772+0000 7fea5c51e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fea54113530 con 0x7fea541037f0 2024-05-14T16:52:11.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.772+0000 7fea5c51e700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fea54113830 con 0x7fea541088f0 2024-05-14T16:52:11.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.772+0000 7fea59ab9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea541041a0 0x7fea5419fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:11.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.772+0000 7fea5a2ba700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea541037f0 0x7fea5419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:11.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.772+0000 7fea59ab9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea541041a0 0x7fea5419fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36486/0 (socket says 172.21.15.2:36486) 2024-05-14T16:52:11.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.772+0000 7fea59ab9700 1 -- 172.21.15.2:0/1471942552 learned_addr learned my addr 172.21.15.2:0/1471942552 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:52:11.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.772+0000 7fea5a2ba700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea541037f0 0x7fea5419f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:46634/0 (socket says 172.21.15.2:46634) 2024-05-14T16:52:11.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.772+0000 7fea5aabb700 1 --2- 172.21.15.2:0/1471942552 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea541088f0 0x7fea54199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:11.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.772+0000 7fea5a2ba700 1 -- 172.21.15.2:0/1471942552 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea541088f0 msgr2=0x7fea54199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:11.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.772+0000 7fea5a2ba700 1 --2- 172.21.15.2:0/1471942552 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea541088f0 0x7fea54199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:11.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.772+0000 7fea5a2ba700 1 -- 172.21.15.2:0/1471942552 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea541041a0 msgr2=0x7fea5419fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:11.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.772+0000 7fea5a2ba700 1 --2- 172.21.15.2:0/1471942552 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea541041a0 0x7fea5419fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:11.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.772+0000 7fea5a2ba700 1 -- 172.21.15.2:0/1471942552 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea5419a090 con 0x7fea541037f0 2024-05-14T16:52:11.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.772+0000 7fea59ab9700 1 --2- 172.21.15.2:0/1471942552 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea541041a0 0x7fea5419fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:52:11.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.773+0000 7fea5a2ba700 1 --2- 172.21.15.2:0/1471942552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea541037f0 0x7fea5419f720 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fea5000c970 tx=0x7fea5000cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:11.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.773+0000 7fea4b7fe700 1 -- 172.21.15.2:0/1471942552 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50014070 con 0x7fea541037f0 2024-05-14T16:52:11.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.773+0000 7fea5c51e700 1 -- 172.21.15.2:0/1471942552 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea5419a380 con 0x7fea541037f0 2024-05-14T16:52:11.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.773+0000 7fea4b7fe700 1 -- 172.21.15.2:0/1471942552 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea50004440 con 0x7fea541037f0 2024-05-14T16:52:11.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.773+0000 7fea5c51e700 1 -- 172.21.15.2:0/1471942552 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea541a65f0 con 0x7fea541037f0 2024-05-14T16:52:11.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.773+0000 7fea4b7fe700 1 -- 172.21.15.2:0/1471942552 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50007a30 con 0x7fea541037f0 2024-05-14T16:52:11.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.774+0000 7fea5c51e700 1 -- 172.21.15.2:0/1471942552 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea38005330 con 0x7fea541037f0 2024-05-14T16:52:11.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.776+0000 7fea4b7fe700 1 -- 172.21.15.2:0/1471942552 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fea50017070 con 0x7fea541037f0 2024-05-14T16:52:11.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.776+0000 7fea4b7fe700 1 --2- 172.21.15.2:0/1471942552 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fea400b2910 0x7fea400b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:11.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.776+0000 7fea59ab9700 1 --2- 172.21.15.2:0/1471942552 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fea400b2910 0x7fea400b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:11.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.776+0000 7fea4b7fe700 1 -- 172.21.15.2:0/1471942552 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7fea500e2290 con 0x7fea541037f0 2024-05-14T16:52:11.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.777+0000 7fea59ab9700 1 --2- 172.21.15.2:0/1471942552 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fea400b2910 0x7fea400b4dd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fea44003800 tx=0x7fea44005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:11.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:11.780+0000 7fea4b7fe700 1 -- 172.21.15.2:0/1471942552 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fea5001e030 con 0x7fea541037f0 2024-05-14T16:52:12.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:12.051+0000 7fea5c51e700 1 -- 172.21.15.2:0/1471942552 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea38005620 con 0x7fea541037f0 2024-05-14T16:52:12.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:12.052+0000 7fea4b7fe700 1 -- 172.21.15.2:0/1471942552 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11850 (secure 0 0 0) 0x7fea500a9f00 con 0x7fea541037f0 2024-05-14T16:52:12.052 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:52:12.052 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2275696923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2275696923},{"type":"v1","addr":"172.21.15.2:6835","nonce":2275696923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/716718761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":716718761},{"type":"v1","addr":"172.21.15.2:6839","nonce":716718761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/3644815179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3644815179},{"type":"v1","addr":"172.21.15.2:6841","nonce":3644815179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1880336731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1880336731},{"type":"v1","addr":"172.21.15.151:6833","nonce":1880336731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2415605548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2415605548},{"type":"v1","addr":"172.21.15.2:6837","nonce":2415605548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2635167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2635167},{"type":"v1","addr":"172.21.15.123:6833","nonce":2635167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:51:34.106919+0000","modified":"2024-05-14T16:51:53.879252+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24455,"mds_2":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24415":{"gid":24415,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/762044339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":762044339},{"type":"v1","addr":"172.21.15.151:6835","nonce":762044339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/132043151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":132043151},{"type":"v1","addr":"172.21.15.151:6837","nonce":132043151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/4107834375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":4107834375},{"type":"v1","addr":"172.21.15.151:6839","nonce":4107834375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.123:6835/1198670013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1198670013},{"type":"v1","addr":"172.21.15.123:6835","nonce":1198670013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/333426645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":333426645},{"type":"v1","addr":"172.21.15.123:6837","nonce":333426645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/580275877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":580275877},{"type":"v1","addr":"172.21.15.123:6839","nonce":580275877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24455,24479,24431]},"id":1}]} 2024-05-14T16:52:12.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:12.055+0000 7fea5c51e700 1 -- 172.21.15.2:0/1471942552 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fea400b2910 msgr2=0x7fea400b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:12.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:12.055+0000 7fea5c51e700 1 --2- 172.21.15.2:0/1471942552 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fea400b2910 0x7fea400b4dd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fea44003800 tx=0x7fea44005fb0 comp rx=0 tx=0).stop 2024-05-14T16:52:12.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:12.055+0000 7fea5c51e700 1 -- 172.21.15.2:0/1471942552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea541037f0 msgr2=0x7fea5419f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:12.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:12.055+0000 7fea5c51e700 1 --2- 172.21.15.2:0/1471942552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea541037f0 0x7fea5419f720 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fea5000c970 tx=0x7fea5000cc80 comp rx=0 tx=0).stop 2024-05-14T16:52:12.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:12.055+0000 7fea5c51e700 1 -- 172.21.15.2:0/1471942552 shutdown_connections 2024-05-14T16:52:12.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:12.055+0000 7fea5c51e700 1 --2- 172.21.15.2:0/1471942552 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea541088f0 0x7fea54199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:12.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:12.055+0000 7fea5c51e700 1 --2- 172.21.15.2:0/1471942552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea541037f0 0x7fea5419f720 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:12.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:12.055+0000 7fea5c51e700 1 --2- 172.21.15.2:0/1471942552 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fea400b2910 0x7fea400b4dd0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:12.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:12.055+0000 7fea5c51e700 1 --2- 172.21.15.2:0/1471942552 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea541041a0 0x7fea5419fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:12.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:12.055+0000 7fea5c51e700 1 -- 172.21.15.2:0/1471942552 >> 172.21.15.2:0/1471942552 conn(0x7fea5407a6a0 msgr2=0x7fea54102d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:12.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:12.055+0000 7fea5c51e700 1 -- 172.21.15.2:0/1471942552 shutdown_connections 2024-05-14T16:52:12.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:12.055+0000 7fea5c51e700 1 -- 172.21.15.2:0/1471942552 wait complete. 2024-05-14T16:52:12.056 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 24 2024-05-14T16:52:12.366 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:12 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-05-14T16:52:12.366 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:12 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-14T16:52:12.366 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:12 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2024-05-14T16:52:12.366 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:12 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-05-14T16:52:12.366 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:12 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-05-14T16:52:12.366 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:12 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-05-14T16:52:12.366 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:12 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-05-14T16:52:12.366 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:12 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-05-14T16:52:12.366 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:12 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/1471942552' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:52:12.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:12 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-05-14T16:52:12.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:12 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-14T16:52:12.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:12 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2024-05-14T16:52:12.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:12 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-05-14T16:52:12.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:12 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-05-14T16:52:12.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:12 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-05-14T16:52:12.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:12 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-05-14T16:52:12.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:12 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-05-14T16:52:12.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:12 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/1471942552' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:52:12.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:12 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-05-14T16:52:12.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:12 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-14T16:52:12.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:12 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2024-05-14T16:52:12.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:12 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-05-14T16:52:12.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:12 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-05-14T16:52:12.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:12 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-05-14T16:52:12.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:12 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-05-14T16:52:12.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:12 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-05-14T16:52:12.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:12 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/1471942552' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:52:12.697 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T16:52:12.947 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:52:13.336 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:13 smithi002 ceph-mon[31538]: pgmap v308: 97 pgs: 97 active+clean; 630 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.1 KiB/s wr, 7 op/s 2024-05-14T16:52:13.336 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:13 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-05-14T16:52:13.336 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:13 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-14T16:52:13.336 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:13 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-05-14T16:52:13.336 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:13 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-05-14T16:52:13.336 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:13 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-05-14T16:52:13.337 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:13 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-05-14T16:52:13.337 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:13 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-05-14T16:52:13.337 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:13 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-05-14T16:52:13.337 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:13 smithi002 ceph-mon[31538]: osdmap e79: 12 total, 12 up, 12 in 2024-05-14T16:52:13.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:13 smithi151 ceph-mon[36066]: pgmap v308: 97 pgs: 97 active+clean; 630 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.1 KiB/s wr, 7 op/s 2024-05-14T16:52:13.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:13 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-05-14T16:52:13.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:13 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-14T16:52:13.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:13 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-05-14T16:52:13.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:13 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-05-14T16:52:13.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:13 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-05-14T16:52:13.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:13 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-05-14T16:52:13.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:13 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-05-14T16:52:13.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:13 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-05-14T16:52:13.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:13 smithi151 ceph-mon[36066]: osdmap e79: 12 total, 12 up, 12 in 2024-05-14T16:52:13.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:13 smithi123 ceph-mon[35971]: pgmap v308: 97 pgs: 97 active+clean; 630 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.1 KiB/s wr, 7 op/s 2024-05-14T16:52:13.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:13 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-05-14T16:52:13.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:13 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-14T16:52:13.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:13 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-05-14T16:52:13.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:13 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-05-14T16:52:13.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:13 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-05-14T16:52:13.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:13 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-05-14T16:52:13.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:13 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-05-14T16:52:13.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:13 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-05-14T16:52:13.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:13 smithi123 ceph-mon[35971]: osdmap e79: 12 total, 12 up, 12 in 2024-05-14T16:52:14.343 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:14 smithi002 ceph-mon[31538]: osdmap e80: 12 total, 12 up, 12 in 2024-05-14T16:52:14.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:14 smithi151 ceph-mon[36066]: osdmap e80: 12 total, 12 up, 12 in 2024-05-14T16:52:14.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.620+0000 7f5cab50c700 1 -- 172.21.15.2:0/2329564347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ca406c470 msgr2=0x7f5ca41103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:14.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.620+0000 7f5cab50c700 1 --2- 172.21.15.2:0/2329564347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ca406c470 0x7f5ca41103c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00b0a0 tx=0x7f5c9c00b3b0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.620+0000 7f5cab50c700 1 -- 172.21.15.2:0/2329564347 shutdown_connections 2024-05-14T16:52:14.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.620+0000 7f5cab50c700 1 --2- 172.21.15.2:0/2329564347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ca406c470 0x7f5ca41103c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.620+0000 7f5cab50c700 1 --2- 172.21.15.2:0/2329564347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ca41095a0 0x7f5ca4109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.620+0000 7f5cab50c700 1 --2- 172.21.15.2:0/2329564347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ca406bab0 0x7f5ca406bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.620+0000 7f5cab50c700 1 -- 172.21.15.2:0/2329564347 >> 172.21.15.2:0/2329564347 conn(0x7f5ca40793b0 msgr2=0x7f5ca40797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:14.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.620+0000 7f5cab50c700 1 -- 172.21.15.2:0/2329564347 shutdown_connections 2024-05-14T16:52:14.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.620+0000 7f5cab50c700 1 -- 172.21.15.2:0/2329564347 wait complete. 2024-05-14T16:52:14.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.621+0000 7f5cab50c700 1 Processor -- start 2024-05-14T16:52:14.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.621+0000 7f5cab50c700 1 -- start start 2024-05-14T16:52:14.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.621+0000 7f5cab50c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ca406bab0 0x7f5ca4102500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:14.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.621+0000 7f5cab50c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ca406c470 0x7f5ca4102a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:14.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.621+0000 7f5cab50c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ca41095a0 0x7f5ca4102f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:14.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.621+0000 7f5cab50c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca41136d0 con 0x7f5ca406bab0 2024-05-14T16:52:14.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.621+0000 7f5cab50c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca4113550 con 0x7f5ca406c470 2024-05-14T16:52:14.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.621+0000 7f5cab50c700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca4113850 con 0x7f5ca41095a0 2024-05-14T16:52:14.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.621+0000 7f5ca92a8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ca406bab0 0x7f5ca4102500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:14.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.621+0000 7f5ca92a8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ca406bab0 0x7f5ca4102500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51580/0 (socket says 172.21.15.2:51580) 2024-05-14T16:52:14.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.621+0000 7f5ca92a8700 1 -- 172.21.15.2:0/1026038788 learned_addr learned my addr 172.21.15.2:0/1026038788 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:52:14.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.621+0000 7f5ca8aa7700 1 --2- 172.21.15.2:0/1026038788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ca406c470 0x7f5ca4102a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:14.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.621+0000 7f5ca9aa9700 1 --2- 172.21.15.2:0/1026038788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ca41095a0 0x7f5ca4102f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:14.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.621+0000 7f5ca92a8700 1 -- 172.21.15.2:0/1026038788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ca41095a0 msgr2=0x7f5ca4102f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:14.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.621+0000 7f5ca92a8700 1 --2- 172.21.15.2:0/1026038788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ca41095a0 0x7f5ca4102f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.621+0000 7f5ca92a8700 1 -- 172.21.15.2:0/1026038788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ca406c470 msgr2=0x7f5ca4102a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:14.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.621+0000 7f5ca92a8700 1 --2- 172.21.15.2:0/1026038788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ca406c470 0x7f5ca4102a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.622+0000 7f5ca92a8700 1 -- 172.21.15.2:0/1026038788 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca41a80d0 con 0x7f5ca406bab0 2024-05-14T16:52:14.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.622+0000 7f5ca9aa9700 1 --2- 172.21.15.2:0/1026038788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ca41095a0 0x7f5ca4102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:52:14.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.622+0000 7f5ca8aa7700 1 --2- 172.21.15.2:0/1026038788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ca406c470 0x7f5ca4102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:52:14.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.622+0000 7f5ca92a8700 1 --2- 172.21.15.2:0/1026038788 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ca406bab0 0x7f5ca4102500 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5ca000d980 tx=0x7f5ca000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:14.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.622+0000 7f5c9a7fc700 1 -- 172.21.15.2:0/1026038788 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca0017070 con 0x7f5ca406bab0 2024-05-14T16:52:14.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.622+0000 7f5c9a7fc700 1 -- 172.21.15.2:0/1026038788 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ca0005020 con 0x7f5ca406bab0 2024-05-14T16:52:14.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.622+0000 7f5cab50c700 1 -- 172.21.15.2:0/1026038788 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ca41a83c0 con 0x7f5ca406bab0 2024-05-14T16:52:14.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.622+0000 7f5c9a7fc700 1 -- 172.21.15.2:0/1026038788 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca0009b50 con 0x7f5ca406bab0 2024-05-14T16:52:14.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.622+0000 7f5cab50c700 1 -- 172.21.15.2:0/1026038788 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ca41a88b0 con 0x7f5ca406bab0 2024-05-14T16:52:14.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.623+0000 7f5cab50c700 1 -- 172.21.15.2:0/1026038788 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ca410ba80 con 0x7f5ca406bab0 2024-05-14T16:52:14.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.624+0000 7f5c9a7fc700 1 -- 172.21.15.2:0/1026038788 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f5ca00051d0 con 0x7f5ca406bab0 2024-05-14T16:52:14.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.625+0000 7f5c9a7fc700 1 --2- 172.21.15.2:0/1026038788 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f5c900b2910 0x7f5c900b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:14.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.625+0000 7f5ca8aa7700 1 --2- 172.21.15.2:0/1026038788 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f5c900b2910 0x7f5c900b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:14.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.625+0000 7f5c9a7fc700 1 -- 172.21.15.2:0/1026038788 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f5ca00e2700 con 0x7f5ca406bab0 2024-05-14T16:52:14.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.626+0000 7f5ca8aa7700 1 --2- 172.21.15.2:0/1026038788 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f5c900b2910 0x7f5c900b4dd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4079a90 tx=0x7f5c94009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:14.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.629+0000 7f5c9a7fc700 1 -- 172.21.15.2:0/1026038788 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5ca0020020 con 0x7f5ca406bab0 2024-05-14T16:52:14.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:14 smithi123 ceph-mon[35971]: osdmap e80: 12 total, 12 up, 12 in 2024-05-14T16:52:14.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.843+0000 7f5cab50c700 1 -- 172.21.15.2:0/1026038788 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5ca4103a30 con 0x7f5ca406bab0 2024-05-14T16:52:14.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.845+0000 7f5c9a7fc700 1 -- 172.21.15.2:0/1026038788 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22575 (secure 0 0 0) 0x7f5ca00cae20 con 0x7f5ca406bab0 2024-05-14T16:52:14.845 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:52:14.845 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":80,"fsid":"ed40717a-1210-11ef-bc98-c7b262605968","created":"2024-05-14T16:43:09.415298+0000","modified":"2024-05-14T16:52:13.205039+0000","last_up_change":"2024-05-14T16:50:29.787654+0000","last_in_change":"2024-05-14T16:50:09.287932+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T16:46:38.090903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T16:51:27.296955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T16:51:31.103019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"153f20ec-5bcf-4786-9ede-2e512a1e3d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6803","nonce":2654237820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6805","nonce":2654237820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6809","nonce":2654237820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6807","nonce":2654237820}]},"public_addr":"172.21.15.2:6803/2654237820","cluster_addr":"172.21.15.2:6805/2654237820","heartbeat_back_addr":"172.21.15.2:6809/2654237820","heartbeat_front_addr":"172.21.15.2:6807/2654237820","state":["exists","up"]},{"osd":1,"uuid":"0e95419f-9a9d-4f7a-b55d-ecb54971918a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6801","nonce":2668156992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6803","nonce":2668156992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6807","nonce":2668156992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6805","nonce":2668156992}]},"public_addr":"172.21.15.123:6801/2668156992","cluster_addr":"172.21.15.123:6803/2668156992","heartbeat_back_addr":"172.21.15.123:6807/2668156992","heartbeat_front_addr":"172.21.15.123:6805/2668156992","state":["exists","up"]},{"osd":2,"uuid":"b6ce7ac7-be6c-4549-a356-59cb81b5b492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6801","nonce":2160189636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6803","nonce":2160189636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6807","nonce":2160189636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6805","nonce":2160189636}]},"public_addr":"172.21.15.151:6801/2160189636","cluster_addr":"172.21.15.151:6803/2160189636","heartbeat_back_addr":"172.21.15.151:6807/2160189636","heartbeat_front_addr":"172.21.15.151:6805/2160189636","state":["exists","up"]},{"osd":3,"uuid":"2fd592bf-c212-411a-afbe-a7c498e5ac93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6811","nonce":3451424073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6813","nonce":3451424073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6817","nonce":3451424073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6815","nonce":3451424073}]},"public_addr":"172.21.15.2:6811/3451424073","cluster_addr":"172.21.15.2:6813/3451424073","heartbeat_back_addr":"172.21.15.2:6817/3451424073","heartbeat_front_addr":"172.21.15.2:6815/3451424073","state":["exists","up"]},{"osd":4,"uuid":"af1905e7-e036-47c6-be59-b085bc41fede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6809","nonce":2101935762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6811","nonce":2101935762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6815","nonce":2101935762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6813","nonce":2101935762}]},"public_addr":"172.21.15.123:6809/2101935762","cluster_addr":"172.21.15.123:6811/2101935762","heartbeat_back_addr":"172.21.15.123:6815/2101935762","heartbeat_front_addr":"172.21.15.123:6813/2101935762","state":["exists","up"]},{"osd":5,"uuid":"9c166916-af78-4cc2-bd14-d1220b6e7d9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6809","nonce":3695961886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6811","nonce":3695961886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6815","nonce":3695961886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6813","nonce":3695961886}]},"public_addr":"172.21.15.151:6809/3695961886","cluster_addr":"172.21.15.151:6811/3695961886","heartbeat_back_addr":"172.21.15.151:6815/3695961886","heartbeat_front_addr":"172.21.15.151:6813/3695961886","state":["exists","up"]},{"osd":6,"uuid":"60e9bf66-2338-41e8-9280-398d4e3b322f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6819","nonce":2711893600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6821","nonce":2711893600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6825","nonce":2711893600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6823","nonce":2711893600}]},"public_addr":"172.21.15.2:6819/2711893600","cluster_addr":"172.21.15.2:6821/2711893600","heartbeat_back_addr":"172.21.15.2:6825/2711893600","heartbeat_front_addr":"172.21.15.2:6823/2711893600","state":["exists","up"]},{"osd":7,"uuid":"c93e07a1-e941-493e-8468-33429df2dff0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6817","nonce":3634155507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6819","nonce":3634155507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6823","nonce":3634155507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6821","nonce":3634155507}]},"public_addr":"172.21.15.123:6817/3634155507","cluster_addr":"172.21.15.123:6819/3634155507","heartbeat_back_addr":"172.21.15.123:6823/3634155507","heartbeat_front_addr":"172.21.15.123:6821/3634155507","state":["exists","up"]},{"osd":8,"uuid":"1155fb58-d614-4f45-91e2-5da3a93e7d9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6817","nonce":4271107010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6819","nonce":4271107010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6823","nonce":4271107010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6821","nonce":4271107010}]},"public_addr":"172.21.15.151:6817/4271107010","cluster_addr":"172.21.15.151:6819/4271107010","heartbeat_back_addr":"172.21.15.151:6823/4271107010","heartbeat_front_addr":"172.21.15.151:6821/4271107010","state":["exists","up"]},{"osd":9,"uuid":"82b34e3b-0017-4ca4-8f86-0993ca78ce7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6827","nonce":3479431021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6829","nonce":3479431021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6833","nonce":3479431021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6831","nonce":3479431021}]},"public_addr":"172.21.15.2:6827/3479431021","cluster_addr":"172.21.15.2:6829/3479431021","heartbeat_back_addr":"172.21.15.2:6833/3479431021","heartbeat_front_addr":"172.21.15.2:6831/3479431021","state":["exists","up"]},{"osd":10,"uuid":"32e2ace9-eb5d-4258-a2ac-49b782be2b5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6825","nonce":4266374435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6827","nonce":4266374435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6831","nonce":4266374435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6829","nonce":4266374435}]},"public_addr":"172.21.15.123:6825/4266374435","cluster_addr":"172.21.15.123:6827/4266374435","heartbeat_back_addr":"172.21.15.123:6831/4266374435","heartbeat_front_addr":"172.21.15.123:6829/4266374435","state":["exists","up"]},{"osd":11,"uuid":"6e3c2ead-1af6-4bda-87cd-12770c81fa67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6825","nonce":4129194698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6827","nonce":4129194698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6831","nonce":4129194698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6829","nonce":4129194698}]},"public_addr":"172.21.15.151:6825/4129194698","cluster_addr":"172.21.15.151:6827/4129194698","heartbeat_back_addr":"172.21.15.151:6831/4129194698","heartbeat_front_addr":"172.21.15.151:6829/4129194698","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:45:46.540023+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:46:10.845186+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:46:34.283112+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:47:00.072467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:47:25.315644+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:47:49.748688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:48:16.964258+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:48:42.875515+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:49:08.596500+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:49:35.450943+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:50:01.293368+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:50:27.793713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/4267809161":"2024-05-15T16:44:11.998214+0000","172.21.15.2:6801/3582988344":"2024-05-15T16:44:11.998214+0000","172.21.15.2:6800/2574015037":"2024-05-15T16:43:43.407184+0000","172.21.15.2:0/2100732571":"2024-05-15T16:44:11.998214+0000","172.21.15.2:0/2954199324":"2024-05-15T16:43:43.407184+0000","172.21.15.2:6801/2574015037":"2024-05-15T16:43:43.407184+0000","172.21.15.2:0/3171550944":"2024-05-15T16:43:43.407184+0000","172.21.15.2:0/1679761136":"2024-05-15T16:44:11.998214+0000","172.21.15.2:0/1746766591":"2024-05-15T16:43:43.407184+0000","172.21.15.2:6800/3582988344":"2024-05-15T16:44:11.998214+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T16:52:14.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.848+0000 7f5cab50c700 1 -- 172.21.15.2:0/1026038788 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f5c900b2910 msgr2=0x7f5c900b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:14.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.848+0000 7f5cab50c700 1 --2- 172.21.15.2:0/1026038788 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f5c900b2910 0x7f5c900b4dd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4079a90 tx=0x7f5c94009260 comp rx=0 tx=0).stop 2024-05-14T16:52:14.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.848+0000 7f5cab50c700 1 -- 172.21.15.2:0/1026038788 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ca406bab0 msgr2=0x7f5ca4102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:14.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.848+0000 7f5cab50c700 1 --2- 172.21.15.2:0/1026038788 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ca406bab0 0x7f5ca4102500 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5ca000d980 tx=0x7f5ca000dc90 comp rx=0 tx=0).stop 2024-05-14T16:52:14.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.848+0000 7f5cab50c700 1 -- 172.21.15.2:0/1026038788 shutdown_connections 2024-05-14T16:52:14.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.848+0000 7f5cab50c700 1 --2- 172.21.15.2:0/1026038788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ca41095a0 0x7f5ca4102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.848+0000 7f5cab50c700 1 --2- 172.21.15.2:0/1026038788 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f5c900b2910 0x7f5c900b4dd0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.848+0000 7f5cab50c700 1 --2- 172.21.15.2:0/1026038788 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5ca406bab0 0x7f5ca4102500 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.848+0000 7f5cab50c700 1 --2- 172.21.15.2:0/1026038788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ca406c470 0x7f5ca4102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:14.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.848+0000 7f5cab50c700 1 -- 172.21.15.2:0/1026038788 >> 172.21.15.2:0/1026038788 conn(0x7f5ca40793b0 msgr2=0x7f5ca4101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:14.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.848+0000 7f5cab50c700 1 -- 172.21.15.2:0/1026038788 shutdown_connections 2024-05-14T16:52:14.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:14.848+0000 7f5cab50c700 1 -- 172.21.15.2:0/1026038788 wait complete. 2024-05-14T16:52:15.482 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:15 smithi002 ceph-mon[31538]: pgmap v311: 97 pgs: 3 peering, 94 active+clean; 634 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-05-14T16:52:15.482 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:15 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/1026038788' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:52:15.512 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-14T16:52:15.512 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-14T16:52:15.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:15 smithi151 ceph-mon[36066]: pgmap v311: 97 pgs: 3 peering, 94 active+clean; 634 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-05-14T16:52:15.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:15 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/1026038788' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:52:15.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:15 smithi123 ceph-mon[35971]: pgmap v311: 97 pgs: 3 peering, 94 active+clean; 634 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-05-14T16:52:15.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:15 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/1026038788' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:52:15.764 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:52:17.408 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:17 smithi002 ceph-mon[31538]: pgmap v312: 97 pgs: 3 peering, 94 active+clean; 634 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.0 KiB/s wr, 8 op/s; 14 B/s, 0 objects/s recovering 2024-05-14T16:52:17.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.408+0000 7fdb7a4a8700 1 -- 172.21.15.2:0/1222327050 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb7406c610 msgr2=0x7fdb7410fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:17.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.408+0000 7fdb7a4a8700 1 --2- 172.21.15.2:0/1222327050 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb7406c610 0x7fdb7410fa50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c009990 tx=0x7fdb5c009ca0 comp rx=0 tx=0).stop 2024-05-14T16:52:17.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.408+0000 7fdb7a4a8700 1 -- 172.21.15.2:0/1222327050 shutdown_connections 2024-05-14T16:52:17.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.408+0000 7fdb7a4a8700 1 --2- 172.21.15.2:0/1222327050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb7410ff90 0x7fdb74112380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:17.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.408+0000 7fdb7a4a8700 1 --2- 172.21.15.2:0/1222327050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb7406bcf0 0x7fdb7406c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:17.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.408+0000 7fdb7a4a8700 1 --2- 172.21.15.2:0/1222327050 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb7406c610 0x7fdb7410fa50 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:17.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.408+0000 7fdb7a4a8700 1 -- 172.21.15.2:0/1222327050 >> 172.21.15.2:0/1222327050 conn(0x7fdb740794e0 msgr2=0x7fdb740798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:17.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.408+0000 7fdb7a4a8700 1 -- 172.21.15.2:0/1222327050 shutdown_connections 2024-05-14T16:52:17.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.408+0000 7fdb7a4a8700 1 -- 172.21.15.2:0/1222327050 wait complete. 2024-05-14T16:52:17.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.409+0000 7fdb7a4a8700 1 Processor -- start 2024-05-14T16:52:17.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.409+0000 7fdb7a4a8700 1 -- start start 2024-05-14T16:52:17.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.409+0000 7fdb7a4a8700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb7406bcf0 0x7fdb7419be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:17.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.409+0000 7fdb7a4a8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb7406c610 0x7fdb7419c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:17.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.409+0000 7fdb7a4a8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb7410ff90 0x7fdb741a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:17.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.409+0000 7fdb7a4a8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fdb74115950 con 0x7fdb7406c610 2024-05-14T16:52:17.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.409+0000 7fdb7a4a8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdb741157d0 con 0x7fdb7410ff90 2024-05-14T16:52:17.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.409+0000 7fdb7a4a8700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdb74115ad0 con 0x7fdb7406bcf0 2024-05-14T16:52:17.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.409+0000 7fdb6bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb7406c610 0x7fdb7419c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:17.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.409+0000 7fdb6bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb7406c610 0x7fdb7419c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51594/0 (socket says 172.21.15.2:51594) 2024-05-14T16:52:17.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.409+0000 7fdb6bfff700 1 -- 172.21.15.2:0/4148388325 learned_addr learned my addr 172.21.15.2:0/4148388325 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:52:17.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.409+0000 7fdb78a45700 1 --2- 172.21.15.2:0/4148388325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb7410ff90 0x7fdb741a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:17.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.409+0000 7fdb73fff700 1 --2- 172.21.15.2:0/4148388325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb7406bcf0 0x7fdb7419be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:17.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.410+0000 7fdb6bfff700 1 -- 172.21.15.2:0/4148388325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb7406bcf0 msgr2=0x7fdb7419be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:17.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.410+0000 7fdb6bfff700 1 --2- 172.21.15.2:0/4148388325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb7406bcf0 0x7fdb7419be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:17.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.410+0000 7fdb6bfff700 1 -- 172.21.15.2:0/4148388325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb7410ff90 msgr2=0x7fdb741a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:17.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.410+0000 7fdb6bfff700 1 --2- 172.21.15.2:0/4148388325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb7410ff90 0x7fdb741a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:17.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.410+0000 7fdb6bfff700 1 -- 172.21.15.2:0/4148388325 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb741a0e30 con 0x7fdb7406c610 2024-05-14T16:52:17.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.410+0000 7fdb6bfff700 1 --2- 172.21.15.2:0/4148388325 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb7406c610 0x7fdb7419c360 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c0057c0 tx=0x7fdb5c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:17.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.410+0000 7fdb71ffb700 1 -- 172.21.15.2:0/4148388325 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c027070 con 0x7fdb7406c610 2024-05-14T16:52:17.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.410+0000 7fdb71ffb700 1 -- 172.21.15.2:0/4148388325 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb5c003f20 con 0x7fdb7406c610 2024-05-14T16:52:17.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.410+0000 7fdb7a4a8700 1 -- 172.21.15.2:0/4148388325 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb741a10c0 con 0x7fdb7406c610 2024-05-14T16:52:17.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.410+0000 7fdb71ffb700 1 -- 172.21.15.2:0/4148388325 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c022ba0 con 0x7fdb7406c610 2024-05-14T16:52:17.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.410+0000 7fdb7a4a8700 1 -- 172.21.15.2:0/4148388325 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb741a89a0 con 0x7fdb7406c610 2024-05-14T16:52:17.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.411+0000 7fdb7a4a8700 1 -- 172.21.15.2:0/4148388325 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb7406a230 con 0x7fdb7406c610 2024-05-14T16:52:17.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.412+0000 7fdb71ffb700 1 -- 172.21.15.2:0/4148388325 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fdb5c02a070 con 0x7fdb7406c610 2024-05-14T16:52:17.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.413+0000 7fdb71ffb700 1 --2- 172.21.15.2:0/4148388325 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fdb600b2910 0x7fdb600b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:17.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.413+0000 7fdb73fff700 1 --2- 172.21.15.2:0/4148388325 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fdb600b2910 0x7fdb600b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:17.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.413+0000 7fdb71ffb700 1 -- 172.21.15.2:0/4148388325 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7fdb5c0e3b80 con 0x7fdb7406c610 2024-05-14T16:52:17.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.414+0000 7fdb73fff700 1 --2- 172.21.15.2:0/4148388325 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fdb600b2910 0x7fdb600b4dd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdb64005ea0 tx=0x7fdb64005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:17.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.417+0000 7fdb71ffb700 1 -- 172.21.15.2:0/4148388325 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdb5c031020 con 0x7fdb7406c610 2024-05-14T16:52:17.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:17 smithi151 ceph-mon[36066]: pgmap v312: 97 pgs: 3 peering, 94 active+clean; 634 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.0 KiB/s wr, 8 op/s; 14 B/s, 0 objects/s recovering 2024-05-14T16:52:17.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.626+0000 7fdb7a4a8700 1 -- 172.21.15.2:0/4148388325 --> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdb7419d180 con 0x7fdb600b2910 2024-05-14T16:52:17.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.634+0000 7fdb71ffb700 1 -- 172.21.15.2:0/4148388325 <== mgr.14150 v2:172.21.15.2:6800/1498936028 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258773 (secure 0 0 0) 0x7fdb7419d180 con 0x7fdb600b2910 2024-05-14T16:52:17.637 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:52:17.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.637+0000 7fdb7a4a8700 1 -- 172.21.15.2:0/4148388325 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fdb600b2910 msgr2=0x7fdb600b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:17.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.637+0000 7fdb7a4a8700 1 --2- 172.21.15.2:0/4148388325 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fdb600b2910 0x7fdb600b4dd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdb64005ea0 tx=0x7fdb64005e30 comp rx=0 tx=0).stop 2024-05-14T16:52:17.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.637+0000 7fdb7a4a8700 1 -- 172.21.15.2:0/4148388325 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb7406c610 msgr2=0x7fdb7419c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:17.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.637+0000 7fdb7a4a8700 1 --2- 172.21.15.2:0/4148388325 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb7406c610 0x7fdb7419c360 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c0057c0 tx=0x7fdb5c0057f0 comp rx=0 tx=0).stop 2024-05-14T16:52:17.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.637+0000 7fdb7a4a8700 1 -- 172.21.15.2:0/4148388325 shutdown_connections 2024-05-14T16:52:17.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.637+0000 7fdb7a4a8700 1 --2- 172.21.15.2:0/4148388325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb7410ff90 0x7fdb741a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:17.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.637+0000 7fdb7a4a8700 1 --2- 172.21.15.2:0/4148388325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb7406bcf0 0x7fdb7419be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:17.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.637+0000 7fdb7a4a8700 1 --2- 172.21.15.2:0/4148388325 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fdb600b2910 0x7fdb600b4dd0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:17.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.637+0000 7fdb7a4a8700 1 --2- 172.21.15.2:0/4148388325 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb7406c610 0x7fdb7419c360 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:17.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.637+0000 7fdb7a4a8700 1 -- 172.21.15.2:0/4148388325 >> 172.21.15.2:0/4148388325 conn(0x7fdb740794e0 msgr2=0x7fdb741113e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:17.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.637+0000 7fdb7a4a8700 1 -- 172.21.15.2:0/4148388325 shutdown_connections 2024-05-14T16:52:17.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:17.637+0000 7fdb7a4a8700 1 -- 172.21.15.2:0/4148388325 wait complete. 2024-05-14T16:52:17.640 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-05-14T16:52:17.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:17 smithi123 ceph-mon[35971]: pgmap v312: 97 pgs: 3 peering, 94 active+clean; 634 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.0 KiB/s wr, 8 op/s; 14 B/s, 0 objects/s recovering 2024-05-14T16:52:18.315 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":312,"stamp":"2024-05-14T16:52:16.095016+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649599,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":244,"num_read_kb":209,"num_write":283,"num_write_kb":2256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2951957,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":229,"ondisk_log_size":229,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":293,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":384628,"kb_used_data":10476,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124557196,"statfs":{"total":1151940427776,"available":1151546568704,"internally_reserved":0,"allocated":10727424,"data_stored":6918056,"data_compressed":108486,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27030,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":53,"num_read_kb":58,"num_write":14,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002253"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215138+0000","last_change":"2024-05-14T16:51:32.298580+0000","last_active":"2024-05-14T16:52:13.215138+0000","last_peered":"2024-05-14T16:52:13.215138+0000","last_clean":"2024-05-14T16:52:13.215138+0000","last_became_active":"2024-05-14T16:51:32.296945+0000","last_became_peered":"2024-05-14T16:51:32.296945+0000","last_unstale":"2024-05-14T16:52:13.215138+0000","last_undegraded":"2024-05-14T16:52:13.215138+0000","last_fullsized":"2024-05-14T16:52:13.215138+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:50:17.664928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215759+0000","last_change":"2024-05-14T16:51:32.297618+0000","last_active":"2024-05-14T16:52:13.215759+0000","last_peered":"2024-05-14T16:52:13.215759+0000","last_clean":"2024-05-14T16:52:13.215759+0000","last_became_active":"2024-05-14T16:51:32.297260+0000","last_became_peered":"2024-05-14T16:51:32.297260+0000","last_unstale":"2024-05-14T16:52:13.215759+0000","last_undegraded":"2024-05-14T16:52:13.215759+0000","last_fullsized":"2024-05-14T16:52:13.215759+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:43:50.542587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215551+0000","last_change":"2024-05-14T16:51:32.295525+0000","last_active":"2024-05-14T16:52:13.215551+0000","last_peered":"2024-05-14T16:52:13.215551+0000","last_clean":"2024-05-14T16:52:13.215551+0000","last_became_active":"2024-05-14T16:51:32.295215+0000","last_became_peered":"2024-05-14T16:51:32.295215+0000","last_unstale":"2024-05-14T16:52:13.215551+0000","last_undegraded":"2024-05-14T16:52:13.215551+0000","last_fullsized":"2024-05-14T16:52:13.215551+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:40:05.516280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-14T16:52:12.209932+0000","last_change":"2024-05-14T16:51:32.307883+0000","last_active":"2024-05-14T16:52:12.209932+0000","last_peered":"2024-05-14T16:52:12.209932+0000","last_clean":"2024-05-14T16:52:12.209932+0000","last_became_active":"2024-05-14T16:51:32.307499+0000","last_became_peered":"2024-05-14T16:51:32.307499+0000","last_unstale":"2024-05-14T16:52:12.209932+0000","last_undegraded":"2024-05-14T16:52:12.209932+0000","last_fullsized":"2024-05-14T16:52:12.209932+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:30:59.978059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:51:34.298243+0000","last_change":"2024-05-14T16:51:32.294429+0000","last_active":"2024-05-14T16:51:34.298243+0000","last_peered":"2024-05-14T16:51:34.298243+0000","last_clean":"2024-05-14T16:51:34.298243+0000","last_became_active":"2024-05-14T16:51:32.294088+0000","last_became_peered":"2024-05-14T16:51:32.294088+0000","last_unstale":"2024-05-14T16:51:34.298243+0000","last_undegraded":"2024-05-14T16:51:34.298243+0000","last_fullsized":"2024-05-14T16:51:34.298243+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:25:19.383586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.445204+0000","last_change":"2024-05-14T16:51:32.296169+0000","last_active":"2024-05-14T16:52:13.445204+0000","last_peered":"2024-05-14T16:52:13.445204+0000","last_clean":"2024-05-14T16:52:13.445204+0000","last_became_active":"2024-05-14T16:51:32.295819+0000","last_became_peered":"2024-05-14T16:51:32.295819+0000","last_unstale":"2024-05-14T16:52:13.445204+0000","last_undegraded":"2024-05-14T16:52:13.445204+0000","last_fullsized":"2024-05-14T16:52:13.445204+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:56:23.932039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.214939+0000","last_change":"2024-05-14T16:51:32.309609+0000","last_active":"2024-05-14T16:52:13.214939+0000","last_peered":"2024-05-14T16:52:13.214939+0000","last_clean":"2024-05-14T16:52:13.214939+0000","last_became_active":"2024-05-14T16:51:32.309175+0000","last_became_peered":"2024-05-14T16:51:32.309175+0000","last_unstale":"2024-05-14T16:52:13.214939+0000","last_undegraded":"2024-05-14T16:52:13.214939+0000","last_fullsized":"2024-05-14T16:52:13.214939+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:06:46.717592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":2,"reported_epoch":79,"state":"peering","last_fresh":"2024-05-14T16:51:34.300883+0000","last_change":"2024-05-14T16:52:12.314039+0000","last_active":"2024-05-14T16:52:12.314039+0000","last_peered":"2024-05-14T16:51:34.300883+0000","last_clean":"2024-05-14T16:51:34.300883+0000","last_became_active":"2024-05-14T16:51:32.299523+0000","last_became_peered":"2024-05-14T16:51:32.299523+0000","last_unstale":"2024-05-14T16:51:34.300883+0000","last_undegraded":"2024-05-14T16:51:34.300883+0000","last_fullsized":"2024-05-14T16:51:34.300883+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-14T16:52:12.207459+0000","last_change":"2024-05-14T16:51:32.307571+0000","last_active":"2024-05-14T16:52:12.207459+0000","last_peered":"2024-05-14T16:52:12.207459+0000","last_clean":"2024-05-14T16:52:12.207459+0000","last_became_active":"2024-05-14T16:51:32.307151+0000","last_became_peered":"2024-05-14T16:51:32.307151+0000","last_unstale":"2024-05-14T16:52:12.207459+0000","last_undegraded":"2024-05-14T16:52:12.207459+0000","last_fullsized":"2024-05-14T16:52:12.207459+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:17:26.325393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.445700+0000","last_change":"2024-05-14T16:51:32.298773+0000","last_active":"2024-05-14T16:52:13.445700+0000","last_peered":"2024-05-14T16:52:13.445700+0000","last_clean":"2024-05-14T16:52:13.445700+0000","last_became_active":"2024-05-14T16:51:32.298283+0000","last_became_peered":"2024-05-14T16:51:32.298283+0000","last_unstale":"2024-05-14T16:52:13.445700+0000","last_undegraded":"2024-05-14T16:52:13.445700+0000","last_fullsized":"2024-05-14T16:52:13.445700+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:52:18.471201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215163+0000","last_change":"2024-05-14T16:51:32.302443+0000","last_active":"2024-05-14T16:52:13.215163+0000","last_peered":"2024-05-14T16:52:13.215163+0000","last_clean":"2024-05-14T16:52:13.215163+0000","last_became_active":"2024-05-14T16:51:32.302034+0000","last_became_peered":"2024-05-14T16:51:32.302034+0000","last_unstale":"2024-05-14T16:52:13.215163+0000","last_undegraded":"2024-05-14T16:52:13.215163+0000","last_fullsized":"2024-05-14T16:52:13.215163+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:53:08.041293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-14T16:52:12.210042+0000","last_change":"2024-05-14T16:51:32.304837+0000","last_active":"2024-05-14T16:52:12.210042+0000","last_peered":"2024-05-14T16:52:12.210042+0000","last_clean":"2024-05-14T16:52:12.210042+0000","last_became_active":"2024-05-14T16:51:32.304500+0000","last_became_peered":"2024-05-14T16:51:32.304500+0000","last_unstale":"2024-05-14T16:52:12.210042+0000","last_undegraded":"2024-05-14T16:52:12.210042+0000","last_fullsized":"2024-05-14T16:52:12.210042+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:32:02.713044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.921639+0000","last_change":"2024-05-14T16:51:32.299247+0000","last_active":"2024-05-14T16:52:13.921639+0000","last_peered":"2024-05-14T16:52:13.921639+0000","last_clean":"2024-05-14T16:52:13.921639+0000","last_became_active":"2024-05-14T16:51:32.298925+0000","last_became_peered":"2024-05-14T16:51:32.298925+0000","last_unstale":"2024-05-14T16:52:13.921639+0000","last_undegraded":"2024-05-14T16:52:13.921639+0000","last_fullsized":"2024-05-14T16:52:13.921639+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:10:38.847877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215377+0000","last_change":"2024-05-14T16:51:32.293743+0000","last_active":"2024-05-14T16:52:13.215377+0000","last_peered":"2024-05-14T16:52:13.215377+0000","last_clean":"2024-05-14T16:52:13.215377+0000","last_became_active":"2024-05-14T16:51:32.293429+0000","last_became_peered":"2024-05-14T16:51:32.293429+0000","last_unstale":"2024-05-14T16:52:13.215377+0000","last_undegraded":"2024-05-14T16:52:13.215377+0000","last_fullsized":"2024-05-14T16:52:13.215377+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:12:33.909940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215027+0000","last_change":"2024-05-14T16:51:32.304940+0000","last_active":"2024-05-14T16:52:13.215027+0000","last_peered":"2024-05-14T16:52:13.215027+0000","last_clean":"2024-05-14T16:52:13.215027+0000","last_became_active":"2024-05-14T16:51:32.304482+0000","last_became_peered":"2024-05-14T16:51:32.304482+0000","last_unstale":"2024-05-14T16:52:13.215027+0000","last_undegraded":"2024-05-14T16:52:13.215027+0000","last_fullsized":"2024-05-14T16:52:13.215027+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:54:17.771620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-14T16:52:12.206815+0000","last_change":"2024-05-14T16:51:32.301784+0000","last_active":"2024-05-14T16:52:12.206815+0000","last_peered":"2024-05-14T16:52:12.206815+0000","last_clean":"2024-05-14T16:52:12.206815+0000","last_became_active":"2024-05-14T16:51:32.301472+0000","last_became_peered":"2024-05-14T16:51:32.301472+0000","last_unstale":"2024-05-14T16:52:12.206815+0000","last_undegraded":"2024-05-14T16:52:12.206815+0000","last_fullsized":"2024-05-14T16:52:12.206815+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:10:56.735424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215272+0000","last_change":"2024-05-14T16:51:32.297951+0000","last_active":"2024-05-14T16:52:13.215272+0000","last_peered":"2024-05-14T16:52:13.215272+0000","last_clean":"2024-05-14T16:52:13.215272+0000","last_became_active":"2024-05-14T16:51:32.296602+0000","last_became_peered":"2024-05-14T16:51:32.296602+0000","last_unstale":"2024-05-14T16:52:13.215272+0000","last_undegraded":"2024-05-14T16:52:13.215272+0000","last_fullsized":"2024-05-14T16:52:13.215272+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:16:36.039077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-14T16:52:12.210182+0000","last_change":"2024-05-14T16:51:32.290398+0000","last_active":"2024-05-14T16:52:12.210182+0000","last_peered":"2024-05-14T16:52:12.210182+0000","last_clean":"2024-05-14T16:52:12.210182+0000","last_became_active":"2024-05-14T16:51:32.290083+0000","last_became_peered":"2024-05-14T16:51:32.290083+0000","last_unstale":"2024-05-14T16:52:12.210182+0000","last_undegraded":"2024-05-14T16:52:12.210182+0000","last_fullsized":"2024-05-14T16:52:12.210182+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:48:05.795209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:51:34.298714+0000","last_change":"2024-05-14T16:51:32.294398+0000","last_active":"2024-05-14T16:51:34.298714+0000","last_peered":"2024-05-14T16:51:34.298714+0000","last_clean":"2024-05-14T16:51:34.298714+0000","last_became_active":"2024-05-14T16:51:32.294080+0000","last_became_peered":"2024-05-14T16:51:32.294080+0000","last_unstale":"2024-05-14T16:51:34.298714+0000","last_undegraded":"2024-05-14T16:51:34.298714+0000","last_fullsized":"2024-05-14T16:51:34.298714+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:15:03.859045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":34,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-14T16:52:12.209793+0000","last_change":"2024-05-14T16:51:29.274482+0000","last_active":"2024-05-14T16:52:12.209793+0000","last_peered":"2024-05-14T16:52:12.209793+0000","last_clean":"2024-05-14T16:52:12.209793+0000","last_became_active":"2024-05-14T16:51:29.274152+0000","last_became_peered":"2024-05-14T16:51:29.274152+0000","last_unstale":"2024-05-14T16:52:12.209793+0000","last_undegraded":"2024-05-14T16:52:12.209793+0000","last_fullsized":"2024-05-14T16:52:12.209793+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:05:38.274248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:51:34.301045+0000","last_change":"2024-05-14T16:51:32.300956+0000","last_active":"2024-05-14T16:51:34.301045+0000","last_peered":"2024-05-14T16:51:34.301045+0000","last_clean":"2024-05-14T16:51:34.301045+0000","last_became_active":"2024-05-14T16:51:32.300546+0000","last_became_peered":"2024-05-14T16:51:32.300546+0000","last_unstale":"2024-05-14T16:51:34.301045+0000","last_undegraded":"2024-05-14T16:51:34.301045+0000","last_fullsized":"2024-05-14T16:51:34.301045+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:54:09.181036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:51:34.304585+0000","last_change":"2024-05-14T16:51:29.345206+0000","last_active":"2024-05-14T16:51:34.304585+0000","last_peered":"2024-05-14T16:51:34.304585+0000","last_clean":"2024-05-14T16:51:34.304585+0000","last_became_active":"2024-05-14T16:51:29.344699+0000","last_became_peered":"2024-05-14T16:51:29.344699+0000","last_unstale":"2024-05-14T16:51:34.304585+0000","last_undegraded":"2024-05-14T16:51:34.304585+0000","last_fullsized":"2024-05-14T16:51:34.304585+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:14:02.364661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215868+0000","last_change":"2024-05-14T16:51:32.297522+0000","last_active":"2024-05-14T16:52:13.215868+0000","last_peered":"2024-05-14T16:52:13.215868+0000","last_clean":"2024-05-14T16:52:13.215868+0000","last_became_active":"2024-05-14T16:51:32.296866+0000","last_became_peered":"2024-05-14T16:51:32.296866+0000","last_unstale":"2024-05-14T16:52:13.215868+0000","last_undegraded":"2024-05-14T16:52:13.215868+0000","last_fullsized":"2024-05-14T16:52:13.215868+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:49:43.409575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:51:47.987913+0000","last_change":"2024-05-14T16:51:29.280632+0000","last_active":"2024-05-14T16:51:47.987913+0000","last_peered":"2024-05-14T16:51:47.987913+0000","last_clean":"2024-05-14T16:51:47.987913+0000","last_became_active":"2024-05-14T16:51:29.280246+0000","last_became_peered":"2024-05-14T16:51:29.280246+0000","last_unstale":"2024-05-14T16:51:47.987913+0000","last_undegraded":"2024-05-14T16:51:47.987913+0000","last_fullsized":"2024-05-14T16:51:47.987913+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:34:47.925344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-14T16:52:12.207113+0000","last_change":"2024-05-14T16:51:32.298284+0000","last_active":"2024-05-14T16:52:12.207113+0000","last_peered":"2024-05-14T16:52:12.207113+0000","last_clean":"2024-05-14T16:52:12.207113+0000","last_became_active":"2024-05-14T16:51:32.294202+0000","last_became_peered":"2024-05-14T16:51:32.294202+0000","last_unstale":"2024-05-14T16:52:12.207113+0000","last_undegraded":"2024-05-14T16:52:12.207113+0000","last_fullsized":"2024-05-14T16:52:12.207113+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:27:37.561275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:51:34.306143+0000","last_change":"2024-05-14T16:51:29.276227+0000","last_active":"2024-05-14T16:51:34.306143+0000","last_peered":"2024-05-14T16:51:34.306143+0000","last_clean":"2024-05-14T16:51:34.306143+0000","last_became_active":"2024-05-14T16:51:29.275837+0000","last_became_peered":"2024-05-14T16:51:29.275837+0000","last_unstale":"2024-05-14T16:51:34.306143+0000","last_undegraded":"2024-05-14T16:51:34.306143+0000","last_fullsized":"2024-05-14T16:51:34.306143+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:03:29.402985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-14T16:52:12.214531+0000","last_change":"2024-05-14T16:51:32.308173+0000","last_active":"2024-05-14T16:52:12.214531+0000","last_peered":"2024-05-14T16:52:12.214531+0000","last_clean":"2024-05-14T16:52:12.214531+0000","last_became_active":"2024-05-14T16:51:32.307135+0000","last_became_peered":"2024-05-14T16:51:32.307135+0000","last_unstale":"2024-05-14T16:52:12.214531+0000","last_undegraded":"2024-05-14T16:52:12.214531+0000","last_fullsized":"2024-05-14T16:52:12.214531+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:01:46.579968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"78'6","reported_seq":55,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-14T16:52:12.214280+0000","last_change":"2024-05-14T16:51:29.279486+0000","last_active":"2024-05-14T16:52:12.214280+0000","last_peered":"2024-05-14T16:52:12.214280+0000","last_clean":"2024-05-14T16:52:12.214280+0000","last_became_active":"2024-05-14T16:51:29.279133+0000","last_became_peered":"2024-05-14T16:51:29.279133+0000","last_unstale":"2024-05-14T16:52:12.214280+0000","last_undegraded":"2024-05-14T16:52:12.214280+0000","last_fullsized":"2024-05-14T16:52:12.214280+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:26:18.620421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:51:34.298487+0000","last_change":"2024-05-14T16:51:32.299852+0000","last_active":"2024-05-14T16:51:34.298487+0000","last_peered":"2024-05-14T16:51:34.298487+0000","last_clean":"2024-05-14T16:51:34.298487+0000","last_became_active":"2024-05-14T16:51:32.299541+0000","last_became_peered":"2024-05-14T16:51:32.299541+0000","last_unstale":"2024-05-14T16:51:34.298487+0000","last_undegraded":"2024-05-14T16:51:34.298487+0000","last_fullsized":"2024-05-14T16:51:34.298487+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:36:38.191009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.445109+0000","last_change":"2024-05-14T16:51:29.275269+0000","last_active":"2024-05-14T16:52:13.445109+0000","last_peered":"2024-05-14T16:52:13.445109+0000","last_clean":"2024-05-14T16:52:13.445109+0000","last_became_active":"2024-05-14T16:51:29.274792+0000","last_became_peered":"2024-05-14T16:51:29.274792+0000","last_unstale":"2024-05-14T16:52:13.445109+0000","last_undegraded":"2024-05-14T16:52:13.445109+0000","last_fullsized":"2024-05-14T16:52:13.445109+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:17:08.925289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-14T16:52:12.214115+0000","last_change":"2024-05-14T16:51:32.306536+0000","last_active":"2024-05-14T16:52:12.214115+0000","last_peered":"2024-05-14T16:52:12.214115+0000","last_clean":"2024-05-14T16:52:12.214115+0000","last_became_active":"2024-05-14T16:51:32.304976+0000","last_became_peered":"2024-05-14T16:51:32.304976+0000","last_unstale":"2024-05-14T16:52:12.214115+0000","last_undegraded":"2024-05-14T16:52:12.214115+0000","last_fullsized":"2024-05-14T16:52:12.214115+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:48:13.284022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.921481+0000","last_change":"2024-05-14T16:51:29.266783+0000","last_active":"2024-05-14T16:52:13.921481+0000","last_peered":"2024-05-14T16:52:13.921481+0000","last_clean":"2024-05-14T16:52:13.921481+0000","last_became_active":"2024-05-14T16:51:29.266409+0000","last_became_peered":"2024-05-14T16:51:29.266409+0000","last_unstale":"2024-05-14T16:52:13.921481+0000","last_undegraded":"2024-05-14T16:52:13.921481+0000","last_fullsized":"2024-05-14T16:52:13.921481+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:33:14.981709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:51:34.299764+0000","last_change":"2024-05-14T16:51:32.289288+0000","last_active":"2024-05-14T16:51:34.299764+0000","last_peered":"2024-05-14T16:51:34.299764+0000","last_clean":"2024-05-14T16:51:34.299764+0000","last_became_active":"2024-05-14T16:51:32.288920+0000","last_became_peered":"2024-05-14T16:51:32.288920+0000","last_unstale":"2024-05-14T16:51:34.299764+0000","last_undegraded":"2024-05-14T16:51:34.299764+0000","last_fullsized":"2024-05-14T16:51:34.299764+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:23:43.659417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-14T16:52:12.214592+0000","last_change":"2024-05-14T16:51:29.273926+0000","last_active":"2024-05-14T16:52:12.214592+0000","last_peered":"2024-05-14T16:52:12.214592+0000","last_clean":"2024-05-14T16:52:12.214592+0000","last_became_active":"2024-05-14T16:51:29.273059+0000","last_became_peered":"2024-05-14T16:51:29.273059+0000","last_unstale":"2024-05-14T16:52:12.214592+0000","last_undegraded":"2024-05-14T16:52:12.214592+0000","last_fullsized":"2024-05-14T16:52:12.214592+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:46:27.800481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-14T16:52:12.209545+0000","last_change":"2024-05-14T16:51:32.287554+0000","last_active":"2024-05-14T16:52:12.209545+0000","last_peered":"2024-05-14T16:52:12.209545+0000","last_clean":"2024-05-14T16:52:12.209545+0000","last_became_active":"2024-05-14T16:51:32.287204+0000","last_became_peered":"2024-05-14T16:51:32.287204+0000","last_unstale":"2024-05-14T16:52:12.209545+0000","last_undegraded":"2024-05-14T16:52:12.209545+0000","last_fullsized":"2024-05-14T16:52:12.209545+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:29:40.675378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":52,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.445296+0000","last_change":"2024-05-14T16:51:29.274819+0000","last_active":"2024-05-14T16:52:13.445296+0000","last_peered":"2024-05-14T16:52:13.445296+0000","last_clean":"2024-05-14T16:52:13.445296+0000","last_became_active":"2024-05-14T16:51:29.274454+0000","last_became_peered":"2024-05-14T16:51:29.274454+0000","last_unstale":"2024-05-14T16:52:13.445296+0000","last_undegraded":"2024-05-14T16:52:13.445296+0000","last_fullsized":"2024-05-14T16:52:13.445296+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:36:30.496064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-14T16:52:12.207276+0000","last_change":"2024-05-14T16:51:32.295732+0000","last_active":"2024-05-14T16:52:12.207276+0000","last_peered":"2024-05-14T16:52:12.207276+0000","last_clean":"2024-05-14T16:52:12.207276+0000","last_became_active":"2024-05-14T16:51:32.294919+0000","last_became_peered":"2024-05-14T16:51:32.294919+0000","last_unstale":"2024-05-14T16:52:12.207276+0000","last_undegraded":"2024-05-14T16:52:12.207276+0000","last_fullsized":"2024-05-14T16:52:12.207276+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:08:54.566601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215472+0000","last_change":"2024-05-14T16:51:29.279338+0000","last_active":"2024-05-14T16:52:13.215472+0000","last_peered":"2024-05-14T16:52:13.215472+0000","last_clean":"2024-05-14T16:52:13.215472+0000","last_became_active":"2024-05-14T16:51:29.278928+0000","last_became_peered":"2024-05-14T16:51:29.278928+0000","last_unstale":"2024-05-14T16:52:13.215472+0000","last_undegraded":"2024-05-14T16:52:13.215472+0000","last_fullsized":"2024-05-14T16:52:13.215472+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:24:34.803515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215986+0000","last_change":"2024-05-14T16:51:32.305294+0000","last_active":"2024-05-14T16:52:13.215986+0000","last_peered":"2024-05-14T16:52:13.215986+0000","last_clean":"2024-05-14T16:52:13.215986+0000","last_became_active":"2024-05-14T16:51:32.304563+0000","last_became_peered":"2024-05-14T16:51:32.304563+0000","last_unstale":"2024-05-14T16:52:13.215986+0000","last_undegraded":"2024-05-14T16:52:13.215986+0000","last_fullsized":"2024-05-14T16:52:13.215986+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:56:06.259475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-14T16:52:12.214660+0000","last_change":"2024-05-14T16:51:29.345400+0000","last_active":"2024-05-14T16:52:12.214660+0000","last_peered":"2024-05-14T16:52:12.214660+0000","last_clean":"2024-05-14T16:52:12.214660+0000","last_became_active":"2024-05-14T16:51:29.345010+0000","last_became_peered":"2024-05-14T16:51:29.345010+0000","last_unstale":"2024-05-14T16:52:12.214660+0000","last_undegraded":"2024-05-14T16:52:12.214660+0000","last_fullsized":"2024-05-14T16:52:12.214660+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:24:22.933005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"peering","last_fresh":"2024-05-14T16:51:34.303803+0000","last_change":"2024-05-14T16:52:12.207691+0000","last_active":"2024-05-14T16:52:12.207691+0000","last_peered":"2024-05-14T16:51:34.303803+0000","last_clean":"2024-05-14T16:51:34.303803+0000","last_became_active":"2024-05-14T16:51:32.304626+0000","last_became_peered":"2024-05-14T16:51:32.304626+0000","last_unstale":"2024-05-14T16:51:34.303803+0000","last_undegraded":"2024-05-14T16:51:34.303803+0000","last_fullsized":"2024-05-14T16:51:34.303803+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6,8],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'28","reported_seq":107,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.447996+0000","last_change":"2024-05-14T16:51:29.277591+0000","last_active":"2024-05-14T16:52:13.447996+0000","last_peered":"2024-05-14T16:52:13.447996+0000","last_clean":"2024-05-14T16:52:13.447996+0000","last_became_active":"2024-05-14T16:51:29.276612+0000","last_became_peered":"2024-05-14T16:51:29.276612+0000","last_unstale":"2024-05-14T16:52:13.447996+0000","last_undegraded":"2024-05-14T16:52:13.447996+0000","last_fullsized":"2024-05-14T16:52:13.447996+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:22:28.934173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54996,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":53,"num_read_kb":74,"num_write":29,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:51:34.299785+0000","last_change":"2024-05-14T16:51:32.302321+0000","last_active":"2024-05-14T16:51:34.299785+0000","last_peered":"2024-05-14T16:51:34.299785+0000","last_clean":"2024-05-14T16:51:34.299785+0000","last_became_active":"2024-05-14T16:51:32.301973+0000","last_became_peered":"2024-05-14T16:51:32.301973+0000","last_unstale":"2024-05-14T16:51:34.299785+0000","last_undegraded":"2024-05-14T16:51:34.299785+0000","last_fullsized":"2024-05-14T16:51:34.299785+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:49:24.054379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:51:47.988060+0000","last_change":"2024-05-14T16:51:29.278776+0000","last_active":"2024-05-14T16:51:47.988060+0000","last_peered":"2024-05-14T16:51:47.988060+0000","last_clean":"2024-05-14T16:51:47.988060+0000","last_became_active":"2024-05-14T16:51:29.278395+0000","last_became_peered":"2024-05-14T16:51:29.278395+0000","last_unstale":"2024-05-14T16:51:47.988060+0000","last_undegraded":"2024-05-14T16:51:47.988060+0000","last_fullsized":"2024-05-14T16:51:47.988060+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:29:20.942783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.921447+0000","last_change":"2024-05-14T16:51:32.292107+0000","last_active":"2024-05-14T16:52:13.921447+0000","last_peered":"2024-05-14T16:52:13.921447+0000","last_clean":"2024-05-14T16:52:13.921447+0000","last_became_active":"2024-05-14T16:51:32.291729+0000","last_became_peered":"2024-05-14T16:51:32.291729+0000","last_unstale":"2024-05-14T16:52:13.921447+0000","last_undegraded":"2024-05-14T16:52:13.921447+0000","last_fullsized":"2024-05-14T16:52:13.921447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:30:15.440599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-14T16:52:12.214202+0000","last_change":"2024-05-14T16:51:29.274862+0000","last_active":"2024-05-14T16:52:12.214202+0000","last_peered":"2024-05-14T16:52:12.214202+0000","last_clean":"2024-05-14T16:52:12.214202+0000","last_became_active":"2024-05-14T16:51:29.274527+0000","last_became_peered":"2024-05-14T16:51:29.274527+0000","last_unstale":"2024-05-14T16:52:12.214202+0000","last_undegraded":"2024-05-14T16:52:12.214202+0000","last_fullsized":"2024-05-14T16:52:12.214202+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:43:15.634585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215503+0000","last_change":"2024-05-14T16:51:32.306158+0000","last_active":"2024-05-14T16:52:13.215503+0000","last_peered":"2024-05-14T16:52:13.215503+0000","last_clean":"2024-05-14T16:52:13.215503+0000","last_became_active":"2024-05-14T16:51:32.305834+0000","last_became_peered":"2024-05-14T16:51:32.305834+0000","last_unstale":"2024-05-14T16:52:13.215503+0000","last_undegraded":"2024-05-14T16:52:13.215503+0000","last_fullsized":"2024-05-14T16:52:13.215503+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:31:47.645862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'3","reported_seq":51,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-14T16:52:12.209298+0000","last_change":"2024-05-14T16:51:29.270735+0000","last_active":"2024-05-14T16:52:12.209298+0000","last_peered":"2024-05-14T16:52:12.209298+0000","last_clean":"2024-05-14T16:52:12.209298+0000","last_became_active":"2024-05-14T16:51:29.270362+0000","last_became_peered":"2024-05-14T16:51:29.270362+0000","last_unstale":"2024-05-14T16:52:12.209298+0000","last_undegraded":"2024-05-14T16:52:12.209298+0000","last_fullsized":"2024-05-14T16:52:12.209298+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:40:12.288655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.214882+0000","last_change":"2024-05-14T16:51:32.305007+0000","last_active":"2024-05-14T16:52:13.214882+0000","last_peered":"2024-05-14T16:52:13.214882+0000","last_clean":"2024-05-14T16:52:13.214882+0000","last_became_active":"2024-05-14T16:51:32.303927+0000","last_became_peered":"2024-05-14T16:51:32.303927+0000","last_unstale":"2024-05-14T16:52:13.214882+0000","last_undegraded":"2024-05-14T16:52:13.214882+0000","last_fullsized":"2024-05-14T16:52:13.214882+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:42:28.018492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:51:47.989054+0000","last_change":"2024-05-14T16:51:29.345902+0000","last_active":"2024-05-14T16:51:47.989054+0000","last_peered":"2024-05-14T16:51:47.989054+0000","last_clean":"2024-05-14T16:51:47.989054+0000","last_became_active":"2024-05-14T16:51:29.344904+0000","last_became_peered":"2024-05-14T16:51:29.344904+0000","last_unstale":"2024-05-14T16:51:47.989054+0000","last_undegraded":"2024-05-14T16:51:47.989054+0000","last_fullsized":"2024-05-14T16:51:47.989054+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:56:22.209067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-14T16:52:12.213891+0000","last_change":"2024-05-14T16:51:32.306459+0000","last_active":"2024-05-14T16:52:12.213891+0000","last_peered":"2024-05-14T16:52:12.213891+0000","last_clean":"2024-05-14T16:52:12.213891+0000","last_became_active":"2024-05-14T16:51:32.304897+0000","last_became_peered":"2024-05-14T16:51:32.304897+0000","last_unstale":"2024-05-14T16:52:12.213891+0000","last_undegraded":"2024-05-14T16:52:12.213891+0000","last_fullsized":"2024-05-14T16:52:12.213891+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:09:51.216631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:51:49.984231+0000","last_change":"2024-05-14T16:51:29.275219+0000","last_active":"2024-05-14T16:51:49.984231+0000","last_peered":"2024-05-14T16:51:49.984231+0000","last_clean":"2024-05-14T16:51:49.984231+0000","last_became_active":"2024-05-14T16:51:29.274816+0000","last_became_peered":"2024-05-14T16:51:29.274816+0000","last_unstale":"2024-05-14T16:51:49.984231+0000","last_undegraded":"2024-05-14T16:51:49.984231+0000","last_fullsized":"2024-05-14T16:51:49.984231+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:55:49.667387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215144+0000","last_change":"2024-05-14T16:51:32.299284+0000","last_active":"2024-05-14T16:52:13.215144+0000","last_peered":"2024-05-14T16:52:13.215144+0000","last_clean":"2024-05-14T16:52:13.215144+0000","last_became_active":"2024-05-14T16:51:32.298933+0000","last_became_peered":"2024-05-14T16:51:32.298933+0000","last_unstale":"2024-05-14T16:52:13.215144+0000","last_undegraded":"2024-05-14T16:52:13.215144+0000","last_fullsized":"2024-05-14T16:52:13.215144+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:35:42.266665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-14T16:52:12.209732+0000","last_change":"2024-05-14T16:51:29.345454+0000","last_active":"2024-05-14T16:52:12.209732+0000","last_peered":"2024-05-14T16:52:12.209732+0000","last_clean":"2024-05-14T16:52:12.209732+0000","last_became_active":"2024-05-14T16:51:29.345074+0000","last_became_peered":"2024-05-14T16:51:29.345074+0000","last_unstale":"2024-05-14T16:52:12.209732+0000","last_undegraded":"2024-05-14T16:52:12.209732+0000","last_fullsized":"2024-05-14T16:52:12.209732+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:56:11.661144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.445213+0000","last_change":"2024-05-14T16:51:32.288804+0000","last_active":"2024-05-14T16:52:13.445213+0000","last_peered":"2024-05-14T16:52:13.445213+0000","last_clean":"2024-05-14T16:52:13.445213+0000","last_became_active":"2024-05-14T16:51:32.288441+0000","last_became_peered":"2024-05-14T16:51:32.288441+0000","last_unstale":"2024-05-14T16:52:13.445213+0000","last_undegraded":"2024-05-14T16:52:13.445213+0000","last_fullsized":"2024-05-14T16:52:13.445213+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:28:31.720342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215130+0000","last_change":"2024-05-14T16:51:29.272032+0000","last_active":"2024-05-14T16:52:13.215130+0000","last_peered":"2024-05-14T16:52:13.215130+0000","last_clean":"2024-05-14T16:52:13.215130+0000","last_became_active":"2024-05-14T16:51:29.271677+0000","last_became_peered":"2024-05-14T16:51:29.271677+0000","last_unstale":"2024-05-14T16:52:13.215130+0000","last_undegraded":"2024-05-14T16:52:13.215130+0000","last_fullsized":"2024-05-14T16:52:13.215130+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:39:41.377591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.445317+0000","last_change":"2024-05-14T16:51:32.295159+0000","last_active":"2024-05-14T16:52:13.445317+0000","last_peered":"2024-05-14T16:52:13.445317+0000","last_clean":"2024-05-14T16:52:13.445317+0000","last_became_active":"2024-05-14T16:51:32.294812+0000","last_became_peered":"2024-05-14T16:51:32.294812+0000","last_unstale":"2024-05-14T16:52:13.445317+0000","last_undegraded":"2024-05-14T16:52:13.445317+0000","last_fullsized":"2024-05-14T16:52:13.445317+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:08:22.411800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-14T16:52:12.215039+0000","last_change":"2024-05-14T16:51:32.306582+0000","last_active":"2024-05-14T16:52:12.215039+0000","last_peered":"2024-05-14T16:52:12.215039+0000","last_clean":"2024-05-14T16:52:12.215039+0000","last_became_active":"2024-05-14T16:51:32.306209+0000","last_became_peered":"2024-05-14T16:51:32.306209+0000","last_unstale":"2024-05-14T16:52:12.215039+0000","last_undegraded":"2024-05-14T16:52:12.215039+0000","last_fullsized":"2024-05-14T16:52:12.215039+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:30:46.264097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"78'4","reported_seq":54,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.921396+0000","last_change":"2024-05-14T16:51:29.345057+0000","last_active":"2024-05-14T16:52:13.921396+0000","last_peered":"2024-05-14T16:52:13.921396+0000","last_clean":"2024-05-14T16:52:13.921396+0000","last_became_active":"2024-05-14T16:51:29.344658+0000","last_became_peered":"2024-05-14T16:51:29.344658+0000","last_unstale":"2024-05-14T16:52:13.921396+0000","last_undegraded":"2024-05-14T16:52:13.921396+0000","last_fullsized":"2024-05-14T16:52:13.921396+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:43:53.007543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:51:34.300364+0000","last_change":"2024-05-14T16:51:32.298958+0000","last_active":"2024-05-14T16:51:34.300364+0000","last_peered":"2024-05-14T16:51:34.300364+0000","last_clean":"2024-05-14T16:51:34.300364+0000","last_became_active":"2024-05-14T16:51:32.298420+0000","last_became_peered":"2024-05-14T16:51:32.298420+0000","last_unstale":"2024-05-14T16:51:34.300364+0000","last_undegraded":"2024-05-14T16:51:34.300364+0000","last_fullsized":"2024-05-14T16:51:34.300364+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:15:21.262026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-14T16:52:12.209496+0000","last_change":"2024-05-14T16:51:29.272082+0000","last_active":"2024-05-14T16:52:12.209496+0000","last_peered":"2024-05-14T16:52:12.209496+0000","last_clean":"2024-05-14T16:52:12.209496+0000","last_became_active":"2024-05-14T16:51:29.271711+0000","last_became_peered":"2024-05-14T16:51:29.271711+0000","last_unstale":"2024-05-14T16:52:12.209496+0000","last_undegraded":"2024-05-14T16:52:12.209496+0000","last_fullsized":"2024-05-14T16:52:12.209496+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:27:44.320423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":99,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215003+0000","last_change":"2024-05-14T16:50:05.320136+0000","last_active":"2024-05-14T16:52:13.215003+0000","last_peered":"2024-05-14T16:52:13.215003+0000","last_clean":"2024-05-14T16:52:13.215003+0000","last_became_active":"2024-05-14T16:50:05.315016+0000","last_became_peered":"2024-05-14T16:50:05.315016+0000","last_unstale":"2024-05-14T16:52:13.215003+0000","last_undegraded":"2024-05-14T16:52:13.215003+0000","last_fullsized":"2024-05-14T16:52:13.215003+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:38.138211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:38.138211+0000","last_clean_scrub_stamp":"2024-05-14T16:46:38.138211+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:19:09.012578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.921326+0000","last_change":"2024-05-14T16:51:32.295995+0000","last_active":"2024-05-14T16:52:13.921326+0000","last_peered":"2024-05-14T16:52:13.921326+0000","last_clean":"2024-05-14T16:52:13.921326+0000","last_became_active":"2024-05-14T16:51:32.295640+0000","last_became_peered":"2024-05-14T16:51:32.295640+0000","last_unstale":"2024-05-14T16:52:13.921326+0000","last_undegraded":"2024-05-14T16:52:13.921326+0000","last_fullsized":"2024-05-14T16:52:13.921326+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:42:26.774916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215122+0000","last_change":"2024-05-14T16:51:29.274892+0000","last_active":"2024-05-14T16:52:13.215122+0000","last_peered":"2024-05-14T16:52:13.215122+0000","last_clean":"2024-05-14T16:52:13.215122+0000","last_became_active":"2024-05-14T16:51:29.274386+0000","last_became_peered":"2024-05-14T16:51:29.274386+0000","last_unstale":"2024-05-14T16:52:13.215122+0000","last_undegraded":"2024-05-14T16:52:13.215122+0000","last_fullsized":"2024-05-14T16:52:13.215122+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:16:54.899320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:51:46.980632+0000","last_change":"2024-05-14T16:51:29.345672+0000","last_active":"2024-05-14T16:51:46.980632+0000","last_peered":"2024-05-14T16:51:46.980632+0000","last_clean":"2024-05-14T16:51:46.980632+0000","last_became_active":"2024-05-14T16:51:29.344948+0000","last_became_peered":"2024-05-14T16:51:29.344948+0000","last_unstale":"2024-05-14T16:51:46.980632+0000","last_undegraded":"2024-05-14T16:51:46.980632+0000","last_fullsized":"2024-05-14T16:51:46.980632+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:19:26.577799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.921944+0000","last_change":"2024-05-14T16:51:32.297360+0000","last_active":"2024-05-14T16:52:13.921944+0000","last_peered":"2024-05-14T16:52:13.921944+0000","last_clean":"2024-05-14T16:52:13.921944+0000","last_became_active":"2024-05-14T16:51:32.297025+0000","last_became_peered":"2024-05-14T16:51:32.297025+0000","last_unstale":"2024-05-14T16:52:13.921944+0000","last_undegraded":"2024-05-14T16:52:13.921944+0000","last_fullsized":"2024-05-14T16:52:13.921944+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:05:39.705860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'8","reported_seq":31,"reported_epoch":78,"state":"peering","last_fresh":"2024-05-14T16:52:09.301575+0000","last_change":"2024-05-14T16:52:12.209956+0000","last_active":"2024-05-14T16:52:12.209956+0000","last_peered":"2024-05-14T16:52:09.301575+0000","last_clean":"2024-05-14T16:52:09.301575+0000","last_became_active":"2024-05-14T16:51:29.277383+0000","last_became_peered":"2024-05-14T16:51:29.277383+0000","last_unstale":"2024-05-14T16:52:09.301575+0000","last_undegraded":"2024-05-14T16:52:09.301575+0000","last_fullsized":"2024-05-14T16:52:09.301575+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,8,11],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:51:34.300276+0000","last_change":"2024-05-14T16:51:32.292112+0000","last_active":"2024-05-14T16:51:34.300276+0000","last_peered":"2024-05-14T16:51:34.300276+0000","last_clean":"2024-05-14T16:51:34.300276+0000","last_became_active":"2024-05-14T16:51:32.291688+0000","last_became_peered":"2024-05-14T16:51:32.291688+0000","last_unstale":"2024-05-14T16:51:34.300276+0000","last_undegraded":"2024-05-14T16:51:34.300276+0000","last_fullsized":"2024-05-14T16:51:34.300276+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:13:06.728608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-14T16:52:12.209528+0000","last_change":"2024-05-14T16:51:29.412805+0000","last_active":"2024-05-14T16:52:12.209528+0000","last_peered":"2024-05-14T16:52:12.209528+0000","last_clean":"2024-05-14T16:52:12.209528+0000","last_became_active":"2024-05-14T16:51:29.412394+0000","last_became_peered":"2024-05-14T16:51:29.412394+0000","last_unstale":"2024-05-14T16:52:12.209528+0000","last_undegraded":"2024-05-14T16:52:12.209528+0000","last_fullsized":"2024-05-14T16:52:12.209528+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:49:48.721104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.218227+0000","last_change":"2024-05-14T16:52:13.218227+0000","last_active":"2024-05-14T16:52:13.218227+0000","last_peered":"2024-05-14T16:52:13.218227+0000","last_clean":"2024-05-14T16:52:13.218227+0000","last_became_active":"2024-05-14T16:52:13.217787+0000","last_became_peered":"2024-05-14T16:52:13.217787+0000","last_unstale":"2024-05-14T16:52:13.218227+0000","last_undegraded":"2024-05-14T16:52:13.218227+0000","last_fullsized":"2024-05-14T16:52:13.218227+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:52:33.348804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-14T16:52:12.206817+0000","last_change":"2024-05-14T16:51:29.345427+0000","last_active":"2024-05-14T16:52:12.206817+0000","last_peered":"2024-05-14T16:52:12.206817+0000","last_clean":"2024-05-14T16:52:12.206817+0000","last_became_active":"2024-05-14T16:51:29.345049+0000","last_became_peered":"2024-05-14T16:51:29.345049+0000","last_unstale":"2024-05-14T16:52:12.206817+0000","last_undegraded":"2024-05-14T16:52:12.206817+0000","last_fullsized":"2024-05-14T16:52:12.206817+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:06:19.799660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:51:34.298193+0000","last_change":"2024-05-14T16:51:32.298757+0000","last_active":"2024-05-14T16:51:34.298193+0000","last_peered":"2024-05-14T16:51:34.298193+0000","last_clean":"2024-05-14T16:51:34.298193+0000","last_became_active":"2024-05-14T16:51:32.297193+0000","last_became_peered":"2024-05-14T16:51:32.297193+0000","last_unstale":"2024-05-14T16:51:34.298193+0000","last_undegraded":"2024-05-14T16:51:34.298193+0000","last_fullsized":"2024-05-14T16:51:34.298193+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:10:19.672230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:51:46.981060+0000","last_change":"2024-05-14T16:51:29.270545+0000","last_active":"2024-05-14T16:51:46.981060+0000","last_peered":"2024-05-14T16:51:46.981060+0000","last_clean":"2024-05-14T16:51:46.981060+0000","last_became_active":"2024-05-14T16:51:29.270182+0000","last_became_peered":"2024-05-14T16:51:29.270182+0000","last_unstale":"2024-05-14T16:51:46.981060+0000","last_undegraded":"2024-05-14T16:51:46.981060+0000","last_fullsized":"2024-05-14T16:51:46.981060+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:19:49.977916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.214764+0000","last_change":"2024-05-14T16:51:32.294767+0000","last_active":"2024-05-14T16:52:13.214764+0000","last_peered":"2024-05-14T16:52:13.214764+0000","last_clean":"2024-05-14T16:52:13.214764+0000","last_became_active":"2024-05-14T16:51:32.293947+0000","last_became_peered":"2024-05-14T16:51:32.293947+0000","last_unstale":"2024-05-14T16:52:13.214764+0000","last_undegraded":"2024-05-14T16:52:13.214764+0000","last_fullsized":"2024-05-14T16:52:13.214764+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:30:21.731492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.226988+0000","last_change":"2024-05-14T16:52:13.226509+0000","last_active":"2024-05-14T16:52:13.226988+0000","last_peered":"2024-05-14T16:52:13.226988+0000","last_clean":"2024-05-14T16:52:13.226988+0000","last_became_active":"2024-05-14T16:52:13.220923+0000","last_became_peered":"2024-05-14T16:52:13.220923+0000","last_unstale":"2024-05-14T16:52:13.226988+0000","last_undegraded":"2024-05-14T16:52:13.226988+0000","last_fullsized":"2024-05-14T16:52:13.226988+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:25:52.655481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:51:34.300787+0000","last_change":"2024-05-14T16:51:32.288810+0000","last_active":"2024-05-14T16:51:34.300787+0000","last_peered":"2024-05-14T16:51:34.300787+0000","last_clean":"2024-05-14T16:51:34.300787+0000","last_became_active":"2024-05-14T16:51:32.288492+0000","last_became_peered":"2024-05-14T16:51:32.288492+0000","last_unstale":"2024-05-14T16:51:34.300787+0000","last_undegraded":"2024-05-14T16:51:34.300787+0000","last_fullsized":"2024-05-14T16:51:34.300787+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:44:53.393371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:51:46.982475+0000","last_change":"2024-05-14T16:51:29.281181+0000","last_active":"2024-05-14T16:51:46.982475+0000","last_peered":"2024-05-14T16:51:46.982475+0000","last_clean":"2024-05-14T16:51:46.982475+0000","last_became_active":"2024-05-14T16:51:29.279965+0000","last_became_peered":"2024-05-14T16:51:29.279965+0000","last_unstale":"2024-05-14T16:51:46.982475+0000","last_undegraded":"2024-05-14T16:51:46.982475+0000","last_fullsized":"2024-05-14T16:51:46.982475+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:41:00.735291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215266+0000","last_change":"2024-05-14T16:51:32.296971+0000","last_active":"2024-05-14T16:52:13.215266+0000","last_peered":"2024-05-14T16:52:13.215266+0000","last_clean":"2024-05-14T16:52:13.215266+0000","last_became_active":"2024-05-14T16:51:32.296600+0000","last_became_peered":"2024-05-14T16:51:32.296600+0000","last_unstale":"2024-05-14T16:52:13.215266+0000","last_undegraded":"2024-05-14T16:52:13.215266+0000","last_fullsized":"2024-05-14T16:52:13.215266+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:13:14.354127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:51:47.989117+0000","last_change":"2024-05-14T16:51:29.412682+0000","last_active":"2024-05-14T16:51:47.989117+0000","last_peered":"2024-05-14T16:51:47.989117+0000","last_clean":"2024-05-14T16:51:47.989117+0000","last_became_active":"2024-05-14T16:51:29.412186+0000","last_became_peered":"2024-05-14T16:51:29.412186+0000","last_unstale":"2024-05-14T16:51:47.989117+0000","last_undegraded":"2024-05-14T16:51:47.989117+0000","last_fullsized":"2024-05-14T16:51:47.989117+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:56:24.570276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215347+0000","last_change":"2024-05-14T16:51:32.300626+0000","last_active":"2024-05-14T16:52:13.215347+0000","last_peered":"2024-05-14T16:52:13.215347+0000","last_clean":"2024-05-14T16:52:13.215347+0000","last_became_active":"2024-05-14T16:51:32.300278+0000","last_became_peered":"2024-05-14T16:51:32.300278+0000","last_unstale":"2024-05-14T16:52:13.215347+0000","last_undegraded":"2024-05-14T16:52:13.215347+0000","last_fullsized":"2024-05-14T16:52:13.215347+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:59:14.196315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-14T16:52:12.214623+0000","last_change":"2024-05-14T16:51:29.270826+0000","last_active":"2024-05-14T16:52:12.214623+0000","last_peered":"2024-05-14T16:52:12.214623+0000","last_clean":"2024-05-14T16:52:12.214623+0000","last_became_active":"2024-05-14T16:51:29.270266+0000","last_became_peered":"2024-05-14T16:51:29.270266+0000","last_unstale":"2024-05-14T16:52:12.214623+0000","last_undegraded":"2024-05-14T16:52:12.214623+0000","last_fullsized":"2024-05-14T16:52:12.214623+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:00:55.349278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.223994+0000","last_change":"2024-05-14T16:52:13.223994+0000","last_active":"2024-05-14T16:52:13.223994+0000","last_peered":"2024-05-14T16:52:13.223994+0000","last_clean":"2024-05-14T16:52:13.223994+0000","last_became_active":"2024-05-14T16:52:13.221071+0000","last_became_peered":"2024-05-14T16:52:13.221071+0000","last_unstale":"2024-05-14T16:52:13.223994+0000","last_undegraded":"2024-05-14T16:52:13.223994+0000","last_fullsized":"2024-05-14T16:52:13.223994+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:29:58.775171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.921947+0000","last_change":"2024-05-14T16:51:29.276705+0000","last_active":"2024-05-14T16:52:13.921947+0000","last_peered":"2024-05-14T16:52:13.921947+0000","last_clean":"2024-05-14T16:52:13.921947+0000","last_became_active":"2024-05-14T16:51:29.276347+0000","last_became_peered":"2024-05-14T16:51:29.276347+0000","last_unstale":"2024-05-14T16:52:13.921947+0000","last_undegraded":"2024-05-14T16:52:13.921947+0000","last_fullsized":"2024-05-14T16:52:13.921947+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:32:56.064427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215638+0000","last_change":"2024-05-14T16:51:32.301617+0000","last_active":"2024-05-14T16:52:13.215638+0000","last_peered":"2024-05-14T16:52:13.215638+0000","last_clean":"2024-05-14T16:52:13.215638+0000","last_became_active":"2024-05-14T16:51:32.301288+0000","last_became_peered":"2024-05-14T16:51:32.301288+0000","last_unstale":"2024-05-14T16:52:13.215638+0000","last_undegraded":"2024-05-14T16:52:13.215638+0000","last_fullsized":"2024-05-14T16:52:13.215638+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:19:24.253633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.221694+0000","last_change":"2024-05-14T16:52:13.221694+0000","last_active":"2024-05-14T16:52:13.221694+0000","last_peered":"2024-05-14T16:52:13.221694+0000","last_clean":"2024-05-14T16:52:13.221694+0000","last_became_active":"2024-05-14T16:52:13.221264+0000","last_became_peered":"2024-05-14T16:52:13.221264+0000","last_unstale":"2024-05-14T16:52:13.221694+0000","last_undegraded":"2024-05-14T16:52:13.221694+0000","last_fullsized":"2024-05-14T16:52:13.221694+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:22:52.738772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:51:34.299778+0000","last_change":"2024-05-14T16:51:32.301789+0000","last_active":"2024-05-14T16:51:34.299778+0000","last_peered":"2024-05-14T16:51:34.299778+0000","last_clean":"2024-05-14T16:51:34.299778+0000","last_became_active":"2024-05-14T16:51:32.301350+0000","last_became_peered":"2024-05-14T16:51:32.301350+0000","last_unstale":"2024-05-14T16:51:34.299778+0000","last_undegraded":"2024-05-14T16:51:34.299778+0000","last_fullsized":"2024-05-14T16:51:34.299778+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:46:24.896855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:51:34.298732+0000","last_change":"2024-05-14T16:51:32.295978+0000","last_active":"2024-05-14T16:51:34.298732+0000","last_peered":"2024-05-14T16:51:34.298732+0000","last_clean":"2024-05-14T16:51:34.298732+0000","last_became_active":"2024-05-14T16:51:32.295652+0000","last_became_peered":"2024-05-14T16:51:32.295652+0000","last_unstale":"2024-05-14T16:51:34.298732+0000","last_undegraded":"2024-05-14T16:51:34.298732+0000","last_fullsized":"2024-05-14T16:51:34.298732+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:32:17.743734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215754+0000","last_change":"2024-05-14T16:51:32.305568+0000","last_active":"2024-05-14T16:52:13.215754+0000","last_peered":"2024-05-14T16:52:13.215754+0000","last_clean":"2024-05-14T16:52:13.215754+0000","last_became_active":"2024-05-14T16:51:32.304580+0000","last_became_peered":"2024-05-14T16:51:32.304580+0000","last_unstale":"2024-05-14T16:52:13.215754+0000","last_undegraded":"2024-05-14T16:52:13.215754+0000","last_fullsized":"2024-05-14T16:52:13.215754+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:32:44.465054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.922077+0000","last_change":"2024-05-14T16:51:32.295029+0000","last_active":"2024-05-14T16:52:13.922077+0000","last_peered":"2024-05-14T16:52:13.922077+0000","last_clean":"2024-05-14T16:52:13.922077+0000","last_became_active":"2024-05-14T16:51:32.294621+0000","last_became_peered":"2024-05-14T16:51:32.294621+0000","last_unstale":"2024-05-14T16:52:13.922077+0000","last_undegraded":"2024-05-14T16:52:13.922077+0000","last_fullsized":"2024-05-14T16:52:13.922077+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:52:05.390141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.214978+0000","last_change":"2024-05-14T16:51:32.297671+0000","last_active":"2024-05-14T16:52:13.214978+0000","last_peered":"2024-05-14T16:52:13.214978+0000","last_clean":"2024-05-14T16:52:13.214978+0000","last_became_active":"2024-05-14T16:51:32.296210+0000","last_became_peered":"2024-05-14T16:51:32.296210+0000","last_unstale":"2024-05-14T16:52:13.214978+0000","last_undegraded":"2024-05-14T16:52:13.214978+0000","last_fullsized":"2024-05-14T16:52:13.214978+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:55:04.789021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:51:34.300696+0000","last_change":"2024-05-14T16:51:32.297404+0000","last_active":"2024-05-14T16:51:34.300696+0000","last_peered":"2024-05-14T16:51:34.300696+0000","last_clean":"2024-05-14T16:51:34.300696+0000","last_became_active":"2024-05-14T16:51:32.296856+0000","last_became_peered":"2024-05-14T16:51:32.296856+0000","last_unstale":"2024-05-14T16:51:34.300696+0000","last_undegraded":"2024-05-14T16:51:34.300696+0000","last_fullsized":"2024-05-14T16:51:34.300696+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:25:29.992875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:51:34.300108+0000","last_change":"2024-05-14T16:51:32.295931+0000","last_active":"2024-05-14T16:51:34.300108+0000","last_peered":"2024-05-14T16:51:34.300108+0000","last_clean":"2024-05-14T16:51:34.300108+0000","last_became_active":"2024-05-14T16:51:32.295456+0000","last_became_peered":"2024-05-14T16:51:32.295456+0000","last_unstale":"2024-05-14T16:51:34.300108+0000","last_undegraded":"2024-05-14T16:51:34.300108+0000","last_fullsized":"2024-05-14T16:51:34.300108+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:53:20.265632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-14T16:52:12.214743+0000","last_change":"2024-05-14T16:51:32.303570+0000","last_active":"2024-05-14T16:52:12.214743+0000","last_peered":"2024-05-14T16:52:12.214743+0000","last_clean":"2024-05-14T16:52:12.214743+0000","last_became_active":"2024-05-14T16:51:32.303234+0000","last_became_peered":"2024-05-14T16:51:32.303234+0000","last_unstale":"2024-05-14T16:52:12.214743+0000","last_undegraded":"2024-05-14T16:52:12.214743+0000","last_fullsized":"2024-05-14T16:52:12.214743+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:27:57.960120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-14T16:51:34.300038+0000","last_change":"2024-05-14T16:51:32.298028+0000","last_active":"2024-05-14T16:51:34.300038+0000","last_peered":"2024-05-14T16:51:34.300038+0000","last_clean":"2024-05-14T16:51:34.300038+0000","last_became_active":"2024-05-14T16:51:32.295104+0000","last_became_peered":"2024-05-14T16:51:32.295104+0000","last_unstale":"2024-05-14T16:51:34.300038+0000","last_undegraded":"2024-05-14T16:51:34.300038+0000","last_fullsized":"2024-05-14T16:51:34.300038+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:58:08.441590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-05-14T16:52:12.207064+0000","last_change":"2024-05-14T16:51:32.304458+0000","last_active":"2024-05-14T16:52:12.207064+0000","last_peered":"2024-05-14T16:52:12.207064+0000","last_clean":"2024-05-14T16:52:12.207064+0000","last_became_active":"2024-05-14T16:51:32.302952+0000","last_became_peered":"2024-05-14T16:51:32.302952+0000","last_unstale":"2024-05-14T16:52:12.207064+0000","last_undegraded":"2024-05-14T16:52:12.207064+0000","last_fullsized":"2024-05-14T16:52:12.207064+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:40:14.220075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.921611+0000","last_change":"2024-05-14T16:51:32.292334+0000","last_active":"2024-05-14T16:52:13.921611+0000","last_peered":"2024-05-14T16:52:13.921611+0000","last_clean":"2024-05-14T16:52:13.921611+0000","last_became_active":"2024-05-14T16:51:32.291769+0000","last_became_peered":"2024-05-14T16:51:32.291769+0000","last_unstale":"2024-05-14T16:52:13.921611+0000","last_undegraded":"2024-05-14T16:52:13.921611+0000","last_fullsized":"2024-05-14T16:52:13.921611+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:55:25.694827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.214789+0000","last_change":"2024-05-14T16:51:32.300044+0000","last_active":"2024-05-14T16:52:13.214789+0000","last_peered":"2024-05-14T16:52:13.214789+0000","last_clean":"2024-05-14T16:52:13.214789+0000","last_became_active":"2024-05-14T16:51:32.299630+0000","last_became_peered":"2024-05-14T16:51:32.299630+0000","last_unstale":"2024-05-14T16:52:13.214789+0000","last_undegraded":"2024-05-14T16:52:13.214789+0000","last_fullsized":"2024-05-14T16:52:13.214789+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:50:40.516973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284433,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":58997,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":163,"num_read_kb":140,"num_write":158,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":167371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710744,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":864256,"data_stored":425130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906973,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":995328,"data_stored":1058338,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070497,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":868352,"data_stored":416907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266727,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":856064,"data_stored":408184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462956,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":1017216,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626482,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36188,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708964,"statfs":{"total":95995035648,"available":95957979136,"internally_reserved":0,"allocated":868352,"data_stored":424755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":909312,"data_stored":466275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36212,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708940,"statfs":{"total":95995035648,"available":95957954560,"internally_reserved":0,"allocated":888832,"data_stored":426026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":921600,"data_stored":459373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182464,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36176,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708976,"statfs":{"total":95995035648,"available":95957991424,"internally_reserved":0,"allocated":860160,"data_stored":407964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986236,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32004,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713148,"statfs":{"total":95995035648,"available":95962263552,"internally_reserved":0,"allocated":847872,"data_stored":408090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822711,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":917504,"data_stored":999798,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34541,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34541,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34541,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":50186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":51927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":56574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T16:52:18.320 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-14T16:52:18.577 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:52:19.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:19 smithi002 ceph-mon[31538]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:52:19.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:19 smithi002 ceph-mon[31538]: pgmap v313: 97 pgs: 1 peering, 96 active+clean; 634 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1023 B/s wr, 6 op/s; 14 B/s, 1 keys/s, 0 objects/s recovering 2024-05-14T16:52:19.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:19 smithi151 ceph-mon[36066]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:52:19.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:19 smithi151 ceph-mon[36066]: pgmap v313: 97 pgs: 1 peering, 96 active+clean; 634 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1023 B/s wr, 6 op/s; 14 B/s, 1 keys/s, 0 objects/s recovering 2024-05-14T16:52:19.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:19 smithi123 ceph-mon[35971]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:52:19.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:19 smithi123 ceph-mon[35971]: pgmap v313: 97 pgs: 1 peering, 96 active+clean; 634 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1023 B/s wr, 6 op/s; 14 B/s, 1 keys/s, 0 objects/s recovering 2024-05-14T16:52:20.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.201+0000 7fac83ecf700 1 -- 172.21.15.2:0/1245530446 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac7c06c610 msgr2=0x7fac7c10fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:20.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.201+0000 7fac83ecf700 1 --2- 172.21.15.2:0/1245530446 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac7c06c610 0x7fac7c10fa50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fac6c009990 tx=0x7fac6c009ca0 comp rx=0 tx=0).stop 2024-05-14T16:52:20.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.201+0000 7fac83ecf700 1 -- 172.21.15.2:0/1245530446 shutdown_connections 2024-05-14T16:52:20.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.201+0000 7fac83ecf700 1 --2- 172.21.15.2:0/1245530446 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac7c10ff90 0x7fac7c112380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:20.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.201+0000 7fac83ecf700 1 --2- 172.21.15.2:0/1245530446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac7c06bcf0 0x7fac7c06c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:20.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.201+0000 7fac83ecf700 1 --2- 172.21.15.2:0/1245530446 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac7c06c610 0x7fac7c10fa50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:20.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.201+0000 7fac83ecf700 1 -- 172.21.15.2:0/1245530446 >> 172.21.15.2:0/1245530446 conn(0x7fac7c0794e0 msgr2=0x7fac7c0798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:20.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.201+0000 7fac83ecf700 1 -- 172.21.15.2:0/1245530446 shutdown_connections 2024-05-14T16:52:20.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.201+0000 7fac83ecf700 1 -- 172.21.15.2:0/1245530446 wait complete. 2024-05-14T16:52:20.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.201+0000 7fac83ecf700 1 Processor -- start 2024-05-14T16:52:20.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.202+0000 7fac83ecf700 1 -- start start 2024-05-14T16:52:20.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.202+0000 7fac83ecf700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac7c06bcf0 0x7fac7c19bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:20.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.202+0000 7fac83ecf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac7c06c610 0x7fac7c19c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:20.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.202+0000 7fac83ecf700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac7c10ff90 0x7fac7c1a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:20.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.202+0000 7fac83ecf700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fac7c1159c0 con 0x7fac7c06c610 2024-05-14T16:52:20.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.202+0000 7fac83ecf700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fac7c115840 con 0x7fac7c06bcf0 2024-05-14T16:52:20.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.202+0000 7fac83ecf700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fac7c115b40 con 0x7fac7c10ff90 2024-05-14T16:52:20.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.202+0000 7fac8146a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac7c06c610 0x7fac7c19c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:20.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.202+0000 7fac8146a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac7c06c610 0x7fac7c19c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51604/0 (socket says 172.21.15.2:51604) 2024-05-14T16:52:20.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.202+0000 7fac8146a700 1 -- 172.21.15.2:0/3049258207 learned_addr learned my addr 172.21.15.2:0/3049258207 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:52:20.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.202+0000 7fac8246c700 1 --2- 172.21.15.2:0/3049258207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac7c10ff90 0x7fac7c1a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:20.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.202+0000 7fac81c6b700 1 --2- 172.21.15.2:0/3049258207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac7c06bcf0 0x7fac7c19bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:20.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.202+0000 7fac8146a700 1 -- 172.21.15.2:0/3049258207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac7c10ff90 msgr2=0x7fac7c1a07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:20.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.202+0000 7fac8146a700 1 --2- 172.21.15.2:0/3049258207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac7c10ff90 0x7fac7c1a07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:20.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.202+0000 7fac8146a700 1 -- 172.21.15.2:0/3049258207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac7c06bcf0 msgr2=0x7fac7c19bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:20.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.202+0000 7fac8146a700 1 --2- 172.21.15.2:0/3049258207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac7c06bcf0 0x7fac7c19bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:20.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.202+0000 7fac8146a700 1 -- 172.21.15.2:0/3049258207 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac7c1a0f00 con 0x7fac7c06c610 2024-05-14T16:52:20.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.202+0000 7fac81c6b700 1 --2- 172.21.15.2:0/3049258207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac7c06bcf0 0x7fac7c19bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:52:20.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.203+0000 7fac8146a700 1 --2- 172.21.15.2:0/3049258207 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac7c06c610 0x7fac7c19c430 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fac6c000c00 tx=0x7fac6c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:20.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.203+0000 7fac72ffd700 1 -- 172.21.15.2:0/3049258207 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c027070 con 0x7fac7c06c610 2024-05-14T16:52:20.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.203+0000 7fac72ffd700 1 -- 172.21.15.2:0/3049258207 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac6c0040e0 con 0x7fac7c06c610 2024-05-14T16:52:20.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.203+0000 7fac83ecf700 1 -- 172.21.15.2:0/3049258207 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac7c1a1190 con 0x7fac7c06c610 2024-05-14T16:52:20.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.203+0000 7fac72ffd700 1 -- 172.21.15.2:0/3049258207 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c00f460 con 0x7fac7c06c610 2024-05-14T16:52:20.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.203+0000 7fac83ecf700 1 -- 172.21.15.2:0/3049258207 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac7c1a8a70 con 0x7fac7c06c610 2024-05-14T16:52:20.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.204+0000 7fac83ecf700 1 -- 172.21.15.2:0/3049258207 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac7c06a230 con 0x7fac7c06c610 2024-05-14T16:52:20.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.205+0000 7fac72ffd700 1 -- 172.21.15.2:0/3049258207 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fac6c02a070 con 0x7fac7c06c610 2024-05-14T16:52:20.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.206+0000 7fac72ffd700 1 --2- 172.21.15.2:0/3049258207 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fac680b2990 0x7fac680b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:20.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.206+0000 7fac81c6b700 1 --2- 172.21.15.2:0/3049258207 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fac680b2990 0x7fac680b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:20.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.206+0000 7fac72ffd700 1 -- 172.21.15.2:0/3049258207 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7fac6c0e3790 con 0x7fac7c06c610 2024-05-14T16:52:20.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.207+0000 7fac81c6b700 1 --2- 172.21.15.2:0/3049258207 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fac680b2990 0x7fac680b4e50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fac780099d0 tx=0x7fac78008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:20.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.210+0000 7fac72ffd700 1 -- 172.21.15.2:0/3049258207 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fac6c031070 con 0x7fac7c06c610 2024-05-14T16:52:20.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.425+0000 7fac83ecf700 1 -- 172.21.15.2:0/3049258207 --> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fac7c19d250 con 0x7fac680b2990 2024-05-14T16:52:20.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.432+0000 7fac72ffd700 1 -- 172.21.15.2:0/3049258207 <== mgr.14150 v2:172.21.15.2:6800/1498936028 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258915 (secure 0 0 0) 0x7fac7c19d250 con 0x7fac680b2990 2024-05-14T16:52:20.436 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:52:20.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.435+0000 7fac83ecf700 1 -- 172.21.15.2:0/3049258207 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fac680b2990 msgr2=0x7fac680b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:20.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.435+0000 7fac83ecf700 1 --2- 172.21.15.2:0/3049258207 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fac680b2990 0x7fac680b4e50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fac780099d0 tx=0x7fac78008040 comp rx=0 tx=0).stop 2024-05-14T16:52:20.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.436+0000 7fac83ecf700 1 -- 172.21.15.2:0/3049258207 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac7c06c610 msgr2=0x7fac7c19c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:20.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.436+0000 7fac83ecf700 1 --2- 172.21.15.2:0/3049258207 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac7c06c610 0x7fac7c19c430 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fac6c000c00 tx=0x7fac6c0047b0 comp rx=0 tx=0).stop 2024-05-14T16:52:20.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.436+0000 7fac83ecf700 1 -- 172.21.15.2:0/3049258207 shutdown_connections 2024-05-14T16:52:20.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.436+0000 7fac83ecf700 1 --2- 172.21.15.2:0/3049258207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac7c10ff90 0x7fac7c1a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:20.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.436+0000 7fac83ecf700 1 --2- 172.21.15.2:0/3049258207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac7c06bcf0 0x7fac7c19bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:20.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.436+0000 7fac83ecf700 1 --2- 172.21.15.2:0/3049258207 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fac680b2990 0x7fac680b4e50 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:20.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.436+0000 7fac83ecf700 1 --2- 172.21.15.2:0/3049258207 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac7c06c610 0x7fac7c19c430 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:20.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.436+0000 7fac83ecf700 1 -- 172.21.15.2:0/3049258207 >> 172.21.15.2:0/3049258207 conn(0x7fac7c0794e0 msgr2=0x7fac7c1115d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:20.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.436+0000 7fac83ecf700 1 -- 172.21.15.2:0/3049258207 shutdown_connections 2024-05-14T16:52:20.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:20.436+0000 7fac83ecf700 1 -- 172.21.15.2:0/3049258207 wait complete. 2024-05-14T16:52:20.439 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-05-14T16:52:21.137 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-05-14T16:52:20.096134+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655844,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":275,"num_read_kb":229,"num_write":289,"num_write_kb":2263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":2951957,"num_keys_recovered":9,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":384836,"kb_used_data":10644,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124556988,"statfs":{"total":1151940427776,"available":1151546355712,"internally_reserved":0,"allocated":10899456,"data_stored":7043373,"data_compressed":108486,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11055,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":43,"num_write":12,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":117,"num_keys_recovered":9,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002077"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215138+0000","last_change":"2024-05-14T16:51:32.298580+0000","last_active":"2024-05-14T16:52:13.215138+0000","last_peered":"2024-05-14T16:52:13.215138+0000","last_clean":"2024-05-14T16:52:13.215138+0000","last_became_active":"2024-05-14T16:51:32.296945+0000","last_became_peered":"2024-05-14T16:51:32.296945+0000","last_unstale":"2024-05-14T16:52:13.215138+0000","last_undegraded":"2024-05-14T16:52:13.215138+0000","last_fullsized":"2024-05-14T16:52:13.215138+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:50:17.664928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215759+0000","last_change":"2024-05-14T16:51:32.297618+0000","last_active":"2024-05-14T16:52:13.215759+0000","last_peered":"2024-05-14T16:52:13.215759+0000","last_clean":"2024-05-14T16:52:13.215759+0000","last_became_active":"2024-05-14T16:51:32.297260+0000","last_became_peered":"2024-05-14T16:51:32.297260+0000","last_unstale":"2024-05-14T16:52:13.215759+0000","last_undegraded":"2024-05-14T16:52:13.215759+0000","last_fullsized":"2024-05-14T16:52:13.215759+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:43:50.542587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215551+0000","last_change":"2024-05-14T16:51:32.295525+0000","last_active":"2024-05-14T16:52:13.215551+0000","last_peered":"2024-05-14T16:52:13.215551+0000","last_clean":"2024-05-14T16:52:13.215551+0000","last_became_active":"2024-05-14T16:51:32.295215+0000","last_became_peered":"2024-05-14T16:51:32.295215+0000","last_unstale":"2024-05-14T16:52:13.215551+0000","last_undegraded":"2024-05-14T16:52:13.215551+0000","last_fullsized":"2024-05-14T16:52:13.215551+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:40:05.516280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.921725+0000","last_change":"2024-05-14T16:51:32.307883+0000","last_active":"2024-05-14T16:52:13.921725+0000","last_peered":"2024-05-14T16:52:13.921725+0000","last_clean":"2024-05-14T16:52:13.921725+0000","last_became_active":"2024-05-14T16:51:32.307499+0000","last_became_peered":"2024-05-14T16:51:32.307499+0000","last_unstale":"2024-05-14T16:52:13.921725+0000","last_undegraded":"2024-05-14T16:52:13.921725+0000","last_fullsized":"2024-05-14T16:52:13.921725+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:30:59.978059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.218739+0000","last_change":"2024-05-14T16:51:32.294429+0000","last_active":"2024-05-14T16:52:13.218739+0000","last_peered":"2024-05-14T16:52:13.218739+0000","last_clean":"2024-05-14T16:52:13.218739+0000","last_became_active":"2024-05-14T16:51:32.294088+0000","last_became_peered":"2024-05-14T16:51:32.294088+0000","last_unstale":"2024-05-14T16:52:13.218739+0000","last_undegraded":"2024-05-14T16:52:13.218739+0000","last_fullsized":"2024-05-14T16:52:13.218739+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:25:19.383586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.445204+0000","last_change":"2024-05-14T16:51:32.296169+0000","last_active":"2024-05-14T16:52:13.445204+0000","last_peered":"2024-05-14T16:52:13.445204+0000","last_clean":"2024-05-14T16:52:13.445204+0000","last_became_active":"2024-05-14T16:51:32.295819+0000","last_became_peered":"2024-05-14T16:51:32.295819+0000","last_unstale":"2024-05-14T16:52:13.445204+0000","last_undegraded":"2024-05-14T16:52:13.445204+0000","last_fullsized":"2024-05-14T16:52:13.445204+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:56:23.932039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.214939+0000","last_change":"2024-05-14T16:51:32.309609+0000","last_active":"2024-05-14T16:52:13.214939+0000","last_peered":"2024-05-14T16:52:13.214939+0000","last_clean":"2024-05-14T16:52:13.214939+0000","last_became_active":"2024-05-14T16:51:32.309175+0000","last_became_peered":"2024-05-14T16:51:32.309175+0000","last_unstale":"2024-05-14T16:52:13.214939+0000","last_undegraded":"2024-05-14T16:52:13.214939+0000","last_fullsized":"2024-05-14T16:52:13.214939+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:06:46.717592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.446592+0000","last_change":"2024-05-14T16:52:13.446592+0000","last_active":"2024-05-14T16:52:13.446592+0000","last_peered":"2024-05-14T16:52:13.446592+0000","last_clean":"2024-05-14T16:52:13.446592+0000","last_became_active":"2024-05-14T16:52:13.446238+0000","last_became_peered":"2024-05-14T16:52:13.446238+0000","last_unstale":"2024-05-14T16:52:13.446592+0000","last_undegraded":"2024-05-14T16:52:13.446592+0000","last_fullsized":"2024-05-14T16:52:13.446592+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:33:14.216797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.213149+0000","last_change":"2024-05-14T16:51:32.307571+0000","last_active":"2024-05-14T16:52:13.213149+0000","last_peered":"2024-05-14T16:52:13.213149+0000","last_clean":"2024-05-14T16:52:13.213149+0000","last_became_active":"2024-05-14T16:51:32.307151+0000","last_became_peered":"2024-05-14T16:51:32.307151+0000","last_unstale":"2024-05-14T16:52:13.213149+0000","last_undegraded":"2024-05-14T16:52:13.213149+0000","last_fullsized":"2024-05-14T16:52:13.213149+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:17:26.325393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.445700+0000","last_change":"2024-05-14T16:51:32.298773+0000","last_active":"2024-05-14T16:52:13.445700+0000","last_peered":"2024-05-14T16:52:13.445700+0000","last_clean":"2024-05-14T16:52:13.445700+0000","last_became_active":"2024-05-14T16:51:32.298283+0000","last_became_peered":"2024-05-14T16:51:32.298283+0000","last_unstale":"2024-05-14T16:52:13.445700+0000","last_undegraded":"2024-05-14T16:52:13.445700+0000","last_fullsized":"2024-05-14T16:52:13.445700+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:52:18.471201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215163+0000","last_change":"2024-05-14T16:51:32.302443+0000","last_active":"2024-05-14T16:52:13.215163+0000","last_peered":"2024-05-14T16:52:13.215163+0000","last_clean":"2024-05-14T16:52:13.215163+0000","last_became_active":"2024-05-14T16:51:32.302034+0000","last_became_peered":"2024-05-14T16:51:32.302034+0000","last_unstale":"2024-05-14T16:52:13.215163+0000","last_undegraded":"2024-05-14T16:52:13.215163+0000","last_fullsized":"2024-05-14T16:52:13.215163+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:53:08.041293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.921862+0000","last_change":"2024-05-14T16:51:32.304837+0000","last_active":"2024-05-14T16:52:13.921862+0000","last_peered":"2024-05-14T16:52:13.921862+0000","last_clean":"2024-05-14T16:52:13.921862+0000","last_became_active":"2024-05-14T16:51:32.304500+0000","last_became_peered":"2024-05-14T16:51:32.304500+0000","last_unstale":"2024-05-14T16:52:13.921862+0000","last_undegraded":"2024-05-14T16:52:13.921862+0000","last_fullsized":"2024-05-14T16:52:13.921862+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:32:02.713044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.921639+0000","last_change":"2024-05-14T16:51:32.299247+0000","last_active":"2024-05-14T16:52:13.921639+0000","last_peered":"2024-05-14T16:52:13.921639+0000","last_clean":"2024-05-14T16:52:13.921639+0000","last_became_active":"2024-05-14T16:51:32.298925+0000","last_became_peered":"2024-05-14T16:51:32.298925+0000","last_unstale":"2024-05-14T16:52:13.921639+0000","last_undegraded":"2024-05-14T16:52:13.921639+0000","last_fullsized":"2024-05-14T16:52:13.921639+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:10:38.847877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215377+0000","last_change":"2024-05-14T16:51:32.293743+0000","last_active":"2024-05-14T16:52:13.215377+0000","last_peered":"2024-05-14T16:52:13.215377+0000","last_clean":"2024-05-14T16:52:13.215377+0000","last_became_active":"2024-05-14T16:51:32.293429+0000","last_became_peered":"2024-05-14T16:51:32.293429+0000","last_unstale":"2024-05-14T16:52:13.215377+0000","last_undegraded":"2024-05-14T16:52:13.215377+0000","last_fullsized":"2024-05-14T16:52:13.215377+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:12:33.909940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215027+0000","last_change":"2024-05-14T16:51:32.304940+0000","last_active":"2024-05-14T16:52:13.215027+0000","last_peered":"2024-05-14T16:52:13.215027+0000","last_clean":"2024-05-14T16:52:13.215027+0000","last_became_active":"2024-05-14T16:51:32.304482+0000","last_became_peered":"2024-05-14T16:51:32.304482+0000","last_unstale":"2024-05-14T16:52:13.215027+0000","last_undegraded":"2024-05-14T16:52:13.215027+0000","last_fullsized":"2024-05-14T16:52:13.215027+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:54:17.771620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.212590+0000","last_change":"2024-05-14T16:51:32.301784+0000","last_active":"2024-05-14T16:52:13.212590+0000","last_peered":"2024-05-14T16:52:13.212590+0000","last_clean":"2024-05-14T16:52:13.212590+0000","last_became_active":"2024-05-14T16:51:32.301472+0000","last_became_peered":"2024-05-14T16:51:32.301472+0000","last_unstale":"2024-05-14T16:52:13.212590+0000","last_undegraded":"2024-05-14T16:52:13.212590+0000","last_fullsized":"2024-05-14T16:52:13.212590+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:10:56.735424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215272+0000","last_change":"2024-05-14T16:51:32.297951+0000","last_active":"2024-05-14T16:52:13.215272+0000","last_peered":"2024-05-14T16:52:13.215272+0000","last_clean":"2024-05-14T16:52:13.215272+0000","last_became_active":"2024-05-14T16:51:32.296602+0000","last_became_peered":"2024-05-14T16:51:32.296602+0000","last_unstale":"2024-05-14T16:52:13.215272+0000","last_undegraded":"2024-05-14T16:52:13.215272+0000","last_fullsized":"2024-05-14T16:52:13.215272+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:16:36.039077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.214959+0000","last_change":"2024-05-14T16:51:32.290398+0000","last_active":"2024-05-14T16:52:13.214959+0000","last_peered":"2024-05-14T16:52:13.214959+0000","last_clean":"2024-05-14T16:52:13.214959+0000","last_became_active":"2024-05-14T16:51:32.290083+0000","last_became_peered":"2024-05-14T16:51:32.290083+0000","last_unstale":"2024-05-14T16:52:13.214959+0000","last_undegraded":"2024-05-14T16:52:13.214959+0000","last_fullsized":"2024-05-14T16:52:13.214959+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:48:05.795209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.218065+0000","last_change":"2024-05-14T16:51:32.294398+0000","last_active":"2024-05-14T16:52:13.218065+0000","last_peered":"2024-05-14T16:52:13.218065+0000","last_clean":"2024-05-14T16:52:13.218065+0000","last_became_active":"2024-05-14T16:51:32.294080+0000","last_became_peered":"2024-05-14T16:51:32.294080+0000","last_unstale":"2024-05-14T16:52:13.218065+0000","last_undegraded":"2024-05-14T16:52:13.218065+0000","last_fullsized":"2024-05-14T16:52:13.218065+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:15:03.859045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.214939+0000","last_change":"2024-05-14T16:51:29.274482+0000","last_active":"2024-05-14T16:52:13.214939+0000","last_peered":"2024-05-14T16:52:13.214939+0000","last_clean":"2024-05-14T16:52:13.214939+0000","last_became_active":"2024-05-14T16:51:29.274152+0000","last_became_peered":"2024-05-14T16:51:29.274152+0000","last_unstale":"2024-05-14T16:52:13.214939+0000","last_undegraded":"2024-05-14T16:52:13.214939+0000","last_fullsized":"2024-05-14T16:52:13.214939+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:05:38.274248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.212486+0000","last_change":"2024-05-14T16:51:32.300956+0000","last_active":"2024-05-14T16:52:13.212486+0000","last_peered":"2024-05-14T16:52:13.212486+0000","last_clean":"2024-05-14T16:52:13.212486+0000","last_became_active":"2024-05-14T16:51:32.300546+0000","last_became_peered":"2024-05-14T16:51:32.300546+0000","last_unstale":"2024-05-14T16:52:13.212486+0000","last_undegraded":"2024-05-14T16:52:13.212486+0000","last_fullsized":"2024-05-14T16:52:13.212486+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:54:09.181036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.212599+0000","last_change":"2024-05-14T16:51:29.345206+0000","last_active":"2024-05-14T16:52:13.212599+0000","last_peered":"2024-05-14T16:52:13.212599+0000","last_clean":"2024-05-14T16:52:13.212599+0000","last_became_active":"2024-05-14T16:51:29.344699+0000","last_became_peered":"2024-05-14T16:51:29.344699+0000","last_unstale":"2024-05-14T16:52:13.212599+0000","last_undegraded":"2024-05-14T16:52:13.212599+0000","last_fullsized":"2024-05-14T16:52:13.212599+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:14:02.364661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215868+0000","last_change":"2024-05-14T16:51:32.297522+0000","last_active":"2024-05-14T16:52:13.215868+0000","last_peered":"2024-05-14T16:52:13.215868+0000","last_clean":"2024-05-14T16:52:13.215868+0000","last_became_active":"2024-05-14T16:51:32.296866+0000","last_became_peered":"2024-05-14T16:51:32.296866+0000","last_unstale":"2024-05-14T16:52:13.215868+0000","last_undegraded":"2024-05-14T16:52:13.215868+0000","last_fullsized":"2024-05-14T16:52:13.215868+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:49:43.409575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.219763+0000","last_change":"2024-05-14T16:51:29.280632+0000","last_active":"2024-05-14T16:52:13.219763+0000","last_peered":"2024-05-14T16:52:13.219763+0000","last_clean":"2024-05-14T16:52:13.219763+0000","last_became_active":"2024-05-14T16:51:29.280246+0000","last_became_peered":"2024-05-14T16:51:29.280246+0000","last_unstale":"2024-05-14T16:52:13.219763+0000","last_undegraded":"2024-05-14T16:52:13.219763+0000","last_fullsized":"2024-05-14T16:52:13.219763+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:34:47.925344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.212665+0000","last_change":"2024-05-14T16:51:32.298284+0000","last_active":"2024-05-14T16:52:13.212665+0000","last_peered":"2024-05-14T16:52:13.212665+0000","last_clean":"2024-05-14T16:52:13.212665+0000","last_became_active":"2024-05-14T16:51:32.294202+0000","last_became_peered":"2024-05-14T16:51:32.294202+0000","last_unstale":"2024-05-14T16:52:13.212665+0000","last_undegraded":"2024-05-14T16:52:13.212665+0000","last_fullsized":"2024-05-14T16:52:13.212665+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:27:37.561275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.219221+0000","last_change":"2024-05-14T16:51:29.276227+0000","last_active":"2024-05-14T16:52:13.219221+0000","last_peered":"2024-05-14T16:52:13.219221+0000","last_clean":"2024-05-14T16:52:13.219221+0000","last_became_active":"2024-05-14T16:51:29.275837+0000","last_became_peered":"2024-05-14T16:51:29.275837+0000","last_unstale":"2024-05-14T16:52:13.219221+0000","last_undegraded":"2024-05-14T16:52:13.219221+0000","last_fullsized":"2024-05-14T16:52:13.219221+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:03:29.402985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.217958+0000","last_change":"2024-05-14T16:51:32.308173+0000","last_active":"2024-05-14T16:52:13.217958+0000","last_peered":"2024-05-14T16:52:13.217958+0000","last_clean":"2024-05-14T16:52:13.217958+0000","last_became_active":"2024-05-14T16:51:32.307135+0000","last_became_peered":"2024-05-14T16:51:32.307135+0000","last_unstale":"2024-05-14T16:52:13.217958+0000","last_undegraded":"2024-05-14T16:52:13.217958+0000","last_fullsized":"2024-05-14T16:52:13.217958+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:01:46.579968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"78'6","reported_seq":62,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:16.047801+0000","last_change":"2024-05-14T16:51:29.279486+0000","last_active":"2024-05-14T16:52:16.047801+0000","last_peered":"2024-05-14T16:52:16.047801+0000","last_clean":"2024-05-14T16:52:16.047801+0000","last_became_active":"2024-05-14T16:51:29.279133+0000","last_became_peered":"2024-05-14T16:51:29.279133+0000","last_unstale":"2024-05-14T16:52:16.047801+0000","last_undegraded":"2024-05-14T16:52:16.047801+0000","last_fullsized":"2024-05-14T16:52:16.047801+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:26:18.620421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.219450+0000","last_change":"2024-05-14T16:51:32.299852+0000","last_active":"2024-05-14T16:52:13.219450+0000","last_peered":"2024-05-14T16:52:13.219450+0000","last_clean":"2024-05-14T16:52:13.219450+0000","last_became_active":"2024-05-14T16:51:32.299541+0000","last_became_peered":"2024-05-14T16:51:32.299541+0000","last_unstale":"2024-05-14T16:52:13.219450+0000","last_undegraded":"2024-05-14T16:52:13.219450+0000","last_fullsized":"2024-05-14T16:52:13.219450+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:36:38.191009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.445109+0000","last_change":"2024-05-14T16:51:29.275269+0000","last_active":"2024-05-14T16:52:13.445109+0000","last_peered":"2024-05-14T16:52:13.445109+0000","last_clean":"2024-05-14T16:52:13.445109+0000","last_became_active":"2024-05-14T16:51:29.274792+0000","last_became_peered":"2024-05-14T16:51:29.274792+0000","last_unstale":"2024-05-14T16:52:13.445109+0000","last_undegraded":"2024-05-14T16:52:13.445109+0000","last_fullsized":"2024-05-14T16:52:13.445109+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:17:08.925289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.218110+0000","last_change":"2024-05-14T16:51:32.306536+0000","last_active":"2024-05-14T16:52:13.218110+0000","last_peered":"2024-05-14T16:52:13.218110+0000","last_clean":"2024-05-14T16:52:13.218110+0000","last_became_active":"2024-05-14T16:51:32.304976+0000","last_became_peered":"2024-05-14T16:51:32.304976+0000","last_unstale":"2024-05-14T16:52:13.218110+0000","last_undegraded":"2024-05-14T16:52:13.218110+0000","last_fullsized":"2024-05-14T16:52:13.218110+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:48:13.284022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.921481+0000","last_change":"2024-05-14T16:51:29.266783+0000","last_active":"2024-05-14T16:52:13.921481+0000","last_peered":"2024-05-14T16:52:13.921481+0000","last_clean":"2024-05-14T16:52:13.921481+0000","last_became_active":"2024-05-14T16:51:29.266409+0000","last_became_peered":"2024-05-14T16:51:29.266409+0000","last_unstale":"2024-05-14T16:52:13.921481+0000","last_undegraded":"2024-05-14T16:52:13.921481+0000","last_fullsized":"2024-05-14T16:52:13.921481+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:33:14.981709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.212099+0000","last_change":"2024-05-14T16:51:32.289288+0000","last_active":"2024-05-14T16:52:13.212099+0000","last_peered":"2024-05-14T16:52:13.212099+0000","last_clean":"2024-05-14T16:52:13.212099+0000","last_became_active":"2024-05-14T16:51:32.288920+0000","last_became_peered":"2024-05-14T16:51:32.288920+0000","last_unstale":"2024-05-14T16:52:13.212099+0000","last_undegraded":"2024-05-14T16:52:13.212099+0000","last_fullsized":"2024-05-14T16:52:13.212099+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:23:43.659417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.216697+0000","last_change":"2024-05-14T16:51:29.273926+0000","last_active":"2024-05-14T16:52:13.216697+0000","last_peered":"2024-05-14T16:52:13.216697+0000","last_clean":"2024-05-14T16:52:13.216697+0000","last_became_active":"2024-05-14T16:51:29.273059+0000","last_became_peered":"2024-05-14T16:51:29.273059+0000","last_unstale":"2024-05-14T16:52:13.216697+0000","last_undegraded":"2024-05-14T16:52:13.216697+0000","last_fullsized":"2024-05-14T16:52:13.216697+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:46:27.800481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.214785+0000","last_change":"2024-05-14T16:51:32.287554+0000","last_active":"2024-05-14T16:52:13.214785+0000","last_peered":"2024-05-14T16:52:13.214785+0000","last_clean":"2024-05-14T16:52:13.214785+0000","last_became_active":"2024-05-14T16:51:32.287204+0000","last_became_peered":"2024-05-14T16:51:32.287204+0000","last_unstale":"2024-05-14T16:52:13.214785+0000","last_undegraded":"2024-05-14T16:52:13.214785+0000","last_fullsized":"2024-05-14T16:52:13.214785+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:29:40.675378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":57,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:17.048685+0000","last_change":"2024-05-14T16:51:29.274819+0000","last_active":"2024-05-14T16:52:17.048685+0000","last_peered":"2024-05-14T16:52:17.048685+0000","last_clean":"2024-05-14T16:52:17.048685+0000","last_became_active":"2024-05-14T16:51:29.274454+0000","last_became_peered":"2024-05-14T16:51:29.274454+0000","last_unstale":"2024-05-14T16:52:17.048685+0000","last_undegraded":"2024-05-14T16:52:17.048685+0000","last_fullsized":"2024-05-14T16:52:17.048685+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:36:30.496064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":28,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.213180+0000","last_change":"2024-05-14T16:51:32.295732+0000","last_active":"2024-05-14T16:52:13.213180+0000","last_peered":"2024-05-14T16:52:13.213180+0000","last_clean":"2024-05-14T16:52:13.213180+0000","last_became_active":"2024-05-14T16:51:32.294919+0000","last_became_peered":"2024-05-14T16:51:32.294919+0000","last_unstale":"2024-05-14T16:52:13.213180+0000","last_undegraded":"2024-05-14T16:52:13.213180+0000","last_fullsized":"2024-05-14T16:52:13.213180+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:08:54.566601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215472+0000","last_change":"2024-05-14T16:51:29.279338+0000","last_active":"2024-05-14T16:52:13.215472+0000","last_peered":"2024-05-14T16:52:13.215472+0000","last_clean":"2024-05-14T16:52:13.215472+0000","last_became_active":"2024-05-14T16:51:29.278928+0000","last_became_peered":"2024-05-14T16:51:29.278928+0000","last_unstale":"2024-05-14T16:52:13.215472+0000","last_undegraded":"2024-05-14T16:52:13.215472+0000","last_fullsized":"2024-05-14T16:52:13.215472+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:24:34.803515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215986+0000","last_change":"2024-05-14T16:51:32.305294+0000","last_active":"2024-05-14T16:52:13.215986+0000","last_peered":"2024-05-14T16:52:13.215986+0000","last_clean":"2024-05-14T16:52:13.215986+0000","last_became_active":"2024-05-14T16:51:32.304563+0000","last_became_peered":"2024-05-14T16:51:32.304563+0000","last_unstale":"2024-05-14T16:52:13.215986+0000","last_undegraded":"2024-05-14T16:52:13.215986+0000","last_fullsized":"2024-05-14T16:52:13.215986+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:56:06.259475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.218071+0000","last_change":"2024-05-14T16:51:29.345400+0000","last_active":"2024-05-14T16:52:13.218071+0000","last_peered":"2024-05-14T16:52:13.218071+0000","last_clean":"2024-05-14T16:52:13.218071+0000","last_became_active":"2024-05-14T16:51:29.345010+0000","last_became_peered":"2024-05-14T16:51:29.345010+0000","last_unstale":"2024-05-14T16:52:13.218071+0000","last_undegraded":"2024-05-14T16:52:13.218071+0000","last_fullsized":"2024-05-14T16:52:13.218071+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:24:22.933005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.923167+0000","last_change":"2024-05-14T16:52:13.923167+0000","last_active":"2024-05-14T16:52:13.923167+0000","last_peered":"2024-05-14T16:52:13.923167+0000","last_clean":"2024-05-14T16:52:13.923167+0000","last_became_active":"2024-05-14T16:52:13.922813+0000","last_became_peered":"2024-05-14T16:52:13.922813+0000","last_unstale":"2024-05-14T16:52:13.923167+0000","last_undegraded":"2024-05-14T16:52:13.923167+0000","last_fullsized":"2024-05-14T16:52:13.923167+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:50:38.672408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'30","reported_seq":120,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:18.445801+0000","last_change":"2024-05-14T16:51:29.277591+0000","last_active":"2024-05-14T16:52:18.445801+0000","last_peered":"2024-05-14T16:52:18.445801+0000","last_clean":"2024-05-14T16:52:18.445801+0000","last_became_active":"2024-05-14T16:51:29.276612+0000","last_became_peered":"2024-05-14T16:51:29.276612+0000","last_unstale":"2024-05-14T16:52:18.445801+0000","last_undegraded":"2024-05-14T16:52:18.445801+0000","last_fullsized":"2024-05-14T16:52:18.445801+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:22:28.934173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61241,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":64,"num_read_kb":84,"num_write":31,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.218460+0000","last_change":"2024-05-14T16:51:32.302321+0000","last_active":"2024-05-14T16:52:13.218460+0000","last_peered":"2024-05-14T16:52:13.218460+0000","last_clean":"2024-05-14T16:52:13.218460+0000","last_became_active":"2024-05-14T16:51:32.301973+0000","last_became_peered":"2024-05-14T16:51:32.301973+0000","last_unstale":"2024-05-14T16:52:13.218460+0000","last_undegraded":"2024-05-14T16:52:13.218460+0000","last_fullsized":"2024-05-14T16:52:13.218460+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:49:24.054379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.218367+0000","last_change":"2024-05-14T16:51:29.278776+0000","last_active":"2024-05-14T16:52:13.218367+0000","last_peered":"2024-05-14T16:52:13.218367+0000","last_clean":"2024-05-14T16:52:13.218367+0000","last_became_active":"2024-05-14T16:51:29.278395+0000","last_became_peered":"2024-05-14T16:51:29.278395+0000","last_unstale":"2024-05-14T16:52:13.218367+0000","last_undegraded":"2024-05-14T16:52:13.218367+0000","last_fullsized":"2024-05-14T16:52:13.218367+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:29:20.942783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.921447+0000","last_change":"2024-05-14T16:51:32.292107+0000","last_active":"2024-05-14T16:52:13.921447+0000","last_peered":"2024-05-14T16:52:13.921447+0000","last_clean":"2024-05-14T16:52:13.921447+0000","last_became_active":"2024-05-14T16:51:32.291729+0000","last_became_peered":"2024-05-14T16:51:32.291729+0000","last_unstale":"2024-05-14T16:52:13.921447+0000","last_undegraded":"2024-05-14T16:52:13.921447+0000","last_fullsized":"2024-05-14T16:52:13.921447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:30:15.440599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.216788+0000","last_change":"2024-05-14T16:51:29.274862+0000","last_active":"2024-05-14T16:52:13.216788+0000","last_peered":"2024-05-14T16:52:13.216788+0000","last_clean":"2024-05-14T16:52:13.216788+0000","last_became_active":"2024-05-14T16:51:29.274527+0000","last_became_peered":"2024-05-14T16:51:29.274527+0000","last_unstale":"2024-05-14T16:52:13.216788+0000","last_undegraded":"2024-05-14T16:52:13.216788+0000","last_fullsized":"2024-05-14T16:52:13.216788+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:43:15.634585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215503+0000","last_change":"2024-05-14T16:51:32.306158+0000","last_active":"2024-05-14T16:52:13.215503+0000","last_peered":"2024-05-14T16:52:13.215503+0000","last_clean":"2024-05-14T16:52:13.215503+0000","last_became_active":"2024-05-14T16:51:32.305834+0000","last_became_peered":"2024-05-14T16:51:32.305834+0000","last_unstale":"2024-05-14T16:52:13.215503+0000","last_undegraded":"2024-05-14T16:52:13.215503+0000","last_fullsized":"2024-05-14T16:52:13.215503+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:31:47.645862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'3","reported_seq":58,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:16.047649+0000","last_change":"2024-05-14T16:51:29.270735+0000","last_active":"2024-05-14T16:52:16.047649+0000","last_peered":"2024-05-14T16:52:16.047649+0000","last_clean":"2024-05-14T16:52:16.047649+0000","last_became_active":"2024-05-14T16:51:29.270362+0000","last_became_peered":"2024-05-14T16:51:29.270362+0000","last_unstale":"2024-05-14T16:52:16.047649+0000","last_undegraded":"2024-05-14T16:52:16.047649+0000","last_fullsized":"2024-05-14T16:52:16.047649+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:40:12.288655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.214882+0000","last_change":"2024-05-14T16:51:32.305007+0000","last_active":"2024-05-14T16:52:13.214882+0000","last_peered":"2024-05-14T16:52:13.214882+0000","last_clean":"2024-05-14T16:52:13.214882+0000","last_became_active":"2024-05-14T16:51:32.303927+0000","last_became_peered":"2024-05-14T16:51:32.303927+0000","last_unstale":"2024-05-14T16:52:13.214882+0000","last_undegraded":"2024-05-14T16:52:13.214882+0000","last_fullsized":"2024-05-14T16:52:13.214882+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:42:28.018492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":42,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.218464+0000","last_change":"2024-05-14T16:51:29.345902+0000","last_active":"2024-05-14T16:52:13.218464+0000","last_peered":"2024-05-14T16:52:13.218464+0000","last_clean":"2024-05-14T16:52:13.218464+0000","last_became_active":"2024-05-14T16:51:29.344904+0000","last_became_peered":"2024-05-14T16:51:29.344904+0000","last_unstale":"2024-05-14T16:52:13.218464+0000","last_undegraded":"2024-05-14T16:52:13.218464+0000","last_fullsized":"2024-05-14T16:52:13.218464+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:56:22.209067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.217001+0000","last_change":"2024-05-14T16:51:32.306459+0000","last_active":"2024-05-14T16:52:13.217001+0000","last_peered":"2024-05-14T16:52:13.217001+0000","last_clean":"2024-05-14T16:52:13.217001+0000","last_became_active":"2024-05-14T16:51:32.304897+0000","last_became_peered":"2024-05-14T16:51:32.304897+0000","last_unstale":"2024-05-14T16:52:13.217001+0000","last_undegraded":"2024-05-14T16:52:13.217001+0000","last_fullsized":"2024-05-14T16:52:13.217001+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:09:51.216631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.219952+0000","last_change":"2024-05-14T16:51:29.275219+0000","last_active":"2024-05-14T16:52:13.219952+0000","last_peered":"2024-05-14T16:52:13.219952+0000","last_clean":"2024-05-14T16:52:13.219952+0000","last_became_active":"2024-05-14T16:51:29.274816+0000","last_became_peered":"2024-05-14T16:51:29.274816+0000","last_unstale":"2024-05-14T16:52:13.219952+0000","last_undegraded":"2024-05-14T16:52:13.219952+0000","last_fullsized":"2024-05-14T16:52:13.219952+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:55:49.667387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215144+0000","last_change":"2024-05-14T16:51:32.299284+0000","last_active":"2024-05-14T16:52:13.215144+0000","last_peered":"2024-05-14T16:52:13.215144+0000","last_clean":"2024-05-14T16:52:13.215144+0000","last_became_active":"2024-05-14T16:51:32.298933+0000","last_became_peered":"2024-05-14T16:51:32.298933+0000","last_unstale":"2024-05-14T16:52:13.215144+0000","last_undegraded":"2024-05-14T16:52:13.215144+0000","last_fullsized":"2024-05-14T16:52:13.215144+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:35:42.266665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.214563+0000","last_change":"2024-05-14T16:51:29.345454+0000","last_active":"2024-05-14T16:52:13.214563+0000","last_peered":"2024-05-14T16:52:13.214563+0000","last_clean":"2024-05-14T16:52:13.214563+0000","last_became_active":"2024-05-14T16:51:29.345074+0000","last_became_peered":"2024-05-14T16:51:29.345074+0000","last_unstale":"2024-05-14T16:52:13.214563+0000","last_undegraded":"2024-05-14T16:52:13.214563+0000","last_fullsized":"2024-05-14T16:52:13.214563+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:56:11.661144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.445213+0000","last_change":"2024-05-14T16:51:32.288804+0000","last_active":"2024-05-14T16:52:13.445213+0000","last_peered":"2024-05-14T16:52:13.445213+0000","last_clean":"2024-05-14T16:52:13.445213+0000","last_became_active":"2024-05-14T16:51:32.288441+0000","last_became_peered":"2024-05-14T16:51:32.288441+0000","last_unstale":"2024-05-14T16:52:13.445213+0000","last_undegraded":"2024-05-14T16:52:13.445213+0000","last_fullsized":"2024-05-14T16:52:13.445213+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:28:31.720342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215130+0000","last_change":"2024-05-14T16:51:29.272032+0000","last_active":"2024-05-14T16:52:13.215130+0000","last_peered":"2024-05-14T16:52:13.215130+0000","last_clean":"2024-05-14T16:52:13.215130+0000","last_became_active":"2024-05-14T16:51:29.271677+0000","last_became_peered":"2024-05-14T16:51:29.271677+0000","last_unstale":"2024-05-14T16:52:13.215130+0000","last_undegraded":"2024-05-14T16:52:13.215130+0000","last_fullsized":"2024-05-14T16:52:13.215130+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:39:41.377591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.445317+0000","last_change":"2024-05-14T16:51:32.295159+0000","last_active":"2024-05-14T16:52:13.445317+0000","last_peered":"2024-05-14T16:52:13.445317+0000","last_clean":"2024-05-14T16:52:13.445317+0000","last_became_active":"2024-05-14T16:51:32.294812+0000","last_became_peered":"2024-05-14T16:51:32.294812+0000","last_unstale":"2024-05-14T16:52:13.445317+0000","last_undegraded":"2024-05-14T16:52:13.445317+0000","last_fullsized":"2024-05-14T16:52:13.445317+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:08:22.411800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.217087+0000","last_change":"2024-05-14T16:51:32.306582+0000","last_active":"2024-05-14T16:52:13.217087+0000","last_peered":"2024-05-14T16:52:13.217087+0000","last_clean":"2024-05-14T16:52:13.217087+0000","last_became_active":"2024-05-14T16:51:32.306209+0000","last_became_peered":"2024-05-14T16:51:32.306209+0000","last_unstale":"2024-05-14T16:52:13.217087+0000","last_undegraded":"2024-05-14T16:52:13.217087+0000","last_fullsized":"2024-05-14T16:52:13.217087+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:30:46.264097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"78'4","reported_seq":59,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:18.049917+0000","last_change":"2024-05-14T16:51:29.345057+0000","last_active":"2024-05-14T16:52:18.049917+0000","last_peered":"2024-05-14T16:52:18.049917+0000","last_clean":"2024-05-14T16:52:18.049917+0000","last_became_active":"2024-05-14T16:51:29.344658+0000","last_became_peered":"2024-05-14T16:51:29.344658+0000","last_unstale":"2024-05-14T16:52:18.049917+0000","last_undegraded":"2024-05-14T16:52:18.049917+0000","last_fullsized":"2024-05-14T16:52:18.049917+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:43:53.007543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.212440+0000","last_change":"2024-05-14T16:51:32.298958+0000","last_active":"2024-05-14T16:52:13.212440+0000","last_peered":"2024-05-14T16:52:13.212440+0000","last_clean":"2024-05-14T16:52:13.212440+0000","last_became_active":"2024-05-14T16:51:32.298420+0000","last_became_peered":"2024-05-14T16:51:32.298420+0000","last_unstale":"2024-05-14T16:52:13.212440+0000","last_undegraded":"2024-05-14T16:52:13.212440+0000","last_fullsized":"2024-05-14T16:52:13.212440+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:15:21.262026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.921575+0000","last_change":"2024-05-14T16:51:29.272082+0000","last_active":"2024-05-14T16:52:13.921575+0000","last_peered":"2024-05-14T16:52:13.921575+0000","last_clean":"2024-05-14T16:52:13.921575+0000","last_became_active":"2024-05-14T16:51:29.271711+0000","last_became_peered":"2024-05-14T16:51:29.271711+0000","last_unstale":"2024-05-14T16:52:13.921575+0000","last_undegraded":"2024-05-14T16:52:13.921575+0000","last_fullsized":"2024-05-14T16:52:13.921575+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:27:44.320423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":99,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215003+0000","last_change":"2024-05-14T16:50:05.320136+0000","last_active":"2024-05-14T16:52:13.215003+0000","last_peered":"2024-05-14T16:52:13.215003+0000","last_clean":"2024-05-14T16:52:13.215003+0000","last_became_active":"2024-05-14T16:50:05.315016+0000","last_became_peered":"2024-05-14T16:50:05.315016+0000","last_unstale":"2024-05-14T16:52:13.215003+0000","last_undegraded":"2024-05-14T16:52:13.215003+0000","last_fullsized":"2024-05-14T16:52:13.215003+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:38.138211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:38.138211+0000","last_clean_scrub_stamp":"2024-05-14T16:46:38.138211+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:19:09.012578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.921326+0000","last_change":"2024-05-14T16:51:32.295995+0000","last_active":"2024-05-14T16:52:13.921326+0000","last_peered":"2024-05-14T16:52:13.921326+0000","last_clean":"2024-05-14T16:52:13.921326+0000","last_became_active":"2024-05-14T16:51:32.295640+0000","last_became_peered":"2024-05-14T16:51:32.295640+0000","last_unstale":"2024-05-14T16:52:13.921326+0000","last_undegraded":"2024-05-14T16:52:13.921326+0000","last_fullsized":"2024-05-14T16:52:13.921326+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:42:26.774916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215122+0000","last_change":"2024-05-14T16:51:29.274892+0000","last_active":"2024-05-14T16:52:13.215122+0000","last_peered":"2024-05-14T16:52:13.215122+0000","last_clean":"2024-05-14T16:52:13.215122+0000","last_became_active":"2024-05-14T16:51:29.274386+0000","last_became_peered":"2024-05-14T16:51:29.274386+0000","last_unstale":"2024-05-14T16:52:13.215122+0000","last_undegraded":"2024-05-14T16:52:13.215122+0000","last_fullsized":"2024-05-14T16:52:13.215122+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:16:54.899320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.219605+0000","last_change":"2024-05-14T16:51:29.345672+0000","last_active":"2024-05-14T16:52:13.219605+0000","last_peered":"2024-05-14T16:52:13.219605+0000","last_clean":"2024-05-14T16:52:13.219605+0000","last_became_active":"2024-05-14T16:51:29.344948+0000","last_became_peered":"2024-05-14T16:51:29.344948+0000","last_unstale":"2024-05-14T16:52:13.219605+0000","last_undegraded":"2024-05-14T16:52:13.219605+0000","last_fullsized":"2024-05-14T16:52:13.219605+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:19:26.577799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.921944+0000","last_change":"2024-05-14T16:51:32.297360+0000","last_active":"2024-05-14T16:52:13.921944+0000","last_peered":"2024-05-14T16:52:13.921944+0000","last_clean":"2024-05-14T16:52:13.921944+0000","last_became_active":"2024-05-14T16:51:32.297025+0000","last_became_peered":"2024-05-14T16:51:32.297025+0000","last_unstale":"2024-05-14T16:52:13.921944+0000","last_undegraded":"2024-05-14T16:52:13.921944+0000","last_fullsized":"2024-05-14T16:52:13.921944+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:05:39.705860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'10","reported_seq":48,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.224732+0000","last_change":"2024-05-14T16:52:13.224328+0000","last_active":"2024-05-14T16:52:13.224732+0000","last_peered":"2024-05-14T16:52:13.224732+0000","last_clean":"2024-05-14T16:52:13.224732+0000","last_became_active":"2024-05-14T16:52:13.220271+0000","last_became_peered":"2024-05-14T16:52:13.220271+0000","last_unstale":"2024-05-14T16:52:13.224732+0000","last_undegraded":"2024-05-14T16:52:13.224732+0000","last_fullsized":"2024-05-14T16:52:13.224732+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:06:31.955274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.212576+0000","last_change":"2024-05-14T16:51:32.292112+0000","last_active":"2024-05-14T16:52:13.212576+0000","last_peered":"2024-05-14T16:52:13.212576+0000","last_clean":"2024-05-14T16:52:13.212576+0000","last_became_active":"2024-05-14T16:51:32.291688+0000","last_became_peered":"2024-05-14T16:51:32.291688+0000","last_unstale":"2024-05-14T16:52:13.212576+0000","last_undegraded":"2024-05-14T16:52:13.212576+0000","last_fullsized":"2024-05-14T16:52:13.212576+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:13:06.728608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.921942+0000","last_change":"2024-05-14T16:51:29.412805+0000","last_active":"2024-05-14T16:52:13.921942+0000","last_peered":"2024-05-14T16:52:13.921942+0000","last_clean":"2024-05-14T16:52:13.921942+0000","last_became_active":"2024-05-14T16:51:29.412394+0000","last_became_peered":"2024-05-14T16:51:29.412394+0000","last_unstale":"2024-05-14T16:52:13.921942+0000","last_undegraded":"2024-05-14T16:52:13.921942+0000","last_fullsized":"2024-05-14T16:52:13.921942+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:49:48.721104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.218227+0000","last_change":"2024-05-14T16:52:13.218227+0000","last_active":"2024-05-14T16:52:13.218227+0000","last_peered":"2024-05-14T16:52:13.218227+0000","last_clean":"2024-05-14T16:52:13.218227+0000","last_became_active":"2024-05-14T16:52:13.217787+0000","last_became_peered":"2024-05-14T16:52:13.217787+0000","last_unstale":"2024-05-14T16:52:13.218227+0000","last_undegraded":"2024-05-14T16:52:13.218227+0000","last_fullsized":"2024-05-14T16:52:13.218227+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:52:33.348804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.212605+0000","last_change":"2024-05-14T16:51:29.345427+0000","last_active":"2024-05-14T16:52:13.212605+0000","last_peered":"2024-05-14T16:52:13.212605+0000","last_clean":"2024-05-14T16:52:13.212605+0000","last_became_active":"2024-05-14T16:51:29.345049+0000","last_became_peered":"2024-05-14T16:51:29.345049+0000","last_unstale":"2024-05-14T16:52:13.212605+0000","last_undegraded":"2024-05-14T16:52:13.212605+0000","last_fullsized":"2024-05-14T16:52:13.212605+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:06:19.799660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.218616+0000","last_change":"2024-05-14T16:51:32.298757+0000","last_active":"2024-05-14T16:52:13.218616+0000","last_peered":"2024-05-14T16:52:13.218616+0000","last_clean":"2024-05-14T16:52:13.218616+0000","last_became_active":"2024-05-14T16:51:32.297193+0000","last_became_peered":"2024-05-14T16:51:32.297193+0000","last_unstale":"2024-05-14T16:52:13.218616+0000","last_undegraded":"2024-05-14T16:52:13.218616+0000","last_fullsized":"2024-05-14T16:52:13.218616+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:10:19.672230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.212297+0000","last_change":"2024-05-14T16:51:29.270545+0000","last_active":"2024-05-14T16:52:13.212297+0000","last_peered":"2024-05-14T16:52:13.212297+0000","last_clean":"2024-05-14T16:52:13.212297+0000","last_became_active":"2024-05-14T16:51:29.270182+0000","last_became_peered":"2024-05-14T16:51:29.270182+0000","last_unstale":"2024-05-14T16:52:13.212297+0000","last_undegraded":"2024-05-14T16:52:13.212297+0000","last_fullsized":"2024-05-14T16:52:13.212297+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:19:49.977916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.214764+0000","last_change":"2024-05-14T16:51:32.294767+0000","last_active":"2024-05-14T16:52:13.214764+0000","last_peered":"2024-05-14T16:52:13.214764+0000","last_clean":"2024-05-14T16:52:13.214764+0000","last_became_active":"2024-05-14T16:51:32.293947+0000","last_became_peered":"2024-05-14T16:51:32.293947+0000","last_unstale":"2024-05-14T16:52:13.214764+0000","last_undegraded":"2024-05-14T16:52:13.214764+0000","last_fullsized":"2024-05-14T16:52:13.214764+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:30:21.731492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.226988+0000","last_change":"2024-05-14T16:52:13.226509+0000","last_active":"2024-05-14T16:52:13.226988+0000","last_peered":"2024-05-14T16:52:13.226988+0000","last_clean":"2024-05-14T16:52:13.226988+0000","last_became_active":"2024-05-14T16:52:13.220923+0000","last_became_peered":"2024-05-14T16:52:13.220923+0000","last_unstale":"2024-05-14T16:52:13.226988+0000","last_undegraded":"2024-05-14T16:52:13.226988+0000","last_fullsized":"2024-05-14T16:52:13.226988+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:25:52.655481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.212639+0000","last_change":"2024-05-14T16:51:32.288810+0000","last_active":"2024-05-14T16:52:13.212639+0000","last_peered":"2024-05-14T16:52:13.212639+0000","last_clean":"2024-05-14T16:52:13.212639+0000","last_became_active":"2024-05-14T16:51:32.288492+0000","last_became_peered":"2024-05-14T16:51:32.288492+0000","last_unstale":"2024-05-14T16:52:13.212639+0000","last_undegraded":"2024-05-14T16:52:13.212639+0000","last_fullsized":"2024-05-14T16:52:13.212639+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:44:53.393371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.218848+0000","last_change":"2024-05-14T16:51:29.281181+0000","last_active":"2024-05-14T16:52:13.218848+0000","last_peered":"2024-05-14T16:52:13.218848+0000","last_clean":"2024-05-14T16:52:13.218848+0000","last_became_active":"2024-05-14T16:51:29.279965+0000","last_became_peered":"2024-05-14T16:51:29.279965+0000","last_unstale":"2024-05-14T16:52:13.218848+0000","last_undegraded":"2024-05-14T16:52:13.218848+0000","last_fullsized":"2024-05-14T16:52:13.218848+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:41:00.735291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215266+0000","last_change":"2024-05-14T16:51:32.296971+0000","last_active":"2024-05-14T16:52:13.215266+0000","last_peered":"2024-05-14T16:52:13.215266+0000","last_clean":"2024-05-14T16:52:13.215266+0000","last_became_active":"2024-05-14T16:51:32.296600+0000","last_became_peered":"2024-05-14T16:51:32.296600+0000","last_unstale":"2024-05-14T16:52:13.215266+0000","last_undegraded":"2024-05-14T16:52:13.215266+0000","last_fullsized":"2024-05-14T16:52:13.215266+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:13:14.354127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.219192+0000","last_change":"2024-05-14T16:51:29.412682+0000","last_active":"2024-05-14T16:52:13.219192+0000","last_peered":"2024-05-14T16:52:13.219192+0000","last_clean":"2024-05-14T16:52:13.219192+0000","last_became_active":"2024-05-14T16:51:29.412186+0000","last_became_peered":"2024-05-14T16:51:29.412186+0000","last_unstale":"2024-05-14T16:52:13.219192+0000","last_undegraded":"2024-05-14T16:52:13.219192+0000","last_fullsized":"2024-05-14T16:52:13.219192+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:56:24.570276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215347+0000","last_change":"2024-05-14T16:51:32.300626+0000","last_active":"2024-05-14T16:52:13.215347+0000","last_peered":"2024-05-14T16:52:13.215347+0000","last_clean":"2024-05-14T16:52:13.215347+0000","last_became_active":"2024-05-14T16:51:32.300278+0000","last_became_peered":"2024-05-14T16:51:32.300278+0000","last_unstale":"2024-05-14T16:52:13.215347+0000","last_undegraded":"2024-05-14T16:52:13.215347+0000","last_fullsized":"2024-05-14T16:52:13.215347+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:59:14.196315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.216758+0000","last_change":"2024-05-14T16:51:29.270826+0000","last_active":"2024-05-14T16:52:13.216758+0000","last_peered":"2024-05-14T16:52:13.216758+0000","last_clean":"2024-05-14T16:52:13.216758+0000","last_became_active":"2024-05-14T16:51:29.270266+0000","last_became_peered":"2024-05-14T16:51:29.270266+0000","last_unstale":"2024-05-14T16:52:13.216758+0000","last_undegraded":"2024-05-14T16:52:13.216758+0000","last_fullsized":"2024-05-14T16:52:13.216758+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:00:55.349278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.223994+0000","last_change":"2024-05-14T16:52:13.223994+0000","last_active":"2024-05-14T16:52:13.223994+0000","last_peered":"2024-05-14T16:52:13.223994+0000","last_clean":"2024-05-14T16:52:13.223994+0000","last_became_active":"2024-05-14T16:52:13.221071+0000","last_became_peered":"2024-05-14T16:52:13.221071+0000","last_unstale":"2024-05-14T16:52:13.223994+0000","last_undegraded":"2024-05-14T16:52:13.223994+0000","last_fullsized":"2024-05-14T16:52:13.223994+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:29:58.775171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.921947+0000","last_change":"2024-05-14T16:51:29.276705+0000","last_active":"2024-05-14T16:52:13.921947+0000","last_peered":"2024-05-14T16:52:13.921947+0000","last_clean":"2024-05-14T16:52:13.921947+0000","last_became_active":"2024-05-14T16:51:29.276347+0000","last_became_peered":"2024-05-14T16:51:29.276347+0000","last_unstale":"2024-05-14T16:52:13.921947+0000","last_undegraded":"2024-05-14T16:52:13.921947+0000","last_fullsized":"2024-05-14T16:52:13.921947+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:32:56.064427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215638+0000","last_change":"2024-05-14T16:51:32.301617+0000","last_active":"2024-05-14T16:52:13.215638+0000","last_peered":"2024-05-14T16:52:13.215638+0000","last_clean":"2024-05-14T16:52:13.215638+0000","last_became_active":"2024-05-14T16:51:32.301288+0000","last_became_peered":"2024-05-14T16:51:32.301288+0000","last_unstale":"2024-05-14T16:52:13.215638+0000","last_undegraded":"2024-05-14T16:52:13.215638+0000","last_fullsized":"2024-05-14T16:52:13.215638+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:19:24.253633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.221694+0000","last_change":"2024-05-14T16:52:13.221694+0000","last_active":"2024-05-14T16:52:13.221694+0000","last_peered":"2024-05-14T16:52:13.221694+0000","last_clean":"2024-05-14T16:52:13.221694+0000","last_became_active":"2024-05-14T16:52:13.221264+0000","last_became_peered":"2024-05-14T16:52:13.221264+0000","last_unstale":"2024-05-14T16:52:13.221694+0000","last_undegraded":"2024-05-14T16:52:13.221694+0000","last_fullsized":"2024-05-14T16:52:13.221694+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:22:52.738772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.218817+0000","last_change":"2024-05-14T16:51:32.301789+0000","last_active":"2024-05-14T16:52:13.218817+0000","last_peered":"2024-05-14T16:52:13.218817+0000","last_clean":"2024-05-14T16:52:13.218817+0000","last_became_active":"2024-05-14T16:51:32.301350+0000","last_became_peered":"2024-05-14T16:51:32.301350+0000","last_unstale":"2024-05-14T16:52:13.218817+0000","last_undegraded":"2024-05-14T16:52:13.218817+0000","last_fullsized":"2024-05-14T16:52:13.218817+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:46:24.896855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.218398+0000","last_change":"2024-05-14T16:51:32.295978+0000","last_active":"2024-05-14T16:52:13.218398+0000","last_peered":"2024-05-14T16:52:13.218398+0000","last_clean":"2024-05-14T16:52:13.218398+0000","last_became_active":"2024-05-14T16:51:32.295652+0000","last_became_peered":"2024-05-14T16:51:32.295652+0000","last_unstale":"2024-05-14T16:52:13.218398+0000","last_undegraded":"2024-05-14T16:52:13.218398+0000","last_fullsized":"2024-05-14T16:52:13.218398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:32:17.743734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215754+0000","last_change":"2024-05-14T16:51:32.305568+0000","last_active":"2024-05-14T16:52:13.215754+0000","last_peered":"2024-05-14T16:52:13.215754+0000","last_clean":"2024-05-14T16:52:13.215754+0000","last_became_active":"2024-05-14T16:51:32.304580+0000","last_became_peered":"2024-05-14T16:51:32.304580+0000","last_unstale":"2024-05-14T16:52:13.215754+0000","last_undegraded":"2024-05-14T16:52:13.215754+0000","last_fullsized":"2024-05-14T16:52:13.215754+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:32:44.465054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.922077+0000","last_change":"2024-05-14T16:51:32.295029+0000","last_active":"2024-05-14T16:52:13.922077+0000","last_peered":"2024-05-14T16:52:13.922077+0000","last_clean":"2024-05-14T16:52:13.922077+0000","last_became_active":"2024-05-14T16:51:32.294621+0000","last_became_peered":"2024-05-14T16:51:32.294621+0000","last_unstale":"2024-05-14T16:52:13.922077+0000","last_undegraded":"2024-05-14T16:52:13.922077+0000","last_fullsized":"2024-05-14T16:52:13.922077+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:52:05.390141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.214978+0000","last_change":"2024-05-14T16:51:32.297671+0000","last_active":"2024-05-14T16:52:13.214978+0000","last_peered":"2024-05-14T16:52:13.214978+0000","last_clean":"2024-05-14T16:52:13.214978+0000","last_became_active":"2024-05-14T16:51:32.296210+0000","last_became_peered":"2024-05-14T16:51:32.296210+0000","last_unstale":"2024-05-14T16:52:13.214978+0000","last_undegraded":"2024-05-14T16:52:13.214978+0000","last_fullsized":"2024-05-14T16:52:13.214978+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:55:04.789021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.219098+0000","last_change":"2024-05-14T16:51:32.297404+0000","last_active":"2024-05-14T16:52:13.219098+0000","last_peered":"2024-05-14T16:52:13.219098+0000","last_clean":"2024-05-14T16:52:13.219098+0000","last_became_active":"2024-05-14T16:51:32.296856+0000","last_became_peered":"2024-05-14T16:51:32.296856+0000","last_unstale":"2024-05-14T16:52:13.219098+0000","last_undegraded":"2024-05-14T16:52:13.219098+0000","last_fullsized":"2024-05-14T16:52:13.219098+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:25:29.992875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.221379+0000","last_change":"2024-05-14T16:52:13.221379+0000","last_active":"2024-05-14T16:52:13.221379+0000","last_peered":"2024-05-14T16:52:13.221379+0000","last_clean":"2024-05-14T16:52:13.221379+0000","last_became_active":"2024-05-14T16:52:13.220973+0000","last_became_peered":"2024-05-14T16:52:13.220973+0000","last_unstale":"2024-05-14T16:52:13.221379+0000","last_undegraded":"2024-05-14T16:52:13.221379+0000","last_fullsized":"2024-05-14T16:52:13.221379+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:55:08.212344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.216825+0000","last_change":"2024-05-14T16:51:32.303570+0000","last_active":"2024-05-14T16:52:13.216825+0000","last_peered":"2024-05-14T16:52:13.216825+0000","last_clean":"2024-05-14T16:52:13.216825+0000","last_became_active":"2024-05-14T16:51:32.303234+0000","last_became_peered":"2024-05-14T16:51:32.303234+0000","last_unstale":"2024-05-14T16:52:13.216825+0000","last_undegraded":"2024-05-14T16:52:13.216825+0000","last_fullsized":"2024-05-14T16:52:13.216825+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:27:57.960120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.212132+0000","last_change":"2024-05-14T16:51:32.298028+0000","last_active":"2024-05-14T16:52:13.212132+0000","last_peered":"2024-05-14T16:52:13.212132+0000","last_clean":"2024-05-14T16:52:13.212132+0000","last_became_active":"2024-05-14T16:51:32.295104+0000","last_became_peered":"2024-05-14T16:51:32.295104+0000","last_unstale":"2024-05-14T16:52:13.212132+0000","last_undegraded":"2024-05-14T16:52:13.212132+0000","last_fullsized":"2024-05-14T16:52:13.212132+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:58:08.441590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.212921+0000","last_change":"2024-05-14T16:51:32.304458+0000","last_active":"2024-05-14T16:52:13.212921+0000","last_peered":"2024-05-14T16:52:13.212921+0000","last_clean":"2024-05-14T16:52:13.212921+0000","last_became_active":"2024-05-14T16:51:32.302952+0000","last_became_peered":"2024-05-14T16:51:32.302952+0000","last_unstale":"2024-05-14T16:52:13.212921+0000","last_undegraded":"2024-05-14T16:52:13.212921+0000","last_fullsized":"2024-05-14T16:52:13.212921+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:40:14.220075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.921611+0000","last_change":"2024-05-14T16:51:32.292334+0000","last_active":"2024-05-14T16:52:13.921611+0000","last_peered":"2024-05-14T16:52:13.921611+0000","last_clean":"2024-05-14T16:52:13.921611+0000","last_became_active":"2024-05-14T16:51:32.291769+0000","last_became_peered":"2024-05-14T16:51:32.291769+0000","last_unstale":"2024-05-14T16:52:13.921611+0000","last_undegraded":"2024-05-14T16:52:13.921611+0000","last_fullsized":"2024-05-14T16:52:13.921611+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:55:25.694827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.214789+0000","last_change":"2024-05-14T16:51:32.300044+0000","last_active":"2024-05-14T16:52:13.214789+0000","last_peered":"2024-05-14T16:52:13.214789+0000","last_clean":"2024-05-14T16:52:13.214789+0000","last_became_active":"2024-05-14T16:51:32.299630+0000","last_became_peered":"2024-05-14T16:51:32.299630+0000","last_unstale":"2024-05-14T16:52:13.214789+0000","last_undegraded":"2024-05-14T16:52:13.214789+0000","last_fullsized":"2024-05-14T16:52:13.214789+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:50:40.516973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284433,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65242,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":194,"num_read_kb":160,"num_write":164,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":183236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710744,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":864256,"data_stored":425130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906974,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1011712,"data_stored":1071607,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070498,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":425483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266728,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":425286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462957,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":1017216,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626483,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36188,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708964,"statfs":{"total":95995035648,"available":95957979136,"internally_reserved":0,"allocated":868352,"data_stored":424755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":479544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36212,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708940,"statfs":{"total":95995035648,"available":95957954560,"internally_reserved":0,"allocated":888832,"data_stored":426026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":950272,"data_stored":481285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182465,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36204,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708948,"statfs":{"total":95995035648,"available":95957962752,"internally_reserved":0,"allocated":880640,"data_stored":424949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986237,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":872448,"data_stored":425192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822712,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":942080,"data_stored":1016900,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34541,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34541,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34541,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":54996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":56737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T16:52:21.142 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-14T16:52:21.143 INFO:tasks.cephadm:Setting up client nodes... 2024-05-14T16:52:21.143 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-14T16:52:21.401 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:52:21.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:21 smithi002 ceph-mon[31538]: pgmap v314: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.9 KiB/s wr, 9 op/s; 14 B/s, 1 keys/s, 0 objects/s recovering 2024-05-14T16:52:21.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:21 smithi002 ceph-mon[31538]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:52:21.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:21 smithi151 ceph-mon[36066]: pgmap v314: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.9 KiB/s wr, 9 op/s; 14 B/s, 1 keys/s, 0 objects/s recovering 2024-05-14T16:52:21.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:21 smithi151 ceph-mon[36066]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:52:21.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:21 smithi123 ceph-mon[35971]: pgmap v314: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.9 KiB/s wr, 9 op/s; 14 B/s, 1 keys/s, 0 objects/s recovering 2024-05-14T16:52:21.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:21 smithi123 ceph-mon[35971]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:52:23.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.021+0000 7fb3fce4e700 1 -- 172.21.15.2:0/479950864 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3f81035c0 msgr2=0x7fb3f81039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:23.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.021+0000 7fb3fce4e700 1 --2- 172.21.15.2:0/479950864 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3f81035c0 0x7fb3f81039e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb3e80099c0 tx=0x7fb3e8009cd0 comp rx=0 tx=0).stop 2024-05-14T16:52:23.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.021+0000 7fb3fce4e700 1 -- 172.21.15.2:0/479950864 shutdown_connections 2024-05-14T16:52:23.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.021+0000 7fb3fce4e700 1 --2- 172.21.15.2:0/479950864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3f8105160 0x7fb3f810ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:23.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.021+0000 7fb3fce4e700 1 --2- 172.21.15.2:0/479950864 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3f81035c0 0x7fb3f81039e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:23.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.021+0000 7fb3fce4e700 1 --2- 172.21.15.2:0/479950864 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3f81047c0 0x7fb3f8104c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:23.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.021+0000 7fb3fce4e700 1 -- 172.21.15.2:0/479950864 >> 172.21.15.2:0/479950864 conn(0x7fb3f80feb60 msgr2=0x7fb3f8100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:23.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.021+0000 7fb3fce4e700 1 -- 172.21.15.2:0/479950864 shutdown_connections 2024-05-14T16:52:23.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.021+0000 7fb3fce4e700 1 -- 172.21.15.2:0/479950864 wait complete. 2024-05-14T16:52:23.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.022+0000 7fb3fce4e700 1 Processor -- start 2024-05-14T16:52:23.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.022+0000 7fb3fce4e700 1 -- start start 2024-05-14T16:52:23.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.022+0000 7fb3fce4e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3f81035c0 0x7fb3f8199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:23.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.022+0000 7fb3fce4e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3f81047c0 0x7fb3f8199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:23.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.022+0000 7fb3fce4e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3f8105160 0x7fb3f81a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:23.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.022+0000 7fb3fce4e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f810f030 con 0x7fb3f81047c0 2024-05-14T16:52:23.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.022+0000 7fb3fce4e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f810eeb0 con 0x7fb3f81035c0 2024-05-14T16:52:23.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.022+0000 7fb3fce4e700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f810f1b0 con 0x7fb3f8105160 2024-05-14T16:52:23.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.023+0000 7fb3f5d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3f81047c0 0x7fb3f8199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:23.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.023+0000 7fb3f5d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3f81047c0 0x7fb3f8199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51634/0 (socket says 172.21.15.2:51634) 2024-05-14T16:52:23.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.023+0000 7fb3f5d9b700 1 -- 172.21.15.2:0/3119172676 learned_addr learned my addr 172.21.15.2:0/3119172676 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:52:23.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.023+0000 7fb3f659c700 1 --2- 172.21.15.2:0/3119172676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3f81035c0 0x7fb3f8199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:23.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.023+0000 7fb3f6d9d700 1 --2- 172.21.15.2:0/3119172676 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3f8105160 0x7fb3f81a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:23.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.023+0000 7fb3f5d9b700 1 -- 172.21.15.2:0/3119172676 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3f8105160 msgr2=0x7fb3f81a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:23.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.023+0000 7fb3f5d9b700 1 --2- 172.21.15.2:0/3119172676 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3f8105160 0x7fb3f81a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:23.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.023+0000 7fb3f5d9b700 1 -- 172.21.15.2:0/3119172676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3f81035c0 msgr2=0x7fb3f8199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:23.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.023+0000 7fb3f5d9b700 1 --2- 172.21.15.2:0/3119172676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3f81035c0 0x7fb3f8199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:23.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.023+0000 7fb3f5d9b700 1 -- 172.21.15.2:0/3119172676 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f81a1270 con 0x7fb3f81047c0 2024-05-14T16:52:23.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.023+0000 7fb3f5d9b700 1 --2- 172.21.15.2:0/3119172676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3f81047c0 0x7fb3f8199b50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec00c950 tx=0x7fb3ec00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:23.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.023+0000 7fb3e77fe700 1 -- 172.21.15.2:0/3119172676 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec007ac0 con 0x7fb3f81047c0 2024-05-14T16:52:23.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.023+0000 7fb3e77fe700 1 -- 172.21.15.2:0/3119172676 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3ec004ce0 con 0x7fb3f81047c0 2024-05-14T16:52:23.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.023+0000 7fb3fce4e700 1 -- 172.21.15.2:0/3119172676 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3f81a1560 con 0x7fb3f81047c0 2024-05-14T16:52:23.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.023+0000 7fb3e77fe700 1 -- 172.21.15.2:0/3119172676 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec005580 con 0x7fb3f81047c0 2024-05-14T16:52:23.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.023+0000 7fb3fce4e700 1 -- 172.21.15.2:0/3119172676 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3f81a1aa0 con 0x7fb3f81047c0 2024-05-14T16:52:23.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.025+0000 7fb3fce4e700 1 -- 172.21.15.2:0/3119172676 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3f8051a80 con 0x7fb3f81047c0 2024-05-14T16:52:23.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.026+0000 7fb3e77fe700 1 -- 172.21.15.2:0/3119172676 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fb3ec007c60 con 0x7fb3f81047c0 2024-05-14T16:52:23.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.026+0000 7fb3e77fe700 1 --2- 172.21.15.2:0/3119172676 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fb3e00b29b0 0x7fb3e00b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:23.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.027+0000 7fb3f659c700 1 --2- 172.21.15.2:0/3119172676 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fb3e00b29b0 0x7fb3e00b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:23.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.027+0000 7fb3e77fe700 1 -- 172.21.15.2:0/3119172676 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7fb3ec0e1ef0 con 0x7fb3f81047c0 2024-05-14T16:52:23.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.027+0000 7fb3f659c700 1 --2- 172.21.15.2:0/3119172676 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fb3e00b29b0 0x7fb3e00b4e70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb3e80038c0 tx=0x7fb3e800b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:23.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.031+0000 7fb3e77fe700 1 -- 172.21.15.2:0/3119172676 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb3ec014030 con 0x7fb3f81047c0 2024-05-14T16:52:23.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.334+0000 7fb3fce4e700 1 -- 172.21.15.2:0/3119172676 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fb3f802e590 con 0x7fb3f81047c0 2024-05-14T16:52:23.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.336+0000 7fb3e77fe700 1 -- 172.21.15.2:0/3119172676 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fb3ec0a9ac0 con 0x7fb3f81047c0 2024-05-14T16:52:23.336 INFO:teuthology.orchestra.run.smithi002.stdout:[client.0] 2024-05-14T16:52:23.336 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQDHlkNmP2MJFBAAXrjZh4kQMeN92oiDT+kHWw== 2024-05-14T16:52:23.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.339+0000 7fb3fce4e700 1 -- 172.21.15.2:0/3119172676 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fb3e00b29b0 msgr2=0x7fb3e00b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:23.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.339+0000 7fb3fce4e700 1 --2- 172.21.15.2:0/3119172676 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fb3e00b29b0 0x7fb3e00b4e70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb3e80038c0 tx=0x7fb3e800b540 comp rx=0 tx=0).stop 2024-05-14T16:52:23.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.339+0000 7fb3fce4e700 1 -- 172.21.15.2:0/3119172676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3f81047c0 msgr2=0x7fb3f8199b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:23.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.339+0000 7fb3fce4e700 1 --2- 172.21.15.2:0/3119172676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3f81047c0 0x7fb3f8199b50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec00c950 tx=0x7fb3ec00cc60 comp rx=0 tx=0).stop 2024-05-14T16:52:23.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.339+0000 7fb3fce4e700 1 -- 172.21.15.2:0/3119172676 shutdown_connections 2024-05-14T16:52:23.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.339+0000 7fb3fce4e700 1 --2- 172.21.15.2:0/3119172676 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3f8105160 0x7fb3f81a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:23.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.339+0000 7fb3fce4e700 1 --2- 172.21.15.2:0/3119172676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3f81035c0 0x7fb3f8199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:23.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.339+0000 7fb3fce4e700 1 --2- 172.21.15.2:0/3119172676 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fb3e00b29b0 0x7fb3e00b4e70 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:23.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.339+0000 7fb3fce4e700 1 --2- 172.21.15.2:0/3119172676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3f81047c0 0x7fb3f8199b50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:23.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.339+0000 7fb3fce4e700 1 -- 172.21.15.2:0/3119172676 >> 172.21.15.2:0/3119172676 conn(0x7fb3f80feb60 msgr2=0x7fb3f8107ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:23.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.339+0000 7fb3fce4e700 1 -- 172.21.15.2:0/3119172676 shutdown_connections 2024-05-14T16:52:23.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:23.339+0000 7fb3fce4e700 1 -- 172.21.15.2:0/3119172676 wait complete. 2024-05-14T16:52:23.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:23 smithi002 ceph-mon[31538]: pgmap v315: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.5 KiB/s wr, 7 op/s; 11 B/s, 0 keys/s, 0 objects/s recovering 2024-05-14T16:52:23.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:23 smithi151 ceph-mon[36066]: pgmap v315: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.5 KiB/s wr, 7 op/s; 11 B/s, 0 keys/s, 0 objects/s recovering 2024-05-14T16:52:23.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:23 smithi123 ceph-mon[35971]: pgmap v315: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.5 KiB/s wr, 7 op/s; 11 B/s, 0 keys/s, 0 objects/s recovering 2024-05-14T16:52:24.041 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T16:52:24.041 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-14T16:52:24.041 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-14T16:52:24.082 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-14T16:52:24.082 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-14T16:52:24.082 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph mgr dump --format=json 2024-05-14T16:52:24.372 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:52:24.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:24 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/3119172676' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-14T16:52:24.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:24 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/3119172676' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-14T16:52:24.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:24 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/3119172676' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-14T16:52:24.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:24 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/3119172676' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-14T16:52:24.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:24 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/3119172676' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-14T16:52:24.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:24 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/3119172676' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-14T16:52:25.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:25 smithi151 ceph-mon[36066]: pgmap v316: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 7 op/s; 10 B/s, 0 keys/s, 0 objects/s recovering 2024-05-14T16:52:25.636 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:25 smithi002 ceph-mon[31538]: pgmap v316: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 7 op/s; 10 B/s, 0 keys/s, 0 objects/s recovering 2024-05-14T16:52:25.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:25 smithi123 ceph-mon[35971]: pgmap v316: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 7 op/s; 10 B/s, 0 keys/s, 0 objects/s recovering 2024-05-14T16:52:26.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.005+0000 7fa59d9bd700 1 -- 172.21.15.2:0/1741234308 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa598109630 msgr2=0x7fa598109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:26.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.005+0000 7fa59d9bd700 1 --2- 172.21.15.2:0/1741234308 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa598109630 0x7fa598109a10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa58800b0d0 tx=0x7fa58800b3e0 comp rx=0 tx=0).stop 2024-05-14T16:52:26.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.005+0000 7fa59d9bd700 1 -- 172.21.15.2:0/1741234308 shutdown_connections 2024-05-14T16:52:26.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.005+0000 7fa59d9bd700 1 --2- 172.21.15.2:0/1741234308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa59806c6d0 0x7fa598110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:26.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.005+0000 7fa59d9bd700 1 --2- 172.21.15.2:0/1741234308 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa598109630 0x7fa598109a10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:26.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.005+0000 7fa59d9bd700 1 --2- 172.21.15.2:0/1741234308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa59806bd10 0x7fa59806c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:26.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.005+0000 7fa59d9bd700 1 -- 172.21.15.2:0/1741234308 >> 172.21.15.2:0/1741234308 conn(0x7fa5980794e0 msgr2=0x7fa5980798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:26.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.005+0000 7fa59d9bd700 1 -- 172.21.15.2:0/1741234308 shutdown_connections 2024-05-14T16:52:26.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.005+0000 7fa59d9bd700 1 -- 172.21.15.2:0/1741234308 wait complete. 2024-05-14T16:52:26.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.006+0000 7fa59d9bd700 1 Processor -- start 2024-05-14T16:52:26.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.006+0000 7fa59d9bd700 1 -- start start 2024-05-14T16:52:26.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.006+0000 7fa59d9bd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa59806bd10 0x7fa59819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:26.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.006+0000 7fa59d9bd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa59806c6d0 0x7fa59819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:26.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.006+0000 7fa59d9bd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa598109630 0x7fa598199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:26.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.006+0000 7fa59d9bd700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa598113740 con 0x7fa59806bd10 2024-05-14T16:52:26.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.006+0000 7fa59d9bd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa5981135c0 con 0x7fa59806c6d0 2024-05-14T16:52:26.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.006+0000 7fa59d9bd700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa5981138c0 con 0x7fa598109630 2024-05-14T16:52:26.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.007+0000 7fa596ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa59806bd10 0x7fa59819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:26.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.007+0000 7fa596ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa59806bd10 0x7fa59819f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58678/0 (socket says 172.21.15.2:58678) 2024-05-14T16:52:26.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.007+0000 7fa596ffd700 1 -- 172.21.15.2:0/2103043398 learned_addr learned my addr 172.21.15.2:0/2103043398 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:52:26.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.007+0000 7fa5977fe700 1 --2- 172.21.15.2:0/2103043398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa598109630 0x7fa598199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:26.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.007+0000 7fa5967fc700 1 --2- 172.21.15.2:0/2103043398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa59806c6d0 0x7fa59819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:26.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.007+0000 7fa596ffd700 1 -- 172.21.15.2:0/2103043398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa598109630 msgr2=0x7fa598199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:26.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.007+0000 7fa596ffd700 1 --2- 172.21.15.2:0/2103043398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa598109630 0x7fa598199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:26.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.007+0000 7fa596ffd700 1 -- 172.21.15.2:0/2103043398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa59806c6d0 msgr2=0x7fa59819fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:26.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.007+0000 7fa596ffd700 1 --2- 172.21.15.2:0/2103043398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa59806c6d0 0x7fa59819fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:26.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.007+0000 7fa596ffd700 1 -- 172.21.15.2:0/2103043398 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa59819a0e0 con 0x7fa59806bd10 2024-05-14T16:52:26.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.007+0000 7fa596ffd700 1 --2- 172.21.15.2:0/2103043398 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa59806bd10 0x7fa59819f7c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa58800d010 tx=0x7fa588007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:26.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.007+0000 7fa59c9bb700 1 -- 172.21.15.2:0/2103043398 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58801f070 con 0x7fa59806bd10 2024-05-14T16:52:26.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.007+0000 7fa59c9bb700 1 -- 172.21.15.2:0/2103043398 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa588003c00 con 0x7fa59806bd10 2024-05-14T16:52:26.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.007+0000 7fa59d9bd700 1 -- 172.21.15.2:0/2103043398 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa59819a370 con 0x7fa59806bd10 2024-05-14T16:52:26.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.007+0000 7fa59c9bb700 1 -- 172.21.15.2:0/2103043398 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58801b6a0 con 0x7fa59806bd10 2024-05-14T16:52:26.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.007+0000 7fa59d9bd700 1 -- 172.21.15.2:0/2103043398 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5981a6690 con 0x7fa59806bd10 2024-05-14T16:52:26.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.008+0000 7fa59d9bd700 1 -- 172.21.15.2:0/2103043398 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa59810bac0 con 0x7fa59806bd10 2024-05-14T16:52:26.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.010+0000 7fa59c9bb700 1 -- 172.21.15.2:0/2103043398 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fa588022070 con 0x7fa59806bd10 2024-05-14T16:52:26.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.010+0000 7fa59c9bb700 1 --2- 172.21.15.2:0/2103043398 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fa5800b2910 0x7fa5800b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:26.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.011+0000 7fa5967fc700 1 --2- 172.21.15.2:0/2103043398 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fa5800b2910 0x7fa5800b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:26.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.011+0000 7fa59c9bb700 1 -- 172.21.15.2:0/2103043398 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7fa58800f020 con 0x7fa59806bd10 2024-05-14T16:52:26.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.011+0000 7fa5967fc700 1 --2- 172.21.15.2:0/2103043398 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fa5800b2910 0x7fa5800b4dd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa598079a70 tx=0x7fa58c005cf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:26.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.014+0000 7fa59c9bb700 1 -- 172.21.15.2:0/2103043398 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa588029020 con 0x7fa59806bd10 2024-05-14T16:52:26.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.304+0000 7fa59d9bd700 1 -- 172.21.15.2:0/2103043398 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fa59819aeb0 con 0x7fa59806bd10 2024-05-14T16:52:26.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.309+0000 7fa59c9bb700 1 -- 172.21.15.2:0/2103043398 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288637 (secure 0 0 0) 0x7fa5880ab5b0 con 0x7fa59806bd10 2024-05-14T16:52:26.313 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:52:26.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.312+0000 7fa59d9bd700 1 -- 172.21.15.2:0/2103043398 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fa5800b2910 msgr2=0x7fa5800b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:26.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.312+0000 7fa59d9bd700 1 --2- 172.21.15.2:0/2103043398 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fa5800b2910 0x7fa5800b4dd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa598079a70 tx=0x7fa58c005cf0 comp rx=0 tx=0).stop 2024-05-14T16:52:26.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.312+0000 7fa59d9bd700 1 -- 172.21.15.2:0/2103043398 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa59806bd10 msgr2=0x7fa59819f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:26.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.312+0000 7fa59d9bd700 1 --2- 172.21.15.2:0/2103043398 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa59806bd10 0x7fa59819f7c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa58800d010 tx=0x7fa588007a20 comp rx=0 tx=0).stop 2024-05-14T16:52:26.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.312+0000 7fa59d9bd700 1 -- 172.21.15.2:0/2103043398 shutdown_connections 2024-05-14T16:52:26.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.312+0000 7fa59d9bd700 1 --2- 172.21.15.2:0/2103043398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa598109630 0x7fa598199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:26.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.312+0000 7fa59d9bd700 1 --2- 172.21.15.2:0/2103043398 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fa5800b2910 0x7fa5800b4dd0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:26.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.312+0000 7fa59d9bd700 1 --2- 172.21.15.2:0/2103043398 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa59806bd10 0x7fa59819f7c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:26.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.312+0000 7fa59d9bd700 1 --2- 172.21.15.2:0/2103043398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa59806c6d0 0x7fa59819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:26.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.312+0000 7fa59d9bd700 1 -- 172.21.15.2:0/2103043398 >> 172.21.15.2:0/2103043398 conn(0x7fa5980794e0 msgr2=0x7fa598100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:26.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.312+0000 7fa59d9bd700 1 -- 172.21.15.2:0/2103043398 shutdown_connections 2024-05-14T16:52:26.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:26.313+0000 7fa59d9bd700 1 -- 172.21.15.2:0/2103043398 wait complete. 2024-05-14T16:52:26.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:26 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/2103043398' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-14T16:52:26.627 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:26 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/2103043398' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-14T16:52:26.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:26 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/2103043398' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-14T16:52:27.006 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":1498936028},{"type":"v1","addr":"172.21.15.2:6801","nonce":1498936028}]},"active_addr":"172.21.15.2:6801/1498936028","active_change":"2024-05-14T16:44:11.998326+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.2:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.2:0","nonce":4090012382}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.2:0","nonce":2111644395}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.2:0","nonce":2907436778}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.2:0","nonce":156580230}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.2:0","nonce":4193886449}]}]} 2024-05-14T16:52:27.012 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-14T16:52:27.013 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-14T16:52:27.013 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T16:52:27.264 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:52:27.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:27 smithi151 ceph-mon[36066]: pgmap v317: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 6 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2024-05-14T16:52:27.607 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:27 smithi002 ceph-mon[31538]: pgmap v317: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 6 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2024-05-14T16:52:27.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:27 smithi123 ceph-mon[35971]: pgmap v317: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 6 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2024-05-14T16:52:28.538 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:28 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:52:28.544 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:28 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:52:28.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:28 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:52:28.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.769+0000 7f3539d99700 1 -- 172.21.15.2:0/3211066249 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35341077d0 msgr2=0x7f353410f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:28.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.769+0000 7f3539d99700 1 --2- 172.21.15.2:0/3211066249 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35341077d0 0x7f353410f370 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f352c00b0d0 tx=0x7f352c00b3e0 comp rx=0 tx=0).stop 2024-05-14T16:52:28.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.769+0000 7f3539d99700 1 -- 172.21.15.2:0/3211066249 shutdown_connections 2024-05-14T16:52:28.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.769+0000 7f3539d99700 1 --2- 172.21.15.2:0/3211066249 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35341077d0 0x7f353410f370 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:28.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.769+0000 7f3539d99700 1 --2- 172.21.15.2:0/3211066249 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3534102a80 0x7f3534102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:28.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.769+0000 7f3539d99700 1 --2- 172.21.15.2:0/3211066249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3534103430 0x7f35341070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:28.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.769+0000 7f3539d99700 1 -- 172.21.15.2:0/3211066249 >> 172.21.15.2:0/3211066249 conn(0x7f35340793f0 msgr2=0x7f3534079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:28.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.769+0000 7f3539d99700 1 -- 172.21.15.2:0/3211066249 shutdown_connections 2024-05-14T16:52:28.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.769+0000 7f3539d99700 1 -- 172.21.15.2:0/3211066249 wait complete. 2024-05-14T16:52:28.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.770+0000 7f3539d99700 1 Processor -- start 2024-05-14T16:52:28.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.770+0000 7f3539d99700 1 -- start start 2024-05-14T16:52:28.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.770+0000 7f3539d99700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3534102a80 0x7f353419e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:28.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.770+0000 7f3539d99700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3534103430 0x7f353419ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:28.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.770+0000 7f3539d99700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f35341077d0 0x7f35341988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:28.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.770+0000 7f3539d99700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3534112640 con 0x7f3534102a80 2024-05-14T16:52:28.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.770+0000 7f3539d99700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f35341124c0 con 0x7f3534103430 2024-05-14T16:52:28.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.770+0000 7f3539d99700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f35341127c0 con 0x7f35341077d0 2024-05-14T16:52:28.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.770+0000 7f35337fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3534102a80 0x7f353419e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:28.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.770+0000 7f35337fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3534102a80 0x7f353419e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58690/0 (socket says 172.21.15.2:58690) 2024-05-14T16:52:28.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.770+0000 7f35337fe700 1 -- 172.21.15.2:0/2906797896 learned_addr learned my addr 172.21.15.2:0/2906797896 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:52:28.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.770+0000 7f3533fff700 1 --2- 172.21.15.2:0/2906797896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f35341077d0 0x7f35341988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:28.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.770+0000 7f3532ffd700 1 --2- 172.21.15.2:0/2906797896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3534103430 0x7f353419ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:28.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.771+0000 7f3533fff700 1 -- 172.21.15.2:0/2906797896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3534103430 msgr2=0x7f353419ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:28.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.771+0000 7f3533fff700 1 --2- 172.21.15.2:0/2906797896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3534103430 0x7f353419ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:28.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.771+0000 7f3533fff700 1 -- 172.21.15.2:0/2906797896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3534102a80 msgr2=0x7f353419e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:28.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.771+0000 7f3533fff700 1 --2- 172.21.15.2:0/2906797896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3534102a80 0x7f353419e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:28.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.771+0000 7f3533fff700 1 -- 172.21.15.2:0/2906797896 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3534199040 con 0x7f35341077d0 2024-05-14T16:52:28.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.771+0000 7f35337fe700 1 --2- 172.21.15.2:0/2906797896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3534102a80 0x7f353419e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:52:28.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.771+0000 7f3532ffd700 1 --2- 172.21.15.2:0/2906797896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3534103430 0x7f353419ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:52:28.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.771+0000 7f3533fff700 1 --2- 172.21.15.2:0/2906797896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f35341077d0 0x7f35341988e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f352c009fd0 tx=0x7f352c007a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:28.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.771+0000 7f3530ff9700 1 -- 172.21.15.2:0/2906797896 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c021020 con 0x7f35341077d0 2024-05-14T16:52:28.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.771+0000 7f3530ff9700 1 -- 172.21.15.2:0/2906797896 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f352c007e40 con 0x7f35341077d0 2024-05-14T16:52:28.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.771+0000 7f3539d99700 1 -- 172.21.15.2:0/2906797896 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35341992d0 con 0x7f35341077d0 2024-05-14T16:52:28.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.771+0000 7f3530ff9700 1 -- 172.21.15.2:0/2906797896 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c005050 con 0x7f35341077d0 2024-05-14T16:52:28.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.771+0000 7f3539d99700 1 -- 172.21.15.2:0/2906797896 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35341a55f0 con 0x7f35341077d0 2024-05-14T16:52:28.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.772+0000 7f3539d99700 1 -- 172.21.15.2:0/2906797896 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3534103b80 con 0x7f35341077d0 2024-05-14T16:52:28.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.774+0000 7f3530ff9700 1 -- 172.21.15.2:0/2906797896 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f352c024020 con 0x7f35341077d0 2024-05-14T16:52:28.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.774+0000 7f3530ff9700 1 --2- 172.21.15.2:0/2906797896 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f351c0b2910 0x7f351c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:28.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.775+0000 7f35337fe700 1 --2- 172.21.15.2:0/2906797896 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f351c0b2910 0x7f351c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:28.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.775+0000 7f3530ff9700 1 -- 172.21.15.2:0/2906797896 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f352c00f020 con 0x7f35341077d0 2024-05-14T16:52:28.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.775+0000 7f35337fe700 1 --2- 172.21.15.2:0/2906797896 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f351c0b2910 0x7f351c0b4dd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3534199a20 tx=0x7f352400b500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:28.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:28.778+0000 7f3530ff9700 1 -- 172.21.15.2:0/2906797896 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f352c0ab440 con 0x7f35341077d0 2024-05-14T16:52:29.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:29.013+0000 7f3539d99700 1 -- 172.21.15.2:0/2906797896 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f353402e560 con 0x7f35341077d0 2024-05-14T16:52:29.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:29.015+0000 7f3530ff9700 1 -- 172.21.15.2:0/2906797896 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22575 (secure 0 0 0) 0x7f352c0074e0 con 0x7f35341077d0 2024-05-14T16:52:29.015 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:52:29.016 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":80,"fsid":"ed40717a-1210-11ef-bc98-c7b262605968","created":"2024-05-14T16:43:09.415298+0000","modified":"2024-05-14T16:52:13.205039+0000","last_up_change":"2024-05-14T16:50:29.787654+0000","last_in_change":"2024-05-14T16:50:09.287932+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T16:46:38.090903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T16:51:27.296955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T16:51:31.103019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"153f20ec-5bcf-4786-9ede-2e512a1e3d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6803","nonce":2654237820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6805","nonce":2654237820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6809","nonce":2654237820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6807","nonce":2654237820}]},"public_addr":"172.21.15.2:6803/2654237820","cluster_addr":"172.21.15.2:6805/2654237820","heartbeat_back_addr":"172.21.15.2:6809/2654237820","heartbeat_front_addr":"172.21.15.2:6807/2654237820","state":["exists","up"]},{"osd":1,"uuid":"0e95419f-9a9d-4f7a-b55d-ecb54971918a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6801","nonce":2668156992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6803","nonce":2668156992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6807","nonce":2668156992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6805","nonce":2668156992}]},"public_addr":"172.21.15.123:6801/2668156992","cluster_addr":"172.21.15.123:6803/2668156992","heartbeat_back_addr":"172.21.15.123:6807/2668156992","heartbeat_front_addr":"172.21.15.123:6805/2668156992","state":["exists","up"]},{"osd":2,"uuid":"b6ce7ac7-be6c-4549-a356-59cb81b5b492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6801","nonce":2160189636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6803","nonce":2160189636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6807","nonce":2160189636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6805","nonce":2160189636}]},"public_addr":"172.21.15.151:6801/2160189636","cluster_addr":"172.21.15.151:6803/2160189636","heartbeat_back_addr":"172.21.15.151:6807/2160189636","heartbeat_front_addr":"172.21.15.151:6805/2160189636","state":["exists","up"]},{"osd":3,"uuid":"2fd592bf-c212-411a-afbe-a7c498e5ac93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6811","nonce":3451424073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6813","nonce":3451424073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6817","nonce":3451424073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6815","nonce":3451424073}]},"public_addr":"172.21.15.2:6811/3451424073","cluster_addr":"172.21.15.2:6813/3451424073","heartbeat_back_addr":"172.21.15.2:6817/3451424073","heartbeat_front_addr":"172.21.15.2:6815/3451424073","state":["exists","up"]},{"osd":4,"uuid":"af1905e7-e036-47c6-be59-b085bc41fede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6809","nonce":2101935762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6811","nonce":2101935762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6815","nonce":2101935762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6813","nonce":2101935762}]},"public_addr":"172.21.15.123:6809/2101935762","cluster_addr":"172.21.15.123:6811/2101935762","heartbeat_back_addr":"172.21.15.123:6815/2101935762","heartbeat_front_addr":"172.21.15.123:6813/2101935762","state":["exists","up"]},{"osd":5,"uuid":"9c166916-af78-4cc2-bd14-d1220b6e7d9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6809","nonce":3695961886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6811","nonce":3695961886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6815","nonce":3695961886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6813","nonce":3695961886}]},"public_addr":"172.21.15.151:6809/3695961886","cluster_addr":"172.21.15.151:6811/3695961886","heartbeat_back_addr":"172.21.15.151:6815/3695961886","heartbeat_front_addr":"172.21.15.151:6813/3695961886","state":["exists","up"]},{"osd":6,"uuid":"60e9bf66-2338-41e8-9280-398d4e3b322f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6819","nonce":2711893600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6821","nonce":2711893600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6825","nonce":2711893600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6823","nonce":2711893600}]},"public_addr":"172.21.15.2:6819/2711893600","cluster_addr":"172.21.15.2:6821/2711893600","heartbeat_back_addr":"172.21.15.2:6825/2711893600","heartbeat_front_addr":"172.21.15.2:6823/2711893600","state":["exists","up"]},{"osd":7,"uuid":"c93e07a1-e941-493e-8468-33429df2dff0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6817","nonce":3634155507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6819","nonce":3634155507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6823","nonce":3634155507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6821","nonce":3634155507}]},"public_addr":"172.21.15.123:6817/3634155507","cluster_addr":"172.21.15.123:6819/3634155507","heartbeat_back_addr":"172.21.15.123:6823/3634155507","heartbeat_front_addr":"172.21.15.123:6821/3634155507","state":["exists","up"]},{"osd":8,"uuid":"1155fb58-d614-4f45-91e2-5da3a93e7d9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6817","nonce":4271107010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6819","nonce":4271107010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6823","nonce":4271107010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6821","nonce":4271107010}]},"public_addr":"172.21.15.151:6817/4271107010","cluster_addr":"172.21.15.151:6819/4271107010","heartbeat_back_addr":"172.21.15.151:6823/4271107010","heartbeat_front_addr":"172.21.15.151:6821/4271107010","state":["exists","up"]},{"osd":9,"uuid":"82b34e3b-0017-4ca4-8f86-0993ca78ce7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6827","nonce":3479431021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6829","nonce":3479431021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6833","nonce":3479431021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6831","nonce":3479431021}]},"public_addr":"172.21.15.2:6827/3479431021","cluster_addr":"172.21.15.2:6829/3479431021","heartbeat_back_addr":"172.21.15.2:6833/3479431021","heartbeat_front_addr":"172.21.15.2:6831/3479431021","state":["exists","up"]},{"osd":10,"uuid":"32e2ace9-eb5d-4258-a2ac-49b782be2b5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6825","nonce":4266374435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6827","nonce":4266374435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6831","nonce":4266374435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6829","nonce":4266374435}]},"public_addr":"172.21.15.123:6825/4266374435","cluster_addr":"172.21.15.123:6827/4266374435","heartbeat_back_addr":"172.21.15.123:6831/4266374435","heartbeat_front_addr":"172.21.15.123:6829/4266374435","state":["exists","up"]},{"osd":11,"uuid":"6e3c2ead-1af6-4bda-87cd-12770c81fa67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6825","nonce":4129194698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6827","nonce":4129194698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6831","nonce":4129194698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6829","nonce":4129194698}]},"public_addr":"172.21.15.151:6825/4129194698","cluster_addr":"172.21.15.151:6827/4129194698","heartbeat_back_addr":"172.21.15.151:6831/4129194698","heartbeat_front_addr":"172.21.15.151:6829/4129194698","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:45:46.540023+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:46:10.845186+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:46:34.283112+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:47:00.072467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:47:25.315644+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:47:49.748688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:48:16.964258+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:48:42.875515+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:49:08.596500+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:49:35.450943+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:50:01.293368+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:50:27.793713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/4267809161":"2024-05-15T16:44:11.998214+0000","172.21.15.2:6801/3582988344":"2024-05-15T16:44:11.998214+0000","172.21.15.2:6800/2574015037":"2024-05-15T16:43:43.407184+0000","172.21.15.2:0/2100732571":"2024-05-15T16:44:11.998214+0000","172.21.15.2:0/2954199324":"2024-05-15T16:43:43.407184+0000","172.21.15.2:6801/2574015037":"2024-05-15T16:43:43.407184+0000","172.21.15.2:0/3171550944":"2024-05-15T16:43:43.407184+0000","172.21.15.2:0/1679761136":"2024-05-15T16:44:11.998214+0000","172.21.15.2:0/1746766591":"2024-05-15T16:43:43.407184+0000","172.21.15.2:6800/3582988344":"2024-05-15T16:44:11.998214+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T16:52:29.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:29.018+0000 7f3539d99700 1 -- 172.21.15.2:0/2906797896 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f351c0b2910 msgr2=0x7f351c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:29.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:29.018+0000 7f3539d99700 1 --2- 172.21.15.2:0/2906797896 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f351c0b2910 0x7f351c0b4dd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3534199a20 tx=0x7f352400b500 comp rx=0 tx=0).stop 2024-05-14T16:52:29.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:29.018+0000 7f3539d99700 1 -- 172.21.15.2:0/2906797896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f35341077d0 msgr2=0x7f35341988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:29.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:29.018+0000 7f3539d99700 1 --2- 172.21.15.2:0/2906797896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f35341077d0 0x7f35341988e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f352c009fd0 tx=0x7f352c007a00 comp rx=0 tx=0).stop 2024-05-14T16:52:29.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:29.018+0000 7f3539d99700 1 -- 172.21.15.2:0/2906797896 shutdown_connections 2024-05-14T16:52:29.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:29.018+0000 7f3539d99700 1 --2- 172.21.15.2:0/2906797896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f35341077d0 0x7f35341988e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:29.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:29.018+0000 7f3539d99700 1 --2- 172.21.15.2:0/2906797896 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f351c0b2910 0x7f351c0b4dd0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:29.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:29.018+0000 7f3539d99700 1 --2- 172.21.15.2:0/2906797896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3534102a80 0x7f353419e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:29.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:29.018+0000 7f3539d99700 1 --2- 172.21.15.2:0/2906797896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3534103430 0x7f353419ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:29.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:29.018+0000 7f3539d99700 1 -- 172.21.15.2:0/2906797896 >> 172.21.15.2:0/2906797896 conn(0x7f35340793f0 msgr2=0x7f3534101fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:29.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:29.018+0000 7f3539d99700 1 -- 172.21.15.2:0/2906797896 shutdown_connections 2024-05-14T16:52:29.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:29.018+0000 7f3539d99700 1 -- 172.21.15.2:0/2906797896 wait complete. 2024-05-14T16:52:29.558 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:29 smithi002 ceph-mon[31538]: pgmap v318: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 597 B/s wr, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-14T16:52:29.558 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:29 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/2906797896' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:52:29.559 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:29 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:52:29.559 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:29 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:52:29.559 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:29 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:52:29.559 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:29 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:52:29.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:29 smithi151 ceph-mon[36066]: pgmap v318: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 597 B/s wr, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-14T16:52:29.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:29 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/2906797896' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:52:29.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:29 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:52:29.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:29 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:52:29.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:29 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:52:29.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:29 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:52:29.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:29 smithi123 ceph-mon[35971]: pgmap v318: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 597 B/s wr, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-14T16:52:29.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:29 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/2906797896' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:52:29.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:29 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:52:29.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:29 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:52:29.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:29 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:52:29.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:29 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:52:30.175 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-14T16:52:30.175 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T16:52:30.445 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:52:31.571 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:31 smithi002 ceph-mon[31538]: pgmap v319: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-14T16:52:31.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:31 smithi123 ceph-mon[35971]: pgmap v319: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-14T16:52:31.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:31 smithi151 ceph-mon[36066]: pgmap v319: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-14T16:52:32.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.210+0000 7f15d617e700 1 -- 172.21.15.2:0/992737090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15d0102e00 msgr2=0x7f15d010f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:32.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.210+0000 7f15d617e700 1 --2- 172.21.15.2:0/992737090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15d0102e00 0x7f15d010f2f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f15c800b0a0 tx=0x7f15c800b3b0 comp rx=0 tx=0).stop 2024-05-14T16:52:32.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.210+0000 7f15d617e700 1 -- 172.21.15.2:0/992737090 shutdown_connections 2024-05-14T16:52:32.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.210+0000 7f15d617e700 1 --2- 172.21.15.2:0/992737090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15d0102e00 0x7f15d010f2f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:32.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.210+0000 7f15d617e700 1 --2- 172.21.15.2:0/992737090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15d0108520 0x7f15d0108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:32.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.210+0000 7f15d617e700 1 --2- 172.21.15.2:0/992737090 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15d0102440 0x7f15d01028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:32.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.210+0000 7f15d617e700 1 -- 172.21.15.2:0/992737090 >> 172.21.15.2:0/992737090 conn(0x7f15d00793f0 msgr2=0x7f15d0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:32.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.210+0000 7f15d617e700 1 -- 172.21.15.2:0/992737090 shutdown_connections 2024-05-14T16:52:32.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.210+0000 7f15d617e700 1 -- 172.21.15.2:0/992737090 wait complete. 2024-05-14T16:52:32.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.210+0000 7f15d617e700 1 Processor -- start 2024-05-14T16:52:32.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.211+0000 7f15d617e700 1 -- start start 2024-05-14T16:52:32.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.211+0000 7f15d617e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15d0102440 0x7f15d019f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:32.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.211+0000 7f15d617e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15d0102e00 0x7f15d019fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:32.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.211+0000 7f15d617e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15d0108520 0x7f15d0199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:32.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.211+0000 7f15d617e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f15d01125f0 con 0x7f15d0102e00 2024-05-14T16:52:32.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.211+0000 7f15d617e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f15d0112470 con 0x7f15d0102440 2024-05-14T16:52:32.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.211+0000 7f15d617e700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f15d0112770 con 0x7f15d0108520 2024-05-14T16:52:32.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.211+0000 7f15ceffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15d0102e00 0x7f15d019fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:32.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.211+0000 7f15ceffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15d0102e00 0x7f15d019fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58720/0 (socket says 172.21.15.2:58720) 2024-05-14T16:52:32.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.211+0000 7f15ceffd700 1 -- 172.21.15.2:0/2548043768 learned_addr learned my addr 172.21.15.2:0/2548043768 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:52:32.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.211+0000 7f15cf7fe700 1 --2- 172.21.15.2:0/2548043768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15d0102440 0x7f15d019f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:32.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.211+0000 7f15cffff700 1 --2- 172.21.15.2:0/2548043768 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15d0108520 0x7f15d0199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:32.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.212+0000 7f15ceffd700 1 -- 172.21.15.2:0/2548043768 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15d0108520 msgr2=0x7f15d0199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:32.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.212+0000 7f15ceffd700 1 --2- 172.21.15.2:0/2548043768 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15d0108520 0x7f15d0199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:32.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.212+0000 7f15ceffd700 1 -- 172.21.15.2:0/2548043768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15d0102440 msgr2=0x7f15d019f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:32.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.212+0000 7f15ceffd700 1 --2- 172.21.15.2:0/2548043768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15d0102440 0x7f15d019f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:32.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.212+0000 7f15ceffd700 1 -- 172.21.15.2:0/2548043768 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d019a170 con 0x7f15d0102e00 2024-05-14T16:52:32.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.212+0000 7f15cf7fe700 1 --2- 172.21.15.2:0/2548043768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15d0102440 0x7f15d019f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:52:32.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.212+0000 7f15ceffd700 1 --2- 172.21.15.2:0/2548043768 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15d0102e00 0x7f15d019fd40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f15c400d980 tx=0x7f15c400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:32.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.212+0000 7f15ccff9700 1 -- 172.21.15.2:0/2548043768 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c4017070 con 0x7f15d0102e00 2024-05-14T16:52:32.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.212+0000 7f15d617e700 1 -- 172.21.15.2:0/2548043768 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15d019a460 con 0x7f15d0102e00 2024-05-14T16:52:32.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.212+0000 7f15d617e700 1 -- 172.21.15.2:0/2548043768 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15d01a66d0 con 0x7f15d0102e00 2024-05-14T16:52:32.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.212+0000 7f15ccff9700 1 -- 172.21.15.2:0/2548043768 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15c4005020 con 0x7f15d0102e00 2024-05-14T16:52:32.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.212+0000 7f15ccff9700 1 -- 172.21.15.2:0/2548043768 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c400fa10 con 0x7f15d0102e00 2024-05-14T16:52:32.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.213+0000 7f15d617e700 1 -- 172.21.15.2:0/2548043768 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15d006a230 con 0x7f15d0102e00 2024-05-14T16:52:32.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.216+0000 7f15ccff9700 1 -- 172.21.15.2:0/2548043768 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f15c4004a30 con 0x7f15d0102e00 2024-05-14T16:52:32.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.216+0000 7f15ccff9700 1 --2- 172.21.15.2:0/2548043768 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f15b80b2910 0x7f15b80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:32.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.217+0000 7f15cf7fe700 1 --2- 172.21.15.2:0/2548043768 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f15b80b2910 0x7f15b80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:32.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.217+0000 7f15ccff9700 1 -- 172.21.15.2:0/2548043768 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f15c40e2640 con 0x7f15d0102e00 2024-05-14T16:52:32.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.217+0000 7f15cf7fe700 1 --2- 172.21.15.2:0/2548043768 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f15b80b2910 0x7f15b80b4dd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f15d01078b0 tx=0x7f15c0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:32.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.219+0000 7f15ccff9700 1 -- 172.21.15.2:0/2548043768 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f15c4020020 con 0x7f15d0102e00 2024-05-14T16:52:32.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.449+0000 7f15d617e700 1 -- 172.21.15.2:0/2548043768 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f15d002e560 con 0x7f15d0102e00 2024-05-14T16:52:32.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.451+0000 7f15ccff9700 1 -- 172.21.15.2:0/2548043768 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22575 (secure 0 0 0) 0x7f15c401ae40 con 0x7f15d0102e00 2024-05-14T16:52:32.451 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:52:32.451 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":80,"fsid":"ed40717a-1210-11ef-bc98-c7b262605968","created":"2024-05-14T16:43:09.415298+0000","modified":"2024-05-14T16:52:13.205039+0000","last_up_change":"2024-05-14T16:50:29.787654+0000","last_in_change":"2024-05-14T16:50:09.287932+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T16:46:38.090903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T16:51:27.296955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T16:51:31.103019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"153f20ec-5bcf-4786-9ede-2e512a1e3d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6803","nonce":2654237820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6805","nonce":2654237820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6809","nonce":2654237820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6807","nonce":2654237820}]},"public_addr":"172.21.15.2:6803/2654237820","cluster_addr":"172.21.15.2:6805/2654237820","heartbeat_back_addr":"172.21.15.2:6809/2654237820","heartbeat_front_addr":"172.21.15.2:6807/2654237820","state":["exists","up"]},{"osd":1,"uuid":"0e95419f-9a9d-4f7a-b55d-ecb54971918a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6801","nonce":2668156992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6803","nonce":2668156992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6807","nonce":2668156992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6805","nonce":2668156992}]},"public_addr":"172.21.15.123:6801/2668156992","cluster_addr":"172.21.15.123:6803/2668156992","heartbeat_back_addr":"172.21.15.123:6807/2668156992","heartbeat_front_addr":"172.21.15.123:6805/2668156992","state":["exists","up"]},{"osd":2,"uuid":"b6ce7ac7-be6c-4549-a356-59cb81b5b492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6801","nonce":2160189636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6803","nonce":2160189636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6807","nonce":2160189636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6805","nonce":2160189636}]},"public_addr":"172.21.15.151:6801/2160189636","cluster_addr":"172.21.15.151:6803/2160189636","heartbeat_back_addr":"172.21.15.151:6807/2160189636","heartbeat_front_addr":"172.21.15.151:6805/2160189636","state":["exists","up"]},{"osd":3,"uuid":"2fd592bf-c212-411a-afbe-a7c498e5ac93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6811","nonce":3451424073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6813","nonce":3451424073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6817","nonce":3451424073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6815","nonce":3451424073}]},"public_addr":"172.21.15.2:6811/3451424073","cluster_addr":"172.21.15.2:6813/3451424073","heartbeat_back_addr":"172.21.15.2:6817/3451424073","heartbeat_front_addr":"172.21.15.2:6815/3451424073","state":["exists","up"]},{"osd":4,"uuid":"af1905e7-e036-47c6-be59-b085bc41fede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6809","nonce":2101935762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6811","nonce":2101935762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6815","nonce":2101935762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6813","nonce":2101935762}]},"public_addr":"172.21.15.123:6809/2101935762","cluster_addr":"172.21.15.123:6811/2101935762","heartbeat_back_addr":"172.21.15.123:6815/2101935762","heartbeat_front_addr":"172.21.15.123:6813/2101935762","state":["exists","up"]},{"osd":5,"uuid":"9c166916-af78-4cc2-bd14-d1220b6e7d9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6809","nonce":3695961886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6811","nonce":3695961886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6815","nonce":3695961886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6813","nonce":3695961886}]},"public_addr":"172.21.15.151:6809/3695961886","cluster_addr":"172.21.15.151:6811/3695961886","heartbeat_back_addr":"172.21.15.151:6815/3695961886","heartbeat_front_addr":"172.21.15.151:6813/3695961886","state":["exists","up"]},{"osd":6,"uuid":"60e9bf66-2338-41e8-9280-398d4e3b322f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6819","nonce":2711893600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6821","nonce":2711893600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6825","nonce":2711893600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6823","nonce":2711893600}]},"public_addr":"172.21.15.2:6819/2711893600","cluster_addr":"172.21.15.2:6821/2711893600","heartbeat_back_addr":"172.21.15.2:6825/2711893600","heartbeat_front_addr":"172.21.15.2:6823/2711893600","state":["exists","up"]},{"osd":7,"uuid":"c93e07a1-e941-493e-8468-33429df2dff0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6817","nonce":3634155507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6819","nonce":3634155507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6823","nonce":3634155507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6821","nonce":3634155507}]},"public_addr":"172.21.15.123:6817/3634155507","cluster_addr":"172.21.15.123:6819/3634155507","heartbeat_back_addr":"172.21.15.123:6823/3634155507","heartbeat_front_addr":"172.21.15.123:6821/3634155507","state":["exists","up"]},{"osd":8,"uuid":"1155fb58-d614-4f45-91e2-5da3a93e7d9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6817","nonce":4271107010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6819","nonce":4271107010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6823","nonce":4271107010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6821","nonce":4271107010}]},"public_addr":"172.21.15.151:6817/4271107010","cluster_addr":"172.21.15.151:6819/4271107010","heartbeat_back_addr":"172.21.15.151:6823/4271107010","heartbeat_front_addr":"172.21.15.151:6821/4271107010","state":["exists","up"]},{"osd":9,"uuid":"82b34e3b-0017-4ca4-8f86-0993ca78ce7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6827","nonce":3479431021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6829","nonce":3479431021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6833","nonce":3479431021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6831","nonce":3479431021}]},"public_addr":"172.21.15.2:6827/3479431021","cluster_addr":"172.21.15.2:6829/3479431021","heartbeat_back_addr":"172.21.15.2:6833/3479431021","heartbeat_front_addr":"172.21.15.2:6831/3479431021","state":["exists","up"]},{"osd":10,"uuid":"32e2ace9-eb5d-4258-a2ac-49b782be2b5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6825","nonce":4266374435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6827","nonce":4266374435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6831","nonce":4266374435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6829","nonce":4266374435}]},"public_addr":"172.21.15.123:6825/4266374435","cluster_addr":"172.21.15.123:6827/4266374435","heartbeat_back_addr":"172.21.15.123:6831/4266374435","heartbeat_front_addr":"172.21.15.123:6829/4266374435","state":["exists","up"]},{"osd":11,"uuid":"6e3c2ead-1af6-4bda-87cd-12770c81fa67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6825","nonce":4129194698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6827","nonce":4129194698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6831","nonce":4129194698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6829","nonce":4129194698}]},"public_addr":"172.21.15.151:6825/4129194698","cluster_addr":"172.21.15.151:6827/4129194698","heartbeat_back_addr":"172.21.15.151:6831/4129194698","heartbeat_front_addr":"172.21.15.151:6829/4129194698","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:45:46.540023+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:46:10.845186+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:46:34.283112+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:47:00.072467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:47:25.315644+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:47:49.748688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:48:16.964258+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:48:42.875515+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:49:08.596500+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:49:35.450943+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:50:01.293368+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:50:27.793713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/4267809161":"2024-05-15T16:44:11.998214+0000","172.21.15.2:6801/3582988344":"2024-05-15T16:44:11.998214+0000","172.21.15.2:6800/2574015037":"2024-05-15T16:43:43.407184+0000","172.21.15.2:0/2100732571":"2024-05-15T16:44:11.998214+0000","172.21.15.2:0/2954199324":"2024-05-15T16:43:43.407184+0000","172.21.15.2:6801/2574015037":"2024-05-15T16:43:43.407184+0000","172.21.15.2:0/3171550944":"2024-05-15T16:43:43.407184+0000","172.21.15.2:0/1679761136":"2024-05-15T16:44:11.998214+0000","172.21.15.2:0/1746766591":"2024-05-15T16:43:43.407184+0000","172.21.15.2:6800/3582988344":"2024-05-15T16:44:11.998214+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T16:52:32.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.454+0000 7f15d617e700 1 -- 172.21.15.2:0/2548043768 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f15b80b2910 msgr2=0x7f15b80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:32.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.454+0000 7f15d617e700 1 --2- 172.21.15.2:0/2548043768 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f15b80b2910 0x7f15b80b4dd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f15d01078b0 tx=0x7f15c0009260 comp rx=0 tx=0).stop 2024-05-14T16:52:32.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.454+0000 7f15d617e700 1 -- 172.21.15.2:0/2548043768 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15d0102e00 msgr2=0x7f15d019fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:32.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.454+0000 7f15d617e700 1 --2- 172.21.15.2:0/2548043768 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15d0102e00 0x7f15d019fd40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f15c400d980 tx=0x7f15c400dc90 comp rx=0 tx=0).stop 2024-05-14T16:52:32.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.454+0000 7f15d617e700 1 -- 172.21.15.2:0/2548043768 shutdown_connections 2024-05-14T16:52:32.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.454+0000 7f15d617e700 1 --2- 172.21.15.2:0/2548043768 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15d0108520 0x7f15d0199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:32.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.454+0000 7f15d617e700 1 --2- 172.21.15.2:0/2548043768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15d0102440 0x7f15d019f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:32.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.454+0000 7f15d617e700 1 --2- 172.21.15.2:0/2548043768 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f15b80b2910 0x7f15b80b4dd0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:32.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.454+0000 7f15d617e700 1 --2- 172.21.15.2:0/2548043768 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15d0102e00 0x7f15d019fd40 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:32.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.454+0000 7f15d617e700 1 -- 172.21.15.2:0/2548043768 >> 172.21.15.2:0/2548043768 conn(0x7f15d00793f0 msgr2=0x7f15d006b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:32.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.454+0000 7f15d617e700 1 -- 172.21.15.2:0/2548043768 shutdown_connections 2024-05-14T16:52:32.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:32.454+0000 7f15d617e700 1 -- 172.21.15.2:0/2548043768 wait complete. 2024-05-14T16:52:33.079 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:32 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:52:33.079 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:32 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:52:33.079 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:32 smithi002 ceph-mon[31538]: pgmap v320: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-14T16:52:33.079 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:32 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/2548043768' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:52:33.079 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:32 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:52:33.079 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:32 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:52:33.079 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:32 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:52:33.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:32 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:52:33.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:32 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:52:33.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:32 smithi151 ceph-mon[36066]: pgmap v320: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-14T16:52:33.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:32 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/2548043768' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:52:33.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:32 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:52:33.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:32 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:52:33.105 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:32 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:52:33.107 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-14T16:52:33.108 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-14T16:52:33.108 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-14T16:52:33.109 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-14T16:52:33.109 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-14T16:52:33.109 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-14T16:52:33.110 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-14T16:52:33.110 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-14T16:52:33.111 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-14T16:52:33.111 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-14T16:52:33.111 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-14T16:52:33.112 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-14T16:52:33.159 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:32 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:52:33.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:32 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:52:33.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:32 smithi123 ceph-mon[35971]: pgmap v320: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-14T16:52:33.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:32 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/2548043768' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:52:33.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:32 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:52:33.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:32 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:52:33.160 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:32 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:52:33.763 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:52:33.769 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:52:33.775 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:52:33.870 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:52:33.870 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:52:33.870 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:52:33.874 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:52:33.879 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:52:33.886 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:52:35.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:35 smithi123 ceph-mon[35971]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-05-14T16:52:35.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:35 smithi002 ceph-mon[31538]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-05-14T16:52:35.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:35 smithi151 ceph-mon[36066]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-05-14T16:52:37.409 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:37 smithi123 ceph-mon[35971]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:52:37.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:37 smithi002 ceph-mon[31538]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:52:37.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:37 smithi151 ceph-mon[36066]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:52:39.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:39 smithi123 ceph-mon[35971]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T16:52:39.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:39 smithi002 ceph-mon[31538]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T16:52:39.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:39 smithi151 ceph-mon[36066]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T16:52:41.425 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:41 smithi002 ceph-mon[31538]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T16:52:41.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:41 smithi151 ceph-mon[36066]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T16:52:41.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:41 smithi123 ceph-mon[35971]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T16:52:41.718 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:52:41.719 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:52:41.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.970+0000 7f081bf82700 1 -- 172.21.15.2:0/1357721240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0814108520 msgr2=0x7f0814108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:41.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.970+0000 7f081bf82700 1 --2- 172.21.15.2:0/1357721240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0814108520 0x7f0814108900 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0810009990 tx=0x7f0810009ca0 comp rx=0 tx=0).stop 2024-05-14T16:52:41.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.970+0000 7f081bf82700 1 -- 172.21.15.2:0/1357721240 shutdown_connections 2024-05-14T16:52:41.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.970+0000 7f081bf82700 1 --2- 172.21.15.2:0/1357721240 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0814102e00 0x7f081410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:41.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.970+0000 7f081bf82700 1 --2- 172.21.15.2:0/1357721240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0814108520 0x7f0814108900 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:41.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.970+0000 7f081bf82700 1 --2- 172.21.15.2:0/1357721240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0814102440 0x7f08141028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:41.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.970+0000 7f081bf82700 1 -- 172.21.15.2:0/1357721240 >> 172.21.15.2:0/1357721240 conn(0x7f08140793f0 msgr2=0x7f0814079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:41.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.970+0000 7f081bf82700 1 -- 172.21.15.2:0/1357721240 shutdown_connections 2024-05-14T16:52:41.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.971+0000 7f081bf82700 1 -- 172.21.15.2:0/1357721240 wait complete. 2024-05-14T16:52:41.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.971+0000 7f081bf82700 1 Processor -- start 2024-05-14T16:52:41.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.971+0000 7f081bf82700 1 -- start start 2024-05-14T16:52:41.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.972+0000 7f081bf82700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0814102440 0x7f081419f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:41.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.972+0000 7f081bf82700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0814102e00 0x7f081419fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:41.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.972+0000 7f081bf82700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0814108520 0x7f0814199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:41.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.972+0000 7f081bf82700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f08141125f0 con 0x7f0814102e00 2024-05-14T16:52:41.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.972+0000 7f081bf82700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0814112470 con 0x7f0814108520 2024-05-14T16:52:41.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.972+0000 7f081bf82700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0814112770 con 0x7f0814102440 2024-05-14T16:52:41.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.972+0000 7f081951d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0814102e00 0x7f081419fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:41.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.972+0000 7f081951d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0814102e00 0x7f081419fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39272/0 (socket says 172.21.15.2:39272) 2024-05-14T16:52:41.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.972+0000 7f081951d700 1 -- 172.21.15.2:0/907882051 learned_addr learned my addr 172.21.15.2:0/907882051 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:52:41.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.972+0000 7f0819d1e700 1 --2- 172.21.15.2:0/907882051 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0814102440 0x7f081419f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:41.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.972+0000 7f081a51f700 1 --2- 172.21.15.2:0/907882051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0814108520 0x7f0814199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:41.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.972+0000 7f081951d700 1 -- 172.21.15.2:0/907882051 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0814102440 msgr2=0x7f081419f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:41.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.972+0000 7f081951d700 1 --2- 172.21.15.2:0/907882051 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0814102440 0x7f081419f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:41.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.972+0000 7f081951d700 1 -- 172.21.15.2:0/907882051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0814108520 msgr2=0x7f0814199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:41.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.972+0000 7f081951d700 1 --2- 172.21.15.2:0/907882051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0814108520 0x7f0814199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:41.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.972+0000 7f081951d700 1 -- 172.21.15.2:0/907882051 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f081419a160 con 0x7f0814102e00 2024-05-14T16:52:41.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.972+0000 7f081951d700 1 --2- 172.21.15.2:0/907882051 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0814102e00 0x7f081419fd80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f080400c920 tx=0x7f080400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:41.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.972+0000 7f080affd700 1 -- 172.21.15.2:0/907882051 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0804007b00 con 0x7f0814102e00 2024-05-14T16:52:41.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.973+0000 7f081bf82700 1 -- 172.21.15.2:0/907882051 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f081419a450 con 0x7f0814102e00 2024-05-14T16:52:41.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.973+0000 7f080affd700 1 -- 172.21.15.2:0/907882051 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0804004ce0 con 0x7f0814102e00 2024-05-14T16:52:41.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.973+0000 7f080affd700 1 -- 172.21.15.2:0/907882051 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0804005550 con 0x7f0814102e00 2024-05-14T16:52:41.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.973+0000 7f081bf82700 1 -- 172.21.15.2:0/907882051 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08141a6710 con 0x7f0814102e00 2024-05-14T16:52:41.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.973+0000 7f081bf82700 1 -- 172.21.15.2:0/907882051 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f08141074e0 con 0x7f0814102e00 2024-05-14T16:52:41.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.975+0000 7f080affd700 1 -- 172.21.15.2:0/907882051 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f0804007ca0 con 0x7f0814102e00 2024-05-14T16:52:41.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.976+0000 7f080affd700 1 --2- 172.21.15.2:0/907882051 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f08000b2910 0x7f08000b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:41.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.976+0000 7f0819d1e700 1 --2- 172.21.15.2:0/907882051 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f08000b2910 0x7f08000b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:41.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.976+0000 7f080affd700 1 -- 172.21.15.2:0/907882051 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f08040e2570 con 0x7f0814102e00 2024-05-14T16:52:41.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.976+0000 7f080affd700 1 --2- 172.21.15.2:0/907882051 >> [v2:172.21.15.123:6808/2101935762,v1:172.21.15.123:6809/2101935762] conn(0x7f08000bc450 0x7f08000be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:41.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.976+0000 7f080affd700 1 -- 172.21.15.2:0/907882051 --> [v2:172.21.15.123:6808/2101935762,v1:172.21.15.123:6809/2101935762] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0804015e50 con 0x7f08000bc450 2024-05-14T16:52:41.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.976+0000 7f080affd700 1 -- 172.21.15.2:0/907882051 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f08040e2920 con 0x7f0814102e00 2024-05-14T16:52:41.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.976+0000 7f0819d1e700 1 --2- 172.21.15.2:0/907882051 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f08000b2910 0x7f08000b4dd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f081000b5c0 tx=0x7f0810005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:41.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.977+0000 7f081a51f700 1 --2- 172.21.15.2:0/907882051 >> [v2:172.21.15.123:6808/2101935762,v1:172.21.15.123:6809/2101935762] conn(0x7f08000bc450 0x7f08000be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:41.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.977+0000 7f081a51f700 1 --2- 172.21.15.2:0/907882051 >> [v2:172.21.15.123:6808/2101935762,v1:172.21.15.123:6809/2101935762] conn(0x7f08000bc450 0x7f08000be8f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:41.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:41.978+0000 7f080affd700 1 -- 172.21.15.2:0/907882051 <== osd.4 v2:172.21.15.123:6808/2101935762 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0804015e50 con 0x7f08000bc450 2024-05-14T16:52:42.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:42.005+0000 7f081bf82700 1 -- 172.21.15.2:0/907882051 --> [v2:172.21.15.123:6808/2101935762,v1:172.21.15.123:6809/2101935762] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0814103b60 con 0x7f08000bc450 2024-05-14T16:52:42.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:42.005+0000 7f080affd700 1 -- 172.21.15.2:0/907882051 <== osd.4 v2:172.21.15.123:6808/2101935762 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0814103b60 con 0x7f08000bc450 2024-05-14T16:52:42.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:42.005+0000 7f081bf82700 1 -- 172.21.15.2:0/907882051 >> [v2:172.21.15.123:6808/2101935762,v1:172.21.15.123:6809/2101935762] conn(0x7f08000bc450 msgr2=0x7f08000be8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:42.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:42.005+0000 7f081bf82700 1 --2- 172.21.15.2:0/907882051 >> [v2:172.21.15.123:6808/2101935762,v1:172.21.15.123:6809/2101935762] conn(0x7f08000bc450 0x7f08000be8f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:42.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:42.005+0000 7f081bf82700 1 -- 172.21.15.2:0/907882051 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f08000b2910 msgr2=0x7f08000b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:42.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:42.005+0000 7f081bf82700 1 --2- 172.21.15.2:0/907882051 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f08000b2910 0x7f08000b4dd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f081000b5c0 tx=0x7f0810005960 comp rx=0 tx=0).stop 2024-05-14T16:52:42.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:42.005+0000 7f081bf82700 1 -- 172.21.15.2:0/907882051 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0814102e00 msgr2=0x7f081419fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:42.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:42.005+0000 7f081bf82700 1 --2- 172.21.15.2:0/907882051 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0814102e00 0x7f081419fd80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f080400c920 tx=0x7f080400cc30 comp rx=0 tx=0).stop 2024-05-14T16:52:42.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:42.005+0000 7f081a51f700 1 -- 172.21.15.2:0/907882051 reap_dead start 2024-05-14T16:52:42.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:42.005+0000 7f081bf82700 1 -- 172.21.15.2:0/907882051 shutdown_connections 2024-05-14T16:52:42.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:42.005+0000 7f081bf82700 1 -- 172.21.15.2:0/907882051 >> 172.21.15.2:0/907882051 conn(0x7f08140793f0 msgr2=0x7f081406b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:42.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:42.006+0000 7f081bf82700 1 -- 172.21.15.2:0/907882051 shutdown_connections 2024-05-14T16:52:42.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:42.006+0000 7f081bf82700 1 -- 172.21.15.2:0/907882051 wait complete. 2024-05-14T16:52:43.377 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:43 smithi002 ceph-mon[31538]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T16:52:43.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:43 smithi151 ceph-mon[36066]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T16:52:43.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:43 smithi123 ceph-mon[35971]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T16:52:44.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.167+0000 7fb4790b7700 1 -- 172.21.15.2:0/2665266194 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb474102e00 msgr2=0x7fb47410f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:44.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.167+0000 7fb4790b7700 1 --2- 172.21.15.2:0/2665266194 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb474102e00 0x7fb47410f2f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb46c00b0a0 tx=0x7fb46c00b3b0 comp rx=0 tx=0).stop 2024-05-14T16:52:44.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.167+0000 7fb4790b7700 1 -- 172.21.15.2:0/2665266194 shutdown_connections 2024-05-14T16:52:44.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.167+0000 7fb4790b7700 1 --2- 172.21.15.2:0/2665266194 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb474102e00 0x7fb47410f2f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:44.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.167+0000 7fb4790b7700 1 --2- 172.21.15.2:0/2665266194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb474108520 0x7fb474108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:44.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.167+0000 7fb4790b7700 1 --2- 172.21.15.2:0/2665266194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb474102440 0x7fb4741028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:44.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.167+0000 7fb4790b7700 1 -- 172.21.15.2:0/2665266194 >> 172.21.15.2:0/2665266194 conn(0x7fb4740793f0 msgr2=0x7fb474079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:44.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.167+0000 7fb4790b7700 1 -- 172.21.15.2:0/2665266194 shutdown_connections 2024-05-14T16:52:44.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.167+0000 7fb4790b7700 1 -- 172.21.15.2:0/2665266194 wait complete. 2024-05-14T16:52:44.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.168+0000 7fb4790b7700 1 Processor -- start 2024-05-14T16:52:44.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.168+0000 7fb4790b7700 1 -- start start 2024-05-14T16:52:44.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.168+0000 7fb4790b7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb474102440 0x7fb47419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:44.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.169+0000 7fb4790b7700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb474102e00 0x7fb47419afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:44.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.169+0000 7fb4790b7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb474108520 0x7fb4741a5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:44.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.169+0000 7fb4790b7700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb474112620 con 0x7fb474102440 2024-05-14T16:52:44.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.169+0000 7fb4790b7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb4741124a0 con 0x7fb474108520 2024-05-14T16:52:44.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.169+0000 7fb4790b7700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb4741127a0 con 0x7fb474102e00 2024-05-14T16:52:44.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.169+0000 7fb472d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb474102440 0x7fb47419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:44.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.169+0000 7fb472d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb474102440 0x7fb47419aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56206/0 (socket says 172.21.15.2:56206) 2024-05-14T16:52:44.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.169+0000 7fb472d9d700 1 -- 172.21.15.2:0/3595801232 learned_addr learned my addr 172.21.15.2:0/3595801232 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:52:44.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.169+0000 7fb46a59c700 1 --2- 172.21.15.2:0/3595801232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb474102e00 0x7fb47419afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:44.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.169+0000 7fb47359e700 1 --2- 172.21.15.2:0/3595801232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb474108520 0x7fb4741a5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:44.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.169+0000 7fb46a59c700 1 -- 172.21.15.2:0/3595801232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb474108520 msgr2=0x7fb4741a5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:44.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.169+0000 7fb46a59c700 1 --2- 172.21.15.2:0/3595801232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb474108520 0x7fb4741a5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:44.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.169+0000 7fb46a59c700 1 -- 172.21.15.2:0/3595801232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb474102440 msgr2=0x7fb47419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:44.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.169+0000 7fb46a59c700 1 --2- 172.21.15.2:0/3595801232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb474102440 0x7fb47419aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:44.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.169+0000 7fb46a59c700 1 -- 172.21.15.2:0/3595801232 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4741a6040 con 0x7fb474102e00 2024-05-14T16:52:44.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.169+0000 7fb47359e700 1 --2- 172.21.15.2:0/3595801232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb474108520 0x7fb4741a5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:52:44.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.169+0000 7fb46a59c700 1 --2- 172.21.15.2:0/3595801232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb474102e00 0x7fb47419afc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb45c00d750 tx=0x7fb45c00da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:44.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.170+0000 7fb470d99700 1 -- 172.21.15.2:0/3595801232 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c016070 con 0x7fb474102e00 2024-05-14T16:52:44.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.170+0000 7fb4790b7700 1 -- 172.21.15.2:0/3595801232 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4741a6210 con 0x7fb474102e00 2024-05-14T16:52:44.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.170+0000 7fb470d99700 1 -- 172.21.15.2:0/3595801232 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb45c004830 con 0x7fb474102e00 2024-05-14T16:52:44.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.170+0000 7fb470d99700 1 -- 172.21.15.2:0/3595801232 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c0055d0 con 0x7fb474102e00 2024-05-14T16:52:44.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.170+0000 7fb4790b7700 1 -- 172.21.15.2:0/3595801232 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4741a6750 con 0x7fb474102e00 2024-05-14T16:52:44.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.170+0000 7fb4790b7700 1 -- 172.21.15.2:0/3595801232 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb474063c50 con 0x7fb474102e00 2024-05-14T16:52:44.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.172+0000 7fb470d99700 1 -- 172.21.15.2:0/3595801232 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fb45c004220 con 0x7fb474102e00 2024-05-14T16:52:44.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.173+0000 7fb470d99700 1 --2- 172.21.15.2:0/3595801232 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fb4600b2910 0x7fb4600b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:44.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.173+0000 7fb472d9d700 1 --2- 172.21.15.2:0/3595801232 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fb4600b2910 0x7fb4600b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:44.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.173+0000 7fb470d99700 1 -- 172.21.15.2:0/3595801232 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7fb45c0e2760 con 0x7fb474102e00 2024-05-14T16:52:44.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.173+0000 7fb470d99700 1 --2- 172.21.15.2:0/3595801232 >> [v2:172.21.15.151:6808/3695961886,v1:172.21.15.151:6809/3695961886] conn(0x7fb4600bc450 0x7fb4600be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:44.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.173+0000 7fb470d99700 1 -- 172.21.15.2:0/3595801232 --> [v2:172.21.15.151:6808/3695961886,v1:172.21.15.151:6809/3695961886] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb4600befc0 con 0x7fb4600bc450 2024-05-14T16:52:44.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.173+0000 7fb470d99700 1 -- 172.21.15.2:0/3595801232 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fb45c0e2b10 con 0x7fb474102e00 2024-05-14T16:52:44.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.174+0000 7fb472d9d700 1 --2- 172.21.15.2:0/3595801232 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fb4600b2910 0x7fb4600b4dd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb464009ca0 tx=0x7fb464009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:44.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.174+0000 7fb47359e700 1 --2- 172.21.15.2:0/3595801232 >> [v2:172.21.15.151:6808/3695961886,v1:172.21.15.151:6809/3695961886] conn(0x7fb4600bc450 0x7fb4600be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:44.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.174+0000 7fb47359e700 1 --2- 172.21.15.2:0/3595801232 >> [v2:172.21.15.151:6808/3695961886,v1:172.21.15.151:6809/3695961886] conn(0x7fb4600bc450 0x7fb4600be8f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:44.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.175+0000 7fb470d99700 1 -- 172.21.15.2:0/3595801232 <== osd.5 v2:172.21.15.151:6808/3695961886 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb4600befc0 con 0x7fb4600bc450 2024-05-14T16:52:44.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.192+0000 7fb4790b7700 1 -- 172.21.15.2:0/3595801232 --> [v2:172.21.15.151:6808/3695961886,v1:172.21.15.151:6809/3695961886] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb4740640d0 con 0x7fb4600bc450 2024-05-14T16:52:44.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.192+0000 7fb470d99700 1 -- 172.21.15.2:0/3595801232 <== osd.5 v2:172.21.15.151:6808/3695961886 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb4740640d0 con 0x7fb4600bc450 2024-05-14T16:52:44.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.193+0000 7fb4790b7700 1 -- 172.21.15.2:0/3595801232 >> [v2:172.21.15.151:6808/3695961886,v1:172.21.15.151:6809/3695961886] conn(0x7fb4600bc450 msgr2=0x7fb4600be8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:44.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.193+0000 7fb4790b7700 1 --2- 172.21.15.2:0/3595801232 >> [v2:172.21.15.151:6808/3695961886,v1:172.21.15.151:6809/3695961886] conn(0x7fb4600bc450 0x7fb4600be8f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:44.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.193+0000 7fb4790b7700 1 -- 172.21.15.2:0/3595801232 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fb4600b2910 msgr2=0x7fb4600b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:44.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.193+0000 7fb4790b7700 1 --2- 172.21.15.2:0/3595801232 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fb4600b2910 0x7fb4600b4dd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb464009ca0 tx=0x7fb464009260 comp rx=0 tx=0).stop 2024-05-14T16:52:44.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.193+0000 7fb4790b7700 1 -- 172.21.15.2:0/3595801232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb474102e00 msgr2=0x7fb47419afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:44.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.193+0000 7fb4790b7700 1 --2- 172.21.15.2:0/3595801232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb474102e00 0x7fb47419afc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb45c00d750 tx=0x7fb45c00da60 comp rx=0 tx=0).stop 2024-05-14T16:52:44.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.193+0000 7fb47359e700 1 -- 172.21.15.2:0/3595801232 reap_dead start 2024-05-14T16:52:44.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.193+0000 7fb4790b7700 1 -- 172.21.15.2:0/3595801232 shutdown_connections 2024-05-14T16:52:44.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.193+0000 7fb4790b7700 1 -- 172.21.15.2:0/3595801232 >> 172.21.15.2:0/3595801232 conn(0x7fb4740793f0 msgr2=0x7fb474100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:44.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.193+0000 7fb4790b7700 1 -- 172.21.15.2:0/3595801232 shutdown_connections 2024-05-14T16:52:44.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:44.193+0000 7fb4790b7700 1 -- 172.21.15.2:0/3595801232 wait complete. 2024-05-14T16:52:45.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:45 smithi002 ceph-mon[31538]: pgmap v326: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:52:45.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:45 smithi151 ceph-mon[36066]: pgmap v326: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:52:45.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:45 smithi123 ceph-mon[35971]: pgmap v326: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:52:45.766 INFO:teuthology.orchestra.run.smithi002.stdout:154618822718 2024-05-14T16:52:45.767 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-14T16:52:47.173 INFO:teuthology.orchestra.run.smithi002.stdout:133143986242 2024-05-14T16:52:47.174 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-14T16:52:47.404 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:47 smithi002 ceph-mon[31538]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:52:47.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.408+0000 7effe41a4700 1 -- 172.21.15.2:0/1892031407 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7effdc105750 msgr2=0x7effdc105bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:47.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.408+0000 7effe41a4700 1 --2- 172.21.15.2:0/1892031407 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7effdc105750 0x7effdc105bd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7effcc009990 tx=0x7effcc009ca0 comp rx=0 tx=0).stop 2024-05-14T16:52:47.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.408+0000 7effe41a4700 1 -- 172.21.15.2:0/1892031407 shutdown_connections 2024-05-14T16:52:47.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.408+0000 7effe41a4700 1 --2- 172.21.15.2:0/1892031407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effdc106110 0x7effdc112620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:47.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.408+0000 7effe41a4700 1 --2- 172.21.15.2:0/1892031407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7effdc10b750 0x7effdc10bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:47.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.408+0000 7effe41a4700 1 --2- 172.21.15.2:0/1892031407 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7effdc105750 0x7effdc105bd0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:47.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.408+0000 7effe41a4700 1 -- 172.21.15.2:0/1892031407 >> 172.21.15.2:0/1892031407 conn(0x7effdc101240 msgr2=0x7effdc103660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:47.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.408+0000 7effe41a4700 1 -- 172.21.15.2:0/1892031407 shutdown_connections 2024-05-14T16:52:47.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.408+0000 7effe41a4700 1 -- 172.21.15.2:0/1892031407 wait complete. 2024-05-14T16:52:47.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.409+0000 7effe41a4700 1 Processor -- start 2024-05-14T16:52:47.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.409+0000 7effe41a4700 1 -- start start 2024-05-14T16:52:47.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.409+0000 7effe41a4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effdc105750 0x7effdc19be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:47.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.409+0000 7effe41a4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7effdc106110 0x7effdc19c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:47.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.409+0000 7effe41a4700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7effdc10b750 0x7effdc1a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:47.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.409+0000 7effe41a4700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7effdc115990 con 0x7effdc106110 2024-05-14T16:52:47.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.409+0000 7effe41a4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7effdc115810 con 0x7effdc105750 2024-05-14T16:52:47.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.409+0000 7effe41a4700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7effdc115b10 con 0x7effdc10b750 2024-05-14T16:52:47.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.409+0000 7effe173f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7effdc106110 0x7effdc19c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:47.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.410+0000 7effe1f40700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effdc105750 0x7effdc19be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:47.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.410+0000 7effe173f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7effdc106110 0x7effdc19c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56236/0 (socket says 172.21.15.2:56236) 2024-05-14T16:52:47.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.410+0000 7effe173f700 1 -- 172.21.15.2:0/3388081741 learned_addr learned my addr 172.21.15.2:0/3388081741 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:52:47.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.410+0000 7effe2741700 1 --2- 172.21.15.2:0/3388081741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7effdc10b750 0x7effdc1a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:47.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.410+0000 7effe173f700 1 -- 172.21.15.2:0/3388081741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7effdc10b750 msgr2=0x7effdc1a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:47.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.410+0000 7effe173f700 1 --2- 172.21.15.2:0/3388081741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7effdc10b750 0x7effdc1a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:47.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.410+0000 7effe173f700 1 -- 172.21.15.2:0/3388081741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effdc105750 msgr2=0x7effdc19be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:47.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.410+0000 7effe173f700 1 --2- 172.21.15.2:0/3388081741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effdc105750 0x7effdc19be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:47.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.410+0000 7effe173f700 1 -- 172.21.15.2:0/3388081741 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effdc1a0e20 con 0x7effdc106110 2024-05-14T16:52:47.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.410+0000 7effe1f40700 1 --2- 172.21.15.2:0/3388081741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effdc105750 0x7effdc19be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:52:47.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.410+0000 7effe173f700 1 --2- 172.21.15.2:0/3388081741 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7effdc106110 0x7effdc19c3a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7effcc000c00 tx=0x7effcc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:47.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.410+0000 7effd2ffd700 1 -- 172.21.15.2:0/3388081741 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc027070 con 0x7effdc106110 2024-05-14T16:52:47.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.410+0000 7effd2ffd700 1 -- 172.21.15.2:0/3388081741 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effcc003fa0 con 0x7effdc106110 2024-05-14T16:52:47.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.411+0000 7effe41a4700 1 -- 172.21.15.2:0/3388081741 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effdc1a10b0 con 0x7effdc106110 2024-05-14T16:52:47.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.411+0000 7effe41a4700 1 -- 172.21.15.2:0/3388081741 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effdc1a8990 con 0x7effdc106110 2024-05-14T16:52:47.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.411+0000 7effd2ffd700 1 -- 172.21.15.2:0/3388081741 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc022d40 con 0x7effdc106110 2024-05-14T16:52:47.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.411+0000 7effe41a4700 1 -- 172.21.15.2:0/3388081741 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7effc0001030 con 0x7effdc106110 2024-05-14T16:52:47.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.413+0000 7effd2ffd700 1 -- 172.21.15.2:0/3388081741 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7effcc02a070 con 0x7effdc106110 2024-05-14T16:52:47.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.414+0000 7effd2ffd700 1 --2- 172.21.15.2:0/3388081741 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7effc80b2990 0x7effc80b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:47.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.414+0000 7effd2ffd700 1 -- 172.21.15.2:0/3388081741 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7effcc0e3c50 con 0x7effdc106110 2024-05-14T16:52:47.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.415+0000 7effd2ffd700 1 --2- 172.21.15.2:0/3388081741 >> [v2:172.21.15.123:6816/3634155507,v1:172.21.15.123:6817/3634155507] conn(0x7effc80bc4d0 0x7effc80be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:47.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.415+0000 7effd2ffd700 1 -- 172.21.15.2:0/3388081741 --> [v2:172.21.15.123:6816/3634155507,v1:172.21.15.123:6817/3634155507] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7effcc031e80 con 0x7effc80bc4d0 2024-05-14T16:52:47.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.415+0000 7effd2ffd700 1 -- 172.21.15.2:0/3388081741 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7effcc0e4000 con 0x7effdc106110 2024-05-14T16:52:47.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.415+0000 7effe1f40700 1 --2- 172.21.15.2:0/3388081741 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7effc80b2990 0x7effc80b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:47.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.415+0000 7effe2741700 1 --2- 172.21.15.2:0/3388081741 >> [v2:172.21.15.123:6816/3634155507,v1:172.21.15.123:6817/3634155507] conn(0x7effc80bc4d0 0x7effc80be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:47.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.416+0000 7effe2741700 1 --2- 172.21.15.2:0/3388081741 >> [v2:172.21.15.123:6816/3634155507,v1:172.21.15.123:6817/3634155507] conn(0x7effc80bc4d0 0x7effc80be970 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:47.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.416+0000 7effe1f40700 1 --2- 172.21.15.2:0/3388081741 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7effc80b2990 0x7effc80b4e50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7effd8009d50 tx=0x7effd8009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:47.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.417+0000 7effd2ffd700 1 -- 172.21.15.2:0/3388081741 <== osd.7 v2:172.21.15.123:6816/3634155507 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7effcc031e80 con 0x7effc80bc4d0 2024-05-14T16:52:47.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.440+0000 7effe41a4700 1 -- 172.21.15.2:0/3388081741 --> [v2:172.21.15.123:6816/3634155507,v1:172.21.15.123:6817/3634155507] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7effc0002e60 con 0x7effc80bc4d0 2024-05-14T16:52:47.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.441+0000 7effd2ffd700 1 -- 172.21.15.2:0/3388081741 <== osd.7 v2:172.21.15.123:6816/3634155507 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7effc0002e60 con 0x7effc80bc4d0 2024-05-14T16:52:47.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.441+0000 7effe41a4700 1 -- 172.21.15.2:0/3388081741 >> [v2:172.21.15.123:6816/3634155507,v1:172.21.15.123:6817/3634155507] conn(0x7effc80bc4d0 msgr2=0x7effc80be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:47.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.441+0000 7effe41a4700 1 --2- 172.21.15.2:0/3388081741 >> [v2:172.21.15.123:6816/3634155507,v1:172.21.15.123:6817/3634155507] conn(0x7effc80bc4d0 0x7effc80be970 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:47.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.441+0000 7effe41a4700 1 -- 172.21.15.2:0/3388081741 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7effc80b2990 msgr2=0x7effc80b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:47.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.441+0000 7effe41a4700 1 --2- 172.21.15.2:0/3388081741 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7effc80b2990 0x7effc80b4e50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7effd8009d50 tx=0x7effd8009040 comp rx=0 tx=0).stop 2024-05-14T16:52:47.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.441+0000 7effe41a4700 1 -- 172.21.15.2:0/3388081741 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7effdc106110 msgr2=0x7effdc19c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:47.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.441+0000 7effe41a4700 1 --2- 172.21.15.2:0/3388081741 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7effdc106110 0x7effdc19c3a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7effcc000c00 tx=0x7effcc0047b0 comp rx=0 tx=0).stop 2024-05-14T16:52:47.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.442+0000 7effe2741700 1 -- 172.21.15.2:0/3388081741 reap_dead start 2024-05-14T16:52:47.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.442+0000 7effe41a4700 1 -- 172.21.15.2:0/3388081741 shutdown_connections 2024-05-14T16:52:47.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.442+0000 7effe41a4700 1 -- 172.21.15.2:0/3388081741 >> 172.21.15.2:0/3388081741 conn(0x7effdc101240 msgr2=0x7effdc101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:47.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.442+0000 7effe41a4700 1 -- 172.21.15.2:0/3388081741 shutdown_connections 2024-05-14T16:52:47.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:47.442+0000 7effe41a4700 1 -- 172.21.15.2:0/3388081741 wait complete. 2024-05-14T16:52:47.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:47 smithi151 ceph-mon[36066]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:52:47.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.484+0000 7fbb4691e700 1 -- 172.21.15.2:0/246858558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbb40102e20 msgr2=0x7fbb4010f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:47.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.484+0000 7fbb4691e700 1 --2- 172.21.15.2:0/246858558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbb40102e20 0x7fbb4010f330 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbb3800b0a0 tx=0x7fbb3800b3b0 comp rx=0 tx=0).stop 2024-05-14T16:52:47.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.484+0000 7fbb4691e700 1 -- 172.21.15.2:0/246858558 shutdown_connections 2024-05-14T16:52:47.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.484+0000 7fbb4691e700 1 --2- 172.21.15.2:0/246858558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbb40102e20 0x7fbb4010f330 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:47.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.484+0000 7fbb4691e700 1 --2- 172.21.15.2:0/246858558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb401084f0 0x7fbb401088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:47.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.484+0000 7fbb4691e700 1 --2- 172.21.15.2:0/246858558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb40102460 0x7fbb401028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:47.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.484+0000 7fbb4691e700 1 -- 172.21.15.2:0/246858558 >> 172.21.15.2:0/246858558 conn(0x7fbb400794e0 msgr2=0x7fbb400798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:47.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.484+0000 7fbb4691e700 1 -- 172.21.15.2:0/246858558 shutdown_connections 2024-05-14T16:52:47.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.484+0000 7fbb4691e700 1 -- 172.21.15.2:0/246858558 wait complete. 2024-05-14T16:52:47.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.485+0000 7fbb4691e700 1 Processor -- start 2024-05-14T16:52:47.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.485+0000 7fbb4691e700 1 -- start start 2024-05-14T16:52:47.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.485+0000 7fbb4691e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb40102460 0x7fbb4019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:47.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.485+0000 7fbb4691e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb40102e20 0x7fbb4019fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:47.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.485+0000 7fbb4691e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbb401084f0 0x7fbb401999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:47.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.485+0000 7fbb4691e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbb40112620 con 0x7fbb401084f0 2024-05-14T16:52:47.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.485+0000 7fbb4691e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbb401124a0 con 0x7fbb40102e20 2024-05-14T16:52:47.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.485+0000 7fbb4691e700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbb401127a0 con 0x7fbb40102460 2024-05-14T16:52:47.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.485+0000 7fbb44ebb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbb401084f0 0x7fbb401999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:47.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.485+0000 7fbb44ebb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbb401084f0 0x7fbb401999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39292/0 (socket says 172.21.15.2:39292) 2024-05-14T16:52:47.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.485+0000 7fbb44ebb700 1 -- 172.21.15.2:0/3218188881 learned_addr learned my addr 172.21.15.2:0/3218188881 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:52:47.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.485+0000 7fbb37fff700 1 --2- 172.21.15.2:0/3218188881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb40102e20 0x7fbb4019fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:47.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.485+0000 7fbb3ffff700 1 --2- 172.21.15.2:0/3218188881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb40102460 0x7fbb4019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:47.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.486+0000 7fbb44ebb700 1 -- 172.21.15.2:0/3218188881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb40102460 msgr2=0x7fbb4019f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:47.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.486+0000 7fbb44ebb700 1 --2- 172.21.15.2:0/3218188881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb40102460 0x7fbb4019f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:47.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.486+0000 7fbb44ebb700 1 -- 172.21.15.2:0/3218188881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb40102e20 msgr2=0x7fbb4019fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:47.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.486+0000 7fbb44ebb700 1 --2- 172.21.15.2:0/3218188881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb40102e20 0x7fbb4019fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:47.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.486+0000 7fbb44ebb700 1 -- 172.21.15.2:0/3218188881 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb4019a130 con 0x7fbb401084f0 2024-05-14T16:52:47.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.486+0000 7fbb44ebb700 1 --2- 172.21.15.2:0/3218188881 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbb401084f0 0x7fbb401999d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbb3800b830 tx=0x7fbb38004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:47.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.486+0000 7fbb3dffb700 1 -- 172.21.15.2:0/3218188881 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb38021070 con 0x7fbb401084f0 2024-05-14T16:52:47.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.486+0000 7fbb4691e700 1 -- 172.21.15.2:0/3218188881 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb4019a3c0 con 0x7fbb401084f0 2024-05-14T16:52:47.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.486+0000 7fbb3dffb700 1 -- 172.21.15.2:0/3218188881 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb38007e30 con 0x7fbb401084f0 2024-05-14T16:52:47.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.486+0000 7fbb3dffb700 1 -- 172.21.15.2:0/3218188881 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb380047d0 con 0x7fbb401084f0 2024-05-14T16:52:47.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.486+0000 7fbb4691e700 1 -- 172.21.15.2:0/3218188881 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb401a66e0 con 0x7fbb401084f0 2024-05-14T16:52:47.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.487+0000 7fbb4691e700 1 -- 172.21.15.2:0/3218188881 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbb40063cd0 con 0x7fbb401084f0 2024-05-14T16:52:47.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.488+0000 7fbb3dffb700 1 -- 172.21.15.2:0/3218188881 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fbb38007980 con 0x7fbb401084f0 2024-05-14T16:52:47.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.489+0000 7fbb3dffb700 1 --2- 172.21.15.2:0/3218188881 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fbb2c0b2910 0x7fbb2c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:47.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.489+0000 7fbb3ffff700 1 --2- 172.21.15.2:0/3218188881 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fbb2c0b2910 0x7fbb2c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:47.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.489+0000 7fbb3dffb700 1 -- 172.21.15.2:0/3218188881 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7fbb3800f020 con 0x7fbb401084f0 2024-05-14T16:52:47.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.489+0000 7fbb3dffb700 1 --2- 172.21.15.2:0/3218188881 >> [v2:172.21.15.2:6802/2654237820,v1:172.21.15.2:6803/2654237820] conn(0x7fbb2c0bc450 0x7fbb2c0be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:47.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.489+0000 7fbb3dffb700 1 -- 172.21.15.2:0/3218188881 --> [v2:172.21.15.2:6802/2654237820,v1:172.21.15.2:6803/2654237820] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbb2c0befc0 con 0x7fbb2c0bc450 2024-05-14T16:52:47.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.489+0000 7fbb3dffb700 1 -- 172.21.15.2:0/3218188881 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fbb380e3f60 con 0x7fbb401084f0 2024-05-14T16:52:47.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.489+0000 7fbb37fff700 1 --2- 172.21.15.2:0/3218188881 >> [v2:172.21.15.2:6802/2654237820,v1:172.21.15.2:6803/2654237820] conn(0x7fbb2c0bc450 0x7fbb2c0be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:47.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.489+0000 7fbb3ffff700 1 --2- 172.21.15.2:0/3218188881 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fbb2c0b2910 0x7fbb2c0b4dd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbb30005fd0 tx=0x7fbb30005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:47.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.490+0000 7fbb37fff700 1 --2- 172.21.15.2:0/3218188881 >> [v2:172.21.15.2:6802/2654237820,v1:172.21.15.2:6803/2654237820] conn(0x7fbb2c0bc450 0x7fbb2c0be8f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:47.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.490+0000 7fbb3dffb700 1 -- 172.21.15.2:0/3218188881 <== osd.0 v2:172.21.15.2:6802/2654237820 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbb2c0befc0 con 0x7fbb2c0bc450 2024-05-14T16:52:47.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.507+0000 7fbb4691e700 1 -- 172.21.15.2:0/3218188881 --> [v2:172.21.15.2:6802/2654237820,v1:172.21.15.2:6803/2654237820] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbb400640d0 con 0x7fbb2c0bc450 2024-05-14T16:52:47.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.508+0000 7fbb3dffb700 1 -- 172.21.15.2:0/3218188881 <== osd.0 v2:172.21.15.2:6802/2654237820 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbb400640d0 con 0x7fbb2c0bc450 2024-05-14T16:52:47.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.508+0000 7fbb4691e700 1 -- 172.21.15.2:0/3218188881 >> [v2:172.21.15.2:6802/2654237820,v1:172.21.15.2:6803/2654237820] conn(0x7fbb2c0bc450 msgr2=0x7fbb2c0be8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:47.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.508+0000 7fbb4691e700 1 --2- 172.21.15.2:0/3218188881 >> [v2:172.21.15.2:6802/2654237820,v1:172.21.15.2:6803/2654237820] conn(0x7fbb2c0bc450 0x7fbb2c0be8f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:47.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.508+0000 7fbb4691e700 1 -- 172.21.15.2:0/3218188881 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fbb2c0b2910 msgr2=0x7fbb2c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:47.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.508+0000 7fbb4691e700 1 --2- 172.21.15.2:0/3218188881 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fbb2c0b2910 0x7fbb2c0b4dd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbb30005fd0 tx=0x7fbb30005d90 comp rx=0 tx=0).stop 2024-05-14T16:52:47.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.508+0000 7fbb4691e700 1 -- 172.21.15.2:0/3218188881 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbb401084f0 msgr2=0x7fbb401999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:47.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.508+0000 7fbb4691e700 1 --2- 172.21.15.2:0/3218188881 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbb401084f0 0x7fbb401999d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbb3800b830 tx=0x7fbb38004ce0 comp rx=0 tx=0).stop 2024-05-14T16:52:47.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.508+0000 7fbb44ebb700 1 -- 172.21.15.2:0/3218188881 reap_dead start 2024-05-14T16:52:47.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.508+0000 7fbb4691e700 1 -- 172.21.15.2:0/3218188881 shutdown_connections 2024-05-14T16:52:47.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.508+0000 7fbb4691e700 1 -- 172.21.15.2:0/3218188881 >> 172.21.15.2:0/3218188881 conn(0x7fbb400794e0 msgr2=0x7fbb4006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:47.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.508+0000 7fbb4691e700 1 -- 172.21.15.2:0/3218188881 shutdown_connections 2024-05-14T16:52:47.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:43.508+0000 7fbb4691e700 1 -- 172.21.15.2:0/3218188881 wait complete. 2024-05-14T16:52:47.663 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:47 smithi123 ceph-mon[35971]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:52:49.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:49 smithi002 ceph-mon[31538]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T16:52:49.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:49 smithi151 ceph-mon[36066]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T16:52:49.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:49 smithi123 ceph-mon[35971]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T16:52:49.703 INFO:teuthology.orchestra.run.smithi002.stdout:201863462963 2024-05-14T16:52:49.704 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-14T16:52:49.835 INFO:teuthology.orchestra.run.smithi002.stdout:34359738453 2024-05-14T16:52:49.836 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-14T16:52:50.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.277+0000 7fbab091e700 1 -- 172.21.15.2:0/347062180 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbaac103fc0 msgr2=0x7fbaac1103b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:50.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.277+0000 7fbab091e700 1 --2- 172.21.15.2:0/347062180 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbaac103fc0 0x7fbaac1103b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbaa400b0d0 tx=0x7fbaa400b3e0 comp rx=0 tx=0).stop 2024-05-14T16:52:50.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.277+0000 7fbab091e700 1 -- 172.21.15.2:0/347062180 shutdown_connections 2024-05-14T16:52:50.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.277+0000 7fbab091e700 1 --2- 172.21.15.2:0/347062180 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbaac103fc0 0x7fbaac1103b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:50.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.277+0000 7fbab091e700 1 --2- 172.21.15.2:0/347062180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaac1095e0 0x7fbaac1099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:50.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.277+0000 7fbab091e700 1 --2- 172.21.15.2:0/347062180 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbaac103600 0x7fbaac103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:50.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.277+0000 7fbab091e700 1 -- 172.21.15.2:0/347062180 >> 172.21.15.2:0/347062180 conn(0x7fbaac07a700 msgr2=0x7fbaac07ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:50.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.277+0000 7fbab091e700 1 -- 172.21.15.2:0/347062180 shutdown_connections 2024-05-14T16:52:50.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.277+0000 7fbab091e700 1 -- 172.21.15.2:0/347062180 wait complete. 2024-05-14T16:52:50.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.278+0000 7fbab091e700 1 Processor -- start 2024-05-14T16:52:50.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.278+0000 7fbab091e700 1 -- start start 2024-05-14T16:52:50.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.278+0000 7fbab091e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbaac103600 0x7fbaac19f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:50.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.278+0000 7fbab091e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbaac103fc0 0x7fbaac19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:50.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.278+0000 7fbab091e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaac1095e0 0x7fbaac199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:50.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.278+0000 7fbab091e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac113740 con 0x7fbaac103fc0 2024-05-14T16:52:50.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.278+0000 7fbab091e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac1135c0 con 0x7fbaac1095e0 2024-05-14T16:52:50.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.278+0000 7fbab091e700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac1138c0 con 0x7fbaac103600 2024-05-14T16:52:50.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.278+0000 7fbaa9d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbaac103fc0 0x7fbaac19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:50.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.278+0000 7fbaa9d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbaac103fc0 0x7fbaac19fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56254/0 (socket says 172.21.15.2:56254) 2024-05-14T16:52:50.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.278+0000 7fbaa9d9b700 1 -- 172.21.15.2:0/1684824185 learned_addr learned my addr 172.21.15.2:0/1684824185 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:52:50.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.278+0000 7fbaaa59c700 1 --2- 172.21.15.2:0/1684824185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbaac103600 0x7fbaac19f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:50.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.278+0000 7fbaaad9d700 1 --2- 172.21.15.2:0/1684824185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaac1095e0 0x7fbaac199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:50.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.279+0000 7fbaa9d9b700 1 -- 172.21.15.2:0/1684824185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbaac103600 msgr2=0x7fbaac19f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:50.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.279+0000 7fbaa9d9b700 1 --2- 172.21.15.2:0/1684824185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbaac103600 0x7fbaac19f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:50.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.279+0000 7fbaa9d9b700 1 -- 172.21.15.2:0/1684824185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaac1095e0 msgr2=0x7fbaac199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:50.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.279+0000 7fbaa9d9b700 1 --2- 172.21.15.2:0/1684824185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaac1095e0 0x7fbaac199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:50.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.279+0000 7fbaa9d9b700 1 -- 172.21.15.2:0/1684824185 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaac19a080 con 0x7fbaac103fc0 2024-05-14T16:52:50.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.279+0000 7fbaaa59c700 1 --2- 172.21.15.2:0/1684824185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbaac103600 0x7fbaac19f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:52:50.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.279+0000 7fbaa9d9b700 1 --2- 172.21.15.2:0/1684824185 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbaac103fc0 0x7fbaac19fca0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbaa000d980 tx=0x7fbaa000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:50.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.279+0000 7fba9b7fe700 1 -- 172.21.15.2:0/1684824185 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0017070 con 0x7fbaac103fc0 2024-05-14T16:52:50.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.279+0000 7fbab091e700 1 -- 172.21.15.2:0/1684824185 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaac19a370 con 0x7fbaac103fc0 2024-05-14T16:52:50.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.279+0000 7fbab091e700 1 -- 172.21.15.2:0/1684824185 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaac1a6630 con 0x7fbaac103fc0 2024-05-14T16:52:50.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.279+0000 7fba9b7fe700 1 -- 172.21.15.2:0/1684824185 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbaa0005020 con 0x7fbaac103fc0 2024-05-14T16:52:50.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.279+0000 7fba9b7fe700 1 -- 172.21.15.2:0/1684824185 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0009b50 con 0x7fbaac103fc0 2024-05-14T16:52:50.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.280+0000 7fbab091e700 1 -- 172.21.15.2:0/1684824185 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fba8c001030 con 0x7fbaac103fc0 2024-05-14T16:52:50.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.282+0000 7fba9b7fe700 1 -- 172.21.15.2:0/1684824185 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fbaa0009cf0 con 0x7fbaac103fc0 2024-05-14T16:52:50.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.283+0000 7fba9b7fe700 1 --2- 172.21.15.2:0/1684824185 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fba940b2990 0x7fba940b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:50.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.283+0000 7fbaaa59c700 1 --2- 172.21.15.2:0/1684824185 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fba940b2990 0x7fba940b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:50.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.283+0000 7fba9b7fe700 1 -- 172.21.15.2:0/1684824185 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7fbaa00e2110 con 0x7fbaac103fc0 2024-05-14T16:52:50.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.283+0000 7fba9b7fe700 1 --2- 172.21.15.2:0/1684824185 >> [v2:172.21.15.123:6800/2668156992,v1:172.21.15.123:6801/2668156992] conn(0x7fba940bc4d0 0x7fba940be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:50.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.283+0000 7fba9b7fe700 1 -- 172.21.15.2:0/1684824185 --> [v2:172.21.15.123:6800/2668156992,v1:172.21.15.123:6801/2668156992] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbaa0019ec0 con 0x7fba940bc4d0 2024-05-14T16:52:50.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.283+0000 7fba9b7fe700 1 -- 172.21.15.2:0/1684824185 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fbaa00e9050 con 0x7fbaac103fc0 2024-05-14T16:52:50.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.284+0000 7fbaaad9d700 1 --2- 172.21.15.2:0/1684824185 >> [v2:172.21.15.123:6800/2668156992,v1:172.21.15.123:6801/2668156992] conn(0x7fba940bc4d0 0x7fba940be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:50.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.284+0000 7fbaaa59c700 1 --2- 172.21.15.2:0/1684824185 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fba940b2990 0x7fba940b4e50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fba9c005950 tx=0x7fba9c0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:50.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.285+0000 7fbaaad9d700 1 --2- 172.21.15.2:0/1684824185 >> [v2:172.21.15.123:6800/2668156992,v1:172.21.15.123:6801/2668156992] conn(0x7fba940bc4d0 0x7fba940be970 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:50.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.286+0000 7fba9b7fe700 1 -- 172.21.15.2:0/1684824185 <== osd.1 v2:172.21.15.123:6800/2668156992 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbaa0019ec0 con 0x7fba940bc4d0 2024-05-14T16:52:50.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.303+0000 7fbab091e700 1 -- 172.21.15.2:0/1684824185 --> [v2:172.21.15.123:6800/2668156992,v1:172.21.15.123:6801/2668156992] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fba8c002e60 con 0x7fba940bc4d0 2024-05-14T16:52:50.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.303+0000 7fba9b7fe700 1 -- 172.21.15.2:0/1684824185 <== osd.1 v2:172.21.15.123:6800/2668156992 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fba8c002e60 con 0x7fba940bc4d0 2024-05-14T16:52:50.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.303+0000 7fbab091e700 1 -- 172.21.15.2:0/1684824185 >> [v2:172.21.15.123:6800/2668156992,v1:172.21.15.123:6801/2668156992] conn(0x7fba940bc4d0 msgr2=0x7fba940be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:50.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.304+0000 7fbab091e700 1 --2- 172.21.15.2:0/1684824185 >> [v2:172.21.15.123:6800/2668156992,v1:172.21.15.123:6801/2668156992] conn(0x7fba940bc4d0 0x7fba940be970 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:50.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.304+0000 7fbab091e700 1 -- 172.21.15.2:0/1684824185 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fba940b2990 msgr2=0x7fba940b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:50.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.304+0000 7fbab091e700 1 --2- 172.21.15.2:0/1684824185 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fba940b2990 0x7fba940b4e50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fba9c005950 tx=0x7fba9c0058e0 comp rx=0 tx=0).stop 2024-05-14T16:52:50.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.304+0000 7fbab091e700 1 -- 172.21.15.2:0/1684824185 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbaac103fc0 msgr2=0x7fbaac19fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:50.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.304+0000 7fbab091e700 1 --2- 172.21.15.2:0/1684824185 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbaac103fc0 0x7fbaac19fca0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbaa000d980 tx=0x7fbaa000dc90 comp rx=0 tx=0).stop 2024-05-14T16:52:50.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.304+0000 7fbaaad9d700 1 -- 172.21.15.2:0/1684824185 reap_dead start 2024-05-14T16:52:50.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.304+0000 7fbab091e700 1 -- 172.21.15.2:0/1684824185 shutdown_connections 2024-05-14T16:52:50.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.304+0000 7fbab091e700 1 -- 172.21.15.2:0/1684824185 >> 172.21.15.2:0/1684824185 conn(0x7fbaac07a700 msgr2=0x7fbaac102280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:50.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.304+0000 7fbab091e700 1 -- 172.21.15.2:0/1684824185 shutdown_connections 2024-05-14T16:52:50.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.304+0000 7fbab091e700 1 -- 172.21.15.2:0/1684824185 wait complete. 2024-05-14T16:52:50.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.773+0000 7fa5a9864700 1 -- 172.21.15.2:0/4261807026 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5a406c5d0 msgr2=0x7fa5a4110420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:50.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.773+0000 7fa5a9864700 1 --2- 172.21.15.2:0/4261807026 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5a406c5d0 0x7fa5a4110420 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009990 tx=0x7fa59c009ca0 comp rx=0 tx=0).stop 2024-05-14T16:52:50.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.773+0000 7fa5a9864700 1 -- 172.21.15.2:0/4261807026 shutdown_connections 2024-05-14T16:52:50.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.773+0000 7fa5a9864700 1 --2- 172.21.15.2:0/4261807026 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5a406c5d0 0x7fa5a4110420 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:50.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.773+0000 7fa5a9864700 1 --2- 172.21.15.2:0/4261807026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5a4109720 0x7fa5a4109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:50.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.773+0000 7fa5a9864700 1 --2- 172.21.15.2:0/4261807026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5a406bc10 0x7fa5a406c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:50.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.773+0000 7fa5a9864700 1 -- 172.21.15.2:0/4261807026 >> 172.21.15.2:0/4261807026 conn(0x7fa5a40793f0 msgr2=0x7fa5a4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:50.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.773+0000 7fa5a9864700 1 -- 172.21.15.2:0/4261807026 shutdown_connections 2024-05-14T16:52:50.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.773+0000 7fa5a9864700 1 -- 172.21.15.2:0/4261807026 wait complete. 2024-05-14T16:52:50.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.774+0000 7fa5a9864700 1 Processor -- start 2024-05-14T16:52:50.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.774+0000 7fa5a9864700 1 -- start start 2024-05-14T16:52:50.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.775+0000 7fa5a9864700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5a406bc10 0x7fa5a41028f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:50.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.775+0000 7fa5a9864700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5a406c5d0 0x7fa5a4102e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:50.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.775+0000 7fa5a9864700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5a4109720 0x7fa5a4103370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:50.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.775+0000 7fa5a9864700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a4113700 con 0x7fa5a406c5d0 2024-05-14T16:52:50.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.775+0000 7fa5a9864700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a4113580 con 0x7fa5a4109720 2024-05-14T16:52:50.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.775+0000 7fa5a9864700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a4113880 con 0x7fa5a406bc10 2024-05-14T16:52:50.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.775+0000 7fa5a27fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5a406c5d0 0x7fa5a4102e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:50.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.775+0000 7fa5a27fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5a406c5d0 0x7fa5a4102e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56264/0 (socket says 172.21.15.2:56264) 2024-05-14T16:52:50.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.775+0000 7fa5a27fc700 1 -- 172.21.15.2:0/4180125210 learned_addr learned my addr 172.21.15.2:0/4180125210 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:52:50.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.775+0000 7fa5a37fe700 1 --2- 172.21.15.2:0/4180125210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5a4109720 0x7fa5a4103370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:50.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.775+0000 7fa5a2ffd700 1 --2- 172.21.15.2:0/4180125210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5a406bc10 0x7fa5a41028f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:50.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.775+0000 7fa5a27fc700 1 -- 172.21.15.2:0/4180125210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5a406bc10 msgr2=0x7fa5a41028f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:50.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.775+0000 7fa5a27fc700 1 --2- 172.21.15.2:0/4180125210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5a406bc10 0x7fa5a41028f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:50.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.775+0000 7fa5a27fc700 1 -- 172.21.15.2:0/4180125210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5a4109720 msgr2=0x7fa5a4103370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:50.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.775+0000 7fa5a27fc700 1 --2- 172.21.15.2:0/4180125210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5a4109720 0x7fa5a4103370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:50.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.775+0000 7fa5a27fc700 1 -- 172.21.15.2:0/4180125210 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a41a80e0 con 0x7fa5a406c5d0 2024-05-14T16:52:50.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.775+0000 7fa5a27fc700 1 --2- 172.21.15.2:0/4180125210 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5a406c5d0 0x7fa5a4102e30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa59800d980 tx=0x7fa59800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:50.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.775+0000 7fa5a8862700 1 -- 172.21.15.2:0/4180125210 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa598016070 con 0x7fa5a406c5d0 2024-05-14T16:52:50.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.775+0000 7fa5a9864700 1 -- 172.21.15.2:0/4180125210 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a41a83d0 con 0x7fa5a406c5d0 2024-05-14T16:52:50.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.775+0000 7fa5a8862700 1 -- 172.21.15.2:0/4180125210 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa598004830 con 0x7fa5a406c5d0 2024-05-14T16:52:50.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.775+0000 7fa5a8862700 1 -- 172.21.15.2:0/4180125210 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa598005540 con 0x7fa5a406c5d0 2024-05-14T16:52:50.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.775+0000 7fa5a9864700 1 -- 172.21.15.2:0/4180125210 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a41a8960 con 0x7fa5a406c5d0 2024-05-14T16:52:50.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.776+0000 7fa5a9864700 1 -- 172.21.15.2:0/4180125210 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa5a4063c50 con 0x7fa5a406c5d0 2024-05-14T16:52:50.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.778+0000 7fa5a8862700 1 -- 172.21.15.2:0/4180125210 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fa598004220 con 0x7fa5a406c5d0 2024-05-14T16:52:50.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.778+0000 7fa5a8862700 1 --2- 172.21.15.2:0/4180125210 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fa58c0b2910 0x7fa58c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:50.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.779+0000 7fa5a2ffd700 1 --2- 172.21.15.2:0/4180125210 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fa58c0b2910 0x7fa58c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:50.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.779+0000 7fa5a8862700 1 -- 172.21.15.2:0/4180125210 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7fa5980e2820 con 0x7fa5a406c5d0 2024-05-14T16:52:50.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.779+0000 7fa5a8862700 1 --2- 172.21.15.2:0/4180125210 >> [v2:172.21.15.151:6816/4271107010,v1:172.21.15.151:6817/4271107010] conn(0x7fa58c0bc450 0x7fa58c0be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:50.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.779+0000 7fa5a8862700 1 -- 172.21.15.2:0/4180125210 --> [v2:172.21.15.151:6816/4271107010,v1:172.21.15.151:6817/4271107010] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa59800d130 con 0x7fa58c0bc450 2024-05-14T16:52:50.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.779+0000 7fa5a8862700 1 -- 172.21.15.2:0/4180125210 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fa5980e2c70 con 0x7fa5a406c5d0 2024-05-14T16:52:50.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.779+0000 7fa5a2ffd700 1 --2- 172.21.15.2:0/4180125210 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fa58c0b2910 0x7fa58c0b4dd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa594009550 tx=0x7fa5940091b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:50.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.779+0000 7fa5a37fe700 1 --2- 172.21.15.2:0/4180125210 >> [v2:172.21.15.151:6816/4271107010,v1:172.21.15.151:6817/4271107010] conn(0x7fa58c0bc450 0x7fa58c0be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:50.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.780+0000 7fa5a37fe700 1 --2- 172.21.15.2:0/4180125210 >> [v2:172.21.15.151:6816/4271107010,v1:172.21.15.151:6817/4271107010] conn(0x7fa58c0bc450 0x7fa58c0be8f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:50.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.780+0000 7fa5a8862700 1 -- 172.21.15.2:0/4180125210 <== osd.8 v2:172.21.15.151:6816/4271107010 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa59800d130 con 0x7fa58c0bc450 2024-05-14T16:52:50.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.797+0000 7fa5a9864700 1 -- 172.21.15.2:0/4180125210 --> [v2:172.21.15.151:6816/4271107010,v1:172.21.15.151:6817/4271107010] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa5a4064050 con 0x7fa58c0bc450 2024-05-14T16:52:50.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.798+0000 7fa5a8862700 1 -- 172.21.15.2:0/4180125210 <== osd.8 v2:172.21.15.151:6816/4271107010 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa5a4064050 con 0x7fa58c0bc450 2024-05-14T16:52:50.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.798+0000 7fa5a9864700 1 -- 172.21.15.2:0/4180125210 >> [v2:172.21.15.151:6816/4271107010,v1:172.21.15.151:6817/4271107010] conn(0x7fa58c0bc450 msgr2=0x7fa58c0be8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:50.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.798+0000 7fa5a9864700 1 --2- 172.21.15.2:0/4180125210 >> [v2:172.21.15.151:6816/4271107010,v1:172.21.15.151:6817/4271107010] conn(0x7fa58c0bc450 0x7fa58c0be8f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:50.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.798+0000 7fa5a9864700 1 -- 172.21.15.2:0/4180125210 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fa58c0b2910 msgr2=0x7fa58c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:50.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.798+0000 7fa5a9864700 1 --2- 172.21.15.2:0/4180125210 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fa58c0b2910 0x7fa58c0b4dd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa594009550 tx=0x7fa5940091b0 comp rx=0 tx=0).stop 2024-05-14T16:52:50.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.798+0000 7fa5a9864700 1 -- 172.21.15.2:0/4180125210 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5a406c5d0 msgr2=0x7fa5a4102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:50.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.798+0000 7fa5a9864700 1 --2- 172.21.15.2:0/4180125210 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa5a406c5d0 0x7fa5a4102e30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa59800d980 tx=0x7fa59800dc90 comp rx=0 tx=0).stop 2024-05-14T16:52:50.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.798+0000 7fa5a37fe700 1 -- 172.21.15.2:0/4180125210 reap_dead start 2024-05-14T16:52:50.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.798+0000 7fa5a9864700 1 -- 172.21.15.2:0/4180125210 shutdown_connections 2024-05-14T16:52:50.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.798+0000 7fa5a9864700 1 -- 172.21.15.2:0/4180125210 >> 172.21.15.2:0/4180125210 conn(0x7fa5a40793f0 msgr2=0x7fa5a4101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:50.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.799+0000 7fa5a9864700 1 -- 172.21.15.2:0/4180125210 shutdown_connections 2024-05-14T16:52:50.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:50.799+0000 7fa5a9864700 1 -- 172.21.15.2:0/4180125210 wait complete. 2024-05-14T16:52:51.460 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:51 smithi002 ceph-mon[31538]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T16:52:51.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:51 smithi151 ceph-mon[36066]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T16:52:51.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:51 smithi123 ceph-mon[35971]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T16:52:52.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.449+0000 7f937189f700 1 -- 172.21.15.2:0/810532350 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f936c109630 msgr2=0x7f936c109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:52.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.449+0000 7f937189f700 1 --2- 172.21.15.2:0/810532350 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f936c109630 0x7f936c109a10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f93540099c0 tx=0x7f9354009cd0 comp rx=0 tx=0).stop 2024-05-14T16:52:52.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.449+0000 7f937189f700 1 -- 172.21.15.2:0/810532350 shutdown_connections 2024-05-14T16:52:52.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.449+0000 7f937189f700 1 --2- 172.21.15.2:0/810532350 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f936c06c6d0 0x7f936c110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:52.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.449+0000 7f937189f700 1 --2- 172.21.15.2:0/810532350 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f936c109630 0x7f936c109a10 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:52.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.449+0000 7f937189f700 1 --2- 172.21.15.2:0/810532350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f936c06bd10 0x7f936c06c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:52.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.449+0000 7f937189f700 1 -- 172.21.15.2:0/810532350 >> 172.21.15.2:0/810532350 conn(0x7f936c0794e0 msgr2=0x7f936c0798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:52.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.449+0000 7f937189f700 1 -- 172.21.15.2:0/810532350 shutdown_connections 2024-05-14T16:52:52.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.449+0000 7f937189f700 1 -- 172.21.15.2:0/810532350 wait complete. 2024-05-14T16:52:52.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.449+0000 7f937189f700 1 Processor -- start 2024-05-14T16:52:52.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.449+0000 7f937189f700 1 -- start start 2024-05-14T16:52:52.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.450+0000 7f937189f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f936c06bd10 0x7f936c19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:52.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.450+0000 7f937189f700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f936c06c6d0 0x7f936c19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:52.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.450+0000 7f937189f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f936c109630 0x7f936c199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:52.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.450+0000 7f937189f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f936c113750 con 0x7f936c109630 2024-05-14T16:52:52.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.450+0000 7f937189f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f936c1135d0 con 0x7f936c06bd10 2024-05-14T16:52:52.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.450+0000 7f937189f700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f936c1138d0 con 0x7f936c06c6d0 2024-05-14T16:52:52.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.450+0000 7f936b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f936c109630 0x7f936c199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:52.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.450+0000 7f936b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f936c109630 0x7f936c199910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56318/0 (socket says 172.21.15.2:56318) 2024-05-14T16:52:52.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.450+0000 7f936b7fe700 1 -- 172.21.15.2:0/4112848280 learned_addr learned my addr 172.21.15.2:0/4112848280 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:52:52.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.450+0000 7f936affd700 1 --2- 172.21.15.2:0/4112848280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f936c06bd10 0x7f936c19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:52.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.450+0000 7f936a7fc700 1 --2- 172.21.15.2:0/4112848280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f936c06c6d0 0x7f936c19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:52.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.450+0000 7f936b7fe700 1 -- 172.21.15.2:0/4112848280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f936c06c6d0 msgr2=0x7f936c19fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:52.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.450+0000 7f936b7fe700 1 --2- 172.21.15.2:0/4112848280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f936c06c6d0 0x7f936c19fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:52.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.450+0000 7f936b7fe700 1 -- 172.21.15.2:0/4112848280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f936c06bd10 msgr2=0x7f936c19f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:52.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.450+0000 7f936b7fe700 1 --2- 172.21.15.2:0/4112848280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f936c06bd10 0x7f936c19f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:52.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.450+0000 7f936b7fe700 1 -- 172.21.15.2:0/4112848280 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f936c19a070 con 0x7f936c109630 2024-05-14T16:52:52.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.450+0000 7f936b7fe700 1 --2- 172.21.15.2:0/4112848280 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f936c109630 0x7f936c199910 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9364007ee0 tx=0x7f936400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:52.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.451+0000 7f937089d700 1 -- 172.21.15.2:0/4112848280 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936400dbd0 con 0x7f936c109630 2024-05-14T16:52:52.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.451+0000 7f937189f700 1 -- 172.21.15.2:0/4112848280 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f936c19a360 con 0x7f936c109630 2024-05-14T16:52:52.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.451+0000 7f937089d700 1 -- 172.21.15.2:0/4112848280 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f936400dd70 con 0x7f936c109630 2024-05-14T16:52:52.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.451+0000 7f937089d700 1 -- 172.21.15.2:0/4112848280 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9364005990 con 0x7f936c109630 2024-05-14T16:52:52.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.451+0000 7f937189f700 1 -- 172.21.15.2:0/4112848280 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f936c1a6620 con 0x7f936c109630 2024-05-14T16:52:52.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.451+0000 7f937189f700 1 -- 172.21.15.2:0/4112848280 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f936c063cd0 con 0x7f936c109630 2024-05-14T16:52:52.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.453+0000 7f937089d700 1 -- 172.21.15.2:0/4112848280 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f936401a020 con 0x7f936c109630 2024-05-14T16:52:52.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.454+0000 7f937089d700 1 --2- 172.21.15.2:0/4112848280 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f93580b2910 0x7f93580b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:52.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.454+0000 7f936affd700 1 --2- 172.21.15.2:0/4112848280 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f93580b2910 0x7f93580b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:52.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.454+0000 7f937089d700 1 -- 172.21.15.2:0/4112848280 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f93640e2290 con 0x7f936c109630 2024-05-14T16:52:52.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.454+0000 7f937089d700 1 --2- 172.21.15.2:0/4112848280 >> [v2:172.21.15.2:6810/3451424073,v1:172.21.15.2:6811/3451424073] conn(0x7f93580bc450 0x7f93580be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:52.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.454+0000 7f937089d700 1 -- 172.21.15.2:0/4112848280 --> [v2:172.21.15.2:6810/3451424073,v1:172.21.15.2:6811/3451424073] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9364012ec0 con 0x7f93580bc450 2024-05-14T16:52:52.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.454+0000 7f937089d700 1 -- 172.21.15.2:0/4112848280 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f93640e2640 con 0x7f936c109630 2024-05-14T16:52:52.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.454+0000 7f936a7fc700 1 --2- 172.21.15.2:0/4112848280 >> [v2:172.21.15.2:6810/3451424073,v1:172.21.15.2:6811/3451424073] conn(0x7f93580bc450 0x7f93580be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:52.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.454+0000 7f936affd700 1 --2- 172.21.15.2:0/4112848280 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f93580b2910 0x7f93580b4dd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f936c19aa50 tx=0x7f9354005c40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:52.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.455+0000 7f936a7fc700 1 --2- 172.21.15.2:0/4112848280 >> [v2:172.21.15.2:6810/3451424073,v1:172.21.15.2:6811/3451424073] conn(0x7f93580bc450 0x7f93580be8f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:52.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.455+0000 7f937089d700 1 -- 172.21.15.2:0/4112848280 <== osd.3 v2:172.21.15.2:6810/3451424073 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9364012ec0 con 0x7f93580bc450 2024-05-14T16:52:52.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.473+0000 7f937189f700 1 -- 172.21.15.2:0/4112848280 --> [v2:172.21.15.2:6810/3451424073,v1:172.21.15.2:6811/3451424073] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f936c0640d0 con 0x7f93580bc450 2024-05-14T16:52:52.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.473+0000 7f937089d700 1 -- 172.21.15.2:0/4112848280 <== osd.3 v2:172.21.15.2:6810/3451424073 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f936c0640d0 con 0x7f93580bc450 2024-05-14T16:52:52.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.473+0000 7f937189f700 1 -- 172.21.15.2:0/4112848280 >> [v2:172.21.15.2:6810/3451424073,v1:172.21.15.2:6811/3451424073] conn(0x7f93580bc450 msgr2=0x7f93580be8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:52.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.473+0000 7f937189f700 1 --2- 172.21.15.2:0/4112848280 >> [v2:172.21.15.2:6810/3451424073,v1:172.21.15.2:6811/3451424073] conn(0x7f93580bc450 0x7f93580be8f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:52.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.474+0000 7f937189f700 1 -- 172.21.15.2:0/4112848280 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f93580b2910 msgr2=0x7f93580b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:52.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.474+0000 7f937189f700 1 --2- 172.21.15.2:0/4112848280 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f93580b2910 0x7f93580b4dd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f936c19aa50 tx=0x7f9354005c40 comp rx=0 tx=0).stop 2024-05-14T16:52:52.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.474+0000 7f937189f700 1 -- 172.21.15.2:0/4112848280 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f936c109630 msgr2=0x7f936c199910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:52.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.474+0000 7f937189f700 1 --2- 172.21.15.2:0/4112848280 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f936c109630 0x7f936c199910 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9364007ee0 tx=0x7f936400d3b0 comp rx=0 tx=0).stop 2024-05-14T16:52:52.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.474+0000 7f936b7fe700 1 -- 172.21.15.2:0/4112848280 reap_dead start 2024-05-14T16:52:52.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.474+0000 7f937189f700 1 -- 172.21.15.2:0/4112848280 shutdown_connections 2024-05-14T16:52:52.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.474+0000 7f937189f700 1 -- 172.21.15.2:0/4112848280 >> 172.21.15.2:0/4112848280 conn(0x7f936c0794e0 msgr2=0x7f936c100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:52.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.474+0000 7f937189f700 1 -- 172.21.15.2:0/4112848280 shutdown_connections 2024-05-14T16:52:52.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.474+0000 7f937189f700 1 -- 172.21.15.2:0/4112848280 wait complete. 2024-05-14T16:52:52.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.343+0000 7f0815b18700 1 -- 172.21.15.2:0/171917557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0810108920 msgr2=0x7f08101104a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:52.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.343+0000 7f0815b18700 1 --2- 172.21.15.2:0/171917557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0810108920 0x7f08101104a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f080800b0d0 tx=0x7f080800b3e0 comp rx=0 tx=0).stop 2024-05-14T16:52:52.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.343+0000 7f0815b18700 1 -- 172.21.15.2:0/171917557 shutdown_connections 2024-05-14T16:52:52.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.343+0000 7f0815b18700 1 --2- 172.21.15.2:0/171917557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0810108920 0x7f08101104a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:52.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.343+0000 7f0815b18700 1 --2- 172.21.15.2:0/171917557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0810103850 0x7f0810103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:52.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.343+0000 7f0815b18700 1 --2- 172.21.15.2:0/171917557 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0810104200 0x7f08101081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:52.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.343+0000 7f0815b18700 1 -- 172.21.15.2:0/171917557 >> 172.21.15.2:0/171917557 conn(0x7f081007a5d0 msgr2=0x7f081007a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:52.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.343+0000 7f0815b18700 1 -- 172.21.15.2:0/171917557 shutdown_connections 2024-05-14T16:52:52.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.343+0000 7f0815b18700 1 -- 172.21.15.2:0/171917557 wait complete. 2024-05-14T16:52:52.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.344+0000 7f0815b18700 1 Processor -- start 2024-05-14T16:52:52.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.344+0000 7f0815b18700 1 -- start start 2024-05-14T16:52:52.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.345+0000 7f0815b18700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0810103850 0x7f081019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:52.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.345+0000 7f0815b18700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0810104200 0x7f081019fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:52.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.345+0000 7f0815b18700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0810108920 0x7f0810199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:52.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.345+0000 7f0815b18700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f08101136f0 con 0x7f0810103850 2024-05-14T16:52:52.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.345+0000 7f0815b18700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0810113570 con 0x7f0810108920 2024-05-14T16:52:52.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.345+0000 7f0815b18700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0810113870 con 0x7f0810104200 2024-05-14T16:52:52.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.345+0000 7f080f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0810103850 0x7f081019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:52.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.345+0000 7f080f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0810103850 0x7f081019f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56292/0 (socket says 172.21.15.2:56292) 2024-05-14T16:52:52.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.345+0000 7f080f7fe700 1 -- 172.21.15.2:0/531545356 learned_addr learned my addr 172.21.15.2:0/531545356 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:52:52.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.345+0000 7f080effd700 1 --2- 172.21.15.2:0/531545356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0810104200 0x7f081019fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:52.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.345+0000 7f080ffff700 1 --2- 172.21.15.2:0/531545356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0810108920 0x7f0810199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:52.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.345+0000 7f080f7fe700 1 -- 172.21.15.2:0/531545356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0810104200 msgr2=0x7f081019fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:52.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.345+0000 7f080f7fe700 1 --2- 172.21.15.2:0/531545356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0810104200 0x7f081019fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:52.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.345+0000 7f080f7fe700 1 -- 172.21.15.2:0/531545356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0810108920 msgr2=0x7f0810199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:52.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.345+0000 7f080f7fe700 1 --2- 172.21.15.2:0/531545356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0810108920 0x7f0810199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:52.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.345+0000 7f080f7fe700 1 -- 172.21.15.2:0/531545356 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f081019a070 con 0x7f0810103850 2024-05-14T16:52:52.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.345+0000 7f080ffff700 1 --2- 172.21.15.2:0/531545356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0810108920 0x7f0810199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:52:52.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.345+0000 7f080f7fe700 1 --2- 172.21.15.2:0/531545356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0810103850 0x7f081019f750 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f080000c920 tx=0x7f080000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:52.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.346+0000 7f080cff9700 1 -- 172.21.15.2:0/531545356 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0800007b00 con 0x7f0810103850 2024-05-14T16:52:52.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.346+0000 7f0815b18700 1 -- 172.21.15.2:0/531545356 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f081019a360 con 0x7f0810103850 2024-05-14T16:52:52.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.346+0000 7f080cff9700 1 -- 172.21.15.2:0/531545356 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0800004ce0 con 0x7f0810103850 2024-05-14T16:52:52.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.346+0000 7f080cff9700 1 -- 172.21.15.2:0/531545356 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0800005550 con 0x7f0810103850 2024-05-14T16:52:52.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.346+0000 7f0815b18700 1 -- 172.21.15.2:0/531545356 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08101a6620 con 0x7f0810103850 2024-05-14T16:52:52.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.347+0000 7f0815b18700 1 -- 172.21.15.2:0/531545356 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0810063cd0 con 0x7f0810103850 2024-05-14T16:52:52.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.348+0000 7f080cff9700 1 -- 172.21.15.2:0/531545356 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f0800007ca0 con 0x7f0810103850 2024-05-14T16:52:52.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.349+0000 7f080cff9700 1 --2- 172.21.15.2:0/531545356 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f07fc0b2910 0x7f07fc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:52.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.349+0000 7f080effd700 1 --2- 172.21.15.2:0/531545356 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f07fc0b2910 0x7f07fc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:52.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.349+0000 7f080cff9700 1 -- 172.21.15.2:0/531545356 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f08000e24e0 con 0x7f0810103850 2024-05-14T16:52:52.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.349+0000 7f080cff9700 1 --2- 172.21.15.2:0/531545356 >> [v2:172.21.15.2:6826/3479431021,v1:172.21.15.2:6827/3479431021] conn(0x7f07fc0bc450 0x7f07fc0be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:52.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.349+0000 7f080cff9700 1 -- 172.21.15.2:0/531545356 --> [v2:172.21.15.2:6826/3479431021,v1:172.21.15.2:6827/3479431021] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0800012ec0 con 0x7f07fc0bc450 2024-05-14T16:52:52.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.349+0000 7f080cff9700 1 -- 172.21.15.2:0/531545356 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f08000e2890 con 0x7f0810103850 2024-05-14T16:52:52.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.349+0000 7f080ffff700 1 --2- 172.21.15.2:0/531545356 >> [v2:172.21.15.2:6826/3479431021,v1:172.21.15.2:6827/3479431021] conn(0x7f07fc0bc450 0x7f07fc0be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:52.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.349+0000 7f080effd700 1 --2- 172.21.15.2:0/531545356 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f07fc0b2910 0x7f07fc0b4dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0810103110 tx=0x7f07f80073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:52.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.350+0000 7f080ffff700 1 --2- 172.21.15.2:0/531545356 >> [v2:172.21.15.2:6826/3479431021,v1:172.21.15.2:6827/3479431021] conn(0x7f07fc0bc450 0x7f07fc0be8f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:52.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.350+0000 7f080cff9700 1 -- 172.21.15.2:0/531545356 <== osd.9 v2:172.21.15.2:6826/3479431021 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0800012ec0 con 0x7f07fc0bc450 2024-05-14T16:52:52.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.367+0000 7f0815b18700 1 -- 172.21.15.2:0/531545356 --> [v2:172.21.15.2:6826/3479431021,v1:172.21.15.2:6827/3479431021] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f08100640d0 con 0x7f07fc0bc450 2024-05-14T16:52:52.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.368+0000 7f080cff9700 1 -- 172.21.15.2:0/531545356 <== osd.9 v2:172.21.15.2:6826/3479431021 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f08100640d0 con 0x7f07fc0bc450 2024-05-14T16:52:52.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.368+0000 7f0815b18700 1 -- 172.21.15.2:0/531545356 >> [v2:172.21.15.2:6826/3479431021,v1:172.21.15.2:6827/3479431021] conn(0x7f07fc0bc450 msgr2=0x7f07fc0be8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:52.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.368+0000 7f0815b18700 1 --2- 172.21.15.2:0/531545356 >> [v2:172.21.15.2:6826/3479431021,v1:172.21.15.2:6827/3479431021] conn(0x7f07fc0bc450 0x7f07fc0be8f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:52.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.368+0000 7f0815b18700 1 -- 172.21.15.2:0/531545356 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f07fc0b2910 msgr2=0x7f07fc0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:52.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.368+0000 7f0815b18700 1 --2- 172.21.15.2:0/531545356 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f07fc0b2910 0x7f07fc0b4dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0810103110 tx=0x7f07f80073d0 comp rx=0 tx=0).stop 2024-05-14T16:52:52.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.368+0000 7f0815b18700 1 -- 172.21.15.2:0/531545356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0810103850 msgr2=0x7f081019f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:52.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.368+0000 7f0815b18700 1 --2- 172.21.15.2:0/531545356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0810103850 0x7f081019f750 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f080000c920 tx=0x7f080000cc30 comp rx=0 tx=0).stop 2024-05-14T16:52:52.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.368+0000 7f080ffff700 1 -- 172.21.15.2:0/531545356 reap_dead start 2024-05-14T16:52:52.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.368+0000 7f0815b18700 1 -- 172.21.15.2:0/531545356 shutdown_connections 2024-05-14T16:52:52.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.368+0000 7f0815b18700 1 -- 172.21.15.2:0/531545356 >> 172.21.15.2:0/531545356 conn(0x7f081007a5d0 msgr2=0x7f0810101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:52.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.368+0000 7f0815b18700 1 -- 172.21.15.2:0/531545356 shutdown_connections 2024-05-14T16:52:52.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.368+0000 7f0815b18700 1 -- 172.21.15.2:0/531545356 wait complete. 2024-05-14T16:52:52.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.978+0000 7fedccd4c700 1 -- 172.21.15.2:0/3613771061 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedc81095b0 msgr2=0x7fedc8109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:52.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.978+0000 7fedccd4c700 1 --2- 172.21.15.2:0/3613771061 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedc81095b0 0x7fedc8109990 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fedb8009990 tx=0x7fedb8009ca0 comp rx=0 tx=0).stop 2024-05-14T16:52:52.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.978+0000 7fedccd4c700 1 -- 172.21.15.2:0/3613771061 shutdown_connections 2024-05-14T16:52:52.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.978+0000 7fedccd4c700 1 --2- 172.21.15.2:0/3613771061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedc806bbe0 0x7fedc81103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:52.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.978+0000 7fedccd4c700 1 --2- 172.21.15.2:0/3613771061 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedc81095b0 0x7fedc8109990 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:52.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.978+0000 7fedccd4c700 1 --2- 172.21.15.2:0/3613771061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fedc806b220 0x7fedc806b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:52.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.978+0000 7fedccd4c700 1 -- 172.21.15.2:0/3613771061 >> 172.21.15.2:0/3613771061 conn(0x7fedc80793f0 msgr2=0x7fedc8079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:52.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.978+0000 7fedccd4c700 1 -- 172.21.15.2:0/3613771061 shutdown_connections 2024-05-14T16:52:52.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.979+0000 7fedccd4c700 1 -- 172.21.15.2:0/3613771061 wait complete. 2024-05-14T16:52:52.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.979+0000 7fedccd4c700 1 Processor -- start 2024-05-14T16:52:52.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.979+0000 7fedccd4c700 1 -- start start 2024-05-14T16:52:52.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.980+0000 7fedccd4c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fedc806b220 0x7fedc819be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:52.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.980+0000 7fedccd4c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedc806bbe0 0x7fedc819c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:52.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.980+0000 7fedccd4c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedc81095b0 0x7fedc81a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:52.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.980+0000 7fedccd4c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fedc81136e0 con 0x7fedc81095b0 2024-05-14T16:52:52.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.980+0000 7fedccd4c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fedc8113560 con 0x7fedc806bbe0 2024-05-14T16:52:52.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.980+0000 7fedccd4c700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fedc8113860 con 0x7fedc806b220 2024-05-14T16:52:52.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.980+0000 7fedc6d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedc81095b0 0x7fedc81a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:52.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.980+0000 7fedc6d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedc81095b0 0x7fedc81a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56342/0 (socket says 172.21.15.2:56342) 2024-05-14T16:52:52.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.980+0000 7fedc6d9d700 1 -- 172.21.15.2:0/2599984114 learned_addr learned my addr 172.21.15.2:0/2599984114 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:52:52.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.980+0000 7fedc5d9b700 1 --2- 172.21.15.2:0/2599984114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedc806bbe0 0x7fedc819c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:52.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.980+0000 7fedc659c700 1 --2- 172.21.15.2:0/2599984114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fedc806b220 0x7fedc819be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:52.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.980+0000 7fedc6d9d700 1 -- 172.21.15.2:0/2599984114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fedc806b220 msgr2=0x7fedc819be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:52.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.980+0000 7fedc6d9d700 1 --2- 172.21.15.2:0/2599984114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fedc806b220 0x7fedc819be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:52.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.980+0000 7fedc6d9d700 1 -- 172.21.15.2:0/2599984114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedc806bbe0 msgr2=0x7fedc819c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:52.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.980+0000 7fedc6d9d700 1 --2- 172.21.15.2:0/2599984114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedc806bbe0 0x7fedc819c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:52.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.980+0000 7fedc6d9d700 1 -- 172.21.15.2:0/2599984114 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc81a0e20 con 0x7fedc81095b0 2024-05-14T16:52:52.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.980+0000 7fedc659c700 1 --2- 172.21.15.2:0/2599984114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fedc806b220 0x7fedc819be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:52:52.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.980+0000 7fedc6d9d700 1 --2- 172.21.15.2:0/2599984114 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedc81095b0 0x7fedc81a06e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fedc000ef30 tx=0x7fedc000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:52.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.980+0000 7fedb77fe700 1 -- 172.21.15.2:0/2599984114 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc00092e0 con 0x7fedc81095b0 2024-05-14T16:52:52.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.980+0000 7fedb77fe700 1 -- 172.21.15.2:0/2599984114 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedc0004bb0 con 0x7fedc81095b0 2024-05-14T16:52:52.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.980+0000 7fedccd4c700 1 -- 172.21.15.2:0/2599984114 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedc81a10b0 con 0x7fedc81095b0 2024-05-14T16:52:52.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.980+0000 7fedb77fe700 1 -- 172.21.15.2:0/2599984114 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc0004280 con 0x7fedc81095b0 2024-05-14T16:52:52.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.980+0000 7fedccd4c700 1 -- 172.21.15.2:0/2599984114 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedc81a8990 con 0x7fedc81095b0 2024-05-14T16:52:52.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.981+0000 7fedccd4c700 1 -- 172.21.15.2:0/2599984114 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fedc810f180 con 0x7fedc81095b0 2024-05-14T16:52:52.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.983+0000 7fedb77fe700 1 -- 172.21.15.2:0/2599984114 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fedc001b030 con 0x7fedc81095b0 2024-05-14T16:52:52.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.983+0000 7fedb77fe700 1 --2- 172.21.15.2:0/2599984114 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fedb00b2910 0x7fedb00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:52.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.984+0000 7fedc659c700 1 --2- 172.21.15.2:0/2599984114 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fedb00b2910 0x7fedb00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:52.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.984+0000 7fedb77fe700 1 -- 172.21.15.2:0/2599984114 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7fedc00e2810 con 0x7fedc81095b0 2024-05-14T16:52:52.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.984+0000 7fedb77fe700 1 --2- 172.21.15.2:0/2599984114 >> [v2:172.21.15.151:6824/4129194698,v1:172.21.15.151:6825/4129194698] conn(0x7fedb00bc450 0x7fedb00be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:52.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.984+0000 7fedb77fe700 1 -- 172.21.15.2:0/2599984114 --> [v2:172.21.15.151:6824/4129194698,v1:172.21.15.151:6825/4129194698] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fedb00befc0 con 0x7fedb00bc450 2024-05-14T16:52:52.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.984+0000 7fedb77fe700 1 -- 172.21.15.2:0/2599984114 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fedc00e2c60 con 0x7fedc81095b0 2024-05-14T16:52:52.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.984+0000 7fedc659c700 1 --2- 172.21.15.2:0/2599984114 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fedb00b2910 0x7fedb00b4dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fedb800b2e0 tx=0x7fedb801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:52.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.984+0000 7fedc5d9b700 1 --2- 172.21.15.2:0/2599984114 >> [v2:172.21.15.151:6824/4129194698,v1:172.21.15.151:6825/4129194698] conn(0x7fedb00bc450 0x7fedb00be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:52.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.985+0000 7fedc5d9b700 1 --2- 172.21.15.2:0/2599984114 >> [v2:172.21.15.151:6824/4129194698,v1:172.21.15.151:6825/4129194698] conn(0x7fedb00bc450 0x7fedb00be8f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:52.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:52.986+0000 7fedb77fe700 1 -- 172.21.15.2:0/2599984114 <== osd.11 v2:172.21.15.151:6824/4129194698 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fedb00befc0 con 0x7fedb00bc450 2024-05-14T16:52:53.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.004+0000 7fedccd4c700 1 -- 172.21.15.2:0/2599984114 --> [v2:172.21.15.151:6824/4129194698,v1:172.21.15.151:6825/4129194698] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fedc8104c70 con 0x7fedb00bc450 2024-05-14T16:52:53.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.006+0000 7fedb77fe700 1 -- 172.21.15.2:0/2599984114 <== osd.11 v2:172.21.15.151:6824/4129194698 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fedc8104c70 con 0x7fedb00bc450 2024-05-14T16:52:53.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.007+0000 7fedccd4c700 1 -- 172.21.15.2:0/2599984114 >> [v2:172.21.15.151:6824/4129194698,v1:172.21.15.151:6825/4129194698] conn(0x7fedb00bc450 msgr2=0x7fedb00be8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:53.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.007+0000 7fedccd4c700 1 --2- 172.21.15.2:0/2599984114 >> [v2:172.21.15.151:6824/4129194698,v1:172.21.15.151:6825/4129194698] conn(0x7fedb00bc450 0x7fedb00be8f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:53.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.007+0000 7fedccd4c700 1 -- 172.21.15.2:0/2599984114 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fedb00b2910 msgr2=0x7fedb00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:53.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.007+0000 7fedccd4c700 1 --2- 172.21.15.2:0/2599984114 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fedb00b2910 0x7fedb00b4dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fedb800b2e0 tx=0x7fedb801a040 comp rx=0 tx=0).stop 2024-05-14T16:52:53.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.007+0000 7fedccd4c700 1 -- 172.21.15.2:0/2599984114 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedc81095b0 msgr2=0x7fedc81a06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:53.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.007+0000 7fedccd4c700 1 --2- 172.21.15.2:0/2599984114 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fedc81095b0 0x7fedc81a06e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fedc000ef30 tx=0x7fedc000c3a0 comp rx=0 tx=0).stop 2024-05-14T16:52:53.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.007+0000 7fedc6d9d700 1 -- 172.21.15.2:0/2599984114 reap_dead start 2024-05-14T16:52:53.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.007+0000 7fedccd4c700 1 -- 172.21.15.2:0/2599984114 shutdown_connections 2024-05-14T16:52:53.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.007+0000 7fedccd4c700 1 -- 172.21.15.2:0/2599984114 >> 172.21.15.2:0/2599984114 conn(0x7fedc80793f0 msgr2=0x7fedc810e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:53.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.007+0000 7fedccd4c700 1 -- 172.21.15.2:0/2599984114 shutdown_connections 2024-05-14T16:52:53.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.007+0000 7fedccd4c700 1 -- 172.21.15.2:0/2599984114 wait complete. 2024-05-14T16:52:53.010 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:52:53.462 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:53 smithi123 ceph-mon[35971]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:52:53.484 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:53 smithi002 ceph-mon[31538]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:52:53.512 INFO:teuthology.orchestra.run.smithi002.stdout:55834574930 2024-05-14T16:52:53.512 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-14T16:52:53.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:53 smithi151 ceph-mon[36066]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:52:53.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.844+0000 7f2e602cb700 1 -- 172.21.15.2:0/2858783709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e5806bab0 msgr2=0x7f2e5806bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:53.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.844+0000 7f2e602cb700 1 --2- 172.21.15.2:0/2858783709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e5806bab0 0x7f2e5806bf30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2e48009990 tx=0x7f2e48009ca0 comp rx=0 tx=0).stop 2024-05-14T16:52:53.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.844+0000 7f2e602cb700 1 -- 172.21.15.2:0/2858783709 shutdown_connections 2024-05-14T16:52:53.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.844+0000 7f2e602cb700 1 --2- 172.21.15.2:0/2858783709 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2e5806c470 0x7f2e581103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:53.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.844+0000 7f2e602cb700 1 --2- 172.21.15.2:0/2858783709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e581095a0 0x7f2e58109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:53.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.844+0000 7f2e602cb700 1 --2- 172.21.15.2:0/2858783709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e5806bab0 0x7f2e5806bf30 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:53.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.844+0000 7f2e602cb700 1 -- 172.21.15.2:0/2858783709 >> 172.21.15.2:0/2858783709 conn(0x7f2e580793b0 msgr2=0x7f2e580797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:53.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.844+0000 7f2e602cb700 1 -- 172.21.15.2:0/2858783709 shutdown_connections 2024-05-14T16:52:53.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.844+0000 7f2e602cb700 1 -- 172.21.15.2:0/2858783709 wait complete. 2024-05-14T16:52:53.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.845+0000 7f2e602cb700 1 Processor -- start 2024-05-14T16:52:53.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.845+0000 7f2e602cb700 1 -- start start 2024-05-14T16:52:53.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.845+0000 7f2e602cb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e5806bab0 0x7f2e5819f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:53.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.845+0000 7f2e602cb700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2e5806c470 0x7f2e5819fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:53.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.845+0000 7f2e602cb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e581095a0 0x7f2e58199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:53.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.845+0000 7f2e602cb700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2e58113730 con 0x7f2e581095a0 2024-05-14T16:52:53.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.845+0000 7f2e602cb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2e581135b0 con 0x7f2e5806bab0 2024-05-14T16:52:53.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.845+0000 7f2e602cb700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2e581138b0 con 0x7f2e5806c470 2024-05-14T16:52:53.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.845+0000 7f2e5d866700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2e5806c470 0x7f2e5819fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:53.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.845+0000 7f2e5d866700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2e5806c470 0x7f2e5819fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:52576/0 (socket says 172.21.15.2:52576) 2024-05-14T16:52:53.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.845+0000 7f2e5d866700 1 -- 172.21.15.2:0/2701622790 learned_addr learned my addr 172.21.15.2:0/2701622790 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:52:53.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.845+0000 7f2e5e868700 1 --2- 172.21.15.2:0/2701622790 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e581095a0 0x7f2e58199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:53.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.845+0000 7f2e5e067700 1 --2- 172.21.15.2:0/2701622790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e5806bab0 0x7f2e5819f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:53.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.846+0000 7f2e5e868700 1 -- 172.21.15.2:0/2701622790 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2e5806c470 msgr2=0x7f2e5819fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:53.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.846+0000 7f2e5e868700 1 --2- 172.21.15.2:0/2701622790 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2e5806c470 0x7f2e5819fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:53.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.846+0000 7f2e5e868700 1 -- 172.21.15.2:0/2701622790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e5806bab0 msgr2=0x7f2e5819f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:53.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.846+0000 7f2e5e868700 1 --2- 172.21.15.2:0/2701622790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e5806bab0 0x7f2e5819f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:53.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.846+0000 7f2e5e868700 1 -- 172.21.15.2:0/2701622790 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e5819a230 con 0x7f2e581095a0 2024-05-14T16:52:53.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.846+0000 7f2e5d866700 1 --2- 172.21.15.2:0/2701622790 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2e5806c470 0x7f2e5819fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:52:53.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.846+0000 7f2e5e868700 1 --2- 172.21.15.2:0/2701622790 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e581095a0 0x7f2e58199ad0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2e50007f10 tx=0x7f2e5000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:53.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.846+0000 7f2e4f7fe700 1 -- 172.21.15.2:0/2701622790 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e50004db0 con 0x7f2e581095a0 2024-05-14T16:52:53.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.846+0000 7f2e602cb700 1 -- 172.21.15.2:0/2701622790 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e5819a520 con 0x7f2e581095a0 2024-05-14T16:52:53.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.846+0000 7f2e4f7fe700 1 -- 172.21.15.2:0/2701622790 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e50009e20 con 0x7f2e581095a0 2024-05-14T16:52:53.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.846+0000 7f2e4f7fe700 1 -- 172.21.15.2:0/2701622790 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e50015400 con 0x7f2e581095a0 2024-05-14T16:52:53.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.846+0000 7f2e602cb700 1 -- 172.21.15.2:0/2701622790 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e581a67e0 con 0x7f2e581095a0 2024-05-14T16:52:53.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.847+0000 7f2e602cb700 1 -- 172.21.15.2:0/2701622790 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2e58063c50 con 0x7f2e581095a0 2024-05-14T16:52:53.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.849+0000 7f2e4f7fe700 1 -- 172.21.15.2:0/2701622790 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f2e50005260 con 0x7f2e581095a0 2024-05-14T16:52:53.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.849+0000 7f2e4f7fe700 1 --2- 172.21.15.2:0/2701622790 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f2e440b2910 0x7f2e440b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:53.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.849+0000 7f2e5e067700 1 --2- 172.21.15.2:0/2701622790 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f2e440b2910 0x7f2e440b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:53.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.849+0000 7f2e4f7fe700 1 -- 172.21.15.2:0/2701622790 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f2e500e2140 con 0x7f2e581095a0 2024-05-14T16:52:53.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.850+0000 7f2e4f7fe700 1 --2- 172.21.15.2:0/2701622790 >> [v2:172.21.15.2:6818/2711893600,v1:172.21.15.2:6819/2711893600] conn(0x7f2e440bc450 0x7f2e440be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:53.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.850+0000 7f2e4f7fe700 1 -- 172.21.15.2:0/2701622790 --> [v2:172.21.15.2:6818/2711893600,v1:172.21.15.2:6819/2711893600] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2e440befc0 con 0x7f2e440bc450 2024-05-14T16:52:53.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.850+0000 7f2e4f7fe700 1 -- 172.21.15.2:0/2701622790 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f2e500e2590 con 0x7f2e581095a0 2024-05-14T16:52:53.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.850+0000 7f2e5d866700 1 --2- 172.21.15.2:0/2701622790 >> [v2:172.21.15.2:6818/2711893600,v1:172.21.15.2:6819/2711893600] conn(0x7f2e440bc450 0x7f2e440be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:53.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.850+0000 7f2e5e067700 1 --2- 172.21.15.2:0/2701622790 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f2e440b2910 0x7f2e440b4dd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2e54005950 tx=0x7f2e5400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:53.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.850+0000 7f2e5d866700 1 --2- 172.21.15.2:0/2701622790 >> [v2:172.21.15.2:6818/2711893600,v1:172.21.15.2:6819/2711893600] conn(0x7f2e440bc450 0x7f2e440be8f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:53.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.851+0000 7f2e4f7fe700 1 -- 172.21.15.2:0/2701622790 <== osd.6 v2:172.21.15.2:6818/2711893600 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2e440befc0 con 0x7f2e440bc450 2024-05-14T16:52:53.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.868+0000 7f2e602cb700 1 -- 172.21.15.2:0/2701622790 --> [v2:172.21.15.2:6818/2711893600,v1:172.21.15.2:6819/2711893600] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2e580640d0 con 0x7f2e440bc450 2024-05-14T16:52:53.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.868+0000 7f2e4f7fe700 1 -- 172.21.15.2:0/2701622790 <== osd.6 v2:172.21.15.2:6818/2711893600 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2e580640d0 con 0x7f2e440bc450 2024-05-14T16:52:53.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.868+0000 7f2e602cb700 1 -- 172.21.15.2:0/2701622790 >> [v2:172.21.15.2:6818/2711893600,v1:172.21.15.2:6819/2711893600] conn(0x7f2e440bc450 msgr2=0x7f2e440be8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:53.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.868+0000 7f2e602cb700 1 --2- 172.21.15.2:0/2701622790 >> [v2:172.21.15.2:6818/2711893600,v1:172.21.15.2:6819/2711893600] conn(0x7f2e440bc450 0x7f2e440be8f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:53.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.868+0000 7f2e602cb700 1 -- 172.21.15.2:0/2701622790 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f2e440b2910 msgr2=0x7f2e440b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:53.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.868+0000 7f2e602cb700 1 --2- 172.21.15.2:0/2701622790 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f2e440b2910 0x7f2e440b4dd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2e54005950 tx=0x7f2e5400b410 comp rx=0 tx=0).stop 2024-05-14T16:52:53.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.868+0000 7f2e602cb700 1 -- 172.21.15.2:0/2701622790 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e581095a0 msgr2=0x7f2e58199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:53.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.868+0000 7f2e602cb700 1 --2- 172.21.15.2:0/2701622790 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e581095a0 0x7f2e58199ad0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2e50007f10 tx=0x7f2e5000d3b0 comp rx=0 tx=0).stop 2024-05-14T16:52:53.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.868+0000 7f2e5e868700 1 -- 172.21.15.2:0/2701622790 reap_dead start 2024-05-14T16:52:53.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.868+0000 7f2e602cb700 1 -- 172.21.15.2:0/2701622790 shutdown_connections 2024-05-14T16:52:53.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.868+0000 7f2e602cb700 1 -- 172.21.15.2:0/2701622790 >> 172.21.15.2:0/2701622790 conn(0x7f2e580793b0 msgr2=0x7f2e58101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:53.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.868+0000 7f2e602cb700 1 -- 172.21.15.2:0/2701622790 shutdown_connections 2024-05-14T16:52:53.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:53.868+0000 7f2e602cb700 1 -- 172.21.15.2:0/2701622790 wait complete. 2024-05-14T16:52:54.282 INFO:teuthology.orchestra.run.smithi002.stdout:227633266735 2024-05-14T16:52:54.283 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-14T16:52:55.007 INFO:teuthology.orchestra.run.smithi002.stdout:253403070506 2024-05-14T16:52:55.007 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-14T16:52:55.243 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:55 smithi002 ceph-mon[31538]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T16:52:55.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:55 smithi151 ceph-mon[36066]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T16:52:55.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:55 smithi123 ceph-mon[35971]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T16:52:55.880 INFO:teuthology.orchestra.run.smithi002.stdout:107374182473 2024-05-14T16:52:55.881 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-14T16:52:56.496 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:52:56.496 INFO:teuthology.orchestra.run.smithi002.stdout:300647710752 2024-05-14T16:52:56.497 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-14T16:52:57.418 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:57 smithi002 ceph-mon[31538]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:52:57.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.527+0000 7fb1fac97700 1 -- 172.21.15.2:0/1642236906 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1f4102e00 msgr2=0x7fb1f410f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:57.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.527+0000 7fb1fac97700 1 --2- 172.21.15.2:0/1642236906 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1f4102e00 0x7fb1f410f2f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec009990 tx=0x7fb1ec009ca0 comp rx=0 tx=0).stop 2024-05-14T16:52:57.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.527+0000 7fb1fac97700 1 -- 172.21.15.2:0/1642236906 shutdown_connections 2024-05-14T16:52:57.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.527+0000 7fb1fac97700 1 --2- 172.21.15.2:0/1642236906 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1f4102e00 0x7fb1f410f2f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:57.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.527+0000 7fb1fac97700 1 --2- 172.21.15.2:0/1642236906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1f4108520 0x7fb1f4108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:57.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.527+0000 7fb1fac97700 1 --2- 172.21.15.2:0/1642236906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1f4102440 0x7fb1f41028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:57.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.527+0000 7fb1fac97700 1 -- 172.21.15.2:0/1642236906 >> 172.21.15.2:0/1642236906 conn(0x7fb1f40793f0 msgr2=0x7fb1f4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:57.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.527+0000 7fb1fac97700 1 -- 172.21.15.2:0/1642236906 shutdown_connections 2024-05-14T16:52:57.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.528+0000 7fb1fac97700 1 -- 172.21.15.2:0/1642236906 wait complete. 2024-05-14T16:52:57.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.528+0000 7fb1fac97700 1 Processor -- start 2024-05-14T16:52:57.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.528+0000 7fb1fac97700 1 -- start start 2024-05-14T16:52:57.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.528+0000 7fb1fac97700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1f4102440 0x7fb1f4104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:57.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.528+0000 7fb1fac97700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1f4102e00 0x7fb1f41045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:57.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.528+0000 7fb1fac97700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1f4108520 0x7fb1f4104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:57.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.528+0000 7fb1fac97700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f4112650 con 0x7fb1f4102440 2024-05-14T16:52:57.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.528+0000 7fb1fac97700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f41124d0 con 0x7fb1f4102e00 2024-05-14T16:52:57.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.528+0000 7fb1fac97700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f41127d0 con 0x7fb1f4108520 2024-05-14T16:52:57.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.529+0000 7fb1f8a33700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1f4102440 0x7fb1f4104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:57.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.529+0000 7fb1f8a33700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1f4102440 0x7fb1f4104080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59696/0 (socket says 172.21.15.2:59696) 2024-05-14T16:52:57.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.529+0000 7fb1f8a33700 1 -- 172.21.15.2:0/2983552309 learned_addr learned my addr 172.21.15.2:0/2983552309 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:52:57.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.529+0000 7fb1f3fff700 1 --2- 172.21.15.2:0/2983552309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1f4102e00 0x7fb1f41045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:57.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.529+0000 7fb1f9234700 1 --2- 172.21.15.2:0/2983552309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1f4108520 0x7fb1f4104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:57.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.529+0000 7fb1f8a33700 1 -- 172.21.15.2:0/2983552309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1f4108520 msgr2=0x7fb1f4104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:57.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.529+0000 7fb1f8a33700 1 --2- 172.21.15.2:0/2983552309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1f4108520 0x7fb1f4104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:57.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.529+0000 7fb1f8a33700 1 -- 172.21.15.2:0/2983552309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1f4102e00 msgr2=0x7fb1f41045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:57.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.529+0000 7fb1f8a33700 1 --2- 172.21.15.2:0/2983552309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1f4102e00 0x7fb1f41045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:57.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.529+0000 7fb1f8a33700 1 -- 172.21.15.2:0/2983552309 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f41a8160 con 0x7fb1f4102440 2024-05-14T16:52:57.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.529+0000 7fb1f8a33700 1 --2- 172.21.15.2:0/2983552309 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1f4102440 0x7fb1f4104080 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc00b530 tx=0x7fb1dc00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:57.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.529+0000 7fb1f1ffb700 1 -- 172.21.15.2:0/2983552309 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc015020 con 0x7fb1f4102440 2024-05-14T16:52:57.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.529+0000 7fb1f1ffb700 1 -- 172.21.15.2:0/2983552309 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1dc011ce0 con 0x7fb1f4102440 2024-05-14T16:52:57.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.529+0000 7fb1fac97700 1 -- 172.21.15.2:0/2983552309 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1f41a8450 con 0x7fb1f4102440 2024-05-14T16:52:57.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.530+0000 7fb1f1ffb700 1 -- 172.21.15.2:0/2983552309 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc010dd0 con 0x7fb1f4102440 2024-05-14T16:52:57.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.530+0000 7fb1fac97700 1 -- 172.21.15.2:0/2983552309 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1f41a8990 con 0x7fb1f4102440 2024-05-14T16:52:57.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.531+0000 7fb1fac97700 1 -- 172.21.15.2:0/2983552309 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb1f4063c50 con 0x7fb1f4102440 2024-05-14T16:52:57.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.532+0000 7fb1f1ffb700 1 -- 172.21.15.2:0/2983552309 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fb1dc018020 con 0x7fb1f4102440 2024-05-14T16:52:57.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.533+0000 7fb1f1ffb700 1 --2- 172.21.15.2:0/2983552309 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fb1e00b2910 0x7fb1e00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:57.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.533+0000 7fb1f3fff700 1 --2- 172.21.15.2:0/2983552309 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fb1e00b2910 0x7fb1e00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:57.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.533+0000 7fb1f1ffb700 1 -- 172.21.15.2:0/2983552309 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7fb1dc0e2110 con 0x7fb1f4102440 2024-05-14T16:52:57.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.533+0000 7fb1f1ffb700 1 --2- 172.21.15.2:0/2983552309 >> [v2:172.21.15.151:6800/2160189636,v1:172.21.15.151:6801/2160189636] conn(0x7fb1e00bc450 0x7fb1e00be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:52:57.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.533+0000 7fb1f1ffb700 1 -- 172.21.15.2:0/2983552309 --> [v2:172.21.15.151:6800/2160189636,v1:172.21.15.151:6801/2160189636] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb1dc01be50 con 0x7fb1e00bc450 2024-05-14T16:52:57.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.533+0000 7fb1f1ffb700 1 -- 172.21.15.2:0/2983552309 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fb1dc0e2560 con 0x7fb1f4102440 2024-05-14T16:52:57.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.533+0000 7fb1f3fff700 1 --2- 172.21.15.2:0/2983552309 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fb1e00b2910 0x7fb1e00b4dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4009550 tx=0x7fb1e40091b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:57.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.534+0000 7fb1f9234700 1 --2- 172.21.15.2:0/2983552309 >> [v2:172.21.15.151:6800/2160189636,v1:172.21.15.151:6801/2160189636] conn(0x7fb1e00bc450 0x7fb1e00be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:52:57.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.534+0000 7fb1f9234700 1 --2- 172.21.15.2:0/2983552309 >> [v2:172.21.15.151:6800/2160189636,v1:172.21.15.151:6801/2160189636] conn(0x7fb1e00bc450 0x7fb1e00be8f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:52:57.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.535+0000 7fb1f1ffb700 1 -- 172.21.15.2:0/2983552309 <== osd.2 v2:172.21.15.151:6800/2160189636 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb1dc01be50 con 0x7fb1e00bc450 2024-05-14T16:52:57.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.553+0000 7fb1fac97700 1 -- 172.21.15.2:0/2983552309 --> [v2:172.21.15.151:6800/2160189636,v1:172.21.15.151:6801/2160189636] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb1f40640d0 con 0x7fb1e00bc450 2024-05-14T16:52:57.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.553+0000 7fb1f1ffb700 1 -- 172.21.15.2:0/2983552309 <== osd.2 v2:172.21.15.151:6800/2160189636 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb1f40640d0 con 0x7fb1e00bc450 2024-05-14T16:52:57.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.554+0000 7fb1fac97700 1 -- 172.21.15.2:0/2983552309 >> [v2:172.21.15.151:6800/2160189636,v1:172.21.15.151:6801/2160189636] conn(0x7fb1e00bc450 msgr2=0x7fb1e00be8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:57.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.554+0000 7fb1fac97700 1 --2- 172.21.15.2:0/2983552309 >> [v2:172.21.15.151:6800/2160189636,v1:172.21.15.151:6801/2160189636] conn(0x7fb1e00bc450 0x7fb1e00be8f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:52:57.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.554+0000 7fb1fac97700 1 -- 172.21.15.2:0/2983552309 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fb1e00b2910 msgr2=0x7fb1e00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:57.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.554+0000 7fb1fac97700 1 --2- 172.21.15.2:0/2983552309 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fb1e00b2910 0x7fb1e00b4dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4009550 tx=0x7fb1e40091b0 comp rx=0 tx=0).stop 2024-05-14T16:52:57.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.554+0000 7fb1fac97700 1 -- 172.21.15.2:0/2983552309 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1f4102440 msgr2=0x7fb1f4104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:52:57.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.554+0000 7fb1fac97700 1 --2- 172.21.15.2:0/2983552309 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1f4102440 0x7fb1f4104080 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc00b530 tx=0x7fb1dc00b840 comp rx=0 tx=0).stop 2024-05-14T16:52:57.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.554+0000 7fb1f9234700 1 -- 172.21.15.2:0/2983552309 reap_dead start 2024-05-14T16:52:57.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.554+0000 7fb1fac97700 1 -- 172.21.15.2:0/2983552309 shutdown_connections 2024-05-14T16:52:57.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.554+0000 7fb1fac97700 1 -- 172.21.15.2:0/2983552309 >> 172.21.15.2:0/2983552309 conn(0x7fb1f40793f0 msgr2=0x7fb1f406b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:52:57.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.554+0000 7fb1fac97700 1 -- 172.21.15.2:0/2983552309 shutdown_connections 2024-05-14T16:52:57.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:52:57.554+0000 7fb1fac97700 1 -- 172.21.15.2:0/2983552309 wait complete. 2024-05-14T16:52:57.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:57 smithi151 ceph-mon[36066]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:52:57.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:57 smithi123 ceph-mon[35971]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:52:59.044 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:52:59.414 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:52:59 smithi002 ceph-mon[31538]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T16:52:59.447 INFO:teuthology.orchestra.run.smithi002.stdout:180388626490 2024-05-14T16:52:59.447 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-14T16:52:59.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:52:59 smithi151 ceph-mon[36066]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T16:52:59.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:52:59 smithi123 ceph-mon[35971]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T16:52:59.873 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:52:59.882 INFO:teuthology.orchestra.run.smithi002.stdout:77309411408 2024-05-14T16:52:59.882 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-14T16:52:59.930 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:52:59.934 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:53:00.610 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:53:01.352 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:01 smithi002 ceph-mon[31538]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T16:53:01.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:01 smithi151 ceph-mon[36066]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T16:53:01.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:01 smithi123 ceph-mon[35971]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T16:53:03.356 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:03 smithi002 ceph-mon[31538]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:53:03.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:03 smithi151 ceph-mon[36066]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:53:03.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:03 smithi123 ceph-mon[35971]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:53:03.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.960+0000 7fa420592700 1 -- 172.21.15.2:0/187326746 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa418101b30 msgr2=0x7fa418101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:03.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.960+0000 7fa420592700 1 --2- 172.21.15.2:0/187326746 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa418101b30 0x7fa418101fb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa4080099c0 tx=0x7fa408009cd0 comp rx=0 tx=0).stop 2024-05-14T16:53:03.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.960+0000 7fa420592700 1 -- 172.21.15.2:0/187326746 shutdown_connections 2024-05-14T16:53:03.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.960+0000 7fa420592700 1 --2- 172.21.15.2:0/187326746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4181024f0 0x7fa4181103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:03.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.960+0000 7fa420592700 1 --2- 172.21.15.2:0/187326746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa418109520 0x7fa418109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:03.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.960+0000 7fa420592700 1 --2- 172.21.15.2:0/187326746 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa418101b30 0x7fa418101fb0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:03.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.960+0000 7fa420592700 1 -- 172.21.15.2:0/187326746 >> 172.21.15.2:0/187326746 conn(0x7fa418078ab0 msgr2=0x7fa4180ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:03.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.960+0000 7fa420592700 1 -- 172.21.15.2:0/187326746 shutdown_connections 2024-05-14T16:53:03.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.960+0000 7fa420592700 1 -- 172.21.15.2:0/187326746 wait complete. 2024-05-14T16:53:03.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.961+0000 7fa420592700 1 Processor -- start 2024-05-14T16:53:03.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.961+0000 7fa420592700 1 -- start start 2024-05-14T16:53:03.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.962+0000 7fa420592700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa418101b30 0x7fa41819f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:03.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.962+0000 7fa420592700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4181024f0 0x7fa41819fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:03.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.962+0000 7fa420592700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa418109520 0x7fa418199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:03.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.962+0000 7fa420592700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa418113760 con 0x7fa4181024f0 2024-05-14T16:53:03.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.962+0000 7fa420592700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa4181135e0 con 0x7fa418101b30 2024-05-14T16:53:03.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.962+0000 7fa420592700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa4181138e0 con 0x7fa418109520 2024-05-14T16:53:03.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.962+0000 7fa41db2d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4181024f0 0x7fa41819fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:03.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.962+0000 7fa41db2d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4181024f0 0x7fa41819fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52300/0 (socket says 172.21.15.2:52300) 2024-05-14T16:53:03.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.962+0000 7fa41db2d700 1 -- 172.21.15.2:0/3101412987 learned_addr learned my addr 172.21.15.2:0/3101412987 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:53:03.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.962+0000 7fa41eb2f700 1 --2- 172.21.15.2:0/3101412987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa418109520 0x7fa418199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:03.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.962+0000 7fa41e32e700 1 --2- 172.21.15.2:0/3101412987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa418101b30 0x7fa41819f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:03.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.962+0000 7fa41db2d700 1 -- 172.21.15.2:0/3101412987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa418109520 msgr2=0x7fa418199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:03.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.962+0000 7fa41db2d700 1 --2- 172.21.15.2:0/3101412987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa418109520 0x7fa418199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:03.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.962+0000 7fa41db2d700 1 -- 172.21.15.2:0/3101412987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa418101b30 msgr2=0x7fa41819f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:03.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.962+0000 7fa41db2d700 1 --2- 172.21.15.2:0/3101412987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa418101b30 0x7fa41819f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:03.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.962+0000 7fa41db2d700 1 -- 172.21.15.2:0/3101412987 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa41819a160 con 0x7fa4181024f0 2024-05-14T16:53:03.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.963+0000 7fa41db2d700 1 --2- 172.21.15.2:0/3101412987 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4181024f0 0x7fa41819fd80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa40800b580 tx=0x7fa4080047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:03.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.963+0000 7fa40f7fe700 1 -- 172.21.15.2:0/3101412987 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa408027070 con 0x7fa4181024f0 2024-05-14T16:53:03.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.963+0000 7fa420592700 1 -- 172.21.15.2:0/3101412987 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa41819a3f0 con 0x7fa4181024f0 2024-05-14T16:53:03.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.963+0000 7fa40f7fe700 1 -- 172.21.15.2:0/3101412987 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa408003f20 con 0x7fa4181024f0 2024-05-14T16:53:03.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.964+0000 7fa40f7fe700 1 -- 172.21.15.2:0/3101412987 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa408022d10 con 0x7fa4181024f0 2024-05-14T16:53:03.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.964+0000 7fa420592700 1 -- 172.21.15.2:0/3101412987 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4181a6710 con 0x7fa4181024f0 2024-05-14T16:53:03.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.964+0000 7fa420592700 1 -- 172.21.15.2:0/3101412987 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa418063cd0 con 0x7fa4181024f0 2024-05-14T16:53:03.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.966+0000 7fa40f7fe700 1 -- 172.21.15.2:0/3101412987 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fa408022470 con 0x7fa4181024f0 2024-05-14T16:53:03.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.967+0000 7fa40f7fe700 1 --2- 172.21.15.2:0/3101412987 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fa4040b2910 0x7fa4040b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:03.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.967+0000 7fa41e32e700 1 --2- 172.21.15.2:0/3101412987 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fa4040b2910 0x7fa4040b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:03.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.967+0000 7fa40f7fe700 1 -- 172.21.15.2:0/3101412987 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7fa4080e35c0 con 0x7fa4181024f0 2024-05-14T16:53:03.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.967+0000 7fa40f7fe700 1 --2- 172.21.15.2:0/3101412987 >> [v2:172.21.15.123:6824/4266374435,v1:172.21.15.123:6825/4266374435] conn(0x7fa4040bc450 0x7fa4040be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:03.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.968+0000 7fa40f7fe700 1 -- 172.21.15.2:0/3101412987 --> [v2:172.21.15.123:6824/4266374435,v1:172.21.15.123:6825/4266374435] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa408007e80 con 0x7fa4040bc450 2024-05-14T16:53:03.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.968+0000 7fa40f7fe700 1 -- 172.21.15.2:0/3101412987 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fa4080e3a10 con 0x7fa4181024f0 2024-05-14T16:53:03.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.968+0000 7fa41eb2f700 1 --2- 172.21.15.2:0/3101412987 >> [v2:172.21.15.123:6824/4266374435,v1:172.21.15.123:6825/4266374435] conn(0x7fa4040bc450 0x7fa4040be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:03.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.968+0000 7fa41e32e700 1 --2- 172.21.15.2:0/3101412987 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fa4040b2910 0x7fa4040b4dd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa41819ab40 tx=0x7fa41400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:03.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.968+0000 7fa41eb2f700 1 --2- 172.21.15.2:0/3101412987 >> [v2:172.21.15.123:6824/4266374435,v1:172.21.15.123:6825/4266374435] conn(0x7fa4040bc450 0x7fa4040be8f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:03.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.969+0000 7fa40f7fe700 1 -- 172.21.15.2:0/3101412987 <== osd.10 v2:172.21.15.123:6824/4266374435 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa408007e80 con 0x7fa4040bc450 2024-05-14T16:53:03.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.987+0000 7fa420592700 1 -- 172.21.15.2:0/3101412987 --> [v2:172.21.15.123:6824/4266374435,v1:172.21.15.123:6825/4266374435] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa4180640d0 con 0x7fa4040bc450 2024-05-14T16:53:03.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.988+0000 7fa40f7fe700 1 -- 172.21.15.2:0/3101412987 <== osd.10 v2:172.21.15.123:6824/4266374435 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa4180640d0 con 0x7fa4040bc450 2024-05-14T16:53:03.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.988+0000 7fa420592700 1 -- 172.21.15.2:0/3101412987 >> [v2:172.21.15.123:6824/4266374435,v1:172.21.15.123:6825/4266374435] conn(0x7fa4040bc450 msgr2=0x7fa4040be8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:03.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.988+0000 7fa420592700 1 --2- 172.21.15.2:0/3101412987 >> [v2:172.21.15.123:6824/4266374435,v1:172.21.15.123:6825/4266374435] conn(0x7fa4040bc450 0x7fa4040be8f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:03.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.988+0000 7fa420592700 1 -- 172.21.15.2:0/3101412987 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fa4040b2910 msgr2=0x7fa4040b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:03.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.988+0000 7fa420592700 1 --2- 172.21.15.2:0/3101412987 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fa4040b2910 0x7fa4040b4dd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa41819ab40 tx=0x7fa41400b410 comp rx=0 tx=0).stop 2024-05-14T16:53:03.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.988+0000 7fa420592700 1 -- 172.21.15.2:0/3101412987 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4181024f0 msgr2=0x7fa41819fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:03.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.988+0000 7fa420592700 1 --2- 172.21.15.2:0/3101412987 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4181024f0 0x7fa41819fd80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa40800b580 tx=0x7fa4080047b0 comp rx=0 tx=0).stop 2024-05-14T16:53:03.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.988+0000 7fa41eb2f700 1 -- 172.21.15.2:0/3101412987 reap_dead start 2024-05-14T16:53:03.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.988+0000 7fa420592700 1 -- 172.21.15.2:0/3101412987 shutdown_connections 2024-05-14T16:53:03.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.988+0000 7fa420592700 1 -- 172.21.15.2:0/3101412987 >> 172.21.15.2:0/3101412987 conn(0x7fa418078ab0 msgr2=0x7fa418100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:03.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.988+0000 7fa420592700 1 -- 172.21.15.2:0/3101412987 shutdown_connections 2024-05-14T16:53:03.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:03.988+0000 7fa420592700 1 -- 172.21.15.2:0/3101412987 wait complete. 2024-05-14T16:53:04.859 INFO:teuthology.orchestra.run.smithi002.stdout:274877906984 2024-05-14T16:53:04.859 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-14T16:53:05.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.040+0000 7f07de04e700 1 -- 172.21.15.2:0/3741843840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07d806c8c0 msgr2=0x7f07d8108180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:05.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.040+0000 7f07de04e700 1 --2- 172.21.15.2:0/3741843840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07d806c8c0 0x7f07d8108180 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f07c00099c0 tx=0x7f07c0009cd0 comp rx=0 tx=0).stop 2024-05-14T16:53:05.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.040+0000 7f07de04e700 1 -- 172.21.15.2:0/3741843840 shutdown_connections 2024-05-14T16:53:05.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.040+0000 7f07de04e700 1 --2- 172.21.15.2:0/3741843840 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07d8108860 0x7f07d8110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:05.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.040+0000 7f07de04e700 1 --2- 172.21.15.2:0/3741843840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07d806bf10 0x7f07d806c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:05.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.040+0000 7f07de04e700 1 --2- 172.21.15.2:0/3741843840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07d806c8c0 0x7f07d8108180 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:05.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.040+0000 7f07de04e700 1 -- 172.21.15.2:0/3741843840 >> 172.21.15.2:0/3741843840 conn(0x7f07d8078490 msgr2=0x7f07d80788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:05.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.041+0000 7f07de04e700 1 -- 172.21.15.2:0/3741843840 shutdown_connections 2024-05-14T16:53:05.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.041+0000 7f07de04e700 1 -- 172.21.15.2:0/3741843840 wait complete. 2024-05-14T16:53:05.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.041+0000 7f07de04e700 1 Processor -- start 2024-05-14T16:53:05.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.041+0000 7f07de04e700 1 -- start start 2024-05-14T16:53:05.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.042+0000 7f07de04e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07d806bf10 0x7f07d819f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:05.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.042+0000 7f07de04e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07d806c8c0 0x7f07d819fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:05.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.042+0000 7f07de04e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07d8108860 0x7f07d8199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:05.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.042+0000 7f07de04e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f07d8113730 con 0x7f07d806c8c0 2024-05-14T16:53:05.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.042+0000 7f07de04e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f07d81135b0 con 0x7f07d806bf10 2024-05-14T16:53:05.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.042+0000 7f07de04e700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f07d81138b0 con 0x7f07d8108860 2024-05-14T16:53:05.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.042+0000 7f07cffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07d806c8c0 0x7f07d819fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:05.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.042+0000 7f07cffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07d806c8c0 0x7f07d819fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52314/0 (socket says 172.21.15.2:52314) 2024-05-14T16:53:05.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.042+0000 7f07cffff700 1 -- 172.21.15.2:0/80451413 learned_addr learned my addr 172.21.15.2:0/80451413 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:53:05.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.042+0000 7f07d77fe700 1 --2- 172.21.15.2:0/80451413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07d806bf10 0x7f07d819f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:05.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.042+0000 7f07d7fff700 1 --2- 172.21.15.2:0/80451413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07d8108860 0x7f07d8199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:05.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.042+0000 7f07cffff700 1 -- 172.21.15.2:0/80451413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07d8108860 msgr2=0x7f07d8199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:05.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.042+0000 7f07cffff700 1 --2- 172.21.15.2:0/80451413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07d8108860 0x7f07d8199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:05.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.042+0000 7f07cffff700 1 -- 172.21.15.2:0/80451413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07d806bf10 msgr2=0x7f07d819f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:05.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.042+0000 7f07cffff700 1 --2- 172.21.15.2:0/80451413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07d806bf10 0x7f07d819f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:05.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.042+0000 7f07cffff700 1 -- 172.21.15.2:0/80451413 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07d819a1b0 con 0x7f07d806c8c0 2024-05-14T16:53:05.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.042+0000 7f07cffff700 1 --2- 172.21.15.2:0/80451413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07d806c8c0 0x7f07d819fdd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f07c0006010 tx=0x7f07c00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:05.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.042+0000 7f07d57fa700 1 -- 172.21.15.2:0/80451413 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c002b070 con 0x7f07d806c8c0 2024-05-14T16:53:05.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.043+0000 7f07d57fa700 1 -- 172.21.15.2:0/80451413 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07c0004120 con 0x7f07d806c8c0 2024-05-14T16:53:05.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.043+0000 7f07de04e700 1 -- 172.21.15.2:0/80451413 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07d819a440 con 0x7f07d806c8c0 2024-05-14T16:53:05.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.043+0000 7f07d57fa700 1 -- 172.21.15.2:0/80451413 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c001d450 con 0x7f07d806c8c0 2024-05-14T16:53:05.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.043+0000 7f07de04e700 1 -- 172.21.15.2:0/80451413 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07d81024d0 con 0x7f07d806c8c0 2024-05-14T16:53:05.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.044+0000 7f07de04e700 1 -- 172.21.15.2:0/80451413 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07d8104cd0 con 0x7f07d806c8c0 2024-05-14T16:53:05.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.046+0000 7f07d57fa700 1 -- 172.21.15.2:0/80451413 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f07c002e070 con 0x7f07d806c8c0 2024-05-14T16:53:05.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.047+0000 7f07d57fa700 1 --2- 172.21.15.2:0/80451413 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f07c40b2910 0x7f07c40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:05.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.047+0000 7f07d57fa700 1 -- 172.21.15.2:0/80451413 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f07c00e7390 con 0x7f07d806c8c0 2024-05-14T16:53:05.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.047+0000 7f07d77fe700 1 --2- 172.21.15.2:0/80451413 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f07c40b2910 0x7f07c40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:05.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.048+0000 7f07d77fe700 1 --2- 172.21.15.2:0/80451413 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f07c40b2910 0x7f07c40b4dd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f07d819ab90 tx=0x7f07c80073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:05.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.051+0000 7f07d57fa700 1 -- 172.21.15.2:0/80451413 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f07c0035080 con 0x7f07d806c8c0 2024-05-14T16:53:05.059 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:53:05.059 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:53:05.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.294+0000 7f07de04e700 1 -- 172.21.15.2:0/80451413 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f07d819af80 con 0x7f07d806c8c0 2024-05-14T16:53:05.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.294+0000 7f07d57fa700 1 -- 172.21.15.2:0/80451413 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f07c0023070 con 0x7f07d806c8c0 2024-05-14T16:53:05.294 INFO:teuthology.orchestra.run.smithi002.stdout:154618822722 2024-05-14T16:53:05.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.297+0000 7f07de04e700 1 -- 172.21.15.2:0/80451413 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f07c40b2910 msgr2=0x7f07c40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:05.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.297+0000 7f07de04e700 1 --2- 172.21.15.2:0/80451413 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f07c40b2910 0x7f07c40b4dd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f07d819ab90 tx=0x7f07c80073d0 comp rx=0 tx=0).stop 2024-05-14T16:53:05.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.297+0000 7f07de04e700 1 -- 172.21.15.2:0/80451413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07d806c8c0 msgr2=0x7f07d819fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:05.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.297+0000 7f07de04e700 1 --2- 172.21.15.2:0/80451413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07d806c8c0 0x7f07d819fdd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f07c0006010 tx=0x7f07c00047b0 comp rx=0 tx=0).stop 2024-05-14T16:53:05.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.297+0000 7f07de04e700 1 -- 172.21.15.2:0/80451413 shutdown_connections 2024-05-14T16:53:05.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.297+0000 7f07de04e700 1 --2- 172.21.15.2:0/80451413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07d8108860 0x7f07d8199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:05.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.297+0000 7f07de04e700 1 --2- 172.21.15.2:0/80451413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07d806bf10 0x7f07d819f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:05.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.297+0000 7f07de04e700 1 --2- 172.21.15.2:0/80451413 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f07c40b2910 0x7f07c40b4dd0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:05.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.297+0000 7f07de04e700 1 --2- 172.21.15.2:0/80451413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07d806c8c0 0x7f07d819fdd0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:05.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.297+0000 7f07de04e700 1 -- 172.21.15.2:0/80451413 >> 172.21.15.2:0/80451413 conn(0x7f07d8078490 msgr2=0x7f07d81074a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:05.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.297+0000 7f07de04e700 1 -- 172.21.15.2:0/80451413 shutdown_connections 2024-05-14T16:53:05.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.297+0000 7f07de04e700 1 -- 172.21.15.2:0/80451413 wait complete. 2024-05-14T16:53:05.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:05 smithi002 ceph-mon[31538]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:53:05.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:05 smithi151 ceph-mon[36066]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:53:05.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:05 smithi123 ceph-mon[35971]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:53:05.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.764+0000 7f938cf12700 1 -- 172.21.15.2:0/536921975 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9388102440 msgr2=0x7f93881028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:05.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.764+0000 7f938cf12700 1 --2- 172.21.15.2:0/536921975 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9388102440 0x7f93881028c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f937c009990 tx=0x7f937c009ca0 comp rx=0 tx=0).stop 2024-05-14T16:53:05.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.764+0000 7f938cf12700 1 -- 172.21.15.2:0/536921975 shutdown_connections 2024-05-14T16:53:05.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.764+0000 7f938cf12700 1 --2- 172.21.15.2:0/536921975 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9388102e00 0x7f938810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:05.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.764+0000 7f938cf12700 1 --2- 172.21.15.2:0/536921975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9388108520 0x7f9388108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:05.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.764+0000 7f938cf12700 1 --2- 172.21.15.2:0/536921975 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9388102440 0x7f93881028c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:05.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.764+0000 7f938cf12700 1 -- 172.21.15.2:0/536921975 >> 172.21.15.2:0/536921975 conn(0x7f93880793f0 msgr2=0x7f9388079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:05.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.764+0000 7f938cf12700 1 -- 172.21.15.2:0/536921975 shutdown_connections 2024-05-14T16:53:05.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.764+0000 7f938cf12700 1 -- 172.21.15.2:0/536921975 wait complete. 2024-05-14T16:53:05.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.765+0000 7f938cf12700 1 Processor -- start 2024-05-14T16:53:05.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.765+0000 7f938cf12700 1 -- start start 2024-05-14T16:53:05.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.765+0000 7f938cf12700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9388102440 0x7f938819f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:05.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.765+0000 7f938cf12700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9388102e00 0x7f938819fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:05.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.765+0000 7f938cf12700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9388108520 0x7f9388199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:05.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.765+0000 7f938cf12700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9388112650 con 0x7f9388108520 2024-05-14T16:53:05.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.765+0000 7f938cf12700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f93881124d0 con 0x7f9388102440 2024-05-14T16:53:05.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.765+0000 7f938cf12700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f93881127d0 con 0x7f9388102e00 2024-05-14T16:53:05.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.765+0000 7f9386d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9388108520 0x7f9388199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:05.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.765+0000 7f9386d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9388108520 0x7f9388199ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52332/0 (socket says 172.21.15.2:52332) 2024-05-14T16:53:05.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.765+0000 7f9386d9d700 1 -- 172.21.15.2:0/1292377211 learned_addr learned my addr 172.21.15.2:0/1292377211 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:53:05.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.766+0000 7f9385d9b700 1 --2- 172.21.15.2:0/1292377211 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9388102e00 0x7f938819fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:05.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.766+0000 7f938659c700 1 --2- 172.21.15.2:0/1292377211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9388102440 0x7f938819f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:05.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.766+0000 7f9386d9d700 1 -- 172.21.15.2:0/1292377211 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9388102e00 msgr2=0x7f938819fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:05.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.766+0000 7f9386d9d700 1 --2- 172.21.15.2:0/1292377211 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9388102e00 0x7f938819fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:05.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.766+0000 7f9386d9d700 1 -- 172.21.15.2:0/1292377211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9388102440 msgr2=0x7f938819f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:05.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.766+0000 7f9386d9d700 1 --2- 172.21.15.2:0/1292377211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9388102440 0x7f938819f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:05.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.766+0000 7f9386d9d700 1 -- 172.21.15.2:0/1292377211 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f938819a220 con 0x7f9388108520 2024-05-14T16:53:05.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.766+0000 7f938659c700 1 --2- 172.21.15.2:0/1292377211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9388102440 0x7f938819f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:53:05.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.766+0000 7f9386d9d700 1 --2- 172.21.15.2:0/1292377211 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9388108520 0x7f9388199ac0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f938000b970 tx=0x7f938000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:05.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.766+0000 7f93777fe700 1 -- 172.21.15.2:0/1292377211 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9380019070 con 0x7f9388108520 2024-05-14T16:53:05.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.766+0000 7f93777fe700 1 -- 172.21.15.2:0/1292377211 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f938000cde0 con 0x7f9388108520 2024-05-14T16:53:05.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.766+0000 7f938cf12700 1 -- 172.21.15.2:0/1292377211 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f938819a510 con 0x7f9388108520 2024-05-14T16:53:05.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.766+0000 7f93777fe700 1 -- 172.21.15.2:0/1292377211 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9380007370 con 0x7f9388108520 2024-05-14T16:53:05.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.766+0000 7f938cf12700 1 -- 172.21.15.2:0/1292377211 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93881a67d0 con 0x7f9388108520 2024-05-14T16:53:05.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.767+0000 7f938cf12700 1 -- 172.21.15.2:0/1292377211 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9388103b60 con 0x7f9388108520 2024-05-14T16:53:05.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.769+0000 7f93777fe700 1 -- 172.21.15.2:0/1292377211 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f938001c070 con 0x7f9388108520 2024-05-14T16:53:05.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.769+0000 7f93777fe700 1 --2- 172.21.15.2:0/1292377211 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f93700b2910 0x7f93700b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:05.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.770+0000 7f938659c700 1 --2- 172.21.15.2:0/1292377211 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f93700b2910 0x7f93700b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:05.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.770+0000 7f93777fe700 1 -- 172.21.15.2:0/1292377211 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f93800e1ee0 con 0x7f9388108520 2024-05-14T16:53:05.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.770+0000 7f938659c700 1 --2- 172.21.15.2:0/1292377211 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f93700b2910 0x7f93700b4dd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9378005950 tx=0x7f9378005f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:05.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.773+0000 7f93777fe700 1 -- 172.21.15.2:0/1292377211 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9380023070 con 0x7f9388108520 2024-05-14T16:53:05.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:05.999+0000 7f938cf12700 1 -- 172.21.15.2:0/1292377211 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f938806a230 con 0x7f9388108520 2024-05-14T16:53:05.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:06.000+0000 7f93777fe700 1 -- 172.21.15.2:0/1292377211 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f938806a230 con 0x7f9388108520 2024-05-14T16:53:05.999 INFO:teuthology.orchestra.run.smithi002.stdout:201863462966 2024-05-14T16:53:06.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:06.002+0000 7f938cf12700 1 -- 172.21.15.2:0/1292377211 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f93700b2910 msgr2=0x7f93700b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:06.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:06.002+0000 7f938cf12700 1 --2- 172.21.15.2:0/1292377211 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f93700b2910 0x7f93700b4dd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9378005950 tx=0x7f9378005f50 comp rx=0 tx=0).stop 2024-05-14T16:53:06.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:06.002+0000 7f938cf12700 1 -- 172.21.15.2:0/1292377211 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9388108520 msgr2=0x7f9388199ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:06.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:06.002+0000 7f938cf12700 1 --2- 172.21.15.2:0/1292377211 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9388108520 0x7f9388199ac0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f938000b970 tx=0x7f938000bc80 comp rx=0 tx=0).stop 2024-05-14T16:53:06.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:06.002+0000 7f938cf12700 1 -- 172.21.15.2:0/1292377211 shutdown_connections 2024-05-14T16:53:06.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:06.002+0000 7f938cf12700 1 --2- 172.21.15.2:0/1292377211 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f93700b2910 0x7f93700b4dd0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:06.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:06.002+0000 7f938cf12700 1 --2- 172.21.15.2:0/1292377211 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9388108520 0x7f9388199ac0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:06.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:06.002+0000 7f938cf12700 1 --2- 172.21.15.2:0/1292377211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9388102440 0x7f938819f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:06.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:06.002+0000 7f938cf12700 1 --2- 172.21.15.2:0/1292377211 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9388102e00 0x7f938819fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:06.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:06.002+0000 7f938cf12700 1 -- 172.21.15.2:0/1292377211 >> 172.21.15.2:0/1292377211 conn(0x7f93880793f0 msgr2=0x7f9388106400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:06.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:06.002+0000 7f938cf12700 1 -- 172.21.15.2:0/1292377211 shutdown_connections 2024-05-14T16:53:06.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:06.002+0000 7f938cf12700 1 -- 172.21.15.2:0/1292377211 wait complete. 2024-05-14T16:53:06.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:06 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/80451413' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-14T16:53:06.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:06 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/1292377211' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-14T16:53:06.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:06 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/80451413' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-14T16:53:06.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:06 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/1292377211' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-14T16:53:06.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:06 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/80451413' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-14T16:53:06.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:06 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/1292377211' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-14T16:53:07.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:07 smithi002 ceph-mon[31538]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-14T16:53:07.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:07 smithi151 ceph-mon[36066]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-14T16:53:07.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:07 smithi123 ceph-mon[35971]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-14T16:53:08.079 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:53:08.079 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:53:08.596 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:53:08.629 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822718 got 154618822722 for osd.5 2024-05-14T16:53:08.629 DEBUG:teuthology.parallel:result is None 2024-05-14T16:53:09.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.021+0000 7f70dd1ec700 1 -- 172.21.15.2:0/1319630378 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70d8103f80 msgr2=0x7f70d8110410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:09.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.021+0000 7f70dd1ec700 1 --2- 172.21.15.2:0/1319630378 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70d8103f80 0x7f70d8110410 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f70d000b0a0 tx=0x7f70d000b3b0 comp rx=0 tx=0).stop 2024-05-14T16:53:09.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.021+0000 7f70dd1ec700 1 -- 172.21.15.2:0/1319630378 shutdown_connections 2024-05-14T16:53:09.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.021+0000 7f70dd1ec700 1 --2- 172.21.15.2:0/1319630378 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70d8103f80 0x7f70d8110410 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:09.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.021+0000 7f70dd1ec700 1 --2- 172.21.15.2:0/1319630378 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70d81095d0 0x7f70d81099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:09.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.021+0000 7f70dd1ec700 1 --2- 172.21.15.2:0/1319630378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70d81035c0 0x7f70d8103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:09.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.021+0000 7f70dd1ec700 1 -- 172.21.15.2:0/1319630378 >> 172.21.15.2:0/1319630378 conn(0x7f70d807a610 msgr2=0x7f70d807aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:09.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.021+0000 7f70dd1ec700 1 -- 172.21.15.2:0/1319630378 shutdown_connections 2024-05-14T16:53:09.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.021+0000 7f70dd1ec700 1 -- 172.21.15.2:0/1319630378 wait complete. 2024-05-14T16:53:09.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.022+0000 7f70dd1ec700 1 Processor -- start 2024-05-14T16:53:09.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.022+0000 7f70dd1ec700 1 -- start start 2024-05-14T16:53:09.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.023+0000 7f70dd1ec700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70d81035c0 0x7f70d819f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:09.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.023+0000 7f70dd1ec700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70d8103f80 0x7f70d819fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:09.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.023+0000 7f70dd1ec700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70d81095d0 0x7f70d81999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:09.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.023+0000 7f70dd1ec700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f70d81137d0 con 0x7f70d81035c0 2024-05-14T16:53:09.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.023+0000 7f70dd1ec700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f70d8113650 con 0x7f70d81095d0 2024-05-14T16:53:09.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.023+0000 7f70dd1ec700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f70d8113950 con 0x7f70d8103f80 2024-05-14T16:53:09.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.023+0000 7f70d6d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70d81035c0 0x7f70d819f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:09.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.023+0000 7f70d6d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70d81035c0 0x7f70d819f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52356/0 (socket says 172.21.15.2:52356) 2024-05-14T16:53:09.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.023+0000 7f70d6d9d700 1 -- 172.21.15.2:0/1412628140 learned_addr learned my addr 172.21.15.2:0/1412628140 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:53:09.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.023+0000 7f70d659c700 1 --2- 172.21.15.2:0/1412628140 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70d8103f80 0x7f70d819fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:09.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.023+0000 7f70d759e700 1 --2- 172.21.15.2:0/1412628140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70d81095d0 0x7f70d81999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:09.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.023+0000 7f70d6d9d700 1 -- 172.21.15.2:0/1412628140 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70d8103f80 msgr2=0x7f70d819fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:09.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.023+0000 7f70d6d9d700 1 --2- 172.21.15.2:0/1412628140 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70d8103f80 0x7f70d819fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:09.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.023+0000 7f70d6d9d700 1 -- 172.21.15.2:0/1412628140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70d81095d0 msgr2=0x7f70d81999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:09.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.023+0000 7f70d6d9d700 1 --2- 172.21.15.2:0/1412628140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70d81095d0 0x7f70d81999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:09.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.023+0000 7f70d6d9d700 1 -- 172.21.15.2:0/1412628140 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d819a130 con 0x7f70d81035c0 2024-05-14T16:53:09.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.023+0000 7f70d659c700 1 --2- 172.21.15.2:0/1412628140 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70d8103f80 0x7f70d819fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:53:09.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.023+0000 7f70d6d9d700 1 --2- 172.21.15.2:0/1412628140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70d81035c0 0x7f70d819f810 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f70c800d980 tx=0x7f70c800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:09.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.024+0000 7f70bffff700 1 -- 172.21.15.2:0/1412628140 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c8017070 con 0x7f70d81035c0 2024-05-14T16:53:09.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.024+0000 7f70dd1ec700 1 -- 172.21.15.2:0/1412628140 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70d819a420 con 0x7f70d81035c0 2024-05-14T16:53:09.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.024+0000 7f70bffff700 1 -- 172.21.15.2:0/1412628140 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70c8004830 con 0x7f70d81035c0 2024-05-14T16:53:09.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.024+0000 7f70bffff700 1 -- 172.21.15.2:0/1412628140 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c8005440 con 0x7f70d81035c0 2024-05-14T16:53:09.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.024+0000 7f70dd1ec700 1 -- 172.21.15.2:0/1412628140 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70d81a66e0 con 0x7f70d81035c0 2024-05-14T16:53:09.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.025+0000 7f70dd1ec700 1 -- 172.21.15.2:0/1412628140 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70d806a230 con 0x7f70d81035c0 2024-05-14T16:53:09.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.026+0000 7f70bffff700 1 -- 172.21.15.2:0/1412628140 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f70c80055e0 con 0x7f70d81035c0 2024-05-14T16:53:09.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.027+0000 7f70bffff700 1 --2- 172.21.15.2:0/1412628140 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f70c00b2910 0x7f70c00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:09.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.027+0000 7f70d659c700 1 --2- 172.21.15.2:0/1412628140 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f70c00b2910 0x7f70c00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:09.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.027+0000 7f70bffff700 1 -- 172.21.15.2:0/1412628140 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f70c80e2710 con 0x7f70d81035c0 2024-05-14T16:53:09.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.028+0000 7f70d659c700 1 --2- 172.21.15.2:0/1412628140 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f70c00b2910 0x7f70c00b4dd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f70cc005950 tx=0x7f70cc0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:09.032 INFO:teuthology.orchestra.run.smithi002.stdout:34359738458 2024-05-14T16:53:09.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.031+0000 7f70bffff700 1 -- 172.21.15.2:0/1412628140 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f70c8020020 con 0x7f70d81035c0 2024-05-14T16:53:09.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.263+0000 7f70dd1ec700 1 -- 172.21.15.2:0/1412628140 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f70d802e560 con 0x7f70d81035c0 2024-05-14T16:53:09.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.264+0000 7f70bffff700 1 -- 172.21.15.2:0/1412628140 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f70c80aa2e0 con 0x7f70d81035c0 2024-05-14T16:53:09.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.266+0000 7f70dd1ec700 1 -- 172.21.15.2:0/1412628140 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f70c00b2910 msgr2=0x7f70c00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:09.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.266+0000 7f70dd1ec700 1 --2- 172.21.15.2:0/1412628140 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f70c00b2910 0x7f70c00b4dd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f70cc005950 tx=0x7f70cc0058e0 comp rx=0 tx=0).stop 2024-05-14T16:53:09.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.266+0000 7f70dd1ec700 1 -- 172.21.15.2:0/1412628140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70d81035c0 msgr2=0x7f70d819f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:09.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.266+0000 7f70dd1ec700 1 --2- 172.21.15.2:0/1412628140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70d81035c0 0x7f70d819f810 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f70c800d980 tx=0x7f70c800dd40 comp rx=0 tx=0).stop 2024-05-14T16:53:09.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.266+0000 7f70dd1ec700 1 -- 172.21.15.2:0/1412628140 shutdown_connections 2024-05-14T16:53:09.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.266+0000 7f70dd1ec700 1 --2- 172.21.15.2:0/1412628140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70d81095d0 0x7f70d81999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:09.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.266+0000 7f70dd1ec700 1 --2- 172.21.15.2:0/1412628140 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f70c00b2910 0x7f70c00b4dd0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:09.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.266+0000 7f70dd1ec700 1 --2- 172.21.15.2:0/1412628140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70d81035c0 0x7f70d819f810 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:09.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.266+0000 7f70dd1ec700 1 --2- 172.21.15.2:0/1412628140 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70d8103f80 0x7f70d819fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:09.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.266+0000 7f70dd1ec700 1 -- 172.21.15.2:0/1412628140 >> 172.21.15.2:0/1412628140 conn(0x7f70d807a610 msgr2=0x7f70d8102280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:09.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.266+0000 7f70dd1ec700 1 -- 172.21.15.2:0/1412628140 shutdown_connections 2024-05-14T16:53:09.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:08.266+0000 7f70dd1ec700 1 -- 172.21.15.2:0/1412628140 wait complete. 2024-05-14T16:53:09.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:09 smithi002 ceph-mon[31538]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T16:53:09.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:09 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/1412628140' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-14T16:53:09.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:09 smithi151 ceph-mon[36066]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T16:53:09.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:09 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/1412628140' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-14T16:53:09.626 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738453 got 34359738458 for osd.0 2024-05-14T16:53:09.626 DEBUG:teuthology.parallel:result is None 2024-05-14T16:53:09.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:09 smithi123 ceph-mon[35971]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T16:53:09.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:09 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/1412628140' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-14T16:53:10.882 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462963 got 201863462966 for osd.7 2024-05-14T16:53:10.882 DEBUG:teuthology.parallel:result is None 2024-05-14T16:53:11.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:11 smithi002 ceph-mon[31538]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T16:53:11.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:11 smithi151 ceph-mon[36066]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T16:53:11.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.606+0000 7f53c1445700 1 -- 172.21.15.2:0/2464577546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53bc109630 msgr2=0x7f53bc109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:11.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.606+0000 7f53c1445700 1 --2- 172.21.15.2:0/2464577546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53bc109630 0x7f53bc109a10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f53ac00b0d0 tx=0x7f53ac00b3e0 comp rx=0 tx=0).stop 2024-05-14T16:53:11.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.606+0000 7f53c1445700 1 -- 172.21.15.2:0/2464577546 shutdown_connections 2024-05-14T16:53:11.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.606+0000 7f53c1445700 1 --2- 172.21.15.2:0/2464577546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53bc06c6d0 0x7f53bc110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:11.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.606+0000 7f53c1445700 1 --2- 172.21.15.2:0/2464577546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53bc109630 0x7f53bc109a10 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:11.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.606+0000 7f53c1445700 1 --2- 172.21.15.2:0/2464577546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53bc06bd10 0x7f53bc06c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:11.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.606+0000 7f53c1445700 1 -- 172.21.15.2:0/2464577546 >> 172.21.15.2:0/2464577546 conn(0x7f53bc0794e0 msgr2=0x7f53bc0798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:11.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.606+0000 7f53c1445700 1 -- 172.21.15.2:0/2464577546 shutdown_connections 2024-05-14T16:53:11.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.606+0000 7f53c1445700 1 -- 172.21.15.2:0/2464577546 wait complete. 2024-05-14T16:53:11.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.607+0000 7f53c1445700 1 Processor -- start 2024-05-14T16:53:11.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.607+0000 7f53c1445700 1 -- start start 2024-05-14T16:53:11.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.607+0000 7f53c1445700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53bc06bd10 0x7f53bc1024b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:11.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.607+0000 7f53c1445700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53bc06c6d0 0x7f53bc1029f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:11.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.607+0000 7f53c1445700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53bc109630 0x7f53bc102f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:11.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.608+0000 7f53c1445700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f53bc113800 con 0x7f53bc06c6d0 2024-05-14T16:53:11.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.608+0000 7f53c1445700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f53bc113680 con 0x7f53bc06bd10 2024-05-14T16:53:11.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.608+0000 7f53c1445700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f53bc113980 con 0x7f53bc109630 2024-05-14T16:53:11.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.608+0000 7f53ba7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53bc06c6d0 0x7f53bc1029f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:11.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.608+0000 7f53ba7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53bc06c6d0 0x7f53bc1029f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52372/0 (socket says 172.21.15.2:52372) 2024-05-14T16:53:11.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.608+0000 7f53ba7fc700 1 -- 172.21.15.2:0/3120633822 learned_addr learned my addr 172.21.15.2:0/3120633822 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:53:11.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.608+0000 7f53bb7fe700 1 --2- 172.21.15.2:0/3120633822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53bc109630 0x7f53bc102f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:11.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.608+0000 7f53baffd700 1 --2- 172.21.15.2:0/3120633822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53bc06bd10 0x7f53bc1024b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:11.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.608+0000 7f53ba7fc700 1 -- 172.21.15.2:0/3120633822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53bc109630 msgr2=0x7f53bc102f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:11.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.608+0000 7f53ba7fc700 1 --2- 172.21.15.2:0/3120633822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53bc109630 0x7f53bc102f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:11.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.608+0000 7f53ba7fc700 1 -- 172.21.15.2:0/3120633822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53bc06bd10 msgr2=0x7f53bc1024b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:11.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.608+0000 7f53ba7fc700 1 --2- 172.21.15.2:0/3120633822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53bc06bd10 0x7f53bc1024b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:11.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.608+0000 7f53ba7fc700 1 -- 172.21.15.2:0/3120633822 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53bc1a81b0 con 0x7f53bc06c6d0 2024-05-14T16:53:11.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.608+0000 7f53ba7fc700 1 --2- 172.21.15.2:0/3120633822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53bc06c6d0 0x7f53bc1029f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f53b000b530 tx=0x7f53b000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:11.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.608+0000 7f53a3fff700 1 -- 172.21.15.2:0/3120633822 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b0015020 con 0x7f53bc06c6d0 2024-05-14T16:53:11.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.608+0000 7f53a3fff700 1 -- 172.21.15.2:0/3120633822 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53b0011ce0 con 0x7f53bc06c6d0 2024-05-14T16:53:11.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.608+0000 7f53a3fff700 1 -- 172.21.15.2:0/3120633822 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b0010e40 con 0x7f53bc06c6d0 2024-05-14T16:53:11.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.608+0000 7f53c1445700 1 -- 172.21.15.2:0/3120633822 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53bc1a84a0 con 0x7f53bc06c6d0 2024-05-14T16:53:11.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.609+0000 7f53c1445700 1 -- 172.21.15.2:0/3120633822 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53bc1a8990 con 0x7f53bc06c6d0 2024-05-14T16:53:11.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.610+0000 7f53c1445700 1 -- 172.21.15.2:0/3120633822 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53bc10bac0 con 0x7f53bc06c6d0 2024-05-14T16:53:11.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.612+0000 7f53a3fff700 1 -- 172.21.15.2:0/3120633822 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f53b0018020 con 0x7f53bc06c6d0 2024-05-14T16:53:11.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.613+0000 7f53a3fff700 1 --2- 172.21.15.2:0/3120633822 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f53a40b2990 0x7f53a40b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:11.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.613+0000 7f53baffd700 1 --2- 172.21.15.2:0/3120633822 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f53a40b2990 0x7f53a40b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:11.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.613+0000 7f53a3fff700 1 -- 172.21.15.2:0/3120633822 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f53b00e1ff0 con 0x7f53bc06c6d0 2024-05-14T16:53:11.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.613+0000 7f53baffd700 1 --2- 172.21.15.2:0/3120633822 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f53a40b2990 0x7f53a40b4e50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f53ac009fd0 tx=0x7f53ac00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:11.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.616+0000 7f53a3fff700 1 -- 172.21.15.2:0/3120633822 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f53b001f020 con 0x7f53bc06c6d0 2024-05-14T16:53:11.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:11 smithi123 ceph-mon[35971]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T16:53:11.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.835+0000 7f53c1445700 1 -- 172.21.15.2:0/3120633822 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f53bc06a230 con 0x7f53bc06c6d0 2024-05-14T16:53:11.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.835+0000 7f53a3fff700 1 -- 172.21.15.2:0/3120633822 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f53bc06a230 con 0x7f53bc06c6d0 2024-05-14T16:53:11.835 INFO:teuthology.orchestra.run.smithi002.stdout:133143986248 2024-05-14T16:53:11.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.838+0000 7f53c1445700 1 -- 172.21.15.2:0/3120633822 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f53a40b2990 msgr2=0x7f53a40b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:11.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.838+0000 7f53c1445700 1 --2- 172.21.15.2:0/3120633822 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f53a40b2990 0x7f53a40b4e50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f53ac009fd0 tx=0x7f53ac00b040 comp rx=0 tx=0).stop 2024-05-14T16:53:11.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.838+0000 7f53c1445700 1 -- 172.21.15.2:0/3120633822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53bc06c6d0 msgr2=0x7f53bc1029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:11.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.838+0000 7f53c1445700 1 --2- 172.21.15.2:0/3120633822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53bc06c6d0 0x7f53bc1029f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f53b000b530 tx=0x7f53b000b840 comp rx=0 tx=0).stop 2024-05-14T16:53:11.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.838+0000 7f53c1445700 1 -- 172.21.15.2:0/3120633822 shutdown_connections 2024-05-14T16:53:11.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.838+0000 7f53c1445700 1 --2- 172.21.15.2:0/3120633822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53bc109630 0x7f53bc102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:11.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.838+0000 7f53c1445700 1 --2- 172.21.15.2:0/3120633822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53bc06bd10 0x7f53bc1024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:11.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.838+0000 7f53c1445700 1 --2- 172.21.15.2:0/3120633822 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f53a40b2990 0x7f53a40b4e50 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:11.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.838+0000 7f53c1445700 1 --2- 172.21.15.2:0/3120633822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53bc06c6d0 0x7f53bc1029f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:11.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.838+0000 7f53c1445700 1 -- 172.21.15.2:0/3120633822 >> 172.21.15.2:0/3120633822 conn(0x7f53bc0794e0 msgr2=0x7f53bc101030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:11.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.838+0000 7f53c1445700 1 -- 172.21.15.2:0/3120633822 shutdown_connections 2024-05-14T16:53:11.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:11.838+0000 7f53c1445700 1 -- 172.21.15.2:0/3120633822 wait complete. 2024-05-14T16:53:12.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.411+0000 7f4096dfc700 1 -- 172.21.15.2:0/2363649316 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f409006bf10 msgr2=0x7f409006c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:12.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.411+0000 7f4096dfc700 1 --2- 172.21.15.2:0/2363649316 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f409006bf10 0x7f409006c2f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f40840099c0 tx=0x7f4084009cd0 comp rx=0 tx=0).stop 2024-05-14T16:53:12.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.411+0000 7f4096dfc700 1 -- 172.21.15.2:0/2363649316 shutdown_connections 2024-05-14T16:53:12.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.411+0000 7f4096dfc700 1 --2- 172.21.15.2:0/2363649316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4090108860 0x7f4090110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:12.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.411+0000 7f4096dfc700 1 --2- 172.21.15.2:0/2363649316 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f409006bf10 0x7f409006c2f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:12.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.411+0000 7f4096dfc700 1 --2- 172.21.15.2:0/2363649316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f409006c8c0 0x7f4090108180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:12.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.411+0000 7f4096dfc700 1 -- 172.21.15.2:0/2363649316 >> 172.21.15.2:0/2363649316 conn(0x7f4090078490 msgr2=0x7f40900788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:12.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.411+0000 7f4096dfc700 1 -- 172.21.15.2:0/2363649316 shutdown_connections 2024-05-14T16:53:12.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.411+0000 7f4096dfc700 1 -- 172.21.15.2:0/2363649316 wait complete. 2024-05-14T16:53:12.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.411+0000 7f4096dfc700 1 Processor -- start 2024-05-14T16:53:12.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.411+0000 7f4096dfc700 1 -- start start 2024-05-14T16:53:12.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.412+0000 7f4096dfc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f409006bf10 0x7f40901025a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:12.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.412+0000 7f4096dfc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f409006c8c0 0x7f4090102ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:12.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.412+0000 7f4096dfc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4090108860 0x7f4090103020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:12.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.412+0000 7f4096dfc700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4090113790 con 0x7f4090108860 2024-05-14T16:53:12.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.412+0000 7f4096dfc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4090113610 con 0x7f409006c8c0 2024-05-14T16:53:12.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.412+0000 7f4096dfc700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4090113910 con 0x7f409006bf10 2024-05-14T16:53:12.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.412+0000 7f4095399700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4090108860 0x7f4090103020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:12.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.412+0000 7f4095399700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4090108860 0x7f4090103020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52396/0 (socket says 172.21.15.2:52396) 2024-05-14T16:53:12.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.412+0000 7f4095399700 1 -- 172.21.15.2:0/1418609162 learned_addr learned my addr 172.21.15.2:0/1418609162 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:53:12.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.412+0000 7f4094b98700 1 --2- 172.21.15.2:0/1418609162 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f409006bf10 0x7f40901025a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:12.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.412+0000 7f408ffff700 1 --2- 172.21.15.2:0/1418609162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f409006c8c0 0x7f4090102ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:12.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.412+0000 7f4095399700 1 -- 172.21.15.2:0/1418609162 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f409006bf10 msgr2=0x7f40901025a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:12.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.412+0000 7f4095399700 1 --2- 172.21.15.2:0/1418609162 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f409006bf10 0x7f40901025a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:12.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.412+0000 7f4095399700 1 -- 172.21.15.2:0/1418609162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f409006c8c0 msgr2=0x7f4090102ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:12.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.412+0000 7f4095399700 1 --2- 172.21.15.2:0/1418609162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f409006c8c0 0x7f4090102ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:12.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.412+0000 7f4095399700 1 -- 172.21.15.2:0/1418609162 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40901a8200 con 0x7f4090108860 2024-05-14T16:53:12.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.412+0000 7f408ffff700 1 --2- 172.21.15.2:0/1418609162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f409006c8c0 0x7f4090102ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:53:12.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.412+0000 7f4095399700 1 --2- 172.21.15.2:0/1418609162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4090108860 0x7f4090103020 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f408800ef30 tx=0x7f408800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:12.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.413+0000 7f408dffb700 1 -- 172.21.15.2:0/1418609162 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40880092e0 con 0x7f4090108860 2024-05-14T16:53:12.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.413+0000 7f408dffb700 1 -- 172.21.15.2:0/1418609162 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4088004bb0 con 0x7f4090108860 2024-05-14T16:53:12.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.413+0000 7f408dffb700 1 -- 172.21.15.2:0/1418609162 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4088004280 con 0x7f4090108860 2024-05-14T16:53:12.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.413+0000 7f4096dfc700 1 -- 172.21.15.2:0/1418609162 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40901a8490 con 0x7f4090108860 2024-05-14T16:53:12.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.413+0000 7f4096dfc700 1 -- 172.21.15.2:0/1418609162 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40901a89d0 con 0x7f4090108860 2024-05-14T16:53:12.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.414+0000 7f4096dfc700 1 -- 172.21.15.2:0/1418609162 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f409006a230 con 0x7f4090108860 2024-05-14T16:53:12.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.415+0000 7f408dffb700 1 -- 172.21.15.2:0/1418609162 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f408801b030 con 0x7f4090108860 2024-05-14T16:53:12.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.416+0000 7f408dffb700 1 --2- 172.21.15.2:0/1418609162 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f40800b2900 0x7f40800b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:12.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.416+0000 7f4094b98700 1 --2- 172.21.15.2:0/1418609162 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f40800b2900 0x7f40800b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:12.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.416+0000 7f408dffb700 1 -- 172.21.15.2:0/1418609162 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f40880e2640 con 0x7f4090108860 2024-05-14T16:53:12.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.417+0000 7f4094b98700 1 --2- 172.21.15.2:0/1418609162 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f40800b2900 0x7f40800b4dc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4084000c00 tx=0x7f4084005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:12.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.419+0000 7f408dffb700 1 -- 172.21.15.2:0/1418609162 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4088022020 con 0x7f4090108860 2024-05-14T16:53:12.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:12 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/3120633822' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-14T16:53:12.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:12 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/3120633822' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-14T16:53:12.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.641+0000 7f4096dfc700 1 -- 172.21.15.2:0/1418609162 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f409002e560 con 0x7f4090108860 2024-05-14T16:53:12.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.641+0000 7f408dffb700 1 -- 172.21.15.2:0/1418609162 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4088010040 con 0x7f4090108860 2024-05-14T16:53:12.641 INFO:teuthology.orchestra.run.smithi002.stdout:180388626494 2024-05-14T16:53:12.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.643+0000 7f4096dfc700 1 -- 172.21.15.2:0/1418609162 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f40800b2900 msgr2=0x7f40800b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:12.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.643+0000 7f4096dfc700 1 --2- 172.21.15.2:0/1418609162 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f40800b2900 0x7f40800b4dc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4084000c00 tx=0x7f4084005fb0 comp rx=0 tx=0).stop 2024-05-14T16:53:12.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.643+0000 7f4096dfc700 1 -- 172.21.15.2:0/1418609162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4090108860 msgr2=0x7f4090103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:12.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.643+0000 7f4096dfc700 1 --2- 172.21.15.2:0/1418609162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4090108860 0x7f4090103020 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f408800ef30 tx=0x7f408800c3a0 comp rx=0 tx=0).stop 2024-05-14T16:53:12.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.643+0000 7f4096dfc700 1 -- 172.21.15.2:0/1418609162 shutdown_connections 2024-05-14T16:53:12.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.643+0000 7f4096dfc700 1 --2- 172.21.15.2:0/1418609162 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f40800b2900 0x7f40800b4dc0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:12.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.643+0000 7f4096dfc700 1 --2- 172.21.15.2:0/1418609162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4090108860 0x7f4090103020 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:12.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.643+0000 7f4096dfc700 1 --2- 172.21.15.2:0/1418609162 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f409006bf10 0x7f40901025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:12.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.643+0000 7f4096dfc700 1 --2- 172.21.15.2:0/1418609162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f409006c8c0 0x7f4090102ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:12.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.643+0000 7f4096dfc700 1 -- 172.21.15.2:0/1418609162 >> 172.21.15.2:0/1418609162 conn(0x7f4090078490 msgr2=0x7f4090107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:12.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.643+0000 7f4096dfc700 1 -- 172.21.15.2:0/1418609162 shutdown_connections 2024-05-14T16:53:12.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:12.643+0000 7f4096dfc700 1 -- 172.21.15.2:0/1418609162 wait complete. 2024-05-14T16:53:12.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:12 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/3120633822' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-14T16:53:13.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:13 smithi002 ceph-mon[31538]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:53:13.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:13 smithi002 ceph-mon[31538]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-14T16:53:13.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:13 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/1418609162' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-14T16:53:13.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:13 smithi151 ceph-mon[36066]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:53:13.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:13 smithi151 ceph-mon[36066]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-14T16:53:13.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:13 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/1418609162' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-14T16:53:13.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:13 smithi123 ceph-mon[35971]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:53:13.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:13 smithi123 ceph-mon[35971]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-14T16:53:13.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:13 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/1418609162' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-14T16:53:14.967 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986242 got 133143986248 for osd.4 2024-05-14T16:53:14.967 DEBUG:teuthology.parallel:result is None 2024-05-14T16:53:15.477 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:15 smithi002 ceph-mon[31538]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-14T16:53:15.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:15 smithi151 ceph-mon[36066]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-14T16:53:15.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:15 smithi123 ceph-mon[35971]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-14T16:53:16.212 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626490 got 180388626494 for osd.6 2024-05-14T16:53:16.213 DEBUG:teuthology.parallel:result is None 2024-05-14T16:53:16.686 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:53:17.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:17 smithi002 ceph-mon[31538]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-14T16:53:17.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:17 smithi151 ceph-mon[36066]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-14T16:53:17.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:17 smithi123 ceph-mon[35971]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-14T16:53:17.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:17.990+0000 7f2048d34700 1 -- 172.21.15.2:0/1297191299 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2044102e00 msgr2=0x7f204410f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:18.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:17.990+0000 7f2048d34700 1 --2- 172.21.15.2:0/1297191299 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2044102e00 0x7f204410f2f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f203c00b0a0 tx=0x7f203c00b3b0 comp rx=0 tx=0).stop 2024-05-14T16:53:18.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:17.990+0000 7f2048d34700 1 -- 172.21.15.2:0/1297191299 shutdown_connections 2024-05-14T16:53:18.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:17.990+0000 7f2048d34700 1 --2- 172.21.15.2:0/1297191299 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2044102e00 0x7f204410f2f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:18.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:17.990+0000 7f2048d34700 1 --2- 172.21.15.2:0/1297191299 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2044108520 0x7f2044108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:18.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:17.990+0000 7f2048d34700 1 --2- 172.21.15.2:0/1297191299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2044102440 0x7f20441028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:18.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:17.990+0000 7f2048d34700 1 -- 172.21.15.2:0/1297191299 >> 172.21.15.2:0/1297191299 conn(0x7f20440793f0 msgr2=0x7f2044079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:18.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:17.990+0000 7f2048d34700 1 -- 172.21.15.2:0/1297191299 shutdown_connections 2024-05-14T16:53:18.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:17.990+0000 7f2048d34700 1 -- 172.21.15.2:0/1297191299 wait complete. 2024-05-14T16:53:18.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:17.991+0000 7f2048d34700 1 Processor -- start 2024-05-14T16:53:18.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:17.991+0000 7f2048d34700 1 -- start start 2024-05-14T16:53:18.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:17.992+0000 7f2048d34700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2044102440 0x7f204419f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:18.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:17.992+0000 7f2048d34700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2044102e00 0x7f204419fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:18.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:17.992+0000 7f2048d34700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2044108520 0x7f2044199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:18.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:17.992+0000 7f2048d34700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f20441125d0 con 0x7f2044102440 2024-05-14T16:53:18.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:17.992+0000 7f2048d34700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2044112450 con 0x7f2044102e00 2024-05-14T16:53:18.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:17.992+0000 7f2048d34700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2044112750 con 0x7f2044108520 2024-05-14T16:53:18.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:17.992+0000 7f204259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2044102440 0x7f204419f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:18.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:17.992+0000 7f204259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2044102440 0x7f204419f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36326/0 (socket says 172.21.15.2:36326) 2024-05-14T16:53:18.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:17.992+0000 7f204259c700 1 -- 172.21.15.2:0/1827539025 learned_addr learned my addr 172.21.15.2:0/1827539025 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:53:18.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:17.992+0000 7f2039d9b700 1 --2- 172.21.15.2:0/1827539025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2044102e00 0x7f204419fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:18.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:17.992+0000 7f2042d9d700 1 --2- 172.21.15.2:0/1827539025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2044108520 0x7f2044199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:18.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:17.992+0000 7f204259c700 1 -- 172.21.15.2:0/1827539025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2044108520 msgr2=0x7f2044199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:18.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:17.992+0000 7f204259c700 1 --2- 172.21.15.2:0/1827539025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2044108520 0x7f2044199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:18.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:17.992+0000 7f204259c700 1 -- 172.21.15.2:0/1827539025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2044102e00 msgr2=0x7f204419fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:18.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:17.992+0000 7f204259c700 1 --2- 172.21.15.2:0/1827539025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2044102e00 0x7f204419fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:18.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:17.992+0000 7f204259c700 1 -- 172.21.15.2:0/1827539025 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f204419a170 con 0x7f2044102440 2024-05-14T16:53:18.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:17.992+0000 7f204259c700 1 --2- 172.21.15.2:0/1827539025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2044102440 0x7f204419f800 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f203400d950 tx=0x7f203400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:18.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:17.992+0000 7f203bfff700 1 -- 172.21.15.2:0/1827539025 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2034016070 con 0x7f2044102440 2024-05-14T16:53:18.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:17.993+0000 7f203bfff700 1 -- 172.21.15.2:0/1827539025 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f203400fd40 con 0x7f2044102440 2024-05-14T16:53:18.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:17.993+0000 7f2048d34700 1 -- 172.21.15.2:0/1827539025 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f204419a460 con 0x7f2044102440 2024-05-14T16:53:18.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:17.993+0000 7f203bfff700 1 -- 172.21.15.2:0/1827539025 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203400e3b0 con 0x7f2044102440 2024-05-14T16:53:18.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:17.993+0000 7f2048d34700 1 -- 172.21.15.2:0/1827539025 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20441a66d0 con 0x7f2044102440 2024-05-14T16:53:18.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:17.994+0000 7f2048d34700 1 -- 172.21.15.2:0/1827539025 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f204406a230 con 0x7f2044102440 2024-05-14T16:53:18.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:17.995+0000 7f203bfff700 1 -- 172.21.15.2:0/1827539025 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f2034019070 con 0x7f2044102440 2024-05-14T16:53:18.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:17.996+0000 7f203bfff700 1 --2- 172.21.15.2:0/1827539025 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f20300b2880 0x7f20300b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:18.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:17.996+0000 7f2039d9b700 1 --2- 172.21.15.2:0/1827539025 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f20300b2880 0x7f20300b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:18.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:17.996+0000 7f203bfff700 1 -- 172.21.15.2:0/1827539025 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f20340e1bc0 con 0x7f2044102440 2024-05-14T16:53:18.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:17.996+0000 7f2039d9b700 1 --2- 172.21.15.2:0/1827539025 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f20300b2880 0x7f20300b4d40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2044079b40 tx=0x7f202c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:18.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:17.999+0000 7f203bfff700 1 -- 172.21.15.2:0/1827539025 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f20340a98f0 con 0x7f2044102440 2024-05-14T16:53:18.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:18.216+0000 7f2048d34700 1 -- 172.21.15.2:0/1827539025 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f204402e560 con 0x7f2044102440 2024-05-14T16:53:18.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:18.216+0000 7f203bfff700 1 -- 172.21.15.2:0/1827539025 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f20340a91f0 con 0x7f2044102440 2024-05-14T16:53:18.216 INFO:teuthology.orchestra.run.smithi002.stdout:227633266740 2024-05-14T16:53:18.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:18.218+0000 7f2048d34700 1 -- 172.21.15.2:0/1827539025 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f20300b2880 msgr2=0x7f20300b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:18.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:18.218+0000 7f2048d34700 1 --2- 172.21.15.2:0/1827539025 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f20300b2880 0x7f20300b4d40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2044079b40 tx=0x7f202c00b410 comp rx=0 tx=0).stop 2024-05-14T16:53:18.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:18.218+0000 7f2048d34700 1 -- 172.21.15.2:0/1827539025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2044102440 msgr2=0x7f204419f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:18.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:18.218+0000 7f2048d34700 1 --2- 172.21.15.2:0/1827539025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2044102440 0x7f204419f800 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f203400d950 tx=0x7f203400dc60 comp rx=0 tx=0).stop 2024-05-14T16:53:18.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:18.218+0000 7f2048d34700 1 -- 172.21.15.2:0/1827539025 shutdown_connections 2024-05-14T16:53:18.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:18.218+0000 7f2048d34700 1 --2- 172.21.15.2:0/1827539025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2044108520 0x7f2044199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:18.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:18.218+0000 7f2048d34700 1 --2- 172.21.15.2:0/1827539025 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f20300b2880 0x7f20300b4d40 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:18.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:18.218+0000 7f2048d34700 1 --2- 172.21.15.2:0/1827539025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2044102440 0x7f204419f800 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:18.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:18.218+0000 7f2048d34700 1 --2- 172.21.15.2:0/1827539025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2044102e00 0x7f204419fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:18.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:18.218+0000 7f2048d34700 1 -- 172.21.15.2:0/1827539025 >> 172.21.15.2:0/1827539025 conn(0x7f20440793f0 msgr2=0x7f204406b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:18.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:18.219+0000 7f2048d34700 1 -- 172.21.15.2:0/1827539025 shutdown_connections 2024-05-14T16:53:18.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:18.219+0000 7f2048d34700 1 -- 172.21.15.2:0/1827539025 wait complete. 2024-05-14T16:53:18.485 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:18 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/1827539025' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-14T16:53:18.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:18 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/1827539025' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-14T16:53:18.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:18 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/1827539025' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-14T16:53:19.329 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:19 smithi002 ceph-mon[31538]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-14T16:53:19.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:19 smithi151 ceph-mon[36066]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-14T16:53:19.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:19 smithi123 ceph-mon[35971]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-14T16:53:20.135 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266735 got 227633266740 for osd.8 2024-05-14T16:53:20.136 DEBUG:teuthology.parallel:result is None 2024-05-14T16:53:20.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.340+0000 7f07eac17700 1 -- 172.21.15.2:0/4261704758 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07e406bbe0 msgr2=0x7f07e41103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:20.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.340+0000 7f07eac17700 1 --2- 172.21.15.2:0/4261704758 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07e406bbe0 0x7f07e41103d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f07dc00b0a0 tx=0x7f07dc00b3b0 comp rx=0 tx=0).stop 2024-05-14T16:53:20.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.340+0000 7f07eac17700 1 -- 172.21.15.2:0/4261704758 shutdown_connections 2024-05-14T16:53:20.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.340+0000 7f07eac17700 1 --2- 172.21.15.2:0/4261704758 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07e406bbe0 0x7f07e41103d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:20.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.340+0000 7f07eac17700 1 --2- 172.21.15.2:0/4261704758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07e41095b0 0x7f07e4109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:20.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.340+0000 7f07eac17700 1 --2- 172.21.15.2:0/4261704758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07e406b220 0x7f07e406b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:20.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.340+0000 7f07eac17700 1 -- 172.21.15.2:0/4261704758 >> 172.21.15.2:0/4261704758 conn(0x7f07e40793f0 msgr2=0x7f07e4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:20.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.340+0000 7f07eac17700 1 -- 172.21.15.2:0/4261704758 shutdown_connections 2024-05-14T16:53:20.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.340+0000 7f07eac17700 1 -- 172.21.15.2:0/4261704758 wait complete. 2024-05-14T16:53:20.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.341+0000 7f07eac17700 1 Processor -- start 2024-05-14T16:53:20.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.341+0000 7f07eac17700 1 -- start start 2024-05-14T16:53:20.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.341+0000 7f07eac17700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07e406b220 0x7f07e419f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:20.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.341+0000 7f07eac17700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07e406bbe0 0x7f07e419fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:20.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.341+0000 7f07eac17700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07e41095b0 0x7f07e41999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:20.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.341+0000 7f07eac17700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f07e41136e0 con 0x7f07e406bbe0 2024-05-14T16:53:20.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.341+0000 7f07eac17700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f07e4113560 con 0x7f07e41095b0 2024-05-14T16:53:20.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.341+0000 7f07eac17700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f07e4113860 con 0x7f07e406b220 2024-05-14T16:53:20.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.341+0000 7f07e3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07e406bbe0 0x7f07e419fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:20.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.341+0000 7f07e3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07e406bbe0 0x7f07e419fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36378/0 (socket says 172.21.15.2:36378) 2024-05-14T16:53:20.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.341+0000 7f07e3fff700 1 -- 172.21.15.2:0/3124348677 learned_addr learned my addr 172.21.15.2:0/3124348677 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:53:20.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.341+0000 7f07e91b4700 1 --2- 172.21.15.2:0/3124348677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07e41095b0 0x7f07e41999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:20.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.341+0000 7f07e89b3700 1 --2- 172.21.15.2:0/3124348677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07e406b220 0x7f07e419f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:20.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.342+0000 7f07e3fff700 1 -- 172.21.15.2:0/3124348677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07e406b220 msgr2=0x7f07e419f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:20.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.342+0000 7f07e3fff700 1 --2- 172.21.15.2:0/3124348677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07e406b220 0x7f07e419f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:20.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.342+0000 7f07e3fff700 1 -- 172.21.15.2:0/3124348677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07e41095b0 msgr2=0x7f07e41999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:20.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.342+0000 7f07e3fff700 1 --2- 172.21.15.2:0/3124348677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07e41095b0 0x7f07e41999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:20.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.342+0000 7f07e3fff700 1 -- 172.21.15.2:0/3124348677 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e419a120 con 0x7f07e406bbe0 2024-05-14T16:53:20.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.342+0000 7f07e3fff700 1 --2- 172.21.15.2:0/3124348677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07e406bbe0 0x7f07e419fd40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f07d000d980 tx=0x7f07d000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:20.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.342+0000 7f07e1ffb700 1 -- 172.21.15.2:0/3124348677 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d0017070 con 0x7f07e406bbe0 2024-05-14T16:53:20.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.342+0000 7f07e1ffb700 1 -- 172.21.15.2:0/3124348677 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07d0005020 con 0x7f07e406bbe0 2024-05-14T16:53:20.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.342+0000 7f07eac17700 1 -- 172.21.15.2:0/3124348677 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07e419a410 con 0x7f07e406bbe0 2024-05-14T16:53:20.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.342+0000 7f07e1ffb700 1 -- 172.21.15.2:0/3124348677 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d0009b50 con 0x7f07e406bbe0 2024-05-14T16:53:20.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.342+0000 7f07eac17700 1 -- 172.21.15.2:0/3124348677 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07e41a66d0 con 0x7f07e406bbe0 2024-05-14T16:53:20.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.343+0000 7f07eac17700 1 -- 172.21.15.2:0/3124348677 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07e406a230 con 0x7f07e406bbe0 2024-05-14T16:53:20.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.345+0000 7f07e1ffb700 1 -- 172.21.15.2:0/3124348677 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f07d00051d0 con 0x7f07e406bbe0 2024-05-14T16:53:20.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.345+0000 7f07e1ffb700 1 --2- 172.21.15.2:0/3124348677 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f07d40b2900 0x7f07d40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:20.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.345+0000 7f07e1ffb700 1 -- 172.21.15.2:0/3124348677 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f07d00e20e0 con 0x7f07e406bbe0 2024-05-14T16:53:20.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.345+0000 7f07e89b3700 1 --2- 172.21.15.2:0/3124348677 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f07d40b2900 0x7f07d40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:20.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.346+0000 7f07e89b3700 1 --2- 172.21.15.2:0/3124348677 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f07d40b2900 0x7f07d40b4dc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f07e419ab00 tx=0x7f07d800b5b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:20.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.350+0000 7f07e1ffb700 1 -- 172.21.15.2:0/3124348677 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f07d0020020 con 0x7f07e406bbe0 2024-05-14T16:53:20.389 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:20 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/638011896' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-14T16:53:20.390 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:20 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/2133031756' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-14T16:53:20.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.567+0000 7f07eac17700 1 -- 172.21.15.2:0/3124348677 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f07e402e560 con 0x7f07e406bbe0 2024-05-14T16:53:20.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.568+0000 7f07e1ffb700 1 -- 172.21.15.2:0/3124348677 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f07d00a9e10 con 0x7f07e406bbe0 2024-05-14T16:53:20.568 INFO:teuthology.orchestra.run.smithi002.stdout:55834574936 2024-05-14T16:53:20.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.570+0000 7f07eac17700 1 -- 172.21.15.2:0/3124348677 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f07d40b2900 msgr2=0x7f07d40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:20.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.570+0000 7f07eac17700 1 --2- 172.21.15.2:0/3124348677 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f07d40b2900 0x7f07d40b4dc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f07e419ab00 tx=0x7f07d800b5b0 comp rx=0 tx=0).stop 2024-05-14T16:53:20.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.570+0000 7f07eac17700 1 -- 172.21.15.2:0/3124348677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07e406bbe0 msgr2=0x7f07e419fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:20.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.570+0000 7f07eac17700 1 --2- 172.21.15.2:0/3124348677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07e406bbe0 0x7f07e419fd40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f07d000d980 tx=0x7f07d000dc90 comp rx=0 tx=0).stop 2024-05-14T16:53:20.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.570+0000 7f07eac17700 1 -- 172.21.15.2:0/3124348677 shutdown_connections 2024-05-14T16:53:20.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.570+0000 7f07eac17700 1 --2- 172.21.15.2:0/3124348677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07e41095b0 0x7f07e41999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:20.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.570+0000 7f07eac17700 1 --2- 172.21.15.2:0/3124348677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07e406b220 0x7f07e419f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:20.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.570+0000 7f07eac17700 1 --2- 172.21.15.2:0/3124348677 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f07d40b2900 0x7f07d40b4dc0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:20.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.570+0000 7f07eac17700 1 --2- 172.21.15.2:0/3124348677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07e406bbe0 0x7f07e419fd40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:20.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.570+0000 7f07eac17700 1 -- 172.21.15.2:0/3124348677 >> 172.21.15.2:0/3124348677 conn(0x7f07e40793f0 msgr2=0x7f07e4101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:20.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.570+0000 7f07eac17700 1 -- 172.21.15.2:0/3124348677 shutdown_connections 2024-05-14T16:53:20.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:20.570+0000 7f07eac17700 1 -- 172.21.15.2:0/3124348677 wait complete. 2024-05-14T16:53:20.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:20 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/638011896' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-14T16:53:20.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:20 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/2133031756' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-14T16:53:20.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:20 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/638011896' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-14T16:53:20.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:20 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/2133031756' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-14T16:53:21.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.567+0000 7f4649494700 1 -- 172.21.15.2:0/2235268678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4644102930 msgr2=0x7f4644110430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:21.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.567+0000 7f4649494700 1 --2- 172.21.15.2:0/2235268678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4644102930 0x7f4644110430 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f463c009990 tx=0x7f463c009ca0 comp rx=0 tx=0).stop 2024-05-14T16:53:21.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.567+0000 7f4649494700 1 -- 172.21.15.2:0/2235268678 shutdown_connections 2024-05-14T16:53:21.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.567+0000 7f4649494700 1 --2- 172.21.15.2:0/2235268678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4644102930 0x7f4644110430 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:21.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.567+0000 7f4649494700 1 --2- 172.21.15.2:0/2235268678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4644109720 0x7f4644109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:21.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.567+0000 7f4649494700 1 --2- 172.21.15.2:0/2235268678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4644101f70 0x7f46441023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:21.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.567+0000 7f4649494700 1 -- 172.21.15.2:0/2235268678 >> 172.21.15.2:0/2235268678 conn(0x7f46440776b0 msgr2=0x7f4644077ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:21.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.567+0000 7f4649494700 1 -- 172.21.15.2:0/2235268678 shutdown_connections 2024-05-14T16:53:21.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.567+0000 7f4649494700 1 -- 172.21.15.2:0/2235268678 wait complete. 2024-05-14T16:53:21.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.568+0000 7f4649494700 1 Processor -- start 2024-05-14T16:53:21.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.568+0000 7f4649494700 1 -- start start 2024-05-14T16:53:21.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.568+0000 7f4649494700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4644101f70 0x7f464419f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:21.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.568+0000 7f4649494700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4644102930 0x7f464419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:21.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.568+0000 7f4649494700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4644109720 0x7f4644199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:21.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.568+0000 7f4649494700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4644113750 con 0x7f4644101f70 2024-05-14T16:53:21.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.568+0000 7f4649494700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f46441135d0 con 0x7f4644109720 2024-05-14T16:53:21.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.568+0000 7f4649494700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f46441138d0 con 0x7f4644102930 2024-05-14T16:53:21.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.568+0000 7f4642ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4644101f70 0x7f464419f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:21.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.568+0000 7f4642ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4644101f70 0x7f464419f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36360/0 (socket says 172.21.15.2:36360) 2024-05-14T16:53:21.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.568+0000 7f4642ffd700 1 -- 172.21.15.2:0/2133031756 learned_addr learned my addr 172.21.15.2:0/2133031756 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:53:21.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.568+0000 7f46437fe700 1 --2- 172.21.15.2:0/2133031756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4644109720 0x7f4644199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:21.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.569+0000 7f46427fc700 1 --2- 172.21.15.2:0/2133031756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4644102930 0x7f464419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:21.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.569+0000 7f4642ffd700 1 -- 172.21.15.2:0/2133031756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4644102930 msgr2=0x7f464419fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:21.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.569+0000 7f4642ffd700 1 --2- 172.21.15.2:0/2133031756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4644102930 0x7f464419fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:21.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.569+0000 7f4642ffd700 1 -- 172.21.15.2:0/2133031756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4644109720 msgr2=0x7f4644199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:21.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.569+0000 7f4642ffd700 1 --2- 172.21.15.2:0/2133031756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4644109720 0x7f4644199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:21.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.569+0000 7f4642ffd700 1 -- 172.21.15.2:0/2133031756 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f464419a0d0 con 0x7f4644101f70 2024-05-14T16:53:21.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.569+0000 7f4642ffd700 1 --2- 172.21.15.2:0/2133031756 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4644101f70 0x7f464419f760 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4634007f10 tx=0x7f463400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:21.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.569+0000 7f463bfff700 1 -- 172.21.15.2:0/2133031756 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463400dbd0 con 0x7f4644101f70 2024-05-14T16:53:21.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.569+0000 7f463bfff700 1 -- 172.21.15.2:0/2133031756 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f463400dd70 con 0x7f4644101f70 2024-05-14T16:53:21.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.569+0000 7f4649494700 1 -- 172.21.15.2:0/2133031756 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f464419a3c0 con 0x7f4644101f70 2024-05-14T16:53:21.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.569+0000 7f463bfff700 1 -- 172.21.15.2:0/2133031756 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4634005a10 con 0x7f4644101f70 2024-05-14T16:53:21.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.569+0000 7f4649494700 1 -- 172.21.15.2:0/2133031756 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46441a6630 con 0x7f4644101f70 2024-05-14T16:53:21.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.570+0000 7f4649494700 1 -- 172.21.15.2:0/2133031756 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f464406a230 con 0x7f4644101f70 2024-05-14T16:53:21.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.572+0000 7f463bfff700 1 -- 172.21.15.2:0/2133031756 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f463401a020 con 0x7f4644101f70 2024-05-14T16:53:21.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.573+0000 7f463bfff700 1 --2- 172.21.15.2:0/2133031756 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f46300b2900 0x7f46300b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:21.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.573+0000 7f46427fc700 1 --2- 172.21.15.2:0/2133031756 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f46300b2900 0x7f46300b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:21.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.573+0000 7f463bfff700 1 -- 172.21.15.2:0/2133031756 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f46340e1fc0 con 0x7f4644101f70 2024-05-14T16:53:21.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.573+0000 7f46427fc700 1 --2- 172.21.15.2:0/2133031756 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f46300b2900 0x7f46300b4dc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f462c005fd0 tx=0x7f462c00a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:21.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.577+0000 7f463bfff700 1 -- 172.21.15.2:0/2133031756 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4634010070 con 0x7f4644101f70 2024-05-14T16:53:21.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.797+0000 7f4649494700 1 -- 172.21.15.2:0/2133031756 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f464402e560 con 0x7f4644101f70 2024-05-14T16:53:21.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.798+0000 7f463bfff700 1 -- 172.21.15.2:0/2133031756 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f46340a9cf0 con 0x7f4644101f70 2024-05-14T16:53:21.294 INFO:teuthology.orchestra.run.smithi002.stdout:77309411412 2024-05-14T16:53:21.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.800+0000 7f4649494700 1 -- 172.21.15.2:0/2133031756 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f46300b2900 msgr2=0x7f46300b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:21.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.800+0000 7f4649494700 1 --2- 172.21.15.2:0/2133031756 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f46300b2900 0x7f46300b4dc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f462c005fd0 tx=0x7f462c00a560 comp rx=0 tx=0).stop 2024-05-14T16:53:21.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.800+0000 7f4649494700 1 -- 172.21.15.2:0/2133031756 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4644101f70 msgr2=0x7f464419f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:21.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.800+0000 7f4649494700 1 --2- 172.21.15.2:0/2133031756 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4644101f70 0x7f464419f760 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4634007f10 tx=0x7f463400d3b0 comp rx=0 tx=0).stop 2024-05-14T16:53:21.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.800+0000 7f4649494700 1 -- 172.21.15.2:0/2133031756 shutdown_connections 2024-05-14T16:53:21.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.800+0000 7f4649494700 1 --2- 172.21.15.2:0/2133031756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4644109720 0x7f4644199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:21.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.800+0000 7f4649494700 1 --2- 172.21.15.2:0/2133031756 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f46300b2900 0x7f46300b4dc0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:21.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.800+0000 7f4649494700 1 --2- 172.21.15.2:0/2133031756 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4644101f70 0x7f464419f760 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:21.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.800+0000 7f4649494700 1 --2- 172.21.15.2:0/2133031756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4644102930 0x7f464419fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:21.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.800+0000 7f4649494700 1 -- 172.21.15.2:0/2133031756 >> 172.21.15.2:0/2133031756 conn(0x7f46440776b0 msgr2=0x7f4644100a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:21.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.800+0000 7f4649494700 1 -- 172.21.15.2:0/2133031756 shutdown_connections 2024-05-14T16:53:21.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.800+0000 7f4649494700 1 -- 172.21.15.2:0/2133031756 wait complete. 2024-05-14T16:53:21.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:21 smithi002 ceph-mon[31538]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-14T16:53:21.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:21 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/3124348677' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-14T16:53:21.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:21 smithi151 ceph-mon[36066]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-14T16:53:21.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:21 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/3124348677' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-14T16:53:21.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:21 smithi123 ceph-mon[35971]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-14T16:53:21.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:21 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/3124348677' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-14T16:53:22.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.506+0000 7f55f69f0700 1 -- 172.21.15.2:0/969631647 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55f01041a0 msgr2=0x7f55f01081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:22.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.506+0000 7f55f69f0700 1 --2- 172.21.15.2:0/969631647 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55f01041a0 0x7f55f01081c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f55e40099c0 tx=0x7f55e4009cd0 comp rx=0 tx=0).stop 2024-05-14T16:53:22.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.506+0000 7f55f69f0700 1 -- 172.21.15.2:0/969631647 shutdown_connections 2024-05-14T16:53:22.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.506+0000 7f55f69f0700 1 --2- 172.21.15.2:0/969631647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55f01088f0 0x7f55f0110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:22.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.506+0000 7f55f69f0700 1 --2- 172.21.15.2:0/969631647 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55f01037f0 0x7f55f0103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:22.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.506+0000 7f55f69f0700 1 --2- 172.21.15.2:0/969631647 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55f01041a0 0x7f55f01081c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:22.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.506+0000 7f55f69f0700 1 -- 172.21.15.2:0/969631647 >> 172.21.15.2:0/969631647 conn(0x7f55f007a6a0 msgr2=0x7f55f007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:22.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.506+0000 7f55f69f0700 1 -- 172.21.15.2:0/969631647 shutdown_connections 2024-05-14T16:53:22.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.506+0000 7f55f69f0700 1 -- 172.21.15.2:0/969631647 wait complete. 2024-05-14T16:53:22.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.507+0000 7f55f69f0700 1 Processor -- start 2024-05-14T16:53:22.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.507+0000 7f55f69f0700 1 -- start start 2024-05-14T16:53:22.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.507+0000 7f55f69f0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55f01037f0 0x7f55f019f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:22.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.507+0000 7f55f69f0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55f01041a0 0x7f55f019fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:22.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.507+0000 7f55f69f0700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55f01088f0 0x7f55f01999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:22.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.507+0000 7f55f69f0700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f55f0113700 con 0x7f55f01037f0 2024-05-14T16:53:22.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.507+0000 7f55f69f0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f55f0113580 con 0x7f55f01041a0 2024-05-14T16:53:22.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.507+0000 7f55f69f0700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f55f0113880 con 0x7f55f01088f0 2024-05-14T16:53:22.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.507+0000 7f55effff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55f01037f0 0x7f55f019f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:22.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.507+0000 7f55effff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55f01037f0 0x7f55f019f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36398/0 (socket says 172.21.15.2:36398) 2024-05-14T16:53:22.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.507+0000 7f55effff700 1 -- 172.21.15.2:0/566740517 learned_addr learned my addr 172.21.15.2:0/566740517 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:53:22.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.508+0000 7f55ef7fe700 1 --2- 172.21.15.2:0/566740517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55f01041a0 0x7f55f019fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:22.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.508+0000 7f55f4f8d700 1 --2- 172.21.15.2:0/566740517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55f01088f0 0x7f55f01999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:22.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.508+0000 7f55effff700 1 -- 172.21.15.2:0/566740517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55f01088f0 msgr2=0x7f55f01999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:22.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.508+0000 7f55effff700 1 --2- 172.21.15.2:0/566740517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55f01088f0 0x7f55f01999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:22.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.508+0000 7f55effff700 1 -- 172.21.15.2:0/566740517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55f01041a0 msgr2=0x7f55f019fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:22.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.508+0000 7f55effff700 1 --2- 172.21.15.2:0/566740517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55f01041a0 0x7f55f019fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:22.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.508+0000 7f55effff700 1 -- 172.21.15.2:0/566740517 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f019a140 con 0x7f55f01037f0 2024-05-14T16:53:22.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.508+0000 7f55f4f8d700 1 --2- 172.21.15.2:0/566740517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55f01088f0 0x7f55f01999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:53:22.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.508+0000 7f55effff700 1 --2- 172.21.15.2:0/566740517 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55f01037f0 0x7f55f019f820 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f55e000d950 tx=0x7f55e000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:22.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.508+0000 7f55ed7fa700 1 -- 172.21.15.2:0/566740517 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e0016070 con 0x7f55f01037f0 2024-05-14T16:53:22.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.508+0000 7f55ed7fa700 1 -- 172.21.15.2:0/566740517 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55e0009df0 con 0x7f55f01037f0 2024-05-14T16:53:22.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.508+0000 7f55ed7fa700 1 -- 172.21.15.2:0/566740517 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e000f3c0 con 0x7f55f01037f0 2024-05-14T16:53:22.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.508+0000 7f55f69f0700 1 -- 172.21.15.2:0/566740517 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55f019a430 con 0x7f55f01037f0 2024-05-14T16:53:22.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.508+0000 7f55f69f0700 1 -- 172.21.15.2:0/566740517 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55f01a66f0 con 0x7f55f01037f0 2024-05-14T16:53:22.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.509+0000 7f55f69f0700 1 -- 172.21.15.2:0/566740517 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55f006a230 con 0x7f55f01037f0 2024-05-14T16:53:22.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.510+0000 7f55ed7fa700 1 -- 172.21.15.2:0/566740517 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f55e0019070 con 0x7f55f01037f0 2024-05-14T16:53:22.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.511+0000 7f55ed7fa700 1 --2- 172.21.15.2:0/566740517 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f55d80b2880 0x7f55d80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:22.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.511+0000 7f55ef7fe700 1 --2- 172.21.15.2:0/566740517 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f55d80b2880 0x7f55d80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:22.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.511+0000 7f55ed7fa700 1 -- 172.21.15.2:0/566740517 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f55e00e1df0 con 0x7f55f01037f0 2024-05-14T16:53:22.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.512+0000 7f55ef7fe700 1 --2- 172.21.15.2:0/566740517 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f55d80b2880 0x7f55d80b4d40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f55e400b580 tx=0x7f55e4005a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:22.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.515+0000 7f55ed7fa700 1 -- 172.21.15.2:0/566740517 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f55e00a9b20 con 0x7f55f01037f0 2024-05-14T16:53:22.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.730+0000 7f55f69f0700 1 -- 172.21.15.2:0/566740517 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f55f002e560 con 0x7f55f01037f0 2024-05-14T16:53:22.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.730+0000 7f55ed7fa700 1 -- 172.21.15.2:0/566740517 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f55e00a9420 con 0x7f55f01037f0 2024-05-14T16:53:22.731 INFO:teuthology.orchestra.run.smithi002.stdout:253403070511 2024-05-14T16:53:22.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.733+0000 7f55f69f0700 1 -- 172.21.15.2:0/566740517 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f55d80b2880 msgr2=0x7f55d80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:22.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.733+0000 7f55f69f0700 1 --2- 172.21.15.2:0/566740517 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f55d80b2880 0x7f55d80b4d40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f55e400b580 tx=0x7f55e4005a90 comp rx=0 tx=0).stop 2024-05-14T16:53:22.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.733+0000 7f55f69f0700 1 -- 172.21.15.2:0/566740517 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55f01037f0 msgr2=0x7f55f019f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:22.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.733+0000 7f55f69f0700 1 --2- 172.21.15.2:0/566740517 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55f01037f0 0x7f55f019f820 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f55e000d950 tx=0x7f55e000dc60 comp rx=0 tx=0).stop 2024-05-14T16:53:22.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.733+0000 7f55f69f0700 1 -- 172.21.15.2:0/566740517 shutdown_connections 2024-05-14T16:53:22.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.733+0000 7f55f69f0700 1 --2- 172.21.15.2:0/566740517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55f01088f0 0x7f55f01999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:22.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.733+0000 7f55f69f0700 1 --2- 172.21.15.2:0/566740517 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f55d80b2880 0x7f55d80b4d40 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:22.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.733+0000 7f55f69f0700 1 --2- 172.21.15.2:0/566740517 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55f01037f0 0x7f55f019f820 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:22.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.733+0000 7f55f69f0700 1 --2- 172.21.15.2:0/566740517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55f01041a0 0x7f55f019fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:22.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.733+0000 7f55f69f0700 1 -- 172.21.15.2:0/566740517 >> 172.21.15.2:0/566740517 conn(0x7f55f007a6a0 msgr2=0x7f55f0101680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:22.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.733+0000 7f55f69f0700 1 -- 172.21.15.2:0/566740517 shutdown_connections 2024-05-14T16:53:22.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.733+0000 7f55f69f0700 1 -- 172.21.15.2:0/566740517 wait complete. 2024-05-14T16:53:22.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.749+0000 7f70bb830700 1 -- 172.21.15.2:0/419016570 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70b406b220 msgr2=0x7f70b406b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:22.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.749+0000 7f70bb830700 1 --2- 172.21.15.2:0/419016570 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70b406b220 0x7f70b406b6a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f70a4009990 tx=0x7f70a4009ca0 comp rx=0 tx=0).stop 2024-05-14T16:53:22.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.749+0000 7f70bb830700 1 -- 172.21.15.2:0/419016570 shutdown_connections 2024-05-14T16:53:22.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.749+0000 7f70bb830700 1 --2- 172.21.15.2:0/419016570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70b406bbe0 0x7f70b41103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:22.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.749+0000 7f70bb830700 1 --2- 172.21.15.2:0/419016570 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70b41095b0 0x7f70b4109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:22.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.749+0000 7f70bb830700 1 --2- 172.21.15.2:0/419016570 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70b406b220 0x7f70b406b6a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:22.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.749+0000 7f70bb830700 1 -- 172.21.15.2:0/419016570 >> 172.21.15.2:0/419016570 conn(0x7f70b40793f0 msgr2=0x7f70b4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:22.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.750+0000 7f70bb830700 1 -- 172.21.15.2:0/419016570 shutdown_connections 2024-05-14T16:53:22.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.750+0000 7f70bb830700 1 -- 172.21.15.2:0/419016570 wait complete. 2024-05-14T16:53:22.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.750+0000 7f70bb830700 1 Processor -- start 2024-05-14T16:53:22.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.750+0000 7f70bb830700 1 -- start start 2024-05-14T16:53:22.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.751+0000 7f70bb830700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70b406b220 0x7f70b419f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:22.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.751+0000 7f70bb830700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70b406bbe0 0x7f70b419fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:22.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.751+0000 7f70bb830700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70b41095b0 0x7f70b4199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:22.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.751+0000 7f70bb830700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f70b41136e0 con 0x7f70b406bbe0 2024-05-14T16:53:22.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.751+0000 7f70bb830700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f70b4113560 con 0x7f70b406b220 2024-05-14T16:53:22.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.751+0000 7f70bb830700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f70b4113860 con 0x7f70b41095b0 2024-05-14T16:53:22.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.751+0000 7f70b8dcb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70b406bbe0 0x7f70b419fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:22.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.751+0000 7f70b8dcb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70b406bbe0 0x7f70b419fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36414/0 (socket says 172.21.15.2:36414) 2024-05-14T16:53:22.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.751+0000 7f70b8dcb700 1 -- 172.21.15.2:0/1196231087 learned_addr learned my addr 172.21.15.2:0/1196231087 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:53:22.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.751+0000 7f70b9dcd700 1 --2- 172.21.15.2:0/1196231087 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70b41095b0 0x7f70b4199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:22.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.751+0000 7f70b95cc700 1 --2- 172.21.15.2:0/1196231087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70b406b220 0x7f70b419f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:22.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.751+0000 7f70b8dcb700 1 -- 172.21.15.2:0/1196231087 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70b41095b0 msgr2=0x7f70b4199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:22.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.751+0000 7f70b8dcb700 1 --2- 172.21.15.2:0/1196231087 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70b41095b0 0x7f70b4199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:22.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.751+0000 7f70b8dcb700 1 -- 172.21.15.2:0/1196231087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70b406b220 msgr2=0x7f70b419f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:22.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.751+0000 7f70b8dcb700 1 --2- 172.21.15.2:0/1196231087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70b406b220 0x7f70b419f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:22.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.751+0000 7f70b8dcb700 1 -- 172.21.15.2:0/1196231087 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70b419a180 con 0x7f70b406bbe0 2024-05-14T16:53:22.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.751+0000 7f70b8dcb700 1 --2- 172.21.15.2:0/1196231087 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70b406bbe0 0x7f70b419fda0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f70a4000c00 tx=0x7f70a40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:22.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.752+0000 7f70aa7fc700 1 -- 172.21.15.2:0/1196231087 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a4027070 con 0x7f70b406bbe0 2024-05-14T16:53:22.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.752+0000 7f70aa7fc700 1 -- 172.21.15.2:0/1196231087 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70a4003fa0 con 0x7f70b406bbe0 2024-05-14T16:53:22.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.752+0000 7f70bb830700 1 -- 172.21.15.2:0/1196231087 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70b419a410 con 0x7f70b406bbe0 2024-05-14T16:53:22.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.752+0000 7f70aa7fc700 1 -- 172.21.15.2:0/1196231087 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a4022db0 con 0x7f70b406bbe0 2024-05-14T16:53:22.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.752+0000 7f70bb830700 1 -- 172.21.15.2:0/1196231087 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70b41a6730 con 0x7f70b406bbe0 2024-05-14T16:53:22.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.753+0000 7f70bb830700 1 -- 172.21.15.2:0/1196231087 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70b410ba90 con 0x7f70b406bbe0 2024-05-14T16:53:22.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.754+0000 7f70aa7fc700 1 -- 172.21.15.2:0/1196231087 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f70a402a070 con 0x7f70b406bbe0 2024-05-14T16:53:22.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.755+0000 7f70aa7fc700 1 --2- 172.21.15.2:0/1196231087 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f70a00b2880 0x7f70a00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:22.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.755+0000 7f70aa7fc700 1 -- 172.21.15.2:0/1196231087 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f70a40e3110 con 0x7f70b406bbe0 2024-05-14T16:53:22.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.755+0000 7f70b95cc700 1 --2- 172.21.15.2:0/1196231087 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f70a00b2880 0x7f70a00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:22.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.756+0000 7f70b95cc700 1 --2- 172.21.15.2:0/1196231087 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f70a00b2880 0x7f70a00b4d40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f70b0005e10 tx=0x7f70b0005d40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:22.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.763+0000 7f70aa7fc700 1 -- 172.21.15.2:0/1196231087 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f70a4013070 con 0x7f70b406bbe0 2024-05-14T16:53:22.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.979+0000 7f70bb830700 1 -- 172.21.15.2:0/1196231087 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f70b419af50 con 0x7f70b406bbe0 2024-05-14T16:53:22.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.980+0000 7f70aa7fc700 1 -- 172.21.15.2:0/1196231087 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f70a40aaed0 con 0x7f70b406bbe0 2024-05-14T16:53:22.981 INFO:teuthology.orchestra.run.smithi002.stdout:274877906987 2024-05-14T16:53:22.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.982+0000 7f70bb830700 1 -- 172.21.15.2:0/1196231087 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f70a00b2880 msgr2=0x7f70a00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:22.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.982+0000 7f70bb830700 1 --2- 172.21.15.2:0/1196231087 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f70a00b2880 0x7f70a00b4d40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f70b0005e10 tx=0x7f70b0005d40 comp rx=0 tx=0).stop 2024-05-14T16:53:22.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.982+0000 7f70bb830700 1 -- 172.21.15.2:0/1196231087 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70b406bbe0 msgr2=0x7f70b419fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:22.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.982+0000 7f70bb830700 1 --2- 172.21.15.2:0/1196231087 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70b406bbe0 0x7f70b419fda0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f70a4000c00 tx=0x7f70a40047b0 comp rx=0 tx=0).stop 2024-05-14T16:53:22.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.982+0000 7f70bb830700 1 -- 172.21.15.2:0/1196231087 shutdown_connections 2024-05-14T16:53:22.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.982+0000 7f70bb830700 1 --2- 172.21.15.2:0/1196231087 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70b41095b0 0x7f70b4199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:22.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.983+0000 7f70bb830700 1 --2- 172.21.15.2:0/1196231087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70b406b220 0x7f70b419f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:22.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.983+0000 7f70bb830700 1 --2- 172.21.15.2:0/1196231087 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f70a00b2880 0x7f70a00b4d40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:22.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.983+0000 7f70bb830700 1 --2- 172.21.15.2:0/1196231087 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70b406bbe0 0x7f70b419fda0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:22.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.983+0000 7f70bb830700 1 -- 172.21.15.2:0/1196231087 >> 172.21.15.2:0/1196231087 conn(0x7f70b40793f0 msgr2=0x7f70b41011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:22.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.983+0000 7f70bb830700 1 -- 172.21.15.2:0/1196231087 shutdown_connections 2024-05-14T16:53:22.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:22.983+0000 7f70bb830700 1 -- 172.21.15.2:0/1196231087 wait complete. 2024-05-14T16:53:23.407 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574930 got 55834574936 for osd.1 2024-05-14T16:53:23.407 DEBUG:teuthology.parallel:result is None 2024-05-14T16:53:23.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:23 smithi002 ceph-mon[31538]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-14T16:53:23.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:23 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/566740517' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-14T16:53:23.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:23 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/1196231087' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-14T16:53:23.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:23 smithi151 ceph-mon[36066]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-14T16:53:23.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:23 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/566740517' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-14T16:53:23.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:23 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/1196231087' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-14T16:53:23.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:23 smithi123 ceph-mon[35971]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-14T16:53:23.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:23 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/566740517' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-14T16:53:23.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:23 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/1196231087' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-14T16:53:23.780 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070506 got 253403070511 for osd.9 2024-05-14T16:53:23.780 DEBUG:teuthology.parallel:result is None 2024-05-14T16:53:23.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.029+0000 7f6a1d859700 1 -- 172.21.15.2:0/3971728629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a1810ff70 msgr2=0x7f6a18112360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:23.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.029+0000 7f6a1d859700 1 --2- 172.21.15.2:0/3971728629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a1810ff70 0x7f6a18112360 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6a10009990 tx=0x7f6a10009ca0 comp rx=0 tx=0).stop 2024-05-14T16:53:23.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.029+0000 7f6a1d859700 1 -- 172.21.15.2:0/3971728629 shutdown_connections 2024-05-14T16:53:23.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.029+0000 7f6a1d859700 1 --2- 172.21.15.2:0/3971728629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a1810ff70 0x7f6a18112360 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:23.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.029+0000 7f6a1d859700 1 --2- 172.21.15.2:0/3971728629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a18075770 0x7f6a18075b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:23.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.029+0000 7f6a1d859700 1 --2- 172.21.15.2:0/3971728629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a18076090 0x7f6a1810fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:23.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.029+0000 7f6a1d859700 1 -- 172.21.15.2:0/3971728629 >> 172.21.15.2:0/3971728629 conn(0x7f6a1807a610 msgr2=0x7f6a1807aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:23.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.029+0000 7f6a1d859700 1 -- 172.21.15.2:0/3971728629 shutdown_connections 2024-05-14T16:53:23.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.029+0000 7f6a1d859700 1 -- 172.21.15.2:0/3971728629 wait complete. 2024-05-14T16:53:23.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.030+0000 7f6a1d859700 1 Processor -- start 2024-05-14T16:53:23.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.030+0000 7f6a1d859700 1 -- start start 2024-05-14T16:53:23.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.030+0000 7f6a1d859700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a18075770 0x7f6a1819bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:23.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.030+0000 7f6a1d859700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a18076090 0x7f6a1819c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:23.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.030+0000 7f6a1d859700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a1810ff70 0x7f6a181a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:23.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.030+0000 7f6a1d859700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6a18115910 con 0x7f6a1810ff70 2024-05-14T16:53:23.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.030+0000 7f6a1d859700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6a18115790 con 0x7f6a18075770 2024-05-14T16:53:23.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.030+0000 7f6a1d859700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6a18115a90 con 0x7f6a18076090 2024-05-14T16:53:23.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.030+0000 7f6a177fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a1810ff70 0x7f6a181a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:23.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.030+0000 7f6a177fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a1810ff70 0x7f6a181a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36340/0 (socket says 172.21.15.2:36340) 2024-05-14T16:53:23.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.030+0000 7f6a177fe700 1 -- 172.21.15.2:0/638011896 learned_addr learned my addr 172.21.15.2:0/638011896 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:53:23.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.030+0000 7f6a16ffd700 1 --2- 172.21.15.2:0/638011896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a18075770 0x7f6a1819bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:23.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.031+0000 7f6a167fc700 1 --2- 172.21.15.2:0/638011896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a18076090 0x7f6a1819c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:23.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.031+0000 7f6a177fe700 1 -- 172.21.15.2:0/638011896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a18076090 msgr2=0x7f6a1819c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:23.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.031+0000 7f6a177fe700 1 --2- 172.21.15.2:0/638011896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a18076090 0x7f6a1819c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:23.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.031+0000 7f6a177fe700 1 -- 172.21.15.2:0/638011896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a18075770 msgr2=0x7f6a1819bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:23.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.031+0000 7f6a177fe700 1 --2- 172.21.15.2:0/638011896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a18075770 0x7f6a1819bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:23.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.031+0000 7f6a177fe700 1 -- 172.21.15.2:0/638011896 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a181a0d40 con 0x7f6a1810ff70 2024-05-14T16:53:23.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.031+0000 7f6a167fc700 1 --2- 172.21.15.2:0/638011896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a18076090 0x7f6a1819c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:53:23.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.031+0000 7f6a177fe700 1 --2- 172.21.15.2:0/638011896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a1810ff70 0x7f6a181a0600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6a100059d0 tx=0x7f6a10009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:23.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.031+0000 7f6a1c857700 1 -- 172.21.15.2:0/638011896 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a10026070 con 0x7f6a1810ff70 2024-05-14T16:53:23.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.031+0000 7f6a1d859700 1 -- 172.21.15.2:0/638011896 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a181a0fd0 con 0x7f6a1810ff70 2024-05-14T16:53:23.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.031+0000 7f6a1c857700 1 -- 172.21.15.2:0/638011896 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a10004ca0 con 0x7f6a1810ff70 2024-05-14T16:53:23.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.031+0000 7f6a1c857700 1 -- 172.21.15.2:0/638011896 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a10003cd0 con 0x7f6a1810ff70 2024-05-14T16:53:23.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.031+0000 7f6a1d859700 1 -- 172.21.15.2:0/638011896 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a181a88b0 con 0x7f6a1810ff70 2024-05-14T16:53:23.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.032+0000 7f6a1d859700 1 -- 172.21.15.2:0/638011896 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a1806a230 con 0x7f6a1810ff70 2024-05-14T16:53:23.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.034+0000 7f6a1c857700 1 -- 172.21.15.2:0/638011896 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f6a10029070 con 0x7f6a1810ff70 2024-05-14T16:53:23.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.034+0000 7f6a1c857700 1 --2- 172.21.15.2:0/638011896 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f6a000b2880 0x7f6a000b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:23.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.034+0000 7f6a16ffd700 1 --2- 172.21.15.2:0/638011896 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f6a000b2880 0x7f6a000b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:23.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.034+0000 7f6a1c857700 1 -- 172.21.15.2:0/638011896 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f6a100e2e40 con 0x7f6a1810ff70 2024-05-14T16:53:23.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.035+0000 7f6a16ffd700 1 --2- 172.21.15.2:0/638011896 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f6a000b2880 0x7f6a000b4d40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6a08009550 tx=0x7f6a080091b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:23.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.038+0000 7f6a1c857700 1 -- 172.21.15.2:0/638011896 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6a10030020 con 0x7f6a1810ff70 2024-05-14T16:53:23.854 INFO:teuthology.orchestra.run.smithi002.stdout:107374182478 2024-05-14T16:53:23.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.271+0000 7f6a1d859700 1 -- 172.21.15.2:0/638011896 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f6a1819d090 con 0x7f6a1810ff70 2024-05-14T16:53:23.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.272+0000 7f6a1c857700 1 -- 172.21.15.2:0/638011896 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6a10013070 con 0x7f6a1810ff70 2024-05-14T16:53:23.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.274+0000 7f6a1d859700 1 -- 172.21.15.2:0/638011896 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f6a000b2880 msgr2=0x7f6a000b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:23.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.274+0000 7f6a1d859700 1 --2- 172.21.15.2:0/638011896 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f6a000b2880 0x7f6a000b4d40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6a08009550 tx=0x7f6a080091b0 comp rx=0 tx=0).stop 2024-05-14T16:53:23.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.274+0000 7f6a1d859700 1 -- 172.21.15.2:0/638011896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a1810ff70 msgr2=0x7f6a181a0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:23.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.274+0000 7f6a1d859700 1 --2- 172.21.15.2:0/638011896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a1810ff70 0x7f6a181a0600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6a100059d0 tx=0x7f6a10009580 comp rx=0 tx=0).stop 2024-05-14T16:53:23.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.274+0000 7f6a1d859700 1 -- 172.21.15.2:0/638011896 shutdown_connections 2024-05-14T16:53:23.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.274+0000 7f6a1d859700 1 --2- 172.21.15.2:0/638011896 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f6a000b2880 0x7f6a000b4d40 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:23.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.274+0000 7f6a1d859700 1 --2- 172.21.15.2:0/638011896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a1810ff70 0x7f6a181a0600 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:23.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.274+0000 7f6a1d859700 1 --2- 172.21.15.2:0/638011896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a18075770 0x7f6a1819bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:23.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.274+0000 7f6a1d859700 1 --2- 172.21.15.2:0/638011896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a18076090 0x7f6a1819c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:23.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.274+0000 7f6a1d859700 1 -- 172.21.15.2:0/638011896 >> 172.21.15.2:0/638011896 conn(0x7f6a1807a610 msgr2=0x7f6a18100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:23.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.274+0000 7f6a1d859700 1 -- 172.21.15.2:0/638011896 shutdown_connections 2024-05-14T16:53:23.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:19.274+0000 7f6a1d859700 1 -- 172.21.15.2:0/638011896 wait complete. 2024-05-14T16:53:24.466 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182473 got 107374182478 for osd.3 2024-05-14T16:53:24.466 DEBUG:teuthology.parallel:result is None 2024-05-14T16:53:25.490 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906984 got 274877906987 for osd.10 2024-05-14T16:53:25.490 DEBUG:teuthology.parallel:result is None 2024-05-14T16:53:25.560 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:25 smithi002 ceph-mon[31538]: pgmap v346: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-14T16:53:25.560 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:25 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/2231462918' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-14T16:53:25.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:25 smithi151 ceph-mon[36066]: pgmap v346: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-14T16:53:25.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:25 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/2231462918' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-14T16:53:25.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:25 smithi123 ceph-mon[35971]: pgmap v346: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-14T16:53:25.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:25 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/2231462918' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-14T16:53:25.920 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411408 got 77309411412 for osd.2 2024-05-14T16:53:25.920 DEBUG:teuthology.parallel:result is None 2024-05-14T16:53:26.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.934+0000 7fab5f789700 1 -- 172.21.15.2:0/1355216096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fab58102df0 msgr2=0x7fab5810f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:26.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.934+0000 7fab5f789700 1 --2- 172.21.15.2:0/1355216096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fab58102df0 0x7fab5810f2e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fab5000b0a0 tx=0x7fab5000b3b0 comp rx=0 tx=0).stop 2024-05-14T16:53:26.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.934+0000 7fab5f789700 1 -- 172.21.15.2:0/1355216096 shutdown_connections 2024-05-14T16:53:26.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.934+0000 7fab5f789700 1 --2- 172.21.15.2:0/1355216096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fab58102df0 0x7fab5810f2e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:26.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.934+0000 7fab5f789700 1 --2- 172.21.15.2:0/1355216096 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab58108510 0x7fab581088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:26.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.934+0000 7fab5f789700 1 --2- 172.21.15.2:0/1355216096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab58102430 0x7fab581028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:26.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.934+0000 7fab5f789700 1 -- 172.21.15.2:0/1355216096 >> 172.21.15.2:0/1355216096 conn(0x7fab580793b0 msgr2=0x7fab580797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:26.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.934+0000 7fab5f789700 1 -- 172.21.15.2:0/1355216096 shutdown_connections 2024-05-14T16:53:26.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.934+0000 7fab5f789700 1 -- 172.21.15.2:0/1355216096 wait complete. 2024-05-14T16:53:26.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.935+0000 7fab5f789700 1 Processor -- start 2024-05-14T16:53:26.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.935+0000 7fab5f789700 1 -- start start 2024-05-14T16:53:26.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.935+0000 7fab5f789700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab58102430 0x7fab5819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:26.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.935+0000 7fab5f789700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fab58102df0 0x7fab5819af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:26.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.935+0000 7fab5f789700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab58108510 0x7fab581a5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:26.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.935+0000 7fab5f789700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fab581125e0 con 0x7fab58102df0 2024-05-14T16:53:26.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.935+0000 7fab5f789700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fab58112460 con 0x7fab58102430 2024-05-14T16:53:26.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.935+0000 7fab5f789700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fab58112760 con 0x7fab58108510 2024-05-14T16:53:26.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.935+0000 7fab5cd24700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fab58102df0 0x7fab5819af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:26.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.935+0000 7fab5cd24700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fab58102df0 0x7fab5819af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38702/0 (socket says 172.21.15.2:38702) 2024-05-14T16:53:26.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.935+0000 7fab5cd24700 1 -- 172.21.15.2:0/2231462918 learned_addr learned my addr 172.21.15.2:0/2231462918 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:53:26.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.935+0000 7fab5d525700 1 --2- 172.21.15.2:0/2231462918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab58102430 0x7fab5819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:26.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.935+0000 7fab5dd26700 1 --2- 172.21.15.2:0/2231462918 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab58108510 0x7fab581a5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:26.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.935+0000 7fab5cd24700 1 -- 172.21.15.2:0/2231462918 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab58108510 msgr2=0x7fab581a5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:26.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.935+0000 7fab5cd24700 1 --2- 172.21.15.2:0/2231462918 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab58108510 0x7fab581a5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:26.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.935+0000 7fab5cd24700 1 -- 172.21.15.2:0/2231462918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab58102430 msgr2=0x7fab5819a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:26.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.935+0000 7fab5cd24700 1 --2- 172.21.15.2:0/2231462918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab58102430 0x7fab5819a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:26.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.935+0000 7fab5cd24700 1 -- 172.21.15.2:0/2231462918 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab581a5fa0 con 0x7fab58102df0 2024-05-14T16:53:26.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.935+0000 7fab5dd26700 1 --2- 172.21.15.2:0/2231462918 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab58108510 0x7fab581a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:53:26.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.936+0000 7fab5cd24700 1 --2- 172.21.15.2:0/2231462918 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fab58102df0 0x7fab5819af20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fab4800c9b0 tx=0x7fab4800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:26.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.936+0000 7fab4e7fc700 1 -- 172.21.15.2:0/2231462918 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab48007b00 con 0x7fab58102df0 2024-05-14T16:53:26.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.936+0000 7fab4e7fc700 1 -- 172.21.15.2:0/2231462918 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab48007ca0 con 0x7fab58102df0 2024-05-14T16:53:26.061 INFO:teuthology.orchestra.run.smithi002.stdout:300647710758 2024-05-14T16:53:26.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.936+0000 7fab5f789700 1 -- 172.21.15.2:0/2231462918 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab581a6170 con 0x7fab58102df0 2024-05-14T16:53:26.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.936+0000 7fab4e7fc700 1 -- 172.21.15.2:0/2231462918 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab48018660 con 0x7fab58102df0 2024-05-14T16:53:26.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.936+0000 7fab5f789700 1 -- 172.21.15.2:0/2231462918 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab581a6680 con 0x7fab58102df0 2024-05-14T16:53:26.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.937+0000 7fab5f789700 1 -- 172.21.15.2:0/2231462918 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab5810a9a0 con 0x7fab58102df0 2024-05-14T16:53:26.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.938+0000 7fab4e7fc700 1 -- 172.21.15.2:0/2231462918 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fab4800f450 con 0x7fab58102df0 2024-05-14T16:53:26.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.939+0000 7fab4e7fc700 1 --2- 172.21.15.2:0/2231462918 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fab440b2880 0x7fab440b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:26.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.939+0000 7fab5d525700 1 --2- 172.21.15.2:0/2231462918 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fab440b2880 0x7fab440b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:26.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.939+0000 7fab4e7fc700 1 -- 172.21.15.2:0/2231462918 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7fab480e24a0 con 0x7fab58102df0 2024-05-14T16:53:26.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.939+0000 7fab5d525700 1 --2- 172.21.15.2:0/2231462918 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fab440b2880 0x7fab440b4d40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fab54009be0 tx=0x7fab54009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:26.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:24.942+0000 7fab4e7fc700 1 -- 172.21.15.2:0/2231462918 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fab48014030 con 0x7fab58102df0 2024-05-14T16:53:26.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:25.155+0000 7fab5f789700 1 -- 172.21.15.2:0/2231462918 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fab5810a380 con 0x7fab58102df0 2024-05-14T16:53:26.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:25.155+0000 7fab4e7fc700 1 -- 172.21.15.2:0/2231462918 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fab480aa260 con 0x7fab58102df0 2024-05-14T16:53:26.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:25.157+0000 7fab5f789700 1 -- 172.21.15.2:0/2231462918 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fab440b2880 msgr2=0x7fab440b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:26.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:25.157+0000 7fab5f789700 1 --2- 172.21.15.2:0/2231462918 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fab440b2880 0x7fab440b4d40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fab54009be0 tx=0x7fab54009260 comp rx=0 tx=0).stop 2024-05-14T16:53:26.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:25.157+0000 7fab5f789700 1 -- 172.21.15.2:0/2231462918 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fab58102df0 msgr2=0x7fab5819af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:26.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:25.157+0000 7fab5f789700 1 --2- 172.21.15.2:0/2231462918 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fab58102df0 0x7fab5819af20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fab4800c9b0 tx=0x7fab4800ccc0 comp rx=0 tx=0).stop 2024-05-14T16:53:26.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:25.157+0000 7fab5f789700 1 -- 172.21.15.2:0/2231462918 shutdown_connections 2024-05-14T16:53:26.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:25.157+0000 7fab5f789700 1 --2- 172.21.15.2:0/2231462918 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab58108510 0x7fab581a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:26.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:25.157+0000 7fab5f789700 1 --2- 172.21.15.2:0/2231462918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab58102430 0x7fab5819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:26.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:25.157+0000 7fab5f789700 1 --2- 172.21.15.2:0/2231462918 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fab440b2880 0x7fab440b4d40 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:26.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:25.157+0000 7fab5f789700 1 --2- 172.21.15.2:0/2231462918 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fab58102df0 0x7fab5819af20 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:26.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:25.157+0000 7fab5f789700 1 -- 172.21.15.2:0/2231462918 >> 172.21.15.2:0/2231462918 conn(0x7fab580793b0 msgr2=0x7fab58100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:26.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:25.158+0000 7fab5f789700 1 -- 172.21.15.2:0/2231462918 shutdown_connections 2024-05-14T16:53:26.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:25.158+0000 7fab5f789700 1 -- 172.21.15.2:0/2231462918 wait complete. 2024-05-14T16:53:26.705 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710752 got 300647710758 for osd.11 2024-05-14T16:53:26.706 DEBUG:teuthology.parallel:result is None 2024-05-14T16:53:26.706 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-14T16:53:26.706 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-14T16:53:26.966 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:53:27.349 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:27 smithi002 ceph-mon[31538]: pgmap v347: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:53:27.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:27 smithi151 ceph-mon[36066]: pgmap v347: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:53:27.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:27 smithi123 ceph-mon[35971]: pgmap v347: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:53:28.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.615+0000 7f4fffbbe700 1 -- 172.21.15.2:0/859442698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ff806bbe0 msgr2=0x7f4ff81103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:28.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.615+0000 7f4fffbbe700 1 --2- 172.21.15.2:0/859442698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ff806bbe0 0x7f4ff81103d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4ff000b0a0 tx=0x7f4ff000b3b0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.615+0000 7f4fffbbe700 1 -- 172.21.15.2:0/859442698 shutdown_connections 2024-05-14T16:53:28.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.615+0000 7f4fffbbe700 1 --2- 172.21.15.2:0/859442698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ff806bbe0 0x7f4ff81103d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.615+0000 7f4fffbbe700 1 --2- 172.21.15.2:0/859442698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ff81095b0 0x7f4ff8109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.615+0000 7f4fffbbe700 1 --2- 172.21.15.2:0/859442698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ff806b220 0x7f4ff806b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.615+0000 7f4fffbbe700 1 -- 172.21.15.2:0/859442698 >> 172.21.15.2:0/859442698 conn(0x7f4ff80793f0 msgr2=0x7f4ff8079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:28.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.615+0000 7f4fffbbe700 1 -- 172.21.15.2:0/859442698 shutdown_connections 2024-05-14T16:53:28.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.615+0000 7f4fffbbe700 1 -- 172.21.15.2:0/859442698 wait complete. 2024-05-14T16:53:28.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.616+0000 7f4fffbbe700 1 Processor -- start 2024-05-14T16:53:28.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.616+0000 7f4fffbbe700 1 -- start start 2024-05-14T16:53:28.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.616+0000 7f4fffbbe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ff806b220 0x7f4ff819f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:28.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.616+0000 7f4fffbbe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ff806bbe0 0x7f4ff819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:28.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.616+0000 7f4fffbbe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ff81095b0 0x7f4ff81999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:28.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.616+0000 7f4fffbbe700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff8113750 con 0x7f4ff806bbe0 2024-05-14T16:53:28.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.616+0000 7f4fffbbe700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff81135d0 con 0x7f4ff81095b0 2024-05-14T16:53:28.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.616+0000 7f4fffbbe700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff81138d0 con 0x7f4ff806b220 2024-05-14T16:53:28.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.616+0000 7f4ffd159700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ff806bbe0 0x7f4ff819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:28.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.616+0000 7f4ffd159700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ff806bbe0 0x7f4ff819fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38722/0 (socket says 172.21.15.2:38722) 2024-05-14T16:53:28.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.616+0000 7f4ffd159700 1 -- 172.21.15.2:0/2473867079 learned_addr learned my addr 172.21.15.2:0/2473867079 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:53:28.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.616+0000 7f4ffd95a700 1 --2- 172.21.15.2:0/2473867079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ff806b220 0x7f4ff819f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:28.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.616+0000 7f4ffe15b700 1 --2- 172.21.15.2:0/2473867079 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ff81095b0 0x7f4ff81999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:28.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.617+0000 7f4ffd159700 1 -- 172.21.15.2:0/2473867079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ff806b220 msgr2=0x7f4ff819f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:28.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.617+0000 7f4ffd159700 1 --2- 172.21.15.2:0/2473867079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ff806b220 0x7f4ff819f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.617+0000 7f4ffd159700 1 -- 172.21.15.2:0/2473867079 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ff81095b0 msgr2=0x7f4ff81999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:28.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.617+0000 7f4ffd159700 1 --2- 172.21.15.2:0/2473867079 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ff81095b0 0x7f4ff81999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.617+0000 7f4ffd159700 1 -- 172.21.15.2:0/2473867079 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ff819a150 con 0x7f4ff806bbe0 2024-05-14T16:53:28.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.617+0000 7f4ffd159700 1 --2- 172.21.15.2:0/2473867079 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ff806bbe0 0x7f4ff819fd20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4fe800b530 tx=0x7f4fe800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:28.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.617+0000 7f4feeffd700 1 -- 172.21.15.2:0/2473867079 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8015020 con 0x7f4ff806bbe0 2024-05-14T16:53:28.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.617+0000 7f4feeffd700 1 -- 172.21.15.2:0/2473867079 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fe8011ce0 con 0x7f4ff806bbe0 2024-05-14T16:53:28.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.617+0000 7f4fffbbe700 1 -- 172.21.15.2:0/2473867079 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ff819a440 con 0x7f4ff806bbe0 2024-05-14T16:53:28.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.617+0000 7f4feeffd700 1 -- 172.21.15.2:0/2473867079 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8010dd0 con 0x7f4ff806bbe0 2024-05-14T16:53:28.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.617+0000 7f4fffbbe700 1 -- 172.21.15.2:0/2473867079 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ff81a66b0 con 0x7f4ff806bbe0 2024-05-14T16:53:28.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.618+0000 7f4fffbbe700 1 -- 172.21.15.2:0/2473867079 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ff806a230 con 0x7f4ff806bbe0 2024-05-14T16:53:28.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.619+0000 7f4feeffd700 1 -- 172.21.15.2:0/2473867079 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f4fe8018020 con 0x7f4ff806bbe0 2024-05-14T16:53:28.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.620+0000 7f4feeffd700 1 --2- 172.21.15.2:0/2473867079 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f4fe40b2880 0x7f4fe40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:28.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.620+0000 7f4ffd95a700 1 --2- 172.21.15.2:0/2473867079 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f4fe40b2880 0x7f4fe40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:28.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.620+0000 7f4feeffd700 1 -- 172.21.15.2:0/2473867079 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f4fe80e1eb0 con 0x7f4ff806bbe0 2024-05-14T16:53:28.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.621+0000 7f4ffd95a700 1 --2- 172.21.15.2:0/2473867079 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f4fe40b2880 0x7f4fe40b4d40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4ff81090b0 tx=0x7f4ff400a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:28.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.624+0000 7f4feeffd700 1 -- 172.21.15.2:0/2473867079 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4fe801f020 con 0x7f4ff806bbe0 2024-05-14T16:53:28.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.834+0000 7f4fffbbe700 1 -- 172.21.15.2:0/2473867079 --> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4ff8063980 con 0x7f4fe40b2880 2024-05-14T16:53:28.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.841+0000 7f4feeffd700 1 -- 172.21.15.2:0/2473867079 <== mgr.14150 v2:172.21.15.2:6800/1498936028 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258986 (secure 0 0 0) 0x7f4ff8063980 con 0x7f4fe40b2880 2024-05-14T16:53:28.841 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:53:28.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.845+0000 7f4fffbbe700 1 -- 172.21.15.2:0/2473867079 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f4fe40b2880 msgr2=0x7f4fe40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:28.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.845+0000 7f4fffbbe700 1 --2- 172.21.15.2:0/2473867079 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f4fe40b2880 0x7f4fe40b4d40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4ff81090b0 tx=0x7f4ff400a380 comp rx=0 tx=0).stop 2024-05-14T16:53:28.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.845+0000 7f4fffbbe700 1 -- 172.21.15.2:0/2473867079 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ff806bbe0 msgr2=0x7f4ff819fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:28.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.845+0000 7f4fffbbe700 1 --2- 172.21.15.2:0/2473867079 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ff806bbe0 0x7f4ff819fd20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4fe800b530 tx=0x7f4fe800b840 comp rx=0 tx=0).stop 2024-05-14T16:53:28.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.846+0000 7f4fffbbe700 1 -- 172.21.15.2:0/2473867079 shutdown_connections 2024-05-14T16:53:28.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.846+0000 7f4fffbbe700 1 --2- 172.21.15.2:0/2473867079 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ff81095b0 0x7f4ff81999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.846+0000 7f4fffbbe700 1 --2- 172.21.15.2:0/2473867079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ff806b220 0x7f4ff819f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.846+0000 7f4fffbbe700 1 --2- 172.21.15.2:0/2473867079 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f4fe40b2880 0x7f4fe40b4d40 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.846+0000 7f4fffbbe700 1 --2- 172.21.15.2:0/2473867079 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ff806bbe0 0x7f4ff819fd20 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:28.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.846+0000 7f4fffbbe700 1 -- 172.21.15.2:0/2473867079 >> 172.21.15.2:0/2473867079 conn(0x7f4ff80793f0 msgr2=0x7f4ff8101070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:28.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.846+0000 7f4fffbbe700 1 -- 172.21.15.2:0/2473867079 shutdown_connections 2024-05-14T16:53:28.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:28.846+0000 7f4fffbbe700 1 -- 172.21.15.2:0/2473867079 wait complete. 2024-05-14T16:53:28.857 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-05-14T16:53:29.470 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:29 smithi002 ceph-mon[31538]: pgmap v348: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T16:53:29.500 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":348,"stamp":"2024-05-14T16:53:28.113393+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656234,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":678,"num_read_kb":433,"num_write":296,"num_write_kb":2268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":2951957,"num_keys_recovered":9,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":739,"ondisk_log_size":739,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":384768,"kb_used_data":10656,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124557056,"statfs":{"total":1151940427776,"available":1151546425344,"internally_reserved":0,"allocated":10911744,"data_stored":7063278,"data_compressed":108486,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003130"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.129790+0000","last_change":"2024-05-14T16:51:32.298580+0000","last_active":"2024-05-14T16:53:12.129790+0000","last_peered":"2024-05-14T16:53:12.129790+0000","last_clean":"2024-05-14T16:53:12.129790+0000","last_became_active":"2024-05-14T16:51:32.296945+0000","last_became_peered":"2024-05-14T16:51:32.296945+0000","last_unstale":"2024-05-14T16:53:12.129790+0000","last_undegraded":"2024-05-14T16:53:12.129790+0000","last_fullsized":"2024-05-14T16:53:12.129790+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:50:17.664928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.116489+0000","last_change":"2024-05-14T16:51:32.297618+0000","last_active":"2024-05-14T16:53:12.116489+0000","last_peered":"2024-05-14T16:53:12.116489+0000","last_clean":"2024-05-14T16:53:12.116489+0000","last_became_active":"2024-05-14T16:51:32.297260+0000","last_became_peered":"2024-05-14T16:51:32.297260+0000","last_unstale":"2024-05-14T16:53:12.116489+0000","last_undegraded":"2024-05-14T16:53:12.116489+0000","last_fullsized":"2024-05-14T16:53:12.116489+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:43:50.542587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.128312+0000","last_change":"2024-05-14T16:51:32.295525+0000","last_active":"2024-05-14T16:53:12.128312+0000","last_peered":"2024-05-14T16:53:12.128312+0000","last_clean":"2024-05-14T16:53:12.128312+0000","last_became_active":"2024-05-14T16:51:32.295215+0000","last_became_peered":"2024-05-14T16:51:32.295215+0000","last_unstale":"2024-05-14T16:53:12.128312+0000","last_undegraded":"2024-05-14T16:53:12.128312+0000","last_fullsized":"2024-05-14T16:53:12.128312+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:40:05.516280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.121502+0000","last_change":"2024-05-14T16:51:32.307883+0000","last_active":"2024-05-14T16:53:12.121502+0000","last_peered":"2024-05-14T16:53:12.121502+0000","last_clean":"2024-05-14T16:53:12.121502+0000","last_became_active":"2024-05-14T16:51:32.307499+0000","last_became_peered":"2024-05-14T16:51:32.307499+0000","last_unstale":"2024-05-14T16:53:12.121502+0000","last_undegraded":"2024-05-14T16:53:12.121502+0000","last_fullsized":"2024-05-14T16:53:12.121502+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:30:59.978059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.105483+0000","last_change":"2024-05-14T16:51:32.294429+0000","last_active":"2024-05-14T16:53:12.105483+0000","last_peered":"2024-05-14T16:53:12.105483+0000","last_clean":"2024-05-14T16:53:12.105483+0000","last_became_active":"2024-05-14T16:51:32.294088+0000","last_became_peered":"2024-05-14T16:51:32.294088+0000","last_unstale":"2024-05-14T16:53:12.105483+0000","last_undegraded":"2024-05-14T16:53:12.105483+0000","last_fullsized":"2024-05-14T16:53:12.105483+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:25:19.383586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.115996+0000","last_change":"2024-05-14T16:51:32.296169+0000","last_active":"2024-05-14T16:53:12.115996+0000","last_peered":"2024-05-14T16:53:12.115996+0000","last_clean":"2024-05-14T16:53:12.115996+0000","last_became_active":"2024-05-14T16:51:32.295819+0000","last_became_peered":"2024-05-14T16:51:32.295819+0000","last_unstale":"2024-05-14T16:53:12.115996+0000","last_undegraded":"2024-05-14T16:53:12.115996+0000","last_fullsized":"2024-05-14T16:53:12.115996+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:56:23.932039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.125249+0000","last_change":"2024-05-14T16:51:32.309609+0000","last_active":"2024-05-14T16:53:12.125249+0000","last_peered":"2024-05-14T16:53:12.125249+0000","last_clean":"2024-05-14T16:53:12.125249+0000","last_became_active":"2024-05-14T16:51:32.309175+0000","last_became_peered":"2024-05-14T16:51:32.309175+0000","last_unstale":"2024-05-14T16:53:12.125249+0000","last_undegraded":"2024-05-14T16:53:12.125249+0000","last_fullsized":"2024-05-14T16:53:12.125249+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:06:46.717592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.117730+0000","last_change":"2024-05-14T16:52:13.446592+0000","last_active":"2024-05-14T16:53:12.117730+0000","last_peered":"2024-05-14T16:53:12.117730+0000","last_clean":"2024-05-14T16:53:12.117730+0000","last_became_active":"2024-05-14T16:52:13.446238+0000","last_became_peered":"2024-05-14T16:52:13.446238+0000","last_unstale":"2024-05-14T16:53:12.117730+0000","last_undegraded":"2024-05-14T16:53:12.117730+0000","last_fullsized":"2024-05-14T16:53:12.117730+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:33:14.216797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.131561+0000","last_change":"2024-05-14T16:51:32.307571+0000","last_active":"2024-05-14T16:53:12.131561+0000","last_peered":"2024-05-14T16:53:12.131561+0000","last_clean":"2024-05-14T16:53:12.131561+0000","last_became_active":"2024-05-14T16:51:32.307151+0000","last_became_peered":"2024-05-14T16:51:32.307151+0000","last_unstale":"2024-05-14T16:53:12.131561+0000","last_undegraded":"2024-05-14T16:53:12.131561+0000","last_fullsized":"2024-05-14T16:53:12.131561+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:17:26.325393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.111451+0000","last_change":"2024-05-14T16:51:32.298773+0000","last_active":"2024-05-14T16:53:12.111451+0000","last_peered":"2024-05-14T16:53:12.111451+0000","last_clean":"2024-05-14T16:53:12.111451+0000","last_became_active":"2024-05-14T16:51:32.298283+0000","last_became_peered":"2024-05-14T16:51:32.298283+0000","last_unstale":"2024-05-14T16:53:12.111451+0000","last_undegraded":"2024-05-14T16:53:12.111451+0000","last_fullsized":"2024-05-14T16:53:12.111451+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:52:18.471201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.129073+0000","last_change":"2024-05-14T16:51:32.302443+0000","last_active":"2024-05-14T16:53:12.129073+0000","last_peered":"2024-05-14T16:53:12.129073+0000","last_clean":"2024-05-14T16:53:12.129073+0000","last_became_active":"2024-05-14T16:51:32.302034+0000","last_became_peered":"2024-05-14T16:51:32.302034+0000","last_unstale":"2024-05-14T16:53:12.129073+0000","last_undegraded":"2024-05-14T16:53:12.129073+0000","last_fullsized":"2024-05-14T16:53:12.129073+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:53:08.041293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.121182+0000","last_change":"2024-05-14T16:51:32.304837+0000","last_active":"2024-05-14T16:53:12.121182+0000","last_peered":"2024-05-14T16:53:12.121182+0000","last_clean":"2024-05-14T16:53:12.121182+0000","last_became_active":"2024-05-14T16:51:32.304500+0000","last_became_peered":"2024-05-14T16:51:32.304500+0000","last_unstale":"2024-05-14T16:53:12.121182+0000","last_undegraded":"2024-05-14T16:53:12.121182+0000","last_fullsized":"2024-05-14T16:53:12.121182+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:32:02.713044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.117386+0000","last_change":"2024-05-14T16:51:32.299247+0000","last_active":"2024-05-14T16:53:12.117386+0000","last_peered":"2024-05-14T16:53:12.117386+0000","last_clean":"2024-05-14T16:53:12.117386+0000","last_became_active":"2024-05-14T16:51:32.298925+0000","last_became_peered":"2024-05-14T16:51:32.298925+0000","last_unstale":"2024-05-14T16:53:12.117386+0000","last_undegraded":"2024-05-14T16:53:12.117386+0000","last_fullsized":"2024-05-14T16:53:12.117386+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:10:38.847877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.117049+0000","last_change":"2024-05-14T16:51:32.293743+0000","last_active":"2024-05-14T16:53:12.117049+0000","last_peered":"2024-05-14T16:53:12.117049+0000","last_clean":"2024-05-14T16:53:12.117049+0000","last_became_active":"2024-05-14T16:51:32.293429+0000","last_became_peered":"2024-05-14T16:51:32.293429+0000","last_unstale":"2024-05-14T16:53:12.117049+0000","last_undegraded":"2024-05-14T16:53:12.117049+0000","last_fullsized":"2024-05-14T16:53:12.117049+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:12:33.909940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.113527+0000","last_change":"2024-05-14T16:51:32.304940+0000","last_active":"2024-05-14T16:53:12.113527+0000","last_peered":"2024-05-14T16:53:12.113527+0000","last_clean":"2024-05-14T16:53:12.113527+0000","last_became_active":"2024-05-14T16:51:32.304482+0000","last_became_peered":"2024-05-14T16:51:32.304482+0000","last_unstale":"2024-05-14T16:53:12.113527+0000","last_undegraded":"2024-05-14T16:53:12.113527+0000","last_fullsized":"2024-05-14T16:53:12.113527+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:54:17.771620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.131342+0000","last_change":"2024-05-14T16:51:32.301784+0000","last_active":"2024-05-14T16:53:12.131342+0000","last_peered":"2024-05-14T16:53:12.131342+0000","last_clean":"2024-05-14T16:53:12.131342+0000","last_became_active":"2024-05-14T16:51:32.301472+0000","last_became_peered":"2024-05-14T16:51:32.301472+0000","last_unstale":"2024-05-14T16:53:12.131342+0000","last_undegraded":"2024-05-14T16:53:12.131342+0000","last_fullsized":"2024-05-14T16:53:12.131342+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:10:56.735424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.130053+0000","last_change":"2024-05-14T16:51:32.297951+0000","last_active":"2024-05-14T16:53:12.130053+0000","last_peered":"2024-05-14T16:53:12.130053+0000","last_clean":"2024-05-14T16:53:12.130053+0000","last_became_active":"2024-05-14T16:51:32.296602+0000","last_became_peered":"2024-05-14T16:51:32.296602+0000","last_unstale":"2024-05-14T16:53:12.130053+0000","last_undegraded":"2024-05-14T16:53:12.130053+0000","last_fullsized":"2024-05-14T16:53:12.130053+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:16:36.039077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.116374+0000","last_change":"2024-05-14T16:51:32.290398+0000","last_active":"2024-05-14T16:53:12.116374+0000","last_peered":"2024-05-14T16:53:12.116374+0000","last_clean":"2024-05-14T16:53:12.116374+0000","last_became_active":"2024-05-14T16:51:32.290083+0000","last_became_peered":"2024-05-14T16:51:32.290083+0000","last_unstale":"2024-05-14T16:53:12.116374+0000","last_undegraded":"2024-05-14T16:53:12.116374+0000","last_fullsized":"2024-05-14T16:53:12.116374+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:48:05.795209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.117279+0000","last_change":"2024-05-14T16:51:32.294398+0000","last_active":"2024-05-14T16:53:12.117279+0000","last_peered":"2024-05-14T16:53:12.117279+0000","last_clean":"2024-05-14T16:53:12.117279+0000","last_became_active":"2024-05-14T16:51:32.294080+0000","last_became_peered":"2024-05-14T16:51:32.294080+0000","last_unstale":"2024-05-14T16:53:12.117279+0000","last_undegraded":"2024-05-14T16:53:12.117279+0000","last_fullsized":"2024-05-14T16:53:12.117279+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:15:03.859045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.214939+0000","last_change":"2024-05-14T16:51:29.274482+0000","last_active":"2024-05-14T16:52:13.214939+0000","last_peered":"2024-05-14T16:52:13.214939+0000","last_clean":"2024-05-14T16:52:13.214939+0000","last_became_active":"2024-05-14T16:51:29.274152+0000","last_became_peered":"2024-05-14T16:51:29.274152+0000","last_unstale":"2024-05-14T16:52:13.214939+0000","last_undegraded":"2024-05-14T16:52:13.214939+0000","last_fullsized":"2024-05-14T16:52:13.214939+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:05:38.274248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.112062+0000","last_change":"2024-05-14T16:51:32.300956+0000","last_active":"2024-05-14T16:53:12.112062+0000","last_peered":"2024-05-14T16:53:12.112062+0000","last_clean":"2024-05-14T16:53:12.112062+0000","last_became_active":"2024-05-14T16:51:32.300546+0000","last_became_peered":"2024-05-14T16:51:32.300546+0000","last_unstale":"2024-05-14T16:53:12.112062+0000","last_undegraded":"2024-05-14T16:53:12.112062+0000","last_fullsized":"2024-05-14T16:53:12.112062+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:54:09.181036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.212599+0000","last_change":"2024-05-14T16:51:29.345206+0000","last_active":"2024-05-14T16:52:13.212599+0000","last_peered":"2024-05-14T16:52:13.212599+0000","last_clean":"2024-05-14T16:52:13.212599+0000","last_became_active":"2024-05-14T16:51:29.344699+0000","last_became_peered":"2024-05-14T16:51:29.344699+0000","last_unstale":"2024-05-14T16:52:13.212599+0000","last_undegraded":"2024-05-14T16:52:13.212599+0000","last_fullsized":"2024-05-14T16:52:13.212599+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:14:02.364661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.128243+0000","last_change":"2024-05-14T16:51:32.297522+0000","last_active":"2024-05-14T16:53:12.128243+0000","last_peered":"2024-05-14T16:53:12.128243+0000","last_clean":"2024-05-14T16:53:12.128243+0000","last_became_active":"2024-05-14T16:51:32.296866+0000","last_became_peered":"2024-05-14T16:51:32.296866+0000","last_unstale":"2024-05-14T16:53:12.128243+0000","last_undegraded":"2024-05-14T16:53:12.128243+0000","last_fullsized":"2024-05-14T16:53:12.128243+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:49:43.409575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.219763+0000","last_change":"2024-05-14T16:51:29.280632+0000","last_active":"2024-05-14T16:52:13.219763+0000","last_peered":"2024-05-14T16:52:13.219763+0000","last_clean":"2024-05-14T16:52:13.219763+0000","last_became_active":"2024-05-14T16:51:29.280246+0000","last_became_peered":"2024-05-14T16:51:29.280246+0000","last_unstale":"2024-05-14T16:52:13.219763+0000","last_undegraded":"2024-05-14T16:52:13.219763+0000","last_fullsized":"2024-05-14T16:52:13.219763+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:34:47.925344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.131865+0000","last_change":"2024-05-14T16:51:32.298284+0000","last_active":"2024-05-14T16:53:12.131865+0000","last_peered":"2024-05-14T16:53:12.131865+0000","last_clean":"2024-05-14T16:53:12.131865+0000","last_became_active":"2024-05-14T16:51:32.294202+0000","last_became_peered":"2024-05-14T16:51:32.294202+0000","last_unstale":"2024-05-14T16:53:12.131865+0000","last_undegraded":"2024-05-14T16:53:12.131865+0000","last_fullsized":"2024-05-14T16:53:12.131865+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:27:37.561275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.219221+0000","last_change":"2024-05-14T16:51:29.276227+0000","last_active":"2024-05-14T16:52:13.219221+0000","last_peered":"2024-05-14T16:52:13.219221+0000","last_clean":"2024-05-14T16:52:13.219221+0000","last_became_active":"2024-05-14T16:51:29.275837+0000","last_became_peered":"2024-05-14T16:51:29.275837+0000","last_unstale":"2024-05-14T16:52:13.219221+0000","last_undegraded":"2024-05-14T16:52:13.219221+0000","last_fullsized":"2024-05-14T16:52:13.219221+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:03:29.402985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.129798+0000","last_change":"2024-05-14T16:51:32.308173+0000","last_active":"2024-05-14T16:53:12.129798+0000","last_peered":"2024-05-14T16:53:12.129798+0000","last_clean":"2024-05-14T16:53:12.129798+0000","last_became_active":"2024-05-14T16:51:32.307135+0000","last_became_peered":"2024-05-14T16:51:32.307135+0000","last_unstale":"2024-05-14T16:53:12.129798+0000","last_undegraded":"2024-05-14T16:53:12.129798+0000","last_fullsized":"2024-05-14T16:53:12.129798+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:01:46.579968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'7","reported_seq":134,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:26.157545+0000","last_change":"2024-05-14T16:51:29.279486+0000","last_active":"2024-05-14T16:53:26.157545+0000","last_peered":"2024-05-14T16:53:26.157545+0000","last_clean":"2024-05-14T16:53:26.157545+0000","last_became_active":"2024-05-14T16:51:29.279133+0000","last_became_peered":"2024-05-14T16:51:29.279133+0000","last_unstale":"2024-05-14T16:53:26.157545+0000","last_undegraded":"2024-05-14T16:53:26.157545+0000","last_fullsized":"2024-05-14T16:53:26.157545+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:26:18.620421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.110557+0000","last_change":"2024-05-14T16:51:32.299852+0000","last_active":"2024-05-14T16:53:12.110557+0000","last_peered":"2024-05-14T16:53:12.110557+0000","last_clean":"2024-05-14T16:53:12.110557+0000","last_became_active":"2024-05-14T16:51:32.299541+0000","last_became_peered":"2024-05-14T16:51:32.299541+0000","last_unstale":"2024-05-14T16:53:12.110557+0000","last_undegraded":"2024-05-14T16:53:12.110557+0000","last_fullsized":"2024-05-14T16:53:12.110557+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:36:38.191009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.445109+0000","last_change":"2024-05-14T16:51:29.275269+0000","last_active":"2024-05-14T16:52:13.445109+0000","last_peered":"2024-05-14T16:52:13.445109+0000","last_clean":"2024-05-14T16:52:13.445109+0000","last_became_active":"2024-05-14T16:51:29.274792+0000","last_became_peered":"2024-05-14T16:51:29.274792+0000","last_unstale":"2024-05-14T16:52:13.445109+0000","last_undegraded":"2024-05-14T16:52:13.445109+0000","last_fullsized":"2024-05-14T16:52:13.445109+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:17:08.925289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.110428+0000","last_change":"2024-05-14T16:51:32.306536+0000","last_active":"2024-05-14T16:53:12.110428+0000","last_peered":"2024-05-14T16:53:12.110428+0000","last_clean":"2024-05-14T16:53:12.110428+0000","last_became_active":"2024-05-14T16:51:32.304976+0000","last_became_peered":"2024-05-14T16:51:32.304976+0000","last_unstale":"2024-05-14T16:53:12.110428+0000","last_undegraded":"2024-05-14T16:53:12.110428+0000","last_fullsized":"2024-05-14T16:53:12.110428+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:48:13.284022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.921481+0000","last_change":"2024-05-14T16:51:29.266783+0000","last_active":"2024-05-14T16:52:13.921481+0000","last_peered":"2024-05-14T16:52:13.921481+0000","last_clean":"2024-05-14T16:52:13.921481+0000","last_became_active":"2024-05-14T16:51:29.266409+0000","last_became_peered":"2024-05-14T16:51:29.266409+0000","last_unstale":"2024-05-14T16:52:13.921481+0000","last_undegraded":"2024-05-14T16:52:13.921481+0000","last_fullsized":"2024-05-14T16:52:13.921481+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:33:14.981709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.116994+0000","last_change":"2024-05-14T16:51:32.289288+0000","last_active":"2024-05-14T16:53:12.116994+0000","last_peered":"2024-05-14T16:53:12.116994+0000","last_clean":"2024-05-14T16:53:12.116994+0000","last_became_active":"2024-05-14T16:51:32.288920+0000","last_became_peered":"2024-05-14T16:51:32.288920+0000","last_unstale":"2024-05-14T16:53:12.116994+0000","last_undegraded":"2024-05-14T16:53:12.116994+0000","last_fullsized":"2024-05-14T16:53:12.116994+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:23:43.659417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.216697+0000","last_change":"2024-05-14T16:51:29.273926+0000","last_active":"2024-05-14T16:52:13.216697+0000","last_peered":"2024-05-14T16:52:13.216697+0000","last_clean":"2024-05-14T16:52:13.216697+0000","last_became_active":"2024-05-14T16:51:29.273059+0000","last_became_peered":"2024-05-14T16:51:29.273059+0000","last_unstale":"2024-05-14T16:52:13.216697+0000","last_undegraded":"2024-05-14T16:52:13.216697+0000","last_fullsized":"2024-05-14T16:52:13.216697+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:46:27.800481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.117050+0000","last_change":"2024-05-14T16:51:32.287554+0000","last_active":"2024-05-14T16:53:12.117050+0000","last_peered":"2024-05-14T16:53:12.117050+0000","last_clean":"2024-05-14T16:53:12.117050+0000","last_became_active":"2024-05-14T16:51:32.287204+0000","last_became_peered":"2024-05-14T16:51:32.287204+0000","last_unstale":"2024-05-14T16:53:12.117050+0000","last_undegraded":"2024-05-14T16:53:12.117050+0000","last_fullsized":"2024-05-14T16:53:12.117050+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:29:40.675378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":123,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:22.150102+0000","last_change":"2024-05-14T16:51:29.274819+0000","last_active":"2024-05-14T16:53:22.150102+0000","last_peered":"2024-05-14T16:53:22.150102+0000","last_clean":"2024-05-14T16:53:22.150102+0000","last_became_active":"2024-05-14T16:51:29.274454+0000","last_became_peered":"2024-05-14T16:51:29.274454+0000","last_unstale":"2024-05-14T16:53:22.150102+0000","last_undegraded":"2024-05-14T16:53:22.150102+0000","last_fullsized":"2024-05-14T16:53:22.150102+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:36:30.496064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":93,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.130381+0000","last_change":"2024-05-14T16:51:32.295732+0000","last_active":"2024-05-14T16:53:12.130381+0000","last_peered":"2024-05-14T16:53:12.130381+0000","last_clean":"2024-05-14T16:53:12.130381+0000","last_became_active":"2024-05-14T16:51:32.294919+0000","last_became_peered":"2024-05-14T16:51:32.294919+0000","last_unstale":"2024-05-14T16:53:12.130381+0000","last_undegraded":"2024-05-14T16:53:12.130381+0000","last_fullsized":"2024-05-14T16:53:12.130381+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:08:54.566601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215472+0000","last_change":"2024-05-14T16:51:29.279338+0000","last_active":"2024-05-14T16:52:13.215472+0000","last_peered":"2024-05-14T16:52:13.215472+0000","last_clean":"2024-05-14T16:52:13.215472+0000","last_became_active":"2024-05-14T16:51:29.278928+0000","last_became_peered":"2024-05-14T16:51:29.278928+0000","last_unstale":"2024-05-14T16:52:13.215472+0000","last_undegraded":"2024-05-14T16:52:13.215472+0000","last_fullsized":"2024-05-14T16:52:13.215472+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:24:34.803515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.113307+0000","last_change":"2024-05-14T16:51:32.305294+0000","last_active":"2024-05-14T16:53:12.113307+0000","last_peered":"2024-05-14T16:53:12.113307+0000","last_clean":"2024-05-14T16:53:12.113307+0000","last_became_active":"2024-05-14T16:51:32.304563+0000","last_became_peered":"2024-05-14T16:51:32.304563+0000","last_unstale":"2024-05-14T16:53:12.113307+0000","last_undegraded":"2024-05-14T16:53:12.113307+0000","last_fullsized":"2024-05-14T16:53:12.113307+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:56:06.259475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.218071+0000","last_change":"2024-05-14T16:51:29.345400+0000","last_active":"2024-05-14T16:52:13.218071+0000","last_peered":"2024-05-14T16:52:13.218071+0000","last_clean":"2024-05-14T16:52:13.218071+0000","last_became_active":"2024-05-14T16:51:29.345010+0000","last_became_peered":"2024-05-14T16:51:29.345010+0000","last_unstale":"2024-05-14T16:52:13.218071+0000","last_undegraded":"2024-05-14T16:52:13.218071+0000","last_fullsized":"2024-05-14T16:52:13.218071+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:24:22.933005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.131849+0000","last_change":"2024-05-14T16:52:13.923167+0000","last_active":"2024-05-14T16:53:12.131849+0000","last_peered":"2024-05-14T16:53:12.131849+0000","last_clean":"2024-05-14T16:53:12.131849+0000","last_became_active":"2024-05-14T16:52:13.922813+0000","last_became_peered":"2024-05-14T16:52:13.922813+0000","last_unstale":"2024-05-14T16:53:12.131849+0000","last_undegraded":"2024-05-14T16:53:12.131849+0000","last_fullsized":"2024-05-14T16:53:12.131849+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:50:38.672408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'33","reported_seq":255,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:23.151736+0000","last_change":"2024-05-14T16:51:29.277591+0000","last_active":"2024-05-14T16:53:23.151736+0000","last_peered":"2024-05-14T16:53:23.151736+0000","last_clean":"2024-05-14T16:53:23.151736+0000","last_became_active":"2024-05-14T16:51:29.276612+0000","last_became_peered":"2024-05-14T16:51:29.276612+0000","last_unstale":"2024-05-14T16:53:23.151736+0000","last_undegraded":"2024-05-14T16:53:23.151736+0000","last_fullsized":"2024-05-14T16:53:23.151736+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:22:28.934173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61490,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":196,"num_read_kb":157,"num_write":34,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.117319+0000","last_change":"2024-05-14T16:51:32.302321+0000","last_active":"2024-05-14T16:53:12.117319+0000","last_peered":"2024-05-14T16:53:12.117319+0000","last_clean":"2024-05-14T16:53:12.117319+0000","last_became_active":"2024-05-14T16:51:32.301973+0000","last_became_peered":"2024-05-14T16:51:32.301973+0000","last_unstale":"2024-05-14T16:53:12.117319+0000","last_undegraded":"2024-05-14T16:53:12.117319+0000","last_fullsized":"2024-05-14T16:53:12.117319+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:49:24.054379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.218367+0000","last_change":"2024-05-14T16:51:29.278776+0000","last_active":"2024-05-14T16:52:13.218367+0000","last_peered":"2024-05-14T16:52:13.218367+0000","last_clean":"2024-05-14T16:52:13.218367+0000","last_became_active":"2024-05-14T16:51:29.278395+0000","last_became_peered":"2024-05-14T16:51:29.278395+0000","last_unstale":"2024-05-14T16:52:13.218367+0000","last_undegraded":"2024-05-14T16:52:13.218367+0000","last_fullsized":"2024-05-14T16:52:13.218367+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:29:20.942783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.123308+0000","last_change":"2024-05-14T16:51:32.292107+0000","last_active":"2024-05-14T16:53:12.123308+0000","last_peered":"2024-05-14T16:53:12.123308+0000","last_clean":"2024-05-14T16:53:12.123308+0000","last_became_active":"2024-05-14T16:51:32.291729+0000","last_became_peered":"2024-05-14T16:51:32.291729+0000","last_unstale":"2024-05-14T16:53:12.123308+0000","last_undegraded":"2024-05-14T16:53:12.123308+0000","last_fullsized":"2024-05-14T16:53:12.123308+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:30:15.440599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.216788+0000","last_change":"2024-05-14T16:51:29.274862+0000","last_active":"2024-05-14T16:52:13.216788+0000","last_peered":"2024-05-14T16:52:13.216788+0000","last_clean":"2024-05-14T16:52:13.216788+0000","last_became_active":"2024-05-14T16:51:29.274527+0000","last_became_peered":"2024-05-14T16:51:29.274527+0000","last_unstale":"2024-05-14T16:52:13.216788+0000","last_undegraded":"2024-05-14T16:52:13.216788+0000","last_fullsized":"2024-05-14T16:52:13.216788+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:43:15.634585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.126844+0000","last_change":"2024-05-14T16:51:32.306158+0000","last_active":"2024-05-14T16:53:12.126844+0000","last_peered":"2024-05-14T16:53:12.126844+0000","last_clean":"2024-05-14T16:53:12.126844+0000","last_became_active":"2024-05-14T16:51:32.305834+0000","last_became_peered":"2024-05-14T16:51:32.305834+0000","last_unstale":"2024-05-14T16:53:12.126844+0000","last_undegraded":"2024-05-14T16:53:12.126844+0000","last_fullsized":"2024-05-14T16:53:12.126844+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:31:47.645862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'3","reported_seq":128,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:26.156857+0000","last_change":"2024-05-14T16:51:29.270735+0000","last_active":"2024-05-14T16:53:26.156857+0000","last_peered":"2024-05-14T16:53:26.156857+0000","last_clean":"2024-05-14T16:53:26.156857+0000","last_became_active":"2024-05-14T16:51:29.270362+0000","last_became_peered":"2024-05-14T16:51:29.270362+0000","last_unstale":"2024-05-14T16:53:26.156857+0000","last_undegraded":"2024-05-14T16:53:26.156857+0000","last_fullsized":"2024-05-14T16:53:26.156857+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:40:12.288655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.125112+0000","last_change":"2024-05-14T16:51:32.305007+0000","last_active":"2024-05-14T16:53:12.125112+0000","last_peered":"2024-05-14T16:53:12.125112+0000","last_clean":"2024-05-14T16:53:12.125112+0000","last_became_active":"2024-05-14T16:51:32.303927+0000","last_became_peered":"2024-05-14T16:51:32.303927+0000","last_unstale":"2024-05-14T16:53:12.125112+0000","last_undegraded":"2024-05-14T16:53:12.125112+0000","last_fullsized":"2024-05-14T16:53:12.125112+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:42:28.018492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":42,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.218464+0000","last_change":"2024-05-14T16:51:29.345902+0000","last_active":"2024-05-14T16:52:13.218464+0000","last_peered":"2024-05-14T16:52:13.218464+0000","last_clean":"2024-05-14T16:52:13.218464+0000","last_became_active":"2024-05-14T16:51:29.344904+0000","last_became_peered":"2024-05-14T16:51:29.344904+0000","last_unstale":"2024-05-14T16:52:13.218464+0000","last_undegraded":"2024-05-14T16:52:13.218464+0000","last_fullsized":"2024-05-14T16:52:13.218464+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:56:22.209067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.114101+0000","last_change":"2024-05-14T16:51:32.306459+0000","last_active":"2024-05-14T16:53:12.114101+0000","last_peered":"2024-05-14T16:53:12.114101+0000","last_clean":"2024-05-14T16:53:12.114101+0000","last_became_active":"2024-05-14T16:51:32.304897+0000","last_became_peered":"2024-05-14T16:51:32.304897+0000","last_unstale":"2024-05-14T16:53:12.114101+0000","last_undegraded":"2024-05-14T16:53:12.114101+0000","last_fullsized":"2024-05-14T16:53:12.114101+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:09:51.216631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.219952+0000","last_change":"2024-05-14T16:51:29.275219+0000","last_active":"2024-05-14T16:52:13.219952+0000","last_peered":"2024-05-14T16:52:13.219952+0000","last_clean":"2024-05-14T16:52:13.219952+0000","last_became_active":"2024-05-14T16:51:29.274816+0000","last_became_peered":"2024-05-14T16:51:29.274816+0000","last_unstale":"2024-05-14T16:52:13.219952+0000","last_undegraded":"2024-05-14T16:52:13.219952+0000","last_fullsized":"2024-05-14T16:52:13.219952+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:55:49.667387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.118605+0000","last_change":"2024-05-14T16:51:32.299284+0000","last_active":"2024-05-14T16:53:12.118605+0000","last_peered":"2024-05-14T16:53:12.118605+0000","last_clean":"2024-05-14T16:53:12.118605+0000","last_became_active":"2024-05-14T16:51:32.298933+0000","last_became_peered":"2024-05-14T16:51:32.298933+0000","last_unstale":"2024-05-14T16:53:12.118605+0000","last_undegraded":"2024-05-14T16:53:12.118605+0000","last_fullsized":"2024-05-14T16:53:12.118605+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:35:42.266665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.214563+0000","last_change":"2024-05-14T16:51:29.345454+0000","last_active":"2024-05-14T16:52:13.214563+0000","last_peered":"2024-05-14T16:52:13.214563+0000","last_clean":"2024-05-14T16:52:13.214563+0000","last_became_active":"2024-05-14T16:51:29.345074+0000","last_became_peered":"2024-05-14T16:51:29.345074+0000","last_unstale":"2024-05-14T16:52:13.214563+0000","last_undegraded":"2024-05-14T16:52:13.214563+0000","last_fullsized":"2024-05-14T16:52:13.214563+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:56:11.661144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.116755+0000","last_change":"2024-05-14T16:51:32.288804+0000","last_active":"2024-05-14T16:53:12.116755+0000","last_peered":"2024-05-14T16:53:12.116755+0000","last_clean":"2024-05-14T16:53:12.116755+0000","last_became_active":"2024-05-14T16:51:32.288441+0000","last_became_peered":"2024-05-14T16:51:32.288441+0000","last_unstale":"2024-05-14T16:53:12.116755+0000","last_undegraded":"2024-05-14T16:53:12.116755+0000","last_fullsized":"2024-05-14T16:53:12.116755+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:28:31.720342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215130+0000","last_change":"2024-05-14T16:51:29.272032+0000","last_active":"2024-05-14T16:52:13.215130+0000","last_peered":"2024-05-14T16:52:13.215130+0000","last_clean":"2024-05-14T16:52:13.215130+0000","last_became_active":"2024-05-14T16:51:29.271677+0000","last_became_peered":"2024-05-14T16:51:29.271677+0000","last_unstale":"2024-05-14T16:52:13.215130+0000","last_undegraded":"2024-05-14T16:52:13.215130+0000","last_fullsized":"2024-05-14T16:52:13.215130+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:39:41.377591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.121432+0000","last_change":"2024-05-14T16:51:32.295159+0000","last_active":"2024-05-14T16:53:12.121432+0000","last_peered":"2024-05-14T16:53:12.121432+0000","last_clean":"2024-05-14T16:53:12.121432+0000","last_became_active":"2024-05-14T16:51:32.294812+0000","last_became_peered":"2024-05-14T16:51:32.294812+0000","last_unstale":"2024-05-14T16:53:12.121432+0000","last_undegraded":"2024-05-14T16:53:12.121432+0000","last_fullsized":"2024-05-14T16:53:12.121432+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:08:22.411800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.116699+0000","last_change":"2024-05-14T16:51:32.306582+0000","last_active":"2024-05-14T16:53:12.116699+0000","last_peered":"2024-05-14T16:53:12.116699+0000","last_clean":"2024-05-14T16:53:12.116699+0000","last_became_active":"2024-05-14T16:51:32.306209+0000","last_became_peered":"2024-05-14T16:51:32.306209+0000","last_unstale":"2024-05-14T16:53:12.116699+0000","last_undegraded":"2024-05-14T16:53:12.116699+0000","last_fullsized":"2024-05-14T16:53:12.116699+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:30:46.264097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"78'4","reported_seq":124,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:23.150903+0000","last_change":"2024-05-14T16:51:29.345057+0000","last_active":"2024-05-14T16:53:23.150903+0000","last_peered":"2024-05-14T16:53:23.150903+0000","last_clean":"2024-05-14T16:53:23.150903+0000","last_became_active":"2024-05-14T16:51:29.344658+0000","last_became_peered":"2024-05-14T16:51:29.344658+0000","last_unstale":"2024-05-14T16:53:23.150903+0000","last_undegraded":"2024-05-14T16:53:23.150903+0000","last_fullsized":"2024-05-14T16:53:23.150903+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:43:53.007543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":94,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.129043+0000","last_change":"2024-05-14T16:51:32.298958+0000","last_active":"2024-05-14T16:53:12.129043+0000","last_peered":"2024-05-14T16:53:12.129043+0000","last_clean":"2024-05-14T16:53:12.129043+0000","last_became_active":"2024-05-14T16:51:32.298420+0000","last_became_peered":"2024-05-14T16:51:32.298420+0000","last_unstale":"2024-05-14T16:53:12.129043+0000","last_undegraded":"2024-05-14T16:53:12.129043+0000","last_fullsized":"2024-05-14T16:53:12.129043+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:15:21.262026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.921575+0000","last_change":"2024-05-14T16:51:29.272082+0000","last_active":"2024-05-14T16:52:13.921575+0000","last_peered":"2024-05-14T16:52:13.921575+0000","last_clean":"2024-05-14T16:52:13.921575+0000","last_became_active":"2024-05-14T16:51:29.271711+0000","last_became_peered":"2024-05-14T16:51:29.271711+0000","last_unstale":"2024-05-14T16:52:13.921575+0000","last_undegraded":"2024-05-14T16:52:13.921575+0000","last_fullsized":"2024-05-14T16:52:13.921575+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:27:44.320423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":100,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215066+0000","last_change":"2024-05-14T16:50:05.320136+0000","last_active":"2024-05-14T16:52:13.215066+0000","last_peered":"2024-05-14T16:52:13.215066+0000","last_clean":"2024-05-14T16:52:13.215066+0000","last_became_active":"2024-05-14T16:50:05.315016+0000","last_became_peered":"2024-05-14T16:50:05.315016+0000","last_unstale":"2024-05-14T16:52:13.215066+0000","last_undegraded":"2024-05-14T16:52:13.215066+0000","last_fullsized":"2024-05-14T16:52:13.215066+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:38.138211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:38.138211+0000","last_clean_scrub_stamp":"2024-05-14T16:46:38.138211+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:19:09.012578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.115955+0000","last_change":"2024-05-14T16:51:32.295995+0000","last_active":"2024-05-14T16:53:12.115955+0000","last_peered":"2024-05-14T16:53:12.115955+0000","last_clean":"2024-05-14T16:53:12.115955+0000","last_became_active":"2024-05-14T16:51:32.295640+0000","last_became_peered":"2024-05-14T16:51:32.295640+0000","last_unstale":"2024-05-14T16:53:12.115955+0000","last_undegraded":"2024-05-14T16:53:12.115955+0000","last_fullsized":"2024-05-14T16:53:12.115955+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:42:26.774916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215122+0000","last_change":"2024-05-14T16:51:29.274892+0000","last_active":"2024-05-14T16:52:13.215122+0000","last_peered":"2024-05-14T16:52:13.215122+0000","last_clean":"2024-05-14T16:52:13.215122+0000","last_became_active":"2024-05-14T16:51:29.274386+0000","last_became_peered":"2024-05-14T16:51:29.274386+0000","last_unstale":"2024-05-14T16:52:13.215122+0000","last_undegraded":"2024-05-14T16:52:13.215122+0000","last_fullsized":"2024-05-14T16:52:13.215122+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:16:54.899320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.219605+0000","last_change":"2024-05-14T16:51:29.345672+0000","last_active":"2024-05-14T16:52:13.219605+0000","last_peered":"2024-05-14T16:52:13.219605+0000","last_clean":"2024-05-14T16:52:13.219605+0000","last_became_active":"2024-05-14T16:51:29.344948+0000","last_became_peered":"2024-05-14T16:51:29.344948+0000","last_unstale":"2024-05-14T16:52:13.219605+0000","last_undegraded":"2024-05-14T16:52:13.219605+0000","last_fullsized":"2024-05-14T16:52:13.219605+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:19:26.577799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.116080+0000","last_change":"2024-05-14T16:51:32.297360+0000","last_active":"2024-05-14T16:53:12.116080+0000","last_peered":"2024-05-14T16:53:12.116080+0000","last_clean":"2024-05-14T16:53:12.116080+0000","last_became_active":"2024-05-14T16:51:32.297025+0000","last_became_peered":"2024-05-14T16:51:32.297025+0000","last_unstale":"2024-05-14T16:53:12.116080+0000","last_undegraded":"2024-05-14T16:53:12.116080+0000","last_fullsized":"2024-05-14T16:53:12.116080+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:05:39.705860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'11","reported_seq":49,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:15.302454+0000","last_change":"2024-05-14T16:52:13.224328+0000","last_active":"2024-05-14T16:52:15.302454+0000","last_peered":"2024-05-14T16:52:15.302454+0000","last_clean":"2024-05-14T16:52:15.302454+0000","last_became_active":"2024-05-14T16:52:13.220271+0000","last_became_peered":"2024-05-14T16:52:13.220271+0000","last_unstale":"2024-05-14T16:52:15.302454+0000","last_undegraded":"2024-05-14T16:52:15.302454+0000","last_fullsized":"2024-05-14T16:52:15.302454+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:06:31.955274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.126103+0000","last_change":"2024-05-14T16:51:32.292112+0000","last_active":"2024-05-14T16:53:12.126103+0000","last_peered":"2024-05-14T16:53:12.126103+0000","last_clean":"2024-05-14T16:53:12.126103+0000","last_became_active":"2024-05-14T16:51:32.291688+0000","last_became_peered":"2024-05-14T16:51:32.291688+0000","last_unstale":"2024-05-14T16:53:12.126103+0000","last_undegraded":"2024-05-14T16:53:12.126103+0000","last_fullsized":"2024-05-14T16:53:12.126103+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:13:06.728608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.921942+0000","last_change":"2024-05-14T16:51:29.412805+0000","last_active":"2024-05-14T16:52:13.921942+0000","last_peered":"2024-05-14T16:52:13.921942+0000","last_clean":"2024-05-14T16:52:13.921942+0000","last_became_active":"2024-05-14T16:51:29.412394+0000","last_became_peered":"2024-05-14T16:51:29.412394+0000","last_unstale":"2024-05-14T16:52:13.921942+0000","last_undegraded":"2024-05-14T16:52:13.921942+0000","last_fullsized":"2024-05-14T16:52:13.921942+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:49:48.721104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.123859+0000","last_change":"2024-05-14T16:52:13.218227+0000","last_active":"2024-05-14T16:53:12.123859+0000","last_peered":"2024-05-14T16:53:12.123859+0000","last_clean":"2024-05-14T16:53:12.123859+0000","last_became_active":"2024-05-14T16:52:13.217787+0000","last_became_peered":"2024-05-14T16:52:13.217787+0000","last_unstale":"2024-05-14T16:53:12.123859+0000","last_undegraded":"2024-05-14T16:53:12.123859+0000","last_fullsized":"2024-05-14T16:53:12.123859+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:52:33.348804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.212605+0000","last_change":"2024-05-14T16:51:29.345427+0000","last_active":"2024-05-14T16:52:13.212605+0000","last_peered":"2024-05-14T16:52:13.212605+0000","last_clean":"2024-05-14T16:52:13.212605+0000","last_became_active":"2024-05-14T16:51:29.345049+0000","last_became_peered":"2024-05-14T16:51:29.345049+0000","last_unstale":"2024-05-14T16:52:13.212605+0000","last_undegraded":"2024-05-14T16:52:13.212605+0000","last_fullsized":"2024-05-14T16:52:13.212605+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:06:19.799660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.076075+0000","last_change":"2024-05-14T16:51:32.298757+0000","last_active":"2024-05-14T16:53:12.076075+0000","last_peered":"2024-05-14T16:53:12.076075+0000","last_clean":"2024-05-14T16:53:12.076075+0000","last_became_active":"2024-05-14T16:51:32.297193+0000","last_became_peered":"2024-05-14T16:51:32.297193+0000","last_unstale":"2024-05-14T16:53:12.076075+0000","last_undegraded":"2024-05-14T16:53:12.076075+0000","last_fullsized":"2024-05-14T16:53:12.076075+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:10:19.672230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.212297+0000","last_change":"2024-05-14T16:51:29.270545+0000","last_active":"2024-05-14T16:52:13.212297+0000","last_peered":"2024-05-14T16:52:13.212297+0000","last_clean":"2024-05-14T16:52:13.212297+0000","last_became_active":"2024-05-14T16:51:29.270182+0000","last_became_peered":"2024-05-14T16:51:29.270182+0000","last_unstale":"2024-05-14T16:52:13.212297+0000","last_undegraded":"2024-05-14T16:52:13.212297+0000","last_fullsized":"2024-05-14T16:52:13.212297+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:19:49.977916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.115401+0000","last_change":"2024-05-14T16:51:32.294767+0000","last_active":"2024-05-14T16:53:12.115401+0000","last_peered":"2024-05-14T16:53:12.115401+0000","last_clean":"2024-05-14T16:53:12.115401+0000","last_became_active":"2024-05-14T16:51:32.293947+0000","last_became_peered":"2024-05-14T16:51:32.293947+0000","last_unstale":"2024-05-14T16:53:12.115401+0000","last_undegraded":"2024-05-14T16:53:12.115401+0000","last_fullsized":"2024-05-14T16:53:12.115401+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:30:21.731492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.226988+0000","last_change":"2024-05-14T16:52:13.226509+0000","last_active":"2024-05-14T16:52:13.226988+0000","last_peered":"2024-05-14T16:52:13.226988+0000","last_clean":"2024-05-14T16:52:13.226988+0000","last_became_active":"2024-05-14T16:52:13.220923+0000","last_became_peered":"2024-05-14T16:52:13.220923+0000","last_unstale":"2024-05-14T16:52:13.226988+0000","last_undegraded":"2024-05-14T16:52:13.226988+0000","last_fullsized":"2024-05-14T16:52:13.226988+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:25:52.655481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.110072+0000","last_change":"2024-05-14T16:51:32.288810+0000","last_active":"2024-05-14T16:53:12.110072+0000","last_peered":"2024-05-14T16:53:12.110072+0000","last_clean":"2024-05-14T16:53:12.110072+0000","last_became_active":"2024-05-14T16:51:32.288492+0000","last_became_peered":"2024-05-14T16:51:32.288492+0000","last_unstale":"2024-05-14T16:53:12.110072+0000","last_undegraded":"2024-05-14T16:53:12.110072+0000","last_fullsized":"2024-05-14T16:53:12.110072+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:44:53.393371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.218848+0000","last_change":"2024-05-14T16:51:29.281181+0000","last_active":"2024-05-14T16:52:13.218848+0000","last_peered":"2024-05-14T16:52:13.218848+0000","last_clean":"2024-05-14T16:52:13.218848+0000","last_became_active":"2024-05-14T16:51:29.279965+0000","last_became_peered":"2024-05-14T16:51:29.279965+0000","last_unstale":"2024-05-14T16:52:13.218848+0000","last_undegraded":"2024-05-14T16:52:13.218848+0000","last_fullsized":"2024-05-14T16:52:13.218848+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:41:00.735291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.130940+0000","last_change":"2024-05-14T16:51:32.296971+0000","last_active":"2024-05-14T16:53:12.130940+0000","last_peered":"2024-05-14T16:53:12.130940+0000","last_clean":"2024-05-14T16:53:12.130940+0000","last_became_active":"2024-05-14T16:51:32.296600+0000","last_became_peered":"2024-05-14T16:51:32.296600+0000","last_unstale":"2024-05-14T16:53:12.130940+0000","last_undegraded":"2024-05-14T16:53:12.130940+0000","last_fullsized":"2024-05-14T16:53:12.130940+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:13:14.354127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.219192+0000","last_change":"2024-05-14T16:51:29.412682+0000","last_active":"2024-05-14T16:52:13.219192+0000","last_peered":"2024-05-14T16:52:13.219192+0000","last_clean":"2024-05-14T16:52:13.219192+0000","last_became_active":"2024-05-14T16:51:29.412186+0000","last_became_peered":"2024-05-14T16:51:29.412186+0000","last_unstale":"2024-05-14T16:52:13.219192+0000","last_undegraded":"2024-05-14T16:52:13.219192+0000","last_fullsized":"2024-05-14T16:52:13.219192+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:56:24.570276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.130044+0000","last_change":"2024-05-14T16:51:32.300626+0000","last_active":"2024-05-14T16:53:12.130044+0000","last_peered":"2024-05-14T16:53:12.130044+0000","last_clean":"2024-05-14T16:53:12.130044+0000","last_became_active":"2024-05-14T16:51:32.300278+0000","last_became_peered":"2024-05-14T16:51:32.300278+0000","last_unstale":"2024-05-14T16:53:12.130044+0000","last_undegraded":"2024-05-14T16:53:12.130044+0000","last_fullsized":"2024-05-14T16:53:12.130044+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:59:14.196315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.216758+0000","last_change":"2024-05-14T16:51:29.270826+0000","last_active":"2024-05-14T16:52:13.216758+0000","last_peered":"2024-05-14T16:52:13.216758+0000","last_clean":"2024-05-14T16:52:13.216758+0000","last_became_active":"2024-05-14T16:51:29.270266+0000","last_became_peered":"2024-05-14T16:51:29.270266+0000","last_unstale":"2024-05-14T16:52:13.216758+0000","last_undegraded":"2024-05-14T16:52:13.216758+0000","last_fullsized":"2024-05-14T16:52:13.216758+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:00:55.349278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.125213+0000","last_change":"2024-05-14T16:52:13.223994+0000","last_active":"2024-05-14T16:53:12.125213+0000","last_peered":"2024-05-14T16:53:12.125213+0000","last_clean":"2024-05-14T16:53:12.125213+0000","last_became_active":"2024-05-14T16:52:13.221071+0000","last_became_peered":"2024-05-14T16:52:13.221071+0000","last_unstale":"2024-05-14T16:53:12.125213+0000","last_undegraded":"2024-05-14T16:53:12.125213+0000","last_fullsized":"2024-05-14T16:53:12.125213+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:29:58.775171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.921947+0000","last_change":"2024-05-14T16:51:29.276705+0000","last_active":"2024-05-14T16:52:13.921947+0000","last_peered":"2024-05-14T16:52:13.921947+0000","last_clean":"2024-05-14T16:52:13.921947+0000","last_became_active":"2024-05-14T16:51:29.276347+0000","last_became_peered":"2024-05-14T16:51:29.276347+0000","last_unstale":"2024-05-14T16:52:13.921947+0000","last_undegraded":"2024-05-14T16:52:13.921947+0000","last_fullsized":"2024-05-14T16:52:13.921947+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:32:56.064427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.125997+0000","last_change":"2024-05-14T16:51:32.301617+0000","last_active":"2024-05-14T16:53:12.125997+0000","last_peered":"2024-05-14T16:53:12.125997+0000","last_clean":"2024-05-14T16:53:12.125997+0000","last_became_active":"2024-05-14T16:51:32.301288+0000","last_became_peered":"2024-05-14T16:51:32.301288+0000","last_unstale":"2024-05-14T16:53:12.125997+0000","last_undegraded":"2024-05-14T16:53:12.125997+0000","last_fullsized":"2024-05-14T16:53:12.125997+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:19:24.253633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.115012+0000","last_change":"2024-05-14T16:52:13.221694+0000","last_active":"2024-05-14T16:53:12.115012+0000","last_peered":"2024-05-14T16:53:12.115012+0000","last_clean":"2024-05-14T16:53:12.115012+0000","last_became_active":"2024-05-14T16:52:13.221264+0000","last_became_peered":"2024-05-14T16:52:13.221264+0000","last_unstale":"2024-05-14T16:53:12.115012+0000","last_undegraded":"2024-05-14T16:53:12.115012+0000","last_fullsized":"2024-05-14T16:53:12.115012+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:22:52.738772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.119141+0000","last_change":"2024-05-14T16:51:32.301789+0000","last_active":"2024-05-14T16:53:12.119141+0000","last_peered":"2024-05-14T16:53:12.119141+0000","last_clean":"2024-05-14T16:53:12.119141+0000","last_became_active":"2024-05-14T16:51:32.301350+0000","last_became_peered":"2024-05-14T16:51:32.301350+0000","last_unstale":"2024-05-14T16:53:12.119141+0000","last_undegraded":"2024-05-14T16:53:12.119141+0000","last_fullsized":"2024-05-14T16:53:12.119141+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:46:24.896855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.114989+0000","last_change":"2024-05-14T16:51:32.295978+0000","last_active":"2024-05-14T16:53:12.114989+0000","last_peered":"2024-05-14T16:53:12.114989+0000","last_clean":"2024-05-14T16:53:12.114989+0000","last_became_active":"2024-05-14T16:51:32.295652+0000","last_became_peered":"2024-05-14T16:51:32.295652+0000","last_unstale":"2024-05-14T16:53:12.114989+0000","last_undegraded":"2024-05-14T16:53:12.114989+0000","last_fullsized":"2024-05-14T16:53:12.114989+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:32:17.743734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.110277+0000","last_change":"2024-05-14T16:51:32.305568+0000","last_active":"2024-05-14T16:53:12.110277+0000","last_peered":"2024-05-14T16:53:12.110277+0000","last_clean":"2024-05-14T16:53:12.110277+0000","last_became_active":"2024-05-14T16:51:32.304580+0000","last_became_peered":"2024-05-14T16:51:32.304580+0000","last_unstale":"2024-05-14T16:53:12.110277+0000","last_undegraded":"2024-05-14T16:53:12.110277+0000","last_fullsized":"2024-05-14T16:53:12.110277+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:32:44.465054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.115959+0000","last_change":"2024-05-14T16:51:32.295029+0000","last_active":"2024-05-14T16:53:12.115959+0000","last_peered":"2024-05-14T16:53:12.115959+0000","last_clean":"2024-05-14T16:53:12.115959+0000","last_became_active":"2024-05-14T16:51:32.294621+0000","last_became_peered":"2024-05-14T16:51:32.294621+0000","last_unstale":"2024-05-14T16:53:12.115959+0000","last_undegraded":"2024-05-14T16:53:12.115959+0000","last_fullsized":"2024-05-14T16:53:12.115959+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:52:05.390141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.115051+0000","last_change":"2024-05-14T16:51:32.297671+0000","last_active":"2024-05-14T16:53:12.115051+0000","last_peered":"2024-05-14T16:53:12.115051+0000","last_clean":"2024-05-14T16:53:12.115051+0000","last_became_active":"2024-05-14T16:51:32.296210+0000","last_became_peered":"2024-05-14T16:51:32.296210+0000","last_unstale":"2024-05-14T16:53:12.115051+0000","last_undegraded":"2024-05-14T16:53:12.115051+0000","last_fullsized":"2024-05-14T16:53:12.115051+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:55:04.789021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.128900+0000","last_change":"2024-05-14T16:51:32.297404+0000","last_active":"2024-05-14T16:53:12.128900+0000","last_peered":"2024-05-14T16:53:12.128900+0000","last_clean":"2024-05-14T16:53:12.128900+0000","last_became_active":"2024-05-14T16:51:32.296856+0000","last_became_peered":"2024-05-14T16:51:32.296856+0000","last_unstale":"2024-05-14T16:53:12.128900+0000","last_undegraded":"2024-05-14T16:53:12.128900+0000","last_fullsized":"2024-05-14T16:53:12.128900+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:25:29.992875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.111593+0000","last_change":"2024-05-14T16:52:13.221379+0000","last_active":"2024-05-14T16:53:12.111593+0000","last_peered":"2024-05-14T16:53:12.111593+0000","last_clean":"2024-05-14T16:53:12.111593+0000","last_became_active":"2024-05-14T16:52:13.220973+0000","last_became_peered":"2024-05-14T16:52:13.220973+0000","last_unstale":"2024-05-14T16:53:12.111593+0000","last_undegraded":"2024-05-14T16:53:12.111593+0000","last_fullsized":"2024-05-14T16:53:12.111593+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:55:08.212344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.115631+0000","last_change":"2024-05-14T16:51:32.303570+0000","last_active":"2024-05-14T16:53:12.115631+0000","last_peered":"2024-05-14T16:53:12.115631+0000","last_clean":"2024-05-14T16:53:12.115631+0000","last_became_active":"2024-05-14T16:51:32.303234+0000","last_became_peered":"2024-05-14T16:51:32.303234+0000","last_unstale":"2024-05-14T16:53:12.115631+0000","last_undegraded":"2024-05-14T16:53:12.115631+0000","last_fullsized":"2024-05-14T16:53:12.115631+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:27:57.960120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.117705+0000","last_change":"2024-05-14T16:51:32.298028+0000","last_active":"2024-05-14T16:53:12.117705+0000","last_peered":"2024-05-14T16:53:12.117705+0000","last_clean":"2024-05-14T16:53:12.117705+0000","last_became_active":"2024-05-14T16:51:32.295104+0000","last_became_peered":"2024-05-14T16:51:32.295104+0000","last_unstale":"2024-05-14T16:53:12.117705+0000","last_undegraded":"2024-05-14T16:53:12.117705+0000","last_fullsized":"2024-05-14T16:53:12.117705+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:58:08.441590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.131613+0000","last_change":"2024-05-14T16:51:32.304458+0000","last_active":"2024-05-14T16:53:12.131613+0000","last_peered":"2024-05-14T16:53:12.131613+0000","last_clean":"2024-05-14T16:53:12.131613+0000","last_became_active":"2024-05-14T16:51:32.302952+0000","last_became_peered":"2024-05-14T16:51:32.302952+0000","last_unstale":"2024-05-14T16:53:12.131613+0000","last_undegraded":"2024-05-14T16:53:12.131613+0000","last_fullsized":"2024-05-14T16:53:12.131613+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:40:14.220075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.123290+0000","last_change":"2024-05-14T16:51:32.292334+0000","last_active":"2024-05-14T16:53:12.123290+0000","last_peered":"2024-05-14T16:53:12.123290+0000","last_clean":"2024-05-14T16:53:12.123290+0000","last_became_active":"2024-05-14T16:51:32.291769+0000","last_became_peered":"2024-05-14T16:51:32.291769+0000","last_unstale":"2024-05-14T16:53:12.123290+0000","last_undegraded":"2024-05-14T16:53:12.123290+0000","last_fullsized":"2024-05-14T16:53:12.123290+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:55:25.694827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.118892+0000","last_change":"2024-05-14T16:51:32.300044+0000","last_active":"2024-05-14T16:53:12.118892+0000","last_peered":"2024-05-14T16:53:12.118892+0000","last_clean":"2024-05-14T16:53:12.118892+0000","last_became_active":"2024-05-14T16:51:32.299630+0000","last_became_peered":"2024-05-14T16:51:32.299630+0000","last_unstale":"2024-05-14T16:53:12.118892+0000","last_undegraded":"2024-05-14T16:53:12.118892+0000","last_fullsized":"2024-05-14T16:53:12.118892+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:50:40.516973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284433,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65632,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":597,"num_read_kb":364,"num_write":171,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":196896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710759,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":864256,"data_stored":425130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906988,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1015808,"data_stored":1078101,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070513,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":425483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266743,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":880640,"data_stored":425286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462971,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":929792,"data_stored":1017357,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626497,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36184,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708968,"statfs":{"total":95995035648,"available":95957983232,"internally_reserved":0,"allocated":868352,"data_stored":424755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":929792,"data_stored":486038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36204,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708948,"statfs":{"total":95995035648,"available":95957962752,"internally_reserved":0,"allocated":888832,"data_stored":426026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":487920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182480,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36196,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708956,"statfs":{"total":95995035648,"available":95957970944,"internally_reserved":0,"allocated":880640,"data_stored":424949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986252,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":872448,"data_stored":425192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822727,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":1017041,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34541,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34541,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34541,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T16:53:29.505 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-14T16:53:29.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:29 smithi151 ceph-mon[36066]: pgmap v348: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T16:53:29.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:29 smithi123 ceph-mon[35971]: pgmap v348: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T16:53:29.758 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:53:30.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:30 smithi002 ceph-mon[31538]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:53:30.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:30 smithi151 ceph-mon[36066]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:53:30.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:30 smithi123 ceph-mon[35971]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:53:31.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.436+0000 7f925da31700 1 -- 172.21.15.2:0/3999067245 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9258103850 msgr2=0x7f9258103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:31.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.437+0000 7f925da31700 1 --2- 172.21.15.2:0/3999067245 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9258103850 0x7f9258103c30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f92480099c0 tx=0x7f9248009cd0 comp rx=0 tx=0).stop 2024-05-14T16:53:31.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.437+0000 7f925da31700 1 -- 172.21.15.2:0/3999067245 shutdown_connections 2024-05-14T16:53:31.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.437+0000 7f925da31700 1 --2- 172.21.15.2:0/3999067245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9258108920 0x7f92581104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:31.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.437+0000 7f925da31700 1 --2- 172.21.15.2:0/3999067245 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9258103850 0x7f9258103c30 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:31.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.437+0000 7f925da31700 1 --2- 172.21.15.2:0/3999067245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9258104200 0x7f92581081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:31.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.437+0000 7f925da31700 1 -- 172.21.15.2:0/3999067245 >> 172.21.15.2:0/3999067245 conn(0x7f925807a620 msgr2=0x7f925807aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:31.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.437+0000 7f925da31700 1 -- 172.21.15.2:0/3999067245 shutdown_connections 2024-05-14T16:53:31.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.437+0000 7f925da31700 1 -- 172.21.15.2:0/3999067245 wait complete. 2024-05-14T16:53:31.438 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:31 smithi002 ceph-mon[31538]: pgmap v349: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T16:53:31.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.437+0000 7f925da31700 1 Processor -- start 2024-05-14T16:53:31.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.438+0000 7f925da31700 1 -- start start 2024-05-14T16:53:31.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.438+0000 7f925da31700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9258103850 0x7f925819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:31.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.438+0000 7f925da31700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9258104200 0x7f925819fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:31.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.438+0000 7f925da31700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9258108920 0x7f92581999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:31.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.438+0000 7f925da31700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9258113720 con 0x7f9258108920 2024-05-14T16:53:31.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.438+0000 7f925da31700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f92581135a0 con 0x7f9258103850 2024-05-14T16:53:31.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.438+0000 7f925da31700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f92581138a0 con 0x7f9258104200 2024-05-14T16:53:31.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.438+0000 7f92577fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9258108920 0x7f92581999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:31.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.438+0000 7f9256ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9258103850 0x7f925819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:31.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.438+0000 7f9256ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9258103850 0x7f925819f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:34692/0 (socket says 172.21.15.2:34692) 2024-05-14T16:53:31.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.438+0000 7f9256ffd700 1 -- 172.21.15.2:0/1822336645 learned_addr learned my addr 172.21.15.2:0/1822336645 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:53:31.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.438+0000 7f92567fc700 1 --2- 172.21.15.2:0/1822336645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9258104200 0x7f925819fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:31.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.439+0000 7f92577fe700 1 -- 172.21.15.2:0/1822336645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9258104200 msgr2=0x7f925819fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:31.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.439+0000 7f92577fe700 1 --2- 172.21.15.2:0/1822336645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9258104200 0x7f925819fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:31.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.439+0000 7f92577fe700 1 -- 172.21.15.2:0/1822336645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9258103850 msgr2=0x7f925819f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:31.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.439+0000 7f92577fe700 1 --2- 172.21.15.2:0/1822336645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9258103850 0x7f925819f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:31.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.439+0000 7f92577fe700 1 -- 172.21.15.2:0/1822336645 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f925819a140 con 0x7f9258108920 2024-05-14T16:53:31.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.439+0000 7f9256ffd700 1 --2- 172.21.15.2:0/1822336645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9258103850 0x7f925819f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:53:31.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.439+0000 7f92577fe700 1 --2- 172.21.15.2:0/1822336645 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9258108920 0x7f92581999e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9250007f10 tx=0x7f925000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:31.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.439+0000 7f925ca2f700 1 -- 172.21.15.2:0/1822336645 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9250004db0 con 0x7f9258108920 2024-05-14T16:53:31.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.439+0000 7f925ca2f700 1 -- 172.21.15.2:0/1822336645 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9250009e20 con 0x7f9258108920 2024-05-14T16:53:31.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.439+0000 7f925da31700 1 -- 172.21.15.2:0/1822336645 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f925819a430 con 0x7f9258108920 2024-05-14T16:53:31.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.439+0000 7f925ca2f700 1 -- 172.21.15.2:0/1822336645 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9250015400 con 0x7f9258108920 2024-05-14T16:53:31.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.439+0000 7f925da31700 1 -- 172.21.15.2:0/1822336645 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9258075770 con 0x7f9258108920 2024-05-14T16:53:31.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.440+0000 7f925da31700 1 -- 172.21.15.2:0/1822336645 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f925810bb60 con 0x7f9258108920 2024-05-14T16:53:31.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.442+0000 7f925ca2f700 1 -- 172.21.15.2:0/1822336645 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f9250005260 con 0x7f9258108920 2024-05-14T16:53:31.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.443+0000 7f925ca2f700 1 --2- 172.21.15.2:0/1822336645 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f92400b2880 0x7f92400b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:31.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.443+0000 7f9256ffd700 1 --2- 172.21.15.2:0/1822336645 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f92400b2880 0x7f92400b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:31.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.443+0000 7f925ca2f700 1 -- 172.21.15.2:0/1822336645 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f92500e2020 con 0x7f9258108920 2024-05-14T16:53:31.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.443+0000 7f9256ffd700 1 --2- 172.21.15.2:0/1822336645 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f92400b2880 0x7f92400b4d40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9248003800 tx=0x7f9248005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:31.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.446+0000 7f925ca2f700 1 -- 172.21.15.2:0/1822336645 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9250010070 con 0x7f9258108920 2024-05-14T16:53:31.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:31 smithi151 ceph-mon[36066]: pgmap v349: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T16:53:31.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:31 smithi123 ceph-mon[35971]: pgmap v349: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T16:53:31.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.663+0000 7f925da31700 1 -- 172.21.15.2:0/1822336645 --> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9258063980 con 0x7f92400b2880 2024-05-14T16:53:31.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.670+0000 7f925ca2f700 1 -- 172.21.15.2:0/1822336645 <== mgr.14150 v2:172.21.15.2:6800/1498936028 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258986 (secure 0 0 0) 0x7f9258063980 con 0x7f92400b2880 2024-05-14T16:53:31.673 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:53:31.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.674+0000 7f925da31700 1 -- 172.21.15.2:0/1822336645 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f92400b2880 msgr2=0x7f92400b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:31.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.674+0000 7f925da31700 1 --2- 172.21.15.2:0/1822336645 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f92400b2880 0x7f92400b4d40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9248003800 tx=0x7f9248005fb0 comp rx=0 tx=0).stop 2024-05-14T16:53:31.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.674+0000 7f925da31700 1 -- 172.21.15.2:0/1822336645 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9258108920 msgr2=0x7f92581999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:31.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.674+0000 7f925da31700 1 --2- 172.21.15.2:0/1822336645 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9258108920 0x7f92581999e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9250007f10 tx=0x7f925000d3b0 comp rx=0 tx=0).stop 2024-05-14T16:53:31.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.674+0000 7f925da31700 1 -- 172.21.15.2:0/1822336645 shutdown_connections 2024-05-14T16:53:31.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.674+0000 7f925da31700 1 --2- 172.21.15.2:0/1822336645 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f92400b2880 0x7f92400b4d40 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:31.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.674+0000 7f925da31700 1 --2- 172.21.15.2:0/1822336645 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9258108920 0x7f92581999e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:31.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.674+0000 7f925da31700 1 --2- 172.21.15.2:0/1822336645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9258103850 0x7f925819f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:31.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.674+0000 7f925da31700 1 --2- 172.21.15.2:0/1822336645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9258104200 0x7f925819fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:31.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.674+0000 7f925da31700 1 -- 172.21.15.2:0/1822336645 >> 172.21.15.2:0/1822336645 conn(0x7f925807a620 msgr2=0x7f9258109a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:31.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.674+0000 7f925da31700 1 -- 172.21.15.2:0/1822336645 shutdown_connections 2024-05-14T16:53:31.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:31.674+0000 7f925da31700 1 -- 172.21.15.2:0/1822336645 wait complete. 2024-05-14T16:53:31.677 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-05-14T16:53:32.332 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2024-05-14T16:53:30.113969+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656234,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":698,"num_read_kb":448,"num_write":296,"num_write_kb":2268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":2951957,"num_keys_recovered":9,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":739,"ondisk_log_size":739,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":384768,"kb_used_data":10656,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124557056,"statfs":{"total":1151940427776,"available":1151546425344,"internally_reserved":0,"allocated":10911744,"data_stored":7063278,"data_compressed":108486,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":45,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003122"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.129790+0000","last_change":"2024-05-14T16:51:32.298580+0000","last_active":"2024-05-14T16:53:12.129790+0000","last_peered":"2024-05-14T16:53:12.129790+0000","last_clean":"2024-05-14T16:53:12.129790+0000","last_became_active":"2024-05-14T16:51:32.296945+0000","last_became_peered":"2024-05-14T16:51:32.296945+0000","last_unstale":"2024-05-14T16:53:12.129790+0000","last_undegraded":"2024-05-14T16:53:12.129790+0000","last_fullsized":"2024-05-14T16:53:12.129790+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:50:17.664928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.116489+0000","last_change":"2024-05-14T16:51:32.297618+0000","last_active":"2024-05-14T16:53:12.116489+0000","last_peered":"2024-05-14T16:53:12.116489+0000","last_clean":"2024-05-14T16:53:12.116489+0000","last_became_active":"2024-05-14T16:51:32.297260+0000","last_became_peered":"2024-05-14T16:51:32.297260+0000","last_unstale":"2024-05-14T16:53:12.116489+0000","last_undegraded":"2024-05-14T16:53:12.116489+0000","last_fullsized":"2024-05-14T16:53:12.116489+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:43:50.542587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.128312+0000","last_change":"2024-05-14T16:51:32.295525+0000","last_active":"2024-05-14T16:53:12.128312+0000","last_peered":"2024-05-14T16:53:12.128312+0000","last_clean":"2024-05-14T16:53:12.128312+0000","last_became_active":"2024-05-14T16:51:32.295215+0000","last_became_peered":"2024-05-14T16:51:32.295215+0000","last_unstale":"2024-05-14T16:53:12.128312+0000","last_undegraded":"2024-05-14T16:53:12.128312+0000","last_fullsized":"2024-05-14T16:53:12.128312+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:40:05.516280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.121502+0000","last_change":"2024-05-14T16:51:32.307883+0000","last_active":"2024-05-14T16:53:12.121502+0000","last_peered":"2024-05-14T16:53:12.121502+0000","last_clean":"2024-05-14T16:53:12.121502+0000","last_became_active":"2024-05-14T16:51:32.307499+0000","last_became_peered":"2024-05-14T16:51:32.307499+0000","last_unstale":"2024-05-14T16:53:12.121502+0000","last_undegraded":"2024-05-14T16:53:12.121502+0000","last_fullsized":"2024-05-14T16:53:12.121502+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:30:59.978059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.105483+0000","last_change":"2024-05-14T16:51:32.294429+0000","last_active":"2024-05-14T16:53:12.105483+0000","last_peered":"2024-05-14T16:53:12.105483+0000","last_clean":"2024-05-14T16:53:12.105483+0000","last_became_active":"2024-05-14T16:51:32.294088+0000","last_became_peered":"2024-05-14T16:51:32.294088+0000","last_unstale":"2024-05-14T16:53:12.105483+0000","last_undegraded":"2024-05-14T16:53:12.105483+0000","last_fullsized":"2024-05-14T16:53:12.105483+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:25:19.383586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.115996+0000","last_change":"2024-05-14T16:51:32.296169+0000","last_active":"2024-05-14T16:53:12.115996+0000","last_peered":"2024-05-14T16:53:12.115996+0000","last_clean":"2024-05-14T16:53:12.115996+0000","last_became_active":"2024-05-14T16:51:32.295819+0000","last_became_peered":"2024-05-14T16:51:32.295819+0000","last_unstale":"2024-05-14T16:53:12.115996+0000","last_undegraded":"2024-05-14T16:53:12.115996+0000","last_fullsized":"2024-05-14T16:53:12.115996+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:56:23.932039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.125249+0000","last_change":"2024-05-14T16:51:32.309609+0000","last_active":"2024-05-14T16:53:12.125249+0000","last_peered":"2024-05-14T16:53:12.125249+0000","last_clean":"2024-05-14T16:53:12.125249+0000","last_became_active":"2024-05-14T16:51:32.309175+0000","last_became_peered":"2024-05-14T16:51:32.309175+0000","last_unstale":"2024-05-14T16:53:12.125249+0000","last_undegraded":"2024-05-14T16:53:12.125249+0000","last_fullsized":"2024-05-14T16:53:12.125249+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:06:46.717592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.117730+0000","last_change":"2024-05-14T16:52:13.446592+0000","last_active":"2024-05-14T16:53:12.117730+0000","last_peered":"2024-05-14T16:53:12.117730+0000","last_clean":"2024-05-14T16:53:12.117730+0000","last_became_active":"2024-05-14T16:52:13.446238+0000","last_became_peered":"2024-05-14T16:52:13.446238+0000","last_unstale":"2024-05-14T16:53:12.117730+0000","last_undegraded":"2024-05-14T16:53:12.117730+0000","last_fullsized":"2024-05-14T16:53:12.117730+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:33:14.216797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.131561+0000","last_change":"2024-05-14T16:51:32.307571+0000","last_active":"2024-05-14T16:53:12.131561+0000","last_peered":"2024-05-14T16:53:12.131561+0000","last_clean":"2024-05-14T16:53:12.131561+0000","last_became_active":"2024-05-14T16:51:32.307151+0000","last_became_peered":"2024-05-14T16:51:32.307151+0000","last_unstale":"2024-05-14T16:53:12.131561+0000","last_undegraded":"2024-05-14T16:53:12.131561+0000","last_fullsized":"2024-05-14T16:53:12.131561+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:17:26.325393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.111451+0000","last_change":"2024-05-14T16:51:32.298773+0000","last_active":"2024-05-14T16:53:12.111451+0000","last_peered":"2024-05-14T16:53:12.111451+0000","last_clean":"2024-05-14T16:53:12.111451+0000","last_became_active":"2024-05-14T16:51:32.298283+0000","last_became_peered":"2024-05-14T16:51:32.298283+0000","last_unstale":"2024-05-14T16:53:12.111451+0000","last_undegraded":"2024-05-14T16:53:12.111451+0000","last_fullsized":"2024-05-14T16:53:12.111451+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:52:18.471201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.129073+0000","last_change":"2024-05-14T16:51:32.302443+0000","last_active":"2024-05-14T16:53:12.129073+0000","last_peered":"2024-05-14T16:53:12.129073+0000","last_clean":"2024-05-14T16:53:12.129073+0000","last_became_active":"2024-05-14T16:51:32.302034+0000","last_became_peered":"2024-05-14T16:51:32.302034+0000","last_unstale":"2024-05-14T16:53:12.129073+0000","last_undegraded":"2024-05-14T16:53:12.129073+0000","last_fullsized":"2024-05-14T16:53:12.129073+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:53:08.041293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.121182+0000","last_change":"2024-05-14T16:51:32.304837+0000","last_active":"2024-05-14T16:53:12.121182+0000","last_peered":"2024-05-14T16:53:12.121182+0000","last_clean":"2024-05-14T16:53:12.121182+0000","last_became_active":"2024-05-14T16:51:32.304500+0000","last_became_peered":"2024-05-14T16:51:32.304500+0000","last_unstale":"2024-05-14T16:53:12.121182+0000","last_undegraded":"2024-05-14T16:53:12.121182+0000","last_fullsized":"2024-05-14T16:53:12.121182+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:32:02.713044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.117386+0000","last_change":"2024-05-14T16:51:32.299247+0000","last_active":"2024-05-14T16:53:12.117386+0000","last_peered":"2024-05-14T16:53:12.117386+0000","last_clean":"2024-05-14T16:53:12.117386+0000","last_became_active":"2024-05-14T16:51:32.298925+0000","last_became_peered":"2024-05-14T16:51:32.298925+0000","last_unstale":"2024-05-14T16:53:12.117386+0000","last_undegraded":"2024-05-14T16:53:12.117386+0000","last_fullsized":"2024-05-14T16:53:12.117386+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:10:38.847877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.117049+0000","last_change":"2024-05-14T16:51:32.293743+0000","last_active":"2024-05-14T16:53:12.117049+0000","last_peered":"2024-05-14T16:53:12.117049+0000","last_clean":"2024-05-14T16:53:12.117049+0000","last_became_active":"2024-05-14T16:51:32.293429+0000","last_became_peered":"2024-05-14T16:51:32.293429+0000","last_unstale":"2024-05-14T16:53:12.117049+0000","last_undegraded":"2024-05-14T16:53:12.117049+0000","last_fullsized":"2024-05-14T16:53:12.117049+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:12:33.909940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.113527+0000","last_change":"2024-05-14T16:51:32.304940+0000","last_active":"2024-05-14T16:53:12.113527+0000","last_peered":"2024-05-14T16:53:12.113527+0000","last_clean":"2024-05-14T16:53:12.113527+0000","last_became_active":"2024-05-14T16:51:32.304482+0000","last_became_peered":"2024-05-14T16:51:32.304482+0000","last_unstale":"2024-05-14T16:53:12.113527+0000","last_undegraded":"2024-05-14T16:53:12.113527+0000","last_fullsized":"2024-05-14T16:53:12.113527+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:54:17.771620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.131342+0000","last_change":"2024-05-14T16:51:32.301784+0000","last_active":"2024-05-14T16:53:12.131342+0000","last_peered":"2024-05-14T16:53:12.131342+0000","last_clean":"2024-05-14T16:53:12.131342+0000","last_became_active":"2024-05-14T16:51:32.301472+0000","last_became_peered":"2024-05-14T16:51:32.301472+0000","last_unstale":"2024-05-14T16:53:12.131342+0000","last_undegraded":"2024-05-14T16:53:12.131342+0000","last_fullsized":"2024-05-14T16:53:12.131342+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:10:56.735424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.130053+0000","last_change":"2024-05-14T16:51:32.297951+0000","last_active":"2024-05-14T16:53:12.130053+0000","last_peered":"2024-05-14T16:53:12.130053+0000","last_clean":"2024-05-14T16:53:12.130053+0000","last_became_active":"2024-05-14T16:51:32.296602+0000","last_became_peered":"2024-05-14T16:51:32.296602+0000","last_unstale":"2024-05-14T16:53:12.130053+0000","last_undegraded":"2024-05-14T16:53:12.130053+0000","last_fullsized":"2024-05-14T16:53:12.130053+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:16:36.039077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.116374+0000","last_change":"2024-05-14T16:51:32.290398+0000","last_active":"2024-05-14T16:53:12.116374+0000","last_peered":"2024-05-14T16:53:12.116374+0000","last_clean":"2024-05-14T16:53:12.116374+0000","last_became_active":"2024-05-14T16:51:32.290083+0000","last_became_peered":"2024-05-14T16:51:32.290083+0000","last_unstale":"2024-05-14T16:53:12.116374+0000","last_undegraded":"2024-05-14T16:53:12.116374+0000","last_fullsized":"2024-05-14T16:53:12.116374+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:48:05.795209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.117279+0000","last_change":"2024-05-14T16:51:32.294398+0000","last_active":"2024-05-14T16:53:12.117279+0000","last_peered":"2024-05-14T16:53:12.117279+0000","last_clean":"2024-05-14T16:53:12.117279+0000","last_became_active":"2024-05-14T16:51:32.294080+0000","last_became_peered":"2024-05-14T16:51:32.294080+0000","last_unstale":"2024-05-14T16:53:12.117279+0000","last_undegraded":"2024-05-14T16:53:12.117279+0000","last_fullsized":"2024-05-14T16:53:12.117279+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:15:03.859045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.214939+0000","last_change":"2024-05-14T16:51:29.274482+0000","last_active":"2024-05-14T16:52:13.214939+0000","last_peered":"2024-05-14T16:52:13.214939+0000","last_clean":"2024-05-14T16:52:13.214939+0000","last_became_active":"2024-05-14T16:51:29.274152+0000","last_became_peered":"2024-05-14T16:51:29.274152+0000","last_unstale":"2024-05-14T16:52:13.214939+0000","last_undegraded":"2024-05-14T16:52:13.214939+0000","last_fullsized":"2024-05-14T16:52:13.214939+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:05:38.274248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.112062+0000","last_change":"2024-05-14T16:51:32.300956+0000","last_active":"2024-05-14T16:53:12.112062+0000","last_peered":"2024-05-14T16:53:12.112062+0000","last_clean":"2024-05-14T16:53:12.112062+0000","last_became_active":"2024-05-14T16:51:32.300546+0000","last_became_peered":"2024-05-14T16:51:32.300546+0000","last_unstale":"2024-05-14T16:53:12.112062+0000","last_undegraded":"2024-05-14T16:53:12.112062+0000","last_fullsized":"2024-05-14T16:53:12.112062+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:54:09.181036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.212599+0000","last_change":"2024-05-14T16:51:29.345206+0000","last_active":"2024-05-14T16:52:13.212599+0000","last_peered":"2024-05-14T16:52:13.212599+0000","last_clean":"2024-05-14T16:52:13.212599+0000","last_became_active":"2024-05-14T16:51:29.344699+0000","last_became_peered":"2024-05-14T16:51:29.344699+0000","last_unstale":"2024-05-14T16:52:13.212599+0000","last_undegraded":"2024-05-14T16:52:13.212599+0000","last_fullsized":"2024-05-14T16:52:13.212599+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:14:02.364661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.128243+0000","last_change":"2024-05-14T16:51:32.297522+0000","last_active":"2024-05-14T16:53:12.128243+0000","last_peered":"2024-05-14T16:53:12.128243+0000","last_clean":"2024-05-14T16:53:12.128243+0000","last_became_active":"2024-05-14T16:51:32.296866+0000","last_became_peered":"2024-05-14T16:51:32.296866+0000","last_unstale":"2024-05-14T16:53:12.128243+0000","last_undegraded":"2024-05-14T16:53:12.128243+0000","last_fullsized":"2024-05-14T16:53:12.128243+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:49:43.409575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.219763+0000","last_change":"2024-05-14T16:51:29.280632+0000","last_active":"2024-05-14T16:52:13.219763+0000","last_peered":"2024-05-14T16:52:13.219763+0000","last_clean":"2024-05-14T16:52:13.219763+0000","last_became_active":"2024-05-14T16:51:29.280246+0000","last_became_peered":"2024-05-14T16:51:29.280246+0000","last_unstale":"2024-05-14T16:52:13.219763+0000","last_undegraded":"2024-05-14T16:52:13.219763+0000","last_fullsized":"2024-05-14T16:52:13.219763+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:34:47.925344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.131865+0000","last_change":"2024-05-14T16:51:32.298284+0000","last_active":"2024-05-14T16:53:12.131865+0000","last_peered":"2024-05-14T16:53:12.131865+0000","last_clean":"2024-05-14T16:53:12.131865+0000","last_became_active":"2024-05-14T16:51:32.294202+0000","last_became_peered":"2024-05-14T16:51:32.294202+0000","last_unstale":"2024-05-14T16:53:12.131865+0000","last_undegraded":"2024-05-14T16:53:12.131865+0000","last_fullsized":"2024-05-14T16:53:12.131865+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:27:37.561275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.219221+0000","last_change":"2024-05-14T16:51:29.276227+0000","last_active":"2024-05-14T16:52:13.219221+0000","last_peered":"2024-05-14T16:52:13.219221+0000","last_clean":"2024-05-14T16:52:13.219221+0000","last_became_active":"2024-05-14T16:51:29.275837+0000","last_became_peered":"2024-05-14T16:51:29.275837+0000","last_unstale":"2024-05-14T16:52:13.219221+0000","last_undegraded":"2024-05-14T16:52:13.219221+0000","last_fullsized":"2024-05-14T16:52:13.219221+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:03:29.402985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.129798+0000","last_change":"2024-05-14T16:51:32.308173+0000","last_active":"2024-05-14T16:53:12.129798+0000","last_peered":"2024-05-14T16:53:12.129798+0000","last_clean":"2024-05-14T16:53:12.129798+0000","last_became_active":"2024-05-14T16:51:32.307135+0000","last_became_peered":"2024-05-14T16:51:32.307135+0000","last_unstale":"2024-05-14T16:53:12.129798+0000","last_undegraded":"2024-05-14T16:53:12.129798+0000","last_fullsized":"2024-05-14T16:53:12.129798+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:01:46.579968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'7","reported_seq":134,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:26.157545+0000","last_change":"2024-05-14T16:51:29.279486+0000","last_active":"2024-05-14T16:53:26.157545+0000","last_peered":"2024-05-14T16:53:26.157545+0000","last_clean":"2024-05-14T16:53:26.157545+0000","last_became_active":"2024-05-14T16:51:29.279133+0000","last_became_peered":"2024-05-14T16:51:29.279133+0000","last_unstale":"2024-05-14T16:53:26.157545+0000","last_undegraded":"2024-05-14T16:53:26.157545+0000","last_fullsized":"2024-05-14T16:53:26.157545+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:26:18.620421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.110557+0000","last_change":"2024-05-14T16:51:32.299852+0000","last_active":"2024-05-14T16:53:12.110557+0000","last_peered":"2024-05-14T16:53:12.110557+0000","last_clean":"2024-05-14T16:53:12.110557+0000","last_became_active":"2024-05-14T16:51:32.299541+0000","last_became_peered":"2024-05-14T16:51:32.299541+0000","last_unstale":"2024-05-14T16:53:12.110557+0000","last_undegraded":"2024-05-14T16:53:12.110557+0000","last_fullsized":"2024-05-14T16:53:12.110557+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:36:38.191009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.445109+0000","last_change":"2024-05-14T16:51:29.275269+0000","last_active":"2024-05-14T16:52:13.445109+0000","last_peered":"2024-05-14T16:52:13.445109+0000","last_clean":"2024-05-14T16:52:13.445109+0000","last_became_active":"2024-05-14T16:51:29.274792+0000","last_became_peered":"2024-05-14T16:51:29.274792+0000","last_unstale":"2024-05-14T16:52:13.445109+0000","last_undegraded":"2024-05-14T16:52:13.445109+0000","last_fullsized":"2024-05-14T16:52:13.445109+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:17:08.925289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.110428+0000","last_change":"2024-05-14T16:51:32.306536+0000","last_active":"2024-05-14T16:53:12.110428+0000","last_peered":"2024-05-14T16:53:12.110428+0000","last_clean":"2024-05-14T16:53:12.110428+0000","last_became_active":"2024-05-14T16:51:32.304976+0000","last_became_peered":"2024-05-14T16:51:32.304976+0000","last_unstale":"2024-05-14T16:53:12.110428+0000","last_undegraded":"2024-05-14T16:53:12.110428+0000","last_fullsized":"2024-05-14T16:53:12.110428+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:48:13.284022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.921481+0000","last_change":"2024-05-14T16:51:29.266783+0000","last_active":"2024-05-14T16:52:13.921481+0000","last_peered":"2024-05-14T16:52:13.921481+0000","last_clean":"2024-05-14T16:52:13.921481+0000","last_became_active":"2024-05-14T16:51:29.266409+0000","last_became_peered":"2024-05-14T16:51:29.266409+0000","last_unstale":"2024-05-14T16:52:13.921481+0000","last_undegraded":"2024-05-14T16:52:13.921481+0000","last_fullsized":"2024-05-14T16:52:13.921481+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:33:14.981709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.116994+0000","last_change":"2024-05-14T16:51:32.289288+0000","last_active":"2024-05-14T16:53:12.116994+0000","last_peered":"2024-05-14T16:53:12.116994+0000","last_clean":"2024-05-14T16:53:12.116994+0000","last_became_active":"2024-05-14T16:51:32.288920+0000","last_became_peered":"2024-05-14T16:51:32.288920+0000","last_unstale":"2024-05-14T16:53:12.116994+0000","last_undegraded":"2024-05-14T16:53:12.116994+0000","last_fullsized":"2024-05-14T16:53:12.116994+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:23:43.659417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.216697+0000","last_change":"2024-05-14T16:51:29.273926+0000","last_active":"2024-05-14T16:52:13.216697+0000","last_peered":"2024-05-14T16:52:13.216697+0000","last_clean":"2024-05-14T16:52:13.216697+0000","last_became_active":"2024-05-14T16:51:29.273059+0000","last_became_peered":"2024-05-14T16:51:29.273059+0000","last_unstale":"2024-05-14T16:52:13.216697+0000","last_undegraded":"2024-05-14T16:52:13.216697+0000","last_fullsized":"2024-05-14T16:52:13.216697+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:46:27.800481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.117050+0000","last_change":"2024-05-14T16:51:32.287554+0000","last_active":"2024-05-14T16:53:12.117050+0000","last_peered":"2024-05-14T16:53:12.117050+0000","last_clean":"2024-05-14T16:53:12.117050+0000","last_became_active":"2024-05-14T16:51:32.287204+0000","last_became_peered":"2024-05-14T16:51:32.287204+0000","last_unstale":"2024-05-14T16:53:12.117050+0000","last_undegraded":"2024-05-14T16:53:12.117050+0000","last_fullsized":"2024-05-14T16:53:12.117050+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:29:40.675378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":128,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:27.158345+0000","last_change":"2024-05-14T16:51:29.274819+0000","last_active":"2024-05-14T16:53:27.158345+0000","last_peered":"2024-05-14T16:53:27.158345+0000","last_clean":"2024-05-14T16:53:27.158345+0000","last_became_active":"2024-05-14T16:51:29.274454+0000","last_became_peered":"2024-05-14T16:51:29.274454+0000","last_unstale":"2024-05-14T16:53:27.158345+0000","last_undegraded":"2024-05-14T16:53:27.158345+0000","last_fullsized":"2024-05-14T16:53:27.158345+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:36:30.496064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.130381+0000","last_change":"2024-05-14T16:51:32.295732+0000","last_active":"2024-05-14T16:53:12.130381+0000","last_peered":"2024-05-14T16:53:12.130381+0000","last_clean":"2024-05-14T16:53:12.130381+0000","last_became_active":"2024-05-14T16:51:32.294919+0000","last_became_peered":"2024-05-14T16:51:32.294919+0000","last_unstale":"2024-05-14T16:53:12.130381+0000","last_undegraded":"2024-05-14T16:53:12.130381+0000","last_fullsized":"2024-05-14T16:53:12.130381+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:08:54.566601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215472+0000","last_change":"2024-05-14T16:51:29.279338+0000","last_active":"2024-05-14T16:52:13.215472+0000","last_peered":"2024-05-14T16:52:13.215472+0000","last_clean":"2024-05-14T16:52:13.215472+0000","last_became_active":"2024-05-14T16:51:29.278928+0000","last_became_peered":"2024-05-14T16:51:29.278928+0000","last_unstale":"2024-05-14T16:52:13.215472+0000","last_undegraded":"2024-05-14T16:52:13.215472+0000","last_fullsized":"2024-05-14T16:52:13.215472+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:24:34.803515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.113307+0000","last_change":"2024-05-14T16:51:32.305294+0000","last_active":"2024-05-14T16:53:12.113307+0000","last_peered":"2024-05-14T16:53:12.113307+0000","last_clean":"2024-05-14T16:53:12.113307+0000","last_became_active":"2024-05-14T16:51:32.304563+0000","last_became_peered":"2024-05-14T16:51:32.304563+0000","last_unstale":"2024-05-14T16:53:12.113307+0000","last_undegraded":"2024-05-14T16:53:12.113307+0000","last_fullsized":"2024-05-14T16:53:12.113307+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:56:06.259475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.218071+0000","last_change":"2024-05-14T16:51:29.345400+0000","last_active":"2024-05-14T16:52:13.218071+0000","last_peered":"2024-05-14T16:52:13.218071+0000","last_clean":"2024-05-14T16:52:13.218071+0000","last_became_active":"2024-05-14T16:51:29.345010+0000","last_became_peered":"2024-05-14T16:51:29.345010+0000","last_unstale":"2024-05-14T16:52:13.218071+0000","last_undegraded":"2024-05-14T16:52:13.218071+0000","last_fullsized":"2024-05-14T16:52:13.218071+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:24:22.933005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.131849+0000","last_change":"2024-05-14T16:52:13.923167+0000","last_active":"2024-05-14T16:53:12.131849+0000","last_peered":"2024-05-14T16:53:12.131849+0000","last_clean":"2024-05-14T16:53:12.131849+0000","last_became_active":"2024-05-14T16:52:13.922813+0000","last_became_peered":"2024-05-14T16:52:13.922813+0000","last_unstale":"2024-05-14T16:53:12.131849+0000","last_undegraded":"2024-05-14T16:53:12.131849+0000","last_fullsized":"2024-05-14T16:53:12.131849+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:50:38.672408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'33","reported_seq":265,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:28.160123+0000","last_change":"2024-05-14T16:51:29.277591+0000","last_active":"2024-05-14T16:53:28.160123+0000","last_peered":"2024-05-14T16:53:28.160123+0000","last_clean":"2024-05-14T16:53:28.160123+0000","last_became_active":"2024-05-14T16:51:29.276612+0000","last_became_peered":"2024-05-14T16:51:29.276612+0000","last_unstale":"2024-05-14T16:53:28.160123+0000","last_undegraded":"2024-05-14T16:53:28.160123+0000","last_fullsized":"2024-05-14T16:53:28.160123+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:22:28.934173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61490,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":206,"num_read_kb":162,"num_write":34,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.117319+0000","last_change":"2024-05-14T16:51:32.302321+0000","last_active":"2024-05-14T16:53:12.117319+0000","last_peered":"2024-05-14T16:53:12.117319+0000","last_clean":"2024-05-14T16:53:12.117319+0000","last_became_active":"2024-05-14T16:51:32.301973+0000","last_became_peered":"2024-05-14T16:51:32.301973+0000","last_unstale":"2024-05-14T16:53:12.117319+0000","last_undegraded":"2024-05-14T16:53:12.117319+0000","last_fullsized":"2024-05-14T16:53:12.117319+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:49:24.054379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.218367+0000","last_change":"2024-05-14T16:51:29.278776+0000","last_active":"2024-05-14T16:52:13.218367+0000","last_peered":"2024-05-14T16:52:13.218367+0000","last_clean":"2024-05-14T16:52:13.218367+0000","last_became_active":"2024-05-14T16:51:29.278395+0000","last_became_peered":"2024-05-14T16:51:29.278395+0000","last_unstale":"2024-05-14T16:52:13.218367+0000","last_undegraded":"2024-05-14T16:52:13.218367+0000","last_fullsized":"2024-05-14T16:52:13.218367+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:29:20.942783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.123308+0000","last_change":"2024-05-14T16:51:32.292107+0000","last_active":"2024-05-14T16:53:12.123308+0000","last_peered":"2024-05-14T16:53:12.123308+0000","last_clean":"2024-05-14T16:53:12.123308+0000","last_became_active":"2024-05-14T16:51:32.291729+0000","last_became_peered":"2024-05-14T16:51:32.291729+0000","last_unstale":"2024-05-14T16:53:12.123308+0000","last_undegraded":"2024-05-14T16:53:12.123308+0000","last_fullsized":"2024-05-14T16:53:12.123308+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:30:15.440599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.216788+0000","last_change":"2024-05-14T16:51:29.274862+0000","last_active":"2024-05-14T16:52:13.216788+0000","last_peered":"2024-05-14T16:52:13.216788+0000","last_clean":"2024-05-14T16:52:13.216788+0000","last_became_active":"2024-05-14T16:51:29.274527+0000","last_became_peered":"2024-05-14T16:51:29.274527+0000","last_unstale":"2024-05-14T16:52:13.216788+0000","last_undegraded":"2024-05-14T16:52:13.216788+0000","last_fullsized":"2024-05-14T16:52:13.216788+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:43:15.634585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.126844+0000","last_change":"2024-05-14T16:51:32.306158+0000","last_active":"2024-05-14T16:53:12.126844+0000","last_peered":"2024-05-14T16:53:12.126844+0000","last_clean":"2024-05-14T16:53:12.126844+0000","last_became_active":"2024-05-14T16:51:32.305834+0000","last_became_peered":"2024-05-14T16:51:32.305834+0000","last_unstale":"2024-05-14T16:53:12.126844+0000","last_undegraded":"2024-05-14T16:53:12.126844+0000","last_fullsized":"2024-05-14T16:53:12.126844+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:31:47.645862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'3","reported_seq":128,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:26.156857+0000","last_change":"2024-05-14T16:51:29.270735+0000","last_active":"2024-05-14T16:53:26.156857+0000","last_peered":"2024-05-14T16:53:26.156857+0000","last_clean":"2024-05-14T16:53:26.156857+0000","last_became_active":"2024-05-14T16:51:29.270362+0000","last_became_peered":"2024-05-14T16:51:29.270362+0000","last_unstale":"2024-05-14T16:53:26.156857+0000","last_undegraded":"2024-05-14T16:53:26.156857+0000","last_fullsized":"2024-05-14T16:53:26.156857+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:40:12.288655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.125112+0000","last_change":"2024-05-14T16:51:32.305007+0000","last_active":"2024-05-14T16:53:12.125112+0000","last_peered":"2024-05-14T16:53:12.125112+0000","last_clean":"2024-05-14T16:53:12.125112+0000","last_became_active":"2024-05-14T16:51:32.303927+0000","last_became_peered":"2024-05-14T16:51:32.303927+0000","last_unstale":"2024-05-14T16:53:12.125112+0000","last_undegraded":"2024-05-14T16:53:12.125112+0000","last_fullsized":"2024-05-14T16:53:12.125112+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:42:28.018492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":42,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.218464+0000","last_change":"2024-05-14T16:51:29.345902+0000","last_active":"2024-05-14T16:52:13.218464+0000","last_peered":"2024-05-14T16:52:13.218464+0000","last_clean":"2024-05-14T16:52:13.218464+0000","last_became_active":"2024-05-14T16:51:29.344904+0000","last_became_peered":"2024-05-14T16:51:29.344904+0000","last_unstale":"2024-05-14T16:52:13.218464+0000","last_undegraded":"2024-05-14T16:52:13.218464+0000","last_fullsized":"2024-05-14T16:52:13.218464+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:56:22.209067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.114101+0000","last_change":"2024-05-14T16:51:32.306459+0000","last_active":"2024-05-14T16:53:12.114101+0000","last_peered":"2024-05-14T16:53:12.114101+0000","last_clean":"2024-05-14T16:53:12.114101+0000","last_became_active":"2024-05-14T16:51:32.304897+0000","last_became_peered":"2024-05-14T16:51:32.304897+0000","last_unstale":"2024-05-14T16:53:12.114101+0000","last_undegraded":"2024-05-14T16:53:12.114101+0000","last_fullsized":"2024-05-14T16:53:12.114101+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:09:51.216631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.219952+0000","last_change":"2024-05-14T16:51:29.275219+0000","last_active":"2024-05-14T16:52:13.219952+0000","last_peered":"2024-05-14T16:52:13.219952+0000","last_clean":"2024-05-14T16:52:13.219952+0000","last_became_active":"2024-05-14T16:51:29.274816+0000","last_became_peered":"2024-05-14T16:51:29.274816+0000","last_unstale":"2024-05-14T16:52:13.219952+0000","last_undegraded":"2024-05-14T16:52:13.219952+0000","last_fullsized":"2024-05-14T16:52:13.219952+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:55:49.667387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.118605+0000","last_change":"2024-05-14T16:51:32.299284+0000","last_active":"2024-05-14T16:53:12.118605+0000","last_peered":"2024-05-14T16:53:12.118605+0000","last_clean":"2024-05-14T16:53:12.118605+0000","last_became_active":"2024-05-14T16:51:32.298933+0000","last_became_peered":"2024-05-14T16:51:32.298933+0000","last_unstale":"2024-05-14T16:53:12.118605+0000","last_undegraded":"2024-05-14T16:53:12.118605+0000","last_fullsized":"2024-05-14T16:53:12.118605+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:35:42.266665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.214563+0000","last_change":"2024-05-14T16:51:29.345454+0000","last_active":"2024-05-14T16:52:13.214563+0000","last_peered":"2024-05-14T16:52:13.214563+0000","last_clean":"2024-05-14T16:52:13.214563+0000","last_became_active":"2024-05-14T16:51:29.345074+0000","last_became_peered":"2024-05-14T16:51:29.345074+0000","last_unstale":"2024-05-14T16:52:13.214563+0000","last_undegraded":"2024-05-14T16:52:13.214563+0000","last_fullsized":"2024-05-14T16:52:13.214563+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:56:11.661144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.116755+0000","last_change":"2024-05-14T16:51:32.288804+0000","last_active":"2024-05-14T16:53:12.116755+0000","last_peered":"2024-05-14T16:53:12.116755+0000","last_clean":"2024-05-14T16:53:12.116755+0000","last_became_active":"2024-05-14T16:51:32.288441+0000","last_became_peered":"2024-05-14T16:51:32.288441+0000","last_unstale":"2024-05-14T16:53:12.116755+0000","last_undegraded":"2024-05-14T16:53:12.116755+0000","last_fullsized":"2024-05-14T16:53:12.116755+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:28:31.720342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215130+0000","last_change":"2024-05-14T16:51:29.272032+0000","last_active":"2024-05-14T16:52:13.215130+0000","last_peered":"2024-05-14T16:52:13.215130+0000","last_clean":"2024-05-14T16:52:13.215130+0000","last_became_active":"2024-05-14T16:51:29.271677+0000","last_became_peered":"2024-05-14T16:51:29.271677+0000","last_unstale":"2024-05-14T16:52:13.215130+0000","last_undegraded":"2024-05-14T16:52:13.215130+0000","last_fullsized":"2024-05-14T16:52:13.215130+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:39:41.377591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.121432+0000","last_change":"2024-05-14T16:51:32.295159+0000","last_active":"2024-05-14T16:53:12.121432+0000","last_peered":"2024-05-14T16:53:12.121432+0000","last_clean":"2024-05-14T16:53:12.121432+0000","last_became_active":"2024-05-14T16:51:32.294812+0000","last_became_peered":"2024-05-14T16:51:32.294812+0000","last_unstale":"2024-05-14T16:53:12.121432+0000","last_undegraded":"2024-05-14T16:53:12.121432+0000","last_fullsized":"2024-05-14T16:53:12.121432+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:08:22.411800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.116699+0000","last_change":"2024-05-14T16:51:32.306582+0000","last_active":"2024-05-14T16:53:12.116699+0000","last_peered":"2024-05-14T16:53:12.116699+0000","last_clean":"2024-05-14T16:53:12.116699+0000","last_became_active":"2024-05-14T16:51:32.306209+0000","last_became_peered":"2024-05-14T16:51:32.306209+0000","last_unstale":"2024-05-14T16:53:12.116699+0000","last_undegraded":"2024-05-14T16:53:12.116699+0000","last_fullsized":"2024-05-14T16:53:12.116699+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:30:46.264097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"78'4","reported_seq":129,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:28.159461+0000","last_change":"2024-05-14T16:51:29.345057+0000","last_active":"2024-05-14T16:53:28.159461+0000","last_peered":"2024-05-14T16:53:28.159461+0000","last_clean":"2024-05-14T16:53:28.159461+0000","last_became_active":"2024-05-14T16:51:29.344658+0000","last_became_peered":"2024-05-14T16:51:29.344658+0000","last_unstale":"2024-05-14T16:53:28.159461+0000","last_undegraded":"2024-05-14T16:53:28.159461+0000","last_fullsized":"2024-05-14T16:53:28.159461+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:43:53.007543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.129043+0000","last_change":"2024-05-14T16:51:32.298958+0000","last_active":"2024-05-14T16:53:12.129043+0000","last_peered":"2024-05-14T16:53:12.129043+0000","last_clean":"2024-05-14T16:53:12.129043+0000","last_became_active":"2024-05-14T16:51:32.298420+0000","last_became_peered":"2024-05-14T16:51:32.298420+0000","last_unstale":"2024-05-14T16:53:12.129043+0000","last_undegraded":"2024-05-14T16:53:12.129043+0000","last_fullsized":"2024-05-14T16:53:12.129043+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:15:21.262026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.921575+0000","last_change":"2024-05-14T16:51:29.272082+0000","last_active":"2024-05-14T16:52:13.921575+0000","last_peered":"2024-05-14T16:52:13.921575+0000","last_clean":"2024-05-14T16:52:13.921575+0000","last_became_active":"2024-05-14T16:51:29.271711+0000","last_became_peered":"2024-05-14T16:51:29.271711+0000","last_unstale":"2024-05-14T16:52:13.921575+0000","last_undegraded":"2024-05-14T16:52:13.921575+0000","last_fullsized":"2024-05-14T16:52:13.921575+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:27:44.320423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":100,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215066+0000","last_change":"2024-05-14T16:50:05.320136+0000","last_active":"2024-05-14T16:52:13.215066+0000","last_peered":"2024-05-14T16:52:13.215066+0000","last_clean":"2024-05-14T16:52:13.215066+0000","last_became_active":"2024-05-14T16:50:05.315016+0000","last_became_peered":"2024-05-14T16:50:05.315016+0000","last_unstale":"2024-05-14T16:52:13.215066+0000","last_undegraded":"2024-05-14T16:52:13.215066+0000","last_fullsized":"2024-05-14T16:52:13.215066+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:38.138211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:38.138211+0000","last_clean_scrub_stamp":"2024-05-14T16:46:38.138211+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:19:09.012578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.115955+0000","last_change":"2024-05-14T16:51:32.295995+0000","last_active":"2024-05-14T16:53:12.115955+0000","last_peered":"2024-05-14T16:53:12.115955+0000","last_clean":"2024-05-14T16:53:12.115955+0000","last_became_active":"2024-05-14T16:51:32.295640+0000","last_became_peered":"2024-05-14T16:51:32.295640+0000","last_unstale":"2024-05-14T16:53:12.115955+0000","last_undegraded":"2024-05-14T16:53:12.115955+0000","last_fullsized":"2024-05-14T16:53:12.115955+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:42:26.774916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.215178+0000","last_change":"2024-05-14T16:51:29.274892+0000","last_active":"2024-05-14T16:52:13.215178+0000","last_peered":"2024-05-14T16:52:13.215178+0000","last_clean":"2024-05-14T16:52:13.215178+0000","last_became_active":"2024-05-14T16:51:29.274386+0000","last_became_peered":"2024-05-14T16:51:29.274386+0000","last_unstale":"2024-05-14T16:52:13.215178+0000","last_undegraded":"2024-05-14T16:52:13.215178+0000","last_fullsized":"2024-05-14T16:52:13.215178+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:16:54.899320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.219605+0000","last_change":"2024-05-14T16:51:29.345672+0000","last_active":"2024-05-14T16:52:13.219605+0000","last_peered":"2024-05-14T16:52:13.219605+0000","last_clean":"2024-05-14T16:52:13.219605+0000","last_became_active":"2024-05-14T16:51:29.344948+0000","last_became_peered":"2024-05-14T16:51:29.344948+0000","last_unstale":"2024-05-14T16:52:13.219605+0000","last_undegraded":"2024-05-14T16:52:13.219605+0000","last_fullsized":"2024-05-14T16:52:13.219605+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:19:26.577799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.116080+0000","last_change":"2024-05-14T16:51:32.297360+0000","last_active":"2024-05-14T16:53:12.116080+0000","last_peered":"2024-05-14T16:53:12.116080+0000","last_clean":"2024-05-14T16:53:12.116080+0000","last_became_active":"2024-05-14T16:51:32.297025+0000","last_became_peered":"2024-05-14T16:51:32.297025+0000","last_unstale":"2024-05-14T16:53:12.116080+0000","last_undegraded":"2024-05-14T16:53:12.116080+0000","last_fullsized":"2024-05-14T16:53:12.116080+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:05:39.705860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'11","reported_seq":49,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:15.302454+0000","last_change":"2024-05-14T16:52:13.224328+0000","last_active":"2024-05-14T16:52:15.302454+0000","last_peered":"2024-05-14T16:52:15.302454+0000","last_clean":"2024-05-14T16:52:15.302454+0000","last_became_active":"2024-05-14T16:52:13.220271+0000","last_became_peered":"2024-05-14T16:52:13.220271+0000","last_unstale":"2024-05-14T16:52:15.302454+0000","last_undegraded":"2024-05-14T16:52:15.302454+0000","last_fullsized":"2024-05-14T16:52:15.302454+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:06:31.955274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.126103+0000","last_change":"2024-05-14T16:51:32.292112+0000","last_active":"2024-05-14T16:53:12.126103+0000","last_peered":"2024-05-14T16:53:12.126103+0000","last_clean":"2024-05-14T16:53:12.126103+0000","last_became_active":"2024-05-14T16:51:32.291688+0000","last_became_peered":"2024-05-14T16:51:32.291688+0000","last_unstale":"2024-05-14T16:53:12.126103+0000","last_undegraded":"2024-05-14T16:53:12.126103+0000","last_fullsized":"2024-05-14T16:53:12.126103+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:13:06.728608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.921942+0000","last_change":"2024-05-14T16:51:29.412805+0000","last_active":"2024-05-14T16:52:13.921942+0000","last_peered":"2024-05-14T16:52:13.921942+0000","last_clean":"2024-05-14T16:52:13.921942+0000","last_became_active":"2024-05-14T16:51:29.412394+0000","last_became_peered":"2024-05-14T16:51:29.412394+0000","last_unstale":"2024-05-14T16:52:13.921942+0000","last_undegraded":"2024-05-14T16:52:13.921942+0000","last_fullsized":"2024-05-14T16:52:13.921942+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:49:48.721104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.123859+0000","last_change":"2024-05-14T16:52:13.218227+0000","last_active":"2024-05-14T16:53:12.123859+0000","last_peered":"2024-05-14T16:53:12.123859+0000","last_clean":"2024-05-14T16:53:12.123859+0000","last_became_active":"2024-05-14T16:52:13.217787+0000","last_became_peered":"2024-05-14T16:52:13.217787+0000","last_unstale":"2024-05-14T16:53:12.123859+0000","last_undegraded":"2024-05-14T16:53:12.123859+0000","last_fullsized":"2024-05-14T16:53:12.123859+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:52:33.348804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.212605+0000","last_change":"2024-05-14T16:51:29.345427+0000","last_active":"2024-05-14T16:52:13.212605+0000","last_peered":"2024-05-14T16:52:13.212605+0000","last_clean":"2024-05-14T16:52:13.212605+0000","last_became_active":"2024-05-14T16:51:29.345049+0000","last_became_peered":"2024-05-14T16:51:29.345049+0000","last_unstale":"2024-05-14T16:52:13.212605+0000","last_undegraded":"2024-05-14T16:52:13.212605+0000","last_fullsized":"2024-05-14T16:52:13.212605+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:06:19.799660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.076075+0000","last_change":"2024-05-14T16:51:32.298757+0000","last_active":"2024-05-14T16:53:12.076075+0000","last_peered":"2024-05-14T16:53:12.076075+0000","last_clean":"2024-05-14T16:53:12.076075+0000","last_became_active":"2024-05-14T16:51:32.297193+0000","last_became_peered":"2024-05-14T16:51:32.297193+0000","last_unstale":"2024-05-14T16:53:12.076075+0000","last_undegraded":"2024-05-14T16:53:12.076075+0000","last_fullsized":"2024-05-14T16:53:12.076075+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:10:19.672230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.212297+0000","last_change":"2024-05-14T16:51:29.270545+0000","last_active":"2024-05-14T16:52:13.212297+0000","last_peered":"2024-05-14T16:52:13.212297+0000","last_clean":"2024-05-14T16:52:13.212297+0000","last_became_active":"2024-05-14T16:51:29.270182+0000","last_became_peered":"2024-05-14T16:51:29.270182+0000","last_unstale":"2024-05-14T16:52:13.212297+0000","last_undegraded":"2024-05-14T16:52:13.212297+0000","last_fullsized":"2024-05-14T16:52:13.212297+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:19:49.977916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.115401+0000","last_change":"2024-05-14T16:51:32.294767+0000","last_active":"2024-05-14T16:53:12.115401+0000","last_peered":"2024-05-14T16:53:12.115401+0000","last_clean":"2024-05-14T16:53:12.115401+0000","last_became_active":"2024-05-14T16:51:32.293947+0000","last_became_peered":"2024-05-14T16:51:32.293947+0000","last_unstale":"2024-05-14T16:53:12.115401+0000","last_undegraded":"2024-05-14T16:53:12.115401+0000","last_fullsized":"2024-05-14T16:53:12.115401+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:30:21.731492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.226988+0000","last_change":"2024-05-14T16:52:13.226509+0000","last_active":"2024-05-14T16:52:13.226988+0000","last_peered":"2024-05-14T16:52:13.226988+0000","last_clean":"2024-05-14T16:52:13.226988+0000","last_became_active":"2024-05-14T16:52:13.220923+0000","last_became_peered":"2024-05-14T16:52:13.220923+0000","last_unstale":"2024-05-14T16:52:13.226988+0000","last_undegraded":"2024-05-14T16:52:13.226988+0000","last_fullsized":"2024-05-14T16:52:13.226988+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:25:52.655481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.110072+0000","last_change":"2024-05-14T16:51:32.288810+0000","last_active":"2024-05-14T16:53:12.110072+0000","last_peered":"2024-05-14T16:53:12.110072+0000","last_clean":"2024-05-14T16:53:12.110072+0000","last_became_active":"2024-05-14T16:51:32.288492+0000","last_became_peered":"2024-05-14T16:51:32.288492+0000","last_unstale":"2024-05-14T16:53:12.110072+0000","last_undegraded":"2024-05-14T16:53:12.110072+0000","last_fullsized":"2024-05-14T16:53:12.110072+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:44:53.393371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.218848+0000","last_change":"2024-05-14T16:51:29.281181+0000","last_active":"2024-05-14T16:52:13.218848+0000","last_peered":"2024-05-14T16:52:13.218848+0000","last_clean":"2024-05-14T16:52:13.218848+0000","last_became_active":"2024-05-14T16:51:29.279965+0000","last_became_peered":"2024-05-14T16:51:29.279965+0000","last_unstale":"2024-05-14T16:52:13.218848+0000","last_undegraded":"2024-05-14T16:52:13.218848+0000","last_fullsized":"2024-05-14T16:52:13.218848+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:41:00.735291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.130940+0000","last_change":"2024-05-14T16:51:32.296971+0000","last_active":"2024-05-14T16:53:12.130940+0000","last_peered":"2024-05-14T16:53:12.130940+0000","last_clean":"2024-05-14T16:53:12.130940+0000","last_became_active":"2024-05-14T16:51:32.296600+0000","last_became_peered":"2024-05-14T16:51:32.296600+0000","last_unstale":"2024-05-14T16:53:12.130940+0000","last_undegraded":"2024-05-14T16:53:12.130940+0000","last_fullsized":"2024-05-14T16:53:12.130940+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:13:14.354127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.219192+0000","last_change":"2024-05-14T16:51:29.412682+0000","last_active":"2024-05-14T16:52:13.219192+0000","last_peered":"2024-05-14T16:52:13.219192+0000","last_clean":"2024-05-14T16:52:13.219192+0000","last_became_active":"2024-05-14T16:51:29.412186+0000","last_became_peered":"2024-05-14T16:51:29.412186+0000","last_unstale":"2024-05-14T16:52:13.219192+0000","last_undegraded":"2024-05-14T16:52:13.219192+0000","last_fullsized":"2024-05-14T16:52:13.219192+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:56:24.570276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.130044+0000","last_change":"2024-05-14T16:51:32.300626+0000","last_active":"2024-05-14T16:53:12.130044+0000","last_peered":"2024-05-14T16:53:12.130044+0000","last_clean":"2024-05-14T16:53:12.130044+0000","last_became_active":"2024-05-14T16:51:32.300278+0000","last_became_peered":"2024-05-14T16:51:32.300278+0000","last_unstale":"2024-05-14T16:53:12.130044+0000","last_undegraded":"2024-05-14T16:53:12.130044+0000","last_fullsized":"2024-05-14T16:53:12.130044+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:59:14.196315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.216758+0000","last_change":"2024-05-14T16:51:29.270826+0000","last_active":"2024-05-14T16:52:13.216758+0000","last_peered":"2024-05-14T16:52:13.216758+0000","last_clean":"2024-05-14T16:52:13.216758+0000","last_became_active":"2024-05-14T16:51:29.270266+0000","last_became_peered":"2024-05-14T16:51:29.270266+0000","last_unstale":"2024-05-14T16:52:13.216758+0000","last_undegraded":"2024-05-14T16:52:13.216758+0000","last_fullsized":"2024-05-14T16:52:13.216758+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:00:55.349278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.125213+0000","last_change":"2024-05-14T16:52:13.223994+0000","last_active":"2024-05-14T16:53:12.125213+0000","last_peered":"2024-05-14T16:53:12.125213+0000","last_clean":"2024-05-14T16:53:12.125213+0000","last_became_active":"2024-05-14T16:52:13.221071+0000","last_became_peered":"2024-05-14T16:52:13.221071+0000","last_unstale":"2024-05-14T16:53:12.125213+0000","last_undegraded":"2024-05-14T16:53:12.125213+0000","last_fullsized":"2024-05-14T16:53:12.125213+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:29:58.775171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:52:13.921947+0000","last_change":"2024-05-14T16:51:29.276705+0000","last_active":"2024-05-14T16:52:13.921947+0000","last_peered":"2024-05-14T16:52:13.921947+0000","last_clean":"2024-05-14T16:52:13.921947+0000","last_became_active":"2024-05-14T16:51:29.276347+0000","last_became_peered":"2024-05-14T16:51:29.276347+0000","last_unstale":"2024-05-14T16:52:13.921947+0000","last_undegraded":"2024-05-14T16:52:13.921947+0000","last_fullsized":"2024-05-14T16:52:13.921947+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:32:56.064427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.125997+0000","last_change":"2024-05-14T16:51:32.301617+0000","last_active":"2024-05-14T16:53:12.125997+0000","last_peered":"2024-05-14T16:53:12.125997+0000","last_clean":"2024-05-14T16:53:12.125997+0000","last_became_active":"2024-05-14T16:51:32.301288+0000","last_became_peered":"2024-05-14T16:51:32.301288+0000","last_unstale":"2024-05-14T16:53:12.125997+0000","last_undegraded":"2024-05-14T16:53:12.125997+0000","last_fullsized":"2024-05-14T16:53:12.125997+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:19:24.253633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.115012+0000","last_change":"2024-05-14T16:52:13.221694+0000","last_active":"2024-05-14T16:53:12.115012+0000","last_peered":"2024-05-14T16:53:12.115012+0000","last_clean":"2024-05-14T16:53:12.115012+0000","last_became_active":"2024-05-14T16:52:13.221264+0000","last_became_peered":"2024-05-14T16:52:13.221264+0000","last_unstale":"2024-05-14T16:53:12.115012+0000","last_undegraded":"2024-05-14T16:53:12.115012+0000","last_fullsized":"2024-05-14T16:53:12.115012+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:22:52.738772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.119141+0000","last_change":"2024-05-14T16:51:32.301789+0000","last_active":"2024-05-14T16:53:12.119141+0000","last_peered":"2024-05-14T16:53:12.119141+0000","last_clean":"2024-05-14T16:53:12.119141+0000","last_became_active":"2024-05-14T16:51:32.301350+0000","last_became_peered":"2024-05-14T16:51:32.301350+0000","last_unstale":"2024-05-14T16:53:12.119141+0000","last_undegraded":"2024-05-14T16:53:12.119141+0000","last_fullsized":"2024-05-14T16:53:12.119141+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:46:24.896855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.114989+0000","last_change":"2024-05-14T16:51:32.295978+0000","last_active":"2024-05-14T16:53:12.114989+0000","last_peered":"2024-05-14T16:53:12.114989+0000","last_clean":"2024-05-14T16:53:12.114989+0000","last_became_active":"2024-05-14T16:51:32.295652+0000","last_became_peered":"2024-05-14T16:51:32.295652+0000","last_unstale":"2024-05-14T16:53:12.114989+0000","last_undegraded":"2024-05-14T16:53:12.114989+0000","last_fullsized":"2024-05-14T16:53:12.114989+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:32:17.743734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.110277+0000","last_change":"2024-05-14T16:51:32.305568+0000","last_active":"2024-05-14T16:53:12.110277+0000","last_peered":"2024-05-14T16:53:12.110277+0000","last_clean":"2024-05-14T16:53:12.110277+0000","last_became_active":"2024-05-14T16:51:32.304580+0000","last_became_peered":"2024-05-14T16:51:32.304580+0000","last_unstale":"2024-05-14T16:53:12.110277+0000","last_undegraded":"2024-05-14T16:53:12.110277+0000","last_fullsized":"2024-05-14T16:53:12.110277+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:32:44.465054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.115959+0000","last_change":"2024-05-14T16:51:32.295029+0000","last_active":"2024-05-14T16:53:12.115959+0000","last_peered":"2024-05-14T16:53:12.115959+0000","last_clean":"2024-05-14T16:53:12.115959+0000","last_became_active":"2024-05-14T16:51:32.294621+0000","last_became_peered":"2024-05-14T16:51:32.294621+0000","last_unstale":"2024-05-14T16:53:12.115959+0000","last_undegraded":"2024-05-14T16:53:12.115959+0000","last_fullsized":"2024-05-14T16:53:12.115959+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:52:05.390141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.115051+0000","last_change":"2024-05-14T16:51:32.297671+0000","last_active":"2024-05-14T16:53:12.115051+0000","last_peered":"2024-05-14T16:53:12.115051+0000","last_clean":"2024-05-14T16:53:12.115051+0000","last_became_active":"2024-05-14T16:51:32.296210+0000","last_became_peered":"2024-05-14T16:51:32.296210+0000","last_unstale":"2024-05-14T16:53:12.115051+0000","last_undegraded":"2024-05-14T16:53:12.115051+0000","last_fullsized":"2024-05-14T16:53:12.115051+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:55:04.789021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.128900+0000","last_change":"2024-05-14T16:51:32.297404+0000","last_active":"2024-05-14T16:53:12.128900+0000","last_peered":"2024-05-14T16:53:12.128900+0000","last_clean":"2024-05-14T16:53:12.128900+0000","last_became_active":"2024-05-14T16:51:32.296856+0000","last_became_peered":"2024-05-14T16:51:32.296856+0000","last_unstale":"2024-05-14T16:53:12.128900+0000","last_undegraded":"2024-05-14T16:53:12.128900+0000","last_fullsized":"2024-05-14T16:53:12.128900+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:25:29.992875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.111593+0000","last_change":"2024-05-14T16:52:13.221379+0000","last_active":"2024-05-14T16:53:12.111593+0000","last_peered":"2024-05-14T16:53:12.111593+0000","last_clean":"2024-05-14T16:53:12.111593+0000","last_became_active":"2024-05-14T16:52:13.220973+0000","last_became_peered":"2024-05-14T16:52:13.220973+0000","last_unstale":"2024-05-14T16:53:12.111593+0000","last_undegraded":"2024-05-14T16:53:12.111593+0000","last_fullsized":"2024-05-14T16:53:12.111593+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:55:08.212344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.115631+0000","last_change":"2024-05-14T16:51:32.303570+0000","last_active":"2024-05-14T16:53:12.115631+0000","last_peered":"2024-05-14T16:53:12.115631+0000","last_clean":"2024-05-14T16:53:12.115631+0000","last_became_active":"2024-05-14T16:51:32.303234+0000","last_became_peered":"2024-05-14T16:51:32.303234+0000","last_unstale":"2024-05-14T16:53:12.115631+0000","last_undegraded":"2024-05-14T16:53:12.115631+0000","last_fullsized":"2024-05-14T16:53:12.115631+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:27:57.960120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.117705+0000","last_change":"2024-05-14T16:51:32.298028+0000","last_active":"2024-05-14T16:53:12.117705+0000","last_peered":"2024-05-14T16:53:12.117705+0000","last_clean":"2024-05-14T16:53:12.117705+0000","last_became_active":"2024-05-14T16:51:32.295104+0000","last_became_peered":"2024-05-14T16:51:32.295104+0000","last_unstale":"2024-05-14T16:53:12.117705+0000","last_undegraded":"2024-05-14T16:53:12.117705+0000","last_fullsized":"2024-05-14T16:53:12.117705+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:58:08.441590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.131613+0000","last_change":"2024-05-14T16:51:32.304458+0000","last_active":"2024-05-14T16:53:12.131613+0000","last_peered":"2024-05-14T16:53:12.131613+0000","last_clean":"2024-05-14T16:53:12.131613+0000","last_became_active":"2024-05-14T16:51:32.302952+0000","last_became_peered":"2024-05-14T16:51:32.302952+0000","last_unstale":"2024-05-14T16:53:12.131613+0000","last_undegraded":"2024-05-14T16:53:12.131613+0000","last_fullsized":"2024-05-14T16:53:12.131613+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:40:14.220075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.123290+0000","last_change":"2024-05-14T16:51:32.292334+0000","last_active":"2024-05-14T16:53:12.123290+0000","last_peered":"2024-05-14T16:53:12.123290+0000","last_clean":"2024-05-14T16:53:12.123290+0000","last_became_active":"2024-05-14T16:51:32.291769+0000","last_became_peered":"2024-05-14T16:51:32.291769+0000","last_unstale":"2024-05-14T16:53:12.123290+0000","last_undegraded":"2024-05-14T16:53:12.123290+0000","last_fullsized":"2024-05-14T16:53:12.123290+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:55:25.694827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-14T16:53:12.118892+0000","last_change":"2024-05-14T16:51:32.300044+0000","last_active":"2024-05-14T16:53:12.118892+0000","last_peered":"2024-05-14T16:53:12.118892+0000","last_clean":"2024-05-14T16:53:12.118892+0000","last_became_active":"2024-05-14T16:51:32.299630+0000","last_became_peered":"2024-05-14T16:51:32.299630+0000","last_unstale":"2024-05-14T16:53:12.118892+0000","last_undegraded":"2024-05-14T16:53:12.118892+0000","last_fullsized":"2024-05-14T16:53:12.118892+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:50:40.516973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284433,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65632,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":617,"num_read_kb":379,"num_write":171,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":196896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710759,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":864256,"data_stored":425130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906989,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1015808,"data_stored":1078101,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070513,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":425483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266743,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":880640,"data_stored":425286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462972,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":929792,"data_stored":1017357,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626498,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36184,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708968,"statfs":{"total":95995035648,"available":95957983232,"internally_reserved":0,"allocated":868352,"data_stored":424755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":929792,"data_stored":486038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36204,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708948,"statfs":{"total":95995035648,"available":95957962752,"internally_reserved":0,"allocated":888832,"data_stored":426026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":487920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182480,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36196,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708956,"statfs":{"total":95995035648,"available":95957970944,"internally_reserved":0,"allocated":880640,"data_stored":424949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986252,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":872448,"data_stored":425192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822727,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":1017041,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34541,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34541,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34541,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T16:53:32.336 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-14T16:53:32.336 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-14T16:53:32.336 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-14T16:53:32.336 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph health --format=json 2024-05-14T16:53:32.586 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:53:33.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:33 smithi151 ceph-mon[36066]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:53:33.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:33 smithi151 ceph-mon[36066]: pgmap v350: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:53:33.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:33 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:53:33.631 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:33 smithi123 ceph-mon[35971]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:53:33.631 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:33 smithi123 ceph-mon[35971]: pgmap v350: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:53:33.631 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:33 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:53:33.731 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:33 smithi002 ceph-mon[31538]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:53:33.731 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:33 smithi002 ceph-mon[31538]: pgmap v350: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:53:33.731 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:33 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:53:34.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.386+0000 7f25b454f700 1 -- 172.21.15.2:0/2357877715 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25ac06bbe0 msgr2=0x7f25ac1103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:34.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.386+0000 7f25b454f700 1 --2- 172.21.15.2:0/2357877715 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25ac06bbe0 0x7f25ac1103d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f25a400b0a0 tx=0x7f25a400b3b0 comp rx=0 tx=0).stop 2024-05-14T16:53:34.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.386+0000 7f25b454f700 1 -- 172.21.15.2:0/2357877715 shutdown_connections 2024-05-14T16:53:34.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.386+0000 7f25b454f700 1 --2- 172.21.15.2:0/2357877715 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25ac06bbe0 0x7f25ac1103d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:34.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.386+0000 7f25b454f700 1 --2- 172.21.15.2:0/2357877715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25ac1095b0 0x7f25ac109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:34.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.386+0000 7f25b454f700 1 --2- 172.21.15.2:0/2357877715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25ac06b220 0x7f25ac06b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:34.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.386+0000 7f25b454f700 1 -- 172.21.15.2:0/2357877715 >> 172.21.15.2:0/2357877715 conn(0x7f25ac0793f0 msgr2=0x7f25ac079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:34.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.386+0000 7f25b454f700 1 -- 172.21.15.2:0/2357877715 shutdown_connections 2024-05-14T16:53:34.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.386+0000 7f25b454f700 1 -- 172.21.15.2:0/2357877715 wait complete. 2024-05-14T16:53:34.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.387+0000 7f25b454f700 1 Processor -- start 2024-05-14T16:53:34.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.387+0000 7f25b454f700 1 -- start start 2024-05-14T16:53:34.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.387+0000 7f25b454f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25ac06b220 0x7f25ac19bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:34.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.387+0000 7f25b454f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25ac06bbe0 0x7f25ac19c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:34.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.387+0000 7f25b454f700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25ac1095b0 0x7f25ac1a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:34.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.387+0000 7f25b454f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f25ac1136c0 con 0x7f25ac06bbe0 2024-05-14T16:53:34.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.387+0000 7f25b454f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f25ac113540 con 0x7f25ac06b220 2024-05-14T16:53:34.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.387+0000 7f25b454f700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f25ac113840 con 0x7f25ac1095b0 2024-05-14T16:53:34.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.387+0000 7f25b1aea700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25ac06bbe0 0x7f25ac19c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:34.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.387+0000 7f25b1aea700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25ac06bbe0 0x7f25ac19c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49966/0 (socket says 172.21.15.2:49966) 2024-05-14T16:53:34.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.387+0000 7f25b1aea700 1 -- 172.21.15.2:0/2159148345 learned_addr learned my addr 172.21.15.2:0/2159148345 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:53:34.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.387+0000 7f25b2aec700 1 --2- 172.21.15.2:0/2159148345 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25ac1095b0 0x7f25ac1a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:34.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.387+0000 7f25b22eb700 1 --2- 172.21.15.2:0/2159148345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25ac06b220 0x7f25ac19bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:34.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.388+0000 7f25b1aea700 1 -- 172.21.15.2:0/2159148345 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25ac1095b0 msgr2=0x7f25ac1a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:34.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.388+0000 7f25b1aea700 1 --2- 172.21.15.2:0/2159148345 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25ac1095b0 0x7f25ac1a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:34.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.388+0000 7f25b1aea700 1 -- 172.21.15.2:0/2159148345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25ac06b220 msgr2=0x7f25ac19bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:34.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.388+0000 7f25b1aea700 1 --2- 172.21.15.2:0/2159148345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25ac06b220 0x7f25ac19bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:34.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.388+0000 7f25b1aea700 1 -- 172.21.15.2:0/2159148345 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25ac1a0da0 con 0x7f25ac06bbe0 2024-05-14T16:53:34.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.388+0000 7f25b2aec700 1 --2- 172.21.15.2:0/2159148345 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25ac1095b0 0x7f25ac1a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:53:34.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.388+0000 7f25b1aea700 1 --2- 172.21.15.2:0/2159148345 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25ac06bbe0 0x7f25ac19c2d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f259c00e9a0 tx=0x7f259c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:34.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.390+0000 7f25a37fe700 1 -- 172.21.15.2:0/2159148345 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f259c00ca10 con 0x7f25ac06bbe0 2024-05-14T16:53:34.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.390+0000 7f25b454f700 1 -- 172.21.15.2:0/2159148345 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25ac1a1090 con 0x7f25ac06bbe0 2024-05-14T16:53:34.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.390+0000 7f25a37fe700 1 -- 172.21.15.2:0/2159148345 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f259c00cbb0 con 0x7f25ac06bbe0 2024-05-14T16:53:34.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.390+0000 7f25a37fe700 1 -- 172.21.15.2:0/2159148345 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f259c018420 con 0x7f25ac06bbe0 2024-05-14T16:53:34.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.391+0000 7f25b454f700 1 -- 172.21.15.2:0/2159148345 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25ac1a8910 con 0x7f25ac06bbe0 2024-05-14T16:53:34.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.392+0000 7f25b454f700 1 -- 172.21.15.2:0/2159148345 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25ac06a230 con 0x7f25ac06bbe0 2024-05-14T16:53:34.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.393+0000 7f25a37fe700 1 -- 172.21.15.2:0/2159148345 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f259c010430 con 0x7f25ac06bbe0 2024-05-14T16:53:34.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.394+0000 7f25a37fe700 1 --2- 172.21.15.2:0/2159148345 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f25980b2900 0x7f25980b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:34.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.394+0000 7f25b22eb700 1 --2- 172.21.15.2:0/2159148345 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f25980b2900 0x7f25980b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:34.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.394+0000 7f25a37fe700 1 -- 172.21.15.2:0/2159148345 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f259c0e23a0 con 0x7f25ac06bbe0 2024-05-14T16:53:34.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.394+0000 7f25b22eb700 1 --2- 172.21.15.2:0/2159148345 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f25980b2900 0x7f25980b4dc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f25a8009be0 tx=0x7f25a8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:34.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.398+0000 7f25a37fe700 1 -- 172.21.15.2:0/2159148345 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f259c0aa0d0 con 0x7f25ac06bbe0 2024-05-14T16:53:34.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.674+0000 7f25b454f700 1 -- 172.21.15.2:0/2159148345 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f25ac19d0f0 con 0x7f25ac06bbe0 2024-05-14T16:53:34.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.674+0000 7f25a37fe700 1 -- 172.21.15.2:0/2159148345 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f25ac19d0f0 con 0x7f25ac06bbe0 2024-05-14T16:53:34.675 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:53:34.675 INFO:teuthology.orchestra.run.smithi002.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-14T16:53:34.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.678+0000 7f25b454f700 1 -- 172.21.15.2:0/2159148345 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f25980b2900 msgr2=0x7f25980b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:34.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.678+0000 7f25b454f700 1 --2- 172.21.15.2:0/2159148345 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f25980b2900 0x7f25980b4dc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f25a8009be0 tx=0x7f25a8009260 comp rx=0 tx=0).stop 2024-05-14T16:53:34.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.678+0000 7f25b454f700 1 -- 172.21.15.2:0/2159148345 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25ac06bbe0 msgr2=0x7f25ac19c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:34.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.678+0000 7f25b454f700 1 --2- 172.21.15.2:0/2159148345 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25ac06bbe0 0x7f25ac19c2d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f259c00e9a0 tx=0x7f259c00ecb0 comp rx=0 tx=0).stop 2024-05-14T16:53:34.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.678+0000 7f25b454f700 1 -- 172.21.15.2:0/2159148345 shutdown_connections 2024-05-14T16:53:34.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.678+0000 7f25b454f700 1 --2- 172.21.15.2:0/2159148345 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25ac1095b0 0x7f25ac1a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:34.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.678+0000 7f25b454f700 1 --2- 172.21.15.2:0/2159148345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25ac06b220 0x7f25ac19bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:34.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.678+0000 7f25b454f700 1 --2- 172.21.15.2:0/2159148345 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f25980b2900 0x7f25980b4dc0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:34.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.678+0000 7f25b454f700 1 --2- 172.21.15.2:0/2159148345 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25ac06bbe0 0x7f25ac19c2d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:34.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.678+0000 7f25b454f700 1 -- 172.21.15.2:0/2159148345 >> 172.21.15.2:0/2159148345 conn(0x7f25ac0793f0 msgr2=0x7f25ac10e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:34.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.678+0000 7f25b454f700 1 -- 172.21.15.2:0/2159148345 shutdown_connections 2024-05-14T16:53:34.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:34.679+0000 7f25b454f700 1 -- 172.21.15.2:0/2159148345 wait complete. 2024-05-14T16:53:35.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:35 smithi123 ceph-mon[35971]: pgmap v351: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:53:35.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:35 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:53:35.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:35 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:53:35.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:35 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:53:35.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:35 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:53:35.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:35 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/2159148345' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-14T16:53:35.727 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:35 smithi002 ceph-mon[31538]: pgmap v351: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:53:35.727 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:35 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:53:35.727 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:35 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:53:35.727 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:35 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:53:35.727 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:35 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:53:35.727 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:35 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/2159148345' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-14T16:53:35.785 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-14T16:53:35.786 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:53:35.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:35 smithi151 ceph-mon[36066]: pgmap v351: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:53:35.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:35 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:53:35.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:35 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:53:35.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:35 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:53:35.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:35 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:53:35.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:35 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/2159148345' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-14T16:53:36.089 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:53:37.635 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:37 smithi002 ceph-mon[31538]: pgmap v352: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:53:37.635 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:37 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:53:37.635 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:37 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:53:37.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:37 smithi123 ceph-mon[35971]: pgmap v352: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:53:37.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:37 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:53:37.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:37 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:53:37.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:37.747+0000 7f72b0e6a700 1 -- 172.21.15.2:0/1063875503 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72ac06c630 msgr2=0x7f72ac110420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:37.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:37.747+0000 7f72b0e6a700 1 --2- 172.21.15.2:0/1063875503 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72ac06c630 0x7f72ac110420 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f72a400b0d0 tx=0x7f72a400b3e0 comp rx=0 tx=0).stop 2024-05-14T16:53:37.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:37.747+0000 7f72b0e6a700 1 -- 172.21.15.2:0/1063875503 shutdown_connections 2024-05-14T16:53:37.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:37.747+0000 7f72b0e6a700 1 --2- 172.21.15.2:0/1063875503 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72ac06c630 0x7f72ac110420 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:37.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:37.747+0000 7f72b0e6a700 1 --2- 172.21.15.2:0/1063875503 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72ac109650 0x7f72ac109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:37.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:37.747+0000 7f72b0e6a700 1 --2- 172.21.15.2:0/1063875503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72ac06bc70 0x7f72ac06c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:37.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:37.747+0000 7f72b0e6a700 1 -- 172.21.15.2:0/1063875503 >> 172.21.15.2:0/1063875503 conn(0x7f72ac0782b0 msgr2=0x7f72ac0786c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:37.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:37.747+0000 7f72b0e6a700 1 -- 172.21.15.2:0/1063875503 shutdown_connections 2024-05-14T16:53:37.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:37.747+0000 7f72b0e6a700 1 -- 172.21.15.2:0/1063875503 wait complete. 2024-05-14T16:53:37.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:37.748+0000 7f72b0e6a700 1 Processor -- start 2024-05-14T16:53:37.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:37.748+0000 7f72b0e6a700 1 -- start start 2024-05-14T16:53:37.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:37.748+0000 7f72b0e6a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72ac06bc70 0x7f72ac102860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:37.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:37.748+0000 7f72b0e6a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72ac06c630 0x7f72ac102da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:37.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:37.748+0000 7f72b0e6a700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72ac109650 0x7f72ac1032e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:37.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:37.748+0000 7f72b0e6a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac113760 con 0x7f72ac06bc70 2024-05-14T16:53:37.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:37.748+0000 7f72b0e6a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac1135e0 con 0x7f72ac06c630 2024-05-14T16:53:37.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:37.748+0000 7f72b0e6a700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac1138e0 con 0x7f72ac109650 2024-05-14T16:53:37.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:37.748+0000 7f72aa59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72ac06bc70 0x7f72ac102860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:37.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:37.748+0000 7f72aa59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72ac06bc70 0x7f72ac102860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49990/0 (socket says 172.21.15.2:49990) 2024-05-14T16:53:37.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:37.748+0000 7f72aa59c700 1 -- 172.21.15.2:0/1377060000 learned_addr learned my addr 172.21.15.2:0/1377060000 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:53:37.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:37.748+0000 7f72a9d9b700 1 --2- 172.21.15.2:0/1377060000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72ac06c630 0x7f72ac102da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:37.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:37.749+0000 7f72aad9d700 1 --2- 172.21.15.2:0/1377060000 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72ac109650 0x7f72ac1032e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:37.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:37.749+0000 7f72aa59c700 1 -- 172.21.15.2:0/1377060000 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72ac109650 msgr2=0x7f72ac1032e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:37.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:37.749+0000 7f72aa59c700 1 --2- 172.21.15.2:0/1377060000 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72ac109650 0x7f72ac1032e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:37.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:37.749+0000 7f72aa59c700 1 -- 172.21.15.2:0/1377060000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72ac06c630 msgr2=0x7f72ac102da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:37.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:37.749+0000 7f72aa59c700 1 --2- 172.21.15.2:0/1377060000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72ac06c630 0x7f72ac102da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:37.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:37.749+0000 7f72aa59c700 1 -- 172.21.15.2:0/1377060000 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72ac1a8040 con 0x7f72ac06bc70 2024-05-14T16:53:37.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:37.749+0000 7f72aa59c700 1 --2- 172.21.15.2:0/1377060000 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72ac06bc70 0x7f72ac102860 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f729c00c9b0 tx=0x7f729c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:37.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:37.749+0000 7f729b7fe700 1 -- 172.21.15.2:0/1377060000 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c007b00 con 0x7f72ac06bc70 2024-05-14T16:53:37.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:37.749+0000 7f729b7fe700 1 -- 172.21.15.2:0/1377060000 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f729c007ca0 con 0x7f72ac06bc70 2024-05-14T16:53:37.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:37.749+0000 7f729b7fe700 1 -- 172.21.15.2:0/1377060000 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c018660 con 0x7f72ac06bc70 2024-05-14T16:53:37.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:37.749+0000 7f72b0e6a700 1 -- 172.21.15.2:0/1377060000 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72ac1a8330 con 0x7f72ac06bc70 2024-05-14T16:53:37.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:37.749+0000 7f72b0e6a700 1 -- 172.21.15.2:0/1377060000 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72ac1a88c0 con 0x7f72ac06bc70 2024-05-14T16:53:37.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:37.750+0000 7f72b0e6a700 1 -- 172.21.15.2:0/1377060000 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72ac104ca0 con 0x7f72ac06bc70 2024-05-14T16:53:37.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:37.752+0000 7f729b7fe700 1 -- 172.21.15.2:0/1377060000 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f729c00f450 con 0x7f72ac06bc70 2024-05-14T16:53:37.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:37.753+0000 7f729b7fe700 1 --2- 172.21.15.2:0/1377060000 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f72940b2900 0x7f72940b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:37.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:37.753+0000 7f72a9d9b700 1 --2- 172.21.15.2:0/1377060000 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f72940b2900 0x7f72940b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:37.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:37.753+0000 7f729b7fe700 1 -- 172.21.15.2:0/1377060000 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f729c0e2350 con 0x7f72ac06bc70 2024-05-14T16:53:37.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:37.754+0000 7f72a9d9b700 1 --2- 172.21.15.2:0/1377060000 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f72940b2900 0x7f72940b4dc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f72ac078840 tx=0x7f72a0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:37.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:37.758+0000 7f729b7fe700 1 -- 172.21.15.2:0/1377060000 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f729c014030 con 0x7f72ac06bc70 2024-05-14T16:53:37.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:37 smithi151 ceph-mon[36066]: pgmap v352: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:53:37.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:37 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:53:37.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:37 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:53:38.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:38.021+0000 7f72b0e6a700 1 -- 172.21.15.2:0/1377060000 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72ac103dc0 con 0x7f72ac06bc70 2024-05-14T16:53:38.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:38.021+0000 7f729b7fe700 1 -- 172.21.15.2:0/1377060000 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11850 (secure 0 0 0) 0x7f72ac103dc0 con 0x7f72ac06bc70 2024-05-14T16:53:38.021 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:53:38.021 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2275696923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2275696923},{"type":"v1","addr":"172.21.15.2:6835","nonce":2275696923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/716718761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":716718761},{"type":"v1","addr":"172.21.15.2:6839","nonce":716718761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/3644815179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3644815179},{"type":"v1","addr":"172.21.15.2:6841","nonce":3644815179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1880336731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1880336731},{"type":"v1","addr":"172.21.15.151:6833","nonce":1880336731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2415605548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2415605548},{"type":"v1","addr":"172.21.15.2:6837","nonce":2415605548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2635167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2635167},{"type":"v1","addr":"172.21.15.123:6833","nonce":2635167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:51:34.106919+0000","modified":"2024-05-14T16:51:53.879252+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24455,"mds_2":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24415":{"gid":24415,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/762044339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":762044339},{"type":"v1","addr":"172.21.15.151:6835","nonce":762044339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/132043151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":132043151},{"type":"v1","addr":"172.21.15.151:6837","nonce":132043151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/4107834375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":4107834375},{"type":"v1","addr":"172.21.15.151:6839","nonce":4107834375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.123:6835/1198670013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1198670013},{"type":"v1","addr":"172.21.15.123:6835","nonce":1198670013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/333426645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":333426645},{"type":"v1","addr":"172.21.15.123:6837","nonce":333426645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/580275877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":580275877},{"type":"v1","addr":"172.21.15.123:6839","nonce":580275877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24455,24479,24431]},"id":1}]} 2024-05-14T16:53:38.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:38.024+0000 7f72b0e6a700 1 -- 172.21.15.2:0/1377060000 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f72940b2900 msgr2=0x7f72940b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:38.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:38.024+0000 7f72b0e6a700 1 --2- 172.21.15.2:0/1377060000 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f72940b2900 0x7f72940b4dc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f72ac078840 tx=0x7f72a0009260 comp rx=0 tx=0).stop 2024-05-14T16:53:38.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:38.024+0000 7f72b0e6a700 1 -- 172.21.15.2:0/1377060000 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72ac06bc70 msgr2=0x7f72ac102860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:38.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:38.024+0000 7f72b0e6a700 1 --2- 172.21.15.2:0/1377060000 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72ac06bc70 0x7f72ac102860 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f729c00c9b0 tx=0x7f729c00ccc0 comp rx=0 tx=0).stop 2024-05-14T16:53:38.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:38.024+0000 7f72b0e6a700 1 -- 172.21.15.2:0/1377060000 shutdown_connections 2024-05-14T16:53:38.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:38.024+0000 7f72b0e6a700 1 --2- 172.21.15.2:0/1377060000 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72ac109650 0x7f72ac1032e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:38.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:38.024+0000 7f72b0e6a700 1 --2- 172.21.15.2:0/1377060000 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f72940b2900 0x7f72940b4dc0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:38.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:38.024+0000 7f72b0e6a700 1 --2- 172.21.15.2:0/1377060000 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72ac06bc70 0x7f72ac102860 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:38.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:38.024+0000 7f72b0e6a700 1 --2- 172.21.15.2:0/1377060000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72ac06c630 0x7f72ac102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:38.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:38.024+0000 7f72b0e6a700 1 -- 172.21.15.2:0/1377060000 >> 172.21.15.2:0/1377060000 conn(0x7f72ac0782b0 msgr2=0x7f72ac1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:38.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:38.024+0000 7f72b0e6a700 1 -- 172.21.15.2:0/1377060000 shutdown_connections 2024-05-14T16:53:38.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:38.024+0000 7f72b0e6a700 1 -- 172.21.15.2:0/1377060000 wait complete. 2024-05-14T16:53:38.025 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 24 2024-05-14T16:53:38.656 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:38 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:53:38.656 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:38 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:53:38.657 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:38 smithi002 ceph-mon[31538]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:53:38.657 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:38 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/1377060000' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:53:38.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:38 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:53:38.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:38 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:53:38.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:38 smithi123 ceph-mon[35971]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:53:38.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:38 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/1377060000' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:53:38.688 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:53:38.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:38 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:53:38.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:38 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:53:38.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:38 smithi151 ceph-mon[36066]: from='mgr.14150 172.21.15.2:0/1082130724' entity='mgr.x' 2024-05-14T16:53:38.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:38 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/1377060000' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:53:38.939 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:53:39.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:39 smithi123 ceph-mon[35971]: pgmap v353: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T16:53:39.730 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:39 smithi002 ceph-mon[31538]: pgmap v353: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T16:53:39.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:39 smithi151 ceph-mon[36066]: pgmap v353: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T16:53:40.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.590+0000 7efdcd513700 1 -- 172.21.15.2:0/178472748 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdc8103820 msgr2=0x7efdc8103c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:40.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.590+0000 7efdcd513700 1 --2- 172.21.15.2:0/178472748 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdc8103820 0x7efdc8103c00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efdb80099c0 tx=0x7efdb8009cd0 comp rx=0 tx=0).stop 2024-05-14T16:53:40.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.590+0000 7efdcd513700 1 -- 172.21.15.2:0/178472748 shutdown_connections 2024-05-14T16:53:40.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.590+0000 7efdcd513700 1 --2- 172.21.15.2:0/178472748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdc8108910 0x7efdc8110490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:40.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.590+0000 7efdcd513700 1 --2- 172.21.15.2:0/178472748 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdc8103820 0x7efdc8103c00 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:40.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.590+0000 7efdcd513700 1 --2- 172.21.15.2:0/178472748 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efdc81041d0 0x7efdc81081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:40.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.590+0000 7efdcd513700 1 -- 172.21.15.2:0/178472748 >> 172.21.15.2:0/178472748 conn(0x7efdc807a6b0 msgr2=0x7efdc807aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:40.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.590+0000 7efdcd513700 1 -- 172.21.15.2:0/178472748 shutdown_connections 2024-05-14T16:53:40.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.590+0000 7efdcd513700 1 -- 172.21.15.2:0/178472748 wait complete. 2024-05-14T16:53:40.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.590+0000 7efdcd513700 1 Processor -- start 2024-05-14T16:53:40.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.591+0000 7efdcd513700 1 -- start start 2024-05-14T16:53:40.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.591+0000 7efdcd513700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdc8103820 0x7efdc819f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:40.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.591+0000 7efdcd513700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efdc81041d0 0x7efdc819fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:40.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.591+0000 7efdcd513700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdc8108910 0x7efdc8199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:40.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.591+0000 7efdcd513700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7efdc81137d0 con 0x7efdc8103820 2024-05-14T16:53:40.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.591+0000 7efdcd513700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efdc8113650 con 0x7efdc8108910 2024-05-14T16:53:40.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.591+0000 7efdcd513700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efdc8113950 con 0x7efdc81041d0 2024-05-14T16:53:40.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.591+0000 7efdc6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdc8103820 0x7efdc819f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:40.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.591+0000 7efdc6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdc8103820 0x7efdc819f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50008/0 (socket says 172.21.15.2:50008) 2024-05-14T16:53:40.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.591+0000 7efdc6ffd700 1 -- 172.21.15.2:0/3649690874 learned_addr learned my addr 172.21.15.2:0/3649690874 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:53:40.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.591+0000 7efdc67fc700 1 --2- 172.21.15.2:0/3649690874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efdc81041d0 0x7efdc819fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:40.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.591+0000 7efdc77fe700 1 --2- 172.21.15.2:0/3649690874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdc8108910 0x7efdc8199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:40.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.591+0000 7efdc6ffd700 1 -- 172.21.15.2:0/3649690874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efdc81041d0 msgr2=0x7efdc819fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:40.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.591+0000 7efdc6ffd700 1 --2- 172.21.15.2:0/3649690874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efdc81041d0 0x7efdc819fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:40.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.591+0000 7efdc6ffd700 1 -- 172.21.15.2:0/3649690874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdc8108910 msgr2=0x7efdc8199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:40.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.591+0000 7efdc6ffd700 1 --2- 172.21.15.2:0/3649690874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdc8108910 0x7efdc8199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:40.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.591+0000 7efdc6ffd700 1 -- 172.21.15.2:0/3649690874 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc819a170 con 0x7efdc8103820 2024-05-14T16:53:40.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.591+0000 7efdc77fe700 1 --2- 172.21.15.2:0/3649690874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdc8108910 0x7efdc8199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:53:40.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.591+0000 7efdc6ffd700 1 --2- 172.21.15.2:0/3649690874 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdc8103820 0x7efdc819f850 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7efdb800b580 tx=0x7efdb80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:40.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.592+0000 7efdbffff700 1 -- 172.21.15.2:0/3649690874 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb8027070 con 0x7efdc8103820 2024-05-14T16:53:40.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.592+0000 7efdcd513700 1 -- 172.21.15.2:0/3649690874 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdc819a400 con 0x7efdc8103820 2024-05-14T16:53:40.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.592+0000 7efdbffff700 1 -- 172.21.15.2:0/3649690874 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdb80040a0 con 0x7efdc8103820 2024-05-14T16:53:40.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.592+0000 7efdbffff700 1 -- 172.21.15.2:0/3649690874 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb800f460 con 0x7efdc8103820 2024-05-14T16:53:40.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.592+0000 7efdcd513700 1 -- 172.21.15.2:0/3649690874 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdc81a6720 con 0x7efdc8103820 2024-05-14T16:53:40.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.593+0000 7efdcd513700 1 -- 172.21.15.2:0/3649690874 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdc806a230 con 0x7efdc8103820 2024-05-14T16:53:40.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.594+0000 7efdbffff700 1 -- 172.21.15.2:0/3649690874 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7efdb802a070 con 0x7efdc8103820 2024-05-14T16:53:40.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.595+0000 7efdbffff700 1 --2- 172.21.15.2:0/3649690874 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7efdb40b2880 0x7efdb40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:40.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.595+0000 7efdc67fc700 1 --2- 172.21.15.2:0/3649690874 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7efdb40b2880 0x7efdb40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:40.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.595+0000 7efdbffff700 1 -- 172.21.15.2:0/3649690874 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7efdb80e3700 con 0x7efdc8103820 2024-05-14T16:53:40.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.595+0000 7efdc67fc700 1 --2- 172.21.15.2:0/3649690874 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7efdb40b2880 0x7efdb40b4d40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efdc8101ff0 tx=0x7efdb0009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:40.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.598+0000 7efdbffff700 1 -- 172.21.15.2:0/3649690874 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efdb8031070 con 0x7efdc8103820 2024-05-14T16:53:40.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.860+0000 7efdcd513700 1 -- 172.21.15.2:0/3649690874 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efdc819af40 con 0x7efdc8103820 2024-05-14T16:53:40.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.860+0000 7efdbffff700 1 -- 172.21.15.2:0/3649690874 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11850 (secure 0 0 0) 0x7efdb8013070 con 0x7efdc8103820 2024-05-14T16:53:40.860 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:53:40.860 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2275696923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2275696923},{"type":"v1","addr":"172.21.15.2:6835","nonce":2275696923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/716718761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":716718761},{"type":"v1","addr":"172.21.15.2:6839","nonce":716718761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/3644815179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3644815179},{"type":"v1","addr":"172.21.15.2:6841","nonce":3644815179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1880336731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1880336731},{"type":"v1","addr":"172.21.15.151:6833","nonce":1880336731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2415605548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2415605548},{"type":"v1","addr":"172.21.15.2:6837","nonce":2415605548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2635167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2635167},{"type":"v1","addr":"172.21.15.123:6833","nonce":2635167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:51:34.106919+0000","modified":"2024-05-14T16:51:53.879252+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24455,"mds_2":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24415":{"gid":24415,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/762044339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":762044339},{"type":"v1","addr":"172.21.15.151:6835","nonce":762044339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/132043151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":132043151},{"type":"v1","addr":"172.21.15.151:6837","nonce":132043151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/4107834375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":4107834375},{"type":"v1","addr":"172.21.15.151:6839","nonce":4107834375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.123:6835/1198670013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1198670013},{"type":"v1","addr":"172.21.15.123:6835","nonce":1198670013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/333426645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":333426645},{"type":"v1","addr":"172.21.15.123:6837","nonce":333426645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/580275877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":580275877},{"type":"v1","addr":"172.21.15.123:6839","nonce":580275877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24455,24479,24431]},"id":1}]} 2024-05-14T16:53:40.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.863+0000 7efdcd513700 1 -- 172.21.15.2:0/3649690874 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7efdb40b2880 msgr2=0x7efdb40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:40.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.863+0000 7efdcd513700 1 --2- 172.21.15.2:0/3649690874 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7efdb40b2880 0x7efdb40b4d40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efdc8101ff0 tx=0x7efdb0009190 comp rx=0 tx=0).stop 2024-05-14T16:53:40.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.863+0000 7efdcd513700 1 -- 172.21.15.2:0/3649690874 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdc8103820 msgr2=0x7efdc819f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:40.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.863+0000 7efdcd513700 1 --2- 172.21.15.2:0/3649690874 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdc8103820 0x7efdc819f850 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7efdb800b580 tx=0x7efdb80047b0 comp rx=0 tx=0).stop 2024-05-14T16:53:40.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.863+0000 7efdcd513700 1 -- 172.21.15.2:0/3649690874 shutdown_connections 2024-05-14T16:53:40.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.863+0000 7efdcd513700 1 --2- 172.21.15.2:0/3649690874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdc8108910 0x7efdc8199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:40.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.863+0000 7efdcd513700 1 --2- 172.21.15.2:0/3649690874 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7efdb40b2880 0x7efdb40b4d40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:40.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.863+0000 7efdcd513700 1 --2- 172.21.15.2:0/3649690874 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdc8103820 0x7efdc819f850 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:40.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.863+0000 7efdcd513700 1 --2- 172.21.15.2:0/3649690874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efdc81041d0 0x7efdc819fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:40.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.863+0000 7efdcd513700 1 -- 172.21.15.2:0/3649690874 >> 172.21.15.2:0/3649690874 conn(0x7efdc807a6b0 msgr2=0x7efdc8102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:40.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.863+0000 7efdcd513700 1 -- 172.21.15.2:0/3649690874 shutdown_connections 2024-05-14T16:53:40.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:40.863+0000 7efdcd513700 1 -- 172.21.15.2:0/3649690874 wait complete. 2024-05-14T16:53:40.864 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 24 2024-05-14T16:53:41.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:41 smithi002 ceph-mon[31538]: pgmap v354: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T16:53:41.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:41 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/3649690874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:53:41.594 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T16:53:41.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:41 smithi123 ceph-mon[35971]: pgmap v354: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T16:53:41.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:41 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/3649690874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:53:41.846 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:53:41.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:41 smithi151 ceph-mon[36066]: pgmap v354: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T16:53:41.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:41 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/3649690874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:53:43.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.535+0000 7f2670bb2700 1 -- 172.21.15.2:0/2251154680 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f266c1041d0 msgr2=0x7f266c1081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:43.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.535+0000 7f2670bb2700 1 --2- 172.21.15.2:0/2251154680 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f266c1041d0 0x7f266c1081e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f26600099c0 tx=0x7f2660009cd0 comp rx=0 tx=0).stop 2024-05-14T16:53:43.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.535+0000 7f2670bb2700 1 -- 172.21.15.2:0/2251154680 shutdown_connections 2024-05-14T16:53:43.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.535+0000 7f2670bb2700 1 --2- 172.21.15.2:0/2251154680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f266c108910 0x7f266c110490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:43.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.535+0000 7f2670bb2700 1 --2- 172.21.15.2:0/2251154680 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f266c103820 0x7f266c103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:43.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.535+0000 7f2670bb2700 1 --2- 172.21.15.2:0/2251154680 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f266c1041d0 0x7f266c1081e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:43.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.535+0000 7f2670bb2700 1 -- 172.21.15.2:0/2251154680 >> 172.21.15.2:0/2251154680 conn(0x7f266c07a6b0 msgr2=0x7f266c07aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:43.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.535+0000 7f2670bb2700 1 -- 172.21.15.2:0/2251154680 shutdown_connections 2024-05-14T16:53:43.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.535+0000 7f2670bb2700 1 -- 172.21.15.2:0/2251154680 wait complete. 2024-05-14T16:53:43.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.536+0000 7f2670bb2700 1 Processor -- start 2024-05-14T16:53:43.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.536+0000 7f2670bb2700 1 -- start start 2024-05-14T16:53:43.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.536+0000 7f2670bb2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f266c103820 0x7f266c19f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:43.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.536+0000 7f2670bb2700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f266c1041d0 0x7f266c19fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:43.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.536+0000 7f2670bb2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f266c108910 0x7f266c199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:43.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.536+0000 7f2670bb2700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f266c1137d0 con 0x7f266c103820 2024-05-14T16:53:43.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.536+0000 7f2670bb2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f266c113650 con 0x7f266c108910 2024-05-14T16:53:43.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.536+0000 7f2670bb2700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f266c113950 con 0x7f266c1041d0 2024-05-14T16:53:43.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.536+0000 7f266a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f266c103820 0x7f266c19f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:43.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.536+0000 7f266a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f266c103820 0x7f266c19f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50028/0 (socket says 172.21.15.2:50028) 2024-05-14T16:53:43.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.536+0000 7f266a59c700 1 -- 172.21.15.2:0/3834420876 learned_addr learned my addr 172.21.15.2:0/3834420876 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:53:43.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.536+0000 7f266ad9d700 1 --2- 172.21.15.2:0/3834420876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f266c108910 0x7f266c199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:43.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.536+0000 7f2669d9b700 1 --2- 172.21.15.2:0/3834420876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f266c1041d0 0x7f266c19fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:43.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.537+0000 7f266a59c700 1 -- 172.21.15.2:0/3834420876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f266c1041d0 msgr2=0x7f266c19fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:43.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.537+0000 7f266a59c700 1 --2- 172.21.15.2:0/3834420876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f266c1041d0 0x7f266c19fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:43.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.537+0000 7f266a59c700 1 -- 172.21.15.2:0/3834420876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f266c108910 msgr2=0x7f266c199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:43.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.537+0000 7f266a59c700 1 --2- 172.21.15.2:0/3834420876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f266c108910 0x7f266c199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:43.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.537+0000 7f266a59c700 1 -- 172.21.15.2:0/3834420876 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f266c19a1d0 con 0x7f266c103820 2024-05-14T16:53:43.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.537+0000 7f2669d9b700 1 --2- 172.21.15.2:0/3834420876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f266c1041d0 0x7f266c19fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:53:43.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.537+0000 7f266ad9d700 1 --2- 172.21.15.2:0/3834420876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f266c108910 0x7f266c199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:53:43.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.537+0000 7f266a59c700 1 --2- 172.21.15.2:0/3834420876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f266c103820 0x7f266c19f860 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f265c00b530 tx=0x7f265c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:43.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.537+0000 7f265b7fe700 1 -- 172.21.15.2:0/3834420876 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265c015020 con 0x7f266c103820 2024-05-14T16:53:43.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.537+0000 7f265b7fe700 1 -- 172.21.15.2:0/3834420876 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f265c011ce0 con 0x7f266c103820 2024-05-14T16:53:43.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.537+0000 7f2670bb2700 1 -- 172.21.15.2:0/3834420876 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f266c19a4c0 con 0x7f266c103820 2024-05-14T16:53:43.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.537+0000 7f265b7fe700 1 -- 172.21.15.2:0/3834420876 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265c010d20 con 0x7f266c103820 2024-05-14T16:53:43.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.537+0000 7f2670bb2700 1 -- 172.21.15.2:0/3834420876 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f266c1a6730 con 0x7f266c103820 2024-05-14T16:53:43.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.538+0000 7f2670bb2700 1 -- 172.21.15.2:0/3834420876 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f266c10bb50 con 0x7f266c103820 2024-05-14T16:53:43.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.539+0000 7f265b7fe700 1 -- 172.21.15.2:0/3834420876 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f265c018020 con 0x7f266c103820 2024-05-14T16:53:43.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.540+0000 7f265b7fe700 1 --2- 172.21.15.2:0/3834420876 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f26540b2900 0x7f26540b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:43.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.540+0000 7f2669d9b700 1 --2- 172.21.15.2:0/3834420876 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f26540b2900 0x7f26540b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:43.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.540+0000 7f265b7fe700 1 -- 172.21.15.2:0/3834420876 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f265c0e1fa0 con 0x7f266c103820 2024-05-14T16:53:43.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.540+0000 7f2669d9b700 1 --2- 172.21.15.2:0/3834420876 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f26540b2900 0x7f26540b4dc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f266000b580 tx=0x7f2660009f10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:43.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.544+0000 7f265b7fe700 1 -- 172.21.15.2:0/3834420876 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f265c01f020 con 0x7f266c103820 2024-05-14T16:53:43.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:43 smithi123 ceph-mon[35971]: pgmap v355: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:53:43.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:43 smithi002 ceph-mon[31538]: pgmap v355: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:53:43.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.766+0000 7f2670bb2700 1 -- 172.21.15.2:0/3834420876 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f266c10b530 con 0x7f266c103820 2024-05-14T16:53:43.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.768+0000 7f265b7fe700 1 -- 172.21.15.2:0/3834420876 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22575 (secure 0 0 0) 0x7f265c0a9cd0 con 0x7f266c103820 2024-05-14T16:53:43.768 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:53:43.768 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":80,"fsid":"ed40717a-1210-11ef-bc98-c7b262605968","created":"2024-05-14T16:43:09.415298+0000","modified":"2024-05-14T16:52:13.205039+0000","last_up_change":"2024-05-14T16:50:29.787654+0000","last_in_change":"2024-05-14T16:50:09.287932+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T16:46:38.090903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T16:51:27.296955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T16:51:31.103019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"153f20ec-5bcf-4786-9ede-2e512a1e3d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6803","nonce":2654237820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6805","nonce":2654237820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6809","nonce":2654237820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6807","nonce":2654237820}]},"public_addr":"172.21.15.2:6803/2654237820","cluster_addr":"172.21.15.2:6805/2654237820","heartbeat_back_addr":"172.21.15.2:6809/2654237820","heartbeat_front_addr":"172.21.15.2:6807/2654237820","state":["exists","up"]},{"osd":1,"uuid":"0e95419f-9a9d-4f7a-b55d-ecb54971918a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6801","nonce":2668156992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6803","nonce":2668156992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6807","nonce":2668156992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6805","nonce":2668156992}]},"public_addr":"172.21.15.123:6801/2668156992","cluster_addr":"172.21.15.123:6803/2668156992","heartbeat_back_addr":"172.21.15.123:6807/2668156992","heartbeat_front_addr":"172.21.15.123:6805/2668156992","state":["exists","up"]},{"osd":2,"uuid":"b6ce7ac7-be6c-4549-a356-59cb81b5b492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6801","nonce":2160189636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6803","nonce":2160189636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6807","nonce":2160189636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6805","nonce":2160189636}]},"public_addr":"172.21.15.151:6801/2160189636","cluster_addr":"172.21.15.151:6803/2160189636","heartbeat_back_addr":"172.21.15.151:6807/2160189636","heartbeat_front_addr":"172.21.15.151:6805/2160189636","state":["exists","up"]},{"osd":3,"uuid":"2fd592bf-c212-411a-afbe-a7c498e5ac93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6811","nonce":3451424073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6813","nonce":3451424073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6817","nonce":3451424073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6815","nonce":3451424073}]},"public_addr":"172.21.15.2:6811/3451424073","cluster_addr":"172.21.15.2:6813/3451424073","heartbeat_back_addr":"172.21.15.2:6817/3451424073","heartbeat_front_addr":"172.21.15.2:6815/3451424073","state":["exists","up"]},{"osd":4,"uuid":"af1905e7-e036-47c6-be59-b085bc41fede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6809","nonce":2101935762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6811","nonce":2101935762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6815","nonce":2101935762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6813","nonce":2101935762}]},"public_addr":"172.21.15.123:6809/2101935762","cluster_addr":"172.21.15.123:6811/2101935762","heartbeat_back_addr":"172.21.15.123:6815/2101935762","heartbeat_front_addr":"172.21.15.123:6813/2101935762","state":["exists","up"]},{"osd":5,"uuid":"9c166916-af78-4cc2-bd14-d1220b6e7d9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6809","nonce":3695961886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6811","nonce":3695961886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6815","nonce":3695961886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6813","nonce":3695961886}]},"public_addr":"172.21.15.151:6809/3695961886","cluster_addr":"172.21.15.151:6811/3695961886","heartbeat_back_addr":"172.21.15.151:6815/3695961886","heartbeat_front_addr":"172.21.15.151:6813/3695961886","state":["exists","up"]},{"osd":6,"uuid":"60e9bf66-2338-41e8-9280-398d4e3b322f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6819","nonce":2711893600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6821","nonce":2711893600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6825","nonce":2711893600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6823","nonce":2711893600}]},"public_addr":"172.21.15.2:6819/2711893600","cluster_addr":"172.21.15.2:6821/2711893600","heartbeat_back_addr":"172.21.15.2:6825/2711893600","heartbeat_front_addr":"172.21.15.2:6823/2711893600","state":["exists","up"]},{"osd":7,"uuid":"c93e07a1-e941-493e-8468-33429df2dff0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6817","nonce":3634155507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6819","nonce":3634155507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6823","nonce":3634155507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6821","nonce":3634155507}]},"public_addr":"172.21.15.123:6817/3634155507","cluster_addr":"172.21.15.123:6819/3634155507","heartbeat_back_addr":"172.21.15.123:6823/3634155507","heartbeat_front_addr":"172.21.15.123:6821/3634155507","state":["exists","up"]},{"osd":8,"uuid":"1155fb58-d614-4f45-91e2-5da3a93e7d9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6817","nonce":4271107010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6819","nonce":4271107010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6823","nonce":4271107010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6821","nonce":4271107010}]},"public_addr":"172.21.15.151:6817/4271107010","cluster_addr":"172.21.15.151:6819/4271107010","heartbeat_back_addr":"172.21.15.151:6823/4271107010","heartbeat_front_addr":"172.21.15.151:6821/4271107010","state":["exists","up"]},{"osd":9,"uuid":"82b34e3b-0017-4ca4-8f86-0993ca78ce7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6827","nonce":3479431021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6829","nonce":3479431021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6833","nonce":3479431021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6831","nonce":3479431021}]},"public_addr":"172.21.15.2:6827/3479431021","cluster_addr":"172.21.15.2:6829/3479431021","heartbeat_back_addr":"172.21.15.2:6833/3479431021","heartbeat_front_addr":"172.21.15.2:6831/3479431021","state":["exists","up"]},{"osd":10,"uuid":"32e2ace9-eb5d-4258-a2ac-49b782be2b5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6825","nonce":4266374435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6827","nonce":4266374435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6831","nonce":4266374435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6829","nonce":4266374435}]},"public_addr":"172.21.15.123:6825/4266374435","cluster_addr":"172.21.15.123:6827/4266374435","heartbeat_back_addr":"172.21.15.123:6831/4266374435","heartbeat_front_addr":"172.21.15.123:6829/4266374435","state":["exists","up"]},{"osd":11,"uuid":"6e3c2ead-1af6-4bda-87cd-12770c81fa67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6825","nonce":4129194698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6827","nonce":4129194698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6831","nonce":4129194698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6829","nonce":4129194698}]},"public_addr":"172.21.15.151:6825/4129194698","cluster_addr":"172.21.15.151:6827/4129194698","heartbeat_back_addr":"172.21.15.151:6831/4129194698","heartbeat_front_addr":"172.21.15.151:6829/4129194698","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:45:46.540023+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:46:10.845186+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:46:34.283112+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:47:00.072467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:47:25.315644+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:47:49.748688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:48:16.964258+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:48:42.875515+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:49:08.596500+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:49:35.450943+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:50:01.293368+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:50:27.793713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/4267809161":"2024-05-15T16:44:11.998214+0000","172.21.15.2:6801/3582988344":"2024-05-15T16:44:11.998214+0000","172.21.15.2:6800/2574015037":"2024-05-15T16:43:43.407184+0000","172.21.15.2:0/2100732571":"2024-05-15T16:44:11.998214+0000","172.21.15.2:0/2954199324":"2024-05-15T16:43:43.407184+0000","172.21.15.2:6801/2574015037":"2024-05-15T16:43:43.407184+0000","172.21.15.2:0/3171550944":"2024-05-15T16:43:43.407184+0000","172.21.15.2:0/1679761136":"2024-05-15T16:44:11.998214+0000","172.21.15.2:0/1746766591":"2024-05-15T16:43:43.407184+0000","172.21.15.2:6800/3582988344":"2024-05-15T16:44:11.998214+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T16:53:43.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.770+0000 7f2670bb2700 1 -- 172.21.15.2:0/3834420876 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f26540b2900 msgr2=0x7f26540b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:43.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.770+0000 7f2670bb2700 1 --2- 172.21.15.2:0/3834420876 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f26540b2900 0x7f26540b4dc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f266000b580 tx=0x7f2660009f10 comp rx=0 tx=0).stop 2024-05-14T16:53:43.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.770+0000 7f2670bb2700 1 -- 172.21.15.2:0/3834420876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f266c103820 msgr2=0x7f266c19f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:43.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.770+0000 7f2670bb2700 1 --2- 172.21.15.2:0/3834420876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f266c103820 0x7f266c19f860 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f265c00b530 tx=0x7f265c00b8f0 comp rx=0 tx=0).stop 2024-05-14T16:53:43.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.770+0000 7f2670bb2700 1 -- 172.21.15.2:0/3834420876 shutdown_connections 2024-05-14T16:53:43.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.770+0000 7f2670bb2700 1 --2- 172.21.15.2:0/3834420876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f266c108910 0x7f266c199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:43.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.770+0000 7f2670bb2700 1 --2- 172.21.15.2:0/3834420876 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f26540b2900 0x7f26540b4dc0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:43.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.770+0000 7f2670bb2700 1 --2- 172.21.15.2:0/3834420876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f266c103820 0x7f266c19f860 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:43.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.770+0000 7f2670bb2700 1 --2- 172.21.15.2:0/3834420876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f266c1041d0 0x7f266c19fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:43.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.770+0000 7f2670bb2700 1 -- 172.21.15.2:0/3834420876 >> 172.21.15.2:0/3834420876 conn(0x7f266c07a6b0 msgr2=0x7f266c102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:43.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.771+0000 7f2670bb2700 1 -- 172.21.15.2:0/3834420876 shutdown_connections 2024-05-14T16:53:43.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:43.771+0000 7f2670bb2700 1 -- 172.21.15.2:0/3834420876 wait complete. 2024-05-14T16:53:43.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:43 smithi151 ceph-mon[36066]: pgmap v355: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:53:44.429 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:53:44.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:44 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/3834420876' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:53:44.684 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:53:44.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:44 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/3834420876' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:53:44.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:44 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/3834420876' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:53:45.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:45 smithi123 ceph-mon[35971]: pgmap v356: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:53:45.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:45 smithi002 ceph-mon[31538]: pgmap v356: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:53:45.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:45 smithi151 ceph-mon[36066]: pgmap v356: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:53:46.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.312+0000 7fd20c979700 1 -- 172.21.15.2:0/2378653312 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd208109680 msgr2=0x7fd208109a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:46.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.312+0000 7fd20c979700 1 --2- 172.21.15.2:0/2378653312 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd208109680 0x7fd208109a60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd1f80099c0 tx=0x7fd1f8009cd0 comp rx=0 tx=0).stop 2024-05-14T16:53:46.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.312+0000 7fd20c979700 1 -- 172.21.15.2:0/2378653312 shutdown_connections 2024-05-14T16:53:46.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.312+0000 7fd20c979700 1 --2- 172.21.15.2:0/2378653312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd208103fa0 0x7fd208110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:46.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.312+0000 7fd20c979700 1 --2- 172.21.15.2:0/2378653312 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd208109680 0x7fd208109a60 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:46.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.312+0000 7fd20c979700 1 --2- 172.21.15.2:0/2378653312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd2081035e0 0x7fd208103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:46.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.312+0000 7fd20c979700 1 -- 172.21.15.2:0/2378653312 >> 172.21.15.2:0/2378653312 conn(0x7fd20807a6f0 msgr2=0x7fd20807ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:46.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.313+0000 7fd20c979700 1 -- 172.21.15.2:0/2378653312 shutdown_connections 2024-05-14T16:53:46.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.313+0000 7fd20c979700 1 -- 172.21.15.2:0/2378653312 wait complete. 2024-05-14T16:53:46.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.313+0000 7fd20c979700 1 Processor -- start 2024-05-14T16:53:46.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.313+0000 7fd20c979700 1 -- start start 2024-05-14T16:53:46.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.314+0000 7fd20c979700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2081035e0 0x7fd208075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:46.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.314+0000 7fd20c979700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd208103fa0 0x7fd208075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:46.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.314+0000 7fd20c979700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd208109680 0x7fd2080762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:46.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.314+0000 7fd20c979700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd2081137d0 con 0x7fd208103fa0 2024-05-14T16:53:46.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.314+0000 7fd20c979700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd208113650 con 0x7fd2081035e0 2024-05-14T16:53:46.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.314+0000 7fd20c979700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd208113950 con 0x7fd208109680 2024-05-14T16:53:46.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.314+0000 7fd205d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd208103fa0 0x7fd208075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:46.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.314+0000 7fd205d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd208103fa0 0x7fd208075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58502/0 (socket says 172.21.15.2:58502) 2024-05-14T16:53:46.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.314+0000 7fd205d9b700 1 -- 172.21.15.2:0/1916380813 learned_addr learned my addr 172.21.15.2:0/1916380813 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:53:46.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.314+0000 7fd206d9d700 1 --2- 172.21.15.2:0/1916380813 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd208109680 0x7fd2080762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:46.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.314+0000 7fd20659c700 1 --2- 172.21.15.2:0/1916380813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2081035e0 0x7fd208075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:46.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.314+0000 7fd205d9b700 1 -- 172.21.15.2:0/1916380813 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd208109680 msgr2=0x7fd2080762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:46.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.314+0000 7fd205d9b700 1 --2- 172.21.15.2:0/1916380813 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd208109680 0x7fd2080762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:46.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.314+0000 7fd205d9b700 1 -- 172.21.15.2:0/1916380813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2081035e0 msgr2=0x7fd208075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:46.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.314+0000 7fd205d9b700 1 --2- 172.21.15.2:0/1916380813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2081035e0 0x7fd208075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:46.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.314+0000 7fd205d9b700 1 -- 172.21.15.2:0/1916380813 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2081a8130 con 0x7fd208103fa0 2024-05-14T16:53:46.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.314+0000 7fd205d9b700 1 --2- 172.21.15.2:0/1916380813 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd208103fa0 0x7fd208075d80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc007f10 tx=0x7fd1fc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:46.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.315+0000 7fd1f77fe700 1 -- 172.21.15.2:0/1916380813 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc00ddd0 con 0x7fd208103fa0 2024-05-14T16:53:46.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.315+0000 7fd1f77fe700 1 -- 172.21.15.2:0/1916380813 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1fc004bb0 con 0x7fd208103fa0 2024-05-14T16:53:46.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.315+0000 7fd20c979700 1 -- 172.21.15.2:0/1916380813 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2081a8420 con 0x7fd208103fa0 2024-05-14T16:53:46.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.315+0000 7fd1f77fe700 1 -- 172.21.15.2:0/1916380813 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc0046a0 con 0x7fd208103fa0 2024-05-14T16:53:46.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.315+0000 7fd20c979700 1 -- 172.21.15.2:0/1916380813 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2081a8960 con 0x7fd208103fa0 2024-05-14T16:53:46.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.316+0000 7fd20c979700 1 -- 172.21.15.2:0/1916380813 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd20810bb10 con 0x7fd208103fa0 2024-05-14T16:53:46.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.317+0000 7fd1f77fe700 1 -- 172.21.15.2:0/1916380813 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fd1fc01c020 con 0x7fd208103fa0 2024-05-14T16:53:46.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.318+0000 7fd1f77fe700 1 --2- 172.21.15.2:0/1916380813 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fd1f00b2880 0x7fd1f00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:46.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.318+0000 7fd20659c700 1 --2- 172.21.15.2:0/1916380813 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fd1f00b2880 0x7fd1f00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:46.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.318+0000 7fd1f77fe700 1 -- 172.21.15.2:0/1916380813 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7fd1fc0e24e0 con 0x7fd208103fa0 2024-05-14T16:53:46.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.318+0000 7fd20659c700 1 --2- 172.21.15.2:0/1916380813 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fd1f00b2880 0x7fd1f00b4d40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8003800 tx=0x7fd1f801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:46.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.321+0000 7fd1f77fe700 1 -- 172.21.15.2:0/1916380813 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd1fc023030 con 0x7fd208103fa0 2024-05-14T16:53:46.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.584+0000 7fd20c979700 1 -- 172.21.15.2:0/1916380813 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd208104cf0 con 0x7fd208103fa0 2024-05-14T16:53:46.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.584+0000 7fd1f77fe700 1 -- 172.21.15.2:0/1916380813 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11850 (secure 0 0 0) 0x7fd1fc010070 con 0x7fd208103fa0 2024-05-14T16:53:46.584 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:53:46.584 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2275696923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2275696923},{"type":"v1","addr":"172.21.15.2:6835","nonce":2275696923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/716718761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":716718761},{"type":"v1","addr":"172.21.15.2:6839","nonce":716718761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/3644815179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3644815179},{"type":"v1","addr":"172.21.15.2:6841","nonce":3644815179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1880336731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1880336731},{"type":"v1","addr":"172.21.15.151:6833","nonce":1880336731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2415605548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2415605548},{"type":"v1","addr":"172.21.15.2:6837","nonce":2415605548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2635167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2635167},{"type":"v1","addr":"172.21.15.123:6833","nonce":2635167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:51:34.106919+0000","modified":"2024-05-14T16:51:53.879252+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24455,"mds_2":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24415":{"gid":24415,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/762044339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":762044339},{"type":"v1","addr":"172.21.15.151:6835","nonce":762044339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/132043151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":132043151},{"type":"v1","addr":"172.21.15.151:6837","nonce":132043151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/4107834375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":4107834375},{"type":"v1","addr":"172.21.15.151:6839","nonce":4107834375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.123:6835/1198670013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1198670013},{"type":"v1","addr":"172.21.15.123:6835","nonce":1198670013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/333426645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":333426645},{"type":"v1","addr":"172.21.15.123:6837","nonce":333426645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/580275877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":580275877},{"type":"v1","addr":"172.21.15.123:6839","nonce":580275877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24455,24479,24431]},"id":1}]} 2024-05-14T16:53:46.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.587+0000 7fd20c979700 1 -- 172.21.15.2:0/1916380813 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fd1f00b2880 msgr2=0x7fd1f00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:46.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.587+0000 7fd20c979700 1 --2- 172.21.15.2:0/1916380813 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fd1f00b2880 0x7fd1f00b4d40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8003800 tx=0x7fd1f801a040 comp rx=0 tx=0).stop 2024-05-14T16:53:46.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.587+0000 7fd20c979700 1 -- 172.21.15.2:0/1916380813 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd208103fa0 msgr2=0x7fd208075d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:46.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.587+0000 7fd20c979700 1 --2- 172.21.15.2:0/1916380813 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd208103fa0 0x7fd208075d80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc007f10 tx=0x7fd1fc00d3b0 comp rx=0 tx=0).stop 2024-05-14T16:53:46.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.587+0000 7fd20c979700 1 -- 172.21.15.2:0/1916380813 shutdown_connections 2024-05-14T16:53:46.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.587+0000 7fd20c979700 1 --2- 172.21.15.2:0/1916380813 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd208109680 0x7fd2080762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:46.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.587+0000 7fd20c979700 1 --2- 172.21.15.2:0/1916380813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2081035e0 0x7fd208075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:46.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.587+0000 7fd20c979700 1 --2- 172.21.15.2:0/1916380813 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fd1f00b2880 0x7fd1f00b4d40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:46.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.587+0000 7fd20c979700 1 --2- 172.21.15.2:0/1916380813 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd208103fa0 0x7fd208075d80 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:46.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.587+0000 7fd20c979700 1 -- 172.21.15.2:0/1916380813 >> 172.21.15.2:0/1916380813 conn(0x7fd20807a6f0 msgr2=0x7fd208102210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:46.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.587+0000 7fd20c979700 1 -- 172.21.15.2:0/1916380813 shutdown_connections 2024-05-14T16:53:46.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:46.587+0000 7fd20c979700 1 -- 172.21.15.2:0/1916380813 wait complete. 2024-05-14T16:53:46.588 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 24 2024-05-14T16:53:47.259 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-14T16:51:34.106919+0000', 'modified': '2024-05-14T16:51:53.879252+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24479, 'mds_1': 24455, 'mds_2': 24431}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24415': {'gid': 24415, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.151:6835/762044339', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6834', 'nonce': 762044339}, {'type': 'v1', 'addr': '172.21.15.151:6835', 'nonce': 762044339}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24421': {'gid': 24421, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.151:6837/132043151', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6836', 'nonce': 132043151}, {'type': 'v1', 'addr': '172.21.15.151:6837', 'nonce': 132043151}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24427': {'gid': 24427, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.151:6839/4107834375', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6838', 'nonce': 4107834375}, {'type': 'v1', 'addr': '172.21.15.151:6839', 'nonce': 4107834375}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24431': {'gid': 24431, 'name': 'e', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.123:6835/1198670013', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6834', 'nonce': 1198670013}, {'type': 'v1', 'addr': '172.21.15.123:6835', 'nonce': 1198670013}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'h', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.123:6837/333426645', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6836', 'nonce': 333426645}, {'type': 'v1', 'addr': '172.21.15.123:6837', 'nonce': 333426645}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.123:6839/580275877', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6838', 'nonce': 580275877}, {'type': 'v1', 'addr': '172.21.15.123:6839', 'nonce': 580275877}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24479, 'qdb_cluster': [24455, 24479, 24431]} 2024-05-14T16:53:47.259 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-14T16:53:47.259 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph tell mds.e status 2024-05-14T16:53:47.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:47 smithi002 ceph-mon[31538]: pgmap v357: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:53:47.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:47 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/1916380813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:53:47.511 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:53:47.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:47 smithi123 ceph-mon[35971]: pgmap v357: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:53:47.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:47 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/1916380813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:53:47.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:47 smithi151 ceph-mon[36066]: pgmap v357: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:53:47.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:47 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/1916380813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:53:49.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.164+0000 7fc720a97700 1 -- 172.21.15.2:0/2527701968 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc71c06c5d0 msgr2=0x7fc71c10fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:49.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.164+0000 7fc720a97700 1 --2- 172.21.15.2:0/2527701968 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc71c06c5d0 0x7fc71c10fb40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc710009990 tx=0x7fc710009ca0 comp rx=0 tx=0).stop 2024-05-14T16:53:49.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.164+0000 7fc720a97700 1 -- 172.21.15.2:0/2527701968 shutdown_connections 2024-05-14T16:53:49.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.164+0000 7fc720a97700 1 --2- 172.21.15.2:0/2527701968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc71c110080 0x7fc71c112470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:49.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.164+0000 7fc720a97700 1 --2- 172.21.15.2:0/2527701968 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc71c06bcb0 0x7fc71c06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:49.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.164+0000 7fc720a97700 1 --2- 172.21.15.2:0/2527701968 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc71c06c5d0 0x7fc71c10fb40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:49.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.164+0000 7fc720a97700 1 -- 172.21.15.2:0/2527701968 >> 172.21.15.2:0/2527701968 conn(0x7fc71c078490 msgr2=0x7fc71c0788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:49.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.164+0000 7fc720a97700 1 -- 172.21.15.2:0/2527701968 shutdown_connections 2024-05-14T16:53:49.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.164+0000 7fc720a97700 1 -- 172.21.15.2:0/2527701968 wait complete. 2024-05-14T16:53:49.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.165+0000 7fc720a97700 1 Processor -- start 2024-05-14T16:53:49.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.165+0000 7fc720a97700 1 -- start start 2024-05-14T16:53:49.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.165+0000 7fc720a97700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc71c06bcb0 0x7fc71c19be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:49.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.165+0000 7fc720a97700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc71c06c5d0 0x7fc71c19c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:49.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.165+0000 7fc720a97700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc71c110080 0x7fc71c1a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:49.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.165+0000 7fc720a97700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c115980 con 0x7fc71c06bcb0 2024-05-14T16:53:49.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.165+0000 7fc720a97700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c115800 con 0x7fc71c110080 2024-05-14T16:53:49.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.165+0000 7fc720a97700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c115b00 con 0x7fc71c06c5d0 2024-05-14T16:53:49.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.165+0000 7fc71a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc71c06bcb0 0x7fc71c19be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:49.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.165+0000 7fc71a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc71c06bcb0 0x7fc71c19be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58514/0 (socket says 172.21.15.2:58514) 2024-05-14T16:53:49.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.165+0000 7fc71a59c700 1 -- 172.21.15.2:0/880527655 learned_addr learned my addr 172.21.15.2:0/880527655 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:53:49.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.165+0000 7fc71ad9d700 1 --2- 172.21.15.2:0/880527655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc71c110080 0x7fc71c1a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:49.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.165+0000 7fc719d9b700 1 --2- 172.21.15.2:0/880527655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc71c06c5d0 0x7fc71c19c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:49.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.166+0000 7fc71a59c700 1 -- 172.21.15.2:0/880527655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc71c06c5d0 msgr2=0x7fc71c19c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:49.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.166+0000 7fc71a59c700 1 --2- 172.21.15.2:0/880527655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc71c06c5d0 0x7fc71c19c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:49.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.166+0000 7fc71a59c700 1 -- 172.21.15.2:0/880527655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc71c110080 msgr2=0x7fc71c1a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:49.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.166+0000 7fc71a59c700 1 --2- 172.21.15.2:0/880527655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc71c110080 0x7fc71c1a0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:49.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.166+0000 7fc71a59c700 1 -- 172.21.15.2:0/880527655 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc71c1a0e50 con 0x7fc71c06bcb0 2024-05-14T16:53:49.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.166+0000 7fc719d9b700 1 --2- 172.21.15.2:0/880527655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc71c06c5d0 0x7fc71c19c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:53:49.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.166+0000 7fc71a59c700 1 --2- 172.21.15.2:0/880527655 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc71c06bcb0 0x7fc71c19be90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc70c00d980 tx=0x7fc70c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:49.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.166+0000 7fc70b7fe700 1 -- 172.21.15.2:0/880527655 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c017070 con 0x7fc71c06bcb0 2024-05-14T16:53:49.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.166+0000 7fc720a97700 1 -- 172.21.15.2:0/880527655 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc71c1a1140 con 0x7fc71c06bcb0 2024-05-14T16:53:49.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.166+0000 7fc70b7fe700 1 -- 172.21.15.2:0/880527655 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc70c004830 con 0x7fc71c06bcb0 2024-05-14T16:53:49.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.166+0000 7fc70b7fe700 1 -- 172.21.15.2:0/880527655 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c005440 con 0x7fc71c06bcb0 2024-05-14T16:53:49.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.166+0000 7fc720a97700 1 -- 172.21.15.2:0/880527655 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc71c1a89c0 con 0x7fc71c06bcb0 2024-05-14T16:53:49.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.168+0000 7fc70b7fe700 1 -- 172.21.15.2:0/880527655 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fc70c0055e0 con 0x7fc71c06bcb0 2024-05-14T16:53:49.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.169+0000 7fc70b7fe700 1 --2- 172.21.15.2:0/880527655 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fc7040b2880 0x7fc7040b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:49.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.169+0000 7fc719d9b700 1 --2- 172.21.15.2:0/880527655 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fc7040b2880 0x7fc7040b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:49.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.169+0000 7fc70b7fe700 1 -- 172.21.15.2:0/880527655 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7fc70c0e2210 con 0x7fc71c06bcb0 2024-05-14T16:53:49.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.170+0000 7fc72b32db80 1 Processor -- start 2024-05-14T16:53:49.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.170+0000 7fc719d9b700 1 --2- 172.21.15.2:0/880527655 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fc7040b2880 0x7fc7040b4d40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc71000b5c0 tx=0x7fc710005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:49.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.170+0000 7fc72b32db80 1 -- start start 2024-05-14T16:53:49.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.170+0000 7fc72b32db80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563965843fa0 0x563965844380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:49.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.170+0000 7fc72b32db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5639658448c0 0x56396583e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:49.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.170+0000 7fc72b32db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56396583edc0 0x56396583f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:49.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.170+0000 7fc72b32db80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56396579cd20 con 0x563965843fa0 2024-05-14T16:53:49.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.170+0000 7fc72b32db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56396578c8c0 con 0x56396583edc0 2024-05-14T16:53:49.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.170+0000 7fc72b32db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563965790900 con 0x5639658448c0 2024-05-14T16:53:49.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.170+0000 7fc71a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563965843fa0 0x563965844380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:49.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.170+0000 7fc71a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563965843fa0 0x563965844380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58524/0 (socket says 172.21.15.2:58524) 2024-05-14T16:53:49.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.170+0000 7fc71a59c700 1 -- 172.21.15.2:0/2408825537 learned_addr learned my addr 172.21.15.2:0/2408825537 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:53:49.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.171+0000 7fc71ad9d700 1 --2- 172.21.15.2:0/2408825537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56396583edc0 0x56396583f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:49.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.171+0000 7fc719d9b700 1 --2- 172.21.15.2:0/2408825537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5639658448c0 0x56396583e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:49.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.171+0000 7fc71a59c700 1 -- 172.21.15.2:0/2408825537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5639658448c0 msgr2=0x56396583e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:49.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.171+0000 7fc71a59c700 1 --2- 172.21.15.2:0/2408825537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5639658448c0 0x56396583e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:49.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.171+0000 7fc71a59c700 1 -- 172.21.15.2:0/2408825537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56396583edc0 msgr2=0x56396583f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:49.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.171+0000 7fc71a59c700 1 --2- 172.21.15.2:0/2408825537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56396583edc0 0x56396583f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:49.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.171+0000 7fc71a59c700 1 -- 172.21.15.2:0/2408825537 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563965735f60 con 0x563965843fa0 2024-05-14T16:53:49.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.171+0000 7fc71ad9d700 1 --2- 172.21.15.2:0/2408825537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56396583edc0 0x56396583f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:53:49.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.171+0000 7fc719d9b700 1 --2- 172.21.15.2:0/2408825537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5639658448c0 0x56396583e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:53:49.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.171+0000 7fc71a59c700 1 --2- 172.21.15.2:0/2408825537 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563965843fa0 0x563965844380 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc70c0a6a50 tx=0x7fc70c0a93c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a7831b6e84488c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:49.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.171+0000 7fc708ff9700 1 -- 172.21.15.2:0/2408825537 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c0b2640 con 0x563965843fa0 2024-05-14T16:53:49.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.171+0000 7fc708ff9700 1 -- 172.21.15.2:0/2408825537 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc70c0e1030 con 0x563965843fa0 2024-05-14T16:53:49.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.171+0000 7fc708ff9700 1 -- 172.21.15.2:0/2408825537 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c0b21b0 con 0x563965843fa0 2024-05-14T16:53:49.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.172+0000 7fc72b32db80 1 -- 172.21.15.2:0/2408825537 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563965843fa0 msgr2=0x563965844380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:49.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.172+0000 7fc72b32db80 1 --2- 172.21.15.2:0/2408825537 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563965843fa0 0x563965844380 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc70c0a6a50 tx=0x7fc70c0a93c0 comp rx=0 tx=0).stop 2024-05-14T16:53:49.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.172+0000 7fc72b32db80 1 -- 172.21.15.2:0/2408825537 shutdown_connections 2024-05-14T16:53:49.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.172+0000 7fc72b32db80 1 --2- 172.21.15.2:0/2408825537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56396583edc0 0x56396583f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:49.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.172+0000 7fc72b32db80 1 --2- 172.21.15.2:0/2408825537 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563965843fa0 0x563965844380 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:49.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.172+0000 7fc72b32db80 1 --2- 172.21.15.2:0/2408825537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5639658448c0 0x56396583e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:49.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.172+0000 7fc72b32db80 1 -- 172.21.15.2:0/2408825537 >> 172.21.15.2:0/2408825537 conn(0x56396576bd60 msgr2=0x563965838e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:49.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.172+0000 7fc72b32db80 1 -- 172.21.15.2:0/2408825537 shutdown_connections 2024-05-14T16:53:49.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.172+0000 7fc72b32db80 1 -- 172.21.15.2:0/2408825537 wait complete. 2024-05-14T16:53:49.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.172+0000 7fc72b32db80 1 -- start start 2024-05-14T16:53:49.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.172+0000 7fc72b32db80 1 Processor -- start 2024-05-14T16:53:49.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.172+0000 7fc72b32db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56396583edc0 0x56396584a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:49.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.172+0000 7fc72b32db80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563965843fa0 0x5639658478a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:49.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.173+0000 7fc72b32db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5639658448c0 0x563965847de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:49.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.173+0000 7fc72b32db80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563965692f50 con 0x563965843fa0 2024-05-14T16:53:49.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.173+0000 7fc72b32db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5639656f4d50 con 0x56396583edc0 2024-05-14T16:53:49.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.173+0000 7fc72b32db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56396584b210 con 0x5639658448c0 2024-05-14T16:53:49.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.173+0000 7fc72b32db80 10 client.0 _ll_register_callbacks cb 0x563965727900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:53:49.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.173+0000 7fc719d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563965843fa0 0x5639658478a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:49.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.173+0000 7fc719d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563965843fa0 0x5639658478a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58540/0 (socket says 172.21.15.2:58540) 2024-05-14T16:53:49.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.173+0000 7fc719d9b700 1 -- 172.21.15.2:0/918484433 learned_addr learned my addr 172.21.15.2:0/918484433 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:53:49.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.173+0000 7fc71ad9d700 1 --2- 172.21.15.2:0/918484433 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5639658448c0 0x563965847de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:49.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.173+0000 7fc71a59c700 1 --2- 172.21.15.2:0/918484433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56396583edc0 0x56396584a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:49.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.173+0000 7fc719d9b700 1 -- 172.21.15.2:0/918484433 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5639658448c0 msgr2=0x563965847de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:49.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.173+0000 7fc719d9b700 1 --2- 172.21.15.2:0/918484433 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5639658448c0 0x563965847de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:49.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.173+0000 7fc719d9b700 1 -- 172.21.15.2:0/918484433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56396583edc0 msgr2=0x56396584a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:49.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.173+0000 7fc719d9b700 1 --2- 172.21.15.2:0/918484433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56396583edc0 0x56396584a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:49.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.173+0000 7fc719d9b700 1 -- 172.21.15.2:0/918484433 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639656dc880 con 0x563965843fa0 2024-05-14T16:53:49.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.173+0000 7fc71ad9d700 1 --2- 172.21.15.2:0/918484433 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5639658448c0 0x563965847de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:53:49.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.173+0000 7fc719d9b700 1 --2- 172.21.15.2:0/918484433 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563965843fa0 0x5639658478a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc71c102150 tx=0x7fc710009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c373ee1cd27ed0d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:49.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.173+0000 7fc701ffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:53:49.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.173+0000 7fc701ffb700 1 -- 172.21.15.2:0/918484433 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc710003c70 con 0x563965843fa0 2024-05-14T16:53:49.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.173+0000 7fc701ffb700 1 -- 172.21.15.2:0/918484433 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7fc71000f840 con 0x563965843fa0 2024-05-14T16:53:49.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.173+0000 7fc72b32db80 1 -- 172.21.15.2:0/918484433 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc70c0e1030 con 0x563965843fa0 2024-05-14T16:53:49.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.174+0000 7fc701ffb700 1 -- 172.21.15.2:0/918484433 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc710003e10 con 0x563965843fa0 2024-05-14T16:53:49.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.174+0000 7fc72b32db80 10 client.14940 fetch_fsmap learned FSMap version 24 2024-05-14T16:53:49.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.174+0000 7fc72b32db80 1 -- 172.21.15.2:0/918484433 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563965746710 con 0x563965843fa0 2024-05-14T16:53:49.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.174+0000 7fc701ffb700 1 -- 172.21.15.2:0/918484433 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc70c0e1030 con 0x563965843fa0 2024-05-14T16:53:49.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.174+0000 7fc701ffb700 1 -- 172.21.15.2:0/918484433 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fc71002c070 con 0x563965843fa0 2024-05-14T16:53:49.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.174+0000 7fc72b32db80 10 client.14940 fetch_fsmap finished waiting for FSMap version 24 2024-05-14T16:53:49.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.174+0000 7fc72b32db80 10 client.14940 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-14T16:53:49.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.174+0000 7fc72b32db80 20 client.14940 populate_metadata read hostname 'smithi002' 2024-05-14T16:53:49.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.174+0000 7fc72b32db80 1 --2- 172.21.15.2:0/918484433 >> [v2:172.21.15.123:6834/1198670013,v1:172.21.15.123:6835/1198670013] conn(0x563965857c50 0x5639658590c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:49.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.174+0000 7fc72b32db80 4 client.14940 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:53:49.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.174+0000 7fc72b32db80 1 -- 172.21.15.2:0/918484433 --> [v2:172.21.15.123:6834/1198670013,v1:172.21.15.123:6835/1198670013] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56396579ec70 con 0x563965857c50 2024-05-14T16:53:49.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.174+0000 7fc71a59c700 1 --2- 172.21.15.2:0/918484433 >> [v2:172.21.15.123:6834/1198670013,v1:172.21.15.123:6835/1198670013] conn(0x563965857c50 0x5639658590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:49.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.175+0000 7fc71a59c700 1 --2- 172.21.15.2:0/918484433 >> [v2:172.21.15.123:6834/1198670013,v1:172.21.15.123:6835/1198670013] conn(0x563965857c50 0x5639658590c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9ed86420a5cc4305 server_cookie=87925eb716c95c81 in_seq=0 out_seq=0 2024-05-14T16:53:49.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.175+0000 7fc701ffb700 10 client.14940 ms_handle_connect on v2:172.21.15.123:6834/1198670013 2024-05-14T16:53:49.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.176+0000 7fc701ffb700 1 -- 172.21.15.2:0/918484433 <== mds.2 v2:172.21.15.123:6834/1198670013 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56396579ec70 con 0x563965857c50 2024-05-14T16:53:49.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.176+0000 7fc701ffb700 10 client.14940 handle_command_reply: tid=0 2024-05-14T16:53:49.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.176+0000 7fc72b32db80 1 client.14940 shutdown 2024-05-14T16:53:49.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.176+0000 7fc72b32db80 1 -- 172.21.15.2:0/918484433 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563965843fa0 msgr2=0x5639658478a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:49.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.176+0000 7fc72b32db80 1 --2- 172.21.15.2:0/918484433 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563965843fa0 0x5639658478a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc71c102150 tx=0x7fc710009ca0 comp rx=0 tx=0).stop 2024-05-14T16:53:49.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.176+0000 7fc72b32db80 1 -- 172.21.15.2:0/918484433 shutdown_connections 2024-05-14T16:53:49.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.176+0000 7fc72b32db80 1 --2- 172.21.15.2:0/918484433 >> [v2:172.21.15.123:6834/1198670013,v1:172.21.15.123:6835/1198670013] conn(0x563965857c50 0x5639658590c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:49.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.176+0000 7fc72b32db80 1 --2- 172.21.15.2:0/918484433 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5639658448c0 0x563965847de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:49.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.176+0000 7fc72b32db80 1 --2- 172.21.15.2:0/918484433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56396583edc0 0x56396584a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:49.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.176+0000 7fc701ffb700 0 client.14940 ms_handle_reset on v2:172.21.15.123:6834/1198670013 2024-05-14T16:53:49.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.176+0000 7fc72b32db80 1 --2- 172.21.15.2:0/918484433 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563965843fa0 0x5639658478a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:49.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.176+0000 7fc72b32db80 1 -- 172.21.15.2:0/918484433 >> 172.21.15.2:0/918484433 conn(0x56396576bd60 msgr2=0x563965838630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:49.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.176+0000 7fc72b32db80 1 -- 172.21.15.2:0/918484433 shutdown_connections 2024-05-14T16:53:49.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.176+0000 7fc72b32db80 1 -- 172.21.15.2:0/918484433 wait complete. 2024-05-14T16:53:49.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.176+0000 7fc72b32db80 20 client.14940 trim_cache size 0 max 16384 2024-05-14T16:53:49.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.197+0000 7fc72b32db80 1 Processor -- start 2024-05-14T16:53:49.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.197+0000 7fc72b32db80 1 -- start start 2024-05-14T16:53:49.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.197+0000 7fc72b32db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56396583edc0 0x563965847d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:49.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.197+0000 7fc72b32db80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563965843fa0 0x563965838db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:49.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.197+0000 7fc72b32db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639658448c0 0x563965862520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:49.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.197+0000 7fc72b32db80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563965857400 con 0x563965843fa0 2024-05-14T16:53:49.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.197+0000 7fc72b32db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563965857ad0 con 0x5639658448c0 2024-05-14T16:53:49.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.197+0000 7fc72b32db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5639658581a0 con 0x56396583edc0 2024-05-14T16:53:49.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.198+0000 7fc719d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563965843fa0 0x563965838db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:49.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.198+0000 7fc719d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563965843fa0 0x563965838db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58542/0 (socket says 172.21.15.2:58542) 2024-05-14T16:53:49.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.198+0000 7fc719d9b700 1 -- 172.21.15.2:0/798937338 learned_addr learned my addr 172.21.15.2:0/798937338 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:53:49.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.198+0000 7fc71ad9d700 1 --2- 172.21.15.2:0/798937338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639658448c0 0x563965862520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:49.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.198+0000 7fc71a59c700 1 --2- 172.21.15.2:0/798937338 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56396583edc0 0x563965847d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:49.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.198+0000 7fc719d9b700 1 -- 172.21.15.2:0/798937338 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56396583edc0 msgr2=0x563965847d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:49.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.198+0000 7fc719d9b700 1 --2- 172.21.15.2:0/798937338 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56396583edc0 0x563965847d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:49.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.198+0000 7fc719d9b700 1 -- 172.21.15.2:0/798937338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639658448c0 msgr2=0x563965862520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:49.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.198+0000 7fc719d9b700 1 --2- 172.21.15.2:0/798937338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639658448c0 0x563965862520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:49.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.198+0000 7fc719d9b700 1 -- 172.21.15.2:0/798937338 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563965839a20 con 0x563965843fa0 2024-05-14T16:53:49.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.198+0000 7fc71a59c700 1 --2- 172.21.15.2:0/798937338 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56396583edc0 0x563965847d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:53:49.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.198+0000 7fc71ad9d700 1 --2- 172.21.15.2:0/798937338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639658448c0 0x563965862520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:53:49.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.198+0000 7fc719d9b700 1 --2- 172.21.15.2:0/798937338 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563965843fa0 0x563965838db0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc71002e490 tx=0x7fc710009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=600f783e2f0226b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:49.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.198+0000 7fc703fff700 1 -- 172.21.15.2:0/798937338 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7100052f0 con 0x563965843fa0 2024-05-14T16:53:49.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.198+0000 7fc703fff700 1 -- 172.21.15.2:0/798937338 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc710034030 con 0x563965843fa0 2024-05-14T16:53:49.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.198+0000 7fc703fff700 1 -- 172.21.15.2:0/798937338 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71002eae0 con 0x563965843fa0 2024-05-14T16:53:49.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.199+0000 7fc72b32db80 1 -- 172.21.15.2:0/798937338 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563965843fa0 msgr2=0x563965838db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:49.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.199+0000 7fc72b32db80 1 --2- 172.21.15.2:0/798937338 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563965843fa0 0x563965838db0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc71002e490 tx=0x7fc710009310 comp rx=0 tx=0).stop 2024-05-14T16:53:49.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.199+0000 7fc72b32db80 1 -- 172.21.15.2:0/798937338 shutdown_connections 2024-05-14T16:53:49.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.199+0000 7fc72b32db80 1 --2- 172.21.15.2:0/798937338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639658448c0 0x563965862520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:49.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.199+0000 7fc72b32db80 1 --2- 172.21.15.2:0/798937338 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56396583edc0 0x563965847d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:49.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.199+0000 7fc72b32db80 1 --2- 172.21.15.2:0/798937338 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563965843fa0 0x563965838db0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:49.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.199+0000 7fc72b32db80 1 -- 172.21.15.2:0/798937338 >> 172.21.15.2:0/798937338 conn(0x56396576bd60 msgr2=0x563965847680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:49.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.199+0000 7fc72b32db80 1 -- 172.21.15.2:0/798937338 shutdown_connections 2024-05-14T16:53:49.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.199+0000 7fc72b32db80 1 -- 172.21.15.2:0/798937338 wait complete. 2024-05-14T16:53:49.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.199+0000 7fc72b32db80 1 -- start start 2024-05-14T16:53:49.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.200+0000 7fc72b32db80 1 Processor -- start 2024-05-14T16:53:49.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.200+0000 7fc72b32db80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56396583edc0 0x563965862ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:49.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.200+0000 7fc72b32db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563965843fa0 0x56396585d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:49.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.200+0000 7fc72b32db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5639658448c0 0x56396585da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:49.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.200+0000 7fc72b32db80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563965858f40 con 0x56396583edc0 2024-05-14T16:53:49.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.200+0000 7fc72b32db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563965858870 con 0x563965843fa0 2024-05-14T16:53:49.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.200+0000 7fc72b32db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563965859610 con 0x5639658448c0 2024-05-14T16:53:49.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.200+0000 7fc72b32db80 10 client.0 _ll_register_callbacks cb 0x563965727900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:53:49.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.200+0000 7fc71a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56396583edc0 0x563965862ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:49.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.200+0000 7fc71a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56396583edc0 0x563965862ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58556/0 (socket says 172.21.15.2:58556) 2024-05-14T16:53:49.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.200+0000 7fc71a59c700 1 -- 172.21.15.2:0/3796442495 learned_addr learned my addr 172.21.15.2:0/3796442495 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:53:49.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.200+0000 7fc71ad9d700 1 --2- 172.21.15.2:0/3796442495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5639658448c0 0x56396585da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:49.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.200+0000 7fc71a59c700 1 -- 172.21.15.2:0/3796442495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5639658448c0 msgr2=0x56396585da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:49.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.200+0000 7fc719d9b700 1 --2- 172.21.15.2:0/3796442495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563965843fa0 0x56396585d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:49.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.200+0000 7fc71a59c700 1 --2- 172.21.15.2:0/3796442495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5639658448c0 0x56396585da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:49.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.200+0000 7fc71a59c700 1 -- 172.21.15.2:0/3796442495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563965843fa0 msgr2=0x56396585d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:49.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.200+0000 7fc71a59c700 1 --2- 172.21.15.2:0/3796442495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563965843fa0 0x56396585d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:49.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.200+0000 7fc71a59c700 1 -- 172.21.15.2:0/3796442495 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56396585e2d0 con 0x56396583edc0 2024-05-14T16:53:49.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.200+0000 7fc719d9b700 1 --2- 172.21.15.2:0/3796442495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563965843fa0 0x56396585d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:53:49.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.200+0000 7fc71a59c700 1 --2- 172.21.15.2:0/3796442495 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56396583edc0 0x563965862ae0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc70c00d950 tx=0x7fc70c0a5040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f82910eabfec27 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:49.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.200+0000 7fc701ffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:53:49.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.201+0000 7fc701ffb700 1 -- 172.21.15.2:0/3796442495 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c0a95b0 con 0x56396583edc0 2024-05-14T16:53:49.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.201+0000 7fc701ffb700 1 -- 172.21.15.2:0/3796442495 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7fc70c0a9c40 con 0x56396583edc0 2024-05-14T16:53:49.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.201+0000 7fc72b32db80 1 -- 172.21.15.2:0/3796442495 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc710034030 con 0x56396583edc0 2024-05-14T16:53:49.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.201+0000 7fc701ffb700 1 -- 172.21.15.2:0/3796442495 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc70c0b21b0 con 0x56396583edc0 2024-05-14T16:53:49.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.201+0000 7fc72b32db80 10 client.14946 fetch_fsmap learned FSMap version 24 2024-05-14T16:53:49.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.201+0000 7fc72b32db80 1 -- 172.21.15.2:0/3796442495 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5639658c4190 con 0x56396583edc0 2024-05-14T16:53:49.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.201+0000 7fc701ffb700 1 -- 172.21.15.2:0/3796442495 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc710034030 con 0x56396583edc0 2024-05-14T16:53:49.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.201+0000 7fc701ffb700 1 -- 172.21.15.2:0/3796442495 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fc70c0ca020 con 0x56396583edc0 2024-05-14T16:53:49.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.201+0000 7fc72b32db80 10 client.14946 fetch_fsmap finished waiting for FSMap version 24 2024-05-14T16:53:49.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.201+0000 7fc72b32db80 10 client.14946 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-14T16:53:49.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.201+0000 7fc72b32db80 20 client.14946 populate_metadata read hostname 'smithi002' 2024-05-14T16:53:49.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.201+0000 7fc72b32db80 1 --2- 172.21.15.2:0/3796442495 >> [v2:172.21.15.123:6834/1198670013,v1:172.21.15.123:6835/1198670013] conn(0x5639658c8910 0x5639658c7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:49.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.201+0000 7fc72b32db80 4 client.14946 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T16:53:49.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.201+0000 7fc72b32db80 1 -- 172.21.15.2:0/3796442495 --> [v2:172.21.15.123:6834/1198670013,v1:172.21.15.123:6835/1198670013] -- command(tid 0: {"prefix": "status"}) -- 0x563965798450 con 0x5639658c8910 2024-05-14T16:53:49.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.202+0000 7fc719d9b700 1 --2- 172.21.15.2:0/3796442495 >> [v2:172.21.15.123:6834/1198670013,v1:172.21.15.123:6835/1198670013] conn(0x5639658c8910 0x5639658c7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:49.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.202+0000 7fc719d9b700 1 --2- 172.21.15.2:0/3796442495 >> [v2:172.21.15.123:6834/1198670013,v1:172.21.15.123:6835/1198670013] conn(0x5639658c8910 0x5639658c7620 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9936b29347695d server_cookie=6df856ea5e44d855 in_seq=0 out_seq=0 2024-05-14T16:53:49.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.202+0000 7fc701ffb700 10 client.14946 ms_handle_connect on v2:172.21.15.123:6834/1198670013 2024-05-14T16:53:49.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.202+0000 7fc701ffb700 1 -- 172.21.15.2:0/3796442495 <== mds.2 v2:172.21.15.123:6834/1198670013 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x563965798450 con 0x5639658c8910 2024-05-14T16:53:49.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.202+0000 7fc701ffb700 10 client.14946 handle_command_reply: tid=0 2024-05-14T16:53:49.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.202+0000 7fc72b32db80 1 client.14946 shutdown 2024-05-14T16:53:49.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.202+0000 7fc72b32db80 1 -- 172.21.15.2:0/3796442495 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56396583edc0 msgr2=0x563965862ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:49.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.202+0000 7fc72b32db80 1 --2- 172.21.15.2:0/3796442495 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56396583edc0 0x563965862ae0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc70c00d950 tx=0x7fc70c0a5040 comp rx=0 tx=0).stop 2024-05-14T16:53:49.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.203+0000 7fc72b32db80 1 -- 172.21.15.2:0/3796442495 shutdown_connections 2024-05-14T16:53:49.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.203+0000 7fc72b32db80 1 --2- 172.21.15.2:0/3796442495 >> [v2:172.21.15.123:6834/1198670013,v1:172.21.15.123:6835/1198670013] conn(0x5639658c8910 0x5639658c7620 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:49.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.203+0000 7fc72b32db80 1 --2- 172.21.15.2:0/3796442495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5639658448c0 0x56396585da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:49.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.203+0000 7fc72b32db80 1 --2- 172.21.15.2:0/3796442495 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56396583edc0 0x563965862ae0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:49.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.203+0000 7fc72b32db80 1 --2- 172.21.15.2:0/3796442495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563965843fa0 0x56396585d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:49.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.203+0000 7fc72b32db80 1 -- 172.21.15.2:0/3796442495 >> 172.21.15.2:0/3796442495 conn(0x56396576bd60 msgr2=0x563965844ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:49.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.203+0000 7fc701ffb700 0 client.14946 ms_handle_reset on v2:172.21.15.123:6834/1198670013 2024-05-14T16:53:49.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.203+0000 7fc72b32db80 1 -- 172.21.15.2:0/3796442495 shutdown_connections 2024-05-14T16:53:49.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.203+0000 7fc72b32db80 1 -- 172.21.15.2:0/3796442495 wait complete. 2024-05-14T16:53:49.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.203+0000 7fc72b32db80 20 client.14946 trim_cache size 0 max 16384 2024-05-14T16:53:49.215 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T16:53:49.215 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "ed40717a-1210-11ef-bc98-c7b262605968", 2024-05-14T16:53:49.215 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 2, 2024-05-14T16:53:49.215 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24431, 2024-05-14T16:53:49.215 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2024-05-14T16:53:49.215 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2024-05-14T16:53:49.215 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2024-05-14T16:53:49.215 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 121.224812562, 2024-05-14T16:53:49.215 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 24, 2024-05-14T16:53:49.215 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 80, 2024-05-14T16:53:49.216 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 78, 2024-05-14T16:53:49.216 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 172.30778712099999 2024-05-14T16:53:49.216 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T16:53:49.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.203+0000 7fc720a97700 1 -- 172.21.15.2:0/880527655 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fc7040b2880 msgr2=0x7fc7040b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:49.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.203+0000 7fc720a97700 1 --2- 172.21.15.2:0/880527655 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fc7040b2880 0x7fc7040b4d40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc71000b5c0 tx=0x7fc710005fb0 comp rx=0 tx=0).stop 2024-05-14T16:53:49.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.203+0000 7fc720a97700 1 -- 172.21.15.2:0/880527655 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc71c06bcb0 msgr2=0x7fc71c19be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:49.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.203+0000 7fc720a97700 1 --2- 172.21.15.2:0/880527655 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc71c06bcb0 0x7fc71c19be90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc70c00d980 tx=0x7fc70c00dd40 comp rx=0 tx=0).stop 2024-05-14T16:53:49.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.203+0000 7fc720a97700 1 -- 172.21.15.2:0/880527655 shutdown_connections 2024-05-14T16:53:49.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.203+0000 7fc720a97700 1 --2- 172.21.15.2:0/880527655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc71c110080 0x7fc71c1a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:49.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.203+0000 7fc720a97700 1 --2- 172.21.15.2:0/880527655 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fc7040b2880 0x7fc7040b4d40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:49.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.203+0000 7fc720a97700 1 --2- 172.21.15.2:0/880527655 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc71c06bcb0 0x7fc71c19be90 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:49.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.203+0000 7fc720a97700 1 --2- 172.21.15.2:0/880527655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc71c06c5d0 0x7fc71c19c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:49.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.203+0000 7fc720a97700 1 -- 172.21.15.2:0/880527655 >> 172.21.15.2:0/880527655 conn(0x7fc71c078490 msgr2=0x7fc71c1115f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:49.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.204+0000 7fc720a97700 1 -- 172.21.15.2:0/880527655 shutdown_connections 2024-05-14T16:53:49.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:49.204+0000 7fc720a97700 1 -- 172.21.15.2:0/880527655 wait complete. 2024-05-14T16:53:49.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:49 smithi002 ceph-mon[31538]: pgmap v358: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T16:53:49.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:49 smithi123 ceph-mon[35971]: pgmap v358: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T16:53:49.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:49 smithi151 ceph-mon[36066]: pgmap v358: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T16:53:49.916 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph tell mds.h status 2024-05-14T16:53:50.171 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:53:51.655 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:51 smithi002 ceph-mon[31538]: pgmap v359: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T16:53:51.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.763+0000 7f7b35675700 1 -- 172.21.15.2:0/3431547261 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b301041a0 msgr2=0x7f7b301081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:51.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.763+0000 7f7b35675700 1 --2- 172.21.15.2:0/3431547261 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b301041a0 0x7f7b301081c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7b180099c0 tx=0x7f7b18009cd0 comp rx=0 tx=0).stop 2024-05-14T16:53:51.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.763+0000 7f7b35675700 1 -- 172.21.15.2:0/3431547261 shutdown_connections 2024-05-14T16:53:51.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.763+0000 7f7b35675700 1 --2- 172.21.15.2:0/3431547261 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b301088f0 0x7f7b30110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:51.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.763+0000 7f7b35675700 1 --2- 172.21.15.2:0/3431547261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b301037f0 0x7f7b30103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:51.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.763+0000 7f7b35675700 1 --2- 172.21.15.2:0/3431547261 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b301041a0 0x7f7b301081c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:51.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.763+0000 7f7b35675700 1 -- 172.21.15.2:0/3431547261 >> 172.21.15.2:0/3431547261 conn(0x7f7b3007a6a0 msgr2=0x7f7b3007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:51.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.763+0000 7f7b35675700 1 -- 172.21.15.2:0/3431547261 shutdown_connections 2024-05-14T16:53:51.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.763+0000 7f7b35675700 1 -- 172.21.15.2:0/3431547261 wait complete. 2024-05-14T16:53:51.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.764+0000 7f7b35675700 1 Processor -- start 2024-05-14T16:53:51.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.764+0000 7f7b35675700 1 -- start start 2024-05-14T16:53:51.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.765+0000 7f7b35675700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b301037f0 0x7f7b3019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:51.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.765+0000 7f7b35675700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b301041a0 0x7f7b3019fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:51.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.765+0000 7f7b35675700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b301088f0 0x7f7b301999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:51.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.765+0000 7f7b35675700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7b301136d0 con 0x7f7b301088f0 2024-05-14T16:53:51.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.765+0000 7f7b35675700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7b30113550 con 0x7f7b301037f0 2024-05-14T16:53:51.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.765+0000 7f7b35675700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7b30113850 con 0x7f7b301041a0 2024-05-14T16:53:51.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.765+0000 7f7b2f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b301088f0 0x7f7b301999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:51.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.765+0000 7f7b2f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b301088f0 0x7f7b301999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58572/0 (socket says 172.21.15.2:58572) 2024-05-14T16:53:51.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.765+0000 7f7b2f7fe700 1 -- 172.21.15.2:0/2071449081 learned_addr learned my addr 172.21.15.2:0/2071449081 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:53:51.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.765+0000 7f7b2effd700 1 --2- 172.21.15.2:0/2071449081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b301037f0 0x7f7b3019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:51.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.765+0000 7f7b267fc700 1 --2- 172.21.15.2:0/2071449081 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b301041a0 0x7f7b3019fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:51.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.765+0000 7f7b2f7fe700 1 -- 172.21.15.2:0/2071449081 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b301041a0 msgr2=0x7f7b3019fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:51.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.765+0000 7f7b2f7fe700 1 --2- 172.21.15.2:0/2071449081 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b301041a0 0x7f7b3019fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:51.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.765+0000 7f7b2f7fe700 1 -- 172.21.15.2:0/2071449081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b301037f0 msgr2=0x7f7b3019f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:51.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.765+0000 7f7b2f7fe700 1 --2- 172.21.15.2:0/2071449081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b301037f0 0x7f7b3019f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:51.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.765+0000 7f7b2f7fe700 1 -- 172.21.15.2:0/2071449081 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b3019a110 con 0x7f7b301088f0 2024-05-14T16:53:51.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.765+0000 7f7b267fc700 1 --2- 172.21.15.2:0/2071449081 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b301041a0 0x7f7b3019fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:53:51.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.765+0000 7f7b2f7fe700 1 --2- 172.21.15.2:0/2071449081 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b301088f0 0x7f7b301999b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7b28007ee0 tx=0x7f7b2800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:51.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.766+0000 7f7b2cff9700 1 -- 172.21.15.2:0/2071449081 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2800dbd0 con 0x7f7b301088f0 2024-05-14T16:53:51.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.766+0000 7f7b35675700 1 -- 172.21.15.2:0/2071449081 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b3019a400 con 0x7f7b301088f0 2024-05-14T16:53:51.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.766+0000 7f7b2cff9700 1 -- 172.21.15.2:0/2071449081 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b2800dd70 con 0x7f7b301088f0 2024-05-14T16:53:51.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.766+0000 7f7b2cff9700 1 -- 172.21.15.2:0/2071449081 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b28005990 con 0x7f7b301088f0 2024-05-14T16:53:51.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.766+0000 7f7b35675700 1 -- 172.21.15.2:0/2071449081 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b301a66c0 con 0x7f7b301088f0 2024-05-14T16:53:51.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.768+0000 7f7b2cff9700 1 -- 172.21.15.2:0/2071449081 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f7b2801a020 con 0x7f7b301088f0 2024-05-14T16:53:51.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.769+0000 7f7b2cff9700 1 --2- 172.21.15.2:0/2071449081 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f7b1c0b2900 0x7f7b1c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:51.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.769+0000 7f7b2effd700 1 --2- 172.21.15.2:0/2071449081 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f7b1c0b2900 0x7f7b1c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:51.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.769+0000 7f7b2cff9700 1 -- 172.21.15.2:0/2071449081 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f7b280e1930 con 0x7f7b301088f0 2024-05-14T16:53:51.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.769+0000 7f7b2effd700 1 --2- 172.21.15.2:0/2071449081 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f7b1c0b2900 0x7f7b1c0b4dc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7b20005950 tx=0x7f7b2000a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:51.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.769+0000 7f7b3ff0bb80 1 Processor -- start 2024-05-14T16:53:51.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.770+0000 7f7b3ff0bb80 1 -- start start 2024-05-14T16:53:51.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.770+0000 7f7b3ff0bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555efafd5700 0x555efafd5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:51.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.770+0000 7f7b3ff0bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555efafd6020 0x555efafd97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:51.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.770+0000 7f7b3ff0bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555efafd9d20 0x555efafda160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:51.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.770+0000 7f7b3ff0bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x555efaf2fc40 con 0x555efafd5700 2024-05-14T16:53:51.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.770+0000 7f7b3ff0bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555efaf1f7e0 con 0x555efafd9d20 2024-05-14T16:53:51.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.770+0000 7f7b3ff0bb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555efaf23820 con 0x555efafd6020 2024-05-14T16:53:51.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.770+0000 7f7b2effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555efafd5700 0x555efafd5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:51.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.770+0000 7f7b2effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555efafd5700 0x555efafd5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58586/0 (socket says 172.21.15.2:58586) 2024-05-14T16:53:51.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.770+0000 7f7b2effd700 1 -- 172.21.15.2:0/2582694314 learned_addr learned my addr 172.21.15.2:0/2582694314 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:53:51.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.770+0000 7f7b2f7fe700 1 --2- 172.21.15.2:0/2582694314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555efafd9d20 0x555efafda160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:51.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.770+0000 7f7b267fc700 1 --2- 172.21.15.2:0/2582694314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555efafd6020 0x555efafd97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:51.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.770+0000 7f7b2effd700 1 -- 172.21.15.2:0/2582694314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555efafd6020 msgr2=0x555efafd97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:51.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.770+0000 7f7b2effd700 1 --2- 172.21.15.2:0/2582694314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555efafd6020 0x555efafd97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:51.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.770+0000 7f7b2effd700 1 -- 172.21.15.2:0/2582694314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555efafd9d20 msgr2=0x555efafda160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:51.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.770+0000 7f7b2effd700 1 --2- 172.21.15.2:0/2582694314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555efafd9d20 0x555efafda160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:51.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.770+0000 7f7b2effd700 1 -- 172.21.15.2:0/2582694314 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555efae6f880 con 0x555efafd5700 2024-05-14T16:53:51.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.770+0000 7f7b2f7fe700 1 --2- 172.21.15.2:0/2582694314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555efafd9d20 0x555efafda160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:53:51.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.771+0000 7f7b2effd700 1 --2- 172.21.15.2:0/2582694314 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555efafd5700 0x555efafd5ae0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000fda0 tx=0x7f7b20012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee2e826726302d39 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:51.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.771+0000 7f7b257fa700 1 -- 172.21.15.2:0/2582694314 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b20016070 con 0x555efafd5700 2024-05-14T16:53:51.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.771+0000 7f7b257fa700 1 -- 172.21.15.2:0/2582694314 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b2000fe80 con 0x555efafd5700 2024-05-14T16:53:51.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.771+0000 7f7b257fa700 1 -- 172.21.15.2:0/2582694314 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2001b420 con 0x555efafd5700 2024-05-14T16:53:51.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.772+0000 7f7b3ff0bb80 1 -- 172.21.15.2:0/2582694314 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555efafd5700 msgr2=0x555efafd5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:51.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.772+0000 7f7b3ff0bb80 1 --2- 172.21.15.2:0/2582694314 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555efafd5700 0x555efafd5ae0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000fda0 tx=0x7f7b20012000 comp rx=0 tx=0).stop 2024-05-14T16:53:51.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.772+0000 7f7b3ff0bb80 1 -- 172.21.15.2:0/2582694314 shutdown_connections 2024-05-14T16:53:51.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.772+0000 7f7b3ff0bb80 1 --2- 172.21.15.2:0/2582694314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555efafd9d20 0x555efafda160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:51.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.772+0000 7f7b3ff0bb80 1 --2- 172.21.15.2:0/2582694314 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555efafd5700 0x555efafd5ae0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:51.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.772+0000 7f7b3ff0bb80 1 --2- 172.21.15.2:0/2582694314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555efafd6020 0x555efafd97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:51.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.772+0000 7f7b3ff0bb80 1 -- 172.21.15.2:0/2582694314 >> 172.21.15.2:0/2582694314 conn(0x555efaefc200 msgr2=0x555efafcb930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:51.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.772+0000 7f7b3ff0bb80 1 -- 172.21.15.2:0/2582694314 shutdown_connections 2024-05-14T16:53:51.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.772+0000 7f7b3ff0bb80 1 -- 172.21.15.2:0/2582694314 wait complete. 2024-05-14T16:53:51.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.772+0000 7f7b3ff0bb80 1 -- start start 2024-05-14T16:53:51.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.772+0000 7f7b3ff0bb80 1 Processor -- start 2024-05-14T16:53:51.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.772+0000 7f7b3ff0bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555efafd5700 0x555efafe11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:51.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.772+0000 7f7b3ff0bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555efafd6020 0x555efafde540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:51.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.772+0000 7f7b3ff0bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555efafd9d20 0x555efafdea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:51.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.772+0000 7f7b3ff0bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x555efae25f50 con 0x555efafd6020 2024-05-14T16:53:51.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.772+0000 7f7b3ff0bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555efaec1770 con 0x555efafd5700 2024-05-14T16:53:51.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.772+0000 7f7b3ff0bb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555efafe1f80 con 0x555efafd9d20 2024-05-14T16:53:51.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.773+0000 7f7b3ff0bb80 10 client.0 _ll_register_callbacks cb 0x555efaeb6530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:53:51.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.773+0000 7f7b267fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555efafd6020 0x555efafde540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:51.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.773+0000 7f7b267fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555efafd6020 0x555efafde540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58600/0 (socket says 172.21.15.2:58600) 2024-05-14T16:53:51.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.773+0000 7f7b267fc700 1 -- 172.21.15.2:0/1466249884 learned_addr learned my addr 172.21.15.2:0/1466249884 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:53:51.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.773+0000 7f7b2effd700 1 --2- 172.21.15.2:0/1466249884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555efafd5700 0x555efafe11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:51.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.773+0000 7f7b2f7fe700 1 --2- 172.21.15.2:0/1466249884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555efafd9d20 0x555efafdea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:51.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.773+0000 7f7b267fc700 1 -- 172.21.15.2:0/1466249884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555efafd9d20 msgr2=0x555efafdea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:51.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.773+0000 7f7b267fc700 1 --2- 172.21.15.2:0/1466249884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555efafd9d20 0x555efafdea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:51.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.773+0000 7f7b267fc700 1 -- 172.21.15.2:0/1466249884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555efafd5700 msgr2=0x555efafe11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:51.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.773+0000 7f7b267fc700 1 --2- 172.21.15.2:0/1466249884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555efafd5700 0x555efafe11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:51.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.773+0000 7f7b267fc700 1 -- 172.21.15.2:0/1466249884 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555efaebe640 con 0x555efafd6020 2024-05-14T16:53:51.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.773+0000 7f7b2f7fe700 1 --2- 172.21.15.2:0/1466249884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555efafd9d20 0x555efafdea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:53:51.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.773+0000 7f7b267fc700 1 --2- 172.21.15.2:0/1466249884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555efafd6020 0x555efafde540 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7b180098e0 tx=0x7f7b1800bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=750ea465545d3628 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:51.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.773+0000 7f7b167fc700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:53:51.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.773+0000 7f7b167fc700 1 -- 172.21.15.2:0/1466249884 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b18003a60 con 0x555efafd6020 2024-05-14T16:53:51.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.773+0000 7f7b3ff0bb80 1 -- 172.21.15.2:0/1466249884 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555efaf9d310 con 0x555efafd6020 2024-05-14T16:53:51.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.773+0000 7f7b167fc700 1 -- 172.21.15.2:0/1466249884 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f7b18003c00 con 0x555efafd6020 2024-05-14T16:53:51.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.774+0000 7f7b167fc700 1 -- 172.21.15.2:0/1466249884 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7b18003e00 con 0x555efafd6020 2024-05-14T16:53:51.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.774+0000 7f7b3ff0bb80 10 client.14958 fetch_fsmap learned FSMap version 24 2024-05-14T16:53:51.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.774+0000 7f7b3ff0bb80 1 -- 172.21.15.2:0/1466249884 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555efafec690 con 0x555efafd6020 2024-05-14T16:53:51.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.774+0000 7f7b167fc700 1 -- 172.21.15.2:0/1466249884 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555efaf9d310 con 0x555efafd6020 2024-05-14T16:53:51.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.774+0000 7f7b167fc700 1 -- 172.21.15.2:0/1466249884 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f7b1800f950 con 0x555efafd6020 2024-05-14T16:53:51.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.774+0000 7f7b3ff0bb80 10 client.14958 fetch_fsmap finished waiting for FSMap version 24 2024-05-14T16:53:51.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.774+0000 7f7b3ff0bb80 10 client.14958 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-14T16:53:51.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.774+0000 7f7b3ff0bb80 20 client.14958 populate_metadata read hostname 'smithi002' 2024-05-14T16:53:51.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.774+0000 7f7b3ff0bb80 1 --2- 172.21.15.2:0/1466249884 >> [v2:172.21.15.123:6836/333426645,v1:172.21.15.123:6837/333426645] conn(0x555efafeed30 0x555efaff01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:51.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.774+0000 7f7b3ff0bb80 4 client.14958 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:53:51.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.774+0000 7f7b3ff0bb80 1 -- 172.21.15.2:0/1466249884 --> [v2:172.21.15.123:6836/333426645,v1:172.21.15.123:6837/333426645] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555efaf31b90 con 0x555efafeed30 2024-05-14T16:53:51.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.774+0000 7f7b2effd700 1 --2- 172.21.15.2:0/1466249884 >> [v2:172.21.15.123:6836/333426645,v1:172.21.15.123:6837/333426645] conn(0x555efafeed30 0x555efaff01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:51.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.775+0000 7f7b2effd700 1 --2- 172.21.15.2:0/1466249884 >> [v2:172.21.15.123:6836/333426645,v1:172.21.15.123:6837/333426645] conn(0x555efafeed30 0x555efaff01a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b47b2990b1e1fd3c server_cookie=f2c084422f57cc37 in_seq=0 out_seq=0 2024-05-14T16:53:51.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.775+0000 7f7b167fc700 10 client.14958 ms_handle_connect on v2:172.21.15.123:6836/333426645 2024-05-14T16:53:51.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.776+0000 7f7b167fc700 1 -- 172.21.15.2:0/1466249884 <== mds.1 v2:172.21.15.123:6836/333426645 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555efaf31b90 con 0x555efafeed30 2024-05-14T16:53:51.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.776+0000 7f7b167fc700 10 client.14958 handle_command_reply: tid=0 2024-05-14T16:53:51.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.776+0000 7f7b3ff0bb80 1 client.14958 shutdown 2024-05-14T16:53:51.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.776+0000 7f7b3ff0bb80 1 -- 172.21.15.2:0/1466249884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555efafd6020 msgr2=0x555efafde540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:51.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.776+0000 7f7b3ff0bb80 1 --2- 172.21.15.2:0/1466249884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555efafd6020 0x555efafde540 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7b180098e0 tx=0x7f7b1800bf20 comp rx=0 tx=0).stop 2024-05-14T16:53:51.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.776+0000 7f7b3ff0bb80 1 -- 172.21.15.2:0/1466249884 shutdown_connections 2024-05-14T16:53:51.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.776+0000 7f7b3ff0bb80 1 --2- 172.21.15.2:0/1466249884 >> [v2:172.21.15.123:6836/333426645,v1:172.21.15.123:6837/333426645] conn(0x555efafeed30 0x555efaff01a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:51.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.776+0000 7f7b3ff0bb80 1 --2- 172.21.15.2:0/1466249884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555efafd9d20 0x555efafdea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:51.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.776+0000 7f7b3ff0bb80 1 --2- 172.21.15.2:0/1466249884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555efafd5700 0x555efafe11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:51.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.776+0000 7f7b167fc700 0 client.14958 ms_handle_reset on v2:172.21.15.123:6836/333426645 2024-05-14T16:53:51.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.776+0000 7f7b3ff0bb80 1 --2- 172.21.15.2:0/1466249884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555efafd6020 0x555efafde540 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:51.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.776+0000 7f7b3ff0bb80 1 -- 172.21.15.2:0/1466249884 >> 172.21.15.2:0/1466249884 conn(0x555efaefc200 msgr2=0x555efafdb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:51.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.776+0000 7f7b3ff0bb80 1 -- 172.21.15.2:0/1466249884 shutdown_connections 2024-05-14T16:53:51.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.776+0000 7f7b3ff0bb80 1 -- 172.21.15.2:0/1466249884 wait complete. 2024-05-14T16:53:51.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.776+0000 7f7b3ff0bb80 20 client.14958 trim_cache size 0 max 16384 2024-05-14T16:53:51.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.797+0000 7f7b3ff0bb80 1 Processor -- start 2024-05-14T16:53:51.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.797+0000 7f7b3ff0bb80 1 -- start start 2024-05-14T16:53:51.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.798+0000 7f7b3ff0bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555efaefc200 0x555efafd7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:51.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.798+0000 7f7b3ff0bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555efafd5700 0x555efafd7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:51.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.798+0000 7f7b3ff0bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555efafd6020 0x555efafdddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:51.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.798+0000 7f7b3ff0bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x555efafee4e0 con 0x555efaefc200 2024-05-14T16:53:51.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.798+0000 7f7b3ff0bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555efafeebb0 con 0x555efafd5700 2024-05-14T16:53:51.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.798+0000 7f7b3ff0bb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555efafef280 con 0x555efafd6020 2024-05-14T16:53:51.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.798+0000 7f7b2effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555efaefc200 0x555efafd7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:51.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.798+0000 7f7b2effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555efaefc200 0x555efafd7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58616/0 (socket says 172.21.15.2:58616) 2024-05-14T16:53:51.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.798+0000 7f7b2effd700 1 -- 172.21.15.2:0/3140043876 learned_addr learned my addr 172.21.15.2:0/3140043876 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:53:51.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.798+0000 7f7b267fc700 1 --2- 172.21.15.2:0/3140043876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555efafd5700 0x555efafd7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:51.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.798+0000 7f7b2f7fe700 1 --2- 172.21.15.2:0/3140043876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555efafd6020 0x555efafdddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:51.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.798+0000 7f7b2effd700 1 -- 172.21.15.2:0/3140043876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555efafd6020 msgr2=0x555efafdddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:51.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.798+0000 7f7b2effd700 1 --2- 172.21.15.2:0/3140043876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555efafd6020 0x555efafdddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:51.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.798+0000 7f7b2effd700 1 -- 172.21.15.2:0/3140043876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555efafd5700 msgr2=0x555efafd7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:51.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.798+0000 7f7b2effd700 1 --2- 172.21.15.2:0/3140043876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555efafd5700 0x555efafd7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:51.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.798+0000 7f7b2effd700 1 -- 172.21.15.2:0/3140043876 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555efafdd500 con 0x555efaefc200 2024-05-14T16:53:51.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.798+0000 7f7b2effd700 1 --2- 172.21.15.2:0/3140043876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555efaefc200 0x555efafd7750 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7b20025af0 tx=0x7f7b20025a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e178312f2decfdff server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:51.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.799+0000 7f7b24ff9700 1 -- 172.21.15.2:0/3140043876 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b20039070 con 0x555efaefc200 2024-05-14T16:53:51.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.799+0000 7f7b24ff9700 1 -- 172.21.15.2:0/3140043876 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b20010d70 con 0x555efaefc200 2024-05-14T16:53:51.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.799+0000 7f7b24ff9700 1 -- 172.21.15.2:0/3140043876 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b20016040 con 0x555efaefc200 2024-05-14T16:53:51.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.799+0000 7f7b3ff0bb80 1 -- 172.21.15.2:0/3140043876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555efaefc200 msgr2=0x555efafd7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:51.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.799+0000 7f7b3ff0bb80 1 --2- 172.21.15.2:0/3140043876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555efaefc200 0x555efafd7750 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7b20025af0 tx=0x7f7b20025a80 comp rx=0 tx=0).stop 2024-05-14T16:53:51.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.799+0000 7f7b3ff0bb80 1 -- 172.21.15.2:0/3140043876 shutdown_connections 2024-05-14T16:53:51.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.799+0000 7f7b3ff0bb80 1 --2- 172.21.15.2:0/3140043876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555efafd6020 0x555efafdddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:51.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.799+0000 7f7b3ff0bb80 1 --2- 172.21.15.2:0/3140043876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555efaefc200 0x555efafd7750 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:51.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.799+0000 7f7b3ff0bb80 1 --2- 172.21.15.2:0/3140043876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555efafd5700 0x555efafd7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:51.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.799+0000 7f7b3ff0bb80 1 -- 172.21.15.2:0/3140043876 >> 172.21.15.2:0/3140043876 conn(0x555efafdf430 msgr2=0x555efafd0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:51.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.800+0000 7f7b3ff0bb80 1 -- 172.21.15.2:0/3140043876 shutdown_connections 2024-05-14T16:53:51.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.800+0000 7f7b3ff0bb80 1 -- 172.21.15.2:0/3140043876 wait complete. 2024-05-14T16:53:51.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.800+0000 7f7b3ff0bb80 1 -- start start 2024-05-14T16:53:51.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.800+0000 7f7b3ff0bb80 1 Processor -- start 2024-05-14T16:53:51.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.800+0000 7f7b3ff0bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555efaefc200 0x555efaff9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:51.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.800+0000 7f7b3ff0bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555efafd5700 0x555efaff9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:51.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.800+0000 7f7b3ff0bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555efafd6020 0x555efaff45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:51.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.800+0000 7f7b3ff0bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x555efaff0020 con 0x555efafd5700 2024-05-14T16:53:51.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.800+0000 7f7b3ff0bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555efafef950 con 0x555efaefc200 2024-05-14T16:53:51.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.800+0000 7f7b3ff0bb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555efaff06f0 con 0x555efafd6020 2024-05-14T16:53:51.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.800+0000 7f7b3ff0bb80 10 client.0 _ll_register_callbacks cb 0x555efaeb6530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:53:51.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.800+0000 7f7b267fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555efafd5700 0x555efaff9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:51.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.800+0000 7f7b267fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555efafd5700 0x555efaff9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58626/0 (socket says 172.21.15.2:58626) 2024-05-14T16:53:51.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.800+0000 7f7b267fc700 1 -- 172.21.15.2:0/3374165688 learned_addr learned my addr 172.21.15.2:0/3374165688 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:53:51.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.800+0000 7f7b2effd700 1 --2- 172.21.15.2:0/3374165688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555efaefc200 0x555efaff9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:51.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.801+0000 7f7b267fc700 1 -- 172.21.15.2:0/3374165688 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555efafd6020 msgr2=0x555efaff45d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T16:53:51.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.801+0000 7f7b267fc700 1 --2- 172.21.15.2:0/3374165688 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555efafd6020 0x555efaff45d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:51.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.801+0000 7f7b267fc700 1 -- 172.21.15.2:0/3374165688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555efaefc200 msgr2=0x555efaff9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:51.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.801+0000 7f7b267fc700 1 --2- 172.21.15.2:0/3374165688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555efaefc200 0x555efaff9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:51.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.801+0000 7f7b267fc700 1 -- 172.21.15.2:0/3374165688 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555efaff4bc0 con 0x555efafd5700 2024-05-14T16:53:51.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.801+0000 7f7b2effd700 1 --2- 172.21.15.2:0/3374165688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555efaefc200 0x555efaff9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:53:51.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.801+0000 7f7b267fc700 1 --2- 172.21.15.2:0/3374165688 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555efafd5700 0x555efaff9b60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x555efafdfb10 tx=0x7f7b1800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ce12f85f48697e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:51.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.801+0000 7f7b167fc700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:53:51.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.801+0000 7f7b167fc700 1 -- 172.21.15.2:0/3374165688 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1803c060 con 0x555efafd5700 2024-05-14T16:53:51.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.801+0000 7f7b167fc700 1 -- 172.21.15.2:0/3374165688 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f7b180054d0 con 0x555efafd5700 2024-05-14T16:53:51.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.801+0000 7f7b3ff0bb80 1 -- 172.21.15.2:0/3374165688 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b20010d70 con 0x555efafd5700 2024-05-14T16:53:51.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.801+0000 7f7b167fc700 1 -- 172.21.15.2:0/3374165688 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7b18005c00 con 0x555efafd5700 2024-05-14T16:53:51.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.801+0000 7f7b3ff0bb80 10 client.14964 fetch_fsmap learned FSMap version 24 2024-05-14T16:53:51.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.801+0000 7f7b3ff0bb80 1 -- 172.21.15.2:0/3374165688 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555efb05aa80 con 0x555efafd5700 2024-05-14T16:53:51.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.802+0000 7f7b167fc700 1 -- 172.21.15.2:0/3374165688 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b20010d70 con 0x555efafd5700 2024-05-14T16:53:51.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.802+0000 7f7b167fc700 1 -- 172.21.15.2:0/3374165688 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f7b1802b850 con 0x555efafd5700 2024-05-14T16:53:51.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.802+0000 7f7b3ff0bb80 10 client.14964 fetch_fsmap finished waiting for FSMap version 24 2024-05-14T16:53:51.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.802+0000 7f7b3ff0bb80 10 client.14964 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-14T16:53:51.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.802+0000 7f7b3ff0bb80 20 client.14964 populate_metadata read hostname 'smithi002' 2024-05-14T16:53:51.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.802+0000 7f7b3ff0bb80 1 --2- 172.21.15.2:0/3374165688 >> [v2:172.21.15.123:6836/333426645,v1:172.21.15.123:6837/333426645] conn(0x555efafd9d20 0x555efb05e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:51.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.802+0000 7f7b3ff0bb80 4 client.14964 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T16:53:51.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.802+0000 7f7b3ff0bb80 1 -- 172.21.15.2:0/3374165688 --> [v2:172.21.15.123:6836/333426645,v1:172.21.15.123:6837/333426645] -- command(tid 0: {"prefix": "status"}) -- 0x555efaf2b370 con 0x555efafd9d20 2024-05-14T16:53:51.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.802+0000 7f7b2effd700 1 --2- 172.21.15.2:0/3374165688 >> [v2:172.21.15.123:6836/333426645,v1:172.21.15.123:6837/333426645] conn(0x555efafd9d20 0x555efb05e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:51.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.803+0000 7f7b2effd700 1 --2- 172.21.15.2:0/3374165688 >> [v2:172.21.15.123:6836/333426645,v1:172.21.15.123:6837/333426645] conn(0x555efafd9d20 0x555efb05e290 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=af025e43cceecc5d server_cookie=749964da32f4b682 in_seq=0 out_seq=0 2024-05-14T16:53:51.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.803+0000 7f7b167fc700 10 client.14964 ms_handle_connect on v2:172.21.15.123:6836/333426645 2024-05-14T16:53:51.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.803+0000 7f7b167fc700 1 -- 172.21.15.2:0/3374165688 <== mds.1 v2:172.21.15.123:6836/333426645 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x555efaf2b370 con 0x555efafd9d20 2024-05-14T16:53:51.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.803+0000 7f7b167fc700 10 client.14964 handle_command_reply: tid=0 2024-05-14T16:53:51.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.803+0000 7f7b3ff0bb80 1 client.14964 shutdown 2024-05-14T16:53:51.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.803+0000 7f7b3ff0bb80 1 -- 172.21.15.2:0/3374165688 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555efafd5700 msgr2=0x555efaff9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:51.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.803+0000 7f7b3ff0bb80 1 --2- 172.21.15.2:0/3374165688 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555efafd5700 0x555efaff9b60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x555efafdfb10 tx=0x7f7b1800be80 comp rx=0 tx=0).stop 2024-05-14T16:53:51.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.803+0000 7f7b3ff0bb80 1 -- 172.21.15.2:0/3374165688 shutdown_connections 2024-05-14T16:53:51.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.803+0000 7f7b3ff0bb80 1 --2- 172.21.15.2:0/3374165688 >> [v2:172.21.15.123:6836/333426645,v1:172.21.15.123:6837/333426645] conn(0x555efafd9d20 0x555efb05e290 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:51.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.803+0000 7f7b3ff0bb80 1 --2- 172.21.15.2:0/3374165688 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555efafd6020 0x555efaff45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:51.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.803+0000 7f7b3ff0bb80 1 --2- 172.21.15.2:0/3374165688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555efaefc200 0x555efaff9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:51.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.803+0000 7f7b3ff0bb80 1 --2- 172.21.15.2:0/3374165688 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555efafd5700 0x555efaff9b60 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:51.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.803+0000 7f7b167fc700 0 client.14964 ms_handle_reset on v2:172.21.15.123:6836/333426645 2024-05-14T16:53:51.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.803+0000 7f7b3ff0bb80 1 -- 172.21.15.2:0/3374165688 >> 172.21.15.2:0/3374165688 conn(0x555efafdf430 msgr2=0x555efafd0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:51.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.803+0000 7f7b3ff0bb80 1 -- 172.21.15.2:0/3374165688 shutdown_connections 2024-05-14T16:53:51.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.803+0000 7f7b3ff0bb80 1 -- 172.21.15.2:0/3374165688 wait complete. 2024-05-14T16:53:51.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.803+0000 7f7b3ff0bb80 20 client.14964 trim_cache size 0 max 16384 2024-05-14T16:53:51.809 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T16:53:51.809 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "ed40717a-1210-11ef-bc98-c7b262605968", 2024-05-14T16:53:51.809 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 1, 2024-05-14T16:53:51.809 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24455, 2024-05-14T16:53:51.809 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2024-05-14T16:53:51.809 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2024-05-14T16:53:51.809 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2024-05-14T16:53:51.809 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 124.831367267, 2024-05-14T16:53:51.809 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 24, 2024-05-14T16:53:51.809 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 80, 2024-05-14T16:53:51.809 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 78, 2024-05-14T16:53:51.809 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 166.34415834699999 2024-05-14T16:53:51.810 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T16:53:51.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.804+0000 7f7b35675700 1 -- 172.21.15.2:0/2071449081 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f7b1c0b2900 msgr2=0x7f7b1c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:51.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.804+0000 7f7b35675700 1 --2- 172.21.15.2:0/2071449081 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f7b1c0b2900 0x7f7b1c0b4dc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7b20005950 tx=0x7f7b2000a300 comp rx=0 tx=0).stop 2024-05-14T16:53:51.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.804+0000 7f7b35675700 1 -- 172.21.15.2:0/2071449081 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b301088f0 msgr2=0x7f7b301999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:51.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.804+0000 7f7b35675700 1 --2- 172.21.15.2:0/2071449081 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b301088f0 0x7f7b301999b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7b28007ee0 tx=0x7f7b2800d3b0 comp rx=0 tx=0).stop 2024-05-14T16:53:51.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.804+0000 7f7b35675700 1 -- 172.21.15.2:0/2071449081 shutdown_connections 2024-05-14T16:53:51.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.804+0000 7f7b35675700 1 --2- 172.21.15.2:0/2071449081 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f7b1c0b2900 0x7f7b1c0b4dc0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:51.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.804+0000 7f7b35675700 1 --2- 172.21.15.2:0/2071449081 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b301088f0 0x7f7b301999b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:51.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.804+0000 7f7b35675700 1 --2- 172.21.15.2:0/2071449081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b301037f0 0x7f7b3019f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:51.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.804+0000 7f7b35675700 1 --2- 172.21.15.2:0/2071449081 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b301041a0 0x7f7b3019fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:51.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.804+0000 7f7b35675700 1 -- 172.21.15.2:0/2071449081 >> 172.21.15.2:0/2071449081 conn(0x7f7b3007a6a0 msgr2=0x7f7b30109a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:51.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.804+0000 7f7b35675700 1 -- 172.21.15.2:0/2071449081 shutdown_connections 2024-05-14T16:53:51.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:51.804+0000 7f7b35675700 1 -- 172.21.15.2:0/2071449081 wait complete. 2024-05-14T16:53:51.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:51 smithi151 ceph-mon[36066]: pgmap v359: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T16:53:51.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:51 smithi123 ceph-mon[35971]: pgmap v359: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T16:53:52.445 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph tell mds.b status 2024-05-14T16:53:52.693 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:53:53.675 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:53 smithi151 ceph-mon[36066]: pgmap v360: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:53:53.712 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:53 smithi123 ceph-mon[35971]: pgmap v360: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:53:53.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:53 smithi002 ceph-mon[31538]: pgmap v360: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:53:54.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.302+0000 7f4ec0418700 1 -- 172.21.15.2:0/2831976126 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4eb81041a0 msgr2=0x7f4eb81081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:54.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.302+0000 7f4ec0418700 1 --2- 172.21.15.2:0/2831976126 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4eb81041a0 0x7f4eb81081c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4ea80099c0 tx=0x7f4ea8009cd0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.302+0000 7f4ec0418700 1 -- 172.21.15.2:0/2831976126 shutdown_connections 2024-05-14T16:53:54.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.302+0000 7f4ec0418700 1 --2- 172.21.15.2:0/2831976126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4eb81088f0 0x7f4eb8110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.302+0000 7f4ec0418700 1 --2- 172.21.15.2:0/2831976126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4eb81037f0 0x7f4eb8103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.302+0000 7f4ec0418700 1 --2- 172.21.15.2:0/2831976126 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4eb81041a0 0x7f4eb81081c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.302+0000 7f4ec0418700 1 -- 172.21.15.2:0/2831976126 >> 172.21.15.2:0/2831976126 conn(0x7f4eb807a6a0 msgr2=0x7f4eb807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:54.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.302+0000 7f4ec0418700 1 -- 172.21.15.2:0/2831976126 shutdown_connections 2024-05-14T16:53:54.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.302+0000 7f4ec0418700 1 -- 172.21.15.2:0/2831976126 wait complete. 2024-05-14T16:53:54.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.303+0000 7f4ec0418700 1 Processor -- start 2024-05-14T16:53:54.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.303+0000 7f4ec0418700 1 -- start start 2024-05-14T16:53:54.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.303+0000 7f4ec0418700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4eb81037f0 0x7f4eb819bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:54.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.303+0000 7f4ec0418700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4eb81041a0 0x7f4eb819c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:54.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.303+0000 7f4ec0418700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4eb81088f0 0x7f4eb81a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:54.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.303+0000 7f4ec0418700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb81136d0 con 0x7f4eb81088f0 2024-05-14T16:53:54.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.303+0000 7f4ec0418700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb8113550 con 0x7f4eb81041a0 2024-05-14T16:53:54.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.303+0000 7f4ec0418700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb8113850 con 0x7f4eb81037f0 2024-05-14T16:53:54.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.303+0000 7f4ebe1b4700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4eb81037f0 0x7f4eb819bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:54.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.303+0000 7f4ebd9b3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4eb81041a0 0x7f4eb819c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:54.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.303+0000 7f4ebe9b5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4eb81088f0 0x7f4eb81a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:54.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.303+0000 7f4ebe1b4700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4eb81037f0 0x7f4eb819bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:54790/0 (socket says 172.21.15.2:54790) 2024-05-14T16:53:54.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.303+0000 7f4ebe9b5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4eb81088f0 0x7f4eb81a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43994/0 (socket says 172.21.15.2:43994) 2024-05-14T16:53:54.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.303+0000 7f4ebd9b3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4eb81041a0 0x7f4eb819c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:36146/0 (socket says 172.21.15.2:36146) 2024-05-14T16:53:54.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.303+0000 7f4ebe1b4700 1 -- 172.21.15.2:0/1719688130 learned_addr learned my addr 172.21.15.2:0/1719688130 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:53:54.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.304+0000 7f4ebe1b4700 1 -- 172.21.15.2:0/1719688130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4eb81041a0 msgr2=0x7f4eb819c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:54.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.304+0000 7f4ebe1b4700 1 --2- 172.21.15.2:0/1719688130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4eb81041a0 0x7f4eb819c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.304+0000 7f4ebe1b4700 1 -- 172.21.15.2:0/1719688130 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4eb81088f0 msgr2=0x7f4eb81a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:54.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.304+0000 7f4ebe1b4700 1 --2- 172.21.15.2:0/1719688130 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4eb81088f0 0x7f4eb81a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.304+0000 7f4ebe1b4700 1 -- 172.21.15.2:0/1719688130 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb81a0d30 con 0x7f4eb81037f0 2024-05-14T16:53:54.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.304+0000 7f4ebe9b5700 1 --2- 172.21.15.2:0/1719688130 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4eb81088f0 0x7f4eb81a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:53:54.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.304+0000 7f4ebd9b3700 1 --2- 172.21.15.2:0/1719688130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4eb81041a0 0x7f4eb819c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:53:54.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.304+0000 7f4ebe1b4700 1 --2- 172.21.15.2:0/1719688130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4eb81037f0 0x7f4eb819bd20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4eb400c9b0 tx=0x7f4eb400ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:54.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.304+0000 7f4eaf7fe700 1 -- 172.21.15.2:0/1719688130 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4007b00 con 0x7f4eb81037f0 2024-05-14T16:53:54.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.304+0000 7f4eaf7fe700 1 -- 172.21.15.2:0/1719688130 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4eb400ce50 con 0x7f4eb81037f0 2024-05-14T16:53:54.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.304+0000 7f4ec0418700 1 -- 172.21.15.2:0/1719688130 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eb81a1020 con 0x7f4eb81037f0 2024-05-14T16:53:54.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.304+0000 7f4eaf7fe700 1 -- 172.21.15.2:0/1719688130 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4018660 con 0x7f4eb81037f0 2024-05-14T16:53:54.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.304+0000 7f4ec0418700 1 -- 172.21.15.2:0/1719688130 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eb81a88a0 con 0x7f4eb81037f0 2024-05-14T16:53:54.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.307+0000 7f4eaf7fe700 1 -- 172.21.15.2:0/1719688130 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f4eb400f450 con 0x7f4eb81037f0 2024-05-14T16:53:54.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.307+0000 7f4eaf7fe700 1 --2- 172.21.15.2:0/1719688130 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f4ea40b2880 0x7f4ea40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:54.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.307+0000 7f4ebd9b3700 1 --2- 172.21.15.2:0/1719688130 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f4ea40b2880 0x7f4ea40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:54.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.307+0000 7f4eaf7fe700 1 -- 172.21.15.2:0/1719688130 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f4eb40e2190 con 0x7f4eb81037f0 2024-05-14T16:53:54.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.308+0000 7f4ebd9b3700 1 --2- 172.21.15.2:0/1719688130 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f4ea40b2880 0x7f4ea40b4d40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8009990 tx=0x7f4ea8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:54.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.308+0000 7f4ecacaeb80 1 Processor -- start 2024-05-14T16:53:54.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.308+0000 7f4ecacaeb80 1 -- start start 2024-05-14T16:53:54.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.309+0000 7f4ecacaeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a6d14bfa0 0x564a6d14c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:54.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.309+0000 7f4ecacaeb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a6d14c8c0 0x564a6d1467a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:54.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.309+0000 7f4ecacaeb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a6d146dc0 0x564a6d147220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:54.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.309+0000 7f4ecacaeb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x564a6d0a4d20 con 0x564a6d14c8c0 2024-05-14T16:53:54.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.309+0000 7f4ecacaeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564a6d0948c0 con 0x564a6d14bfa0 2024-05-14T16:53:54.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.309+0000 7f4ecacaeb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564a6d098900 con 0x564a6d146dc0 2024-05-14T16:53:54.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.309+0000 7f4ebd9b3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a6d14c8c0 0x564a6d1467a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:54.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.309+0000 7f4ebd9b3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a6d14c8c0 0x564a6d1467a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44004/0 (socket says 172.21.15.2:44004) 2024-05-14T16:53:54.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.309+0000 7f4ebd9b3700 1 -- 172.21.15.2:0/1459918544 learned_addr learned my addr 172.21.15.2:0/1459918544 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:53:54.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.309+0000 7f4ebe9b5700 1 --2- 172.21.15.2:0/1459918544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a6d146dc0 0x564a6d147220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:54.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.309+0000 7f4ebe1b4700 1 --2- 172.21.15.2:0/1459918544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a6d14bfa0 0x564a6d14c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:54.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.309+0000 7f4ebd9b3700 1 -- 172.21.15.2:0/1459918544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a6d146dc0 msgr2=0x564a6d147220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:54.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.309+0000 7f4ebd9b3700 1 --2- 172.21.15.2:0/1459918544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a6d146dc0 0x564a6d147220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.309+0000 7f4ebd9b3700 1 -- 172.21.15.2:0/1459918544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a6d14bfa0 msgr2=0x564a6d14c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:54.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.309+0000 7f4ebd9b3700 1 --2- 172.21.15.2:0/1459918544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a6d14bfa0 0x564a6d14c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.309+0000 7f4ebd9b3700 1 -- 172.21.15.2:0/1459918544 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a6d03df60 con 0x564a6d14c8c0 2024-05-14T16:53:54.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.309+0000 7f4ebe9b5700 1 --2- 172.21.15.2:0/1459918544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a6d146dc0 0x564a6d147220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:53:54.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.309+0000 7f4ebd9b3700 1 --2- 172.21.15.2:0/1459918544 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a6d14c8c0 0x564a6d1467a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8015000 tx=0x7f4ea8020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8726210993efbf6b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:54.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.309+0000 7f4eacd70700 1 -- 172.21.15.2:0/1459918544 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea80050c0 con 0x564a6d14c8c0 2024-05-14T16:53:54.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.309+0000 7f4eacd70700 1 -- 172.21.15.2:0/1459918544 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ea800f460 con 0x564a6d14c8c0 2024-05-14T16:53:54.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.310+0000 7f4eacd70700 1 -- 172.21.15.2:0/1459918544 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8024670 con 0x564a6d14c8c0 2024-05-14T16:53:54.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.310+0000 7f4ecacaeb80 1 -- 172.21.15.2:0/1459918544 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a6d14c8c0 msgr2=0x564a6d1467a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:54.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.310+0000 7f4ecacaeb80 1 --2- 172.21.15.2:0/1459918544 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a6d14c8c0 0x564a6d1467a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8015000 tx=0x7f4ea8020000 comp rx=0 tx=0).stop 2024-05-14T16:53:54.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.310+0000 7f4ecacaeb80 1 -- 172.21.15.2:0/1459918544 shutdown_connections 2024-05-14T16:53:54.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.310+0000 7f4ecacaeb80 1 --2- 172.21.15.2:0/1459918544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a6d146dc0 0x564a6d147220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.310+0000 7f4ecacaeb80 1 --2- 172.21.15.2:0/1459918544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a6d14bfa0 0x564a6d14c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.310+0000 7f4ecacaeb80 1 --2- 172.21.15.2:0/1459918544 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a6d14c8c0 0x564a6d1467a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.310+0000 7f4ecacaeb80 1 -- 172.21.15.2:0/1459918544 >> 172.21.15.2:0/1459918544 conn(0x564a6d073d60 msgr2=0x564a6d140e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:54.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.310+0000 7f4ecacaeb80 1 -- 172.21.15.2:0/1459918544 shutdown_connections 2024-05-14T16:53:54.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.310+0000 7f4ecacaeb80 1 -- 172.21.15.2:0/1459918544 wait complete. 2024-05-14T16:53:54.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.311+0000 7f4ecacaeb80 1 -- start start 2024-05-14T16:53:54.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.311+0000 7f4ecacaeb80 1 Processor -- start 2024-05-14T16:53:54.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.311+0000 7f4ecacaeb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a6d146dc0 0x564a6d152560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:54.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.311+0000 7f4ecacaeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a6d14bfa0 0x564a6d14f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:54.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.311+0000 7f4ecacaeb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a6d14c8c0 0x564a6d14fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:54.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.311+0000 7f4ecacaeb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x564a6cf9af50 con 0x564a6d14c8c0 2024-05-14T16:53:54.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.311+0000 7f4ecacaeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564a6cffcd50 con 0x564a6d14bfa0 2024-05-14T16:53:54.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.311+0000 7f4ecacaeb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564a6d153270 con 0x564a6d146dc0 2024-05-14T16:53:54.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.311+0000 7f4ecacaeb80 10 client.0 _ll_register_callbacks cb 0x564a6d02f900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:53:54.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.311+0000 7f4ebe9b5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a6d14c8c0 0x564a6d14fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:54.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.311+0000 7f4ebe9b5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a6d14c8c0 0x564a6d14fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44014/0 (socket says 172.21.15.2:44014) 2024-05-14T16:53:54.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.311+0000 7f4ebe9b5700 1 -- 172.21.15.2:0/3185886397 learned_addr learned my addr 172.21.15.2:0/3185886397 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:53:54.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.311+0000 7f4ebe1b4700 1 --2- 172.21.15.2:0/3185886397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a6d146dc0 0x564a6d152560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:54.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.311+0000 7f4ebd9b3700 1 --2- 172.21.15.2:0/3185886397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a6d14bfa0 0x564a6d14f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:54.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.312+0000 7f4ebe9b5700 1 -- 172.21.15.2:0/3185886397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a6d146dc0 msgr2=0x564a6d152560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:54.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.312+0000 7f4ebe9b5700 1 --2- 172.21.15.2:0/3185886397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a6d146dc0 0x564a6d152560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.312+0000 7f4ebe9b5700 1 -- 172.21.15.2:0/3185886397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a6d14bfa0 msgr2=0x564a6d14f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:54.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.312+0000 7f4ebe9b5700 1 --2- 172.21.15.2:0/3185886397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a6d14bfa0 0x564a6d14f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.312+0000 7f4ebe9b5700 1 -- 172.21.15.2:0/3185886397 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a6cfe4880 con 0x564a6d14c8c0 2024-05-14T16:53:54.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.312+0000 7f4ebe1b4700 1 --2- 172.21.15.2:0/3185886397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a6d146dc0 0x564a6d152560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:53:54.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.312+0000 7f4ebe9b5700 1 --2- 172.21.15.2:0/3185886397 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a6d14c8c0 0x564a6d14fe40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0007fc0 tx=0x7f4eb0012520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae9b8e1c12b2287 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:54.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.312+0000 7f4ea1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:53:54.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.312+0000 7f4ea1ffb700 1 -- 172.21.15.2:0/3185886397 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb001c020 con 0x564a6d14c8c0 2024-05-14T16:53:54.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.312+0000 7f4ea1ffb700 1 -- 172.21.15.2:0/3185886397 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f4eb0018470 con 0x564a6d14c8c0 2024-05-14T16:53:54.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.312+0000 7f4ecacaeb80 1 -- 172.21.15.2:0/3185886397 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ea800f460 con 0x564a6d14c8c0 2024-05-14T16:53:54.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.312+0000 7f4ea1ffb700 1 -- 172.21.15.2:0/3185886397 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4eb0018670 con 0x564a6d14c8c0 2024-05-14T16:53:54.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.312+0000 7f4ecacaeb80 10 client.14976 fetch_fsmap learned FSMap version 24 2024-05-14T16:53:54.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.312+0000 7f4ecacaeb80 1 -- 172.21.15.2:0/3185886397 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564a6d04e710 con 0x564a6d14c8c0 2024-05-14T16:53:54.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.312+0000 7f4ea1ffb700 1 -- 172.21.15.2:0/3185886397 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ea800f460 con 0x564a6d14c8c0 2024-05-14T16:53:54.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.313+0000 7f4ea1ffb700 1 -- 172.21.15.2:0/3185886397 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f4eb0023910 con 0x564a6d14c8c0 2024-05-14T16:53:54.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.313+0000 7f4ecacaeb80 10 client.14976 fetch_fsmap finished waiting for FSMap version 24 2024-05-14T16:53:54.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.313+0000 7f4ecacaeb80 10 client.14976 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T16:53:54.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.313+0000 7f4ecacaeb80 20 client.14976 populate_metadata read hostname 'smithi002' 2024-05-14T16:53:54.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.313+0000 7f4ecacaeb80 1 --2- 172.21.15.2:0/3185886397 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564a6d15fe70 0x564a6d1612e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:54.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.313+0000 7f4ecacaeb80 4 client.14976 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:53:54.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.313+0000 7f4ecacaeb80 1 -- 172.21.15.2:0/3185886397 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a6d0a6c70 con 0x564a6d15fe70 2024-05-14T16:53:54.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.313+0000 7f4ebe1b4700 1 --2- 172.21.15.2:0/3185886397 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564a6d15fe70 0x564a6d1612e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:54.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.313+0000 7f4ebe1b4700 1 --2- 172.21.15.2:0/3185886397 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564a6d15fe70 0x564a6d1612e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68560f38b2ac0319 server_cookie=62f1104ef508d192 in_seq=0 out_seq=0 2024-05-14T16:53:54.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.313+0000 7f4ea1ffb700 10 client.14976 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:53:54.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.314+0000 7f4ea1ffb700 1 -- 172.21.15.2:0/3185886397 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564a6d0a6c70 con 0x564a6d15fe70 2024-05-14T16:53:54.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.314+0000 7f4ea1ffb700 10 client.14976 handle_command_reply: tid=0 2024-05-14T16:53:54.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.314+0000 7f4ecacaeb80 1 client.14976 shutdown 2024-05-14T16:53:54.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.314+0000 7f4ecacaeb80 1 -- 172.21.15.2:0/3185886397 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a6d14c8c0 msgr2=0x564a6d14fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:54.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.314+0000 7f4ecacaeb80 1 --2- 172.21.15.2:0/3185886397 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a6d14c8c0 0x564a6d14fe40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0007fc0 tx=0x7f4eb0012520 comp rx=0 tx=0).stop 2024-05-14T16:53:54.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.315+0000 7f4ecacaeb80 1 -- 172.21.15.2:0/3185886397 shutdown_connections 2024-05-14T16:53:54.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.315+0000 7f4ecacaeb80 1 --2- 172.21.15.2:0/3185886397 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564a6d15fe70 0x564a6d1612e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.315+0000 7f4ecacaeb80 1 --2- 172.21.15.2:0/3185886397 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a6d14c8c0 0x564a6d14fe40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.315+0000 7f4ecacaeb80 1 --2- 172.21.15.2:0/3185886397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a6d146dc0 0x564a6d152560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.315+0000 7f4ecacaeb80 1 --2- 172.21.15.2:0/3185886397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a6d14bfa0 0x564a6d14f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.315+0000 7f4ecacaeb80 1 -- 172.21.15.2:0/3185886397 >> 172.21.15.2:0/3185886397 conn(0x564a6d073d60 msgr2=0x564a6d140630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:54.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.315+0000 7f4ea1ffb700 0 client.14976 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:53:54.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.315+0000 7f4ecacaeb80 1 -- 172.21.15.2:0/3185886397 shutdown_connections 2024-05-14T16:53:54.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.315+0000 7f4ecacaeb80 1 -- 172.21.15.2:0/3185886397 wait complete. 2024-05-14T16:53:54.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.315+0000 7f4ecacaeb80 20 client.14976 trim_cache size 0 max 16384 2024-05-14T16:53:54.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.336+0000 7f4ecacaeb80 1 Processor -- start 2024-05-14T16:53:54.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.336+0000 7f4ecacaeb80 1 -- start start 2024-05-14T16:53:54.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.336+0000 7f4ecacaeb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a6d146dc0 0x564a6d1471a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:54.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.336+0000 7f4ecacaeb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a6d14bfa0 0x564a6d14ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:54.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.336+0000 7f4ecacaeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a6d14c8c0 0x564a6d140db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:54.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.336+0000 7f4ecacaeb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x564a6d15f620 con 0x564a6d146dc0 2024-05-14T16:53:54.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.336+0000 7f4ecacaeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564a6d15fcf0 con 0x564a6d14c8c0 2024-05-14T16:53:54.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.336+0000 7f4ecacaeb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564a6d1603c0 con 0x564a6d14bfa0 2024-05-14T16:53:54.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.336+0000 7f4ebe1b4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a6d146dc0 0x564a6d1471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:54.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.336+0000 7f4ebe1b4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a6d146dc0 0x564a6d1471a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44022/0 (socket says 172.21.15.2:44022) 2024-05-14T16:53:54.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.336+0000 7f4ebe1b4700 1 -- 172.21.15.2:0/462015164 learned_addr learned my addr 172.21.15.2:0/462015164 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:53:54.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.336+0000 7f4ebd9b3700 1 --2- 172.21.15.2:0/462015164 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a6d14bfa0 0x564a6d14ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:54.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.336+0000 7f4ebe1b4700 1 -- 172.21.15.2:0/462015164 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a6d14bfa0 msgr2=0x564a6d14ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:54.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.336+0000 7f4ebe1b4700 1 --2- 172.21.15.2:0/462015164 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a6d14bfa0 0x564a6d14ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.336+0000 7f4ebe1b4700 1 -- 172.21.15.2:0/462015164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a6d14c8c0 msgr2=0x564a6d140db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:54.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.336+0000 7f4ebe1b4700 1 --2- 172.21.15.2:0/462015164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a6d14c8c0 0x564a6d140db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.336+0000 7f4ebe1b4700 1 -- 172.21.15.2:0/462015164 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a6d1601e0 con 0x564a6d146dc0 2024-05-14T16:53:54.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.337+0000 7f4ebe1b4700 1 --2- 172.21.15.2:0/462015164 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a6d146dc0 0x564a6d1471a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4eb400fba0 tx=0x7f4eb40cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=726e8dc82684116f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:54.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.337+0000 7f4ea3fff700 1 -- 172.21.15.2:0/462015164 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb40adb10 con 0x564a6d146dc0 2024-05-14T16:53:54.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.337+0000 7f4ea3fff700 1 -- 172.21.15.2:0/462015164 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4eb40b25c0 con 0x564a6d146dc0 2024-05-14T16:53:54.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.337+0000 7f4ea3fff700 1 -- 172.21.15.2:0/462015164 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb40b2130 con 0x564a6d146dc0 2024-05-14T16:53:54.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.337+0000 7f4ecacaeb80 1 -- 172.21.15.2:0/462015164 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a6d146dc0 msgr2=0x564a6d1471a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:54.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.337+0000 7f4ecacaeb80 1 --2- 172.21.15.2:0/462015164 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a6d146dc0 0x564a6d1471a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4eb400fba0 tx=0x7f4eb40cc040 comp rx=0 tx=0).stop 2024-05-14T16:53:54.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.337+0000 7f4ecacaeb80 1 -- 172.21.15.2:0/462015164 shutdown_connections 2024-05-14T16:53:54.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.337+0000 7f4ecacaeb80 1 --2- 172.21.15.2:0/462015164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a6d14c8c0 0x564a6d140db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.337+0000 7f4ecacaeb80 1 --2- 172.21.15.2:0/462015164 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a6d146dc0 0x564a6d1471a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.337+0000 7f4ecacaeb80 1 --2- 172.21.15.2:0/462015164 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a6d14bfa0 0x564a6d14ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.337+0000 7f4ecacaeb80 1 -- 172.21.15.2:0/462015164 >> 172.21.15.2:0/462015164 conn(0x564a6d073d60 msgr2=0x564a6d14e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:54.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.337+0000 7f4ecacaeb80 1 -- 172.21.15.2:0/462015164 shutdown_connections 2024-05-14T16:53:54.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.337+0000 7f4ecacaeb80 1 -- 172.21.15.2:0/462015164 wait complete. 2024-05-14T16:53:54.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.338+0000 7f4ecacaeb80 1 -- start start 2024-05-14T16:53:54.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.338+0000 7f4ecacaeb80 1 Processor -- start 2024-05-14T16:53:54.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.338+0000 7f4ecacaeb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a6d146dc0 0x564a6d1401a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:54.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.338+0000 7f4ecacaeb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a6d14bfa0 0x564a6d1406e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:54.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.338+0000 7f4ecacaeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a6d14c8c0 0x564a6d140c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:54.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.338+0000 7f4ecacaeb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x564a6d161160 con 0x564a6d146dc0 2024-05-14T16:53:54.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.338+0000 7f4ecacaeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564a6d160a90 con 0x564a6d14c8c0 2024-05-14T16:53:54.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.338+0000 7f4ecacaeb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564a6d161830 con 0x564a6d14bfa0 2024-05-14T16:53:54.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.338+0000 7f4ecacaeb80 10 client.0 _ll_register_callbacks cb 0x564a6d02f900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:53:54.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.338+0000 7f4ebe1b4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a6d146dc0 0x564a6d1401a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:54.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.338+0000 7f4ebe1b4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a6d146dc0 0x564a6d1401a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44034/0 (socket says 172.21.15.2:44034) 2024-05-14T16:53:54.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.338+0000 7f4ebe1b4700 1 -- 172.21.15.2:0/3358608298 learned_addr learned my addr 172.21.15.2:0/3358608298 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:53:54.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.338+0000 7f4ebe9b5700 1 --2- 172.21.15.2:0/3358608298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a6d14c8c0 0x564a6d140c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:54.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.338+0000 7f4ebd9b3700 1 --2- 172.21.15.2:0/3358608298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a6d14bfa0 0x564a6d1406e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:54.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.339+0000 7f4ebe1b4700 1 -- 172.21.15.2:0/3358608298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a6d14bfa0 msgr2=0x564a6d1406e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:54.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.339+0000 7f4ebe1b4700 1 --2- 172.21.15.2:0/3358608298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a6d14bfa0 0x564a6d1406e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.339+0000 7f4ebe1b4700 1 -- 172.21.15.2:0/3358608298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a6d14c8c0 msgr2=0x564a6d140c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:53:54.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.339+0000 7f4ebe1b4700 1 --2- 172.21.15.2:0/3358608298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a6d14c8c0 0x564a6d140c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.339+0000 7f4ebe1b4700 1 -- 172.21.15.2:0/3358608298 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a6d168ac0 con 0x564a6d146dc0 2024-05-14T16:53:54.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.339+0000 7f4ebe1b4700 1 --2- 172.21.15.2:0/3358608298 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a6d146dc0 0x564a6d1401a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4eb40dfc10 tx=0x7f4eb400fbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cea86fb5edf1d1b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:54.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.339+0000 7f4ea1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:53:54.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.339+0000 7f4ea1ffb700 1 -- 172.21.15.2:0/3358608298 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4007ca0 con 0x564a6d146dc0 2024-05-14T16:53:54.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.339+0000 7f4ea1ffb700 1 -- 172.21.15.2:0/3358608298 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f4eb40a99c0 con 0x564a6d146dc0 2024-05-14T16:53:54.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.339+0000 7f4ecacaeb80 1 -- 172.21.15.2:0/3358608298 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4eb40b25c0 con 0x564a6d146dc0 2024-05-14T16:53:54.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.339+0000 7f4ea1ffb700 1 -- 172.21.15.2:0/3358608298 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4eb40a9bc0 con 0x564a6d146dc0 2024-05-14T16:53:54.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.339+0000 7f4ecacaeb80 10 client.14982 fetch_fsmap learned FSMap version 24 2024-05-14T16:53:54.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.339+0000 7f4ecacaeb80 1 -- 172.21.15.2:0/3358608298 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564a6d1cb5d0 con 0x564a6d146dc0 2024-05-14T16:53:54.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.339+0000 7f4ea1ffb700 1 -- 172.21.15.2:0/3358608298 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4eb40b25c0 con 0x564a6d146dc0 2024-05-14T16:53:54.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.339+0000 7f4ea1ffb700 1 -- 172.21.15.2:0/3358608298 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f4eb40becd0 con 0x564a6d146dc0 2024-05-14T16:53:54.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.339+0000 7f4ecacaeb80 10 client.14982 fetch_fsmap finished waiting for FSMap version 24 2024-05-14T16:53:54.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.340+0000 7f4ecacaeb80 10 client.14982 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T16:53:54.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.340+0000 7f4ecacaeb80 20 client.14982 populate_metadata read hostname 'smithi002' 2024-05-14T16:53:54.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.340+0000 7f4ecacaeb80 1 --2- 172.21.15.2:0/3358608298 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564a6d1cfcf0 0x564a6d1cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:54.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.340+0000 7f4ecacaeb80 4 client.14982 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T16:53:54.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.340+0000 7f4ecacaeb80 1 -- 172.21.15.2:0/3358608298 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "status"}) -- 0x564a6d0a0450 con 0x564a6d1cfcf0 2024-05-14T16:53:54.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.340+0000 7f4ebd9b3700 1 --2- 172.21.15.2:0/3358608298 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564a6d1cfcf0 0x564a6d1cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:54.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.340+0000 7f4ebd9b3700 1 --2- 172.21.15.2:0/3358608298 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564a6d1cfcf0 0x564a6d1cea00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb33d5bd835f84ac server_cookie=1a52dd938356e3ca in_seq=0 out_seq=0 2024-05-14T16:53:54.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.340+0000 7f4ea1ffb700 10 client.14982 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:53:54.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.341+0000 7f4ea1ffb700 1 -- 172.21.15.2:0/3358608298 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x564a6d0a0450 con 0x564a6d1cfcf0 2024-05-14T16:53:54.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.341+0000 7f4ea1ffb700 10 client.14982 handle_command_reply: tid=0 2024-05-14T16:53:54.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.341+0000 7f4ecacaeb80 1 client.14982 shutdown 2024-05-14T16:53:54.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.341+0000 7f4ecacaeb80 1 -- 172.21.15.2:0/3358608298 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a6d146dc0 msgr2=0x564a6d1401a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:54.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.341+0000 7f4ecacaeb80 1 --2- 172.21.15.2:0/3358608298 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a6d146dc0 0x564a6d1401a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4eb40dfc10 tx=0x7f4eb400fbf0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.341+0000 7f4ecacaeb80 1 -- 172.21.15.2:0/3358608298 shutdown_connections 2024-05-14T16:53:54.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.341+0000 7f4ecacaeb80 1 --2- 172.21.15.2:0/3358608298 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564a6d1cfcf0 0x564a6d1cea00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.341+0000 7f4ecacaeb80 1 --2- 172.21.15.2:0/3358608298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a6d14c8c0 0x564a6d140c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.341+0000 7f4ecacaeb80 1 --2- 172.21.15.2:0/3358608298 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564a6d146dc0 0x564a6d1401a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.341+0000 7f4ecacaeb80 1 --2- 172.21.15.2:0/3358608298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564a6d14bfa0 0x564a6d1406e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.341+0000 7f4ecacaeb80 1 -- 172.21.15.2:0/3358608298 >> 172.21.15.2:0/3358608298 conn(0x564a6d073d60 msgr2=0x564a6d14ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:54.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.341+0000 7f4ea1ffb700 0 client.14982 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:53:54.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.341+0000 7f4ecacaeb80 1 -- 172.21.15.2:0/3358608298 shutdown_connections 2024-05-14T16:53:54.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.341+0000 7f4ecacaeb80 1 -- 172.21.15.2:0/3358608298 wait complete. 2024-05-14T16:53:54.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.341+0000 7f4ecacaeb80 20 client.14982 trim_cache size 0 max 16384 2024-05-14T16:53:54.348 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T16:53:54.348 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "ed40717a-1210-11ef-bc98-c7b262605968", 2024-05-14T16:53:54.348 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 0, 2024-05-14T16:53:54.348 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24479, 2024-05-14T16:53:54.348 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2024-05-14T16:53:54.348 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2024-05-14T16:53:54.349 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2024-05-14T16:53:54.349 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 140.050036205, 2024-05-14T16:53:54.349 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 24, 2024-05-14T16:53:54.349 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 80, 2024-05-14T16:53:54.349 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 78, 2024-05-14T16:53:54.349 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 155.90124331199999 2024-05-14T16:53:54.349 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T16:53:54.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.342+0000 7f4ec0418700 1 -- 172.21.15.2:0/1719688130 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f4ea40b2880 msgr2=0x7f4ea40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:54.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.342+0000 7f4ec0418700 1 --2- 172.21.15.2:0/1719688130 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f4ea40b2880 0x7f4ea40b4d40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8009990 tx=0x7f4ea8000bc0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.342+0000 7f4ec0418700 1 -- 172.21.15.2:0/1719688130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4eb81037f0 msgr2=0x7f4eb819bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:54.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.342+0000 7f4ec0418700 1 --2- 172.21.15.2:0/1719688130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4eb81037f0 0x7f4eb819bd20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4eb400c9b0 tx=0x7f4eb400ccc0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.342+0000 7f4ec0418700 1 -- 172.21.15.2:0/1719688130 shutdown_connections 2024-05-14T16:53:54.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.342+0000 7f4ec0418700 1 --2- 172.21.15.2:0/1719688130 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f4ea40b2880 0x7f4ea40b4d40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.342+0000 7f4ec0418700 1 --2- 172.21.15.2:0/1719688130 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4eb81088f0 0x7f4eb81a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.342+0000 7f4ec0418700 1 --2- 172.21.15.2:0/1719688130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4eb81037f0 0x7f4eb819bd20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.342+0000 7f4ec0418700 1 --2- 172.21.15.2:0/1719688130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4eb81041a0 0x7f4eb819c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:54.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.342+0000 7f4ec0418700 1 -- 172.21.15.2:0/1719688130 >> 172.21.15.2:0/1719688130 conn(0x7f4eb807a6a0 msgr2=0x7f4eb8101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:54.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.342+0000 7f4ec0418700 1 -- 172.21.15.2:0/1719688130 shutdown_connections 2024-05-14T16:53:54.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:54.342+0000 7f4ec0418700 1 -- 172.21.15.2:0/1719688130 wait complete. 2024-05-14T16:53:55.122 INFO:tasks.cephadm:Setup complete, yielding 2024-05-14T16:53:55.122 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-14T16:53:55.272 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi002.front.sepia.ceph.com 2024-05-14T16:53:55.272 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- bash -c 'ceph orch status' 2024-05-14T16:53:55.534 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:53:55.656 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:55 smithi002 ceph-mon[31538]: pgmap v361: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:53:55.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:55 smithi151 ceph-mon[36066]: pgmap v361: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:53:55.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:55 smithi123 ceph-mon[35971]: pgmap v361: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:53:57.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.170+0000 7f68e4d0e700 1 -- 172.21.15.2:0/2414446126 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68e0106960 msgr2=0x7f68e0108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:57.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.170+0000 7f68e4d0e700 1 --2- 172.21.15.2:0/2414446126 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68e0106960 0x7f68e0108d50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f68d4009990 tx=0x7f68d4009ca0 comp rx=0 tx=0).stop 2024-05-14T16:53:57.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.170+0000 7f68e4d0e700 1 -- 172.21.15.2:0/2414446126 shutdown_connections 2024-05-14T16:53:57.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.170+0000 7f68e4d0e700 1 --2- 172.21.15.2:0/2414446126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f68e0109290 0x7f68e010b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:57.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.170+0000 7f68e4d0e700 1 --2- 172.21.15.2:0/2414446126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68e0104030 0x7f68e0106420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:57.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.170+0000 7f68e4d0e700 1 --2- 172.21.15.2:0/2414446126 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68e0106960 0x7f68e0108d50 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:57.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.170+0000 7f68e4d0e700 1 -- 172.21.15.2:0/2414446126 >> 172.21.15.2:0/2414446126 conn(0x7f68e00fda40 msgr2=0x7f68e00ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:57.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.170+0000 7f68e4d0e700 1 -- 172.21.15.2:0/2414446126 shutdown_connections 2024-05-14T16:53:57.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.170+0000 7f68e4d0e700 1 -- 172.21.15.2:0/2414446126 wait complete. 2024-05-14T16:53:57.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.171+0000 7f68e4d0e700 1 Processor -- start 2024-05-14T16:53:57.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.171+0000 7f68e4d0e700 1 -- start start 2024-05-14T16:53:57.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.171+0000 7f68e4d0e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68e0104030 0x7f68e019b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:57.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.171+0000 7f68e4d0e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f68e0106960 0x7f68e019bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:57.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.171+0000 7f68e4d0e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68e0109290 0x7f68e01a2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:57.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.171+0000 7f68e4d0e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f68e010f060 con 0x7f68e0104030 2024-05-14T16:53:57.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.171+0000 7f68e4d0e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f68e010eee0 con 0x7f68e0109290 2024-05-14T16:53:57.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.171+0000 7f68e4d0e700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f68e010f1e0 con 0x7f68e0106960 2024-05-14T16:53:57.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.171+0000 7f68de59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68e0104030 0x7f68e019b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:57.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.171+0000 7f68de59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68e0104030 0x7f68e019b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44044/0 (socket says 172.21.15.2:44044) 2024-05-14T16:53:57.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.171+0000 7f68de59c700 1 -- 172.21.15.2:0/912998076 learned_addr learned my addr 172.21.15.2:0/912998076 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:53:57.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.171+0000 7f68ded9d700 1 --2- 172.21.15.2:0/912998076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68e0109290 0x7f68e01a2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:57.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.171+0000 7f68ddd9b700 1 --2- 172.21.15.2:0/912998076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f68e0106960 0x7f68e019bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:57.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.172+0000 7f68de59c700 1 -- 172.21.15.2:0/912998076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f68e0106960 msgr2=0x7f68e019bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:57.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.172+0000 7f68de59c700 1 --2- 172.21.15.2:0/912998076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f68e0106960 0x7f68e019bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:57.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.172+0000 7f68de59c700 1 -- 172.21.15.2:0/912998076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68e0109290 msgr2=0x7f68e01a2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:57.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.172+0000 7f68de59c700 1 --2- 172.21.15.2:0/912998076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68e0109290 0x7f68e01a2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:57.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.172+0000 7f68de59c700 1 -- 172.21.15.2:0/912998076 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e01a34e0 con 0x7f68e0104030 2024-05-14T16:53:57.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.172+0000 7f68ded9d700 1 --2- 172.21.15.2:0/912998076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68e0109290 0x7f68e01a2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:53:57.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.172+0000 7f68ddd9b700 1 --2- 172.21.15.2:0/912998076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f68e0106960 0x7f68e019bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:53:57.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.172+0000 7f68de59c700 1 --2- 172.21.15.2:0/912998076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68e0104030 0x7f68e019b880 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f68d0007e50 tx=0x7f68d000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:57.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.172+0000 7f68cf7fe700 1 -- 172.21.15.2:0/912998076 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d0016020 con 0x7f68e0104030 2024-05-14T16:53:57.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.172+0000 7f68cf7fe700 1 -- 172.21.15.2:0/912998076 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68d000adc0 con 0x7f68e0104030 2024-05-14T16:53:57.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.172+0000 7f68e4d0e700 1 -- 172.21.15.2:0/912998076 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68e01a37d0 con 0x7f68e0104030 2024-05-14T16:53:57.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.172+0000 7f68cf7fe700 1 -- 172.21.15.2:0/912998076 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d0011700 con 0x7f68e0104030 2024-05-14T16:53:57.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.172+0000 7f68e4d0e700 1 -- 172.21.15.2:0/912998076 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68e01a3d60 con 0x7f68e0104030 2024-05-14T16:53:57.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.173+0000 7f68e4d0e700 1 -- 172.21.15.2:0/912998076 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68e006a230 con 0x7f68e0104030 2024-05-14T16:53:57.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.175+0000 7f68cf7fe700 1 -- 172.21.15.2:0/912998076 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f68d0019020 con 0x7f68e0104030 2024-05-14T16:53:57.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.176+0000 7f68cf7fe700 1 --2- 172.21.15.2:0/912998076 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f68c80b2900 0x7f68c80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:53:57.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.176+0000 7f68ddd9b700 1 --2- 172.21.15.2:0/912998076 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f68c80b2900 0x7f68c80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:53:57.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.176+0000 7f68cf7fe700 1 -- 172.21.15.2:0/912998076 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f68d00e23f0 con 0x7f68e0104030 2024-05-14T16:53:57.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.177+0000 7f68ddd9b700 1 --2- 172.21.15.2:0/912998076 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f68c80b2900 0x7f68c80b4dc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f68d400b5c0 tx=0x7f68d4005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:53:57.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.180+0000 7f68cf7fe700 1 -- 172.21.15.2:0/912998076 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f68d0020070 con 0x7f68e0104030 2024-05-14T16:53:57.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.417+0000 7f68e4d0e700 1 -- 172.21.15.2:0/912998076 --> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f68e0063980 con 0x7f68c80b2900 2024-05-14T16:53:57.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.418+0000 7f68cf7fe700 1 -- 172.21.15.2:0/912998076 <== mgr.14150 v2:172.21.15.2:6800/1498936028 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f68e0063980 con 0x7f68c80b2900 2024-05-14T16:53:57.418 INFO:teuthology.orchestra.run.smithi002.stdout:Backend: cephadm 2024-05-14T16:53:57.418 INFO:teuthology.orchestra.run.smithi002.stdout:Available: Yes 2024-05-14T16:53:57.418 INFO:teuthology.orchestra.run.smithi002.stdout:Paused: No 2024-05-14T16:53:57.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.421+0000 7f68e4d0e700 1 -- 172.21.15.2:0/912998076 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f68c80b2900 msgr2=0x7f68c80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:57.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.421+0000 7f68e4d0e700 1 --2- 172.21.15.2:0/912998076 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f68c80b2900 0x7f68c80b4dc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f68d400b5c0 tx=0x7f68d4005fd0 comp rx=0 tx=0).stop 2024-05-14T16:53:57.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.421+0000 7f68e4d0e700 1 -- 172.21.15.2:0/912998076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68e0104030 msgr2=0x7f68e019b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:53:57.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.421+0000 7f68e4d0e700 1 --2- 172.21.15.2:0/912998076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68e0104030 0x7f68e019b880 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f68d0007e50 tx=0x7f68d000a340 comp rx=0 tx=0).stop 2024-05-14T16:53:57.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.421+0000 7f68e4d0e700 1 -- 172.21.15.2:0/912998076 shutdown_connections 2024-05-14T16:53:57.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.421+0000 7f68e4d0e700 1 --2- 172.21.15.2:0/912998076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68e0109290 0x7f68e01a2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:57.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.421+0000 7f68e4d0e700 1 --2- 172.21.15.2:0/912998076 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f68c80b2900 0x7f68c80b4dc0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:57.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.421+0000 7f68e4d0e700 1 --2- 172.21.15.2:0/912998076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f68e0104030 0x7f68e019b880 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:57.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.421+0000 7f68e4d0e700 1 --2- 172.21.15.2:0/912998076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f68e0106960 0x7f68e019bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:53:57.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.421+0000 7f68e4d0e700 1 -- 172.21.15.2:0/912998076 >> 172.21.15.2:0/912998076 conn(0x7f68e00fda40 msgr2=0x7f68e01036d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:53:57.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.421+0000 7f68e4d0e700 1 -- 172.21.15.2:0/912998076 shutdown_connections 2024-05-14T16:53:57.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:53:57.421+0000 7f68e4d0e700 1 -- 172.21.15.2:0/912998076 wait complete. 2024-05-14T16:53:57.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:57 smithi002 ceph-mon[31538]: pgmap v362: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:53:57.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:57 smithi151 ceph-mon[36066]: pgmap v362: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:53:57.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:57 smithi123 ceph-mon[35971]: pgmap v362: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:53:58.125 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-14T16:53:58.383 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:53:58.701 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:58 smithi002 ceph-mon[31538]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:53:58.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:58 smithi151 ceph-mon[36066]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:53:58.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:58 smithi123 ceph-mon[35971]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:53:59.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:53:59 smithi002 ceph-mon[31538]: pgmap v363: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T16:53:59.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:53:59 smithi151 ceph-mon[36066]: pgmap v363: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T16:53:59.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:53:59 smithi123 ceph-mon[35971]: pgmap v363: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T16:54:00.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.030+0000 7f8a88292700 1 -- 172.21.15.2:0/3390901601 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a801069c0 msgr2=0x7f8a80108db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:00.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.030+0000 7f8a88292700 1 --2- 172.21.15.2:0/3390901601 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a801069c0 0x7f8a80108db0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8a70009990 tx=0x7f8a70009ca0 comp rx=0 tx=0).stop 2024-05-14T16:54:00.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.030+0000 7f8a88292700 1 -- 172.21.15.2:0/3390901601 shutdown_connections 2024-05-14T16:54:00.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.030+0000 7f8a88292700 1 --2- 172.21.15.2:0/3390901601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a801092f0 0x7f8a8010b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:00.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.030+0000 7f8a88292700 1 --2- 172.21.15.2:0/3390901601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a80104090 0x7f8a80106480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:00.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.030+0000 7f8a88292700 1 --2- 172.21.15.2:0/3390901601 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a801069c0 0x7f8a80108db0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:00.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.030+0000 7f8a88292700 1 -- 172.21.15.2:0/3390901601 >> 172.21.15.2:0/3390901601 conn(0x7f8a800fda40 msgr2=0x7f8a800ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:00.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.030+0000 7f8a88292700 1 -- 172.21.15.2:0/3390901601 shutdown_connections 2024-05-14T16:54:00.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.030+0000 7f8a88292700 1 -- 172.21.15.2:0/3390901601 wait complete. 2024-05-14T16:54:00.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.031+0000 7f8a88292700 1 Processor -- start 2024-05-14T16:54:00.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.031+0000 7f8a88292700 1 -- start start 2024-05-14T16:54:00.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.031+0000 7f8a88292700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a80104090 0x7f8a801996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:00.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.031+0000 7f8a88292700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a801069c0 0x7f8a80199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:00.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.031+0000 7f8a88292700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a801092f0 0x7f8a801a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:00.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.031+0000 7f8a88292700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8010f020 con 0x7f8a801092f0 2024-05-14T16:54:00.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.031+0000 7f8a88292700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8010eea0 con 0x7f8a80104090 2024-05-14T16:54:00.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.031+0000 7f8a88292700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8010f1a0 con 0x7f8a801069c0 2024-05-14T16:54:00.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.031+0000 7f8a8682f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a801092f0 0x7f8a801a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:00.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.031+0000 7f8a8602e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a80104090 0x7f8a801996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:00.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.031+0000 7f8a8602e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a80104090 0x7f8a801996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:36224/0 (socket says 172.21.15.2:36224) 2024-05-14T16:54:00.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.031+0000 7f8a8682f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a801092f0 0x7f8a801a0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44060/0 (socket says 172.21.15.2:44060) 2024-05-14T16:54:00.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.031+0000 7f8a8602e700 1 -- 172.21.15.2:0/2090788661 learned_addr learned my addr 172.21.15.2:0/2090788661 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:00.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.032+0000 7f8a8582d700 1 --2- 172.21.15.2:0/2090788661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a801069c0 0x7f8a80199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:00.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.032+0000 7f8a8682f700 1 -- 172.21.15.2:0/2090788661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a801069c0 msgr2=0x7f8a80199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:00.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.032+0000 7f8a8682f700 1 --2- 172.21.15.2:0/2090788661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a801069c0 0x7f8a80199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:00.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.032+0000 7f8a8682f700 1 -- 172.21.15.2:0/2090788661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a80104090 msgr2=0x7f8a801996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:00.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.032+0000 7f8a8682f700 1 --2- 172.21.15.2:0/2090788661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a80104090 0x7f8a801996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:00.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.032+0000 7f8a8682f700 1 -- 172.21.15.2:0/2090788661 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a801a1300 con 0x7f8a801092f0 2024-05-14T16:54:00.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.032+0000 7f8a8602e700 1 --2- 172.21.15.2:0/2090788661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a80104090 0x7f8a801996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:54:00.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.032+0000 7f8a8682f700 1 --2- 172.21.15.2:0/2090788661 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a801092f0 0x7f8a801a0c60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8a78007f10 tx=0x7f8a7800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:00.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.032+0000 7f8a777fe700 1 -- 172.21.15.2:0/2090788661 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a78004db0 con 0x7f8a801092f0 2024-05-14T16:54:00.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.032+0000 7f8a88292700 1 -- 172.21.15.2:0/2090788661 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a801a15f0 con 0x7f8a801092f0 2024-05-14T16:54:00.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.032+0000 7f8a777fe700 1 -- 172.21.15.2:0/2090788661 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a78009e20 con 0x7f8a801092f0 2024-05-14T16:54:00.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.032+0000 7f8a777fe700 1 -- 172.21.15.2:0/2090788661 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a78015400 con 0x7f8a801092f0 2024-05-14T16:54:00.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.032+0000 7f8a88292700 1 -- 172.21.15.2:0/2090788661 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a801a1b80 con 0x7f8a801092f0 2024-05-14T16:54:00.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.033+0000 7f8a88292700 1 -- 172.21.15.2:0/2090788661 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a8006a230 con 0x7f8a801092f0 2024-05-14T16:54:00.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.035+0000 7f8a777fe700 1 -- 172.21.15.2:0/2090788661 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f8a780051d0 con 0x7f8a801092f0 2024-05-14T16:54:00.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.035+0000 7f8a777fe700 1 --2- 172.21.15.2:0/2090788661 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f8a6c0b2880 0x7f8a6c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:00.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.036+0000 7f8a8602e700 1 --2- 172.21.15.2:0/2090788661 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f8a6c0b2880 0x7f8a6c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:00.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.036+0000 7f8a777fe700 1 -- 172.21.15.2:0/2090788661 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f8a780e22c0 con 0x7f8a801092f0 2024-05-14T16:54:00.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.036+0000 7f8a8602e700 1 --2- 172.21.15.2:0/2090788661 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f8a6c0b2880 0x7f8a6c0b4d40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c0104e0 tx=0x7f8a7c007480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:00.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.039+0000 7f8a777fe700 1 -- 172.21.15.2:0/2090788661 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8a78010070 con 0x7f8a801092f0 2024-05-14T16:54:00.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.264+0000 7f8a88292700 1 -- 172.21.15.2:0/2090788661 --> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f8a80063980 con 0x7f8a6c0b2880 2024-05-14T16:54:00.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.278+0000 7f8a777fe700 1 -- 172.21.15.2:0/2090788661 <== mgr.14150 v2:172.21.15.2:6800/1498936028 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f8a80063980 con 0x7f8a6c0b2880 2024-05-14T16:54:00.278 INFO:teuthology.orchestra.run.smithi002.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-14T16:54:00.278 INFO:teuthology.orchestra.run.smithi002.stdout:mds.a smithi002 running (2m) 23s ago 2m 16.1M - 19.0.0-3687-g9b638668 c922e207469e b0216c3abe81 2024-05-14T16:54:00.278 INFO:teuthology.orchestra.run.smithi002.stdout:mds.b smithi123 running (2m) 25s ago 2m 29.7M - 19.0.0-3687-g9b638668 c922e207469e 0cf46c0bdaab 2024-05-14T16:54:00.278 INFO:teuthology.orchestra.run.smithi002.stdout:mds.c smithi151 running (2m) 25s ago 2m 21.0M - 19.0.0-3687-g9b638668 c922e207469e 0371af3365c2 2024-05-14T16:54:00.278 INFO:teuthology.orchestra.run.smithi002.stdout:mds.d smithi002 running (3m) 23s ago 3m 17.3M - 19.0.0-3687-g9b638668 c922e207469e edf605f2baaa 2024-05-14T16:54:00.278 INFO:teuthology.orchestra.run.smithi002.stdout:mds.e smithi123 running (3m) 25s ago 3m 21.3M - 19.0.0-3687-g9b638668 c922e207469e 1b372958c4df 2024-05-14T16:54:00.278 INFO:teuthology.orchestra.run.smithi002.stdout:mds.f smithi151 running (3m) 25s ago 3m 19.2M - 19.0.0-3687-g9b638668 c922e207469e 681def580df5 2024-05-14T16:54:00.278 INFO:teuthology.orchestra.run.smithi002.stdout:mds.g smithi002 running (2m) 23s ago 2m 17.0M - 19.0.0-3687-g9b638668 c922e207469e 441f07e027cc 2024-05-14T16:54:00.278 INFO:teuthology.orchestra.run.smithi002.stdout:mds.h smithi123 running (2m) 25s ago 2m 22.7M - 19.0.0-3687-g9b638668 c922e207469e 14bb6f7ef1ce 2024-05-14T16:54:00.279 INFO:teuthology.orchestra.run.smithi002.stdout:mds.i smithi151 running (2m) 25s ago 2m 20.5M - 19.0.0-3687-g9b638668 c922e207469e d2272915ee3c 2024-05-14T16:54:00.279 INFO:teuthology.orchestra.run.smithi002.stdout:mds.j smithi002 running (3m) 23s ago 3m 14.8M - 19.0.0-3687-g9b638668 c922e207469e 327406706a1b 2024-05-14T16:54:00.279 INFO:teuthology.orchestra.run.smithi002.stdout:mds.k smithi123 running (3m) 25s ago 3m 14.6M - 19.0.0-3687-g9b638668 c922e207469e 34721ed81fd7 2024-05-14T16:54:00.279 INFO:teuthology.orchestra.run.smithi002.stdout:mds.l smithi151 running (3m) 25s ago 3m 14.0M - 19.0.0-3687-g9b638668 c922e207469e d132161610f9 2024-05-14T16:54:00.279 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.x smithi002 *:9283,8765 running (10m) 23s ago 10m 531M - 19.0.0-3687-g9b638668 c922e207469e 619f0e34026d 2024-05-14T16:54:00.279 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.y smithi123 *:8443,8765 running (8m) 25s ago 8m 467M - 19.0.0-3687-g9b638668 c922e207469e f4debe38cf60 2024-05-14T16:54:00.279 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.z smithi151 *:8443,8765 running (8m) 25s ago 8m 470M - 19.0.0-3687-g9b638668 c922e207469e dda976e47705 2024-05-14T16:54:00.279 INFO:teuthology.orchestra.run.smithi002.stdout:mon.a smithi002 running (10m) 23s ago 10m 54.4M 2048M 19.0.0-3687-g9b638668 c922e207469e c3b1d2255195 2024-05-14T16:54:00.279 INFO:teuthology.orchestra.run.smithi002.stdout:mon.b smithi123 running (8m) 25s ago 8m 43.8M 2048M 19.0.0-3687-g9b638668 c922e207469e b249d40e5d46 2024-05-14T16:54:00.279 INFO:teuthology.orchestra.run.smithi002.stdout:mon.c smithi151 running (9m) 25s ago 9m 41.5M 2048M 19.0.0-3687-g9b638668 c922e207469e 3e252dc6012f 2024-05-14T16:54:00.279 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 smithi002 running (8m) 23s ago 8m 56.3M 4273M 19.0.0-3687-g9b638668 c922e207469e 5e05179ffa24 2024-05-14T16:54:00.279 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1 smithi123 running (7m) 25s ago 7m 60.5M 4272M 19.0.0-3687-g9b638668 c922e207469e 874f71783269 2024-05-14T16:54:00.279 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2 smithi151 running (7m) 25s ago 7m 57.1M 4272M 19.0.0-3687-g9b638668 c922e207469e cd1945556869 2024-05-14T16:54:00.279 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 smithi002 running (7m) 23s ago 7m 56.1M 4273M 19.0.0-3687-g9b638668 c922e207469e df0c61db7c5f 2024-05-14T16:54:00.279 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4 smithi123 running (6m) 25s ago 6m 55.6M 4272M 19.0.0-3687-g9b638668 c922e207469e 5a66038a280a 2024-05-14T16:54:00.279 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 smithi151 running (6m) 25s ago 6m 57.0M 4272M 19.0.0-3687-g9b638668 c922e207469e 8175c9a84326 2024-05-14T16:54:00.280 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6 smithi002 running (5m) 23s ago 5m 53.0M 4273M 19.0.0-3687-g9b638668 c922e207469e 81f7195a86f0 2024-05-14T16:54:00.280 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7 smithi123 running (5m) 25s ago 5m 55.5M 4272M 19.0.0-3687-g9b638668 c922e207469e 375d9d2ed387 2024-05-14T16:54:00.280 INFO:teuthology.orchestra.run.smithi002.stdout:osd.8 smithi151 running (4m) 25s ago 4m 43.5M 4272M 19.0.0-3687-g9b638668 c922e207469e 83888fbd4ff8 2024-05-14T16:54:00.280 INFO:teuthology.orchestra.run.smithi002.stdout:osd.9 smithi002 running (4m) 23s ago 4m 42.4M 4273M 19.0.0-3687-g9b638668 c922e207469e 0e2866076f28 2024-05-14T16:54:00.280 INFO:teuthology.orchestra.run.smithi002.stdout:osd.10 smithi123 running (4m) 25s ago 4m 57.5M 4272M 19.0.0-3687-g9b638668 c922e207469e 51861c521ec9 2024-05-14T16:54:00.280 INFO:teuthology.orchestra.run.smithi002.stdout:osd.11 smithi151 running (3m) 25s ago 3m 45.7M 4272M 19.0.0-3687-g9b638668 c922e207469e 6c1e96c9e205 2024-05-14T16:54:00.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.280+0000 7f8a88292700 1 -- 172.21.15.2:0/2090788661 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f8a6c0b2880 msgr2=0x7f8a6c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:00.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.280+0000 7f8a88292700 1 --2- 172.21.15.2:0/2090788661 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f8a6c0b2880 0x7f8a6c0b4d40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c0104e0 tx=0x7f8a7c007480 comp rx=0 tx=0).stop 2024-05-14T16:54:00.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.280+0000 7f8a88292700 1 -- 172.21.15.2:0/2090788661 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a801092f0 msgr2=0x7f8a801a0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:00.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.280+0000 7f8a88292700 1 --2- 172.21.15.2:0/2090788661 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a801092f0 0x7f8a801a0c60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8a78007f10 tx=0x7f8a7800d3b0 comp rx=0 tx=0).stop 2024-05-14T16:54:00.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.280+0000 7f8a88292700 1 -- 172.21.15.2:0/2090788661 shutdown_connections 2024-05-14T16:54:00.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.280+0000 7f8a88292700 1 --2- 172.21.15.2:0/2090788661 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f8a6c0b2880 0x7f8a6c0b4d40 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:00.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.280+0000 7f8a88292700 1 --2- 172.21.15.2:0/2090788661 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a801092f0 0x7f8a801a0c60 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:00.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.280+0000 7f8a88292700 1 --2- 172.21.15.2:0/2090788661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a80104090 0x7f8a801996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:00.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.280+0000 7f8a88292700 1 --2- 172.21.15.2:0/2090788661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a801069c0 0x7f8a80199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:00.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.280+0000 7f8a88292700 1 -- 172.21.15.2:0/2090788661 >> 172.21.15.2:0/2090788661 conn(0x7f8a800fda40 msgr2=0x7f8a80104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:00.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.281+0000 7f8a88292700 1 -- 172.21.15.2:0/2090788661 shutdown_connections 2024-05-14T16:54:00.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:00.281+0000 7f8a88292700 1 -- 172.21.15.2:0/2090788661 wait complete. 2024-05-14T16:54:00.952 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-14T16:54:01.202 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:54:01.659 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:01 smithi002 ceph-mon[31538]: pgmap v364: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T16:54:01.659 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:01 smithi002 ceph-mon[31538]: from='client.14994 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:54:01.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:01 smithi151 ceph-mon[36066]: pgmap v364: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T16:54:01.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:01 smithi151 ceph-mon[36066]: from='client.14994 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:54:01.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:01 smithi123 ceph-mon[35971]: pgmap v364: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T16:54:01.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:01 smithi123 ceph-mon[35971]: from='client.14994 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:54:02.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:02.842+0000 7f57d0673700 1 -- 172.21.15.2:0/920007842 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f57c81035a0 msgr2=0x7f57c81039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:02.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:02.842+0000 7f57d0673700 1 --2- 172.21.15.2:0/920007842 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f57c81035a0 0x7f57c81039c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f57c40099c0 tx=0x7f57c4009cd0 comp rx=0 tx=0).stop 2024-05-14T16:54:02.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:02.843+0000 7f57d0673700 1 -- 172.21.15.2:0/920007842 shutdown_connections 2024-05-14T16:54:02.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:02.843+0000 7f57d0673700 1 --2- 172.21.15.2:0/920007842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57c8105140 0x7f57c810b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:02.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:02.843+0000 7f57d0673700 1 --2- 172.21.15.2:0/920007842 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f57c81035a0 0x7f57c81039c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:02.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:02.843+0000 7f57d0673700 1 --2- 172.21.15.2:0/920007842 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57c81047a0 0x7f57c8104c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:02.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:02.843+0000 7f57d0673700 1 -- 172.21.15.2:0/920007842 >> 172.21.15.2:0/920007842 conn(0x7f57c80feb40 msgr2=0x7f57c8100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:02.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:02.843+0000 7f57d0673700 1 -- 172.21.15.2:0/920007842 shutdown_connections 2024-05-14T16:54:02.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:02.843+0000 7f57d0673700 1 -- 172.21.15.2:0/920007842 wait complete. 2024-05-14T16:54:02.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:02.843+0000 7f57d0673700 1 Processor -- start 2024-05-14T16:54:02.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:02.843+0000 7f57d0673700 1 -- start start 2024-05-14T16:54:02.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:02.843+0000 7f57d0673700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57c81035a0 0x7f57c8199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:02.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:02.843+0000 7f57d0673700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f57c81047a0 0x7f57c8199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:02.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:02.844+0000 7f57d0673700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57c8105140 0x7f57c81a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:02.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:02.844+0000 7f57d0673700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f57c810f090 con 0x7f57c81047a0 2024-05-14T16:54:02.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:02.844+0000 7f57d0673700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f57c810ef10 con 0x7f57c81035a0 2024-05-14T16:54:02.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:02.844+0000 7f57d0673700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f57c810f210 con 0x7f57c8105140 2024-05-14T16:54:02.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:02.844+0000 7f57cdc0e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f57c81047a0 0x7f57c8199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:02.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:02.844+0000 7f57cdc0e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f57c81047a0 0x7f57c8199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44078/0 (socket says 172.21.15.2:44078) 2024-05-14T16:54:02.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:02.844+0000 7f57cdc0e700 1 -- 172.21.15.2:0/1212702661 learned_addr learned my addr 172.21.15.2:0/1212702661 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:02.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:02.844+0000 7f57ce40f700 1 --2- 172.21.15.2:0/1212702661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57c81035a0 0x7f57c8199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:02.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:02.844+0000 7f57cec10700 1 --2- 172.21.15.2:0/1212702661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57c8105140 0x7f57c81a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:02.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:02.844+0000 7f57cdc0e700 1 -- 172.21.15.2:0/1212702661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57c8105140 msgr2=0x7f57c81a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:02.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:02.844+0000 7f57cdc0e700 1 --2- 172.21.15.2:0/1212702661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57c8105140 0x7f57c81a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:02.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:02.844+0000 7f57cdc0e700 1 -- 172.21.15.2:0/1212702661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57c81035a0 msgr2=0x7f57c8199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:02.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:02.844+0000 7f57cdc0e700 1 --2- 172.21.15.2:0/1212702661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57c81035a0 0x7f57c8199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:02.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:02.844+0000 7f57cdc0e700 1 -- 172.21.15.2:0/1212702661 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57c81a1390 con 0x7f57c81047a0 2024-05-14T16:54:02.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:02.844+0000 7f57cdc0e700 1 --2- 172.21.15.2:0/1212702661 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f57c81047a0 0x7f57c8199c70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f57bc00b560 tx=0x7f57bc00b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:02.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:02.844+0000 7f57bb7fe700 1 -- 172.21.15.2:0/1212702661 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57bc015020 con 0x7f57c81047a0 2024-05-14T16:54:02.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:02.844+0000 7f57d0673700 1 -- 172.21.15.2:0/1212702661 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57c81a1680 con 0x7f57c81047a0 2024-05-14T16:54:02.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:02.844+0000 7f57bb7fe700 1 -- 172.21.15.2:0/1212702661 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57bc011ce0 con 0x7f57c81047a0 2024-05-14T16:54:02.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:02.844+0000 7f57bb7fe700 1 -- 172.21.15.2:0/1212702661 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57bc010da0 con 0x7f57c81047a0 2024-05-14T16:54:02.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:02.844+0000 7f57d0673700 1 -- 172.21.15.2:0/1212702661 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57c81a1b70 con 0x7f57c81047a0 2024-05-14T16:54:02.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:02.845+0000 7f57d0673700 1 -- 172.21.15.2:0/1212702661 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57c806a230 con 0x7f57c81047a0 2024-05-14T16:54:02.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:02.847+0000 7f57bb7fe700 1 -- 172.21.15.2:0/1212702661 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f57bc018020 con 0x7f57c81047a0 2024-05-14T16:54:02.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:02.848+0000 7f57bb7fe700 1 --2- 172.21.15.2:0/1212702661 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f57b40b2880 0x7f57b40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:02.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:02.848+0000 7f57ce40f700 1 --2- 172.21.15.2:0/1212702661 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f57b40b2880 0x7f57b40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:02.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:02.848+0000 7f57bb7fe700 1 -- 172.21.15.2:0/1212702661 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f57bc0e2060 con 0x7f57c81047a0 2024-05-14T16:54:02.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:02.848+0000 7f57ce40f700 1 --2- 172.21.15.2:0/1212702661 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f57b40b2880 0x7f57b40b4d40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f57c400b580 tx=0x7f57c4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:02.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:02.851+0000 7f57bb7fe700 1 -- 172.21.15.2:0/1212702661 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f57bc01f020 con 0x7f57c81047a0 2024-05-14T16:54:03.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:03.073+0000 7f57d0673700 1 -- 172.21.15.2:0/1212702661 --> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f57c8063980 con 0x7f57b40b2880 2024-05-14T16:54:03.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:03.079+0000 7f57bb7fe700 1 -- 172.21.15.2:0/1212702661 <== mgr.14150 v2:172.21.15.2:6800/1498936028 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f57c8063980 con 0x7f57b40b2880 2024-05-14T16:54:03.079 INFO:teuthology.orchestra.run.smithi002.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-14T16:54:03.079 INFO:teuthology.orchestra.run.smithi002.stdout:mds.all 12/12 28s ago 3m smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l;count:12 2024-05-14T16:54:03.079 INFO:teuthology.orchestra.run.smithi002.stdout:mgr 3/3 28s ago 8m smithi002=x;smithi123=y;smithi151=z;count:3 2024-05-14T16:54:03.079 INFO:teuthology.orchestra.run.smithi002.stdout:mon 3/3 28s ago 9m smithi002:172.21.15.2=a;smithi123:172.21.15.123=b;smithi151:172.21.15.151=c;count:3 2024-05-14T16:54:03.079 INFO:teuthology.orchestra.run.smithi002.stdout:osd 12 28s ago - 2024-05-14T16:54:03.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:03.081+0000 7f57d0673700 1 -- 172.21.15.2:0/1212702661 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f57b40b2880 msgr2=0x7f57b40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:03.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:03.081+0000 7f57d0673700 1 --2- 172.21.15.2:0/1212702661 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f57b40b2880 0x7f57b40b4d40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f57c400b580 tx=0x7f57c4005fb0 comp rx=0 tx=0).stop 2024-05-14T16:54:03.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:03.081+0000 7f57d0673700 1 -- 172.21.15.2:0/1212702661 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f57c81047a0 msgr2=0x7f57c8199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:03.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:03.081+0000 7f57d0673700 1 --2- 172.21.15.2:0/1212702661 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f57c81047a0 0x7f57c8199c70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f57bc00b560 tx=0x7f57bc00b920 comp rx=0 tx=0).stop 2024-05-14T16:54:03.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:03.081+0000 7f57d0673700 1 -- 172.21.15.2:0/1212702661 shutdown_connections 2024-05-14T16:54:03.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:03.081+0000 7f57d0673700 1 --2- 172.21.15.2:0/1212702661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57c8105140 0x7f57c81a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:03.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:03.081+0000 7f57d0673700 1 --2- 172.21.15.2:0/1212702661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57c81035a0 0x7f57c8199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:03.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:03.081+0000 7f57d0673700 1 --2- 172.21.15.2:0/1212702661 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f57b40b2880 0x7f57b40b4d40 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:03.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:03.081+0000 7f57d0673700 1 --2- 172.21.15.2:0/1212702661 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f57c81047a0 0x7f57c8199c70 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:03.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:03.081+0000 7f57d0673700 1 -- 172.21.15.2:0/1212702661 >> 172.21.15.2:0/1212702661 conn(0x7f57c80feb40 msgr2=0x7f57c8107af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:03.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:03.081+0000 7f57d0673700 1 -- 172.21.15.2:0/1212702661 shutdown_connections 2024-05-14T16:54:03.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:03.081+0000 7f57d0673700 1 -- 172.21.15.2:0/1212702661 wait complete. 2024-05-14T16:54:03.714 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:03 smithi002 ceph-mon[31538]: pgmap v365: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:54:03.749 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-14T16:54:03.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:03 smithi151 ceph-mon[36066]: pgmap v365: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:54:03.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:03 smithi123 ceph-mon[35971]: pgmap v365: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:54:04.006 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:54:04.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:04 smithi002 ceph-mon[31538]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:54:04.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:04 smithi151 ceph-mon[36066]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:54:04.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:04 smithi123 ceph-mon[35971]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:54:05.617 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:05 smithi002 ceph-mon[31538]: pgmap v366: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:54:05.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.727+0000 7fcd1a553700 1 -- 172.21.15.2:0/3460598779 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd14104040 msgr2=0x7fcd1410a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:05.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.727+0000 7fcd1a553700 1 --2- 172.21.15.2:0/3460598779 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd14104040 0x7fcd1410a8f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c00b0d0 tx=0x7fcd0c00b3e0 comp rx=0 tx=0).stop 2024-05-14T16:54:05.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.727+0000 7fcd1a553700 1 -- 172.21.15.2:0/3460598779 shutdown_connections 2024-05-14T16:54:05.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.727+0000 7fcd1a553700 1 --2- 172.21.15.2:0/3460598779 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd14104040 0x7fcd1410a8f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:05.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.727+0000 7fcd1a553700 1 --2- 172.21.15.2:0/3460598779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd141024a0 0x7fcd141028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:05.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.727+0000 7fcd1a553700 1 --2- 172.21.15.2:0/3460598779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd141036a0 0x7fcd14103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:05.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.727+0000 7fcd1a553700 1 -- 172.21.15.2:0/3460598779 >> 172.21.15.2:0/3460598779 conn(0x7fcd140fda40 msgr2=0x7fcd140ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:05.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.727+0000 7fcd1a553700 1 -- 172.21.15.2:0/3460598779 shutdown_connections 2024-05-14T16:54:05.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.727+0000 7fcd1a553700 1 -- 172.21.15.2:0/3460598779 wait complete. 2024-05-14T16:54:05.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.728+0000 7fcd1a553700 1 Processor -- start 2024-05-14T16:54:05.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.728+0000 7fcd1a553700 1 -- start start 2024-05-14T16:54:05.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.728+0000 7fcd1a553700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd141024a0 0x7fcd141985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:05.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.728+0000 7fcd1a553700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd141036a0 0x7fcd14198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:05.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.728+0000 7fcd1a553700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd14104040 0x7fcd1419fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:05.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.728+0000 7fcd1a553700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1410df40 con 0x7fcd141036a0 2024-05-14T16:54:05.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.728+0000 7fcd1a553700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1410ddc0 con 0x7fcd141024a0 2024-05-14T16:54:05.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.728+0000 7fcd1a553700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1410e0c0 con 0x7fcd14104040 2024-05-14T16:54:05.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.728+0000 7fcd137fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd141036a0 0x7fcd14198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:05.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.728+0000 7fcd137fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd141036a0 0x7fcd14198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34978/0 (socket says 172.21.15.2:34978) 2024-05-14T16:54:05.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.728+0000 7fcd137fe700 1 -- 172.21.15.2:0/2897365637 learned_addr learned my addr 172.21.15.2:0/2897365637 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:05.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.728+0000 7fcd13fff700 1 --2- 172.21.15.2:0/2897365637 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd141024a0 0x7fcd141985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:05.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.728+0000 7fcd18af0700 1 --2- 172.21.15.2:0/2897365637 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd14104040 0x7fcd1419fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:05.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.729+0000 7fcd13fff700 1 -- 172.21.15.2:0/2897365637 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd14104040 msgr2=0x7fcd1419fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:05.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.729+0000 7fcd13fff700 1 --2- 172.21.15.2:0/2897365637 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd14104040 0x7fcd1419fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:05.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.729+0000 7fcd13fff700 1 -- 172.21.15.2:0/2897365637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd141036a0 msgr2=0x7fcd14198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:05.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.729+0000 7fcd13fff700 1 --2- 172.21.15.2:0/2897365637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd141036a0 0x7fcd14198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:05.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.729+0000 7fcd13fff700 1 -- 172.21.15.2:0/2897365637 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd141a0210 con 0x7fcd141024a0 2024-05-14T16:54:05.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.729+0000 7fcd137fe700 1 --2- 172.21.15.2:0/2897365637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd141036a0 0x7fcd14198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:54:05.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.729+0000 7fcd13fff700 1 --2- 172.21.15.2:0/2897365637 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd141024a0 0x7fcd141985b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcd0400e9c0 tx=0x7fcd0400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:05.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.729+0000 7fcd117fa700 1 -- 172.21.15.2:0/2897365637 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd04004230 con 0x7fcd141024a0 2024-05-14T16:54:05.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.729+0000 7fcd1a553700 1 -- 172.21.15.2:0/2897365637 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd1406b280 con 0x7fcd141024a0 2024-05-14T16:54:05.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.729+0000 7fcd117fa700 1 -- 172.21.15.2:0/2897365637 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd040043d0 con 0x7fcd141024a0 2024-05-14T16:54:05.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.729+0000 7fcd117fa700 1 -- 172.21.15.2:0/2897365637 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd04003d80 con 0x7fcd141024a0 2024-05-14T16:54:05.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.729+0000 7fcd1a553700 1 -- 172.21.15.2:0/2897365637 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd1406b7c0 con 0x7fcd141024a0 2024-05-14T16:54:05.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.730+0000 7fcd1a553700 1 -- 172.21.15.2:0/2897365637 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd1406a230 con 0x7fcd141024a0 2024-05-14T16:54:05.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.732+0000 7fcd117fa700 1 -- 172.21.15.2:0/2897365637 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fcd0400c7c0 con 0x7fcd141024a0 2024-05-14T16:54:05.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.732+0000 7fcd117fa700 1 --2- 172.21.15.2:0/2897365637 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fccfc0b2880 0x7fccfc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:05.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.732+0000 7fcd137fe700 1 --2- 172.21.15.2:0/2897365637 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fccfc0b2880 0x7fccfc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:05.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.732+0000 7fcd117fa700 1 -- 172.21.15.2:0/2897365637 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7fcd040e2480 con 0x7fcd141024a0 2024-05-14T16:54:05.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.733+0000 7fcd137fe700 1 --2- 172.21.15.2:0/2897365637 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fccfc0b2880 0x7fccfc0b4d40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcd0800e5f0 tx=0x7fcd08010040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:05.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.736+0000 7fcd117fa700 1 -- 172.21.15.2:0/2897365637 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcd040aa150 con 0x7fcd141024a0 2024-05-14T16:54:05.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:05 smithi151 ceph-mon[36066]: pgmap v366: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:54:05.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:05 smithi123 ceph-mon[35971]: pgmap v366: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:54:05.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.961+0000 7fcd1a553700 1 -- 172.21.15.2:0/2897365637 --> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fcd1406ba80 con 0x7fccfc0b2880 2024-05-14T16:54:05.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.962+0000 7fcd117fa700 1 -- 172.21.15.2:0/2897365637 <== mgr.14150 v2:172.21.15.2:6800/1498936028 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fcd1406ba80 con 0x7fccfc0b2880 2024-05-14T16:54:05.962 INFO:teuthology.orchestra.run.smithi002.stdout:HOST ADDR LABELS STATUS 2024-05-14T16:54:05.962 INFO:teuthology.orchestra.run.smithi002.stdout:smithi002 172.21.15.2 2024-05-14T16:54:05.962 INFO:teuthology.orchestra.run.smithi002.stdout:smithi123 172.21.15.123 2024-05-14T16:54:05.962 INFO:teuthology.orchestra.run.smithi002.stdout:smithi151 172.21.15.151 2024-05-14T16:54:05.962 INFO:teuthology.orchestra.run.smithi002.stdout:3 hosts in cluster 2024-05-14T16:54:05.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.965+0000 7fcd1a553700 1 -- 172.21.15.2:0/2897365637 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fccfc0b2880 msgr2=0x7fccfc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:05.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.965+0000 7fcd1a553700 1 --2- 172.21.15.2:0/2897365637 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fccfc0b2880 0x7fccfc0b4d40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcd0800e5f0 tx=0x7fcd08010040 comp rx=0 tx=0).stop 2024-05-14T16:54:05.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.965+0000 7fcd1a553700 1 -- 172.21.15.2:0/2897365637 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd141024a0 msgr2=0x7fcd141985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:05.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.965+0000 7fcd1a553700 1 --2- 172.21.15.2:0/2897365637 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd141024a0 0x7fcd141985b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcd0400e9c0 tx=0x7fcd0400ecd0 comp rx=0 tx=0).stop 2024-05-14T16:54:05.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.965+0000 7fcd1a553700 1 -- 172.21.15.2:0/2897365637 shutdown_connections 2024-05-14T16:54:05.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.965+0000 7fcd1a553700 1 --2- 172.21.15.2:0/2897365637 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd14104040 0x7fcd1419fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:05.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.965+0000 7fcd1a553700 1 --2- 172.21.15.2:0/2897365637 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd141024a0 0x7fcd141985b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:05.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.965+0000 7fcd1a553700 1 --2- 172.21.15.2:0/2897365637 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fccfc0b2880 0x7fccfc0b4d40 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:05.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.965+0000 7fcd1a553700 1 --2- 172.21.15.2:0/2897365637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd141036a0 0x7fcd14198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:05.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.965+0000 7fcd1a553700 1 -- 172.21.15.2:0/2897365637 >> 172.21.15.2:0/2897365637 conn(0x7fcd140fda40 msgr2=0x7fcd141069c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:05.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.965+0000 7fcd1a553700 1 -- 172.21.15.2:0/2897365637 shutdown_connections 2024-05-14T16:54:05.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:05.965+0000 7fcd1a553700 1 -- 172.21.15.2:0/2897365637 wait complete. 2024-05-14T16:54:06.661 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-14T16:54:06.912 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:54:07.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:07 smithi002 ceph-mon[31538]: from='client.24803 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:54:07.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:07 smithi002 ceph-mon[31538]: pgmap v367: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:54:07.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:07 smithi151 ceph-mon[36066]: from='client.24803 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:54:07.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:07 smithi151 ceph-mon[36066]: pgmap v367: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:54:07.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:07 smithi123 ceph-mon[35971]: from='client.24803 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:54:07.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:07 smithi123 ceph-mon[35971]: pgmap v367: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:54:08.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.533+0000 7fd2c9468700 1 -- 172.21.15.2:0/1727068654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2c41035a0 msgr2=0x7fd2c41039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:08.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.533+0000 7fd2c9468700 1 --2- 172.21.15.2:0/1727068654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2c41035a0 0x7fd2c41039c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac0099c0 tx=0x7fd2ac009cd0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.533+0000 7fd2c9468700 1 -- 172.21.15.2:0/1727068654 shutdown_connections 2024-05-14T16:54:08.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.533+0000 7fd2c9468700 1 --2- 172.21.15.2:0/1727068654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2c4105140 0x7fd2c410b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.533+0000 7fd2c9468700 1 --2- 172.21.15.2:0/1727068654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2c41035a0 0x7fd2c41039c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.533+0000 7fd2c9468700 1 --2- 172.21.15.2:0/1727068654 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd2c41047a0 0x7fd2c4104c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.533+0000 7fd2c9468700 1 -- 172.21.15.2:0/1727068654 >> 172.21.15.2:0/1727068654 conn(0x7fd2c40feb40 msgr2=0x7fd2c4100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:08.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.533+0000 7fd2c9468700 1 -- 172.21.15.2:0/1727068654 shutdown_connections 2024-05-14T16:54:08.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.533+0000 7fd2c9468700 1 -- 172.21.15.2:0/1727068654 wait complete. 2024-05-14T16:54:08.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.534+0000 7fd2c9468700 1 Processor -- start 2024-05-14T16:54:08.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.534+0000 7fd2c9468700 1 -- start start 2024-05-14T16:54:08.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.534+0000 7fd2c9468700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2c41035a0 0x7fd2c40772e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:08.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.534+0000 7fd2c9468700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2c41047a0 0x7fd2c4075930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:08.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.534+0000 7fd2c9468700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd2c4105140 0x7fd2c4075e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:08.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.534+0000 7fd2c9468700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c410f020 con 0x7fd2c41047a0 2024-05-14T16:54:08.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.534+0000 7fd2c9468700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c410eea0 con 0x7fd2c41035a0 2024-05-14T16:54:08.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.534+0000 7fd2c9468700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c410f1a0 con 0x7fd2c4105140 2024-05-14T16:54:08.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.534+0000 7fd2c27fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2c41047a0 0x7fd2c4075930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:08.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.534+0000 7fd2c27fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2c41047a0 0x7fd2c4075930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34996/0 (socket says 172.21.15.2:34996) 2024-05-14T16:54:08.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.534+0000 7fd2c27fc700 1 -- 172.21.15.2:0/911676665 learned_addr learned my addr 172.21.15.2:0/911676665 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:08.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.535+0000 7fd2c2ffd700 1 --2- 172.21.15.2:0/911676665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2c41035a0 0x7fd2c40772e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:08.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.535+0000 7fd2c37fe700 1 --2- 172.21.15.2:0/911676665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd2c4105140 0x7fd2c4075e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:08.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.535+0000 7fd2c27fc700 1 -- 172.21.15.2:0/911676665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd2c4105140 msgr2=0x7fd2c4075e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:08.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.535+0000 7fd2c27fc700 1 --2- 172.21.15.2:0/911676665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd2c4105140 0x7fd2c4075e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.535+0000 7fd2c27fc700 1 -- 172.21.15.2:0/911676665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2c41035a0 msgr2=0x7fd2c40772e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:08.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.535+0000 7fd2c27fc700 1 --2- 172.21.15.2:0/911676665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2c41035a0 0x7fd2c40772e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.535+0000 7fd2c27fc700 1 -- 172.21.15.2:0/911676665 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c4076720 con 0x7fd2c41047a0 2024-05-14T16:54:08.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.535+0000 7fd2c37fe700 1 --2- 172.21.15.2:0/911676665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd2c4105140 0x7fd2c4075e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:54:08.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.535+0000 7fd2c27fc700 1 --2- 172.21.15.2:0/911676665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2c41047a0 0x7fd2c4075930 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd2b400e9a0 tx=0x7fd2b400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:08.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.535+0000 7fd2bbfff700 1 -- 172.21.15.2:0/911676665 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b400cbd0 con 0x7fd2c41047a0 2024-05-14T16:54:08.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.535+0000 7fd2bbfff700 1 -- 172.21.15.2:0/911676665 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2b400cd70 con 0x7fd2c41047a0 2024-05-14T16:54:08.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.535+0000 7fd2c9468700 1 -- 172.21.15.2:0/911676665 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2c41a38e0 con 0x7fd2c41047a0 2024-05-14T16:54:08.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.535+0000 7fd2bbfff700 1 -- 172.21.15.2:0/911676665 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b4018680 con 0x7fd2c41047a0 2024-05-14T16:54:08.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.535+0000 7fd2c9468700 1 -- 172.21.15.2:0/911676665 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2c41a3df0 con 0x7fd2c41047a0 2024-05-14T16:54:08.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.536+0000 7fd2c9468700 1 -- 172.21.15.2:0/911676665 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2c406a230 con 0x7fd2c41047a0 2024-05-14T16:54:08.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.538+0000 7fd2bbfff700 1 -- 172.21.15.2:0/911676665 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fd2b4010430 con 0x7fd2c41047a0 2024-05-14T16:54:08.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.539+0000 7fd2bbfff700 1 --2- 172.21.15.2:0/911676665 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fd2b00b2880 0x7fd2b00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:08.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.539+0000 7fd2c2ffd700 1 --2- 172.21.15.2:0/911676665 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fd2b00b2880 0x7fd2b00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:08.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.539+0000 7fd2bbfff700 1 -- 172.21.15.2:0/911676665 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7fd2b40e2980 con 0x7fd2c41047a0 2024-05-14T16:54:08.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.539+0000 7fd2c2ffd700 1 --2- 172.21.15.2:0/911676665 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fd2b00b2880 0x7fd2b00b4d40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac00b580 tx=0x7fd2ac005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:08.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.543+0000 7fd2bbfff700 1 -- 172.21.15.2:0/911676665 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd2b40aa6b0 con 0x7fd2c41047a0 2024-05-14T16:54:08.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.765+0000 7fd2c9468700 1 -- 172.21.15.2:0/911676665 --> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fd2c402e9b0 con 0x7fd2b00b2880 2024-05-14T16:54:08.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.766+0000 7fd2bbfff700 1 -- 172.21.15.2:0/911676665 <== mgr.14150 v2:172.21.15.2:6800/1498936028 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fd2c402e9b0 con 0x7fd2b00b2880 2024-05-14T16:54:08.767 INFO:teuthology.orchestra.run.smithi002.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-14T16:54:08.767 INFO:teuthology.orchestra.run.smithi002.stdout:smithi002 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300EV480DGN 447G No 4m ago Has a FileSystem, LVM detected 2024-05-14T16:54:08.767 INFO:teuthology.orchestra.run.smithi002.stdout:smithi123 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400YK400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-05-14T16:54:08.767 INFO:teuthology.orchestra.run.smithi002.stdout:smithi151 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300JJ400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-05-14T16:54:08.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.769+0000 7fd2c9468700 1 -- 172.21.15.2:0/911676665 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fd2b00b2880 msgr2=0x7fd2b00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:08.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.769+0000 7fd2c9468700 1 --2- 172.21.15.2:0/911676665 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fd2b00b2880 0x7fd2b00b4d40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac00b580 tx=0x7fd2ac005fb0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.769+0000 7fd2c9468700 1 -- 172.21.15.2:0/911676665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2c41047a0 msgr2=0x7fd2c4075930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:08.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.769+0000 7fd2c9468700 1 --2- 172.21.15.2:0/911676665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2c41047a0 0x7fd2c4075930 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd2b400e9a0 tx=0x7fd2b400ecb0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.769+0000 7fd2c9468700 1 -- 172.21.15.2:0/911676665 shutdown_connections 2024-05-14T16:54:08.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.769+0000 7fd2c9468700 1 --2- 172.21.15.2:0/911676665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd2c4105140 0x7fd2c4075e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.769+0000 7fd2c9468700 1 --2- 172.21.15.2:0/911676665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2c41035a0 0x7fd2c40772e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.769+0000 7fd2c9468700 1 --2- 172.21.15.2:0/911676665 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fd2b00b2880 0x7fd2b00b4d40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.769+0000 7fd2c9468700 1 --2- 172.21.15.2:0/911676665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2c41047a0 0x7fd2c4075930 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:08.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.769+0000 7fd2c9468700 1 -- 172.21.15.2:0/911676665 >> 172.21.15.2:0/911676665 conn(0x7fd2c40feb40 msgr2=0x7fd2c4107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:08.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.769+0000 7fd2c9468700 1 -- 172.21.15.2:0/911676665 shutdown_connections 2024-05-14T16:54:08.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:08.769+0000 7fd2c9468700 1 -- 172.21.15.2:0/911676665 wait complete. 2024-05-14T16:54:09.495 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-14T16:54:09.504 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi002.front.sepia.ceph.com 2024-05-14T16:54:09.504 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-14T16:54:09.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:09 smithi002 ceph-mon[31538]: pgmap v368: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T16:54:09.761 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:54:09.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:09 smithi151 ceph-mon[36066]: pgmap v368: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T16:54:09.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:09 smithi123 ceph-mon[35971]: pgmap v368: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T16:54:10.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:10 smithi002 ceph-mon[31538]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:54:10.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:10 smithi151 ceph-mon[36066]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:54:10.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:10 smithi123 ceph-mon[35971]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:54:11.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.426+0000 7f31c2655700 1 -- 172.21.15.2:0/2959937795 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31bc1092b0 msgr2=0x7f31bc10b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:11.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.426+0000 7f31c2655700 1 --2- 172.21.15.2:0/2959937795 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31bc1092b0 0x7f31bc10b720 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f31b400b0a0 tx=0x7f31b400b3b0 comp rx=0 tx=0).stop 2024-05-14T16:54:11.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.426+0000 7f31c2655700 1 -- 172.21.15.2:0/2959937795 shutdown_connections 2024-05-14T16:54:11.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.426+0000 7f31c2655700 1 --2- 172.21.15.2:0/2959937795 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31bc1092b0 0x7f31bc10b720 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:11.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.426+0000 7f31c2655700 1 --2- 172.21.15.2:0/2959937795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31bc104050 0x7f31bc106440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:11.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.426+0000 7f31c2655700 1 --2- 172.21.15.2:0/2959937795 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f31bc106980 0x7f31bc108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:11.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.426+0000 7f31c2655700 1 -- 172.21.15.2:0/2959937795 >> 172.21.15.2:0/2959937795 conn(0x7f31bc0fdaa0 msgr2=0x7f31bc0ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:11.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.426+0000 7f31c2655700 1 -- 172.21.15.2:0/2959937795 shutdown_connections 2024-05-14T16:54:11.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.426+0000 7f31c2655700 1 -- 172.21.15.2:0/2959937795 wait complete. 2024-05-14T16:54:11.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.427+0000 7f31c2655700 1 Processor -- start 2024-05-14T16:54:11.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.427+0000 7f31c2655700 1 -- start start 2024-05-14T16:54:11.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.427+0000 7f31c2655700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f31bc104050 0x7f31bc1996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:11.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.427+0000 7f31c2655700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31bc106980 0x7f31bc199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:11.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.427+0000 7f31c2655700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31bc1092b0 0x7f31bc1a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:11.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.427+0000 7f31c2655700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f31bc10f0a0 con 0x7f31bc106980 2024-05-14T16:54:11.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.427+0000 7f31c2655700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f31bc10ef20 con 0x7f31bc1092b0 2024-05-14T16:54:11.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.427+0000 7f31c2655700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f31bc10f220 con 0x7f31bc104050 2024-05-14T16:54:11.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.427+0000 7f31b3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31bc106980 0x7f31bc199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:11.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.427+0000 7f31b3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31bc106980 0x7f31bc199c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35006/0 (socket says 172.21.15.2:35006) 2024-05-14T16:54:11.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.427+0000 7f31b3fff700 1 -- 172.21.15.2:0/2054224168 learned_addr learned my addr 172.21.15.2:0/2054224168 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:11.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.427+0000 7f31bbfff700 1 --2- 172.21.15.2:0/2054224168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f31bc104050 0x7f31bc1996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:11.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.427+0000 7f31c0bf2700 1 --2- 172.21.15.2:0/2054224168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31bc1092b0 0x7f31bc1a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:11.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.427+0000 7f31b3fff700 1 -- 172.21.15.2:0/2054224168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f31bc104050 msgr2=0x7f31bc1996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:11.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.427+0000 7f31b3fff700 1 --2- 172.21.15.2:0/2054224168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f31bc104050 0x7f31bc1996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:11.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.427+0000 7f31b3fff700 1 -- 172.21.15.2:0/2054224168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31bc1092b0 msgr2=0x7f31bc1a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:11.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.427+0000 7f31b3fff700 1 --2- 172.21.15.2:0/2054224168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31bc1092b0 0x7f31bc1a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:11.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.427+0000 7f31b3fff700 1 -- 172.21.15.2:0/2054224168 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31bc1a12d0 con 0x7f31bc106980 2024-05-14T16:54:11.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.427+0000 7f31b3fff700 1 --2- 172.21.15.2:0/2054224168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31bc106980 0x7f31bc199c20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f31a400b560 tx=0x7f31a400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:11.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.428+0000 7f31b9ffb700 1 -- 172.21.15.2:0/2054224168 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a4015020 con 0x7f31bc106980 2024-05-14T16:54:11.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.428+0000 7f31c2655700 1 -- 172.21.15.2:0/2054224168 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31bc1a15c0 con 0x7f31bc106980 2024-05-14T16:54:11.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.428+0000 7f31b9ffb700 1 -- 172.21.15.2:0/2054224168 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31a4011ce0 con 0x7f31bc106980 2024-05-14T16:54:11.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.428+0000 7f31b9ffb700 1 -- 172.21.15.2:0/2054224168 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a4010d40 con 0x7f31bc106980 2024-05-14T16:54:11.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.428+0000 7f31c2655700 1 -- 172.21.15.2:0/2054224168 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31bc1a1b80 con 0x7f31bc106980 2024-05-14T16:54:11.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.429+0000 7f31c2655700 1 -- 172.21.15.2:0/2054224168 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31bc06a230 con 0x7f31bc106980 2024-05-14T16:54:11.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.430+0000 7f31b9ffb700 1 -- 172.21.15.2:0/2054224168 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f31a4018020 con 0x7f31bc106980 2024-05-14T16:54:11.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.431+0000 7f31b9ffb700 1 --2- 172.21.15.2:0/2054224168 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f31a80b2880 0x7f31a80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:11.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.431+0000 7f31bbfff700 1 --2- 172.21.15.2:0/2054224168 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f31a80b2880 0x7f31a80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:11.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.431+0000 7f31b9ffb700 1 -- 172.21.15.2:0/2054224168 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f31a40e1ed0 con 0x7f31bc106980 2024-05-14T16:54:11.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.431+0000 7f31bbfff700 1 --2- 172.21.15.2:0/2054224168 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f31a80b2880 0x7f31a80b4d40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f31ac00ac50 tx=0x7f31ac00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:11.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.434+0000 7f31b9ffb700 1 -- 172.21.15.2:0/2054224168 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f31a401f020 con 0x7f31bc106980 2024-05-14T16:54:11.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:11 smithi002 ceph-mon[31538]: pgmap v369: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T16:54:11.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.696+0000 7f31c2655700 1 -- 172.21.15.2:0/2054224168 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f31bc02e560 con 0x7f31bc106980 2024-05-14T16:54:11.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.697+0000 7f31b9ffb700 1 -- 172.21.15.2:0/2054224168 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3036 (secure 0 0 0) 0x7f31a40a9ba0 con 0x7f31bc106980 2024-05-14T16:54:11.698 INFO:teuthology.orchestra.run.smithi002.stdout:e24 2024-05-14T16:54:11.698 INFO:teuthology.orchestra.run.smithi002.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-14T16:54:11.698 INFO:teuthology.orchestra.run.smithi002.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-14T16:54:11.698 INFO:teuthology.orchestra.run.smithi002.stdout:legacy client fscid: 1 2024-05-14T16:54:11.698 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:54:11.698 INFO:teuthology.orchestra.run.smithi002.stdout:Filesystem 'cephfs' (1) 2024-05-14T16:54:11.698 INFO:teuthology.orchestra.run.smithi002.stdout:fs_name cephfs 2024-05-14T16:54:11.698 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 24 2024-05-14T16:54:11.698 INFO:teuthology.orchestra.run.smithi002.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-14T16:54:11.698 INFO:teuthology.orchestra.run.smithi002.stdout:created 2024-05-14T16:51:34.106919+0000 2024-05-14T16:54:11.698 INFO:teuthology.orchestra.run.smithi002.stdout:modified 2024-05-14T16:51:53.879252+0000 2024-05-14T16:54:11.698 INFO:teuthology.orchestra.run.smithi002.stdout:tableserver 0 2024-05-14T16:54:11.699 INFO:teuthology.orchestra.run.smithi002.stdout:root 0 2024-05-14T16:54:11.699 INFO:teuthology.orchestra.run.smithi002.stdout:session_timeout 300 2024-05-14T16:54:11.699 INFO:teuthology.orchestra.run.smithi002.stdout:session_autoclose 300 2024-05-14T16:54:11.699 INFO:teuthology.orchestra.run.smithi002.stdout:max_file_size 1099511627776 2024-05-14T16:54:11.699 INFO:teuthology.orchestra.run.smithi002.stdout:max_xattr_size 65536 2024-05-14T16:54:11.699 INFO:teuthology.orchestra.run.smithi002.stdout:required_client_features {} 2024-05-14T16:54:11.699 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure 0 2024-05-14T16:54:11.699 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure_osd_epoch 0 2024-05-14T16:54:11.699 INFO:teuthology.orchestra.run.smithi002.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-14T16:54:11.699 INFO:teuthology.orchestra.run.smithi002.stdout:max_mds 3 2024-05-14T16:54:11.699 INFO:teuthology.orchestra.run.smithi002.stdout:in 0,1,2 2024-05-14T16:54:11.699 INFO:teuthology.orchestra.run.smithi002.stdout:up {0=24479,1=24455,2=24431} 2024-05-14T16:54:11.699 INFO:teuthology.orchestra.run.smithi002.stdout:failed 2024-05-14T16:54:11.700 INFO:teuthology.orchestra.run.smithi002.stdout:damaged 2024-05-14T16:54:11.700 INFO:teuthology.orchestra.run.smithi002.stdout:stopped 2024-05-14T16:54:11.700 INFO:teuthology.orchestra.run.smithi002.stdout:data_pools [3] 2024-05-14T16:54:11.700 INFO:teuthology.orchestra.run.smithi002.stdout:metadata_pool 2 2024-05-14T16:54:11.700 INFO:teuthology.orchestra.run.smithi002.stdout:inline_data disabled 2024-05-14T16:54:11.700 INFO:teuthology.orchestra.run.smithi002.stdout:balancer 2024-05-14T16:54:11.700 INFO:teuthology.orchestra.run.smithi002.stdout:bal_rank_mask -1 2024-05-14T16:54:11.700 INFO:teuthology.orchestra.run.smithi002.stdout:standby_count_wanted 0 2024-05-14T16:54:11.700 INFO:teuthology.orchestra.run.smithi002.stdout:qdb_cluster leader: 24479 members: 24455,24479,24431 2024-05-14T16:54:11.700 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.b{0:24479} state up:active seq 5 addr [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T16:54:11.700 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.i{0:24427} state up:standby-replay seq 1 addr [v2:172.21.15.151:6838/4107834375,v1:172.21.15.151:6839/4107834375] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T16:54:11.700 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.h{1:24455} state up:active seq 12 addr [v2:172.21.15.123:6836/333426645,v1:172.21.15.123:6837/333426645] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T16:54:11.700 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.c{1:24421} state up:standby-replay seq 1 addr [v2:172.21.15.151:6836/132043151,v1:172.21.15.151:6837/132043151] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T16:54:11.700 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.e{2:24431} state up:active seq 14 addr [v2:172.21.15.123:6834/1198670013,v1:172.21.15.123:6835/1198670013] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T16:54:11.700 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.f{2:24415} state up:standby-replay seq 1 addr [v2:172.21.15.151:6834/762044339,v1:172.21.15.151:6835/762044339] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T16:54:11.700 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:54:11.701 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:54:11.701 INFO:teuthology.orchestra.run.smithi002.stdout:Standby daemons: 2024-05-14T16:54:11.701 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:54:11.701 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.j{-1:14508} state up:standby seq 1 addr [v2:172.21.15.2:6834/2275696923,v1:172.21.15.2:6835/2275696923] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T16:54:11.701 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.g{-1:14574} state up:standby seq 1 addr [v2:172.21.15.2:6838/716718761,v1:172.21.15.2:6839/716718761] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T16:54:11.701 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.a{-1:14586} state up:standby seq 1 addr [v2:172.21.15.2:6840/3644815179,v1:172.21.15.2:6841/3644815179] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T16:54:11.701 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.l{-1:24391} state up:standby seq 1 addr [v2:172.21.15.151:6832/1880336731,v1:172.21.15.151:6833/1880336731] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T16:54:11.701 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.d{-1:24406} state up:standby seq 1 addr [v2:172.21.15.2:6836/2415605548,v1:172.21.15.2:6837/2415605548] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T16:54:11.701 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.k{-1:24410} state up:standby seq 1 addr [v2:172.21.15.123:6832/2635167,v1:172.21.15.123:6833/2635167] compat {c=[1],r=[1],i=[1fff]}] 2024-05-14T16:54:11.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.699+0000 7f31c2655700 1 -- 172.21.15.2:0/2054224168 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f31a80b2880 msgr2=0x7f31a80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:11.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.699+0000 7f31c2655700 1 --2- 172.21.15.2:0/2054224168 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f31a80b2880 0x7f31a80b4d40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f31ac00ac50 tx=0x7f31ac00a380 comp rx=0 tx=0).stop 2024-05-14T16:54:11.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.699+0000 7f31c2655700 1 -- 172.21.15.2:0/2054224168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31bc106980 msgr2=0x7f31bc199c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:11.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.699+0000 7f31c2655700 1 --2- 172.21.15.2:0/2054224168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31bc106980 0x7f31bc199c20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f31a400b560 tx=0x7f31a400b870 comp rx=0 tx=0).stop 2024-05-14T16:54:11.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.699+0000 7f31c2655700 1 -- 172.21.15.2:0/2054224168 shutdown_connections 2024-05-14T16:54:11.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.699+0000 7f31c2655700 1 --2- 172.21.15.2:0/2054224168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31bc1092b0 0x7f31bc1a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:11.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.699+0000 7f31c2655700 1 --2- 172.21.15.2:0/2054224168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f31bc104050 0x7f31bc1996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:11.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.699+0000 7f31c2655700 1 --2- 172.21.15.2:0/2054224168 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f31a80b2880 0x7f31a80b4d40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:11.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.699+0000 7f31c2655700 1 --2- 172.21.15.2:0/2054224168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31bc106980 0x7f31bc199c20 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:11.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.699+0000 7f31c2655700 1 -- 172.21.15.2:0/2054224168 >> 172.21.15.2:0/2054224168 conn(0x7f31bc0fdaa0 msgr2=0x7f31bc1049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:11.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.699+0000 7f31c2655700 1 -- 172.21.15.2:0/2054224168 shutdown_connections 2024-05-14T16:54:11.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:11.699+0000 7f31c2655700 1 -- 172.21.15.2:0/2054224168 wait complete. 2024-05-14T16:54:11.703 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 24 2024-05-14T16:54:11.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:11 smithi151 ceph-mon[36066]: pgmap v369: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T16:54:11.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:11 smithi123 ceph-mon[35971]: pgmap v369: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T16:54:12.499 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-14T16:54:12.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:12 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/2054224168' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-14T16:54:12.750 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:54:12.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:12 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/2054224168' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-14T16:54:12.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:12 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/2054224168' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-14T16:54:13.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:13 smithi002 ceph-mon[31538]: pgmap v370: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:54:13.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:13 smithi151 ceph-mon[36066]: pgmap v370: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:54:13.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:13 smithi123 ceph-mon[35971]: pgmap v370: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:54:14.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.367+0000 7f87d6260700 1 -- 172.21.15.2:0/4187764423 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87d01035a0 msgr2=0x7f87d01039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:14.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.367+0000 7f87d6260700 1 --2- 172.21.15.2:0/4187764423 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87d01035a0 0x7f87d01039c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f87c00099c0 tx=0x7f87c0009cd0 comp rx=0 tx=0).stop 2024-05-14T16:54:14.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.367+0000 7f87d6260700 1 -- 172.21.15.2:0/4187764423 shutdown_connections 2024-05-14T16:54:14.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.367+0000 7f87d6260700 1 --2- 172.21.15.2:0/4187764423 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87d0105140 0x7f87d010b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:14.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.367+0000 7f87d6260700 1 --2- 172.21.15.2:0/4187764423 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87d01035a0 0x7f87d01039c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:14.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.367+0000 7f87d6260700 1 --2- 172.21.15.2:0/4187764423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87d01047a0 0x7f87d0104c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:14.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.367+0000 7f87d6260700 1 -- 172.21.15.2:0/4187764423 >> 172.21.15.2:0/4187764423 conn(0x7f87d00feb40 msgr2=0x7f87d0100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:14.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.367+0000 7f87d6260700 1 -- 172.21.15.2:0/4187764423 shutdown_connections 2024-05-14T16:54:14.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.367+0000 7f87d6260700 1 -- 172.21.15.2:0/4187764423 wait complete. 2024-05-14T16:54:14.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.368+0000 7f87d6260700 1 Processor -- start 2024-05-14T16:54:14.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.368+0000 7f87d6260700 1 -- start start 2024-05-14T16:54:14.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.368+0000 7f87d6260700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87d01035a0 0x7f87d0199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:14.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.368+0000 7f87d6260700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87d01047a0 0x7f87d0199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:14.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.368+0000 7f87d6260700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87d0105140 0x7f87d01a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:14.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.368+0000 7f87d6260700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f87d010f040 con 0x7f87d0105140 2024-05-14T16:54:14.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.368+0000 7f87d6260700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f87d010eec0 con 0x7f87d01047a0 2024-05-14T16:54:14.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.368+0000 7f87d6260700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f87d010f1c0 con 0x7f87d01035a0 2024-05-14T16:54:14.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.368+0000 7f87cffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87d0105140 0x7f87d01a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:14.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.368+0000 7f87cffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87d0105140 0x7f87d01a0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57128/0 (socket says 172.21.15.2:57128) 2024-05-14T16:54:14.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.368+0000 7f87cffff700 1 -- 172.21.15.2:0/1827500966 learned_addr learned my addr 172.21.15.2:0/1827500966 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:14.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.368+0000 7f87ceffd700 1 --2- 172.21.15.2:0/1827500966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87d01047a0 0x7f87d0199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:14.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.368+0000 7f87cf7fe700 1 --2- 172.21.15.2:0/1827500966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87d01035a0 0x7f87d0199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:14.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.369+0000 7f87cffff700 1 -- 172.21.15.2:0/1827500966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87d01035a0 msgr2=0x7f87d0199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:14.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.369+0000 7f87cffff700 1 --2- 172.21.15.2:0/1827500966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87d01035a0 0x7f87d0199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:14.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.369+0000 7f87cffff700 1 -- 172.21.15.2:0/1827500966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87d01047a0 msgr2=0x7f87d0199c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:14.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.369+0000 7f87cffff700 1 --2- 172.21.15.2:0/1827500966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87d01047a0 0x7f87d0199c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:14.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.369+0000 7f87cffff700 1 -- 172.21.15.2:0/1827500966 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d01a1370 con 0x7f87d0105140 2024-05-14T16:54:14.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.369+0000 7f87cffff700 1 --2- 172.21.15.2:0/1827500966 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87d0105140 0x7f87d01a0cd0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f87c8007f10 tx=0x7f87c800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:14.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.369+0000 7f87ccff9700 1 -- 172.21.15.2:0/1827500966 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c8004bb0 con 0x7f87d0105140 2024-05-14T16:54:14.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.369+0000 7f87d6260700 1 -- 172.21.15.2:0/1827500966 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87d01a1660 con 0x7f87d0105140 2024-05-14T16:54:14.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.369+0000 7f87ccff9700 1 -- 172.21.15.2:0/1827500966 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87c8004d50 con 0x7f87d0105140 2024-05-14T16:54:14.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.369+0000 7f87ccff9700 1 -- 172.21.15.2:0/1827500966 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c80058e0 con 0x7f87d0105140 2024-05-14T16:54:14.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.369+0000 7f87d6260700 1 -- 172.21.15.2:0/1827500966 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87d01a1b50 con 0x7f87d0105140 2024-05-14T16:54:14.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.370+0000 7f87d6260700 1 -- 172.21.15.2:0/1827500966 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87d006a230 con 0x7f87d0105140 2024-05-14T16:54:14.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.371+0000 7f87ccff9700 1 -- 172.21.15.2:0/1827500966 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f87c801a020 con 0x7f87d0105140 2024-05-14T16:54:14.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.372+0000 7f87ccff9700 1 --2- 172.21.15.2:0/1827500966 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f87b80b2880 0x7f87b80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:14.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.372+0000 7f87cf7fe700 1 --2- 172.21.15.2:0/1827500966 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f87b80b2880 0x7f87b80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:14.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.372+0000 7f87ccff9700 1 -- 172.21.15.2:0/1827500966 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f87c80e1fd0 con 0x7f87d0105140 2024-05-14T16:54:14.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.373+0000 7f87cf7fe700 1 --2- 172.21.15.2:0/1827500966 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f87b80b2880 0x7f87b80b4d40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f87c00038c0 tx=0x7f87c000b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:14.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.375+0000 7f87ccff9700 1 -- 172.21.15.2:0/1827500966 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f87c8010070 con 0x7f87d0105140 2024-05-14T16:54:14.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.600+0000 7f87d6260700 1 -- 172.21.15.2:0/1827500966 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f87d002e590 con 0x7f87d0105140 2024-05-14T16:54:14.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.601+0000 7f87ccff9700 1 -- 172.21.15.2:0/1827500966 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v80) ==== 56+0+5270 (secure 0 0 0) 0x7f87c80a9ca0 con 0x7f87d0105140 2024-05-14T16:54:14.601 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 80 2024-05-14T16:54:14.601 INFO:teuthology.orchestra.run.smithi002.stdout:fsid ed40717a-1210-11ef-bc98-c7b262605968 2024-05-14T16:54:14.601 INFO:teuthology.orchestra.run.smithi002.stdout:created 2024-05-14T16:43:09.415298+0000 2024-05-14T16:54:14.601 INFO:teuthology.orchestra.run.smithi002.stdout:modified 2024-05-14T16:52:13.205039+0000 2024-05-14T16:54:14.601 INFO:teuthology.orchestra.run.smithi002.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-14T16:54:14.602 INFO:teuthology.orchestra.run.smithi002.stdout:crush_version 26 2024-05-14T16:54:14.602 INFO:teuthology.orchestra.run.smithi002.stdout:full_ratio 0.95 2024-05-14T16:54:14.602 INFO:teuthology.orchestra.run.smithi002.stdout:backfillfull_ratio 0.9 2024-05-14T16:54:14.602 INFO:teuthology.orchestra.run.smithi002.stdout:nearfull_ratio 0.85 2024-05-14T16:54:14.602 INFO:teuthology.orchestra.run.smithi002.stdout:require_min_compat_client luminous 2024-05-14T16:54:14.602 INFO:teuthology.orchestra.run.smithi002.stdout:min_compat_client luminous 2024-05-14T16:54:14.602 INFO:teuthology.orchestra.run.smithi002.stdout:require_osd_release squid 2024-05-14T16:54:14.602 INFO:teuthology.orchestra.run.smithi002.stdout:stretch_mode_enabled false 2024-05-14T16:54:14.602 INFO:teuthology.orchestra.run.smithi002.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-14T16:54:14.602 INFO:teuthology.orchestra.run.smithi002.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-05-14T16:54:14.603 INFO:teuthology.orchestra.run.smithi002.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-05-14T16:54:14.603 INFO:teuthology.orchestra.run.smithi002.stdout:max_osd 12 2024-05-14T16:54:14.603 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 up in weight 1 up_from 8 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6802/2654237820,v1:172.21.15.2:6803/2654237820] [v2:172.21.15.2:6804/2654237820,v1:172.21.15.2:6805/2654237820] exists,up 153f20ec-5bcf-4786-9ede-2e512a1e3d13 2024-05-14T16:54:14.603 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1 up in weight 1 up_from 13 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6800/2668156992,v1:172.21.15.123:6801/2668156992] [v2:172.21.15.123:6802/2668156992,v1:172.21.15.123:6803/2668156992] exists,up 0e95419f-9a9d-4f7a-b55d-ecb54971918a 2024-05-14T16:54:14.603 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2 up in weight 1 up_from 18 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6800/2160189636,v1:172.21.15.151:6801/2160189636] [v2:172.21.15.151:6802/2160189636,v1:172.21.15.151:6803/2160189636] exists,up b6ce7ac7-be6c-4549-a356-59cb81b5b492 2024-05-14T16:54:14.604 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 up in weight 1 up_from 25 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6810/3451424073,v1:172.21.15.2:6811/3451424073] [v2:172.21.15.2:6812/3451424073,v1:172.21.15.2:6813/3451424073] exists,up 2fd592bf-c212-411a-afbe-a7c498e5ac93 2024-05-14T16:54:14.604 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4 up in weight 1 up_from 31 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6808/2101935762,v1:172.21.15.123:6809/2101935762] [v2:172.21.15.123:6810/2101935762,v1:172.21.15.123:6811/2101935762] exists,up af1905e7-e036-47c6-be59-b085bc41fede 2024-05-14T16:54:14.604 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 up in weight 1 up_from 36 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6808/3695961886,v1:172.21.15.151:6809/3695961886] [v2:172.21.15.151:6810/3695961886,v1:172.21.15.151:6811/3695961886] exists,up 9c166916-af78-4cc2-bd14-d1220b6e7d9a 2024-05-14T16:54:14.604 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6 up in weight 1 up_from 42 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6818/2711893600,v1:172.21.15.2:6819/2711893600] [v2:172.21.15.2:6820/2711893600,v1:172.21.15.2:6821/2711893600] exists,up 60e9bf66-2338-41e8-9280-398d4e3b322f 2024-05-14T16:54:14.604 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7 up in weight 1 up_from 47 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6816/3634155507,v1:172.21.15.123:6817/3634155507] [v2:172.21.15.123:6818/3634155507,v1:172.21.15.123:6819/3634155507] exists,up c93e07a1-e941-493e-8468-33429df2dff0 2024-05-14T16:54:14.604 INFO:teuthology.orchestra.run.smithi002.stdout:osd.8 up in weight 1 up_from 53 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6816/4271107010,v1:172.21.15.151:6817/4271107010] [v2:172.21.15.151:6818/4271107010,v1:172.21.15.151:6819/4271107010] exists,up 1155fb58-d614-4f45-91e2-5da3a93e7d9c 2024-05-14T16:54:14.604 INFO:teuthology.orchestra.run.smithi002.stdout:osd.9 up in weight 1 up_from 59 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6826/3479431021,v1:172.21.15.2:6827/3479431021] [v2:172.21.15.2:6828/3479431021,v1:172.21.15.2:6829/3479431021] exists,up 82b34e3b-0017-4ca4-8f86-0993ca78ce7e 2024-05-14T16:54:14.605 INFO:teuthology.orchestra.run.smithi002.stdout:osd.10 up in weight 1 up_from 64 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6824/4266374435,v1:172.21.15.123:6825/4266374435] [v2:172.21.15.123:6826/4266374435,v1:172.21.15.123:6827/4266374435] exists,up 32e2ace9-eb5d-4258-a2ac-49b782be2b5c 2024-05-14T16:54:14.605 INFO:teuthology.orchestra.run.smithi002.stdout:osd.11 up in weight 1 up_from 70 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6824/4129194698,v1:172.21.15.151:6825/4129194698] [v2:172.21.15.151:6826/4129194698,v1:172.21.15.151:6827/4129194698] exists,up 6e3c2ead-1af6-4bda-87cd-12770c81fa67 2024-05-14T16:54:14.605 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 2.6 [8,11] 2024-05-14T16:54:14.605 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 2.a [2,11] 2024-05-14T16:54:14.605 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 3.6 [4,7] 2024-05-14T16:54:14.606 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 3.c [3,9] 2024-05-14T16:54:14.606 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 3.14 [4,6] 2024-05-14T16:54:14.606 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 3.20 [5,9] 2024-05-14T16:54:14.606 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 3.27 [4,8] 2024-05-14T16:54:14.606 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 3.38 [3,9] 2024-05-14T16:54:14.606 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:0/4267809161 expires 2024-05-15T16:44:11.998214+0000 2024-05-14T16:54:14.607 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:6801/3582988344 expires 2024-05-15T16:44:11.998214+0000 2024-05-14T16:54:14.607 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:6800/2574015037 expires 2024-05-15T16:43:43.407184+0000 2024-05-14T16:54:14.607 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:0/2100732571 expires 2024-05-15T16:44:11.998214+0000 2024-05-14T16:54:14.607 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:0/2954199324 expires 2024-05-15T16:43:43.407184+0000 2024-05-14T16:54:14.607 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:6801/2574015037 expires 2024-05-15T16:43:43.407184+0000 2024-05-14T16:54:14.607 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:0/3171550944 expires 2024-05-15T16:43:43.407184+0000 2024-05-14T16:54:14.608 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:0/1679761136 expires 2024-05-15T16:44:11.998214+0000 2024-05-14T16:54:14.608 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:0/1746766591 expires 2024-05-15T16:43:43.407184+0000 2024-05-14T16:54:14.608 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:6800/3582988344 expires 2024-05-15T16:44:11.998214+0000 2024-05-14T16:54:14.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.604+0000 7f87d6260700 1 -- 172.21.15.2:0/1827500966 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f87b80b2880 msgr2=0x7f87b80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:14.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.604+0000 7f87d6260700 1 --2- 172.21.15.2:0/1827500966 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f87b80b2880 0x7f87b80b4d40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f87c00038c0 tx=0x7f87c000b540 comp rx=0 tx=0).stop 2024-05-14T16:54:14.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.604+0000 7f87d6260700 1 -- 172.21.15.2:0/1827500966 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87d0105140 msgr2=0x7f87d01a0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:14.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.604+0000 7f87d6260700 1 --2- 172.21.15.2:0/1827500966 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87d0105140 0x7f87d01a0cd0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f87c8007f10 tx=0x7f87c800d3b0 comp rx=0 tx=0).stop 2024-05-14T16:54:14.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.604+0000 7f87d6260700 1 -- 172.21.15.2:0/1827500966 shutdown_connections 2024-05-14T16:54:14.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.604+0000 7f87d6260700 1 --2- 172.21.15.2:0/1827500966 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f87b80b2880 0x7f87b80b4d40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:14.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.604+0000 7f87d6260700 1 --2- 172.21.15.2:0/1827500966 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87d0105140 0x7f87d01a0cd0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:14.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.604+0000 7f87d6260700 1 --2- 172.21.15.2:0/1827500966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87d01035a0 0x7f87d0199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:14.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.604+0000 7f87d6260700 1 --2- 172.21.15.2:0/1827500966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87d01047a0 0x7f87d0199c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:14.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.604+0000 7f87d6260700 1 -- 172.21.15.2:0/1827500966 >> 172.21.15.2:0/1827500966 conn(0x7f87d00feb40 msgr2=0x7f87d0107bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:14.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.604+0000 7f87d6260700 1 -- 172.21.15.2:0/1827500966 shutdown_connections 2024-05-14T16:54:14.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:14.604+0000 7f87d6260700 1 -- 172.21.15.2:0/1827500966 wait complete. 2024-05-14T16:54:15.388 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-14T16:54:15.438 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:54:15.695 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:54:15.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:15 smithi002 ceph-mon[31538]: pgmap v371: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:54:15.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:15 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/1827500966' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-14T16:54:15.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:15 smithi151 ceph-mon[36066]: pgmap v371: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:54:15.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:15 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/1827500966' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-14T16:54:15.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:15 smithi123 ceph-mon[35971]: pgmap v371: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:54:15.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:15 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/1827500966' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-14T16:54:17.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.372+0000 7fa6d1f5d700 1 -- 172.21.15.2:0/20725096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6cc109630 msgr2=0x7fa6cc109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:17.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.372+0000 7fa6d1f5d700 1 --2- 172.21.15.2:0/20725096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6cc109630 0x7fa6cc109a10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa6b40099c0 tx=0x7fa6b4009cd0 comp rx=0 tx=0).stop 2024-05-14T16:54:17.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.372+0000 7fa6d1f5d700 1 -- 172.21.15.2:0/20725096 shutdown_connections 2024-05-14T16:54:17.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.372+0000 7fa6d1f5d700 1 --2- 172.21.15.2:0/20725096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6cc06c6d0 0x7fa6cc110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:17.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.372+0000 7fa6d1f5d700 1 --2- 172.21.15.2:0/20725096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6cc109630 0x7fa6cc109a10 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:17.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.372+0000 7fa6d1f5d700 1 --2- 172.21.15.2:0/20725096 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6cc06bd10 0x7fa6cc06c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:17.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.372+0000 7fa6d1f5d700 1 -- 172.21.15.2:0/20725096 >> 172.21.15.2:0/20725096 conn(0x7fa6cc0794e0 msgr2=0x7fa6cc0798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:17.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.372+0000 7fa6d1f5d700 1 -- 172.21.15.2:0/20725096 shutdown_connections 2024-05-14T16:54:17.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.372+0000 7fa6d1f5d700 1 -- 172.21.15.2:0/20725096 wait complete. 2024-05-14T16:54:17.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.372+0000 7fa6d1f5d700 1 Processor -- start 2024-05-14T16:54:17.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.373+0000 7fa6d1f5d700 1 -- start start 2024-05-14T16:54:17.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.373+0000 7fa6d1f5d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6cc06bd10 0x7fa6cc19beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:17.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.373+0000 7fa6d1f5d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6cc06c6d0 0x7fa6cc19c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:17.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.373+0000 7fa6d1f5d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6cc109630 0x7fa6cc1a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:17.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.373+0000 7fa6d1f5d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa6cc113750 con 0x7fa6cc06bd10 2024-05-14T16:54:17.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.373+0000 7fa6d1f5d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa6cc1135d0 con 0x7fa6cc06c6d0 2024-05-14T16:54:17.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.373+0000 7fa6d1f5d700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa6cc1138d0 con 0x7fa6cc109630 2024-05-14T16:54:17.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.373+0000 7fa6cb7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6cc06bd10 0x7fa6cc19beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:17.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.373+0000 7fa6cb7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6cc06bd10 0x7fa6cc19beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57152/0 (socket says 172.21.15.2:57152) 2024-05-14T16:54:17.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.373+0000 7fa6cb7fe700 1 -- 172.21.15.2:0/2609295524 learned_addr learned my addr 172.21.15.2:0/2609295524 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:17.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.373+0000 7fa6cbfff700 1 --2- 172.21.15.2:0/2609295524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6cc109630 0x7fa6cc1a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:17.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.373+0000 7fa6caffd700 1 --2- 172.21.15.2:0/2609295524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6cc06c6d0 0x7fa6cc19c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:17.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.373+0000 7fa6cb7fe700 1 -- 172.21.15.2:0/2609295524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6cc109630 msgr2=0x7fa6cc1a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:17.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.373+0000 7fa6cb7fe700 1 --2- 172.21.15.2:0/2609295524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6cc109630 0x7fa6cc1a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:17.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.373+0000 7fa6cb7fe700 1 -- 172.21.15.2:0/2609295524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6cc06c6d0 msgr2=0x7fa6cc19c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:17.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.373+0000 7fa6cb7fe700 1 --2- 172.21.15.2:0/2609295524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6cc06c6d0 0x7fa6cc19c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:17.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.373+0000 7fa6cb7fe700 1 -- 172.21.15.2:0/2609295524 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6cc1a0ec0 con 0x7fa6cc06bd10 2024-05-14T16:54:17.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.374+0000 7fa6cb7fe700 1 --2- 172.21.15.2:0/2609295524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6cc06bd10 0x7fa6cc19beb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400b580 tx=0x7fa6b40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:17.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.374+0000 7fa6c8ff9700 1 -- 172.21.15.2:0/2609295524 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4027070 con 0x7fa6cc06bd10 2024-05-14T16:54:17.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.374+0000 7fa6d1f5d700 1 -- 172.21.15.2:0/2609295524 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6cc1a1150 con 0x7fa6cc06bd10 2024-05-14T16:54:17.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.374+0000 7fa6c8ff9700 1 -- 172.21.15.2:0/2609295524 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6b4003f20 con 0x7fa6cc06bd10 2024-05-14T16:54:17.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.374+0000 7fa6c8ff9700 1 -- 172.21.15.2:0/2609295524 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4022d10 con 0x7fa6cc06bd10 2024-05-14T16:54:17.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.374+0000 7fa6d1f5d700 1 -- 172.21.15.2:0/2609295524 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6cc1a8a30 con 0x7fa6cc06bd10 2024-05-14T16:54:17.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.375+0000 7fa6d1f5d700 1 -- 172.21.15.2:0/2609295524 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6cc06a230 con 0x7fa6cc06bd10 2024-05-14T16:54:17.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.377+0000 7fa6c8ff9700 1 -- 172.21.15.2:0/2609295524 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fa6b4022470 con 0x7fa6cc06bd10 2024-05-14T16:54:17.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.378+0000 7fa6c8ff9700 1 --2- 172.21.15.2:0/2609295524 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fa6b80b2880 0x7fa6b80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:17.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.378+0000 7fa6caffd700 1 --2- 172.21.15.2:0/2609295524 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fa6b80b2880 0x7fa6b80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:17.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.378+0000 7fa6c8ff9700 1 -- 172.21.15.2:0/2609295524 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7fa6b40e3200 con 0x7fa6cc06bd10 2024-05-14T16:54:17.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.378+0000 7fa6caffd700 1 --2- 172.21.15.2:0/2609295524 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fa6b80b2880 0x7fa6b80b4d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc104520 tx=0x7fa6bc00a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:17.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.381+0000 7fa6c8ff9700 1 -- 172.21.15.2:0/2609295524 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa6b4013070 con 0x7fa6cc06bd10 2024-05-14T16:54:17.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.643+0000 7fa6d1f5d700 1 -- 172.21.15.2:0/2609295524 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6cc1a8c00 con 0x7fa6cc06bd10 2024-05-14T16:54:17.643 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:17 smithi002 ceph-mon[31538]: pgmap v372: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:54:17.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.644+0000 7fa6c8ff9700 1 -- 172.21.15.2:0/2609295524 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11850 (secure 0 0 0) 0x7fa6b40aaf60 con 0x7fa6cc06bd10 2024-05-14T16:54:17.644 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:54:17.644 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2275696923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2275696923},{"type":"v1","addr":"172.21.15.2:6835","nonce":2275696923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/716718761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":716718761},{"type":"v1","addr":"172.21.15.2:6839","nonce":716718761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/3644815179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3644815179},{"type":"v1","addr":"172.21.15.2:6841","nonce":3644815179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1880336731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1880336731},{"type":"v1","addr":"172.21.15.151:6833","nonce":1880336731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2415605548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2415605548},{"type":"v1","addr":"172.21.15.2:6837","nonce":2415605548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2635167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2635167},{"type":"v1","addr":"172.21.15.123:6833","nonce":2635167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:51:34.106919+0000","modified":"2024-05-14T16:51:53.879252+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24455,"mds_2":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24415":{"gid":24415,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/762044339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":762044339},{"type":"v1","addr":"172.21.15.151:6835","nonce":762044339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/132043151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":132043151},{"type":"v1","addr":"172.21.15.151:6837","nonce":132043151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/4107834375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":4107834375},{"type":"v1","addr":"172.21.15.151:6839","nonce":4107834375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.123:6835/1198670013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1198670013},{"type":"v1","addr":"172.21.15.123:6835","nonce":1198670013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/333426645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":333426645},{"type":"v1","addr":"172.21.15.123:6837","nonce":333426645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/580275877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":580275877},{"type":"v1","addr":"172.21.15.123:6839","nonce":580275877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24455,24479,24431]},"id":1}]} 2024-05-14T16:54:17.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.646+0000 7fa6d1f5d700 1 -- 172.21.15.2:0/2609295524 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fa6b80b2880 msgr2=0x7fa6b80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:17.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.646+0000 7fa6d1f5d700 1 --2- 172.21.15.2:0/2609295524 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fa6b80b2880 0x7fa6b80b4d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc104520 tx=0x7fa6bc00a300 comp rx=0 tx=0).stop 2024-05-14T16:54:17.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.646+0000 7fa6d1f5d700 1 -- 172.21.15.2:0/2609295524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6cc06bd10 msgr2=0x7fa6cc19beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:17.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.646+0000 7fa6d1f5d700 1 --2- 172.21.15.2:0/2609295524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6cc06bd10 0x7fa6cc19beb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400b580 tx=0x7fa6b40047b0 comp rx=0 tx=0).stop 2024-05-14T16:54:17.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.646+0000 7fa6d1f5d700 1 -- 172.21.15.2:0/2609295524 shutdown_connections 2024-05-14T16:54:17.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.646+0000 7fa6d1f5d700 1 --2- 172.21.15.2:0/2609295524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6cc109630 0x7fa6cc1a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:17.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.646+0000 7fa6d1f5d700 1 --2- 172.21.15.2:0/2609295524 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fa6b80b2880 0x7fa6b80b4d40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:17.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.646+0000 7fa6d1f5d700 1 --2- 172.21.15.2:0/2609295524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6cc06bd10 0x7fa6cc19beb0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:17.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.646+0000 7fa6d1f5d700 1 --2- 172.21.15.2:0/2609295524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6cc06c6d0 0x7fa6cc19c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:17.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.646+0000 7fa6d1f5d700 1 -- 172.21.15.2:0/2609295524 >> 172.21.15.2:0/2609295524 conn(0x7fa6cc0794e0 msgr2=0x7fa6cc10e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:17.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.646+0000 7fa6d1f5d700 1 -- 172.21.15.2:0/2609295524 shutdown_connections 2024-05-14T16:54:17.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:17.646+0000 7fa6d1f5d700 1 -- 172.21.15.2:0/2609295524 wait complete. 2024-05-14T16:54:17.647 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 24 2024-05-14T16:54:17.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:17 smithi151 ceph-mon[36066]: pgmap v372: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:54:17.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:17 smithi123 ceph-mon[35971]: pgmap v372: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:54:18.280 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:54:18.530 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:54:18.653 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:18 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/2609295524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:54:18.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:18 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/2609295524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:54:18.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:18 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/2609295524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:54:19.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:19 smithi002 ceph-mon[31538]: pgmap v373: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T16:54:19.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:19 smithi151 ceph-mon[36066]: pgmap v373: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T16:54:19.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:19 smithi123 ceph-mon[35971]: pgmap v373: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T16:54:20.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.154+0000 7f9f5cf2f700 1 -- 172.21.15.2:0/4680212 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f58102e20 msgr2=0x7f9f5810f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:20.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.154+0000 7f9f5cf2f700 1 --2- 172.21.15.2:0/4680212 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f58102e20 0x7f9f5810f330 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9f50009990 tx=0x7f9f50009ca0 comp rx=0 tx=0).stop 2024-05-14T16:54:20.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.154+0000 7f9f5cf2f700 1 -- 172.21.15.2:0/4680212 shutdown_connections 2024-05-14T16:54:20.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.154+0000 7f9f5cf2f700 1 --2- 172.21.15.2:0/4680212 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f58102e20 0x7f9f5810f330 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:20.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.154+0000 7f9f5cf2f700 1 --2- 172.21.15.2:0/4680212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f581084f0 0x7f9f581088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:20.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.154+0000 7f9f5cf2f700 1 --2- 172.21.15.2:0/4680212 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f58102460 0x7f9f581028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:20.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.154+0000 7f9f5cf2f700 1 -- 172.21.15.2:0/4680212 >> 172.21.15.2:0/4680212 conn(0x7f9f580794e0 msgr2=0x7f9f580798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:20.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.154+0000 7f9f5cf2f700 1 -- 172.21.15.2:0/4680212 shutdown_connections 2024-05-14T16:54:20.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.154+0000 7f9f5cf2f700 1 -- 172.21.15.2:0/4680212 wait complete. 2024-05-14T16:54:20.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.155+0000 7f9f5cf2f700 1 Processor -- start 2024-05-14T16:54:20.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.155+0000 7f9f5cf2f700 1 -- start start 2024-05-14T16:54:20.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.155+0000 7f9f5cf2f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f58102460 0x7f9f5819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:20.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.155+0000 7f9f5cf2f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f58102e20 0x7f9f5819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:20.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.155+0000 7f9f5cf2f700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f581084f0 0x7f9f58199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:20.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.155+0000 7f9f5cf2f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9f58112630 con 0x7f9f58102e20 2024-05-14T16:54:20.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.155+0000 7f9f5cf2f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9f581124b0 con 0x7f9f58102460 2024-05-14T16:54:20.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.155+0000 7f9f5cf2f700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9f581127b0 con 0x7f9f581084f0 2024-05-14T16:54:20.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.155+0000 7f9f55d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f58102e20 0x7f9f5819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:20.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.155+0000 7f9f55d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f58102e20 0x7f9f5819fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57170/0 (socket says 172.21.15.2:57170) 2024-05-14T16:54:20.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.155+0000 7f9f55d9b700 1 -- 172.21.15.2:0/2488937762 learned_addr learned my addr 172.21.15.2:0/2488937762 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:20.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.155+0000 7f9f5659c700 1 --2- 172.21.15.2:0/2488937762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f58102460 0x7f9f5819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:20.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.155+0000 7f9f56d9d700 1 --2- 172.21.15.2:0/2488937762 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f581084f0 0x7f9f58199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:20.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.156+0000 7f9f55d9b700 1 -- 172.21.15.2:0/2488937762 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f581084f0 msgr2=0x7f9f58199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:20.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.156+0000 7f9f55d9b700 1 --2- 172.21.15.2:0/2488937762 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f581084f0 0x7f9f58199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:20.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.156+0000 7f9f55d9b700 1 -- 172.21.15.2:0/2488937762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f58102460 msgr2=0x7f9f5819f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:20.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.156+0000 7f9f55d9b700 1 --2- 172.21.15.2:0/2488937762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f58102460 0x7f9f5819f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:20.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.156+0000 7f9f55d9b700 1 -- 172.21.15.2:0/2488937762 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f5819a0e0 con 0x7f9f58102e20 2024-05-14T16:54:20.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.156+0000 7f9f55d9b700 1 --2- 172.21.15.2:0/2488937762 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f58102e20 0x7f9f5819fd00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9f4000d980 tx=0x7f9f4000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:20.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.156+0000 7f9f4f7fe700 1 -- 172.21.15.2:0/2488937762 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f40017070 con 0x7f9f58102e20 2024-05-14T16:54:20.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.156+0000 7f9f5cf2f700 1 -- 172.21.15.2:0/2488937762 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f5819a3d0 con 0x7f9f58102e20 2024-05-14T16:54:20.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.156+0000 7f9f4f7fe700 1 -- 172.21.15.2:0/2488937762 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f40005020 con 0x7f9f58102e20 2024-05-14T16:54:20.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.156+0000 7f9f4f7fe700 1 -- 172.21.15.2:0/2488937762 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4000fa10 con 0x7f9f58102e20 2024-05-14T16:54:20.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.156+0000 7f9f5cf2f700 1 -- 172.21.15.2:0/2488937762 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f581a6690 con 0x7f9f58102e20 2024-05-14T16:54:20.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.157+0000 7f9f5cf2f700 1 -- 172.21.15.2:0/2488937762 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f5806a230 con 0x7f9f58102e20 2024-05-14T16:54:20.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.158+0000 7f9f4f7fe700 1 -- 172.21.15.2:0/2488937762 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f9f40004a30 con 0x7f9f58102e20 2024-05-14T16:54:20.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.159+0000 7f9f4f7fe700 1 --2- 172.21.15.2:0/2488937762 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f9f440b2880 0x7f9f440b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:20.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.159+0000 7f9f5659c700 1 --2- 172.21.15.2:0/2488937762 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f9f440b2880 0x7f9f440b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:20.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.159+0000 7f9f4f7fe700 1 -- 172.21.15.2:0/2488937762 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f9f400e24b0 con 0x7f9f58102e20 2024-05-14T16:54:20.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.160+0000 7f9f5659c700 1 --2- 172.21.15.2:0/2488937762 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f9f440b2880 0x7f9f440b4d40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9f5819aac0 tx=0x7f9f480079e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:20.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.163+0000 7f9f4f7fe700 1 -- 172.21.15.2:0/2488937762 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9f40020020 con 0x7f9f58102e20 2024-05-14T16:54:20.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.424+0000 7f9f5cf2f700 1 -- 172.21.15.2:0/2488937762 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f5819aeb0 con 0x7f9f58102e20 2024-05-14T16:54:20.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.424+0000 7f9f4f7fe700 1 -- 172.21.15.2:0/2488937762 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11850 (secure 0 0 0) 0x7f9f400aa1e0 con 0x7f9f58102e20 2024-05-14T16:54:20.424 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:54:20.424 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2275696923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2275696923},{"type":"v1","addr":"172.21.15.2:6835","nonce":2275696923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/716718761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":716718761},{"type":"v1","addr":"172.21.15.2:6839","nonce":716718761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/3644815179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3644815179},{"type":"v1","addr":"172.21.15.2:6841","nonce":3644815179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1880336731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1880336731},{"type":"v1","addr":"172.21.15.151:6833","nonce":1880336731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2415605548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2415605548},{"type":"v1","addr":"172.21.15.2:6837","nonce":2415605548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2635167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2635167},{"type":"v1","addr":"172.21.15.123:6833","nonce":2635167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:51:34.106919+0000","modified":"2024-05-14T16:51:53.879252+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24455,"mds_2":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24415":{"gid":24415,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/762044339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":762044339},{"type":"v1","addr":"172.21.15.151:6835","nonce":762044339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/132043151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":132043151},{"type":"v1","addr":"172.21.15.151:6837","nonce":132043151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/4107834375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":4107834375},{"type":"v1","addr":"172.21.15.151:6839","nonce":4107834375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.123:6835/1198670013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1198670013},{"type":"v1","addr":"172.21.15.123:6835","nonce":1198670013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/333426645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":333426645},{"type":"v1","addr":"172.21.15.123:6837","nonce":333426645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/580275877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":580275877},{"type":"v1","addr":"172.21.15.123:6839","nonce":580275877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24455,24479,24431]},"id":1}]} 2024-05-14T16:54:20.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.426+0000 7f9f5cf2f700 1 -- 172.21.15.2:0/2488937762 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f9f440b2880 msgr2=0x7f9f440b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:20.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.426+0000 7f9f5cf2f700 1 --2- 172.21.15.2:0/2488937762 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f9f440b2880 0x7f9f440b4d40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9f5819aac0 tx=0x7f9f480079e0 comp rx=0 tx=0).stop 2024-05-14T16:54:20.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.426+0000 7f9f5cf2f700 1 -- 172.21.15.2:0/2488937762 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f58102e20 msgr2=0x7f9f5819fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:20.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.426+0000 7f9f5cf2f700 1 --2- 172.21.15.2:0/2488937762 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f58102e20 0x7f9f5819fd00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9f4000d980 tx=0x7f9f4000dc90 comp rx=0 tx=0).stop 2024-05-14T16:54:20.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.427+0000 7f9f5cf2f700 1 -- 172.21.15.2:0/2488937762 shutdown_connections 2024-05-14T16:54:20.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.427+0000 7f9f5cf2f700 1 --2- 172.21.15.2:0/2488937762 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f581084f0 0x7f9f58199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:20.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.427+0000 7f9f5cf2f700 1 --2- 172.21.15.2:0/2488937762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f58102460 0x7f9f5819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:20.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.427+0000 7f9f5cf2f700 1 --2- 172.21.15.2:0/2488937762 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f9f440b2880 0x7f9f440b4d40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:20.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.427+0000 7f9f5cf2f700 1 --2- 172.21.15.2:0/2488937762 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f58102e20 0x7f9f5819fd00 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:20.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.427+0000 7f9f5cf2f700 1 -- 172.21.15.2:0/2488937762 >> 172.21.15.2:0/2488937762 conn(0x7f9f580794e0 msgr2=0x7f9f5806beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:20.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.427+0000 7f9f5cf2f700 1 -- 172.21.15.2:0/2488937762 shutdown_connections 2024-05-14T16:54:20.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:20.427+0000 7f9f5cf2f700 1 -- 172.21.15.2:0/2488937762 wait complete. 2024-05-14T16:54:20.428 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 24 2024-05-14T16:54:20.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:20 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/2488937762' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:54:20.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:20 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/2488937762' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:54:20.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:20 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/2488937762' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:54:21.190 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T16:54:21.442 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:54:21.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:21 smithi002 ceph-mon[31538]: pgmap v374: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T16:54:21.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:21 smithi151 ceph-mon[36066]: pgmap v374: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T16:54:21.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:21 smithi123 ceph-mon[35971]: pgmap v374: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T16:54:23.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.269+0000 7ff8b9faa700 1 -- 172.21.15.2:0/1450902296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8b41088f0 msgr2=0x7ff8b4110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:23.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.269+0000 7ff8b9faa700 1 --2- 172.21.15.2:0/1450902296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8b41088f0 0x7ff8b4110470 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac00b0d0 tx=0x7ff8ac00b3e0 comp rx=0 tx=0).stop 2024-05-14T16:54:23.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.269+0000 7ff8b9faa700 1 -- 172.21.15.2:0/1450902296 shutdown_connections 2024-05-14T16:54:23.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.269+0000 7ff8b9faa700 1 --2- 172.21.15.2:0/1450902296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8b41088f0 0x7ff8b4110470 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:23.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.269+0000 7ff8b9faa700 1 --2- 172.21.15.2:0/1450902296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8b41037f0 0x7ff8b4103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:23.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.269+0000 7ff8b9faa700 1 --2- 172.21.15.2:0/1450902296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8b41041a0 0x7ff8b41081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:23.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.270+0000 7ff8b9faa700 1 -- 172.21.15.2:0/1450902296 >> 172.21.15.2:0/1450902296 conn(0x7ff8b407a6a0 msgr2=0x7ff8b407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:23.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.270+0000 7ff8b9faa700 1 -- 172.21.15.2:0/1450902296 shutdown_connections 2024-05-14T16:54:23.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.270+0000 7ff8b9faa700 1 -- 172.21.15.2:0/1450902296 wait complete. 2024-05-14T16:54:23.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.270+0000 7ff8b9faa700 1 Processor -- start 2024-05-14T16:54:23.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.270+0000 7ff8b9faa700 1 -- start start 2024-05-14T16:54:23.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.271+0000 7ff8b9faa700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8b41037f0 0x7ff8b4075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:23.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.271+0000 7ff8b9faa700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8b41041a0 0x7ff8b4075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:23.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.271+0000 7ff8b9faa700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8b41088f0 0x7ff8b40762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:23.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.271+0000 7ff8b9faa700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b41136e0 con 0x7ff8b41088f0 2024-05-14T16:54:23.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.271+0000 7ff8b9faa700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b4113560 con 0x7ff8b41041a0 2024-05-14T16:54:23.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.271+0000 7ff8b9faa700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b4113860 con 0x7ff8b41037f0 2024-05-14T16:54:23.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.271+0000 7ff8b3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8b41088f0 0x7ff8b40762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:23.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.271+0000 7ff8b3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8b41088f0 0x7ff8b40762c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57188/0 (socket says 172.21.15.2:57188) 2024-05-14T16:54:23.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.271+0000 7ff8b3fff700 1 -- 172.21.15.2:0/710940009 learned_addr learned my addr 172.21.15.2:0/710940009 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:23.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.271+0000 7ff8b2ffd700 1 --2- 172.21.15.2:0/710940009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8b41041a0 0x7ff8b4075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:23.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.271+0000 7ff8b37fe700 1 --2- 172.21.15.2:0/710940009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8b41037f0 0x7ff8b4075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:23.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.271+0000 7ff8b3fff700 1 -- 172.21.15.2:0/710940009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8b41037f0 msgr2=0x7ff8b4075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:23.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.271+0000 7ff8b3fff700 1 --2- 172.21.15.2:0/710940009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8b41037f0 0x7ff8b4075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:23.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.271+0000 7ff8b3fff700 1 -- 172.21.15.2:0/710940009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8b41041a0 msgr2=0x7ff8b4075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:23.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.271+0000 7ff8b3fff700 1 --2- 172.21.15.2:0/710940009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8b41041a0 0x7ff8b4075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:23.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.271+0000 7ff8b3fff700 1 -- 172.21.15.2:0/710940009 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b41a8000 con 0x7ff8b41088f0 2024-05-14T16:54:23.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.271+0000 7ff8b2ffd700 1 --2- 172.21.15.2:0/710940009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8b41041a0 0x7ff8b4075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:54:23.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.271+0000 7ff8b3fff700 1 --2- 172.21.15.2:0/710940009 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8b41088f0 0x7ff8b40762c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac0095c0 tx=0x7ff8ac007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:23.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.271+0000 7ff8b0ff9700 1 -- 172.21.15.2:0/710940009 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac01f070 con 0x7ff8b41088f0 2024-05-14T16:54:23.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.272+0000 7ff8b9faa700 1 -- 172.21.15.2:0/710940009 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8b41a8290 con 0x7ff8b41088f0 2024-05-14T16:54:23.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.272+0000 7ff8b0ff9700 1 -- 172.21.15.2:0/710940009 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8ac003c00 con 0x7ff8b41088f0 2024-05-14T16:54:23.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.272+0000 7ff8b0ff9700 1 -- 172.21.15.2:0/710940009 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac01b6a0 con 0x7ff8b41088f0 2024-05-14T16:54:23.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.272+0000 7ff8b9faa700 1 -- 172.21.15.2:0/710940009 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8b41a8740 con 0x7ff8b41088f0 2024-05-14T16:54:23.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.272+0000 7ff8b9faa700 1 -- 172.21.15.2:0/710940009 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8b410bac0 con 0x7ff8b41088f0 2024-05-14T16:54:23.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.274+0000 7ff8b0ff9700 1 -- 172.21.15.2:0/710940009 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7ff8ac022070 con 0x7ff8b41088f0 2024-05-14T16:54:23.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.275+0000 7ff8b0ff9700 1 --2- 172.21.15.2:0/710940009 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7ff89c0b2880 0x7ff89c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:23.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.275+0000 7ff8b37fe700 1 --2- 172.21.15.2:0/710940009 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7ff89c0b2880 0x7ff89c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:23.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.275+0000 7ff8b0ff9700 1 -- 172.21.15.2:0/710940009 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7ff8ac00f020 con 0x7ff8b41088f0 2024-05-14T16:54:23.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.275+0000 7ff8b37fe700 1 --2- 172.21.15.2:0/710940009 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7ff89c0b2880 0x7ff89c0b4d40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4004630 tx=0x7ff8a400b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:23.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.278+0000 7ff8b0ff9700 1 -- 172.21.15.2:0/710940009 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff8ac029020 con 0x7ff8b41088f0 2024-05-14T16:54:23.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.492+0000 7ff8b9faa700 1 -- 172.21.15.2:0/710940009 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff8b410b4a0 con 0x7ff8b41088f0 2024-05-14T16:54:23.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.494+0000 7ff8b0ff9700 1 -- 172.21.15.2:0/710940009 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22575 (secure 0 0 0) 0x7ff8ac0ab5d0 con 0x7ff8b41088f0 2024-05-14T16:54:23.494 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:54:23.494 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":80,"fsid":"ed40717a-1210-11ef-bc98-c7b262605968","created":"2024-05-14T16:43:09.415298+0000","modified":"2024-05-14T16:52:13.205039+0000","last_up_change":"2024-05-14T16:50:29.787654+0000","last_in_change":"2024-05-14T16:50:09.287932+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T16:46:38.090903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T16:51:27.296955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T16:51:31.103019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"153f20ec-5bcf-4786-9ede-2e512a1e3d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6803","nonce":2654237820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6805","nonce":2654237820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6809","nonce":2654237820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6807","nonce":2654237820}]},"public_addr":"172.21.15.2:6803/2654237820","cluster_addr":"172.21.15.2:6805/2654237820","heartbeat_back_addr":"172.21.15.2:6809/2654237820","heartbeat_front_addr":"172.21.15.2:6807/2654237820","state":["exists","up"]},{"osd":1,"uuid":"0e95419f-9a9d-4f7a-b55d-ecb54971918a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6801","nonce":2668156992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6803","nonce":2668156992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6807","nonce":2668156992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6805","nonce":2668156992}]},"public_addr":"172.21.15.123:6801/2668156992","cluster_addr":"172.21.15.123:6803/2668156992","heartbeat_back_addr":"172.21.15.123:6807/2668156992","heartbeat_front_addr":"172.21.15.123:6805/2668156992","state":["exists","up"]},{"osd":2,"uuid":"b6ce7ac7-be6c-4549-a356-59cb81b5b492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6801","nonce":2160189636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6803","nonce":2160189636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6807","nonce":2160189636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6805","nonce":2160189636}]},"public_addr":"172.21.15.151:6801/2160189636","cluster_addr":"172.21.15.151:6803/2160189636","heartbeat_back_addr":"172.21.15.151:6807/2160189636","heartbeat_front_addr":"172.21.15.151:6805/2160189636","state":["exists","up"]},{"osd":3,"uuid":"2fd592bf-c212-411a-afbe-a7c498e5ac93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6811","nonce":3451424073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6813","nonce":3451424073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6817","nonce":3451424073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6815","nonce":3451424073}]},"public_addr":"172.21.15.2:6811/3451424073","cluster_addr":"172.21.15.2:6813/3451424073","heartbeat_back_addr":"172.21.15.2:6817/3451424073","heartbeat_front_addr":"172.21.15.2:6815/3451424073","state":["exists","up"]},{"osd":4,"uuid":"af1905e7-e036-47c6-be59-b085bc41fede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6809","nonce":2101935762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6811","nonce":2101935762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6815","nonce":2101935762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6813","nonce":2101935762}]},"public_addr":"172.21.15.123:6809/2101935762","cluster_addr":"172.21.15.123:6811/2101935762","heartbeat_back_addr":"172.21.15.123:6815/2101935762","heartbeat_front_addr":"172.21.15.123:6813/2101935762","state":["exists","up"]},{"osd":5,"uuid":"9c166916-af78-4cc2-bd14-d1220b6e7d9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6809","nonce":3695961886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6811","nonce":3695961886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6815","nonce":3695961886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6813","nonce":3695961886}]},"public_addr":"172.21.15.151:6809/3695961886","cluster_addr":"172.21.15.151:6811/3695961886","heartbeat_back_addr":"172.21.15.151:6815/3695961886","heartbeat_front_addr":"172.21.15.151:6813/3695961886","state":["exists","up"]},{"osd":6,"uuid":"60e9bf66-2338-41e8-9280-398d4e3b322f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6819","nonce":2711893600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6821","nonce":2711893600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6825","nonce":2711893600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6823","nonce":2711893600}]},"public_addr":"172.21.15.2:6819/2711893600","cluster_addr":"172.21.15.2:6821/2711893600","heartbeat_back_addr":"172.21.15.2:6825/2711893600","heartbeat_front_addr":"172.21.15.2:6823/2711893600","state":["exists","up"]},{"osd":7,"uuid":"c93e07a1-e941-493e-8468-33429df2dff0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6817","nonce":3634155507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6819","nonce":3634155507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6823","nonce":3634155507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6821","nonce":3634155507}]},"public_addr":"172.21.15.123:6817/3634155507","cluster_addr":"172.21.15.123:6819/3634155507","heartbeat_back_addr":"172.21.15.123:6823/3634155507","heartbeat_front_addr":"172.21.15.123:6821/3634155507","state":["exists","up"]},{"osd":8,"uuid":"1155fb58-d614-4f45-91e2-5da3a93e7d9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6817","nonce":4271107010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6819","nonce":4271107010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6823","nonce":4271107010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6821","nonce":4271107010}]},"public_addr":"172.21.15.151:6817/4271107010","cluster_addr":"172.21.15.151:6819/4271107010","heartbeat_back_addr":"172.21.15.151:6823/4271107010","heartbeat_front_addr":"172.21.15.151:6821/4271107010","state":["exists","up"]},{"osd":9,"uuid":"82b34e3b-0017-4ca4-8f86-0993ca78ce7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6827","nonce":3479431021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6829","nonce":3479431021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6833","nonce":3479431021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6831","nonce":3479431021}]},"public_addr":"172.21.15.2:6827/3479431021","cluster_addr":"172.21.15.2:6829/3479431021","heartbeat_back_addr":"172.21.15.2:6833/3479431021","heartbeat_front_addr":"172.21.15.2:6831/3479431021","state":["exists","up"]},{"osd":10,"uuid":"32e2ace9-eb5d-4258-a2ac-49b782be2b5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6825","nonce":4266374435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6827","nonce":4266374435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6831","nonce":4266374435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6829","nonce":4266374435}]},"public_addr":"172.21.15.123:6825/4266374435","cluster_addr":"172.21.15.123:6827/4266374435","heartbeat_back_addr":"172.21.15.123:6831/4266374435","heartbeat_front_addr":"172.21.15.123:6829/4266374435","state":["exists","up"]},{"osd":11,"uuid":"6e3c2ead-1af6-4bda-87cd-12770c81fa67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6825","nonce":4129194698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6827","nonce":4129194698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6831","nonce":4129194698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6829","nonce":4129194698}]},"public_addr":"172.21.15.151:6825/4129194698","cluster_addr":"172.21.15.151:6827/4129194698","heartbeat_back_addr":"172.21.15.151:6831/4129194698","heartbeat_front_addr":"172.21.15.151:6829/4129194698","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:45:46.540023+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:46:10.845186+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:46:34.283112+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:47:00.072467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:47:25.315644+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:47:49.748688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:48:16.964258+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:48:42.875515+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:49:08.596500+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:49:35.450943+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:50:01.293368+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:50:27.793713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/4267809161":"2024-05-15T16:44:11.998214+0000","172.21.15.2:6801/3582988344":"2024-05-15T16:44:11.998214+0000","172.21.15.2:6800/2574015037":"2024-05-15T16:43:43.407184+0000","172.21.15.2:0/2100732571":"2024-05-15T16:44:11.998214+0000","172.21.15.2:0/2954199324":"2024-05-15T16:43:43.407184+0000","172.21.15.2:6801/2574015037":"2024-05-15T16:43:43.407184+0000","172.21.15.2:0/3171550944":"2024-05-15T16:43:43.407184+0000","172.21.15.2:0/1679761136":"2024-05-15T16:44:11.998214+0000","172.21.15.2:0/1746766591":"2024-05-15T16:43:43.407184+0000","172.21.15.2:6800/3582988344":"2024-05-15T16:44:11.998214+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T16:54:23.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.496+0000 7ff8b9faa700 1 -- 172.21.15.2:0/710940009 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7ff89c0b2880 msgr2=0x7ff89c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:23.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.496+0000 7ff8b9faa700 1 --2- 172.21.15.2:0/710940009 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7ff89c0b2880 0x7ff89c0b4d40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4004630 tx=0x7ff8a400b3c0 comp rx=0 tx=0).stop 2024-05-14T16:54:23.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.496+0000 7ff8b9faa700 1 -- 172.21.15.2:0/710940009 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8b41088f0 msgr2=0x7ff8b40762c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:23.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.496+0000 7ff8b9faa700 1 --2- 172.21.15.2:0/710940009 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8b41088f0 0x7ff8b40762c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac0095c0 tx=0x7ff8ac007a20 comp rx=0 tx=0).stop 2024-05-14T16:54:23.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.496+0000 7ff8b9faa700 1 -- 172.21.15.2:0/710940009 shutdown_connections 2024-05-14T16:54:23.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.496+0000 7ff8b9faa700 1 --2- 172.21.15.2:0/710940009 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7ff89c0b2880 0x7ff89c0b4d40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:23.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.496+0000 7ff8b9faa700 1 --2- 172.21.15.2:0/710940009 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8b41088f0 0x7ff8b40762c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:23.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.496+0000 7ff8b9faa700 1 --2- 172.21.15.2:0/710940009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8b41037f0 0x7ff8b4075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:23.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.496+0000 7ff8b9faa700 1 --2- 172.21.15.2:0/710940009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8b41041a0 0x7ff8b4075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:23.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.496+0000 7ff8b9faa700 1 -- 172.21.15.2:0/710940009 >> 172.21.15.2:0/710940009 conn(0x7ff8b407a6a0 msgr2=0x7ff8b4109a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:23.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.496+0000 7ff8b9faa700 1 -- 172.21.15.2:0/710940009 shutdown_connections 2024-05-14T16:54:23.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:23.496+0000 7ff8b9faa700 1 -- 172.21.15.2:0/710940009 wait complete. 2024-05-14T16:54:23.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:23 smithi002 ceph-mon[31538]: pgmap v375: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:54:23.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:23 smithi151 ceph-mon[36066]: pgmap v375: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:54:23.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:23 smithi123 ceph-mon[35971]: pgmap v375: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:54:24.225 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-14T16:51:34.106919+0000', 'modified': '2024-05-14T16:51:53.879252+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24479, 'mds_1': 24455, 'mds_2': 24431}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24415': {'gid': 24415, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.151:6835/762044339', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6834', 'nonce': 762044339}, {'type': 'v1', 'addr': '172.21.15.151:6835', 'nonce': 762044339}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24421': {'gid': 24421, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.151:6837/132043151', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6836', 'nonce': 132043151}, {'type': 'v1', 'addr': '172.21.15.151:6837', 'nonce': 132043151}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24427': {'gid': 24427, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.151:6839/4107834375', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6838', 'nonce': 4107834375}, {'type': 'v1', 'addr': '172.21.15.151:6839', 'nonce': 4107834375}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24431': {'gid': 24431, 'name': 'e', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.123:6835/1198670013', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6834', 'nonce': 1198670013}, {'type': 'v1', 'addr': '172.21.15.123:6835', 'nonce': 1198670013}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'h', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.123:6837/333426645', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6836', 'nonce': 333426645}, {'type': 'v1', 'addr': '172.21.15.123:6837', 'nonce': 333426645}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.123:6839/580275877', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6838', 'nonce': 580275877}, {'type': 'v1', 'addr': '172.21.15.123:6839', 'nonce': 580275877}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24479, 'qdb_cluster': [24455, 24479, 24431]} 2024-05-14T16:54:24.225 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-14T16:54:24.225 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph tell mds.e status 2024-05-14T16:54:24.477 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:54:24.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:24 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/710940009' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:54:24.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:24 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/710940009' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:54:24.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:24 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/710940009' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:54:25.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:25 smithi002 ceph-mon[31538]: pgmap v376: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:54:25.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:25 smithi151 ceph-mon[36066]: pgmap v376: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:54:25.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:25 smithi123 ceph-mon[35971]: pgmap v376: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-14T16:54:26.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.267+0000 7f933da4f700 1 -- 172.21.15.2:0/4272507969 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9338103820 msgr2=0x7f9338103c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:26.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.267+0000 7f933da4f700 1 --2- 172.21.15.2:0/4272507969 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9338103820 0x7f9338103c00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f93280099c0 tx=0x7f9328009cd0 comp rx=0 tx=0).stop 2024-05-14T16:54:26.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.267+0000 7f933da4f700 1 -- 172.21.15.2:0/4272507969 shutdown_connections 2024-05-14T16:54:26.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.267+0000 7f933da4f700 1 --2- 172.21.15.2:0/4272507969 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9338108910 0x7f9338110490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:26.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.267+0000 7f933da4f700 1 --2- 172.21.15.2:0/4272507969 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9338103820 0x7f9338103c00 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:26.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.267+0000 7f933da4f700 1 --2- 172.21.15.2:0/4272507969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93381041d0 0x7f93381081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:26.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.267+0000 7f933da4f700 1 -- 172.21.15.2:0/4272507969 >> 172.21.15.2:0/4272507969 conn(0x7f933807a6b0 msgr2=0x7f933807aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:26.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.267+0000 7f933da4f700 1 -- 172.21.15.2:0/4272507969 shutdown_connections 2024-05-14T16:54:26.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.267+0000 7f933da4f700 1 -- 172.21.15.2:0/4272507969 wait complete. 2024-05-14T16:54:26.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.267+0000 7f933da4f700 1 Processor -- start 2024-05-14T16:54:26.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.268+0000 7f933da4f700 1 -- start start 2024-05-14T16:54:26.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.268+0000 7f933da4f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9338103820 0x7f9338075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:26.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.268+0000 7f933da4f700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f93381041d0 0x7f9338075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:26.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.268+0000 7f933da4f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9338108910 0x7f93380762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:26.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.268+0000 7f933da4f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9338113750 con 0x7f9338103820 2024-05-14T16:54:26.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.268+0000 7f933da4f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f93381135d0 con 0x7f9338108910 2024-05-14T16:54:26.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.268+0000 7f933da4f700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f93381138d0 con 0x7f93381041d0 2024-05-14T16:54:26.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.268+0000 7f9336ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9338103820 0x7f9338075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:26.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.268+0000 7f9336ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9338103820 0x7f9338075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49052/0 (socket says 172.21.15.2:49052) 2024-05-14T16:54:26.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.268+0000 7f9336ffd700 1 -- 172.21.15.2:0/2545699048 learned_addr learned my addr 172.21.15.2:0/2545699048 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:26.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.268+0000 7f93377fe700 1 --2- 172.21.15.2:0/2545699048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9338108910 0x7f93380762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:26.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.268+0000 7f93367fc700 1 --2- 172.21.15.2:0/2545699048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f93381041d0 0x7f9338075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:26.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.268+0000 7f9336ffd700 1 -- 172.21.15.2:0/2545699048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f93381041d0 msgr2=0x7f9338075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:26.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.268+0000 7f9336ffd700 1 --2- 172.21.15.2:0/2545699048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f93381041d0 0x7f9338075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:26.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.268+0000 7f9336ffd700 1 -- 172.21.15.2:0/2545699048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9338108910 msgr2=0x7f93380762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:26.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.268+0000 7f9336ffd700 1 --2- 172.21.15.2:0/2545699048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9338108910 0x7f93380762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:26.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.268+0000 7f9336ffd700 1 -- 172.21.15.2:0/2545699048 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93381a80e0 con 0x7f9338103820 2024-05-14T16:54:26.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.268+0000 7f93367fc700 1 --2- 172.21.15.2:0/2545699048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f93381041d0 0x7f9338075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:54:26.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.269+0000 7f9336ffd700 1 --2- 172.21.15.2:0/2545699048 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9338103820 0x7f9338075840 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9328009990 tx=0x7f93280047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:26.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.269+0000 7f933ca4d700 1 -- 172.21.15.2:0/2545699048 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9328026070 con 0x7f9338103820 2024-05-14T16:54:26.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.269+0000 7f933da4f700 1 -- 172.21.15.2:0/2545699048 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93381a8370 con 0x7f9338103820 2024-05-14T16:54:26.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.269+0000 7f933ca4d700 1 -- 172.21.15.2:0/2545699048 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9328003f20 con 0x7f9338103820 2024-05-14T16:54:26.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.269+0000 7f933ca4d700 1 -- 172.21.15.2:0/2545699048 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932800fcc0 con 0x7f9338103820 2024-05-14T16:54:26.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.269+0000 7f933da4f700 1 -- 172.21.15.2:0/2545699048 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93381a8820 con 0x7f9338103820 2024-05-14T16:54:26.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.272+0000 7f933ca4d700 1 -- 172.21.15.2:0/2545699048 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f9328029070 con 0x7f9338103820 2024-05-14T16:54:26.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.273+0000 7f933ca4d700 1 --2- 172.21.15.2:0/2545699048 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f93200b2880 0x7f93200b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:26.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.273+0000 7f93367fc700 1 --2- 172.21.15.2:0/2545699048 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f93200b2880 0x7f93200b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:26.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.273+0000 7f93482e5b80 1 Processor -- start 2024-05-14T16:54:26.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.273+0000 7f933ca4d700 1 -- 172.21.15.2:0/2545699048 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f93280e2c00 con 0x7f9338103820 2024-05-14T16:54:26.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.273+0000 7f93482e5b80 1 -- start start 2024-05-14T16:54:26.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.273+0000 7f93482e5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7ee9082f0 0x55e7ee9086d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:26.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.273+0000 7f93482e5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7ee908c10 0x55e7ee90c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:26.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.273+0000 7f93482e5b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e7ee90c910 0x55e7ee90cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:26.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.273+0000 7f93482e5b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e7ee866c40 con 0x55e7ee90c910 2024-05-14T16:54:26.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.273+0000 7f93482e5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e7ee8567e0 con 0x55e7ee908c10 2024-05-14T16:54:26.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.273+0000 7f93482e5b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e7ee85a820 con 0x55e7ee9082f0 2024-05-14T16:54:26.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.273+0000 7f93367fc700 1 --2- 172.21.15.2:0/2545699048 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f93200b2880 0x7f93200b4d40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f932c005950 tx=0x7f932c0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:26.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.274+0000 7f93377fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e7ee90c910 0x55e7ee90cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:26.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.274+0000 7f93367fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7ee908c10 0x55e7ee90c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:26.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.274+0000 7f93377fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e7ee90c910 0x55e7ee90cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49058/0 (socket says 172.21.15.2:49058) 2024-05-14T16:54:26.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.274+0000 7f93377fe700 1 -- 172.21.15.2:0/3339599760 learned_addr learned my addr 172.21.15.2:0/3339599760 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:26.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.274+0000 7f9336ffd700 1 --2- 172.21.15.2:0/3339599760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7ee9082f0 0x55e7ee9086d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:26.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.274+0000 7f93367fc700 1 -- 172.21.15.2:0/3339599760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7ee9082f0 msgr2=0x55e7ee9086d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:26.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.274+0000 7f93367fc700 1 --2- 172.21.15.2:0/3339599760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7ee9082f0 0x55e7ee9086d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:26.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.274+0000 7f93367fc700 1 -- 172.21.15.2:0/3339599760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e7ee90c910 msgr2=0x55e7ee90cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:26.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.274+0000 7f93367fc700 1 --2- 172.21.15.2:0/3339599760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e7ee90c910 0x55e7ee90cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:26.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.274+0000 7f93367fc700 1 -- 172.21.15.2:0/3339599760 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7ee7a6880 con 0x55e7ee908c10 2024-05-14T16:54:26.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.274+0000 7f93367fc700 1 --2- 172.21.15.2:0/3339599760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7ee908c10 0x55e7ee90c3d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f932c00b5d0 tx=0x7f932c012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=294eb49437289c73 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:26.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.274+0000 7f931dffb700 1 -- 172.21.15.2:0/3339599760 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932c003880 con 0x55e7ee908c10 2024-05-14T16:54:26.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.274+0000 7f931dffb700 1 -- 172.21.15.2:0/3339599760 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f932c024070 con 0x55e7ee908c10 2024-05-14T16:54:26.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.274+0000 7f931dffb700 1 -- 172.21.15.2:0/3339599760 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932c01fbe0 con 0x55e7ee908c10 2024-05-14T16:54:26.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.275+0000 7f93482e5b80 1 -- 172.21.15.2:0/3339599760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7ee908c10 msgr2=0x55e7ee90c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:26.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.275+0000 7f93482e5b80 1 --2- 172.21.15.2:0/3339599760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7ee908c10 0x55e7ee90c3d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f932c00b5d0 tx=0x7f932c012000 comp rx=0 tx=0).stop 2024-05-14T16:54:26.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.275+0000 7f93482e5b80 1 -- 172.21.15.2:0/3339599760 shutdown_connections 2024-05-14T16:54:26.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.275+0000 7f93482e5b80 1 --2- 172.21.15.2:0/3339599760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e7ee90c910 0x55e7ee90cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:26.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.275+0000 7f93482e5b80 1 --2- 172.21.15.2:0/3339599760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7ee9082f0 0x55e7ee9086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:26.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.275+0000 7f93482e5b80 1 --2- 172.21.15.2:0/3339599760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7ee908c10 0x55e7ee90c3d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:26.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.275+0000 7f93482e5b80 1 -- 172.21.15.2:0/3339599760 >> 172.21.15.2:0/3339599760 conn(0x55e7ee833200 msgr2=0x55e7ee902930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:26.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.275+0000 7f93482e5b80 1 -- 172.21.15.2:0/3339599760 shutdown_connections 2024-05-14T16:54:26.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.275+0000 7f93482e5b80 1 -- 172.21.15.2:0/3339599760 wait complete. 2024-05-14T16:54:26.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.275+0000 7f93482e5b80 1 -- start start 2024-05-14T16:54:26.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.276+0000 7f93482e5b80 1 Processor -- start 2024-05-14T16:54:26.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.276+0000 7f93482e5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7ee9082f0 0x55e7ee913df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:26.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.276+0000 7f93482e5b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e7ee908c10 0x55e7ee911190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:26.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.276+0000 7f93482e5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7ee90c910 0x55e7ee9116d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:26.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.276+0000 7f93482e5b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e7ee75cf50 con 0x55e7ee908c10 2024-05-14T16:54:26.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.276+0000 7f93482e5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e7ee7f8770 con 0x55e7ee9082f0 2024-05-14T16:54:26.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.276+0000 7f93482e5b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e7ee914b70 con 0x55e7ee90c910 2024-05-14T16:54:26.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.276+0000 7f93482e5b80 10 client.0 _ll_register_callbacks cb 0x55e7ee7ed530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:54:26.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.276+0000 7f93367fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e7ee908c10 0x55e7ee911190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:26.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.276+0000 7f93367fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e7ee908c10 0x55e7ee911190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49064/0 (socket says 172.21.15.2:49064) 2024-05-14T16:54:26.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.276+0000 7f9336ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7ee9082f0 0x55e7ee913df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:26.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.276+0000 7f93367fc700 1 -- 172.21.15.2:0/694014769 learned_addr learned my addr 172.21.15.2:0/694014769 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:26.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.276+0000 7f93377fe700 1 --2- 172.21.15.2:0/694014769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7ee90c910 0x55e7ee9116d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:26.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.276+0000 7f93367fc700 1 -- 172.21.15.2:0/694014769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7ee90c910 msgr2=0x55e7ee9116d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:26.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.276+0000 7f93367fc700 1 --2- 172.21.15.2:0/694014769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7ee90c910 0x55e7ee9116d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:26.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.276+0000 7f93367fc700 1 -- 172.21.15.2:0/694014769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7ee9082f0 msgr2=0x55e7ee913df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:26.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.276+0000 7f93367fc700 1 --2- 172.21.15.2:0/694014769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7ee9082f0 0x55e7ee913df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:26.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.276+0000 7f93367fc700 1 -- 172.21.15.2:0/694014769 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7ee7f5640 con 0x55e7ee908c10 2024-05-14T16:54:26.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.276+0000 7f9336ffd700 1 --2- 172.21.15.2:0/694014769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7ee9082f0 0x55e7ee913df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:54:26.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.277+0000 7f93367fc700 1 --2- 172.21.15.2:0/694014769 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e7ee908c10 0x55e7ee911190 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f932c00b9b0 tx=0x7f932c01f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c434315d6d2335 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:26.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.277+0000 7f9316ffd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:54:26.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.277+0000 7f9316ffd700 1 -- 172.21.15.2:0/694014769 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932c020c80 con 0x55e7ee908c10 2024-05-14T16:54:26.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.277+0000 7f9316ffd700 1 -- 172.21.15.2:0/694014769 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f932c020e20 con 0x55e7ee908c10 2024-05-14T16:54:26.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.277+0000 7f93482e5b80 1 -- 172.21.15.2:0/694014769 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f932c024070 con 0x55e7ee908c10 2024-05-14T16:54:26.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.277+0000 7f9316ffd700 1 -- 172.21.15.2:0/694014769 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f932c003680 con 0x55e7ee908c10 2024-05-14T16:54:26.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.277+0000 7f93482e5b80 10 client.15054 fetch_fsmap learned FSMap version 24 2024-05-14T16:54:26.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.277+0000 7f93482e5b80 1 -- 172.21.15.2:0/694014769 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e7ee91f2a0 con 0x55e7ee908c10 2024-05-14T16:54:26.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.277+0000 7f9316ffd700 1 -- 172.21.15.2:0/694014769 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f932c024070 con 0x55e7ee908c10 2024-05-14T16:54:26.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.277+0000 7f9316ffd700 1 -- 172.21.15.2:0/694014769 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f932c032070 con 0x55e7ee908c10 2024-05-14T16:54:26.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.277+0000 7f93482e5b80 10 client.15054 fetch_fsmap finished waiting for FSMap version 24 2024-05-14T16:54:26.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.278+0000 7f93482e5b80 10 client.15054 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-14T16:54:26.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.278+0000 7f93482e5b80 20 client.15054 populate_metadata read hostname 'smithi002' 2024-05-14T16:54:26.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.278+0000 7f93482e5b80 1 --2- 172.21.15.2:0/694014769 >> [v2:172.21.15.123:6834/1198670013,v1:172.21.15.123:6835/1198670013] conn(0x55e7ee921670 0x55e7ee922ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:26.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.278+0000 7f93482e5b80 4 client.15054 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:54:26.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.278+0000 7f93482e5b80 1 -- 172.21.15.2:0/694014769 --> [v2:172.21.15.123:6834/1198670013,v1:172.21.15.123:6835/1198670013] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7ee8d4310 con 0x55e7ee921670 2024-05-14T16:54:26.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.278+0000 7f9336ffd700 1 --2- 172.21.15.2:0/694014769 >> [v2:172.21.15.123:6834/1198670013,v1:172.21.15.123:6835/1198670013] conn(0x55e7ee921670 0x55e7ee922ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:26.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.278+0000 7f9336ffd700 1 --2- 172.21.15.2:0/694014769 >> [v2:172.21.15.123:6834/1198670013,v1:172.21.15.123:6835/1198670013] conn(0x55e7ee921670 0x55e7ee922ae0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7a05847f186750c4 server_cookie=686bf9a06ff4fd40 in_seq=0 out_seq=0 2024-05-14T16:54:26.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.278+0000 7f9316ffd700 10 client.15054 ms_handle_connect on v2:172.21.15.123:6834/1198670013 2024-05-14T16:54:26.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.279+0000 7f9316ffd700 1 -- 172.21.15.2:0/694014769 <== mds.2 v2:172.21.15.123:6834/1198670013 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e7ee8d4310 con 0x55e7ee921670 2024-05-14T16:54:26.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.279+0000 7f9316ffd700 10 client.15054 handle_command_reply: tid=0 2024-05-14T16:54:26.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.279+0000 7f93482e5b80 1 client.15054 shutdown 2024-05-14T16:54:26.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.279+0000 7f93482e5b80 1 -- 172.21.15.2:0/694014769 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e7ee908c10 msgr2=0x55e7ee911190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:26.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.279+0000 7f93482e5b80 1 --2- 172.21.15.2:0/694014769 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e7ee908c10 0x55e7ee911190 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f932c00b9b0 tx=0x7f932c01f740 comp rx=0 tx=0).stop 2024-05-14T16:54:26.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.279+0000 7f93482e5b80 1 -- 172.21.15.2:0/694014769 shutdown_connections 2024-05-14T16:54:26.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.279+0000 7f93482e5b80 1 --2- 172.21.15.2:0/694014769 >> [v2:172.21.15.123:6834/1198670013,v1:172.21.15.123:6835/1198670013] conn(0x55e7ee921670 0x55e7ee922ae0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:26.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.279+0000 7f93482e5b80 1 --2- 172.21.15.2:0/694014769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7ee90c910 0x55e7ee9116d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:26.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.279+0000 7f93482e5b80 1 --2- 172.21.15.2:0/694014769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7ee9082f0 0x55e7ee913df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:26.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.279+0000 7f93482e5b80 1 --2- 172.21.15.2:0/694014769 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e7ee908c10 0x55e7ee911190 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:26.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.279+0000 7f93482e5b80 1 -- 172.21.15.2:0/694014769 >> 172.21.15.2:0/694014769 conn(0x55e7ee833200 msgr2=0x55e7ee9028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:26.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.279+0000 7f9316ffd700 0 client.15054 ms_handle_reset on v2:172.21.15.123:6834/1198670013 2024-05-14T16:54:26.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.280+0000 7f93482e5b80 1 -- 172.21.15.2:0/694014769 shutdown_connections 2024-05-14T16:54:26.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.280+0000 7f93482e5b80 1 -- 172.21.15.2:0/694014769 wait complete. 2024-05-14T16:54:26.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.280+0000 7f93482e5b80 20 client.15054 trim_cache size 0 max 16384 2024-05-14T16:54:26.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.301+0000 7f93482e5b80 1 Processor -- start 2024-05-14T16:54:26.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.301+0000 7f93482e5b80 1 -- start start 2024-05-14T16:54:26.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.301+0000 7f93482e5b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e7ee9082f0 0x55e7ee911700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:26.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.301+0000 7f93482e5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7ee908c10 0x55e7ee8fcc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:26.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.301+0000 7f93482e5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7ee90c910 0x55e7ee90a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:26.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.301+0000 7f93482e5b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e7ee920e20 con 0x55e7ee9082f0 2024-05-14T16:54:26.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.301+0000 7f93482e5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e7ee9214f0 con 0x55e7ee90c910 2024-05-14T16:54:26.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.301+0000 7f93482e5b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e7ee921bc0 con 0x55e7ee908c10 2024-05-14T16:54:26.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.301+0000 7f9336ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e7ee9082f0 0x55e7ee911700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:26.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.301+0000 7f9336ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e7ee9082f0 0x55e7ee911700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49070/0 (socket says 172.21.15.2:49070) 2024-05-14T16:54:26.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.301+0000 7f9336ffd700 1 -- 172.21.15.2:0/1775320481 learned_addr learned my addr 172.21.15.2:0/1775320481 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:26.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.301+0000 7f93367fc700 1 --2- 172.21.15.2:0/1775320481 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7ee908c10 0x55e7ee8fcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:26.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.301+0000 7f93377fe700 1 --2- 172.21.15.2:0/1775320481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7ee90c910 0x55e7ee90a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:26.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.301+0000 7f9336ffd700 1 -- 172.21.15.2:0/1775320481 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7ee908c10 msgr2=0x55e7ee8fcc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:26.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.301+0000 7f9336ffd700 1 --2- 172.21.15.2:0/1775320481 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7ee908c10 0x55e7ee8fcc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:26.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.301+0000 7f9336ffd700 1 -- 172.21.15.2:0/1775320481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7ee90c910 msgr2=0x55e7ee90a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:26.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.301+0000 7f9336ffd700 1 --2- 172.21.15.2:0/1775320481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7ee90c910 0x55e7ee90a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:26.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.301+0000 7f9336ffd700 1 -- 172.21.15.2:0/1775320481 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7ee912070 con 0x55e7ee9082f0 2024-05-14T16:54:26.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.301+0000 7f93377fe700 1 --2- 172.21.15.2:0/1775320481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7ee90c910 0x55e7ee90a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:54:26.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.301+0000 7f9336ffd700 1 --2- 172.21.15.2:0/1775320481 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e7ee9082f0 0x55e7ee911700 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9328000c00 tx=0x7f932800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74e9f75b8537131e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:26.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.302+0000 7f931d7fa700 1 -- 172.21.15.2:0/1775320481 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93280ae610 con 0x55e7ee9082f0 2024-05-14T16:54:26.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.302+0000 7f931d7fa700 1 -- 172.21.15.2:0/1775320481 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93280b3030 con 0x55e7ee9082f0 2024-05-14T16:54:26.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.302+0000 7f931d7fa700 1 -- 172.21.15.2:0/1775320481 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93280b2e00 con 0x55e7ee9082f0 2024-05-14T16:54:26.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.302+0000 7f93482e5b80 1 -- 172.21.15.2:0/1775320481 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e7ee9082f0 msgr2=0x55e7ee911700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:26.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.302+0000 7f93482e5b80 1 --2- 172.21.15.2:0/1775320481 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e7ee9082f0 0x55e7ee911700 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9328000c00 tx=0x7f932800f790 comp rx=0 tx=0).stop 2024-05-14T16:54:26.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.302+0000 7f93482e5b80 1 -- 172.21.15.2:0/1775320481 shutdown_connections 2024-05-14T16:54:26.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.302+0000 7f93482e5b80 1 --2- 172.21.15.2:0/1775320481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7ee90c910 0x55e7ee90a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:26.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.302+0000 7f93482e5b80 1 --2- 172.21.15.2:0/1775320481 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e7ee9082f0 0x55e7ee911700 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:26.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.302+0000 7f93482e5b80 1 --2- 172.21.15.2:0/1775320481 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7ee908c10 0x55e7ee8fcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:26.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.302+0000 7f93482e5b80 1 -- 172.21.15.2:0/1775320481 >> 172.21.15.2:0/1775320481 conn(0x55e7ee833200 msgr2=0x55e7ee8fc530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:26.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.302+0000 7f93482e5b80 1 -- 172.21.15.2:0/1775320481 shutdown_connections 2024-05-14T16:54:26.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.302+0000 7f93482e5b80 1 -- 172.21.15.2:0/1775320481 wait complete. 2024-05-14T16:54:26.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.303+0000 7f93482e5b80 1 -- start start 2024-05-14T16:54:26.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.303+0000 7f93482e5b80 1 Processor -- start 2024-05-14T16:54:26.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.303+0000 7f93482e5b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e7ee9082f0 0x55e7ee9096b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:26.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.303+0000 7f93482e5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7ee908c10 0x55e7ee909bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:26.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.303+0000 7f93482e5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7ee90c910 0x55e7ee90a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:26.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.303+0000 7f93482e5b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e7ee922960 con 0x55e7ee9082f0 2024-05-14T16:54:26.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.303+0000 7f93482e5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e7ee922290 con 0x55e7ee908c10 2024-05-14T16:54:26.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.303+0000 7f93482e5b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e7ee923030 con 0x55e7ee90c910 2024-05-14T16:54:26.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.303+0000 7f93482e5b80 10 client.0 _ll_register_callbacks cb 0x55e7ee7ed530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:54:26.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.303+0000 7f9336ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e7ee9082f0 0x55e7ee9096b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:26.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.303+0000 7f9336ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e7ee9082f0 0x55e7ee9096b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49078/0 (socket says 172.21.15.2:49078) 2024-05-14T16:54:26.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.303+0000 7f9336ffd700 1 -- 172.21.15.2:0/3675131644 learned_addr learned my addr 172.21.15.2:0/3675131644 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:26.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.303+0000 7f93377fe700 1 --2- 172.21.15.2:0/3675131644 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7ee90c910 0x55e7ee90a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:26.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.304+0000 7f93367fc700 1 --2- 172.21.15.2:0/3675131644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7ee908c10 0x55e7ee909bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:26.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.304+0000 7f9336ffd700 1 -- 172.21.15.2:0/3675131644 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7ee90c910 msgr2=0x55e7ee90a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:26.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.304+0000 7f9336ffd700 1 --2- 172.21.15.2:0/3675131644 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7ee90c910 0x55e7ee90a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:26.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.304+0000 7f9336ffd700 1 -- 172.21.15.2:0/3675131644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7ee908c10 msgr2=0x55e7ee909bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:26.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.304+0000 7f9336ffd700 1 --2- 172.21.15.2:0/3675131644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7ee908c10 0x55e7ee909bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:26.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.304+0000 7f9336ffd700 1 -- 172.21.15.2:0/3675131644 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7ee92b200 con 0x55e7ee9082f0 2024-05-14T16:54:26.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.304+0000 7f9336ffd700 1 --2- 172.21.15.2:0/3675131644 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e7ee9082f0 0x55e7ee9096b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f93280aa8c0 tx=0x7f93280a9e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c13cbe5370f6d3b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:26.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.304+0000 7f9316ffd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:54:26.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.304+0000 7f9316ffd700 1 -- 172.21.15.2:0/3675131644 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93280040d0 con 0x55e7ee9082f0 2024-05-14T16:54:26.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.304+0000 7f9316ffd700 1 -- 172.21.15.2:0/3675131644 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f93280b2ba0 con 0x55e7ee9082f0 2024-05-14T16:54:26.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.304+0000 7f93482e5b80 1 -- 172.21.15.2:0/3675131644 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93280b3030 con 0x55e7ee9082f0 2024-05-14T16:54:26.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.304+0000 7f9316ffd700 1 -- 172.21.15.2:0/3675131644 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f93280b2da0 con 0x55e7ee9082f0 2024-05-14T16:54:26.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.304+0000 7f93482e5b80 10 client.15060 fetch_fsmap learned FSMap version 24 2024-05-14T16:54:26.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.304+0000 7f93482e5b80 1 -- 172.21.15.2:0/3675131644 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e7ee92c450 con 0x55e7ee9082f0 2024-05-14T16:54:26.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.304+0000 7f9316ffd700 1 -- 172.21.15.2:0/3675131644 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93280b3030 con 0x55e7ee9082f0 2024-05-14T16:54:26.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.305+0000 7f9316ffd700 1 -- 172.21.15.2:0/3675131644 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f93280a94c0 con 0x55e7ee9082f0 2024-05-14T16:54:26.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.305+0000 7f93482e5b80 10 client.15060 fetch_fsmap finished waiting for FSMap version 24 2024-05-14T16:54:26.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.305+0000 7f93482e5b80 10 client.15060 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-14T16:54:26.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.305+0000 7f93482e5b80 20 client.15060 populate_metadata read hostname 'smithi002' 2024-05-14T16:54:26.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.305+0000 7f93482e5b80 1 --2- 172.21.15.2:0/3675131644 >> [v2:172.21.15.123:6834/1198670013,v1:172.21.15.123:6835/1198670013] conn(0x55e7ee990dd0 0x55e7ee98fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:26.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.305+0000 7f93482e5b80 4 client.15060 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T16:54:26.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.305+0000 7f93482e5b80 1 -- 172.21.15.2:0/3675131644 --> [v2:172.21.15.123:6834/1198670013,v1:172.21.15.123:6835/1198670013] -- command(tid 0: {"prefix": "status"}) -- 0x55e7ee868b90 con 0x55e7ee990dd0 2024-05-14T16:54:26.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.305+0000 7f93367fc700 1 --2- 172.21.15.2:0/3675131644 >> [v2:172.21.15.123:6834/1198670013,v1:172.21.15.123:6835/1198670013] conn(0x55e7ee990dd0 0x55e7ee98fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:26.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.305+0000 7f93367fc700 1 --2- 172.21.15.2:0/3675131644 >> [v2:172.21.15.123:6834/1198670013,v1:172.21.15.123:6835/1198670013] conn(0x55e7ee990dd0 0x55e7ee98fae0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5040798a30d618ce server_cookie=f1abf1bf586ca9e3 in_seq=0 out_seq=0 2024-05-14T16:54:26.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.305+0000 7f9316ffd700 10 client.15060 ms_handle_connect on v2:172.21.15.123:6834/1198670013 2024-05-14T16:54:26.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.306+0000 7f9316ffd700 1 -- 172.21.15.2:0/3675131644 <== mds.2 v2:172.21.15.123:6834/1198670013 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55e7ee868b90 con 0x55e7ee990dd0 2024-05-14T16:54:26.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.306+0000 7f9316ffd700 10 client.15060 handle_command_reply: tid=0 2024-05-14T16:54:26.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.306+0000 7f93482e5b80 1 client.15060 shutdown 2024-05-14T16:54:26.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.306+0000 7f93482e5b80 1 -- 172.21.15.2:0/3675131644 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e7ee9082f0 msgr2=0x55e7ee9096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:26.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.306+0000 7f93482e5b80 1 --2- 172.21.15.2:0/3675131644 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e7ee9082f0 0x55e7ee9096b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f93280aa8c0 tx=0x7f93280a9e60 comp rx=0 tx=0).stop 2024-05-14T16:54:26.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.306+0000 7f93482e5b80 1 -- 172.21.15.2:0/3675131644 shutdown_connections 2024-05-14T16:54:26.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.306+0000 7f93482e5b80 1 --2- 172.21.15.2:0/3675131644 >> [v2:172.21.15.123:6834/1198670013,v1:172.21.15.123:6835/1198670013] conn(0x55e7ee990dd0 0x55e7ee98fae0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:26.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.306+0000 7f93482e5b80 1 --2- 172.21.15.2:0/3675131644 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7ee90c910 0x55e7ee90a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:26.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.306+0000 7f93482e5b80 1 --2- 172.21.15.2:0/3675131644 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e7ee9082f0 0x55e7ee9096b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:26.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.306+0000 7f93482e5b80 1 --2- 172.21.15.2:0/3675131644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7ee908c10 0x55e7ee909bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:26.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.306+0000 7f93482e5b80 1 -- 172.21.15.2:0/3675131644 >> 172.21.15.2:0/3675131644 conn(0x55e7ee833200 msgr2=0x55e7ee8fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:26.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.306+0000 7f9316ffd700 0 client.15060 ms_handle_reset on v2:172.21.15.123:6834/1198670013 2024-05-14T16:54:26.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.306+0000 7f93482e5b80 1 -- 172.21.15.2:0/3675131644 shutdown_connections 2024-05-14T16:54:26.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.306+0000 7f93482e5b80 1 -- 172.21.15.2:0/3675131644 wait complete. 2024-05-14T16:54:26.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.306+0000 7f93482e5b80 20 client.15060 trim_cache size 0 max 16384 2024-05-14T16:54:26.314 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T16:54:26.314 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "ed40717a-1210-11ef-bc98-c7b262605968", 2024-05-14T16:54:26.314 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 2, 2024-05-14T16:54:26.314 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24431, 2024-05-14T16:54:26.314 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2024-05-14T16:54:26.314 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2024-05-14T16:54:26.314 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2024-05-14T16:54:26.314 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 158.328055328, 2024-05-14T16:54:26.314 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 24, 2024-05-14T16:54:26.314 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 80, 2024-05-14T16:54:26.314 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 78, 2024-05-14T16:54:26.315 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 209.41102973900001 2024-05-14T16:54:26.315 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T16:54:26.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.307+0000 7f933da4f700 1 -- 172.21.15.2:0/2545699048 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f93200b2880 msgr2=0x7f93200b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:26.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.307+0000 7f933da4f700 1 --2- 172.21.15.2:0/2545699048 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f93200b2880 0x7f93200b4d40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f932c005950 tx=0x7f932c0058e0 comp rx=0 tx=0).stop 2024-05-14T16:54:26.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.307+0000 7f933da4f700 1 -- 172.21.15.2:0/2545699048 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9338103820 msgr2=0x7f9338075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:26.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.307+0000 7f933da4f700 1 --2- 172.21.15.2:0/2545699048 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9338103820 0x7f9338075840 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9328009990 tx=0x7f93280047b0 comp rx=0 tx=0).stop 2024-05-14T16:54:26.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.307+0000 7f933da4f700 1 -- 172.21.15.2:0/2545699048 shutdown_connections 2024-05-14T16:54:26.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.307+0000 7f933da4f700 1 --2- 172.21.15.2:0/2545699048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9338108910 0x7f93380762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:26.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.307+0000 7f933da4f700 1 --2- 172.21.15.2:0/2545699048 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f93200b2880 0x7f93200b4d40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:26.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.307+0000 7f933da4f700 1 --2- 172.21.15.2:0/2545699048 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9338103820 0x7f9338075840 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:26.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.307+0000 7f933da4f700 1 --2- 172.21.15.2:0/2545699048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f93381041d0 0x7f9338075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:26.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.307+0000 7f933da4f700 1 -- 172.21.15.2:0/2545699048 >> 172.21.15.2:0/2545699048 conn(0x7f933807a6b0 msgr2=0x7f9338101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:26.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.307+0000 7f933da4f700 1 -- 172.21.15.2:0/2545699048 shutdown_connections 2024-05-14T16:54:26.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:26.307+0000 7f933da4f700 1 -- 172.21.15.2:0/2545699048 wait complete. 2024-05-14T16:54:26.953 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph tell mds.h status 2024-05-14T16:54:27.207 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:54:27.649 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:27 smithi002 ceph-mon[31538]: pgmap v377: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:54:27.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:27 smithi151 ceph-mon[36066]: pgmap v377: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:54:27.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:27 smithi123 ceph-mon[35971]: pgmap v377: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T16:54:28.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.932+0000 7f503613c700 1 -- 172.21.15.2:0/1297016556 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5030102430 msgr2=0x7f50301028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:28.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.932+0000 7f503613c700 1 --2- 172.21.15.2:0/1297016556 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5030102430 0x7f50301028b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5024009990 tx=0x7f5024009ca0 comp rx=0 tx=0).stop 2024-05-14T16:54:28.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.932+0000 7f503613c700 1 -- 172.21.15.2:0/1297016556 shutdown_connections 2024-05-14T16:54:28.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.932+0000 7f503613c700 1 --2- 172.21.15.2:0/1297016556 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5030102df0 0x7f503010f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:28.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.932+0000 7f503613c700 1 --2- 172.21.15.2:0/1297016556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5030108510 0x7f50301088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:28.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.932+0000 7f503613c700 1 --2- 172.21.15.2:0/1297016556 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5030102430 0x7f50301028b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:28.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.932+0000 7f503613c700 1 -- 172.21.15.2:0/1297016556 >> 172.21.15.2:0/1297016556 conn(0x7f50300793b0 msgr2=0x7f50300797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:28.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.932+0000 7f503613c700 1 -- 172.21.15.2:0/1297016556 shutdown_connections 2024-05-14T16:54:28.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.932+0000 7f503613c700 1 -- 172.21.15.2:0/1297016556 wait complete. 2024-05-14T16:54:28.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.933+0000 7f503613c700 1 Processor -- start 2024-05-14T16:54:28.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.933+0000 7f503613c700 1 -- start start 2024-05-14T16:54:28.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.933+0000 7f503613c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5030102430 0x7f503019f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:28.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.933+0000 7f503613c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5030102df0 0x7f503019fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:28.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.933+0000 7f503613c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5030108510 0x7f5030199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:28.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.933+0000 7f503613c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5030112640 con 0x7f5030108510 2024-05-14T16:54:28.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.933+0000 7f503613c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f50301124c0 con 0x7f5030102df0 2024-05-14T16:54:28.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.933+0000 7f503613c700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f50301127c0 con 0x7f5030102430 2024-05-14T16:54:28.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.934+0000 7f502ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5030108510 0x7f5030199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:28.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.934+0000 7f502effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5030102df0 0x7f503019fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:28.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.934+0000 7f502ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5030108510 0x7f5030199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49102/0 (socket says 172.21.15.2:49102) 2024-05-14T16:54:28.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.934+0000 7f502ffff700 1 -- 172.21.15.2:0/2480430436 learned_addr learned my addr 172.21.15.2:0/2480430436 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:28.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.934+0000 7f502f7fe700 1 --2- 172.21.15.2:0/2480430436 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5030102430 0x7f503019f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:28.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.934+0000 7f502ffff700 1 -- 172.21.15.2:0/2480430436 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5030102430 msgr2=0x7f503019f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:28.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.934+0000 7f502ffff700 1 --2- 172.21.15.2:0/2480430436 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5030102430 0x7f503019f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:28.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.934+0000 7f502ffff700 1 -- 172.21.15.2:0/2480430436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5030102df0 msgr2=0x7f503019fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:28.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.934+0000 7f502ffff700 1 --2- 172.21.15.2:0/2480430436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5030102df0 0x7f503019fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:28.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.934+0000 7f502ffff700 1 -- 172.21.15.2:0/2480430436 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f503019a1b0 con 0x7f5030108510 2024-05-14T16:54:28.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.934+0000 7f502effd700 1 --2- 172.21.15.2:0/2480430436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5030102df0 0x7f503019fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:54:28.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.934+0000 7f502ffff700 1 --2- 172.21.15.2:0/2480430436 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5030108510 0x7f5030199a50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f502800b550 tx=0x7f502800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:28.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.934+0000 7f502cff9700 1 -- 172.21.15.2:0/2480430436 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5028015020 con 0x7f5030108510 2024-05-14T16:54:28.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.934+0000 7f503613c700 1 -- 172.21.15.2:0/2480430436 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f503019a4a0 con 0x7f5030108510 2024-05-14T16:54:28.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.934+0000 7f502cff9700 1 -- 172.21.15.2:0/2480430436 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5028011ce0 con 0x7f5030108510 2024-05-14T16:54:28.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.934+0000 7f502cff9700 1 -- 172.21.15.2:0/2480430436 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5028010c90 con 0x7f5030108510 2024-05-14T16:54:28.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.934+0000 7f503613c700 1 -- 172.21.15.2:0/2480430436 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50301a6760 con 0x7f5030108510 2024-05-14T16:54:28.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.937+0000 7f502cff9700 1 -- 172.21.15.2:0/2480430436 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f5028018020 con 0x7f5030108510 2024-05-14T16:54:28.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.938+0000 7f502cff9700 1 --2- 172.21.15.2:0/2480430436 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f50180b2880 0x7f50180b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:28.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.938+0000 7f502f7fe700 1 --2- 172.21.15.2:0/2480430436 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f50180b2880 0x7f50180b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:28.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.938+0000 7f502cff9700 1 -- 172.21.15.2:0/2480430436 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f50280e18b0 con 0x7f5030108510 2024-05-14T16:54:28.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.938+0000 7f50409d2b80 1 Processor -- start 2024-05-14T16:54:28.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.938+0000 7f50409d2b80 1 -- start start 2024-05-14T16:54:28.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.938+0000 7f502f7fe700 1 --2- 172.21.15.2:0/2480430436 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f50180b2880 0x7f50180b4d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f503019ab90 tx=0x7f50200079e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:28.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.938+0000 7f50409d2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c97432e700 0x55c97432eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:28.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.938+0000 7f50409d2b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97432f020 0x55c9743327e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:28.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.938+0000 7f50409d2b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c974332d20 0x55c974333160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:28.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.938+0000 7f50409d2b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c974288c40 con 0x55c97432f020 2024-05-14T16:54:28.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.938+0000 7f50409d2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c9742787e0 con 0x55c97432e700 2024-05-14T16:54:28.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.938+0000 7f50409d2b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c97427c820 con 0x55c974332d20 2024-05-14T16:54:28.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.938+0000 7f502effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97432f020 0x55c9743327e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:28.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.938+0000 7f502effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97432f020 0x55c9743327e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49116/0 (socket says 172.21.15.2:49116) 2024-05-14T16:54:28.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.939+0000 7f502effd700 1 -- 172.21.15.2:0/3097571155 learned_addr learned my addr 172.21.15.2:0/3097571155 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:28.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.939+0000 7f502f7fe700 1 --2- 172.21.15.2:0/3097571155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c97432e700 0x55c97432eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:28.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.939+0000 7f502ffff700 1 --2- 172.21.15.2:0/3097571155 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c974332d20 0x55c974333160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:28.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.939+0000 7f502effd700 1 -- 172.21.15.2:0/3097571155 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c974332d20 msgr2=0x55c974333160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:28.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.939+0000 7f502effd700 1 --2- 172.21.15.2:0/3097571155 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c974332d20 0x55c974333160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:28.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.939+0000 7f502effd700 1 -- 172.21.15.2:0/3097571155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c97432e700 msgr2=0x55c97432eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:28.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.939+0000 7f502effd700 1 --2- 172.21.15.2:0/3097571155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c97432e700 0x55c97432eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:28.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.939+0000 7f502effd700 1 -- 172.21.15.2:0/3097571155 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9741c8880 con 0x55c97432f020 2024-05-14T16:54:28.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.939+0000 7f502f7fe700 1 --2- 172.21.15.2:0/3097571155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c97432e700 0x55c97432eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:54:28.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.939+0000 7f502ffff700 1 --2- 172.21.15.2:0/3097571155 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c974332d20 0x55c974333160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:54:28.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.939+0000 7f502effd700 1 --2- 172.21.15.2:0/3097571155 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97432f020 0x55c9743327e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5024009990 tx=0x7f5024003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e127abafe2562851 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:28.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.939+0000 7f5015ffb700 1 -- 172.21.15.2:0/3097571155 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5024028070 con 0x55c97432f020 2024-05-14T16:54:28.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.939+0000 7f5015ffb700 1 -- 172.21.15.2:0/3097571155 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f502400be80 con 0x55c97432f020 2024-05-14T16:54:28.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.939+0000 7f5015ffb700 1 -- 172.21.15.2:0/3097571155 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5024023680 con 0x55c97432f020 2024-05-14T16:54:28.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.940+0000 7f50409d2b80 1 -- 172.21.15.2:0/3097571155 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97432f020 msgr2=0x55c9743327e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:28.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.940+0000 7f50409d2b80 1 --2- 172.21.15.2:0/3097571155 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97432f020 0x55c9743327e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5024009990 tx=0x7f5024003680 comp rx=0 tx=0).stop 2024-05-14T16:54:28.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.940+0000 7f50409d2b80 1 -- 172.21.15.2:0/3097571155 shutdown_connections 2024-05-14T16:54:28.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.940+0000 7f50409d2b80 1 --2- 172.21.15.2:0/3097571155 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c974332d20 0x55c974333160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:28.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.940+0000 7f50409d2b80 1 --2- 172.21.15.2:0/3097571155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c97432e700 0x55c97432eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:28.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.940+0000 7f50409d2b80 1 --2- 172.21.15.2:0/3097571155 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97432f020 0x55c9743327e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:28.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.940+0000 7f50409d2b80 1 -- 172.21.15.2:0/3097571155 >> 172.21.15.2:0/3097571155 conn(0x55c974255200 msgr2=0x55c974324930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:28.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.940+0000 7f50409d2b80 1 -- 172.21.15.2:0/3097571155 shutdown_connections 2024-05-14T16:54:28.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.940+0000 7f50409d2b80 1 -- 172.21.15.2:0/3097571155 wait complete. 2024-05-14T16:54:28.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.940+0000 7f50409d2b80 1 -- start start 2024-05-14T16:54:28.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.941+0000 7f50409d2b80 1 Processor -- start 2024-05-14T16:54:28.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.941+0000 7f50409d2b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c97432e700 0x55c97433a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:28.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.941+0000 7f50409d2b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97432f020 0x55c9743374c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:28.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.941+0000 7f50409d2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c974332d20 0x55c974337a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:28.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.941+0000 7f50409d2b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c97417ef50 con 0x55c97432f020 2024-05-14T16:54:28.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.941+0000 7f50409d2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c97421a770 con 0x55c974332d20 2024-05-14T16:54:28.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.941+0000 7f50409d2b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c97433af20 con 0x55c97432e700 2024-05-14T16:54:28.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.941+0000 7f50409d2b80 10 client.0 _ll_register_callbacks cb 0x55c97420f530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:54:28.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.941+0000 7f502effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97432f020 0x55c9743374c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:28.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.941+0000 7f502effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97432f020 0x55c9743374c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49120/0 (socket says 172.21.15.2:49120) 2024-05-14T16:54:28.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.941+0000 7f502effd700 1 -- 172.21.15.2:0/562074602 learned_addr learned my addr 172.21.15.2:0/562074602 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:28.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.941+0000 7f502f7fe700 1 --2- 172.21.15.2:0/562074602 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c97432e700 0x55c97433a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:28.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.941+0000 7f502ffff700 1 --2- 172.21.15.2:0/562074602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c974332d20 0x55c974337a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:28.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.941+0000 7f502effd700 1 -- 172.21.15.2:0/562074602 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c97432e700 msgr2=0x55c97433a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:28.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.941+0000 7f502effd700 1 --2- 172.21.15.2:0/562074602 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c97432e700 0x55c97433a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:28.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.941+0000 7f502effd700 1 -- 172.21.15.2:0/562074602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c974332d20 msgr2=0x55c974337a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:28.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.941+0000 7f502effd700 1 --2- 172.21.15.2:0/562074602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c974332d20 0x55c974337a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:28.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.941+0000 7f502effd700 1 -- 172.21.15.2:0/562074602 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c974217640 con 0x55c97432f020 2024-05-14T16:54:28.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.941+0000 7f502f7fe700 1 --2- 172.21.15.2:0/562074602 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c97432e700 0x55c97433a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:54:28.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.942+0000 7f502effd700 1 --2- 172.21.15.2:0/562074602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97432f020 0x55c9743374c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5024009ce0 tx=0x7f5024009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c103a5f0abc488 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:28.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.942+0000 7f500effd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:54:28.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.942+0000 7f500effd700 1 -- 172.21.15.2:0/562074602 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5024033060 con 0x55c97432f020 2024-05-14T16:54:28.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.942+0000 7f500effd700 1 -- 172.21.15.2:0/562074602 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f5024003d30 con 0x55c97432f020 2024-05-14T16:54:28.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.942+0000 7f50409d2b80 1 -- 172.21.15.2:0/562074602 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c9742f6310 con 0x55c97432f020 2024-05-14T16:54:28.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.942+0000 7f500effd700 1 -- 172.21.15.2:0/562074602 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5024028070 con 0x55c97432f020 2024-05-14T16:54:28.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.942+0000 7f50409d2b80 10 client.15072 fetch_fsmap learned FSMap version 24 2024-05-14T16:54:28.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.942+0000 7f50409d2b80 1 -- 172.21.15.2:0/562074602 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c9743454c0 con 0x55c97432f020 2024-05-14T16:54:28.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.942+0000 7f500effd700 1 -- 172.21.15.2:0/562074602 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c9742f6310 con 0x55c97432f020 2024-05-14T16:54:28.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.943+0000 7f500effd700 1 -- 172.21.15.2:0/562074602 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f502400f510 con 0x55c97432f020 2024-05-14T16:54:28.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.943+0000 7f50409d2b80 10 client.15072 fetch_fsmap finished waiting for FSMap version 24 2024-05-14T16:54:28.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.943+0000 7f50409d2b80 10 client.15072 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-14T16:54:28.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.943+0000 7f50409d2b80 20 client.15072 populate_metadata read hostname 'smithi002' 2024-05-14T16:54:28.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.943+0000 7f50409d2b80 1 --2- 172.21.15.2:0/562074602 >> [v2:172.21.15.123:6836/333426645,v1:172.21.15.123:6837/333426645] conn(0x55c974346810 0x55c9743489e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:28.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.943+0000 7f50409d2b80 4 client.15072 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:54:28.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.943+0000 7f50409d2b80 1 -- 172.21.15.2:0/562074602 --> [v2:172.21.15.123:6836/333426645,v1:172.21.15.123:6837/333426645] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c97428ab90 con 0x55c974346810 2024-05-14T16:54:28.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.943+0000 7f502f7fe700 1 --2- 172.21.15.2:0/562074602 >> [v2:172.21.15.123:6836/333426645,v1:172.21.15.123:6837/333426645] conn(0x55c974346810 0x55c9743489e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:28.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.943+0000 7f500effd700 10 client.15072 ms_handle_connect on v2:172.21.15.123:6836/333426645 2024-05-14T16:54:28.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.943+0000 7f502f7fe700 1 --2- 172.21.15.2:0/562074602 >> [v2:172.21.15.123:6836/333426645,v1:172.21.15.123:6837/333426645] conn(0x55c974346810 0x55c9743489e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=732d5592fde6b3e1 server_cookie=dd6afaab3f35abaf in_seq=0 out_seq=0 2024-05-14T16:54:28.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.944+0000 7f500effd700 1 -- 172.21.15.2:0/562074602 <== mds.1 v2:172.21.15.123:6836/333426645 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c97428ab90 con 0x55c974346810 2024-05-14T16:54:28.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.944+0000 7f500effd700 10 client.15072 handle_command_reply: tid=0 2024-05-14T16:54:28.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.944+0000 7f50409d2b80 1 client.15072 shutdown 2024-05-14T16:54:28.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.944+0000 7f50409d2b80 1 -- 172.21.15.2:0/562074602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97432f020 msgr2=0x55c9743374c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:28.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.944+0000 7f50409d2b80 1 --2- 172.21.15.2:0/562074602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97432f020 0x55c9743374c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5024009ce0 tx=0x7f5024009580 comp rx=0 tx=0).stop 2024-05-14T16:54:28.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.945+0000 7f50409d2b80 1 -- 172.21.15.2:0/562074602 shutdown_connections 2024-05-14T16:54:28.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.945+0000 7f50409d2b80 1 --2- 172.21.15.2:0/562074602 >> [v2:172.21.15.123:6836/333426645,v1:172.21.15.123:6837/333426645] conn(0x55c974346810 0x55c9743489e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:28.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.945+0000 7f50409d2b80 1 --2- 172.21.15.2:0/562074602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c974332d20 0x55c974337a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:28.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.945+0000 7f50409d2b80 1 --2- 172.21.15.2:0/562074602 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c97432e700 0x55c97433a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:28.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.945+0000 7f50409d2b80 1 --2- 172.21.15.2:0/562074602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97432f020 0x55c9743374c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:28.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.945+0000 7f500effd700 0 client.15072 ms_handle_reset on v2:172.21.15.123:6836/333426645 2024-05-14T16:54:28.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.945+0000 7f50409d2b80 1 -- 172.21.15.2:0/562074602 >> 172.21.15.2:0/562074602 conn(0x55c974255200 msgr2=0x55c9743341f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:28.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.945+0000 7f50409d2b80 1 -- 172.21.15.2:0/562074602 shutdown_connections 2024-05-14T16:54:28.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.945+0000 7f50409d2b80 1 -- 172.21.15.2:0/562074602 wait complete. 2024-05-14T16:54:28.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.945+0000 7f50409d2b80 20 client.15072 trim_cache size 0 max 16384 2024-05-14T16:54:28.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.970+0000 7f50409d2b80 1 Processor -- start 2024-05-14T16:54:28.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.970+0000 7f50409d2b80 1 -- start start 2024-05-14T16:54:28.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.970+0000 7f50409d2b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97432e700 0x55c974330750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:28.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.970+0000 7f50409d2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c97432f020 0x55c974330c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:28.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.970+0000 7f50409d2b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c974332d20 0x55c9743431e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:28.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.970+0000 7f50409d2b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c974346d20 con 0x55c97432e700 2024-05-14T16:54:28.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.970+0000 7f50409d2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c9743473f0 con 0x55c97432f020 2024-05-14T16:54:28.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.970+0000 7f50409d2b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c974347ac0 con 0x55c974332d20 2024-05-14T16:54:28.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.970+0000 7f502f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97432e700 0x55c974330750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:28.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.971+0000 7f502f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97432e700 0x55c974330750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49128/0 (socket says 172.21.15.2:49128) 2024-05-14T16:54:28.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.971+0000 7f502f7fe700 1 -- 172.21.15.2:0/3023174302 learned_addr learned my addr 172.21.15.2:0/3023174302 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:28.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.971+0000 7f502effd700 1 --2- 172.21.15.2:0/3023174302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c97432f020 0x55c974330c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:28.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.971+0000 7f502ffff700 1 --2- 172.21.15.2:0/3023174302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c974332d20 0x55c9743431e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:28.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.971+0000 7f502f7fe700 1 -- 172.21.15.2:0/3023174302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c974332d20 msgr2=0x55c9743431e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:28.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.971+0000 7f502f7fe700 1 --2- 172.21.15.2:0/3023174302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c974332d20 0x55c9743431e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:28.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.971+0000 7f502f7fe700 1 -- 172.21.15.2:0/3023174302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c97432f020 msgr2=0x55c974330c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:28.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.971+0000 7f502f7fe700 1 --2- 172.21.15.2:0/3023174302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c97432f020 0x55c974330c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:28.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.971+0000 7f502f7fe700 1 -- 172.21.15.2:0/3023174302 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c974324f60 con 0x55c97432e700 2024-05-14T16:54:28.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.971+0000 7f502f7fe700 1 --2- 172.21.15.2:0/3023174302 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97432e700 0x55c974330750 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5020007720 tx=0x7f5020015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e9396da3ce5bb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:28.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.971+0000 7f50157fa700 1 -- 172.21.15.2:0/3023174302 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502001b040 con 0x55c97432e700 2024-05-14T16:54:28.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.971+0000 7f50157fa700 1 -- 172.21.15.2:0/3023174302 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f502002e070 con 0x55c97432e700 2024-05-14T16:54:28.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.971+0000 7f50157fa700 1 -- 172.21.15.2:0/3023174302 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5020022630 con 0x55c97432e700 2024-05-14T16:54:28.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.972+0000 7f50409d2b80 1 -- 172.21.15.2:0/3023174302 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97432e700 msgr2=0x55c974330750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:28.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.972+0000 7f50409d2b80 1 --2- 172.21.15.2:0/3023174302 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97432e700 0x55c974330750 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5020007720 tx=0x7f5020015000 comp rx=0 tx=0).stop 2024-05-14T16:54:28.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.972+0000 7f50409d2b80 1 -- 172.21.15.2:0/3023174302 shutdown_connections 2024-05-14T16:54:28.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.972+0000 7f50409d2b80 1 --2- 172.21.15.2:0/3023174302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c974332d20 0x55c9743431e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:28.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.972+0000 7f50409d2b80 1 --2- 172.21.15.2:0/3023174302 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97432e700 0x55c974330750 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:28.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.972+0000 7f50409d2b80 1 --2- 172.21.15.2:0/3023174302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c97432f020 0x55c974330c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:28.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.972+0000 7f50409d2b80 1 -- 172.21.15.2:0/3023174302 >> 172.21.15.2:0/3023174302 conn(0x55c974255200 msgr2=0x55c974329990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:28.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.972+0000 7f50409d2b80 1 -- 172.21.15.2:0/3023174302 shutdown_connections 2024-05-14T16:54:28.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.972+0000 7f50409d2b80 1 -- 172.21.15.2:0/3023174302 wait complete. 2024-05-14T16:54:28.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.972+0000 7f50409d2b80 1 -- start start 2024-05-14T16:54:28.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.973+0000 7f50409d2b80 1 Processor -- start 2024-05-14T16:54:28.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.973+0000 7f50409d2b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97432e700 0x55c974351e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:28.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.973+0000 7f50409d2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c97432f020 0x55c9743523a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:28.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.973+0000 7f50409d2b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c974332d20 0x55c97434ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:28.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.973+0000 7f50409d2b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c974348860 con 0x55c97432e700 2024-05-14T16:54:28.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.973+0000 7f50409d2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c974348190 con 0x55c97432f020 2024-05-14T16:54:28.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.973+0000 7f50409d2b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c974348f30 con 0x55c974332d20 2024-05-14T16:54:28.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.973+0000 7f50409d2b80 10 client.0 _ll_register_callbacks cb 0x55c97420f530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:54:28.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.973+0000 7f502f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97432e700 0x55c974351e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:28.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.973+0000 7f502effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c97432f020 0x55c9743523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:28.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.973+0000 7f502f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97432e700 0x55c974351e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49144/0 (socket says 172.21.15.2:49144) 2024-05-14T16:54:28.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.973+0000 7f502f7fe700 1 -- 172.21.15.2:0/483650760 learned_addr learned my addr 172.21.15.2:0/483650760 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:28.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.973+0000 7f502ffff700 1 --2- 172.21.15.2:0/483650760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c974332d20 0x55c97434ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:28.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.973+0000 7f502f7fe700 1 -- 172.21.15.2:0/483650760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c974332d20 msgr2=0x55c97434ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:28.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.973+0000 7f502f7fe700 1 --2- 172.21.15.2:0/483650760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c974332d20 0x55c97434ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:28.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.973+0000 7f502f7fe700 1 -- 172.21.15.2:0/483650760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c97432f020 msgr2=0x55c9743523a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:28.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.973+0000 7f502f7fe700 1 --2- 172.21.15.2:0/483650760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c97432f020 0x55c9743523a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:28.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.973+0000 7f502f7fe700 1 -- 172.21.15.2:0/483650760 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c97434d560 con 0x55c97432e700 2024-05-14T16:54:28.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.973+0000 7f502effd700 1 --2- 172.21.15.2:0/483650760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c97432f020 0x55c9743523a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:54:28.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.973+0000 7f502f7fe700 1 --2- 172.21.15.2:0/483650760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97432e700 0x55c974351e60 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5020005b90 tx=0x7f5020014e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b49d141c96bb122 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:28.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.973+0000 7f500effd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:54:28.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.974+0000 7f500effd700 1 -- 172.21.15.2:0/483650760 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5020037060 con 0x55c97432e700 2024-05-14T16:54:28.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.974+0000 7f500effd700 1 -- 172.21.15.2:0/483650760 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f5020028040 con 0x55c97432e700 2024-05-14T16:54:28.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.974+0000 7f50409d2b80 1 -- 172.21.15.2:0/483650760 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f502002e070 con 0x55c97432e700 2024-05-14T16:54:28.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.974+0000 7f500effd700 1 -- 172.21.15.2:0/483650760 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f50200296a0 con 0x55c97432e700 2024-05-14T16:54:28.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.974+0000 7f50409d2b80 10 client.15078 fetch_fsmap learned FSMap version 24 2024-05-14T16:54:28.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.974+0000 7f50409d2b80 1 -- 172.21.15.2:0/483650760 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c9743b3420 con 0x55c97432e700 2024-05-14T16:54:28.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.974+0000 7f500effd700 1 -- 172.21.15.2:0/483650760 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f502002e070 con 0x55c97432e700 2024-05-14T16:54:28.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.974+0000 7f500effd700 1 -- 172.21.15.2:0/483650760 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f5020029ce0 con 0x55c97432e700 2024-05-14T16:54:28.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.974+0000 7f50409d2b80 10 client.15078 fetch_fsmap finished waiting for FSMap version 24 2024-05-14T16:54:28.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.974+0000 7f50409d2b80 10 client.15078 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-14T16:54:28.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.974+0000 7f50409d2b80 20 client.15078 populate_metadata read hostname 'smithi002' 2024-05-14T16:54:28.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.974+0000 7f50409d2b80 1 --2- 172.21.15.2:0/483650760 >> [v2:172.21.15.123:6836/333426645,v1:172.21.15.123:6837/333426645] conn(0x55c9743b7c20 0x55c9743b6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:28.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.974+0000 7f50409d2b80 4 client.15078 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T16:54:28.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.974+0000 7f50409d2b80 1 -- 172.21.15.2:0/483650760 --> [v2:172.21.15.123:6836/333426645,v1:172.21.15.123:6837/333426645] -- command(tid 0: {"prefix": "status"}) -- 0x55c974284370 con 0x55c9743b7c20 2024-05-14T16:54:28.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.974+0000 7f502effd700 1 --2- 172.21.15.2:0/483650760 >> [v2:172.21.15.123:6836/333426645,v1:172.21.15.123:6837/333426645] conn(0x55c9743b7c20 0x55c9743b6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:28.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.975+0000 7f502effd700 1 --2- 172.21.15.2:0/483650760 >> [v2:172.21.15.123:6836/333426645,v1:172.21.15.123:6837/333426645] conn(0x55c9743b7c20 0x55c9743b6930 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=611bc622486c57ae server_cookie=9b968b8f02f0ef79 in_seq=0 out_seq=0 2024-05-14T16:54:28.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.975+0000 7f500effd700 10 client.15078 ms_handle_connect on v2:172.21.15.123:6836/333426645 2024-05-14T16:54:28.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.975+0000 7f500effd700 1 -- 172.21.15.2:0/483650760 <== mds.1 v2:172.21.15.123:6836/333426645 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c974284370 con 0x55c9743b7c20 2024-05-14T16:54:28.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.975+0000 7f500effd700 10 client.15078 handle_command_reply: tid=0 2024-05-14T16:54:28.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.975+0000 7f50409d2b80 1 client.15078 shutdown 2024-05-14T16:54:28.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.975+0000 7f50409d2b80 1 -- 172.21.15.2:0/483650760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97432e700 msgr2=0x55c974351e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:28.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.975+0000 7f50409d2b80 1 --2- 172.21.15.2:0/483650760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97432e700 0x55c974351e60 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5020005b90 tx=0x7f5020014e40 comp rx=0 tx=0).stop 2024-05-14T16:54:28.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.975+0000 7f50409d2b80 1 -- 172.21.15.2:0/483650760 shutdown_connections 2024-05-14T16:54:28.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.975+0000 7f50409d2b80 1 --2- 172.21.15.2:0/483650760 >> [v2:172.21.15.123:6836/333426645,v1:172.21.15.123:6837/333426645] conn(0x55c9743b7c20 0x55c9743b6930 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:28.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.975+0000 7f50409d2b80 1 --2- 172.21.15.2:0/483650760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c974332d20 0x55c97434ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:28.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.975+0000 7f50409d2b80 1 --2- 172.21.15.2:0/483650760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c97432e700 0x55c974351e60 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:28.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.975+0000 7f500effd700 0 client.15078 ms_handle_reset on v2:172.21.15.123:6836/333426645 2024-05-14T16:54:28.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.975+0000 7f50409d2b80 1 --2- 172.21.15.2:0/483650760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c97432f020 0x55c9743523a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:28.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.975+0000 7f50409d2b80 1 -- 172.21.15.2:0/483650760 >> 172.21.15.2:0/483650760 conn(0x55c974255200 msgr2=0x55c974335ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:28.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.975+0000 7f50409d2b80 1 -- 172.21.15.2:0/483650760 shutdown_connections 2024-05-14T16:54:28.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.975+0000 7f50409d2b80 1 -- 172.21.15.2:0/483650760 wait complete. 2024-05-14T16:54:28.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.976+0000 7f50409d2b80 20 client.15078 trim_cache size 0 max 16384 2024-05-14T16:54:28.981 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T16:54:28.981 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "ed40717a-1210-11ef-bc98-c7b262605968", 2024-05-14T16:54:28.981 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 1, 2024-05-14T16:54:28.981 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24455, 2024-05-14T16:54:28.981 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2024-05-14T16:54:28.981 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2024-05-14T16:54:28.981 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2024-05-14T16:54:28.981 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 162.00359992400001, 2024-05-14T16:54:28.981 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 24, 2024-05-14T16:54:28.981 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 80, 2024-05-14T16:54:28.981 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 78, 2024-05-14T16:54:28.981 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 203.51638893000001 2024-05-14T16:54:28.982 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T16:54:28.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.976+0000 7f503613c700 1 -- 172.21.15.2:0/2480430436 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f50180b2880 msgr2=0x7f50180b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:28.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.976+0000 7f503613c700 1 --2- 172.21.15.2:0/2480430436 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f50180b2880 0x7f50180b4d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f503019ab90 tx=0x7f50200079e0 comp rx=0 tx=0).stop 2024-05-14T16:54:28.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.976+0000 7f503613c700 1 -- 172.21.15.2:0/2480430436 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5030108510 msgr2=0x7f5030199a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:28.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.976+0000 7f503613c700 1 --2- 172.21.15.2:0/2480430436 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5030108510 0x7f5030199a50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f502800b550 tx=0x7f502800b860 comp rx=0 tx=0).stop 2024-05-14T16:54:28.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.976+0000 7f503613c700 1 -- 172.21.15.2:0/2480430436 shutdown_connections 2024-05-14T16:54:28.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.976+0000 7f503613c700 1 --2- 172.21.15.2:0/2480430436 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f50180b2880 0x7f50180b4d40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:28.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.976+0000 7f503613c700 1 --2- 172.21.15.2:0/2480430436 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5030108510 0x7f5030199a50 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:28.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.976+0000 7f503613c700 1 --2- 172.21.15.2:0/2480430436 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5030102430 0x7f503019f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:28.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.976+0000 7f503613c700 1 --2- 172.21.15.2:0/2480430436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5030102df0 0x7f503019fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:28.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.976+0000 7f503613c700 1 -- 172.21.15.2:0/2480430436 >> 172.21.15.2:0/2480430436 conn(0x7f50300793b0 msgr2=0x7f5030106350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:28.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.976+0000 7f503613c700 1 -- 172.21.15.2:0/2480430436 shutdown_connections 2024-05-14T16:54:28.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:28.976+0000 7f503613c700 1 -- 172.21.15.2:0/2480430436 wait complete. 2024-05-14T16:54:29.683 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:29 smithi002 ceph-mon[31538]: pgmap v378: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T16:54:29.715 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph tell mds.b status 2024-05-14T16:54:29.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:29 smithi151 ceph-mon[36066]: pgmap v378: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T16:54:29.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:29 smithi123 ceph-mon[35971]: pgmap v378: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-14T16:54:29.969 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:54:31.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.642+0000 7f4348394700 1 -- 172.21.15.2:0/42551818 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4340076090 msgr2=0x7f434010fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:31.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.642+0000 7f4348394700 1 --2- 172.21.15.2:0/42551818 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4340076090 0x7f434010fa00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4334009990 tx=0x7f4334009ca0 comp rx=0 tx=0).stop 2024-05-14T16:54:31.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.642+0000 7f4348394700 1 -- 172.21.15.2:0/42551818 shutdown_connections 2024-05-14T16:54:31.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.642+0000 7f4348394700 1 --2- 172.21.15.2:0/42551818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f434010ff40 0x7f4340112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:31.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.642+0000 7f4348394700 1 --2- 172.21.15.2:0/42551818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4340075770 0x7f4340075b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:31.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.642+0000 7f4348394700 1 --2- 172.21.15.2:0/42551818 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4340076090 0x7f434010fa00 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:31.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.642+0000 7f4348394700 1 -- 172.21.15.2:0/42551818 >> 172.21.15.2:0/42551818 conn(0x7f434007a6a0 msgr2=0x7f434007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:31.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.642+0000 7f4348394700 1 -- 172.21.15.2:0/42551818 shutdown_connections 2024-05-14T16:54:31.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.642+0000 7f4348394700 1 -- 172.21.15.2:0/42551818 wait complete. 2024-05-14T16:54:31.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.643+0000 7f4348394700 1 Processor -- start 2024-05-14T16:54:31.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.643+0000 7f4348394700 1 -- start start 2024-05-14T16:54:31.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.643+0000 7f4348394700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4340075770 0x7f434019be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:31.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.643+0000 7f4348394700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4340076090 0x7f434019c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:31.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.643+0000 7f4348394700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f434010ff40 0x7f43401a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:31.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.643+0000 7f4348394700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4340115900 con 0x7f4340076090 2024-05-14T16:54:31.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.643+0000 7f4348394700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4340115780 con 0x7f434010ff40 2024-05-14T16:54:31.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.643+0000 7f4348394700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4340115a80 con 0x7f4340075770 2024-05-14T16:54:31.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.643+0000 7f434592f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4340076090 0x7f434019c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:31.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.643+0000 7f434592f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4340076090 0x7f434019c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49158/0 (socket says 172.21.15.2:49158) 2024-05-14T16:54:31.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.643+0000 7f434592f700 1 -- 172.21.15.2:0/846686442 learned_addr learned my addr 172.21.15.2:0/846686442 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:31.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.644+0000 7f4346130700 1 --2- 172.21.15.2:0/846686442 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4340075770 0x7f434019be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:31.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.644+0000 7f4346931700 1 --2- 172.21.15.2:0/846686442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f434010ff40 0x7f43401a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:31.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.644+0000 7f434592f700 1 -- 172.21.15.2:0/846686442 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4340075770 msgr2=0x7f434019be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:31.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.644+0000 7f434592f700 1 --2- 172.21.15.2:0/846686442 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4340075770 0x7f434019be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:31.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.644+0000 7f434592f700 1 -- 172.21.15.2:0/846686442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f434010ff40 msgr2=0x7f43401a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:31.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.644+0000 7f434592f700 1 --2- 172.21.15.2:0/846686442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f434010ff40 0x7f43401a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:31.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.644+0000 7f434592f700 1 -- 172.21.15.2:0/846686442 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43401a0dd0 con 0x7f4340076090 2024-05-14T16:54:31.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.644+0000 7f4346931700 1 --2- 172.21.15.2:0/846686442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f434010ff40 0x7f43401a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:54:31.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.644+0000 7f434592f700 1 --2- 172.21.15.2:0/846686442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4340076090 0x7f434019c350 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4334000c00 tx=0x7f43340047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:31.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.644+0000 7f433f7fe700 1 -- 172.21.15.2:0/846686442 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4334027070 con 0x7f4340076090 2024-05-14T16:54:31.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.644+0000 7f4348394700 1 -- 172.21.15.2:0/846686442 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43401a1060 con 0x7f4340076090 2024-05-14T16:54:31.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.644+0000 7f433f7fe700 1 -- 172.21.15.2:0/846686442 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4334003f20 con 0x7f4340076090 2024-05-14T16:54:31.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.644+0000 7f4348394700 1 -- 172.21.15.2:0/846686442 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43401a8940 con 0x7f4340076090 2024-05-14T16:54:31.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.644+0000 7f433f7fe700 1 -- 172.21.15.2:0/846686442 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4334022cd0 con 0x7f4340076090 2024-05-14T16:54:31.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.647+0000 7f433f7fe700 1 -- 172.21.15.2:0/846686442 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f433402a070 con 0x7f4340076090 2024-05-14T16:54:31.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.648+0000 7f433f7fe700 1 --2- 172.21.15.2:0/846686442 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f43280b2880 0x7f43280b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:31.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.648+0000 7f4346130700 1 --2- 172.21.15.2:0/846686442 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f43280b2880 0x7f43280b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:31.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.648+0000 7f4352c2ab80 1 Processor -- start 2024-05-14T16:54:31.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.648+0000 7f433f7fe700 1 -- 172.21.15.2:0/846686442 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f43340e2fe0 con 0x7f4340076090 2024-05-14T16:54:31.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.648+0000 7f4352c2ab80 1 -- start start 2024-05-14T16:54:31.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.648+0000 7f4352c2ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55653ac9e2f0 0x55653ac9e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:31.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.648+0000 7f4352c2ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55653ac9ec10 0x55653aca23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:31.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.648+0000 7f4352c2ab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55653aca2910 0x55653aca2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:31.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.648+0000 7f4352c2ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55653abfcc40 con 0x55653ac9e2f0 2024-05-14T16:54:31.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.648+0000 7f4352c2ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55653abec7e0 con 0x55653ac9ec10 2024-05-14T16:54:31.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.648+0000 7f4352c2ab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55653abf0820 con 0x55653aca2910 2024-05-14T16:54:31.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.648+0000 7f4346130700 1 --2- 172.21.15.2:0/846686442 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f43280b2880 0x7f43280b4d40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4330009550 tx=0x7f43300091b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:31.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.649+0000 7f4346130700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55653ac9e2f0 0x55653ac9e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:31.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.649+0000 7f4346130700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55653ac9e2f0 0x55653ac9e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49160/0 (socket says 172.21.15.2:49160) 2024-05-14T16:54:31.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.649+0000 7f4346130700 1 -- 172.21.15.2:0/3750831822 learned_addr learned my addr 172.21.15.2:0/3750831822 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:31.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.649+0000 7f4346130700 1 -- 172.21.15.2:0/3750831822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55653aca2910 msgr2=0x55653aca2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:31.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.649+0000 7f4346130700 1 --2- 172.21.15.2:0/3750831822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55653aca2910 0x55653aca2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:31.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.649+0000 7f4346130700 1 -- 172.21.15.2:0/3750831822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55653ac9ec10 msgr2=0x55653aca23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:31.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.649+0000 7f4346130700 1 --2- 172.21.15.2:0/3750831822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55653ac9ec10 0x55653aca23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:31.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.649+0000 7f4346130700 1 -- 172.21.15.2:0/3750831822 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55653ab3c880 con 0x55653ac9e2f0 2024-05-14T16:54:31.654 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:31 smithi002 ceph-mon[31538]: pgmap v379: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T16:54:31.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.649+0000 7f4346130700 1 --2- 172.21.15.2:0/3750831822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55653ac9e2f0 0x55653ac9e6d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4330004700 tx=0x7f4330013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a936cc61f4c82d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:31.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.649+0000 7f433cd70700 1 -- 172.21.15.2:0/3750831822 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4330026070 con 0x55653ac9e2f0 2024-05-14T16:54:31.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.649+0000 7f433cd70700 1 -- 172.21.15.2:0/3750831822 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f433001d040 con 0x55653ac9e2f0 2024-05-14T16:54:31.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.649+0000 7f433cd70700 1 -- 172.21.15.2:0/3750831822 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433001c050 con 0x55653ac9e2f0 2024-05-14T16:54:31.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.650+0000 7f4352c2ab80 1 -- 172.21.15.2:0/3750831822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55653ac9e2f0 msgr2=0x55653ac9e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:31.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.650+0000 7f4352c2ab80 1 --2- 172.21.15.2:0/3750831822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55653ac9e2f0 0x55653ac9e6d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4330004700 tx=0x7f4330013000 comp rx=0 tx=0).stop 2024-05-14T16:54:31.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.650+0000 7f4352c2ab80 1 -- 172.21.15.2:0/3750831822 shutdown_connections 2024-05-14T16:54:31.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.650+0000 7f4352c2ab80 1 --2- 172.21.15.2:0/3750831822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55653aca2910 0x55653aca2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:31.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.650+0000 7f4352c2ab80 1 --2- 172.21.15.2:0/3750831822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55653ac9e2f0 0x55653ac9e6d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:31.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.650+0000 7f4352c2ab80 1 --2- 172.21.15.2:0/3750831822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55653ac9ec10 0x55653aca23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:31.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.650+0000 7f4352c2ab80 1 -- 172.21.15.2:0/3750831822 >> 172.21.15.2:0/3750831822 conn(0x55653abc9200 msgr2=0x55653ac98930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:31.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.650+0000 7f4352c2ab80 1 -- 172.21.15.2:0/3750831822 shutdown_connections 2024-05-14T16:54:31.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.650+0000 7f4352c2ab80 1 -- 172.21.15.2:0/3750831822 wait complete. 2024-05-14T16:54:31.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.651+0000 7f4352c2ab80 1 -- start start 2024-05-14T16:54:31.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.651+0000 7f4352c2ab80 1 Processor -- start 2024-05-14T16:54:31.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.651+0000 7f4352c2ab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55653ac9e2f0 0x55653aca9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:31.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.651+0000 7f4352c2ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55653ac9ec10 0x55653aca71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:31.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.651+0000 7f4352c2ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55653aca2910 0x55653aca7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:31.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.651+0000 7f4352c2ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55653aaf2f50 con 0x55653ac9ec10 2024-05-14T16:54:31.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.651+0000 7f4352c2ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55653ab8e770 con 0x55653aca2910 2024-05-14T16:54:31.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.651+0000 7f4352c2ab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55653acaab70 con 0x55653ac9e2f0 2024-05-14T16:54:31.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.651+0000 7f4352c2ab80 10 client.0 _ll_register_callbacks cb 0x55653ab83530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:54:31.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.651+0000 7f434592f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55653ac9ec10 0x55653aca71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:31.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.651+0000 7f434592f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55653ac9ec10 0x55653aca71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49162/0 (socket says 172.21.15.2:49162) 2024-05-14T16:54:31.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.651+0000 7f434592f700 1 -- 172.21.15.2:0/1730099368 learned_addr learned my addr 172.21.15.2:0/1730099368 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:31.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.651+0000 7f4346130700 1 --2- 172.21.15.2:0/1730099368 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55653ac9e2f0 0x55653aca9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:31.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.652+0000 7f4346931700 1 --2- 172.21.15.2:0/1730099368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55653aca2910 0x55653aca7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.652+0000 7f434592f700 1 -- 172.21.15.2:0/1730099368 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55653ac9e2f0 msgr2=0x55653aca9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.652+0000 7f434592f700 1 --2- 172.21.15.2:0/1730099368 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55653ac9e2f0 0x55653aca9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.652+0000 7f434592f700 1 -- 172.21.15.2:0/1730099368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55653aca2910 msgr2=0x55653aca7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.652+0000 7f434592f700 1 --2- 172.21.15.2:0/1730099368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55653aca2910 0x55653aca7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.652+0000 7f434592f700 1 -- 172.21.15.2:0/1730099368 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55653ab8b640 con 0x55653ac9ec10 2024-05-14T16:54:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.652+0000 7f4346931700 1 --2- 172.21.15.2:0/1730099368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55653aca2910 0x55653aca7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:54:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.652+0000 7f434592f700 1 --2- 172.21.15.2:0/1730099368 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55653ac9ec10 0x55653aca71f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f43340aa610 tx=0x7f43340aa240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=721bc2c8b0b4a4bf server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.652+0000 7f4325ffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:54:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.652+0000 7f4325ffb700 1 -- 172.21.15.2:0/1730099368 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43340aea30 con 0x55653ac9ec10 2024-05-14T16:54:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.652+0000 7f4325ffb700 1 -- 172.21.15.2:0/1730099368 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f4334092110 con 0x55653ac9ec10 2024-05-14T16:54:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.652+0000 7f4352c2ab80 1 -- 172.21.15.2:0/1730099368 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f433001d040 con 0x55653ac9ec10 2024-05-14T16:54:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.652+0000 7f4325ffb700 1 -- 172.21.15.2:0/1730099368 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f43340b3450 con 0x55653ac9ec10 2024-05-14T16:54:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.652+0000 7f4352c2ab80 10 client.15090 fetch_fsmap learned FSMap version 24 2024-05-14T16:54:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.652+0000 7f4352c2ab80 1 -- 172.21.15.2:0/1730099368 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55653acb52a0 con 0x55653ac9ec10 2024-05-14T16:54:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.652+0000 7f4325ffb700 1 -- 172.21.15.2:0/1730099368 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f433001d040 con 0x55653ac9ec10 2024-05-14T16:54:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.653+0000 7f4325ffb700 1 -- 172.21.15.2:0/1730099368 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f43340afbc0 con 0x55653ac9ec10 2024-05-14T16:54:31.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.653+0000 7f4352c2ab80 10 client.15090 fetch_fsmap finished waiting for FSMap version 24 2024-05-14T16:54:31.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.653+0000 7f4352c2ab80 10 client.15090 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T16:54:31.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.653+0000 7f4352c2ab80 20 client.15090 populate_metadata read hostname 'smithi002' 2024-05-14T16:54:31.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.653+0000 7f4352c2ab80 1 --2- 172.21.15.2:0/1730099368 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55653acb7860 0x55653acb8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:31.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.653+0000 7f4352c2ab80 4 client.15090 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:54:31.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.653+0000 7f4352c2ab80 1 -- 172.21.15.2:0/1730099368 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55653ac6a310 con 0x55653acb7860 2024-05-14T16:54:31.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.653+0000 7f4346130700 1 --2- 172.21.15.2:0/1730099368 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55653acb7860 0x55653acb8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:31.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.654+0000 7f4346130700 1 --2- 172.21.15.2:0/1730099368 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55653acb7860 0x55653acb8cd0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7518f5d39f0409fe server_cookie=35cd2ea68c2f6a1d in_seq=0 out_seq=0 2024-05-14T16:54:31.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.654+0000 7f4325ffb700 10 client.15090 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:54:31.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.654+0000 7f4325ffb700 1 -- 172.21.15.2:0/1730099368 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55653ac6a310 con 0x55653acb7860 2024-05-14T16:54:31.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.654+0000 7f4325ffb700 10 client.15090 handle_command_reply: tid=0 2024-05-14T16:54:31.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.655+0000 7f4352c2ab80 1 client.15090 shutdown 2024-05-14T16:54:31.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.655+0000 7f4352c2ab80 1 -- 172.21.15.2:0/1730099368 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55653ac9ec10 msgr2=0x55653aca71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:31.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.655+0000 7f4352c2ab80 1 --2- 172.21.15.2:0/1730099368 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55653ac9ec10 0x55653aca71f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f43340aa610 tx=0x7f43340aa240 comp rx=0 tx=0).stop 2024-05-14T16:54:31.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.655+0000 7f4352c2ab80 1 -- 172.21.15.2:0/1730099368 shutdown_connections 2024-05-14T16:54:31.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.655+0000 7f4352c2ab80 1 --2- 172.21.15.2:0/1730099368 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55653acb7860 0x55653acb8cd0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:31.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.655+0000 7f4352c2ab80 1 --2- 172.21.15.2:0/1730099368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55653aca2910 0x55653aca7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:31.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.655+0000 7f4352c2ab80 1 --2- 172.21.15.2:0/1730099368 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55653ac9e2f0 0x55653aca9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:31.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.655+0000 7f4352c2ab80 1 --2- 172.21.15.2:0/1730099368 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55653ac9ec10 0x55653aca71f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:31.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.655+0000 7f4325ffb700 0 client.15090 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:54:31.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.655+0000 7f4352c2ab80 1 -- 172.21.15.2:0/1730099368 >> 172.21.15.2:0/1730099368 conn(0x55653abc9200 msgr2=0x55653aca3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:31.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.655+0000 7f4352c2ab80 1 -- 172.21.15.2:0/1730099368 shutdown_connections 2024-05-14T16:54:31.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.655+0000 7f4352c2ab80 1 -- 172.21.15.2:0/1730099368 wait complete. 2024-05-14T16:54:31.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.655+0000 7f4352c2ab80 20 client.15090 trim_cache size 0 max 16384 2024-05-14T16:54:31.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.678+0000 7f4352c2ab80 1 Processor -- start 2024-05-14T16:54:31.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.678+0000 7f4352c2ab80 1 -- start start 2024-05-14T16:54:31.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.678+0000 7f4352c2ab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55653ac9e2f0 0x55653aca6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:31.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.678+0000 7f4352c2ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55653ac9ec10 0x55653aca0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:31.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.678+0000 7f4352c2ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55653aca2910 0x55653aca0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:31.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.678+0000 7f4352c2ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55653acb7010 con 0x55653ac9ec10 2024-05-14T16:54:31.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.678+0000 7f4352c2ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55653acb76e0 con 0x55653aca2910 2024-05-14T16:54:31.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.678+0000 7f4352c2ab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55653acb7db0 con 0x55653ac9e2f0 2024-05-14T16:54:31.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.678+0000 7f434592f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55653ac9ec10 0x55653aca0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:31.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.678+0000 7f434592f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55653ac9ec10 0x55653aca0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49176/0 (socket says 172.21.15.2:49176) 2024-05-14T16:54:31.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.678+0000 7f434592f700 1 -- 172.21.15.2:0/1307896459 learned_addr learned my addr 172.21.15.2:0/1307896459 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:31.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.678+0000 7f4346931700 1 --2- 172.21.15.2:0/1307896459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55653aca2910 0x55653aca0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:31.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.678+0000 7f4346130700 1 --2- 172.21.15.2:0/1307896459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55653ac9e2f0 0x55653aca6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:31.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.678+0000 7f434592f700 1 -- 172.21.15.2:0/1307896459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55653ac9e2f0 msgr2=0x55653aca6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:31.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.678+0000 7f434592f700 1 --2- 172.21.15.2:0/1307896459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55653ac9e2f0 0x55653aca6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:31.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.678+0000 7f434592f700 1 -- 172.21.15.2:0/1307896459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55653aca2910 msgr2=0x55653aca0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:31.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.678+0000 7f434592f700 1 --2- 172.21.15.2:0/1307896459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55653aca2910 0x55653aca0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:31.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.678+0000 7f434592f700 1 -- 172.21.15.2:0/1307896459 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55653acb8cd0 con 0x55653ac9ec10 2024-05-14T16:54:31.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.678+0000 7f4346931700 1 --2- 172.21.15.2:0/1307896459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55653aca2910 0x55653aca0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:54:31.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.678+0000 7f434592f700 1 --2- 172.21.15.2:0/1307896459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55653ac9ec10 0x55653aca0340 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f43340e3280 tx=0x7f43340a9df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30723ddc76ae6c3b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:31.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.679+0000 7f4327fff700 1 -- 172.21.15.2:0/1307896459 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43340aea30 con 0x55653ac9ec10 2024-05-14T16:54:31.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.679+0000 7f4327fff700 1 -- 172.21.15.2:0/1307896459 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4334092110 con 0x55653ac9ec10 2024-05-14T16:54:31.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.679+0000 7f4327fff700 1 -- 172.21.15.2:0/1307896459 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43340afe20 con 0x55653ac9ec10 2024-05-14T16:54:31.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.679+0000 7f4352c2ab80 1 -- 172.21.15.2:0/1307896459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55653ac9ec10 msgr2=0x55653aca0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:31.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.679+0000 7f4352c2ab80 1 --2- 172.21.15.2:0/1307896459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55653ac9ec10 0x55653aca0340 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f43340e3280 tx=0x7f43340a9df0 comp rx=0 tx=0).stop 2024-05-14T16:54:31.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.679+0000 7f4352c2ab80 1 -- 172.21.15.2:0/1307896459 shutdown_connections 2024-05-14T16:54:31.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.679+0000 7f4352c2ab80 1 --2- 172.21.15.2:0/1307896459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55653aca2910 0x55653aca0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:31.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.679+0000 7f4352c2ab80 1 --2- 172.21.15.2:0/1307896459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55653ac9e2f0 0x55653aca6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:31.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.679+0000 7f4352c2ab80 1 --2- 172.21.15.2:0/1307896459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55653ac9ec10 0x55653aca0340 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:31.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.679+0000 7f4352c2ab80 1 -- 172.21.15.2:0/1307896459 >> 172.21.15.2:0/1307896459 conn(0x55653abc9200 msgr2=0x55653ac9d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:31.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.679+0000 7f4352c2ab80 1 -- 172.21.15.2:0/1307896459 shutdown_connections 2024-05-14T16:54:31.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.679+0000 7f4352c2ab80 1 -- 172.21.15.2:0/1307896459 wait complete. 2024-05-14T16:54:31.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.680+0000 7f4352c2ab80 1 -- start start 2024-05-14T16:54:31.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.680+0000 7f4352c2ab80 1 Processor -- start 2024-05-14T16:54:31.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.680+0000 7f4352c2ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55653ac9e2f0 0x55653acc2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:31.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.680+0000 7f4352c2ab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55653ac9ec10 0x55653acc2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:31.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.680+0000 7f4352c2ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55653aca2910 0x55653acbd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:31.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.680+0000 7f4352c2ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55653acb8b50 con 0x55653aca2910 2024-05-14T16:54:31.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.680+0000 7f4352c2ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55653acb8480 con 0x55653ac9e2f0 2024-05-14T16:54:31.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.680+0000 7f4352c2ab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55653acb9220 con 0x55653ac9ec10 2024-05-14T16:54:31.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.680+0000 7f4352c2ab80 10 client.0 _ll_register_callbacks cb 0x55653ab83530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:54:31.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.680+0000 7f4346931700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55653aca2910 0x55653acbd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:31.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.680+0000 7f4346130700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55653ac9e2f0 0x55653acc2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:31.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.680+0000 7f4346931700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55653aca2910 0x55653acbd100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49180/0 (socket says 172.21.15.2:49180) 2024-05-14T16:54:31.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.680+0000 7f4346931700 1 -- 172.21.15.2:0/1694852989 learned_addr learned my addr 172.21.15.2:0/1694852989 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:31.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.680+0000 7f434592f700 1 --2- 172.21.15.2:0/1694852989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55653ac9ec10 0x55653acc2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:31.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.680+0000 7f4346931700 1 -- 172.21.15.2:0/1694852989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55653ac9ec10 msgr2=0x55653acc2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:31.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.680+0000 7f4346931700 1 --2- 172.21.15.2:0/1694852989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55653ac9ec10 0x55653acc2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:31.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.680+0000 7f4346931700 1 -- 172.21.15.2:0/1694852989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55653ac9e2f0 msgr2=0x55653acc2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:31.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.680+0000 7f4346931700 1 --2- 172.21.15.2:0/1694852989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55653ac9e2f0 0x55653acc2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:31.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.681+0000 7f4346931700 1 -- 172.21.15.2:0/1694852989 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55653acbd670 con 0x55653aca2910 2024-05-14T16:54:31.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.681+0000 7f4346931700 1 --2- 172.21.15.2:0/1694852989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55653aca2910 0x55653acbd100 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f433800dec0 tx=0x7f43380126d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda90561509f426b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:31.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.681+0000 7f4325ffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:54:31.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.681+0000 7f4325ffb700 1 -- 172.21.15.2:0/1694852989 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4338010860 con 0x55653aca2910 2024-05-14T16:54:31.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.681+0000 7f4325ffb700 1 -- 172.21.15.2:0/1694852989 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f4338010a00 con 0x55653aca2910 2024-05-14T16:54:31.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.681+0000 7f4352c2ab80 1 -- 172.21.15.2:0/1694852989 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4334092110 con 0x55653aca2910 2024-05-14T16:54:31.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.681+0000 7f4325ffb700 1 -- 172.21.15.2:0/1694852989 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4338010c00 con 0x55653aca2910 2024-05-14T16:54:31.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.681+0000 7f4352c2ab80 10 client.15096 fetch_fsmap learned FSMap version 24 2024-05-14T16:54:31.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.681+0000 7f4352c2ab80 1 -- 172.21.15.2:0/1694852989 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55653ad23590 con 0x55653aca2910 2024-05-14T16:54:31.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.681+0000 7f4325ffb700 1 -- 172.21.15.2:0/1694852989 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4334092110 con 0x55653aca2910 2024-05-14T16:54:31.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.681+0000 7f4325ffb700 1 -- 172.21.15.2:0/1694852989 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f4338019070 con 0x55653aca2910 2024-05-14T16:54:31.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.681+0000 7f4352c2ab80 10 client.15096 fetch_fsmap finished waiting for FSMap version 24 2024-05-14T16:54:31.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.681+0000 7f4352c2ab80 10 client.15096 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T16:54:31.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.681+0000 7f4352c2ab80 20 client.15096 populate_metadata read hostname 'smithi002' 2024-05-14T16:54:31.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.682+0000 7f4352c2ab80 1 --2- 172.21.15.2:0/1694852989 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55653ad28110 0x55653ad26e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:31.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.682+0000 7f4352c2ab80 4 client.15096 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T16:54:31.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.682+0000 7f4352c2ab80 1 -- 172.21.15.2:0/1694852989 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "status"}) -- 0x55653abfeb90 con 0x55653ad28110 2024-05-14T16:54:31.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.682+0000 7f4346130700 1 --2- 172.21.15.2:0/1694852989 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55653ad28110 0x55653ad26e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:31.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.682+0000 7f4346130700 1 --2- 172.21.15.2:0/1694852989 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55653ad28110 0x55653ad26e20 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1800641f238aa67 server_cookie=88a8acf4d14573e5 in_seq=0 out_seq=0 2024-05-14T16:54:31.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.682+0000 7f4325ffb700 10 client.15096 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:54:31.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.682+0000 7f4325ffb700 1 -- 172.21.15.2:0/1694852989 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55653abfeb90 con 0x55653ad28110 2024-05-14T16:54:31.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.682+0000 7f4325ffb700 10 client.15096 handle_command_reply: tid=0 2024-05-14T16:54:31.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.682+0000 7f4352c2ab80 1 client.15096 shutdown 2024-05-14T16:54:31.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.683+0000 7f4352c2ab80 1 -- 172.21.15.2:0/1694852989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55653aca2910 msgr2=0x55653acbd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:31.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.683+0000 7f4352c2ab80 1 --2- 172.21.15.2:0/1694852989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55653aca2910 0x55653acbd100 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f433800dec0 tx=0x7f43380126d0 comp rx=0 tx=0).stop 2024-05-14T16:54:31.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.683+0000 7f4352c2ab80 1 -- 172.21.15.2:0/1694852989 shutdown_connections 2024-05-14T16:54:31.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.683+0000 7f4352c2ab80 1 --2- 172.21.15.2:0/1694852989 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55653ad28110 0x55653ad26e20 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:31.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.683+0000 7f4352c2ab80 1 --2- 172.21.15.2:0/1694852989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55653aca2910 0x55653acbd100 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:31.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.683+0000 7f4352c2ab80 1 --2- 172.21.15.2:0/1694852989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55653ac9e2f0 0x55653acc2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:31.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.683+0000 7f4352c2ab80 1 --2- 172.21.15.2:0/1694852989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55653ac9ec10 0x55653acc2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:31.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.683+0000 7f4325ffb700 0 client.15096 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:54:31.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.683+0000 7f4352c2ab80 1 -- 172.21.15.2:0/1694852989 >> 172.21.15.2:0/1694852989 conn(0x55653abc9200 msgr2=0x55653aca1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:31.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.683+0000 7f4352c2ab80 1 -- 172.21.15.2:0/1694852989 shutdown_connections 2024-05-14T16:54:31.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.683+0000 7f4352c2ab80 1 -- 172.21.15.2:0/1694852989 wait complete. 2024-05-14T16:54:31.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.683+0000 7f4352c2ab80 20 client.15096 trim_cache size 0 max 16384 2024-05-14T16:54:31.695 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T16:54:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "ed40717a-1210-11ef-bc98-c7b262605968", 2024-05-14T16:54:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 0, 2024-05-14T16:54:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24479, 2024-05-14T16:54:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2024-05-14T16:54:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2024-05-14T16:54:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2024-05-14T16:54:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 177.391732631, 2024-05-14T16:54:31.696 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 24, 2024-05-14T16:54:31.696 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 80, 2024-05-14T16:54:31.696 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 78, 2024-05-14T16:54:31.696 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 193.24294262000001 2024-05-14T16:54:31.696 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T16:54:31.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.683+0000 7f4348394700 1 -- 172.21.15.2:0/846686442 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f43280b2880 msgr2=0x7f43280b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:31.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.683+0000 7f4348394700 1 --2- 172.21.15.2:0/846686442 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f43280b2880 0x7f43280b4d40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4330009550 tx=0x7f43300091b0 comp rx=0 tx=0).stop 2024-05-14T16:54:31.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.683+0000 7f4348394700 1 -- 172.21.15.2:0/846686442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4340076090 msgr2=0x7f434019c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:31.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.683+0000 7f4348394700 1 --2- 172.21.15.2:0/846686442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4340076090 0x7f434019c350 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4334000c00 tx=0x7f43340047b0 comp rx=0 tx=0).stop 2024-05-14T16:54:31.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.683+0000 7f4348394700 1 -- 172.21.15.2:0/846686442 shutdown_connections 2024-05-14T16:54:31.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.683+0000 7f4348394700 1 --2- 172.21.15.2:0/846686442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f434010ff40 0x7f43401a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:31.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.683+0000 7f4348394700 1 --2- 172.21.15.2:0/846686442 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4340075770 0x7f434019be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:31.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.683+0000 7f4348394700 1 --2- 172.21.15.2:0/846686442 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f43280b2880 0x7f43280b4d40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:31.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.683+0000 7f4348394700 1 --2- 172.21.15.2:0/846686442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4340076090 0x7f434019c350 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:31.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.683+0000 7f4348394700 1 -- 172.21.15.2:0/846686442 >> 172.21.15.2:0/846686442 conn(0x7f434007a6a0 msgr2=0x7f4340100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:31.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.684+0000 7f4348394700 1 -- 172.21.15.2:0/846686442 shutdown_connections 2024-05-14T16:54:31.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:31.684+0000 7f4348394700 1 -- 172.21.15.2:0/846686442 wait complete. 2024-05-14T16:54:31.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:31 smithi151 ceph-mon[36066]: pgmap v379: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T16:54:31.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:31 smithi123 ceph-mon[35971]: pgmap v379: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T16:54:32.486 INFO:teuthology.run_tasks:Running task sequential... 2024-05-14T16:54:32.588 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-14T16:54:32.589 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-14T16:54:32.589 INFO:teuthology.task.print:Enabling mgr modules 2024-05-14T16:54:32.589 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-05-14T16:54:32.590 INFO:teuthology.task.exec:Executing custom commands... 2024-05-14T16:54:32.590 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi002.front.sepia.ceph.com 2024-05-14T16:54:32.590 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-05-14T16:54:32.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.705+0000 7fbccde54640 1 -- 172.21.15.2:0/3251116419 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbcc8103610 msgr2=0x7fbcc8109ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:32.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.705+0000 7fbccde54640 1 --2- 172.21.15.2:0/3251116419 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbcc8103610 0x7fbcc8109ea0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0009ca0 tx=0x7fbcc002a8c0 comp rx=0 tx=0).stop 2024-05-14T16:54:32.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.705+0000 7fbccde54640 1 -- 172.21.15.2:0/3251116419 shutdown_connections 2024-05-14T16:54:32.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.705+0000 7fbccde54640 1 --2- 172.21.15.2:0/3251116419 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbcc8103610 0x7fbcc8109ea0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:32.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.705+0000 7fbccde54640 1 --2- 172.21.15.2:0/3251116419 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbcc8102c50 0x7fbcc81030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:32.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.705+0000 7fbccde54640 1 --2- 172.21.15.2:0/3251116419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcc8101a60 0x7fbcc8101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:32.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.705+0000 7fbccde54640 1 -- 172.21.15.2:0/3251116419 >> 172.21.15.2:0/3251116419 conn(0x7fbcc80fcfd0 msgr2=0x7fbcc80ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:32.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.705+0000 7fbccde54640 1 -- 172.21.15.2:0/3251116419 shutdown_connections 2024-05-14T16:54:32.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.705+0000 7fbccde54640 1 -- 172.21.15.2:0/3251116419 wait complete. 2024-05-14T16:54:32.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.705+0000 7fbccde54640 1 Processor -- start 2024-05-14T16:54:32.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.706+0000 7fbccde54640 1 -- start start 2024-05-14T16:54:32.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.706+0000 7fbccde54640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbcc8101a60 0x7fbcc810c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:32.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.706+0000 7fbccde54640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcc8102c50 0x7fbcc810a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:32.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.706+0000 7fbccde54640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbcc8103610 0x7fbcc810ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:32.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.706+0000 7fbccde54640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc810e630 con 0x7fbcc8103610 2024-05-14T16:54:32.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.706+0000 7fbccde54640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc810e4b0 con 0x7fbcc8102c50 2024-05-14T16:54:32.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.706+0000 7fbccde54640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc810e7b0 con 0x7fbcc8101a60 2024-05-14T16:54:32.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.706+0000 7fbcc7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbcc8103610 0x7fbcc810ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:32.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.706+0000 7fbcc7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbcc8103610 0x7fbcc810ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49206/0 (socket says 172.21.15.2:49206) 2024-05-14T16:54:32.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.706+0000 7fbcc7fff640 1 -- 172.21.15.2:0/2158740081 learned_addr learned my addr 172.21.15.2:0/2158740081 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:32.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.706+0000 7fbcc6ffd640 1 --2- 172.21.15.2:0/2158740081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcc8102c50 0x7fbcc810a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:32.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.706+0000 7fbcc77fe640 1 --2- 172.21.15.2:0/2158740081 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbcc8101a60 0x7fbcc810c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:32.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.706+0000 7fbcc7fff640 1 -- 172.21.15.2:0/2158740081 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbcc8101a60 msgr2=0x7fbcc810c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:32.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.706+0000 7fbcc7fff640 1 --2- 172.21.15.2:0/2158740081 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbcc8101a60 0x7fbcc810c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:32.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.706+0000 7fbcc7fff640 1 -- 172.21.15.2:0/2158740081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcc8102c50 msgr2=0x7fbcc810a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:32.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.706+0000 7fbcc7fff640 1 --2- 172.21.15.2:0/2158740081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcc8102c50 0x7fbcc810a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:32.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.706+0000 7fbcc7fff640 1 -- 172.21.15.2:0/2158740081 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc810b6a0 con 0x7fbcc8103610 2024-05-14T16:54:32.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.706+0000 7fbcc77fe640 1 --2- 172.21.15.2:0/2158740081 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbcc8101a60 0x7fbcc810c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:54:32.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.706+0000 7fbcc7fff640 1 --2- 172.21.15.2:0/2158740081 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbcc8103610 0x7fbcc810ade0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbcc00097b0 tx=0x7fbcc0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:32.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.707+0000 7fbcc4ff9640 1 -- 172.21.15.2:0/2158740081 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0004240 con 0x7fbcc8103610 2024-05-14T16:54:32.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.707+0000 7fbcc4ff9640 1 -- 172.21.15.2:0/2158740081 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcc00317c0 con 0x7fbcc8103610 2024-05-14T16:54:32.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.707+0000 7fbccde54640 1 -- 172.21.15.2:0/2158740081 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcc81a2c90 con 0x7fbcc8103610 2024-05-14T16:54:32.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.707+0000 7fbcc4ff9640 1 -- 172.21.15.2:0/2158740081 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc002fd80 con 0x7fbcc8103610 2024-05-14T16:54:32.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.707+0000 7fbccde54640 1 -- 172.21.15.2:0/2158740081 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcc81a31b0 con 0x7fbcc8103610 2024-05-14T16:54:32.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.708+0000 7fbccde54640 1 -- 172.21.15.2:0/2158740081 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcc8108d80 con 0x7fbcc8103610 2024-05-14T16:54:32.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.709+0000 7fbcc4ff9640 1 -- 172.21.15.2:0/2158740081 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fbcc00043e0 con 0x7fbcc8103610 2024-05-14T16:54:32.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.710+0000 7fbcc4ff9640 1 --2- 172.21.15.2:0/2158740081 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fbcb00b2920 0x7fbcb00b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:32.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.710+0000 7fbcc77fe640 1 --2- 172.21.15.2:0/2158740081 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fbcb00b2920 0x7fbcb00b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:32.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.710+0000 7fbcc4ff9640 1 -- 172.21.15.2:0/2158740081 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7fbcc0103c00 con 0x7fbcc8103610 2024-05-14T16:54:32.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.711+0000 7fbcc77fe640 1 --2- 172.21.15.2:0/2158740081 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fbcb00b2920 0x7fbcb00b4de0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbcb80058c0 tx=0x7fbcb8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:32.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.714+0000 7fbcc4ff9640 1 -- 172.21.15.2:0/2158740081 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbcc00cb8d0 con 0x7fbcc8103610 2024-05-14T16:54:32.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:32.956+0000 7fbccde54640 1 -- 172.21.15.2:0/2158740081 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fbcc802dba0 con 0x7fbcc8103610 2024-05-14T16:54:33.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:33.499+0000 7fbcc4ff9640 1 -- 172.21.15.2:0/2158740081 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7fbcc00cb1d0 con 0x7fbcc8103610 2024-05-14T16:54:33.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:33.503+0000 7fbccde54640 1 -- 172.21.15.2:0/2158740081 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fbcb00b2920 msgr2=0x7fbcb00b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:33.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:33.503+0000 7fbccde54640 1 --2- 172.21.15.2:0/2158740081 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fbcb00b2920 0x7fbcb00b4de0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbcb80058c0 tx=0x7fbcb8005850 comp rx=0 tx=0).stop 2024-05-14T16:54:33.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:33.503+0000 7fbccde54640 1 -- 172.21.15.2:0/2158740081 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbcc8103610 msgr2=0x7fbcc810ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:33.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:33.503+0000 7fbccde54640 1 --2- 172.21.15.2:0/2158740081 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbcc8103610 0x7fbcc810ade0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbcc00097b0 tx=0x7fbcc0003c20 comp rx=0 tx=0).stop 2024-05-14T16:54:33.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:33.503+0000 7fbccde54640 1 -- 172.21.15.2:0/2158740081 shutdown_connections 2024-05-14T16:54:33.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:33.503+0000 7fbccde54640 1 --2- 172.21.15.2:0/2158740081 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fbcb00b2920 0x7fbcb00b4de0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:33.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:33.503+0000 7fbccde54640 1 --2- 172.21.15.2:0/2158740081 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbcc8103610 0x7fbcc810ade0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:33.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:33.503+0000 7fbccde54640 1 --2- 172.21.15.2:0/2158740081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcc8102c50 0x7fbcc810a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:33.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:33.503+0000 7fbccde54640 1 --2- 172.21.15.2:0/2158740081 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbcc8101a60 0x7fbcc810c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:33.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:33.503+0000 7fbccde54640 1 -- 172.21.15.2:0/2158740081 >> 172.21.15.2:0/2158740081 conn(0x7fbcc80fcfd0 msgr2=0x7fbcc8105f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:33.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:33.503+0000 7fbccde54640 1 -- 172.21.15.2:0/2158740081 shutdown_connections 2024-05-14T16:54:33.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:33.503+0000 7fbccde54640 1 -- 172.21.15.2:0/2158740081 wait complete. 2024-05-14T16:54:33.523 INFO:teuthology.run_tasks:Running task kclient... 2024-05-14T16:54:33.546 INFO:tasks.kclient:Mounting kernel clients... 2024-05-14T16:54:33.546 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-14T16:54:33.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T16:54:33.546 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list 2024-05-14T16:54:33.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T16:54:33.581 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link delete ceph-brx 2024-05-14T16:54:33.650 INFO:teuthology.orchestra.run.smithi002.stderr:Cannot find device "ceph-brx" 2024-05-14T16:54:33.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T16:54:33.652 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v2', 'mount_subvol_num': 1, 'mountpoint': '/tmp/cephfs'} 2024-05-14T16:54:33.653 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-14T16:54:33.653 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2024-05-14T16:54:33.653 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2024-05-14T16:54:33.653 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs ls 2024-05-14T16:54:33.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:33 smithi151 ceph-mon[36066]: pgmap v380: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:54:33.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:33 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/2158740081' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-14T16:54:33.855 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:54:33 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: ignoring --setuser ceph since I am not root 2024-05-14T16:54:33.855 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:54:33 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: ignoring --setgroup ceph since I am not root 2024-05-14T16:54:33.855 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:54:33 smithi151 ceph-mgr[37903]: -- 172.21.15.151:0/3980573195 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557f79fb85a0 con 0x557f79f8c800 2024-05-14T16:54:33.855 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:54:33 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:54:33.684+0000 7f52a1261200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-14T16:54:33.866 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:33 smithi123 ceph-mon[35971]: pgmap v380: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:54:33.866 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:33 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/2158740081' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-14T16:54:33.866 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:54:33 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: ignoring --setuser ceph since I am not root 2024-05-14T16:54:33.866 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:54:33 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: ignoring --setgroup ceph since I am not root 2024-05-14T16:54:33.867 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:54:33 smithi123 ceph-mgr[37342]: -- 172.21.15.123:0/3195513219 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55afe91725a0 con 0x55afe9146c00 2024-05-14T16:54:33.867 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:54:33 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:54:33.682+0000 7f547f656200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-14T16:54:33.867 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:54:33 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:54:33.866+0000 7f547f656200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:54:33.894 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:33 smithi002 ceph-mon[31538]: pgmap v380: 97 pgs: 97 active+clean; 641 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T16:54:33.894 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:33 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/2158740081' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-14T16:54:33.895 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:54:33 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: ignoring --setuser ceph since I am not root 2024-05-14T16:54:33.895 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:54:33 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: ignoring --setgroup ceph since I am not root 2024-05-14T16:54:33.895 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:54:33 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:54:33.693+0000 7f67e5f41200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-14T16:54:33.895 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:54:33 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:54:33.894+0000 7f67e5f41200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:54:33.951 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:54:34.159 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:54:33 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:54:33.968+0000 7f547f656200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-14T16:54:34.183 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:54:34 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:54:34.018+0000 7f67e5f41200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-14T16:54:34.275 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:54:33 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:54:33.868+0000 7f52a1261200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:54:34.275 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:54:33 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:54:33.969+0000 7f52a1261200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-14T16:54:34.463 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:54:34 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:54:34.369+0000 7f67e5f41200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-14T16:54:34.464 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:54:34 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:54:34.463+0000 7f67e5f41200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-14T16:54:34.604 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:54:34 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:54:34.275+0000 7f52a1261200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-14T16:54:34.604 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:54:34 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:54:34.360+0000 7f52a1261200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-14T16:54:34.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:34 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/2158740081' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-14T16:54:34.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:34 smithi151 ceph-mon[36066]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-05-14T16:54:34.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:34 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/2158740081' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-14T16:54:34.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:34 smithi123 ceph-mon[35971]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-05-14T16:54:34.660 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:54:34 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:54:34.277+0000 7f547f656200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-14T16:54:34.660 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:54:34 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:54:34.363+0000 7f547f656200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-14T16:54:34.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:34 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/2158740081' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-14T16:54:34.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:34 smithi002 ceph-mon[31538]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-05-14T16:54:35.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.815+0000 7f52be625700 1 -- 172.21.15.2:0/3550798239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52b806c470 msgr2=0x7f52b81103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:35.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.815+0000 7f52be625700 1 --2- 172.21.15.2:0/3550798239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52b806c470 0x7f52b81103c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f52b000b0a0 tx=0x7f52b000b3b0 comp rx=0 tx=0).stop 2024-05-14T16:54:35.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.815+0000 7f52be625700 1 -- 172.21.15.2:0/3550798239 shutdown_connections 2024-05-14T16:54:35.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.815+0000 7f52be625700 1 --2- 172.21.15.2:0/3550798239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52b806c470 0x7f52b81103c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:35.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.815+0000 7f52be625700 1 --2- 172.21.15.2:0/3550798239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b81095a0 0x7f52b8109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:35.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.815+0000 7f52be625700 1 --2- 172.21.15.2:0/3550798239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52b806bab0 0x7f52b806bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:35.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.815+0000 7f52be625700 1 -- 172.21.15.2:0/3550798239 >> 172.21.15.2:0/3550798239 conn(0x7f52b80793b0 msgr2=0x7f52b80797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:35.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.815+0000 7f52be625700 1 -- 172.21.15.2:0/3550798239 shutdown_connections 2024-05-14T16:54:35.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.815+0000 7f52be625700 1 -- 172.21.15.2:0/3550798239 wait complete. 2024-05-14T16:54:35.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.816+0000 7f52be625700 1 Processor -- start 2024-05-14T16:54:35.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.816+0000 7f52be625700 1 -- start start 2024-05-14T16:54:35.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.816+0000 7f52be625700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52b806bab0 0x7f52b8102500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:35.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.816+0000 7f52be625700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b806c470 0x7f52b8102a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:35.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.816+0000 7f52be625700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52b81095a0 0x7f52b8102f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:35.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.816+0000 7f52be625700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f52b81136d0 con 0x7f52b81095a0 2024-05-14T16:54:35.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.816+0000 7f52be625700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f52b8113550 con 0x7f52b806c470 2024-05-14T16:54:35.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.816+0000 7f52be625700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f52b8113850 con 0x7f52b806bab0 2024-05-14T16:54:35.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.816+0000 7f52bcbc2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52b81095a0 0x7f52b8102f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:35.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.817+0000 7f52bcbc2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52b81095a0 0x7f52b8102f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40716/0 (socket says 172.21.15.2:40716) 2024-05-14T16:54:35.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.817+0000 7f52bcbc2700 1 -- 172.21.15.2:0/170759041 learned_addr learned my addr 172.21.15.2:0/170759041 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:35.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.817+0000 7f52b7fff700 1 --2- 172.21.15.2:0/170759041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52b806bab0 0x7f52b8102500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:35.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.817+0000 7f52b77fe700 1 --2- 172.21.15.2:0/170759041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b806c470 0x7f52b8102a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:35.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.817+0000 7f52b77fe700 1 -- 172.21.15.2:0/170759041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52b806bab0 msgr2=0x7f52b8102500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:35.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.817+0000 7f52b77fe700 1 --2- 172.21.15.2:0/170759041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52b806bab0 0x7f52b8102500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:35.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.817+0000 7f52b77fe700 1 -- 172.21.15.2:0/170759041 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52b81095a0 msgr2=0x7f52b8102f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:35.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.817+0000 7f52b77fe700 1 --2- 172.21.15.2:0/170759041 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52b81095a0 0x7f52b8102f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:35.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.817+0000 7f52b77fe700 1 -- 172.21.15.2:0/170759041 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b81a80f0 con 0x7f52b806c470 2024-05-14T16:54:35.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.817+0000 7f52bcbc2700 1 --2- 172.21.15.2:0/170759041 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52b81095a0 0x7f52b8102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:54:35.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.817+0000 7f52b77fe700 1 --2- 172.21.15.2:0/170759041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b806c470 0x7f52b8102a40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f52a000e9f0 tx=0x7f52a000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:35.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.817+0000 7f52b57fa700 1 -- 172.21.15.2:0/170759041 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a0004220 con 0x7f52b806c470 2024-05-14T16:54:35.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.817+0000 7f52be625700 1 -- 172.21.15.2:0/170759041 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b81a83e0 con 0x7f52b806c470 2024-05-14T16:54:35.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.817+0000 7f52b57fa700 1 -- 172.21.15.2:0/170759041 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52a00043c0 con 0x7f52b806c470 2024-05-14T16:54:35.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.817+0000 7f52b57fa700 1 -- 172.21.15.2:0/170759041 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a0005020 con 0x7f52b806c470 2024-05-14T16:54:35.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.817+0000 7f52be625700 1 -- 172.21.15.2:0/170759041 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b81a88d0 con 0x7f52b806c470 2024-05-14T16:54:35.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.818+0000 7f52be625700 1 -- 172.21.15.2:0/170759041 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52b806a230 con 0x7f52b806c470 2024-05-14T16:54:35.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.820+0000 7f52b57fa700 1 -- 172.21.15.2:0/170759041 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 19) ==== 149668+0+0 (secure 0 0 0) 0x7f52a0004750 con 0x7f52b806c470 2024-05-14T16:54:35.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.820+0000 7f52b57fa700 1 --2- 172.21.15.2:0/170759041 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f52a40b28d0 0x7f52a40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:35.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.820+0000 7f52b7fff700 1 -- 172.21.15.2:0/170759041 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f52a40b28d0 msgr2=0x7f52a40b4d90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/1498936028 2024-05-14T16:54:35.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.821+0000 7f52b57fa700 1 -- 172.21.15.2:0/170759041 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f52a00e22d0 con 0x7f52b806c470 2024-05-14T16:54:35.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.821+0000 7f52b7fff700 1 --2- 172.21.15.2:0/170759041 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f52a40b28d0 0x7f52a40b4d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T16:54:35.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:35.824+0000 7f52b57fa700 1 -- 172.21.15.2:0/170759041 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f52a00aa030 con 0x7f52b806c470 2024-05-14T16:54:35.854 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:54:35 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:54:35.537+0000 7f52a1261200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-14T16:54:35.854 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:54:35 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:54:35.708+0000 7f52a1261200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-14T16:54:35.876 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:54:35 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:54:35.538+0000 7f547f656200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-14T16:54:35.876 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:54:35 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:54:35.706+0000 7f547f656200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-14T16:54:35.876 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:54:35 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:54:35.876+0000 7f547f656200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-14T16:54:35.993 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:54:35 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:54:35.741+0000 7f67e5f41200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-14T16:54:35.993 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:54:35 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:54:35.926+0000 7f67e5f41200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-14T16:54:36.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:36.022+0000 7f52b7fff700 1 -- 172.21.15.2:0/170759041 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f52a40b28d0 msgr2=0x7f52a40b4d90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/1498936028 2024-05-14T16:54:36.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:36.022+0000 7f52b7fff700 1 --2- 172.21.15.2:0/170759041 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f52a40b28d0 0x7f52a40b4d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-14T16:54:36.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:36.086+0000 7f52be625700 1 -- 172.21.15.2:0/170759041 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f52b802e560 con 0x7f52b806c470 2024-05-14T16:54:36.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:36.086+0000 7f52b57fa700 1 -- 172.21.15.2:0/170759041 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7f52a00a9930 con 0x7f52b806c470 2024-05-14T16:54:36.086 INFO:teuthology.orchestra.run.smithi002.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-14T16:54:36.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:36.089+0000 7f52be625700 1 -- 172.21.15.2:0/170759041 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f52a40b28d0 msgr2=0x7f52a40b4d90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T16:54:36.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:36.089+0000 7f52be625700 1 --2- 172.21.15.2:0/170759041 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f52a40b28d0 0x7f52a40b4d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:36.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:36.089+0000 7f52be625700 1 -- 172.21.15.2:0/170759041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b806c470 msgr2=0x7f52b8102a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:36.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:36.089+0000 7f52be625700 1 --2- 172.21.15.2:0/170759041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b806c470 0x7f52b8102a40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f52a000e9f0 tx=0x7f52a000ed00 comp rx=0 tx=0).stop 2024-05-14T16:54:36.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:36.089+0000 7f52be625700 1 -- 172.21.15.2:0/170759041 shutdown_connections 2024-05-14T16:54:36.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:36.089+0000 7f52be625700 1 --2- 172.21.15.2:0/170759041 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f52a40b28d0 0x7f52a40b4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:36.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:36.089+0000 7f52be625700 1 --2- 172.21.15.2:0/170759041 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52b81095a0 0x7f52b8102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:36.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:36.089+0000 7f52be625700 1 --2- 172.21.15.2:0/170759041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52b806bab0 0x7f52b8102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:36.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:36.089+0000 7f52be625700 1 --2- 172.21.15.2:0/170759041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b806c470 0x7f52b8102a40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:36.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:36.089+0000 7f52be625700 1 -- 172.21.15.2:0/170759041 >> 172.21.15.2:0/170759041 conn(0x7f52b80793b0 msgr2=0x7f52b81010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:36.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:36.089+0000 7f52be625700 1 -- 172.21.15.2:0/170759041 shutdown_connections 2024-05-14T16:54:36.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:36.089+0000 7f52be625700 1 -- 172.21.15.2:0/170759041 wait complete. 2024-05-14T16:54:36.138 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:54:35 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:54:35.880+0000 7f52a1261200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-14T16:54:36.409 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:36 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/170759041' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-14T16:54:36.410 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:54:36 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:54:36.166+0000 7f547f656200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-14T16:54:36.410 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:54:36 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:54:36.257+0000 7f547f656200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-14T16:54:36.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:36 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/170759041' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-14T16:54:36.412 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:54:36 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:54:36.109+0000 7f67e5f41200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-14T16:54:36.471 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:36 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/170759041' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-14T16:54:36.472 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:54:36 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:54:36.171+0000 7f52a1261200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-14T16:54:36.472 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:54:36 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:54:36.264+0000 7f52a1261200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-14T16:54:36.732 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:54:36 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:54:36.420+0000 7f67e5f41200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-14T16:54:36.732 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:54:36 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:54:36.518+0000 7f67e5f41200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-14T16:54:36.732 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:54:36 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:54:36.732+0000 7f67e5f41200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-14T16:54:36.768 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-14T16:54:36.768 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-14T16:54:36.768 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi002.front.sepia.ceph.com 2024-05-14T16:54:36.768 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-14T16:54:36.768 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2024-05-14T16:54:36.768 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-14T16:54:36.768 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-14T16:54:36.768 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-14T16:54:36.768 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2024-05-14T16:54:36.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T16:54:36.768 DEBUG:teuthology.orchestra.run.smithi002:> ip addr 2024-05-14T16:54:36.785 INFO:teuthology.orchestra.run.smithi002.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-14T16:54:36.786 INFO:teuthology.orchestra.run.smithi002.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-14T16:54:36.786 INFO:teuthology.orchestra.run.smithi002.stdout: inet 127.0.0.1/8 scope host lo 2024-05-14T16:54:36.786 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2024-05-14T16:54:36.786 INFO:teuthology.orchestra.run.smithi002.stdout: inet6 ::1/128 scope host 2024-05-14T16:54:36.786 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2024-05-14T16:54:36.786 INFO:teuthology.orchestra.run.smithi002.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-14T16:54:36.786 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:6b:fe:02 brd ff:ff:ff:ff:ff:ff 2024-05-14T16:54:36.786 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp7s0f0 2024-05-14T16:54:36.786 INFO:teuthology.orchestra.run.smithi002.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-14T16:54:36.786 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:6b:fe:03 brd ff:ff:ff:ff:ff:ff 2024-05-14T16:54:36.786 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp7s0f1 2024-05-14T16:54:36.786 INFO:teuthology.orchestra.run.smithi002.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-14T16:54:36.787 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:bd:19:20 brd ff:ff:ff:ff:ff:ff 2024-05-14T16:54:36.787 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp1s0f0 2024-05-14T16:54:36.787 INFO:teuthology.orchestra.run.smithi002.stdout: inet 172.21.15.2/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-14T16:54:36.787 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft 42071sec preferred_lft 42071sec 2024-05-14T16:54:36.787 INFO:teuthology.orchestra.run.smithi002.stdout: inet6 fe80::ec4:7aff:febd:1920/64 scope link noprefixroute 2024-05-14T16:54:36.787 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2024-05-14T16:54:36.787 INFO:teuthology.orchestra.run.smithi002.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-14T16:54:36.787 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:bd:19:21 brd ff:ff:ff:ff:ff:ff 2024-05-14T16:54:36.787 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp1s0f1 2024-05-14T16:54:36.787 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-14T16:54:36.788 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T16:54:36.788 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-05-14T16:54:36.788 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link add name ceph-brx type bridge 2024-05-14T16:54:36.788 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip addr flush dev ceph-brx 2024-05-14T16:54:36.788 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set ceph-brx up 2024-05-14T16:54:36.788 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-14T16:54:36.788 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-05-14T16:54:36.854 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:54:36 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:54:36.471+0000 7f52a1261200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-14T16:54:36.854 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:54:36 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:54:36.667+0000 7f52a1261200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:54:36.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T16:54:36.909 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:54:36 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:54:36.460+0000 7f547f656200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-14T16:54:36.909 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:54:36 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:54:36.652+0000 7f547f656200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:54:36.982 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:54:36 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:54:36.943+0000 7f67e5f41200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-14T16:54:36.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T16:54:37.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T16:54:37.004 DEBUG:teuthology.orchestra.run.smithi002:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-14T16:54:37.032 INFO:teuthology.orchestra.run.smithi002.stdout:1 2024-05-14T16:54:37.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T16:54:37.034 DEBUG:teuthology.orchestra.run.smithi002:> route 2024-05-14T16:54:37.105 INFO:teuthology.orchestra.run.smithi002.stdout:Kernel IP routing table 2024-05-14T16:54:37.105 INFO:teuthology.orchestra.run.smithi002.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-14T16:54:37.105 INFO:teuthology.orchestra.run.smithi002.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-14T16:54:37.105 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-14T16:54:37.105 INFO:teuthology.orchestra.run.smithi002.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-14T16:54:37.106 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T16:54:37.106 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-05-14T16:54:37.106 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-14T16:54:37.106 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-14T16:54:37.106 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-14T16:54:37.106 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-05-14T16:54:37.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T16:54:37.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T16:54:37.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T16:54:37.391 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list 2024-05-14T16:54:37.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T16:54:37.408 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list-id 2024-05-14T16:54:37.465 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T16:54:37.465 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-05-14T16:54:37.465 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns add ceph-ns--tmp-cephfs 2024-05-14T16:54:37.465 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns set ceph-ns--tmp-cephfs 0 2024-05-14T16:54:37.465 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-05-14T16:54:37.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T16:54:37.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T16:54:37.588 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2024-05-14T16:54:37.588 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T16:54:37.588 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-05-14T16:54:37.588 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2024-05-14T16:54:37.588 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-14T16:54:37.588 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2024-05-14T16:54:37.588 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2024-05-14T16:54:37.588 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2024-05-14T16:54:37.588 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-05-14T16:54:37.604 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:54:37 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:54:37.270+0000 7f52a1261200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-14T16:54:37.604 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:54:37 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:54:37.492+0000 7f52a1261200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-14T16:54:37.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T16:54:37.659 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:54:37 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:54:37.247+0000 7f547f656200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-14T16:54:37.659 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:54:37 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:54:37.466+0000 7f547f656200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-14T16:54:37.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T16:54:37.731 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-14T16:54:37.732 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-05-14T16:54:37.732 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set brx.0 up 2024-05-14T16:54:37.732 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set dev brx.0 master ceph-brx 2024-05-14T16:54:37.732 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-05-14T16:54:37.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T16:54:37.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T16:54:37.815 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T16:54:37.815 DEBUG:teuthology.orchestra.run.smithi002:> mkdir -p -v /tmp/cephfs 2024-05-14T16:54:37.842 INFO:teuthology.orchestra.run.smithi002.stdout:mkdir: created directory '/tmp/cephfs' 2024-05-14T16:54:37.843 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-14T16:54:37.843 DEBUG:teuthology.orchestra.run.smithi002:> chmod 0000 /tmp/cephfs 2024-05-14T16:54:37.855 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:54:37 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:54:37.609+0000 7f67e5f41200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-14T16:54:37.855 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:54:37 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:54:37.855+0000 7f67e5f41200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-14T16:54:37.870 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-14T16:54:37.870 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-14T16:54:37.870 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/83bdfcd3-fc78-423b-b57a-a691f0db09d1 2024-05-14T16:54:37.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T16:54:37.871 DEBUG:teuthology.orchestra.run.smithi002:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/83bdfcd3-fc78-423b-b57a-a691f0db09d1 /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,context=system_u:object_r:postgresql_db_t:s0,nofallback 2024-05-14T16:54:37.982 INFO:teuthology.orchestra.run.smithi002.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2024-05-14T16:54:37.982 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-05-14T16:54:38.110 INFO:teuthology.orchestra.run.smithi002.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2024-05-14T16:54:38.110 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-05-14T16:54:38.110 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: resolved to: "172.21.15.2:3300,172.21.15.123:3300,172.21.15.151:3300" 2024-05-14T16:54:38.110 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: trying mount with new device syntax: 0@ed40717a-1210-11ef-bc98-c7b262605968.cephfs=/volumes/qa/sv_1/83bdfcd3-fc78-423b-b57a-a691f0db09d1 2024-05-14T16:54:38.110 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync,context="system_u:object_r:postgresql_db_t:s0",name=0,key=0,mon_addr=172.21.15.2:3300/172.21.15.123:3300/172.21.15.151:3300" will pass to kernel 2024-05-14T16:54:38.113 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-14T16:54:38.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T16:54:38.113 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 1777 /tmp/cephfs 2024-05-14T16:54:38.144 DEBUG:teuthology.orchestra.run.smithi002:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-14T16:54:38.159 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:54:37 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:54:37.887+0000 7f547f656200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-14T16:54:38.159 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:54:37 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:54:37.982+0000 7f547f656200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-14T16:54:38.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-14T16:54:38.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T16:54:38.244 INFO:teuthology.orchestra.run.smithi002.stdout:client15129 2024-05-14T16:54:38.244 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T16:54:38.244 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T16:54:38.261 DEBUG:teuthology.orchestra.run.smithi002:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-14T16:54:38.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-14T16:54:38.354 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:54:37 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:54:37.921+0000 7f52a1261200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-14T16:54:38.354 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:54:38 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:54:38.018+0000 7f52a1261200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-14T16:54:38.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T16:54:38.362 INFO:teuthology.orchestra.run.smithi002.stdout:client15129 2024-05-14T16:54:38.362 DEBUG:teuthology.orchestra.run.smithi002:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-14T16:54:38.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-14T16:54:38.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T16:54:38.421 INFO:teuthology.orchestra.run.smithi002.stdout:client15129 2024-05-14T16:54:38.421 DEBUG:teuthology.orchestra.run.smithi002:> (cd /tmp/cephfs && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/ed40717a-1210-11ef-bc98-c7b262605968.client15129/status') 2024-05-14T16:54:38.431 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:54:38 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:54:38.324+0000 7f67e5f41200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-14T16:54:38.431 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:54:38 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:54:38.431+0000 7f67e5f41200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-14T16:54:38.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-14T16:54:38.482 INFO:teuthology.orchestra.run.smithi002.stdout:instance: client.15129 (3)192.168.0.1:0/131110669 2024-05-14T16:54:38.482 INFO:teuthology.orchestra.run.smithi002.stdout:blocklisted: false 2024-05-14T16:54:38.482 INFO:teuthology.orchestra.run.smithi002.stderr:0+1 records in 2024-05-14T16:54:38.482 INFO:teuthology.orchestra.run.smithi002.stderr:0+1 records out 2024-05-14T16:54:38.482 INFO:teuthology.orchestra.run.smithi002.stderr:69 bytes copied, 2.472e-05 s, 2.8 MB/s 2024-05-14T16:54:38.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-14T16:54:38.490 INFO:teuthology.run_tasks:Running task exec... 2024-05-14T16:54:38.498 INFO:teuthology.task.exec:Executing custom commands... 2024-05-14T16:54:38.498 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi002.front.sepia.ceph.com 2024-05-14T16:54:38.498 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-05-14T16:54:38.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.641+0000 7fc67c683640 1 -- 172.21.15.2:0/3530905559 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc674104670 msgr2=0x7fc67410af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:38.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.641+0000 7fc67c683640 1 --2- 172.21.15.2:0/3530905559 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc674104670 0x7fc67410af00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc66c009ca0 tx=0x7fc66c02a8c0 comp rx=0 tx=0).stop 2024-05-14T16:54:38.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.641+0000 7fc67c683640 1 -- 172.21.15.2:0/3530905559 shutdown_connections 2024-05-14T16:54:38.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.641+0000 7fc67c683640 1 --2- 172.21.15.2:0/3530905559 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc674104670 0x7fc67410af00 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:38.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.641+0000 7fc67c683640 1 --2- 172.21.15.2:0/3530905559 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc674103cb0 0x7fc674104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:38.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.641+0000 7fc67c683640 1 --2- 172.21.15.2:0/3530905559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc674102ac0 0x7fc674102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:38.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.641+0000 7fc67c683640 1 -- 172.21.15.2:0/3530905559 >> 172.21.15.2:0/3530905559 conn(0x7fc6740fe030 msgr2=0x7fc674100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:38.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.641+0000 7fc67c683640 1 -- 172.21.15.2:0/3530905559 shutdown_connections 2024-05-14T16:54:38.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.641+0000 7fc67c683640 1 -- 172.21.15.2:0/3530905559 wait complete. 2024-05-14T16:54:38.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.642+0000 7fc67c683640 1 Processor -- start 2024-05-14T16:54:38.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.642+0000 7fc67c683640 1 -- start start 2024-05-14T16:54:38.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.642+0000 7fc67c683640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc674102ac0 0x7fc6741989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:38.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.642+0000 7fc67c683640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc674103cb0 0x7fc674198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:38.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.642+0000 7fc67c683640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc674104670 0x7fc67419ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:38.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.642+0000 7fc67c683640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc67410e5f0 con 0x7fc674103cb0 2024-05-14T16:54:38.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.642+0000 7fc67c683640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc67410e470 con 0x7fc674104670 2024-05-14T16:54:38.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.642+0000 7fc67c683640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc67410e770 con 0x7fc674102ac0 2024-05-14T16:54:38.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.642+0000 7fc679c1e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc674103cb0 0x7fc674198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:38.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.642+0000 7fc679c1e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc674103cb0 0x7fc674198f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40732/0 (socket says 172.21.15.2:40732) 2024-05-14T16:54:38.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.642+0000 7fc679c1e640 1 -- 172.21.15.2:0/225385962 learned_addr learned my addr 172.21.15.2:0/225385962 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:38.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.642+0000 7fc67a41f640 1 --2- 172.21.15.2:0/225385962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc674102ac0 0x7fc6741989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:38.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.642+0000 7fc67ac20640 1 --2- 172.21.15.2:0/225385962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc674104670 0x7fc67419ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:38.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.643+0000 7fc679c1e640 1 -- 172.21.15.2:0/225385962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc674102ac0 msgr2=0x7fc6741989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:38.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.643+0000 7fc679c1e640 1 --2- 172.21.15.2:0/225385962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc674102ac0 0x7fc6741989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:38.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.643+0000 7fc679c1e640 1 -- 172.21.15.2:0/225385962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc674104670 msgr2=0x7fc67419ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:38.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.643+0000 7fc679c1e640 1 --2- 172.21.15.2:0/225385962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc674104670 0x7fc67419ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:38.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.643+0000 7fc679c1e640 1 -- 172.21.15.2:0/225385962 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6741a0600 con 0x7fc674103cb0 2024-05-14T16:54:38.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.643+0000 7fc67ac20640 1 --2- 172.21.15.2:0/225385962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc674104670 0x7fc67419ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:54:38.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.643+0000 7fc679c1e640 1 --2- 172.21.15.2:0/225385962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc674103cb0 0x7fc674198f20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc66400e9f0 tx=0x7fc66400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:38.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.643+0000 7fc66b7fe640 1 -- 172.21.15.2:0/225385962 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66400cd80 con 0x7fc674103cb0 2024-05-14T16:54:38.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.643+0000 7fc67c683640 1 -- 172.21.15.2:0/225385962 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6741a08f0 con 0x7fc674103cb0 2024-05-14T16:54:38.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.643+0000 7fc66b7fe640 1 -- 172.21.15.2:0/225385962 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6640054d0 con 0x7fc674103cb0 2024-05-14T16:54:38.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.643+0000 7fc66b7fe640 1 -- 172.21.15.2:0/225385962 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc664010690 con 0x7fc674103cb0 2024-05-14T16:54:38.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.643+0000 7fc67c683640 1 -- 172.21.15.2:0/225385962 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6741a0df0 con 0x7fc674103cb0 2024-05-14T16:54:38.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.644+0000 7fc67c683640 1 -- 172.21.15.2:0/225385962 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc674109d10 con 0x7fc674103cb0 2024-05-14T16:54:38.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.646+0000 7fc66b7fe640 1 -- 172.21.15.2:0/225385962 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) ==== 149668+0+0 (secure 0 0 0) 0x7fc664004e90 con 0x7fc674103cb0 2024-05-14T16:54:38.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.646+0000 7fc66b7fe640 1 --2- 172.21.15.2:0/225385962 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fc6600b28f0 0x7fc6600b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:38.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.646+0000 7fc67a41f640 1 -- 172.21.15.2:0/225385962 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fc6600b28f0 msgr2=0x7fc6600b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/1498936028 2024-05-14T16:54:38.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.646+0000 7fc66b7fe640 1 -- 172.21.15.2:0/225385962 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7fc6640e26e0 con 0x7fc674103cb0 2024-05-14T16:54:38.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.646+0000 7fc67a41f640 1 --2- 172.21.15.2:0/225385962 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fc6600b28f0 0x7fc6600b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T16:54:38.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.651+0000 7fc66b7fe640 1 -- 172.21.15.2:0/225385962 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc6640aa3b0 con 0x7fc674103cb0 2024-05-14T16:54:38.659 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:54:38 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:54:38.353+0000 7f547f656200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-14T16:54:38.752 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:54:38 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:54:38.394+0000 7f52a1261200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-14T16:54:38.753 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:54:38 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:54:38.752+0000 7f52a1261200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-14T16:54:38.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.847+0000 7fc67a41f640 1 -- 172.21.15.2:0/225385962 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fc6600b28f0 msgr2=0x7fc6600b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/1498936028 2024-05-14T16:54:38.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.847+0000 7fc67a41f640 1 --2- 172.21.15.2:0/225385962 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fc6600b28f0 0x7fc6600b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-14T16:54:38.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.885+0000 7fc67c683640 1 -- 172.21.15.2:0/225385962 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7fc67402dba0 con 0x7fc674103cb0 2024-05-14T16:54:38.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.936+0000 7fc66b7fe640 1 -- 172.21.15.2:0/225385962 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v25) ==== 117+0+0 (secure 0 0 0) 0x7fc6640a9cb0 con 0x7fc674103cb0 2024-05-14T16:54:38.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.939+0000 7fc67c683640 1 -- 172.21.15.2:0/225385962 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fc6600b28f0 msgr2=0x7fc6600b4db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T16:54:38.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.939+0000 7fc67c683640 1 --2- 172.21.15.2:0/225385962 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fc6600b28f0 0x7fc6600b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:38.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.939+0000 7fc67c683640 1 -- 172.21.15.2:0/225385962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc674103cb0 msgr2=0x7fc674198f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:38.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.939+0000 7fc67c683640 1 --2- 172.21.15.2:0/225385962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc674103cb0 0x7fc674198f20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc66400e9f0 tx=0x7fc66400edf0 comp rx=0 tx=0).stop 2024-05-14T16:54:38.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.939+0000 7fc67c683640 1 -- 172.21.15.2:0/225385962 shutdown_connections 2024-05-14T16:54:38.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.939+0000 7fc67c683640 1 --2- 172.21.15.2:0/225385962 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fc6600b28f0 0x7fc6600b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:38.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.939+0000 7fc67c683640 1 --2- 172.21.15.2:0/225385962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc674104670 0x7fc67419ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:38.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.939+0000 7fc67c683640 1 --2- 172.21.15.2:0/225385962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc674103cb0 0x7fc674198f20 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:38.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.939+0000 7fc67c683640 1 --2- 172.21.15.2:0/225385962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc674102ac0 0x7fc6741989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:38.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.939+0000 7fc67c683640 1 -- 172.21.15.2:0/225385962 >> 172.21.15.2:0/225385962 conn(0x7fc6740fe030 msgr2=0x7fc674106ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:38.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.939+0000 7fc67c683640 1 -- 172.21.15.2:0/225385962 shutdown_connections 2024-05-14T16:54:38.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:38.939+0000 7fc67c683640 1 -- 172.21.15.2:0/225385962 wait complete. 2024-05-14T16:54:38.950 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-14T16:54:38.961 INFO:teuthology.run_tasks:Running task exec... 2024-05-14T16:54:39.011 INFO:teuthology.task.exec:Executing custom commands... 2024-05-14T16:54:39.011 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi002.front.sepia.ceph.com 2024-05-14T16:54:39.011 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-05-14T16:54:39.043 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:38 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/225385962' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-14T16:54:39.043 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:54:38 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:54:38.709+0000 7f547f656200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-14T16:54:39.043 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:54:38 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:54:38.800+0000 7f547f656200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-14T16:54:39.094 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:38 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/225385962' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-14T16:54:39.094 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:54:38 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:54:38.847+0000 7f52a1261200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-14T16:54:39.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.118+0000 7f3b9075f640 1 -- 172.21.15.2:0/3820376655 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b881019b0 msgr2=0x7f3b88101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:39.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.118+0000 7f3b9075f640 1 --2- 172.21.15.2:0/3820376655 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b881019b0 0x7f3b88101dd0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3b84009900 tx=0x7f3b8402a410 comp rx=0 tx=0).stop 2024-05-14T16:54:39.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.119+0000 7f3b9075f640 1 -- 172.21.15.2:0/3820376655 shutdown_connections 2024-05-14T16:54:39.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.119+0000 7f3b9075f640 1 --2- 172.21.15.2:0/3820376655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b88103560 0x7f3b88109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:39.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.119+0000 7f3b9075f640 1 --2- 172.21.15.2:0/3820376655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b88102ba0 0x7f3b88103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:39.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.119+0000 7f3b9075f640 1 --2- 172.21.15.2:0/3820376655 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b881019b0 0x7f3b88101dd0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:39.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.119+0000 7f3b9075f640 1 -- 172.21.15.2:0/3820376655 >> 172.21.15.2:0/3820376655 conn(0x7f3b880fcf20 msgr2=0x7f3b880ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:39.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.119+0000 7f3b9075f640 1 -- 172.21.15.2:0/3820376655 shutdown_connections 2024-05-14T16:54:39.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.119+0000 7f3b9075f640 1 -- 172.21.15.2:0/3820376655 wait complete. 2024-05-14T16:54:39.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.119+0000 7f3b9075f640 1 Processor -- start 2024-05-14T16:54:39.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.119+0000 7f3b9075f640 1 -- start start 2024-05-14T16:54:39.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.119+0000 7f3b9075f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b881019b0 0x7f3b88198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:39.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.119+0000 7f3b9075f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b88102ba0 0x7f3b88199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:39.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.119+0000 7f3b9075f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b88103560 0x7f3b881a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:39.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.119+0000 7f3b9075f640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8810d4b0 con 0x7f3b88103560 2024-05-14T16:54:39.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.119+0000 7f3b9075f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8810d330 con 0x7f3b88102ba0 2024-05-14T16:54:39.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.119+0000 7f3b9075f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8810d630 con 0x7f3b881019b0 2024-05-14T16:54:39.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.120+0000 7f3b8e4fb640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b881019b0 0x7f3b88198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:39.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.120+0000 7f3b8ecfc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b88103560 0x7f3b881a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:39.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.120+0000 7f3b8e4fb640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b881019b0 0x7f3b88198cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:48860/0 (socket says 172.21.15.2:48860) 2024-05-14T16:54:39.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.120+0000 7f3b8e4fb640 1 -- 172.21.15.2:0/516243126 learned_addr learned my addr 172.21.15.2:0/516243126 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:39.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.120+0000 7f3b8dcfa640 1 --2- 172.21.15.2:0/516243126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b88102ba0 0x7f3b88199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:39.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.120+0000 7f3b8e4fb640 1 -- 172.21.15.2:0/516243126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b88102ba0 msgr2=0x7f3b88199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:39.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.120+0000 7f3b8e4fb640 1 --2- 172.21.15.2:0/516243126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b88102ba0 0x7f3b88199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:39.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.120+0000 7f3b8e4fb640 1 -- 172.21.15.2:0/516243126 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b88103560 msgr2=0x7f3b881a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:39.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.120+0000 7f3b8e4fb640 1 --2- 172.21.15.2:0/516243126 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b88103560 0x7f3b881a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:39.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.120+0000 7f3b8e4fb640 1 -- 172.21.15.2:0/516243126 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b881a0890 con 0x7f3b881019b0 2024-05-14T16:54:39.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.120+0000 7f3b8ecfc640 1 --2- 172.21.15.2:0/516243126 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b88103560 0x7f3b881a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:54:39.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.120+0000 7f3b8dcfa640 1 --2- 172.21.15.2:0/516243126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b88102ba0 0x7f3b88199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:54:39.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.120+0000 7f3b8e4fb640 1 --2- 172.21.15.2:0/516243126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b881019b0 0x7f3b88198cd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3b84006010 tx=0x7f3b840042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:39.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.120+0000 7f3b7f7fe640 1 -- 172.21.15.2:0/516243126 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8403f030 con 0x7f3b881019b0 2024-05-14T16:54:39.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.120+0000 7f3b9075f640 1 -- 172.21.15.2:0/516243126 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b881a0b20 con 0x7f3b881019b0 2024-05-14T16:54:39.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.120+0000 7f3b7f7fe640 1 -- 172.21.15.2:0/516243126 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b84031910 con 0x7f3b881019b0 2024-05-14T16:54:39.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.120+0000 7f3b7f7fe640 1 -- 172.21.15.2:0/516243126 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b84049060 con 0x7f3b881019b0 2024-05-14T16:54:39.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.120+0000 7f3b9075f640 1 -- 172.21.15.2:0/516243126 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b881a1010 con 0x7f3b881019b0 2024-05-14T16:54:39.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.121+0000 7f3b9075f640 1 -- 172.21.15.2:0/516243126 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b881a11e0 con 0x7f3b881019b0 2024-05-14T16:54:39.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.123+0000 7f3b7f7fe640 1 -- 172.21.15.2:0/516243126 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 19) ==== 149668+0+0 (secure 0 0 0) 0x7f3b84042030 con 0x7f3b881019b0 2024-05-14T16:54:39.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.123+0000 7f3b7f7fe640 1 --2- 172.21.15.2:0/516243126 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f3b700b28f0 0x7f3b700b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:39.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.123+0000 7f3b8dcfa640 1 -- 172.21.15.2:0/516243126 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f3b700b28f0 msgr2=0x7f3b700b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/1498936028 2024-05-14T16:54:39.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.124+0000 7f3b7f7fe640 1 -- 172.21.15.2:0/516243126 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f3b84102e90 con 0x7f3b881019b0 2024-05-14T16:54:39.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.124+0000 7f3b8dcfa640 1 --2- 172.21.15.2:0/516243126 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f3b700b28f0 0x7f3b700b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T16:54:39.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.127+0000 7f3b7f7fe640 1 -- 172.21.15.2:0/516243126 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3b840ca570 con 0x7f3b881019b0 2024-05-14T16:54:39.218 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:38 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/225385962' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-14T16:54:39.219 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:54:38 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:54:38.830+0000 7f67e5f41200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-14T16:54:39.219 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:54:39 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:54:39.218+0000 7f67e5f41200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-14T16:54:39.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.319+0000 7f3b9075f640 1 -- 172.21.15.2:0/516243126 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f3b881a13b0 con 0x7f3b881019b0 2024-05-14T16:54:39.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.325+0000 7f3b8dcfa640 1 -- 172.21.15.2:0/516243126 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f3b700b28f0 msgr2=0x7f3b700b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/1498936028 2024-05-14T16:54:39.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.325+0000 7f3b8dcfa640 1 --2- 172.21.15.2:0/516243126 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f3b700b28f0 0x7f3b700b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-14T16:54:39.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.325+0000 7f3b7f7fe640 1 -- 172.21.15.2:0/516243126 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v32)=0 v32) ==== 129+0+0 (secure 0 0 0) 0x7f3b840ce870 con 0x7f3b881019b0 2024-05-14T16:54:39.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.327+0000 7f3b9075f640 1 -- 172.21.15.2:0/516243126 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f3b700b28f0 msgr2=0x7f3b700b4db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T16:54:39.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.327+0000 7f3b9075f640 1 --2- 172.21.15.2:0/516243126 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f3b700b28f0 0x7f3b700b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:39.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.327+0000 7f3b9075f640 1 -- 172.21.15.2:0/516243126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b881019b0 msgr2=0x7f3b88198cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:39.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.327+0000 7f3b9075f640 1 --2- 172.21.15.2:0/516243126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b881019b0 0x7f3b88198cd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3b84006010 tx=0x7f3b840042f0 comp rx=0 tx=0).stop 2024-05-14T16:54:39.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.327+0000 7f3b9075f640 1 -- 172.21.15.2:0/516243126 shutdown_connections 2024-05-14T16:54:39.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.327+0000 7f3b9075f640 1 --2- 172.21.15.2:0/516243126 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f3b700b28f0 0x7f3b700b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:39.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.327+0000 7f3b9075f640 1 --2- 172.21.15.2:0/516243126 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b88103560 0x7f3b881a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:39.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.327+0000 7f3b9075f640 1 --2- 172.21.15.2:0/516243126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b88102ba0 0x7f3b88199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:39.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.327+0000 7f3b9075f640 1 --2- 172.21.15.2:0/516243126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b881019b0 0x7f3b88198cd0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:39.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.327+0000 7f3b9075f640 1 -- 172.21.15.2:0/516243126 >> 172.21.15.2:0/516243126 conn(0x7f3b880fcf20 msgr2=0x7f3b8806a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:39.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.327+0000 7f3b9075f640 1 -- 172.21.15.2:0/516243126 shutdown_connections 2024-05-14T16:54:39.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.327+0000 7f3b9075f640 1 -- 172.21.15.2:0/516243126 wait complete. 2024-05-14T16:54:39.338 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-05-14T16:54:39.354 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:54:39 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:54:39.094+0000 7f52a1261200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-14T16:54:39.354 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:54:39 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:54:39.230+0000 7f52a1261200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-14T16:54:39.354 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:54:39 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:54:39.329+0000 7f52a1261200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-14T16:54:39.409 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:54:39 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:54:39.044+0000 7f547f656200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-14T16:54:39.410 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:54:39 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:54:39.178+0000 7f547f656200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-14T16:54:39.410 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:54:39 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:54:39.273+0000 7f547f656200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-14T16:54:39.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.445+0000 7facc6d40640 1 -- 172.21.15.2:0/737837716 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facc0105e60 msgr2=0x7facc0108250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:39.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.445+0000 7facc6d40640 1 --2- 172.21.15.2:0/737837716 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facc0105e60 0x7facc0108250 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7facac009900 tx=0x7facac02a410 comp rx=0 tx=0).stop 2024-05-14T16:54:39.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.445+0000 7facc6d40640 1 -- 172.21.15.2:0/737837716 shutdown_connections 2024-05-14T16:54:39.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.445+0000 7facc6d40640 1 --2- 172.21.15.2:0/737837716 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7facc0108790 0x7facc010abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:39.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.445+0000 7facc6d40640 1 --2- 172.21.15.2:0/737837716 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facc0105e60 0x7facc0108250 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:39.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.445+0000 7facc6d40640 1 --2- 172.21.15.2:0/737837716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facc0103530 0x7facc0105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:39.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.445+0000 7facc6d40640 1 -- 172.21.15.2:0/737837716 >> 172.21.15.2:0/737837716 conn(0x7facc00fcf20 msgr2=0x7facc00ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:39.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.445+0000 7facc6d40640 1 -- 172.21.15.2:0/737837716 shutdown_connections 2024-05-14T16:54:39.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.445+0000 7facc6d40640 1 -- 172.21.15.2:0/737837716 wait complete. 2024-05-14T16:54:39.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.445+0000 7facc6d40640 1 Processor -- start 2024-05-14T16:54:39.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.445+0000 7facc6d40640 1 -- start start 2024-05-14T16:54:39.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.446+0000 7facc6d40640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facc0103530 0x7facc0198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:39.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.446+0000 7facc6d40640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7facc0105e60 0x7facc0199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:39.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.446+0000 7facc6d40640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facc0108790 0x7facc01a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:39.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.446+0000 7facc6d40640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7facc010e5d0 con 0x7facc0103530 2024-05-14T16:54:39.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.446+0000 7facc6d40640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7facc010e450 con 0x7facc0108790 2024-05-14T16:54:39.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.446+0000 7facc6d40640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7facc010e750 con 0x7facc0105e60 2024-05-14T16:54:39.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.446+0000 7facc4adc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facc0103530 0x7facc0198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:39.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.446+0000 7facc4adc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facc0103530 0x7facc0198ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40778/0 (socket says 172.21.15.2:40778) 2024-05-14T16:54:39.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.446+0000 7facc4adc640 1 -- 172.21.15.2:0/795045916 learned_addr learned my addr 172.21.15.2:0/795045916 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:39.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.446+0000 7facbffff640 1 --2- 172.21.15.2:0/795045916 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7facc0105e60 0x7facc0199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:39.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.446+0000 7facc52dd640 1 --2- 172.21.15.2:0/795045916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facc0108790 0x7facc01a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:39.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.446+0000 7facbffff640 1 -- 172.21.15.2:0/795045916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facc0108790 msgr2=0x7facc01a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:39.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.446+0000 7facbffff640 1 --2- 172.21.15.2:0/795045916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facc0108790 0x7facc01a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:39.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.446+0000 7facbffff640 1 -- 172.21.15.2:0/795045916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facc0103530 msgr2=0x7facc0198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:39.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.446+0000 7facbffff640 1 --2- 172.21.15.2:0/795045916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facc0103530 0x7facc0198ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:39.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.446+0000 7facbffff640 1 -- 172.21.15.2:0/795045916 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc01a08d0 con 0x7facc0105e60 2024-05-14T16:54:39.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.446+0000 7facc4adc640 1 --2- 172.21.15.2:0/795045916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facc0103530 0x7facc0198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:54:39.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.446+0000 7facc52dd640 1 --2- 172.21.15.2:0/795045916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facc0108790 0x7facc01a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:54:39.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.446+0000 7facbffff640 1 --2- 172.21.15.2:0/795045916 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7facc0105e60 0x7facc0199220 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7facac006010 tx=0x7facac0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:39.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.447+0000 7facbdffb640 1 -- 172.21.15.2:0/795045916 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facac03f030 con 0x7facc0105e60 2024-05-14T16:54:39.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.447+0000 7facbdffb640 1 -- 172.21.15.2:0/795045916 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facac031910 con 0x7facc0105e60 2024-05-14T16:54:39.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.447+0000 7facc6d40640 1 -- 172.21.15.2:0/795045916 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facc01a0b60 con 0x7facc0105e60 2024-05-14T16:54:39.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.447+0000 7facbdffb640 1 -- 172.21.15.2:0/795045916 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facac035070 con 0x7facc0105e60 2024-05-14T16:54:39.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.447+0000 7facc6d40640 1 -- 172.21.15.2:0/795045916 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facc01a1050 con 0x7facc0105e60 2024-05-14T16:54:39.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.448+0000 7facc6d40640 1 -- 172.21.15.2:0/795045916 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facc00fea10 con 0x7facc0105e60 2024-05-14T16:54:39.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.449+0000 7facbdffb640 1 -- 172.21.15.2:0/795045916 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 19) ==== 149668+0+0 (secure 0 0 0) 0x7facac042030 con 0x7facc0105e60 2024-05-14T16:54:39.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.450+0000 7facbdffb640 1 --2- 172.21.15.2:0/795045916 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7facb00b28f0 0x7facb00b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:39.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.450+0000 7facc4adc640 1 -- 172.21.15.2:0/795045916 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7facb00b28f0 msgr2=0x7facb00b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/1498936028 2024-05-14T16:54:39.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.450+0000 7facbdffb640 1 -- 172.21.15.2:0/795045916 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7facac102840 con 0x7facc0105e60 2024-05-14T16:54:39.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.450+0000 7facc4adc640 1 --2- 172.21.15.2:0/795045916 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7facb00b28f0 0x7facb00b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T16:54:39.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.453+0000 7facbdffb640 1 -- 172.21.15.2:0/795045916 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7facac0ca510 con 0x7facc0105e60 2024-05-14T16:54:39.493 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:54:39 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:54:39.320+0000 7f67e5f41200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-14T16:54:39.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.651+0000 7facc4adc640 1 -- 172.21.15.2:0/795045916 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7facb00b28f0 msgr2=0x7facb00b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/1498936028 2024-05-14T16:54:39.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.651+0000 7facc4adc640 1 --2- 172.21.15.2:0/795045916 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7facb00b28f0 0x7facb00b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-14T16:54:39.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.652+0000 7facc6d40640 1 -- 172.21.15.2:0/795045916 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7facc002dba0 con 0x7facc0105e60 2024-05-14T16:54:39.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.658+0000 7facbdffb640 1 -- 172.21.15.2:0/795045916 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v33)=0 v33) ==== 138+0+0 (secure 0 0 0) 0x7facac0c9e10 con 0x7facc0105e60 2024-05-14T16:54:39.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.661+0000 7facc6d40640 1 -- 172.21.15.2:0/795045916 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7facb00b28f0 msgr2=0x7facb00b4db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T16:54:39.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.661+0000 7facc6d40640 1 --2- 172.21.15.2:0/795045916 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7facb00b28f0 0x7facb00b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:39.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.661+0000 7facc6d40640 1 -- 172.21.15.2:0/795045916 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7facc0105e60 msgr2=0x7facc0199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:39.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.661+0000 7facc6d40640 1 --2- 172.21.15.2:0/795045916 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7facc0105e60 0x7facc0199220 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7facac006010 tx=0x7facac0042f0 comp rx=0 tx=0).stop 2024-05-14T16:54:39.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.661+0000 7facc6d40640 1 -- 172.21.15.2:0/795045916 shutdown_connections 2024-05-14T16:54:39.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.661+0000 7facc6d40640 1 --2- 172.21.15.2:0/795045916 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7facb00b28f0 0x7facb00b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:39.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.661+0000 7facc6d40640 1 --2- 172.21.15.2:0/795045916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facc0108790 0x7facc01a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:39.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.661+0000 7facc6d40640 1 --2- 172.21.15.2:0/795045916 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7facc0105e60 0x7facc0199220 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:39.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.661+0000 7facc6d40640 1 --2- 172.21.15.2:0/795045916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facc0103530 0x7facc0198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:39.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.661+0000 7facc6d40640 1 -- 172.21.15.2:0/795045916 >> 172.21.15.2:0/795045916 conn(0x7facc00fcf20 msgr2=0x7facc0103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:39.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.661+0000 7facc6d40640 1 -- 172.21.15.2:0/795045916 shutdown_connections 2024-05-14T16:54:39.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.661+0000 7facc6d40640 1 -- 172.21.15.2:0/795045916 wait complete. 2024-05-14T16:54:39.671 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-05-14T16:54:39.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.779+0000 7f69f759e640 1 -- 172.21.15.2:0/2031836140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69f81017d0 msgr2=0x7f69f8101bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:39.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.779+0000 7f69f759e640 1 --2- 172.21.15.2:0/2031836140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69f81017d0 0x7f69f8101bf0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f69e8009900 tx=0x7f69e802a410 comp rx=0 tx=0).stop 2024-05-14T16:54:39.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.779+0000 7f69f759e640 1 -- 172.21.15.2:0/2031836140 shutdown_connections 2024-05-14T16:54:39.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.779+0000 7f69f759e640 1 --2- 172.21.15.2:0/2031836140 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69f8103380 0x7f69f8109c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:39.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.779+0000 7f69f759e640 1 --2- 172.21.15.2:0/2031836140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69f81029c0 0x7f69f8102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:39.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.779+0000 7f69f759e640 1 --2- 172.21.15.2:0/2031836140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69f81017d0 0x7f69f8101bf0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:39.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.779+0000 7f69f759e640 1 -- 172.21.15.2:0/2031836140 >> 172.21.15.2:0/2031836140 conn(0x7f69f80fcd40 msgr2=0x7f69f80ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:39.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.779+0000 7f69f759e640 1 -- 172.21.15.2:0/2031836140 shutdown_connections 2024-05-14T16:54:39.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.779+0000 7f69f759e640 1 -- 172.21.15.2:0/2031836140 wait complete. 2024-05-14T16:54:39.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.779+0000 7f69f759e640 1 Processor -- start 2024-05-14T16:54:39.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.780+0000 7f69f759e640 1 -- start start 2024-05-14T16:54:39.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.780+0000 7f69f759e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69f81017d0 0x7f69f8105760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:39.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.780+0000 7f69f759e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69f81029c0 0x7f69f8103da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:39.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.780+0000 7f69f759e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69f8103380 0x7f69f81042e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:39.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.780+0000 7f69f759e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f69f810d300 con 0x7f69f81017d0 2024-05-14T16:54:39.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.780+0000 7f69f759e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f69f810d180 con 0x7f69f8103380 2024-05-14T16:54:39.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.780+0000 7f69f759e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f69f810d480 con 0x7f69f81029c0 2024-05-14T16:54:39.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.780+0000 7f69f659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69f81017d0 0x7f69f8105760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:39.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.780+0000 7f69f659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69f81017d0 0x7f69f8105760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40800/0 (socket says 172.21.15.2:40800) 2024-05-14T16:54:39.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.780+0000 7f69f659c640 1 -- 172.21.15.2:0/4033259496 learned_addr learned my addr 172.21.15.2:0/4033259496 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:39.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.780+0000 7f69f6d9d640 1 --2- 172.21.15.2:0/4033259496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69f8103380 0x7f69f81042e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:39.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.780+0000 7f69effff640 1 --2- 172.21.15.2:0/4033259496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69f81029c0 0x7f69f8103da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:39.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.780+0000 7f69f659c640 1 -- 172.21.15.2:0/4033259496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69f81029c0 msgr2=0x7f69f8103da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:39.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.780+0000 7f69f659c640 1 --2- 172.21.15.2:0/4033259496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69f81029c0 0x7f69f8103da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:39.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.780+0000 7f69f659c640 1 -- 172.21.15.2:0/4033259496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69f8103380 msgr2=0x7f69f81042e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:39.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.780+0000 7f69f659c640 1 --2- 172.21.15.2:0/4033259496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69f8103380 0x7f69f81042e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:39.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.780+0000 7f69f659c640 1 -- 172.21.15.2:0/4033259496 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f8104b70 con 0x7f69f81017d0 2024-05-14T16:54:39.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.780+0000 7f69f659c640 1 --2- 172.21.15.2:0/4033259496 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69f81017d0 0x7f69f8105760 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f69e8006010 tx=0x7f69e8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:39.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.781+0000 7f69ef7fe640 1 -- 172.21.15.2:0/4033259496 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e803f030 con 0x7f69f81017d0 2024-05-14T16:54:39.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.781+0000 7f69ef7fe640 1 -- 172.21.15.2:0/4033259496 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69e8031910 con 0x7f69f81017d0 2024-05-14T16:54:39.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.781+0000 7f69f759e640 1 -- 172.21.15.2:0/4033259496 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69f81a2ca0 con 0x7f69f81017d0 2024-05-14T16:54:39.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.781+0000 7f69ef7fe640 1 -- 172.21.15.2:0/4033259496 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e8035070 con 0x7f69f81017d0 2024-05-14T16:54:39.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.781+0000 7f69f759e640 1 -- 172.21.15.2:0/4033259496 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69f81a31a0 con 0x7f69f81017d0 2024-05-14T16:54:39.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.782+0000 7f69f759e640 1 -- 172.21.15.2:0/4033259496 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69f81a3370 con 0x7f69f81017d0 2024-05-14T16:54:39.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.783+0000 7f69ef7fe640 1 -- 172.21.15.2:0/4033259496 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 19) ==== 149668+0+0 (secure 0 0 0) 0x7f69e8042030 con 0x7f69f81017d0 2024-05-14T16:54:39.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.784+0000 7f69ef7fe640 1 --2- 172.21.15.2:0/4033259496 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f69e40b2830 0x7f69e40b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:39.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.785+0000 7f69effff640 1 -- 172.21.15.2:0/4033259496 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f69e40b2830 msgr2=0x7f69e40b4cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/1498936028 2024-05-14T16:54:39.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.785+0000 7f69ef7fe640 1 -- 172.21.15.2:0/4033259496 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7f69e8102a40 con 0x7f69f81017d0 2024-05-14T16:54:39.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.785+0000 7f69effff640 1 --2- 172.21.15.2:0/4033259496 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f69e40b2830 0x7f69e40b4cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T16:54:39.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.788+0000 7f69ef7fe640 1 -- 172.21.15.2:0/4033259496 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f69e80ca0c0 con 0x7f69f81017d0 2024-05-14T16:54:39.846 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:54:39 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:54:39.592+0000 7f67e5f41200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-14T16:54:39.846 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:54:39 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:54:39.739+0000 7f67e5f41200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-14T16:54:39.846 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:54:39 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:54:39.846+0000 7f67e5f41200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-14T16:54:39.909 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:54:39 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:54:39.598+0000 7f547f656200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-14T16:54:39.910 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:54:39 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:54:39.690+0000 7f547f656200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-14T16:54:39.942 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:54:39 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:54:39.663+0000 7f52a1261200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-14T16:54:39.942 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:54:39 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:54:39.757+0000 7f52a1261200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-14T16:54:39.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.980+0000 7f69f759e640 1 -- 172.21.15.2:0/4033259496 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f69f81a3b00 con 0x7f69f81017d0 2024-05-14T16:54:39.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.984+0000 7f69ef7fe640 1 -- 172.21.15.2:0/4033259496 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v34)=0 v34) ==== 133+0+0 (secure 0 0 0) 0x7f69e80ce3c0 con 0x7f69f81017d0 2024-05-14T16:54:39.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.986+0000 7f69effff640 1 -- 172.21.15.2:0/4033259496 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f69e40b2830 msgr2=0x7f69e40b4cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/1498936028 2024-05-14T16:54:39.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.986+0000 7f69effff640 1 --2- 172.21.15.2:0/4033259496 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f69e40b2830 0x7f69e40b4cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-14T16:54:39.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.986+0000 7f69f759e640 1 -- 172.21.15.2:0/4033259496 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f69e40b2830 msgr2=0x7f69e40b4cf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T16:54:39.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.986+0000 7f69f759e640 1 --2- 172.21.15.2:0/4033259496 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f69e40b2830 0x7f69e40b4cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:39.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.986+0000 7f69f759e640 1 -- 172.21.15.2:0/4033259496 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69f81017d0 msgr2=0x7f69f8105760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:39.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.986+0000 7f69f759e640 1 --2- 172.21.15.2:0/4033259496 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69f81017d0 0x7f69f8105760 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f69e8006010 tx=0x7f69e8004270 comp rx=0 tx=0).stop 2024-05-14T16:54:39.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.986+0000 7f69f759e640 1 -- 172.21.15.2:0/4033259496 shutdown_connections 2024-05-14T16:54:39.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.986+0000 7f69f759e640 1 --2- 172.21.15.2:0/4033259496 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7f69e40b2830 0x7f69e40b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:39.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.986+0000 7f69f759e640 1 --2- 172.21.15.2:0/4033259496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69f8103380 0x7f69f81042e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:39.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.986+0000 7f69f759e640 1 --2- 172.21.15.2:0/4033259496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69f81029c0 0x7f69f8103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:39.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.986+0000 7f69f759e640 1 --2- 172.21.15.2:0/4033259496 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69f81017d0 0x7f69f8105760 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:39.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.986+0000 7f69f759e640 1 -- 172.21.15.2:0/4033259496 >> 172.21.15.2:0/4033259496 conn(0x7f69f80fcd40 msgr2=0x7f69f806a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:39.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.986+0000 7f69f759e640 1 -- 172.21.15.2:0/4033259496 shutdown_connections 2024-05-14T16:54:39.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:39.986+0000 7f69f759e640 1 -- 172.21.15.2:0/4033259496 wait complete. 2024-05-14T16:54:39.998 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-05-14T16:54:40.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.107+0000 7fe539245640 1 -- 172.21.15.2:0/1380398912 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe534108790 msgr2=0x7fe53410abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:40.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.107+0000 7fe539245640 1 --2- 172.21.15.2:0/1380398912 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe534108790 0x7fe53410abe0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe52c009ca0 tx=0x7fe52c02a8c0 comp rx=0 tx=0).stop 2024-05-14T16:54:40.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.107+0000 7fe539245640 1 -- 172.21.15.2:0/1380398912 shutdown_connections 2024-05-14T16:54:40.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.107+0000 7fe539245640 1 --2- 172.21.15.2:0/1380398912 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe534108790 0x7fe53410abe0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:40.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.108+0000 7fe539245640 1 --2- 172.21.15.2:0/1380398912 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe534105e60 0x7fe534108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:40.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.108+0000 7fe539245640 1 --2- 172.21.15.2:0/1380398912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe534103530 0x7fe534105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:40.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.108+0000 7fe539245640 1 -- 172.21.15.2:0/1380398912 >> 172.21.15.2:0/1380398912 conn(0x7fe5340fcf20 msgr2=0x7fe5340ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:40.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.108+0000 7fe539245640 1 -- 172.21.15.2:0/1380398912 shutdown_connections 2024-05-14T16:54:40.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.108+0000 7fe539245640 1 -- 172.21.15.2:0/1380398912 wait complete. 2024-05-14T16:54:40.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.109+0000 7fe539245640 1 Processor -- start 2024-05-14T16:54:40.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.109+0000 7fe539245640 1 -- start start 2024-05-14T16:54:40.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.109+0000 7fe539245640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe534103530 0x7fe534198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:40.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.109+0000 7fe539245640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe534105e60 0x7fe534199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:40.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.109+0000 7fe539245640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe534108790 0x7fe5341a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:40.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.109+0000 7fe539245640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe53410e5e0 con 0x7fe534105e60 2024-05-14T16:54:40.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.109+0000 7fe539245640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe53410e460 con 0x7fe534103530 2024-05-14T16:54:40.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.109+0000 7fe539245640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe53410e760 con 0x7fe534108790 2024-05-14T16:54:40.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.109+0000 7fe532d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe534103530 0x7fe534198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:40.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.109+0000 7fe53259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe534105e60 0x7fe534199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:40.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.109+0000 7fe532d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe534103530 0x7fe534198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:52054/0 (socket says 172.21.15.2:52054) 2024-05-14T16:54:40.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.109+0000 7fe532d9d640 1 -- 172.21.15.2:0/4088439493 learned_addr learned my addr 172.21.15.2:0/4088439493 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:40.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.109+0000 7fe53359e640 1 --2- 172.21.15.2:0/4088439493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe534108790 0x7fe5341a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:40.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.109+0000 7fe532d9d640 1 -- 172.21.15.2:0/4088439493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe534108790 msgr2=0x7fe5341a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:40.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.109+0000 7fe532d9d640 1 --2- 172.21.15.2:0/4088439493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe534108790 0x7fe5341a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:40.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.110+0000 7fe532d9d640 1 -- 172.21.15.2:0/4088439493 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe534105e60 msgr2=0x7fe534199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:40.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.110+0000 7fe532d9d640 1 --2- 172.21.15.2:0/4088439493 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe534105e60 0x7fe534199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:40.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.110+0000 7fe532d9d640 1 -- 172.21.15.2:0/4088439493 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5341a0800 con 0x7fe534103530 2024-05-14T16:54:40.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.110+0000 7fe532d9d640 1 --2- 172.21.15.2:0/4088439493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe534103530 0x7fe534198c30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe52400ea60 tx=0x7fe52400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:40.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.110+0000 7fe51bfff640 1 -- 172.21.15.2:0/4088439493 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52400cde0 con 0x7fe534103530 2024-05-14T16:54:40.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.110+0000 7fe539245640 1 -- 172.21.15.2:0/4088439493 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5341a0af0 con 0x7fe534103530 2024-05-14T16:54:40.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.110+0000 7fe51bfff640 1 -- 172.21.15.2:0/4088439493 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5240035f0 con 0x7fe534103530 2024-05-14T16:54:40.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.110+0000 7fe51bfff640 1 -- 172.21.15.2:0/4088439493 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe524010690 con 0x7fe534103530 2024-05-14T16:54:40.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.110+0000 7fe539245640 1 -- 172.21.15.2:0/4088439493 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5341a1090 con 0x7fe534103530 2024-05-14T16:54:40.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.111+0000 7fe539245640 1 -- 172.21.15.2:0/4088439493 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5340fea10 con 0x7fe534103530 2024-05-14T16:54:40.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.112+0000 7fe51bfff640 1 -- 172.21.15.2:0/4088439493 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 19) ==== 149668+0+0 (secure 0 0 0) 0x7fe5240037a0 con 0x7fe534103530 2024-05-14T16:54:40.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.113+0000 7fe51bfff640 1 --2- 172.21.15.2:0/4088439493 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fe51c0b28f0 0x7fe51c0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:40.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.113+0000 7fe53259c640 1 -- 172.21.15.2:0/4088439493 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fe51c0b28f0 msgr2=0x7fe51c0b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/1498936028 2024-05-14T16:54:40.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.113+0000 7fe51bfff640 1 -- 172.21.15.2:0/4088439493 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8211+0+0 (secure 0 0 0) 0x7fe5240e2500 con 0x7fe534103530 2024-05-14T16:54:40.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.113+0000 7fe53259c640 1 --2- 172.21.15.2:0/4088439493 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fe51c0b28f0 0x7fe51c0b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-14T16:54:40.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.116+0000 7fe51bfff640 1 -- 172.21.15.2:0/4088439493 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe5240aa230 con 0x7fe534103530 2024-05-14T16:54:40.200 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:39 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/225385962' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-14T16:54:40.200 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:39 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:54:40.200 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:39 smithi002 ceph-mon[31538]: from='client.? ' entity='client.admin' 2024-05-14T16:54:40.200 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:39 smithi002 ceph-mon[31538]: from='client.? ' entity='client.admin' 2024-05-14T16:54:40.292 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:39 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/225385962' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-14T16:54:40.292 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:39 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:54:40.292 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:39 smithi123 ceph-mon[35971]: from='client.? ' entity='client.admin' 2024-05-14T16:54:40.292 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:39 smithi123 ceph-mon[35971]: from='client.? ' entity='client.admin' 2024-05-14T16:54:40.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.314+0000 7fe53259c640 1 -- 172.21.15.2:0/4088439493 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fe51c0b28f0 msgr2=0x7fe51c0b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/1498936028 2024-05-14T16:54:40.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.314+0000 7fe53259c640 1 --2- 172.21.15.2:0/4088439493 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fe51c0b28f0 0x7fe51c0b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-14T16:54:40.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.336+0000 7fe539245640 1 -- 172.21.15.2:0/4088439493 --> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fe534062e90 con 0x7fe51c0b28f0 2024-05-14T16:54:40.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:39 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/225385962' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-14T16:54:40.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:39 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:54:40.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:39 smithi151 ceph-mon[36066]: from='client.? ' entity='client.admin' 2024-05-14T16:54:40.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:39 smithi151 ceph-mon[36066]: from='client.? ' entity='client.admin' 2024-05-14T16:54:40.493 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:54:40 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:54:40.200+0000 7f67e5f41200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-14T16:54:40.493 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:54:40 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:54:40.300+0000 7f67e5f41200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-14T16:54:40.659 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:54:40 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:54:40.292+0000 7f547f656200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-14T16:54:40.659 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 16:54:40 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y[37338]: 2024-05-14T16:54:40.508+0000 7f547f656200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-14T16:54:40.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.715+0000 7fe53259c640 1 -- 172.21.15.2:0/4088439493 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fe51c0b28f0 msgr2=0x7fe51c0b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/1498936028 2024-05-14T16:54:40.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:40.715+0000 7fe53259c640 1 --2- 172.21.15.2:0/4088439493 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fe51c0b28f0 0x7fe51c0b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-14T16:54:40.854 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:54:40 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:54:40.366+0000 7f52a1261200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-14T16:54:40.854 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 16:54:40 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z[37899]: 2024-05-14T16:54:40.584+0000 7f52a1261200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-14T16:54:41.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:41.000+0000 7fe51bfff640 1 -- 172.21.15.2:0/4088439493 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mgrmap(e 20) ==== 149668+0+0 (secure 0 0 0) 0x7fe5240a5dc0 con 0x7fe534103530 2024-05-14T16:54:41.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:41.199+0000 7fe51bfff640 1 -- 172.21.15.2:0/4088439493 <== mon.2 v2:172.21.15.123:3300/0 8 ==== mgrmap(e 21) ==== 149337+0+0 (secure 0 0 0) 0x7fe524004000 con 0x7fe534103530 2024-05-14T16:54:41.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:41.200+0000 7fe51bfff640 1 -- 172.21.15.2:0/4088439493 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fe51c0b28f0 msgr2=0x7fe51c0b4db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T16:54:41.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:41.200+0000 7fe51bfff640 1 --2- 172.21.15.2:0/4088439493 >> [v2:172.21.15.2:6800/1498936028,v1:172.21.15.2:6801/1498936028] conn(0x7fe51c0b28f0 0x7fe51c0b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:41.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:40 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/4033259496' entity='client.admin' 2024-05-14T16:54:41.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:40 smithi002 ceph-mon[31538]: Standby manager daemon y restarted 2024-05-14T16:54:41.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:40 smithi002 ceph-mon[31538]: Standby manager daemon y started 2024-05-14T16:54:41.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:40 smithi002 ceph-mon[31538]: from='mgr.? 172.21.15.123:0/2907089231' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-14T16:54:41.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:40 smithi002 ceph-mon[31538]: from='mgr.? 172.21.15.123:0/2907089231' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T16:54:41.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:40 smithi002 ceph-mon[31538]: from='mgr.? 172.21.15.123:0/2907089231' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-14T16:54:41.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:40 smithi002 ceph-mon[31538]: from='mgr.? 172.21.15.123:0/2907089231' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T16:54:41.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:40 smithi002 ceph-mon[31538]: Standby manager daemon z restarted 2024-05-14T16:54:41.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:40 smithi002 ceph-mon[31538]: Standby manager daemon z started 2024-05-14T16:54:41.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:40 smithi002 ceph-mon[31538]: from='mgr.? 172.21.15.151:0/2710278690' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-14T16:54:41.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:40 smithi002 ceph-mon[31538]: from='mgr.? 172.21.15.151:0/2710278690' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T16:54:41.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:40 smithi002 ceph-mon[31538]: from='mgr.? 172.21.15.151:0/2710278690' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-14T16:54:41.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:40 smithi002 ceph-mon[31538]: from='mgr.? 172.21.15.151:0/2710278690' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T16:54:41.245 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:54:40 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:54:40.945+0000 7f67e5f41200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-14T16:54:41.245 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:54:41 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:54:41.185+0000 7f67e5f41200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-14T16:54:41.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:40 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/4033259496' entity='client.admin' 2024-05-14T16:54:41.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:40 smithi151 ceph-mon[36066]: Standby manager daemon y restarted 2024-05-14T16:54:41.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:40 smithi151 ceph-mon[36066]: Standby manager daemon y started 2024-05-14T16:54:41.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:40 smithi151 ceph-mon[36066]: from='mgr.? 172.21.15.123:0/2907089231' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-14T16:54:41.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:40 smithi151 ceph-mon[36066]: from='mgr.? 172.21.15.123:0/2907089231' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T16:54:41.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:40 smithi151 ceph-mon[36066]: from='mgr.? 172.21.15.123:0/2907089231' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-14T16:54:41.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:40 smithi151 ceph-mon[36066]: from='mgr.? 172.21.15.123:0/2907089231' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T16:54:41.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:40 smithi151 ceph-mon[36066]: Standby manager daemon z restarted 2024-05-14T16:54:41.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:40 smithi151 ceph-mon[36066]: Standby manager daemon z started 2024-05-14T16:54:41.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:40 smithi151 ceph-mon[36066]: from='mgr.? 172.21.15.151:0/2710278690' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-14T16:54:41.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:40 smithi151 ceph-mon[36066]: from='mgr.? 172.21.15.151:0/2710278690' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T16:54:41.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:40 smithi151 ceph-mon[36066]: from='mgr.? 172.21.15.151:0/2710278690' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-14T16:54:41.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:40 smithi151 ceph-mon[36066]: from='mgr.? 172.21.15.151:0/2710278690' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T16:54:41.365 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:40 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/4033259496' entity='client.admin' 2024-05-14T16:54:41.365 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:40 smithi123 ceph-mon[35971]: Standby manager daemon y restarted 2024-05-14T16:54:41.365 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:40 smithi123 ceph-mon[35971]: Standby manager daemon y started 2024-05-14T16:54:41.365 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:40 smithi123 ceph-mon[35971]: from='mgr.? 172.21.15.123:0/2907089231' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-14T16:54:41.365 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:40 smithi123 ceph-mon[35971]: from='mgr.? 172.21.15.123:0/2907089231' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T16:54:41.365 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:40 smithi123 ceph-mon[35971]: from='mgr.? 172.21.15.123:0/2907089231' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-14T16:54:41.365 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:40 smithi123 ceph-mon[35971]: from='mgr.? 172.21.15.123:0/2907089231' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T16:54:41.365 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:40 smithi123 ceph-mon[35971]: Standby manager daemon z restarted 2024-05-14T16:54:41.365 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:40 smithi123 ceph-mon[35971]: Standby manager daemon z started 2024-05-14T16:54:41.366 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:40 smithi123 ceph-mon[35971]: from='mgr.? 172.21.15.151:0/2710278690' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-14T16:54:41.366 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:40 smithi123 ceph-mon[35971]: from='mgr.? 172.21.15.151:0/2710278690' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-14T16:54:41.366 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:40 smithi123 ceph-mon[35971]: from='mgr.? 172.21.15.151:0/2710278690' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-14T16:54:41.366 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:40 smithi123 ceph-mon[35971]: from='mgr.? 172.21.15.151:0/2710278690' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-14T16:54:41.744 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:54:41 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:54:41.456+0000 7f674474e700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T16:54:41.744 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:54:41 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:54:41.456+0000 7f674474e700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T16:54:41.744 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:54:41 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:54:41.456+0000 7f674474e700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T16:54:41.744 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:54:41 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:54:41.456+0000 7f674474e700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T16:54:41.744 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:54:41 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:54:41.456+0000 7f674474e700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T16:54:41.744 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:54:41 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:54:41.458+0000 7f6747794700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T16:54:41.744 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:54:41 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:54:41.458+0000 7f6747794700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T16:54:41.744 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:54:41 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:54:41.458+0000 7f6747794700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T16:54:41.745 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:54:41 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:54:41.458+0000 7f6747794700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T16:54:41.745 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:54:41 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:54:41.458+0000 7f6747794700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T16:54:42.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.203+0000 7fe51bfff640 1 -- 172.21.15.2:0/4088439493 <== mon.2 v2:172.21.15.123:3300/0 9 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7fe5240e0060 con 0x7fe534103530 2024-05-14T16:54:42.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.204+0000 7fe51bfff640 1 --2- 172.21.15.2:0/4088439493 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fe51c03a510 0x7fe51c03a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:42.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.204+0000 7fe51bfff640 1 -- 172.21.15.2:0/4088439493 --> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fe52401aec0 con 0x7fe51c03a510 2024-05-14T16:54:42.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.209+0000 7fe53259c640 1 --2- 172.21.15.2:0/4088439493 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fe51c03a510 0x7fe51c03a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:42.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.211+0000 7fe53259c640 1 --2- 172.21.15.2:0/4088439493 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fe51c03a510 0x7fe51c03a8f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe53419a0f0 tx=0x7fe528005f90 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:42.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-05-14T16:54:42.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: Active manager daemon x restarted 2024-05-14T16:54:42.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: Activating manager daemon x 2024-05-14T16:54:42.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: osdmap e81: 12 total, 12 up, 12 in 2024-05-14T16:54:42.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: mgrmap e21: x(active, starting, since 0.0115683s), standbys: z, y 2024-05-14T16:54:42.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:54:42.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:54:42.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:54:42.244 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-14T16:54:42.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-14T16:54:42.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-14T16:54:42.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-14T16:54:42.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-14T16:54:42.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-14T16:54:42.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T16:54:42.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T16:54:42.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T16:54:42.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T16:54:42.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T16:54:42.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T16:54:42.245 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-14T16:54:42.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-14T16:54:42.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-14T16:54:42.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:54:42.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:54:42.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:54:42.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:54:42.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:54:42.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:54:42.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:54:42.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:54:42.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:54:42.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:54:42.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:54:42.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:54:42.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-14T16:54:42.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-14T16:54:42.247 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-14T16:54:42.247 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: Manager daemon x is now available 2024-05-14T16:54:42.247 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:54:42.247 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-14T16:54:42.247 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:41 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-14T16:54:42.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.279+0000 7fe51bfff640 1 -- 172.21.15.2:0/4088439493 <== mgr.15120 v2:172.21.15.2:6800/1339485805 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe52401aec0 con 0x7fe51c03a510 2024-05-14T16:54:42.279 INFO:teuthology.orchestra.run.smithi002.stdout:Schedule set for path / 2024-05-14T16:54:42.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.281+0000 7fe539245640 1 -- 172.21.15.2:0/4088439493 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fe51c03a510 msgr2=0x7fe51c03a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:42.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.281+0000 7fe539245640 1 --2- 172.21.15.2:0/4088439493 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fe51c03a510 0x7fe51c03a8f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe53419a0f0 tx=0x7fe528005f90 comp rx=0 tx=0).stop 2024-05-14T16:54:42.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.281+0000 7fe539245640 1 -- 172.21.15.2:0/4088439493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe534103530 msgr2=0x7fe534198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:42.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.281+0000 7fe539245640 1 --2- 172.21.15.2:0/4088439493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe534103530 0x7fe534198c30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe52400ea60 tx=0x7fe52400ef20 comp rx=0 tx=0).stop 2024-05-14T16:54:42.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.281+0000 7fe53359e640 1 -- 172.21.15.2:0/4088439493 reap_dead start 2024-05-14T16:54:42.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.281+0000 7fe539245640 1 -- 172.21.15.2:0/4088439493 shutdown_connections 2024-05-14T16:54:42.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.281+0000 7fe539245640 1 -- 172.21.15.2:0/4088439493 >> 172.21.15.2:0/4088439493 conn(0x7fe5340fcf20 msgr2=0x7fe534103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:42.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.281+0000 7fe539245640 1 -- 172.21.15.2:0/4088439493 shutdown_connections 2024-05-14T16:54:42.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.281+0000 7fe539245640 1 -- 172.21.15.2:0/4088439493 wait complete. 2024-05-14T16:54:42.292 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-05-14T16:54:42.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-05-14T16:54:42.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: Active manager daemon x restarted 2024-05-14T16:54:42.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: Activating manager daemon x 2024-05-14T16:54:42.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: osdmap e81: 12 total, 12 up, 12 in 2024-05-14T16:54:42.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: mgrmap e21: x(active, starting, since 0.0115683s), standbys: z, y 2024-05-14T16:54:42.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:54:42.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:54:42.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:54:42.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-14T16:54:42.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-14T16:54:42.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-14T16:54:42.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-14T16:54:42.355 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-14T16:54:42.356 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-14T16:54:42.356 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T16:54:42.356 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T16:54:42.356 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T16:54:42.356 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T16:54:42.356 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T16:54:42.356 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T16:54:42.356 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-14T16:54:42.356 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-14T16:54:42.356 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-14T16:54:42.356 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:54:42.356 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:54:42.356 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:54:42.356 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:54:42.357 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:54:42.357 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:54:42.357 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:54:42.357 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:54:42.357 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:54:42.357 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:54:42.357 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:54:42.357 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:54:42.357 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-14T16:54:42.357 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-14T16:54:42.357 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-14T16:54:42.357 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: Manager daemon x is now available 2024-05-14T16:54:42.357 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:54:42.357 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-14T16:54:42.357 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:41 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-14T16:54:42.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.402+0000 7fdf6362a640 1 -- 172.21.15.2:0/3664657318 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf5c105ec0 msgr2=0x7fdf5c1082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:42.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.402+0000 7fdf6362a640 1 --2- 172.21.15.2:0/3664657318 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf5c105ec0 0x7fdf5c1082b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c009900 tx=0x7fdf4c02a410 comp rx=0 tx=0).stop 2024-05-14T16:54:42.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.402+0000 7fdf6362a640 1 -- 172.21.15.2:0/3664657318 shutdown_connections 2024-05-14T16:54:42.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.402+0000 7fdf6362a640 1 --2- 172.21.15.2:0/3664657318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf5c1087f0 0x7fdf5c10abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:42.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.402+0000 7fdf6362a640 1 --2- 172.21.15.2:0/3664657318 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf5c105ec0 0x7fdf5c1082b0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:42.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.402+0000 7fdf6362a640 1 --2- 172.21.15.2:0/3664657318 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf5c103590 0x7fdf5c105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:42.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.402+0000 7fdf6362a640 1 -- 172.21.15.2:0/3664657318 >> 172.21.15.2:0/3664657318 conn(0x7fdf5c0fcf20 msgr2=0x7fdf5c0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:42.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.402+0000 7fdf6362a640 1 -- 172.21.15.2:0/3664657318 shutdown_connections 2024-05-14T16:54:42.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.402+0000 7fdf6362a640 1 -- 172.21.15.2:0/3664657318 wait complete. 2024-05-14T16:54:42.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.403+0000 7fdf6362a640 1 Processor -- start 2024-05-14T16:54:42.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.403+0000 7fdf6362a640 1 -- start start 2024-05-14T16:54:42.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.403+0000 7fdf6362a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf5c103590 0x7fdf5c198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:42.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.403+0000 7fdf6362a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf5c105ec0 0x7fdf5c199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:42.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.403+0000 7fdf6362a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf5c1087f0 0x7fdf5c1a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:42.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.403+0000 7fdf6362a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5c10e5d0 con 0x7fdf5c103590 2024-05-14T16:54:42.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.403+0000 7fdf6362a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5c10e450 con 0x7fdf5c105ec0 2024-05-14T16:54:42.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.403+0000 7fdf6362a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5c10e750 con 0x7fdf5c1087f0 2024-05-14T16:54:42.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.403+0000 7fdf613c6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf5c103590 0x7fdf5c198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:42.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.403+0000 7fdf613c6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf5c103590 0x7fdf5c198cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40966/0 (socket says 172.21.15.2:40966) 2024-05-14T16:54:42.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.403+0000 7fdf613c6640 1 -- 172.21.15.2:0/656845411 learned_addr learned my addr 172.21.15.2:0/656845411 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:42.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.403+0000 7fdf61bc7640 1 --2- 172.21.15.2:0/656845411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf5c1087f0 0x7fdf5c1a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:42.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.403+0000 7fdf60bc5640 1 --2- 172.21.15.2:0/656845411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf5c105ec0 0x7fdf5c199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:42.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.404+0000 7fdf613c6640 1 -- 172.21.15.2:0/656845411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf5c1087f0 msgr2=0x7fdf5c1a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:42.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.404+0000 7fdf613c6640 1 --2- 172.21.15.2:0/656845411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf5c1087f0 0x7fdf5c1a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:42.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.404+0000 7fdf613c6640 1 -- 172.21.15.2:0/656845411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf5c105ec0 msgr2=0x7fdf5c199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:42.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.404+0000 7fdf613c6640 1 --2- 172.21.15.2:0/656845411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf5c105ec0 0x7fdf5c199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:42.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.404+0000 7fdf613c6640 1 -- 172.21.15.2:0/656845411 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf5c1a0900 con 0x7fdf5c103590 2024-05-14T16:54:42.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.404+0000 7fdf613c6640 1 --2- 172.21.15.2:0/656845411 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf5c103590 0x7fdf5c198cf0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdf5800c970 tx=0x7fdf5800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:42.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.404+0000 7fdf527fc640 1 -- 172.21.15.2:0/656845411 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf58007bb0 con 0x7fdf5c103590 2024-05-14T16:54:42.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.404+0000 7fdf6362a640 1 -- 172.21.15.2:0/656845411 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf5c1a0bf0 con 0x7fdf5c103590 2024-05-14T16:54:42.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.404+0000 7fdf527fc640 1 -- 172.21.15.2:0/656845411 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf58007d50 con 0x7fdf5c103590 2024-05-14T16:54:42.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.404+0000 7fdf527fc640 1 -- 172.21.15.2:0/656845411 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5800f450 con 0x7fdf5c103590 2024-05-14T16:54:42.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.404+0000 7fdf6362a640 1 -- 172.21.15.2:0/656845411 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf5c1a10f0 con 0x7fdf5c103590 2024-05-14T16:54:42.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.405+0000 7fdf6362a640 1 -- 172.21.15.2:0/656845411 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf5c0fea10 con 0x7fdf5c103590 2024-05-14T16:54:42.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.407+0000 7fdf527fc640 1 -- 172.21.15.2:0/656845411 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7fdf58004f20 con 0x7fdf5c103590 2024-05-14T16:54:42.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.407+0000 7fdf527fc640 1 --2- 172.21.15.2:0/656845411 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fdf480b2690 0x7fdf480b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:42.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.408+0000 7fdf60bc5640 1 --2- 172.21.15.2:0/656845411 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fdf480b2690 0x7fdf480b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:42.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.408+0000 7fdf527fc640 1 -- 172.21.15.2:0/656845411 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fdf580e1be0 con 0x7fdf5c103590 2024-05-14T16:54:42.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.408+0000 7fdf60bc5640 1 --2- 172.21.15.2:0/656845411 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fdf480b2690 0x7fdf480b4b50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c009230 tx=0x7fdf4c0314a0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:42.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-05-14T16:54:42.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: Active manager daemon x restarted 2024-05-14T16:54:42.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: Activating manager daemon x 2024-05-14T16:54:42.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: osdmap e81: 12 total, 12 up, 12 in 2024-05-14T16:54:42.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: mgrmap e21: x(active, starting, since 0.0115683s), standbys: z, y 2024-05-14T16:54:42.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-14T16:54:42.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-14T16:54:42.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-14T16:54:42.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-14T16:54:42.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-14T16:54:42.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-14T16:54:42.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-14T16:54:42.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-14T16:54:42.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-14T16:54:42.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T16:54:42.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T16:54:42.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T16:54:42.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T16:54:42.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T16:54:42.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T16:54:42.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-14T16:54:42.411 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-14T16:54:42.412 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-14T16:54:42.412 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-14T16:54:42.412 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-14T16:54:42.412 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-14T16:54:42.412 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-14T16:54:42.412 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-14T16:54:42.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.412+0000 7fdf527fc640 1 -- 172.21.15.2:0/656845411 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdf58014030 con 0x7fdf5c103590 2024-05-14T16:54:42.413 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-14T16:54:42.413 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-14T16:54:42.413 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-14T16:54:42.413 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-14T16:54:42.413 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-14T16:54:42.413 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-14T16:54:42.413 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-14T16:54:42.413 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-14T16:54:42.413 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-14T16:54:42.413 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-14T16:54:42.413 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: Manager daemon x is now available 2024-05-14T16:54:42.413 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:54:42.413 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-14T16:54:42.413 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:41 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-14T16:54:42.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.665+0000 7fdf6362a640 1 -- 172.21.15.2:0/656845411 --> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fdf5c062f30 con 0x7fdf480b2690 2024-05-14T16:54:42.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.729+0000 7fdf527fc640 1 -- 172.21.15.2:0/656845411 <== mgr.15120 v2:172.21.15.2:6800/1339485805 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fdf5c062f30 con 0x7fdf480b2690 2024-05-14T16:54:42.729 INFO:teuthology.orchestra.run.smithi002.stdout:Retention added to path / 2024-05-14T16:54:42.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.731+0000 7fdf6362a640 1 -- 172.21.15.2:0/656845411 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fdf480b2690 msgr2=0x7fdf480b4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:42.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.731+0000 7fdf6362a640 1 --2- 172.21.15.2:0/656845411 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fdf480b2690 0x7fdf480b4b50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c009230 tx=0x7fdf4c0314a0 comp rx=0 tx=0).stop 2024-05-14T16:54:42.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.731+0000 7fdf6362a640 1 -- 172.21.15.2:0/656845411 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf5c103590 msgr2=0x7fdf5c198cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:42.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.731+0000 7fdf6362a640 1 --2- 172.21.15.2:0/656845411 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf5c103590 0x7fdf5c198cf0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdf5800c970 tx=0x7fdf5800ce30 comp rx=0 tx=0).stop 2024-05-14T16:54:42.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.731+0000 7fdf6362a640 1 -- 172.21.15.2:0/656845411 shutdown_connections 2024-05-14T16:54:42.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.731+0000 7fdf6362a640 1 --2- 172.21.15.2:0/656845411 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fdf480b2690 0x7fdf480b4b50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:42.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.731+0000 7fdf6362a640 1 --2- 172.21.15.2:0/656845411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf5c1087f0 0x7fdf5c1a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:42.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.731+0000 7fdf6362a640 1 --2- 172.21.15.2:0/656845411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf5c105ec0 0x7fdf5c199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:42.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.731+0000 7fdf6362a640 1 --2- 172.21.15.2:0/656845411 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf5c103590 0x7fdf5c198cf0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:42.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.731+0000 7fdf6362a640 1 -- 172.21.15.2:0/656845411 >> 172.21.15.2:0/656845411 conn(0x7fdf5c0fcf20 msgr2=0x7fdf5c104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:42.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.731+0000 7fdf6362a640 1 -- 172.21.15.2:0/656845411 shutdown_connections 2024-05-14T16:54:42.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.731+0000 7fdf6362a640 1 -- 172.21.15.2:0/656845411 wait complete. 2024-05-14T16:54:42.743 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-14T16:54:42.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.864+0000 7faa2d338640 1 -- 172.21.15.2:0/2941251565 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa28104680 msgr2=0x7faa2810af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:42.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.864+0000 7faa2d338640 1 --2- 172.21.15.2:0/2941251565 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa28104680 0x7faa2810af10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7faa20009ca0 tx=0x7faa2002a8c0 comp rx=0 tx=0).stop 2024-05-14T16:54:42.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.864+0000 7faa2d338640 1 -- 172.21.15.2:0/2941251565 shutdown_connections 2024-05-14T16:54:42.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.864+0000 7faa2d338640 1 --2- 172.21.15.2:0/2941251565 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa28104680 0x7faa2810af10 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:42.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.864+0000 7faa2d338640 1 --2- 172.21.15.2:0/2941251565 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa28103cc0 0x7faa28104140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:42.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.864+0000 7faa2d338640 1 --2- 172.21.15.2:0/2941251565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa28102ad0 0x7faa28102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:42.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.864+0000 7faa2d338640 1 -- 172.21.15.2:0/2941251565 >> 172.21.15.2:0/2941251565 conn(0x7faa280fe040 msgr2=0x7faa28100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:42.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.864+0000 7faa2d338640 1 -- 172.21.15.2:0/2941251565 shutdown_connections 2024-05-14T16:54:42.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.864+0000 7faa2d338640 1 -- 172.21.15.2:0/2941251565 wait complete. 2024-05-14T16:54:42.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.865+0000 7faa2d338640 1 Processor -- start 2024-05-14T16:54:42.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.865+0000 7faa2d338640 1 -- start start 2024-05-14T16:54:42.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.865+0000 7faa2d338640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa28102ad0 0x7faa28076800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:42.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.865+0000 7faa2d338640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa28103cc0 0x7faa28074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:42.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.865+0000 7faa2d338640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa28104680 0x7faa28075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:42.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.865+0000 7faa2d338640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7faa2810e5e0 con 0x7faa28102ad0 2024-05-14T16:54:42.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.865+0000 7faa2d338640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faa2810e460 con 0x7faa28104680 2024-05-14T16:54:42.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.865+0000 7faa2d338640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faa2810e760 con 0x7faa28103cc0 2024-05-14T16:54:42.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.865+0000 7faa26ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa28102ad0 0x7faa28076800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:42.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.865+0000 7faa26ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa28102ad0 0x7faa28076800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40994/0 (socket says 172.21.15.2:40994) 2024-05-14T16:54:42.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.865+0000 7faa26ffd640 1 -- 172.21.15.2:0/1969801007 learned_addr learned my addr 172.21.15.2:0/1969801007 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:42.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.866+0000 7faa277fe640 1 --2- 172.21.15.2:0/1969801007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa28104680 0x7faa28075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:42.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.866+0000 7faa1ffff640 1 --2- 172.21.15.2:0/1969801007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa28103cc0 0x7faa28074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:42.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.866+0000 7faa26ffd640 1 -- 172.21.15.2:0/1969801007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa28103cc0 msgr2=0x7faa28074e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:42.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.866+0000 7faa26ffd640 1 --2- 172.21.15.2:0/1969801007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa28103cc0 0x7faa28074e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:42.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.866+0000 7faa26ffd640 1 -- 172.21.15.2:0/1969801007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa28104680 msgr2=0x7faa28075380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:42.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.866+0000 7faa26ffd640 1 --2- 172.21.15.2:0/1969801007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa28104680 0x7faa28075380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:42.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.866+0000 7faa26ffd640 1 -- 172.21.15.2:0/1969801007 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa28075c10 con 0x7faa28102ad0 2024-05-14T16:54:42.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.866+0000 7faa26ffd640 1 --2- 172.21.15.2:0/1969801007 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa28102ad0 0x7faa28076800 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7faa1800e9c0 tx=0x7faa1800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:42.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.866+0000 7faa24ff9640 1 -- 172.21.15.2:0/1969801007 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1800cde0 con 0x7faa28102ad0 2024-05-14T16:54:42.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.866+0000 7faa24ff9640 1 -- 172.21.15.2:0/1969801007 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa180108e0 con 0x7faa28102ad0 2024-05-14T16:54:42.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.866+0000 7faa2d338640 1 -- 172.21.15.2:0/1969801007 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa281a2d10 con 0x7faa28102ad0 2024-05-14T16:54:42.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.866+0000 7faa24ff9640 1 -- 172.21.15.2:0/1969801007 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa18018960 con 0x7faa28102ad0 2024-05-14T16:54:42.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.866+0000 7faa2d338640 1 -- 172.21.15.2:0/1969801007 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa281a31b0 con 0x7faa28102ad0 2024-05-14T16:54:42.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.867+0000 7faa2d338640 1 -- 172.21.15.2:0/1969801007 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa28109df0 con 0x7faa28102ad0 2024-05-14T16:54:42.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.868+0000 7faa24ff9640 1 -- 172.21.15.2:0/1969801007 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7faa18010a90 con 0x7faa28102ad0 2024-05-14T16:54:42.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.869+0000 7faa24ff9640 1 --2- 172.21.15.2:0/1969801007 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7faa140b2690 0x7faa140b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:42.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.869+0000 7faa1ffff640 1 --2- 172.21.15.2:0/1969801007 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7faa140b2690 0x7faa140b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:42.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.869+0000 7faa24ff9640 1 -- 172.21.15.2:0/1969801007 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7faa180e2c70 con 0x7faa28102ad0 2024-05-14T16:54:42.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.870+0000 7faa1ffff640 1 --2- 172.21.15.2:0/1969801007 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7faa140b2690 0x7faa140b4b50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7faa28076570 tx=0x7faa10005f90 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:42.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:42.872+0000 7faa24ff9640 1 -- 172.21.15.2:0/1969801007 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faa18026070 con 0x7faa28102ad0 2024-05-14T16:54:43.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.088+0000 7faa2d338640 1 -- 172.21.15.2:0/1969801007 --> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7faa28062e90 con 0x7faa140b2690 2024-05-14T16:54:43.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.096+0000 7faa24ff9640 1 -- 172.21.15.2:0/1969801007 <== mgr.15120 v2:172.21.15.2:6800/1339485805 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7faa28062e90 con 0x7faa140b2690 2024-05-14T16:54:43.096 INFO:teuthology.orchestra.run.smithi002.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-14T00:00:00", "created": "2024-05-14T16:54:42", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-05-14T16:54:43.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.099+0000 7faa2d338640 1 -- 172.21.15.2:0/1969801007 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7faa140b2690 msgr2=0x7faa140b4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:43.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.099+0000 7faa2d338640 1 --2- 172.21.15.2:0/1969801007 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7faa140b2690 0x7faa140b4b50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7faa28076570 tx=0x7faa10005f90 comp rx=0 tx=0).stop 2024-05-14T16:54:43.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.099+0000 7faa2d338640 1 -- 172.21.15.2:0/1969801007 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa28102ad0 msgr2=0x7faa28076800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:43.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.099+0000 7faa2d338640 1 --2- 172.21.15.2:0/1969801007 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa28102ad0 0x7faa28076800 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7faa1800e9c0 tx=0x7faa1800ee80 comp rx=0 tx=0).stop 2024-05-14T16:54:43.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.099+0000 7faa2d338640 1 -- 172.21.15.2:0/1969801007 shutdown_connections 2024-05-14T16:54:43.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.099+0000 7faa2d338640 1 --2- 172.21.15.2:0/1969801007 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7faa140b2690 0x7faa140b4b50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:43.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.099+0000 7faa2d338640 1 --2- 172.21.15.2:0/1969801007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa28104680 0x7faa28075380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:43.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.099+0000 7faa2d338640 1 --2- 172.21.15.2:0/1969801007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa28103cc0 0x7faa28074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:43.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.099+0000 7faa2d338640 1 --2- 172.21.15.2:0/1969801007 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa28102ad0 0x7faa28076800 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:43.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.099+0000 7faa2d338640 1 -- 172.21.15.2:0/1969801007 >> 172.21.15.2:0/1969801007 conn(0x7faa280fe040 msgr2=0x7faa28106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:43.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.099+0000 7faa2d338640 1 -- 172.21.15.2:0/1969801007 shutdown_connections 2024-05-14T16:54:43.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.099+0000 7faa2d338640 1 -- 172.21.15.2:0/1969801007 wait complete. 2024-05-14T16:54:43.109 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-14T16:54:43.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.223+0000 7f38a177f640 1 -- 172.21.15.2:0/2197161993 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f389c108790 msgr2=0x7f389c10abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:43.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.223+0000 7f38a177f640 1 --2- 172.21.15.2:0/2197161993 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f389c108790 0x7f389c10abe0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3894009be0 tx=0x7f389402a6c0 comp rx=0 tx=0).stop 2024-05-14T16:54:43.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.223+0000 7f38a177f640 1 -- 172.21.15.2:0/2197161993 shutdown_connections 2024-05-14T16:54:43.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.223+0000 7f38a177f640 1 --2- 172.21.15.2:0/2197161993 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f389c108790 0x7f389c10abe0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:43.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.223+0000 7f38a177f640 1 --2- 172.21.15.2:0/2197161993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f389c105e60 0x7f389c108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:43.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.223+0000 7f38a177f640 1 --2- 172.21.15.2:0/2197161993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f389c103530 0x7f389c105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:43.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.223+0000 7f38a177f640 1 -- 172.21.15.2:0/2197161993 >> 172.21.15.2:0/2197161993 conn(0x7f389c0fcf20 msgr2=0x7f389c0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:43.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.223+0000 7f38a177f640 1 -- 172.21.15.2:0/2197161993 shutdown_connections 2024-05-14T16:54:43.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.223+0000 7f38a177f640 1 -- 172.21.15.2:0/2197161993 wait complete. 2024-05-14T16:54:43.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.224+0000 7f38a177f640 1 Processor -- start 2024-05-14T16:54:43.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.224+0000 7f38a177f640 1 -- start start 2024-05-14T16:54:43.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.224+0000 7f38a177f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f389c103530 0x7f389c102e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:43.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.224+0000 7f38a177f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f389c105e60 0x7f389c1014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:43.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.224+0000 7f38a177f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f389c108790 0x7f389c101a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:43.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.224+0000 7f38a177f640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f389c10e600 con 0x7f389c103530 2024-05-14T16:54:43.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.224+0000 7f38a177f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f389c10e480 con 0x7f389c105e60 2024-05-14T16:54:43.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.224+0000 7f38a177f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f389c10e780 con 0x7f389c108790 2024-05-14T16:54:43.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.224+0000 7f389affd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f389c103530 0x7f389c102e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:43.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.224+0000 7f389affd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f389c103530 0x7f389c102e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41012/0 (socket says 172.21.15.2:41012) 2024-05-14T16:54:43.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.224+0000 7f389affd640 1 -- 172.21.15.2:0/2939111361 learned_addr learned my addr 172.21.15.2:0/2939111361 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:43.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.224+0000 7f389a7fc640 1 --2- 172.21.15.2:0/2939111361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f389c105e60 0x7f389c1014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:43.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.225+0000 7f389b7fe640 1 --2- 172.21.15.2:0/2939111361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f389c108790 0x7f389c101a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:43.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.225+0000 7f389affd640 1 -- 172.21.15.2:0/2939111361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f389c108790 msgr2=0x7f389c101a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:43.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.225+0000 7f389affd640 1 --2- 172.21.15.2:0/2939111361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f389c108790 0x7f389c101a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:43.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.225+0000 7f389affd640 1 -- 172.21.15.2:0/2939111361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f389c105e60 msgr2=0x7f389c1014e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:43.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.225+0000 7f389affd640 1 --2- 172.21.15.2:0/2939111361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f389c105e60 0x7f389c1014e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:43.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.225+0000 7f389affd640 1 -- 172.21.15.2:0/2939111361 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f389c1022e0 con 0x7f389c103530 2024-05-14T16:54:43.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.225+0000 7f389a7fc640 1 --2- 172.21.15.2:0/2939111361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f389c105e60 0x7f389c1014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:54:43.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.225+0000 7f389affd640 1 --2- 172.21.15.2:0/2939111361 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f389c103530 0x7f389c102e50 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f388c00e9f0 tx=0x7f388c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:43.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.225+0000 7f3893fff640 1 -- 172.21.15.2:0/2939111361 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c00ccc0 con 0x7f389c103530 2024-05-14T16:54:43.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.225+0000 7f38a177f640 1 -- 172.21.15.2:0/2939111361 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f389c1a2d70 con 0x7f389c103530 2024-05-14T16:54:43.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.225+0000 7f3893fff640 1 -- 172.21.15.2:0/2939111361 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f388c0054d0 con 0x7f389c103530 2024-05-14T16:54:43.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.225+0000 7f3893fff640 1 -- 172.21.15.2:0/2939111361 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c010430 con 0x7f389c103530 2024-05-14T16:54:43.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.225+0000 7f38a177f640 1 -- 172.21.15.2:0/2939111361 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f389c1a3180 con 0x7f389c103530 2024-05-14T16:54:43.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.226+0000 7f38a177f640 1 -- 172.21.15.2:0/2939111361 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f389c0fea10 con 0x7f389c103530 2024-05-14T16:54:43.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.228+0000 7f3893fff640 1 -- 172.21.15.2:0/2939111361 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7f388c0046c0 con 0x7f389c103530 2024-05-14T16:54:43.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.228+0000 7f3893fff640 1 --2- 172.21.15.2:0/2939111361 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f38880b2690 0x7f38880b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:43.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.228+0000 7f389a7fc640 1 --2- 172.21.15.2:0/2939111361 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f38880b2690 0x7f38880b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:43.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.229+0000 7f3893fff640 1 -- 172.21.15.2:0/2939111361 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f388c0e2190 con 0x7f389c103530 2024-05-14T16:54:43.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.229+0000 7f389a7fc640 1 --2- 172.21.15.2:0/2939111361 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f38880b2690 0x7f38880b4b50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f389c102bc0 tx=0x7f3884005f90 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:43.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.237+0000 7f3893fff640 1 -- 172.21.15.2:0/2939111361 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f388c0aa0b0 con 0x7f389c103530 2024-05-14T16:54:43.369 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:43 smithi002 ceph-mon[31538]: [14/May/2024:16:54:41] ENGINE Bus STARTING 2024-05-14T16:54:43.369 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:43 smithi002 ceph-mon[31538]: [14/May/2024:16:54:41] ENGINE Serving on http://172.21.15.2:8765 2024-05-14T16:54:43.369 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:43 smithi002 ceph-mon[31538]: [14/May/2024:16:54:41] ENGINE Serving on https://172.21.15.2:7150 2024-05-14T16:54:43.369 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:43 smithi002 ceph-mon[31538]: [14/May/2024:16:54:41] ENGINE Bus STARTED 2024-05-14T16:54:43.369 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:43 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:43.370 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:43 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:43.370 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:43 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:43.370 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:43 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:43.370 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:43 smithi002 ceph-mon[31538]: mgrmap e22: x(active, since 1.01885s), standbys: z, y 2024-05-14T16:54:43.370 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:43 smithi002 ceph-mon[31538]: from='client.24917 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:54:43.370 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:43 smithi002 ceph-mon[31538]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:54:43.409 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:43 smithi123 ceph-mon[35971]: [14/May/2024:16:54:41] ENGINE Bus STARTING 2024-05-14T16:54:43.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:43 smithi123 ceph-mon[35971]: [14/May/2024:16:54:41] ENGINE Serving on http://172.21.15.2:8765 2024-05-14T16:54:43.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:43 smithi123 ceph-mon[35971]: [14/May/2024:16:54:41] ENGINE Serving on https://172.21.15.2:7150 2024-05-14T16:54:43.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:43 smithi123 ceph-mon[35971]: [14/May/2024:16:54:41] ENGINE Bus STARTED 2024-05-14T16:54:43.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:43 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:43.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:43 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:43.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:43 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:43.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:43 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:43.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:43 smithi123 ceph-mon[35971]: mgrmap e22: x(active, since 1.01885s), standbys: z, y 2024-05-14T16:54:43.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:43 smithi123 ceph-mon[35971]: from='client.24917 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:54:43.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:43 smithi123 ceph-mon[35971]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:54:43.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.455+0000 7f38a177f640 1 -- 172.21.15.2:0/2939111361 --> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f389c062e90 con 0x7f38880b2690 2024-05-14T16:54:43.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.462+0000 7f3893fff640 1 -- 172.21.15.2:0/2939111361 <== mgr.15120 v2:172.21.15.2:6800/1339485805 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f389c062e90 con 0x7f38880b2690 2024-05-14T16:54:43.461 INFO:teuthology.orchestra.run.smithi002.stdout:/ 1m 6m3h 2024-05-14T16:54:43.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.464+0000 7f38a177f640 1 -- 172.21.15.2:0/2939111361 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f38880b2690 msgr2=0x7f38880b4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:43.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.464+0000 7f38a177f640 1 --2- 172.21.15.2:0/2939111361 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f38880b2690 0x7f38880b4b50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f389c102bc0 tx=0x7f3884005f90 comp rx=0 tx=0).stop 2024-05-14T16:54:43.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.464+0000 7f38a177f640 1 -- 172.21.15.2:0/2939111361 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f389c103530 msgr2=0x7f389c102e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:43.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.464+0000 7f38a177f640 1 --2- 172.21.15.2:0/2939111361 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f389c103530 0x7f389c102e50 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f388c00e9f0 tx=0x7f388c00eeb0 comp rx=0 tx=0).stop 2024-05-14T16:54:43.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.464+0000 7f38a177f640 1 -- 172.21.15.2:0/2939111361 shutdown_connections 2024-05-14T16:54:43.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.464+0000 7f38a177f640 1 --2- 172.21.15.2:0/2939111361 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f38880b2690 0x7f38880b4b50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:43.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.464+0000 7f38a177f640 1 --2- 172.21.15.2:0/2939111361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f389c108790 0x7f389c101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:43.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.464+0000 7f38a177f640 1 --2- 172.21.15.2:0/2939111361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f389c105e60 0x7f389c1014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:43.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.464+0000 7f38a177f640 1 --2- 172.21.15.2:0/2939111361 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f389c103530 0x7f389c102e50 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:43.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.464+0000 7f38a177f640 1 -- 172.21.15.2:0/2939111361 >> 172.21.15.2:0/2939111361 conn(0x7f389c0fcf20 msgr2=0x7f389c103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:43.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.464+0000 7f38a177f640 1 -- 172.21.15.2:0/2939111361 shutdown_connections 2024-05-14T16:54:43.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:43.464+0000 7f38a177f640 1 -- 172.21.15.2:0/2939111361 wait complete. 2024-05-14T16:54:43.475 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-05-14T16:54:43.502 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-05-14T16:54:43.513 INFO:teuthology.run_tasks:Running task background_exec... 2024-05-14T16:54:43.552 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi002.front.sepia.ceph.com 2024-05-14T16:54:43.553 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-05-14T16:54:43.555 INFO:teuthology.run_tasks:Running task quiescer... 2024-05-14T16:54:43.587 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-14T16:54:43.587 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-14T16:54:43.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:43 smithi151 ceph-mon[36066]: [14/May/2024:16:54:41] ENGINE Bus STARTING 2024-05-14T16:54:43.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:43 smithi151 ceph-mon[36066]: [14/May/2024:16:54:41] ENGINE Serving on http://172.21.15.2:8765 2024-05-14T16:54:43.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:43 smithi151 ceph-mon[36066]: [14/May/2024:16:54:41] ENGINE Serving on https://172.21.15.2:7150 2024-05-14T16:54:43.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:43 smithi151 ceph-mon[36066]: [14/May/2024:16:54:41] ENGINE Bus STARTED 2024-05-14T16:54:43.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:43 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:43.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:43 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:43.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:43 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:43.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:43 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:43.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:43 smithi151 ceph-mon[36066]: mgrmap e22: x(active, since 1.01885s), standbys: z, y 2024-05-14T16:54:43.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:43 smithi151 ceph-mon[36066]: from='client.24917 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:54:43.605 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:43 smithi151 ceph-mon[36066]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:54:43.873 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:54:44.653 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:44 smithi002 ceph-mon[31538]: from='client.15201 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:54:44.654 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:44 smithi002 ceph-mon[31538]: from='client.15207 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:54:44.654 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:44 smithi002 ceph-mon[31538]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:54:44.654 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:44 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:44.654 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:44 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:44.654 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:44 smithi002 ceph-mon[31538]: from='client.15213 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:54:44.654 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:44 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:44.654 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:44 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:44.654 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:44 smithi002 ceph-mon[31538]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-05-14T16:54:44.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:44 smithi123 ceph-mon[35971]: from='client.15201 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:54:44.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:44 smithi123 ceph-mon[35971]: from='client.15207 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:54:44.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:44 smithi123 ceph-mon[35971]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:54:44.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:44 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:44.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:44 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:44.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:44 smithi123 ceph-mon[35971]: from='client.15213 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:54:44.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:44 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:44.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:44 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:44.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:44 smithi123 ceph-mon[35971]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-05-14T16:54:44.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:44 smithi151 ceph-mon[36066]: from='client.15201 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:54:44.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:44 smithi151 ceph-mon[36066]: from='client.15207 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:54:44.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:44 smithi151 ceph-mon[36066]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:54:44.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:44 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:44.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:44 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:44.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:44 smithi151 ceph-mon[36066]: from='client.15213 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T16:54:44.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:44 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:44.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:44 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:44.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:44 smithi151 ceph-mon[36066]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-05-14T16:54:45.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.624+0000 7f96a00c1700 1 -- 172.21.15.2:0/4017650420 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9698102440 msgr2=0x7f96981028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:45.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.624+0000 7f96a00c1700 1 --2- 172.21.15.2:0/4017650420 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9698102440 0x7f96981028c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9688009990 tx=0x7f9688009ca0 comp rx=0 tx=0).stop 2024-05-14T16:54:45.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.625+0000 7f96a00c1700 1 -- 172.21.15.2:0/4017650420 shutdown_connections 2024-05-14T16:54:45.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.625+0000 7f96a00c1700 1 --2- 172.21.15.2:0/4017650420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9698102e00 0x7f969810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:45.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.625+0000 7f96a00c1700 1 --2- 172.21.15.2:0/4017650420 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9698108520 0x7f9698108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:45.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.625+0000 7f96a00c1700 1 --2- 172.21.15.2:0/4017650420 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9698102440 0x7f96981028c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:45.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.625+0000 7f96a00c1700 1 -- 172.21.15.2:0/4017650420 >> 172.21.15.2:0/4017650420 conn(0x7f96980793f0 msgr2=0x7f9698079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:45.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.625+0000 7f96a00c1700 1 -- 172.21.15.2:0/4017650420 shutdown_connections 2024-05-14T16:54:45.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.625+0000 7f96a00c1700 1 -- 172.21.15.2:0/4017650420 wait complete. 2024-05-14T16:54:45.626 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:45 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:45.626 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:45 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:45.626 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:45 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:54:45.626 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:45 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:54:45.626 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:45 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:54:45.626 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:45 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:54:45.626 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:45 smithi002 ceph-mon[31538]: Adjusting osd_memory_target on smithi151 to 176.2M 2024-05-14T16:54:45.626 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:45 smithi002 ceph-mon[31538]: Unable to set osd_memory_target on smithi151 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-05-14T16:54:45.626 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:45 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:45.626 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:45 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:45.627 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:45 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:54:45.627 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:45 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:54:45.627 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:45 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:54:45.627 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:45 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:54:45.627 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:45 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:45.627 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:45 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:45.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.625+0000 7f96a00c1700 1 Processor -- start 2024-05-14T16:54:45.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.626+0000 7f96a00c1700 1 -- start start 2024-05-14T16:54:45.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.626+0000 7f96a00c1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9698102440 0x7f969819be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:45.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.626+0000 7f96a00c1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9698102e00 0x7f969819c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:45.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.626+0000 7f96a00c1700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9698108520 0x7f96981a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:45.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.626+0000 7f96a00c1700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f96981125d0 con 0x7f9698102440 2024-05-14T16:54:45.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.626+0000 7f96a00c1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9698112450 con 0x7f9698102e00 2024-05-14T16:54:45.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.626+0000 7f96a00c1700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9698112750 con 0x7f9698108520 2024-05-14T16:54:45.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.626+0000 7f969de5d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9698102440 0x7f969819be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:45.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.626+0000 7f969de5d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9698102440 0x7f969819be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57070/0 (socket says 172.21.15.2:57070) 2024-05-14T16:54:45.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.626+0000 7f969de5d700 1 -- 172.21.15.2:0/1679060157 learned_addr learned my addr 172.21.15.2:0/1679060157 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:45.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.626+0000 7f969e65e700 1 --2- 172.21.15.2:0/1679060157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9698108520 0x7f96981a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:45.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.626+0000 7f969de5d700 1 -- 172.21.15.2:0/1679060157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9698108520 msgr2=0x7f96981a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:45.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.626+0000 7f969de5d700 1 --2- 172.21.15.2:0/1679060157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9698108520 0x7f96981a0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:45.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.626+0000 7f969de5d700 1 -- 172.21.15.2:0/1679060157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9698102e00 msgr2=0x7f969819c380 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-14T16:54:45.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.626+0000 7f969d65c700 1 --2- 172.21.15.2:0/1679060157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9698102e00 0x7f969819c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:45.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.627+0000 7f969de5d700 1 --2- 172.21.15.2:0/1679060157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9698102e00 0x7f969819c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:45.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.627+0000 7f969de5d700 1 -- 172.21.15.2:0/1679060157 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96981a0e50 con 0x7f9698102440 2024-05-14T16:54:45.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.627+0000 7f969de5d700 1 --2- 172.21.15.2:0/1679060157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9698102440 0x7f969819be40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f969400d950 tx=0x7f969400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:45.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.627+0000 7f968effd700 1 -- 172.21.15.2:0/1679060157 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9694016070 con 0x7f9698102440 2024-05-14T16:54:45.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.627+0000 7f968effd700 1 -- 172.21.15.2:0/1679060157 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9694009df0 con 0x7f9698102440 2024-05-14T16:54:45.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.627+0000 7f96a00c1700 1 -- 172.21.15.2:0/1679060157 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96981a1140 con 0x7f9698102440 2024-05-14T16:54:45.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.627+0000 7f968effd700 1 -- 172.21.15.2:0/1679060157 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969400f3c0 con 0x7f9698102440 2024-05-14T16:54:45.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.627+0000 7f96a00c1700 1 -- 172.21.15.2:0/1679060157 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96981a89c0 con 0x7f9698102440 2024-05-14T16:54:45.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.628+0000 7f96a00c1700 1 -- 172.21.15.2:0/1679060157 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f969806a230 con 0x7f9698102440 2024-05-14T16:54:45.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.631+0000 7f968effd700 1 -- 172.21.15.2:0/1679060157 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 23) ==== 149771+0+0 (secure 0 0 0) 0x7f9694019070 con 0x7f9698102440 2024-05-14T16:54:45.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.631+0000 7f968effd700 1 --2- 172.21.15.2:0/1679060157 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f96840b29f0 0x7f96840b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:45.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.631+0000 7f969d65c700 1 --2- 172.21.15.2:0/1679060157 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f96840b29f0 0x7f96840b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:45.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.631+0000 7f968effd700 1 -- 172.21.15.2:0/1679060157 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f96940e20d0 con 0x7f9698102440 2024-05-14T16:54:45.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.632+0000 7f969d65c700 1 --2- 172.21.15.2:0/1679060157 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f96840b29f0 0x7f96840b4eb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f968800b2e0 tx=0x7f9688005e80 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:45.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.636+0000 7f968effd700 1 -- 172.21.15.2:0/1679060157 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f96940a9b80 con 0x7f9698102440 2024-05-14T16:54:45.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:45 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:45.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:45 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:45.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:45 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:54:45.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:45 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:54:45.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:45 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:54:45.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:45 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:54:45.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:45 smithi151 ceph-mon[36066]: Adjusting osd_memory_target on smithi151 to 176.2M 2024-05-14T16:54:45.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:45 smithi151 ceph-mon[36066]: Unable to set osd_memory_target on smithi151 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-05-14T16:54:45.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:45 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:45.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:45 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:45.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:45 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:54:45.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:45 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:54:45.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:45 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:54:45.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:45 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:54:45.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:45 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:45.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:45 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:45.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.856+0000 7f96a00c1700 1 -- 172.21.15.2:0/1679060157 --> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f969806b930 con 0x7f96840b29f0 2024-05-14T16:54:45.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.864+0000 7f968effd700 1 -- 172.21.15.2:0/1679060157 <== mgr.15120 v2:172.21.15.2:6800/1339485805 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257993 (secure 0 0 0) 0x7f969806b930 con 0x7f96840b29f0 2024-05-14T16:54:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:54:45.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.867+0000 7f96a00c1700 1 -- 172.21.15.2:0/1679060157 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f96840b29f0 msgr2=0x7f96840b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:45.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.867+0000 7f96a00c1700 1 --2- 172.21.15.2:0/1679060157 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f96840b29f0 0x7f96840b4eb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f968800b2e0 tx=0x7f9688005e80 comp rx=0 tx=0).stop 2024-05-14T16:54:45.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.867+0000 7f96a00c1700 1 -- 172.21.15.2:0/1679060157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9698102440 msgr2=0x7f969819be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:45.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.867+0000 7f96a00c1700 1 --2- 172.21.15.2:0/1679060157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9698102440 0x7f969819be40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f969400d950 tx=0x7f969400dc60 comp rx=0 tx=0).stop 2024-05-14T16:54:45.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.867+0000 7f96a00c1700 1 -- 172.21.15.2:0/1679060157 shutdown_connections 2024-05-14T16:54:45.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.867+0000 7f96a00c1700 1 --2- 172.21.15.2:0/1679060157 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f96840b29f0 0x7f96840b4eb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:45.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.867+0000 7f96a00c1700 1 --2- 172.21.15.2:0/1679060157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9698108520 0x7f96981a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:45.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.867+0000 7f96a00c1700 1 --2- 172.21.15.2:0/1679060157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9698102440 0x7f969819be40 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:45.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.867+0000 7f96a00c1700 1 --2- 172.21.15.2:0/1679060157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9698102e00 0x7f969819c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:45.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.867+0000 7f96a00c1700 1 -- 172.21.15.2:0/1679060157 >> 172.21.15.2:0/1679060157 conn(0x7f96980793f0 msgr2=0x7f969806b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:45.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.867+0000 7f96a00c1700 1 -- 172.21.15.2:0/1679060157 shutdown_connections 2024-05-14T16:54:45.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:45.867+0000 7f96a00c1700 1 -- 172.21.15.2:0/1679060157 wait complete. 2024-05-14T16:54:45.875 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-05-14T16:54:45.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:45 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:45.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:45 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:45.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:45 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:54:45.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:45 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:54:45.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:45 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:54:45.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:45 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:54:45.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:45 smithi123 ceph-mon[35971]: Adjusting osd_memory_target on smithi151 to 176.2M 2024-05-14T16:54:45.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:45 smithi123 ceph-mon[35971]: Unable to set osd_memory_target on smithi151 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-05-14T16:54:45.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:45 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:45.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:45 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:45.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:45 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:54:45.911 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:45 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:54:45.911 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:45 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:54:45.911 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:45 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:54:45.911 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:45 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:45.911 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:45 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:46.646 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:46 smithi002 ceph-mon[31538]: Adjusting osd_memory_target on smithi123 to 176.2M 2024-05-14T16:54:46.647 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:46 smithi002 ceph-mon[31538]: Unable to set osd_memory_target on smithi123 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-05-14T16:54:46.647 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:46 smithi002 ceph-mon[31538]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:54:46.647 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:46 smithi002 ceph-mon[31538]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-05-14T16:54:46.674 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-05-14T16:54:45.208310+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1187935,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1232,"num_read_kb":752,"num_write":421,"num_write_kb":3592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":2951957,"num_keys_recovered":9,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":825,"ondisk_log_size":825,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":385092,"kb_used_data":10952,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124556732,"statfs":{"total":1151940427776,"available":1151546093568,"internally_reserved":0,"allocated":11214848,"data_stored":8763550,"data_compressed":219420,"data_compressed_allocated":282624,"data_compressed_original":3416064,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.989407"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.197029+0000","last_change":"2024-05-14T16:51:32.298580+0000","last_active":"2024-05-14T16:54:41.197029+0000","last_peered":"2024-05-14T16:54:41.197029+0000","last_clean":"2024-05-14T16:54:41.197029+0000","last_became_active":"2024-05-14T16:51:32.296945+0000","last_became_peered":"2024-05-14T16:51:32.296945+0000","last_unstale":"2024-05-14T16:54:41.197029+0000","last_undegraded":"2024-05-14T16:54:41.197029+0000","last_fullsized":"2024-05-14T16:54:41.197029+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:50:17.664928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206588+0000","last_change":"2024-05-14T16:51:32.297618+0000","last_active":"2024-05-14T16:54:41.206588+0000","last_peered":"2024-05-14T16:54:41.206588+0000","last_clean":"2024-05-14T16:54:41.206588+0000","last_became_active":"2024-05-14T16:51:32.297260+0000","last_became_peered":"2024-05-14T16:51:32.297260+0000","last_unstale":"2024-05-14T16:54:41.206588+0000","last_undegraded":"2024-05-14T16:54:41.206588+0000","last_fullsized":"2024-05-14T16:54:41.206588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:43:50.542587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206503+0000","last_change":"2024-05-14T16:51:32.295525+0000","last_active":"2024-05-14T16:54:41.206503+0000","last_peered":"2024-05-14T16:54:41.206503+0000","last_clean":"2024-05-14T16:54:41.206503+0000","last_became_active":"2024-05-14T16:51:32.295215+0000","last_became_peered":"2024-05-14T16:51:32.295215+0000","last_unstale":"2024-05-14T16:54:41.206503+0000","last_undegraded":"2024-05-14T16:54:41.206503+0000","last_fullsized":"2024-05-14T16:54:41.206503+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:40:05.516280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207077+0000","last_change":"2024-05-14T16:51:32.307883+0000","last_active":"2024-05-14T16:54:41.207077+0000","last_peered":"2024-05-14T16:54:41.207077+0000","last_clean":"2024-05-14T16:54:41.207077+0000","last_became_active":"2024-05-14T16:51:32.307499+0000","last_became_peered":"2024-05-14T16:51:32.307499+0000","last_unstale":"2024-05-14T16:54:41.207077+0000","last_undegraded":"2024-05-14T16:54:41.207077+0000","last_fullsized":"2024-05-14T16:54:41.207077+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:30:59.978059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206362+0000","last_change":"2024-05-14T16:51:32.294429+0000","last_active":"2024-05-14T16:54:41.206362+0000","last_peered":"2024-05-14T16:54:41.206362+0000","last_clean":"2024-05-14T16:54:41.206362+0000","last_became_active":"2024-05-14T16:51:32.294088+0000","last_became_peered":"2024-05-14T16:51:32.294088+0000","last_unstale":"2024-05-14T16:54:41.206362+0000","last_undegraded":"2024-05-14T16:54:41.206362+0000","last_fullsized":"2024-05-14T16:54:41.206362+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:25:19.383586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.194143+0000","last_change":"2024-05-14T16:51:32.296169+0000","last_active":"2024-05-14T16:54:41.194143+0000","last_peered":"2024-05-14T16:54:41.194143+0000","last_clean":"2024-05-14T16:54:41.194143+0000","last_became_active":"2024-05-14T16:51:32.295819+0000","last_became_peered":"2024-05-14T16:51:32.295819+0000","last_unstale":"2024-05-14T16:54:41.194143+0000","last_undegraded":"2024-05-14T16:54:41.194143+0000","last_fullsized":"2024-05-14T16:54:41.194143+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:56:23.932039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206285+0000","last_change":"2024-05-14T16:51:32.309609+0000","last_active":"2024-05-14T16:54:41.206285+0000","last_peered":"2024-05-14T16:54:41.206285+0000","last_clean":"2024-05-14T16:54:41.206285+0000","last_became_active":"2024-05-14T16:51:32.309175+0000","last_became_peered":"2024-05-14T16:51:32.309175+0000","last_unstale":"2024-05-14T16:54:41.206285+0000","last_undegraded":"2024-05-14T16:54:41.206285+0000","last_fullsized":"2024-05-14T16:54:41.206285+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:06:46.717592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207138+0000","last_change":"2024-05-14T16:52:13.446592+0000","last_active":"2024-05-14T16:54:41.207138+0000","last_peered":"2024-05-14T16:54:41.207138+0000","last_clean":"2024-05-14T16:54:41.207138+0000","last_became_active":"2024-05-14T16:52:13.446238+0000","last_became_peered":"2024-05-14T16:52:13.446238+0000","last_unstale":"2024-05-14T16:54:41.207138+0000","last_undegraded":"2024-05-14T16:54:41.207138+0000","last_fullsized":"2024-05-14T16:54:41.207138+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:33:14.216797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206922+0000","last_change":"2024-05-14T16:51:32.307571+0000","last_active":"2024-05-14T16:54:41.206922+0000","last_peered":"2024-05-14T16:54:41.206922+0000","last_clean":"2024-05-14T16:54:41.206922+0000","last_became_active":"2024-05-14T16:51:32.307151+0000","last_became_peered":"2024-05-14T16:51:32.307151+0000","last_unstale":"2024-05-14T16:54:41.206922+0000","last_undegraded":"2024-05-14T16:54:41.206922+0000","last_fullsized":"2024-05-14T16:54:41.206922+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:17:26.325393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.194394+0000","last_change":"2024-05-14T16:51:32.298773+0000","last_active":"2024-05-14T16:54:41.194394+0000","last_peered":"2024-05-14T16:54:41.194394+0000","last_clean":"2024-05-14T16:54:41.194394+0000","last_became_active":"2024-05-14T16:51:32.298283+0000","last_became_peered":"2024-05-14T16:51:32.298283+0000","last_unstale":"2024-05-14T16:54:41.194394+0000","last_undegraded":"2024-05-14T16:54:41.194394+0000","last_fullsized":"2024-05-14T16:54:41.194394+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:52:18.471201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207583+0000","last_change":"2024-05-14T16:51:32.302443+0000","last_active":"2024-05-14T16:54:41.207583+0000","last_peered":"2024-05-14T16:54:41.207583+0000","last_clean":"2024-05-14T16:54:41.207583+0000","last_became_active":"2024-05-14T16:51:32.302034+0000","last_became_peered":"2024-05-14T16:51:32.302034+0000","last_unstale":"2024-05-14T16:54:41.207583+0000","last_undegraded":"2024-05-14T16:54:41.207583+0000","last_fullsized":"2024-05-14T16:54:41.207583+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:53:08.041293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207252+0000","last_change":"2024-05-14T16:51:32.304837+0000","last_active":"2024-05-14T16:54:41.207252+0000","last_peered":"2024-05-14T16:54:41.207252+0000","last_clean":"2024-05-14T16:54:41.207252+0000","last_became_active":"2024-05-14T16:51:32.304500+0000","last_became_peered":"2024-05-14T16:51:32.304500+0000","last_unstale":"2024-05-14T16:54:41.207252+0000","last_undegraded":"2024-05-14T16:54:41.207252+0000","last_fullsized":"2024-05-14T16:54:41.207252+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:32:02.713044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.192789+0000","last_change":"2024-05-14T16:51:32.299247+0000","last_active":"2024-05-14T16:54:41.192789+0000","last_peered":"2024-05-14T16:54:41.192789+0000","last_clean":"2024-05-14T16:54:41.192789+0000","last_became_active":"2024-05-14T16:51:32.298925+0000","last_became_peered":"2024-05-14T16:51:32.298925+0000","last_unstale":"2024-05-14T16:54:41.192789+0000","last_undegraded":"2024-05-14T16:54:41.192789+0000","last_fullsized":"2024-05-14T16:54:41.192789+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:10:38.847877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.205900+0000","last_change":"2024-05-14T16:51:32.293743+0000","last_active":"2024-05-14T16:54:41.205900+0000","last_peered":"2024-05-14T16:54:41.205900+0000","last_clean":"2024-05-14T16:54:41.205900+0000","last_became_active":"2024-05-14T16:51:32.293429+0000","last_became_peered":"2024-05-14T16:51:32.293429+0000","last_unstale":"2024-05-14T16:54:41.205900+0000","last_undegraded":"2024-05-14T16:54:41.205900+0000","last_fullsized":"2024-05-14T16:54:41.205900+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:12:33.909940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.205749+0000","last_change":"2024-05-14T16:51:32.304940+0000","last_active":"2024-05-14T16:54:41.205749+0000","last_peered":"2024-05-14T16:54:41.205749+0000","last_clean":"2024-05-14T16:54:41.205749+0000","last_became_active":"2024-05-14T16:51:32.304482+0000","last_became_peered":"2024-05-14T16:51:32.304482+0000","last_unstale":"2024-05-14T16:54:41.205749+0000","last_undegraded":"2024-05-14T16:54:41.205749+0000","last_fullsized":"2024-05-14T16:54:41.205749+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:54:17.771620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206201+0000","last_change":"2024-05-14T16:51:32.301784+0000","last_active":"2024-05-14T16:54:41.206201+0000","last_peered":"2024-05-14T16:54:41.206201+0000","last_clean":"2024-05-14T16:54:41.206201+0000","last_became_active":"2024-05-14T16:51:32.301472+0000","last_became_peered":"2024-05-14T16:51:32.301472+0000","last_unstale":"2024-05-14T16:54:41.206201+0000","last_undegraded":"2024-05-14T16:54:41.206201+0000","last_fullsized":"2024-05-14T16:54:41.206201+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:10:56.735424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.197151+0000","last_change":"2024-05-14T16:51:32.297951+0000","last_active":"2024-05-14T16:54:41.197151+0000","last_peered":"2024-05-14T16:54:41.197151+0000","last_clean":"2024-05-14T16:54:41.197151+0000","last_became_active":"2024-05-14T16:51:32.296602+0000","last_became_peered":"2024-05-14T16:51:32.296602+0000","last_unstale":"2024-05-14T16:54:41.197151+0000","last_undegraded":"2024-05-14T16:54:41.197151+0000","last_fullsized":"2024-05-14T16:54:41.197151+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:16:36.039077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.208451+0000","last_change":"2024-05-14T16:51:32.290398+0000","last_active":"2024-05-14T16:54:41.208451+0000","last_peered":"2024-05-14T16:54:41.208451+0000","last_clean":"2024-05-14T16:54:41.208451+0000","last_became_active":"2024-05-14T16:51:32.290083+0000","last_became_peered":"2024-05-14T16:51:32.290083+0000","last_unstale":"2024-05-14T16:54:41.208451+0000","last_undegraded":"2024-05-14T16:54:41.208451+0000","last_fullsized":"2024-05-14T16:54:41.208451+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:48:05.795209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207421+0000","last_change":"2024-05-14T16:51:32.294398+0000","last_active":"2024-05-14T16:54:41.207421+0000","last_peered":"2024-05-14T16:54:41.207421+0000","last_clean":"2024-05-14T16:54:41.207421+0000","last_became_active":"2024-05-14T16:51:32.294080+0000","last_became_peered":"2024-05-14T16:51:32.294080+0000","last_unstale":"2024-05-14T16:54:41.207421+0000","last_undegraded":"2024-05-14T16:54:41.207421+0000","last_fullsized":"2024-05-14T16:54:41.207421+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:15:03.859045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.208336+0000","last_change":"2024-05-14T16:51:29.274482+0000","last_active":"2024-05-14T16:54:41.208336+0000","last_peered":"2024-05-14T16:54:41.208336+0000","last_clean":"2024-05-14T16:54:41.208336+0000","last_became_active":"2024-05-14T16:51:29.274152+0000","last_became_peered":"2024-05-14T16:51:29.274152+0000","last_unstale":"2024-05-14T16:54:41.208336+0000","last_undegraded":"2024-05-14T16:54:41.208336+0000","last_fullsized":"2024-05-14T16:54:41.208336+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:05:38.274248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207141+0000","last_change":"2024-05-14T16:51:32.300956+0000","last_active":"2024-05-14T16:54:41.207141+0000","last_peered":"2024-05-14T16:54:41.207141+0000","last_clean":"2024-05-14T16:54:41.207141+0000","last_became_active":"2024-05-14T16:51:32.300546+0000","last_became_peered":"2024-05-14T16:51:32.300546+0000","last_unstale":"2024-05-14T16:54:41.207141+0000","last_undegraded":"2024-05-14T16:54:41.207141+0000","last_fullsized":"2024-05-14T16:54:41.207141+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:54:09.181036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207390+0000","last_change":"2024-05-14T16:51:29.345206+0000","last_active":"2024-05-14T16:54:41.207390+0000","last_peered":"2024-05-14T16:54:41.207390+0000","last_clean":"2024-05-14T16:54:41.207390+0000","last_became_active":"2024-05-14T16:51:29.344699+0000","last_became_peered":"2024-05-14T16:51:29.344699+0000","last_unstale":"2024-05-14T16:54:41.207390+0000","last_undegraded":"2024-05-14T16:54:41.207390+0000","last_fullsized":"2024-05-14T16:54:41.207390+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:14:02.364661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206815+0000","last_change":"2024-05-14T16:51:32.297522+0000","last_active":"2024-05-14T16:54:41.206815+0000","last_peered":"2024-05-14T16:54:41.206815+0000","last_clean":"2024-05-14T16:54:41.206815+0000","last_became_active":"2024-05-14T16:51:32.296866+0000","last_became_peered":"2024-05-14T16:51:32.296866+0000","last_unstale":"2024-05-14T16:54:41.206815+0000","last_undegraded":"2024-05-14T16:54:41.206815+0000","last_fullsized":"2024-05-14T16:54:41.206815+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:49:43.409575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207355+0000","last_change":"2024-05-14T16:51:29.280632+0000","last_active":"2024-05-14T16:54:41.207355+0000","last_peered":"2024-05-14T16:54:41.207355+0000","last_clean":"2024-05-14T16:54:41.207355+0000","last_became_active":"2024-05-14T16:51:29.280246+0000","last_became_peered":"2024-05-14T16:51:29.280246+0000","last_unstale":"2024-05-14T16:54:41.207355+0000","last_undegraded":"2024-05-14T16:54:41.207355+0000","last_fullsized":"2024-05-14T16:54:41.207355+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:34:47.925344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206511+0000","last_change":"2024-05-14T16:51:32.298284+0000","last_active":"2024-05-14T16:54:41.206511+0000","last_peered":"2024-05-14T16:54:41.206511+0000","last_clean":"2024-05-14T16:54:41.206511+0000","last_became_active":"2024-05-14T16:51:32.294202+0000","last_became_peered":"2024-05-14T16:51:32.294202+0000","last_unstale":"2024-05-14T16:54:41.206511+0000","last_undegraded":"2024-05-14T16:54:41.206511+0000","last_fullsized":"2024-05-14T16:54:41.206511+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:27:37.561275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206773+0000","last_change":"2024-05-14T16:51:29.276227+0000","last_active":"2024-05-14T16:54:41.206773+0000","last_peered":"2024-05-14T16:54:41.206773+0000","last_clean":"2024-05-14T16:54:41.206773+0000","last_became_active":"2024-05-14T16:51:29.275837+0000","last_became_peered":"2024-05-14T16:51:29.275837+0000","last_unstale":"2024-05-14T16:54:41.206773+0000","last_undegraded":"2024-05-14T16:54:41.206773+0000","last_fullsized":"2024-05-14T16:54:41.206773+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:03:29.402985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207149+0000","last_change":"2024-05-14T16:51:32.308173+0000","last_active":"2024-05-14T16:54:41.207149+0000","last_peered":"2024-05-14T16:54:41.207149+0000","last_clean":"2024-05-14T16:54:41.207149+0000","last_became_active":"2024-05-14T16:51:32.307135+0000","last_became_peered":"2024-05-14T16:51:32.307135+0000","last_unstale":"2024-05-14T16:54:41.207149+0000","last_undegraded":"2024-05-14T16:54:41.207149+0000","last_fullsized":"2024-05-14T16:54:41.207149+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:01:46.579968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'8","reported_seq":212,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.272035+0000","last_change":"2024-05-14T16:51:29.279486+0000","last_active":"2024-05-14T16:54:41.272035+0000","last_peered":"2024-05-14T16:54:41.272035+0000","last_clean":"2024-05-14T16:54:41.272035+0000","last_became_active":"2024-05-14T16:51:29.279133+0000","last_became_peered":"2024-05-14T16:51:29.279133+0000","last_unstale":"2024-05-14T16:54:41.272035+0000","last_undegraded":"2024-05-14T16:54:41.272035+0000","last_fullsized":"2024-05-14T16:54:41.272035+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:26:18.620421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":175,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206811+0000","last_change":"2024-05-14T16:51:32.299852+0000","last_active":"2024-05-14T16:54:41.206811+0000","last_peered":"2024-05-14T16:54:41.206811+0000","last_clean":"2024-05-14T16:54:41.206811+0000","last_became_active":"2024-05-14T16:51:32.299541+0000","last_became_peered":"2024-05-14T16:51:32.299541+0000","last_unstale":"2024-05-14T16:54:41.206811+0000","last_undegraded":"2024-05-14T16:54:41.206811+0000","last_fullsized":"2024-05-14T16:54:41.206811+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:36:38.191009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.194073+0000","last_change":"2024-05-14T16:51:29.275269+0000","last_active":"2024-05-14T16:54:41.194073+0000","last_peered":"2024-05-14T16:54:41.194073+0000","last_clean":"2024-05-14T16:54:41.194073+0000","last_became_active":"2024-05-14T16:51:29.274792+0000","last_became_peered":"2024-05-14T16:51:29.274792+0000","last_unstale":"2024-05-14T16:54:41.194073+0000","last_undegraded":"2024-05-14T16:54:41.194073+0000","last_fullsized":"2024-05-14T16:54:41.194073+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:17:08.925289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207090+0000","last_change":"2024-05-14T16:51:32.306536+0000","last_active":"2024-05-14T16:54:41.207090+0000","last_peered":"2024-05-14T16:54:41.207090+0000","last_clean":"2024-05-14T16:54:41.207090+0000","last_became_active":"2024-05-14T16:51:32.304976+0000","last_became_peered":"2024-05-14T16:51:32.304976+0000","last_unstale":"2024-05-14T16:54:41.207090+0000","last_undegraded":"2024-05-14T16:54:41.207090+0000","last_fullsized":"2024-05-14T16:54:41.207090+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:48:13.284022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.192664+0000","last_change":"2024-05-14T16:51:29.266783+0000","last_active":"2024-05-14T16:54:41.192664+0000","last_peered":"2024-05-14T16:54:41.192664+0000","last_clean":"2024-05-14T16:54:41.192664+0000","last_became_active":"2024-05-14T16:51:29.266409+0000","last_became_peered":"2024-05-14T16:51:29.266409+0000","last_unstale":"2024-05-14T16:54:41.192664+0000","last_undegraded":"2024-05-14T16:54:41.192664+0000","last_fullsized":"2024-05-14T16:54:41.192664+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:33:14.981709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206568+0000","last_change":"2024-05-14T16:51:32.289288+0000","last_active":"2024-05-14T16:54:41.206568+0000","last_peered":"2024-05-14T16:54:41.206568+0000","last_clean":"2024-05-14T16:54:41.206568+0000","last_became_active":"2024-05-14T16:51:32.288920+0000","last_became_peered":"2024-05-14T16:51:32.288920+0000","last_unstale":"2024-05-14T16:54:41.206568+0000","last_undegraded":"2024-05-14T16:54:41.206568+0000","last_fullsized":"2024-05-14T16:54:41.206568+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:23:43.659417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206346+0000","last_change":"2024-05-14T16:51:29.273926+0000","last_active":"2024-05-14T16:54:41.206346+0000","last_peered":"2024-05-14T16:54:41.206346+0000","last_clean":"2024-05-14T16:54:41.206346+0000","last_became_active":"2024-05-14T16:51:29.273059+0000","last_became_peered":"2024-05-14T16:51:29.273059+0000","last_unstale":"2024-05-14T16:54:41.206346+0000","last_undegraded":"2024-05-14T16:54:41.206346+0000","last_fullsized":"2024-05-14T16:54:41.206346+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:46:27.800481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206716+0000","last_change":"2024-05-14T16:51:32.287554+0000","last_active":"2024-05-14T16:54:41.206716+0000","last_peered":"2024-05-14T16:54:41.206716+0000","last_clean":"2024-05-14T16:54:41.206716+0000","last_became_active":"2024-05-14T16:51:32.287204+0000","last_became_peered":"2024-05-14T16:51:32.287204+0000","last_unstale":"2024-05-14T16:54:41.206716+0000","last_undegraded":"2024-05-14T16:54:41.206716+0000","last_fullsized":"2024-05-14T16:54:41.206716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:29:40.675378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'5","reported_seq":205,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.271284+0000","last_change":"2024-05-14T16:51:29.274819+0000","last_active":"2024-05-14T16:54:41.271284+0000","last_peered":"2024-05-14T16:54:41.271284+0000","last_clean":"2024-05-14T16:54:41.271284+0000","last_became_active":"2024-05-14T16:51:29.274454+0000","last_became_peered":"2024-05-14T16:51:29.274454+0000","last_unstale":"2024-05-14T16:54:41.271284+0000","last_undegraded":"2024-05-14T16:54:41.271284+0000","last_fullsized":"2024-05-14T16:54:41.271284+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:36:30.496064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":171,"num_read_kb":171,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206897+0000","last_change":"2024-05-14T16:51:32.295732+0000","last_active":"2024-05-14T16:54:41.206897+0000","last_peered":"2024-05-14T16:54:41.206897+0000","last_clean":"2024-05-14T16:54:41.206897+0000","last_became_active":"2024-05-14T16:51:32.294919+0000","last_became_peered":"2024-05-14T16:51:32.294919+0000","last_unstale":"2024-05-14T16:54:41.206897+0000","last_undegraded":"2024-05-14T16:54:41.206897+0000","last_fullsized":"2024-05-14T16:54:41.206897+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:08:54.566601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207342+0000","last_change":"2024-05-14T16:51:29.279338+0000","last_active":"2024-05-14T16:54:41.207342+0000","last_peered":"2024-05-14T16:54:41.207342+0000","last_clean":"2024-05-14T16:54:41.207342+0000","last_became_active":"2024-05-14T16:51:29.278928+0000","last_became_peered":"2024-05-14T16:51:29.278928+0000","last_unstale":"2024-05-14T16:54:41.207342+0000","last_undegraded":"2024-05-14T16:54:41.207342+0000","last_fullsized":"2024-05-14T16:54:41.207342+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:24:34.803515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206905+0000","last_change":"2024-05-14T16:51:32.305294+0000","last_active":"2024-05-14T16:54:41.206905+0000","last_peered":"2024-05-14T16:54:41.206905+0000","last_clean":"2024-05-14T16:54:41.206905+0000","last_became_active":"2024-05-14T16:51:32.304563+0000","last_became_peered":"2024-05-14T16:51:32.304563+0000","last_unstale":"2024-05-14T16:54:41.206905+0000","last_undegraded":"2024-05-14T16:54:41.206905+0000","last_fullsized":"2024-05-14T16:54:41.206905+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:56:06.259475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207460+0000","last_change":"2024-05-14T16:51:29.345400+0000","last_active":"2024-05-14T16:54:41.207460+0000","last_peered":"2024-05-14T16:54:41.207460+0000","last_clean":"2024-05-14T16:54:41.207460+0000","last_became_active":"2024-05-14T16:51:29.345010+0000","last_became_peered":"2024-05-14T16:51:29.345010+0000","last_unstale":"2024-05-14T16:54:41.207460+0000","last_undegraded":"2024-05-14T16:54:41.207460+0000","last_fullsized":"2024-05-14T16:54:41.207460+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:24:22.933005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206748+0000","last_change":"2024-05-14T16:52:13.923167+0000","last_active":"2024-05-14T16:54:41.206748+0000","last_peered":"2024-05-14T16:54:41.206748+0000","last_clean":"2024-05-14T16:54:41.206748+0000","last_became_active":"2024-05-14T16:52:13.922813+0000","last_became_peered":"2024-05-14T16:52:13.922813+0000","last_unstale":"2024-05-14T16:54:41.206748+0000","last_undegraded":"2024-05-14T16:54:41.206748+0000","last_fullsized":"2024-05-14T16:54:41.206748+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:50:38.672408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'38","reported_seq":421,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.462964+0000","last_change":"2024-05-14T16:51:29.277591+0000","last_active":"2024-05-14T16:54:41.462964+0000","last_peered":"2024-05-14T16:54:41.462964+0000","last_clean":"2024-05-14T16:54:41.462964+0000","last_became_active":"2024-05-14T16:51:29.276612+0000","last_became_peered":"2024-05-14T16:51:29.276612+0000","last_unstale":"2024-05-14T16:54:41.462964+0000","last_undegraded":"2024-05-14T16:54:41.462964+0000","last_fullsized":"2024-05-14T16:54:41.462964+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:22:28.934173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66894,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":354,"num_read_kb":241,"num_write":39,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207212+0000","last_change":"2024-05-14T16:51:32.302321+0000","last_active":"2024-05-14T16:54:41.207212+0000","last_peered":"2024-05-14T16:54:41.207212+0000","last_clean":"2024-05-14T16:54:41.207212+0000","last_became_active":"2024-05-14T16:51:32.301973+0000","last_became_peered":"2024-05-14T16:51:32.301973+0000","last_unstale":"2024-05-14T16:54:41.207212+0000","last_undegraded":"2024-05-14T16:54:41.207212+0000","last_fullsized":"2024-05-14T16:54:41.207212+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:49:24.054379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207061+0000","last_change":"2024-05-14T16:51:29.278776+0000","last_active":"2024-05-14T16:54:41.207061+0000","last_peered":"2024-05-14T16:54:41.207061+0000","last_clean":"2024-05-14T16:54:41.207061+0000","last_became_active":"2024-05-14T16:51:29.278395+0000","last_became_peered":"2024-05-14T16:51:29.278395+0000","last_unstale":"2024-05-14T16:54:41.207061+0000","last_undegraded":"2024-05-14T16:54:41.207061+0000","last_fullsized":"2024-05-14T16:54:41.207061+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:29:20.942783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.192617+0000","last_change":"2024-05-14T16:51:32.292107+0000","last_active":"2024-05-14T16:54:41.192617+0000","last_peered":"2024-05-14T16:54:41.192617+0000","last_clean":"2024-05-14T16:54:41.192617+0000","last_became_active":"2024-05-14T16:51:32.291729+0000","last_became_peered":"2024-05-14T16:51:32.291729+0000","last_unstale":"2024-05-14T16:54:41.192617+0000","last_undegraded":"2024-05-14T16:54:41.192617+0000","last_fullsized":"2024-05-14T16:54:41.192617+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:30:15.440599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206802+0000","last_change":"2024-05-14T16:51:29.274862+0000","last_active":"2024-05-14T16:54:41.206802+0000","last_peered":"2024-05-14T16:54:41.206802+0000","last_clean":"2024-05-14T16:54:41.206802+0000","last_became_active":"2024-05-14T16:51:29.274527+0000","last_became_peered":"2024-05-14T16:51:29.274527+0000","last_unstale":"2024-05-14T16:54:41.206802+0000","last_undegraded":"2024-05-14T16:54:41.206802+0000","last_fullsized":"2024-05-14T16:54:41.206802+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:43:15.634585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207361+0000","last_change":"2024-05-14T16:51:32.306158+0000","last_active":"2024-05-14T16:54:41.207361+0000","last_peered":"2024-05-14T16:54:41.207361+0000","last_clean":"2024-05-14T16:54:41.207361+0000","last_became_active":"2024-05-14T16:51:32.305834+0000","last_became_peered":"2024-05-14T16:51:32.305834+0000","last_unstale":"2024-05-14T16:54:41.207361+0000","last_undegraded":"2024-05-14T16:54:41.207361+0000","last_fullsized":"2024-05-14T16:54:41.207361+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:31:47.645862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'6","reported_seq":209,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.462297+0000","last_change":"2024-05-14T16:51:29.270735+0000","last_active":"2024-05-14T16:54:41.462297+0000","last_peered":"2024-05-14T16:54:41.462297+0000","last_clean":"2024-05-14T16:54:41.462297+0000","last_became_active":"2024-05-14T16:51:29.270362+0000","last_became_peered":"2024-05-14T16:51:29.270362+0000","last_unstale":"2024-05-14T16:54:41.462297+0000","last_undegraded":"2024-05-14T16:54:41.462297+0000","last_fullsized":"2024-05-14T16:54:41.462297+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:40:12.288655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1678,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":175,"num_read_kb":4,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206222+0000","last_change":"2024-05-14T16:51:32.305007+0000","last_active":"2024-05-14T16:54:41.206222+0000","last_peered":"2024-05-14T16:54:41.206222+0000","last_clean":"2024-05-14T16:54:41.206222+0000","last_became_active":"2024-05-14T16:51:32.303927+0000","last_became_peered":"2024-05-14T16:51:32.303927+0000","last_unstale":"2024-05-14T16:54:41.206222+0000","last_undegraded":"2024-05-14T16:54:41.206222+0000","last_fullsized":"2024-05-14T16:54:41.206222+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:42:28.018492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206550+0000","last_change":"2024-05-14T16:51:29.345902+0000","last_active":"2024-05-14T16:54:41.206550+0000","last_peered":"2024-05-14T16:54:41.206550+0000","last_clean":"2024-05-14T16:54:41.206550+0000","last_became_active":"2024-05-14T16:51:29.344904+0000","last_became_peered":"2024-05-14T16:51:29.344904+0000","last_unstale":"2024-05-14T16:54:41.206550+0000","last_undegraded":"2024-05-14T16:54:41.206550+0000","last_fullsized":"2024-05-14T16:54:41.206550+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:56:22.209067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206556+0000","last_change":"2024-05-14T16:51:32.306459+0000","last_active":"2024-05-14T16:54:41.206556+0000","last_peered":"2024-05-14T16:54:41.206556+0000","last_clean":"2024-05-14T16:54:41.206556+0000","last_became_active":"2024-05-14T16:51:32.304897+0000","last_became_peered":"2024-05-14T16:51:32.304897+0000","last_unstale":"2024-05-14T16:54:41.206556+0000","last_undegraded":"2024-05-14T16:54:41.206556+0000","last_fullsized":"2024-05-14T16:54:41.206556+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:09:51.216631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'4","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206585+0000","last_change":"2024-05-14T16:51:29.275219+0000","last_active":"2024-05-14T16:54:41.206585+0000","last_peered":"2024-05-14T16:54:41.206585+0000","last_clean":"2024-05-14T16:54:41.206585+0000","last_became_active":"2024-05-14T16:51:29.274816+0000","last_became_peered":"2024-05-14T16:51:29.274816+0000","last_unstale":"2024-05-14T16:54:41.206585+0000","last_undegraded":"2024-05-14T16:54:41.206585+0000","last_fullsized":"2024-05-14T16:54:41.206585+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:55:49.667387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.196946+0000","last_change":"2024-05-14T16:51:32.299284+0000","last_active":"2024-05-14T16:54:41.196946+0000","last_peered":"2024-05-14T16:54:41.196946+0000","last_clean":"2024-05-14T16:54:41.196946+0000","last_became_active":"2024-05-14T16:51:32.298933+0000","last_became_peered":"2024-05-14T16:51:32.298933+0000","last_unstale":"2024-05-14T16:54:41.196946+0000","last_undegraded":"2024-05-14T16:54:41.196946+0000","last_fullsized":"2024-05-14T16:54:41.196946+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:35:42.266665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.208227+0000","last_change":"2024-05-14T16:51:29.345454+0000","last_active":"2024-05-14T16:54:41.208227+0000","last_peered":"2024-05-14T16:54:41.208227+0000","last_clean":"2024-05-14T16:54:41.208227+0000","last_became_active":"2024-05-14T16:51:29.345074+0000","last_became_peered":"2024-05-14T16:51:29.345074+0000","last_unstale":"2024-05-14T16:54:41.208227+0000","last_undegraded":"2024-05-14T16:54:41.208227+0000","last_fullsized":"2024-05-14T16:54:41.208227+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:56:11.661144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.194447+0000","last_change":"2024-05-14T16:51:32.288804+0000","last_active":"2024-05-14T16:54:41.194447+0000","last_peered":"2024-05-14T16:54:41.194447+0000","last_clean":"2024-05-14T16:54:41.194447+0000","last_became_active":"2024-05-14T16:51:32.288441+0000","last_became_peered":"2024-05-14T16:51:32.288441+0000","last_unstale":"2024-05-14T16:54:41.194447+0000","last_undegraded":"2024-05-14T16:54:41.194447+0000","last_fullsized":"2024-05-14T16:54:41.194447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:28:31.720342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206860+0000","last_change":"2024-05-14T16:51:29.272032+0000","last_active":"2024-05-14T16:54:41.206860+0000","last_peered":"2024-05-14T16:54:41.206860+0000","last_clean":"2024-05-14T16:54:41.206860+0000","last_became_active":"2024-05-14T16:51:29.271677+0000","last_became_peered":"2024-05-14T16:51:29.271677+0000","last_unstale":"2024-05-14T16:54:41.206860+0000","last_undegraded":"2024-05-14T16:54:41.206860+0000","last_fullsized":"2024-05-14T16:54:41.206860+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:39:41.377591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.194175+0000","last_change":"2024-05-14T16:51:32.295159+0000","last_active":"2024-05-14T16:54:41.194175+0000","last_peered":"2024-05-14T16:54:41.194175+0000","last_clean":"2024-05-14T16:54:41.194175+0000","last_became_active":"2024-05-14T16:51:32.294812+0000","last_became_peered":"2024-05-14T16:51:32.294812+0000","last_unstale":"2024-05-14T16:54:41.194175+0000","last_undegraded":"2024-05-14T16:54:41.194175+0000","last_fullsized":"2024-05-14T16:54:41.194175+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:08:22.411800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206770+0000","last_change":"2024-05-14T16:51:32.306582+0000","last_active":"2024-05-14T16:54:41.206770+0000","last_peered":"2024-05-14T16:54:41.206770+0000","last_clean":"2024-05-14T16:54:41.206770+0000","last_became_active":"2024-05-14T16:51:32.306209+0000","last_became_peered":"2024-05-14T16:51:32.306209+0000","last_unstale":"2024-05-14T16:54:41.206770+0000","last_undegraded":"2024-05-14T16:54:41.206770+0000","last_fullsized":"2024-05-14T16:54:41.206770+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:30:46.264097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":205,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.192557+0000","last_change":"2024-05-14T16:51:29.345057+0000","last_active":"2024-05-14T16:54:41.192557+0000","last_peered":"2024-05-14T16:54:41.192557+0000","last_clean":"2024-05-14T16:54:41.192557+0000","last_became_active":"2024-05-14T16:51:29.344658+0000","last_became_peered":"2024-05-14T16:51:29.344658+0000","last_unstale":"2024-05-14T16:54:41.192557+0000","last_undegraded":"2024-05-14T16:54:41.192557+0000","last_fullsized":"2024-05-14T16:54:41.192557+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:43:53.007543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":171,"num_read_kb":171,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207516+0000","last_change":"2024-05-14T16:51:32.298958+0000","last_active":"2024-05-14T16:54:41.207516+0000","last_peered":"2024-05-14T16:54:41.207516+0000","last_clean":"2024-05-14T16:54:41.207516+0000","last_became_active":"2024-05-14T16:51:32.298420+0000","last_became_peered":"2024-05-14T16:51:32.298420+0000","last_unstale":"2024-05-14T16:54:41.207516+0000","last_undegraded":"2024-05-14T16:54:41.207516+0000","last_fullsized":"2024-05-14T16:54:41.207516+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:15:21.262026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206858+0000","last_change":"2024-05-14T16:51:29.272082+0000","last_active":"2024-05-14T16:54:41.206858+0000","last_peered":"2024-05-14T16:54:41.206858+0000","last_clean":"2024-05-14T16:54:41.206858+0000","last_became_active":"2024-05-14T16:51:29.271711+0000","last_became_peered":"2024-05-14T16:51:29.271711+0000","last_unstale":"2024-05-14T16:54:41.206858+0000","last_undegraded":"2024-05-14T16:54:41.206858+0000","last_fullsized":"2024-05-14T16:54:41.206858+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:27:44.320423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":102,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206476+0000","last_change":"2024-05-14T16:50:05.320136+0000","last_active":"2024-05-14T16:54:41.206476+0000","last_peered":"2024-05-14T16:54:41.206476+0000","last_clean":"2024-05-14T16:54:41.206476+0000","last_became_active":"2024-05-14T16:50:05.315016+0000","last_became_peered":"2024-05-14T16:50:05.315016+0000","last_unstale":"2024-05-14T16:54:41.206476+0000","last_undegraded":"2024-05-14T16:54:41.206476+0000","last_fullsized":"2024-05-14T16:54:41.206476+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:38.138211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:38.138211+0000","last_clean_scrub_stamp":"2024-05-14T16:46:38.138211+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:19:09.012578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.192492+0000","last_change":"2024-05-14T16:51:32.295995+0000","last_active":"2024-05-14T16:54:41.192492+0000","last_peered":"2024-05-14T16:54:41.192492+0000","last_clean":"2024-05-14T16:54:41.192492+0000","last_became_active":"2024-05-14T16:51:32.295640+0000","last_became_peered":"2024-05-14T16:51:32.295640+0000","last_unstale":"2024-05-14T16:54:41.192492+0000","last_undegraded":"2024-05-14T16:54:41.192492+0000","last_fullsized":"2024-05-14T16:54:41.192492+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:42:26.774916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206592+0000","last_change":"2024-05-14T16:51:29.274892+0000","last_active":"2024-05-14T16:54:41.206592+0000","last_peered":"2024-05-14T16:54:41.206592+0000","last_clean":"2024-05-14T16:54:41.206592+0000","last_became_active":"2024-05-14T16:51:29.274386+0000","last_became_peered":"2024-05-14T16:51:29.274386+0000","last_unstale":"2024-05-14T16:54:41.206592+0000","last_undegraded":"2024-05-14T16:54:41.206592+0000","last_fullsized":"2024-05-14T16:54:41.206592+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:16:54.899320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207219+0000","last_change":"2024-05-14T16:51:29.345672+0000","last_active":"2024-05-14T16:54:41.207219+0000","last_peered":"2024-05-14T16:54:41.207219+0000","last_clean":"2024-05-14T16:54:41.207219+0000","last_became_active":"2024-05-14T16:51:29.344948+0000","last_became_peered":"2024-05-14T16:51:29.344948+0000","last_unstale":"2024-05-14T16:54:41.207219+0000","last_undegraded":"2024-05-14T16:54:41.207219+0000","last_fullsized":"2024-05-14T16:54:41.207219+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:19:26.577799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.192847+0000","last_change":"2024-05-14T16:51:32.297360+0000","last_active":"2024-05-14T16:54:41.192847+0000","last_peered":"2024-05-14T16:54:41.192847+0000","last_clean":"2024-05-14T16:54:41.192847+0000","last_became_active":"2024-05-14T16:51:32.297025+0000","last_became_peered":"2024-05-14T16:51:32.297025+0000","last_unstale":"2024-05-14T16:54:41.192847+0000","last_undegraded":"2024-05-14T16:54:41.192847+0000","last_fullsized":"2024-05-14T16:54:41.192847+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:05:39.705860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'46","reported_seq":110,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.417853+0000","last_change":"2024-05-14T16:52:13.224328+0000","last_active":"2024-05-14T16:54:41.417853+0000","last_peered":"2024-05-14T16:54:41.417853+0000","last_clean":"2024-05-14T16:54:41.417853+0000","last_became_active":"2024-05-14T16:52:13.220271+0000","last_became_peered":"2024-05-14T16:52:13.220271+0000","last_unstale":"2024-05-14T16:54:41.417853+0000","last_undegraded":"2024-05-14T16:54:41.417853+0000","last_fullsized":"2024-05-14T16:54:41.417853+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:06:31.955274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207534+0000","last_change":"2024-05-14T16:51:32.292112+0000","last_active":"2024-05-14T16:54:41.207534+0000","last_peered":"2024-05-14T16:54:41.207534+0000","last_clean":"2024-05-14T16:54:41.207534+0000","last_became_active":"2024-05-14T16:51:32.291688+0000","last_became_peered":"2024-05-14T16:51:32.291688+0000","last_unstale":"2024-05-14T16:54:41.207534+0000","last_undegraded":"2024-05-14T16:54:41.207534+0000","last_fullsized":"2024-05-14T16:54:41.207534+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:13:06.728608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207174+0000","last_change":"2024-05-14T16:51:29.412805+0000","last_active":"2024-05-14T16:54:41.207174+0000","last_peered":"2024-05-14T16:54:41.207174+0000","last_clean":"2024-05-14T16:54:41.207174+0000","last_became_active":"2024-05-14T16:51:29.412394+0000","last_became_peered":"2024-05-14T16:51:29.412394+0000","last_unstale":"2024-05-14T16:54:41.207174+0000","last_undegraded":"2024-05-14T16:54:41.207174+0000","last_fullsized":"2024-05-14T16:54:41.207174+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:49:48.721104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206672+0000","last_change":"2024-05-14T16:52:13.218227+0000","last_active":"2024-05-14T16:54:41.206672+0000","last_peered":"2024-05-14T16:54:41.206672+0000","last_clean":"2024-05-14T16:54:41.206672+0000","last_became_active":"2024-05-14T16:52:13.217787+0000","last_became_peered":"2024-05-14T16:52:13.217787+0000","last_unstale":"2024-05-14T16:54:41.206672+0000","last_undegraded":"2024-05-14T16:54:41.206672+0000","last_fullsized":"2024-05-14T16:54:41.206672+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:52:33.348804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206178+0000","last_change":"2024-05-14T16:51:29.345427+0000","last_active":"2024-05-14T16:54:41.206178+0000","last_peered":"2024-05-14T16:54:41.206178+0000","last_clean":"2024-05-14T16:54:41.206178+0000","last_became_active":"2024-05-14T16:51:29.345049+0000","last_became_peered":"2024-05-14T16:51:29.345049+0000","last_unstale":"2024-05-14T16:54:41.206178+0000","last_undegraded":"2024-05-14T16:54:41.206178+0000","last_fullsized":"2024-05-14T16:54:41.206178+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:06:19.799660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206337+0000","last_change":"2024-05-14T16:51:32.298757+0000","last_active":"2024-05-14T16:54:41.206337+0000","last_peered":"2024-05-14T16:54:41.206337+0000","last_clean":"2024-05-14T16:54:41.206337+0000","last_became_active":"2024-05-14T16:51:32.297193+0000","last_became_peered":"2024-05-14T16:51:32.297193+0000","last_unstale":"2024-05-14T16:54:41.206337+0000","last_undegraded":"2024-05-14T16:54:41.206337+0000","last_fullsized":"2024-05-14T16:54:41.206337+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:10:19.672230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207012+0000","last_change":"2024-05-14T16:51:29.270545+0000","last_active":"2024-05-14T16:54:41.207012+0000","last_peered":"2024-05-14T16:54:41.207012+0000","last_clean":"2024-05-14T16:54:41.207012+0000","last_became_active":"2024-05-14T16:51:29.270182+0000","last_became_peered":"2024-05-14T16:51:29.270182+0000","last_unstale":"2024-05-14T16:54:41.207012+0000","last_undegraded":"2024-05-14T16:54:41.207012+0000","last_fullsized":"2024-05-14T16:54:41.207012+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:19:49.977916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.196769+0000","last_change":"2024-05-14T16:51:32.294767+0000","last_active":"2024-05-14T16:54:41.196769+0000","last_peered":"2024-05-14T16:54:41.196769+0000","last_clean":"2024-05-14T16:54:41.196769+0000","last_became_active":"2024-05-14T16:51:32.293947+0000","last_became_peered":"2024-05-14T16:51:32.293947+0000","last_unstale":"2024-05-14T16:54:41.196769+0000","last_undegraded":"2024-05-14T16:54:41.196769+0000","last_fullsized":"2024-05-14T16:54:41.196769+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:30:21.731492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.196766+0000","last_change":"2024-05-14T16:52:13.226509+0000","last_active":"2024-05-14T16:54:41.196766+0000","last_peered":"2024-05-14T16:54:41.196766+0000","last_clean":"2024-05-14T16:54:41.196766+0000","last_became_active":"2024-05-14T16:52:13.220923+0000","last_became_peered":"2024-05-14T16:52:13.220923+0000","last_unstale":"2024-05-14T16:54:41.196766+0000","last_undegraded":"2024-05-14T16:54:41.196766+0000","last_fullsized":"2024-05-14T16:54:41.196766+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:25:52.655481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207841+0000","last_change":"2024-05-14T16:51:32.288810+0000","last_active":"2024-05-14T16:54:41.207841+0000","last_peered":"2024-05-14T16:54:41.207841+0000","last_clean":"2024-05-14T16:54:41.207841+0000","last_became_active":"2024-05-14T16:51:32.288492+0000","last_became_peered":"2024-05-14T16:51:32.288492+0000","last_unstale":"2024-05-14T16:54:41.207841+0000","last_undegraded":"2024-05-14T16:54:41.207841+0000","last_fullsized":"2024-05-14T16:54:41.207841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:44:53.393371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206506+0000","last_change":"2024-05-14T16:51:29.281181+0000","last_active":"2024-05-14T16:54:41.206506+0000","last_peered":"2024-05-14T16:54:41.206506+0000","last_clean":"2024-05-14T16:54:41.206506+0000","last_became_active":"2024-05-14T16:51:29.279965+0000","last_became_peered":"2024-05-14T16:51:29.279965+0000","last_unstale":"2024-05-14T16:54:41.206506+0000","last_undegraded":"2024-05-14T16:54:41.206506+0000","last_fullsized":"2024-05-14T16:54:41.206506+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:41:00.735291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206981+0000","last_change":"2024-05-14T16:51:32.296971+0000","last_active":"2024-05-14T16:54:41.206981+0000","last_peered":"2024-05-14T16:54:41.206981+0000","last_clean":"2024-05-14T16:54:41.206981+0000","last_became_active":"2024-05-14T16:51:32.296600+0000","last_became_peered":"2024-05-14T16:51:32.296600+0000","last_unstale":"2024-05-14T16:54:41.206981+0000","last_undegraded":"2024-05-14T16:54:41.206981+0000","last_fullsized":"2024-05-14T16:54:41.206981+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:13:14.354127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207310+0000","last_change":"2024-05-14T16:51:29.412682+0000","last_active":"2024-05-14T16:54:41.207310+0000","last_peered":"2024-05-14T16:54:41.207310+0000","last_clean":"2024-05-14T16:54:41.207310+0000","last_became_active":"2024-05-14T16:51:29.412186+0000","last_became_peered":"2024-05-14T16:51:29.412186+0000","last_unstale":"2024-05-14T16:54:41.207310+0000","last_undegraded":"2024-05-14T16:54:41.207310+0000","last_fullsized":"2024-05-14T16:54:41.207310+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:56:24.570276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207683+0000","last_change":"2024-05-14T16:51:32.300626+0000","last_active":"2024-05-14T16:54:41.207683+0000","last_peered":"2024-05-14T16:54:41.207683+0000","last_clean":"2024-05-14T16:54:41.207683+0000","last_became_active":"2024-05-14T16:51:32.300278+0000","last_became_peered":"2024-05-14T16:51:32.300278+0000","last_unstale":"2024-05-14T16:54:41.207683+0000","last_undegraded":"2024-05-14T16:54:41.207683+0000","last_fullsized":"2024-05-14T16:54:41.207683+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:59:14.196315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'39","reported_seq":76,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.415378+0000","last_change":"2024-05-14T16:51:29.270826+0000","last_active":"2024-05-14T16:54:41.415378+0000","last_peered":"2024-05-14T16:54:41.415378+0000","last_clean":"2024-05-14T16:54:41.415378+0000","last_became_active":"2024-05-14T16:51:29.270266+0000","last_became_peered":"2024-05-14T16:51:29.270266+0000","last_unstale":"2024-05-14T16:54:41.415378+0000","last_undegraded":"2024-05-14T16:54:41.415378+0000","last_fullsized":"2024-05-14T16:54:41.415378+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:00:55.349278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207426+0000","last_change":"2024-05-14T16:52:13.223994+0000","last_active":"2024-05-14T16:54:41.207426+0000","last_peered":"2024-05-14T16:54:41.207426+0000","last_clean":"2024-05-14T16:54:41.207426+0000","last_became_active":"2024-05-14T16:52:13.221071+0000","last_became_peered":"2024-05-14T16:52:13.221071+0000","last_unstale":"2024-05-14T16:54:41.207426+0000","last_undegraded":"2024-05-14T16:54:41.207426+0000","last_fullsized":"2024-05-14T16:54:41.207426+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:29:58.775171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.192788+0000","last_change":"2024-05-14T16:51:29.276705+0000","last_active":"2024-05-14T16:54:41.192788+0000","last_peered":"2024-05-14T16:54:41.192788+0000","last_clean":"2024-05-14T16:54:41.192788+0000","last_became_active":"2024-05-14T16:51:29.276347+0000","last_became_peered":"2024-05-14T16:51:29.276347+0000","last_unstale":"2024-05-14T16:54:41.192788+0000","last_undegraded":"2024-05-14T16:54:41.192788+0000","last_fullsized":"2024-05-14T16:54:41.192788+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:32:56.064427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207670+0000","last_change":"2024-05-14T16:51:32.301617+0000","last_active":"2024-05-14T16:54:41.207670+0000","last_peered":"2024-05-14T16:54:41.207670+0000","last_clean":"2024-05-14T16:54:41.207670+0000","last_became_active":"2024-05-14T16:51:32.301288+0000","last_became_peered":"2024-05-14T16:51:32.301288+0000","last_unstale":"2024-05-14T16:54:41.207670+0000","last_undegraded":"2024-05-14T16:54:41.207670+0000","last_fullsized":"2024-05-14T16:54:41.207670+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:19:24.253633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.197028+0000","last_change":"2024-05-14T16:52:13.221694+0000","last_active":"2024-05-14T16:54:41.197028+0000","last_peered":"2024-05-14T16:54:41.197028+0000","last_clean":"2024-05-14T16:54:41.197028+0000","last_became_active":"2024-05-14T16:52:13.221264+0000","last_became_peered":"2024-05-14T16:52:13.221264+0000","last_unstale":"2024-05-14T16:54:41.197028+0000","last_undegraded":"2024-05-14T16:54:41.197028+0000","last_fullsized":"2024-05-14T16:54:41.197028+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:22:52.738772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206531+0000","last_change":"2024-05-14T16:51:32.301789+0000","last_active":"2024-05-14T16:54:41.206531+0000","last_peered":"2024-05-14T16:54:41.206531+0000","last_clean":"2024-05-14T16:54:41.206531+0000","last_became_active":"2024-05-14T16:51:32.301350+0000","last_became_peered":"2024-05-14T16:51:32.301350+0000","last_unstale":"2024-05-14T16:54:41.206531+0000","last_undegraded":"2024-05-14T16:54:41.206531+0000","last_fullsized":"2024-05-14T16:54:41.206531+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:46:24.896855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206730+0000","last_change":"2024-05-14T16:51:32.295978+0000","last_active":"2024-05-14T16:54:41.206730+0000","last_peered":"2024-05-14T16:54:41.206730+0000","last_clean":"2024-05-14T16:54:41.206730+0000","last_became_active":"2024-05-14T16:51:32.295652+0000","last_became_peered":"2024-05-14T16:51:32.295652+0000","last_unstale":"2024-05-14T16:54:41.206730+0000","last_undegraded":"2024-05-14T16:54:41.206730+0000","last_fullsized":"2024-05-14T16:54:41.206730+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:32:17.743734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206139+0000","last_change":"2024-05-14T16:51:32.305568+0000","last_active":"2024-05-14T16:54:41.206139+0000","last_peered":"2024-05-14T16:54:41.206139+0000","last_clean":"2024-05-14T16:54:41.206139+0000","last_became_active":"2024-05-14T16:51:32.304580+0000","last_became_peered":"2024-05-14T16:51:32.304580+0000","last_unstale":"2024-05-14T16:54:41.206139+0000","last_undegraded":"2024-05-14T16:54:41.206139+0000","last_fullsized":"2024-05-14T16:54:41.206139+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:32:44.465054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.192870+0000","last_change":"2024-05-14T16:51:32.295029+0000","last_active":"2024-05-14T16:54:41.192870+0000","last_peered":"2024-05-14T16:54:41.192870+0000","last_clean":"2024-05-14T16:54:41.192870+0000","last_became_active":"2024-05-14T16:51:32.294621+0000","last_became_peered":"2024-05-14T16:51:32.294621+0000","last_unstale":"2024-05-14T16:54:41.192870+0000","last_undegraded":"2024-05-14T16:54:41.192870+0000","last_fullsized":"2024-05-14T16:54:41.192870+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:52:05.390141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.196972+0000","last_change":"2024-05-14T16:51:32.297671+0000","last_active":"2024-05-14T16:54:41.196972+0000","last_peered":"2024-05-14T16:54:41.196972+0000","last_clean":"2024-05-14T16:54:41.196972+0000","last_became_active":"2024-05-14T16:51:32.296210+0000","last_became_peered":"2024-05-14T16:51:32.296210+0000","last_unstale":"2024-05-14T16:54:41.196972+0000","last_undegraded":"2024-05-14T16:54:41.196972+0000","last_fullsized":"2024-05-14T16:54:41.196972+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:55:04.789021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207532+0000","last_change":"2024-05-14T16:51:32.297404+0000","last_active":"2024-05-14T16:54:41.207532+0000","last_peered":"2024-05-14T16:54:41.207532+0000","last_clean":"2024-05-14T16:54:41.207532+0000","last_became_active":"2024-05-14T16:51:32.296856+0000","last_became_peered":"2024-05-14T16:51:32.296856+0000","last_unstale":"2024-05-14T16:54:41.207532+0000","last_undegraded":"2024-05-14T16:54:41.207532+0000","last_fullsized":"2024-05-14T16:54:41.207532+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:25:29.992875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207395+0000","last_change":"2024-05-14T16:52:13.221379+0000","last_active":"2024-05-14T16:54:41.207395+0000","last_peered":"2024-05-14T16:54:41.207395+0000","last_clean":"2024-05-14T16:54:41.207395+0000","last_became_active":"2024-05-14T16:52:13.220973+0000","last_became_peered":"2024-05-14T16:52:13.220973+0000","last_unstale":"2024-05-14T16:54:41.207395+0000","last_undegraded":"2024-05-14T16:54:41.207395+0000","last_fullsized":"2024-05-14T16:54:41.207395+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:55:08.212344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206477+0000","last_change":"2024-05-14T16:51:32.303570+0000","last_active":"2024-05-14T16:54:41.206477+0000","last_peered":"2024-05-14T16:54:41.206477+0000","last_clean":"2024-05-14T16:54:41.206477+0000","last_became_active":"2024-05-14T16:51:32.303234+0000","last_became_peered":"2024-05-14T16:51:32.303234+0000","last_unstale":"2024-05-14T16:54:41.206477+0000","last_undegraded":"2024-05-14T16:54:41.206477+0000","last_fullsized":"2024-05-14T16:54:41.206477+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:27:57.960120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206835+0000","last_change":"2024-05-14T16:51:32.298028+0000","last_active":"2024-05-14T16:54:41.206835+0000","last_peered":"2024-05-14T16:54:41.206835+0000","last_clean":"2024-05-14T16:54:41.206835+0000","last_became_active":"2024-05-14T16:51:32.295104+0000","last_became_peered":"2024-05-14T16:51:32.295104+0000","last_unstale":"2024-05-14T16:54:41.206835+0000","last_undegraded":"2024-05-14T16:54:41.206835+0000","last_fullsized":"2024-05-14T16:54:41.206835+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:58:08.441590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206745+0000","last_change":"2024-05-14T16:51:32.304458+0000","last_active":"2024-05-14T16:54:41.206745+0000","last_peered":"2024-05-14T16:54:41.206745+0000","last_clean":"2024-05-14T16:54:41.206745+0000","last_became_active":"2024-05-14T16:51:32.302952+0000","last_became_peered":"2024-05-14T16:51:32.302952+0000","last_unstale":"2024-05-14T16:54:41.206745+0000","last_undegraded":"2024-05-14T16:54:41.206745+0000","last_fullsized":"2024-05-14T16:54:41.206745+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:40:14.220075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.192762+0000","last_change":"2024-05-14T16:51:32.292334+0000","last_active":"2024-05-14T16:54:41.192762+0000","last_peered":"2024-05-14T16:54:41.192762+0000","last_clean":"2024-05-14T16:54:41.192762+0000","last_became_active":"2024-05-14T16:51:32.291769+0000","last_became_peered":"2024-05-14T16:51:32.291769+0000","last_unstale":"2024-05-14T16:54:41.192762+0000","last_undegraded":"2024-05-14T16:54:41.192762+0000","last_fullsized":"2024-05-14T16:54:41.192762+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:55:25.694827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.196919+0000","last_change":"2024-05-14T16:51:32.300044+0000","last_active":"2024-05-14T16:54:41.196919+0000","last_peered":"2024-05-14T16:54:41.196919+0000","last_clean":"2024-05-14T16:54:41.196919+0000","last_became_active":"2024-05-14T16:51:32.299630+0000","last_became_peered":"2024-05-14T16:51:32.299630+0000","last_unstale":"2024-05-14T16:54:41.196919+0000","last_undegraded":"2024-05-14T16:54:41.196919+0000","last_fullsized":"2024-05-14T16:54:41.196919+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:50:40.516973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108486,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597333,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1151,"num_read_kb":683,"num_write":296,"num_write_kb":1440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":1791999,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":231,"ondisk_log_size":231,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710773,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":901120,"data_stored":827273,"data_compressed":22065,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907002,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1056768,"data_stored":1225122,"data_compressed":45237,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070527,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":827626,"data_compressed":22065,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266757,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":892928,"data_stored":434213,"data_compressed":2919,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462985,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":942080,"data_stored":1027765,"data_compressed":39081,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626511,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36200,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708952,"statfs":{"total":95995035648,"available":95957966848,"internally_reserved":0,"allocated":880640,"data_stored":433682,"data_compressed":2919,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574952,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":500369,"data_compressed":2919,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738476,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36220,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708932,"statfs":{"total":95995035648,"available":95957946368,"internally_reserved":0,"allocated":901120,"data_stored":434953,"data_compressed":2919,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411428,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":970752,"data_stored":501469,"data_compressed":2919,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182494,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":937984,"data_stored":958692,"data_compressed":28221,"data_compressed_allocated":36864,"data_compressed_original":524288,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986266,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":884736,"data_stored":434427,"data_compressed":2919,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822741,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":1157959,"data_compressed":45237,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":133999,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":201162,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":393798,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T16:54:46.679 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-14T16:54:46.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:46 smithi151 ceph-mon[36066]: Adjusting osd_memory_target on smithi123 to 176.2M 2024-05-14T16:54:46.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:46 smithi151 ceph-mon[36066]: Unable to set osd_memory_target on smithi123 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-05-14T16:54:46.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:46 smithi151 ceph-mon[36066]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:54:46.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:46 smithi151 ceph-mon[36066]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-05-14T16:54:46.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:46 smithi123 ceph-mon[35971]: Adjusting osd_memory_target on smithi123 to 176.2M 2024-05-14T16:54:46.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:46 smithi123 ceph-mon[35971]: Unable to set osd_memory_target on smithi123 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-05-14T16:54:46.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:46 smithi123 ceph-mon[35971]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:54:46.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:46 smithi123 ceph-mon[35971]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-05-14T16:54:46.944 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:54:47.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:47 smithi002 ceph-mon[31538]: from='client.15225 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:54:47.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:47 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:47.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:47 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:47.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:47 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:54:47.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:47 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:54:47.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:47 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:54:47.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:47 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:54:47.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:47 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:47.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:47 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:54:47.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:47 smithi151 ceph-mon[36066]: from='client.15225 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:54:47.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:47 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:47.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:47 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:47.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:47 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:54:47.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:47 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:54:47.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:47 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:54:47.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:47 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:54:47.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:47 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:47.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:47 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:54:47.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:47 smithi123 ceph-mon[35971]: from='client.15225 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:54:47.911 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:47 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:47.911 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:47 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:47.911 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:47 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:54:47.911 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:47 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:54:47.911 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:47 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:54:47.911 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:47 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T16:54:47.911 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:47 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:47.911 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:47 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:54:48.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:48 smithi002 ceph-mon[31538]: Adjusting osd_memory_target on smithi002 to 177.2M 2024-05-14T16:54:48.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:48 smithi002 ceph-mon[31538]: Unable to set osd_memory_target on smithi002 to 185879142: error parsing value: Value '185879142' is below minimum 939524096 2024-05-14T16:54:48.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:48 smithi002 ceph-mon[31538]: Updating smithi002:/etc/ceph/ceph.conf 2024-05-14T16:54:48.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:48 smithi002 ceph-mon[31538]: Updating smithi123:/etc/ceph/ceph.conf 2024-05-14T16:54:48.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:48 smithi002 ceph-mon[31538]: Updating smithi151:/etc/ceph/ceph.conf 2024-05-14T16:54:48.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:48 smithi002 ceph-mon[31538]: Updating smithi123:/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:54:48.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:48 smithi002 ceph-mon[31538]: Updating smithi151:/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:54:48.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:48 smithi002 ceph-mon[31538]: Updating smithi002:/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:54:48.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:48 smithi002 ceph-mon[31538]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:54:48.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:48 smithi002 ceph-mon[31538]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:54:48.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:48 smithi002 ceph-mon[31538]: Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:54:48.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:48 smithi002 ceph-mon[31538]: Updating smithi002:/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:54:48.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:48 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:48.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:48 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:48.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:48 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:48.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:48 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:48.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:48 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:48.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:48 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:48.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:48 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:48.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:48 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:54:48.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.771+0000 7f3a88eea700 1 -- 172.21.15.2:0/850162450 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a841024f0 msgr2=0x7f3a841103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:48.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.771+0000 7f3a88eea700 1 --2- 172.21.15.2:0/850162450 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a841024f0 0x7f3a841103d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c00b0d0 tx=0x7f3a7c00b3e0 comp rx=0 tx=0).stop 2024-05-14T16:54:48.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.771+0000 7f3a88eea700 1 -- 172.21.15.2:0/850162450 shutdown_connections 2024-05-14T16:54:48.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.771+0000 7f3a88eea700 1 --2- 172.21.15.2:0/850162450 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a841024f0 0x7f3a841103d0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:48.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.771+0000 7f3a88eea700 1 --2- 172.21.15.2:0/850162450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a84109520 0x7f3a84109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:48.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.771+0000 7f3a88eea700 1 --2- 172.21.15.2:0/850162450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a84101b30 0x7f3a84101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:48.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.771+0000 7f3a88eea700 1 -- 172.21.15.2:0/850162450 >> 172.21.15.2:0/850162450 conn(0x7f3a84078ab0 msgr2=0x7f3a840ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:48.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.771+0000 7f3a88eea700 1 -- 172.21.15.2:0/850162450 shutdown_connections 2024-05-14T16:54:48.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.771+0000 7f3a88eea700 1 -- 172.21.15.2:0/850162450 wait complete. 2024-05-14T16:54:48.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.772+0000 7f3a88eea700 1 Processor -- start 2024-05-14T16:54:48.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.772+0000 7f3a88eea700 1 -- start start 2024-05-14T16:54:48.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.772+0000 7f3a88eea700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a84101b30 0x7f3a8419f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:48.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.772+0000 7f3a88eea700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a841024f0 0x7f3a8419fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:48.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.772+0000 7f3a88eea700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a84109520 0x7f3a841999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:48.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.772+0000 7f3a88eea700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3a84113760 con 0x7f3a84101b30 2024-05-14T16:54:48.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.772+0000 7f3a88eea700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3a841135e0 con 0x7f3a841024f0 2024-05-14T16:54:48.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.772+0000 7f3a88eea700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3a841138e0 con 0x7f3a84109520 2024-05-14T16:54:48.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.772+0000 7f3a8259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a84101b30 0x7f3a8419f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:48.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.772+0000 7f3a8259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a84101b30 0x7f3a8419f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57092/0 (socket says 172.21.15.2:57092) 2024-05-14T16:54:48.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.773+0000 7f3a8259c700 1 -- 172.21.15.2:0/1467918589 learned_addr learned my addr 172.21.15.2:0/1467918589 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:48.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.773+0000 7f3a81d9b700 1 --2- 172.21.15.2:0/1467918589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a841024f0 0x7f3a8419fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:48.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.773+0000 7f3a82d9d700 1 --2- 172.21.15.2:0/1467918589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a84109520 0x7f3a841999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:48.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.773+0000 7f3a8259c700 1 -- 172.21.15.2:0/1467918589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a84109520 msgr2=0x7f3a841999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:48.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.773+0000 7f3a8259c700 1 --2- 172.21.15.2:0/1467918589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a84109520 0x7f3a841999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:48.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.773+0000 7f3a8259c700 1 -- 172.21.15.2:0/1467918589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a841024f0 msgr2=0x7f3a8419fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:48.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.773+0000 7f3a8259c700 1 --2- 172.21.15.2:0/1467918589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a841024f0 0x7f3a8419fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:48.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.773+0000 7f3a8259c700 1 -- 172.21.15.2:0/1467918589 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a8419a140 con 0x7f3a84101b30 2024-05-14T16:54:48.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.773+0000 7f3a8259c700 1 --2- 172.21.15.2:0/1467918589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a84101b30 0x7f3a8419f7d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3a7400d950 tx=0x7f3a7400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:48.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.773+0000 7f3a737fe700 1 -- 172.21.15.2:0/1467918589 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a74016070 con 0x7f3a84101b30 2024-05-14T16:54:48.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.773+0000 7f3a88eea700 1 -- 172.21.15.2:0/1467918589 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a8419a430 con 0x7f3a84101b30 2024-05-14T16:54:48.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.773+0000 7f3a737fe700 1 -- 172.21.15.2:0/1467918589 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a74009df0 con 0x7f3a84101b30 2024-05-14T16:54:48.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.773+0000 7f3a737fe700 1 -- 172.21.15.2:0/1467918589 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7400f3c0 con 0x7f3a84101b30 2024-05-14T16:54:48.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.773+0000 7f3a88eea700 1 -- 172.21.15.2:0/1467918589 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a841a66a0 con 0x7f3a84101b30 2024-05-14T16:54:48.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.774+0000 7f3a88eea700 1 -- 172.21.15.2:0/1467918589 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a84104e50 con 0x7f3a84101b30 2024-05-14T16:54:48.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.776+0000 7f3a737fe700 1 -- 172.21.15.2:0/1467918589 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 24) ==== 149827+0+0 (secure 0 0 0) 0x7f3a74019070 con 0x7f3a84101b30 2024-05-14T16:54:48.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.776+0000 7f3a737fe700 1 --2- 172.21.15.2:0/1467918589 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f3a6c0b2a80 0x7f3a6c0b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:48.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.777+0000 7f3a81d9b700 1 --2- 172.21.15.2:0/1467918589 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f3a6c0b2a80 0x7f3a6c0b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:48.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.777+0000 7f3a737fe700 1 -- 172.21.15.2:0/1467918589 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f3a740e22f0 con 0x7f3a84101b30 2024-05-14T16:54:48.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.777+0000 7f3a81d9b700 1 --2- 172.21.15.2:0/1467918589 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f3a6c0b2a80 0x7f3a6c0b4f40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3a84108c10 tx=0x7f3a780073d0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:48.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.780+0000 7f3a737fe700 1 -- 172.21.15.2:0/1467918589 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3a740a9b90 con 0x7f3a84101b30 2024-05-14T16:54:48.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:48 smithi151 ceph-mon[36066]: Adjusting osd_memory_target on smithi002 to 177.2M 2024-05-14T16:54:48.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:48 smithi151 ceph-mon[36066]: Unable to set osd_memory_target on smithi002 to 185879142: error parsing value: Value '185879142' is below minimum 939524096 2024-05-14T16:54:48.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:48 smithi151 ceph-mon[36066]: Updating smithi002:/etc/ceph/ceph.conf 2024-05-14T16:54:48.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:48 smithi151 ceph-mon[36066]: Updating smithi123:/etc/ceph/ceph.conf 2024-05-14T16:54:48.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:48 smithi151 ceph-mon[36066]: Updating smithi151:/etc/ceph/ceph.conf 2024-05-14T16:54:48.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:48 smithi151 ceph-mon[36066]: Updating smithi123:/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:54:48.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:48 smithi151 ceph-mon[36066]: Updating smithi151:/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:54:48.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:48 smithi151 ceph-mon[36066]: Updating smithi002:/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:54:48.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:48 smithi151 ceph-mon[36066]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:54:48.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:48 smithi151 ceph-mon[36066]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:54:48.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:48 smithi151 ceph-mon[36066]: Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:54:48.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:48 smithi151 ceph-mon[36066]: Updating smithi002:/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:54:48.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:48 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:48.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:48 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:48.856 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:48 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:48.856 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:48 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:48.856 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:48 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:48.856 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:48 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:48.856 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:48 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:48.856 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:48 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:54:48.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:48 smithi123 ceph-mon[35971]: Adjusting osd_memory_target on smithi002 to 177.2M 2024-05-14T16:54:48.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:48 smithi123 ceph-mon[35971]: Unable to set osd_memory_target on smithi002 to 185879142: error parsing value: Value '185879142' is below minimum 939524096 2024-05-14T16:54:48.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:48 smithi123 ceph-mon[35971]: Updating smithi002:/etc/ceph/ceph.conf 2024-05-14T16:54:48.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:48 smithi123 ceph-mon[35971]: Updating smithi123:/etc/ceph/ceph.conf 2024-05-14T16:54:48.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:48 smithi123 ceph-mon[35971]: Updating smithi151:/etc/ceph/ceph.conf 2024-05-14T16:54:48.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:48 smithi123 ceph-mon[35971]: Updating smithi123:/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:54:48.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:48 smithi123 ceph-mon[35971]: Updating smithi151:/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:54:48.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:48 smithi123 ceph-mon[35971]: Updating smithi002:/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-14T16:54:48.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:48 smithi123 ceph-mon[35971]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T16:54:48.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:48 smithi123 ceph-mon[35971]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:54:48.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:48 smithi123 ceph-mon[35971]: Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:54:48.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:48 smithi123 ceph-mon[35971]: Updating smithi002:/etc/ceph/ceph.client.admin.keyring 2024-05-14T16:54:48.911 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:48 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:48.911 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:48 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:48.911 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:48 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:48.911 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:48 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:48.911 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:48 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:48.911 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:48 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:48.911 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:48 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:48.911 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:48 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:54:48.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.989+0000 7f3a88eea700 1 -- 172.21.15.2:0/1467918589 --> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3a84063980 con 0x7f3a6c0b2a80 2024-05-14T16:54:48.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:48.997+0000 7f3a737fe700 1 -- 172.21.15.2:0/1467918589 <== mgr.15120 v2:172.21.15.2:6800/1339485805 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257993 (secure 0 0 0) 0x7f3a84063980 con 0x7f3a6c0b2a80 2024-05-14T16:54:48.997 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:54:49.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:49.000+0000 7f3a88eea700 1 -- 172.21.15.2:0/1467918589 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f3a6c0b2a80 msgr2=0x7f3a6c0b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:49.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:49.000+0000 7f3a88eea700 1 --2- 172.21.15.2:0/1467918589 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f3a6c0b2a80 0x7f3a6c0b4f40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3a84108c10 tx=0x7f3a780073d0 comp rx=0 tx=0).stop 2024-05-14T16:54:49.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:49.000+0000 7f3a88eea700 1 -- 172.21.15.2:0/1467918589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a84101b30 msgr2=0x7f3a8419f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:49.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:49.000+0000 7f3a88eea700 1 --2- 172.21.15.2:0/1467918589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a84101b30 0x7f3a8419f7d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3a7400d950 tx=0x7f3a7400dc60 comp rx=0 tx=0).stop 2024-05-14T16:54:49.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:49.000+0000 7f3a88eea700 1 -- 172.21.15.2:0/1467918589 shutdown_connections 2024-05-14T16:54:49.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:49.000+0000 7f3a88eea700 1 --2- 172.21.15.2:0/1467918589 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f3a6c0b2a80 0x7f3a6c0b4f40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:49.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:49.000+0000 7f3a88eea700 1 --2- 172.21.15.2:0/1467918589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a84109520 0x7f3a841999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:49.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:49.000+0000 7f3a88eea700 1 --2- 172.21.15.2:0/1467918589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a84101b30 0x7f3a8419f7d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:49.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:49.000+0000 7f3a88eea700 1 --2- 172.21.15.2:0/1467918589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a841024f0 0x7f3a8419fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:49.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:49.000+0000 7f3a88eea700 1 -- 172.21.15.2:0/1467918589 >> 172.21.15.2:0/1467918589 conn(0x7f3a84078ab0 msgr2=0x7f3a84100260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:49.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:49.000+0000 7f3a88eea700 1 -- 172.21.15.2:0/1467918589 shutdown_connections 2024-05-14T16:54:49.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:49.000+0000 7f3a88eea700 1 -- 172.21.15.2:0/1467918589 wait complete. 2024-05-14T16:54:49.003 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-05-14T16:54:49.601 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:49 smithi002 ceph-mon[31538]: Updating smithi123:/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T16:54:49.601 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:49 smithi002 ceph-mon[31538]: Updating smithi151:/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T16:54:49.601 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:49 smithi002 ceph-mon[31538]: Updating smithi002:/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T16:54:49.601 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:49 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:49.601 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:49 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:49.601 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:49 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:49.601 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:49 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:54:49.601 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:49 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:49.631 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-05-14T16:54:47.208643+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1187935,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1232,"num_read_kb":752,"num_write":421,"num_write_kb":3592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":2951957,"num_keys_recovered":9,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":825,"ondisk_log_size":825,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":385092,"kb_used_data":10952,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124556732,"statfs":{"total":1151940427776,"available":1151546093568,"internally_reserved":0,"allocated":11214848,"data_stored":8763550,"data_compressed":219420,"data_compressed_allocated":282624,"data_compressed_original":3416064,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.989740"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.197029+0000","last_change":"2024-05-14T16:51:32.298580+0000","last_active":"2024-05-14T16:54:41.197029+0000","last_peered":"2024-05-14T16:54:41.197029+0000","last_clean":"2024-05-14T16:54:41.197029+0000","last_became_active":"2024-05-14T16:51:32.296945+0000","last_became_peered":"2024-05-14T16:51:32.296945+0000","last_unstale":"2024-05-14T16:54:41.197029+0000","last_undegraded":"2024-05-14T16:54:41.197029+0000","last_fullsized":"2024-05-14T16:54:41.197029+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:50:17.664928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206588+0000","last_change":"2024-05-14T16:51:32.297618+0000","last_active":"2024-05-14T16:54:41.206588+0000","last_peered":"2024-05-14T16:54:41.206588+0000","last_clean":"2024-05-14T16:54:41.206588+0000","last_became_active":"2024-05-14T16:51:32.297260+0000","last_became_peered":"2024-05-14T16:51:32.297260+0000","last_unstale":"2024-05-14T16:54:41.206588+0000","last_undegraded":"2024-05-14T16:54:41.206588+0000","last_fullsized":"2024-05-14T16:54:41.206588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:43:50.542587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206503+0000","last_change":"2024-05-14T16:51:32.295525+0000","last_active":"2024-05-14T16:54:41.206503+0000","last_peered":"2024-05-14T16:54:41.206503+0000","last_clean":"2024-05-14T16:54:41.206503+0000","last_became_active":"2024-05-14T16:51:32.295215+0000","last_became_peered":"2024-05-14T16:51:32.295215+0000","last_unstale":"2024-05-14T16:54:41.206503+0000","last_undegraded":"2024-05-14T16:54:41.206503+0000","last_fullsized":"2024-05-14T16:54:41.206503+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:40:05.516280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207077+0000","last_change":"2024-05-14T16:51:32.307883+0000","last_active":"2024-05-14T16:54:41.207077+0000","last_peered":"2024-05-14T16:54:41.207077+0000","last_clean":"2024-05-14T16:54:41.207077+0000","last_became_active":"2024-05-14T16:51:32.307499+0000","last_became_peered":"2024-05-14T16:51:32.307499+0000","last_unstale":"2024-05-14T16:54:41.207077+0000","last_undegraded":"2024-05-14T16:54:41.207077+0000","last_fullsized":"2024-05-14T16:54:41.207077+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:30:59.978059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206362+0000","last_change":"2024-05-14T16:51:32.294429+0000","last_active":"2024-05-14T16:54:41.206362+0000","last_peered":"2024-05-14T16:54:41.206362+0000","last_clean":"2024-05-14T16:54:41.206362+0000","last_became_active":"2024-05-14T16:51:32.294088+0000","last_became_peered":"2024-05-14T16:51:32.294088+0000","last_unstale":"2024-05-14T16:54:41.206362+0000","last_undegraded":"2024-05-14T16:54:41.206362+0000","last_fullsized":"2024-05-14T16:54:41.206362+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:25:19.383586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.194143+0000","last_change":"2024-05-14T16:51:32.296169+0000","last_active":"2024-05-14T16:54:41.194143+0000","last_peered":"2024-05-14T16:54:41.194143+0000","last_clean":"2024-05-14T16:54:41.194143+0000","last_became_active":"2024-05-14T16:51:32.295819+0000","last_became_peered":"2024-05-14T16:51:32.295819+0000","last_unstale":"2024-05-14T16:54:41.194143+0000","last_undegraded":"2024-05-14T16:54:41.194143+0000","last_fullsized":"2024-05-14T16:54:41.194143+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:56:23.932039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206285+0000","last_change":"2024-05-14T16:51:32.309609+0000","last_active":"2024-05-14T16:54:41.206285+0000","last_peered":"2024-05-14T16:54:41.206285+0000","last_clean":"2024-05-14T16:54:41.206285+0000","last_became_active":"2024-05-14T16:51:32.309175+0000","last_became_peered":"2024-05-14T16:51:32.309175+0000","last_unstale":"2024-05-14T16:54:41.206285+0000","last_undegraded":"2024-05-14T16:54:41.206285+0000","last_fullsized":"2024-05-14T16:54:41.206285+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:06:46.717592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207138+0000","last_change":"2024-05-14T16:52:13.446592+0000","last_active":"2024-05-14T16:54:41.207138+0000","last_peered":"2024-05-14T16:54:41.207138+0000","last_clean":"2024-05-14T16:54:41.207138+0000","last_became_active":"2024-05-14T16:52:13.446238+0000","last_became_peered":"2024-05-14T16:52:13.446238+0000","last_unstale":"2024-05-14T16:54:41.207138+0000","last_undegraded":"2024-05-14T16:54:41.207138+0000","last_fullsized":"2024-05-14T16:54:41.207138+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:33:14.216797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206922+0000","last_change":"2024-05-14T16:51:32.307571+0000","last_active":"2024-05-14T16:54:41.206922+0000","last_peered":"2024-05-14T16:54:41.206922+0000","last_clean":"2024-05-14T16:54:41.206922+0000","last_became_active":"2024-05-14T16:51:32.307151+0000","last_became_peered":"2024-05-14T16:51:32.307151+0000","last_unstale":"2024-05-14T16:54:41.206922+0000","last_undegraded":"2024-05-14T16:54:41.206922+0000","last_fullsized":"2024-05-14T16:54:41.206922+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:17:26.325393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.194394+0000","last_change":"2024-05-14T16:51:32.298773+0000","last_active":"2024-05-14T16:54:41.194394+0000","last_peered":"2024-05-14T16:54:41.194394+0000","last_clean":"2024-05-14T16:54:41.194394+0000","last_became_active":"2024-05-14T16:51:32.298283+0000","last_became_peered":"2024-05-14T16:51:32.298283+0000","last_unstale":"2024-05-14T16:54:41.194394+0000","last_undegraded":"2024-05-14T16:54:41.194394+0000","last_fullsized":"2024-05-14T16:54:41.194394+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:52:18.471201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207583+0000","last_change":"2024-05-14T16:51:32.302443+0000","last_active":"2024-05-14T16:54:41.207583+0000","last_peered":"2024-05-14T16:54:41.207583+0000","last_clean":"2024-05-14T16:54:41.207583+0000","last_became_active":"2024-05-14T16:51:32.302034+0000","last_became_peered":"2024-05-14T16:51:32.302034+0000","last_unstale":"2024-05-14T16:54:41.207583+0000","last_undegraded":"2024-05-14T16:54:41.207583+0000","last_fullsized":"2024-05-14T16:54:41.207583+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:53:08.041293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207252+0000","last_change":"2024-05-14T16:51:32.304837+0000","last_active":"2024-05-14T16:54:41.207252+0000","last_peered":"2024-05-14T16:54:41.207252+0000","last_clean":"2024-05-14T16:54:41.207252+0000","last_became_active":"2024-05-14T16:51:32.304500+0000","last_became_peered":"2024-05-14T16:51:32.304500+0000","last_unstale":"2024-05-14T16:54:41.207252+0000","last_undegraded":"2024-05-14T16:54:41.207252+0000","last_fullsized":"2024-05-14T16:54:41.207252+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:32:02.713044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.192789+0000","last_change":"2024-05-14T16:51:32.299247+0000","last_active":"2024-05-14T16:54:41.192789+0000","last_peered":"2024-05-14T16:54:41.192789+0000","last_clean":"2024-05-14T16:54:41.192789+0000","last_became_active":"2024-05-14T16:51:32.298925+0000","last_became_peered":"2024-05-14T16:51:32.298925+0000","last_unstale":"2024-05-14T16:54:41.192789+0000","last_undegraded":"2024-05-14T16:54:41.192789+0000","last_fullsized":"2024-05-14T16:54:41.192789+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:10:38.847877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.205900+0000","last_change":"2024-05-14T16:51:32.293743+0000","last_active":"2024-05-14T16:54:41.205900+0000","last_peered":"2024-05-14T16:54:41.205900+0000","last_clean":"2024-05-14T16:54:41.205900+0000","last_became_active":"2024-05-14T16:51:32.293429+0000","last_became_peered":"2024-05-14T16:51:32.293429+0000","last_unstale":"2024-05-14T16:54:41.205900+0000","last_undegraded":"2024-05-14T16:54:41.205900+0000","last_fullsized":"2024-05-14T16:54:41.205900+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:12:33.909940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.205749+0000","last_change":"2024-05-14T16:51:32.304940+0000","last_active":"2024-05-14T16:54:41.205749+0000","last_peered":"2024-05-14T16:54:41.205749+0000","last_clean":"2024-05-14T16:54:41.205749+0000","last_became_active":"2024-05-14T16:51:32.304482+0000","last_became_peered":"2024-05-14T16:51:32.304482+0000","last_unstale":"2024-05-14T16:54:41.205749+0000","last_undegraded":"2024-05-14T16:54:41.205749+0000","last_fullsized":"2024-05-14T16:54:41.205749+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:54:17.771620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206201+0000","last_change":"2024-05-14T16:51:32.301784+0000","last_active":"2024-05-14T16:54:41.206201+0000","last_peered":"2024-05-14T16:54:41.206201+0000","last_clean":"2024-05-14T16:54:41.206201+0000","last_became_active":"2024-05-14T16:51:32.301472+0000","last_became_peered":"2024-05-14T16:51:32.301472+0000","last_unstale":"2024-05-14T16:54:41.206201+0000","last_undegraded":"2024-05-14T16:54:41.206201+0000","last_fullsized":"2024-05-14T16:54:41.206201+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:10:56.735424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.197151+0000","last_change":"2024-05-14T16:51:32.297951+0000","last_active":"2024-05-14T16:54:41.197151+0000","last_peered":"2024-05-14T16:54:41.197151+0000","last_clean":"2024-05-14T16:54:41.197151+0000","last_became_active":"2024-05-14T16:51:32.296602+0000","last_became_peered":"2024-05-14T16:51:32.296602+0000","last_unstale":"2024-05-14T16:54:41.197151+0000","last_undegraded":"2024-05-14T16:54:41.197151+0000","last_fullsized":"2024-05-14T16:54:41.197151+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:16:36.039077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.208451+0000","last_change":"2024-05-14T16:51:32.290398+0000","last_active":"2024-05-14T16:54:41.208451+0000","last_peered":"2024-05-14T16:54:41.208451+0000","last_clean":"2024-05-14T16:54:41.208451+0000","last_became_active":"2024-05-14T16:51:32.290083+0000","last_became_peered":"2024-05-14T16:51:32.290083+0000","last_unstale":"2024-05-14T16:54:41.208451+0000","last_undegraded":"2024-05-14T16:54:41.208451+0000","last_fullsized":"2024-05-14T16:54:41.208451+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:48:05.795209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207421+0000","last_change":"2024-05-14T16:51:32.294398+0000","last_active":"2024-05-14T16:54:41.207421+0000","last_peered":"2024-05-14T16:54:41.207421+0000","last_clean":"2024-05-14T16:54:41.207421+0000","last_became_active":"2024-05-14T16:51:32.294080+0000","last_became_peered":"2024-05-14T16:51:32.294080+0000","last_unstale":"2024-05-14T16:54:41.207421+0000","last_undegraded":"2024-05-14T16:54:41.207421+0000","last_fullsized":"2024-05-14T16:54:41.207421+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:15:03.859045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.208336+0000","last_change":"2024-05-14T16:51:29.274482+0000","last_active":"2024-05-14T16:54:41.208336+0000","last_peered":"2024-05-14T16:54:41.208336+0000","last_clean":"2024-05-14T16:54:41.208336+0000","last_became_active":"2024-05-14T16:51:29.274152+0000","last_became_peered":"2024-05-14T16:51:29.274152+0000","last_unstale":"2024-05-14T16:54:41.208336+0000","last_undegraded":"2024-05-14T16:54:41.208336+0000","last_fullsized":"2024-05-14T16:54:41.208336+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:05:38.274248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207141+0000","last_change":"2024-05-14T16:51:32.300956+0000","last_active":"2024-05-14T16:54:41.207141+0000","last_peered":"2024-05-14T16:54:41.207141+0000","last_clean":"2024-05-14T16:54:41.207141+0000","last_became_active":"2024-05-14T16:51:32.300546+0000","last_became_peered":"2024-05-14T16:51:32.300546+0000","last_unstale":"2024-05-14T16:54:41.207141+0000","last_undegraded":"2024-05-14T16:54:41.207141+0000","last_fullsized":"2024-05-14T16:54:41.207141+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:54:09.181036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207390+0000","last_change":"2024-05-14T16:51:29.345206+0000","last_active":"2024-05-14T16:54:41.207390+0000","last_peered":"2024-05-14T16:54:41.207390+0000","last_clean":"2024-05-14T16:54:41.207390+0000","last_became_active":"2024-05-14T16:51:29.344699+0000","last_became_peered":"2024-05-14T16:51:29.344699+0000","last_unstale":"2024-05-14T16:54:41.207390+0000","last_undegraded":"2024-05-14T16:54:41.207390+0000","last_fullsized":"2024-05-14T16:54:41.207390+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:14:02.364661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206815+0000","last_change":"2024-05-14T16:51:32.297522+0000","last_active":"2024-05-14T16:54:41.206815+0000","last_peered":"2024-05-14T16:54:41.206815+0000","last_clean":"2024-05-14T16:54:41.206815+0000","last_became_active":"2024-05-14T16:51:32.296866+0000","last_became_peered":"2024-05-14T16:51:32.296866+0000","last_unstale":"2024-05-14T16:54:41.206815+0000","last_undegraded":"2024-05-14T16:54:41.206815+0000","last_fullsized":"2024-05-14T16:54:41.206815+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:49:43.409575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207355+0000","last_change":"2024-05-14T16:51:29.280632+0000","last_active":"2024-05-14T16:54:41.207355+0000","last_peered":"2024-05-14T16:54:41.207355+0000","last_clean":"2024-05-14T16:54:41.207355+0000","last_became_active":"2024-05-14T16:51:29.280246+0000","last_became_peered":"2024-05-14T16:51:29.280246+0000","last_unstale":"2024-05-14T16:54:41.207355+0000","last_undegraded":"2024-05-14T16:54:41.207355+0000","last_fullsized":"2024-05-14T16:54:41.207355+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:34:47.925344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206511+0000","last_change":"2024-05-14T16:51:32.298284+0000","last_active":"2024-05-14T16:54:41.206511+0000","last_peered":"2024-05-14T16:54:41.206511+0000","last_clean":"2024-05-14T16:54:41.206511+0000","last_became_active":"2024-05-14T16:51:32.294202+0000","last_became_peered":"2024-05-14T16:51:32.294202+0000","last_unstale":"2024-05-14T16:54:41.206511+0000","last_undegraded":"2024-05-14T16:54:41.206511+0000","last_fullsized":"2024-05-14T16:54:41.206511+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:27:37.561275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206773+0000","last_change":"2024-05-14T16:51:29.276227+0000","last_active":"2024-05-14T16:54:41.206773+0000","last_peered":"2024-05-14T16:54:41.206773+0000","last_clean":"2024-05-14T16:54:41.206773+0000","last_became_active":"2024-05-14T16:51:29.275837+0000","last_became_peered":"2024-05-14T16:51:29.275837+0000","last_unstale":"2024-05-14T16:54:41.206773+0000","last_undegraded":"2024-05-14T16:54:41.206773+0000","last_fullsized":"2024-05-14T16:54:41.206773+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:03:29.402985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207149+0000","last_change":"2024-05-14T16:51:32.308173+0000","last_active":"2024-05-14T16:54:41.207149+0000","last_peered":"2024-05-14T16:54:41.207149+0000","last_clean":"2024-05-14T16:54:41.207149+0000","last_became_active":"2024-05-14T16:51:32.307135+0000","last_became_peered":"2024-05-14T16:51:32.307135+0000","last_unstale":"2024-05-14T16:54:41.207149+0000","last_undegraded":"2024-05-14T16:54:41.207149+0000","last_fullsized":"2024-05-14T16:54:41.207149+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:01:46.579968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'8","reported_seq":212,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.272035+0000","last_change":"2024-05-14T16:51:29.279486+0000","last_active":"2024-05-14T16:54:41.272035+0000","last_peered":"2024-05-14T16:54:41.272035+0000","last_clean":"2024-05-14T16:54:41.272035+0000","last_became_active":"2024-05-14T16:51:29.279133+0000","last_became_peered":"2024-05-14T16:51:29.279133+0000","last_unstale":"2024-05-14T16:54:41.272035+0000","last_undegraded":"2024-05-14T16:54:41.272035+0000","last_fullsized":"2024-05-14T16:54:41.272035+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:26:18.620421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":175,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206811+0000","last_change":"2024-05-14T16:51:32.299852+0000","last_active":"2024-05-14T16:54:41.206811+0000","last_peered":"2024-05-14T16:54:41.206811+0000","last_clean":"2024-05-14T16:54:41.206811+0000","last_became_active":"2024-05-14T16:51:32.299541+0000","last_became_peered":"2024-05-14T16:51:32.299541+0000","last_unstale":"2024-05-14T16:54:41.206811+0000","last_undegraded":"2024-05-14T16:54:41.206811+0000","last_fullsized":"2024-05-14T16:54:41.206811+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:36:38.191009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.194073+0000","last_change":"2024-05-14T16:51:29.275269+0000","last_active":"2024-05-14T16:54:41.194073+0000","last_peered":"2024-05-14T16:54:41.194073+0000","last_clean":"2024-05-14T16:54:41.194073+0000","last_became_active":"2024-05-14T16:51:29.274792+0000","last_became_peered":"2024-05-14T16:51:29.274792+0000","last_unstale":"2024-05-14T16:54:41.194073+0000","last_undegraded":"2024-05-14T16:54:41.194073+0000","last_fullsized":"2024-05-14T16:54:41.194073+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:17:08.925289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207090+0000","last_change":"2024-05-14T16:51:32.306536+0000","last_active":"2024-05-14T16:54:41.207090+0000","last_peered":"2024-05-14T16:54:41.207090+0000","last_clean":"2024-05-14T16:54:41.207090+0000","last_became_active":"2024-05-14T16:51:32.304976+0000","last_became_peered":"2024-05-14T16:51:32.304976+0000","last_unstale":"2024-05-14T16:54:41.207090+0000","last_undegraded":"2024-05-14T16:54:41.207090+0000","last_fullsized":"2024-05-14T16:54:41.207090+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:48:13.284022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.192664+0000","last_change":"2024-05-14T16:51:29.266783+0000","last_active":"2024-05-14T16:54:41.192664+0000","last_peered":"2024-05-14T16:54:41.192664+0000","last_clean":"2024-05-14T16:54:41.192664+0000","last_became_active":"2024-05-14T16:51:29.266409+0000","last_became_peered":"2024-05-14T16:51:29.266409+0000","last_unstale":"2024-05-14T16:54:41.192664+0000","last_undegraded":"2024-05-14T16:54:41.192664+0000","last_fullsized":"2024-05-14T16:54:41.192664+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:33:14.981709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206568+0000","last_change":"2024-05-14T16:51:32.289288+0000","last_active":"2024-05-14T16:54:41.206568+0000","last_peered":"2024-05-14T16:54:41.206568+0000","last_clean":"2024-05-14T16:54:41.206568+0000","last_became_active":"2024-05-14T16:51:32.288920+0000","last_became_peered":"2024-05-14T16:51:32.288920+0000","last_unstale":"2024-05-14T16:54:41.206568+0000","last_undegraded":"2024-05-14T16:54:41.206568+0000","last_fullsized":"2024-05-14T16:54:41.206568+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:23:43.659417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206346+0000","last_change":"2024-05-14T16:51:29.273926+0000","last_active":"2024-05-14T16:54:41.206346+0000","last_peered":"2024-05-14T16:54:41.206346+0000","last_clean":"2024-05-14T16:54:41.206346+0000","last_became_active":"2024-05-14T16:51:29.273059+0000","last_became_peered":"2024-05-14T16:51:29.273059+0000","last_unstale":"2024-05-14T16:54:41.206346+0000","last_undegraded":"2024-05-14T16:54:41.206346+0000","last_fullsized":"2024-05-14T16:54:41.206346+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:46:27.800481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206716+0000","last_change":"2024-05-14T16:51:32.287554+0000","last_active":"2024-05-14T16:54:41.206716+0000","last_peered":"2024-05-14T16:54:41.206716+0000","last_clean":"2024-05-14T16:54:41.206716+0000","last_became_active":"2024-05-14T16:51:32.287204+0000","last_became_peered":"2024-05-14T16:51:32.287204+0000","last_unstale":"2024-05-14T16:54:41.206716+0000","last_undegraded":"2024-05-14T16:54:41.206716+0000","last_fullsized":"2024-05-14T16:54:41.206716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:29:40.675378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'5","reported_seq":205,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.271284+0000","last_change":"2024-05-14T16:51:29.274819+0000","last_active":"2024-05-14T16:54:41.271284+0000","last_peered":"2024-05-14T16:54:41.271284+0000","last_clean":"2024-05-14T16:54:41.271284+0000","last_became_active":"2024-05-14T16:51:29.274454+0000","last_became_peered":"2024-05-14T16:51:29.274454+0000","last_unstale":"2024-05-14T16:54:41.271284+0000","last_undegraded":"2024-05-14T16:54:41.271284+0000","last_fullsized":"2024-05-14T16:54:41.271284+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:36:30.496064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":171,"num_read_kb":171,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206897+0000","last_change":"2024-05-14T16:51:32.295732+0000","last_active":"2024-05-14T16:54:41.206897+0000","last_peered":"2024-05-14T16:54:41.206897+0000","last_clean":"2024-05-14T16:54:41.206897+0000","last_became_active":"2024-05-14T16:51:32.294919+0000","last_became_peered":"2024-05-14T16:51:32.294919+0000","last_unstale":"2024-05-14T16:54:41.206897+0000","last_undegraded":"2024-05-14T16:54:41.206897+0000","last_fullsized":"2024-05-14T16:54:41.206897+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:08:54.566601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207342+0000","last_change":"2024-05-14T16:51:29.279338+0000","last_active":"2024-05-14T16:54:41.207342+0000","last_peered":"2024-05-14T16:54:41.207342+0000","last_clean":"2024-05-14T16:54:41.207342+0000","last_became_active":"2024-05-14T16:51:29.278928+0000","last_became_peered":"2024-05-14T16:51:29.278928+0000","last_unstale":"2024-05-14T16:54:41.207342+0000","last_undegraded":"2024-05-14T16:54:41.207342+0000","last_fullsized":"2024-05-14T16:54:41.207342+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:24:34.803515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206905+0000","last_change":"2024-05-14T16:51:32.305294+0000","last_active":"2024-05-14T16:54:41.206905+0000","last_peered":"2024-05-14T16:54:41.206905+0000","last_clean":"2024-05-14T16:54:41.206905+0000","last_became_active":"2024-05-14T16:51:32.304563+0000","last_became_peered":"2024-05-14T16:51:32.304563+0000","last_unstale":"2024-05-14T16:54:41.206905+0000","last_undegraded":"2024-05-14T16:54:41.206905+0000","last_fullsized":"2024-05-14T16:54:41.206905+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:56:06.259475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207460+0000","last_change":"2024-05-14T16:51:29.345400+0000","last_active":"2024-05-14T16:54:41.207460+0000","last_peered":"2024-05-14T16:54:41.207460+0000","last_clean":"2024-05-14T16:54:41.207460+0000","last_became_active":"2024-05-14T16:51:29.345010+0000","last_became_peered":"2024-05-14T16:51:29.345010+0000","last_unstale":"2024-05-14T16:54:41.207460+0000","last_undegraded":"2024-05-14T16:54:41.207460+0000","last_fullsized":"2024-05-14T16:54:41.207460+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:24:22.933005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206748+0000","last_change":"2024-05-14T16:52:13.923167+0000","last_active":"2024-05-14T16:54:41.206748+0000","last_peered":"2024-05-14T16:54:41.206748+0000","last_clean":"2024-05-14T16:54:41.206748+0000","last_became_active":"2024-05-14T16:52:13.922813+0000","last_became_peered":"2024-05-14T16:52:13.922813+0000","last_unstale":"2024-05-14T16:54:41.206748+0000","last_undegraded":"2024-05-14T16:54:41.206748+0000","last_fullsized":"2024-05-14T16:54:41.206748+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:50:38.672408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'38","reported_seq":421,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.462964+0000","last_change":"2024-05-14T16:51:29.277591+0000","last_active":"2024-05-14T16:54:41.462964+0000","last_peered":"2024-05-14T16:54:41.462964+0000","last_clean":"2024-05-14T16:54:41.462964+0000","last_became_active":"2024-05-14T16:51:29.276612+0000","last_became_peered":"2024-05-14T16:51:29.276612+0000","last_unstale":"2024-05-14T16:54:41.462964+0000","last_undegraded":"2024-05-14T16:54:41.462964+0000","last_fullsized":"2024-05-14T16:54:41.462964+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:22:28.934173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66894,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":354,"num_read_kb":241,"num_write":39,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207212+0000","last_change":"2024-05-14T16:51:32.302321+0000","last_active":"2024-05-14T16:54:41.207212+0000","last_peered":"2024-05-14T16:54:41.207212+0000","last_clean":"2024-05-14T16:54:41.207212+0000","last_became_active":"2024-05-14T16:51:32.301973+0000","last_became_peered":"2024-05-14T16:51:32.301973+0000","last_unstale":"2024-05-14T16:54:41.207212+0000","last_undegraded":"2024-05-14T16:54:41.207212+0000","last_fullsized":"2024-05-14T16:54:41.207212+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:49:24.054379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207061+0000","last_change":"2024-05-14T16:51:29.278776+0000","last_active":"2024-05-14T16:54:41.207061+0000","last_peered":"2024-05-14T16:54:41.207061+0000","last_clean":"2024-05-14T16:54:41.207061+0000","last_became_active":"2024-05-14T16:51:29.278395+0000","last_became_peered":"2024-05-14T16:51:29.278395+0000","last_unstale":"2024-05-14T16:54:41.207061+0000","last_undegraded":"2024-05-14T16:54:41.207061+0000","last_fullsized":"2024-05-14T16:54:41.207061+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:29:20.942783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.192617+0000","last_change":"2024-05-14T16:51:32.292107+0000","last_active":"2024-05-14T16:54:41.192617+0000","last_peered":"2024-05-14T16:54:41.192617+0000","last_clean":"2024-05-14T16:54:41.192617+0000","last_became_active":"2024-05-14T16:51:32.291729+0000","last_became_peered":"2024-05-14T16:51:32.291729+0000","last_unstale":"2024-05-14T16:54:41.192617+0000","last_undegraded":"2024-05-14T16:54:41.192617+0000","last_fullsized":"2024-05-14T16:54:41.192617+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:30:15.440599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206802+0000","last_change":"2024-05-14T16:51:29.274862+0000","last_active":"2024-05-14T16:54:41.206802+0000","last_peered":"2024-05-14T16:54:41.206802+0000","last_clean":"2024-05-14T16:54:41.206802+0000","last_became_active":"2024-05-14T16:51:29.274527+0000","last_became_peered":"2024-05-14T16:51:29.274527+0000","last_unstale":"2024-05-14T16:54:41.206802+0000","last_undegraded":"2024-05-14T16:54:41.206802+0000","last_fullsized":"2024-05-14T16:54:41.206802+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:43:15.634585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207361+0000","last_change":"2024-05-14T16:51:32.306158+0000","last_active":"2024-05-14T16:54:41.207361+0000","last_peered":"2024-05-14T16:54:41.207361+0000","last_clean":"2024-05-14T16:54:41.207361+0000","last_became_active":"2024-05-14T16:51:32.305834+0000","last_became_peered":"2024-05-14T16:51:32.305834+0000","last_unstale":"2024-05-14T16:54:41.207361+0000","last_undegraded":"2024-05-14T16:54:41.207361+0000","last_fullsized":"2024-05-14T16:54:41.207361+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:31:47.645862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'6","reported_seq":209,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.462297+0000","last_change":"2024-05-14T16:51:29.270735+0000","last_active":"2024-05-14T16:54:41.462297+0000","last_peered":"2024-05-14T16:54:41.462297+0000","last_clean":"2024-05-14T16:54:41.462297+0000","last_became_active":"2024-05-14T16:51:29.270362+0000","last_became_peered":"2024-05-14T16:51:29.270362+0000","last_unstale":"2024-05-14T16:54:41.462297+0000","last_undegraded":"2024-05-14T16:54:41.462297+0000","last_fullsized":"2024-05-14T16:54:41.462297+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:40:12.288655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1678,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":175,"num_read_kb":4,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206222+0000","last_change":"2024-05-14T16:51:32.305007+0000","last_active":"2024-05-14T16:54:41.206222+0000","last_peered":"2024-05-14T16:54:41.206222+0000","last_clean":"2024-05-14T16:54:41.206222+0000","last_became_active":"2024-05-14T16:51:32.303927+0000","last_became_peered":"2024-05-14T16:51:32.303927+0000","last_unstale":"2024-05-14T16:54:41.206222+0000","last_undegraded":"2024-05-14T16:54:41.206222+0000","last_fullsized":"2024-05-14T16:54:41.206222+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:42:28.018492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206550+0000","last_change":"2024-05-14T16:51:29.345902+0000","last_active":"2024-05-14T16:54:41.206550+0000","last_peered":"2024-05-14T16:54:41.206550+0000","last_clean":"2024-05-14T16:54:41.206550+0000","last_became_active":"2024-05-14T16:51:29.344904+0000","last_became_peered":"2024-05-14T16:51:29.344904+0000","last_unstale":"2024-05-14T16:54:41.206550+0000","last_undegraded":"2024-05-14T16:54:41.206550+0000","last_fullsized":"2024-05-14T16:54:41.206550+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:56:22.209067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206556+0000","last_change":"2024-05-14T16:51:32.306459+0000","last_active":"2024-05-14T16:54:41.206556+0000","last_peered":"2024-05-14T16:54:41.206556+0000","last_clean":"2024-05-14T16:54:41.206556+0000","last_became_active":"2024-05-14T16:51:32.304897+0000","last_became_peered":"2024-05-14T16:51:32.304897+0000","last_unstale":"2024-05-14T16:54:41.206556+0000","last_undegraded":"2024-05-14T16:54:41.206556+0000","last_fullsized":"2024-05-14T16:54:41.206556+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:09:51.216631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'4","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206585+0000","last_change":"2024-05-14T16:51:29.275219+0000","last_active":"2024-05-14T16:54:41.206585+0000","last_peered":"2024-05-14T16:54:41.206585+0000","last_clean":"2024-05-14T16:54:41.206585+0000","last_became_active":"2024-05-14T16:51:29.274816+0000","last_became_peered":"2024-05-14T16:51:29.274816+0000","last_unstale":"2024-05-14T16:54:41.206585+0000","last_undegraded":"2024-05-14T16:54:41.206585+0000","last_fullsized":"2024-05-14T16:54:41.206585+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:55:49.667387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.196946+0000","last_change":"2024-05-14T16:51:32.299284+0000","last_active":"2024-05-14T16:54:41.196946+0000","last_peered":"2024-05-14T16:54:41.196946+0000","last_clean":"2024-05-14T16:54:41.196946+0000","last_became_active":"2024-05-14T16:51:32.298933+0000","last_became_peered":"2024-05-14T16:51:32.298933+0000","last_unstale":"2024-05-14T16:54:41.196946+0000","last_undegraded":"2024-05-14T16:54:41.196946+0000","last_fullsized":"2024-05-14T16:54:41.196946+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:35:42.266665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.208227+0000","last_change":"2024-05-14T16:51:29.345454+0000","last_active":"2024-05-14T16:54:41.208227+0000","last_peered":"2024-05-14T16:54:41.208227+0000","last_clean":"2024-05-14T16:54:41.208227+0000","last_became_active":"2024-05-14T16:51:29.345074+0000","last_became_peered":"2024-05-14T16:51:29.345074+0000","last_unstale":"2024-05-14T16:54:41.208227+0000","last_undegraded":"2024-05-14T16:54:41.208227+0000","last_fullsized":"2024-05-14T16:54:41.208227+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:56:11.661144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.194447+0000","last_change":"2024-05-14T16:51:32.288804+0000","last_active":"2024-05-14T16:54:41.194447+0000","last_peered":"2024-05-14T16:54:41.194447+0000","last_clean":"2024-05-14T16:54:41.194447+0000","last_became_active":"2024-05-14T16:51:32.288441+0000","last_became_peered":"2024-05-14T16:51:32.288441+0000","last_unstale":"2024-05-14T16:54:41.194447+0000","last_undegraded":"2024-05-14T16:54:41.194447+0000","last_fullsized":"2024-05-14T16:54:41.194447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:28:31.720342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206860+0000","last_change":"2024-05-14T16:51:29.272032+0000","last_active":"2024-05-14T16:54:41.206860+0000","last_peered":"2024-05-14T16:54:41.206860+0000","last_clean":"2024-05-14T16:54:41.206860+0000","last_became_active":"2024-05-14T16:51:29.271677+0000","last_became_peered":"2024-05-14T16:51:29.271677+0000","last_unstale":"2024-05-14T16:54:41.206860+0000","last_undegraded":"2024-05-14T16:54:41.206860+0000","last_fullsized":"2024-05-14T16:54:41.206860+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:39:41.377591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.194175+0000","last_change":"2024-05-14T16:51:32.295159+0000","last_active":"2024-05-14T16:54:41.194175+0000","last_peered":"2024-05-14T16:54:41.194175+0000","last_clean":"2024-05-14T16:54:41.194175+0000","last_became_active":"2024-05-14T16:51:32.294812+0000","last_became_peered":"2024-05-14T16:51:32.294812+0000","last_unstale":"2024-05-14T16:54:41.194175+0000","last_undegraded":"2024-05-14T16:54:41.194175+0000","last_fullsized":"2024-05-14T16:54:41.194175+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:08:22.411800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206770+0000","last_change":"2024-05-14T16:51:32.306582+0000","last_active":"2024-05-14T16:54:41.206770+0000","last_peered":"2024-05-14T16:54:41.206770+0000","last_clean":"2024-05-14T16:54:41.206770+0000","last_became_active":"2024-05-14T16:51:32.306209+0000","last_became_peered":"2024-05-14T16:51:32.306209+0000","last_unstale":"2024-05-14T16:54:41.206770+0000","last_undegraded":"2024-05-14T16:54:41.206770+0000","last_fullsized":"2024-05-14T16:54:41.206770+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:30:46.264097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":205,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.192557+0000","last_change":"2024-05-14T16:51:29.345057+0000","last_active":"2024-05-14T16:54:41.192557+0000","last_peered":"2024-05-14T16:54:41.192557+0000","last_clean":"2024-05-14T16:54:41.192557+0000","last_became_active":"2024-05-14T16:51:29.344658+0000","last_became_peered":"2024-05-14T16:51:29.344658+0000","last_unstale":"2024-05-14T16:54:41.192557+0000","last_undegraded":"2024-05-14T16:54:41.192557+0000","last_fullsized":"2024-05-14T16:54:41.192557+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:43:53.007543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":171,"num_read_kb":171,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207516+0000","last_change":"2024-05-14T16:51:32.298958+0000","last_active":"2024-05-14T16:54:41.207516+0000","last_peered":"2024-05-14T16:54:41.207516+0000","last_clean":"2024-05-14T16:54:41.207516+0000","last_became_active":"2024-05-14T16:51:32.298420+0000","last_became_peered":"2024-05-14T16:51:32.298420+0000","last_unstale":"2024-05-14T16:54:41.207516+0000","last_undegraded":"2024-05-14T16:54:41.207516+0000","last_fullsized":"2024-05-14T16:54:41.207516+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:15:21.262026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206858+0000","last_change":"2024-05-14T16:51:29.272082+0000","last_active":"2024-05-14T16:54:41.206858+0000","last_peered":"2024-05-14T16:54:41.206858+0000","last_clean":"2024-05-14T16:54:41.206858+0000","last_became_active":"2024-05-14T16:51:29.271711+0000","last_became_peered":"2024-05-14T16:51:29.271711+0000","last_unstale":"2024-05-14T16:54:41.206858+0000","last_undegraded":"2024-05-14T16:54:41.206858+0000","last_fullsized":"2024-05-14T16:54:41.206858+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:27:44.320423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":102,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206476+0000","last_change":"2024-05-14T16:50:05.320136+0000","last_active":"2024-05-14T16:54:41.206476+0000","last_peered":"2024-05-14T16:54:41.206476+0000","last_clean":"2024-05-14T16:54:41.206476+0000","last_became_active":"2024-05-14T16:50:05.315016+0000","last_became_peered":"2024-05-14T16:50:05.315016+0000","last_unstale":"2024-05-14T16:54:41.206476+0000","last_undegraded":"2024-05-14T16:54:41.206476+0000","last_fullsized":"2024-05-14T16:54:41.206476+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:38.138211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:38.138211+0000","last_clean_scrub_stamp":"2024-05-14T16:46:38.138211+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:19:09.012578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.192492+0000","last_change":"2024-05-14T16:51:32.295995+0000","last_active":"2024-05-14T16:54:41.192492+0000","last_peered":"2024-05-14T16:54:41.192492+0000","last_clean":"2024-05-14T16:54:41.192492+0000","last_became_active":"2024-05-14T16:51:32.295640+0000","last_became_peered":"2024-05-14T16:51:32.295640+0000","last_unstale":"2024-05-14T16:54:41.192492+0000","last_undegraded":"2024-05-14T16:54:41.192492+0000","last_fullsized":"2024-05-14T16:54:41.192492+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:42:26.774916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206592+0000","last_change":"2024-05-14T16:51:29.274892+0000","last_active":"2024-05-14T16:54:41.206592+0000","last_peered":"2024-05-14T16:54:41.206592+0000","last_clean":"2024-05-14T16:54:41.206592+0000","last_became_active":"2024-05-14T16:51:29.274386+0000","last_became_peered":"2024-05-14T16:51:29.274386+0000","last_unstale":"2024-05-14T16:54:41.206592+0000","last_undegraded":"2024-05-14T16:54:41.206592+0000","last_fullsized":"2024-05-14T16:54:41.206592+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:16:54.899320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207219+0000","last_change":"2024-05-14T16:51:29.345672+0000","last_active":"2024-05-14T16:54:41.207219+0000","last_peered":"2024-05-14T16:54:41.207219+0000","last_clean":"2024-05-14T16:54:41.207219+0000","last_became_active":"2024-05-14T16:51:29.344948+0000","last_became_peered":"2024-05-14T16:51:29.344948+0000","last_unstale":"2024-05-14T16:54:41.207219+0000","last_undegraded":"2024-05-14T16:54:41.207219+0000","last_fullsized":"2024-05-14T16:54:41.207219+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:19:26.577799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.192847+0000","last_change":"2024-05-14T16:51:32.297360+0000","last_active":"2024-05-14T16:54:41.192847+0000","last_peered":"2024-05-14T16:54:41.192847+0000","last_clean":"2024-05-14T16:54:41.192847+0000","last_became_active":"2024-05-14T16:51:32.297025+0000","last_became_peered":"2024-05-14T16:51:32.297025+0000","last_unstale":"2024-05-14T16:54:41.192847+0000","last_undegraded":"2024-05-14T16:54:41.192847+0000","last_fullsized":"2024-05-14T16:54:41.192847+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:05:39.705860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'46","reported_seq":110,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.417853+0000","last_change":"2024-05-14T16:52:13.224328+0000","last_active":"2024-05-14T16:54:41.417853+0000","last_peered":"2024-05-14T16:54:41.417853+0000","last_clean":"2024-05-14T16:54:41.417853+0000","last_became_active":"2024-05-14T16:52:13.220271+0000","last_became_peered":"2024-05-14T16:52:13.220271+0000","last_unstale":"2024-05-14T16:54:41.417853+0000","last_undegraded":"2024-05-14T16:54:41.417853+0000","last_fullsized":"2024-05-14T16:54:41.417853+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:06:31.955274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207534+0000","last_change":"2024-05-14T16:51:32.292112+0000","last_active":"2024-05-14T16:54:41.207534+0000","last_peered":"2024-05-14T16:54:41.207534+0000","last_clean":"2024-05-14T16:54:41.207534+0000","last_became_active":"2024-05-14T16:51:32.291688+0000","last_became_peered":"2024-05-14T16:51:32.291688+0000","last_unstale":"2024-05-14T16:54:41.207534+0000","last_undegraded":"2024-05-14T16:54:41.207534+0000","last_fullsized":"2024-05-14T16:54:41.207534+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:13:06.728608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207174+0000","last_change":"2024-05-14T16:51:29.412805+0000","last_active":"2024-05-14T16:54:41.207174+0000","last_peered":"2024-05-14T16:54:41.207174+0000","last_clean":"2024-05-14T16:54:41.207174+0000","last_became_active":"2024-05-14T16:51:29.412394+0000","last_became_peered":"2024-05-14T16:51:29.412394+0000","last_unstale":"2024-05-14T16:54:41.207174+0000","last_undegraded":"2024-05-14T16:54:41.207174+0000","last_fullsized":"2024-05-14T16:54:41.207174+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:49:48.721104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206672+0000","last_change":"2024-05-14T16:52:13.218227+0000","last_active":"2024-05-14T16:54:41.206672+0000","last_peered":"2024-05-14T16:54:41.206672+0000","last_clean":"2024-05-14T16:54:41.206672+0000","last_became_active":"2024-05-14T16:52:13.217787+0000","last_became_peered":"2024-05-14T16:52:13.217787+0000","last_unstale":"2024-05-14T16:54:41.206672+0000","last_undegraded":"2024-05-14T16:54:41.206672+0000","last_fullsized":"2024-05-14T16:54:41.206672+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:52:33.348804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206178+0000","last_change":"2024-05-14T16:51:29.345427+0000","last_active":"2024-05-14T16:54:41.206178+0000","last_peered":"2024-05-14T16:54:41.206178+0000","last_clean":"2024-05-14T16:54:41.206178+0000","last_became_active":"2024-05-14T16:51:29.345049+0000","last_became_peered":"2024-05-14T16:51:29.345049+0000","last_unstale":"2024-05-14T16:54:41.206178+0000","last_undegraded":"2024-05-14T16:54:41.206178+0000","last_fullsized":"2024-05-14T16:54:41.206178+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:06:19.799660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206337+0000","last_change":"2024-05-14T16:51:32.298757+0000","last_active":"2024-05-14T16:54:41.206337+0000","last_peered":"2024-05-14T16:54:41.206337+0000","last_clean":"2024-05-14T16:54:41.206337+0000","last_became_active":"2024-05-14T16:51:32.297193+0000","last_became_peered":"2024-05-14T16:51:32.297193+0000","last_unstale":"2024-05-14T16:54:41.206337+0000","last_undegraded":"2024-05-14T16:54:41.206337+0000","last_fullsized":"2024-05-14T16:54:41.206337+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:10:19.672230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207012+0000","last_change":"2024-05-14T16:51:29.270545+0000","last_active":"2024-05-14T16:54:41.207012+0000","last_peered":"2024-05-14T16:54:41.207012+0000","last_clean":"2024-05-14T16:54:41.207012+0000","last_became_active":"2024-05-14T16:51:29.270182+0000","last_became_peered":"2024-05-14T16:51:29.270182+0000","last_unstale":"2024-05-14T16:54:41.207012+0000","last_undegraded":"2024-05-14T16:54:41.207012+0000","last_fullsized":"2024-05-14T16:54:41.207012+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:19:49.977916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.196769+0000","last_change":"2024-05-14T16:51:32.294767+0000","last_active":"2024-05-14T16:54:41.196769+0000","last_peered":"2024-05-14T16:54:41.196769+0000","last_clean":"2024-05-14T16:54:41.196769+0000","last_became_active":"2024-05-14T16:51:32.293947+0000","last_became_peered":"2024-05-14T16:51:32.293947+0000","last_unstale":"2024-05-14T16:54:41.196769+0000","last_undegraded":"2024-05-14T16:54:41.196769+0000","last_fullsized":"2024-05-14T16:54:41.196769+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:30:21.731492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.196766+0000","last_change":"2024-05-14T16:52:13.226509+0000","last_active":"2024-05-14T16:54:41.196766+0000","last_peered":"2024-05-14T16:54:41.196766+0000","last_clean":"2024-05-14T16:54:41.196766+0000","last_became_active":"2024-05-14T16:52:13.220923+0000","last_became_peered":"2024-05-14T16:52:13.220923+0000","last_unstale":"2024-05-14T16:54:41.196766+0000","last_undegraded":"2024-05-14T16:54:41.196766+0000","last_fullsized":"2024-05-14T16:54:41.196766+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:25:52.655481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207841+0000","last_change":"2024-05-14T16:51:32.288810+0000","last_active":"2024-05-14T16:54:41.207841+0000","last_peered":"2024-05-14T16:54:41.207841+0000","last_clean":"2024-05-14T16:54:41.207841+0000","last_became_active":"2024-05-14T16:51:32.288492+0000","last_became_peered":"2024-05-14T16:51:32.288492+0000","last_unstale":"2024-05-14T16:54:41.207841+0000","last_undegraded":"2024-05-14T16:54:41.207841+0000","last_fullsized":"2024-05-14T16:54:41.207841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:44:53.393371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206506+0000","last_change":"2024-05-14T16:51:29.281181+0000","last_active":"2024-05-14T16:54:41.206506+0000","last_peered":"2024-05-14T16:54:41.206506+0000","last_clean":"2024-05-14T16:54:41.206506+0000","last_became_active":"2024-05-14T16:51:29.279965+0000","last_became_peered":"2024-05-14T16:51:29.279965+0000","last_unstale":"2024-05-14T16:54:41.206506+0000","last_undegraded":"2024-05-14T16:54:41.206506+0000","last_fullsized":"2024-05-14T16:54:41.206506+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:41:00.735291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206981+0000","last_change":"2024-05-14T16:51:32.296971+0000","last_active":"2024-05-14T16:54:41.206981+0000","last_peered":"2024-05-14T16:54:41.206981+0000","last_clean":"2024-05-14T16:54:41.206981+0000","last_became_active":"2024-05-14T16:51:32.296600+0000","last_became_peered":"2024-05-14T16:51:32.296600+0000","last_unstale":"2024-05-14T16:54:41.206981+0000","last_undegraded":"2024-05-14T16:54:41.206981+0000","last_fullsized":"2024-05-14T16:54:41.206981+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:13:14.354127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207310+0000","last_change":"2024-05-14T16:51:29.412682+0000","last_active":"2024-05-14T16:54:41.207310+0000","last_peered":"2024-05-14T16:54:41.207310+0000","last_clean":"2024-05-14T16:54:41.207310+0000","last_became_active":"2024-05-14T16:51:29.412186+0000","last_became_peered":"2024-05-14T16:51:29.412186+0000","last_unstale":"2024-05-14T16:54:41.207310+0000","last_undegraded":"2024-05-14T16:54:41.207310+0000","last_fullsized":"2024-05-14T16:54:41.207310+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:56:24.570276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207683+0000","last_change":"2024-05-14T16:51:32.300626+0000","last_active":"2024-05-14T16:54:41.207683+0000","last_peered":"2024-05-14T16:54:41.207683+0000","last_clean":"2024-05-14T16:54:41.207683+0000","last_became_active":"2024-05-14T16:51:32.300278+0000","last_became_peered":"2024-05-14T16:51:32.300278+0000","last_unstale":"2024-05-14T16:54:41.207683+0000","last_undegraded":"2024-05-14T16:54:41.207683+0000","last_fullsized":"2024-05-14T16:54:41.207683+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:59:14.196315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'39","reported_seq":76,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.415378+0000","last_change":"2024-05-14T16:51:29.270826+0000","last_active":"2024-05-14T16:54:41.415378+0000","last_peered":"2024-05-14T16:54:41.415378+0000","last_clean":"2024-05-14T16:54:41.415378+0000","last_became_active":"2024-05-14T16:51:29.270266+0000","last_became_peered":"2024-05-14T16:51:29.270266+0000","last_unstale":"2024-05-14T16:54:41.415378+0000","last_undegraded":"2024-05-14T16:54:41.415378+0000","last_fullsized":"2024-05-14T16:54:41.415378+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:00:55.349278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207426+0000","last_change":"2024-05-14T16:52:13.223994+0000","last_active":"2024-05-14T16:54:41.207426+0000","last_peered":"2024-05-14T16:54:41.207426+0000","last_clean":"2024-05-14T16:54:41.207426+0000","last_became_active":"2024-05-14T16:52:13.221071+0000","last_became_peered":"2024-05-14T16:52:13.221071+0000","last_unstale":"2024-05-14T16:54:41.207426+0000","last_undegraded":"2024-05-14T16:54:41.207426+0000","last_fullsized":"2024-05-14T16:54:41.207426+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:29:58.775171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.192788+0000","last_change":"2024-05-14T16:51:29.276705+0000","last_active":"2024-05-14T16:54:41.192788+0000","last_peered":"2024-05-14T16:54:41.192788+0000","last_clean":"2024-05-14T16:54:41.192788+0000","last_became_active":"2024-05-14T16:51:29.276347+0000","last_became_peered":"2024-05-14T16:51:29.276347+0000","last_unstale":"2024-05-14T16:54:41.192788+0000","last_undegraded":"2024-05-14T16:54:41.192788+0000","last_fullsized":"2024-05-14T16:54:41.192788+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:32:56.064427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207670+0000","last_change":"2024-05-14T16:51:32.301617+0000","last_active":"2024-05-14T16:54:41.207670+0000","last_peered":"2024-05-14T16:54:41.207670+0000","last_clean":"2024-05-14T16:54:41.207670+0000","last_became_active":"2024-05-14T16:51:32.301288+0000","last_became_peered":"2024-05-14T16:51:32.301288+0000","last_unstale":"2024-05-14T16:54:41.207670+0000","last_undegraded":"2024-05-14T16:54:41.207670+0000","last_fullsized":"2024-05-14T16:54:41.207670+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:19:24.253633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.197028+0000","last_change":"2024-05-14T16:52:13.221694+0000","last_active":"2024-05-14T16:54:41.197028+0000","last_peered":"2024-05-14T16:54:41.197028+0000","last_clean":"2024-05-14T16:54:41.197028+0000","last_became_active":"2024-05-14T16:52:13.221264+0000","last_became_peered":"2024-05-14T16:52:13.221264+0000","last_unstale":"2024-05-14T16:54:41.197028+0000","last_undegraded":"2024-05-14T16:54:41.197028+0000","last_fullsized":"2024-05-14T16:54:41.197028+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:22:52.738772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206531+0000","last_change":"2024-05-14T16:51:32.301789+0000","last_active":"2024-05-14T16:54:41.206531+0000","last_peered":"2024-05-14T16:54:41.206531+0000","last_clean":"2024-05-14T16:54:41.206531+0000","last_became_active":"2024-05-14T16:51:32.301350+0000","last_became_peered":"2024-05-14T16:51:32.301350+0000","last_unstale":"2024-05-14T16:54:41.206531+0000","last_undegraded":"2024-05-14T16:54:41.206531+0000","last_fullsized":"2024-05-14T16:54:41.206531+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:46:24.896855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206730+0000","last_change":"2024-05-14T16:51:32.295978+0000","last_active":"2024-05-14T16:54:41.206730+0000","last_peered":"2024-05-14T16:54:41.206730+0000","last_clean":"2024-05-14T16:54:41.206730+0000","last_became_active":"2024-05-14T16:51:32.295652+0000","last_became_peered":"2024-05-14T16:51:32.295652+0000","last_unstale":"2024-05-14T16:54:41.206730+0000","last_undegraded":"2024-05-14T16:54:41.206730+0000","last_fullsized":"2024-05-14T16:54:41.206730+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:32:17.743734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206139+0000","last_change":"2024-05-14T16:51:32.305568+0000","last_active":"2024-05-14T16:54:41.206139+0000","last_peered":"2024-05-14T16:54:41.206139+0000","last_clean":"2024-05-14T16:54:41.206139+0000","last_became_active":"2024-05-14T16:51:32.304580+0000","last_became_peered":"2024-05-14T16:51:32.304580+0000","last_unstale":"2024-05-14T16:54:41.206139+0000","last_undegraded":"2024-05-14T16:54:41.206139+0000","last_fullsized":"2024-05-14T16:54:41.206139+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:32:44.465054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.192870+0000","last_change":"2024-05-14T16:51:32.295029+0000","last_active":"2024-05-14T16:54:41.192870+0000","last_peered":"2024-05-14T16:54:41.192870+0000","last_clean":"2024-05-14T16:54:41.192870+0000","last_became_active":"2024-05-14T16:51:32.294621+0000","last_became_peered":"2024-05-14T16:51:32.294621+0000","last_unstale":"2024-05-14T16:54:41.192870+0000","last_undegraded":"2024-05-14T16:54:41.192870+0000","last_fullsized":"2024-05-14T16:54:41.192870+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:52:05.390141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.196972+0000","last_change":"2024-05-14T16:51:32.297671+0000","last_active":"2024-05-14T16:54:41.196972+0000","last_peered":"2024-05-14T16:54:41.196972+0000","last_clean":"2024-05-14T16:54:41.196972+0000","last_became_active":"2024-05-14T16:51:32.296210+0000","last_became_peered":"2024-05-14T16:51:32.296210+0000","last_unstale":"2024-05-14T16:54:41.196972+0000","last_undegraded":"2024-05-14T16:54:41.196972+0000","last_fullsized":"2024-05-14T16:54:41.196972+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:55:04.789021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207532+0000","last_change":"2024-05-14T16:51:32.297404+0000","last_active":"2024-05-14T16:54:41.207532+0000","last_peered":"2024-05-14T16:54:41.207532+0000","last_clean":"2024-05-14T16:54:41.207532+0000","last_became_active":"2024-05-14T16:51:32.296856+0000","last_became_peered":"2024-05-14T16:51:32.296856+0000","last_unstale":"2024-05-14T16:54:41.207532+0000","last_undegraded":"2024-05-14T16:54:41.207532+0000","last_fullsized":"2024-05-14T16:54:41.207532+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:25:29.992875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207395+0000","last_change":"2024-05-14T16:52:13.221379+0000","last_active":"2024-05-14T16:54:41.207395+0000","last_peered":"2024-05-14T16:54:41.207395+0000","last_clean":"2024-05-14T16:54:41.207395+0000","last_became_active":"2024-05-14T16:52:13.220973+0000","last_became_peered":"2024-05-14T16:52:13.220973+0000","last_unstale":"2024-05-14T16:54:41.207395+0000","last_undegraded":"2024-05-14T16:54:41.207395+0000","last_fullsized":"2024-05-14T16:54:41.207395+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:55:08.212344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206477+0000","last_change":"2024-05-14T16:51:32.303570+0000","last_active":"2024-05-14T16:54:41.206477+0000","last_peered":"2024-05-14T16:54:41.206477+0000","last_clean":"2024-05-14T16:54:41.206477+0000","last_became_active":"2024-05-14T16:51:32.303234+0000","last_became_peered":"2024-05-14T16:51:32.303234+0000","last_unstale":"2024-05-14T16:54:41.206477+0000","last_undegraded":"2024-05-14T16:54:41.206477+0000","last_fullsized":"2024-05-14T16:54:41.206477+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:27:57.960120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206835+0000","last_change":"2024-05-14T16:51:32.298028+0000","last_active":"2024-05-14T16:54:41.206835+0000","last_peered":"2024-05-14T16:54:41.206835+0000","last_clean":"2024-05-14T16:54:41.206835+0000","last_became_active":"2024-05-14T16:51:32.295104+0000","last_became_peered":"2024-05-14T16:51:32.295104+0000","last_unstale":"2024-05-14T16:54:41.206835+0000","last_undegraded":"2024-05-14T16:54:41.206835+0000","last_fullsized":"2024-05-14T16:54:41.206835+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:58:08.441590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206745+0000","last_change":"2024-05-14T16:51:32.304458+0000","last_active":"2024-05-14T16:54:41.206745+0000","last_peered":"2024-05-14T16:54:41.206745+0000","last_clean":"2024-05-14T16:54:41.206745+0000","last_became_active":"2024-05-14T16:51:32.302952+0000","last_became_peered":"2024-05-14T16:51:32.302952+0000","last_unstale":"2024-05-14T16:54:41.206745+0000","last_undegraded":"2024-05-14T16:54:41.206745+0000","last_fullsized":"2024-05-14T16:54:41.206745+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:40:14.220075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.192762+0000","last_change":"2024-05-14T16:51:32.292334+0000","last_active":"2024-05-14T16:54:41.192762+0000","last_peered":"2024-05-14T16:54:41.192762+0000","last_clean":"2024-05-14T16:54:41.192762+0000","last_became_active":"2024-05-14T16:51:32.291769+0000","last_became_peered":"2024-05-14T16:51:32.291769+0000","last_unstale":"2024-05-14T16:54:41.192762+0000","last_undegraded":"2024-05-14T16:54:41.192762+0000","last_fullsized":"2024-05-14T16:54:41.192762+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:55:25.694827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.196919+0000","last_change":"2024-05-14T16:51:32.300044+0000","last_active":"2024-05-14T16:54:41.196919+0000","last_peered":"2024-05-14T16:54:41.196919+0000","last_clean":"2024-05-14T16:54:41.196919+0000","last_became_active":"2024-05-14T16:51:32.299630+0000","last_became_peered":"2024-05-14T16:51:32.299630+0000","last_unstale":"2024-05-14T16:54:41.196919+0000","last_undegraded":"2024-05-14T16:54:41.196919+0000","last_fullsized":"2024-05-14T16:54:41.196919+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:50:40.516973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108486,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597333,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1151,"num_read_kb":683,"num_write":296,"num_write_kb":1440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":1791999,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":231,"ondisk_log_size":231,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710773,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":901120,"data_stored":827273,"data_compressed":22065,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907002,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1056768,"data_stored":1225122,"data_compressed":45237,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070527,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":827626,"data_compressed":22065,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266757,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":892928,"data_stored":434213,"data_compressed":2919,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462985,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":942080,"data_stored":1027765,"data_compressed":39081,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626511,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36200,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708952,"statfs":{"total":95995035648,"available":95957966848,"internally_reserved":0,"allocated":880640,"data_stored":433682,"data_compressed":2919,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574952,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":500369,"data_compressed":2919,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738476,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36220,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708932,"statfs":{"total":95995035648,"available":95957946368,"internally_reserved":0,"allocated":901120,"data_stored":434953,"data_compressed":2919,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411428,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":970752,"data_stored":501469,"data_compressed":2919,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182494,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":937984,"data_stored":958692,"data_compressed":28221,"data_compressed_allocated":36864,"data_compressed_original":524288,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986266,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":884736,"data_stored":434427,"data_compressed":2919,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822741,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":1157959,"data_compressed":45237,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":133999,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":201162,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":393798,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T16:54:49.635 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-14T16:54:49.635 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-14T16:54:49.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:49 smithi151 ceph-mon[36066]: Updating smithi123:/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T16:54:49.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:49 smithi151 ceph-mon[36066]: Updating smithi151:/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T16:54:49.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:49 smithi151 ceph-mon[36066]: Updating smithi002:/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T16:54:49.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:49 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:49.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:49 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:49.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:49 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:49.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:49 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:54:49.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:49 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:49.888 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:54:49.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:49 smithi123 ceph-mon[35971]: Updating smithi123:/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T16:54:49.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:49 smithi123 ceph-mon[35971]: Updating smithi151:/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T16:54:49.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:49 smithi123 ceph-mon[35971]: Updating smithi002:/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-14T16:54:49.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:49 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:49.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:49 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:49.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:49 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:54:49.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:49 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:54:49.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:49 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:54:50.725 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:50 smithi002 ceph-mon[31538]: from='client.15231 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:54:50.725 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:50 smithi002 ceph-mon[31538]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 131 KiB/s wr, 69 op/s 2024-05-14T16:54:50.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:50 smithi151 ceph-mon[36066]: from='client.15231 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:54:50.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:50 smithi151 ceph-mon[36066]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 131 KiB/s wr, 69 op/s 2024-05-14T16:54:50.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:50 smithi123 ceph-mon[35971]: from='client.15231 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:54:50.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:50 smithi123 ceph-mon[35971]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 131 KiB/s wr, 69 op/s 2024-05-14T16:54:51.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.700+0000 7f546713c700 1 -- 172.21.15.2:0/2413967158 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5460101ee0 msgr2=0x7f5460102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:51.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.700+0000 7f546713c700 1 --2- 172.21.15.2:0/2413967158 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5460101ee0 0x7f5460102360 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5448009990 tx=0x7f5448009ca0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.700+0000 7f546713c700 1 -- 172.21.15.2:0/2413967158 shutdown_connections 2024-05-14T16:54:51.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.700+0000 7f546713c700 1 --2- 172.21.15.2:0/2413967158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54601028a0 0x7f5460110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.700+0000 7f546713c700 1 --2- 172.21.15.2:0/2413967158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5460108600 0x7f54601089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.700+0000 7f546713c700 1 --2- 172.21.15.2:0/2413967158 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5460101ee0 0x7f5460102360 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.700+0000 7f546713c700 1 -- 172.21.15.2:0/2413967158 >> 172.21.15.2:0/2413967158 conn(0x7f5460077760 msgr2=0x7f5460077b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:51.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.700+0000 7f546713c700 1 -- 172.21.15.2:0/2413967158 shutdown_connections 2024-05-14T16:54:51.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.700+0000 7f546713c700 1 -- 172.21.15.2:0/2413967158 wait complete. 2024-05-14T16:54:51.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.701+0000 7f546713c700 1 Processor -- start 2024-05-14T16:54:51.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.701+0000 7f546713c700 1 -- start start 2024-05-14T16:54:51.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.701+0000 7f546713c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5460101ee0 0x7f54601030c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:51.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.701+0000 7f546713c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54601028a0 0x7f5460103600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:51.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.701+0000 7f546713c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5460108600 0x7f5460103b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:51.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.701+0000 7f546713c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5460113790 con 0x7f5460101ee0 2024-05-14T16:54:51.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.701+0000 7f546713c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5460113610 con 0x7f54601028a0 2024-05-14T16:54:51.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.701+0000 7f546713c700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5460113910 con 0x7f5460108600 2024-05-14T16:54:51.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.701+0000 7f5464ed8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5460101ee0 0x7f54601030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:51.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.701+0000 7f5464ed8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5460101ee0 0x7f54601030c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57114/0 (socket says 172.21.15.2:57114) 2024-05-14T16:54:51.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.701+0000 7f5464ed8700 1 -- 172.21.15.2:0/1358853479 learned_addr learned my addr 172.21.15.2:0/1358853479 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:51.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.701+0000 7f545ffff700 1 --2- 172.21.15.2:0/1358853479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54601028a0 0x7f5460103600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:51.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.701+0000 7f54656d9700 1 --2- 172.21.15.2:0/1358853479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5460108600 0x7f5460103b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:51.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.701+0000 7f5464ed8700 1 -- 172.21.15.2:0/1358853479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5460108600 msgr2=0x7f5460103b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:51.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.701+0000 7f5464ed8700 1 --2- 172.21.15.2:0/1358853479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5460108600 0x7f5460103b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.701+0000 7f5464ed8700 1 -- 172.21.15.2:0/1358853479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54601028a0 msgr2=0x7f5460103600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:51.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.701+0000 7f5464ed8700 1 --2- 172.21.15.2:0/1358853479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54601028a0 0x7f5460103600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.701+0000 7f5464ed8700 1 -- 172.21.15.2:0/1358853479 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54601a8160 con 0x7f5460101ee0 2024-05-14T16:54:51.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.702+0000 7f5464ed8700 1 --2- 172.21.15.2:0/1358853479 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5460101ee0 0x7f54601030c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5450007f10 tx=0x7f545000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:51.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.702+0000 7f545dffb700 1 -- 172.21.15.2:0/1358853479 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545000dd20 con 0x7f5460101ee0 2024-05-14T16:54:51.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.702+0000 7f546713c700 1 -- 172.21.15.2:0/1358853479 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54601a8450 con 0x7f5460101ee0 2024-05-14T16:54:51.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.702+0000 7f545dffb700 1 -- 172.21.15.2:0/1358853479 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5450018070 con 0x7f5460101ee0 2024-05-14T16:54:51.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.702+0000 7f545dffb700 1 -- 172.21.15.2:0/1358853479 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54500125b0 con 0x7f5460101ee0 2024-05-14T16:54:51.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.702+0000 7f546713c700 1 -- 172.21.15.2:0/1358853479 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54601a8960 con 0x7f5460101ee0 2024-05-14T16:54:51.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.703+0000 7f546713c700 1 -- 172.21.15.2:0/1358853479 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f546006a230 con 0x7f5460101ee0 2024-05-14T16:54:51.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.704+0000 7f545dffb700 1 -- 172.21.15.2:0/1358853479 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 24) ==== 149827+0+0 (secure 0 0 0) 0x7f54500133f0 con 0x7f5460101ee0 2024-05-14T16:54:51.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.705+0000 7f545dffb700 1 --2- 172.21.15.2:0/1358853479 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f544c0b2a80 0x7f544c0b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:51.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.705+0000 7f545dffb700 1 -- 172.21.15.2:0/1358853479 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f54500e2490 con 0x7f5460101ee0 2024-05-14T16:54:51.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.705+0000 7f545ffff700 1 --2- 172.21.15.2:0/1358853479 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f544c0b2a80 0x7f544c0b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:51.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.706+0000 7f545ffff700 1 --2- 172.21.15.2:0/1358853479 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f544c0b2a80 0x7f544c0b4f40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f544800b5c0 tx=0x7f5448005960 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:51.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.710+0000 7f545dffb700 1 -- 172.21.15.2:0/1358853479 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5450010070 con 0x7f5460101ee0 2024-05-14T16:54:51.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.925+0000 7f546713c700 1 -- 172.21.15.2:0/1358853479 --> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5460063980 con 0x7f544c0b2a80 2024-05-14T16:54:51.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.933+0000 7f545dffb700 1 -- 172.21.15.2:0/1358853479 <== mgr.15120 v2:172.21.15.2:6800/1339485805 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258026 (secure 0 0 0) 0x7f5460063980 con 0x7f544c0b2a80 2024-05-14T16:54:51.934 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:54:51.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.936+0000 7f546713c700 1 -- 172.21.15.2:0/1358853479 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f544c0b2a80 msgr2=0x7f544c0b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:51.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.936+0000 7f546713c700 1 --2- 172.21.15.2:0/1358853479 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f544c0b2a80 0x7f544c0b4f40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f544800b5c0 tx=0x7f5448005960 comp rx=0 tx=0).stop 2024-05-14T16:54:51.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.936+0000 7f546713c700 1 -- 172.21.15.2:0/1358853479 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5460101ee0 msgr2=0x7f54601030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:51.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.936+0000 7f546713c700 1 --2- 172.21.15.2:0/1358853479 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5460101ee0 0x7f54601030c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5450007f10 tx=0x7f545000d3b0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.936+0000 7f546713c700 1 -- 172.21.15.2:0/1358853479 shutdown_connections 2024-05-14T16:54:51.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.936+0000 7f546713c700 1 --2- 172.21.15.2:0/1358853479 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f544c0b2a80 0x7f544c0b4f40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.936+0000 7f546713c700 1 --2- 172.21.15.2:0/1358853479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5460108600 0x7f5460103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.936+0000 7f546713c700 1 --2- 172.21.15.2:0/1358853479 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5460101ee0 0x7f54601030c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.936+0000 7f546713c700 1 --2- 172.21.15.2:0/1358853479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54601028a0 0x7f5460103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:51.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.936+0000 7f546713c700 1 -- 172.21.15.2:0/1358853479 >> 172.21.15.2:0/1358853479 conn(0x7f5460077760 msgr2=0x7f5460100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:51.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.936+0000 7f546713c700 1 -- 172.21.15.2:0/1358853479 shutdown_connections 2024-05-14T16:54:51.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:51.936+0000 7f546713c700 1 -- 172.21.15.2:0/1358853479 wait complete. 2024-05-14T16:54:51.939 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-05-14T16:54:52.554 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-05-14T16:54:51.209636+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1384567,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1581,"num_read_kb":1179,"num_write":561,"num_write_kb":4507,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":2951957,"num_keys_recovered":9,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":912,"ondisk_log_size":912,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":385184,"kb_used_data":11028,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124556640,"statfs":{"total":1151940427776,"available":1151545999360,"internally_reserved":0,"allocated":11292672,"data_stored":9355401,"data_compressed":246987,"data_compressed_allocated":319488,"data_compressed_original":3969024,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":349,"num_read_kb":427,"num_write":140,"num_write_kb":915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.990733"},"pg_stats":[{"pgid":"3.3f","version":"80'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.197029+0000","last_change":"2024-05-14T16:51:32.298580+0000","last_active":"2024-05-14T16:54:41.197029+0000","last_peered":"2024-05-14T16:54:41.197029+0000","last_clean":"2024-05-14T16:54:41.197029+0000","last_became_active":"2024-05-14T16:51:32.296945+0000","last_became_peered":"2024-05-14T16:51:32.296945+0000","last_unstale":"2024-05-14T16:54:41.197029+0000","last_undegraded":"2024-05-14T16:54:41.197029+0000","last_fullsized":"2024-05-14T16:54:41.197029+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:50:17.664928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"80'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206588+0000","last_change":"2024-05-14T16:51:32.297618+0000","last_active":"2024-05-14T16:54:41.206588+0000","last_peered":"2024-05-14T16:54:41.206588+0000","last_clean":"2024-05-14T16:54:41.206588+0000","last_became_active":"2024-05-14T16:51:32.297260+0000","last_became_peered":"2024-05-14T16:51:32.297260+0000","last_unstale":"2024-05-14T16:54:41.206588+0000","last_undegraded":"2024-05-14T16:54:41.206588+0000","last_fullsized":"2024-05-14T16:54:41.206588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:43:50.542587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"80'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206503+0000","last_change":"2024-05-14T16:51:32.295525+0000","last_active":"2024-05-14T16:54:41.206503+0000","last_peered":"2024-05-14T16:54:41.206503+0000","last_clean":"2024-05-14T16:54:41.206503+0000","last_became_active":"2024-05-14T16:51:32.295215+0000","last_became_peered":"2024-05-14T16:51:32.295215+0000","last_unstale":"2024-05-14T16:54:41.206503+0000","last_undegraded":"2024-05-14T16:54:41.206503+0000","last_fullsized":"2024-05-14T16:54:41.206503+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:40:05.516280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"80'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207077+0000","last_change":"2024-05-14T16:51:32.307883+0000","last_active":"2024-05-14T16:54:41.207077+0000","last_peered":"2024-05-14T16:54:41.207077+0000","last_clean":"2024-05-14T16:54:41.207077+0000","last_became_active":"2024-05-14T16:51:32.307499+0000","last_became_peered":"2024-05-14T16:51:32.307499+0000","last_unstale":"2024-05-14T16:54:41.207077+0000","last_undegraded":"2024-05-14T16:54:41.207077+0000","last_fullsized":"2024-05-14T16:54:41.207077+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:30:59.978059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"80'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206362+0000","last_change":"2024-05-14T16:51:32.294429+0000","last_active":"2024-05-14T16:54:41.206362+0000","last_peered":"2024-05-14T16:54:41.206362+0000","last_clean":"2024-05-14T16:54:41.206362+0000","last_became_active":"2024-05-14T16:51:32.294088+0000","last_became_peered":"2024-05-14T16:51:32.294088+0000","last_unstale":"2024-05-14T16:54:41.206362+0000","last_undegraded":"2024-05-14T16:54:41.206362+0000","last_fullsized":"2024-05-14T16:54:41.206362+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:25:19.383586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"80'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.194143+0000","last_change":"2024-05-14T16:51:32.296169+0000","last_active":"2024-05-14T16:54:41.194143+0000","last_peered":"2024-05-14T16:54:41.194143+0000","last_clean":"2024-05-14T16:54:41.194143+0000","last_became_active":"2024-05-14T16:51:32.295819+0000","last_became_peered":"2024-05-14T16:51:32.295819+0000","last_unstale":"2024-05-14T16:54:41.194143+0000","last_undegraded":"2024-05-14T16:54:41.194143+0000","last_fullsized":"2024-05-14T16:54:41.194143+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:56:23.932039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"80'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206285+0000","last_change":"2024-05-14T16:51:32.309609+0000","last_active":"2024-05-14T16:54:41.206285+0000","last_peered":"2024-05-14T16:54:41.206285+0000","last_clean":"2024-05-14T16:54:41.206285+0000","last_became_active":"2024-05-14T16:51:32.309175+0000","last_became_peered":"2024-05-14T16:51:32.309175+0000","last_unstale":"2024-05-14T16:54:41.206285+0000","last_undegraded":"2024-05-14T16:54:41.206285+0000","last_fullsized":"2024-05-14T16:54:41.206285+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:06:46.717592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'9","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207138+0000","last_change":"2024-05-14T16:52:13.446592+0000","last_active":"2024-05-14T16:54:41.207138+0000","last_peered":"2024-05-14T16:54:41.207138+0000","last_clean":"2024-05-14T16:54:41.207138+0000","last_became_active":"2024-05-14T16:52:13.446238+0000","last_became_peered":"2024-05-14T16:52:13.446238+0000","last_unstale":"2024-05-14T16:54:41.207138+0000","last_undegraded":"2024-05-14T16:54:41.207138+0000","last_fullsized":"2024-05-14T16:54:41.207138+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:33:14.216797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"80'11","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206922+0000","last_change":"2024-05-14T16:51:32.307571+0000","last_active":"2024-05-14T16:54:41.206922+0000","last_peered":"2024-05-14T16:54:41.206922+0000","last_clean":"2024-05-14T16:54:41.206922+0000","last_became_active":"2024-05-14T16:51:32.307151+0000","last_became_peered":"2024-05-14T16:51:32.307151+0000","last_unstale":"2024-05-14T16:54:41.206922+0000","last_undegraded":"2024-05-14T16:54:41.206922+0000","last_fullsized":"2024-05-14T16:54:41.206922+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:17:26.325393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"80'16","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.194394+0000","last_change":"2024-05-14T16:51:32.298773+0000","last_active":"2024-05-14T16:54:41.194394+0000","last_peered":"2024-05-14T16:54:41.194394+0000","last_clean":"2024-05-14T16:54:41.194394+0000","last_became_active":"2024-05-14T16:51:32.298283+0000","last_became_peered":"2024-05-14T16:51:32.298283+0000","last_unstale":"2024-05-14T16:54:41.194394+0000","last_undegraded":"2024-05-14T16:54:41.194394+0000","last_fullsized":"2024-05-14T16:54:41.194394+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:52:18.471201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"80'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207583+0000","last_change":"2024-05-14T16:51:32.302443+0000","last_active":"2024-05-14T16:54:41.207583+0000","last_peered":"2024-05-14T16:54:41.207583+0000","last_clean":"2024-05-14T16:54:41.207583+0000","last_became_active":"2024-05-14T16:51:32.302034+0000","last_became_peered":"2024-05-14T16:51:32.302034+0000","last_unstale":"2024-05-14T16:54:41.207583+0000","last_undegraded":"2024-05-14T16:54:41.207583+0000","last_fullsized":"2024-05-14T16:54:41.207583+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:53:08.041293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"80'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207252+0000","last_change":"2024-05-14T16:51:32.304837+0000","last_active":"2024-05-14T16:54:41.207252+0000","last_peered":"2024-05-14T16:54:41.207252+0000","last_clean":"2024-05-14T16:54:41.207252+0000","last_became_active":"2024-05-14T16:51:32.304500+0000","last_became_peered":"2024-05-14T16:51:32.304500+0000","last_unstale":"2024-05-14T16:54:41.207252+0000","last_undegraded":"2024-05-14T16:54:41.207252+0000","last_fullsized":"2024-05-14T16:54:41.207252+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:32:02.713044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'10","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.192789+0000","last_change":"2024-05-14T16:51:32.299247+0000","last_active":"2024-05-14T16:54:41.192789+0000","last_peered":"2024-05-14T16:54:41.192789+0000","last_clean":"2024-05-14T16:54:41.192789+0000","last_became_active":"2024-05-14T16:51:32.298925+0000","last_became_peered":"2024-05-14T16:51:32.298925+0000","last_unstale":"2024-05-14T16:54:41.192789+0000","last_undegraded":"2024-05-14T16:54:41.192789+0000","last_fullsized":"2024-05-14T16:54:41.192789+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:10:38.847877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"80'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.205900+0000","last_change":"2024-05-14T16:51:32.293743+0000","last_active":"2024-05-14T16:54:41.205900+0000","last_peered":"2024-05-14T16:54:41.205900+0000","last_clean":"2024-05-14T16:54:41.205900+0000","last_became_active":"2024-05-14T16:51:32.293429+0000","last_became_peered":"2024-05-14T16:51:32.293429+0000","last_unstale":"2024-05-14T16:54:41.205900+0000","last_undegraded":"2024-05-14T16:54:41.205900+0000","last_fullsized":"2024-05-14T16:54:41.205900+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:12:33.909940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"80'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.205749+0000","last_change":"2024-05-14T16:51:32.304940+0000","last_active":"2024-05-14T16:54:41.205749+0000","last_peered":"2024-05-14T16:54:41.205749+0000","last_clean":"2024-05-14T16:54:41.205749+0000","last_became_active":"2024-05-14T16:51:32.304482+0000","last_became_peered":"2024-05-14T16:51:32.304482+0000","last_unstale":"2024-05-14T16:54:41.205749+0000","last_undegraded":"2024-05-14T16:54:41.205749+0000","last_fullsized":"2024-05-14T16:54:41.205749+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:54:17.771620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"80'9","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206201+0000","last_change":"2024-05-14T16:51:32.301784+0000","last_active":"2024-05-14T16:54:41.206201+0000","last_peered":"2024-05-14T16:54:41.206201+0000","last_clean":"2024-05-14T16:54:41.206201+0000","last_became_active":"2024-05-14T16:51:32.301472+0000","last_became_peered":"2024-05-14T16:51:32.301472+0000","last_unstale":"2024-05-14T16:54:41.206201+0000","last_undegraded":"2024-05-14T16:54:41.206201+0000","last_fullsized":"2024-05-14T16:54:41.206201+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:10:56.735424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"80'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.197151+0000","last_change":"2024-05-14T16:51:32.297951+0000","last_active":"2024-05-14T16:54:41.197151+0000","last_peered":"2024-05-14T16:54:41.197151+0000","last_clean":"2024-05-14T16:54:41.197151+0000","last_became_active":"2024-05-14T16:51:32.296602+0000","last_became_peered":"2024-05-14T16:51:32.296602+0000","last_unstale":"2024-05-14T16:54:41.197151+0000","last_undegraded":"2024-05-14T16:54:41.197151+0000","last_fullsized":"2024-05-14T16:54:41.197151+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:16:36.039077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"80'10","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.208451+0000","last_change":"2024-05-14T16:51:32.290398+0000","last_active":"2024-05-14T16:54:41.208451+0000","last_peered":"2024-05-14T16:54:41.208451+0000","last_clean":"2024-05-14T16:54:41.208451+0000","last_became_active":"2024-05-14T16:51:32.290083+0000","last_became_peered":"2024-05-14T16:51:32.290083+0000","last_unstale":"2024-05-14T16:54:41.208451+0000","last_undegraded":"2024-05-14T16:54:41.208451+0000","last_fullsized":"2024-05-14T16:54:41.208451+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:48:05.795209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"80'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207421+0000","last_change":"2024-05-14T16:51:32.294398+0000","last_active":"2024-05-14T16:54:41.207421+0000","last_peered":"2024-05-14T16:54:41.207421+0000","last_clean":"2024-05-14T16:54:41.207421+0000","last_became_active":"2024-05-14T16:51:32.294080+0000","last_became_peered":"2024-05-14T16:51:32.294080+0000","last_unstale":"2024-05-14T16:54:41.207421+0000","last_undegraded":"2024-05-14T16:54:41.207421+0000","last_fullsized":"2024-05-14T16:54:41.207421+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:15:03.859045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.208336+0000","last_change":"2024-05-14T16:51:29.274482+0000","last_active":"2024-05-14T16:54:41.208336+0000","last_peered":"2024-05-14T16:54:41.208336+0000","last_clean":"2024-05-14T16:54:41.208336+0000","last_became_active":"2024-05-14T16:51:29.274152+0000","last_became_peered":"2024-05-14T16:51:29.274152+0000","last_unstale":"2024-05-14T16:54:41.208336+0000","last_undegraded":"2024-05-14T16:54:41.208336+0000","last_fullsized":"2024-05-14T16:54:41.208336+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:05:38.274248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"80'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207141+0000","last_change":"2024-05-14T16:51:32.300956+0000","last_active":"2024-05-14T16:54:41.207141+0000","last_peered":"2024-05-14T16:54:41.207141+0000","last_clean":"2024-05-14T16:54:41.207141+0000","last_became_active":"2024-05-14T16:51:32.300546+0000","last_became_peered":"2024-05-14T16:51:32.300546+0000","last_unstale":"2024-05-14T16:54:41.207141+0000","last_undegraded":"2024-05-14T16:54:41.207141+0000","last_fullsized":"2024-05-14T16:54:41.207141+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T16:54:09.181036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207390+0000","last_change":"2024-05-14T16:51:29.345206+0000","last_active":"2024-05-14T16:54:41.207390+0000","last_peered":"2024-05-14T16:54:41.207390+0000","last_clean":"2024-05-14T16:54:41.207390+0000","last_became_active":"2024-05-14T16:51:29.344699+0000","last_became_peered":"2024-05-14T16:51:29.344699+0000","last_unstale":"2024-05-14T16:54:41.207390+0000","last_undegraded":"2024-05-14T16:54:41.207390+0000","last_fullsized":"2024-05-14T16:54:41.207390+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:14:02.364661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"80'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206815+0000","last_change":"2024-05-14T16:51:32.297522+0000","last_active":"2024-05-14T16:54:41.206815+0000","last_peered":"2024-05-14T16:54:41.206815+0000","last_clean":"2024-05-14T16:54:41.206815+0000","last_became_active":"2024-05-14T16:51:32.296866+0000","last_became_peered":"2024-05-14T16:51:32.296866+0000","last_unstale":"2024-05-14T16:54:41.206815+0000","last_undegraded":"2024-05-14T16:54:41.206815+0000","last_fullsized":"2024-05-14T16:54:41.206815+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:49:43.409575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207355+0000","last_change":"2024-05-14T16:51:29.280632+0000","last_active":"2024-05-14T16:54:41.207355+0000","last_peered":"2024-05-14T16:54:41.207355+0000","last_clean":"2024-05-14T16:54:41.207355+0000","last_became_active":"2024-05-14T16:51:29.280246+0000","last_became_peered":"2024-05-14T16:51:29.280246+0000","last_unstale":"2024-05-14T16:54:41.207355+0000","last_undegraded":"2024-05-14T16:54:41.207355+0000","last_fullsized":"2024-05-14T16:54:41.207355+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:34:47.925344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"80'9","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206511+0000","last_change":"2024-05-14T16:51:32.298284+0000","last_active":"2024-05-14T16:54:41.206511+0000","last_peered":"2024-05-14T16:54:41.206511+0000","last_clean":"2024-05-14T16:54:41.206511+0000","last_became_active":"2024-05-14T16:51:32.294202+0000","last_became_peered":"2024-05-14T16:51:32.294202+0000","last_unstale":"2024-05-14T16:54:41.206511+0000","last_undegraded":"2024-05-14T16:54:41.206511+0000","last_fullsized":"2024-05-14T16:54:41.206511+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:27:37.561275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206773+0000","last_change":"2024-05-14T16:51:29.276227+0000","last_active":"2024-05-14T16:54:41.206773+0000","last_peered":"2024-05-14T16:54:41.206773+0000","last_clean":"2024-05-14T16:54:41.206773+0000","last_became_active":"2024-05-14T16:51:29.275837+0000","last_became_peered":"2024-05-14T16:51:29.275837+0000","last_unstale":"2024-05-14T16:54:41.206773+0000","last_undegraded":"2024-05-14T16:54:41.206773+0000","last_fullsized":"2024-05-14T16:54:41.206773+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:03:29.402985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"80'15","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207149+0000","last_change":"2024-05-14T16:51:32.308173+0000","last_active":"2024-05-14T16:54:41.207149+0000","last_peered":"2024-05-14T16:54:41.207149+0000","last_clean":"2024-05-14T16:54:41.207149+0000","last_became_active":"2024-05-14T16:51:32.307135+0000","last_became_peered":"2024-05-14T16:51:32.307135+0000","last_unstale":"2024-05-14T16:54:41.207149+0000","last_undegraded":"2024-05-14T16:54:41.207149+0000","last_fullsized":"2024-05-14T16:54:41.207149+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:01:46.579968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'8","reported_seq":218,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:45.277500+0000","last_change":"2024-05-14T16:51:29.279486+0000","last_active":"2024-05-14T16:54:45.277500+0000","last_peered":"2024-05-14T16:54:45.277500+0000","last_clean":"2024-05-14T16:54:45.277500+0000","last_became_active":"2024-05-14T16:51:29.279133+0000","last_became_peered":"2024-05-14T16:51:29.279133+0000","last_unstale":"2024-05-14T16:54:45.277500+0000","last_undegraded":"2024-05-14T16:54:45.277500+0000","last_fullsized":"2024-05-14T16:54:45.277500+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:26:18.620421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":181,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"80'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206811+0000","last_change":"2024-05-14T16:51:32.299852+0000","last_active":"2024-05-14T16:54:41.206811+0000","last_peered":"2024-05-14T16:54:41.206811+0000","last_clean":"2024-05-14T16:54:41.206811+0000","last_became_active":"2024-05-14T16:51:32.299541+0000","last_became_peered":"2024-05-14T16:51:32.299541+0000","last_unstale":"2024-05-14T16:54:41.206811+0000","last_undegraded":"2024-05-14T16:54:41.206811+0000","last_fullsized":"2024-05-14T16:54:41.206811+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:36:38.191009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.194073+0000","last_change":"2024-05-14T16:51:29.275269+0000","last_active":"2024-05-14T16:54:41.194073+0000","last_peered":"2024-05-14T16:54:41.194073+0000","last_clean":"2024-05-14T16:54:41.194073+0000","last_became_active":"2024-05-14T16:51:29.274792+0000","last_became_peered":"2024-05-14T16:51:29.274792+0000","last_unstale":"2024-05-14T16:54:41.194073+0000","last_undegraded":"2024-05-14T16:54:41.194073+0000","last_fullsized":"2024-05-14T16:54:41.194073+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:17:08.925289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"80'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207090+0000","last_change":"2024-05-14T16:51:32.306536+0000","last_active":"2024-05-14T16:54:41.207090+0000","last_peered":"2024-05-14T16:54:41.207090+0000","last_clean":"2024-05-14T16:54:41.207090+0000","last_became_active":"2024-05-14T16:51:32.304976+0000","last_became_peered":"2024-05-14T16:51:32.304976+0000","last_unstale":"2024-05-14T16:54:41.207090+0000","last_undegraded":"2024-05-14T16:54:41.207090+0000","last_fullsized":"2024-05-14T16:54:41.207090+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:48:13.284022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.192664+0000","last_change":"2024-05-14T16:51:29.266783+0000","last_active":"2024-05-14T16:54:41.192664+0000","last_peered":"2024-05-14T16:54:41.192664+0000","last_clean":"2024-05-14T16:54:41.192664+0000","last_became_active":"2024-05-14T16:51:29.266409+0000","last_became_peered":"2024-05-14T16:51:29.266409+0000","last_unstale":"2024-05-14T16:54:41.192664+0000","last_undegraded":"2024-05-14T16:54:41.192664+0000","last_fullsized":"2024-05-14T16:54:41.192664+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:33:14.981709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"80'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206568+0000","last_change":"2024-05-14T16:51:32.289288+0000","last_active":"2024-05-14T16:54:41.206568+0000","last_peered":"2024-05-14T16:54:41.206568+0000","last_clean":"2024-05-14T16:54:41.206568+0000","last_became_active":"2024-05-14T16:51:32.288920+0000","last_became_peered":"2024-05-14T16:51:32.288920+0000","last_unstale":"2024-05-14T16:54:41.206568+0000","last_undegraded":"2024-05-14T16:54:41.206568+0000","last_fullsized":"2024-05-14T16:54:41.206568+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:23:43.659417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206346+0000","last_change":"2024-05-14T16:51:29.273926+0000","last_active":"2024-05-14T16:54:41.206346+0000","last_peered":"2024-05-14T16:54:41.206346+0000","last_clean":"2024-05-14T16:54:41.206346+0000","last_became_active":"2024-05-14T16:51:29.273059+0000","last_became_peered":"2024-05-14T16:51:29.273059+0000","last_unstale":"2024-05-14T16:54:41.206346+0000","last_undegraded":"2024-05-14T16:54:41.206346+0000","last_fullsized":"2024-05-14T16:54:41.206346+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:46:27.800481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"80'11","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206716+0000","last_change":"2024-05-14T16:51:32.287554+0000","last_active":"2024-05-14T16:54:41.206716+0000","last_peered":"2024-05-14T16:54:41.206716+0000","last_clean":"2024-05-14T16:54:41.206716+0000","last_became_active":"2024-05-14T16:51:32.287204+0000","last_became_peered":"2024-05-14T16:51:32.287204+0000","last_unstale":"2024-05-14T16:54:41.206716+0000","last_undegraded":"2024-05-14T16:54:41.206716+0000","last_fullsized":"2024-05-14T16:54:41.206716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:29:40.675378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'5","reported_seq":210,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:45.276759+0000","last_change":"2024-05-14T16:51:29.274819+0000","last_active":"2024-05-14T16:54:45.276759+0000","last_peered":"2024-05-14T16:54:45.276759+0000","last_clean":"2024-05-14T16:54:45.276759+0000","last_became_active":"2024-05-14T16:51:29.274454+0000","last_became_peered":"2024-05-14T16:51:29.274454+0000","last_unstale":"2024-05-14T16:54:45.276759+0000","last_undegraded":"2024-05-14T16:54:45.276759+0000","last_fullsized":"2024-05-14T16:54:45.276759+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:36:30.496064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":176,"num_read_kb":176,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"80'10","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206897+0000","last_change":"2024-05-14T16:51:32.295732+0000","last_active":"2024-05-14T16:54:41.206897+0000","last_peered":"2024-05-14T16:54:41.206897+0000","last_clean":"2024-05-14T16:54:41.206897+0000","last_became_active":"2024-05-14T16:51:32.294919+0000","last_became_peered":"2024-05-14T16:51:32.294919+0000","last_unstale":"2024-05-14T16:54:41.206897+0000","last_undegraded":"2024-05-14T16:54:41.206897+0000","last_fullsized":"2024-05-14T16:54:41.206897+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:08:54.566601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207342+0000","last_change":"2024-05-14T16:51:29.279338+0000","last_active":"2024-05-14T16:54:41.207342+0000","last_peered":"2024-05-14T16:54:41.207342+0000","last_clean":"2024-05-14T16:54:41.207342+0000","last_became_active":"2024-05-14T16:51:29.278928+0000","last_became_peered":"2024-05-14T16:51:29.278928+0000","last_unstale":"2024-05-14T16:54:41.207342+0000","last_undegraded":"2024-05-14T16:54:41.207342+0000","last_fullsized":"2024-05-14T16:54:41.207342+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:24:34.803515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"80'9","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206905+0000","last_change":"2024-05-14T16:51:32.305294+0000","last_active":"2024-05-14T16:54:41.206905+0000","last_peered":"2024-05-14T16:54:41.206905+0000","last_clean":"2024-05-14T16:54:41.206905+0000","last_became_active":"2024-05-14T16:51:32.304563+0000","last_became_peered":"2024-05-14T16:51:32.304563+0000","last_unstale":"2024-05-14T16:54:41.206905+0000","last_undegraded":"2024-05-14T16:54:41.206905+0000","last_fullsized":"2024-05-14T16:54:41.206905+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:56:06.259475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207460+0000","last_change":"2024-05-14T16:51:29.345400+0000","last_active":"2024-05-14T16:54:41.207460+0000","last_peered":"2024-05-14T16:54:41.207460+0000","last_clean":"2024-05-14T16:54:41.207460+0000","last_became_active":"2024-05-14T16:51:29.345010+0000","last_became_peered":"2024-05-14T16:51:29.345010+0000","last_unstale":"2024-05-14T16:54:41.207460+0000","last_undegraded":"2024-05-14T16:54:41.207460+0000","last_fullsized":"2024-05-14T16:54:41.207460+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:24:22.933005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"80'11","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206748+0000","last_change":"2024-05-14T16:52:13.923167+0000","last_active":"2024-05-14T16:54:41.206748+0000","last_peered":"2024-05-14T16:54:41.206748+0000","last_clean":"2024-05-14T16:54:41.206748+0000","last_became_active":"2024-05-14T16:52:13.922813+0000","last_became_peered":"2024-05-14T16:52:13.922813+0000","last_unstale":"2024-05-14T16:54:41.206748+0000","last_undegraded":"2024-05-14T16:54:41.206748+0000","last_fullsized":"2024-05-14T16:54:41.206748+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:50:38.672408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'38","reported_seq":432,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:46.278326+0000","last_change":"2024-05-14T16:51:29.277591+0000","last_active":"2024-05-14T16:54:46.278326+0000","last_peered":"2024-05-14T16:54:46.278326+0000","last_clean":"2024-05-14T16:54:46.278326+0000","last_became_active":"2024-05-14T16:51:29.276612+0000","last_became_peered":"2024-05-14T16:51:29.276612+0000","last_unstale":"2024-05-14T16:54:46.278326+0000","last_undegraded":"2024-05-14T16:54:46.278326+0000","last_fullsized":"2024-05-14T16:54:46.278326+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:22:28.934173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66894,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":365,"num_read_kb":247,"num_write":39,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"80'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207212+0000","last_change":"2024-05-14T16:51:32.302321+0000","last_active":"2024-05-14T16:54:41.207212+0000","last_peered":"2024-05-14T16:54:41.207212+0000","last_clean":"2024-05-14T16:54:41.207212+0000","last_became_active":"2024-05-14T16:51:32.301973+0000","last_became_peered":"2024-05-14T16:51:32.301973+0000","last_unstale":"2024-05-14T16:54:41.207212+0000","last_undegraded":"2024-05-14T16:54:41.207212+0000","last_fullsized":"2024-05-14T16:54:41.207212+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:49:24.054379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207061+0000","last_change":"2024-05-14T16:51:29.278776+0000","last_active":"2024-05-14T16:54:41.207061+0000","last_peered":"2024-05-14T16:54:41.207061+0000","last_clean":"2024-05-14T16:54:41.207061+0000","last_became_active":"2024-05-14T16:51:29.278395+0000","last_became_peered":"2024-05-14T16:51:29.278395+0000","last_unstale":"2024-05-14T16:54:41.207061+0000","last_undegraded":"2024-05-14T16:54:41.207061+0000","last_fullsized":"2024-05-14T16:54:41.207061+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:29:20.942783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"80'9","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.192617+0000","last_change":"2024-05-14T16:51:32.292107+0000","last_active":"2024-05-14T16:54:41.192617+0000","last_peered":"2024-05-14T16:54:41.192617+0000","last_clean":"2024-05-14T16:54:41.192617+0000","last_became_active":"2024-05-14T16:51:32.291729+0000","last_became_peered":"2024-05-14T16:51:32.291729+0000","last_unstale":"2024-05-14T16:54:41.192617+0000","last_undegraded":"2024-05-14T16:54:41.192617+0000","last_fullsized":"2024-05-14T16:54:41.192617+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:30:15.440599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206802+0000","last_change":"2024-05-14T16:51:29.274862+0000","last_active":"2024-05-14T16:54:41.206802+0000","last_peered":"2024-05-14T16:54:41.206802+0000","last_clean":"2024-05-14T16:54:41.206802+0000","last_became_active":"2024-05-14T16:51:29.274527+0000","last_became_peered":"2024-05-14T16:51:29.274527+0000","last_unstale":"2024-05-14T16:54:41.206802+0000","last_undegraded":"2024-05-14T16:54:41.206802+0000","last_fullsized":"2024-05-14T16:54:41.206802+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:43:15.634585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"80'15","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207361+0000","last_change":"2024-05-14T16:51:32.306158+0000","last_active":"2024-05-14T16:54:41.207361+0000","last_peered":"2024-05-14T16:54:41.207361+0000","last_clean":"2024-05-14T16:54:41.207361+0000","last_became_active":"2024-05-14T16:51:32.305834+0000","last_became_peered":"2024-05-14T16:51:32.305834+0000","last_unstale":"2024-05-14T16:54:41.207361+0000","last_undegraded":"2024-05-14T16:54:41.207361+0000","last_fullsized":"2024-05-14T16:54:41.207361+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:31:47.645862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'6","reported_seq":215,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:45.277498+0000","last_change":"2024-05-14T16:51:29.270735+0000","last_active":"2024-05-14T16:54:45.277498+0000","last_peered":"2024-05-14T16:54:45.277498+0000","last_clean":"2024-05-14T16:54:45.277498+0000","last_became_active":"2024-05-14T16:51:29.270362+0000","last_became_peered":"2024-05-14T16:51:29.270362+0000","last_unstale":"2024-05-14T16:54:45.277498+0000","last_undegraded":"2024-05-14T16:54:45.277498+0000","last_fullsized":"2024-05-14T16:54:45.277498+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:40:12.288655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1678,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":181,"num_read_kb":5,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"80'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206222+0000","last_change":"2024-05-14T16:51:32.305007+0000","last_active":"2024-05-14T16:54:41.206222+0000","last_peered":"2024-05-14T16:54:41.206222+0000","last_clean":"2024-05-14T16:54:41.206222+0000","last_became_active":"2024-05-14T16:51:32.303927+0000","last_became_peered":"2024-05-14T16:51:32.303927+0000","last_unstale":"2024-05-14T16:54:41.206222+0000","last_undegraded":"2024-05-14T16:54:41.206222+0000","last_fullsized":"2024-05-14T16:54:41.206222+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:42:28.018492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206550+0000","last_change":"2024-05-14T16:51:29.345902+0000","last_active":"2024-05-14T16:54:41.206550+0000","last_peered":"2024-05-14T16:54:41.206550+0000","last_clean":"2024-05-14T16:54:41.206550+0000","last_became_active":"2024-05-14T16:51:29.344904+0000","last_became_peered":"2024-05-14T16:51:29.344904+0000","last_unstale":"2024-05-14T16:54:41.206550+0000","last_undegraded":"2024-05-14T16:54:41.206550+0000","last_fullsized":"2024-05-14T16:54:41.206550+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:56:22.209067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"80'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206556+0000","last_change":"2024-05-14T16:51:32.306459+0000","last_active":"2024-05-14T16:54:41.206556+0000","last_peered":"2024-05-14T16:54:41.206556+0000","last_clean":"2024-05-14T16:54:41.206556+0000","last_became_active":"2024-05-14T16:51:32.304897+0000","last_became_peered":"2024-05-14T16:51:32.304897+0000","last_unstale":"2024-05-14T16:54:41.206556+0000","last_undegraded":"2024-05-14T16:54:41.206556+0000","last_fullsized":"2024-05-14T16:54:41.206556+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T18:09:51.216631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'4","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206585+0000","last_change":"2024-05-14T16:51:29.275219+0000","last_active":"2024-05-14T16:54:41.206585+0000","last_peered":"2024-05-14T16:54:41.206585+0000","last_clean":"2024-05-14T16:54:41.206585+0000","last_became_active":"2024-05-14T16:51:29.274816+0000","last_became_peered":"2024-05-14T16:51:29.274816+0000","last_unstale":"2024-05-14T16:54:41.206585+0000","last_undegraded":"2024-05-14T16:54:41.206585+0000","last_fullsized":"2024-05-14T16:54:41.206585+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:55:49.667387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"80'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.196946+0000","last_change":"2024-05-14T16:51:32.299284+0000","last_active":"2024-05-14T16:54:41.196946+0000","last_peered":"2024-05-14T16:54:41.196946+0000","last_clean":"2024-05-14T16:54:41.196946+0000","last_became_active":"2024-05-14T16:51:32.298933+0000","last_became_peered":"2024-05-14T16:51:32.298933+0000","last_unstale":"2024-05-14T16:54:41.196946+0000","last_undegraded":"2024-05-14T16:54:41.196946+0000","last_fullsized":"2024-05-14T16:54:41.196946+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:35:42.266665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.208227+0000","last_change":"2024-05-14T16:51:29.345454+0000","last_active":"2024-05-14T16:54:41.208227+0000","last_peered":"2024-05-14T16:54:41.208227+0000","last_clean":"2024-05-14T16:54:41.208227+0000","last_became_active":"2024-05-14T16:51:29.345074+0000","last_became_peered":"2024-05-14T16:51:29.345074+0000","last_unstale":"2024-05-14T16:54:41.208227+0000","last_undegraded":"2024-05-14T16:54:41.208227+0000","last_fullsized":"2024-05-14T16:54:41.208227+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:56:11.661144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.194447+0000","last_change":"2024-05-14T16:51:32.288804+0000","last_active":"2024-05-14T16:54:41.194447+0000","last_peered":"2024-05-14T16:54:41.194447+0000","last_clean":"2024-05-14T16:54:41.194447+0000","last_became_active":"2024-05-14T16:51:32.288441+0000","last_became_peered":"2024-05-14T16:51:32.288441+0000","last_unstale":"2024-05-14T16:54:41.194447+0000","last_undegraded":"2024-05-14T16:54:41.194447+0000","last_fullsized":"2024-05-14T16:54:41.194447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:28:31.720342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206860+0000","last_change":"2024-05-14T16:51:29.272032+0000","last_active":"2024-05-14T16:54:41.206860+0000","last_peered":"2024-05-14T16:54:41.206860+0000","last_clean":"2024-05-14T16:54:41.206860+0000","last_became_active":"2024-05-14T16:51:29.271677+0000","last_became_peered":"2024-05-14T16:51:29.271677+0000","last_unstale":"2024-05-14T16:54:41.206860+0000","last_undegraded":"2024-05-14T16:54:41.206860+0000","last_fullsized":"2024-05-14T16:54:41.206860+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:39:41.377591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"80'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.194175+0000","last_change":"2024-05-14T16:51:32.295159+0000","last_active":"2024-05-14T16:54:41.194175+0000","last_peered":"2024-05-14T16:54:41.194175+0000","last_clean":"2024-05-14T16:54:41.194175+0000","last_became_active":"2024-05-14T16:51:32.294812+0000","last_became_peered":"2024-05-14T16:51:32.294812+0000","last_unstale":"2024-05-14T16:54:41.194175+0000","last_undegraded":"2024-05-14T16:54:41.194175+0000","last_fullsized":"2024-05-14T16:54:41.194175+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:08:22.411800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"80'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206770+0000","last_change":"2024-05-14T16:51:32.306582+0000","last_active":"2024-05-14T16:54:41.206770+0000","last_peered":"2024-05-14T16:54:41.206770+0000","last_clean":"2024-05-14T16:54:41.206770+0000","last_became_active":"2024-05-14T16:51:32.306209+0000","last_became_peered":"2024-05-14T16:51:32.306209+0000","last_unstale":"2024-05-14T16:54:41.206770+0000","last_undegraded":"2024-05-14T16:54:41.206770+0000","last_fullsized":"2024-05-14T16:54:41.206770+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:30:46.264097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":210,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:45.276865+0000","last_change":"2024-05-14T16:51:29.345057+0000","last_active":"2024-05-14T16:54:45.276865+0000","last_peered":"2024-05-14T16:54:45.276865+0000","last_clean":"2024-05-14T16:54:45.276865+0000","last_became_active":"2024-05-14T16:51:29.344658+0000","last_became_peered":"2024-05-14T16:51:29.344658+0000","last_unstale":"2024-05-14T16:54:45.276865+0000","last_undegraded":"2024-05-14T16:54:45.276865+0000","last_fullsized":"2024-05-14T16:54:45.276865+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:43:53.007543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":176,"num_read_kb":176,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"80'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207516+0000","last_change":"2024-05-14T16:51:32.298958+0000","last_active":"2024-05-14T16:54:41.207516+0000","last_peered":"2024-05-14T16:54:41.207516+0000","last_clean":"2024-05-14T16:54:41.207516+0000","last_became_active":"2024-05-14T16:51:32.298420+0000","last_became_peered":"2024-05-14T16:51:32.298420+0000","last_unstale":"2024-05-14T16:54:41.207516+0000","last_undegraded":"2024-05-14T16:54:41.207516+0000","last_fullsized":"2024-05-14T16:54:41.207516+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:15:21.262026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206858+0000","last_change":"2024-05-14T16:51:29.272082+0000","last_active":"2024-05-14T16:54:41.206858+0000","last_peered":"2024-05-14T16:54:41.206858+0000","last_clean":"2024-05-14T16:54:41.206858+0000","last_became_active":"2024-05-14T16:51:29.271711+0000","last_became_peered":"2024-05-14T16:51:29.271711+0000","last_unstale":"2024-05-14T16:54:41.206858+0000","last_undegraded":"2024-05-14T16:54:41.206858+0000","last_fullsized":"2024-05-14T16:54:41.206858+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:27:44.320423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"81'99","reported_seq":130,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:43.293475+0000","last_change":"2024-05-14T16:50:05.320136+0000","last_active":"2024-05-14T16:54:43.293475+0000","last_peered":"2024-05-14T16:54:43.293475+0000","last_clean":"2024-05-14T16:54:43.293475+0000","last_became_active":"2024-05-14T16:50:05.315016+0000","last_became_peered":"2024-05-14T16:50:05.315016+0000","last_unstale":"2024-05-14T16:54:43.293475+0000","last_undegraded":"2024-05-14T16:54:43.293475+0000","last_fullsized":"2024-05-14T16:54:43.293475+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:46:38.138211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:46:38.138211+0000","last_clean_scrub_stamp":"2024-05-14T16:46:38.138211+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:19:09.012578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.192492+0000","last_change":"2024-05-14T16:51:32.295995+0000","last_active":"2024-05-14T16:54:41.192492+0000","last_peered":"2024-05-14T16:54:41.192492+0000","last_clean":"2024-05-14T16:54:41.192492+0000","last_became_active":"2024-05-14T16:51:32.295640+0000","last_became_peered":"2024-05-14T16:51:32.295640+0000","last_unstale":"2024-05-14T16:54:41.192492+0000","last_undegraded":"2024-05-14T16:54:41.192492+0000","last_fullsized":"2024-05-14T16:54:41.192492+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:42:26.774916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206592+0000","last_change":"2024-05-14T16:51:29.274892+0000","last_active":"2024-05-14T16:54:41.206592+0000","last_peered":"2024-05-14T16:54:41.206592+0000","last_clean":"2024-05-14T16:54:41.206592+0000","last_became_active":"2024-05-14T16:51:29.274386+0000","last_became_peered":"2024-05-14T16:51:29.274386+0000","last_unstale":"2024-05-14T16:54:41.206592+0000","last_undegraded":"2024-05-14T16:54:41.206592+0000","last_fullsized":"2024-05-14T16:54:41.206592+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:16:54.899320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207219+0000","last_change":"2024-05-14T16:51:29.345672+0000","last_active":"2024-05-14T16:54:41.207219+0000","last_peered":"2024-05-14T16:54:41.207219+0000","last_clean":"2024-05-14T16:54:41.207219+0000","last_became_active":"2024-05-14T16:51:29.344948+0000","last_became_peered":"2024-05-14T16:51:29.344948+0000","last_unstale":"2024-05-14T16:54:41.207219+0000","last_undegraded":"2024-05-14T16:54:41.207219+0000","last_fullsized":"2024-05-14T16:54:41.207219+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:19:26.577799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"80'9","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.192847+0000","last_change":"2024-05-14T16:51:32.297360+0000","last_active":"2024-05-14T16:54:41.192847+0000","last_peered":"2024-05-14T16:54:41.192847+0000","last_clean":"2024-05-14T16:54:41.192847+0000","last_became_active":"2024-05-14T16:51:32.297025+0000","last_became_peered":"2024-05-14T16:51:32.297025+0000","last_unstale":"2024-05-14T16:54:41.192847+0000","last_undegraded":"2024-05-14T16:54:41.192847+0000","last_fullsized":"2024-05-14T16:54:41.192847+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:05:39.705860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'100","reported_seq":233,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:43.460974+0000","last_change":"2024-05-14T16:52:13.224328+0000","last_active":"2024-05-14T16:54:43.460974+0000","last_peered":"2024-05-14T16:54:43.460974+0000","last_clean":"2024-05-14T16:54:43.460974+0000","last_became_active":"2024-05-14T16:52:13.220271+0000","last_became_peered":"2024-05-14T16:52:13.220271+0000","last_unstale":"2024-05-14T16:54:43.460974+0000","last_undegraded":"2024-05-14T16:54:43.460974+0000","last_fullsized":"2024-05-14T16:54:43.460974+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:06:31.955274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":185,"num_write_kb":1287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"80'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207534+0000","last_change":"2024-05-14T16:51:32.292112+0000","last_active":"2024-05-14T16:54:41.207534+0000","last_peered":"2024-05-14T16:54:41.207534+0000","last_clean":"2024-05-14T16:54:41.207534+0000","last_became_active":"2024-05-14T16:51:32.291688+0000","last_became_peered":"2024-05-14T16:51:32.291688+0000","last_unstale":"2024-05-14T16:54:41.207534+0000","last_undegraded":"2024-05-14T16:54:41.207534+0000","last_fullsized":"2024-05-14T16:54:41.207534+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:13:06.728608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207174+0000","last_change":"2024-05-14T16:51:29.412805+0000","last_active":"2024-05-14T16:54:41.207174+0000","last_peered":"2024-05-14T16:54:41.207174+0000","last_clean":"2024-05-14T16:54:41.207174+0000","last_became_active":"2024-05-14T16:51:29.412394+0000","last_became_peered":"2024-05-14T16:51:29.412394+0000","last_unstale":"2024-05-14T16:54:41.207174+0000","last_undegraded":"2024-05-14T16:54:41.207174+0000","last_fullsized":"2024-05-14T16:54:41.207174+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:49:48.721104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"80'7","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206672+0000","last_change":"2024-05-14T16:52:13.218227+0000","last_active":"2024-05-14T16:54:41.206672+0000","last_peered":"2024-05-14T16:54:41.206672+0000","last_clean":"2024-05-14T16:54:41.206672+0000","last_became_active":"2024-05-14T16:52:13.217787+0000","last_became_peered":"2024-05-14T16:52:13.217787+0000","last_unstale":"2024-05-14T16:54:41.206672+0000","last_undegraded":"2024-05-14T16:54:41.206672+0000","last_fullsized":"2024-05-14T16:54:41.206672+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:52:33.348804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206178+0000","last_change":"2024-05-14T16:51:29.345427+0000","last_active":"2024-05-14T16:54:41.206178+0000","last_peered":"2024-05-14T16:54:41.206178+0000","last_clean":"2024-05-14T16:54:41.206178+0000","last_became_active":"2024-05-14T16:51:29.345049+0000","last_became_peered":"2024-05-14T16:51:29.345049+0000","last_unstale":"2024-05-14T16:54:41.206178+0000","last_undegraded":"2024-05-14T16:54:41.206178+0000","last_fullsized":"2024-05-14T16:54:41.206178+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:06:19.799660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"80'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206337+0000","last_change":"2024-05-14T16:51:32.298757+0000","last_active":"2024-05-14T16:54:41.206337+0000","last_peered":"2024-05-14T16:54:41.206337+0000","last_clean":"2024-05-14T16:54:41.206337+0000","last_became_active":"2024-05-14T16:51:32.297193+0000","last_became_peered":"2024-05-14T16:51:32.297193+0000","last_unstale":"2024-05-14T16:54:41.206337+0000","last_undegraded":"2024-05-14T16:54:41.206337+0000","last_fullsized":"2024-05-14T16:54:41.206337+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:10:19.672230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207012+0000","last_change":"2024-05-14T16:51:29.270545+0000","last_active":"2024-05-14T16:54:41.207012+0000","last_peered":"2024-05-14T16:54:41.207012+0000","last_clean":"2024-05-14T16:54:41.207012+0000","last_became_active":"2024-05-14T16:51:29.270182+0000","last_became_peered":"2024-05-14T16:51:29.270182+0000","last_unstale":"2024-05-14T16:54:41.207012+0000","last_undegraded":"2024-05-14T16:54:41.207012+0000","last_fullsized":"2024-05-14T16:54:41.207012+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:19:49.977916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"80'9","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.196769+0000","last_change":"2024-05-14T16:51:32.294767+0000","last_active":"2024-05-14T16:54:41.196769+0000","last_peered":"2024-05-14T16:54:41.196769+0000","last_clean":"2024-05-14T16:54:41.196769+0000","last_became_active":"2024-05-14T16:51:32.293947+0000","last_became_peered":"2024-05-14T16:51:32.293947+0000","last_unstale":"2024-05-14T16:54:41.196769+0000","last_undegraded":"2024-05-14T16:54:41.196769+0000","last_fullsized":"2024-05-14T16:54:41.196769+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:30:21.731492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.196766+0000","last_change":"2024-05-14T16:52:13.226509+0000","last_active":"2024-05-14T16:54:41.196766+0000","last_peered":"2024-05-14T16:54:41.196766+0000","last_clean":"2024-05-14T16:54:41.196766+0000","last_became_active":"2024-05-14T16:52:13.220923+0000","last_became_peered":"2024-05-14T16:52:13.220923+0000","last_unstale":"2024-05-14T16:54:41.196766+0000","last_undegraded":"2024-05-14T16:54:41.196766+0000","last_fullsized":"2024-05-14T16:54:41.196766+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:25:52.655481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"80'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207841+0000","last_change":"2024-05-14T16:51:32.288810+0000","last_active":"2024-05-14T16:54:41.207841+0000","last_peered":"2024-05-14T16:54:41.207841+0000","last_clean":"2024-05-14T16:54:41.207841+0000","last_became_active":"2024-05-14T16:51:32.288492+0000","last_became_peered":"2024-05-14T16:51:32.288492+0000","last_unstale":"2024-05-14T16:54:41.207841+0000","last_undegraded":"2024-05-14T16:54:41.207841+0000","last_fullsized":"2024-05-14T16:54:41.207841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T00:44:53.393371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206506+0000","last_change":"2024-05-14T16:51:29.281181+0000","last_active":"2024-05-14T16:54:41.206506+0000","last_peered":"2024-05-14T16:54:41.206506+0000","last_clean":"2024-05-14T16:54:41.206506+0000","last_became_active":"2024-05-14T16:51:29.279965+0000","last_became_peered":"2024-05-14T16:51:29.279965+0000","last_unstale":"2024-05-14T16:54:41.206506+0000","last_undegraded":"2024-05-14T16:54:41.206506+0000","last_fullsized":"2024-05-14T16:54:41.206506+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:41:00.735291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"80'5","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206981+0000","last_change":"2024-05-14T16:51:32.296971+0000","last_active":"2024-05-14T16:54:41.206981+0000","last_peered":"2024-05-14T16:54:41.206981+0000","last_clean":"2024-05-14T16:54:41.206981+0000","last_became_active":"2024-05-14T16:51:32.296600+0000","last_became_peered":"2024-05-14T16:51:32.296600+0000","last_unstale":"2024-05-14T16:54:41.206981+0000","last_undegraded":"2024-05-14T16:54:41.206981+0000","last_fullsized":"2024-05-14T16:54:41.206981+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:13:14.354127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207310+0000","last_change":"2024-05-14T16:51:29.412682+0000","last_active":"2024-05-14T16:54:41.207310+0000","last_peered":"2024-05-14T16:54:41.207310+0000","last_clean":"2024-05-14T16:54:41.207310+0000","last_became_active":"2024-05-14T16:51:29.412186+0000","last_became_peered":"2024-05-14T16:51:29.412186+0000","last_unstale":"2024-05-14T16:54:41.207310+0000","last_undegraded":"2024-05-14T16:54:41.207310+0000","last_fullsized":"2024-05-14T16:54:41.207310+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T03:56:24.570276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"80'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207683+0000","last_change":"2024-05-14T16:51:32.300626+0000","last_active":"2024-05-14T16:54:41.207683+0000","last_peered":"2024-05-14T16:54:41.207683+0000","last_clean":"2024-05-14T16:54:41.207683+0000","last_became_active":"2024-05-14T16:51:32.300278+0000","last_became_peered":"2024-05-14T16:51:32.300278+0000","last_unstale":"2024-05-14T16:54:41.207683+0000","last_undegraded":"2024-05-14T16:54:41.207683+0000","last_fullsized":"2024-05-14T16:54:41.207683+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T01:59:14.196315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'65","reported_seq":125,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:43.093544+0000","last_change":"2024-05-14T16:51:29.270826+0000","last_active":"2024-05-14T16:54:43.093544+0000","last_peered":"2024-05-14T16:54:43.093544+0000","last_clean":"2024-05-14T16:54:43.093544+0000","last_became_active":"2024-05-14T16:51:29.270266+0000","last_became_peered":"2024-05-14T16:51:29.270266+0000","last_unstale":"2024-05-14T16:54:43.093544+0000","last_undegraded":"2024-05-14T16:54:43.093544+0000","last_fullsized":"2024-05-14T16:54:43.093544+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:00:55.349278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'7","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207426+0000","last_change":"2024-05-14T16:52:13.223994+0000","last_active":"2024-05-14T16:54:41.207426+0000","last_peered":"2024-05-14T16:54:41.207426+0000","last_clean":"2024-05-14T16:54:41.207426+0000","last_became_active":"2024-05-14T16:52:13.221071+0000","last_became_peered":"2024-05-14T16:52:13.221071+0000","last_unstale":"2024-05-14T16:54:41.207426+0000","last_undegraded":"2024-05-14T16:54:41.207426+0000","last_fullsized":"2024-05-14T16:54:41.207426+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T17:29:58.775171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.192788+0000","last_change":"2024-05-14T16:51:29.276705+0000","last_active":"2024-05-14T16:54:41.192788+0000","last_peered":"2024-05-14T16:54:41.192788+0000","last_clean":"2024-05-14T16:54:41.192788+0000","last_became_active":"2024-05-14T16:51:29.276347+0000","last_became_peered":"2024-05-14T16:51:29.276347+0000","last_unstale":"2024-05-14T16:54:41.192788+0000","last_undegraded":"2024-05-14T16:54:41.192788+0000","last_fullsized":"2024-05-14T16:54:41.192788+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:28.245277+0000","last_clean_scrub_stamp":"2024-05-14T16:51:28.245277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T21:32:56.064427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"80'9","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207670+0000","last_change":"2024-05-14T16:51:32.301617+0000","last_active":"2024-05-14T16:54:41.207670+0000","last_peered":"2024-05-14T16:54:41.207670+0000","last_clean":"2024-05-14T16:54:41.207670+0000","last_became_active":"2024-05-14T16:51:32.301288+0000","last_became_peered":"2024-05-14T16:51:32.301288+0000","last_unstale":"2024-05-14T16:54:41.207670+0000","last_undegraded":"2024-05-14T16:54:41.207670+0000","last_fullsized":"2024-05-14T16:54:41.207670+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:19:24.253633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"80'9","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.197028+0000","last_change":"2024-05-14T16:52:13.221694+0000","last_active":"2024-05-14T16:54:41.197028+0000","last_peered":"2024-05-14T16:54:41.197028+0000","last_clean":"2024-05-14T16:54:41.197028+0000","last_became_active":"2024-05-14T16:52:13.221264+0000","last_became_peered":"2024-05-14T16:52:13.221264+0000","last_unstale":"2024-05-14T16:54:41.197028+0000","last_undegraded":"2024-05-14T16:54:41.197028+0000","last_fullsized":"2024-05-14T16:54:41.197028+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:22:52.738772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"80'12","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206531+0000","last_change":"2024-05-14T16:51:32.301789+0000","last_active":"2024-05-14T16:54:41.206531+0000","last_peered":"2024-05-14T16:54:41.206531+0000","last_clean":"2024-05-14T16:54:41.206531+0000","last_became_active":"2024-05-14T16:51:32.301350+0000","last_became_peered":"2024-05-14T16:51:32.301350+0000","last_unstale":"2024-05-14T16:54:41.206531+0000","last_undegraded":"2024-05-14T16:54:41.206531+0000","last_fullsized":"2024-05-14T16:54:41.206531+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:46:24.896855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"80'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206730+0000","last_change":"2024-05-14T16:51:32.295978+0000","last_active":"2024-05-14T16:54:41.206730+0000","last_peered":"2024-05-14T16:54:41.206730+0000","last_clean":"2024-05-14T16:54:41.206730+0000","last_became_active":"2024-05-14T16:51:32.295652+0000","last_became_peered":"2024-05-14T16:51:32.295652+0000","last_unstale":"2024-05-14T16:54:41.206730+0000","last_undegraded":"2024-05-14T16:54:41.206730+0000","last_fullsized":"2024-05-14T16:54:41.206730+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:32:17.743734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"80'7","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206139+0000","last_change":"2024-05-14T16:51:32.305568+0000","last_active":"2024-05-14T16:54:41.206139+0000","last_peered":"2024-05-14T16:54:41.206139+0000","last_clean":"2024-05-14T16:54:41.206139+0000","last_became_active":"2024-05-14T16:51:32.304580+0000","last_became_peered":"2024-05-14T16:51:32.304580+0000","last_unstale":"2024-05-14T16:54:41.206139+0000","last_undegraded":"2024-05-14T16:54:41.206139+0000","last_fullsized":"2024-05-14T16:54:41.206139+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:32:44.465054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"80'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.192870+0000","last_change":"2024-05-14T16:51:32.295029+0000","last_active":"2024-05-14T16:54:41.192870+0000","last_peered":"2024-05-14T16:54:41.192870+0000","last_clean":"2024-05-14T16:54:41.192870+0000","last_became_active":"2024-05-14T16:51:32.294621+0000","last_became_peered":"2024-05-14T16:51:32.294621+0000","last_unstale":"2024-05-14T16:54:41.192870+0000","last_undegraded":"2024-05-14T16:54:41.192870+0000","last_fullsized":"2024-05-14T16:54:41.192870+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:52:05.390141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"80'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.196972+0000","last_change":"2024-05-14T16:51:32.297671+0000","last_active":"2024-05-14T16:54:41.196972+0000","last_peered":"2024-05-14T16:54:41.196972+0000","last_clean":"2024-05-14T16:54:41.196972+0000","last_became_active":"2024-05-14T16:51:32.296210+0000","last_became_peered":"2024-05-14T16:51:32.296210+0000","last_unstale":"2024-05-14T16:54:41.196972+0000","last_undegraded":"2024-05-14T16:54:41.196972+0000","last_fullsized":"2024-05-14T16:54:41.196972+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T23:55:04.789021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"80'15","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207532+0000","last_change":"2024-05-14T16:51:32.297404+0000","last_active":"2024-05-14T16:54:41.207532+0000","last_peered":"2024-05-14T16:54:41.207532+0000","last_clean":"2024-05-14T16:54:41.207532+0000","last_became_active":"2024-05-14T16:51:32.296856+0000","last_became_peered":"2024-05-14T16:51:32.296856+0000","last_unstale":"2024-05-14T16:54:41.207532+0000","last_undegraded":"2024-05-14T16:54:41.207532+0000","last_fullsized":"2024-05-14T16:54:41.207532+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T04:25:29.992875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"80'2","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.207395+0000","last_change":"2024-05-14T16:52:13.221379+0000","last_active":"2024-05-14T16:54:41.207395+0000","last_peered":"2024-05-14T16:54:41.207395+0000","last_clean":"2024-05-14T16:54:41.207395+0000","last_became_active":"2024-05-14T16:52:13.220973+0000","last_became_peered":"2024-05-14T16:52:13.220973+0000","last_unstale":"2024-05-14T16:54:41.207395+0000","last_undegraded":"2024-05-14T16:54:41.207395+0000","last_fullsized":"2024-05-14T16:54:41.207395+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T19:55:08.212344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"80'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206477+0000","last_change":"2024-05-14T16:51:32.303570+0000","last_active":"2024-05-14T16:54:41.206477+0000","last_peered":"2024-05-14T16:54:41.206477+0000","last_clean":"2024-05-14T16:54:41.206477+0000","last_became_active":"2024-05-14T16:51:32.303234+0000","last_became_peered":"2024-05-14T16:51:32.303234+0000","last_unstale":"2024-05-14T16:54:41.206477+0000","last_undegraded":"2024-05-14T16:54:41.206477+0000","last_fullsized":"2024-05-14T16:54:41.206477+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:27:57.960120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"80'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206835+0000","last_change":"2024-05-14T16:51:32.298028+0000","last_active":"2024-05-14T16:54:41.206835+0000","last_peered":"2024-05-14T16:54:41.206835+0000","last_clean":"2024-05-14T16:54:41.206835+0000","last_became_active":"2024-05-14T16:51:32.295104+0000","last_became_peered":"2024-05-14T16:51:32.295104+0000","last_unstale":"2024-05-14T16:54:41.206835+0000","last_undegraded":"2024-05-14T16:54:41.206835+0000","last_fullsized":"2024-05-14T16:54:41.206835+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T20:58:08.441590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"80'10","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.206745+0000","last_change":"2024-05-14T16:51:32.304458+0000","last_active":"2024-05-14T16:54:41.206745+0000","last_peered":"2024-05-14T16:54:41.206745+0000","last_clean":"2024-05-14T16:54:41.206745+0000","last_became_active":"2024-05-14T16:51:32.302952+0000","last_became_peered":"2024-05-14T16:51:32.302952+0000","last_unstale":"2024-05-14T16:54:41.206745+0000","last_undegraded":"2024-05-14T16:54:41.206745+0000","last_fullsized":"2024-05-14T16:54:41.206745+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-15T22:40:14.220075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"80'11","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.192762+0000","last_change":"2024-05-14T16:51:32.292334+0000","last_active":"2024-05-14T16:54:41.192762+0000","last_peered":"2024-05-14T16:54:41.192762+0000","last_clean":"2024-05-14T16:54:41.192762+0000","last_became_active":"2024-05-14T16:51:32.291769+0000","last_became_peered":"2024-05-14T16:51:32.291769+0000","last_unstale":"2024-05-14T16:54:41.192762+0000","last_undegraded":"2024-05-14T16:54:41.192762+0000","last_fullsized":"2024-05-14T16:54:41.192762+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:55:25.694827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"80'8","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-14T16:54:41.196919+0000","last_change":"2024-05-14T16:51:32.300044+0000","last_active":"2024-05-14T16:54:41.196919+0000","last_peered":"2024-05-14T16:54:41.196919+0000","last_clean":"2024-05-14T16:54:41.196919+0000","last_became_active":"2024-05-14T16:51:32.299630+0000","last_became_peered":"2024-05-14T16:51:32.299630+0000","last_unstale":"2024-05-14T16:54:41.196919+0000","last_undegraded":"2024-05-14T16:54:41.196919+0000","last_fullsized":"2024-05-14T16:54:41.196919+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-14T16:51:31.266061+0000","last_clean_scrub_stamp":"2024-05-14T16:51:31.266061+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T02:50:40.516973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108486,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":793965,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1437,"num_read_kb":931,"num_write":424,"num_write_kb":2355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":9,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":651264,"data_stored":2381895,"data_compressed":103473,"data_compressed_allocated":135168,"data_compressed_original":2101248,"omap_allocated":0,"internal_metadata":0},"log_size":311,"ondisk_log_size":311,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710774,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":827273,"data_compressed":22208,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907003,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1081344,"data_stored":1421754,"data_compressed":54283,"data_compressed_allocated":69632,"data_compressed_original":905216,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070528,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":827626,"data_compressed":22208,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266758,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":892928,"data_stored":434213,"data_compressed":2919,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462986,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":1027765,"data_compressed":39081,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626512,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36200,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708952,"statfs":{"total":95995035648,"available":95957966848,"internally_reserved":0,"allocated":880640,"data_stored":433682,"data_compressed":2919,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":500369,"data_compressed":2919,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738477,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36220,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708932,"statfs":{"total":95995035648,"available":95957946368,"internally_reserved":0,"allocated":901120,"data_stored":434953,"data_compressed":2919,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411429,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":974848,"data_stored":502642,"data_compressed":2919,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182495,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":962560,"data_stored":1155324,"data_compressed":37410,"data_compressed_allocated":49152,"data_compressed_original":708608,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986267,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":884736,"data_stored":435209,"data_compressed":2919,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822742,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":999424,"data_stored":1354591,"data_compressed":54283,"data_compressed_allocated":69632,"data_compressed_original":905216,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36162,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":721732,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":330631,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":397794,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":393798,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-14T16:54:52.558 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:54:52.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:52 smithi002 ceph-mon[31538]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 102 KiB/s wr, 54 op/s 2024-05-14T16:54:52.808 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:54:52.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:52 smithi151 ceph-mon[36066]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 102 KiB/s wr, 54 op/s 2024-05-14T16:54:52.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:52 smithi123 ceph-mon[35971]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 102 KiB/s wr, 54 op/s 2024-05-14T16:54:53.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:53 smithi002 ceph-mon[31538]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:54:53.755 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:53 smithi151 ceph-mon[36066]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:54:53.811 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:53 smithi123 ceph-mon[35971]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-14T16:54:54.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:54 smithi002 ceph-mon[31538]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 83 KiB/s wr, 47 op/s 2024-05-14T16:54:54.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.513+0000 7faec9526700 1 -- 172.21.15.2:0/30814025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faec4109630 msgr2=0x7faec4109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:54.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.513+0000 7faec9526700 1 --2- 172.21.15.2:0/30814025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faec4109630 0x7faec4109a10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7faeb40099c0 tx=0x7faeb4009cd0 comp rx=0 tx=0).stop 2024-05-14T16:54:54.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.513+0000 7faec9526700 1 -- 172.21.15.2:0/30814025 shutdown_connections 2024-05-14T16:54:54.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.513+0000 7faec9526700 1 --2- 172.21.15.2:0/30814025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faec406c6d0 0x7faec4110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:54.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.513+0000 7faec9526700 1 --2- 172.21.15.2:0/30814025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faec4109630 0x7faec4109a10 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:54.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.513+0000 7faec9526700 1 --2- 172.21.15.2:0/30814025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faec406bd10 0x7faec406c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:54.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.513+0000 7faec9526700 1 -- 172.21.15.2:0/30814025 >> 172.21.15.2:0/30814025 conn(0x7faec40794e0 msgr2=0x7faec40798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:54.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.513+0000 7faec9526700 1 -- 172.21.15.2:0/30814025 shutdown_connections 2024-05-14T16:54:54.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.513+0000 7faec9526700 1 -- 172.21.15.2:0/30814025 wait complete. 2024-05-14T16:54:54.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.513+0000 7faec9526700 1 Processor -- start 2024-05-14T16:54:54.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.514+0000 7faec9526700 1 -- start start 2024-05-14T16:54:54.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.514+0000 7faec9526700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faec406bd10 0x7faec419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:54.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.514+0000 7faec9526700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faec406c6d0 0x7faec419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:54.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.514+0000 7faec9526700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faec4109630 0x7faec41999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:54.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.514+0000 7faec9526700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7faec4113740 con 0x7faec406bd10 2024-05-14T16:54:54.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.514+0000 7faec9526700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faec41135c0 con 0x7faec4109630 2024-05-14T16:54:54.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.514+0000 7faec9526700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faec41138c0 con 0x7faec406c6d0 2024-05-14T16:54:54.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.514+0000 7faec2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faec406bd10 0x7faec419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:54.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.514+0000 7faec2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faec406bd10 0x7faec419f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40504/0 (socket says 172.21.15.2:40504) 2024-05-14T16:54:54.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.514+0000 7faec2ffd700 1 -- 172.21.15.2:0/3671716652 learned_addr learned my addr 172.21.15.2:0/3671716652 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:54.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.514+0000 7faec27fc700 1 --2- 172.21.15.2:0/3671716652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faec406c6d0 0x7faec419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:54.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.514+0000 7faec37fe700 1 --2- 172.21.15.2:0/3671716652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faec4109630 0x7faec41999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:54.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.514+0000 7faec2ffd700 1 -- 172.21.15.2:0/3671716652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faec406c6d0 msgr2=0x7faec419fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:54.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.514+0000 7faec2ffd700 1 --2- 172.21.15.2:0/3671716652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faec406c6d0 0x7faec419fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:54.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.515+0000 7faec2ffd700 1 -- 172.21.15.2:0/3671716652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faec4109630 msgr2=0x7faec41999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:54.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.515+0000 7faec2ffd700 1 --2- 172.21.15.2:0/3671716652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faec4109630 0x7faec41999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:54.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.515+0000 7faec2ffd700 1 -- 172.21.15.2:0/3671716652 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faec419a100 con 0x7faec406bd10 2024-05-14T16:54:54.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.515+0000 7faec2ffd700 1 --2- 172.21.15.2:0/3671716652 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faec406bd10 0x7faec419f7e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7faeb4006010 tx=0x7faeb40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:54.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.515+0000 7faebbfff700 1 -- 172.21.15.2:0/3671716652 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb4026070 con 0x7faec406bd10 2024-05-14T16:54:54.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.515+0000 7faebbfff700 1 -- 172.21.15.2:0/3671716652 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faeb400f460 con 0x7faec406bd10 2024-05-14T16:54:54.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.515+0000 7faebbfff700 1 -- 172.21.15.2:0/3671716652 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb4009e60 con 0x7faec406bd10 2024-05-14T16:54:54.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.515+0000 7faec9526700 1 -- 172.21.15.2:0/3671716652 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faec419a390 con 0x7faec406bd10 2024-05-14T16:54:54.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.515+0000 7faec9526700 1 -- 172.21.15.2:0/3671716652 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faec41a66b0 con 0x7faec406bd10 2024-05-14T16:54:54.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.516+0000 7faec9526700 1 -- 172.21.15.2:0/3671716652 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faec410bac0 con 0x7faec406bd10 2024-05-14T16:54:54.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.517+0000 7faebbfff700 1 -- 172.21.15.2:0/3671716652 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 24) ==== 149827+0+0 (secure 0 0 0) 0x7faeb4029070 con 0x7faec406bd10 2024-05-14T16:54:54.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.518+0000 7faebbfff700 1 --2- 172.21.15.2:0/3671716652 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7faeb00b2a80 0x7faeb00b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:54.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.518+0000 7faec27fc700 1 --2- 172.21.15.2:0/3671716652 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7faeb00b2a80 0x7faeb00b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:54.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.518+0000 7faebbfff700 1 -- 172.21.15.2:0/3671716652 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7faeb40e3380 con 0x7faec406bd10 2024-05-14T16:54:54.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.519+0000 7faec27fc700 1 --2- 172.21.15.2:0/3671716652 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7faeb00b2a80 0x7faeb00b4f40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faec4079ac0 tx=0x7faeac00b410 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:54.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.522+0000 7faebbfff700 1 -- 172.21.15.2:0/3671716652 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faeb4030020 con 0x7faec406bd10 2024-05-14T16:54:54.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.788+0000 7faec9526700 1 -- 172.21.15.2:0/3671716652 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faec419aed0 con 0x7faec406bd10 2024-05-14T16:54:54.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.789+0000 7faebbfff700 1 -- 172.21.15.2:0/3671716652 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11850 (secure 0 0 0) 0x7faeb4013070 con 0x7faec406bd10 2024-05-14T16:54:54.790 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:54:54.790 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2275696923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2275696923},{"type":"v1","addr":"172.21.15.2:6835","nonce":2275696923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/716718761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":716718761},{"type":"v1","addr":"172.21.15.2:6839","nonce":716718761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/3644815179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3644815179},{"type":"v1","addr":"172.21.15.2:6841","nonce":3644815179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1880336731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1880336731},{"type":"v1","addr":"172.21.15.151:6833","nonce":1880336731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2415605548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2415605548},{"type":"v1","addr":"172.21.15.2:6837","nonce":2415605548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2635167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2635167},{"type":"v1","addr":"172.21.15.123:6833","nonce":2635167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:51:34.106919+0000","modified":"2024-05-14T16:54:38.886333+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24455,"mds_2":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24415":{"gid":24415,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/762044339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":762044339},{"type":"v1","addr":"172.21.15.151:6835","nonce":762044339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/132043151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":132043151},{"type":"v1","addr":"172.21.15.151:6837","nonce":132043151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/4107834375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":4107834375},{"type":"v1","addr":"172.21.15.151:6839","nonce":4107834375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.123:6835/1198670013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1198670013},{"type":"v1","addr":"172.21.15.123:6835","nonce":1198670013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/333426645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":333426645},{"type":"v1","addr":"172.21.15.123:6837","nonce":333426645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/580275877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":580275877},{"type":"v1","addr":"172.21.15.123:6839","nonce":580275877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24431,24455,24479]},"id":1}]} 2024-05-14T16:54:54.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.792+0000 7faec9526700 1 -- 172.21.15.2:0/3671716652 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7faeb00b2a80 msgr2=0x7faeb00b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:54.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.792+0000 7faec9526700 1 --2- 172.21.15.2:0/3671716652 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7faeb00b2a80 0x7faeb00b4f40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faec4079ac0 tx=0x7faeac00b410 comp rx=0 tx=0).stop 2024-05-14T16:54:54.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.792+0000 7faec9526700 1 -- 172.21.15.2:0/3671716652 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faec406bd10 msgr2=0x7faec419f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:54.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.792+0000 7faec9526700 1 --2- 172.21.15.2:0/3671716652 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faec406bd10 0x7faec419f7e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7faeb4006010 tx=0x7faeb40045e0 comp rx=0 tx=0).stop 2024-05-14T16:54:54.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.792+0000 7faec9526700 1 -- 172.21.15.2:0/3671716652 shutdown_connections 2024-05-14T16:54:54.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.792+0000 7faec9526700 1 --2- 172.21.15.2:0/3671716652 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7faeb00b2a80 0x7faeb00b4f40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:54.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.792+0000 7faec9526700 1 --2- 172.21.15.2:0/3671716652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faec4109630 0x7faec41999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:54.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.792+0000 7faec9526700 1 --2- 172.21.15.2:0/3671716652 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faec406bd10 0x7faec419f7e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:54.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.792+0000 7faec9526700 1 --2- 172.21.15.2:0/3671716652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faec406c6d0 0x7faec419fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:54.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.792+0000 7faec9526700 1 -- 172.21.15.2:0/3671716652 >> 172.21.15.2:0/3671716652 conn(0x7faec40794e0 msgr2=0x7faec4101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:54.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.792+0000 7faec9526700 1 -- 172.21.15.2:0/3671716652 shutdown_connections 2024-05-14T16:54:54.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:54.792+0000 7faec9526700 1 -- 172.21.15.2:0/3671716652 wait complete. 2024-05-14T16:54:54.794 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 25 2024-05-14T16:54:54.827 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:54 smithi123 ceph-mon[35971]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 83 KiB/s wr, 47 op/s 2024-05-14T16:54:54.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:54 smithi151 ceph-mon[36066]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 83 KiB/s wr, 47 op/s 2024-05-14T16:54:55.407 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:54:55.662 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:54:55.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:55 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/3671716652' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:54:55.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:55 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/3671716652' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:54:55.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:55 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/3671716652' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:54:56.649 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:56 smithi002 ceph-mon[31538]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 43 op/s 2024-05-14T16:54:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.703+0000 7fc1a0503640 1 -- 172.21.15.2:0/153776791 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc198102330 msgr2=0x7fc19810e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.703+0000 7fc1a0503640 1 --2- 172.21.15.2:0/153776791 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc198102330 0x7fc19810e840 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc190009cd0 tx=0x7fc19002a8e0 comp rx=0 tx=0).stop 2024-05-14T16:54:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.703+0000 7fc1a0503640 1 -- 172.21.15.2:0/153776791 shutdown_connections 2024-05-14T16:54:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.703+0000 7fc1a0503640 1 --2- 172.21.15.2:0/153776791 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc198102330 0x7fc19810e840 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.703+0000 7fc1a0503640 1 --2- 172.21.15.2:0/153776791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc198101970 0x7fc198101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.703+0000 7fc1a0503640 1 --2- 172.21.15.2:0/153776791 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc198107980 0x7fc198107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.703+0000 7fc1a0503640 1 -- 172.21.15.2:0/153776791 >> 172.21.15.2:0/153776791 conn(0x7fc198077cd0 msgr2=0x7fc1980780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.704+0000 7fc1a0503640 1 -- 172.21.15.2:0/153776791 shutdown_connections 2024-05-14T16:54:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.704+0000 7fc1a0503640 1 -- 172.21.15.2:0/153776791 wait complete. 2024-05-14T16:54:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.704+0000 7fc1a0503640 1 Processor -- start 2024-05-14T16:54:56.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.704+0000 7fc1a0503640 1 -- start start 2024-05-14T16:54:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.704+0000 7fc1a0503640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc198101970 0x7fc198103540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.704+0000 7fc1a0503640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc198102330 0x7fc198103a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.704+0000 7fc1a0503640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc198107980 0x7fc198103fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.704+0000 7fc1a0503640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc198111c40 con 0x7fc198107980 2024-05-14T16:54:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.704+0000 7fc1a0503640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc198111ac0 con 0x7fc198102330 2024-05-14T16:54:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.704+0000 7fc1a0503640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc198111dc0 con 0x7fc198101970 2024-05-14T16:54:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.705+0000 7fc19eaa0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc198107980 0x7fc198103fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.705+0000 7fc19eaa0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc198107980 0x7fc198103fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40524/0 (socket says 172.21.15.2:40524) 2024-05-14T16:54:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.705+0000 7fc19eaa0640 1 -- 172.21.15.2:0/426195287 learned_addr learned my addr 172.21.15.2:0/426195287 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.705+0000 7fc19e29f640 1 --2- 172.21.15.2:0/426195287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc198101970 0x7fc198103540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.705+0000 7fc19da9e640 1 --2- 172.21.15.2:0/426195287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc198102330 0x7fc198103a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.705+0000 7fc19eaa0640 1 -- 172.21.15.2:0/426195287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc198101970 msgr2=0x7fc198103540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.705+0000 7fc19eaa0640 1 --2- 172.21.15.2:0/426195287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc198101970 0x7fc198103540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.705+0000 7fc19eaa0640 1 -- 172.21.15.2:0/426195287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc198102330 msgr2=0x7fc198103a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.705+0000 7fc19eaa0640 1 --2- 172.21.15.2:0/426195287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc198102330 0x7fc198103a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.705+0000 7fc19eaa0640 1 -- 172.21.15.2:0/426195287 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1981a75d0 con 0x7fc198107980 2024-05-14T16:54:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.705+0000 7fc19eaa0640 1 --2- 172.21.15.2:0/426195287 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc198107980 0x7fc198103fc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc190005cc0 tx=0x7fc190003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.705+0000 7fc18f7fe640 1 -- 172.21.15.2:0/426195287 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc190004450 con 0x7fc198107980 2024-05-14T16:54:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.705+0000 7fc18f7fe640 1 -- 172.21.15.2:0/426195287 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc190031c90 con 0x7fc198107980 2024-05-14T16:54:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.705+0000 7fc1a0503640 1 -- 172.21.15.2:0/426195287 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1981a7860 con 0x7fc198107980 2024-05-14T16:54:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.705+0000 7fc18f7fe640 1 -- 172.21.15.2:0/426195287 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19003b400 con 0x7fc198107980 2024-05-14T16:54:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.706+0000 7fc1a0503640 1 -- 172.21.15.2:0/426195287 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1981a7db0 con 0x7fc198107980 2024-05-14T16:54:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.708+0000 7fc18f7fe640 1 -- 172.21.15.2:0/426195287 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 24) ==== 149827+0+0 (secure 0 0 0) 0x7fc190004680 con 0x7fc198107980 2024-05-14T16:54:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.709+0000 7fc1a3218740 1 Processor -- start 2024-05-14T16:54:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.709+0000 7fc1a3218740 1 -- start start 2024-05-14T16:54:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.709+0000 7fc1a3218740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653487c18f0 0x5653487c1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.709+0000 7fc1a3218740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653487c2210 0x5653487cc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.710+0000 7fc1a3218740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653487ccad0 0x5653487ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.710+0000 7fc1a3218740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x565348734c30 con 0x5653487ccad0 2024-05-14T16:54:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.710+0000 7fc1a3218740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5653487247d0 con 0x5653487c18f0 2024-05-14T16:54:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.710+0000 7fc1a3218740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x565348728810 con 0x5653487c2210 2024-05-14T16:54:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.710+0000 7fc19da9e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653487ccad0 0x5653487ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.710+0000 7fc19da9e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653487ccad0 0x5653487ceec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40526/0 (socket says 172.21.15.2:40526) 2024-05-14T16:54:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.710+0000 7fc19da9e640 1 -- 172.21.15.2:0/4199370310 learned_addr learned my addr 172.21.15.2:0/4199370310 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.710+0000 7fc19da9e640 1 --2- 172.21.15.2:0/4199370310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653487c18f0 0x5653487c1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.710+0000 7fc19e29f640 1 --2- 172.21.15.2:0/4199370310 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653487c2210 0x5653487cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.710+0000 7fc18f7fe640 1 --2- 172.21.15.2:0/426195287 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc1840b2ac0 0x7fc1840b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.710+0000 7fc19da9e640 1 -- 172.21.15.2:0/4199370310 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653487c2210 msgr2=0x5653487cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.710+0000 7fc19da9e640 1 --2- 172.21.15.2:0/4199370310 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653487c2210 0x5653487cc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:56.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.710+0000 7fc19da9e640 1 -- 172.21.15.2:0/4199370310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653487c18f0 msgr2=0x5653487c1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:56.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.710+0000 7fc19da9e640 1 --2- 172.21.15.2:0/4199370310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653487c18f0 0x5653487c1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:56.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.710+0000 7fc19da9e640 1 -- 172.21.15.2:0/4199370310 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565348439210 con 0x5653487ccad0 2024-05-14T16:54:56.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.710+0000 7fc19eaa0640 1 --2- 172.21.15.2:0/426195287 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc1840b2ac0 0x7fc1840b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:56.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.711+0000 7fc19da9e640 1 --2- 172.21.15.2:0/4199370310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653487ccad0 0x5653487ceec0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc1880149b0 tx=0x7fc18800c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d526fa5a8f697b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:56.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.711+0000 7fc18f7fe640 1 -- 172.21.15.2:0/426195287 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fc19002e030 con 0x7fc198107980 2024-05-14T16:54:56.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.711+0000 7fc18cbfe640 1 -- 172.21.15.2:0/4199370310 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1880142d0 con 0x5653487ccad0 2024-05-14T16:54:56.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.711+0000 7fc18cbfe640 1 -- 172.21.15.2:0/4199370310 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1880054d0 con 0x5653487ccad0 2024-05-14T16:54:56.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.711+0000 7fc18cbfe640 1 -- 172.21.15.2:0/4199370310 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18800f760 con 0x5653487ccad0 2024-05-14T16:54:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.712+0000 7fc1a3218740 1 -- 172.21.15.2:0/4199370310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653487ccad0 msgr2=0x5653487ceec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.712+0000 7fc19eaa0640 1 --2- 172.21.15.2:0/426195287 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc1840b2ac0 0x7fc1840b4f80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc190101010 tx=0x7fc190045fb0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.712+0000 7fc1a3218740 1 --2- 172.21.15.2:0/4199370310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653487ccad0 0x5653487ceec0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc1880149b0 tx=0x7fc18800c4b0 comp rx=0 tx=0).stop 2024-05-14T16:54:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.712+0000 7fc1a3218740 1 -- 172.21.15.2:0/4199370310 shutdown_connections 2024-05-14T16:54:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.712+0000 7fc1a3218740 1 --2- 172.21.15.2:0/4199370310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653487ccad0 0x5653487ceec0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.712+0000 7fc1a3218740 1 --2- 172.21.15.2:0/4199370310 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653487c2210 0x5653487cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.712+0000 7fc1a3218740 1 --2- 172.21.15.2:0/4199370310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653487c18f0 0x5653487c1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.712+0000 7fc1a3218740 1 -- 172.21.15.2:0/4199370310 >> 172.21.15.2:0/4199370310 conn(0x5653487bde00 msgr2=0x5653487be1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.712+0000 7fc1a3218740 1 -- 172.21.15.2:0/4199370310 shutdown_connections 2024-05-14T16:54:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.712+0000 7fc1a3218740 1 -- 172.21.15.2:0/4199370310 wait complete. 2024-05-14T16:54:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.712+0000 7fc1a3218740 1 -- start start 2024-05-14T16:54:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.712+0000 7fc1a3218740 1 Processor -- start 2024-05-14T16:54:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.712+0000 7fc1a3218740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653487c18f0 0x5653487cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.712+0000 7fc1a3218740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653487c2210 0x5653487c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.713+0000 7fc1a3218740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653487ccad0 0x5653487c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.713+0000 7fc1a3218740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56534840a790 con 0x5653487c18f0 2024-05-14T16:54:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.713+0000 7fc1a3218740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5653486e0890 con 0x5653487ccad0 2024-05-14T16:54:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.713+0000 7fc1a3218740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5653487cf7a0 con 0x5653487c2210 2024-05-14T16:54:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.713+0000 7fc1a3218740 10 client.0 _ll_register_callbacks cb 0x5653486e3d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:54:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.713+0000 7fc19da9e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653487c18f0 0x5653487cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.713+0000 7fc19da9e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653487c18f0 0x5653487cc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40542/0 (socket says 172.21.15.2:40542) 2024-05-14T16:54:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.713+0000 7fc19da9e640 1 -- 172.21.15.2:0/78474847 learned_addr learned my addr 172.21.15.2:0/78474847 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.713+0000 7fc19e29f640 1 --2- 172.21.15.2:0/78474847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653487c2210 0x5653487c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.713+0000 7fc19da9e640 1 -- 172.21.15.2:0/78474847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653487c2210 msgr2=0x5653487c97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.713+0000 7fc19da9e640 1 --2- 172.21.15.2:0/78474847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653487c2210 0x5653487c97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.713+0000 7fc19da9e640 1 -- 172.21.15.2:0/78474847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653487ccad0 msgr2=0x5653487c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.713+0000 7fc19da9e640 1 --2- 172.21.15.2:0/78474847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653487ccad0 0x5653487c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.713+0000 7fc19da9e640 1 -- 172.21.15.2:0/78474847 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565348505620 con 0x5653487c18f0 2024-05-14T16:54:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.713+0000 7fc19da9e640 1 --2- 172.21.15.2:0/78474847 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653487c18f0 0x5653487cc4b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc188005cf0 tx=0x7fc18800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=608ffc7997d2d2ea server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.713+0000 7fc181ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:54:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.713+0000 7fc181ffb640 1 -- 172.21.15.2:0/78474847 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1880048c0 con 0x5653487c18f0 2024-05-14T16:54:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.713+0000 7fc181ffb640 1 -- 172.21.15.2:0/78474847 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fc188005000 con 0x5653487c18f0 2024-05-14T16:54:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.713+0000 7fc1a3218740 1 -- 172.21.15.2:0/78474847 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1880054d0 con 0x5653487c18f0 2024-05-14T16:54:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.714+0000 7fc181ffb640 1 -- 172.21.15.2:0/78474847 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc188031050 con 0x5653487c18f0 2024-05-14T16:54:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.714+0000 7fc1a3218740 10 client.15255 fetch_fsmap learned FSMap version 25 2024-05-14T16:54:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.714+0000 7fc1a3218740 1 -- 172.21.15.2:0/78474847 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5653487db240 con 0x5653487c18f0 2024-05-14T16:54:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.714+0000 7fc181ffb640 1 -- 172.21.15.2:0/78474847 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc188025020 con 0x5653487c18f0 2024-05-14T16:54:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.714+0000 7fc181ffb640 1 -- 172.21.15.2:0/78474847 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fc18800fa30 con 0x5653487c18f0 2024-05-14T16:54:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.714+0000 7fc1a3218740 10 client.15255 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T16:54:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.714+0000 7fc1a3218740 10 client.15255 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:54:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.714+0000 7fc1a3218740 20 client.15255 populate_metadata read hostname 'smithi002' 2024-05-14T16:54:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.714+0000 7fc1a3218740 1 --2- 172.21.15.2:0/78474847 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5653487dc700 0x5653487deaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.714+0000 7fc1a3218740 4 client.15255 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:54:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.714+0000 7fc1a3218740 1 -- 172.21.15.2:0/78474847 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565348736b80 con 0x5653487dc700 2024-05-14T16:54:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.715+0000 7fc19e29f640 1 --2- 172.21.15.2:0/78474847 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5653487dc700 0x5653487deaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.715+0000 7fc19e29f640 1 --2- 172.21.15.2:0/78474847 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5653487dc700 0x5653487deaf0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ec4e8e7c81bc4ee server_cookie=ad55100121c9e246 in_seq=0 out_seq=0 2024-05-14T16:54:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.715+0000 7fc181ffb640 10 client.15255 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:54:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.716+0000 7fc181ffb640 1 -- 172.21.15.2:0/78474847 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x565348736b80 con 0x5653487dc700 2024-05-14T16:54:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.716+0000 7fc181ffb640 10 client.15255 handle_command_reply: tid=0 2024-05-14T16:54:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.716+0000 7fc1a3218740 1 client.15255 shutdown 2024-05-14T16:54:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.716+0000 7fc1a3218740 1 -- 172.21.15.2:0/78474847 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653487c18f0 msgr2=0x5653487cc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.716+0000 7fc1a3218740 1 --2- 172.21.15.2:0/78474847 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653487c18f0 0x5653487cc4b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc188005cf0 tx=0x7fc18800cd30 comp rx=0 tx=0).stop 2024-05-14T16:54:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.716+0000 7fc1a3218740 1 -- 172.21.15.2:0/78474847 shutdown_connections 2024-05-14T16:54:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.716+0000 7fc1a3218740 1 --2- 172.21.15.2:0/78474847 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5653487dc700 0x5653487deaf0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.716+0000 7fc1a3218740 1 --2- 172.21.15.2:0/78474847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653487ccad0 0x5653487c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.716+0000 7fc1a3218740 1 --2- 172.21.15.2:0/78474847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653487c2210 0x5653487c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.716+0000 7fc181ffb640 0 client.15255 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:54:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.716+0000 7fc1a3218740 1 --2- 172.21.15.2:0/78474847 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653487c18f0 0x5653487cc4b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.716+0000 7fc1a3218740 1 -- 172.21.15.2:0/78474847 >> 172.21.15.2:0/78474847 conn(0x5653487bde00 msgr2=0x5653487bd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.716+0000 7fc1a3218740 1 -- 172.21.15.2:0/78474847 shutdown_connections 2024-05-14T16:54:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.716+0000 7fc1a3218740 1 -- 172.21.15.2:0/78474847 wait complete. 2024-05-14T16:54:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.716+0000 7fc1a3218740 20 client.15255 trim_cache size 0 max 16384 2024-05-14T16:54:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.734+0000 7fc1a3218740 1 Processor -- start 2024-05-14T16:54:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.734+0000 7fc1a3218740 1 -- start start 2024-05-14T16:54:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.734+0000 7fc1a3218740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653487c18f0 0x5653487c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.734+0000 7fc1a3218740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653487c2210 0x5653487e4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.734+0000 7fc1a3218740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653487ccad0 0x5653487e4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.734+0000 7fc1a3218740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc1981b2f40 con 0x5653487ccad0 2024-05-14T16:54:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.734+0000 7fc1a3218740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc1981b0810 con 0x5653487c18f0 2024-05-14T16:54:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.734+0000 7fc1a3218740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc1981ae660 con 0x5653487c2210 2024-05-14T16:54:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.734+0000 7fc19da9e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653487ccad0 0x5653487e4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.734+0000 7fc19da9e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653487ccad0 0x5653487e4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40544/0 (socket says 172.21.15.2:40544) 2024-05-14T16:54:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.734+0000 7fc19da9e640 1 -- 172.21.15.2:0/1160565230 learned_addr learned my addr 172.21.15.2:0/1160565230 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.734+0000 7fc19da9e640 1 --2- 172.21.15.2:0/1160565230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653487c18f0 0x5653487c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.734+0000 7fc19e29f640 1 --2- 172.21.15.2:0/1160565230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653487c2210 0x5653487e4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.734+0000 7fc19da9e640 1 -- 172.21.15.2:0/1160565230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653487c2210 msgr2=0x5653487e4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.734+0000 7fc19da9e640 1 --2- 172.21.15.2:0/1160565230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653487c2210 0x5653487e4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.734+0000 7fc19da9e640 1 -- 172.21.15.2:0/1160565230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653487c18f0 msgr2=0x5653487c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.734+0000 7fc19da9e640 1 --2- 172.21.15.2:0/1160565230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653487c18f0 0x5653487c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.734+0000 7fc19da9e640 1 -- 172.21.15.2:0/1160565230 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653487c7cf0 con 0x5653487ccad0 2024-05-14T16:54:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.734+0000 7fc19da9e640 1 --2- 172.21.15.2:0/1160565230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653487ccad0 0x5653487e4660 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc188004c70 tx=0x7fc18800c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88560b13f08cc72c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.734+0000 7fc183fff640 1 -- 172.21.15.2:0/1160565230 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc188004880 con 0x5653487ccad0 2024-05-14T16:54:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.734+0000 7fc183fff640 1 -- 172.21.15.2:0/1160565230 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc188025070 con 0x5653487ccad0 2024-05-14T16:54:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.734+0000 7fc183fff640 1 -- 172.21.15.2:0/1160565230 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc188020450 con 0x5653487ccad0 2024-05-14T16:54:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.735+0000 7fc1a3218740 1 -- 172.21.15.2:0/1160565230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653487ccad0 msgr2=0x5653487e4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.735+0000 7fc1a3218740 1 --2- 172.21.15.2:0/1160565230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653487ccad0 0x5653487e4660 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc188004c70 tx=0x7fc18800c4b0 comp rx=0 tx=0).stop 2024-05-14T16:54:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.735+0000 7fc1a3218740 1 -- 172.21.15.2:0/1160565230 shutdown_connections 2024-05-14T16:54:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.735+0000 7fc1a3218740 1 --2- 172.21.15.2:0/1160565230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653487ccad0 0x5653487e4660 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.735+0000 7fc1a3218740 1 --2- 172.21.15.2:0/1160565230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653487c2210 0x5653487e4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.735+0000 7fc1a3218740 1 --2- 172.21.15.2:0/1160565230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653487c18f0 0x5653487c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.735+0000 7fc1a3218740 1 -- 172.21.15.2:0/1160565230 >> 172.21.15.2:0/1160565230 conn(0x5653487bde00 msgr2=0x5653487d7660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.735+0000 7fc1a3218740 1 -- 172.21.15.2:0/1160565230 shutdown_connections 2024-05-14T16:54:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.735+0000 7fc1a3218740 1 -- 172.21.15.2:0/1160565230 wait complete. 2024-05-14T16:54:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.735+0000 7fc1a3218740 1 -- start start 2024-05-14T16:54:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.735+0000 7fc1a3218740 1 Processor -- start 2024-05-14T16:54:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.735+0000 7fc1a3218740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653487c18f0 0x5653487d2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.736+0000 7fc1a3218740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653487c2210 0x5653487d2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.736+0000 7fc1a3218740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653487ccad0 0x5653487d30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.736+0000 7fc1a3218740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc1981aecd0 con 0x5653487ccad0 2024-05-14T16:54:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.736+0000 7fc1a3218740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc1981b0140 con 0x5653487c2210 2024-05-14T16:54:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.736+0000 7fc1a3218740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc1981af3a0 con 0x5653487c18f0 2024-05-14T16:54:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.736+0000 7fc1a3218740 10 client.0 _ll_register_callbacks cb 0x7fc188014880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:54:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.736+0000 7fc19da9e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653487ccad0 0x5653487d30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.736+0000 7fc19da9e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653487ccad0 0x5653487d30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40560/0 (socket says 172.21.15.2:40560) 2024-05-14T16:54:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.736+0000 7fc19da9e640 1 -- 172.21.15.2:0/2595068666 learned_addr learned my addr 172.21.15.2:0/2595068666 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.736+0000 7fc19e29f640 1 --2- 172.21.15.2:0/2595068666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653487c2210 0x5653487d2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.736+0000 7fc19da9e640 1 --2- 172.21.15.2:0/2595068666 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653487c18f0 0x5653487d2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.736+0000 7fc19e29f640 1 -- 172.21.15.2:0/2595068666 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653487c18f0 msgr2=0x5653487d2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.736+0000 7fc19e29f640 1 --2- 172.21.15.2:0/2595068666 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653487c18f0 0x5653487d2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.736+0000 7fc19e29f640 1 -- 172.21.15.2:0/2595068666 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653487ccad0 msgr2=0x5653487d30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:54:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.736+0000 7fc19e29f640 1 --2- 172.21.15.2:0/2595068666 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653487ccad0 0x5653487d30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.736+0000 7fc19e29f640 1 -- 172.21.15.2:0/2595068666 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565348845d60 con 0x5653487c2210 2024-05-14T16:54:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.736+0000 7fc19da9e640 1 --2- 172.21.15.2:0/2595068666 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653487ccad0 0x5653487d30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:54:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.736+0000 7fc19e29f640 1 --2- 172.21.15.2:0/2595068666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653487c2210 0x5653487d2b70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x5653487df0c0 tx=0x7fc194011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abeda9b1508b9f5b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.736+0000 7fc181ffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T16:54:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.737+0000 7fc181ffb640 1 -- 172.21.15.2:0/2595068666 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194005220 con 0x5653487c2210 2024-05-14T16:54:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.737+0000 7fc181ffb640 1 -- 172.21.15.2:0/2595068666 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fc1940053c0 con 0x5653487c2210 2024-05-14T16:54:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.737+0000 7fc1a3218740 1 -- 172.21.15.2:0/2595068666 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc188025070 con 0x5653487c2210 2024-05-14T16:54:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.737+0000 7fc181ffb640 1 -- 172.21.15.2:0/2595068666 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc194030050 con 0x5653487c2210 2024-05-14T16:54:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.737+0000 7fc1a3218740 10 client.25004 fetch_fsmap learned FSMap version 25 2024-05-14T16:54:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.737+0000 7fc1a3218740 1 -- 172.21.15.2:0/2595068666 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5653488470b0 con 0x5653487c2210 2024-05-14T16:54:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.737+0000 7fc181ffb640 1 -- 172.21.15.2:0/2595068666 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc188025070 con 0x5653487c2210 2024-05-14T16:54:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.737+0000 7fc181ffb640 1 -- 172.21.15.2:0/2595068666 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fc194020100 con 0x5653487c2210 2024-05-14T16:54:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.737+0000 7fc1a3218740 10 client.25004 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T16:54:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.737+0000 7fc1a3218740 10 client.25004 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:54:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.737+0000 7fc1a3218740 20 client.25004 populate_metadata read hostname 'smithi002' 2024-05-14T16:54:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.737+0000 7fc1a3218740 1 --2- 172.21.15.2:0/2595068666 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5653488477e0 0x565348849c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.737+0000 7fc1a3218740 4 client.25004 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:54:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.737+0000 7fc1a3218740 1 -- 172.21.15.2:0/2595068666 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5653487b9630 con 0x5653488477e0 2024-05-14T16:54:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.739+0000 7fc19da9e640 1 --2- 172.21.15.2:0/2595068666 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5653488477e0 0x565348849c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.739+0000 7fc19da9e640 1 --2- 172.21.15.2:0/2595068666 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5653488477e0 0x565348849c00 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb11ccd3aef87e5c server_cookie=d4f3391684886eed in_seq=0 out_seq=0 2024-05-14T16:54:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.739+0000 7fc181ffb640 10 client.25004 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:54:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.748+0000 7fc181ffb640 1 -- 172.21.15.2:0/2595068666 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5653487b9630 con 0x5653488477e0 2024-05-14T16:54:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.748+0000 7fc181ffb640 10 client.25004 handle_command_reply: tid=0 2024-05-14T16:54:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.748+0000 7fc1a3218740 1 client.25004 shutdown 2024-05-14T16:54:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.749+0000 7fc1a3218740 1 -- 172.21.15.2:0/2595068666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653487c2210 msgr2=0x5653487d2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.749+0000 7fc1a3218740 1 --2- 172.21.15.2:0/2595068666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653487c2210 0x5653487d2b70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x5653487df0c0 tx=0x7fc194011810 comp rx=0 tx=0).stop 2024-05-14T16:54:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.749+0000 7fc1a3218740 1 -- 172.21.15.2:0/2595068666 shutdown_connections 2024-05-14T16:54:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.749+0000 7fc1a3218740 1 --2- 172.21.15.2:0/2595068666 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5653488477e0 0x565348849c00 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.749+0000 7fc1a3218740 1 --2- 172.21.15.2:0/2595068666 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5653487ccad0 0x5653487d30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.749+0000 7fc1a3218740 1 --2- 172.21.15.2:0/2595068666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653487c2210 0x5653487d2b70 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.749+0000 7fc1a3218740 1 --2- 172.21.15.2:0/2595068666 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5653487c18f0 0x5653487d2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.749+0000 7fc181ffb640 0 client.25004 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:54:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.749+0000 7fc1a3218740 1 -- 172.21.15.2:0/2595068666 >> 172.21.15.2:0/2595068666 conn(0x5653487bde00 msgr2=0x5653487c8780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.749+0000 7fc1a3218740 1 -- 172.21.15.2:0/2595068666 shutdown_connections 2024-05-14T16:54:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.749+0000 7fc1a3218740 1 -- 172.21.15.2:0/2595068666 wait complete. 2024-05-14T16:54:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.749+0000 7fc1a3218740 20 client.25004 trim_cache size 0 max 16384 2024-05-14T16:54:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T16:54:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T16:54:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T16:54:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.749+0000 7fc1a0503640 1 -- 172.21.15.2:0/426195287 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc1840b2ac0 msgr2=0x7fc1840b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.749+0000 7fc1a0503640 1 --2- 172.21.15.2:0/426195287 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc1840b2ac0 0x7fc1840b4f80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc190101010 tx=0x7fc190045fb0 comp rx=0 tx=0).stop 2024-05-14T16:54:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.749+0000 7fc1a0503640 1 -- 172.21.15.2:0/426195287 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc198107980 msgr2=0x7fc198103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:56.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.749+0000 7fc1a0503640 1 --2- 172.21.15.2:0/426195287 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc198107980 0x7fc198103fc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc190005cc0 tx=0x7fc190003c20 comp rx=0 tx=0).stop 2024-05-14T16:54:56.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.750+0000 7fc1a0503640 1 -- 172.21.15.2:0/426195287 shutdown_connections 2024-05-14T16:54:56.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.750+0000 7fc1a0503640 1 --2- 172.21.15.2:0/426195287 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc1840b2ac0 0x7fc1840b4f80 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:56.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.750+0000 7fc1a0503640 1 --2- 172.21.15.2:0/426195287 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc198107980 0x7fc198103fc0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:56.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.750+0000 7fc1a0503640 1 --2- 172.21.15.2:0/426195287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc198102330 0x7fc198103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:56.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.750+0000 7fc1a0503640 1 --2- 172.21.15.2:0/426195287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc198101970 0x7fc198103540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:56.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.750+0000 7fc1a0503640 1 -- 172.21.15.2:0/426195287 >> 172.21.15.2:0/426195287 conn(0x7fc198077cd0 msgr2=0x7fc198105720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:56.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.750+0000 7fc1a0503640 1 -- 172.21.15.2:0/426195287 shutdown_connections 2024-05-14T16:54:56.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:54:56.750+0000 7fc1a0503640 1 -- 172.21.15.2:0/426195287 wait complete. 2024-05-14T16:54:56.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:56 smithi151 ceph-mon[36066]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 43 op/s 2024-05-14T16:54:56.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:56 smithi123 ceph-mon[35971]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 43 op/s 2024-05-14T16:54:57.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.285+0000 7f406fe59700 1 -- 172.21.15.2:0/927215962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4068102a80 msgr2=0x7f4068102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:57.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.285+0000 7f406fe59700 1 --2- 172.21.15.2:0/927215962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4068102a80 0x7f4068102e60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f405c0099c0 tx=0x7f405c009cd0 comp rx=0 tx=0).stop 2024-05-14T16:54:57.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.285+0000 7f406fe59700 1 -- 172.21.15.2:0/927215962 shutdown_connections 2024-05-14T16:54:57.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.285+0000 7f406fe59700 1 --2- 172.21.15.2:0/927215962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40681077d0 0x7f406810f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:57.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.285+0000 7f406fe59700 1 --2- 172.21.15.2:0/927215962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4068102a80 0x7f4068102e60 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:57.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.285+0000 7f406fe59700 1 --2- 172.21.15.2:0/927215962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4068103430 0x7f40681070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:57.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.285+0000 7f406fe59700 1 -- 172.21.15.2:0/927215962 >> 172.21.15.2:0/927215962 conn(0x7f40680793f0 msgr2=0x7f4068079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:57.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.285+0000 7f406fe59700 1 -- 172.21.15.2:0/927215962 shutdown_connections 2024-05-14T16:54:57.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.285+0000 7f406fe59700 1 -- 172.21.15.2:0/927215962 wait complete. 2024-05-14T16:54:57.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.286+0000 7f406fe59700 1 Processor -- start 2024-05-14T16:54:57.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.286+0000 7f406fe59700 1 -- start start 2024-05-14T16:54:57.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.286+0000 7f406fe59700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4068102a80 0x7f406819beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:57.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.286+0000 7f406fe59700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4068103430 0x7f406819c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:57.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.286+0000 7f406fe59700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f40681077d0 0x7f40681a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:57.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.286+0000 7f406fe59700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4068112650 con 0x7f40681077d0 2024-05-14T16:54:57.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.286+0000 7f406fe59700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f40681124d0 con 0x7f4068103430 2024-05-14T16:54:57.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.286+0000 7f406fe59700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f40681127d0 con 0x7f4068102a80 2024-05-14T16:54:57.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.286+0000 7f406e3f6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f40681077d0 0x7f40681a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:57.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.286+0000 7f406e3f6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f40681077d0 0x7f40681a0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40586/0 (socket says 172.21.15.2:40586) 2024-05-14T16:54:57.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.286+0000 7f406e3f6700 1 -- 172.21.15.2:0/4200010654 learned_addr learned my addr 172.21.15.2:0/4200010654 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:54:57.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.286+0000 7f406dbf5700 1 --2- 172.21.15.2:0/4200010654 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4068102a80 0x7f406819beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:57.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.287+0000 7f406d3f4700 1 --2- 172.21.15.2:0/4200010654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4068103430 0x7f406819c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:57.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.287+0000 7f406e3f6700 1 -- 172.21.15.2:0/4200010654 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4068102a80 msgr2=0x7f406819beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:57.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.287+0000 7f406e3f6700 1 --2- 172.21.15.2:0/4200010654 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4068102a80 0x7f406819beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:57.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.287+0000 7f406e3f6700 1 -- 172.21.15.2:0/4200010654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4068103430 msgr2=0x7f406819c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:57.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.287+0000 7f406e3f6700 1 --2- 172.21.15.2:0/4200010654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4068103430 0x7f406819c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:57.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.287+0000 7f406e3f6700 1 -- 172.21.15.2:0/4200010654 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40681a0ec0 con 0x7f40681077d0 2024-05-14T16:54:57.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.287+0000 7f406e3f6700 1 --2- 172.21.15.2:0/4200010654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f40681077d0 0x7f40681a0780 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4060007ee0 tx=0x7f406000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:57.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.287+0000 7f405affd700 1 -- 172.21.15.2:0/4200010654 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406000dbd0 con 0x7f40681077d0 2024-05-14T16:54:57.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.287+0000 7f406fe59700 1 -- 172.21.15.2:0/4200010654 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40681a11b0 con 0x7f40681077d0 2024-05-14T16:54:57.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.287+0000 7f405affd700 1 -- 172.21.15.2:0/4200010654 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f406000dd70 con 0x7f40681077d0 2024-05-14T16:54:57.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.287+0000 7f405affd700 1 -- 172.21.15.2:0/4200010654 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4060005990 con 0x7f40681077d0 2024-05-14T16:54:57.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.287+0000 7f406fe59700 1 -- 172.21.15.2:0/4200010654 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40681a8a30 con 0x7f40681077d0 2024-05-14T16:54:57.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.288+0000 7f406fe59700 1 -- 172.21.15.2:0/4200010654 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4068103b80 con 0x7f40681077d0 2024-05-14T16:54:57.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.290+0000 7f405affd700 1 -- 172.21.15.2:0/4200010654 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 24) ==== 149827+0+0 (secure 0 0 0) 0x7f406001a020 con 0x7f40681077d0 2024-05-14T16:54:57.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.291+0000 7f405affd700 1 --2- 172.21.15.2:0/4200010654 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f40540b2a80 0x7f40540b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:54:57.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.291+0000 7f405affd700 1 -- 172.21.15.2:0/4200010654 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f40600e2550 con 0x7f40681077d0 2024-05-14T16:54:57.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.291+0000 7f406dbf5700 1 --2- 172.21.15.2:0/4200010654 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f40540b2a80 0x7f40540b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:54:57.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.291+0000 7f406dbf5700 1 --2- 172.21.15.2:0/4200010654 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f40540b2a80 0x7f40540b4f40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f405c009990 tx=0x7f405c009650 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:54:57.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.294+0000 7f405affd700 1 -- 172.21.15.2:0/4200010654 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4060010070 con 0x7f40681077d0 2024-05-14T16:54:57.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.569+0000 7f406fe59700 1 -- 172.21.15.2:0/4200010654 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f406802e560 con 0x7f40681077d0 2024-05-14T16:54:57.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.570+0000 7f405affd700 1 -- 172.21.15.2:0/4200010654 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11850 (secure 0 0 0) 0x7f40600b0e20 con 0x7f40681077d0 2024-05-14T16:54:57.570 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:54:57.570 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2275696923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2275696923},{"type":"v1","addr":"172.21.15.2:6835","nonce":2275696923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/716718761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":716718761},{"type":"v1","addr":"172.21.15.2:6839","nonce":716718761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/3644815179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3644815179},{"type":"v1","addr":"172.21.15.2:6841","nonce":3644815179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1880336731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1880336731},{"type":"v1","addr":"172.21.15.151:6833","nonce":1880336731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2415605548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2415605548},{"type":"v1","addr":"172.21.15.2:6837","nonce":2415605548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2635167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2635167},{"type":"v1","addr":"172.21.15.123:6833","nonce":2635167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:51:34.106919+0000","modified":"2024-05-14T16:54:38.886333+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24455,"mds_2":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24415":{"gid":24415,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/762044339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":762044339},{"type":"v1","addr":"172.21.15.151:6835","nonce":762044339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/132043151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":132043151},{"type":"v1","addr":"172.21.15.151:6837","nonce":132043151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/4107834375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":4107834375},{"type":"v1","addr":"172.21.15.151:6839","nonce":4107834375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.123:6835/1198670013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1198670013},{"type":"v1","addr":"172.21.15.123:6835","nonce":1198670013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/333426645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":333426645},{"type":"v1","addr":"172.21.15.123:6837","nonce":333426645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/580275877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":580275877},{"type":"v1","addr":"172.21.15.123:6839","nonce":580275877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24431,24455,24479]},"id":1}]} 2024-05-14T16:54:57.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.573+0000 7f406fe59700 1 -- 172.21.15.2:0/4200010654 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f40540b2a80 msgr2=0x7f40540b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:57.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.573+0000 7f406fe59700 1 --2- 172.21.15.2:0/4200010654 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f40540b2a80 0x7f40540b4f40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f405c009990 tx=0x7f405c009650 comp rx=0 tx=0).stop 2024-05-14T16:54:57.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.573+0000 7f406fe59700 1 -- 172.21.15.2:0/4200010654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f40681077d0 msgr2=0x7f40681a0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:54:57.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.573+0000 7f406fe59700 1 --2- 172.21.15.2:0/4200010654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f40681077d0 0x7f40681a0780 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4060007ee0 tx=0x7f406000d3b0 comp rx=0 tx=0).stop 2024-05-14T16:54:57.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.573+0000 7f406fe59700 1 -- 172.21.15.2:0/4200010654 shutdown_connections 2024-05-14T16:54:57.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.573+0000 7f406fe59700 1 --2- 172.21.15.2:0/4200010654 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f40540b2a80 0x7f40540b4f40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:57.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.573+0000 7f406fe59700 1 --2- 172.21.15.2:0/4200010654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f40681077d0 0x7f40681a0780 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:57.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.573+0000 7f406fe59700 1 --2- 172.21.15.2:0/4200010654 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4068102a80 0x7f406819beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:57.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.573+0000 7f406fe59700 1 --2- 172.21.15.2:0/4200010654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4068103430 0x7f406819c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:54:57.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.573+0000 7f406fe59700 1 -- 172.21.15.2:0/4200010654 >> 172.21.15.2:0/4200010654 conn(0x7f40680793f0 msgr2=0x7f4068101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:54:57.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.573+0000 7f406fe59700 1 -- 172.21.15.2:0/4200010654 shutdown_connections 2024-05-14T16:54:57.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:54:57.573+0000 7f406fe59700 1 -- 172.21.15.2:0/4200010654 wait complete. 2024-05-14T16:54:57.573 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 25 2024-05-14T16:54:58.237 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T16:54:58.489 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:54:58.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:58 smithi002 ceph-mon[31538]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 43 op/s 2024-05-14T16:54:58.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:54:58 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/4200010654' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:54:58.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:58 smithi151 ceph-mon[36066]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 43 op/s 2024-05-14T16:54:58.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:54:58 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/4200010654' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:54:58.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:58 smithi123 ceph-mon[35971]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 43 op/s 2024-05-14T16:54:58.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:54:58 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/4200010654' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:55:00.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.104+0000 7fa6bf102700 1 -- 172.21.15.2:0/3659072396 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6b8076090 msgr2=0x7fa6b8114200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:00.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.104+0000 7fa6bf102700 1 --2- 172.21.15.2:0/3659072396 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6b8076090 0x7fa6b8114200 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4009990 tx=0x7fa6a4009ca0 comp rx=0 tx=0).stop 2024-05-14T16:55:00.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.104+0000 7fa6bf102700 1 -- 172.21.15.2:0/3659072396 shutdown_connections 2024-05-14T16:55:00.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.104+0000 7fa6bf102700 1 --2- 172.21.15.2:0/3659072396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6b8114740 0x7fa6b8116b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:00.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.104+0000 7fa6bf102700 1 --2- 172.21.15.2:0/3659072396 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6b8075770 0x7fa6b8075b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:00.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.104+0000 7fa6bf102700 1 --2- 172.21.15.2:0/3659072396 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6b8076090 0x7fa6b8114200 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:00.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.104+0000 7fa6bf102700 1 -- 172.21.15.2:0/3659072396 >> 172.21.15.2:0/3659072396 conn(0x7fa6b807a6b0 msgr2=0x7fa6b807aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:00.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.104+0000 7fa6bf102700 1 -- 172.21.15.2:0/3659072396 shutdown_connections 2024-05-14T16:55:00.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.104+0000 7fa6bf102700 1 -- 172.21.15.2:0/3659072396 wait complete. 2024-05-14T16:55:00.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.105+0000 7fa6bf102700 1 Processor -- start 2024-05-14T16:55:00.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.105+0000 7fa6bf102700 1 -- start start 2024-05-14T16:55:00.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.105+0000 7fa6bf102700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6b8075770 0x7fa6b81a02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:00.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.105+0000 7fa6bf102700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6b8076090 0x7fa6b81a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:00.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.105+0000 7fa6bf102700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6b8114740 0x7fa6b81a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:00.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.105+0000 7fa6bf102700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b8119d80 con 0x7fa6b8075770 2024-05-14T16:55:00.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.105+0000 7fa6bf102700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b8119c00 con 0x7fa6b8114740 2024-05-14T16:55:00.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.105+0000 7fa6bf102700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b8119f00 con 0x7fa6b8076090 2024-05-14T16:55:00.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.105+0000 7fa6bce9e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6b8075770 0x7fa6b81a02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:00.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.105+0000 7fa6bce9e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6b8075770 0x7fa6b81a02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40608/0 (socket says 172.21.15.2:40608) 2024-05-14T16:55:00.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.105+0000 7fa6bce9e700 1 -- 172.21.15.2:0/3108946973 learned_addr learned my addr 172.21.15.2:0/3108946973 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:00.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.105+0000 7fa6b7fff700 1 --2- 172.21.15.2:0/3108946973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6b8076090 0x7fa6b81a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:00.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.105+0000 7fa6bd69f700 1 --2- 172.21.15.2:0/3108946973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6b8114740 0x7fa6b81a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:00.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.105+0000 7fa6bce9e700 1 -- 172.21.15.2:0/3108946973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6b8076090 msgr2=0x7fa6b81a0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:00.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.105+0000 7fa6bce9e700 1 --2- 172.21.15.2:0/3108946973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6b8076090 0x7fa6b81a0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:00.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.105+0000 7fa6bce9e700 1 -- 172.21.15.2:0/3108946973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6b8114740 msgr2=0x7fa6b81a4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:00.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.105+0000 7fa6bce9e700 1 --2- 172.21.15.2:0/3108946973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6b8114740 0x7fa6b81a4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:00.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.105+0000 7fa6bce9e700 1 -- 172.21.15.2:0/3108946973 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b81a52e0 con 0x7fa6b8075770 2024-05-14T16:55:00.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.106+0000 7fa6bce9e700 1 --2- 172.21.15.2:0/3108946973 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6b8075770 0x7fa6b81a02d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac00d950 tx=0x7fa6ac00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:00.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.106+0000 7fa6b5ffb700 1 -- 172.21.15.2:0/3108946973 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac016070 con 0x7fa6b8075770 2024-05-14T16:55:00.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.106+0000 7fa6b5ffb700 1 -- 172.21.15.2:0/3108946973 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6ac00fd40 con 0x7fa6b8075770 2024-05-14T16:55:00.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.106+0000 7fa6b5ffb700 1 -- 172.21.15.2:0/3108946973 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac00e3b0 con 0x7fa6b8075770 2024-05-14T16:55:00.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.106+0000 7fa6bf102700 1 -- 172.21.15.2:0/3108946973 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6b81a55d0 con 0x7fa6b8075770 2024-05-14T16:55:00.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.106+0000 7fa6bf102700 1 -- 172.21.15.2:0/3108946973 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6b81ace50 con 0x7fa6b8075770 2024-05-14T16:55:00.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.107+0000 7fa6bf102700 1 -- 172.21.15.2:0/3108946973 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6b806a230 con 0x7fa6b8075770 2024-05-14T16:55:00.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.108+0000 7fa6b5ffb700 1 -- 172.21.15.2:0/3108946973 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 24) ==== 149827+0+0 (secure 0 0 0) 0x7fa6ac019070 con 0x7fa6b8075770 2024-05-14T16:55:00.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.109+0000 7fa6b5ffb700 1 --2- 172.21.15.2:0/3108946973 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fa6a80b2b00 0x7fa6a80b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:00.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.109+0000 7fa6b7fff700 1 --2- 172.21.15.2:0/3108946973 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fa6a80b2b00 0x7fa6a80b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:00.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.109+0000 7fa6b5ffb700 1 -- 172.21.15.2:0/3108946973 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fa6ac0e21d0 con 0x7fa6b8075770 2024-05-14T16:55:00.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.110+0000 7fa6b7fff700 1 --2- 172.21.15.2:0/3108946973 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fa6a80b2b00 0x7fa6a80b4fc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8113d10 tx=0x7fa6a4005fd0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:00.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.114+0000 7fa6b5ffb700 1 -- 172.21.15.2:0/3108946973 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa6ac0a9a70 con 0x7fa6b8075770 2024-05-14T16:55:00.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.334+0000 7fa6bf102700 1 -- 172.21.15.2:0/3108946973 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6b81ad080 con 0x7fa6b8075770 2024-05-14T16:55:00.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.335+0000 7fa6b5ffb700 1 -- 172.21.15.2:0/3108946973 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22946 (secure 0 0 0) 0x7fa6ac0a9370 con 0x7fa6b8075770 2024-05-14T16:55:00.335 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:55:00.335 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":81,"fsid":"ed40717a-1210-11ef-bc98-c7b262605968","created":"2024-05-14T16:43:09.415298+0000","modified":"2024-05-14T16:54:41.187623+0000","last_up_change":"2024-05-14T16:50:29.787654+0000","last_in_change":"2024-05-14T16:50:09.287932+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T16:46:38.090903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T16:51:27.296955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T16:51:31.103019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"153f20ec-5bcf-4786-9ede-2e512a1e3d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6803","nonce":2654237820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6805","nonce":2654237820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6809","nonce":2654237820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6807","nonce":2654237820}]},"public_addr":"172.21.15.2:6803/2654237820","cluster_addr":"172.21.15.2:6805/2654237820","heartbeat_back_addr":"172.21.15.2:6809/2654237820","heartbeat_front_addr":"172.21.15.2:6807/2654237820","state":["exists","up"]},{"osd":1,"uuid":"0e95419f-9a9d-4f7a-b55d-ecb54971918a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6801","nonce":2668156992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6803","nonce":2668156992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6807","nonce":2668156992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6805","nonce":2668156992}]},"public_addr":"172.21.15.123:6801/2668156992","cluster_addr":"172.21.15.123:6803/2668156992","heartbeat_back_addr":"172.21.15.123:6807/2668156992","heartbeat_front_addr":"172.21.15.123:6805/2668156992","state":["exists","up"]},{"osd":2,"uuid":"b6ce7ac7-be6c-4549-a356-59cb81b5b492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6801","nonce":2160189636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6803","nonce":2160189636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6807","nonce":2160189636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6805","nonce":2160189636}]},"public_addr":"172.21.15.151:6801/2160189636","cluster_addr":"172.21.15.151:6803/2160189636","heartbeat_back_addr":"172.21.15.151:6807/2160189636","heartbeat_front_addr":"172.21.15.151:6805/2160189636","state":["exists","up"]},{"osd":3,"uuid":"2fd592bf-c212-411a-afbe-a7c498e5ac93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6811","nonce":3451424073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6813","nonce":3451424073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6817","nonce":3451424073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6815","nonce":3451424073}]},"public_addr":"172.21.15.2:6811/3451424073","cluster_addr":"172.21.15.2:6813/3451424073","heartbeat_back_addr":"172.21.15.2:6817/3451424073","heartbeat_front_addr":"172.21.15.2:6815/3451424073","state":["exists","up"]},{"osd":4,"uuid":"af1905e7-e036-47c6-be59-b085bc41fede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6809","nonce":2101935762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6811","nonce":2101935762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6815","nonce":2101935762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6813","nonce":2101935762}]},"public_addr":"172.21.15.123:6809/2101935762","cluster_addr":"172.21.15.123:6811/2101935762","heartbeat_back_addr":"172.21.15.123:6815/2101935762","heartbeat_front_addr":"172.21.15.123:6813/2101935762","state":["exists","up"]},{"osd":5,"uuid":"9c166916-af78-4cc2-bd14-d1220b6e7d9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6809","nonce":3695961886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6811","nonce":3695961886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6815","nonce":3695961886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6813","nonce":3695961886}]},"public_addr":"172.21.15.151:6809/3695961886","cluster_addr":"172.21.15.151:6811/3695961886","heartbeat_back_addr":"172.21.15.151:6815/3695961886","heartbeat_front_addr":"172.21.15.151:6813/3695961886","state":["exists","up"]},{"osd":6,"uuid":"60e9bf66-2338-41e8-9280-398d4e3b322f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6819","nonce":2711893600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6821","nonce":2711893600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6825","nonce":2711893600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6823","nonce":2711893600}]},"public_addr":"172.21.15.2:6819/2711893600","cluster_addr":"172.21.15.2:6821/2711893600","heartbeat_back_addr":"172.21.15.2:6825/2711893600","heartbeat_front_addr":"172.21.15.2:6823/2711893600","state":["exists","up"]},{"osd":7,"uuid":"c93e07a1-e941-493e-8468-33429df2dff0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6817","nonce":3634155507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6819","nonce":3634155507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6823","nonce":3634155507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6821","nonce":3634155507}]},"public_addr":"172.21.15.123:6817/3634155507","cluster_addr":"172.21.15.123:6819/3634155507","heartbeat_back_addr":"172.21.15.123:6823/3634155507","heartbeat_front_addr":"172.21.15.123:6821/3634155507","state":["exists","up"]},{"osd":8,"uuid":"1155fb58-d614-4f45-91e2-5da3a93e7d9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6817","nonce":4271107010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6819","nonce":4271107010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6823","nonce":4271107010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6821","nonce":4271107010}]},"public_addr":"172.21.15.151:6817/4271107010","cluster_addr":"172.21.15.151:6819/4271107010","heartbeat_back_addr":"172.21.15.151:6823/4271107010","heartbeat_front_addr":"172.21.15.151:6821/4271107010","state":["exists","up"]},{"osd":9,"uuid":"82b34e3b-0017-4ca4-8f86-0993ca78ce7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6827","nonce":3479431021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6829","nonce":3479431021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6833","nonce":3479431021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6831","nonce":3479431021}]},"public_addr":"172.21.15.2:6827/3479431021","cluster_addr":"172.21.15.2:6829/3479431021","heartbeat_back_addr":"172.21.15.2:6833/3479431021","heartbeat_front_addr":"172.21.15.2:6831/3479431021","state":["exists","up"]},{"osd":10,"uuid":"32e2ace9-eb5d-4258-a2ac-49b782be2b5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6825","nonce":4266374435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6827","nonce":4266374435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6831","nonce":4266374435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6829","nonce":4266374435}]},"public_addr":"172.21.15.123:6825/4266374435","cluster_addr":"172.21.15.123:6827/4266374435","heartbeat_back_addr":"172.21.15.123:6831/4266374435","heartbeat_front_addr":"172.21.15.123:6829/4266374435","state":["exists","up"]},{"osd":11,"uuid":"6e3c2ead-1af6-4bda-87cd-12770c81fa67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6825","nonce":4129194698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6827","nonce":4129194698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6831","nonce":4129194698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6829","nonce":4129194698}]},"public_addr":"172.21.15.151:6825/4129194698","cluster_addr":"172.21.15.151:6827/4129194698","heartbeat_back_addr":"172.21.15.151:6831/4129194698","heartbeat_front_addr":"172.21.15.151:6829/4129194698","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:45:46.540023+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:46:10.845186+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:46:34.283112+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:47:00.072467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:47:25.315644+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:47:49.748688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:48:16.964258+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:48:42.875515+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:49:08.596500+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:49:35.450943+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:50:01.293368+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:50:27.793713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/2907436778":"2024-05-15T16:54:41.187591+0000","172.21.15.2:0/2111644395":"2024-05-15T16:54:41.187591+0000","172.21.15.2:6801/1498936028":"2024-05-15T16:54:41.187591+0000","172.21.15.2:6800/1498936028":"2024-05-15T16:54:41.187591+0000","172.21.15.2:0/156580230":"2024-05-15T16:54:41.187591+0000","172.21.15.2:0/4267809161":"2024-05-15T16:44:11.998214+0000","172.21.15.2:0/4090012382":"2024-05-15T16:54:41.187591+0000","172.21.15.2:6801/3582988344":"2024-05-15T16:44:11.998214+0000","172.21.15.2:6800/2574015037":"2024-05-15T16:43:43.407184+0000","172.21.15.2:0/2100732571":"2024-05-15T16:44:11.998214+0000","172.21.15.2:0/2954199324":"2024-05-15T16:43:43.407184+0000","172.21.15.2:6801/2574015037":"2024-05-15T16:43:43.407184+0000","172.21.15.2:0/3171550944":"2024-05-15T16:43:43.407184+0000","172.21.15.2:0/1679761136":"2024-05-15T16:44:11.998214+0000","172.21.15.2:0/1746766591":"2024-05-15T16:43:43.407184+0000","172.21.15.2:6800/3582988344":"2024-05-15T16:44:11.998214+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T16:55:00.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.337+0000 7fa6bf102700 1 -- 172.21.15.2:0/3108946973 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fa6a80b2b00 msgr2=0x7fa6a80b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:00.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.337+0000 7fa6bf102700 1 --2- 172.21.15.2:0/3108946973 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fa6a80b2b00 0x7fa6a80b4fc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8113d10 tx=0x7fa6a4005fd0 comp rx=0 tx=0).stop 2024-05-14T16:55:00.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.337+0000 7fa6bf102700 1 -- 172.21.15.2:0/3108946973 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6b8075770 msgr2=0x7fa6b81a02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:00.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.337+0000 7fa6bf102700 1 --2- 172.21.15.2:0/3108946973 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6b8075770 0x7fa6b81a02d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac00d950 tx=0x7fa6ac00dc60 comp rx=0 tx=0).stop 2024-05-14T16:55:00.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.338+0000 7fa6bf102700 1 -- 172.21.15.2:0/3108946973 shutdown_connections 2024-05-14T16:55:00.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.338+0000 7fa6bf102700 1 --2- 172.21.15.2:0/3108946973 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fa6a80b2b00 0x7fa6a80b4fc0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:00.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.338+0000 7fa6bf102700 1 --2- 172.21.15.2:0/3108946973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6b8114740 0x7fa6b81a4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:00.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.338+0000 7fa6bf102700 1 --2- 172.21.15.2:0/3108946973 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6b8075770 0x7fa6b81a02d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:00.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.338+0000 7fa6bf102700 1 --2- 172.21.15.2:0/3108946973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6b8076090 0x7fa6b81a0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:00.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.338+0000 7fa6bf102700 1 -- 172.21.15.2:0/3108946973 >> 172.21.15.2:0/3108946973 conn(0x7fa6b807a6b0 msgr2=0x7fa6b81101e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:00.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.338+0000 7fa6bf102700 1 -- 172.21.15.2:0/3108946973 shutdown_connections 2024-05-14T16:55:00.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:00.338+0000 7fa6bf102700 1 -- 172.21.15.2:0/3108946973 wait complete. 2024-05-14T16:55:00.740 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:00 smithi002 ceph-mon[31538]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 78 KiB/s wr, 49 op/s 2024-05-14T16:55:00.740 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:00 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/3108946973' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:55:00.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:00 smithi151 ceph-mon[36066]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 78 KiB/s wr, 49 op/s 2024-05-14T16:55:00.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:00 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/3108946973' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:55:00.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:00 smithi123 ceph-mon[35971]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 78 KiB/s wr, 49 op/s 2024-05-14T16:55:00.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:00 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/3108946973' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T16:55:00.981 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-05-14T16:55:00.994 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:55:00 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:55:00.739+0000 7f6725310700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T16:55:00.994 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:55:00 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:55:00.739+0000 7f6725310700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T16:55:00.994 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:55:00 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:55:00.739+0000 7f6725310700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T16:55:00.994 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:55:00 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:55:00.739+0000 7f6725310700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T16:55:00.994 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 16:55:00 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T16:55:00.739+0000 7f6725310700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-14T16:55:01.232 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:55:02.677 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:02 smithi002 ceph-mon[31538]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-05-14T16:55:02.677 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:02 smithi002 ceph-mon[31538]: mgrmap e25: x(active, since 20s), standbys: z, y 2024-05-14T16:55:02.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:02 smithi151 ceph-mon[36066]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-05-14T16:55:02.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:02 smithi151 ceph-mon[36066]: mgrmap e25: x(active, since 20s), standbys: z, y 2024-05-14T16:55:02.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:02 smithi123 ceph-mon[35971]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-05-14T16:55:02.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:02 smithi123 ceph-mon[35971]: mgrmap e25: x(active, since 20s), standbys: z, y 2024-05-14T16:55:03.601 INFO:teuthology.orchestra.run.smithi002.stdout:{"run_dir":"/var/run/ceph"} 2024-05-14T16:55:03.602 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-05-14T16:55:03.602 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 710 2024-05-14T16:55:03.603 DEBUG:tasks.quiescer:Yielding 2024-05-14T16:55:03.603 INFO:teuthology.run_tasks:Running task exec... 2024-05-14T16:55:03.613 INFO:teuthology.task.exec:Executing custom commands... 2024-05-14T16:55:03.613 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi002.front.sepia.ceph.com 2024-05-14T16:55:03.613 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2024-05-14T16:55:03.614 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:55:03.683 INFO:teuthology.orchestra.run.smithi002.stdout:total 0 2024-05-14T16:55:03.696 INFO:teuthology.orchestra.run.smithi002.stdout:Filesystem Size Used Avail Use% Mounted on 2024-05-14T16:55:03.696 INFO:teuthology.orchestra.run.smithi002.stdout:0@ed40717a-1210-11ef-bc98-c7b262605968.cephfs=/volumes/qa/sv_1/83bdfcd3-fc78-423b-b57a-a691f0db09d1 340G 0 340G 0% /tmp/cephfs 2024-05-14T16:55:03.699 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2024-05-14T16:55:03.793 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2024-05-14T16:55:03.837 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2024-05-14T16:55:03.906 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:55:04.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:04 smithi002 ceph-mon[31538]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-05-14T16:55:04.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:04 smithi151 ceph-mon[36066]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-05-14T16:55:04.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:04 smithi123 ceph-mon[35971]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-05-14T16:55:05.710 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2024-05-14T16:55:05.790 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2024-05-14T16:55:05.792 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2024-05-14T16:55:05.873 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-05-14T16:55:05.954 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/: 2024-05-14T16:55:05.954 INFO:teuthology.orchestra.run.smithi002.stdout:total 0 2024-05-14T16:55:05.954 INFO:teuthology.orchestra.run.smithi002.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 16:55 . 2024-05-14T16:55:05.954 INFO:teuthology.orchestra.run.smithi002.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 May 14 16:55 .. 2024-05-14T16:55:05.954 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 data 2024-05-14T16:55:05.954 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:55:05.955 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data: 2024-05-14T16:55:05.955 INFO:teuthology.orchestra.run.smithi002.stdout:total 0 2024-05-14T16:55:05.955 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 . 2024-05-14T16:55:05.955 INFO:teuthology.orchestra.run.smithi002.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 16:55 .. 2024-05-14T16:55:05.956 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2024-05-14T16:55:06.041 INFO:teuthology.orchestra.run.smithi002.stdout:[Service] 2024-05-14T16:55:06.041 INFO:teuthology.orchestra.run.smithi002.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2024-05-14T16:55:06.042 INFO:teuthology.orchestra.run.smithi002.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2024-05-14T16:55:06.044 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2024-05-14T16:55:06.143 INFO:teuthology.orchestra.run.smithi002.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2024-05-14T16:55:06.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.461+0000 7f7869898700 1 -- 172.21.15.2:0/2070400747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7864102a80 msgr2=0x7f7864102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:06.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.461+0000 7f7869898700 1 --2- 172.21.15.2:0/2070400747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7864102a80 0x7f7864102e60 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f78540099c0 tx=0x7f7854009cd0 comp rx=0 tx=0).stop 2024-05-14T16:55:06.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.462+0000 7f7869898700 1 -- 172.21.15.2:0/2070400747 shutdown_connections 2024-05-14T16:55:06.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.462+0000 7f7869898700 1 --2- 172.21.15.2:0/2070400747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78641077d0 0x7f786410f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:06.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.462+0000 7f7869898700 1 --2- 172.21.15.2:0/2070400747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7864102a80 0x7f7864102e60 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:06.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.462+0000 7f7869898700 1 --2- 172.21.15.2:0/2070400747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7864103430 0x7f78641070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:06.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.462+0000 7f7869898700 1 -- 172.21.15.2:0/2070400747 >> 172.21.15.2:0/2070400747 conn(0x7f78640793f0 msgr2=0x7f7864079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:06.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.462+0000 7f7869898700 1 -- 172.21.15.2:0/2070400747 shutdown_connections 2024-05-14T16:55:06.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.462+0000 7f7869898700 1 -- 172.21.15.2:0/2070400747 wait complete. 2024-05-14T16:55:06.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.462+0000 7f7869898700 1 Processor -- start 2024-05-14T16:55:06.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.462+0000 7f7869898700 1 -- start start 2024-05-14T16:55:06.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.463+0000 7f7869898700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7864102a80 0x7f7864104140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:06.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.463+0000 7f7869898700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7864103430 0x7f7864104680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:06.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.463+0000 7f7869898700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78641077d0 0x7f7864104bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:06.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.463+0000 7f7869898700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7864112630 con 0x7f7864103430 2024-05-14T16:55:06.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.463+0000 7f7869898700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f78641124b0 con 0x7f7864102a80 2024-05-14T16:55:06.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.463+0000 7f7869898700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f78641127b0 con 0x7f78641077d0 2024-05-14T16:55:06.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.463+0000 7f78627fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7864103430 0x7f7864104680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:06.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.463+0000 7f78627fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7864103430 0x7f7864104680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46080/0 (socket says 172.21.15.2:46080) 2024-05-14T16:55:06.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.463+0000 7f7862ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7864102a80 0x7f7864104140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:06.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.463+0000 7f78627fc700 1 -- 172.21.15.2:0/2524780733 learned_addr learned my addr 172.21.15.2:0/2524780733 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:06.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.463+0000 7f78637fe700 1 --2- 172.21.15.2:0/2524780733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78641077d0 0x7f7864104bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:06.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.463+0000 7f78627fc700 1 -- 172.21.15.2:0/2524780733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78641077d0 msgr2=0x7f7864104bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:06.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.463+0000 7f78627fc700 1 --2- 172.21.15.2:0/2524780733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78641077d0 0x7f7864104bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:06.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.463+0000 7f78627fc700 1 -- 172.21.15.2:0/2524780733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7864102a80 msgr2=0x7f7864104140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:06.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.463+0000 7f78627fc700 1 --2- 172.21.15.2:0/2524780733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7864102a80 0x7f7864104140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:06.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.463+0000 7f78627fc700 1 -- 172.21.15.2:0/2524780733 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78641a8110 con 0x7f7864103430 2024-05-14T16:55:06.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.463+0000 7f7862ffd700 1 --2- 172.21.15.2:0/2524780733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7864102a80 0x7f7864104140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:55:06.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.463+0000 7f78627fc700 1 --2- 172.21.15.2:0/2524780733 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7864103430 0x7f7864104680 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f785800c9b0 tx=0x7f785800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:06.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.464+0000 7f7868896700 1 -- 172.21.15.2:0/2524780733 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7858007aa0 con 0x7f7864103430 2024-05-14T16:55:06.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.464+0000 7f7869898700 1 -- 172.21.15.2:0/2524780733 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78641a8400 con 0x7f7864103430 2024-05-14T16:55:06.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.464+0000 7f7868896700 1 -- 172.21.15.2:0/2524780733 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f785800ce50 con 0x7f7864103430 2024-05-14T16:55:06.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.464+0000 7f7868896700 1 -- 172.21.15.2:0/2524780733 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7858018660 con 0x7f7864103430 2024-05-14T16:55:06.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.464+0000 7f7869898700 1 -- 172.21.15.2:0/2524780733 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78641a8990 con 0x7f7864103430 2024-05-14T16:55:06.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.465+0000 7f7869898700 1 -- 172.21.15.2:0/2524780733 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7864103b80 con 0x7f7864103430 2024-05-14T16:55:06.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.467+0000 7f7868896700 1 -- 172.21.15.2:0/2524780733 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) ==== 149884+0+0 (secure 0 0 0) 0x7f785800f450 con 0x7f7864103430 2024-05-14T16:55:06.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.468+0000 7f7868896700 1 --2- 172.21.15.2:0/2524780733 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f784c0b2b10 0x7f784c0b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:06.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.468+0000 7f7868896700 1 -- 172.21.15.2:0/2524780733 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f78580e2900 con 0x7f7864103430 2024-05-14T16:55:06.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.468+0000 7f7862ffd700 1 --2- 172.21.15.2:0/2524780733 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f784c0b2b10 0x7f784c0b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:06.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.468+0000 7f7862ffd700 1 --2- 172.21.15.2:0/2524780733 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f784c0b2b10 0x7f784c0b4fd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7854003760 tx=0x7f7854005fb0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:06.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.471+0000 7f7868896700 1 -- 172.21.15.2:0/2524780733 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7858054df0 con 0x7f7864103430 2024-05-14T16:55:06.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.739+0000 7f7869898700 1 -- 172.21.15.2:0/2524780733 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f786402e560 con 0x7f7864103430 2024-05-14T16:55:06.739 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:06 smithi002 ceph-mon[31538]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-05-14T16:55:06.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.740+0000 7f7868896700 1 -- 172.21.15.2:0/2524780733 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11850 (secure 0 0 0) 0x7f7858014030 con 0x7f7864103430 2024-05-14T16:55:06.740 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:55:06.740 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2275696923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2275696923},{"type":"v1","addr":"172.21.15.2:6835","nonce":2275696923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/716718761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":716718761},{"type":"v1","addr":"172.21.15.2:6839","nonce":716718761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/3644815179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3644815179},{"type":"v1","addr":"172.21.15.2:6841","nonce":3644815179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1880336731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1880336731},{"type":"v1","addr":"172.21.15.151:6833","nonce":1880336731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2415605548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2415605548},{"type":"v1","addr":"172.21.15.2:6837","nonce":2415605548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2635167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2635167},{"type":"v1","addr":"172.21.15.123:6833","nonce":2635167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:51:34.106919+0000","modified":"2024-05-14T16:54:38.886333+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24455,"mds_2":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24415":{"gid":24415,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/762044339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":762044339},{"type":"v1","addr":"172.21.15.151:6835","nonce":762044339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/132043151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":132043151},{"type":"v1","addr":"172.21.15.151:6837","nonce":132043151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/4107834375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":4107834375},{"type":"v1","addr":"172.21.15.151:6839","nonce":4107834375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.123:6835/1198670013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1198670013},{"type":"v1","addr":"172.21.15.123:6835","nonce":1198670013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/333426645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":333426645},{"type":"v1","addr":"172.21.15.123:6837","nonce":333426645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/580275877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":580275877},{"type":"v1","addr":"172.21.15.123:6839","nonce":580275877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24431,24455,24479]},"id":1}]} 2024-05-14T16:55:06.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.742+0000 7f7869898700 1 -- 172.21.15.2:0/2524780733 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f784c0b2b10 msgr2=0x7f784c0b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:06.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.742+0000 7f7869898700 1 --2- 172.21.15.2:0/2524780733 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f784c0b2b10 0x7f784c0b4fd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7854003760 tx=0x7f7854005fb0 comp rx=0 tx=0).stop 2024-05-14T16:55:06.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.742+0000 7f7869898700 1 -- 172.21.15.2:0/2524780733 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7864103430 msgr2=0x7f7864104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:06.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.742+0000 7f7869898700 1 --2- 172.21.15.2:0/2524780733 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7864103430 0x7f7864104680 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f785800c9b0 tx=0x7f785800ccc0 comp rx=0 tx=0).stop 2024-05-14T16:55:06.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.742+0000 7f7869898700 1 -- 172.21.15.2:0/2524780733 shutdown_connections 2024-05-14T16:55:06.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.742+0000 7f7869898700 1 --2- 172.21.15.2:0/2524780733 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f784c0b2b10 0x7f784c0b4fd0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:06.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.742+0000 7f7869898700 1 --2- 172.21.15.2:0/2524780733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78641077d0 0x7f7864104bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:06.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.742+0000 7f7869898700 1 --2- 172.21.15.2:0/2524780733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7864102a80 0x7f7864104140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:06.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.742+0000 7f7869898700 1 --2- 172.21.15.2:0/2524780733 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7864103430 0x7f7864104680 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:06.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.742+0000 7f7869898700 1 -- 172.21.15.2:0/2524780733 >> 172.21.15.2:0/2524780733 conn(0x7f78640793f0 msgr2=0x7f786406bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:06.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.743+0000 7f7869898700 1 -- 172.21.15.2:0/2524780733 shutdown_connections 2024-05-14T16:55:06.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:06.743+0000 7f7869898700 1 -- 172.21.15.2:0/2524780733 wait complete. 2024-05-14T16:55:06.743 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 25 2024-05-14T16:55:06.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:06 smithi151 ceph-mon[36066]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-05-14T16:55:06.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:06 smithi123 ceph-mon[35971]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-05-14T16:55:07.450 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-14T16:51:34.106919+0000', 'modified': '2024-05-14T16:54:38.886333+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24479, 'mds_1': 24455, 'mds_2': 24431}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24415': {'gid': 24415, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.151:6835/762044339', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6834', 'nonce': 762044339}, {'type': 'v1', 'addr': '172.21.15.151:6835', 'nonce': 762044339}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24421': {'gid': 24421, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.151:6837/132043151', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6836', 'nonce': 132043151}, {'type': 'v1', 'addr': '172.21.15.151:6837', 'nonce': 132043151}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24427': {'gid': 24427, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.151:6839/4107834375', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6838', 'nonce': 4107834375}, {'type': 'v1', 'addr': '172.21.15.151:6839', 'nonce': 4107834375}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24431': {'gid': 24431, 'name': 'e', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.123:6835/1198670013', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6834', 'nonce': 1198670013}, {'type': 'v1', 'addr': '172.21.15.123:6835', 'nonce': 1198670013}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'h', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.123:6837/333426645', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6836', 'nonce': 333426645}, {'type': 'v1', 'addr': '172.21.15.123:6837', 'nonce': 333426645}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.123:6839/580275877', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6838', 'nonce': 580275877}, {'type': 'v1', 'addr': '172.21.15.123:6839', 'nonce': 580275877}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24479, 'qdb_cluster': [24431, 24455, 24479]} 2024-05-14T16:55:07.450 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-14T16:55:07.450 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph tell mds.e status 2024-05-14T16:55:07.708 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:55:07.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:07 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/2524780733' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:55:07.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:07 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/2524780733' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:55:07.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:07 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/2524780733' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:55:08.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:08 smithi002 ceph-mon[31538]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 29 KiB/s wr, 25 op/s 2024-05-14T16:55:08.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:08 smithi151 ceph-mon[36066]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 29 KiB/s wr, 25 op/s 2024-05-14T16:55:08.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:08 smithi123 ceph-mon[35971]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 29 KiB/s wr, 25 op/s 2024-05-14T16:55:09.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.392+0000 7fcd7caad700 1 -- 172.21.15.2:0/3371555186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd781088f0 msgr2=0x7fcd78110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:09.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.392+0000 7fcd7caad700 1 --2- 172.21.15.2:0/3371555186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd781088f0 0x7fcd78110470 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fcd7000b0d0 tx=0x7fcd7000b3e0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.392+0000 7fcd7caad700 1 -- 172.21.15.2:0/3371555186 shutdown_connections 2024-05-14T16:55:09.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.392+0000 7fcd7caad700 1 --2- 172.21.15.2:0/3371555186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd781088f0 0x7fcd78110470 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.392+0000 7fcd7caad700 1 --2- 172.21.15.2:0/3371555186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd781037f0 0x7fcd78103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.392+0000 7fcd7caad700 1 --2- 172.21.15.2:0/3371555186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd781041a0 0x7fcd781081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.392+0000 7fcd7caad700 1 -- 172.21.15.2:0/3371555186 >> 172.21.15.2:0/3371555186 conn(0x7fcd7807a6a0 msgr2=0x7fcd7807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:09.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.392+0000 7fcd7caad700 1 -- 172.21.15.2:0/3371555186 shutdown_connections 2024-05-14T16:55:09.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.392+0000 7fcd7caad700 1 -- 172.21.15.2:0/3371555186 wait complete. 2024-05-14T16:55:09.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.393+0000 7fcd7caad700 1 Processor -- start 2024-05-14T16:55:09.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.393+0000 7fcd7caad700 1 -- start start 2024-05-14T16:55:09.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.393+0000 7fcd7caad700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd781037f0 0x7fcd7819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:09.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.393+0000 7fcd7caad700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd781041a0 0x7fcd7819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:09.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.393+0000 7fcd7caad700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd781088f0 0x7fcd781999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:09.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.393+0000 7fcd7caad700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcd78113710 con 0x7fcd781037f0 2024-05-14T16:55:09.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.393+0000 7fcd7caad700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcd78113590 con 0x7fcd781041a0 2024-05-14T16:55:09.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.393+0000 7fcd7caad700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcd78113890 con 0x7fcd781088f0 2024-05-14T16:55:09.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.393+0000 7fcd7659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd781037f0 0x7fcd7819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:09.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.393+0000 7fcd75d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd781041a0 0x7fcd7819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:09.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.393+0000 7fcd7659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd781037f0 0x7fcd7819f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46108/0 (socket says 172.21.15.2:46108) 2024-05-14T16:55:09.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.393+0000 7fcd7659c700 1 -- 172.21.15.2:0/1070567194 learned_addr learned my addr 172.21.15.2:0/1070567194 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:09.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.393+0000 7fcd76d9d700 1 --2- 172.21.15.2:0/1070567194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd781088f0 0x7fcd781999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:09.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.394+0000 7fcd75d9b700 1 -- 172.21.15.2:0/1070567194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd781088f0 msgr2=0x7fcd781999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:09.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.394+0000 7fcd75d9b700 1 --2- 172.21.15.2:0/1070567194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd781088f0 0x7fcd781999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.394+0000 7fcd75d9b700 1 -- 172.21.15.2:0/1070567194 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd781037f0 msgr2=0x7fcd7819f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:09.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.394+0000 7fcd75d9b700 1 --2- 172.21.15.2:0/1070567194 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd781037f0 0x7fcd7819f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.394+0000 7fcd75d9b700 1 -- 172.21.15.2:0/1070567194 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd7819a130 con 0x7fcd781041a0 2024-05-14T16:55:09.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.394+0000 7fcd7659c700 1 --2- 172.21.15.2:0/1070567194 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd781037f0 0x7fcd7819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:55:09.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.394+0000 7fcd75d9b700 1 --2- 172.21.15.2:0/1070567194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd781041a0 0x7fcd7819fd00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00d950 tx=0x7fcd6c00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:09.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.394+0000 7fcd677fe700 1 -- 172.21.15.2:0/1070567194 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c016070 con 0x7fcd781041a0 2024-05-14T16:55:09.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.394+0000 7fcd7caad700 1 -- 172.21.15.2:0/1070567194 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd7819a420 con 0x7fcd781041a0 2024-05-14T16:55:09.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.394+0000 7fcd677fe700 1 -- 172.21.15.2:0/1070567194 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd6c00fd40 con 0x7fcd781041a0 2024-05-14T16:55:09.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.394+0000 7fcd677fe700 1 -- 172.21.15.2:0/1070567194 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c00e3b0 con 0x7fcd781041a0 2024-05-14T16:55:09.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.394+0000 7fcd7caad700 1 -- 172.21.15.2:0/1070567194 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd781a6690 con 0x7fcd781041a0 2024-05-14T16:55:09.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.399+0000 7fcd677fe700 1 -- 172.21.15.2:0/1070567194 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 149884+0+0 (secure 0 0 0) 0x7fcd6c019070 con 0x7fcd781041a0 2024-05-14T16:55:09.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.399+0000 7fcd87343b80 1 Processor -- start 2024-05-14T16:55:09.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.399+0000 7fcd87343b80 1 -- start start 2024-05-14T16:55:09.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.399+0000 7fcd87343b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564900c8e380 0x564900c8e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:09.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.399+0000 7fcd87343b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564900c8eca0 0x564900c88bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:09.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.399+0000 7fcd87343b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564900c891d0 0x564900c89630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:09.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.399+0000 7fcd87343b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x564900be2d20 con 0x564900c8eca0 2024-05-14T16:55:09.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.399+0000 7fcd87343b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564900bd28c0 con 0x564900c8e380 2024-05-14T16:55:09.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.399+0000 7fcd87343b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564900bd6900 con 0x564900c891d0 2024-05-14T16:55:09.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.400+0000 7fcd7659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564900c8eca0 0x564900c88bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:09.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.400+0000 7fcd76d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564900c8e380 0x564900c8e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:09.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.400+0000 7fcd76d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564900c8e380 0x564900c8e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:52260/0 (socket says 172.21.15.2:52260) 2024-05-14T16:55:09.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.400+0000 7fcd76d9d700 1 -- 172.21.15.2:0/3201837514 learned_addr learned my addr 172.21.15.2:0/3201837514 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:09.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.400+0000 7fcd677fe700 1 --2- 172.21.15.2:0/1070567194 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcd600b2b10 0x7fcd600b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:09.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.400+0000 7fcd75d9b700 1 --2- 172.21.15.2:0/3201837514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564900c891d0 0x564900c89630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:09.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.400+0000 7fcd76d9d700 1 -- 172.21.15.2:0/3201837514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564900c891d0 msgr2=0x564900c89630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:09.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.400+0000 7fcd76d9d700 1 --2- 172.21.15.2:0/3201837514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564900c891d0 0x564900c89630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.400+0000 7fcd76d9d700 1 -- 172.21.15.2:0/3201837514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564900c8eca0 msgr2=0x564900c88bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:09.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.400+0000 7fcd76d9d700 1 --2- 172.21.15.2:0/3201837514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564900c8eca0 0x564900c88bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.400+0000 7fcd76d9d700 1 -- 172.21.15.2:0/3201837514 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564900b7bf60 con 0x564900c8e380 2024-05-14T16:55:09.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.400+0000 7fcd677fe700 1 -- 172.21.15.2:0/1070567194 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fcd6c0e22b0 con 0x7fcd781041a0 2024-05-14T16:55:09.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.400+0000 7fcd76d9d700 1 --2- 172.21.15.2:0/1070567194 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcd600b2b10 0x7fcd600b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:09.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.400+0000 7fcd76d9d700 1 --2- 172.21.15.2:0/3201837514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564900c8e380 0x564900c8e760 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcd70009d10 tx=0x7fcd70009d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4cb6a405a304a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:09.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.400+0000 7fcd64ff9700 1 -- 172.21.15.2:0/3201837514 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7002d070 con 0x564900c8e380 2024-05-14T16:55:09.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.401+0000 7fcd64ff9700 1 -- 172.21.15.2:0/3201837514 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd70022070 con 0x564900c8e380 2024-05-14T16:55:09.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.401+0000 7fcd64ff9700 1 -- 172.21.15.2:0/3201837514 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd700076e0 con 0x564900c8e380 2024-05-14T16:55:09.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.401+0000 7fcd87343b80 1 -- 172.21.15.2:0/3201837514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564900c8e380 msgr2=0x564900c8e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:09.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.401+0000 7fcd76d9d700 1 --2- 172.21.15.2:0/1070567194 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcd600b2b10 0x7fcd600b4fd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcd70009e60 tx=0x7fcd70027000 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:09.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.401+0000 7fcd87343b80 1 --2- 172.21.15.2:0/3201837514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564900c8e380 0x564900c8e760 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcd70009d10 tx=0x7fcd70009d40 comp rx=0 tx=0).stop 2024-05-14T16:55:09.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.401+0000 7fcd87343b80 1 -- 172.21.15.2:0/3201837514 shutdown_connections 2024-05-14T16:55:09.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.401+0000 7fcd87343b80 1 --2- 172.21.15.2:0/3201837514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564900c891d0 0x564900c89630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.401+0000 7fcd87343b80 1 --2- 172.21.15.2:0/3201837514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564900c8e380 0x564900c8e760 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.401+0000 7fcd87343b80 1 --2- 172.21.15.2:0/3201837514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564900c8eca0 0x564900c88bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.401+0000 7fcd87343b80 1 -- 172.21.15.2:0/3201837514 >> 172.21.15.2:0/3201837514 conn(0x564900bb1d60 msgr2=0x564900c7ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:09.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.401+0000 7fcd87343b80 1 -- 172.21.15.2:0/3201837514 shutdown_connections 2024-05-14T16:55:09.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.401+0000 7fcd87343b80 1 -- 172.21.15.2:0/3201837514 wait complete. 2024-05-14T16:55:09.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.402+0000 7fcd87343b80 1 -- start start 2024-05-14T16:55:09.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.402+0000 7fcd87343b80 1 Processor -- start 2024-05-14T16:55:09.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.402+0000 7fcd87343b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564900c891d0 0x564900c94b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:09.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.402+0000 7fcd87343b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564900c8e380 0x564900c91eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:09.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.402+0000 7fcd87343b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564900c8eca0 0x564900c923f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:09.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.402+0000 7fcd87343b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x564900ad8f50 con 0x564900c8eca0 2024-05-14T16:55:09.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.402+0000 7fcd87343b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564900b3ad50 con 0x564900c8e380 2024-05-14T16:55:09.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.402+0000 7fcd87343b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564900c95760 con 0x564900c891d0 2024-05-14T16:55:09.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.402+0000 7fcd87343b80 10 client.0 _ll_register_callbacks cb 0x564900b6d900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:55:09.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.402+0000 7fcd75d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564900c8e380 0x564900c91eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:09.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.402+0000 7fcd75d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564900c8e380 0x564900c91eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:52270/0 (socket says 172.21.15.2:52270) 2024-05-14T16:55:09.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.402+0000 7fcd75d9b700 1 -- 172.21.15.2:0/1599889603 learned_addr learned my addr 172.21.15.2:0/1599889603 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:09.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.402+0000 7fcd76d9d700 1 --2- 172.21.15.2:0/1599889603 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564900c8eca0 0x564900c923f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:09.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.402+0000 7fcd7659c700 1 --2- 172.21.15.2:0/1599889603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564900c891d0 0x564900c94b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:09.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.403+0000 7fcd75d9b700 1 -- 172.21.15.2:0/1599889603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564900c891d0 msgr2=0x564900c94b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:09.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.403+0000 7fcd75d9b700 1 --2- 172.21.15.2:0/1599889603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564900c891d0 0x564900c94b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.403+0000 7fcd75d9b700 1 -- 172.21.15.2:0/1599889603 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564900c8eca0 msgr2=0x564900c923f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:09.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.403+0000 7fcd75d9b700 1 --2- 172.21.15.2:0/1599889603 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564900c8eca0 0x564900c923f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.403+0000 7fcd75d9b700 1 -- 172.21.15.2:0/1599889603 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564900b22880 con 0x564900c8e380 2024-05-14T16:55:09.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.403+0000 7fcd76d9d700 1 --2- 172.21.15.2:0/1599889603 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564900c8eca0 0x564900c923f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:55:09.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.403+0000 7fcd75d9b700 1 --2- 172.21.15.2:0/1599889603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564900c8e380 0x564900c91eb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c005950 tx=0x7fcd6c010840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5244b453f0247744 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:09.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.403+0000 7fcd5dffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T16:55:09.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.403+0000 7fcd5dffb700 1 -- 172.21.15.2:0/1599889603 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c020050 con 0x564900c8e380 2024-05-14T16:55:09.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.403+0000 7fcd87343b80 1 -- 172.21.15.2:0/1599889603 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd70022070 con 0x564900c8e380 2024-05-14T16:55:09.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.403+0000 7fcd5dffb700 1 -- 172.21.15.2:0/1599889603 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fcd6c0b1f50 con 0x564900c8e380 2024-05-14T16:55:09.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.403+0000 7fcd5dffb700 1 -- 172.21.15.2:0/1599889603 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcd6c0a8ae0 con 0x564900c8e380 2024-05-14T16:55:09.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.403+0000 7fcd87343b80 10 client.25037 fetch_fsmap learned FSMap version 25 2024-05-14T16:55:09.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.403+0000 7fcd87343b80 1 -- 172.21.15.2:0/1599889603 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564900b8c710 con 0x564900c8e380 2024-05-14T16:55:09.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.403+0000 7fcd5dffb700 1 -- 172.21.15.2:0/1599889603 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd70022070 con 0x564900c8e380 2024-05-14T16:55:09.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.404+0000 7fcd5dffb700 1 -- 172.21.15.2:0/1599889603 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fcd6c0dfb90 con 0x564900c8e380 2024-05-14T16:55:09.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.404+0000 7fcd87343b80 10 client.25037 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T16:55:09.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.404+0000 7fcd87343b80 10 client.25037 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-14T16:55:09.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.404+0000 7fcd87343b80 20 client.25037 populate_metadata read hostname 'smithi002' 2024-05-14T16:55:09.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.404+0000 7fcd87343b80 1 --2- 172.21.15.2:0/1599889603 >> [v2:172.21.15.123:6834/1198670013,v1:172.21.15.123:6835/1198670013] conn(0x564900ca23b0 0x564900ca3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:09.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.404+0000 7fcd87343b80 4 client.25037 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:55:09.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.404+0000 7fcd87343b80 1 -- 172.21.15.2:0/1599889603 --> [v2:172.21.15.123:6834/1198670013,v1:172.21.15.123:6835/1198670013] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564900be4c70 con 0x564900ca23b0 2024-05-14T16:55:09.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.404+0000 7fcd7659c700 1 --2- 172.21.15.2:0/1599889603 >> [v2:172.21.15.123:6834/1198670013,v1:172.21.15.123:6835/1198670013] conn(0x564900ca23b0 0x564900ca3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:09.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.404+0000 7fcd7659c700 1 --2- 172.21.15.2:0/1599889603 >> [v2:172.21.15.123:6834/1198670013,v1:172.21.15.123:6835/1198670013] conn(0x564900ca23b0 0x564900ca3820 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c95b7beb84deb39b server_cookie=1dc8cd53dd1db67c in_seq=0 out_seq=0 2024-05-14T16:55:09.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.404+0000 7fcd5dffb700 10 client.25037 ms_handle_connect on v2:172.21.15.123:6834/1198670013 2024-05-14T16:55:09.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.405+0000 7fcd5dffb700 1 -- 172.21.15.2:0/1599889603 <== mds.2 v2:172.21.15.123:6834/1198670013 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564900be4c70 con 0x564900ca23b0 2024-05-14T16:55:09.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.405+0000 7fcd5dffb700 10 client.25037 handle_command_reply: tid=0 2024-05-14T16:55:09.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.405+0000 7fcd87343b80 1 client.25037 shutdown 2024-05-14T16:55:09.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.405+0000 7fcd87343b80 1 -- 172.21.15.2:0/1599889603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564900c8e380 msgr2=0x564900c91eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:09.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.405+0000 7fcd87343b80 1 --2- 172.21.15.2:0/1599889603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564900c8e380 0x564900c91eb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c005950 tx=0x7fcd6c010840 comp rx=0 tx=0).stop 2024-05-14T16:55:09.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.405+0000 7fcd87343b80 1 -- 172.21.15.2:0/1599889603 shutdown_connections 2024-05-14T16:55:09.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.405+0000 7fcd87343b80 1 --2- 172.21.15.2:0/1599889603 >> [v2:172.21.15.123:6834/1198670013,v1:172.21.15.123:6835/1198670013] conn(0x564900ca23b0 0x564900ca3820 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.405+0000 7fcd87343b80 1 --2- 172.21.15.2:0/1599889603 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564900c8eca0 0x564900c923f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.405+0000 7fcd87343b80 1 --2- 172.21.15.2:0/1599889603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564900c891d0 0x564900c94b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.405+0000 7fcd5dffb700 0 client.25037 ms_handle_reset on v2:172.21.15.123:6834/1198670013 2024-05-14T16:55:09.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.405+0000 7fcd87343b80 1 --2- 172.21.15.2:0/1599889603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564900c8e380 0x564900c91eb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.405+0000 7fcd87343b80 1 -- 172.21.15.2:0/1599889603 >> 172.21.15.2:0/1599889603 conn(0x564900bb1d60 msgr2=0x564900c7e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:09.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.405+0000 7fcd87343b80 1 -- 172.21.15.2:0/1599889603 shutdown_connections 2024-05-14T16:55:09.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.405+0000 7fcd87343b80 1 -- 172.21.15.2:0/1599889603 wait complete. 2024-05-14T16:55:09.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.405+0000 7fcd87343b80 20 client.25037 trim_cache size 0 max 16384 2024-05-14T16:55:09.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.426+0000 7fcd87343b80 1 Processor -- start 2024-05-14T16:55:09.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.426+0000 7fcd87343b80 1 -- start start 2024-05-14T16:55:09.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.427+0000 7fcd87343b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564900c891d0 0x564900c90e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:09.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.427+0000 7fcd87343b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564900c8e380 0x564900c8c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:09.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.427+0000 7fcd87343b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564900c8eca0 0x564900c7edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:09.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.427+0000 7fcd87343b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x564900ca1b60 con 0x564900c8e380 2024-05-14T16:55:09.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.427+0000 7fcd87343b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564900ca2230 con 0x564900c891d0 2024-05-14T16:55:09.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.427+0000 7fcd87343b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564900ca2900 con 0x564900c8eca0 2024-05-14T16:55:09.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.427+0000 7fcd75d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564900c8e380 0x564900c8c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:09.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.427+0000 7fcd75d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564900c8e380 0x564900c8c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46134/0 (socket says 172.21.15.2:46134) 2024-05-14T16:55:09.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.427+0000 7fcd7659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564900c891d0 0x564900c90e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:09.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.427+0000 7fcd75d9b700 1 -- 172.21.15.2:0/1469915698 learned_addr learned my addr 172.21.15.2:0/1469915698 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:09.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.427+0000 7fcd76d9d700 1 --2- 172.21.15.2:0/1469915698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564900c8eca0 0x564900c7edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:09.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.427+0000 7fcd75d9b700 1 -- 172.21.15.2:0/1469915698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564900c8eca0 msgr2=0x564900c7edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:09.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.427+0000 7fcd75d9b700 1 --2- 172.21.15.2:0/1469915698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564900c8eca0 0x564900c7edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.427+0000 7fcd75d9b700 1 -- 172.21.15.2:0/1469915698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564900c891d0 msgr2=0x564900c90e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:09.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.427+0000 7fcd75d9b700 1 --2- 172.21.15.2:0/1469915698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564900c891d0 0x564900c90e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.427+0000 7fcd75d9b700 1 -- 172.21.15.2:0/1469915698 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564900c80db0 con 0x564900c8e380 2024-05-14T16:55:09.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.427+0000 7fcd76d9d700 1 --2- 172.21.15.2:0/1469915698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564900c8eca0 0x564900c7edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:55:09.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.427+0000 7fcd7659c700 1 --2- 172.21.15.2:0/1469915698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564900c891d0 0x564900c90e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:55:09.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.427+0000 7fcd75d9b700 1 --2- 172.21.15.2:0/1469915698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564900c8e380 0x564900c8c560 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c0a9070 tx=0x7fcd6c0ad890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb458e713a031530 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:09.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.428+0000 7fcd5ffff700 1 -- 172.21.15.2:0/1469915698 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c020070 con 0x564900c8e380 2024-05-14T16:55:09.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.428+0000 7fcd5ffff700 1 -- 172.21.15.2:0/1469915698 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd6c0e1070 con 0x564900c8e380 2024-05-14T16:55:09.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.428+0000 7fcd5ffff700 1 -- 172.21.15.2:0/1469915698 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c0a9370 con 0x564900c8e380 2024-05-14T16:55:09.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.428+0000 7fcd87343b80 1 -- 172.21.15.2:0/1469915698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564900c8e380 msgr2=0x564900c8c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:09.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.428+0000 7fcd87343b80 1 --2- 172.21.15.2:0/1469915698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564900c8e380 0x564900c8c560 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c0a9070 tx=0x7fcd6c0ad890 comp rx=0 tx=0).stop 2024-05-14T16:55:09.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.428+0000 7fcd87343b80 1 -- 172.21.15.2:0/1469915698 shutdown_connections 2024-05-14T16:55:09.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.428+0000 7fcd87343b80 1 --2- 172.21.15.2:0/1469915698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564900c8eca0 0x564900c7edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.428+0000 7fcd87343b80 1 --2- 172.21.15.2:0/1469915698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564900c891d0 0x564900c90e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.428+0000 7fcd87343b80 1 --2- 172.21.15.2:0/1469915698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564900c8e380 0x564900c8c560 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.428+0000 7fcd87343b80 1 -- 172.21.15.2:0/1469915698 >> 172.21.15.2:0/1469915698 conn(0x564900bb1d60 msgr2=0x564900c8be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:09.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.428+0000 7fcd87343b80 1 -- 172.21.15.2:0/1469915698 shutdown_connections 2024-05-14T16:55:09.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.428+0000 7fcd87343b80 1 -- 172.21.15.2:0/1469915698 wait complete. 2024-05-14T16:55:09.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.429+0000 7fcd87343b80 1 -- start start 2024-05-14T16:55:09.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.429+0000 7fcd87343b80 1 Processor -- start 2024-05-14T16:55:09.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.429+0000 7fcd87343b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564900c891d0 0x564900c7e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:09.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.429+0000 7fcd87343b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564900c8e380 0x564900c7e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:09.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.429+0000 7fcd87343b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564900c8eca0 0x564900c7ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:09.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.429+0000 7fcd87343b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x564900ca36a0 con 0x564900c891d0 2024-05-14T16:55:09.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.429+0000 7fcd87343b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564900ca2fd0 con 0x564900c8e380 2024-05-14T16:55:09.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.429+0000 7fcd87343b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564900ca3d70 con 0x564900c8eca0 2024-05-14T16:55:09.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.429+0000 7fcd87343b80 10 client.0 _ll_register_callbacks cb 0x564900b6d900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:55:09.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.429+0000 7fcd7659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564900c891d0 0x564900c7e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:09.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.429+0000 7fcd7659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564900c891d0 0x564900c7e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46140/0 (socket says 172.21.15.2:46140) 2024-05-14T16:55:09.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.429+0000 7fcd7659c700 1 -- 172.21.15.2:0/66086534 learned_addr learned my addr 172.21.15.2:0/66086534 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:09.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.429+0000 7fcd75d9b700 1 --2- 172.21.15.2:0/66086534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564900c8e380 0x564900c7e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:09.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.429+0000 7fcd76d9d700 1 --2- 172.21.15.2:0/66086534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564900c8eca0 0x564900c7ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:09.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.429+0000 7fcd7659c700 1 -- 172.21.15.2:0/66086534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564900c8eca0 msgr2=0x564900c7ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:09.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.429+0000 7fcd7659c700 1 --2- 172.21.15.2:0/66086534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564900c8eca0 0x564900c7ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.429+0000 7fcd7659c700 1 -- 172.21.15.2:0/66086534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564900c8e380 msgr2=0x564900c7e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:09.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.429+0000 7fcd7659c700 1 --2- 172.21.15.2:0/66086534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564900c8e380 0x564900c7e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.429+0000 7fcd7659c700 1 -- 172.21.15.2:0/66086534 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564900cab4a0 con 0x564900c891d0 2024-05-14T16:55:09.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.429+0000 7fcd76d9d700 1 --2- 172.21.15.2:0/66086534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564900c8eca0 0x564900c7ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:55:09.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.429+0000 7fcd75d9b700 1 --2- 172.21.15.2:0/66086534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564900c8e380 0x564900c7e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:55:09.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.430+0000 7fcd7659c700 1 --2- 172.21.15.2:0/66086534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564900c891d0 0x564900c7e3c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fcd68000c00 tx=0x7fcd680041a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c41cb6f965c22e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:09.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.430+0000 7fcd5dffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:55:09.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.430+0000 7fcd5dffb700 1 -- 172.21.15.2:0/66086534 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd68022070 con 0x564900c891d0 2024-05-14T16:55:09.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.430+0000 7fcd87343b80 1 -- 172.21.15.2:0/66086534 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd6c0e1070 con 0x564900c891d0 2024-05-14T16:55:09.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.430+0000 7fcd5dffb700 1 -- 172.21.15.2:0/66086534 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fcd68010d20 con 0x564900c891d0 2024-05-14T16:55:09.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.430+0000 7fcd5dffb700 1 -- 172.21.15.2:0/66086534 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcd6802d050 con 0x564900c891d0 2024-05-14T16:55:09.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.430+0000 7fcd87343b80 10 client.15303 fetch_fsmap learned FSMap version 25 2024-05-14T16:55:09.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.430+0000 7fcd87343b80 1 -- 172.21.15.2:0/66086534 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564900d0dd90 con 0x564900c891d0 2024-05-14T16:55:09.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.430+0000 7fcd5dffb700 1 -- 172.21.15.2:0/66086534 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd6c0e1070 con 0x564900c891d0 2024-05-14T16:55:09.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.431+0000 7fcd5dffb700 1 -- 172.21.15.2:0/66086534 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fcd68005020 con 0x564900c891d0 2024-05-14T16:55:09.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.431+0000 7fcd87343b80 10 client.15303 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T16:55:09.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.431+0000 7fcd87343b80 10 client.15303 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-14T16:55:09.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.431+0000 7fcd87343b80 20 client.15303 populate_metadata read hostname 'smithi002' 2024-05-14T16:55:09.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.431+0000 7fcd87343b80 1 --2- 172.21.15.2:0/66086534 >> [v2:172.21.15.123:6834/1198670013,v1:172.21.15.123:6835/1198670013] conn(0x564900d12960 0x564900d11670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:09.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.431+0000 7fcd87343b80 4 client.15303 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T16:55:09.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.431+0000 7fcd87343b80 1 -- 172.21.15.2:0/66086534 --> [v2:172.21.15.123:6834/1198670013,v1:172.21.15.123:6835/1198670013] -- command(tid 0: {"prefix": "status"}) -- 0x564900bde450 con 0x564900d12960 2024-05-14T16:55:09.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.431+0000 7fcd75d9b700 1 --2- 172.21.15.2:0/66086534 >> [v2:172.21.15.123:6834/1198670013,v1:172.21.15.123:6835/1198670013] conn(0x564900d12960 0x564900d11670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:09.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.431+0000 7fcd75d9b700 1 --2- 172.21.15.2:0/66086534 >> [v2:172.21.15.123:6834/1198670013,v1:172.21.15.123:6835/1198670013] conn(0x564900d12960 0x564900d11670 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ea3e59c81dfb0e39 server_cookie=f9ca6b08494c1c51 in_seq=0 out_seq=0 2024-05-14T16:55:09.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.431+0000 7fcd5dffb700 10 client.15303 ms_handle_connect on v2:172.21.15.123:6834/1198670013 2024-05-14T16:55:09.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.431+0000 7fcd5dffb700 1 -- 172.21.15.2:0/66086534 <== mds.2 v2:172.21.15.123:6834/1198670013 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x564900bde450 con 0x564900d12960 2024-05-14T16:55:09.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.431+0000 7fcd5dffb700 10 client.15303 handle_command_reply: tid=0 2024-05-14T16:55:09.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.431+0000 7fcd87343b80 1 client.15303 shutdown 2024-05-14T16:55:09.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.432+0000 7fcd87343b80 1 -- 172.21.15.2:0/66086534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564900c891d0 msgr2=0x564900c7e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:09.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.432+0000 7fcd87343b80 1 --2- 172.21.15.2:0/66086534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564900c891d0 0x564900c7e3c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fcd68000c00 tx=0x7fcd680041a0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.432+0000 7fcd87343b80 1 -- 172.21.15.2:0/66086534 shutdown_connections 2024-05-14T16:55:09.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.432+0000 7fcd87343b80 1 --2- 172.21.15.2:0/66086534 >> [v2:172.21.15.123:6834/1198670013,v1:172.21.15.123:6835/1198670013] conn(0x564900d12960 0x564900d11670 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.432+0000 7fcd87343b80 1 --2- 172.21.15.2:0/66086534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564900c8eca0 0x564900c7ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.432+0000 7fcd87343b80 1 --2- 172.21.15.2:0/66086534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564900c891d0 0x564900c7e3c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.432+0000 7fcd87343b80 1 --2- 172.21.15.2:0/66086534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564900c8e380 0x564900c7e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.432+0000 7fcd5dffb700 0 client.15303 ms_handle_reset on v2:172.21.15.123:6834/1198670013 2024-05-14T16:55:09.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.432+0000 7fcd87343b80 1 -- 172.21.15.2:0/66086534 >> 172.21.15.2:0/66086534 conn(0x564900bb1d60 msgr2=0x564900c83790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:09.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.432+0000 7fcd87343b80 1 -- 172.21.15.2:0/66086534 shutdown_connections 2024-05-14T16:55:09.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.432+0000 7fcd87343b80 1 -- 172.21.15.2:0/66086534 wait complete. 2024-05-14T16:55:09.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.432+0000 7fcd87343b80 20 client.15303 trim_cache size 0 max 16384 2024-05-14T16:55:09.439 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T16:55:09.439 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "ed40717a-1210-11ef-bc98-c7b262605968", 2024-05-14T16:55:09.439 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 2, 2024-05-14T16:55:09.439 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24431, 2024-05-14T16:55:09.439 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2024-05-14T16:55:09.440 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2024-05-14T16:55:09.440 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2024-05-14T16:55:09.440 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 201.45394747700001, 2024-05-14T16:55:09.440 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 25, 2024-05-14T16:55:09.440 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 81, 2024-05-14T16:55:09.440 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 80, 2024-05-14T16:55:09.440 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 252.53691819100001 2024-05-14T16:55:09.440 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T16:55:09.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.433+0000 7fcd7caad700 1 -- 172.21.15.2:0/1070567194 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcd600b2b10 msgr2=0x7fcd600b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:09.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.433+0000 7fcd7caad700 1 --2- 172.21.15.2:0/1070567194 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcd600b2b10 0x7fcd600b4fd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcd70009e60 tx=0x7fcd70027000 comp rx=0 tx=0).stop 2024-05-14T16:55:09.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.433+0000 7fcd7caad700 1 -- 172.21.15.2:0/1070567194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd781041a0 msgr2=0x7fcd7819fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:09.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.433+0000 7fcd7caad700 1 --2- 172.21.15.2:0/1070567194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd781041a0 0x7fcd7819fd00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00d950 tx=0x7fcd6c00dc60 comp rx=0 tx=0).stop 2024-05-14T16:55:09.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.433+0000 7fcd7caad700 1 -- 172.21.15.2:0/1070567194 shutdown_connections 2024-05-14T16:55:09.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.433+0000 7fcd7caad700 1 --2- 172.21.15.2:0/1070567194 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcd600b2b10 0x7fcd600b4fd0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.433+0000 7fcd7caad700 1 --2- 172.21.15.2:0/1070567194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd781088f0 0x7fcd781999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.433+0000 7fcd7caad700 1 --2- 172.21.15.2:0/1070567194 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd781037f0 0x7fcd7819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.433+0000 7fcd7caad700 1 --2- 172.21.15.2:0/1070567194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd781041a0 0x7fcd7819fd00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.433+0000 7fcd7caad700 1 -- 172.21.15.2:0/1070567194 >> 172.21.15.2:0/1070567194 conn(0x7fcd7807a6a0 msgr2=0x7fcd78101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:09.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.433+0000 7fcd7caad700 1 -- 172.21.15.2:0/1070567194 shutdown_connections 2024-05-14T16:55:09.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:09.433+0000 7fcd7caad700 1 -- 172.21.15.2:0/1070567194 wait complete. 2024-05-14T16:55:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.859+0000 7fa1ac82a640 1 -- 172.21.15.2:0/3149690118 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1a8103c00 msgr2=0x7fa1a8103fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.859+0000 7fa1ac82a640 1 --2- 172.21.15.2:0/3149690118 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1a8103c00 0x7fa1a8103fe0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa190009900 tx=0x7fa19002a480 comp rx=0 tx=0).stop 2024-05-14T16:55:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.859+0000 7fa1ac82a640 1 -- 172.21.15.2:0/3149690118 shutdown_connections 2024-05-14T16:55:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.859+0000 7fa1ac82a640 1 --2- 172.21.15.2:0/3149690118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1a810d020 0x7fa1a810f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.859+0000 7fa1ac82a640 1 --2- 172.21.15.2:0/3149690118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1a8104520 0x7fa1a810ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.859+0000 7fa1ac82a640 1 --2- 172.21.15.2:0/3149690118 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1a8103c00 0x7fa1a8103fe0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.859+0000 7fa1ac82a640 1 -- 172.21.15.2:0/3149690118 >> 172.21.15.2:0/3149690118 conn(0x7fa1a80fd420 msgr2=0x7fa1a80ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.859+0000 7fa1ac82a640 1 -- 172.21.15.2:0/3149690118 shutdown_connections 2024-05-14T16:55:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.859+0000 7fa1ac82a640 1 -- 172.21.15.2:0/3149690118 wait complete. 2024-05-14T16:55:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.860+0000 7fa1ac82a640 1 Processor -- start 2024-05-14T16:55:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.860+0000 7fa1ac82a640 1 -- start start 2024-05-14T16:55:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.860+0000 7fa1ac82a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1a8103c00 0x7fa1a8101850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.860+0000 7fa1ac82a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1a8104520 0x7fa1a8101d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.860+0000 7fa1ac82a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1a810d020 0x7fa1a81022d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.860+0000 7fa1ac82a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a8112cd0 con 0x7fa1a8104520 2024-05-14T16:55:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.860+0000 7fa1ac82a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a8112b50 con 0x7fa1a810d020 2024-05-14T16:55:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.860+0000 7fa1ac82a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a8112e50 con 0x7fa1a8103c00 2024-05-14T16:55:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.860+0000 7fa1a5d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1a8104520 0x7fa1a8101d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.860+0000 7fa1a659c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1a8103c00 0x7fa1a8101850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.860+0000 7fa1a659c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1a8103c00 0x7fa1a8101850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:43604/0 (socket says 172.21.15.2:43604) 2024-05-14T16:55:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.860+0000 7fa1a659c640 1 -- 172.21.15.2:0/2440969439 learned_addr learned my addr 172.21.15.2:0/2440969439 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.860+0000 7fa1a5d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1a8104520 0x7fa1a8101d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46170/0 (socket says 172.21.15.2:46170) 2024-05-14T16:55:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.861+0000 7fa1a6d9d640 1 --2- 172.21.15.2:0/2440969439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1a810d020 0x7fa1a81022d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.861+0000 7fa1a659c640 1 -- 172.21.15.2:0/2440969439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1a810d020 msgr2=0x7fa1a81022d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.861+0000 7fa1a659c640 1 --2- 172.21.15.2:0/2440969439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1a810d020 0x7fa1a81022d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.861+0000 7fa1a659c640 1 -- 172.21.15.2:0/2440969439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1a8104520 msgr2=0x7fa1a8101d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.861+0000 7fa1a659c640 1 --2- 172.21.15.2:0/2440969439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1a8104520 0x7fa1a8101d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.861+0000 7fa1a659c640 1 -- 172.21.15.2:0/2440969439 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a81a7670 con 0x7fa1a8103c00 2024-05-14T16:55:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.861+0000 7fa1a659c640 1 --2- 172.21.15.2:0/2440969439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1a8103c00 0x7fa1a8101850 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa190005940 tx=0x7fa1900040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.861+0000 7fa19f7fe640 1 -- 172.21.15.2:0/2440969439 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19003f030 con 0x7fa1a8103c00 2024-05-14T16:55:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.861+0000 7fa1ac82a640 1 -- 172.21.15.2:0/2440969439 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a81a7900 con 0x7fa1a8103c00 2024-05-14T16:55:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.861+0000 7fa19f7fe640 1 -- 172.21.15.2:0/2440969439 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1900044c0 con 0x7fa1a8103c00 2024-05-14T16:55:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.861+0000 7fa19f7fe640 1 -- 172.21.15.2:0/2440969439 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa190035070 con 0x7fa1a8103c00 2024-05-14T16:55:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.862+0000 7fa1ac82a640 1 -- 172.21.15.2:0/2440969439 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a81a7df0 con 0x7fa1a8103c00 2024-05-14T16:55:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.864+0000 7fa19f7fe640 1 -- 172.21.15.2:0/2440969439 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 25) ==== 149884+0+0 (secure 0 0 0) 0x7fa190042030 con 0x7fa1a8103c00 2024-05-14T16:55:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.864+0000 7fa1af625740 1 Processor -- start 2024-05-14T16:55:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.865+0000 7fa1af625740 1 -- start start 2024-05-14T16:55:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.865+0000 7fa1af625740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56113af0d8f0 0x56113af0dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.865+0000 7fa1af625740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56113af0e210 0x56113af18590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.865+0000 7fa1af625740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56113af18ad0 0x56113af1aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.865+0000 7fa1af625740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56113ae80c30 con 0x56113af0d8f0 2024-05-14T16:55:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.865+0000 7fa1af625740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56113ae707d0 con 0x56113af18ad0 2024-05-14T16:55:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.865+0000 7fa1af625740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56113ae74810 con 0x56113af0e210 2024-05-14T16:55:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.865+0000 7fa1a6d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56113af0d8f0 0x56113af0dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.865+0000 7fa1a6d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56113af0d8f0 0x56113af0dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46174/0 (socket says 172.21.15.2:46174) 2024-05-14T16:55:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.865+0000 7fa1a659c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56113af0e210 0x56113af18590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.865+0000 7fa1a6d9d640 1 -- 172.21.15.2:0/2485336828 learned_addr learned my addr 172.21.15.2:0/2485336828 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.865+0000 7fa1a5d9b640 1 --2- 172.21.15.2:0/2485336828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56113af18ad0 0x56113af1aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.865+0000 7fa1a6d9d640 1 -- 172.21.15.2:0/2485336828 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56113af0e210 msgr2=0x56113af18590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.865+0000 7fa1a6d9d640 1 --2- 172.21.15.2:0/2485336828 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56113af0e210 0x56113af18590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.865+0000 7fa1a6d9d640 1 -- 172.21.15.2:0/2485336828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56113af18ad0 msgr2=0x56113af1aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.865+0000 7fa1a6d9d640 1 --2- 172.21.15.2:0/2485336828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56113af18ad0 0x56113af1aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.865+0000 7fa1a6d9d640 1 -- 172.21.15.2:0/2485336828 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56113ab85210 con 0x56113af0d8f0 2024-05-14T16:55:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.865+0000 7fa1a5d9b640 1 --2- 172.21.15.2:0/2485336828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56113af18ad0 0x56113af1aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:55:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.865+0000 7fa19f7fe640 1 --2- 172.21.15.2:0/2440969439 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fa1940b2b50 0x7fa1940b5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.866+0000 7fa1a659c640 1 --2- 172.21.15.2:0/2440969439 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fa1940b2b50 0x7fa1940b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.866+0000 7fa1a6d9d640 1 --2- 172.21.15.2:0/2485336828 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56113af0d8f0 0x56113af0dcd0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0009f50 tx=0x7fa1a00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9cb26076932448f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.866+0000 7fa19f7fe640 1 -- 172.21.15.2:0/2440969439 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fa190102a90 con 0x7fa1a8103c00 2024-05-14T16:55:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.866+0000 7fa19cff9640 1 -- 172.21.15.2:0/2485336828 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0004d20 con 0x56113af0d8f0 2024-05-14T16:55:09.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.866+0000 7fa19cff9640 1 -- 172.21.15.2:0/2485336828 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1a0004050 con 0x56113af0d8f0 2024-05-14T16:55:09.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.866+0000 7fa19cff9640 1 -- 172.21.15.2:0/2485336828 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0005280 con 0x56113af0d8f0 2024-05-14T16:55:09.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.867+0000 7fa1af625740 1 -- 172.21.15.2:0/2485336828 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56113af0d8f0 msgr2=0x56113af0dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:09.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.867+0000 7fa1a659c640 1 --2- 172.21.15.2:0/2440969439 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fa1940b2b50 0x7fa1940b5010 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa1901000f0 tx=0x7fa1901065a0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:09.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.867+0000 7fa1af625740 1 --2- 172.21.15.2:0/2485336828 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56113af0d8f0 0x56113af0dcd0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0009f50 tx=0x7fa1a00118d0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.867+0000 7fa1af625740 1 -- 172.21.15.2:0/2485336828 shutdown_connections 2024-05-14T16:55:09.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.867+0000 7fa1af625740 1 --2- 172.21.15.2:0/2485336828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56113af18ad0 0x56113af1aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.867+0000 7fa1af625740 1 --2- 172.21.15.2:0/2485336828 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56113af0e210 0x56113af18590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.867+0000 7fa1af625740 1 --2- 172.21.15.2:0/2485336828 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56113af0d8f0 0x56113af0dcd0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.867+0000 7fa1af625740 1 -- 172.21.15.2:0/2485336828 >> 172.21.15.2:0/2485336828 conn(0x56113af09e00 msgr2=0x56113af0a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.867+0000 7fa1af625740 1 -- 172.21.15.2:0/2485336828 shutdown_connections 2024-05-14T16:55:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.867+0000 7fa1af625740 1 -- 172.21.15.2:0/2485336828 wait complete. 2024-05-14T16:55:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.867+0000 7fa1af625740 1 -- start start 2024-05-14T16:55:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.867+0000 7fa1af625740 1 Processor -- start 2024-05-14T16:55:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.868+0000 7fa1af625740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56113af0d8f0 0x56113af18430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.868+0000 7fa1af625740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56113af0e210 0x56113af15720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.868+0000 7fa1af625740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56113af18ad0 0x56113af15c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.868+0000 7fa1af625740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56113ab56790 con 0x56113af0d8f0 2024-05-14T16:55:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.868+0000 7fa1af625740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56113ae2c890 con 0x56113af0e210 2024-05-14T16:55:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.868+0000 7fa1af625740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56113af1b7a0 con 0x56113af18ad0 2024-05-14T16:55:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.868+0000 7fa1af625740 10 client.0 _ll_register_callbacks cb 0x56113ae2fd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:55:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.868+0000 7fa1a6d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56113af0d8f0 0x56113af18430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.868+0000 7fa1a6d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56113af0d8f0 0x56113af18430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46186/0 (socket says 172.21.15.2:46186) 2024-05-14T16:55:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.868+0000 7fa1a5d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56113af0e210 0x56113af15720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.868+0000 7fa1a6d9d640 1 -- 172.21.15.2:0/2518807708 learned_addr learned my addr 172.21.15.2:0/2518807708 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.868+0000 7fa1a6d9d640 1 --2- 172.21.15.2:0/2518807708 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56113af18ad0 0x56113af15c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.868+0000 7fa1a6d9d640 1 -- 172.21.15.2:0/2518807708 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56113af18ad0 msgr2=0x56113af15c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.868+0000 7fa1a6d9d640 1 --2- 172.21.15.2:0/2518807708 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56113af18ad0 0x56113af15c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.868+0000 7fa1a6d9d640 1 -- 172.21.15.2:0/2518807708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56113af0e210 msgr2=0x56113af15720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.868+0000 7fa1a6d9d640 1 --2- 172.21.15.2:0/2518807708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56113af0e210 0x56113af15720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.868+0000 7fa1a6d9d640 1 -- 172.21.15.2:0/2518807708 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56113ac51620 con 0x56113af0d8f0 2024-05-14T16:55:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.868+0000 7fa1a6d9d640 1 --2- 172.21.15.2:0/2518807708 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56113af0d8f0 0x56113af18430 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0005c40 tx=0x7fa1a0005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=971778da2fa32530 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.868+0000 7fa18dffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:55:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.868+0000 7fa18dffb640 1 -- 172.21.15.2:0/2518807708 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0004c50 con 0x56113af0d8f0 2024-05-14T16:55:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.869+0000 7fa18dffb640 1 -- 172.21.15.2:0/2518807708 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fa1a0004df0 con 0x56113af0d8f0 2024-05-14T16:55:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.869+0000 7fa1af625740 1 -- 172.21.15.2:0/2518807708 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1a0004050 con 0x56113af0d8f0 2024-05-14T16:55:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.869+0000 7fa18dffb640 1 -- 172.21.15.2:0/2518807708 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa1a0005040 con 0x56113af0d8f0 2024-05-14T16:55:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.869+0000 7fa1af625740 10 client.15315 fetch_fsmap learned FSMap version 25 2024-05-14T16:55:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.869+0000 7fa1af625740 1 -- 172.21.15.2:0/2518807708 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56113af271c0 con 0x56113af0d8f0 2024-05-14T16:55:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.869+0000 7fa18dffb640 1 -- 172.21.15.2:0/2518807708 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1a0004050 con 0x56113af0d8f0 2024-05-14T16:55:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.869+0000 7fa18dffb640 1 -- 172.21.15.2:0/2518807708 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fa1a0027780 con 0x56113af0d8f0 2024-05-14T16:55:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.869+0000 7fa1af625740 10 client.15315 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T16:55:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.869+0000 7fa1af625740 10 client.15315 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:55:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.869+0000 7fa1af625740 20 client.15315 populate_metadata read hostname 'smithi002' 2024-05-14T16:55:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.869+0000 7fa1af625740 1 --2- 172.21.15.2:0/2518807708 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56113af285c0 0x56113af2a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.869+0000 7fa1af625740 4 client.15315 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:55:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.869+0000 7fa1af625740 1 -- 172.21.15.2:0/2518807708 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56113ae82b80 con 0x56113af285c0 2024-05-14T16:55:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.869+0000 7fa1a5d9b640 1 --2- 172.21.15.2:0/2518807708 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56113af285c0 0x56113af2a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.870+0000 7fa1a5d9b640 1 --2- 172.21.15.2:0/2518807708 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56113af285c0 0x56113af2a9b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1f687e5f8d1b467 server_cookie=a4cbb72be1a12d0 in_seq=0 out_seq=0 2024-05-14T16:55:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.870+0000 7fa18dffb640 10 client.15315 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:55:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.873+0000 7fa18dffb640 1 -- 172.21.15.2:0/2518807708 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56113ae82b80 con 0x56113af285c0 2024-05-14T16:55:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.873+0000 7fa18dffb640 10 client.15315 handle_command_reply: tid=0 2024-05-14T16:55:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.873+0000 7fa1af625740 1 client.15315 shutdown 2024-05-14T16:55:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.873+0000 7fa1af625740 1 -- 172.21.15.2:0/2518807708 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56113af0d8f0 msgr2=0x56113af18430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.873+0000 7fa1af625740 1 --2- 172.21.15.2:0/2518807708 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56113af0d8f0 0x56113af18430 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0005c40 tx=0x7fa1a0005bd0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.873+0000 7fa1af625740 1 -- 172.21.15.2:0/2518807708 shutdown_connections 2024-05-14T16:55:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.873+0000 7fa1af625740 1 --2- 172.21.15.2:0/2518807708 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56113af285c0 0x56113af2a9b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.873+0000 7fa1af625740 1 --2- 172.21.15.2:0/2518807708 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56113af18ad0 0x56113af15c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.873+0000 7fa1af625740 1 --2- 172.21.15.2:0/2518807708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56113af0e210 0x56113af15720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.873+0000 7fa18dffb640 0 client.15315 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:55:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.873+0000 7fa1af625740 1 --2- 172.21.15.2:0/2518807708 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56113af0d8f0 0x56113af18430 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.873+0000 7fa1af625740 1 -- 172.21.15.2:0/2518807708 >> 172.21.15.2:0/2518807708 conn(0x56113af09e00 msgr2=0x56113af09100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.873+0000 7fa1af625740 1 -- 172.21.15.2:0/2518807708 shutdown_connections 2024-05-14T16:55:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.873+0000 7fa1af625740 1 -- 172.21.15.2:0/2518807708 wait complete. 2024-05-14T16:55:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.873+0000 7fa1af625740 20 client.15315 trim_cache size 0 max 16384 2024-05-14T16:55:09.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.889+0000 7fa1af625740 1 Processor -- start 2024-05-14T16:55:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.889+0000 7fa1af625740 1 -- start start 2024-05-14T16:55:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.890+0000 7fa1af625740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56113af0d8f0 0x56113af24670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.890+0000 7fa1af625740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56113af0e210 0x56113af17f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.890+0000 7fa1af625740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56113af18ad0 0x56113af30040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.890+0000 7fa1af625740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a81b30a0 con 0x56113af0e210 2024-05-14T16:55:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.890+0000 7fa1af625740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a81b0970 con 0x56113af18ad0 2024-05-14T16:55:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.890+0000 7fa1af625740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a81ae7c0 con 0x56113af0d8f0 2024-05-14T16:55:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.890+0000 7fa1a5d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56113af0e210 0x56113af17f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.890+0000 7fa1a6d9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56113af0d8f0 0x56113af24670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.890+0000 7fa1a5d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56113af0e210 0x56113af17f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46202/0 (socket says 172.21.15.2:46202) 2024-05-14T16:55:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.890+0000 7fa1a5d9b640 1 -- 172.21.15.2:0/2517277467 learned_addr learned my addr 172.21.15.2:0/2517277467 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.890+0000 7fa1a6d9d640 1 --2- 172.21.15.2:0/2517277467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56113af18ad0 0x56113af30040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.890+0000 7fa1a5d9b640 1 -- 172.21.15.2:0/2517277467 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56113af0d8f0 msgr2=0x56113af24670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.890+0000 7fa1a5d9b640 1 --2- 172.21.15.2:0/2517277467 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56113af0d8f0 0x56113af24670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.890+0000 7fa1a5d9b640 1 -- 172.21.15.2:0/2517277467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56113af18ad0 msgr2=0x56113af30040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.890+0000 7fa1a5d9b640 1 --2- 172.21.15.2:0/2517277467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56113af18ad0 0x56113af30040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.890+0000 7fa1a5d9b640 1 -- 172.21.15.2:0/2517277467 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56113af05600 con 0x56113af0e210 2024-05-14T16:55:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.890+0000 7fa1a5d9b640 1 --2- 172.21.15.2:0/2517277467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56113af0e210 0x56113af17f60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa198005e00 tx=0x7fa1980118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b57c27e4fe551c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.890+0000 7fa18ffff640 1 -- 172.21.15.2:0/2517277467 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa198026070 con 0x56113af0e210 2024-05-14T16:55:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.890+0000 7fa18ffff640 1 -- 172.21.15.2:0/2517277467 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1980042f0 con 0x56113af0e210 2024-05-14T16:55:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.890+0000 7fa18ffff640 1 -- 172.21.15.2:0/2517277467 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1980204d0 con 0x56113af0e210 2024-05-14T16:55:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.891+0000 7fa1af625740 1 -- 172.21.15.2:0/2517277467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56113af0e210 msgr2=0x56113af17f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.891+0000 7fa1af625740 1 --2- 172.21.15.2:0/2517277467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56113af0e210 0x56113af17f60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa198005e00 tx=0x7fa1980118d0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.891+0000 7fa1af625740 1 -- 172.21.15.2:0/2517277467 shutdown_connections 2024-05-14T16:55:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.891+0000 7fa1af625740 1 --2- 172.21.15.2:0/2517277467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56113af18ad0 0x56113af30040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.891+0000 7fa1af625740 1 --2- 172.21.15.2:0/2517277467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56113af0e210 0x56113af17f60 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.891+0000 7fa1af625740 1 --2- 172.21.15.2:0/2517277467 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56113af0d8f0 0x56113af24670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.891+0000 7fa1af625740 1 -- 172.21.15.2:0/2517277467 >> 172.21.15.2:0/2517277467 conn(0x56113af09e00 msgr2=0x56113af10ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.891+0000 7fa1af625740 1 -- 172.21.15.2:0/2517277467 shutdown_connections 2024-05-14T16:55:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.891+0000 7fa1af625740 1 -- 172.21.15.2:0/2517277467 wait complete. 2024-05-14T16:55:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.891+0000 7fa1af625740 1 -- start start 2024-05-14T16:55:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.891+0000 7fa1af625740 1 Processor -- start 2024-05-14T16:55:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.892+0000 7fa1af625740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56113af0d8f0 0x56113af20e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.892+0000 7fa1af625740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56113af0e210 0x56113af1c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.892+0000 7fa1af625740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56113af18ad0 0x56113af1c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.892+0000 7fa1af625740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a81aee30 con 0x56113af18ad0 2024-05-14T16:55:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.892+0000 7fa1af625740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a81b02a0 con 0x56113af0e210 2024-05-14T16:55:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.892+0000 7fa1af625740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a81af500 con 0x56113af0d8f0 2024-05-14T16:55:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.892+0000 7fa1af625740 10 client.0 _ll_register_callbacks cb 0x7fa1a0009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:55:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.892+0000 7fa1a6d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56113af18ad0 0x56113af1c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.892+0000 7fa1a6d9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56113af0d8f0 0x56113af20e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.892+0000 7fa1a6d9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56113af0d8f0 0x56113af20e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:43642/0 (socket says 172.21.15.2:43642) 2024-05-14T16:55:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.892+0000 7fa1a6d9d640 1 -- 172.21.15.2:0/1767159970 learned_addr learned my addr 172.21.15.2:0/1767159970 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.892+0000 7fa1a5d9b640 1 --2- 172.21.15.2:0/1767159970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56113af0e210 0x56113af1c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.892+0000 7fa1a6d9d640 1 -- 172.21.15.2:0/1767159970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56113af0e210 msgr2=0x56113af1c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.892+0000 7fa1a6d9d640 1 --2- 172.21.15.2:0/1767159970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56113af0e210 0x56113af1c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.892+0000 7fa1a6d9d640 1 -- 172.21.15.2:0/1767159970 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56113af18ad0 msgr2=0x56113af1c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.892+0000 7fa1a6d9d640 1 --2- 172.21.15.2:0/1767159970 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56113af18ad0 0x56113af1c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.892+0000 7fa1a6d9d640 1 -- 172.21.15.2:0/1767159970 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56113af1cde0 con 0x56113af0d8f0 2024-05-14T16:55:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.892+0000 7fa1a6d9d640 1 --2- 172.21.15.2:0/1767159970 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56113af0d8f0 0x56113af20e40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0011ed0 tx=0x7fa1a0000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=121e9fd2e8a5cf2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.892+0000 7fa18dffb640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-14T16:55:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.892+0000 7fa18dffb640 1 -- 172.21.15.2:0/1767159970 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0027680 con 0x56113af0d8f0 2024-05-14T16:55:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.892+0000 7fa1af625740 1 -- 172.21.15.2:0/1767159970 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1980042f0 con 0x56113af0d8f0 2024-05-14T16:55:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.893+0000 7fa18dffb640 1 -- 172.21.15.2:0/1767159970 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fa1a002c070 con 0x56113af0d8f0 2024-05-14T16:55:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.893+0000 7fa18dffb640 1 -- 172.21.15.2:0/1767159970 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa1a0027820 con 0x56113af0d8f0 2024-05-14T16:55:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.893+0000 7fa1af625740 10 client.25057 fetch_fsmap learned FSMap version 25 2024-05-14T16:55:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.893+0000 7fa1af625740 1 -- 172.21.15.2:0/1767159970 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56113af92b00 con 0x56113af0d8f0 2024-05-14T16:55:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.893+0000 7fa18dffb640 1 -- 172.21.15.2:0/1767159970 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1980042f0 con 0x56113af0d8f0 2024-05-14T16:55:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.893+0000 7fa18dffb640 1 -- 172.21.15.2:0/1767159970 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fa1a0030440 con 0x56113af0d8f0 2024-05-14T16:55:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.893+0000 7fa1af625740 10 client.25057 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T16:55:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.893+0000 7fa1af625740 10 client.25057 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:55:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.893+0000 7fa1af625740 20 client.25057 populate_metadata read hostname 'smithi002' 2024-05-14T16:55:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.893+0000 7fa1af625740 1 --2- 172.21.15.2:0/1767159970 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56113af931c0 0x56113af955b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.893+0000 7fa1af625740 4 client.25057 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:55:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.893+0000 7fa1af625740 1 -- 172.21.15.2:0/1767159970 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56113ae7c360 con 0x56113af931c0 2024-05-14T16:55:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.893+0000 7fa1a5d9b640 1 --2- 172.21.15.2:0/1767159970 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56113af931c0 0x56113af955b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.894+0000 7fa1a5d9b640 1 --2- 172.21.15.2:0/1767159970 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56113af931c0 0x56113af955b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9126c2984acaaa04 server_cookie=8776291a9f32c7a7 in_seq=0 out_seq=0 2024-05-14T16:55:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:09.894+0000 7fa18dffb640 10 client.25057 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:55:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:10.099+0000 7fa18dffb640 1 -- 172.21.15.2:0/1767159970 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56113ae7c360 con 0x56113af931c0 2024-05-14T16:55:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:10.099+0000 7fa18dffb640 10 client.25057 handle_command_reply: tid=0 2024-05-14T16:55:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:10.099+0000 7fa1af625740 1 client.25057 shutdown 2024-05-14T16:55:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:10.099+0000 7fa1af625740 1 -- 172.21.15.2:0/1767159970 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56113af0d8f0 msgr2=0x56113af20e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:10.099+0000 7fa1af625740 1 --2- 172.21.15.2:0/1767159970 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56113af0d8f0 0x56113af20e40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0011ed0 tx=0x7fa1a0000f40 comp rx=0 tx=0).stop 2024-05-14T16:55:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:10.099+0000 7fa1af625740 1 -- 172.21.15.2:0/1767159970 shutdown_connections 2024-05-14T16:55:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:10.099+0000 7fa1af625740 1 --2- 172.21.15.2:0/1767159970 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56113af931c0 0x56113af955b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:10.099+0000 7fa1af625740 1 --2- 172.21.15.2:0/1767159970 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56113af18ad0 0x56113af1c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:10.099+0000 7fa1af625740 1 --2- 172.21.15.2:0/1767159970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56113af0e210 0x56113af1c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:10.099+0000 7fa18dffb640 0 client.25057 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:55:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:10.099+0000 7fa1af625740 1 --2- 172.21.15.2:0/1767159970 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56113af0d8f0 0x56113af20e40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:10.099+0000 7fa1af625740 1 -- 172.21.15.2:0/1767159970 >> 172.21.15.2:0/1767159970 conn(0x56113af09e00 msgr2=0x56113af0f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:10.100+0000 7fa1af625740 1 -- 172.21.15.2:0/1767159970 shutdown_connections 2024-05-14T16:55:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:10.100+0000 7fa1af625740 1 -- 172.21.15.2:0/1767159970 wait complete. 2024-05-14T16:55:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:10.100+0000 7fa1af625740 20 client.25057 trim_cache size 0 max 16384 2024-05-14T16:55:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T16:55:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T16:55:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T16:55:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:10.100+0000 7fa1ac82a640 1 -- 172.21.15.2:0/2440969439 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fa1940b2b50 msgr2=0x7fa1940b5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:10.100+0000 7fa1ac82a640 1 --2- 172.21.15.2:0/2440969439 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fa1940b2b50 0x7fa1940b5010 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa1901000f0 tx=0x7fa1901065a0 comp rx=0 tx=0).stop 2024-05-14T16:55:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:10.100+0000 7fa1ac82a640 1 -- 172.21.15.2:0/2440969439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1a8103c00 msgr2=0x7fa1a8101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:10.100+0000 7fa1ac82a640 1 --2- 172.21.15.2:0/2440969439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1a8103c00 0x7fa1a8101850 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa190005940 tx=0x7fa1900040c0 comp rx=0 tx=0).stop 2024-05-14T16:55:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:10.100+0000 7fa1ac82a640 1 -- 172.21.15.2:0/2440969439 shutdown_connections 2024-05-14T16:55:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:10.100+0000 7fa1ac82a640 1 --2- 172.21.15.2:0/2440969439 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fa1940b2b50 0x7fa1940b5010 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:10.100+0000 7fa1ac82a640 1 --2- 172.21.15.2:0/2440969439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1a810d020 0x7fa1a81022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:10.100+0000 7fa1ac82a640 1 --2- 172.21.15.2:0/2440969439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa1a8104520 0x7fa1a8101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:10.100+0000 7fa1ac82a640 1 --2- 172.21.15.2:0/2440969439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1a8103c00 0x7fa1a8101850 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:10.100+0000 7fa1ac82a640 1 -- 172.21.15.2:0/2440969439 >> 172.21.15.2:0/2440969439 conn(0x7fa1a80fd420 msgr2=0x7fa1a80fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:10.100+0000 7fa1ac82a640 1 -- 172.21.15.2:0/2440969439 shutdown_connections 2024-05-14T16:55:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:10.100+0000 7fa1ac82a640 1 -- 172.21.15.2:0/2440969439 wait complete. 2024-05-14T16:55:10.103 INFO:teuthology.orchestra.run.smithi002.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2024-05-14T16:55:10.105 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-05-14T16:55:10.150 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/: 2024-05-14T16:55:10.151 INFO:teuthology.orchestra.run.smithi002.stdout:total 0 2024-05-14T16:55:10.151 INFO:teuthology.orchestra.run.smithi002.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 16:55 . 2024-05-14T16:55:10.151 INFO:teuthology.orchestra.run.smithi002.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 May 14 16:55 .. 2024-05-14T16:55:10.151 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 14 16:55 data 2024-05-14T16:55:10.151 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:55:10.151 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data: 2024-05-14T16:55:10.151 INFO:teuthology.orchestra.run.smithi002.stdout:total 35 2024-05-14T16:55:10.151 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 14 16:55 . 2024-05-14T16:55:10.151 INFO:teuthology.orchestra.run.smithi002.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 16:55 .. 2024-05-14T16:55:10.152 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 16:55 base 2024-05-14T16:55:10.152 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 May 14 16:55 global 2024-05-14T16:55:10.152 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 log 2024-05-14T16:55:10.152 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 pg_commit_ts 2024-05-14T16:55:10.152 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 pg_dynshmem 2024-05-14T16:55:10.152 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 May 14 16:55 pg_hba.conf 2024-05-14T16:55:10.152 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 May 14 16:55 pg_ident.conf 2024-05-14T16:55:10.152 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 16:55 pg_logical 2024-05-14T16:55:10.152 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 14 16:55 pg_multixact 2024-05-14T16:55:10.152 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 pg_notify 2024-05-14T16:55:10.152 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 pg_replslot 2024-05-14T16:55:10.153 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 pg_serial 2024-05-14T16:55:10.153 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 pg_snapshots 2024-05-14T16:55:10.153 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 pg_stat 2024-05-14T16:55:10.153 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 pg_stat_tmp 2024-05-14T16:55:10.153 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 16:55 pg_subtrans 2024-05-14T16:55:10.153 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 pg_tblspc 2024-05-14T16:55:10.153 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 pg_twophase 2024-05-14T16:55:10.153 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 16:55 PG_VERSION 2024-05-14T16:55:10.153 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 14 16:55 pg_wal 2024-05-14T16:55:10.153 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 16:55 pg_xact 2024-05-14T16:55:10.154 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 May 14 16:55 postgresql.auto.conf 2024-05-14T16:55:10.154 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 May 14 16:55 postgresql.conf 2024-05-14T16:55:10.154 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:55:10.154 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/base: 2024-05-14T16:55:10.154 INFO:teuthology.orchestra.run.smithi002.stdout:total 0 2024-05-14T16:55:10.154 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 16:55 . 2024-05-14T16:55:10.154 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 14 16:55 .. 2024-05-14T16:55:10.154 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 May 14 16:55 1 2024-05-14T16:55:10.154 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 May 14 16:55 13493 2024-05-14T16:55:10.154 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 May 14 16:55 13494 2024-05-14T16:55:10.155 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:55:10.155 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/base/1: 2024-05-14T16:55:10.155 INFO:teuthology.orchestra.run.smithi002.stdout:total 7777 2024-05-14T16:55:10.155 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 May 14 16:55 . 2024-05-14T16:55:10.155 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 16:55 .. 2024-05-14T16:55:10.155 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 112 2024-05-14T16:55:10.155 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 113 2024-05-14T16:55:10.155 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 14 16:55 1247 2024-05-14T16:55:10.155 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 1247_fsm 2024-05-14T16:55:10.155 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1247_vm 2024-05-14T16:55:10.155 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 14 16:55 1249 2024-05-14T16:55:10.155 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 1249_fsm 2024-05-14T16:55:10.156 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1249_vm 2024-05-14T16:55:10.156 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 14 16:55 1255 2024-05-14T16:55:10.156 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 1255_fsm 2024-05-14T16:55:10.156 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1255_vm 2024-05-14T16:55:10.156 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 14 16:55 1259 2024-05-14T16:55:10.156 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 1259_fsm 2024-05-14T16:55:10.156 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1259_vm 2024-05-14T16:55:10.156 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 14 16:55 13344 2024-05-14T16:55:10.156 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 13344_fsm 2024-05-14T16:55:10.156 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13344_vm 2024-05-14T16:55:10.156 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 13346 2024-05-14T16:55:10.156 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13348 2024-05-14T16:55:10.156 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13349 2024-05-14T16:55:10.157 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 13349_fsm 2024-05-14T16:55:10.157 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13349_vm 2024-05-14T16:55:10.157 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 13351 2024-05-14T16:55:10.157 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13353 2024-05-14T16:55:10.157 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13354 2024-05-14T16:55:10.157 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 13354_fsm 2024-05-14T16:55:10.157 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13354_vm 2024-05-14T16:55:10.157 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 13356 2024-05-14T16:55:10.157 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13358 2024-05-14T16:55:10.157 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13359 2024-05-14T16:55:10.157 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 13359_fsm 2024-05-14T16:55:10.157 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13359_vm 2024-05-14T16:55:10.157 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 13361 2024-05-14T16:55:10.157 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13363 2024-05-14T16:55:10.158 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 1417 2024-05-14T16:55:10.158 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 1418 2024-05-14T16:55:10.158 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 174 2024-05-14T16:55:10.158 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 175 2024-05-14T16:55:10.158 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2187 2024-05-14T16:55:10.158 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2224 2024-05-14T16:55:10.158 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2328 2024-05-14T16:55:10.158 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2336 2024-05-14T16:55:10.158 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2337 2024-05-14T16:55:10.158 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2579 2024-05-14T16:55:10.158 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2600 2024-05-14T16:55:10.158 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2600_fsm 2024-05-14T16:55:10.158 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2600_vm 2024-05-14T16:55:10.158 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2601 2024-05-14T16:55:10.159 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2601_fsm 2024-05-14T16:55:10.159 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2601_vm 2024-05-14T16:55:10.159 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 14 16:55 2602 2024-05-14T16:55:10.159 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2602_fsm 2024-05-14T16:55:10.159 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2602_vm 2024-05-14T16:55:10.159 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2603 2024-05-14T16:55:10.159 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2603_fsm 2024-05-14T16:55:10.159 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2603_vm 2024-05-14T16:55:10.159 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2604 2024-05-14T16:55:10.159 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2605 2024-05-14T16:55:10.159 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2605_fsm 2024-05-14T16:55:10.159 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2605_vm 2024-05-14T16:55:10.159 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2606 2024-05-14T16:55:10.159 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2606_fsm 2024-05-14T16:55:10.160 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2606_vm 2024-05-14T16:55:10.160 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2607 2024-05-14T16:55:10.160 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2607_fsm 2024-05-14T16:55:10.160 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2607_vm 2024-05-14T16:55:10.160 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 14 16:55 2608 2024-05-14T16:55:10.160 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2608_fsm 2024-05-14T16:55:10.160 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2608_vm 2024-05-14T16:55:10.160 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 14 16:55 2609 2024-05-14T16:55:10.160 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2609_fsm 2024-05-14T16:55:10.160 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2609_vm 2024-05-14T16:55:10.160 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2610 2024-05-14T16:55:10.160 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2610_fsm 2024-05-14T16:55:10.160 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2610_vm 2024-05-14T16:55:10.160 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2611 2024-05-14T16:55:10.160 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2612 2024-05-14T16:55:10.160 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2612_fsm 2024-05-14T16:55:10.161 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2612_vm 2024-05-14T16:55:10.161 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2613 2024-05-14T16:55:10.161 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2615 2024-05-14T16:55:10.161 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2615_fsm 2024-05-14T16:55:10.161 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2615_vm 2024-05-14T16:55:10.161 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2616 2024-05-14T16:55:10.161 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2616_fsm 2024-05-14T16:55:10.161 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2616_vm 2024-05-14T16:55:10.161 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 14 16:55 2617 2024-05-14T16:55:10.161 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2617_fsm 2024-05-14T16:55:10.161 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2617_vm 2024-05-14T16:55:10.161 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 14 16:55 2618 2024-05-14T16:55:10.161 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2618_fsm 2024-05-14T16:55:10.161 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2618_vm 2024-05-14T16:55:10.161 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 14 16:55 2619 2024-05-14T16:55:10.162 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2619_fsm 2024-05-14T16:55:10.162 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2619_vm 2024-05-14T16:55:10.162 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2620 2024-05-14T16:55:10.162 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2650 2024-05-14T16:55:10.162 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2651 2024-05-14T16:55:10.162 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2652 2024-05-14T16:55:10.162 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:55 2653 2024-05-14T16:55:10.162 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:55 2654 2024-05-14T16:55:10.162 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2655 2024-05-14T16:55:10.162 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2656 2024-05-14T16:55:10.162 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2657 2024-05-14T16:55:10.162 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 14 16:55 2658 2024-05-14T16:55:10.162 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 14 16:55 2659 2024-05-14T16:55:10.163 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2660 2024-05-14T16:55:10.163 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2661 2024-05-14T16:55:10.163 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2662 2024-05-14T16:55:10.163 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 14 16:55 2663 2024-05-14T16:55:10.163 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2664 2024-05-14T16:55:10.163 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2665 2024-05-14T16:55:10.163 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2666 2024-05-14T16:55:10.163 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2667 2024-05-14T16:55:10.163 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2668 2024-05-14T16:55:10.163 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2669 2024-05-14T16:55:10.163 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2670 2024-05-14T16:55:10.163 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 14 16:55 2673 2024-05-14T16:55:10.163 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 14 16:55 2674 2024-05-14T16:55:10.163 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 14 16:55 2675 2024-05-14T16:55:10.163 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2678 2024-05-14T16:55:10.164 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2679 2024-05-14T16:55:10.164 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2680 2024-05-14T16:55:10.164 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2681 2024-05-14T16:55:10.164 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2682 2024-05-14T16:55:10.164 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2683 2024-05-14T16:55:10.164 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2684 2024-05-14T16:55:10.164 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2685 2024-05-14T16:55:10.164 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2686 2024-05-14T16:55:10.164 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2687 2024-05-14T16:55:10.164 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:55 2688 2024-05-14T16:55:10.164 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 14 16:55 2689 2024-05-14T16:55:10.164 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 14 16:55 2690 2024-05-14T16:55:10.164 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 14 16:55 2691 2024-05-14T16:55:10.165 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2692 2024-05-14T16:55:10.165 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2693 2024-05-14T16:55:10.165 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2696 2024-05-14T16:55:10.165 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2699 2024-05-14T16:55:10.165 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2701 2024-05-14T16:55:10.165 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2702 2024-05-14T16:55:10.165 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2703 2024-05-14T16:55:10.165 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:55 2704 2024-05-14T16:55:10.165 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2753 2024-05-14T16:55:10.165 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2753_fsm 2024-05-14T16:55:10.165 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2753_vm 2024-05-14T16:55:10.165 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2754 2024-05-14T16:55:10.165 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2755 2024-05-14T16:55:10.165 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2756 2024-05-14T16:55:10.166 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2757 2024-05-14T16:55:10.166 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2830 2024-05-14T16:55:10.166 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2831 2024-05-14T16:55:10.166 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2832 2024-05-14T16:55:10.166 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2833 2024-05-14T16:55:10.166 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2834 2024-05-14T16:55:10.166 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2835 2024-05-14T16:55:10.166 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2836 2024-05-14T16:55:10.166 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2837 2024-05-14T16:55:10.166 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 14 16:55 2838 2024-05-14T16:55:10.166 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2838_fsm 2024-05-14T16:55:10.166 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2838_vm 2024-05-14T16:55:10.167 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2839 2024-05-14T16:55:10.167 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2840 2024-05-14T16:55:10.167 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2840_fsm 2024-05-14T16:55:10.167 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2840_vm 2024-05-14T16:55:10.167 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2841 2024-05-14T16:55:10.167 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2995 2024-05-14T16:55:10.167 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2996 2024-05-14T16:55:10.167 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3079 2024-05-14T16:55:10.167 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3079_fsm 2024-05-14T16:55:10.167 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3079_vm 2024-05-14T16:55:10.167 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3080 2024-05-14T16:55:10.167 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3081 2024-05-14T16:55:10.167 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:55 3085 2024-05-14T16:55:10.168 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3118 2024-05-14T16:55:10.168 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3119 2024-05-14T16:55:10.168 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 14 16:55 3164 2024-05-14T16:55:10.168 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3256 2024-05-14T16:55:10.168 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3257 2024-05-14T16:55:10.168 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3258 2024-05-14T16:55:10.168 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3350 2024-05-14T16:55:10.168 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3351 2024-05-14T16:55:10.168 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3379 2024-05-14T16:55:10.168 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3380 2024-05-14T16:55:10.168 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3381 2024-05-14T16:55:10.168 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3394 2024-05-14T16:55:10.168 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3394_fsm 2024-05-14T16:55:10.169 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3394_vm 2024-05-14T16:55:10.169 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3395 2024-05-14T16:55:10.169 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3429 2024-05-14T16:55:10.169 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3430 2024-05-14T16:55:10.169 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3431 2024-05-14T16:55:10.169 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3433 2024-05-14T16:55:10.169 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3439 2024-05-14T16:55:10.169 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3440 2024-05-14T16:55:10.169 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:55 3455 2024-05-14T16:55:10.169 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 14 16:55 3456 2024-05-14T16:55:10.169 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3456_fsm 2024-05-14T16:55:10.169 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3456_vm 2024-05-14T16:55:10.169 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3466 2024-05-14T16:55:10.169 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3467 2024-05-14T16:55:10.169 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3468 2024-05-14T16:55:10.170 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3501 2024-05-14T16:55:10.170 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3502 2024-05-14T16:55:10.170 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3503 2024-05-14T16:55:10.170 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3534 2024-05-14T16:55:10.170 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3541 2024-05-14T16:55:10.170 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3541_fsm 2024-05-14T16:55:10.170 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3541_vm 2024-05-14T16:55:10.170 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3542 2024-05-14T16:55:10.170 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3574 2024-05-14T16:55:10.170 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3575 2024-05-14T16:55:10.170 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3576 2024-05-14T16:55:10.170 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3596 2024-05-14T16:55:10.170 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3597 2024-05-14T16:55:10.170 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3598 2024-05-14T16:55:10.171 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3599 2024-05-14T16:55:10.171 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3600 2024-05-14T16:55:10.171 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3600_fsm 2024-05-14T16:55:10.171 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3600_vm 2024-05-14T16:55:10.171 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3601 2024-05-14T16:55:10.171 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3601_fsm 2024-05-14T16:55:10.171 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3601_vm 2024-05-14T16:55:10.171 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3602 2024-05-14T16:55:10.171 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3602_fsm 2024-05-14T16:55:10.171 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3602_vm 2024-05-14T16:55:10.172 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3603 2024-05-14T16:55:10.172 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3603_fsm 2024-05-14T16:55:10.172 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3603_vm 2024-05-14T16:55:10.172 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3604 2024-05-14T16:55:10.172 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3605 2024-05-14T16:55:10.172 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3606 2024-05-14T16:55:10.172 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3607 2024-05-14T16:55:10.172 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3608 2024-05-14T16:55:10.172 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 3609 2024-05-14T16:55:10.172 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3712 2024-05-14T16:55:10.172 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3764 2024-05-14T16:55:10.172 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3764_fsm 2024-05-14T16:55:10.172 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3764_vm 2024-05-14T16:55:10.173 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3766 2024-05-14T16:55:10.173 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3767 2024-05-14T16:55:10.173 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3997 2024-05-14T16:55:10.173 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4143 2024-05-14T16:55:10.173 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4144 2024-05-14T16:55:10.173 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4145 2024-05-14T16:55:10.173 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4146 2024-05-14T16:55:10.173 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4147 2024-05-14T16:55:10.173 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4148 2024-05-14T16:55:10.173 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4149 2024-05-14T16:55:10.173 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4150 2024-05-14T16:55:10.173 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4151 2024-05-14T16:55:10.173 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4152 2024-05-14T16:55:10.174 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4153 2024-05-14T16:55:10.174 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4154 2024-05-14T16:55:10.174 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4155 2024-05-14T16:55:10.174 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4156 2024-05-14T16:55:10.174 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4157 2024-05-14T16:55:10.174 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4158 2024-05-14T16:55:10.174 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4159 2024-05-14T16:55:10.174 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4160 2024-05-14T16:55:10.174 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4161 2024-05-14T16:55:10.174 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4162 2024-05-14T16:55:10.175 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4163 2024-05-14T16:55:10.175 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4164 2024-05-14T16:55:10.175 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4165 2024-05-14T16:55:10.178 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4166 2024-05-14T16:55:10.178 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4167 2024-05-14T16:55:10.178 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4168 2024-05-14T16:55:10.178 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4169 2024-05-14T16:55:10.178 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4170 2024-05-14T16:55:10.178 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4171 2024-05-14T16:55:10.179 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4172 2024-05-14T16:55:10.179 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4173 2024-05-14T16:55:10.179 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4174 2024-05-14T16:55:10.179 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 5002 2024-05-14T16:55:10.179 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 548 2024-05-14T16:55:10.179 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 549 2024-05-14T16:55:10.179 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 6102 2024-05-14T16:55:10.179 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 6104 2024-05-14T16:55:10.179 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 6106 2024-05-14T16:55:10.179 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 6110 2024-05-14T16:55:10.179 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 6111 2024-05-14T16:55:10.179 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 6112 2024-05-14T16:55:10.180 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 6113 2024-05-14T16:55:10.180 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 6117 2024-05-14T16:55:10.180 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 826 2024-05-14T16:55:10.180 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 827 2024-05-14T16:55:10.180 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 828 2024-05-14T16:55:10.180 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 14 16:55 pg_filenode.map 2024-05-14T16:55:10.180 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 16:55 PG_VERSION 2024-05-14T16:55:10.180 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:55:10.180 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-05-14T16:55:10.180 INFO:teuthology.orchestra.run.smithi002.stdout:total 7777 2024-05-14T16:55:10.180 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 May 14 16:55 . 2024-05-14T16:55:10.181 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 16:55 .. 2024-05-14T16:55:10.181 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 112 2024-05-14T16:55:10.181 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 113 2024-05-14T16:55:10.181 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 14 16:55 1247 2024-05-14T16:55:10.181 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 1247_fsm 2024-05-14T16:55:10.181 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1247_vm 2024-05-14T16:55:10.181 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 14 16:55 1249 2024-05-14T16:55:10.181 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 1249_fsm 2024-05-14T16:55:10.181 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1249_vm 2024-05-14T16:55:10.181 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 14 16:55 1255 2024-05-14T16:55:10.181 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 1255_fsm 2024-05-14T16:55:10.181 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1255_vm 2024-05-14T16:55:10.182 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 14 16:55 1259 2024-05-14T16:55:10.182 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 1259_fsm 2024-05-14T16:55:10.182 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1259_vm 2024-05-14T16:55:10.182 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 14 16:55 13344 2024-05-14T16:55:10.182 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 13344_fsm 2024-05-14T16:55:10.182 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13344_vm 2024-05-14T16:55:10.182 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 13346 2024-05-14T16:55:10.182 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13348 2024-05-14T16:55:10.182 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13349 2024-05-14T16:55:10.182 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 13349_fsm 2024-05-14T16:55:10.182 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13349_vm 2024-05-14T16:55:10.182 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 13351 2024-05-14T16:55:10.182 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13353 2024-05-14T16:55:10.182 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13354 2024-05-14T16:55:10.183 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 13354_fsm 2024-05-14T16:55:10.183 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13354_vm 2024-05-14T16:55:10.183 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 13356 2024-05-14T16:55:10.183 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13358 2024-05-14T16:55:10.183 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13359 2024-05-14T16:55:10.183 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 13359_fsm 2024-05-14T16:55:10.183 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13359_vm 2024-05-14T16:55:10.183 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 13361 2024-05-14T16:55:10.183 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13363 2024-05-14T16:55:10.183 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 1417 2024-05-14T16:55:10.183 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 1418 2024-05-14T16:55:10.183 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 174 2024-05-14T16:55:10.183 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 175 2024-05-14T16:55:10.183 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2187 2024-05-14T16:55:10.183 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2224 2024-05-14T16:55:10.184 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2328 2024-05-14T16:55:10.184 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2336 2024-05-14T16:55:10.184 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2337 2024-05-14T16:55:10.184 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2579 2024-05-14T16:55:10.184 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2600 2024-05-14T16:55:10.184 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2600_fsm 2024-05-14T16:55:10.184 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2600_vm 2024-05-14T16:55:10.184 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2601 2024-05-14T16:55:10.184 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2601_fsm 2024-05-14T16:55:10.184 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2601_vm 2024-05-14T16:55:10.184 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 14 16:55 2602 2024-05-14T16:55:10.184 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2602_fsm 2024-05-14T16:55:10.184 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2602_vm 2024-05-14T16:55:10.184 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2603 2024-05-14T16:55:10.184 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2603_fsm 2024-05-14T16:55:10.185 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2603_vm 2024-05-14T16:55:10.185 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2604 2024-05-14T16:55:10.185 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2605 2024-05-14T16:55:10.185 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2605_fsm 2024-05-14T16:55:10.185 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2605_vm 2024-05-14T16:55:10.185 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2606 2024-05-14T16:55:10.185 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2606_fsm 2024-05-14T16:55:10.185 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2606_vm 2024-05-14T16:55:10.185 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2607 2024-05-14T16:55:10.185 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2607_fsm 2024-05-14T16:55:10.185 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2607_vm 2024-05-14T16:55:10.185 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 14 16:55 2608 2024-05-14T16:55:10.185 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2608_fsm 2024-05-14T16:55:10.185 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2608_vm 2024-05-14T16:55:10.185 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 14 16:55 2609 2024-05-14T16:55:10.186 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2609_fsm 2024-05-14T16:55:10.186 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2609_vm 2024-05-14T16:55:10.186 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2610 2024-05-14T16:55:10.186 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2610_fsm 2024-05-14T16:55:10.186 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2610_vm 2024-05-14T16:55:10.186 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2611 2024-05-14T16:55:10.186 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2612 2024-05-14T16:55:10.186 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2612_fsm 2024-05-14T16:55:10.186 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2612_vm 2024-05-14T16:55:10.186 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2613 2024-05-14T16:55:10.186 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2615 2024-05-14T16:55:10.186 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2615_fsm 2024-05-14T16:55:10.186 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2615_vm 2024-05-14T16:55:10.186 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2616 2024-05-14T16:55:10.186 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2616_fsm 2024-05-14T16:55:10.187 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2616_vm 2024-05-14T16:55:10.187 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 14 16:55 2617 2024-05-14T16:55:10.187 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2617_fsm 2024-05-14T16:55:10.187 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2617_vm 2024-05-14T16:55:10.187 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 14 16:55 2618 2024-05-14T16:55:10.187 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2618_fsm 2024-05-14T16:55:10.187 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2618_vm 2024-05-14T16:55:10.187 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 14 16:55 2619 2024-05-14T16:55:10.187 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2619_fsm 2024-05-14T16:55:10.187 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2619_vm 2024-05-14T16:55:10.187 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2620 2024-05-14T16:55:10.187 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2650 2024-05-14T16:55:10.187 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2651 2024-05-14T16:55:10.187 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2652 2024-05-14T16:55:10.188 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:55 2653 2024-05-14T16:55:10.188 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:55 2654 2024-05-14T16:55:10.188 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2655 2024-05-14T16:55:10.188 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2656 2024-05-14T16:55:10.188 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2657 2024-05-14T16:55:10.188 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 14 16:55 2658 2024-05-14T16:55:10.188 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 14 16:55 2659 2024-05-14T16:55:10.188 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2660 2024-05-14T16:55:10.188 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2661 2024-05-14T16:55:10.188 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2662 2024-05-14T16:55:10.188 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 14 16:55 2663 2024-05-14T16:55:10.188 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2664 2024-05-14T16:55:10.188 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2665 2024-05-14T16:55:10.188 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2666 2024-05-14T16:55:10.188 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2667 2024-05-14T16:55:10.189 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2668 2024-05-14T16:55:10.189 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2669 2024-05-14T16:55:10.189 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2670 2024-05-14T16:55:10.189 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 14 16:55 2673 2024-05-14T16:55:10.189 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 14 16:55 2674 2024-05-14T16:55:10.189 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 14 16:55 2675 2024-05-14T16:55:10.189 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2678 2024-05-14T16:55:10.189 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2679 2024-05-14T16:55:10.189 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2680 2024-05-14T16:55:10.189 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2681 2024-05-14T16:55:10.189 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2682 2024-05-14T16:55:10.189 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2683 2024-05-14T16:55:10.189 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2684 2024-05-14T16:55:10.189 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2685 2024-05-14T16:55:10.189 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2686 2024-05-14T16:55:10.190 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2687 2024-05-14T16:55:10.190 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:55 2688 2024-05-14T16:55:10.190 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 14 16:55 2689 2024-05-14T16:55:10.190 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 14 16:55 2690 2024-05-14T16:55:10.190 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 14 16:55 2691 2024-05-14T16:55:10.190 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2692 2024-05-14T16:55:10.190 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2693 2024-05-14T16:55:10.190 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2696 2024-05-14T16:55:10.190 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2699 2024-05-14T16:55:10.190 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2701 2024-05-14T16:55:10.190 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2702 2024-05-14T16:55:10.190 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2703 2024-05-14T16:55:10.190 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:55 2704 2024-05-14T16:55:10.190 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2753 2024-05-14T16:55:10.190 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2753_fsm 2024-05-14T16:55:10.190 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2753_vm 2024-05-14T16:55:10.191 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2754 2024-05-14T16:55:10.191 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2755 2024-05-14T16:55:10.191 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2756 2024-05-14T16:55:10.191 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2757 2024-05-14T16:55:10.191 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2830 2024-05-14T16:55:10.191 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2831 2024-05-14T16:55:10.191 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2832 2024-05-14T16:55:10.191 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2833 2024-05-14T16:55:10.191 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2834 2024-05-14T16:55:10.191 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2835 2024-05-14T16:55:10.191 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2836 2024-05-14T16:55:10.191 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2837 2024-05-14T16:55:10.191 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 14 16:55 2838 2024-05-14T16:55:10.191 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2838_fsm 2024-05-14T16:55:10.191 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2838_vm 2024-05-14T16:55:10.192 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2839 2024-05-14T16:55:10.192 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2840 2024-05-14T16:55:10.192 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2840_fsm 2024-05-14T16:55:10.192 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2840_vm 2024-05-14T16:55:10.192 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2841 2024-05-14T16:55:10.192 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2995 2024-05-14T16:55:10.192 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2996 2024-05-14T16:55:10.192 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3079 2024-05-14T16:55:10.192 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3079_fsm 2024-05-14T16:55:10.192 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3079_vm 2024-05-14T16:55:10.192 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3080 2024-05-14T16:55:10.192 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3081 2024-05-14T16:55:10.192 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:55 3085 2024-05-14T16:55:10.192 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3118 2024-05-14T16:55:10.192 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3119 2024-05-14T16:55:10.193 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 14 16:55 3164 2024-05-14T16:55:10.193 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3256 2024-05-14T16:55:10.193 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3257 2024-05-14T16:55:10.193 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3258 2024-05-14T16:55:10.193 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3350 2024-05-14T16:55:10.193 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3351 2024-05-14T16:55:10.193 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3379 2024-05-14T16:55:10.193 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3380 2024-05-14T16:55:10.193 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3381 2024-05-14T16:55:10.193 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3394 2024-05-14T16:55:10.193 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3394_fsm 2024-05-14T16:55:10.193 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3394_vm 2024-05-14T16:55:10.193 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3395 2024-05-14T16:55:10.193 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3429 2024-05-14T16:55:10.193 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3430 2024-05-14T16:55:10.193 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3431 2024-05-14T16:55:10.194 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3433 2024-05-14T16:55:10.194 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3439 2024-05-14T16:55:10.194 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3440 2024-05-14T16:55:10.194 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:55 3455 2024-05-14T16:55:10.194 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 14 16:55 3456 2024-05-14T16:55:10.194 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3456_fsm 2024-05-14T16:55:10.194 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3456_vm 2024-05-14T16:55:10.194 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3466 2024-05-14T16:55:10.194 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3467 2024-05-14T16:55:10.194 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3468 2024-05-14T16:55:10.194 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3501 2024-05-14T16:55:10.194 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3502 2024-05-14T16:55:10.195 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3503 2024-05-14T16:55:10.195 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3534 2024-05-14T16:55:10.195 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3541 2024-05-14T16:55:10.195 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3541_fsm 2024-05-14T16:55:10.195 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3541_vm 2024-05-14T16:55:10.195 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3542 2024-05-14T16:55:10.195 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3574 2024-05-14T16:55:10.195 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3575 2024-05-14T16:55:10.195 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3576 2024-05-14T16:55:10.195 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3596 2024-05-14T16:55:10.195 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3597 2024-05-14T16:55:10.195 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3598 2024-05-14T16:55:10.195 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3599 2024-05-14T16:55:10.195 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3600 2024-05-14T16:55:10.196 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3600_fsm 2024-05-14T16:55:10.196 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3600_vm 2024-05-14T16:55:10.196 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3601 2024-05-14T16:55:10.196 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3601_fsm 2024-05-14T16:55:10.196 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3601_vm 2024-05-14T16:55:10.196 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3602 2024-05-14T16:55:10.196 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3602_fsm 2024-05-14T16:55:10.196 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3602_vm 2024-05-14T16:55:10.196 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3603 2024-05-14T16:55:10.196 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3603_fsm 2024-05-14T16:55:10.196 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3603_vm 2024-05-14T16:55:10.196 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3604 2024-05-14T16:55:10.196 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3605 2024-05-14T16:55:10.196 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3606 2024-05-14T16:55:10.196 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3607 2024-05-14T16:55:10.197 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3608 2024-05-14T16:55:10.197 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 3609 2024-05-14T16:55:10.197 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3712 2024-05-14T16:55:10.197 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3764 2024-05-14T16:55:10.197 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3764_fsm 2024-05-14T16:55:10.197 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3764_vm 2024-05-14T16:55:10.197 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3766 2024-05-14T16:55:10.197 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3767 2024-05-14T16:55:10.197 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3997 2024-05-14T16:55:10.197 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4143 2024-05-14T16:55:10.197 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4144 2024-05-14T16:55:10.197 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4145 2024-05-14T16:55:10.197 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4146 2024-05-14T16:55:10.198 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4147 2024-05-14T16:55:10.198 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4148 2024-05-14T16:55:10.198 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4149 2024-05-14T16:55:10.198 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4150 2024-05-14T16:55:10.198 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4151 2024-05-14T16:55:10.198 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4152 2024-05-14T16:55:10.198 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4153 2024-05-14T16:55:10.198 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4154 2024-05-14T16:55:10.198 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4155 2024-05-14T16:55:10.198 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4156 2024-05-14T16:55:10.198 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4157 2024-05-14T16:55:10.198 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4158 2024-05-14T16:55:10.198 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4159 2024-05-14T16:55:10.198 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4160 2024-05-14T16:55:10.199 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4161 2024-05-14T16:55:10.199 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4162 2024-05-14T16:55:10.199 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4163 2024-05-14T16:55:10.199 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4164 2024-05-14T16:55:10.199 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4165 2024-05-14T16:55:10.199 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4166 2024-05-14T16:55:10.199 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4167 2024-05-14T16:55:10.199 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4168 2024-05-14T16:55:10.199 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4169 2024-05-14T16:55:10.199 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4170 2024-05-14T16:55:10.199 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4171 2024-05-14T16:55:10.199 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4172 2024-05-14T16:55:10.199 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4173 2024-05-14T16:55:10.199 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4174 2024-05-14T16:55:10.200 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 5002 2024-05-14T16:55:10.200 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 548 2024-05-14T16:55:10.200 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 549 2024-05-14T16:55:10.200 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 6102 2024-05-14T16:55:10.200 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 6104 2024-05-14T16:55:10.200 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 6106 2024-05-14T16:55:10.200 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 6110 2024-05-14T16:55:10.200 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 6111 2024-05-14T16:55:10.200 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 6112 2024-05-14T16:55:10.200 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 6113 2024-05-14T16:55:10.200 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 6117 2024-05-14T16:55:10.200 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 826 2024-05-14T16:55:10.200 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 827 2024-05-14T16:55:10.200 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 828 2024-05-14T16:55:10.200 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 14 16:55 pg_filenode.map 2024-05-14T16:55:10.200 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 16:55 PG_VERSION 2024-05-14T16:55:10.201 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:55:10.201 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-05-14T16:55:10.201 INFO:teuthology.orchestra.run.smithi002.stdout:total 7777 2024-05-14T16:55:10.201 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 May 14 16:55 . 2024-05-14T16:55:10.201 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 16:55 .. 2024-05-14T16:55:10.201 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 112 2024-05-14T16:55:10.201 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 113 2024-05-14T16:55:10.201 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 14 16:55 1247 2024-05-14T16:55:10.201 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 1247_fsm 2024-05-14T16:55:10.201 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1247_vm 2024-05-14T16:55:10.201 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 14 16:55 1249 2024-05-14T16:55:10.201 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 1249_fsm 2024-05-14T16:55:10.201 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1249_vm 2024-05-14T16:55:10.201 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 14 16:55 1255 2024-05-14T16:55:10.201 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 1255_fsm 2024-05-14T16:55:10.202 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1255_vm 2024-05-14T16:55:10.202 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 14 16:55 1259 2024-05-14T16:55:10.202 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 1259_fsm 2024-05-14T16:55:10.202 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1259_vm 2024-05-14T16:55:10.202 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 14 16:55 13344 2024-05-14T16:55:10.202 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 13344_fsm 2024-05-14T16:55:10.202 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13344_vm 2024-05-14T16:55:10.202 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 13346 2024-05-14T16:55:10.202 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13348 2024-05-14T16:55:10.202 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13349 2024-05-14T16:55:10.202 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 13349_fsm 2024-05-14T16:55:10.202 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13349_vm 2024-05-14T16:55:10.202 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 13351 2024-05-14T16:55:10.202 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13353 2024-05-14T16:55:10.203 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13354 2024-05-14T16:55:10.203 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 13354_fsm 2024-05-14T16:55:10.203 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13354_vm 2024-05-14T16:55:10.203 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 13356 2024-05-14T16:55:10.203 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13358 2024-05-14T16:55:10.203 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13359 2024-05-14T16:55:10.203 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 13359_fsm 2024-05-14T16:55:10.203 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13359_vm 2024-05-14T16:55:10.203 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 13361 2024-05-14T16:55:10.203 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13363 2024-05-14T16:55:10.203 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 1417 2024-05-14T16:55:10.203 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 1418 2024-05-14T16:55:10.203 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 174 2024-05-14T16:55:10.203 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 175 2024-05-14T16:55:10.203 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2187 2024-05-14T16:55:10.204 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2224 2024-05-14T16:55:10.204 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2328 2024-05-14T16:55:10.204 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2336 2024-05-14T16:55:10.204 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2337 2024-05-14T16:55:10.204 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2579 2024-05-14T16:55:10.204 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2600 2024-05-14T16:55:10.204 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2600_fsm 2024-05-14T16:55:10.204 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2600_vm 2024-05-14T16:55:10.204 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2601 2024-05-14T16:55:10.204 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2601_fsm 2024-05-14T16:55:10.204 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2601_vm 2024-05-14T16:55:10.204 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 14 16:55 2602 2024-05-14T16:55:10.204 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2602_fsm 2024-05-14T16:55:10.204 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2602_vm 2024-05-14T16:55:10.204 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2603 2024-05-14T16:55:10.205 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2603_fsm 2024-05-14T16:55:10.205 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2603_vm 2024-05-14T16:55:10.205 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2604 2024-05-14T16:55:10.205 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2605 2024-05-14T16:55:10.205 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2605_fsm 2024-05-14T16:55:10.205 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2605_vm 2024-05-14T16:55:10.205 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2606 2024-05-14T16:55:10.205 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2606_fsm 2024-05-14T16:55:10.205 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2606_vm 2024-05-14T16:55:10.205 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2607 2024-05-14T16:55:10.205 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2607_fsm 2024-05-14T16:55:10.205 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2607_vm 2024-05-14T16:55:10.205 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 14 16:55 2608 2024-05-14T16:55:10.205 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2608_fsm 2024-05-14T16:55:10.205 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2608_vm 2024-05-14T16:55:10.206 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 14 16:55 2609 2024-05-14T16:55:10.206 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2609_fsm 2024-05-14T16:55:10.206 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2609_vm 2024-05-14T16:55:10.206 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2610 2024-05-14T16:55:10.206 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2610_fsm 2024-05-14T16:55:10.206 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2610_vm 2024-05-14T16:55:10.206 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2611 2024-05-14T16:55:10.206 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2612 2024-05-14T16:55:10.206 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2612_fsm 2024-05-14T16:55:10.206 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2612_vm 2024-05-14T16:55:10.206 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2613 2024-05-14T16:55:10.206 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2615 2024-05-14T16:55:10.206 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2615_fsm 2024-05-14T16:55:10.207 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2615_vm 2024-05-14T16:55:10.207 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2616 2024-05-14T16:55:10.207 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2616_fsm 2024-05-14T16:55:10.207 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2616_vm 2024-05-14T16:55:10.207 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 14 16:55 2617 2024-05-14T16:55:10.207 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2617_fsm 2024-05-14T16:55:10.207 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2617_vm 2024-05-14T16:55:10.207 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 14 16:55 2618 2024-05-14T16:55:10.207 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2618_fsm 2024-05-14T16:55:10.207 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2618_vm 2024-05-14T16:55:10.207 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 14 16:55 2619 2024-05-14T16:55:10.207 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2619_fsm 2024-05-14T16:55:10.207 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2619_vm 2024-05-14T16:55:10.207 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2620 2024-05-14T16:55:10.207 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2650 2024-05-14T16:55:10.208 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2651 2024-05-14T16:55:10.208 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2652 2024-05-14T16:55:10.208 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:55 2653 2024-05-14T16:55:10.208 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:55 2654 2024-05-14T16:55:10.208 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2655 2024-05-14T16:55:10.208 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2656 2024-05-14T16:55:10.208 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2657 2024-05-14T16:55:10.208 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 14 16:55 2658 2024-05-14T16:55:10.208 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 14 16:55 2659 2024-05-14T16:55:10.208 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2660 2024-05-14T16:55:10.208 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2661 2024-05-14T16:55:10.208 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2662 2024-05-14T16:55:10.208 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 14 16:55 2663 2024-05-14T16:55:10.208 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2664 2024-05-14T16:55:10.208 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2665 2024-05-14T16:55:10.209 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2666 2024-05-14T16:55:10.209 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2667 2024-05-14T16:55:10.209 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2668 2024-05-14T16:55:10.209 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2669 2024-05-14T16:55:10.209 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2670 2024-05-14T16:55:10.209 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 14 16:55 2673 2024-05-14T16:55:10.209 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 14 16:55 2674 2024-05-14T16:55:10.209 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 14 16:55 2675 2024-05-14T16:55:10.209 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2678 2024-05-14T16:55:10.209 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2679 2024-05-14T16:55:10.209 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2680 2024-05-14T16:55:10.209 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2681 2024-05-14T16:55:10.209 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2682 2024-05-14T16:55:10.209 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2683 2024-05-14T16:55:10.210 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2684 2024-05-14T16:55:10.210 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2685 2024-05-14T16:55:10.210 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2686 2024-05-14T16:55:10.210 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2687 2024-05-14T16:55:10.210 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:55 2688 2024-05-14T16:55:10.210 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 14 16:55 2689 2024-05-14T16:55:10.210 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 14 16:55 2690 2024-05-14T16:55:10.210 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 14 16:55 2691 2024-05-14T16:55:10.210 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2692 2024-05-14T16:55:10.210 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2693 2024-05-14T16:55:10.210 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2696 2024-05-14T16:55:10.210 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2699 2024-05-14T16:55:10.210 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2701 2024-05-14T16:55:10.210 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2702 2024-05-14T16:55:10.210 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2703 2024-05-14T16:55:10.211 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:55 2704 2024-05-14T16:55:10.211 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2753 2024-05-14T16:55:10.211 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2753_fsm 2024-05-14T16:55:10.211 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2753_vm 2024-05-14T16:55:10.211 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2754 2024-05-14T16:55:10.211 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2755 2024-05-14T16:55:10.211 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2756 2024-05-14T16:55:10.211 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2757 2024-05-14T16:55:10.211 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2830 2024-05-14T16:55:10.211 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2831 2024-05-14T16:55:10.211 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2832 2024-05-14T16:55:10.211 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2833 2024-05-14T16:55:10.211 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2834 2024-05-14T16:55:10.211 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2835 2024-05-14T16:55:10.212 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2836 2024-05-14T16:55:10.212 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2837 2024-05-14T16:55:10.212 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 14 16:55 2838 2024-05-14T16:55:10.212 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2838_fsm 2024-05-14T16:55:10.212 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2838_vm 2024-05-14T16:55:10.212 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2839 2024-05-14T16:55:10.212 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2840 2024-05-14T16:55:10.212 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2840_fsm 2024-05-14T16:55:10.212 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2840_vm 2024-05-14T16:55:10.212 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2841 2024-05-14T16:55:10.212 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2995 2024-05-14T16:55:10.212 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2996 2024-05-14T16:55:10.212 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3079 2024-05-14T16:55:10.212 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3079_fsm 2024-05-14T16:55:10.212 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3079_vm 2024-05-14T16:55:10.213 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3080 2024-05-14T16:55:10.213 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3081 2024-05-14T16:55:10.213 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:55 3085 2024-05-14T16:55:10.213 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3118 2024-05-14T16:55:10.213 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3119 2024-05-14T16:55:10.213 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 14 16:55 3164 2024-05-14T16:55:10.213 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3256 2024-05-14T16:55:10.213 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3257 2024-05-14T16:55:10.213 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3258 2024-05-14T16:55:10.213 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3350 2024-05-14T16:55:10.213 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3351 2024-05-14T16:55:10.213 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3379 2024-05-14T16:55:10.213 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3380 2024-05-14T16:55:10.213 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3381 2024-05-14T16:55:10.213 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3394 2024-05-14T16:55:10.214 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3394_fsm 2024-05-14T16:55:10.214 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3394_vm 2024-05-14T16:55:10.214 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3395 2024-05-14T16:55:10.214 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3429 2024-05-14T16:55:10.214 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3430 2024-05-14T16:55:10.214 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3431 2024-05-14T16:55:10.214 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3433 2024-05-14T16:55:10.214 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3439 2024-05-14T16:55:10.214 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3440 2024-05-14T16:55:10.214 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:55 3455 2024-05-14T16:55:10.214 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 14 16:55 3456 2024-05-14T16:55:10.214 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3456_fsm 2024-05-14T16:55:10.214 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3456_vm 2024-05-14T16:55:10.214 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3466 2024-05-14T16:55:10.215 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3467 2024-05-14T16:55:10.215 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3468 2024-05-14T16:55:10.215 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3501 2024-05-14T16:55:10.215 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3502 2024-05-14T16:55:10.215 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3503 2024-05-14T16:55:10.215 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3534 2024-05-14T16:55:10.215 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3541 2024-05-14T16:55:10.215 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3541_fsm 2024-05-14T16:55:10.215 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3541_vm 2024-05-14T16:55:10.215 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3542 2024-05-14T16:55:10.215 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3574 2024-05-14T16:55:10.215 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3575 2024-05-14T16:55:10.215 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3576 2024-05-14T16:55:10.215 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3596 2024-05-14T16:55:10.215 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3597 2024-05-14T16:55:10.216 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3598 2024-05-14T16:55:10.216 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3599 2024-05-14T16:55:10.216 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3600 2024-05-14T16:55:10.216 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3600_fsm 2024-05-14T16:55:10.216 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3600_vm 2024-05-14T16:55:10.216 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3601 2024-05-14T16:55:10.216 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3601_fsm 2024-05-14T16:55:10.216 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3601_vm 2024-05-14T16:55:10.216 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3602 2024-05-14T16:55:10.216 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3602_fsm 2024-05-14T16:55:10.216 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3602_vm 2024-05-14T16:55:10.216 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3603 2024-05-14T16:55:10.216 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3603_fsm 2024-05-14T16:55:10.216 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3603_vm 2024-05-14T16:55:10.217 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3604 2024-05-14T16:55:10.217 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3605 2024-05-14T16:55:10.217 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3606 2024-05-14T16:55:10.217 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3607 2024-05-14T16:55:10.217 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3608 2024-05-14T16:55:10.217 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 3609 2024-05-14T16:55:10.217 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3712 2024-05-14T16:55:10.217 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3764 2024-05-14T16:55:10.217 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3764_fsm 2024-05-14T16:55:10.217 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3764_vm 2024-05-14T16:55:10.217 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3766 2024-05-14T16:55:10.217 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3767 2024-05-14T16:55:10.217 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3997 2024-05-14T16:55:10.217 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4143 2024-05-14T16:55:10.218 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4144 2024-05-14T16:55:10.218 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4145 2024-05-14T16:55:10.218 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4146 2024-05-14T16:55:10.218 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4147 2024-05-14T16:55:10.218 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4148 2024-05-14T16:55:10.218 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4149 2024-05-14T16:55:10.218 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4150 2024-05-14T16:55:10.218 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4151 2024-05-14T16:55:10.218 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4152 2024-05-14T16:55:10.218 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4153 2024-05-14T16:55:10.218 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4154 2024-05-14T16:55:10.218 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4155 2024-05-14T16:55:10.218 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4156 2024-05-14T16:55:10.219 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4157 2024-05-14T16:55:10.219 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4158 2024-05-14T16:55:10.219 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4159 2024-05-14T16:55:10.219 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4160 2024-05-14T16:55:10.219 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4161 2024-05-14T16:55:10.219 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4162 2024-05-14T16:55:10.219 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4163 2024-05-14T16:55:10.219 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4164 2024-05-14T16:55:10.219 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4165 2024-05-14T16:55:10.219 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4166 2024-05-14T16:55:10.219 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4167 2024-05-14T16:55:10.219 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4168 2024-05-14T16:55:10.219 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4169 2024-05-14T16:55:10.219 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4170 2024-05-14T16:55:10.219 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4171 2024-05-14T16:55:10.219 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4172 2024-05-14T16:55:10.220 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4173 2024-05-14T16:55:10.220 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4174 2024-05-14T16:55:10.220 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 5002 2024-05-14T16:55:10.220 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 548 2024-05-14T16:55:10.220 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 549 2024-05-14T16:55:10.220 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 6102 2024-05-14T16:55:10.220 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 6104 2024-05-14T16:55:10.220 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 6106 2024-05-14T16:55:10.220 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 6110 2024-05-14T16:55:10.220 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 6111 2024-05-14T16:55:10.220 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 6112 2024-05-14T16:55:10.220 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 6113 2024-05-14T16:55:10.220 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 6117 2024-05-14T16:55:10.220 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 826 2024-05-14T16:55:10.220 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 827 2024-05-14T16:55:10.220 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 828 2024-05-14T16:55:10.221 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 14 16:55 pg_filenode.map 2024-05-14T16:55:10.221 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 16:55 PG_VERSION 2024-05-14T16:55:10.221 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:55:10.221 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/global: 2024-05-14T16:55:10.221 INFO:teuthology.orchestra.run.smithi002.stdout:total 537 2024-05-14T16:55:10.221 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 May 14 16:55 . 2024-05-14T16:55:10.221 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 14 16:55 .. 2024-05-14T16:55:10.221 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1213 2024-05-14T16:55:10.221 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 1213_fsm 2024-05-14T16:55:10.221 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1213_vm 2024-05-14T16:55:10.221 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1214 2024-05-14T16:55:10.221 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 1214_fsm 2024-05-14T16:55:10.221 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1214_vm 2024-05-14T16:55:10.221 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 1232 2024-05-14T16:55:10.221 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 1233 2024-05-14T16:55:10.221 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1260 2024-05-14T16:55:10.222 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 1260_fsm 2024-05-14T16:55:10.222 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1260_vm 2024-05-14T16:55:10.222 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1261 2024-05-14T16:55:10.222 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 1261_fsm 2024-05-14T16:55:10.222 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1261_vm 2024-05-14T16:55:10.222 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1262 2024-05-14T16:55:10.222 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 1262_fsm 2024-05-14T16:55:10.222 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1262_vm 2024-05-14T16:55:10.222 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2396 2024-05-14T16:55:10.222 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2396_fsm 2024-05-14T16:55:10.222 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2396_vm 2024-05-14T16:55:10.222 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2397 2024-05-14T16:55:10.222 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2671 2024-05-14T16:55:10.222 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2672 2024-05-14T16:55:10.223 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2676 2024-05-14T16:55:10.223 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2677 2024-05-14T16:55:10.223 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2694 2024-05-14T16:55:10.223 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2695 2024-05-14T16:55:10.223 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2697 2024-05-14T16:55:10.223 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2698 2024-05-14T16:55:10.223 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2846 2024-05-14T16:55:10.223 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2847 2024-05-14T16:55:10.223 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2964 2024-05-14T16:55:10.223 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2965 2024-05-14T16:55:10.223 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2966 2024-05-14T16:55:10.223 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2967 2024-05-14T16:55:10.223 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3592 2024-05-14T16:55:10.223 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3593 2024-05-14T16:55:10.223 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4060 2024-05-14T16:55:10.224 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4061 2024-05-14T16:55:10.224 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4175 2024-05-14T16:55:10.224 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4176 2024-05-14T16:55:10.224 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4177 2024-05-14T16:55:10.224 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4178 2024-05-14T16:55:10.224 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4181 2024-05-14T16:55:10.224 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4182 2024-05-14T16:55:10.224 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4183 2024-05-14T16:55:10.224 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4184 2024-05-14T16:55:10.224 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4185 2024-05-14T16:55:10.224 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4186 2024-05-14T16:55:10.224 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 6000 2024-05-14T16:55:10.224 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 6001 2024-05-14T16:55:10.224 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 6002 2024-05-14T16:55:10.224 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 6100 2024-05-14T16:55:10.225 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 6114 2024-05-14T16:55:10.225 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 6115 2024-05-14T16:55:10.225 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 pg_control 2024-05-14T16:55:10.225 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 14 16:55 pg_filenode.map 2024-05-14T16:55:10.225 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:55:10.225 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/log: 2024-05-14T16:55:10.225 INFO:teuthology.orchestra.run.smithi002.stdout:total 0 2024-05-14T16:55:10.225 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 . 2024-05-14T16:55:10.225 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 14 16:55 .. 2024-05-14T16:55:10.225 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:55:10.225 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-05-14T16:55:10.225 INFO:teuthology.orchestra.run.smithi002.stdout:total 0 2024-05-14T16:55:10.225 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 . 2024-05-14T16:55:10.225 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 14 16:55 .. 2024-05-14T16:55:10.226 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:55:10.226 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-05-14T16:55:10.226 INFO:teuthology.orchestra.run.smithi002.stdout:total 0 2024-05-14T16:55:10.226 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 . 2024-05-14T16:55:10.226 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 14 16:55 .. 2024-05-14T16:55:10.226 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:55:10.226 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-05-14T16:55:10.226 INFO:teuthology.orchestra.run.smithi002.stdout:total 1 2024-05-14T16:55:10.226 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 16:55 . 2024-05-14T16:55:10.226 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 14 16:55 .. 2024-05-14T16:55:10.226 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 mappings 2024-05-14T16:55:10.226 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 14 16:55 replorigin_checkpoint 2024-05-14T16:55:10.226 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 snapshots 2024-05-14T16:55:10.226 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:55:10.226 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-05-14T16:55:10.227 INFO:teuthology.orchestra.run.smithi002.stdout:total 0 2024-05-14T16:55:10.227 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 . 2024-05-14T16:55:10.227 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 16:55 .. 2024-05-14T16:55:10.227 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:55:10.227 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-05-14T16:55:10.227 INFO:teuthology.orchestra.run.smithi002.stdout:total 0 2024-05-14T16:55:10.227 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 . 2024-05-14T16:55:10.227 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 16:55 .. 2024-05-14T16:55:10.227 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:55:10.227 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-05-14T16:55:10.227 INFO:teuthology.orchestra.run.smithi002.stdout:total 0 2024-05-14T16:55:10.227 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 14 16:55 . 2024-05-14T16:55:10.227 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 14 16:55 .. 2024-05-14T16:55:10.227 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 16:55 members 2024-05-14T16:55:10.227 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 16:55 offsets 2024-05-14T16:55:10.227 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:55:10.228 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-05-14T16:55:10.228 INFO:teuthology.orchestra.run.smithi002.stdout:total 8 2024-05-14T16:55:10.228 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 16:55 . 2024-05-14T16:55:10.228 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 14 16:55 .. 2024-05-14T16:55:10.228 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 0000 2024-05-14T16:55:10.228 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:55:10.228 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-05-14T16:55:10.228 INFO:teuthology.orchestra.run.smithi002.stdout:total 8 2024-05-14T16:55:10.228 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 16:55 . 2024-05-14T16:55:10.228 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 14 16:55 .. 2024-05-14T16:55:10.228 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 0000 2024-05-14T16:55:10.228 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:55:10.228 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-05-14T16:55:10.228 INFO:teuthology.orchestra.run.smithi002.stdout:total 0 2024-05-14T16:55:10.228 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 . 2024-05-14T16:55:10.229 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 14 16:55 .. 2024-05-14T16:55:10.229 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:55:10.229 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-05-14T16:55:10.229 INFO:teuthology.orchestra.run.smithi002.stdout:total 0 2024-05-14T16:55:10.229 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 . 2024-05-14T16:55:10.229 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 14 16:55 .. 2024-05-14T16:55:10.229 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:55:10.229 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-05-14T16:55:10.229 INFO:teuthology.orchestra.run.smithi002.stdout:total 0 2024-05-14T16:55:10.229 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 . 2024-05-14T16:55:10.229 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 14 16:55 .. 2024-05-14T16:55:10.229 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:55:10.229 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-05-14T16:55:10.229 INFO:teuthology.orchestra.run.smithi002.stdout:total 0 2024-05-14T16:55:10.230 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 . 2024-05-14T16:55:10.230 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 14 16:55 .. 2024-05-14T16:55:10.230 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:55:10.230 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-05-14T16:55:10.230 INFO:teuthology.orchestra.run.smithi002.stdout:total 0 2024-05-14T16:55:10.230 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 . 2024-05-14T16:55:10.230 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 14 16:55 .. 2024-05-14T16:55:10.230 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:55:10.230 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-05-14T16:55:10.230 INFO:teuthology.orchestra.run.smithi002.stdout:total 0 2024-05-14T16:55:10.230 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 . 2024-05-14T16:55:10.230 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 14 16:55 .. 2024-05-14T16:55:10.230 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:55:10.230 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-05-14T16:55:10.231 INFO:teuthology.orchestra.run.smithi002.stdout:total 8 2024-05-14T16:55:10.231 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 16:55 . 2024-05-14T16:55:10.231 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 14 16:55 .. 2024-05-14T16:55:10.231 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 0000 2024-05-14T16:55:10.231 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:55:10.231 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-05-14T16:55:10.231 INFO:teuthology.orchestra.run.smithi002.stdout:total 0 2024-05-14T16:55:10.231 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 . 2024-05-14T16:55:10.231 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 14 16:55 .. 2024-05-14T16:55:10.231 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:55:10.231 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-05-14T16:55:10.231 INFO:teuthology.orchestra.run.smithi002.stdout:total 0 2024-05-14T16:55:10.231 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 . 2024-05-14T16:55:10.231 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 14 16:55 .. 2024-05-14T16:55:10.232 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:55:10.232 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-05-14T16:55:10.232 INFO:teuthology.orchestra.run.smithi002.stdout:total 16384 2024-05-14T16:55:10.232 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 14 16:55 . 2024-05-14T16:55:10.232 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 14 16:55 .. 2024-05-14T16:55:10.232 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 16:55 000000010000000000000001 2024-05-14T16:55:10.232 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 archive_status 2024-05-14T16:55:10.232 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:55:10.232 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-05-14T16:55:10.232 INFO:teuthology.orchestra.run.smithi002.stdout:total 0 2024-05-14T16:55:10.232 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 . 2024-05-14T16:55:10.232 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 14 16:55 .. 2024-05-14T16:55:10.232 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:55:10.232 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-05-14T16:55:10.232 INFO:teuthology.orchestra.run.smithi002.stdout:total 8 2024-05-14T16:55:10.233 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 16:55 . 2024-05-14T16:55:10.233 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 May 14 16:55 .. 2024-05-14T16:55:10.233 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 0000 2024-05-14T16:55:10.233 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2024-05-14T16:55:10.234 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph tell mds.h status 2024-05-14T16:55:10.395 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2024-05-14T16:55:10.445 INFO:teuthology.orchestra.run.smithi002.stderr:dropping old tables... 2024-05-14T16:55:10.445 INFO:teuthology.orchestra.run.smithi002.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2024-05-14T16:55:10.445 INFO:teuthology.orchestra.run.smithi002.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2024-05-14T16:55:10.445 INFO:teuthology.orchestra.run.smithi002.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2024-05-14T16:55:10.445 INFO:teuthology.orchestra.run.smithi002.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2024-05-14T16:55:10.445 INFO:teuthology.orchestra.run.smithi002.stderr:creating tables... 2024-05-14T16:55:10.459 INFO:teuthology.orchestra.run.smithi002.stderr:generating data (client-side)... 2024-05-14T16:55:10.538 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:55:10.541 INFO:teuthology.orchestra.run.smithi002.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.06 s, remaining 1.84 s) 2024-05-14T16:55:10.669 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:10 smithi002 ceph-mon[31538]: pgmap v17: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 91 KiB/s wr, 29 op/s 2024-05-14T16:55:10.716 INFO:teuthology.orchestra.run.smithi002.stderr:200000 of 3200000 tuples (6%) done (elapsed 0.23 s, remaining 3.51 s) 2024-05-14T16:55:10.805 INFO:teuthology.orchestra.run.smithi002.stderr:300000 of 3200000 tuples (9%) done (elapsed 0.32 s, remaining 3.12 s) 2024-05-14T16:55:10.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:10 smithi151 ceph-mon[36066]: pgmap v17: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 91 KiB/s wr, 29 op/s 2024-05-14T16:55:10.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:10 smithi123 ceph-mon[35971]: pgmap v17: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 91 KiB/s wr, 29 op/s 2024-05-14T16:55:10.940 INFO:teuthology.orchestra.run.smithi002.stderr:400000 of 3200000 tuples (12%) done (elapsed 0.46 s, remaining 3.21 s) 2024-05-14T16:55:11.085 INFO:teuthology.orchestra.run.smithi002.stderr:500000 of 3200000 tuples (15%) done (elapsed 0.60 s, remaining 3.26 s) 2024-05-14T16:55:11.171 INFO:teuthology.orchestra.run.smithi002.stderr:600000 of 3200000 tuples (18%) done (elapsed 0.69 s, remaining 2.99 s) 2024-05-14T16:55:11.353 INFO:teuthology.orchestra.run.smithi002.stderr:700000 of 3200000 tuples (21%) done (elapsed 0.87 s, remaining 3.11 s) 2024-05-14T16:55:11.501 INFO:teuthology.orchestra.run.smithi002.stderr:800000 of 3200000 tuples (25%) done (elapsed 1.02 s, remaining 3.06 s) 2024-05-14T16:55:11.597 INFO:teuthology.orchestra.run.smithi002.stderr:900000 of 3200000 tuples (28%) done (elapsed 1.12 s, remaining 2.85 s) 2024-05-14T16:55:11.726 INFO:teuthology.orchestra.run.smithi002.stderr:1000000 of 3200000 tuples (31%) done (elapsed 1.24 s, remaining 2.74 s) 2024-05-14T16:55:11.887 INFO:teuthology.orchestra.run.smithi002.stderr:1100000 of 3200000 tuples (34%) done (elapsed 1.40 s, remaining 2.68 s) 2024-05-14T16:55:11.983 INFO:teuthology.orchestra.run.smithi002.stderr:1200000 of 3200000 tuples (37%) done (elapsed 1.50 s, remaining 2.50 s) 2024-05-14T16:55:12.115 INFO:teuthology.orchestra.run.smithi002.stderr:1300000 of 3200000 tuples (40%) done (elapsed 1.63 s, remaining 2.39 s) 2024-05-14T16:55:12.237 INFO:teuthology.orchestra.run.smithi002.stderr:1400000 of 3200000 tuples (43%) done (elapsed 1.75 s, remaining 2.26 s) 2024-05-14T16:55:12.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.314+0000 7f23bcade700 1 -- 172.21.15.2:0/788165648 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23b8101e60 msgr2=0x7f23b8102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:12.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.314+0000 7f23bcade700 1 --2- 172.21.15.2:0/788165648 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23b8101e60 0x7f23b8102240 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f23a80099c0 tx=0x7f23a8009cd0 comp rx=0 tx=0).stop 2024-05-14T16:55:12.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.314+0000 7f23bcade700 1 -- 172.21.15.2:0/788165648 shutdown_connections 2024-05-14T16:55:12.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.314+0000 7f23bcade700 1 --2- 172.21.15.2:0/788165648 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23b8106ef0 0x7f23b810eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:12.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.314+0000 7f23bcade700 1 --2- 172.21.15.2:0/788165648 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23b8101e60 0x7f23b8102240 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:12.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.314+0000 7f23bcade700 1 --2- 172.21.15.2:0/788165648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23b8102810 0x7f23b8106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:12.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.314+0000 7f23bcade700 1 -- 172.21.15.2:0/788165648 >> 172.21.15.2:0/788165648 conn(0x7f23b8078ab0 msgr2=0x7f23b80ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:12.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.314+0000 7f23bcade700 1 -- 172.21.15.2:0/788165648 shutdown_connections 2024-05-14T16:55:12.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.314+0000 7f23bcade700 1 -- 172.21.15.2:0/788165648 wait complete. 2024-05-14T16:55:12.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.315+0000 7f23bcade700 1 Processor -- start 2024-05-14T16:55:12.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.315+0000 7f23bcade700 1 -- start start 2024-05-14T16:55:12.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.316+0000 7f23bcade700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23b8101e60 0x7f23b80fdf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:12.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.316+0000 7f23bcade700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23b8102810 0x7f23b80fe460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:12.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.316+0000 7f23bcade700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23b8106ef0 0x7f23b80fe9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:12.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.316+0000 7f23bcade700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f23b8111e50 con 0x7f23b8106ef0 2024-05-14T16:55:12.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.316+0000 7f23bcade700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f23b8111cd0 con 0x7f23b8102810 2024-05-14T16:55:12.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.316+0000 7f23bcade700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f23b8111fd0 con 0x7f23b8101e60 2024-05-14T16:55:12.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.316+0000 7f23b6d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23b8106ef0 0x7f23b80fe9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:12.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.316+0000 7f23b6d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23b8106ef0 0x7f23b80fe9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46220/0 (socket says 172.21.15.2:46220) 2024-05-14T16:55:12.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.316+0000 7f23b6d9d700 1 -- 172.21.15.2:0/1384035663 learned_addr learned my addr 172.21.15.2:0/1384035663 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:12.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.316+0000 7f23b659c700 1 --2- 172.21.15.2:0/1384035663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23b8101e60 0x7f23b80fdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:12.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.316+0000 7f23b5d9b700 1 --2- 172.21.15.2:0/1384035663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23b8102810 0x7f23b80fe460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:12.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.316+0000 7f23b6d9d700 1 -- 172.21.15.2:0/1384035663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23b8101e60 msgr2=0x7f23b80fdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:12.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.316+0000 7f23b6d9d700 1 --2- 172.21.15.2:0/1384035663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23b8101e60 0x7f23b80fdf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:12.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.316+0000 7f23b6d9d700 1 -- 172.21.15.2:0/1384035663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23b8102810 msgr2=0x7f23b80fe460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:12.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.316+0000 7f23b6d9d700 1 --2- 172.21.15.2:0/1384035663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23b8102810 0x7f23b80fe460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:12.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.316+0000 7f23b6d9d700 1 -- 172.21.15.2:0/1384035663 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23b81a81a0 con 0x7f23b8106ef0 2024-05-14T16:55:12.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.316+0000 7f23b659c700 1 --2- 172.21.15.2:0/1384035663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23b8101e60 0x7f23b80fdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:55:12.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.316+0000 7f23b6d9d700 1 --2- 172.21.15.2:0/1384035663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23b8106ef0 0x7f23b80fe9a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f23b000b970 tx=0x7f23b000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:12.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.317+0000 7f23a77fe700 1 -- 172.21.15.2:0/1384035663 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b000d860 con 0x7f23b8106ef0 2024-05-14T16:55:12.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.317+0000 7f23bcade700 1 -- 172.21.15.2:0/1384035663 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23b81a8490 con 0x7f23b8106ef0 2024-05-14T16:55:12.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.317+0000 7f23bcade700 1 -- 172.21.15.2:0/1384035663 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23b81a8a20 con 0x7f23b8106ef0 2024-05-14T16:55:12.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.317+0000 7f23a77fe700 1 -- 172.21.15.2:0/1384035663 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23b000e430 con 0x7f23b8106ef0 2024-05-14T16:55:12.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.317+0000 7f23a77fe700 1 -- 172.21.15.2:0/1384035663 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b0017660 con 0x7f23b8106ef0 2024-05-14T16:55:12.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.320+0000 7f23a77fe700 1 -- 172.21.15.2:0/1384035663 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) ==== 149884+0+0 (secure 0 0 0) 0x7f23b00178c0 con 0x7f23b8106ef0 2024-05-14T16:55:12.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.321+0000 7f23a77fe700 1 --2- 172.21.15.2:0/1384035663 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f23a00b2b90 0x7f23a00b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:12.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.321+0000 7f23c7374b80 1 Processor -- start 2024-05-14T16:55:12.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.321+0000 7f23b659c700 1 --2- 172.21.15.2:0/1384035663 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f23a00b2b90 0x7f23a00b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:12.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.321+0000 7f23a77fe700 1 -- 172.21.15.2:0/1384035663 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f23b00e2470 con 0x7f23b8106ef0 2024-05-14T16:55:12.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.322+0000 7f23c7374b80 1 -- start start 2024-05-14T16:55:12.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.322+0000 7f23c7374b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef28d392f0 0x55ef28d396d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:12.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.322+0000 7f23c7374b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef28d39c10 0x55ef28d3d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:12.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.322+0000 7f23c7374b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef28d3d910 0x55ef28d3dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:12.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.322+0000 7f23c7374b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ef28c97c40 con 0x55ef28d392f0 2024-05-14T16:55:12.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.322+0000 7f23c7374b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ef28c877e0 con 0x55ef28d3d910 2024-05-14T16:55:12.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.322+0000 7f23c7374b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ef28c8b820 con 0x55ef28d39c10 2024-05-14T16:55:12.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.322+0000 7f23b659c700 1 --2- 172.21.15.2:0/1384035663 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f23a00b2b90 0x7f23a00b5050 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f23a800b580 tx=0x7f23a80058e0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:12.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.322+0000 7f23b659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef28d392f0 0x55ef28d396d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:12.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.322+0000 7f23b6d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef28d3d910 0x55ef28d3dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:12.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.322+0000 7f23b659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef28d392f0 0x55ef28d396d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46224/0 (socket says 172.21.15.2:46224) 2024-05-14T16:55:12.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.322+0000 7f23b659c700 1 -- 172.21.15.2:0/761856053 learned_addr learned my addr 172.21.15.2:0/761856053 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:12.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.322+0000 7f23b5d9b700 1 --2- 172.21.15.2:0/761856053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef28d39c10 0x55ef28d3d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:12.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.322+0000 7f23b6d9d700 1 -- 172.21.15.2:0/761856053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef28d39c10 msgr2=0x55ef28d3d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:12.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.322+0000 7f23b6d9d700 1 --2- 172.21.15.2:0/761856053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef28d39c10 0x55ef28d3d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:12.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.322+0000 7f23b6d9d700 1 -- 172.21.15.2:0/761856053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef28d392f0 msgr2=0x55ef28d396d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:12.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.322+0000 7f23b6d9d700 1 --2- 172.21.15.2:0/761856053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef28d392f0 0x55ef28d396d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:12.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.322+0000 7f23b6d9d700 1 -- 172.21.15.2:0/761856053 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef28bd7880 con 0x55ef28d3d910 2024-05-14T16:55:12.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.323+0000 7f23b6d9d700 1 --2- 172.21.15.2:0/761856053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef28d3d910 0x55ef28d3dd50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f23b00dfa10 tx=0x7f23b00a9060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=daecad5be523a1a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:12.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.323+0000 7f23a4ff9700 1 -- 172.21.15.2:0/761856053 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b00a94d0 con 0x55ef28d3d910 2024-05-14T16:55:12.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.323+0000 7f23a4ff9700 1 -- 172.21.15.2:0/761856053 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23b00ad880 con 0x55ef28d3d910 2024-05-14T16:55:12.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.323+0000 7f23a4ff9700 1 -- 172.21.15.2:0/761856053 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b00b2100 con 0x55ef28d3d910 2024-05-14T16:55:12.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.324+0000 7f23c7374b80 1 -- 172.21.15.2:0/761856053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef28d3d910 msgr2=0x55ef28d3dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:12.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.324+0000 7f23c7374b80 1 --2- 172.21.15.2:0/761856053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef28d3d910 0x55ef28d3dd50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f23b00dfa10 tx=0x7f23b00a9060 comp rx=0 tx=0).stop 2024-05-14T16:55:12.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.324+0000 7f23c7374b80 1 -- 172.21.15.2:0/761856053 shutdown_connections 2024-05-14T16:55:12.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.324+0000 7f23c7374b80 1 --2- 172.21.15.2:0/761856053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef28d3d910 0x55ef28d3dd50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:12.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.324+0000 7f23c7374b80 1 --2- 172.21.15.2:0/761856053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef28d392f0 0x55ef28d396d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:12.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.324+0000 7f23c7374b80 1 --2- 172.21.15.2:0/761856053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef28d39c10 0x55ef28d3d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:12.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.324+0000 7f23c7374b80 1 -- 172.21.15.2:0/761856053 >> 172.21.15.2:0/761856053 conn(0x55ef28c64200 msgr2=0x55ef28d33930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:12.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.324+0000 7f23c7374b80 1 -- 172.21.15.2:0/761856053 shutdown_connections 2024-05-14T16:55:12.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.324+0000 7f23c7374b80 1 -- 172.21.15.2:0/761856053 wait complete. 2024-05-14T16:55:12.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.324+0000 7f23c7374b80 1 -- start start 2024-05-14T16:55:12.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.325+0000 7f23c7374b80 1 Processor -- start 2024-05-14T16:55:12.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.325+0000 7f23c7374b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef28d392f0 0x55ef28d44d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:12.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.325+0000 7f23c7374b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef28d39c10 0x55ef28d42130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:12.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.325+0000 7f23c7374b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef28d3d910 0x55ef28d42670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:12.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.325+0000 7f23c7374b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ef28b8df50 con 0x55ef28d39c10 2024-05-14T16:55:12.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.325+0000 7f23c7374b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ef28c29770 con 0x55ef28d392f0 2024-05-14T16:55:12.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.325+0000 7f23c7374b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ef28d45b70 con 0x55ef28d3d910 2024-05-14T16:55:12.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.325+0000 7f23c7374b80 10 client.0 _ll_register_callbacks cb 0x55ef28c1e530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:55:12.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.325+0000 7f23b5d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef28d39c10 0x55ef28d42130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:12.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.325+0000 7f23b5d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef28d39c10 0x55ef28d42130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46230/0 (socket says 172.21.15.2:46230) 2024-05-14T16:55:12.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.325+0000 7f23b5d9b700 1 -- 172.21.15.2:0/3468776006 learned_addr learned my addr 172.21.15.2:0/3468776006 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:12.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.325+0000 7f23b659c700 1 --2- 172.21.15.2:0/3468776006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef28d392f0 0x55ef28d44d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:12.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.325+0000 7f23b6d9d700 1 --2- 172.21.15.2:0/3468776006 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef28d3d910 0x55ef28d42670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:12.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.325+0000 7f23b5d9b700 1 -- 172.21.15.2:0/3468776006 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef28d3d910 msgr2=0x55ef28d42670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:12.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.325+0000 7f23b5d9b700 1 --2- 172.21.15.2:0/3468776006 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef28d3d910 0x55ef28d42670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:12.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.325+0000 7f23b5d9b700 1 -- 172.21.15.2:0/3468776006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef28d392f0 msgr2=0x55ef28d44d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:12.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.325+0000 7f23b5d9b700 1 --2- 172.21.15.2:0/3468776006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef28d392f0 0x55ef28d44d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:12.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.325+0000 7f23b5d9b700 1 -- 172.21.15.2:0/3468776006 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef28c26640 con 0x55ef28d39c10 2024-05-14T16:55:12.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.325+0000 7f23b6d9d700 1 --2- 172.21.15.2:0/3468776006 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef28d3d910 0x55ef28d42670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:55:12.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.325+0000 7f23b659c700 1 --2- 172.21.15.2:0/3468776006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef28d392f0 0x55ef28d44d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:55:12.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.326+0000 7f23b5d9b700 1 --2- 172.21.15.2:0/3468776006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef28d39c10 0x55ef28d42130 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f23ac0107b0 tx=0x7f23ac010b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff5271a87e01ec3d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:12.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.326+0000 7f239dffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:55:12.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.326+0000 7f239dffb700 1 -- 172.21.15.2:0/3468776006 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ac004320 con 0x55ef28d39c10 2024-05-14T16:55:12.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.326+0000 7f239dffb700 1 -- 172.21.15.2:0/3468776006 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f23ac0044c0 con 0x55ef28d39c10 2024-05-14T16:55:12.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.326+0000 7f23c7374b80 1 -- 172.21.15.2:0/3468776006 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23b00ad880 con 0x55ef28d39c10 2024-05-14T16:55:12.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.326+0000 7f239dffb700 1 -- 172.21.15.2:0/3468776006 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f23ac003680 con 0x55ef28d39c10 2024-05-14T16:55:12.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.326+0000 7f23c7374b80 10 client.15333 fetch_fsmap learned FSMap version 25 2024-05-14T16:55:12.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.326+0000 7f23c7374b80 1 -- 172.21.15.2:0/3468776006 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ef28d50290 con 0x55ef28d39c10 2024-05-14T16:55:12.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.326+0000 7f239dffb700 1 -- 172.21.15.2:0/3468776006 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23b00ad880 con 0x55ef28d39c10 2024-05-14T16:55:12.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.326+0000 7f239dffb700 1 -- 172.21.15.2:0/3468776006 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f23ac00fbe0 con 0x55ef28d39c10 2024-05-14T16:55:12.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.327+0000 7f23c7374b80 10 client.15333 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T16:55:12.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.327+0000 7f23c7374b80 10 client.15333 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-14T16:55:12.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.327+0000 7f23c7374b80 20 client.15333 populate_metadata read hostname 'smithi002' 2024-05-14T16:55:12.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.327+0000 7f23c7374b80 1 --2- 172.21.15.2:0/3468776006 >> [v2:172.21.15.123:6836/333426645,v1:172.21.15.123:6837/333426645] conn(0x55ef28d52930 0x55ef28d53da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:12.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.327+0000 7f23c7374b80 4 client.15333 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:55:12.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.327+0000 7f23c7374b80 1 -- 172.21.15.2:0/3468776006 --> [v2:172.21.15.123:6836/333426645,v1:172.21.15.123:6837/333426645] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef28d05310 con 0x55ef28d52930 2024-05-14T16:55:12.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.327+0000 7f23b659c700 1 --2- 172.21.15.2:0/3468776006 >> [v2:172.21.15.123:6836/333426645,v1:172.21.15.123:6837/333426645] conn(0x55ef28d52930 0x55ef28d53da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:12.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.327+0000 7f239dffb700 10 client.15333 ms_handle_connect on v2:172.21.15.123:6836/333426645 2024-05-14T16:55:12.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.327+0000 7f23b659c700 1 --2- 172.21.15.2:0/3468776006 >> [v2:172.21.15.123:6836/333426645,v1:172.21.15.123:6837/333426645] conn(0x55ef28d52930 0x55ef28d53da0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=feb2e82aac88f4d3 server_cookie=f86641bb2135c3a2 in_seq=0 out_seq=0 2024-05-14T16:55:12.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.328+0000 7f239dffb700 1 -- 172.21.15.2:0/3468776006 <== mds.1 v2:172.21.15.123:6836/333426645 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ef28d05310 con 0x55ef28d52930 2024-05-14T16:55:12.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.328+0000 7f239dffb700 10 client.15333 handle_command_reply: tid=0 2024-05-14T16:55:12.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.328+0000 7f23c7374b80 1 client.15333 shutdown 2024-05-14T16:55:12.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.328+0000 7f23c7374b80 1 -- 172.21.15.2:0/3468776006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef28d39c10 msgr2=0x55ef28d42130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:12.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.328+0000 7f23c7374b80 1 --2- 172.21.15.2:0/3468776006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef28d39c10 0x55ef28d42130 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f23ac0107b0 tx=0x7f23ac010b70 comp rx=0 tx=0).stop 2024-05-14T16:55:12.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.329+0000 7f23c7374b80 1 -- 172.21.15.2:0/3468776006 shutdown_connections 2024-05-14T16:55:12.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.329+0000 7f23c7374b80 1 --2- 172.21.15.2:0/3468776006 >> [v2:172.21.15.123:6836/333426645,v1:172.21.15.123:6837/333426645] conn(0x55ef28d52930 0x55ef28d53da0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:12.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.329+0000 7f23c7374b80 1 --2- 172.21.15.2:0/3468776006 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef28d3d910 0x55ef28d42670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:12.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.329+0000 7f23c7374b80 1 --2- 172.21.15.2:0/3468776006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef28d392f0 0x55ef28d44d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:12.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.329+0000 7f23c7374b80 1 --2- 172.21.15.2:0/3468776006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef28d39c10 0x55ef28d42130 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:12.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.329+0000 7f23c7374b80 1 -- 172.21.15.2:0/3468776006 >> 172.21.15.2:0/3468776006 conn(0x55ef28c64200 msgr2=0x55ef28d3ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:12.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.329+0000 7f239dffb700 0 client.15333 ms_handle_reset on v2:172.21.15.123:6836/333426645 2024-05-14T16:55:12.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.329+0000 7f23c7374b80 1 -- 172.21.15.2:0/3468776006 shutdown_connections 2024-05-14T16:55:12.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.329+0000 7f23c7374b80 1 -- 172.21.15.2:0/3468776006 wait complete. 2024-05-14T16:55:12.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.329+0000 7f23c7374b80 20 client.15333 trim_cache size 0 max 16384 2024-05-14T16:55:12.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.351+0000 7f23c7374b80 1 Processor -- start 2024-05-14T16:55:12.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.351+0000 7f23c7374b80 1 -- start start 2024-05-14T16:55:12.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.351+0000 7f23c7374b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef28d392f0 0x55ef28d2dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:12.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.351+0000 7f23c7374b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef28d39c10 0x55ef28d3b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:12.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.351+0000 7f23c7374b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef28d3d910 0x55ef28d3b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:12.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.351+0000 7f23c7374b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ef28d520e0 con 0x55ef28d39c10 2024-05-14T16:55:12.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.351+0000 7f23c7374b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ef28d527b0 con 0x55ef28d392f0 2024-05-14T16:55:12.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.351+0000 7f23c7374b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ef28d52e80 con 0x55ef28d3d910 2024-05-14T16:55:12.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.351+0000 7f23b5d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef28d39c10 0x55ef28d3b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:12.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.351+0000 7f23b5d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef28d39c10 0x55ef28d3b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46240/0 (socket says 172.21.15.2:46240) 2024-05-14T16:55:12.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.351+0000 7f23b5d9b700 1 -- 172.21.15.2:0/792882209 learned_addr learned my addr 172.21.15.2:0/792882209 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:12.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.351+0000 7f23b659c700 1 --2- 172.21.15.2:0/792882209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef28d392f0 0x55ef28d2dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:12.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.351+0000 7f23b6d9d700 1 --2- 172.21.15.2:0/792882209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef28d3d910 0x55ef28d3b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:12.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.352+0000 7f23b5d9b700 1 -- 172.21.15.2:0/792882209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef28d3d910 msgr2=0x55ef28d3b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:12.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.352+0000 7f23b5d9b700 1 --2- 172.21.15.2:0/792882209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef28d3d910 0x55ef28d3b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:12.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.352+0000 7f23b5d9b700 1 -- 172.21.15.2:0/792882209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef28d392f0 msgr2=0x55ef28d2dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:12.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.352+0000 7f23b5d9b700 1 --2- 172.21.15.2:0/792882209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef28d392f0 0x55ef28d2dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:12.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.352+0000 7f23b5d9b700 1 -- 172.21.15.2:0/792882209 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef28d406a0 con 0x55ef28d39c10 2024-05-14T16:55:12.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.352+0000 7f23b5d9b700 1 --2- 172.21.15.2:0/792882209 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef28d39c10 0x55ef28d3b340 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f23ac004020 tx=0x7f23ac00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5732b11025440050 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:12.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.352+0000 7f239ffff700 1 -- 172.21.15.2:0/792882209 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ac027040 con 0x55ef28d39c10 2024-05-14T16:55:12.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.352+0000 7f239ffff700 1 -- 172.21.15.2:0/792882209 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23ac00a3f0 con 0x55ef28d39c10 2024-05-14T16:55:12.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.352+0000 7f239ffff700 1 -- 172.21.15.2:0/792882209 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ac00fa60 con 0x55ef28d39c10 2024-05-14T16:55:12.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.353+0000 7f23c7374b80 1 -- 172.21.15.2:0/792882209 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef28d39c10 msgr2=0x55ef28d3b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:12.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.353+0000 7f23c7374b80 1 --2- 172.21.15.2:0/792882209 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef28d39c10 0x55ef28d3b340 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f23ac004020 tx=0x7f23ac00ee70 comp rx=0 tx=0).stop 2024-05-14T16:55:12.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.353+0000 7f23c7374b80 1 -- 172.21.15.2:0/792882209 shutdown_connections 2024-05-14T16:55:12.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.353+0000 7f23c7374b80 1 --2- 172.21.15.2:0/792882209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef28d3d910 0x55ef28d3b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:12.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.353+0000 7f23c7374b80 1 --2- 172.21.15.2:0/792882209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef28d392f0 0x55ef28d2dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:12.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.353+0000 7f23c7374b80 1 --2- 172.21.15.2:0/792882209 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef28d39c10 0x55ef28d3b340 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:12.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.353+0000 7f23c7374b80 1 -- 172.21.15.2:0/792882209 >> 172.21.15.2:0/792882209 conn(0x55ef28c64200 msgr2=0x55ef28d2d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:12.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.353+0000 7f23c7374b80 1 -- 172.21.15.2:0/792882209 shutdown_connections 2024-05-14T16:55:12.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.353+0000 7f23c7374b80 1 -- 172.21.15.2:0/792882209 wait complete. 2024-05-14T16:55:12.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.353+0000 7f23c7374b80 1 -- start start 2024-05-14T16:55:12.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.353+0000 7f23c7374b80 1 Processor -- start 2024-05-14T16:55:12.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.354+0000 7f23c7374b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef28d392f0 0x55ef28d5d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:12.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.354+0000 7f23c7374b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef28d39c10 0x55ef28d581d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:12.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.354+0000 7f23c7374b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef28d3d910 0x55ef28d58710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:12.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.354+0000 7f23c7374b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ef28d53c20 con 0x55ef28d392f0 2024-05-14T16:55:12.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.354+0000 7f23c7374b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ef28d53550 con 0x55ef28d3d910 2024-05-14T16:55:12.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.354+0000 7f23c7374b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ef28d542f0 con 0x55ef28d39c10 2024-05-14T16:55:12.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.354+0000 7f23c7374b80 10 client.0 _ll_register_callbacks cb 0x55ef28c1e530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:55:12.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.354+0000 7f23b659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef28d392f0 0x55ef28d5d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:12.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.354+0000 7f23b5d9b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef28d39c10 0x55ef28d581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:12.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.354+0000 7f23b5d9b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef28d39c10 0x55ef28d581d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:43708/0 (socket says 172.21.15.2:43708) 2024-05-14T16:55:12.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.354+0000 7f23b659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef28d392f0 0x55ef28d5d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46252/0 (socket says 172.21.15.2:46252) 2024-05-14T16:55:12.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.354+0000 7f23b5d9b700 1 -- 172.21.15.2:0/367832781 learned_addr learned my addr 172.21.15.2:0/367832781 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:12.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.354+0000 7f23b6d9d700 1 --2- 172.21.15.2:0/367832781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef28d3d910 0x55ef28d58710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:12.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.354+0000 7f23b5d9b700 1 -- 172.21.15.2:0/367832781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef28d3d910 msgr2=0x55ef28d58710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:12.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.354+0000 7f23b5d9b700 1 --2- 172.21.15.2:0/367832781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef28d3d910 0x55ef28d58710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:12.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.354+0000 7f23b5d9b700 1 -- 172.21.15.2:0/367832781 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef28d392f0 msgr2=0x55ef28d5d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:12.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.354+0000 7f23b5d9b700 1 --2- 172.21.15.2:0/367832781 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef28d392f0 0x55ef28d5d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:12.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.354+0000 7f23b5d9b700 1 -- 172.21.15.2:0/367832781 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef28d58c50 con 0x55ef28d39c10 2024-05-14T16:55:12.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.354+0000 7f23b659c700 1 --2- 172.21.15.2:0/367832781 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef28d392f0 0x55ef28d5d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:55:12.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.354+0000 7f23b5d9b700 1 --2- 172.21.15.2:0/367832781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef28d39c10 0x55ef28d581d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f23ac010bb0 tx=0x7f23ac005400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ead99a8b3d41fc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:12.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.354+0000 7f239dffb700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-14T16:55:12.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.355+0000 7f239dffb700 1 -- 172.21.15.2:0/367832781 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ac038020 con 0x55ef28d39c10 2024-05-14T16:55:12.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.355+0000 7f239dffb700 1 -- 172.21.15.2:0/367832781 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f23ac031040 con 0x55ef28d39c10 2024-05-14T16:55:12.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.355+0000 7f23c7374b80 1 -- 172.21.15.2:0/367832781 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23ac00a3f0 con 0x55ef28d39c10 2024-05-14T16:55:12.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.355+0000 7f239dffb700 1 -- 172.21.15.2:0/367832781 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f23ac043050 con 0x55ef28d39c10 2024-05-14T16:55:12.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.355+0000 7f23c7374b80 10 client.25072 fetch_fsmap learned FSMap version 25 2024-05-14T16:55:12.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.355+0000 7f23c7374b80 1 -- 172.21.15.2:0/367832781 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ef28dbea00 con 0x55ef28d39c10 2024-05-14T16:55:12.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.355+0000 7f239dffb700 1 -- 172.21.15.2:0/367832781 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23ac00a3f0 con 0x55ef28d39c10 2024-05-14T16:55:12.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.355+0000 7f239dffb700 1 -- 172.21.15.2:0/367832781 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f23ac01a610 con 0x55ef28d39c10 2024-05-14T16:55:12.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.355+0000 7f23c7374b80 10 client.25072 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T16:55:12.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.356+0000 7f23c7374b80 10 client.25072 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-14T16:55:12.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.356+0000 7f23c7374b80 20 client.25072 populate_metadata read hostname 'smithi002' 2024-05-14T16:55:12.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.356+0000 7f23c7374b80 1 --2- 172.21.15.2:0/367832781 >> [v2:172.21.15.123:6836/333426645,v1:172.21.15.123:6837/333426645] conn(0x55ef28dc3400 0x55ef28dc2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:12.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.356+0000 7f23c7374b80 4 client.25072 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T16:55:12.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.356+0000 7f23c7374b80 1 -- 172.21.15.2:0/367832781 --> [v2:172.21.15.123:6836/333426645,v1:172.21.15.123:6837/333426645] -- command(tid 0: {"prefix": "status"}) -- 0x55ef28c99b90 con 0x55ef28dc3400 2024-05-14T16:55:12.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.356+0000 7f23b659c700 1 --2- 172.21.15.2:0/367832781 >> [v2:172.21.15.123:6836/333426645,v1:172.21.15.123:6837/333426645] conn(0x55ef28dc3400 0x55ef28dc2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:12.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.356+0000 7f23b659c700 1 --2- 172.21.15.2:0/367832781 >> [v2:172.21.15.123:6836/333426645,v1:172.21.15.123:6837/333426645] conn(0x55ef28dc3400 0x55ef28dc2110 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=df8145dc42217cfd server_cookie=df838b84ec6ed934 in_seq=0 out_seq=0 2024-05-14T16:55:12.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.356+0000 7f239dffb700 10 client.25072 ms_handle_connect on v2:172.21.15.123:6836/333426645 2024-05-14T16:55:12.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.356+0000 7f239dffb700 1 -- 172.21.15.2:0/367832781 <== mds.1 v2:172.21.15.123:6836/333426645 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ef28c99b90 con 0x55ef28dc3400 2024-05-14T16:55:12.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.356+0000 7f239dffb700 10 client.25072 handle_command_reply: tid=0 2024-05-14T16:55:12.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.356+0000 7f23c7374b80 1 client.25072 shutdown 2024-05-14T16:55:12.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.357+0000 7f23c7374b80 1 -- 172.21.15.2:0/367832781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef28d39c10 msgr2=0x55ef28d581d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:12.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.357+0000 7f23c7374b80 1 --2- 172.21.15.2:0/367832781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef28d39c10 0x55ef28d581d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f23ac010bb0 tx=0x7f23ac005400 comp rx=0 tx=0).stop 2024-05-14T16:55:12.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.357+0000 7f23c7374b80 1 -- 172.21.15.2:0/367832781 shutdown_connections 2024-05-14T16:55:12.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.357+0000 7f23c7374b80 1 --2- 172.21.15.2:0/367832781 >> [v2:172.21.15.123:6836/333426645,v1:172.21.15.123:6837/333426645] conn(0x55ef28dc3400 0x55ef28dc2110 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:12.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.357+0000 7f239dffb700 0 client.25072 ms_handle_reset on v2:172.21.15.123:6836/333426645 2024-05-14T16:55:12.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.357+0000 7f23c7374b80 1 --2- 172.21.15.2:0/367832781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef28d3d910 0x55ef28d58710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:12.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.357+0000 7f23c7374b80 1 --2- 172.21.15.2:0/367832781 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef28d392f0 0x55ef28d5d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:12.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.357+0000 7f23c7374b80 1 --2- 172.21.15.2:0/367832781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef28d39c10 0x55ef28d581d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:12.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.357+0000 7f23c7374b80 1 -- 172.21.15.2:0/367832781 >> 172.21.15.2:0/367832781 conn(0x55ef28c64200 msgr2=0x55ef28d38100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:12.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.357+0000 7f23c7374b80 1 -- 172.21.15.2:0/367832781 shutdown_connections 2024-05-14T16:55:12.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.357+0000 7f23c7374b80 1 -- 172.21.15.2:0/367832781 wait complete. 2024-05-14T16:55:12.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.357+0000 7f23c7374b80 20 client.25072 trim_cache size 0 max 16384 2024-05-14T16:55:12.370 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T16:55:12.370 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "ed40717a-1210-11ef-bc98-c7b262605968", 2024-05-14T16:55:12.370 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 1, 2024-05-14T16:55:12.370 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24455, 2024-05-14T16:55:12.370 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2024-05-14T16:55:12.370 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2024-05-14T16:55:12.370 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2024-05-14T16:55:12.370 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 205.38493850899999, 2024-05-14T16:55:12.370 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 25, 2024-05-14T16:55:12.370 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 81, 2024-05-14T16:55:12.371 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 80, 2024-05-14T16:55:12.371 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 246.89772547199999 2024-05-14T16:55:12.371 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T16:55:12.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.357+0000 7f23bcade700 1 -- 172.21.15.2:0/1384035663 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f23a00b2b90 msgr2=0x7f23a00b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:12.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.357+0000 7f23bcade700 1 --2- 172.21.15.2:0/1384035663 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f23a00b2b90 0x7f23a00b5050 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f23a800b580 tx=0x7f23a80058e0 comp rx=0 tx=0).stop 2024-05-14T16:55:12.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.357+0000 7f23bcade700 1 -- 172.21.15.2:0/1384035663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23b8106ef0 msgr2=0x7f23b80fe9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:12.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.357+0000 7f23bcade700 1 --2- 172.21.15.2:0/1384035663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23b8106ef0 0x7f23b80fe9a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f23b000b970 tx=0x7f23b000bc80 comp rx=0 tx=0).stop 2024-05-14T16:55:12.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.358+0000 7f23bcade700 1 -- 172.21.15.2:0/1384035663 shutdown_connections 2024-05-14T16:55:12.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.358+0000 7f23bcade700 1 --2- 172.21.15.2:0/1384035663 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f23a00b2b90 0x7f23a00b5050 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:12.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.358+0000 7f23bcade700 1 --2- 172.21.15.2:0/1384035663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23b8106ef0 0x7f23b80fe9a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:12.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.358+0000 7f23bcade700 1 --2- 172.21.15.2:0/1384035663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23b8101e60 0x7f23b80fdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:12.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.358+0000 7f23bcade700 1 --2- 172.21.15.2:0/1384035663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23b8102810 0x7f23b80fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:12.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.358+0000 7f23bcade700 1 -- 172.21.15.2:0/1384035663 >> 172.21.15.2:0/1384035663 conn(0x7f23b8078ab0 msgr2=0x7f23b8104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:12.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.358+0000 7f23bcade700 1 -- 172.21.15.2:0/1384035663 shutdown_connections 2024-05-14T16:55:12.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:12.358+0000 7f23bcade700 1 -- 172.21.15.2:0/1384035663 wait complete. 2024-05-14T16:55:12.405 INFO:teuthology.orchestra.run.smithi002.stderr:1500000 of 3200000 tuples (46%) done (elapsed 1.92 s, remaining 2.18 s) 2024-05-14T16:55:12.555 INFO:teuthology.orchestra.run.smithi002.stderr:1600000 of 3200000 tuples (50%) done (elapsed 2.07 s, remaining 2.07 s) 2024-05-14T16:55:12.663 INFO:teuthology.orchestra.run.smithi002.stderr:1700000 of 3200000 tuples (53%) done (elapsed 2.18 s, remaining 1.92 s) 2024-05-14T16:55:12.683 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:12 smithi002 ceph-mon[31538]: pgmap v18: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 89 KiB/s wr, 23 op/s 2024-05-14T16:55:12.825 INFO:teuthology.orchestra.run.smithi002.stderr:1800000 of 3200000 tuples (56%) done (elapsed 2.34 s, remaining 1.82 s) 2024-05-14T16:55:12.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:12 smithi151 ceph-mon[36066]: pgmap v18: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 89 KiB/s wr, 23 op/s 2024-05-14T16:55:12.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:12 smithi123 ceph-mon[35971]: pgmap v18: 97 pgs: 97 active+clean; 2.0 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 89 KiB/s wr, 23 op/s 2024-05-14T16:55:12.958 INFO:teuthology.orchestra.run.smithi002.stderr:1900000 of 3200000 tuples (59%) done (elapsed 2.48 s, remaining 1.69 s) 2024-05-14T16:55:13.025 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph tell mds.b status 2024-05-14T16:55:13.059 INFO:teuthology.orchestra.run.smithi002.stderr:2000000 of 3200000 tuples (62%) done (elapsed 2.58 s, remaining 1.55 s) 2024-05-14T16:55:13.206 INFO:teuthology.orchestra.run.smithi002.stderr:2100000 of 3200000 tuples (65%) done (elapsed 2.72 s, remaining 1.43 s) 2024-05-14T16:55:13.303 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:55:13.395 INFO:teuthology.orchestra.run.smithi002.stderr:2200000 of 3200000 tuples (68%) done (elapsed 2.91 s, remaining 1.32 s) 2024-05-14T16:55:13.491 INFO:teuthology.orchestra.run.smithi002.stderr:2300000 of 3200000 tuples (71%) done (elapsed 3.01 s, remaining 1.18 s) 2024-05-14T16:55:13.625 INFO:teuthology.orchestra.run.smithi002.stderr:2400000 of 3200000 tuples (75%) done (elapsed 3.14 s, remaining 1.05 s) 2024-05-14T16:55:13.821 INFO:teuthology.orchestra.run.smithi002.stderr:2500000 of 3200000 tuples (78%) done (elapsed 3.34 s, remaining 0.93 s) 2024-05-14T16:55:13.886 INFO:teuthology.orchestra.run.smithi002.stderr:2600000 of 3200000 tuples (81%) done (elapsed 3.40 s, remaining 0.79 s) 2024-05-14T16:55:14.061 INFO:teuthology.orchestra.run.smithi002.stderr:2700000 of 3200000 tuples (84%) done (elapsed 3.58 s, remaining 0.66 s) 2024-05-14T16:55:14.213 INFO:teuthology.orchestra.run.smithi002.stderr:2800000 of 3200000 tuples (87%) done (elapsed 3.73 s, remaining 0.53 s) 2024-05-14T16:55:14.303 INFO:teuthology.orchestra.run.smithi002.stderr:2900000 of 3200000 tuples (90%) done (elapsed 3.82 s, remaining 0.40 s) 2024-05-14T16:55:14.483 INFO:teuthology.orchestra.run.smithi002.stderr:3000000 of 3200000 tuples (93%) done (elapsed 4.00 s, remaining 0.27 s) 2024-05-14T16:55:14.671 INFO:teuthology.orchestra.run.smithi002.stderr:3100000 of 3200000 tuples (96%) done (elapsed 4.19 s, remaining 0.14 s) 2024-05-14T16:55:14.730 INFO:teuthology.orchestra.run.smithi002.stderr:3200000 of 3200000 tuples (100%) done (elapsed 4.25 s, remaining 0.00 s) 2024-05-14T16:55:14.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:14 smithi002 ceph-mon[31538]: pgmap v19: 97 pgs: 97 active+clean; 173 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 26 MiB/s wr, 294 op/s 2024-05-14T16:55:14.773 INFO:teuthology.orchestra.run.smithi002.stderr:vacuuming... 2024-05-14T16:55:14.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:14 smithi151 ceph-mon[36066]: pgmap v19: 97 pgs: 97 active+clean; 173 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 26 MiB/s wr, 294 op/s 2024-05-14T16:55:14.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:14 smithi123 ceph-mon[35971]: pgmap v19: 97 pgs: 97 active+clean; 173 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 26 MiB/s wr, 294 op/s 2024-05-14T16:55:15.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.007+0000 7ff222598700 1 -- 172.21.15.2:0/3932110000 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff21c1095a0 msgr2=0x7ff21c109980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:15.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.007+0000 7ff222598700 1 --2- 172.21.15.2:0/3932110000 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff21c1095a0 0x7ff21c109980 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff20c009990 tx=0x7ff20c009ca0 comp rx=0 tx=0).stop 2024-05-14T16:55:15.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.007+0000 7ff222598700 1 -- 172.21.15.2:0/3932110000 shutdown_connections 2024-05-14T16:55:15.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.007+0000 7ff222598700 1 --2- 172.21.15.2:0/3932110000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff21c06c470 0x7ff21c1103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:15.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.007+0000 7ff222598700 1 --2- 172.21.15.2:0/3932110000 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff21c1095a0 0x7ff21c109980 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:15.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.007+0000 7ff222598700 1 --2- 172.21.15.2:0/3932110000 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff21c06bab0 0x7ff21c06bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:15.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.007+0000 7ff222598700 1 -- 172.21.15.2:0/3932110000 >> 172.21.15.2:0/3932110000 conn(0x7ff21c0793b0 msgr2=0x7ff21c0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:15.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.007+0000 7ff222598700 1 -- 172.21.15.2:0/3932110000 shutdown_connections 2024-05-14T16:55:15.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.007+0000 7ff222598700 1 -- 172.21.15.2:0/3932110000 wait complete. 2024-05-14T16:55:15.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.008+0000 7ff222598700 1 Processor -- start 2024-05-14T16:55:15.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.008+0000 7ff222598700 1 -- start start 2024-05-14T16:55:15.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.008+0000 7ff222598700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff21c06bab0 0x7ff21c102500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:15.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.008+0000 7ff222598700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff21c06c470 0x7ff21c102a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:15.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.008+0000 7ff222598700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff21c1095a0 0x7ff21c102f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:15.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.008+0000 7ff222598700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff21c1136b0 con 0x7ff21c06bab0 2024-05-14T16:55:15.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.008+0000 7ff222598700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff21c113530 con 0x7ff21c1095a0 2024-05-14T16:55:15.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.008+0000 7ff222598700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff21c113830 con 0x7ff21c06c470 2024-05-14T16:55:15.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.008+0000 7ff21bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff21c06bab0 0x7ff21c102500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:15.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.008+0000 7ff21bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff21c06bab0 0x7ff21c102500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33012/0 (socket says 172.21.15.2:33012) 2024-05-14T16:55:15.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.008+0000 7ff21bfff700 1 -- 172.21.15.2:0/358790512 learned_addr learned my addr 172.21.15.2:0/358790512 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:15.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.009+0000 7ff21b7fe700 1 --2- 172.21.15.2:0/358790512 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff21c06c470 0x7ff21c102a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:15.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.009+0000 7ff220b35700 1 --2- 172.21.15.2:0/358790512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff21c1095a0 0x7ff21c102f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:15.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.009+0000 7ff21bfff700 1 -- 172.21.15.2:0/358790512 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff21c06c470 msgr2=0x7ff21c102a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:15.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.009+0000 7ff21bfff700 1 --2- 172.21.15.2:0/358790512 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff21c06c470 0x7ff21c102a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:15.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.009+0000 7ff21bfff700 1 -- 172.21.15.2:0/358790512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff21c1095a0 msgr2=0x7ff21c102f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:15.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.009+0000 7ff21bfff700 1 --2- 172.21.15.2:0/358790512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff21c1095a0 0x7ff21c102f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:15.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.009+0000 7ff21bfff700 1 -- 172.21.15.2:0/358790512 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff21c1a8090 con 0x7ff21c06bab0 2024-05-14T16:55:15.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.009+0000 7ff21bfff700 1 --2- 172.21.15.2:0/358790512 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff21c06bab0 0x7ff21c102500 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff20c0057c0 tx=0x7ff20c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:15.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.009+0000 7ff2197fa700 1 -- 172.21.15.2:0/358790512 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20c027070 con 0x7ff21c06bab0 2024-05-14T16:55:15.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.009+0000 7ff2197fa700 1 -- 172.21.15.2:0/358790512 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff20c003f20 con 0x7ff21c06bab0 2024-05-14T16:55:15.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.009+0000 7ff222598700 1 -- 172.21.15.2:0/358790512 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff21c1a8320 con 0x7ff21c06bab0 2024-05-14T16:55:15.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.009+0000 7ff2197fa700 1 -- 172.21.15.2:0/358790512 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20c022b30 con 0x7ff21c06bab0 2024-05-14T16:55:15.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.009+0000 7ff222598700 1 -- 172.21.15.2:0/358790512 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff21c1a8800 con 0x7ff21c06bab0 2024-05-14T16:55:15.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.012+0000 7ff2197fa700 1 -- 172.21.15.2:0/358790512 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) ==== 149884+0+0 (secure 0 0 0) 0x7ff20c02a070 con 0x7ff21c06bab0 2024-05-14T16:55:15.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.013+0000 7ff22ce2eb80 1 Processor -- start 2024-05-14T16:55:15.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.013+0000 7ff2197fa700 1 --2- 172.21.15.2:0/358790512 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ff2040b2b10 0x7ff2040b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:15.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.013+0000 7ff22ce2eb80 1 -- start start 2024-05-14T16:55:15.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.014+0000 7ff2197fa700 1 -- 172.21.15.2:0/358790512 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7ff20c0e3810 con 0x7ff21c06bab0 2024-05-14T16:55:15.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.014+0000 7ff220b35700 1 --2- 172.21.15.2:0/358790512 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ff2040b2b10 0x7ff2040b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:15.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.014+0000 7ff22ce2eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55baad83cfa0 0x55baad83d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:15.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.014+0000 7ff22ce2eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baad83d8c0 0x55baad8377a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:15.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.014+0000 7ff22ce2eb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55baad837dc0 0x55baad838220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:15.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.014+0000 7ff22ce2eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55baad795d20 con 0x55baad83cfa0 2024-05-14T16:55:15.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.014+0000 7ff22ce2eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55baad7858c0 con 0x55baad83d8c0 2024-05-14T16:55:15.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.014+0000 7ff22ce2eb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55baad789900 con 0x55baad837dc0 2024-05-14T16:55:15.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.014+0000 7ff21b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baad83d8c0 0x55baad8377a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:15.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.014+0000 7ff21b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baad83d8c0 0x55baad8377a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:36952/0 (socket says 172.21.15.2:36952) 2024-05-14T16:55:15.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.014+0000 7ff21b7fe700 1 -- 172.21.15.2:0/3794480188 learned_addr learned my addr 172.21.15.2:0/3794480188 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:15.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.014+0000 7ff21bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55baad83cfa0 0x55baad83d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:15.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.014+0000 7ff220b35700 1 --2- 172.21.15.2:0/3794480188 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55baad837dc0 0x55baad838220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:15.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.015+0000 7ff220b35700 1 --2- 172.21.15.2:0/358790512 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ff2040b2b10 0x7ff2040b4fd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff214003ee0 tx=0x7ff214009100 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:15.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.015+0000 7ff21b7fe700 1 -- 172.21.15.2:0/3794480188 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55baad837dc0 msgr2=0x55baad838220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:15.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.015+0000 7ff21b7fe700 1 --2- 172.21.15.2:0/3794480188 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55baad837dc0 0x55baad838220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:15.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.015+0000 7ff21b7fe700 1 -- 172.21.15.2:0/3794480188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55baad83cfa0 msgr2=0x55baad83d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:15.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.015+0000 7ff21b7fe700 1 --2- 172.21.15.2:0/3794480188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55baad83cfa0 0x55baad83d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:15.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.015+0000 7ff21b7fe700 1 -- 172.21.15.2:0/3794480188 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55baad72ef60 con 0x55baad83d8c0 2024-05-14T16:55:15.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.015+0000 7ff21b7fe700 1 --2- 172.21.15.2:0/3794480188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baad83d8c0 0x55baad8377a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff21000ace0 tx=0x7ff21000f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f50e25b61366d61c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:15.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.015+0000 7ff2027fc700 1 -- 172.21.15.2:0/3794480188 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21000fe50 con 0x55baad83d8c0 2024-05-14T16:55:15.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.015+0000 7ff2027fc700 1 -- 172.21.15.2:0/3794480188 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2100044d0 con 0x55baad83d8c0 2024-05-14T16:55:15.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.015+0000 7ff2027fc700 1 -- 172.21.15.2:0/3794480188 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2100186a0 con 0x55baad83d8c0 2024-05-14T16:55:15.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.016+0000 7ff22ce2eb80 1 -- 172.21.15.2:0/3794480188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baad83d8c0 msgr2=0x55baad8377a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:15.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.016+0000 7ff22ce2eb80 1 --2- 172.21.15.2:0/3794480188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baad83d8c0 0x55baad8377a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff21000ace0 tx=0x7ff21000f3c0 comp rx=0 tx=0).stop 2024-05-14T16:55:15.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.016+0000 7ff22ce2eb80 1 -- 172.21.15.2:0/3794480188 shutdown_connections 2024-05-14T16:55:15.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.016+0000 7ff22ce2eb80 1 --2- 172.21.15.2:0/3794480188 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55baad837dc0 0x55baad838220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:15.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.016+0000 7ff22ce2eb80 1 --2- 172.21.15.2:0/3794480188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55baad83cfa0 0x55baad83d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:15.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.016+0000 7ff22ce2eb80 1 --2- 172.21.15.2:0/3794480188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baad83d8c0 0x55baad8377a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:15.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.016+0000 7ff22ce2eb80 1 -- 172.21.15.2:0/3794480188 >> 172.21.15.2:0/3794480188 conn(0x55baad764d60 msgr2=0x55baad831e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:15.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.016+0000 7ff22ce2eb80 1 -- 172.21.15.2:0/3794480188 shutdown_connections 2024-05-14T16:55:15.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.016+0000 7ff22ce2eb80 1 -- 172.21.15.2:0/3794480188 wait complete. 2024-05-14T16:55:15.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.016+0000 7ff22ce2eb80 1 -- start start 2024-05-14T16:55:15.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.017+0000 7ff22ce2eb80 1 Processor -- start 2024-05-14T16:55:15.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.017+0000 7ff22ce2eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55baad837dc0 0x55baad8435f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:15.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.017+0000 7ff22ce2eb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55baad83cfa0 0x55baad840990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:15.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.017+0000 7ff22ce2eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baad83d8c0 0x55baad840ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:15.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.017+0000 7ff22ce2eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55baad68bf50 con 0x55baad837dc0 2024-05-14T16:55:15.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.017+0000 7ff22ce2eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55baad6edd50 con 0x55baad83d8c0 2024-05-14T16:55:15.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.017+0000 7ff22ce2eb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55baad8442d0 con 0x55baad83cfa0 2024-05-14T16:55:15.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.017+0000 7ff22ce2eb80 10 client.0 _ll_register_callbacks cb 0x55baad720900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:55:15.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.017+0000 7ff21bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55baad837dc0 0x55baad8435f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:15.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.017+0000 7ff220b35700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baad83d8c0 0x55baad840ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:15.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.017+0000 7ff21bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55baad837dc0 0x55baad8435f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33028/0 (socket says 172.21.15.2:33028) 2024-05-14T16:55:15.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.017+0000 7ff21bfff700 1 -- 172.21.15.2:0/1513442780 learned_addr learned my addr 172.21.15.2:0/1513442780 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:15.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.018+0000 7ff220b35700 1 -- 172.21.15.2:0/1513442780 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55baad83cfa0 msgr2=0x55baad840990 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T16:55:15.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.018+0000 7ff220b35700 1 --2- 172.21.15.2:0/1513442780 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55baad83cfa0 0x55baad840990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:15.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.018+0000 7ff220b35700 1 -- 172.21.15.2:0/1513442780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55baad837dc0 msgr2=0x55baad8435f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:15.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.018+0000 7ff220b35700 1 --2- 172.21.15.2:0/1513442780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55baad837dc0 0x55baad8435f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:15.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.018+0000 7ff220b35700 1 -- 172.21.15.2:0/1513442780 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55baad6d5880 con 0x55baad83d8c0 2024-05-14T16:55:15.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.018+0000 7ff220b35700 1 --2- 172.21.15.2:0/1513442780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baad83d8c0 0x55baad840ed0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff214009f30 tx=0x7ff214004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e35f57038319ace0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:15.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.018+0000 7ff1fb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T16:55:15.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.018+0000 7ff1fb7fe700 1 -- 172.21.15.2:0/1513442780 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff214026070 con 0x55baad83d8c0 2024-05-14T16:55:15.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.018+0000 7ff1fb7fe700 1 -- 172.21.15.2:0/1513442780 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7ff214004db0 con 0x55baad83d8c0 2024-05-14T16:55:15.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.018+0000 7ff22ce2eb80 1 -- 172.21.15.2:0/1513442780 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2100044d0 con 0x55baad83d8c0 2024-05-14T16:55:15.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.018+0000 7ff1fb7fe700 1 -- 172.21.15.2:0/1513442780 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff214031050 con 0x55baad83d8c0 2024-05-14T16:55:15.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.018+0000 7ff22ce2eb80 10 client.25088 fetch_fsmap learned FSMap version 25 2024-05-14T16:55:15.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.019+0000 7ff22ce2eb80 1 -- 172.21.15.2:0/1513442780 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55baad73f710 con 0x55baad83d8c0 2024-05-14T16:55:15.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.019+0000 7ff1fb7fe700 1 -- 172.21.15.2:0/1513442780 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2100044d0 con 0x55baad83d8c0 2024-05-14T16:55:15.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.019+0000 7ff1fb7fe700 1 -- 172.21.15.2:0/1513442780 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7ff214015530 con 0x55baad83d8c0 2024-05-14T16:55:15.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.019+0000 7ff22ce2eb80 10 client.25088 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T16:55:15.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.019+0000 7ff22ce2eb80 10 client.25088 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T16:55:15.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.019+0000 7ff22ce2eb80 20 client.25088 populate_metadata read hostname 'smithi002' 2024-05-14T16:55:15.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.019+0000 7ff22ce2eb80 1 --2- 172.21.15.2:0/1513442780 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55baad850f40 0x55baad8523b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:15.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.019+0000 7ff22ce2eb80 4 client.25088 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:55:15.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.019+0000 7ff22ce2eb80 1 -- 172.21.15.2:0/1513442780 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55baad797c70 con 0x55baad850f40 2024-05-14T16:55:15.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.019+0000 7ff21bfff700 1 --2- 172.21.15.2:0/1513442780 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55baad850f40 0x55baad8523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:15.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.020+0000 7ff21bfff700 1 --2- 172.21.15.2:0/1513442780 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55baad850f40 0x55baad8523b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12c78e510b82eb54 server_cookie=8196031c38f68a8f in_seq=0 out_seq=0 2024-05-14T16:55:15.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.020+0000 7ff1fb7fe700 10 client.25088 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:55:15.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.020+0000 7ff1fb7fe700 1 -- 172.21.15.2:0/1513442780 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55baad797c70 con 0x55baad850f40 2024-05-14T16:55:15.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.020+0000 7ff1fb7fe700 10 client.25088 handle_command_reply: tid=0 2024-05-14T16:55:15.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.020+0000 7ff22ce2eb80 1 client.25088 shutdown 2024-05-14T16:55:15.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.021+0000 7ff22ce2eb80 1 -- 172.21.15.2:0/1513442780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baad83d8c0 msgr2=0x55baad840ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:15.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.021+0000 7ff22ce2eb80 1 --2- 172.21.15.2:0/1513442780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baad83d8c0 0x55baad840ed0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff214009f30 tx=0x7ff214004b40 comp rx=0 tx=0).stop 2024-05-14T16:55:15.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.021+0000 7ff22ce2eb80 1 -- 172.21.15.2:0/1513442780 shutdown_connections 2024-05-14T16:55:15.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.021+0000 7ff22ce2eb80 1 --2- 172.21.15.2:0/1513442780 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55baad850f40 0x55baad8523b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:15.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.021+0000 7ff22ce2eb80 1 --2- 172.21.15.2:0/1513442780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baad83d8c0 0x55baad840ed0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:15.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.021+0000 7ff22ce2eb80 1 --2- 172.21.15.2:0/1513442780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55baad837dc0 0x55baad8435f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:15.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.021+0000 7ff22ce2eb80 1 --2- 172.21.15.2:0/1513442780 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55baad83cfa0 0x55baad840990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:15.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.021+0000 7ff22ce2eb80 1 -- 172.21.15.2:0/1513442780 >> 172.21.15.2:0/1513442780 conn(0x55baad764d60 msgr2=0x55baad831630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:15.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.021+0000 7ff1fb7fe700 0 client.25088 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:55:15.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.021+0000 7ff22ce2eb80 1 -- 172.21.15.2:0/1513442780 shutdown_connections 2024-05-14T16:55:15.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.021+0000 7ff22ce2eb80 1 -- 172.21.15.2:0/1513442780 wait complete. 2024-05-14T16:55:15.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.021+0000 7ff22ce2eb80 20 client.25088 trim_cache size 0 max 16384 2024-05-14T16:55:15.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.042+0000 7ff22ce2eb80 1 Processor -- start 2024-05-14T16:55:15.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.042+0000 7ff22ce2eb80 1 -- start start 2024-05-14T16:55:15.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.043+0000 7ff22ce2eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baad837dc0 0x55baad8377a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:15.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.043+0000 7ff22ce2eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55baad83cfa0 0x55baad831db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:15.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.043+0000 7ff22ce2eb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55baad83d8c0 0x55baad85b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:15.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.043+0000 7ff22ce2eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55baad8506f0 con 0x55baad83cfa0 2024-05-14T16:55:15.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.043+0000 7ff22ce2eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55baad850dc0 con 0x55baad837dc0 2024-05-14T16:55:15.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.043+0000 7ff22ce2eb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55baad851490 con 0x55baad83d8c0 2024-05-14T16:55:15.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.043+0000 7ff21b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55baad83cfa0 0x55baad831db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:15.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.043+0000 7ff21bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baad837dc0 0x55baad8377a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:15.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.043+0000 7ff21b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55baad83cfa0 0x55baad831db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33042/0 (socket says 172.21.15.2:33042) 2024-05-14T16:55:15.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.043+0000 7ff21b7fe700 1 -- 172.21.15.2:0/2193770039 learned_addr learned my addr 172.21.15.2:0/2193770039 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:15.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.043+0000 7ff220b35700 1 --2- 172.21.15.2:0/2193770039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55baad83d8c0 0x55baad85b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:15.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.043+0000 7ff21bfff700 1 -- 172.21.15.2:0/2193770039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55baad83d8c0 msgr2=0x55baad85b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:15.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.043+0000 7ff21bfff700 1 --2- 172.21.15.2:0/2193770039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55baad83d8c0 0x55baad85b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:15.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.043+0000 7ff21bfff700 1 -- 172.21.15.2:0/2193770039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55baad83cfa0 msgr2=0x55baad831db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:15.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.043+0000 7ff21bfff700 1 --2- 172.21.15.2:0/2193770039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55baad83cfa0 0x55baad831db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:15.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.043+0000 7ff21bfff700 1 -- 172.21.15.2:0/2193770039 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55baad83b260 con 0x55baad837dc0 2024-05-14T16:55:15.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.043+0000 7ff21bfff700 1 --2- 172.21.15.2:0/2193770039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baad837dc0 0x55baad8377a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff20c005f50 tx=0x7ff20c0cd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6482ab206b588697 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:15.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.044+0000 7ff201ffb700 1 -- 172.21.15.2:0/2193770039 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20c0ab000 con 0x55baad837dc0 2024-05-14T16:55:15.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.044+0000 7ff201ffb700 1 -- 172.21.15.2:0/2193770039 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff20c0aa900 con 0x55baad837dc0 2024-05-14T16:55:15.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.044+0000 7ff201ffb700 1 -- 172.21.15.2:0/2193770039 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20c0aab60 con 0x55baad837dc0 2024-05-14T16:55:15.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.044+0000 7ff22ce2eb80 1 -- 172.21.15.2:0/2193770039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baad837dc0 msgr2=0x55baad8377a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:15.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.044+0000 7ff22ce2eb80 1 --2- 172.21.15.2:0/2193770039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baad837dc0 0x55baad8377a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff20c005f50 tx=0x7ff20c0cd040 comp rx=0 tx=0).stop 2024-05-14T16:55:15.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.044+0000 7ff22ce2eb80 1 -- 172.21.15.2:0/2193770039 shutdown_connections 2024-05-14T16:55:15.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.044+0000 7ff22ce2eb80 1 --2- 172.21.15.2:0/2193770039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55baad83d8c0 0x55baad85b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:15.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.044+0000 7ff22ce2eb80 1 --2- 172.21.15.2:0/2193770039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baad837dc0 0x55baad8377a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:15.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.044+0000 7ff22ce2eb80 1 --2- 172.21.15.2:0/2193770039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55baad83cfa0 0x55baad831db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:15.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.044+0000 7ff22ce2eb80 1 -- 172.21.15.2:0/2193770039 >> 172.21.15.2:0/2193770039 conn(0x55baad764d60 msgr2=0x55baad83f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:15.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.044+0000 7ff22ce2eb80 1 -- 172.21.15.2:0/2193770039 shutdown_connections 2024-05-14T16:55:15.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.045+0000 7ff22ce2eb80 1 -- 172.21.15.2:0/2193770039 wait complete. 2024-05-14T16:55:15.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.045+0000 7ff22ce2eb80 1 -- start start 2024-05-14T16:55:15.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.045+0000 7ff22ce2eb80 1 Processor -- start 2024-05-14T16:55:15.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.045+0000 7ff22ce2eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55baad837dc0 0x55baad85b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:15.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.045+0000 7ff22ce2eb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55baad83cfa0 0x55baad85bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:15.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.045+0000 7ff22ce2eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baad83d8c0 0x55baad8567e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:15.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.045+0000 7ff22ce2eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55baad852230 con 0x55baad837dc0 2024-05-14T16:55:15.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.045+0000 7ff22ce2eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55baad851b60 con 0x55baad83d8c0 2024-05-14T16:55:15.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.045+0000 7ff22ce2eb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55baad852900 con 0x55baad83cfa0 2024-05-14T16:55:15.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.045+0000 7ff22ce2eb80 10 client.0 _ll_register_callbacks cb 0x55baad720900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:55:15.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.046+0000 7ff21bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55baad837dc0 0x55baad85b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:15.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.046+0000 7ff21bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55baad837dc0 0x55baad85b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33048/0 (socket says 172.21.15.2:33048) 2024-05-14T16:55:15.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.046+0000 7ff21bfff700 1 -- 172.21.15.2:0/1876019485 learned_addr learned my addr 172.21.15.2:0/1876019485 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:15.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.046+0000 7ff21b7fe700 1 --2- 172.21.15.2:0/1876019485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55baad83cfa0 0x55baad85bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:15.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.046+0000 7ff220b35700 1 --2- 172.21.15.2:0/1876019485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baad83d8c0 0x55baad8567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:15.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.046+0000 7ff21bfff700 1 -- 172.21.15.2:0/1876019485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55baad83cfa0 msgr2=0x55baad85bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:15.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.046+0000 7ff21bfff700 1 --2- 172.21.15.2:0/1876019485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55baad83cfa0 0x55baad85bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:15.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.046+0000 7ff21bfff700 1 -- 172.21.15.2:0/1876019485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baad83d8c0 msgr2=0x55baad8567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:15.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.046+0000 7ff21bfff700 1 --2- 172.21.15.2:0/1876019485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baad83d8c0 0x55baad8567e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:15.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.046+0000 7ff21bfff700 1 -- 172.21.15.2:0/1876019485 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55baad856e20 con 0x55baad837dc0 2024-05-14T16:55:15.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.046+0000 7ff220b35700 1 --2- 172.21.15.2:0/1876019485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baad83d8c0 0x55baad8567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:55:15.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.046+0000 7ff21bfff700 1 --2- 172.21.15.2:0/1876019485 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55baad837dc0 0x55baad85b830 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff20c0b3010 tx=0x7ff20c0aa800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f864944b4509378 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:15.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.046+0000 7ff1fb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:55:15.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.046+0000 7ff1fb7fe700 1 -- 172.21.15.2:0/1876019485 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20c0b36d0 con 0x55baad837dc0 2024-05-14T16:55:15.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.046+0000 7ff22ce2eb80 1 -- 172.21.15.2:0/1876019485 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff20c0aa900 con 0x55baad837dc0 2024-05-14T16:55:15.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.046+0000 7ff1fb7fe700 1 -- 172.21.15.2:0/1876019485 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7ff20c0aab60 con 0x55baad837dc0 2024-05-14T16:55:15.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.047+0000 7ff1fb7fe700 1 -- 172.21.15.2:0/1876019485 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff20c0aad60 con 0x55baad837dc0 2024-05-14T16:55:15.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.047+0000 7ff22ce2eb80 10 client.15357 fetch_fsmap learned FSMap version 25 2024-05-14T16:55:15.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.047+0000 7ff22ce2eb80 1 -- 172.21.15.2:0/1876019485 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55baad8bccd0 con 0x55baad837dc0 2024-05-14T16:55:15.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.047+0000 7ff1fb7fe700 1 -- 172.21.15.2:0/1876019485 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff20c0aa900 con 0x55baad837dc0 2024-05-14T16:55:15.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.047+0000 7ff1fb7fe700 1 -- 172.21.15.2:0/1876019485 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7ff20c0e1090 con 0x55baad837dc0 2024-05-14T16:55:15.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.047+0000 7ff22ce2eb80 10 client.15357 fetch_fsmap finished waiting for FSMap version 25 2024-05-14T16:55:15.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.047+0000 7ff22ce2eb80 10 client.15357 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-14T16:55:15.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.047+0000 7ff22ce2eb80 20 client.15357 populate_metadata read hostname 'smithi002' 2024-05-14T16:55:15.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.047+0000 7ff22ce2eb80 1 --2- 172.21.15.2:0/1876019485 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55baad8c16e0 0x55baad8c03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:15.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.047+0000 7ff22ce2eb80 4 client.15357 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-14T16:55:15.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.047+0000 7ff22ce2eb80 1 -- 172.21.15.2:0/1876019485 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "status"}) -- 0x55baad791450 con 0x55baad8c16e0 2024-05-14T16:55:15.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.047+0000 7ff21b7fe700 1 --2- 172.21.15.2:0/1876019485 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55baad8c16e0 0x55baad8c03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:15.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.048+0000 7ff1fb7fe700 10 client.15357 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:55:15.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.048+0000 7ff21b7fe700 1 --2- 172.21.15.2:0/1876019485 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55baad8c16e0 0x55baad8c03f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=593c5d3f2f2bd4fd server_cookie=8fb0c57cfe2a6c11 in_seq=0 out_seq=0 2024-05-14T16:55:15.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.048+0000 7ff1fb7fe700 1 -- 172.21.15.2:0/1876019485 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55baad791450 con 0x55baad8c16e0 2024-05-14T16:55:15.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.048+0000 7ff1fb7fe700 10 client.15357 handle_command_reply: tid=0 2024-05-14T16:55:15.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.048+0000 7ff22ce2eb80 1 client.15357 shutdown 2024-05-14T16:55:15.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.048+0000 7ff22ce2eb80 1 -- 172.21.15.2:0/1876019485 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55baad837dc0 msgr2=0x55baad85b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:15.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.048+0000 7ff22ce2eb80 1 --2- 172.21.15.2:0/1876019485 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55baad837dc0 0x55baad85b830 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff20c0b3010 tx=0x7ff20c0aa800 comp rx=0 tx=0).stop 2024-05-14T16:55:15.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.048+0000 7ff22ce2eb80 1 -- 172.21.15.2:0/1876019485 shutdown_connections 2024-05-14T16:55:15.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.048+0000 7ff22ce2eb80 1 --2- 172.21.15.2:0/1876019485 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55baad8c16e0 0x55baad8c03f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:15.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.048+0000 7ff22ce2eb80 1 --2- 172.21.15.2:0/1876019485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baad83d8c0 0x55baad8567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:15.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.048+0000 7ff22ce2eb80 1 --2- 172.21.15.2:0/1876019485 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55baad837dc0 0x55baad85b830 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:15.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.048+0000 7ff22ce2eb80 1 --2- 172.21.15.2:0/1876019485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55baad83cfa0 0x55baad85bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:15.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.048+0000 7ff22ce2eb80 1 -- 172.21.15.2:0/1876019485 >> 172.21.15.2:0/1876019485 conn(0x55baad764d60 msgr2=0x55baad840770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:15.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.048+0000 7ff1fb7fe700 0 client.15357 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:55:15.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.048+0000 7ff22ce2eb80 1 -- 172.21.15.2:0/1876019485 shutdown_connections 2024-05-14T16:55:15.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.048+0000 7ff22ce2eb80 1 -- 172.21.15.2:0/1876019485 wait complete. 2024-05-14T16:55:15.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.049+0000 7ff22ce2eb80 20 client.15357 trim_cache size 0 max 16384 2024-05-14T16:55:15.053 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T16:55:15.053 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "ed40717a-1210-11ef-bc98-c7b262605968", 2024-05-14T16:55:15.053 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 0, 2024-05-14T16:55:15.053 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24479, 2024-05-14T16:55:15.053 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2024-05-14T16:55:15.053 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2024-05-14T16:55:15.053 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2024-05-14T16:55:15.054 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 220.757321429, 2024-05-14T16:55:15.054 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 25, 2024-05-14T16:55:15.054 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 81, 2024-05-14T16:55:15.054 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 80, 2024-05-14T16:55:15.054 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 236.608527353 2024-05-14T16:55:15.054 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T16:55:15.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.049+0000 7ff222598700 1 -- 172.21.15.2:0/358790512 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ff2040b2b10 msgr2=0x7ff2040b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:15.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.049+0000 7ff222598700 1 --2- 172.21.15.2:0/358790512 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ff2040b2b10 0x7ff2040b4fd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff214003ee0 tx=0x7ff214009100 comp rx=0 tx=0).stop 2024-05-14T16:55:15.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.049+0000 7ff222598700 1 -- 172.21.15.2:0/358790512 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff21c06bab0 msgr2=0x7ff21c102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:15.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.049+0000 7ff222598700 1 --2- 172.21.15.2:0/358790512 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff21c06bab0 0x7ff21c102500 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff20c0057c0 tx=0x7ff20c0057f0 comp rx=0 tx=0).stop 2024-05-14T16:55:15.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.049+0000 7ff222598700 1 -- 172.21.15.2:0/358790512 shutdown_connections 2024-05-14T16:55:15.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.049+0000 7ff222598700 1 --2- 172.21.15.2:0/358790512 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ff2040b2b10 0x7ff2040b4fd0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:15.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.049+0000 7ff222598700 1 --2- 172.21.15.2:0/358790512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff21c1095a0 0x7ff21c102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:15.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.049+0000 7ff222598700 1 --2- 172.21.15.2:0/358790512 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff21c06bab0 0x7ff21c102500 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:15.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.049+0000 7ff222598700 1 --2- 172.21.15.2:0/358790512 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff21c06c470 0x7ff21c102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:15.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.049+0000 7ff222598700 1 -- 172.21.15.2:0/358790512 >> 172.21.15.2:0/358790512 conn(0x7ff21c0793b0 msgr2=0x7ff21c1010e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:15.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.049+0000 7ff222598700 1 -- 172.21.15.2:0/358790512 shutdown_connections 2024-05-14T16:55:15.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:15.049+0000 7ff222598700 1 -- 172.21.15.2:0/358790512 wait complete. 2024-05-14T16:55:15.430 INFO:teuthology.orchestra.run.smithi002.stderr:creating primary keys... 2024-05-14T16:55:15.923 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-05-14T16:55:16.674 INFO:teuthology.orchestra.run.smithi002.stderr:done in 6.23 s (drop tables 0.00 s, create tables 0.01 s, client-side generate 4.31 s, vacuum 0.66 s, primary keys 1.24 s). 2024-05-14T16:55:16.678 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2024-05-14T16:55:16.746 INFO:teuthology.orchestra.run.smithi002.stderr:starting vacuum...end. 2024-05-14T16:55:16.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:16 smithi151 ceph-mon[36066]: pgmap v20: 97 pgs: 97 active+clean; 173 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 26 MiB/s wr, 276 op/s 2024-05-14T16:55:16.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:16 smithi123 ceph-mon[35971]: pgmap v20: 97 pgs: 97 active+clean; 173 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 26 MiB/s wr, 276 op/s 2024-05-14T16:55:16.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:16 smithi002 ceph-mon[31538]: pgmap v20: 97 pgs: 97 active+clean; 173 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 26 MiB/s wr, 276 op/s 2024-05-14T16:55:18.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:18 smithi151 ceph-mon[36066]: pgmap v21: 97 pgs: 97 active+clean; 430 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 63 MiB/s wr, 314 op/s 2024-05-14T16:55:18.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:18 smithi123 ceph-mon[35971]: pgmap v21: 97 pgs: 97 active+clean; 430 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 63 MiB/s wr, 314 op/s 2024-05-14T16:55:18.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:18 smithi002 ceph-mon[31538]: pgmap v21: 97 pgs: 97 active+clean; 430 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 63 MiB/s wr, 314 op/s 2024-05-14T16:55:19.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:19 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:55:19.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:19 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:55:19.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:19 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:55:20.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:20 smithi151 ceph-mon[36066]: pgmap v22: 97 pgs: 97 active+clean; 498 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 74 MiB/s wr, 317 op/s 2024-05-14T16:55:20.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:20 smithi123 ceph-mon[35971]: pgmap v22: 97 pgs: 97 active+clean; 498 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 74 MiB/s wr, 317 op/s 2024-05-14T16:55:20.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:20 smithi002 ceph-mon[31538]: pgmap v22: 97 pgs: 97 active+clean; 498 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 74 MiB/s wr, 317 op/s 2024-05-14T16:55:21.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 5.0 s, 3814.4 tps, lat 25.736 ms stddev 26.222 2024-05-14T16:55:22.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:22 smithi151 ceph-mon[36066]: pgmap v23: 97 pgs: 97 active+clean; 498 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 74 MiB/s wr, 312 op/s 2024-05-14T16:55:22.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:22 smithi123 ceph-mon[35971]: pgmap v23: 97 pgs: 97 active+clean; 498 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 74 MiB/s wr, 312 op/s 2024-05-14T16:55:22.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:22 smithi002 ceph-mon[31538]: pgmap v23: 97 pgs: 97 active+clean; 498 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 74 MiB/s wr, 312 op/s 2024-05-14T16:55:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.240+0000 7f6c5ab0a640 1 -- 172.21.15.2:0/2695224494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c54107940 msgr2=0x7f6c54107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.240+0000 7f6c5ab0a640 1 --2- 172.21.15.2:0/2695224494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c54107940 0x7f6c54107d20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6c48009900 tx=0x7f6c4802a410 comp rx=0 tx=0).stop 2024-05-14T16:55:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.240+0000 7f6c5ab0a640 1 -- 172.21.15.2:0/2695224494 shutdown_connections 2024-05-14T16:55:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.240+0000 7f6c5ab0a640 1 --2- 172.21.15.2:0/2695224494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c541022f0 0x7f6c5410e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.240+0000 7f6c5ab0a640 1 --2- 172.21.15.2:0/2695224494 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c54101930 0x7f6c54101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.240+0000 7f6c5ab0a640 1 --2- 172.21.15.2:0/2695224494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c54107940 0x7f6c54107d20 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.240+0000 7f6c5ab0a640 1 -- 172.21.15.2:0/2695224494 >> 172.21.15.2:0/2695224494 conn(0x7f6c540fd420 msgr2=0x7f6c540ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.241+0000 7f6c5ab0a640 1 -- 172.21.15.2:0/2695224494 shutdown_connections 2024-05-14T16:55:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.242+0000 7f6c5ab0a640 1 -- 172.21.15.2:0/2695224494 wait complete. 2024-05-14T16:55:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.243+0000 7f6c5ab0a640 1 Processor -- start 2024-05-14T16:55:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.243+0000 7f6c5ab0a640 1 -- start start 2024-05-14T16:55:23.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.244+0000 7f6c5ab0a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c54101930 0x7f6c5419b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:23.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.244+0000 7f6c5ab0a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c541022f0 0x7f6c5419b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:23.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.244+0000 7f6c5ab0a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c54107940 0x7f6c5419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.244+0000 7f6c5ab0a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6c54110b60 con 0x7f6c541022f0 2024-05-14T16:55:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.244+0000 7f6c5ab0a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6c541109e0 con 0x7f6c54101930 2024-05-14T16:55:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.244+0000 7f6c588a6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c54101930 0x7f6c5419b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.244+0000 7f6c588a6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c54101930 0x7f6c5419b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:37000/0 (socket says 172.21.15.2:37000) 2024-05-14T16:55:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.244+0000 7f6c588a6640 1 -- 172.21.15.2:0/1857910200 learned_addr learned my addr 172.21.15.2:0/1857910200 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.244+0000 7f6c5ab0a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6c54110ce0 con 0x7f6c54107940 2024-05-14T16:55:23.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.244+0000 7f6c590a7640 1 --2- 172.21.15.2:0/1857910200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c54107940 0x7f6c5419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:23.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.244+0000 7f6c53fff640 1 --2- 172.21.15.2:0/1857910200 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c541022f0 0x7f6c5419b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:23.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.244+0000 7f6c588a6640 1 -- 172.21.15.2:0/1857910200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c54107940 msgr2=0x7f6c5419fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:23.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.244+0000 7f6c588a6640 1 --2- 172.21.15.2:0/1857910200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c54107940 0x7f6c5419fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:23.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.244+0000 7f6c588a6640 1 -- 172.21.15.2:0/1857910200 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c541022f0 msgr2=0x7f6c5419b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:23.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.244+0000 7f6c588a6640 1 --2- 172.21.15.2:0/1857910200 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c541022f0 0x7f6c5419b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.244+0000 7f6c588a6640 1 -- 172.21.15.2:0/1857910200 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c541a0400 con 0x7f6c54101930 2024-05-14T16:55:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.245+0000 7f6c588a6640 1 --2- 172.21.15.2:0/1857910200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c54101930 0x7f6c5419b410 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6c48005ab0 tx=0x7f6c48003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.245+0000 7f6c51ffb640 1 -- 172.21.15.2:0/1857910200 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4803f070 con 0x7f6c54101930 2024-05-14T16:55:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.245+0000 7f6c51ffb640 1 -- 172.21.15.2:0/1857910200 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c48031d60 con 0x7f6c54101930 2024-05-14T16:55:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.245+0000 7f6c51ffb640 1 -- 172.21.15.2:0/1857910200 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c48035030 con 0x7f6c54101930 2024-05-14T16:55:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.245+0000 7f6c5ab0a640 1 -- 172.21.15.2:0/1857910200 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c541a0690 con 0x7f6c54101930 2024-05-14T16:55:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.246+0000 7f6c5ab0a640 1 -- 172.21.15.2:0/1857910200 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c541a7f80 con 0x7f6c54101930 2024-05-14T16:55:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.249+0000 7f6c5d81d740 1 Processor -- start 2024-05-14T16:55:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.250+0000 7f6c51ffb640 1 -- 172.21.15.2:0/1857910200 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 149884+0+0 (secure 0 0 0) 0x7f6c48042070 con 0x7f6c54101930 2024-05-14T16:55:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.250+0000 7f6c5d81d740 1 -- start start 2024-05-14T16:55:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.251+0000 7f6c5d81d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55837f16f8f0 0x55837f16fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.251+0000 7f6c5d81d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55837f170210 0x55837f17a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.251+0000 7f6c5d81d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55837f17aad0 0x55837f17cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.251+0000 7f6c5d81d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55837f0e2c30 con 0x55837f170210 2024-05-14T16:55:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.251+0000 7f6c5d81d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55837f0d27d0 con 0x55837f17aad0 2024-05-14T16:55:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.251+0000 7f6c5d81d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55837f0d6810 con 0x55837f16f8f0 2024-05-14T16:55:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.251+0000 7f6c588a6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55837f170210 0x55837f17a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.251+0000 7f6c590a7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55837f16f8f0 0x55837f16fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.251+0000 7f6c590a7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55837f16f8f0 0x55837f16fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:58040/0 (socket says 172.21.15.2:58040) 2024-05-14T16:55:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.251+0000 7f6c590a7640 1 -- 172.21.15.2:0/132813604 learned_addr learned my addr 172.21.15.2:0/132813604 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.251+0000 7f6c53fff640 1 --2- 172.21.15.2:0/132813604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55837f17aad0 0x55837f17cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:23.252 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.252+0000 7f6c53fff640 1 -- 172.21.15.2:0/132813604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55837f16f8f0 msgr2=0x55837f16fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:23.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.252+0000 7f6c53fff640 1 --2- 172.21.15.2:0/132813604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55837f16f8f0 0x55837f16fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:23.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.252+0000 7f6c53fff640 1 -- 172.21.15.2:0/132813604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55837f170210 msgr2=0x55837f17a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:23.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.252+0000 7f6c53fff640 1 --2- 172.21.15.2:0/132813604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55837f170210 0x55837f17a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:23.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.252+0000 7f6c53fff640 1 -- 172.21.15.2:0/132813604 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55837ede7210 con 0x55837f17aad0 2024-05-14T16:55:23.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.252+0000 7f6c51ffb640 1 --2- 172.21.15.2:0/1857910200 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f6c440b2bb0 0x7f6c440b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:23.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.253+0000 7f6c588a6640 1 --2- 172.21.15.2:0/132813604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55837f170210 0x55837f17a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:55:23.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.253+0000 7f6c53fff640 1 --2- 172.21.15.2:0/132813604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55837f17aad0 0x55837f17cec0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000bbb0 tx=0x7f6c4000f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aabeeb288d886f8a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:23.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.253+0000 7f6c51ffb640 1 -- 172.21.15.2:0/1857910200 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f6c48103150 con 0x7f6c54101930 2024-05-14T16:55:23.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.253+0000 7f6c590a7640 1 --2- 172.21.15.2:0/1857910200 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f6c440b2bb0 0x7f6c440b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:23.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.253+0000 7f6c3effd640 1 -- 172.21.15.2:0/132813604 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c40015070 con 0x55837f17aad0 2024-05-14T16:55:23.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.253+0000 7f6c3effd640 1 -- 172.21.15.2:0/132813604 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c400035f0 con 0x55837f17aad0 2024-05-14T16:55:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.253+0000 7f6c3effd640 1 -- 172.21.15.2:0/132813604 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c400116a0 con 0x55837f17aad0 2024-05-14T16:55:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.254+0000 7f6c5d81d740 1 -- 172.21.15.2:0/132813604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55837f17aad0 msgr2=0x55837f17cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.254+0000 7f6c590a7640 1 --2- 172.21.15.2:0/1857910200 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f6c440b2bb0 0x7f6c440b5070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c00b600 tx=0x7f6c4c00e040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:23.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.254+0000 7f6c5d81d740 1 --2- 172.21.15.2:0/132813604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55837f17aad0 0x55837f17cec0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000bbb0 tx=0x7f6c4000f730 comp rx=0 tx=0).stop 2024-05-14T16:55:23.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.254+0000 7f6c5d81d740 1 -- 172.21.15.2:0/132813604 shutdown_connections 2024-05-14T16:55:23.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.254+0000 7f6c5d81d740 1 --2- 172.21.15.2:0/132813604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55837f17aad0 0x55837f17cec0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:23.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.254+0000 7f6c5d81d740 1 --2- 172.21.15.2:0/132813604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55837f170210 0x55837f17a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:23.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.255+0000 7f6c5d81d740 1 --2- 172.21.15.2:0/132813604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55837f16f8f0 0x55837f16fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:23.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.255+0000 7f6c5d81d740 1 -- 172.21.15.2:0/132813604 >> 172.21.15.2:0/132813604 conn(0x55837f16be00 msgr2=0x55837f16c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:23.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.256+0000 7f6c5d81d740 1 -- 172.21.15.2:0/132813604 shutdown_connections 2024-05-14T16:55:23.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.256+0000 7f6c5d81d740 1 -- 172.21.15.2:0/132813604 wait complete. 2024-05-14T16:55:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.256+0000 7f6c5d81d740 1 -- start start 2024-05-14T16:55:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.257+0000 7f6c5d81d740 1 Processor -- start 2024-05-14T16:55:23.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.257+0000 7f6c5d81d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55837f16f8f0 0x55837f17a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:23.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.258+0000 7f6c5d81d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55837f170210 0x55837f1776e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:23.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.258+0000 7f6c53fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55837f170210 0x55837f1776e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:23.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.258+0000 7f6c53fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55837f170210 0x55837f1776e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:37018/0 (socket says 172.21.15.2:37018) 2024-05-14T16:55:23.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.258+0000 7f6c53fff640 1 -- 172.21.15.2:0/4225946617 learned_addr learned my addr 172.21.15.2:0/4225946617 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:23.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.258+0000 7f6c5d81d740 1 --2- 172.21.15.2:0/4225946617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55837f17aad0 0x55837f177c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:23.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.258+0000 7f6c5d81d740 1 -- 172.21.15.2:0/4225946617 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55837edb8790 con 0x55837f16f8f0 2024-05-14T16:55:23.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.259+0000 7f6c5d81d740 1 -- 172.21.15.2:0/4225946617 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55837f08e890 con 0x55837f170210 2024-05-14T16:55:23.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.259+0000 7f6c5d81d740 1 -- 172.21.15.2:0/4225946617 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55837f17d7f0 con 0x55837f17aad0 2024-05-14T16:55:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.259+0000 7f6c590a7640 1 --2- 172.21.15.2:0/4225946617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55837f17aad0 0x55837f177c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.259+0000 7f6c5d81d740 10 client.0 _ll_register_callbacks cb 0x55837f091d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:55:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.259+0000 7f6c588a6640 1 --2- 172.21.15.2:0/4225946617 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55837f16f8f0 0x55837f17a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.259+0000 7f6c53fff640 1 -- 172.21.15.2:0/4225946617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55837f17aad0 msgr2=0x55837f177c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.259+0000 7f6c53fff640 1 --2- 172.21.15.2:0/4225946617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55837f17aad0 0x55837f177c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.259+0000 7f6c53fff640 1 -- 172.21.15.2:0/4225946617 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55837f16f8f0 msgr2=0x55837f17a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.259+0000 7f6c53fff640 1 --2- 172.21.15.2:0/4225946617 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55837f16f8f0 0x55837f17a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.259+0000 7f6c53fff640 1 -- 172.21.15.2:0/4225946617 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55837eeb3620 con 0x55837f170210 2024-05-14T16:55:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.259+0000 7f6c53fff640 1 --2- 172.21.15.2:0/4225946617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55837f170210 0x55837f1776e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000bed0 tx=0x7f6c4000bf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a56a3389a914f91c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.260+0000 7f6c33fff640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T16:55:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.260+0000 7f6c33fff640 1 -- 172.21.15.2:0/4225946617 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c40015070 con 0x55837f170210 2024-05-14T16:55:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.260+0000 7f6c33fff640 1 -- 172.21.15.2:0/4225946617 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f6c400039b0 con 0x55837f170210 2024-05-14T16:55:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.260+0000 7f6c5d81d740 1 -- 172.21.15.2:0/4225946617 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c400035f0 con 0x55837f170210 2024-05-14T16:55:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.260+0000 7f6c33fff640 1 -- 172.21.15.2:0/4225946617 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6c40003f90 con 0x55837f170210 2024-05-14T16:55:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.260+0000 7f6c5d81d740 10 client.25103 fetch_fsmap learned FSMap version 26 2024-05-14T16:55:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.260+0000 7f6c5d81d740 1 -- 172.21.15.2:0/4225946617 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55837f1891e0 con 0x55837f170210 2024-05-14T16:55:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.261+0000 7f6c33fff640 1 -- 172.21.15.2:0/4225946617 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c400035f0 con 0x55837f170210 2024-05-14T16:55:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.261+0000 7f6c33fff640 1 -- 172.21.15.2:0/4225946617 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 26) ==== 8342+0+0 (secure 0 0 0) 0x7f6c400046c0 con 0x55837f170210 2024-05-14T16:55:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.262+0000 7f6c5d81d740 10 client.25103 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T16:55:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.262+0000 7f6c5d81d740 10 client.25103 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:55:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.262+0000 7f6c5d81d740 20 client.25103 populate_metadata read hostname 'smithi002' 2024-05-14T16:55:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.262+0000 7f6c5d81d740 1 --2- 172.21.15.2:0/4225946617 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55837f18a590 0x55837f18c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.262+0000 7f6c5d81d740 4 client.25103 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:55:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.262+0000 7f6c5d81d740 1 -- 172.21.15.2:0/4225946617 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55837f0e4b80 con 0x55837f18a590 2024-05-14T16:55:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.262+0000 7f6c588a6640 1 --2- 172.21.15.2:0/4225946617 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55837f18a590 0x55837f18c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.264+0000 7f6c588a6640 1 --2- 172.21.15.2:0/4225946617 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55837f18a590 0x55837f18c980 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a011c57d5d34be83 server_cookie=1cf804d04a287523 in_seq=0 out_seq=0 2024-05-14T16:55:23.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.264+0000 7f6c33fff640 10 client.25103 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:55:23.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.264+0000 7f6c33fff640 1 -- 172.21.15.2:0/4225946617 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55837f0e4b80 con 0x55837f18a590 2024-05-14T16:55:23.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.264+0000 7f6c33fff640 10 client.25103 handle_command_reply: tid=0 2024-05-14T16:55:23.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.265+0000 7f6c5d81d740 1 client.25103 shutdown 2024-05-14T16:55:23.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.265+0000 7f6c5d81d740 1 -- 172.21.15.2:0/4225946617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55837f170210 msgr2=0x55837f1776e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:23.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.265+0000 7f6c5d81d740 1 --2- 172.21.15.2:0/4225946617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55837f170210 0x55837f1776e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000bed0 tx=0x7f6c4000bf00 comp rx=0 tx=0).stop 2024-05-14T16:55:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.265+0000 7f6c5d81d740 1 -- 172.21.15.2:0/4225946617 shutdown_connections 2024-05-14T16:55:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.265+0000 7f6c5d81d740 1 --2- 172.21.15.2:0/4225946617 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55837f18a590 0x55837f18c980 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.265+0000 7f6c5d81d740 1 --2- 172.21.15.2:0/4225946617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55837f17aad0 0x55837f177c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.265+0000 7f6c5d81d740 1 --2- 172.21.15.2:0/4225946617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55837f170210 0x55837f1776e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.265+0000 7f6c5d81d740 1 --2- 172.21.15.2:0/4225946617 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55837f16f8f0 0x55837f17a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:23.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.265+0000 7f6c5d81d740 1 -- 172.21.15.2:0/4225946617 >> 172.21.15.2:0/4225946617 conn(0x55837f16be00 msgr2=0x55837f16b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:23.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.265+0000 7f6c33fff640 0 client.25103 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:55:23.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.266+0000 7f6c5d81d740 1 -- 172.21.15.2:0/4225946617 shutdown_connections 2024-05-14T16:55:23.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.266+0000 7f6c5d81d740 1 -- 172.21.15.2:0/4225946617 wait complete. 2024-05-14T16:55:23.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.266+0000 7f6c5d81d740 20 client.25103 trim_cache size 0 max 16384 2024-05-14T16:55:23.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.290+0000 7f6c5d81d740 1 Processor -- start 2024-05-14T16:55:23.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.290+0000 7f6c5d81d740 1 -- start start 2024-05-14T16:55:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.290+0000 7f6c5d81d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55837f16f8f0 0x55837f179f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.290+0000 7f6c5d81d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55837f170210 0x55837f172a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.290+0000 7f6c5d81d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55837f17aad0 0x55837f191f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.290+0000 7f6c5d81d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6c541b3120 con 0x55837f17aad0 2024-05-14T16:55:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.290+0000 7f6c5d81d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6c541b09f0 con 0x55837f170210 2024-05-14T16:55:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.290+0000 7f6c5d81d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6c541ae840 con 0x55837f16f8f0 2024-05-14T16:55:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.290+0000 7f6c590a7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55837f17aad0 0x55837f191f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.290+0000 7f6c590a7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55837f17aad0 0x55837f191f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33084/0 (socket says 172.21.15.2:33084) 2024-05-14T16:55:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.290+0000 7f6c590a7640 1 -- 172.21.15.2:0/3973874929 learned_addr learned my addr 172.21.15.2:0/3973874929 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.290+0000 7f6c53fff640 1 --2- 172.21.15.2:0/3973874929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55837f170210 0x55837f172a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.291+0000 7f6c588a6640 1 --2- 172.21.15.2:0/3973874929 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55837f16f8f0 0x55837f179f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.291+0000 7f6c588a6640 1 -- 172.21.15.2:0/3973874929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55837f170210 msgr2=0x55837f172a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.291+0000 7f6c588a6640 1 --2- 172.21.15.2:0/3973874929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55837f170210 0x55837f172a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.291+0000 7f6c588a6640 1 -- 172.21.15.2:0/3973874929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55837f17aad0 msgr2=0x55837f191f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.291+0000 7f6c588a6640 1 --2- 172.21.15.2:0/3973874929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55837f17aad0 0x55837f191f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.291+0000 7f6c588a6640 1 -- 172.21.15.2:0/3973874929 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55837f08a590 con 0x55837f16f8f0 2024-05-14T16:55:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.291+0000 7f6c588a6640 1 --2- 172.21.15.2:0/3973874929 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55837f16f8f0 0x55837f179f20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6c481006f0 tx=0x7f6c48000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfb00d8c7fd5a1ed server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.292+0000 7f6c3e7fc640 1 -- 172.21.15.2:0/3973874929 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c48049040 con 0x55837f16f8f0 2024-05-14T16:55:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.292+0000 7f6c3e7fc640 1 -- 172.21.15.2:0/3973874929 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c4803be50 con 0x55837f16f8f0 2024-05-14T16:55:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.292+0000 7f6c3e7fc640 1 -- 172.21.15.2:0/3973874929 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c480ce670 con 0x55837f16f8f0 2024-05-14T16:55:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.293+0000 7f6c5d81d740 1 -- 172.21.15.2:0/3973874929 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55837f16f8f0 msgr2=0x55837f179f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.293+0000 7f6c5d81d740 1 --2- 172.21.15.2:0/3973874929 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55837f16f8f0 0x55837f179f20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6c481006f0 tx=0x7f6c48000bc0 comp rx=0 tx=0).stop 2024-05-14T16:55:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.293+0000 7f6c5d81d740 1 -- 172.21.15.2:0/3973874929 shutdown_connections 2024-05-14T16:55:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.293+0000 7f6c5d81d740 1 --2- 172.21.15.2:0/3973874929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55837f17aad0 0x55837f191f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.293+0000 7f6c5d81d740 1 --2- 172.21.15.2:0/3973874929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55837f170210 0x55837f172a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.293+0000 7f6c5d81d740 1 --2- 172.21.15.2:0/3973874929 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55837f16f8f0 0x55837f179f20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.293+0000 7f6c5d81d740 1 -- 172.21.15.2:0/3973874929 >> 172.21.15.2:0/3973874929 conn(0x55837f16be00 msgr2=0x55837f172380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.293+0000 7f6c5d81d740 1 -- 172.21.15.2:0/3973874929 shutdown_connections 2024-05-14T16:55:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.293+0000 7f6c5d81d740 1 -- 172.21.15.2:0/3973874929 wait complete. 2024-05-14T16:55:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.294+0000 7f6c5d81d740 1 -- start start 2024-05-14T16:55:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.294+0000 7f6c5d81d740 1 Processor -- start 2024-05-14T16:55:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.294+0000 7f6c5d81d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55837f16f8f0 0x55837f182c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.294+0000 7f6c5d81d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55837f170210 0x55837f173d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.295+0000 7f6c5d81d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55837f17aad0 0x55837f174280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.295+0000 7f6c588a6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55837f16f8f0 0x55837f182c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.295+0000 7f6c5d81d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6c541aeeb0 con 0x55837f17aad0 2024-05-14T16:55:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.295+0000 7f6c5d81d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6c541b0320 con 0x55837f16f8f0 2024-05-14T16:55:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.295+0000 7f6c588a6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55837f16f8f0 0x55837f182c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:37042/0 (socket says 172.21.15.2:37042) 2024-05-14T16:55:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.295+0000 7f6c588a6640 1 -- 172.21.15.2:0/2292199361 learned_addr learned my addr 172.21.15.2:0/2292199361 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.295+0000 7f6c5d81d740 1 -- 172.21.15.2:0/2292199361 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6c541af580 con 0x55837f170210 2024-05-14T16:55:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.295+0000 7f6c590a7640 1 --2- 172.21.15.2:0/2292199361 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55837f17aad0 0x55837f174280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.295+0000 7f6c53fff640 1 --2- 172.21.15.2:0/2292199361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55837f170210 0x55837f173d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.295+0000 7f6c5d81d740 10 client.0 _ll_register_callbacks cb 0x7f6c4000b780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:55:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.295+0000 7f6c588a6640 1 -- 172.21.15.2:0/2292199361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55837f170210 msgr2=0x55837f173d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.295+0000 7f6c588a6640 1 --2- 172.21.15.2:0/2292199361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55837f170210 0x55837f173d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.295+0000 7f6c588a6640 1 -- 172.21.15.2:0/2292199361 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55837f17aad0 msgr2=0x55837f174280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.295+0000 7f6c588a6640 1 --2- 172.21.15.2:0/2292199361 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55837f17aad0 0x55837f174280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:23.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.295+0000 7f6c588a6640 1 -- 172.21.15.2:0/2292199361 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55837f1834a0 con 0x55837f16f8f0 2024-05-14T16:55:23.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.295+0000 7f6c590a7640 1 --2- 172.21.15.2:0/2292199361 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55837f17aad0 0x55837f174280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:55:23.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.295+0000 7f6c588a6640 1 --2- 172.21.15.2:0/2292199361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55837f16f8f0 0x55837f182c30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6c48005ec0 tx=0x7f6c4803b830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76736cbc4e03a5bc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:23.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.295+0000 7f6c33fff640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T16:55:23.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.296+0000 7f6c33fff640 1 -- 172.21.15.2:0/2292199361 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c48049040 con 0x55837f16f8f0 2024-05-14T16:55:23.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.296+0000 7f6c33fff640 1 -- 172.21.15.2:0/2292199361 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f6c480ca6f0 con 0x55837f16f8f0 2024-05-14T16:55:23.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.296+0000 7f6c5d81d740 1 -- 172.21.15.2:0/2292199361 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c4803be50 con 0x55837f16f8f0 2024-05-14T16:55:23.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.296+0000 7f6c33fff640 1 -- 172.21.15.2:0/2292199361 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6c480c99f0 con 0x55837f16f8f0 2024-05-14T16:55:23.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.296+0000 7f6c5d81d740 10 client.25109 fetch_fsmap learned FSMap version 26 2024-05-14T16:55:23.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.296+0000 7f6c5d81d740 1 -- 172.21.15.2:0/2292199361 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55837f1f4620 con 0x55837f16f8f0 2024-05-14T16:55:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.296+0000 7f6c33fff640 1 -- 172.21.15.2:0/2292199361 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c480c9d90 con 0x55837f16f8f0 2024-05-14T16:55:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.297+0000 7f6c33fff640 1 -- 172.21.15.2:0/2292199361 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 26) ==== 8342+0+0 (secure 0 0 0) 0x7f6c480d2c50 con 0x55837f16f8f0 2024-05-14T16:55:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.297+0000 7f6c5d81d740 10 client.25109 fetch_fsmap finished waiting for FSMap version 26 2024-05-14T16:55:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.297+0000 7f6c5d81d740 10 client.25109 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:55:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.297+0000 7f6c5d81d740 20 client.25109 populate_metadata read hostname 'smithi002' 2024-05-14T16:55:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.297+0000 7f6c5d81d740 1 --2- 172.21.15.2:0/2292199361 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55837f1f4c00 0x55837f1f7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.297+0000 7f6c5d81d740 4 client.25109 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:55:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.297+0000 7f6c5d81d740 1 -- 172.21.15.2:0/2292199361 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f6c4803be50 con 0x55837f1f4c00 2024-05-14T16:55:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.297+0000 7f6c53fff640 1 --2- 172.21.15.2:0/2292199361 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55837f1f4c00 0x55837f1f7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.297+0000 7f6c33fff640 10 client.25109 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:55:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.298+0000 7f6c53fff640 1 --2- 172.21.15.2:0/2292199361 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55837f1f4c00 0x55837f1f7020 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48c37b31908d420b server_cookie=58c316787ce589fa in_seq=0 out_seq=0 2024-05-14T16:55:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.326+0000 7f6c33fff640 1 -- 172.21.15.2:0/2292199361 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f6c4803be50 con 0x55837f1f4c00 2024-05-14T16:55:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.326+0000 7f6c33fff640 10 client.25109 handle_command_reply: tid=0 2024-05-14T16:55:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.326+0000 7f6c5d81d740 1 client.25109 shutdown 2024-05-14T16:55:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.326+0000 7f6c5d81d740 1 -- 172.21.15.2:0/2292199361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55837f16f8f0 msgr2=0x55837f182c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.326+0000 7f6c5d81d740 1 --2- 172.21.15.2:0/2292199361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55837f16f8f0 0x55837f182c30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6c48005ec0 tx=0x7f6c4803b830 comp rx=0 tx=0).stop 2024-05-14T16:55:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.326+0000 7f6c5d81d740 1 -- 172.21.15.2:0/2292199361 shutdown_connections 2024-05-14T16:55:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.326+0000 7f6c5d81d740 1 --2- 172.21.15.2:0/2292199361 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55837f1f4c00 0x55837f1f7020 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.327+0000 7f6c5d81d740 1 --2- 172.21.15.2:0/2292199361 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55837f17aad0 0x55837f174280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.327+0000 7f6c5d81d740 1 --2- 172.21.15.2:0/2292199361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55837f170210 0x55837f173d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.327+0000 7f6c5d81d740 1 --2- 172.21.15.2:0/2292199361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55837f16f8f0 0x55837f182c30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.327+0000 7f6c5d81d740 1 -- 172.21.15.2:0/2292199361 >> 172.21.15.2:0/2292199361 conn(0x55837f16be00 msgr2=0x55837f172380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.327+0000 7f6c33fff640 0 client.25109 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:55:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.327+0000 7f6c5d81d740 1 -- 172.21.15.2:0/2292199361 shutdown_connections 2024-05-14T16:55:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.327+0000 7f6c5d81d740 1 -- 172.21.15.2:0/2292199361 wait complete. 2024-05-14T16:55:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.327+0000 7f6c5d81d740 20 client.25109 trim_cache size 0 max 16384 2024-05-14T16:55:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T16:55:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T16:55:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T16:55:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.328+0000 7f6c5ab0a640 1 -- 172.21.15.2:0/1857910200 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f6c440b2bb0 msgr2=0x7f6c440b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.328+0000 7f6c5ab0a640 1 --2- 172.21.15.2:0/1857910200 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f6c440b2bb0 0x7f6c440b5070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c00b600 tx=0x7f6c4c00e040 comp rx=0 tx=0).stop 2024-05-14T16:55:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.328+0000 7f6c5ab0a640 1 -- 172.21.15.2:0/1857910200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c54101930 msgr2=0x7f6c5419b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.328+0000 7f6c5ab0a640 1 --2- 172.21.15.2:0/1857910200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c54101930 0x7f6c5419b410 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6c48005ab0 tx=0x7f6c48003cd0 comp rx=0 tx=0).stop 2024-05-14T16:55:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.328+0000 7f6c5ab0a640 1 -- 172.21.15.2:0/1857910200 shutdown_connections 2024-05-14T16:55:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.328+0000 7f6c5ab0a640 1 --2- 172.21.15.2:0/1857910200 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f6c440b2bb0 0x7f6c440b5070 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.328+0000 7f6c5ab0a640 1 --2- 172.21.15.2:0/1857910200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c54107940 0x7f6c5419fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.328+0000 7f6c5ab0a640 1 --2- 172.21.15.2:0/1857910200 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c541022f0 0x7f6c5419b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.328+0000 7f6c5ab0a640 1 --2- 172.21.15.2:0/1857910200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c54101930 0x7f6c5419b410 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.328+0000 7f6c5ab0a640 1 -- 172.21.15.2:0/1857910200 >> 172.21.15.2:0/1857910200 conn(0x7f6c540fd420 msgr2=0x7f6c540fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.329+0000 7f6c5ab0a640 1 -- 172.21.15.2:0/1857910200 shutdown_connections 2024-05-14T16:55:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:23.329+0000 7f6c5ab0a640 1 -- 172.21.15.2:0/1857910200 wait complete. 2024-05-14T16:55:24.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:24 smithi151 ceph-mon[36066]: pgmap v24: 97 pgs: 97 active+clean; 875 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 79 MiB/s wr, 556 op/s 2024-05-14T16:55:24.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:24 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:55:24.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:24 smithi123 ceph-mon[35971]: pgmap v24: 97 pgs: 97 active+clean; 875 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 79 MiB/s wr, 556 op/s 2024-05-14T16:55:24.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:24 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:55:24.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:24 smithi002 ceph-mon[31538]: pgmap v24: 97 pgs: 97 active+clean; 875 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 79 MiB/s wr, 556 op/s 2024-05-14T16:55:24.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:24 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:55:26.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 10.0 s, 3913.8 tps, lat 25.574 ms stddev 25.727 2024-05-14T16:55:26.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:26 smithi151 ceph-mon[36066]: pgmap v25: 97 pgs: 97 active+clean; 875 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 53 MiB/s wr, 285 op/s 2024-05-14T16:55:26.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:26 smithi123 ceph-mon[35971]: pgmap v25: 97 pgs: 97 active+clean; 875 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 53 MiB/s wr, 285 op/s 2024-05-14T16:55:26.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:26 smithi002 ceph-mon[31538]: pgmap v25: 97 pgs: 97 active+clean; 875 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 53 MiB/s wr, 285 op/s 2024-05-14T16:55:28.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:28 smithi151 ceph-mon[36066]: pgmap v26: 97 pgs: 97 active+clean; 908 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 57 MiB/s wr, 598 op/s 2024-05-14T16:55:28.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:28 smithi123 ceph-mon[35971]: pgmap v26: 97 pgs: 97 active+clean; 908 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 57 MiB/s wr, 598 op/s 2024-05-14T16:55:28.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:28 smithi002 ceph-mon[31538]: pgmap v26: 97 pgs: 97 active+clean; 908 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 57 MiB/s wr, 598 op/s 2024-05-14T16:55:30.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:30 smithi151 ceph-mon[36066]: pgmap v27: 97 pgs: 97 active+clean; 925 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 20 MiB/s wr, 594 op/s 2024-05-14T16:55:30.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:30 smithi123 ceph-mon[35971]: pgmap v27: 97 pgs: 97 active+clean; 925 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 20 MiB/s wr, 594 op/s 2024-05-14T16:55:30.924 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 15.2 2024-05-14T16:55:30.925 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:55:30.959 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:30 smithi002 ceph-mon[31538]: pgmap v27: 97 pgs: 97 active+clean; 925 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 20 MiB/s wr, 594 op/s 2024-05-14T16:55:31.295 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:55:31.712 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:31 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:55:31.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 15.0 s, 3964.8 tps, lat 25.126 ms stddev 26.545 2024-05-14T16:55:31.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:31 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:55:31.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:31 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:55:32.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:32 smithi002 ceph-mon[31538]: pgmap v28: 97 pgs: 97 active+clean; 925 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 9.0 MiB/s wr, 589 op/s 2024-05-14T16:55:32.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:32 smithi151 ceph-mon[36066]: pgmap v28: 97 pgs: 97 active+clean; 925 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 9.0 MiB/s wr, 589 op/s 2024-05-14T16:55:32.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:32 smithi123 ceph-mon[35971]: pgmap v28: 97 pgs: 97 active+clean; 925 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 9.0 MiB/s wr, 589 op/s 2024-05-14T16:55:33.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.191+0000 7fd6f6d1c700 1 -- 172.21.15.2:0/2702377230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6f010a540 msgr2=0x7fd6f010a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:33.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.191+0000 7fd6f6d1c700 1 --2- 172.21.15.2:0/2702377230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6f010a540 0x7fd6f010a920 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec00b0d0 tx=0x7fd6ec00b3e0 comp rx=0 tx=0).stop 2024-05-14T16:55:33.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.191+0000 7fd6f6d1c700 1 -- 172.21.15.2:0/2702377230 shutdown_connections 2024-05-14T16:55:33.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.191+0000 7fd6f6d1c700 1 --2- 172.21.15.2:0/2702377230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6f0116690 0x7fd6f0118a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:33.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.191+0000 7fd6f6d1c700 1 --2- 172.21.15.2:0/2702377230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6f010a540 0x7fd6f010a920 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:33.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.191+0000 7fd6f6d1c700 1 --2- 172.21.15.2:0/2702377230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6f010ae60 0x7fd6f0116150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:33.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.191+0000 7fd6f6d1c700 1 -- 172.21.15.2:0/2702377230 >> 172.21.15.2:0/2702377230 conn(0x7fd6f006ea90 msgr2=0x7fd6f006eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:33.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.191+0000 7fd6f6d1c700 1 -- 172.21.15.2:0/2702377230 shutdown_connections 2024-05-14T16:55:33.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.191+0000 7fd6f6d1c700 1 -- 172.21.15.2:0/2702377230 wait complete. 2024-05-14T16:55:33.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.192+0000 7fd6f6d1c700 1 Processor -- start 2024-05-14T16:55:33.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.192+0000 7fd6f6d1c700 1 -- start start 2024-05-14T16:55:33.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.192+0000 7fd6f6d1c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6f010a540 0x7fd6f010bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:33.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.192+0000 7fd6f6d1c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6f010ae60 0x7fd6f010c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:33.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.192+0000 7fd6f6d1c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6f0116690 0x7fd6f010ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:33.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.192+0000 7fd6f6d1c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f011bf80 con 0x7fd6f010a540 2024-05-14T16:55:33.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.192+0000 7fd6f6d1c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f011be00 con 0x7fd6f0116690 2024-05-14T16:55:33.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.192+0000 7fd6f6d1c700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f011c100 con 0x7fd6f010ae60 2024-05-14T16:55:33.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.192+0000 7fd6f4ab8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6f010a540 0x7fd6f010bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:33.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.192+0000 7fd6e7fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6f010ae60 0x7fd6f010c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:33.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.192+0000 7fd6f4ab8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6f010a540 0x7fd6f010bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55762/0 (socket says 172.21.15.2:55762) 2024-05-14T16:55:33.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.192+0000 7fd6f4ab8700 1 -- 172.21.15.2:0/119101106 learned_addr learned my addr 172.21.15.2:0/119101106 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:33.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.192+0000 7fd6f52b9700 1 --2- 172.21.15.2:0/119101106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6f0116690 0x7fd6f010ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:33.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.193+0000 7fd6f52b9700 1 -- 172.21.15.2:0/119101106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6f010ae60 msgr2=0x7fd6f010c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:33.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.193+0000 7fd6f52b9700 1 --2- 172.21.15.2:0/119101106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6f010ae60 0x7fd6f010c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:33.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.193+0000 7fd6f52b9700 1 -- 172.21.15.2:0/119101106 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6f010a540 msgr2=0x7fd6f010bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:33.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.193+0000 7fd6f52b9700 1 --2- 172.21.15.2:0/119101106 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6f010a540 0x7fd6f010bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:33.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.193+0000 7fd6f52b9700 1 -- 172.21.15.2:0/119101106 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f01b0a20 con 0x7fd6f0116690 2024-05-14T16:55:33.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.193+0000 7fd6f4ab8700 1 --2- 172.21.15.2:0/119101106 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6f010a540 0x7fd6f010bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:55:33.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.193+0000 7fd6f52b9700 1 --2- 172.21.15.2:0/119101106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6f0116690 0x7fd6f010ca30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd6e800e970 tx=0x7fd6e800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:33.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.193+0000 7fd6e7fff700 1 --2- 172.21.15.2:0/119101106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6f010ae60 0x7fd6f010c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:55:33.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.194+0000 7fd6e5ffb700 1 -- 172.21.15.2:0/119101106 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e8004220 con 0x7fd6f0116690 2024-05-14T16:55:33.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.194+0000 7fd6e5ffb700 1 -- 172.21.15.2:0/119101106 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6e80043c0 con 0x7fd6f0116690 2024-05-14T16:55:33.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.194+0000 7fd6f6d1c700 1 -- 172.21.15.2:0/119101106 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6f01b0d10 con 0x7fd6f0116690 2024-05-14T16:55:33.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.194+0000 7fd6f6d1c700 1 -- 172.21.15.2:0/119101106 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6f01b11f0 con 0x7fd6f0116690 2024-05-14T16:55:33.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.194+0000 7fd6e5ffb700 1 -- 172.21.15.2:0/119101106 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e8010700 con 0x7fd6f0116690 2024-05-14T16:55:33.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.196+0000 7fd6f6d1c700 1 -- 172.21.15.2:0/119101106 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6dc005330 con 0x7fd6f0116690 2024-05-14T16:55:33.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.200+0000 7fd6e5ffb700 1 -- 172.21.15.2:0/119101106 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 149884+0+0 (secure 0 0 0) 0x7fd6e8004830 con 0x7fd6f0116690 2024-05-14T16:55:33.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.201+0000 7fd6e5ffb700 1 --2- 172.21.15.2:0/119101106 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fd6d80b2d10 0x7fd6d80b51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:33.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.202+0000 7fd6f4ab8700 1 --2- 172.21.15.2:0/119101106 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fd6d80b2d10 0x7fd6d80b51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:33.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.202+0000 7fd6e5ffb700 1 -- 172.21.15.2:0/119101106 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fd6e80e2460 con 0x7fd6f0116690 2024-05-14T16:55:33.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.205+0000 7fd6f4ab8700 1 --2- 172.21.15.2:0/119101106 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fd6d80b2d10 0x7fd6d80b51d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec009fd0 tx=0x7fd6ec00b040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:33.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.205+0000 7fd6e5ffb700 1 -- 172.21.15.2:0/119101106 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd6e80a9bc0 con 0x7fd6f0116690 2024-05-14T16:55:33.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.688+0000 7fd6f6d1c700 1 -- 172.21.15.2:0/119101106 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6dc0058f0 con 0x7fd6f0116690 2024-05-14T16:55:33.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.689+0000 7fd6e5ffb700 1 -- 172.21.15.2:0/119101106 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11854 (secure 0 0 0) 0x7fd6e80a94c0 con 0x7fd6f0116690 2024-05-14T16:55:33.689 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:55:33.689 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2275696923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2275696923},{"type":"v1","addr":"172.21.15.2:6835","nonce":2275696923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/716718761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":716718761},{"type":"v1","addr":"172.21.15.2:6839","nonce":716718761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/3644815179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3644815179},{"type":"v1","addr":"172.21.15.2:6841","nonce":3644815179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1880336731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1880336731},{"type":"v1","addr":"172.21.15.151:6833","nonce":1880336731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2415605548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2415605548},{"type":"v1","addr":"172.21.15.2:6837","nonce":2415605548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2635167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2635167},{"type":"v1","addr":"172.21.15.123:6833","nonce":2635167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:51:34.106919+0000","modified":"2024-05-14T16:55:30.467925+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24455,"mds_2":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24415":{"gid":24415,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/762044339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":762044339},{"type":"v1","addr":"172.21.15.151:6835","nonce":762044339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/132043151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":132043151},{"type":"v1","addr":"172.21.15.151:6837","nonce":132043151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/4107834375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":4107834375},{"type":"v1","addr":"172.21.15.151:6839","nonce":4107834375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.123:6835/1198670013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1198670013},{"type":"v1","addr":"172.21.15.123:6835","nonce":1198670013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/333426645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":333426645},{"type":"v1","addr":"172.21.15.123:6837","nonce":333426645}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/580275877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":580275877},{"type":"v1","addr":"172.21.15.123:6839","nonce":580275877}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24431,24455,24479]},"id":1}]} 2024-05-14T16:55:33.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.693+0000 7fd6f6d1c700 1 -- 172.21.15.2:0/119101106 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fd6d80b2d10 msgr2=0x7fd6d80b51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:33.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.693+0000 7fd6f6d1c700 1 --2- 172.21.15.2:0/119101106 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fd6d80b2d10 0x7fd6d80b51d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec009fd0 tx=0x7fd6ec00b040 comp rx=0 tx=0).stop 2024-05-14T16:55:33.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.693+0000 7fd6f6d1c700 1 -- 172.21.15.2:0/119101106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6f0116690 msgr2=0x7fd6f010ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:33.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.693+0000 7fd6f6d1c700 1 --2- 172.21.15.2:0/119101106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6f0116690 0x7fd6f010ca30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd6e800e970 tx=0x7fd6e800ec80 comp rx=0 tx=0).stop 2024-05-14T16:55:33.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.693+0000 7fd6f6d1c700 1 -- 172.21.15.2:0/119101106 shutdown_connections 2024-05-14T16:55:33.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.693+0000 7fd6f6d1c700 1 --2- 172.21.15.2:0/119101106 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fd6d80b2d10 0x7fd6d80b51d0 secure :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec009fd0 tx=0x7fd6ec00b040 comp rx=0 tx=0).stop 2024-05-14T16:55:33.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.693+0000 7fd6f6d1c700 1 --2- 172.21.15.2:0/119101106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6f0116690 0x7fd6f010ca30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:33.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.693+0000 7fd6f6d1c700 1 --2- 172.21.15.2:0/119101106 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6f010a540 0x7fd6f010bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:33.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.693+0000 7fd6f6d1c700 1 --2- 172.21.15.2:0/119101106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6f010ae60 0x7fd6f010c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:33.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.693+0000 7fd6f6d1c700 1 -- 172.21.15.2:0/119101106 >> 172.21.15.2:0/119101106 conn(0x7fd6f006ea90 msgr2=0x7fd6f00736e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:33.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.695+0000 7fd6f6d1c700 1 -- 172.21.15.2:0/119101106 shutdown_connections 2024-05-14T16:55:33.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:33.696+0000 7fd6f6d1c700 1 -- 172.21.15.2:0/119101106 wait complete. 2024-05-14T16:55:33.697 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 28 2024-05-14T16:55:34.525 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 135.2 --await-for 45.0' 2024-05-14T16:55:34.525 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 135.2 --await-for 45.0 2024-05-14T16:55:34.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:34 smithi002 ceph-mon[31538]: pgmap v29: 97 pgs: 97 active+clean; 933 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 12 MiB/s wr, 917 op/s 2024-05-14T16:55:34.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:34 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/119101106' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:55:34.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:34 smithi151 ceph-mon[36066]: pgmap v29: 97 pgs: 97 active+clean; 933 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 12 MiB/s wr, 917 op/s 2024-05-14T16:55:34.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:34 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/119101106' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:55:34.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:34 smithi123 ceph-mon[35971]: pgmap v29: 97 pgs: 97 active+clean; 933 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 12 MiB/s wr, 917 op/s 2024-05-14T16:55:34.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:34 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/119101106' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:55:34.986 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:55:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.468+0000 7f71d48c7640 1 -- 172.21.15.2:0/602383929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71c809be10 msgr2=0x7f71c809c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.468+0000 7f71d48c7640 1 --2- 172.21.15.2:0/602383929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71c809be10 0x7f71c809c1f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f71c0009930 tx=0x7f71c002a440 comp rx=0 tx=0).stop 2024-05-14T16:55:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.468+0000 7f71d48c7640 1 -- 172.21.15.2:0/602383929 shutdown_connections 2024-05-14T16:55:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.468+0000 7f71d48c7640 1 --2- 172.21.15.2:0/602383929 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71c8005670 0x7f71c80a2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.468+0000 7f71d48c7640 1 --2- 172.21.15.2:0/602383929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71c8004cb0 0x7f71c8005130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.468+0000 7f71d48c7640 1 --2- 172.21.15.2:0/602383929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71c809be10 0x7f71c809c1f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.468+0000 7f71d48c7640 1 -- 172.21.15.2:0/602383929 >> 172.21.15.2:0/602383929 conn(0x7f71c800a740 msgr2=0x7f71c800ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.468+0000 7f71d48c7640 1 -- 172.21.15.2:0/602383929 shutdown_connections 2024-05-14T16:55:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.468+0000 7f71d48c7640 1 -- 172.21.15.2:0/602383929 wait complete. 2024-05-14T16:55:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.469+0000 7f71d48c7640 1 Processor -- start 2024-05-14T16:55:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.469+0000 7f71d48c7640 1 -- start start 2024-05-14T16:55:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.470+0000 7f71d48c7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71c8004cb0 0x7f71c80a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.470+0000 7f71d48c7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71c8005670 0x7f71c809fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.470+0000 7f71d48c7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71c809be10 0x7f71c80a0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.470+0000 7f71d48c7640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f71c8096860 con 0x7f71c8004cb0 2024-05-14T16:55:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.470+0000 7f71d48c7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f71c80966e0 con 0x7f71c8005670 2024-05-14T16:55:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.470+0000 7f71d48c7640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f71c80969e0 con 0x7f71c809be10 2024-05-14T16:55:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.470+0000 7f71cf59e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71c809be10 0x7f71c80a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.470+0000 7f71cf59e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71c809be10 0x7f71c80a0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:60672/0 (socket says 172.21.15.2:60672) 2024-05-14T16:55:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.470+0000 7f71cf59e640 1 -- 172.21.15.2:0/528161526 learned_addr learned my addr 172.21.15.2:0/528161526 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.470+0000 7f71ce59c640 1 --2- 172.21.15.2:0/528161526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71c8005670 0x7f71c809fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.470+0000 7f71ced9d640 1 --2- 172.21.15.2:0/528161526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71c8004cb0 0x7f71c80a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.470+0000 7f71cf59e640 1 -- 172.21.15.2:0/528161526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71c8005670 msgr2=0x7f71c809fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.470+0000 7f71cf59e640 1 --2- 172.21.15.2:0/528161526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71c8005670 0x7f71c809fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.470+0000 7f71cf59e640 1 -- 172.21.15.2:0/528161526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71c8004cb0 msgr2=0x7f71c80a2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.470+0000 7f71cf59e640 1 --2- 172.21.15.2:0/528161526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71c8004cb0 0x7f71c80a2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.470+0000 7f71cf59e640 1 -- 172.21.15.2:0/528161526 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c80a0950 con 0x7f71c809be10 2024-05-14T16:55:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.470+0000 7f71ce59c640 1 --2- 172.21.15.2:0/528161526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71c8005670 0x7f71c809fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:55:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.471+0000 7f71ced9d640 1 --2- 172.21.15.2:0/528161526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71c8004cb0 0x7f71c80a2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:55:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.471+0000 7f71cf59e640 1 --2- 172.21.15.2:0/528161526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71c809be10 0x7f71c80a0090 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f71d00520e0 tx=0x7f71d007c710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.471+0000 7f71b7fff640 1 -- 172.21.15.2:0/528161526 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d007cd90 con 0x7f71c809be10 2024-05-14T16:55:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.471+0000 7f71b7fff640 1 -- 172.21.15.2:0/528161526 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71d007e8d0 con 0x7f71c809be10 2024-05-14T16:55:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.471+0000 7f71d48c7640 1 -- 172.21.15.2:0/528161526 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71c813efd0 con 0x7f71c809be10 2024-05-14T16:55:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.471+0000 7f71b7fff640 1 -- 172.21.15.2:0/528161526 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d0086830 con 0x7f71c809be10 2024-05-14T16:55:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.471+0000 7f71d48c7640 1 -- 172.21.15.2:0/528161526 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71c813f440 con 0x7f71c809be10 2024-05-14T16:55:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.475+0000 7f71d7dde740 1 Processor -- start 2024-05-14T16:55:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.475+0000 7f71b7fff640 1 -- 172.21.15.2:0/528161526 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 25) ==== 149884+0+0 (secure 0 0 0) 0x7f71d007b040 con 0x7f71c809be10 2024-05-14T16:55:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.476+0000 7f71d7dde740 1 -- start start 2024-05-14T16:55:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.476+0000 7f71d7dde740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f06dabae0 0x556f06dabec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.476+0000 7f71d7dde740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f06dac400 0x556f06db6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.476+0000 7f71d7dde740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f06db6cc0 0x556f06db90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.476+0000 7f71d7dde740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x556f06d1ec30 con 0x556f06db6cc0 2024-05-14T16:55:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.476+0000 7f71d7dde740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556f06d0e7d0 con 0x556f06dac400 2024-05-14T16:55:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.476+0000 7f71d7dde740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556f06d12810 con 0x556f06dabae0 2024-05-14T16:55:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.476+0000 7f71ce59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f06db6cc0 0x556f06db90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.476+0000 7f71ced9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f06dac400 0x556f06db6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.476+0000 7f71ce59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f06db6cc0 0x556f06db90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34464/0 (socket says 172.21.15.2:34464) 2024-05-14T16:55:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.476+0000 7f71ce59c640 1 -- 172.21.15.2:0/3686078208 learned_addr learned my addr 172.21.15.2:0/3686078208 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.476+0000 7f71ce59c640 1 --2- 172.21.15.2:0/3686078208 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f06dabae0 0x556f06dabec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.476+0000 7f71ced9d640 1 -- 172.21.15.2:0/3686078208 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f06dabae0 msgr2=0x556f06dabec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.476+0000 7f71ced9d640 1 --2- 172.21.15.2:0/3686078208 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f06dabae0 0x556f06dabec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.476+0000 7f71ced9d640 1 -- 172.21.15.2:0/3686078208 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f06db6cc0 msgr2=0x556f06db90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.476+0000 7f71ced9d640 1 --2- 172.21.15.2:0/3686078208 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f06db6cc0 0x556f06db90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.476+0000 7f71ced9d640 1 -- 172.21.15.2:0/3686078208 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f06a23210 con 0x556f06dac400 2024-05-14T16:55:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.476+0000 7f71ce59c640 1 --2- 172.21.15.2:0/3686078208 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f06dabae0 0x556f06dabec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:55:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.477+0000 7f71ced9d640 1 --2- 172.21.15.2:0/3686078208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f06dac400 0x556f06db6780 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f71c0005ab0 tx=0x7f71c00042e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7450131d98ef5e94 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.477+0000 7f71b7fff640 1 --2- 172.21.15.2:0/528161526 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f71b80b2b00 0x7f71b80b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.477+0000 7f71ce59c640 1 --2- 172.21.15.2:0/528161526 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f71b80b2b00 0x7f71b80b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.477+0000 7f71b7fff640 1 -- 172.21.15.2:0/528161526 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f71d014f9f0 con 0x7f71c809be10 2024-05-14T16:55:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.478+0000 7f71b57fa640 1 -- 172.21.15.2:0/3686078208 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c003f070 con 0x556f06dac400 2024-05-14T16:55:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.478+0000 7f71b57fa640 1 -- 172.21.15.2:0/3686078208 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71c00044d0 con 0x556f06dac400 2024-05-14T16:55:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.478+0000 7f71b57fa640 1 -- 172.21.15.2:0/3686078208 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c0035030 con 0x556f06dac400 2024-05-14T16:55:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.479+0000 7f71d7dde740 1 -- 172.21.15.2:0/3686078208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f06dac400 msgr2=0x556f06db6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.479+0000 7f71d7dde740 1 --2- 172.21.15.2:0/3686078208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f06dac400 0x556f06db6780 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f71c0005ab0 tx=0x7f71c00042e0 comp rx=0 tx=0).stop 2024-05-14T16:55:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.479+0000 7f71ce59c640 1 --2- 172.21.15.2:0/528161526 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f71b80b2b00 0x7f71b80b4fc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x556f06da9410 tx=0x7f71c4009770 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.479+0000 7f71d7dde740 1 -- 172.21.15.2:0/3686078208 shutdown_connections 2024-05-14T16:55:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.479+0000 7f71d7dde740 1 --2- 172.21.15.2:0/3686078208 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f06db6cc0 0x556f06db90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.479+0000 7f71d7dde740 1 --2- 172.21.15.2:0/3686078208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f06dac400 0x556f06db6780 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.479+0000 7f71d7dde740 1 --2- 172.21.15.2:0/3686078208 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f06dabae0 0x556f06dabec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.479+0000 7f71d7dde740 1 -- 172.21.15.2:0/3686078208 >> 172.21.15.2:0/3686078208 conn(0x556f06da7ff0 msgr2=0x556f06da83d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.479+0000 7f71d7dde740 1 -- 172.21.15.2:0/3686078208 shutdown_connections 2024-05-14T16:55:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.479+0000 7f71d7dde740 1 -- 172.21.15.2:0/3686078208 wait complete. 2024-05-14T16:55:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.479+0000 7f71d7dde740 1 -- start start 2024-05-14T16:55:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.480+0000 7f71d7dde740 1 Processor -- start 2024-05-14T16:55:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.480+0000 7f71d7dde740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f06dabae0 0x556f06db6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.480+0000 7f71d7dde740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f06dac400 0x556f06db3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.480+0000 7f71d7dde740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f06db6cc0 0x556f06db3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.480+0000 7f71d7dde740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x556f069f4790 con 0x556f06dabae0 2024-05-14T16:55:36.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.480+0000 7f71d7dde740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556f06cca890 con 0x556f06db6cc0 2024-05-14T16:55:36.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.480+0000 7f71d7dde740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556f06db9930 con 0x556f06dac400 2024-05-14T16:55:36.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.480+0000 7f71ced9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f06dabae0 0x556f06db6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:36.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.480+0000 7f71ced9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f06dabae0 0x556f06db6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34480/0 (socket says 172.21.15.2:34480) 2024-05-14T16:55:36.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.480+0000 7f71ced9d640 1 -- 172.21.15.2:0/1460873346 learned_addr learned my addr 172.21.15.2:0/1460873346 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:36.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.480+0000 7f71d7dde740 10 client.0 _ll_register_callbacks cb 0x556f06ccdd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:55:36.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.480+0000 7f71cf59e640 1 --2- 172.21.15.2:0/1460873346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f06db6cc0 0x556f06db3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:36.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.480+0000 7f71ce59c640 1 --2- 172.21.15.2:0/1460873346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f06dac400 0x556f06db3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:36.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.480+0000 7f71ced9d640 1 -- 172.21.15.2:0/1460873346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f06dac400 msgr2=0x556f06db3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:36.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.480+0000 7f71ced9d640 1 --2- 172.21.15.2:0/1460873346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f06dac400 0x556f06db3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:36.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.480+0000 7f71ced9d640 1 -- 172.21.15.2:0/1460873346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f06db6cc0 msgr2=0x556f06db3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:36.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.480+0000 7f71ced9d640 1 --2- 172.21.15.2:0/1460873346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f06db6cc0 0x556f06db3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:36.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.480+0000 7f71ced9d640 1 -- 172.21.15.2:0/1460873346 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f06aef620 con 0x556f06dabae0 2024-05-14T16:55:36.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.480+0000 7f71cf59e640 1 --2- 172.21.15.2:0/1460873346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f06db6cc0 0x556f06db3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:55:36.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.481+0000 7f71ced9d640 1 --2- 172.21.15.2:0/1460873346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f06dabae0 0x556f06db6630 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f71c002a960 tx=0x7f71c0004f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8999bce4f21c21e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:36.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.481+0000 7f71b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:55:36.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.481+0000 7f71b2ff5640 1 -- 172.21.15.2:0/1460873346 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c004c020 con 0x556f06dabae0 2024-05-14T16:55:36.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.481+0000 7f71b2ff5640 1 -- 172.21.15.2:0/1460873346 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f71c0005200 con 0x556f06dabae0 2024-05-14T16:55:36.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.481+0000 7f71d7dde740 1 -- 172.21.15.2:0/1460873346 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71c00044d0 con 0x556f06dabae0 2024-05-14T16:55:36.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.481+0000 7f71b2ff5640 1 -- 172.21.15.2:0/1460873346 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f71c003f050 con 0x556f06dabae0 2024-05-14T16:55:36.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.481+0000 7f71d7dde740 10 client.15387 fetch_fsmap learned FSMap version 28 2024-05-14T16:55:36.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.481+0000 7f71d7dde740 1 -- 172.21.15.2:0/1460873346 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556f06dc53d0 con 0x556f06dabae0 2024-05-14T16:55:36.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.482+0000 7f71b2ff5640 1 -- 172.21.15.2:0/1460873346 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71c00044d0 con 0x556f06dabae0 2024-05-14T16:55:36.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.482+0000 7f71b2ff5640 1 -- 172.21.15.2:0/1460873346 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 28) ==== 8350+0+0 (secure 0 0 0) 0x7f71c003b460 con 0x556f06dabae0 2024-05-14T16:55:36.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.482+0000 7f71d7dde740 10 client.15387 fetch_fsmap finished waiting for FSMap version 28 2024-05-14T16:55:36.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.482+0000 7f71d7dde740 10 client.15387 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:55:36.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.482+0000 7f71d7dde740 20 client.15387 populate_metadata read hostname 'smithi002' 2024-05-14T16:55:36.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.482+0000 7f71d7dde740 1 --2- 172.21.15.2:0/1460873346 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x556f06dc6720 0x556f06dc8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:36.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.482+0000 7f71d7dde740 4 client.15387 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:55:36.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.482+0000 7f71d7dde740 1 -- 172.21.15.2:0/1460873346 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f06d20b80 con 0x556f06dc6720 2024-05-14T16:55:36.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.482+0000 7f71ce59c640 1 --2- 172.21.15.2:0/1460873346 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x556f06dc6720 0x556f06dc8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:36.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.483+0000 7f71ce59c640 1 --2- 172.21.15.2:0/1460873346 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x556f06dc6720 0x556f06dc8b10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96cf2c8b9bb0a90f server_cookie=151b8b78b2559348 in_seq=0 out_seq=0 2024-05-14T16:55:36.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.483+0000 7f71b2ff5640 10 client.15387 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:55:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.484+0000 7f71b2ff5640 1 -- 172.21.15.2:0/1460873346 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556f06d20b80 con 0x556f06dc6720 2024-05-14T16:55:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.484+0000 7f71b2ff5640 10 client.15387 handle_command_reply: tid=0 2024-05-14T16:55:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.484+0000 7f71d7dde740 1 client.15387 shutdown 2024-05-14T16:55:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.484+0000 7f71d7dde740 1 -- 172.21.15.2:0/1460873346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f06dabae0 msgr2=0x556f06db6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.484+0000 7f71d7dde740 1 --2- 172.21.15.2:0/1460873346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f06dabae0 0x556f06db6630 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f71c002a960 tx=0x7f71c0004f20 comp rx=0 tx=0).stop 2024-05-14T16:55:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.484+0000 7f71d7dde740 1 -- 172.21.15.2:0/1460873346 shutdown_connections 2024-05-14T16:55:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.484+0000 7f71d7dde740 1 --2- 172.21.15.2:0/1460873346 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x556f06dc6720 0x556f06dc8b10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.484+0000 7f71d7dde740 1 --2- 172.21.15.2:0/1460873346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f06db6cc0 0x556f06db3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.484+0000 7f71b2ff5640 0 client.15387 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:55:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.484+0000 7f71d7dde740 1 --2- 172.21.15.2:0/1460873346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f06dac400 0x556f06db3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.484+0000 7f71d7dde740 1 --2- 172.21.15.2:0/1460873346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f06dabae0 0x556f06db6630 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.484+0000 7f71d7dde740 1 -- 172.21.15.2:0/1460873346 >> 172.21.15.2:0/1460873346 conn(0x556f06da7ff0 msgr2=0x556f06da7350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.484+0000 7f71d7dde740 1 -- 172.21.15.2:0/1460873346 shutdown_connections 2024-05-14T16:55:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.484+0000 7f71d7dde740 1 -- 172.21.15.2:0/1460873346 wait complete. 2024-05-14T16:55:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.485+0000 7f71d7dde740 20 client.15387 trim_cache size 0 max 16384 2024-05-14T16:55:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.506+0000 7f71d7dde740 1 Processor -- start 2024-05-14T16:55:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.506+0000 7f71d7dde740 1 -- start start 2024-05-14T16:55:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.506+0000 7f71d7dde740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f06dabae0 0x556f06da83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.507+0000 7f71d7dde740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f06dac400 0x556f06dc2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.507+0000 7f71d7dde740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f06db6cc0 0x556f06db6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.507+0000 7f71d7dde740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f71c814a6f0 con 0x556f06dac400 2024-05-14T16:55:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.507+0000 7f71d7dde740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f71c8147f90 con 0x556f06db6cc0 2024-05-14T16:55:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.507+0000 7f71d7dde740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f71c8145de0 con 0x556f06dabae0 2024-05-14T16:55:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.507+0000 7f71cf59e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f06db6cc0 0x556f06db6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.507+0000 7f71cf59e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f06db6cc0 0x556f06db6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:38072/0 (socket says 172.21.15.2:38072) 2024-05-14T16:55:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.507+0000 7f71cf59e640 1 -- 172.21.15.2:0/3351982764 learned_addr learned my addr 172.21.15.2:0/3351982764 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.507+0000 7f71ced9d640 1 --2- 172.21.15.2:0/3351982764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f06dabae0 0x556f06da83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.507+0000 7f71ce59c640 1 --2- 172.21.15.2:0/3351982764 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f06dac400 0x556f06dc2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.507+0000 7f71cf59e640 1 -- 172.21.15.2:0/3351982764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f06dabae0 msgr2=0x556f06da83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.507+0000 7f71cf59e640 1 --2- 172.21.15.2:0/3351982764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f06dabae0 0x556f06da83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.507+0000 7f71cf59e640 1 -- 172.21.15.2:0/3351982764 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f06dac400 msgr2=0x556f06dc2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.507+0000 7f71cf59e640 1 --2- 172.21.15.2:0/3351982764 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f06dac400 0x556f06dc2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.507+0000 7f71cf59e640 1 -- 172.21.15.2:0/3351982764 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f06da3d30 con 0x556f06db6cc0 2024-05-14T16:55:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.507+0000 7f71ce59c640 1 --2- 172.21.15.2:0/3351982764 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f06dac400 0x556f06dc2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:55:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.508+0000 7f71cf59e640 1 --2- 172.21.15.2:0/3351982764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f06db6cc0 0x556f06db6160 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f71d0116820 tx=0x7f71d00fe040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d30f55a9db5f3e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.508+0000 7f71b4ff9640 1 -- 172.21.15.2:0/3351982764 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d0116b20 con 0x556f06db6cc0 2024-05-14T16:55:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.508+0000 7f71b4ff9640 1 -- 172.21.15.2:0/3351982764 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71d0117e50 con 0x556f06db6cc0 2024-05-14T16:55:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.508+0000 7f71b4ff9640 1 -- 172.21.15.2:0/3351982764 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d0115d80 con 0x556f06db6cc0 2024-05-14T16:55:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.509+0000 7f71d7dde740 1 -- 172.21.15.2:0/3351982764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f06db6cc0 msgr2=0x556f06db6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.509+0000 7f71d7dde740 1 --2- 172.21.15.2:0/3351982764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f06db6cc0 0x556f06db6160 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f71d0116820 tx=0x7f71d00fe040 comp rx=0 tx=0).stop 2024-05-14T16:55:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.509+0000 7f71d7dde740 1 -- 172.21.15.2:0/3351982764 shutdown_connections 2024-05-14T16:55:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.509+0000 7f71d7dde740 1 --2- 172.21.15.2:0/3351982764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f06db6cc0 0x556f06db6160 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.509+0000 7f71d7dde740 1 --2- 172.21.15.2:0/3351982764 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f06dac400 0x556f06dc2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.509+0000 7f71d7dde740 1 --2- 172.21.15.2:0/3351982764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f06dabae0 0x556f06da83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.509+0000 7f71d7dde740 1 -- 172.21.15.2:0/3351982764 >> 172.21.15.2:0/3351982764 conn(0x556f06da7ff0 msgr2=0x556f06daefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.509+0000 7f71d7dde740 1 -- 172.21.15.2:0/3351982764 shutdown_connections 2024-05-14T16:55:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.509+0000 7f71d7dde740 1 -- 172.21.15.2:0/3351982764 wait complete. 2024-05-14T16:55:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.509+0000 7f71d7dde740 1 -- start start 2024-05-14T16:55:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.510+0000 7f71d7dde740 1 Processor -- start 2024-05-14T16:55:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.510+0000 7f71d7dde740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f06dabae0 0x556f06dbf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.510+0000 7f71d7dde740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f06dac400 0x556f06dba3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.510+0000 7f71d7dde740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f06db6cc0 0x556f06dba8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.510+0000 7f71d7dde740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f71c8146450 con 0x556f06dabae0 2024-05-14T16:55:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.511+0000 7f71d7dde740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f71c81478c0 con 0x556f06db6cc0 2024-05-14T16:55:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.511+0000 7f71d7dde740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f71c8146b20 con 0x556f06dac400 2024-05-14T16:55:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.511+0000 7f71ce59c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f06dac400 0x556f06dba3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.511+0000 7f71ce59c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f06dac400 0x556f06dba3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:60712/0 (socket says 172.21.15.2:60712) 2024-05-14T16:55:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.511+0000 7f71ce59c640 1 -- 172.21.15.2:0/1647870066 learned_addr learned my addr 172.21.15.2:0/1647870066 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.511+0000 7f71d7dde740 10 client.0 _ll_register_callbacks cb 0x7f71c0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:55:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.511+0000 7f71ced9d640 1 --2- 172.21.15.2:0/1647870066 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f06dabae0 0x556f06dbf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.511+0000 7f71cf59e640 1 --2- 172.21.15.2:0/1647870066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f06db6cc0 0x556f06dba8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.511+0000 7f71ce59c640 1 -- 172.21.15.2:0/1647870066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f06db6cc0 msgr2=0x556f06dba8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.511+0000 7f71ce59c640 1 --2- 172.21.15.2:0/1647870066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f06db6cc0 0x556f06dba8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.511+0000 7f71ce59c640 1 -- 172.21.15.2:0/1647870066 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f06dabae0 msgr2=0x556f06dbf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.511+0000 7f71ce59c640 1 --2- 172.21.15.2:0/1647870066 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f06dabae0 0x556f06dbf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.512+0000 7f71ce59c640 1 -- 172.21.15.2:0/1647870066 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f06dbafd0 con 0x556f06dac400 2024-05-14T16:55:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.512+0000 7f71cf59e640 1 --2- 172.21.15.2:0/1647870066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f06db6cc0 0x556f06dba8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:55:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.512+0000 7f71ce59c640 1 --2- 172.21.15.2:0/1647870066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f06dac400 0x556f06dba3a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f71c4011660 tx=0x7f71c4005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2bfaf19a206e7f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.512+0000 7f71b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-14T16:55:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.512+0000 7f71b2ff5640 1 -- 172.21.15.2:0/1647870066 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c40051c0 con 0x556f06dac400 2024-05-14T16:55:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.512+0000 7f71d7dde740 1 -- 172.21.15.2:0/1647870066 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71d0117e50 con 0x556f06dac400 2024-05-14T16:55:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.512+0000 7f71b2ff5640 1 -- 172.21.15.2:0/1647870066 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f71c4005360 con 0x556f06dac400 2024-05-14T16:55:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.513+0000 7f71b2ff5640 1 -- 172.21.15.2:0/1647870066 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f71c4039050 con 0x556f06dac400 2024-05-14T16:55:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.513+0000 7f71d7dde740 10 client.25126 fetch_fsmap learned FSMap version 28 2024-05-14T16:55:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.513+0000 7f71d7dde740 1 -- 172.21.15.2:0/1647870066 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556f06e30cd0 con 0x556f06dac400 2024-05-14T16:55:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.513+0000 7f71b2ff5640 1 -- 172.21.15.2:0/1647870066 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71d0117e50 con 0x556f06dac400 2024-05-14T16:55:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.513+0000 7f71b2ff5640 1 -- 172.21.15.2:0/1647870066 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 28) ==== 8350+0+0 (secure 0 0 0) 0x7f71c402bce0 con 0x556f06dac400 2024-05-14T16:55:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.513+0000 7f71d7dde740 10 client.25126 fetch_fsmap finished waiting for FSMap version 28 2024-05-14T16:55:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.513+0000 7f71d7dde740 10 client.25126 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:55:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.513+0000 7f71d7dde740 20 client.25126 populate_metadata read hostname 'smithi002' 2024-05-14T16:55:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.513+0000 7f71d7dde740 1 --2- 172.21.15.2:0/1647870066 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x556f06e31490 0x556f06e338b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.513+0000 7f71d7dde740 4 client.25126 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:55:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.513+0000 7f71d7dde740 1 -- 172.21.15.2:0/1647870066 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556f06d1a360 con 0x556f06e31490 2024-05-14T16:55:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.513+0000 7f71ced9d640 1 --2- 172.21.15.2:0/1647870066 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x556f06e31490 0x556f06e338b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.514+0000 7f71ced9d640 1 --2- 172.21.15.2:0/1647870066 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x556f06e31490 0x556f06e338b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acb25aceb008a986 server_cookie=79dc4e7775d79a49 in_seq=0 out_seq=0 2024-05-14T16:55:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.514+0000 7f71b2ff5640 10 client.25126 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:55:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.526+0000 7f71b2ff5640 1 -- 172.21.15.2:0/1647870066 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556f06d1a360 con 0x556f06e31490 2024-05-14T16:55:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.526+0000 7f71b2ff5640 10 client.25126 handle_command_reply: tid=0 2024-05-14T16:55:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.526+0000 7f71d7dde740 1 client.25126 shutdown 2024-05-14T16:55:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.526+0000 7f71d7dde740 1 -- 172.21.15.2:0/1647870066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f06dac400 msgr2=0x556f06dba3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.526+0000 7f71d7dde740 1 --2- 172.21.15.2:0/1647870066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f06dac400 0x556f06dba3a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f71c4011660 tx=0x7f71c4005f90 comp rx=0 tx=0).stop 2024-05-14T16:55:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.526+0000 7f71d7dde740 1 -- 172.21.15.2:0/1647870066 shutdown_connections 2024-05-14T16:55:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.526+0000 7f71d7dde740 1 --2- 172.21.15.2:0/1647870066 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x556f06e31490 0x556f06e338b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.526+0000 7f71d7dde740 1 --2- 172.21.15.2:0/1647870066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556f06db6cc0 0x556f06dba8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.526+0000 7f71d7dde740 1 --2- 172.21.15.2:0/1647870066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556f06dac400 0x556f06dba3a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.526+0000 7f71d7dde740 1 --2- 172.21.15.2:0/1647870066 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556f06dabae0 0x556f06dbf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.526+0000 7f71d7dde740 1 -- 172.21.15.2:0/1647870066 >> 172.21.15.2:0/1647870066 conn(0x556f06da7ff0 msgr2=0x556f06da7350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.527+0000 7f71b2ff5640 0 client.25126 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:55:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.527+0000 7f71d7dde740 1 -- 172.21.15.2:0/1647870066 shutdown_connections 2024-05-14T16:55:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.527+0000 7f71d7dde740 1 -- 172.21.15.2:0/1647870066 wait complete. 2024-05-14T16:55:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.527+0000 7f71d7dde740 20 client.25126 trim_cache size 0 max 16384 2024-05-14T16:55:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T16:55:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T16:55:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T16:55:36.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.528+0000 7f71d48c7640 1 -- 172.21.15.2:0/528161526 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f71b80b2b00 msgr2=0x7f71b80b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:36.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.528+0000 7f71d48c7640 1 --2- 172.21.15.2:0/528161526 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f71b80b2b00 0x7f71b80b4fc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x556f06da9410 tx=0x7f71c4009770 comp rx=0 tx=0).stop 2024-05-14T16:55:36.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.528+0000 7f71d48c7640 1 -- 172.21.15.2:0/528161526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71c809be10 msgr2=0x7f71c80a0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:36.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.528+0000 7f71d48c7640 1 --2- 172.21.15.2:0/528161526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71c809be10 0x7f71c80a0090 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f71d00520e0 tx=0x7f71d007c710 comp rx=0 tx=0).stop 2024-05-14T16:55:36.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.528+0000 7f71d48c7640 1 -- 172.21.15.2:0/528161526 shutdown_connections 2024-05-14T16:55:36.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.528+0000 7f71d48c7640 1 --2- 172.21.15.2:0/528161526 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f71b80b2b00 0x7f71b80b4fc0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:36.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.528+0000 7f71d48c7640 1 --2- 172.21.15.2:0/528161526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71c809be10 0x7f71c80a0090 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:36.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.528+0000 7f71d48c7640 1 --2- 172.21.15.2:0/528161526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71c8005670 0x7f71c809fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:36.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.528+0000 7f71d48c7640 1 --2- 172.21.15.2:0/528161526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71c8004cb0 0x7f71c80a2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:36.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.528+0000 7f71d48c7640 1 -- 172.21.15.2:0/528161526 >> 172.21.15.2:0/528161526 conn(0x7f71c800a740 msgr2=0x7f71c8093b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:36.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.528+0000 7f71d48c7640 1 -- 172.21.15.2:0/528161526 shutdown_connections 2024-05-14T16:55:36.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:36.528+0000 7f71d48c7640 1 -- 172.21.15.2:0/528161526 wait complete. 2024-05-14T16:55:36.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 20.0 s, 3775.6 tps, lat 26.521 ms stddev 28.248 2024-05-14T16:55:36.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:36 smithi151 ceph-mon[36066]: pgmap v30: 97 pgs: 97 active+clean; 933 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 8.1 MiB/s wr, 673 op/s 2024-05-14T16:55:36.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:36 smithi123 ceph-mon[35971]: pgmap v30: 97 pgs: 97 active+clean; 933 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 8.1 MiB/s wr, 673 op/s 2024-05-14T16:55:36.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:36 smithi002 ceph-mon[31538]: pgmap v30: 97 pgs: 97 active+clean; 933 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 8.1 MiB/s wr, 673 op/s 2024-05-14T16:55:38.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:38 smithi151 ceph-mon[36066]: pgmap v31: 97 pgs: 97 active+clean; 952 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 12 MiB/s wr, 971 op/s 2024-05-14T16:55:38.868 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:38 smithi002 ceph-mon[31538]: pgmap v31: 97 pgs: 97 active+clean; 952 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 12 MiB/s wr, 971 op/s 2024-05-14T16:55:38.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:38 smithi123 ceph-mon[35971]: pgmap v31: 97 pgs: 97 active+clean; 952 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 12 MiB/s wr, 971 op/s 2024-05-14T16:55:39.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.509+0000 7f90f3c83700 1 -- 172.21.15.2:0/1441872127 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90ec104030 msgr2=0x7f90ec110450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:39.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.509+0000 7f90f3c83700 1 --2- 172.21.15.2:0/1441872127 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90ec104030 0x7f90ec110450 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f90e400b0a0 tx=0x7f90e400b3b0 comp rx=0 tx=0).stop 2024-05-14T16:55:39.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.509+0000 7f90f0a1d700 1 -- 172.21.15.2:0/1441872127 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e4009480 con 0x7f90ec104030 2024-05-14T16:55:39.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.509+0000 7f90f3c83700 1 -- 172.21.15.2:0/1441872127 shutdown_connections 2024-05-14T16:55:39.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.509+0000 7f90f3c83700 1 --2- 172.21.15.2:0/1441872127 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90ec104030 0x7f90ec110450 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:39.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.509+0000 7f90f3c83700 1 --2- 172.21.15.2:0/1441872127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90ec1084e0 0x7f90ec1088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:39.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.509+0000 7f90f3c83700 1 --2- 172.21.15.2:0/1441872127 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90ec103690 0x7f90ec103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:39.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.509+0000 7f90f3c83700 1 -- 172.21.15.2:0/1441872127 >> 172.21.15.2:0/1441872127 conn(0x7f90ec07a730 msgr2=0x7f90ec07ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:39.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.510+0000 7f90f3c83700 1 -- 172.21.15.2:0/1441872127 shutdown_connections 2024-05-14T16:55:39.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.510+0000 7f90f3c83700 1 -- 172.21.15.2:0/1441872127 wait complete. 2024-05-14T16:55:39.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.510+0000 7f90f3c83700 1 Processor -- start 2024-05-14T16:55:39.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.511+0000 7f90f3c83700 1 -- start start 2024-05-14T16:55:39.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.511+0000 7f90f3c83700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90ec103690 0x7f90ec074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:39.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.511+0000 7f90f3c83700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90ec104030 0x7f90ec075040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:39.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.511+0000 7f90f3c83700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90ec1084e0 0x7f90ec06dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:39.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.511+0000 7f90f3c83700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f90ec1137b0 con 0x7f90ec1084e0 2024-05-14T16:55:39.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.511+0000 7f90f3c83700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f90ec113630 con 0x7f90ec103690 2024-05-14T16:55:39.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.511+0000 7f90f3c83700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f90ec113930 con 0x7f90ec104030 2024-05-14T16:55:39.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.511+0000 7f90f2220700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90ec1084e0 0x7f90ec06dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:39.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.511+0000 7f90f121e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90ec104030 0x7f90ec075040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:39.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.512+0000 7f90f2220700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90ec1084e0 0x7f90ec06dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34522/0 (socket says 172.21.15.2:34522) 2024-05-14T16:55:39.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.512+0000 7f90f121e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90ec104030 0x7f90ec075040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:60732/0 (socket says 172.21.15.2:60732) 2024-05-14T16:55:39.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.512+0000 7f90f2220700 1 -- 172.21.15.2:0/2760429106 learned_addr learned my addr 172.21.15.2:0/2760429106 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:39.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.512+0000 7f90f1a1f700 1 --2- 172.21.15.2:0/2760429106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90ec103690 0x7f90ec074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:39.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.512+0000 7f90f2220700 1 -- 172.21.15.2:0/2760429106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90ec104030 msgr2=0x7f90ec075040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:39.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.512+0000 7f90f2220700 1 --2- 172.21.15.2:0/2760429106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90ec104030 0x7f90ec075040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:39.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.512+0000 7f90f2220700 1 -- 172.21.15.2:0/2760429106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90ec103690 msgr2=0x7f90ec074b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:39.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.512+0000 7f90f2220700 1 --2- 172.21.15.2:0/2760429106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90ec103690 0x7f90ec074b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:39.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.512+0000 7f90f2220700 1 -- 172.21.15.2:0/2760429106 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90ec06e4b0 con 0x7f90ec1084e0 2024-05-14T16:55:39.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.512+0000 7f90f1a1f700 1 --2- 172.21.15.2:0/2760429106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90ec103690 0x7f90ec074b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:55:39.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.512+0000 7f90f121e700 1 --2- 172.21.15.2:0/2760429106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90ec104030 0x7f90ec075040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:55:39.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.512+0000 7f90f2220700 1 --2- 172.21.15.2:0/2760429106 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90ec1084e0 0x7f90ec06dbb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f90e40062a0 tx=0x7f90e4004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:39.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.513+0000 7f90e2ffd700 1 -- 172.21.15.2:0/2760429106 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e4021070 con 0x7f90ec1084e0 2024-05-14T16:55:39.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.513+0000 7f90f3c83700 1 -- 172.21.15.2:0/2760429106 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90ec06e740 con 0x7f90ec1084e0 2024-05-14T16:55:39.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.513+0000 7f90e2ffd700 1 -- 172.21.15.2:0/2760429106 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90e4007e30 con 0x7f90ec1084e0 2024-05-14T16:55:39.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.513+0000 7f90f3c83700 1 -- 172.21.15.2:0/2760429106 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90ec072800 con 0x7f90ec1084e0 2024-05-14T16:55:39.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.513+0000 7f90e2ffd700 1 -- 172.21.15.2:0/2760429106 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e4021070 con 0x7f90ec1084e0 2024-05-14T16:55:39.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.516+0000 7f90e2ffd700 1 -- 172.21.15.2:0/2760429106 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) ==== 149884+0+0 (secure 0 0 0) 0x7f90e4007980 con 0x7f90ec1084e0 2024-05-14T16:55:39.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.517+0000 7f90e2ffd700 1 --2- 172.21.15.2:0/2760429106 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f90d80b2e10 0x7f90d80b52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:39.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.517+0000 7f90e2ffd700 1 -- 172.21.15.2:0/2760429106 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f90e400f020 con 0x7f90ec1084e0 2024-05-14T16:55:39.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.517+0000 7f90f1a1f700 1 --2- 172.21.15.2:0/2760429106 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f90d80b2e10 0x7f90d80b52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:39.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.517+0000 7f90fe519b80 1 Processor -- start 2024-05-14T16:55:39.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.517+0000 7f90fe519b80 1 -- start start 2024-05-14T16:55:39.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.517+0000 7f90f1a1f700 1 --2- 172.21.15.2:0/2760429106 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f90d80b2e10 0x7f90d80b52d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f90ec1046c0 tx=0x7f90e8009260 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:39.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.518+0000 7f90fe519b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55800c2f3e30 0x55800c2f4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:39.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.518+0000 7f90fe519b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55800c2f4750 0x55800c2ee540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:39.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.518+0000 7f90fe519b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55800c2eeb60 0x55800c2eefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:39.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.518+0000 7f90fe519b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55800c24ce20 con 0x55800c2f4750 2024-05-14T16:55:39.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.518+0000 7f90fe519b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55800c23c9c0 con 0x55800c2f3e30 2024-05-14T16:55:39.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.518+0000 7f90fe519b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55800c240a00 con 0x55800c2eeb60 2024-05-14T16:55:39.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.518+0000 7f90f1a1f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55800c2f3e30 0x55800c2f4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:39.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.518+0000 7f90f2220700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55800c2eeb60 0x55800c2eefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:39.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.518+0000 7f90f2220700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55800c2eeb60 0x55800c2eefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:60738/0 (socket says 172.21.15.2:60738) 2024-05-14T16:55:39.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.519+0000 7f90f2220700 1 -- 172.21.15.2:0/4168845439 learned_addr learned my addr 172.21.15.2:0/4168845439 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:39.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.519+0000 7f90f121e700 1 --2- 172.21.15.2:0/4168845439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55800c2f4750 0x55800c2ee540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:39.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.519+0000 7f90f2220700 1 -- 172.21.15.2:0/4168845439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55800c2f3e30 msgr2=0x55800c2f4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:39.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.519+0000 7f90f2220700 1 --2- 172.21.15.2:0/4168845439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55800c2f3e30 0x55800c2f4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:39.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.519+0000 7f90f2220700 1 -- 172.21.15.2:0/4168845439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55800c2f4750 msgr2=0x55800c2ee540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:39.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.519+0000 7f90f2220700 1 --2- 172.21.15.2:0/4168845439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55800c2f4750 0x55800c2ee540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:39.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.519+0000 7f90f2220700 1 -- 172.21.15.2:0/4168845439 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55800c18ca60 con 0x55800c2eeb60 2024-05-14T16:55:39.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.520+0000 7f90f2220700 1 --2- 172.21.15.2:0/4168845439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55800c2eeb60 0x55800c2eefe0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f90e40e1020 tx=0x7f90e40097c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92fae055dd533d2b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:39.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.520+0000 7f90d7fff700 1 -- 172.21.15.2:0/4168845439 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e4007c70 con 0x55800c2eeb60 2024-05-14T16:55:39.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.520+0000 7f90d7fff700 1 -- 172.21.15.2:0/4168845439 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90e40a6050 con 0x55800c2eeb60 2024-05-14T16:55:39.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.520+0000 7f90d7fff700 1 -- 172.21.15.2:0/4168845439 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e40b3830 con 0x55800c2eeb60 2024-05-14T16:55:39.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.521+0000 7f90fe519b80 1 -- 172.21.15.2:0/4168845439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55800c2eeb60 msgr2=0x55800c2eefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:39.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.521+0000 7f90fe519b80 1 --2- 172.21.15.2:0/4168845439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55800c2eeb60 0x55800c2eefe0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f90e40e1020 tx=0x7f90e40097c0 comp rx=0 tx=0).stop 2024-05-14T16:55:39.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.521+0000 7f90fe519b80 1 -- 172.21.15.2:0/4168845439 shutdown_connections 2024-05-14T16:55:39.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.521+0000 7f90fe519b80 1 --2- 172.21.15.2:0/4168845439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55800c2eeb60 0x55800c2eefe0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:39.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.521+0000 7f90fe519b80 1 --2- 172.21.15.2:0/4168845439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55800c2f3e30 0x55800c2f4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:39.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.521+0000 7f90fe519b80 1 --2- 172.21.15.2:0/4168845439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55800c2f4750 0x55800c2ee540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:39.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.521+0000 7f90fe519b80 1 -- 172.21.15.2:0/4168845439 >> 172.21.15.2:0/4168845439 conn(0x55800c2193e0 msgr2=0x55800c2e8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:39.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.521+0000 7f90fe519b80 1 -- 172.21.15.2:0/4168845439 shutdown_connections 2024-05-14T16:55:39.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.521+0000 7f90fe519b80 1 -- 172.21.15.2:0/4168845439 wait complete. 2024-05-14T16:55:39.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.522+0000 7f90fe519b80 1 -- start start 2024-05-14T16:55:39.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.522+0000 7f90fe519b80 1 Processor -- start 2024-05-14T16:55:39.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.523+0000 7f90fe519b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55800c2eeb60 0x55800c2fa500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:39.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.523+0000 7f90fe519b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55800c2f3e30 0x55800c2f78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:39.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.523+0000 7f90fe519b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55800c2f4750 0x55800c2f7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:39.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.523+0000 7f90f1a1f700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55800c2eeb60 0x55800c2fa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:39.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.523+0000 7f90f1a1f700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55800c2eeb60 0x55800c2fa500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:60742/0 (socket says 172.21.15.2:60742) 2024-05-14T16:55:39.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.523+0000 7f90f121e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55800c2f3e30 0x55800c2f78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:39.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.523+0000 7f90f121e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55800c2f3e30 0x55800c2f78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34536/0 (socket says 172.21.15.2:34536) 2024-05-14T16:55:39.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.523+0000 7f90fe519b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55800c143130 con 0x55800c2f3e30 2024-05-14T16:55:39.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.524+0000 7f90f1a1f700 1 -- 172.21.15.2:0/3828555131 learned_addr learned my addr 172.21.15.2:0/3828555131 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:39.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.524+0000 7f90f2220700 1 --2- 172.21.15.2:0/3828555131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55800c2f4750 0x55800c2f7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:39.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.524+0000 7f90fe519b80 1 -- 172.21.15.2:0/3828555131 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55800c1de950 con 0x55800c2f4750 2024-05-14T16:55:39.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.524+0000 7f90fe519b80 1 -- 172.21.15.2:0/3828555131 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55800c2faf50 con 0x55800c2eeb60 2024-05-14T16:55:39.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.524+0000 7f90fe519b80 10 client.0 _ll_register_callbacks cb 0x55800c1d3710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:55:39.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.525+0000 7f90f2220700 1 -- 172.21.15.2:0/3828555131 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55800c2eeb60 msgr2=0x55800c2fa500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:39.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.525+0000 7f90f2220700 1 --2- 172.21.15.2:0/3828555131 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55800c2eeb60 0x55800c2fa500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:39.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.525+0000 7f90f2220700 1 -- 172.21.15.2:0/3828555131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55800c2f3e30 msgr2=0x55800c2f78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:39.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.525+0000 7f90f2220700 1 --2- 172.21.15.2:0/3828555131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55800c2f3e30 0x55800c2f78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:39.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.525+0000 7f90f2220700 1 -- 172.21.15.2:0/3828555131 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55800c1db820 con 0x55800c2f4750 2024-05-14T16:55:39.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.525+0000 7f90f2220700 1 --2- 172.21.15.2:0/3828555131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55800c2f4750 0x55800c2f7de0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x55800c2ef520 tx=0x7f90e40aa570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cba4cd5232b7b114 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:39.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.525+0000 7f90d57fa700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T16:55:39.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.526+0000 7f90d57fa700 1 -- 172.21.15.2:0/3828555131 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e4007c70 con 0x55800c2f4750 2024-05-14T16:55:39.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.526+0000 7f90d57fa700 1 -- 172.21.15.2:0/3828555131 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f90e40b33a0 con 0x55800c2f4750 2024-05-14T16:55:39.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.526+0000 7f90fe519b80 1 -- 172.21.15.2:0/3828555131 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90e40a6050 con 0x55800c2f4750 2024-05-14T16:55:39.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.527+0000 7f90d57fa700 1 -- 172.21.15.2:0/3828555131 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f90e40ab160 con 0x55800c2f4750 2024-05-14T16:55:39.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.527+0000 7f90fe519b80 10 client.25145 fetch_fsmap learned FSMap version 28 2024-05-14T16:55:39.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.527+0000 7f90fe519b80 1 -- 172.21.15.2:0/3828555131 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55800c305640 con 0x55800c2f4750 2024-05-14T16:55:39.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.527+0000 7f90d57fa700 1 -- 172.21.15.2:0/3828555131 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90e40b3830 con 0x55800c2f4750 2024-05-14T16:55:39.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.528+0000 7f90d57fa700 1 -- 172.21.15.2:0/3828555131 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 28) ==== 8350+0+0 (secure 0 0 0) 0x7f90e4011070 con 0x55800c2f4750 2024-05-14T16:55:39.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.528+0000 7f90fe519b80 10 client.25145 fetch_fsmap finished waiting for FSMap version 28 2024-05-14T16:55:39.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.528+0000 7f90fe519b80 10 client.25145 resolve_mds: validated gid 24479 aka daemon mds.b 2024-05-14T16:55:39.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.528+0000 7f90fe519b80 20 client.25145 populate_metadata read hostname 'smithi002' 2024-05-14T16:55:39.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.528+0000 7f90fe519b80 1 --2- 172.21.15.2:0/3828555131 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55800c307940 0x55800c309d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:39.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.528+0000 7f90fe519b80 4 client.25145 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:55:39.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.528+0000 7f90fe519b80 1 -- 172.21.15.2:0/3828555131 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f90e40a6050 con 0x55800c307940 2024-05-14T16:55:39.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.528+0000 7f90f1a1f700 1 --2- 172.21.15.2:0/3828555131 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55800c307940 0x55800c309d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:39.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.529+0000 7f90d57fa700 10 client.25145 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:55:39.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.529+0000 7f90f1a1f700 1 --2- 172.21.15.2:0/3828555131 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55800c307940 0x55800c309d30 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fa43b4a6fd20e35 server_cookie=c5c295c6bce73a38 in_seq=0 out_seq=0 2024-05-14T16:55:39.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.530+0000 7f90d57fa700 1 -- 172.21.15.2:0/3828555131 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f90e40a6050 con 0x55800c307940 2024-05-14T16:55:39.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.530+0000 7f90d57fa700 10 client.25145 handle_command_reply: tid=0 2024-05-14T16:55:39.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.530+0000 7f90fe519b80 1 client.25145 shutdown 2024-05-14T16:55:39.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.530+0000 7f90fe519b80 1 -- 172.21.15.2:0/3828555131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55800c2f4750 msgr2=0x55800c2f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:39.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.530+0000 7f90fe519b80 1 --2- 172.21.15.2:0/3828555131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55800c2f4750 0x55800c2f7de0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x55800c2ef520 tx=0x7f90e40aa570 comp rx=0 tx=0).stop 2024-05-14T16:55:39.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.530+0000 7f90fe519b80 1 -- 172.21.15.2:0/3828555131 shutdown_connections 2024-05-14T16:55:39.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.530+0000 7f90fe519b80 1 --2- 172.21.15.2:0/3828555131 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55800c307940 0x55800c309d30 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:39.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.530+0000 7f90fe519b80 1 --2- 172.21.15.2:0/3828555131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55800c2f4750 0x55800c2f7de0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:39.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.530+0000 7f90fe519b80 1 --2- 172.21.15.2:0/3828555131 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55800c2eeb60 0x55800c2fa500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:39.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.530+0000 7f90fe519b80 1 --2- 172.21.15.2:0/3828555131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55800c2f3e30 0x55800c2f78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:39.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.530+0000 7f90d57fa700 0 client.25145 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:55:39.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.530+0000 7f90fe519b80 1 -- 172.21.15.2:0/3828555131 >> 172.21.15.2:0/3828555131 conn(0x55800c2193e0 msgr2=0x55800c2e3860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:39.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.530+0000 7f90fe519b80 1 -- 172.21.15.2:0/3828555131 shutdown_connections 2024-05-14T16:55:39.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.530+0000 7f90fe519b80 1 -- 172.21.15.2:0/3828555131 wait complete. 2024-05-14T16:55:39.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.531+0000 7f90fe519b80 20 client.25145 trim_cache size 0 max 16384 2024-05-14T16:55:39.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.565+0000 7f90fe519b80 1 Processor -- start 2024-05-14T16:55:39.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.565+0000 7f90fe519b80 1 -- start start 2024-05-14T16:55:39.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.566+0000 7f90fe519b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55800c2eeb60 0x55800c2f2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:39.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.566+0000 7f90fe519b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55800c2f3e30 0x55800c2f6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:39.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.566+0000 7f90fe519b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55800c2f4750 0x55800c303e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:39.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.566+0000 7f90fe519b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f90ec1d0350 con 0x55800c2f4750 2024-05-14T16:55:39.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.566+0000 7f90fe519b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f90ec1cdbb0 con 0x55800c2f3e30 2024-05-14T16:55:39.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.566+0000 7f90fe519b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f90ec1a99e0 con 0x55800c2eeb60 2024-05-14T16:55:39.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.566+0000 7f90f1a1f700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55800c2eeb60 0x55800c2f2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:39.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.566+0000 7f90f1a1f700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55800c2eeb60 0x55800c2f2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:60756/0 (socket says 172.21.15.2:60756) 2024-05-14T16:55:39.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.566+0000 7f90f1a1f700 1 -- 172.21.15.2:0/4223290336 learned_addr learned my addr 172.21.15.2:0/4223290336 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:39.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.566+0000 7f90f2220700 1 --2- 172.21.15.2:0/4223290336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55800c2f4750 0x55800c303e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:39.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.566+0000 7f90f121e700 1 --2- 172.21.15.2:0/4223290336 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55800c2f3e30 0x55800c2f6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:39.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.567+0000 7f90f1a1f700 1 -- 172.21.15.2:0/4223290336 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55800c2f3e30 msgr2=0x55800c2f6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:39.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.567+0000 7f90f1a1f700 1 --2- 172.21.15.2:0/4223290336 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55800c2f3e30 0x55800c2f6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:39.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.567+0000 7f90f1a1f700 1 -- 172.21.15.2:0/4223290336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55800c2f4750 msgr2=0x55800c303e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:39.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.567+0000 7f90f1a1f700 1 --2- 172.21.15.2:0/4223290336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55800c2f4750 0x55800c303e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:39.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.567+0000 7f90f1a1f700 1 -- 172.21.15.2:0/4223290336 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55800c2f4460 con 0x55800c2eeb60 2024-05-14T16:55:39.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.568+0000 7f90f1a1f700 1 --2- 172.21.15.2:0/4223290336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55800c2eeb60 0x55800c2f2050 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f90e8013010 tx=0x7f90e8014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e8ea8fd15b7553f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:39.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.568+0000 7f90d77fe700 1 -- 172.21.15.2:0/4223290336 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e8004230 con 0x55800c2eeb60 2024-05-14T16:55:39.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.569+0000 7f90d77fe700 1 -- 172.21.15.2:0/4223290336 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90e8004ce0 con 0x55800c2eeb60 2024-05-14T16:55:39.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.570+0000 7f90fe519b80 1 -- 172.21.15.2:0/4223290336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55800c2eeb60 msgr2=0x55800c2f2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:39.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.570+0000 7f90fe519b80 1 --2- 172.21.15.2:0/4223290336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55800c2eeb60 0x55800c2f2050 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f90e8013010 tx=0x7f90e8014000 comp rx=0 tx=0).stop 2024-05-14T16:55:39.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.571+0000 7f90fe519b80 1 -- 172.21.15.2:0/4223290336 shutdown_connections 2024-05-14T16:55:39.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.571+0000 7f90fe519b80 1 --2- 172.21.15.2:0/4223290336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55800c2f4750 0x55800c303e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:39.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.571+0000 7f90fe519b80 1 --2- 172.21.15.2:0/4223290336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55800c2eeb60 0x55800c2f2050 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:39.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.571+0000 7f90fe519b80 1 --2- 172.21.15.2:0/4223290336 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55800c2f3e30 0x55800c2f6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:39.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.571+0000 7f90fe519b80 1 -- 172.21.15.2:0/4223290336 >> 172.21.15.2:0/4223290336 conn(0x55800c2193e0 msgr2=0x55800c2f32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:39.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.571+0000 7f90fe519b80 1 -- 172.21.15.2:0/4223290336 shutdown_connections 2024-05-14T16:55:39.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.571+0000 7f90fe519b80 1 -- 172.21.15.2:0/4223290336 wait complete. 2024-05-14T16:55:39.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.572+0000 7f90fe519b80 1 -- start start 2024-05-14T16:55:39.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.572+0000 7f90fe519b80 1 Processor -- start 2024-05-14T16:55:39.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.573+0000 7f90fe519b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55800c2eeb60 0x55800c300460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:39.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.573+0000 7f90fe519b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55800c2f3e30 0x55800c2fd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:39.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.573+0000 7f90fe519b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55800c2f4750 0x55800c2fd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:39.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.573+0000 7f90fe519b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f90ec1cc070 con 0x55800c2f3e30 2024-05-14T16:55:39.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.573+0000 7f90fe519b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f90ec1cd4e0 con 0x55800c2f4750 2024-05-14T16:55:39.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.573+0000 7f90fe519b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f90ec1cc740 con 0x55800c2eeb60 2024-05-14T16:55:39.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.573+0000 7f90fe519b80 10 client.0 _ll_register_callbacks cb 0x55800c1d3710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:55:39.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.573+0000 7f90f1a1f700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55800c2eeb60 0x55800c300460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:39.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.573+0000 7f90f1a1f700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55800c2eeb60 0x55800c300460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:60772/0 (socket says 172.21.15.2:60772) 2024-05-14T16:55:39.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.573+0000 7f90f1a1f700 1 -- 172.21.15.2:0/24927043 learned_addr learned my addr 172.21.15.2:0/24927043 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:39.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.574+0000 7f90f2220700 1 --2- 172.21.15.2:0/24927043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55800c2f4750 0x55800c2fd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:39.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.574+0000 7f90f1a1f700 1 -- 172.21.15.2:0/24927043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55800c2f4750 msgr2=0x55800c2fd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:39.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.574+0000 7f90f1a1f700 1 --2- 172.21.15.2:0/24927043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55800c2f4750 0x55800c2fd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:39.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.574+0000 7f90f1a1f700 1 -- 172.21.15.2:0/24927043 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55800c2f3e30 msgr2=0x55800c2fd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:39.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.574+0000 7f90f1a1f700 1 --2- 172.21.15.2:0/24927043 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55800c2f3e30 0x55800c2fd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:39.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.574+0000 7f90f1a1f700 1 -- 172.21.15.2:0/24927043 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55800c2fdbf0 con 0x55800c2eeb60 2024-05-14T16:55:39.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.575+0000 7f90f1a1f700 1 --2- 172.21.15.2:0/24927043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55800c2eeb60 0x55800c300460 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f90e800e6d0 tx=0x7f90e802a520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75064e650df2187a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:39.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.577+0000 7f90d57fa700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-14T16:55:39.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.577+0000 7f90d57fa700 1 -- 172.21.15.2:0/24927043 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e8039020 con 0x55800c2eeb60 2024-05-14T16:55:39.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.577+0000 7f90d57fa700 1 -- 172.21.15.2:0/24927043 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f90e80049b0 con 0x55800c2eeb60 2024-05-14T16:55:39.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.578+0000 7f90fe519b80 1 -- 172.21.15.2:0/24927043 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90e8004ce0 con 0x55800c2eeb60 2024-05-14T16:55:39.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.580+0000 7f90d57fa700 1 -- 172.21.15.2:0/24927043 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f90e8004e90 con 0x55800c2eeb60 2024-05-14T16:55:39.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.581+0000 7f90fe519b80 10 client.25144 fetch_fsmap learned FSMap version 28 2024-05-14T16:55:39.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.581+0000 7f90fe519b80 1 -- 172.21.15.2:0/24927043 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55800c372e60 con 0x55800c2eeb60 2024-05-14T16:55:39.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.582+0000 7f90d57fa700 1 -- 172.21.15.2:0/24927043 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90e8004ce0 con 0x55800c2eeb60 2024-05-14T16:55:39.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.583+0000 7f90d57fa700 1 -- 172.21.15.2:0/24927043 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 28) ==== 8350+0+0 (secure 0 0 0) 0x7f90e8046020 con 0x55800c2eeb60 2024-05-14T16:55:39.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.583+0000 7f90fe519b80 10 client.25144 fetch_fsmap finished waiting for FSMap version 28 2024-05-14T16:55:39.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.583+0000 7f90fe519b80 10 client.25144 resolve_mds: validated gid 24479 aka daemon mds.b 2024-05-14T16:55:39.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.583+0000 7f90fe519b80 20 client.25144 populate_metadata read hostname 'smithi002' 2024-05-14T16:55:39.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.583+0000 7f90fe519b80 1 --2- 172.21.15.2:0/24927043 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55800c3735e0 0x55800c375aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:39.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.583+0000 7f90fe519b80 4 client.25144 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 135.2, "await_for": 45.0}] 2024-05-14T16:55:39.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.583+0000 7f90fe519b80 1 -- 172.21.15.2:0/24927043 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 135.2, "await_for": 45.0}) -- 0x55800c2ba4f0 con 0x55800c3735e0 2024-05-14T16:55:39.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.583+0000 7f90f121e700 1 --2- 172.21.15.2:0/24927043 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55800c3735e0 0x55800c375aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:39.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.584+0000 7f90f121e700 1 --2- 172.21.15.2:0/24927043 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55800c3735e0 0x55800c375aa0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58d24d97ca8097a5 server_cookie=187beb177d1f560f in_seq=0 out_seq=0 2024-05-14T16:55:39.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:39.584+0000 7f90d57fa700 10 client.25144 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:55:40.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:40.713+0000 7f90d57fa700 1 -- 172.21.15.2:0/24927043 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55800c2ba4f0 con 0x55800c3735e0 2024-05-14T16:55:40.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:40.713+0000 7f90d57fa700 10 client.25144 handle_command_reply: tid=0 2024-05-14T16:55:40.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:40.713+0000 7f90fe519b80 1 client.25144 shutdown 2024-05-14T16:55:40.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:40.713+0000 7f90fe519b80 1 -- 172.21.15.2:0/24927043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55800c2eeb60 msgr2=0x55800c300460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:40.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:40.713+0000 7f90fe519b80 1 --2- 172.21.15.2:0/24927043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55800c2eeb60 0x55800c300460 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f90e800e6d0 tx=0x7f90e802a520 comp rx=0 tx=0).stop 2024-05-14T16:55:40.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:40.713+0000 7f90fe519b80 1 -- 172.21.15.2:0/24927043 shutdown_connections 2024-05-14T16:55:40.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:40.713+0000 7f90fe519b80 1 --2- 172.21.15.2:0/24927043 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55800c3735e0 0x55800c375aa0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:40.713+0000 7f90fe519b80 1 --2- 172.21.15.2:0/24927043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55800c2f4750 0x55800c2fd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:40.713+0000 7f90fe519b80 1 --2- 172.21.15.2:0/24927043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55800c2eeb60 0x55800c300460 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:40.713+0000 7f90d57fa700 0 client.25144 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:55:40.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:40.713+0000 7f90fe519b80 1 --2- 172.21.15.2:0/24927043 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55800c2f3e30 0x55800c2fd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:40.713+0000 7f90fe519b80 1 -- 172.21.15.2:0/24927043 >> 172.21.15.2:0/24927043 conn(0x55800c2193e0 msgr2=0x55800c2f32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:40.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:40.713+0000 7f90fe519b80 1 -- 172.21.15.2:0/24927043 shutdown_connections 2024-05-14T16:55:40.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:40.713+0000 7f90fe519b80 1 -- 172.21.15.2:0/24927043 wait complete. 2024-05-14T16:55:40.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:40.713+0000 7f90fe519b80 20 client.25144 trim_cache size 0 max 16384 2024-05-14T16:55:40.716 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T16:55:40.716 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 28, 2024-05-14T16:55:40.716 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24479, 2024-05-14T16:55:40.716 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 2, 2024-05-14T16:55:40.716 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-05-14T16:55:40.716 INFO:teuthology.orchestra.run.smithi002.stdout: "822ccb9f": { 2024-05-14T16:55:40.716 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 2, 2024-05-14T16:55:40.716 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-05-14T16:55:40.716 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-05-14T16:55:40.716 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-05-14T16:55:40.716 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-05-14T16:55:40.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T16:55:40.717 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-05-14T16:55:40.717 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 135.2, 2024-05-14T16:55:40.717 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-05-14T16:55:40.717 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-05-14T16:55:40.717 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-05-14T16:55:40.717 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-05-14T16:55:40.717 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-05-14T16:55:40.717 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-05-14T16:55:40.717 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:55:40.717 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:55:40.717 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:55:40.717 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:55:40.717 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:55:40.717 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T16:55:40.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:40.714+0000 7f90f3c83700 1 -- 172.21.15.2:0/2760429106 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f90d80b2e10 msgr2=0x7f90d80b52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:40.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:40.714+0000 7f90f3c83700 1 --2- 172.21.15.2:0/2760429106 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f90d80b2e10 0x7f90d80b52d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f90ec1046c0 tx=0x7f90e8009260 comp rx=0 tx=0).stop 2024-05-14T16:55:40.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:40.714+0000 7f90f3c83700 1 -- 172.21.15.2:0/2760429106 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90ec1084e0 msgr2=0x7f90ec06dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:40.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:40.714+0000 7f90f3c83700 1 --2- 172.21.15.2:0/2760429106 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90ec1084e0 0x7f90ec06dbb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f90e40062a0 tx=0x7f90e4004c80 comp rx=0 tx=0).stop 2024-05-14T16:55:40.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:40.714+0000 7f90f3c83700 1 -- 172.21.15.2:0/2760429106 shutdown_connections 2024-05-14T16:55:40.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:40.714+0000 7f90f3c83700 1 --2- 172.21.15.2:0/2760429106 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f90d80b2e10 0x7f90d80b52d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:40.714+0000 7f90f3c83700 1 --2- 172.21.15.2:0/2760429106 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90ec1084e0 0x7f90ec06dbb0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:40.714+0000 7f90f3c83700 1 --2- 172.21.15.2:0/2760429106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90ec103690 0x7f90ec074b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:40.714+0000 7f90f3c83700 1 --2- 172.21.15.2:0/2760429106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90ec104030 0x7f90ec075040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:40.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:40.714+0000 7f90f3c83700 1 -- 172.21.15.2:0/2760429106 >> 172.21.15.2:0/2760429106 conn(0x7f90ec07a730 msgr2=0x7f90ec101560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:40.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:40.714+0000 7f90f3c83700 1 -- 172.21.15.2:0/2760429106 shutdown_connections 2024-05-14T16:55:40.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:40.714+0000 7f90f3c83700 1 -- 172.21.15.2:0/2760429106 wait complete. 2024-05-14T16:55:40.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:40 smithi151 ceph-mon[36066]: pgmap v32: 97 pgs: 97 active+clean; 953 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.8 MiB/s wr, 683 op/s 2024-05-14T16:55:40.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:40 smithi123 ceph-mon[35971]: pgmap v32: 97 pgs: 97 active+clean; 953 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.8 MiB/s wr, 683 op/s 2024-05-14T16:55:40.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:40 smithi002 ceph-mon[31538]: pgmap v32: 97 pgs: 97 active+clean; 953 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.8 MiB/s wr, 683 op/s 2024-05-14T16:55:41.674 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '822ccb9f', quiesce took 10.8 seconds. Will release after: 4.399999999999999 2024-05-14T16:55:41.748 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 25.0 s, 1761.9 tps, lat 39.398 ms stddev 68.067 2024-05-14T16:55:42.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:42 smithi151 ceph-mon[36066]: pgmap v33: 97 pgs: 97 active+clean; 953 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.9 MiB/s wr, 649 op/s 2024-05-14T16:55:42.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:42 smithi123 ceph-mon[35971]: pgmap v33: 97 pgs: 97 active+clean; 953 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.9 MiB/s wr, 649 op/s 2024-05-14T16:55:42.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:42 smithi002 ceph-mon[31538]: pgmap v33: 97 pgs: 97 active+clean; 953 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.9 MiB/s wr, 649 op/s 2024-05-14T16:55:44.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:44 smithi151 ceph-mon[36066]: pgmap v34: 97 pgs: 97 active+clean; 958 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 21 MiB/s wr, 1.01k op/s 2024-05-14T16:55:44.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:44 smithi123 ceph-mon[35971]: pgmap v34: 97 pgs: 97 active+clean; 958 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 21 MiB/s wr, 1.01k op/s 2024-05-14T16:55:44.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:44 smithi002 ceph-mon[31538]: pgmap v34: 97 pgs: 97 active+clean; 958 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 21 MiB/s wr, 1.01k op/s 2024-05-14T16:55:46.076 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:55:46.340 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:55:46.714 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:46 smithi002 ceph-mon[31538]: pgmap v35: 97 pgs: 97 active+clean; 958 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 17 MiB/s wr, 679 op/s 2024-05-14T16:55:46.751 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 30.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:55:46.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:46 smithi151 ceph-mon[36066]: pgmap v35: 97 pgs: 97 active+clean; 958 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 17 MiB/s wr, 679 op/s 2024-05-14T16:55:46.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:46 smithi123 ceph-mon[35971]: pgmap v35: 97 pgs: 97 active+clean; 958 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 17 MiB/s wr, 679 op/s 2024-05-14T16:55:48.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.197+0000 7f79b9885700 1 -- 172.21.15.2:0/1472786862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79b406b250 msgr2=0x7f79b406b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:48.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.197+0000 7f79b9885700 1 --2- 172.21.15.2:0/1472786862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79b406b250 0x7f79b406b630 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f79a4009990 tx=0x7f79a4009ca0 comp rx=0 tx=0).stop 2024-05-14T16:55:48.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.197+0000 7f79b9885700 1 -- 172.21.15.2:0/1472786862 shutdown_connections 2024-05-14T16:55:48.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.197+0000 7f79b9885700 1 --2- 172.21.15.2:0/1472786862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79b410fff0 0x7f79b41123e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:48.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.197+0000 7f79b9885700 1 --2- 172.21.15.2:0/1472786862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79b406b250 0x7f79b406b630 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:48.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.197+0000 7f79b9885700 1 --2- 172.21.15.2:0/1472786862 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79b406bb70 0x7f79b410fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:48.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.197+0000 7f79b9885700 1 -- 172.21.15.2:0/1472786862 >> 172.21.15.2:0/1472786862 conn(0x7f79b40793f0 msgr2=0x7f79b4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:48.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.197+0000 7f79b9885700 1 -- 172.21.15.2:0/1472786862 shutdown_connections 2024-05-14T16:55:48.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.197+0000 7f79b9885700 1 -- 172.21.15.2:0/1472786862 wait complete. 2024-05-14T16:55:48.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.198+0000 7f79b9885700 1 Processor -- start 2024-05-14T16:55:48.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.198+0000 7f79b9885700 1 -- start start 2024-05-14T16:55:48.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.198+0000 7f79b9885700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79b406b250 0x7f79b419be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:48.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.198+0000 7f79b9885700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79b406bb70 0x7f79b419c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:48.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.198+0000 7f79b9885700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79b410fff0 0x7f79b41a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:48.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.198+0000 7f79b9885700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f79b4115910 con 0x7f79b406bb70 2024-05-14T16:55:48.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.199+0000 7f79b9885700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f79b4115790 con 0x7f79b406b250 2024-05-14T16:55:48.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.199+0000 7f79b9885700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f79b4115a90 con 0x7f79b410fff0 2024-05-14T16:55:48.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.199+0000 7f79b27fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79b406bb70 0x7f79b419c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:48.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.199+0000 7f79b27fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79b406bb70 0x7f79b419c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34794/0 (socket says 172.21.15.2:34794) 2024-05-14T16:55:48.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.199+0000 7f79b27fc700 1 -- 172.21.15.2:0/256207629 learned_addr learned my addr 172.21.15.2:0/256207629 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:48.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.199+0000 7f79b2ffd700 1 --2- 172.21.15.2:0/256207629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79b406b250 0x7f79b419be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:48.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.199+0000 7f79b37fe700 1 --2- 172.21.15.2:0/256207629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79b410fff0 0x7f79b41a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:48.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.199+0000 7f79b27fc700 1 -- 172.21.15.2:0/256207629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79b410fff0 msgr2=0x7f79b41a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:48.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.199+0000 7f79b27fc700 1 --2- 172.21.15.2:0/256207629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79b410fff0 0x7f79b41a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:48.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.199+0000 7f79b27fc700 1 -- 172.21.15.2:0/256207629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79b406b250 msgr2=0x7f79b419be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:48.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.199+0000 7f79b27fc700 1 --2- 172.21.15.2:0/256207629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79b406b250 0x7f79b419be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:48.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.199+0000 7f79b27fc700 1 -- 172.21.15.2:0/256207629 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b41a0e30 con 0x7f79b406bb70 2024-05-14T16:55:48.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.199+0000 7f79b2ffd700 1 --2- 172.21.15.2:0/256207629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79b406b250 0x7f79b419be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:55:48.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.199+0000 7f79b27fc700 1 --2- 172.21.15.2:0/256207629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79b406bb70 0x7f79b419c360 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f79a800e9c0 tx=0x7f79a800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:48.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.199+0000 7f79b8883700 1 -- 172.21.15.2:0/256207629 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a800cbd0 con 0x7f79b406bb70 2024-05-14T16:55:48.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.199+0000 7f79b9885700 1 -- 172.21.15.2:0/256207629 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79b41a1120 con 0x7f79b406bb70 2024-05-14T16:55:48.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.199+0000 7f79b8883700 1 -- 172.21.15.2:0/256207629 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79a800ee60 con 0x7f79b406bb70 2024-05-14T16:55:48.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.200+0000 7f79b8883700 1 -- 172.21.15.2:0/256207629 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a80055c0 con 0x7f79b406bb70 2024-05-14T16:55:48.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.200+0000 7f79b9885700 1 -- 172.21.15.2:0/256207629 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79b41a89a0 con 0x7f79b406bb70 2024-05-14T16:55:48.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.201+0000 7f799a7fc700 1 -- 172.21.15.2:0/256207629 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7994005330 con 0x7f79b406bb70 2024-05-14T16:55:48.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.203+0000 7f79b8883700 1 -- 172.21.15.2:0/256207629 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) ==== 149884+0+0 (secure 0 0 0) 0x7f79a8004750 con 0x7f79b406bb70 2024-05-14T16:55:48.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.204+0000 7f79b8883700 1 --2- 172.21.15.2:0/256207629 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f799c0b2b10 0x7f799c0b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:48.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.205+0000 7f79b2ffd700 1 --2- 172.21.15.2:0/256207629 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f799c0b2b10 0x7f799c0b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:48.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.205+0000 7f79b8883700 1 -- 172.21.15.2:0/256207629 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f79a80e2d60 con 0x7f79b406bb70 2024-05-14T16:55:48.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.205+0000 7f79b2ffd700 1 --2- 172.21.15.2:0/256207629 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f799c0b2b10 0x7f799c0b4fd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f79a400b2e0 tx=0x7f79a401a040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:48.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.208+0000 7f79b8883700 1 -- 172.21.15.2:0/256207629 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f79a8054df0 con 0x7f79b406bb70 2024-05-14T16:55:48.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.476+0000 7f799a7fc700 1 -- 172.21.15.2:0/256207629 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7994005620 con 0x7f79b406bb70 2024-05-14T16:55:48.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.476+0000 7f79b8883700 1 -- 172.21.15.2:0/256207629 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11854 (secure 0 0 0) 0x7f79a80aa520 con 0x7f79b406bb70 2024-05-14T16:55:48.477 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:55:48.477 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2275696923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2275696923},{"type":"v1","addr":"172.21.15.2:6835","nonce":2275696923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/716718761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":716718761},{"type":"v1","addr":"172.21.15.2:6839","nonce":716718761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/3644815179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3644815179},{"type":"v1","addr":"172.21.15.2:6841","nonce":3644815179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1880336731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1880336731},{"type":"v1","addr":"172.21.15.151:6833","nonce":1880336731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2415605548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2415605548},{"type":"v1","addr":"172.21.15.2:6837","nonce":2415605548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2635167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2635167},{"type":"v1","addr":"172.21.15.123:6833","nonce":2635167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:51:34.106919+0000","modified":"2024-05-14T16:55:30.467925+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24455,"mds_2":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24415":{"gid":24415,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/762044339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":762044339},{"type":"v1","addr":"172.21.15.151:6835","nonce":762044339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/132043151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":132043151},{"type":"v1","addr":"172.21.15.151:6837","nonce":132043151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/4107834375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":4107834375},{"type":"v1","addr":"172.21.15.151:6839","nonce":4107834375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.123:6835/1198670013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1198670013},{"type":"v1","addr":"172.21.15.123:6835","nonce":1198670013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/333426645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":333426645},{"type":"v1","addr":"172.21.15.123:6837","nonce":333426645}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/580275877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":580275877},{"type":"v1","addr":"172.21.15.123:6839","nonce":580275877}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24431,24455,24479]},"id":1}]} 2024-05-14T16:55:48.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.478+0000 7f799a7fc700 1 -- 172.21.15.2:0/256207629 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f799c0b2b10 msgr2=0x7f799c0b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:48.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.478+0000 7f799a7fc700 1 --2- 172.21.15.2:0/256207629 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f799c0b2b10 0x7f799c0b4fd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f79a400b2e0 tx=0x7f79a401a040 comp rx=0 tx=0).stop 2024-05-14T16:55:48.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.478+0000 7f799a7fc700 1 -- 172.21.15.2:0/256207629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79b406bb70 msgr2=0x7f79b419c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:48.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.478+0000 7f799a7fc700 1 --2- 172.21.15.2:0/256207629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79b406bb70 0x7f79b419c360 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f79a800e9c0 tx=0x7f79a800ecd0 comp rx=0 tx=0).stop 2024-05-14T16:55:48.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.479+0000 7f799a7fc700 1 -- 172.21.15.2:0/256207629 shutdown_connections 2024-05-14T16:55:48.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.479+0000 7f799a7fc700 1 --2- 172.21.15.2:0/256207629 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f799c0b2b10 0x7f799c0b4fd0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:48.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.479+0000 7f799a7fc700 1 --2- 172.21.15.2:0/256207629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79b410fff0 0x7f79b41a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:48.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.479+0000 7f799a7fc700 1 --2- 172.21.15.2:0/256207629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79b406b250 0x7f79b419be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:48.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.479+0000 7f799a7fc700 1 --2- 172.21.15.2:0/256207629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79b406bb70 0x7f79b419c360 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:48.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.479+0000 7f799a7fc700 1 -- 172.21.15.2:0/256207629 >> 172.21.15.2:0/256207629 conn(0x7f79b40793f0 msgr2=0x7f79b4111500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:48.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.479+0000 7f799a7fc700 1 -- 172.21.15.2:0/256207629 shutdown_connections 2024-05-14T16:55:48.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:48.479+0000 7f799a7fc700 1 -- 172.21.15.2:0/256207629 wait complete. 2024-05-14T16:55:48.481 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 28 2024-05-14T16:55:48.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:48 smithi002 ceph-mon[31538]: pgmap v36: 97 pgs: 97 active+clean; 958 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 18 MiB/s wr, 691 op/s 2024-05-14T16:55:48.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:48 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/256207629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:55:48.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:48 smithi151 ceph-mon[36066]: pgmap v36: 97 pgs: 97 active+clean; 958 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 18 MiB/s wr, 691 op/s 2024-05-14T16:55:48.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:48 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/256207629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:55:48.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:48 smithi123 ceph-mon[35971]: pgmap v36: 97 pgs: 97 active+clean; 958 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 18 MiB/s wr, 691 op/s 2024-05-14T16:55:48.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:48 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/256207629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:55:49.259 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id 822ccb9f --release --await' 2024-05-14T16:55:49.260 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id 822ccb9f --release --await 2024-05-14T16:55:49.536 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:55:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.624+0000 7f13ec8e3640 1 -- 172.21.15.2:0/3369060641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13e810d020 msgr2=0x7f13e810f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.624+0000 7f13ec8e3640 1 --2- 172.21.15.2:0/3369060641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13e810d020 0x7f13e810f4c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009ca0 tx=0x7f13e002a9e0 comp rx=0 tx=0).stop 2024-05-14T16:55:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.624+0000 7f13ec8e3640 1 -- 172.21.15.2:0/3369060641 shutdown_connections 2024-05-14T16:55:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.624+0000 7f13ec8e3640 1 --2- 172.21.15.2:0/3369060641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13e810d020 0x7f13e810f4c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.624+0000 7f13ec8e3640 1 --2- 172.21.15.2:0/3369060641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13e8104600 0x7f13e810cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.624+0000 7f13ec8e3640 1 --2- 172.21.15.2:0/3369060641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13e8103ce0 0x7f13e81040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.624+0000 7f13ec8e3640 1 -- 172.21.15.2:0/3369060641 >> 172.21.15.2:0/3369060641 conn(0x7f13e8078ad0 msgr2=0x7f13e8078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.624+0000 7f13ec8e3640 1 -- 172.21.15.2:0/3369060641 shutdown_connections 2024-05-14T16:55:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.624+0000 7f13ec8e3640 1 -- 172.21.15.2:0/3369060641 wait complete. 2024-05-14T16:55:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.625+0000 7f13ec8e3640 1 Processor -- start 2024-05-14T16:55:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.625+0000 7f13ec8e3640 1 -- start start 2024-05-14T16:55:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.625+0000 7f13ec8e3640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13e8103ce0 0x7f13e8199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.625+0000 7f13ec8e3640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13e8104600 0x7f13e81996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.625+0000 7f13ec8e3640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13e810d020 0x7f13e819da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.625+0000 7f13ec8e3640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f13e8112d20 con 0x7f13e8103ce0 2024-05-14T16:55:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.625+0000 7f13ec8e3640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f13e8112ba0 con 0x7f13e810d020 2024-05-14T16:55:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.625+0000 7f13ec8e3640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f13e8112ea0 con 0x7f13e8104600 2024-05-14T16:55:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.625+0000 7f13e659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13e8103ce0 0x7f13e8199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.625+0000 7f13e659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13e8103ce0 0x7f13e8199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34824/0 (socket says 172.21.15.2:34824) 2024-05-14T16:55:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.625+0000 7f13e659c640 1 -- 172.21.15.2:0/198787399 learned_addr learned my addr 172.21.15.2:0/198787399 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.625+0000 7f13dffff640 1 --2- 172.21.15.2:0/198787399 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13e8104600 0x7f13e81996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.625+0000 7f13e6d9d640 1 --2- 172.21.15.2:0/198787399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13e810d020 0x7f13e819da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.626+0000 7f13e659c640 1 -- 172.21.15.2:0/198787399 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13e8104600 msgr2=0x7f13e81996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.626+0000 7f13e659c640 1 --2- 172.21.15.2:0/198787399 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13e8104600 0x7f13e81996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.626+0000 7f13e659c640 1 -- 172.21.15.2:0/198787399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13e810d020 msgr2=0x7f13e819da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.626+0000 7f13e659c640 1 --2- 172.21.15.2:0/198787399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13e810d020 0x7f13e819da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.626+0000 7f13e659c640 1 -- 172.21.15.2:0/198787399 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e819e150 con 0x7f13e8103ce0 2024-05-14T16:55:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.626+0000 7f13e659c640 1 --2- 172.21.15.2:0/198787399 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13e8103ce0 0x7f13e8199160 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f13d800d770 tx=0x7f13d800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.626+0000 7f13df7fe640 1 -- 172.21.15.2:0/198787399 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d8017070 con 0x7f13e8103ce0 2024-05-14T16:55:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.626+0000 7f13ec8e3640 1 -- 172.21.15.2:0/198787399 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13e819e440 con 0x7f13e8103ce0 2024-05-14T16:55:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.626+0000 7f13df7fe640 1 -- 172.21.15.2:0/198787399 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13d80054d0 con 0x7f13e8103ce0 2024-05-14T16:55:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.626+0000 7f13df7fe640 1 -- 172.21.15.2:0/198787399 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d8004c30 con 0x7f13e8103ce0 2024-05-14T16:55:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.626+0000 7f13ec8e3640 1 -- 172.21.15.2:0/198787399 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13e81a5cd0 con 0x7f13e8103ce0 2024-05-14T16:55:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.629+0000 7f13df7fe640 1 -- 172.21.15.2:0/198787399 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) ==== 149884+0+0 (secure 0 0 0) 0x7f13d8004e10 con 0x7f13e8103ce0 2024-05-14T16:55:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.629+0000 7f13ef5fd740 1 Processor -- start 2024-05-14T16:55:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.629+0000 7f13ef5fd740 1 -- start start 2024-05-14T16:55:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.629+0000 7f13ef5fd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558f908c68f0 0x558f908c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.629+0000 7f13ef5fd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f908c7210 0x558f908d1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.629+0000 7f13ef5fd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558f908d1ad0 0x558f908d3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.629+0000 7f13ef5fd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x558f90839c30 con 0x558f908c68f0 2024-05-14T16:55:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.629+0000 7f13ef5fd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558f908297d0 con 0x558f908c7210 2024-05-14T16:55:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.629+0000 7f13ef5fd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558f9082d810 con 0x558f908d1ad0 2024-05-14T16:55:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.630+0000 7f13e6d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558f908c68f0 0x558f908c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.630+0000 7f13e6d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558f908c68f0 0x558f908c6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34832/0 (socket says 172.21.15.2:34832) 2024-05-14T16:55:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.630+0000 7f13e6d9d640 1 -- 172.21.15.2:0/3804340552 learned_addr learned my addr 172.21.15.2:0/3804340552 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.630+0000 7f13dffff640 1 --2- 172.21.15.2:0/3804340552 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558f908d1ad0 0x558f908d3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.630+0000 7f13e659c640 1 --2- 172.21.15.2:0/3804340552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f908c7210 0x558f908d1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.630+0000 7f13e6d9d640 1 -- 172.21.15.2:0/3804340552 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558f908d1ad0 msgr2=0x558f908d3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.630+0000 7f13e6d9d640 1 --2- 172.21.15.2:0/3804340552 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558f908d1ad0 0x558f908d3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.630+0000 7f13e6d9d640 1 -- 172.21.15.2:0/3804340552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f908c7210 msgr2=0x558f908d1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.630+0000 7f13e6d9d640 1 --2- 172.21.15.2:0/3804340552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f908c7210 0x558f908d1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.630+0000 7f13e6d9d640 1 -- 172.21.15.2:0/3804340552 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f9053e210 con 0x558f908c68f0 2024-05-14T16:55:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.630+0000 7f13e659c640 1 --2- 172.21.15.2:0/3804340552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f908c7210 0x558f908d1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:55:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.630+0000 7f13df7fe640 1 --2- 172.21.15.2:0/198787399 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f13d40b2b30 0x7f13d40b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.630+0000 7f13e6d9d640 1 --2- 172.21.15.2:0/3804340552 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558f908c68f0 0x558f908c6cd0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009ca0 tx=0x7f13e0009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbdad63988e4cb20 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.630+0000 7f13e659c640 1 --2- 172.21.15.2:0/198787399 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f13d40b2b30 0x7f13d40b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.630+0000 7f13df7fe640 1 -- 172.21.15.2:0/198787399 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f13d80e2320 con 0x7f13e8103ce0 2024-05-14T16:55:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.630+0000 7f13dcff9640 1 -- 172.21.15.2:0/3804340552 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e0049070 con 0x558f908c68f0 2024-05-14T16:55:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.630+0000 7f13dcff9640 1 -- 172.21.15.2:0/3804340552 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13e002ee70 con 0x558f908c68f0 2024-05-14T16:55:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.630+0000 7f13dcff9640 1 -- 172.21.15.2:0/3804340552 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e0033040 con 0x558f908c68f0 2024-05-14T16:55:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.631+0000 7f13ef5fd740 1 -- 172.21.15.2:0/3804340552 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558f908c68f0 msgr2=0x558f908c6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.631+0000 7f13e659c640 1 --2- 172.21.15.2:0/198787399 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f13d40b2b30 0x7f13d40b4ff0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f13d80df960 tx=0x7f13d8009f90 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.631+0000 7f13ef5fd740 1 --2- 172.21.15.2:0/3804340552 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558f908c68f0 0x558f908c6cd0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009ca0 tx=0x7f13e0009a20 comp rx=0 tx=0).stop 2024-05-14T16:55:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.631+0000 7f13ef5fd740 1 -- 172.21.15.2:0/3804340552 shutdown_connections 2024-05-14T16:55:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.631+0000 7f13ef5fd740 1 --2- 172.21.15.2:0/3804340552 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558f908d1ad0 0x558f908d3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.631+0000 7f13ef5fd740 1 --2- 172.21.15.2:0/3804340552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f908c7210 0x558f908d1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.631+0000 7f13ef5fd740 1 --2- 172.21.15.2:0/3804340552 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558f908c68f0 0x558f908c6cd0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.631+0000 7f13ef5fd740 1 -- 172.21.15.2:0/3804340552 >> 172.21.15.2:0/3804340552 conn(0x558f908c2e00 msgr2=0x558f908c31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.631+0000 7f13ef5fd740 1 -- 172.21.15.2:0/3804340552 shutdown_connections 2024-05-14T16:55:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.631+0000 7f13ef5fd740 1 -- 172.21.15.2:0/3804340552 wait complete. 2024-05-14T16:55:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.631+0000 7f13ef5fd740 1 -- start start 2024-05-14T16:55:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.632+0000 7f13ef5fd740 1 Processor -- start 2024-05-14T16:55:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.632+0000 7f13ef5fd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558f908c68f0 0x558f908d1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.632+0000 7f13ef5fd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f908c7210 0x558f908ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.632+0000 7f13ef5fd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558f908d1ad0 0x558f908cef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.632+0000 7f13ef5fd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x558f9050f790 con 0x558f908d1ad0 2024-05-14T16:55:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.632+0000 7f13ef5fd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558f907e5890 con 0x558f908c7210 2024-05-14T16:55:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.632+0000 7f13ef5fd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558f908d4740 con 0x558f908c68f0 2024-05-14T16:55:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.632+0000 7f13ef5fd740 10 client.0 _ll_register_callbacks cb 0x558f907e8d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:55:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.632+0000 7f13e6d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558f908d1ad0 0x558f908cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.632+0000 7f13e6d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558f908d1ad0 0x558f908cef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34840/0 (socket says 172.21.15.2:34840) 2024-05-14T16:55:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.632+0000 7f13dffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f908c7210 0x558f908ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.632+0000 7f13e6d9d640 1 -- 172.21.15.2:0/572894084 learned_addr learned my addr 172.21.15.2:0/572894084 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.632+0000 7f13e6d9d640 1 --2- 172.21.15.2:0/572894084 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558f908c68f0 0x558f908d1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.632+0000 7f13dffff640 1 -- 172.21.15.2:0/572894084 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558f908c68f0 msgr2=0x558f908d1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.632+0000 7f13dffff640 1 --2- 172.21.15.2:0/572894084 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558f908c68f0 0x558f908d1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.632+0000 7f13dffff640 1 -- 172.21.15.2:0/572894084 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558f908d1ad0 msgr2=0x558f908cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.632+0000 7f13dffff640 1 --2- 172.21.15.2:0/572894084 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558f908d1ad0 0x558f908cef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.632+0000 7f13dffff640 1 -- 172.21.15.2:0/572894084 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f9060a620 con 0x558f908c7210 2024-05-14T16:55:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.633+0000 7f13dffff640 1 --2- 172.21.15.2:0/572894084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f908c7210 0x558f908ce7f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x558f908c3f90 tx=0x7f13d000be50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15cbd6b81bcc9132 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.633+0000 7f13cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T16:55:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.633+0000 7f13cdffb640 1 -- 172.21.15.2:0/572894084 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d0003d70 con 0x558f908c7210 2024-05-14T16:55:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.633+0000 7f13ef5fd740 1 -- 172.21.15.2:0/572894084 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13e002ee70 con 0x558f908c7210 2024-05-14T16:55:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.633+0000 7f13cdffb640 1 -- 172.21.15.2:0/572894084 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f13d0005020 con 0x558f908c7210 2024-05-14T16:55:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.633+0000 7f13cdffb640 1 -- 172.21.15.2:0/572894084 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f13d0005220 con 0x558f908c7210 2024-05-14T16:55:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.633+0000 7f13ef5fd740 10 client.25169 fetch_fsmap learned FSMap version 28 2024-05-14T16:55:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.633+0000 7f13ef5fd740 1 -- 172.21.15.2:0/572894084 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558f908e0270 con 0x558f908c7210 2024-05-14T16:55:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.633+0000 7f13cdffb640 1 -- 172.21.15.2:0/572894084 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13e002ee70 con 0x558f908c7210 2024-05-14T16:55:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.633+0000 7f13cdffb640 1 -- 172.21.15.2:0/572894084 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 28) ==== 8350+0+0 (secure 0 0 0) 0x7f13d001b470 con 0x558f908c7210 2024-05-14T16:55:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.633+0000 7f13ef5fd740 10 client.25169 fetch_fsmap finished waiting for FSMap version 28 2024-05-14T16:55:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.633+0000 7f13ef5fd740 10 client.25169 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:55:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.633+0000 7f13ef5fd740 20 client.25169 populate_metadata read hostname 'smithi002' 2024-05-14T16:55:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.633+0000 7f13ef5fd740 1 --2- 172.21.15.2:0/572894084 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x558f908e17d0 0x558f908e3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.633+0000 7f13ef5fd740 4 client.25169 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:55:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.633+0000 7f13ef5fd740 1 -- 172.21.15.2:0/572894084 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f9083bb80 con 0x558f908e17d0 2024-05-14T16:55:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.634+0000 7f13e6d9d640 1 --2- 172.21.15.2:0/572894084 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x558f908e17d0 0x558f908e3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.634+0000 7f13e6d9d640 1 --2- 172.21.15.2:0/572894084 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x558f908e17d0 0x558f908e3bc0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6947c66ba94a11cf server_cookie=e3a679dcfe7f0c88 in_seq=0 out_seq=0 2024-05-14T16:55:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.634+0000 7f13cdffb640 10 client.25169 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:55:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.634+0000 7f13cdffb640 1 -- 172.21.15.2:0/572894084 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558f9083bb80 con 0x558f908e17d0 2024-05-14T16:55:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.635+0000 7f13cdffb640 10 client.25169 handle_command_reply: tid=0 2024-05-14T16:55:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.635+0000 7f13ef5fd740 1 client.25169 shutdown 2024-05-14T16:55:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.635+0000 7f13ef5fd740 1 -- 172.21.15.2:0/572894084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f908c7210 msgr2=0x558f908ce7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.635+0000 7f13ef5fd740 1 --2- 172.21.15.2:0/572894084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f908c7210 0x558f908ce7f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x558f908c3f90 tx=0x7f13d000be50 comp rx=0 tx=0).stop 2024-05-14T16:55:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.635+0000 7f13ef5fd740 1 -- 172.21.15.2:0/572894084 shutdown_connections 2024-05-14T16:55:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.635+0000 7f13ef5fd740 1 --2- 172.21.15.2:0/572894084 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x558f908e17d0 0x558f908e3bc0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.635+0000 7f13ef5fd740 1 --2- 172.21.15.2:0/572894084 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558f908d1ad0 0x558f908cef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.635+0000 7f13ef5fd740 1 --2- 172.21.15.2:0/572894084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f908c7210 0x558f908ce7f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.635+0000 7f13ef5fd740 1 --2- 172.21.15.2:0/572894084 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558f908c68f0 0x558f908d1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.635+0000 7f13cdffb640 0 client.25169 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:55:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.635+0000 7f13ef5fd740 1 -- 172.21.15.2:0/572894084 >> 172.21.15.2:0/572894084 conn(0x558f908c2e00 msgr2=0x558f908c2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.635+0000 7f13ef5fd740 1 -- 172.21.15.2:0/572894084 shutdown_connections 2024-05-14T16:55:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.635+0000 7f13ef5fd740 1 -- 172.21.15.2:0/572894084 wait complete. 2024-05-14T16:55:49.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.635+0000 7f13ef5fd740 20 client.25169 trim_cache size 0 max 16384 2024-05-14T16:55:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.650+0000 7f13ef5fd740 1 Processor -- start 2024-05-14T16:55:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.650+0000 7f13ef5fd740 1 -- start start 2024-05-14T16:55:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.650+0000 7f13ef5fd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f908c68f0 0x558f908dd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.650+0000 7f13ef5fd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558f908c7210 0x558f908d1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.650+0000 7f13ef5fd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558f908d1ad0 0x558f908d1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.650+0000 7f13ef5fd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f13e81b0e40 con 0x558f908c7210 2024-05-14T16:55:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.650+0000 7f13ef5fd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f13e81ae6e0 con 0x558f908c68f0 2024-05-14T16:55:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.650+0000 7f13ef5fd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f13e81ac530 con 0x558f908d1ad0 2024-05-14T16:55:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.650+0000 7f13dffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558f908c7210 0x558f908d1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.650+0000 7f13e6d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f908c68f0 0x558f908dd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.650+0000 7f13dffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558f908c7210 0x558f908d1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34850/0 (socket says 172.21.15.2:34850) 2024-05-14T16:55:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.650+0000 7f13dffff640 1 -- 172.21.15.2:0/266289113 learned_addr learned my addr 172.21.15.2:0/266289113 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:49.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.650+0000 7f13e6d9d640 1 --2- 172.21.15.2:0/266289113 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558f908d1ad0 0x558f908d1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:49.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.651+0000 7f13e6d9d640 1 -- 172.21.15.2:0/266289113 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558f908d1ad0 msgr2=0x558f908d1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:49.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.651+0000 7f13e6d9d640 1 --2- 172.21.15.2:0/266289113 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558f908d1ad0 0x558f908d1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:49.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.651+0000 7f13e6d9d640 1 -- 172.21.15.2:0/266289113 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558f908c7210 msgr2=0x558f908d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:49.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.651+0000 7f13e6d9d640 1 --2- 172.21.15.2:0/266289113 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558f908c7210 0x558f908d1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:49.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.651+0000 7f13e6d9d640 1 -- 172.21.15.2:0/266289113 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f908c66b0 con 0x558f908c68f0 2024-05-14T16:55:49.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.651+0000 7f13e6d9d640 1 --2- 172.21.15.2:0/266289113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f908c68f0 0x558f908dd610 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f13e002fbb0 tx=0x7f13e003b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4433117dd8ef391 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:49.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.651+0000 7f13cffff640 1 -- 172.21.15.2:0/266289113 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e0049050 con 0x558f908c68f0 2024-05-14T16:55:49.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.651+0000 7f13cffff640 1 -- 172.21.15.2:0/266289113 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13e004a040 con 0x558f908c68f0 2024-05-14T16:55:49.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.651+0000 7f13cffff640 1 -- 172.21.15.2:0/266289113 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e0038840 con 0x558f908c68f0 2024-05-14T16:55:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.652+0000 7f13ef5fd740 1 -- 172.21.15.2:0/266289113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f908c68f0 msgr2=0x558f908dd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.652+0000 7f13ef5fd740 1 --2- 172.21.15.2:0/266289113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f908c68f0 0x558f908dd610 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f13e002fbb0 tx=0x7f13e003b040 comp rx=0 tx=0).stop 2024-05-14T16:55:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.652+0000 7f13ef5fd740 1 -- 172.21.15.2:0/266289113 shutdown_connections 2024-05-14T16:55:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.652+0000 7f13ef5fd740 1 --2- 172.21.15.2:0/266289113 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558f908d1ad0 0x558f908d1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.652+0000 7f13ef5fd740 1 --2- 172.21.15.2:0/266289113 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558f908c7210 0x558f908d1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.652+0000 7f13ef5fd740 1 --2- 172.21.15.2:0/266289113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f908c68f0 0x558f908dd610 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.652+0000 7f13ef5fd740 1 -- 172.21.15.2:0/266289113 >> 172.21.15.2:0/266289113 conn(0x558f908c2e00 msgr2=0x558f908c8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.652+0000 7f13ef5fd740 1 -- 172.21.15.2:0/266289113 shutdown_connections 2024-05-14T16:55:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.652+0000 7f13ef5fd740 1 -- 172.21.15.2:0/266289113 wait complete. 2024-05-14T16:55:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.652+0000 7f13ef5fd740 1 -- start start 2024-05-14T16:55:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.652+0000 7f13ef5fd740 1 Processor -- start 2024-05-14T16:55:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.652+0000 7f13ef5fd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558f908c68f0 0x558f908d55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.652+0000 7f13ef5fd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558f908c7210 0x558f908d5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.652+0000 7f13ef5fd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f908d1ad0 0x558f908d6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.652+0000 7f13ef5fd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f13e81acba0 con 0x558f908c68f0 2024-05-14T16:55:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.652+0000 7f13ef5fd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f13e81ae010 con 0x558f908d1ad0 2024-05-14T16:55:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.652+0000 7f13ef5fd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f13e81ad270 con 0x558f908c7210 2024-05-14T16:55:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.652+0000 7f13ef5fd740 10 client.0 _ll_register_callbacks cb 0x7f13d0004420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:55:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.652+0000 7f13e6d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558f908c68f0 0x558f908d55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.652+0000 7f13e6d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558f908c68f0 0x558f908d55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34856/0 (socket says 172.21.15.2:34856) 2024-05-14T16:55:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.653+0000 7f13e6d9d640 1 -- 172.21.15.2:0/1104201670 learned_addr learned my addr 172.21.15.2:0/1104201670 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.653+0000 7f13dffff640 1 --2- 172.21.15.2:0/1104201670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558f908c7210 0x558f908d5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.653+0000 7f13e6d9d640 1 -- 172.21.15.2:0/1104201670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558f908c7210 msgr2=0x558f908d5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.653+0000 7f13e6d9d640 1 --2- 172.21.15.2:0/1104201670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558f908c7210 0x558f908d5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.653+0000 7f13e6d9d640 1 -- 172.21.15.2:0/1104201670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f908d1ad0 msgr2=0x558f908d6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.653+0000 7f13e6d9d640 1 --2- 172.21.15.2:0/1104201670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f908d1ad0 0x558f908d6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.653+0000 7f13e6d9d640 1 -- 172.21.15.2:0/1104201670 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f9094b0b0 con 0x558f908c68f0 2024-05-14T16:55:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.653+0000 7f13dffff640 1 --2- 172.21.15.2:0/1104201670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558f908c7210 0x558f908d5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:55:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.653+0000 7f13e6d9d640 1 --2- 172.21.15.2:0/1104201670 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558f908c68f0 0x558f908d55b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f13e00095c0 tx=0x7f13e002a9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f9fcb354933974 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.653+0000 7f13cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:55:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.653+0000 7f13cdffb640 1 -- 172.21.15.2:0/1104201670 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e0033070 con 0x558f908c68f0 2024-05-14T16:55:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.653+0000 7f13cdffb640 1 -- 172.21.15.2:0/1104201670 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f13e002e390 con 0x558f908c68f0 2024-05-14T16:55:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.653+0000 7f13ef5fd740 1 -- 172.21.15.2:0/1104201670 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13e004a040 con 0x558f908c68f0 2024-05-14T16:55:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.653+0000 7f13cdffb640 1 -- 172.21.15.2:0/1104201670 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f13e0049040 con 0x558f908c68f0 2024-05-14T16:55:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.653+0000 7f13ef5fd740 10 client.15432 fetch_fsmap learned FSMap version 28 2024-05-14T16:55:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.653+0000 7f13ef5fd740 1 -- 172.21.15.2:0/1104201670 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558f9094c3a0 con 0x558f908c68f0 2024-05-14T16:55:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.654+0000 7f13cdffb640 1 -- 172.21.15.2:0/1104201670 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13e004a040 con 0x558f908c68f0 2024-05-14T16:55:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.654+0000 7f13cdffb640 1 -- 172.21.15.2:0/1104201670 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 28) ==== 8350+0+0 (secure 0 0 0) 0x7f13e0043070 con 0x558f908c68f0 2024-05-14T16:55:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.654+0000 7f13ef5fd740 10 client.15432 fetch_fsmap finished waiting for FSMap version 28 2024-05-14T16:55:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.654+0000 7f13ef5fd740 10 client.15432 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:55:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.654+0000 7f13ef5fd740 20 client.15432 populate_metadata read hostname 'smithi002' 2024-05-14T16:55:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.654+0000 7f13ef5fd740 1 --2- 172.21.15.2:0/1104201670 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x558f9094c6a0 0x558f9094ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.654+0000 7f13ef5fd740 4 client.15432 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:55:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.654+0000 7f13ef5fd740 1 -- 172.21.15.2:0/1104201670 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558f90835360 con 0x558f9094c6a0 2024-05-14T16:55:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.654+0000 7f13dffff640 1 --2- 172.21.15.2:0/1104201670 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x558f9094c6a0 0x558f9094ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.655+0000 7f13dffff640 1 --2- 172.21.15.2:0/1104201670 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x558f9094c6a0 0x558f9094ea90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7475bba96911bb33 server_cookie=ceee85cedf6eff96 in_seq=0 out_seq=0 2024-05-14T16:55:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.655+0000 7f13cdffb640 10 client.15432 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:55:49.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.671+0000 7f13cdffb640 1 -- 172.21.15.2:0/1104201670 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558f90835360 con 0x558f9094c6a0 2024-05-14T16:55:49.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.671+0000 7f13cdffb640 10 client.15432 handle_command_reply: tid=0 2024-05-14T16:55:49.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.671+0000 7f13ef5fd740 1 client.15432 shutdown 2024-05-14T16:55:49.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.671+0000 7f13ef5fd740 1 -- 172.21.15.2:0/1104201670 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558f908c68f0 msgr2=0x558f908d55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:49.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.671+0000 7f13ef5fd740 1 --2- 172.21.15.2:0/1104201670 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558f908c68f0 0x558f908d55b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f13e00095c0 tx=0x7f13e002a9e0 comp rx=0 tx=0).stop 2024-05-14T16:55:49.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.672+0000 7f13ef5fd740 1 -- 172.21.15.2:0/1104201670 shutdown_connections 2024-05-14T16:55:49.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.672+0000 7f13ef5fd740 1 --2- 172.21.15.2:0/1104201670 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x558f9094c6a0 0x558f9094ea90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:49.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.672+0000 7f13ef5fd740 1 --2- 172.21.15.2:0/1104201670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f908d1ad0 0x558f908d6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:49.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.672+0000 7f13cdffb640 0 client.15432 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:55:49.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.672+0000 7f13ef5fd740 1 --2- 172.21.15.2:0/1104201670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558f908c7210 0x558f908d5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:49.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.672+0000 7f13ef5fd740 1 --2- 172.21.15.2:0/1104201670 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558f908c68f0 0x558f908d55b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:49.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.672+0000 7f13ef5fd740 1 -- 172.21.15.2:0/1104201670 >> 172.21.15.2:0/1104201670 conn(0x558f908c2e00 msgr2=0x558f908c94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:49.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.672+0000 7f13ef5fd740 1 -- 172.21.15.2:0/1104201670 shutdown_connections 2024-05-14T16:55:49.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.672+0000 7f13ef5fd740 1 -- 172.21.15.2:0/1104201670 wait complete. 2024-05-14T16:55:49.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.672+0000 7f13ef5fd740 20 client.15432 trim_cache size 0 max 16384 2024-05-14T16:55:49.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T16:55:49.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T16:55:49.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T16:55:49.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.672+0000 7f13ec8e3640 1 -- 172.21.15.2:0/198787399 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f13d40b2b30 msgr2=0x7f13d40b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:49.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.672+0000 7f13ec8e3640 1 --2- 172.21.15.2:0/198787399 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f13d40b2b30 0x7f13d40b4ff0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f13d80df960 tx=0x7f13d8009f90 comp rx=0 tx=0).stop 2024-05-14T16:55:49.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.672+0000 7f13ec8e3640 1 -- 172.21.15.2:0/198787399 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13e8103ce0 msgr2=0x7f13e8199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:49.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.672+0000 7f13ec8e3640 1 --2- 172.21.15.2:0/198787399 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13e8103ce0 0x7f13e8199160 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f13d800d770 tx=0x7f13d800dc30 comp rx=0 tx=0).stop 2024-05-14T16:55:49.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.672+0000 7f13ec8e3640 1 -- 172.21.15.2:0/198787399 shutdown_connections 2024-05-14T16:55:49.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.672+0000 7f13ec8e3640 1 --2- 172.21.15.2:0/198787399 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f13d40b2b30 0x7f13d40b4ff0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:49.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.672+0000 7f13ec8e3640 1 --2- 172.21.15.2:0/198787399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13e810d020 0x7f13e819da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:49.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.672+0000 7f13ec8e3640 1 --2- 172.21.15.2:0/198787399 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13e8104600 0x7f13e81996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:49.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.672+0000 7f13ec8e3640 1 --2- 172.21.15.2:0/198787399 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13e8103ce0 0x7f13e8199160 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:49.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.672+0000 7f13ec8e3640 1 -- 172.21.15.2:0/198787399 >> 172.21.15.2:0/198787399 conn(0x7f13e8078ad0 msgr2=0x7f13e80fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:49.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.673+0000 7f13ec8e3640 1 -- 172.21.15.2:0/198787399 shutdown_connections 2024-05-14T16:55:49.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:55:49.673+0000 7f13ec8e3640 1 -- 172.21.15.2:0/198787399 wait complete. 2024-05-14T16:55:49.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:49 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:55:49.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:49 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:55:49.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:49 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:55:49.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:49 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:55:49.906 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:49 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:55:49.907 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:49 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:55:49.907 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:49 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:55:49.907 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:49 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:55:49.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:49 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:55:49.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:49 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:55:49.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:49 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:55:49.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:49 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:55:50.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:50 smithi151 ceph-mon[36066]: pgmap v37: 97 pgs: 97 active+clean; 958 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 14 MiB/s wr, 393 op/s 2024-05-14T16:55:50.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:50 smithi123 ceph-mon[35971]: pgmap v37: 97 pgs: 97 active+clean; 958 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 14 MiB/s wr, 393 op/s 2024-05-14T16:55:50.949 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:50 smithi002 ceph-mon[31538]: pgmap v37: 97 pgs: 97 active+clean; 958 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 14 MiB/s wr, 393 op/s 2024-05-14T16:55:51.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.349+0000 7f867aad0700 1 -- 172.21.15.2:0/1712535839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8674103600 msgr2=0x7f8674103a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:51.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.349+0000 7f867aad0700 1 --2- 172.21.15.2:0/1712535839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8674103600 0x7f8674103a80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f865c00b0d0 tx=0x7f865c00b3e0 comp rx=0 tx=0).stop 2024-05-14T16:55:51.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.349+0000 7f867aad0700 1 -- 172.21.15.2:0/1712535839 shutdown_connections 2024-05-14T16:55:51.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.349+0000 7f867aad0700 1 --2- 172.21.15.2:0/1712535839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8674103fc0 0x7f86741103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:51.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.349+0000 7f867aad0700 1 --2- 172.21.15.2:0/1712535839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86741095e0 0x7f86741099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:51.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.349+0000 7f867aad0700 1 --2- 172.21.15.2:0/1712535839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8674103600 0x7f8674103a80 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:51.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.349+0000 7f867aad0700 1 -- 172.21.15.2:0/1712535839 >> 172.21.15.2:0/1712535839 conn(0x7f867407a700 msgr2=0x7f867407ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:51.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.349+0000 7f867aad0700 1 -- 172.21.15.2:0/1712535839 shutdown_connections 2024-05-14T16:55:51.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.349+0000 7f867aad0700 1 -- 172.21.15.2:0/1712535839 wait complete. 2024-05-14T16:55:51.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.350+0000 7f867aad0700 1 Processor -- start 2024-05-14T16:55:51.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.350+0000 7f867aad0700 1 -- start start 2024-05-14T16:55:51.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.350+0000 7f867aad0700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8674103600 0x7f8674199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:51.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.350+0000 7f867aad0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8674103fc0 0x7f867419a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:51.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.350+0000 7f867aad0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86741095e0 0x7f867419e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:51.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.350+0000 7f867aad0700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8674113700 con 0x7f8674103fc0 2024-05-14T16:55:51.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.350+0000 7f867aad0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8674113580 con 0x7f86741095e0 2024-05-14T16:55:51.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.350+0000 7f867aad0700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8674113880 con 0x7f8674103600 2024-05-14T16:55:51.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.350+0000 7f8673fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8674103fc0 0x7f867419a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:51.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.350+0000 7f8673fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8674103fc0 0x7f867419a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34868/0 (socket says 172.21.15.2:34868) 2024-05-14T16:55:51.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.350+0000 7f8673fff700 1 -- 172.21.15.2:0/1760210834 learned_addr learned my addr 172.21.15.2:0/1760210834 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:51.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.350+0000 7f867886c700 1 --2- 172.21.15.2:0/1760210834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8674103600 0x7f8674199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:51.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.350+0000 7f867906d700 1 --2- 172.21.15.2:0/1760210834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86741095e0 0x7f867419e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:51.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.351+0000 7f8673fff700 1 -- 172.21.15.2:0/1760210834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8674103600 msgr2=0x7f8674199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:51.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.351+0000 7f8673fff700 1 --2- 172.21.15.2:0/1760210834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8674103600 0x7f8674199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:51.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.351+0000 7f8673fff700 1 -- 172.21.15.2:0/1760210834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86741095e0 msgr2=0x7f867419e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:51.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.351+0000 7f8673fff700 1 --2- 172.21.15.2:0/1760210834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86741095e0 0x7f867419e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:51.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.351+0000 7f8673fff700 1 -- 172.21.15.2:0/1760210834 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f867419ec90 con 0x7f8674103fc0 2024-05-14T16:55:51.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.351+0000 7f8673fff700 1 --2- 172.21.15.2:0/1760210834 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8674103fc0 0x7f867419a1d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f865c00d010 tx=0x7f865c007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:51.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.351+0000 7f8671ffb700 1 -- 172.21.15.2:0/1760210834 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c01f070 con 0x7f8674103fc0 2024-05-14T16:55:51.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.351+0000 7f8671ffb700 1 -- 172.21.15.2:0/1760210834 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f865c007e50 con 0x7f8674103fc0 2024-05-14T16:55:51.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.351+0000 7f867aad0700 1 -- 172.21.15.2:0/1760210834 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f867419ef20 con 0x7f8674103fc0 2024-05-14T16:55:51.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.351+0000 7f8671ffb700 1 -- 172.21.15.2:0/1760210834 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c01b4a0 con 0x7f8674103fc0 2024-05-14T16:55:51.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.351+0000 7f867aad0700 1 -- 172.21.15.2:0/1760210834 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86741a6840 con 0x7f8674103fc0 2024-05-14T16:55:51.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.354+0000 7f8671ffb700 1 -- 172.21.15.2:0/1760210834 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 25) ==== 149884+0+0 (secure 0 0 0) 0x7f865c022070 con 0x7f8674103fc0 2024-05-14T16:55:51.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.355+0000 7f8671ffb700 1 --2- 172.21.15.2:0/1760210834 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f86600b2b10 0x7f86600b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:51.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.355+0000 7f867886c700 1 --2- 172.21.15.2:0/1760210834 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f86600b2b10 0x7f86600b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:51.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.355+0000 7f8671ffb700 1 -- 172.21.15.2:0/1760210834 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f865c00f020 con 0x7f8674103fc0 2024-05-14T16:55:51.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.355+0000 7f8685366b80 1 Processor -- start 2024-05-14T16:55:51.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.355+0000 7f867886c700 1 --2- 172.21.15.2:0/1760210834 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f86600b2b10 0x7f86600b4fd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f86640045b0 tx=0x7f866400b3c0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:51.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.355+0000 7f8685366b80 1 -- start start 2024-05-14T16:55:51.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.356+0000 7f8685366b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ee2917420 0x555ee2917800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:51.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.356+0000 7f8685366b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ee2917d40 0x555ee2912ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:51.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.356+0000 7f8685366b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ee29132c0 0x555ee2913720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:51.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.356+0000 7f8685366b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x555ee2870e70 con 0x555ee2917d40 2024-05-14T16:55:51.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.356+0000 7f8685366b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555ee2860a10 con 0x555ee2917420 2024-05-14T16:55:51.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.356+0000 7f8685366b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555ee2864a50 con 0x555ee29132c0 2024-05-14T16:55:51.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.356+0000 7f8673fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ee2917d40 0x555ee2912ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:51.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.356+0000 7f8673fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ee2917d40 0x555ee2912ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34876/0 (socket says 172.21.15.2:34876) 2024-05-14T16:55:51.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.356+0000 7f8673fff700 1 -- 172.21.15.2:0/3645077912 learned_addr learned my addr 172.21.15.2:0/3645077912 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:51.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.356+0000 7f867886c700 1 --2- 172.21.15.2:0/3645077912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ee2917420 0x555ee2917800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:51.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.356+0000 7f867906d700 1 --2- 172.21.15.2:0/3645077912 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ee29132c0 0x555ee2913720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:51.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.356+0000 7f8673fff700 1 -- 172.21.15.2:0/3645077912 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ee29132c0 msgr2=0x555ee2913720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:51.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.356+0000 7f8673fff700 1 --2- 172.21.15.2:0/3645077912 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ee29132c0 0x555ee2913720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:51.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.356+0000 7f8673fff700 1 -- 172.21.15.2:0/3645077912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ee2917420 msgr2=0x555ee2917800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:51.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.356+0000 7f8673fff700 1 --2- 172.21.15.2:0/3645077912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ee2917420 0x555ee2917800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:51.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.356+0000 7f8673fff700 1 -- 172.21.15.2:0/3645077912 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ee280a0b0 con 0x555ee2917d40 2024-05-14T16:55:51.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.356+0000 7f867886c700 1 --2- 172.21.15.2:0/3645077912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ee2917420 0x555ee2917800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:55:51.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.356+0000 7f8673fff700 1 --2- 172.21.15.2:0/3645077912 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ee2917d40 0x555ee2912ca0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f865c0aaa80 tx=0x7f865c0aa330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c211a66721569d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:51.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.356+0000 7f866affd700 1 -- 172.21.15.2:0/3645077912 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c0b3210 con 0x555ee2917d40 2024-05-14T16:55:51.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.357+0000 7f866affd700 1 -- 172.21.15.2:0/3645077912 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f865c0aa600 con 0x555ee2917d40 2024-05-14T16:55:51.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.357+0000 7f866affd700 1 -- 172.21.15.2:0/3645077912 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c0aaf50 con 0x555ee2917d40 2024-05-14T16:55:51.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.357+0000 7f8685366b80 1 -- 172.21.15.2:0/3645077912 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ee2917d40 msgr2=0x555ee2912ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:51.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.357+0000 7f8685366b80 1 --2- 172.21.15.2:0/3645077912 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ee2917d40 0x555ee2912ca0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f865c0aaa80 tx=0x7f865c0aa330 comp rx=0 tx=0).stop 2024-05-14T16:55:51.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.357+0000 7f8685366b80 1 -- 172.21.15.2:0/3645077912 shutdown_connections 2024-05-14T16:55:51.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.357+0000 7f8685366b80 1 --2- 172.21.15.2:0/3645077912 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ee29132c0 0x555ee2913720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:51.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.357+0000 7f8685366b80 1 --2- 172.21.15.2:0/3645077912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ee2917420 0x555ee2917800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:51.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.357+0000 7f8685366b80 1 --2- 172.21.15.2:0/3645077912 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ee2917d40 0x555ee2912ca0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:51.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.357+0000 7f8685366b80 1 -- 172.21.15.2:0/3645077912 >> 172.21.15.2:0/3645077912 conn(0x555ee283feb0 msgr2=0x555ee290d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:51.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.357+0000 7f8685366b80 1 -- 172.21.15.2:0/3645077912 shutdown_connections 2024-05-14T16:55:51.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.357+0000 7f8685366b80 1 -- 172.21.15.2:0/3645077912 wait complete. 2024-05-14T16:55:51.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.358+0000 7f8685366b80 1 -- start start 2024-05-14T16:55:51.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.358+0000 7f8685366b80 1 Processor -- start 2024-05-14T16:55:51.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.358+0000 7f8685366b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ee29132c0 0x555ee291ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:51.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.358+0000 7f8685366b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ee2917420 0x555ee291bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:51.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.358+0000 7f8685366b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ee2917d40 0x555ee291c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:51.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.358+0000 7f8685366b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x555ee27670a0 con 0x555ee29132c0 2024-05-14T16:55:51.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.358+0000 7f8685366b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555ee27c8ea0 con 0x555ee2917420 2024-05-14T16:55:51.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.358+0000 7f8685366b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555ee2870e70 con 0x555ee2917d40 2024-05-14T16:55:51.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.358+0000 7f8685366b80 10 client.0 _ll_register_callbacks cb 0x555ee27fba50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:55:51.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.358+0000 7f867886c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ee29132c0 0x555ee291ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:51.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.359+0000 7f867886c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ee29132c0 0x555ee291ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34884/0 (socket says 172.21.15.2:34884) 2024-05-14T16:55:51.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.359+0000 7f867886c700 1 -- 172.21.15.2:0/1741824912 learned_addr learned my addr 172.21.15.2:0/1741824912 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:51.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.359+0000 7f867906d700 1 --2- 172.21.15.2:0/1741824912 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ee2917d40 0x555ee291c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:51.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.359+0000 7f8673fff700 1 --2- 172.21.15.2:0/1741824912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ee2917420 0x555ee291bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:51.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.359+0000 7f867886c700 1 -- 172.21.15.2:0/1741824912 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ee2917d40 msgr2=0x555ee291c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:51.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.359+0000 7f867886c700 1 --2- 172.21.15.2:0/1741824912 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ee2917d40 0x555ee291c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:51.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.359+0000 7f867886c700 1 -- 172.21.15.2:0/1741824912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ee2917420 msgr2=0x555ee291bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:51.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.359+0000 7f867886c700 1 --2- 172.21.15.2:0/1741824912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ee2917420 0x555ee291bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:51.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.359+0000 7f867886c700 1 -- 172.21.15.2:0/1741824912 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ee27b09d0 con 0x555ee29132c0 2024-05-14T16:55:51.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.359+0000 7f867906d700 1 --2- 172.21.15.2:0/1741824912 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ee2917d40 0x555ee291c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:55:51.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.359+0000 7f8673fff700 1 --2- 172.21.15.2:0/1741824912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ee2917420 0x555ee291bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:55:51.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.359+0000 7f867886c700 1 --2- 172.21.15.2:0/1741824912 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ee29132c0 0x555ee291ebb0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8664003ec0 tx=0x7f86640047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f588f80c12dfec server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:51.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.359+0000 7f8653fff700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:55:51.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.359+0000 7f8653fff700 1 -- 172.21.15.2:0/1741824912 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8664028670 con 0x555ee29132c0 2024-05-14T16:55:51.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.359+0000 7f8653fff700 1 -- 172.21.15.2:0/1741824912 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f866401f430 con 0x555ee29132c0 2024-05-14T16:55:51.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.359+0000 7f8685366b80 1 -- 172.21.15.2:0/1741824912 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f865c0aa600 con 0x555ee29132c0 2024-05-14T16:55:51.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.359+0000 7f8653fff700 1 -- 172.21.15.2:0/1741824912 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8664028810 con 0x555ee29132c0 2024-05-14T16:55:51.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.359+0000 7f8685366b80 10 client.15444 fetch_fsmap learned FSMap version 28 2024-05-14T16:55:51.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.359+0000 7f8685366b80 1 -- 172.21.15.2:0/1741824912 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555ee281a860 con 0x555ee29132c0 2024-05-14T16:55:51.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.360+0000 7f8653fff700 1 -- 172.21.15.2:0/1741824912 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f865c0aa600 con 0x555ee29132c0 2024-05-14T16:55:51.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.360+0000 7f8653fff700 1 -- 172.21.15.2:0/1741824912 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 28) ==== 8350+0+0 (secure 0 0 0) 0x7f866401fb20 con 0x555ee29132c0 2024-05-14T16:55:51.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.360+0000 7f8685366b80 10 client.15444 fetch_fsmap finished waiting for FSMap version 28 2024-05-14T16:55:51.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.360+0000 7f8685366b80 10 client.15444 resolve_mds: validated gid 24479 aka daemon mds.b 2024-05-14T16:55:51.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.360+0000 7f8685366b80 20 client.15444 populate_metadata read hostname 'smithi002' 2024-05-14T16:55:51.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.360+0000 7f8685366b80 1 --2- 172.21.15.2:0/1741824912 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x555ee292c0e0 0x555ee292e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:51.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.360+0000 7f8685366b80 4 client.15444 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:55:51.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.360+0000 7f8685366b80 1 -- 172.21.15.2:0/1741824912 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ee2872dc0 con 0x555ee292c0e0 2024-05-14T16:55:51.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.360+0000 7f8673fff700 1 --2- 172.21.15.2:0/1741824912 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x555ee292c0e0 0x555ee292e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:51.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.361+0000 7f8653fff700 10 client.15444 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:55:51.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.361+0000 7f8673fff700 1 --2- 172.21.15.2:0/1741824912 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x555ee292c0e0 0x555ee292e560 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50072a81ccd42844 server_cookie=7101b5838a627eac in_seq=0 out_seq=0 2024-05-14T16:55:51.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.361+0000 7f8653fff700 1 -- 172.21.15.2:0/1741824912 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555ee2872dc0 con 0x555ee292c0e0 2024-05-14T16:55:51.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.361+0000 7f8653fff700 10 client.15444 handle_command_reply: tid=0 2024-05-14T16:55:51.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.361+0000 7f8685366b80 1 client.15444 shutdown 2024-05-14T16:55:51.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.361+0000 7f8685366b80 1 -- 172.21.15.2:0/1741824912 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ee29132c0 msgr2=0x555ee291ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:51.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.361+0000 7f8685366b80 1 --2- 172.21.15.2:0/1741824912 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ee29132c0 0x555ee291ebb0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8664003ec0 tx=0x7f86640047c0 comp rx=0 tx=0).stop 2024-05-14T16:55:51.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.361+0000 7f8685366b80 1 -- 172.21.15.2:0/1741824912 shutdown_connections 2024-05-14T16:55:51.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.361+0000 7f8685366b80 1 --2- 172.21.15.2:0/1741824912 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x555ee292c0e0 0x555ee292e560 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:51.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.361+0000 7f8685366b80 1 --2- 172.21.15.2:0/1741824912 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ee2917d40 0x555ee291c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:51.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.361+0000 7f8685366b80 1 --2- 172.21.15.2:0/1741824912 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ee29132c0 0x555ee291ebb0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:51.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.362+0000 7f8685366b80 1 --2- 172.21.15.2:0/1741824912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ee2917420 0x555ee291bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:51.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.362+0000 7f8685366b80 1 -- 172.21.15.2:0/1741824912 >> 172.21.15.2:0/1741824912 conn(0x555ee283feb0 msgr2=0x555ee2907fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:51.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.362+0000 7f8653fff700 0 client.15444 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:55:51.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.362+0000 7f8685366b80 1 -- 172.21.15.2:0/1741824912 shutdown_connections 2024-05-14T16:55:51.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.362+0000 7f8685366b80 1 -- 172.21.15.2:0/1741824912 wait complete. 2024-05-14T16:55:51.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.362+0000 7f8685366b80 20 client.15444 trim_cache size 0 max 16384 2024-05-14T16:55:51.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.382+0000 7f8685366b80 1 Processor -- start 2024-05-14T16:55:51.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.382+0000 7f8685366b80 1 -- start start 2024-05-14T16:55:51.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.382+0000 7f8685366b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ee29132c0 0x555ee29080d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:51.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.382+0000 7f8685366b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ee2917420 0x555ee2910c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:51.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.382+0000 7f8685366b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ee2917d40 0x555ee29111d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:51.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.382+0000 7f8685366b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f86741b0690 con 0x555ee2917420 2024-05-14T16:55:51.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.382+0000 7f8685366b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f86741adef0 con 0x555ee2917d40 2024-05-14T16:55:51.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.382+0000 7f8685366b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f86741abd40 con 0x555ee29132c0 2024-05-14T16:55:51.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.382+0000 7f8673fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ee2917420 0x555ee2910c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:51.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.382+0000 7f8673fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ee2917420 0x555ee2910c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34890/0 (socket says 172.21.15.2:34890) 2024-05-14T16:55:51.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.382+0000 7f8673fff700 1 -- 172.21.15.2:0/655977808 learned_addr learned my addr 172.21.15.2:0/655977808 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:51.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.382+0000 7f867906d700 1 --2- 172.21.15.2:0/655977808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ee2917d40 0x555ee29111d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:51.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.382+0000 7f867886c700 1 --2- 172.21.15.2:0/655977808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ee29132c0 0x555ee29080d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:51.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.383+0000 7f8673fff700 1 -- 172.21.15.2:0/655977808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ee29132c0 msgr2=0x555ee29080d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:51.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.383+0000 7f8673fff700 1 --2- 172.21.15.2:0/655977808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ee29132c0 0x555ee29080d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:51.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.383+0000 7f8673fff700 1 -- 172.21.15.2:0/655977808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ee2917d40 msgr2=0x555ee29111d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:51.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.383+0000 7f8673fff700 1 --2- 172.21.15.2:0/655977808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ee2917d40 0x555ee29111d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:51.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.383+0000 7f8673fff700 1 -- 172.21.15.2:0/655977808 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ee2929b30 con 0x555ee2917420 2024-05-14T16:55:51.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.383+0000 7f867886c700 1 --2- 172.21.15.2:0/655977808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ee29132c0 0x555ee29080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:55:51.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.383+0000 7f8673fff700 1 --2- 172.21.15.2:0/655977808 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ee2917420 0x555ee2910c90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f865c0a6b20 tx=0x7f865c0a63b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a06b79875e43e7c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:51.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.383+0000 7f866a7fc700 1 -- 172.21.15.2:0/655977808 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c0aa600 con 0x555ee2917420 2024-05-14T16:55:51.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.383+0000 7f866a7fc700 1 -- 172.21.15.2:0/655977808 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f865c08be50 con 0x555ee2917420 2024-05-14T16:55:51.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.383+0000 7f866a7fc700 1 -- 172.21.15.2:0/655977808 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c0aeea0 con 0x555ee2917420 2024-05-14T16:55:51.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.384+0000 7f8685366b80 1 -- 172.21.15.2:0/655977808 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ee2917420 msgr2=0x555ee2910c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:51.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.384+0000 7f8685366b80 1 --2- 172.21.15.2:0/655977808 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ee2917420 0x555ee2910c90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f865c0a6b20 tx=0x7f865c0a63b0 comp rx=0 tx=0).stop 2024-05-14T16:55:51.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.384+0000 7f8685366b80 1 -- 172.21.15.2:0/655977808 shutdown_connections 2024-05-14T16:55:51.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.384+0000 7f8685366b80 1 --2- 172.21.15.2:0/655977808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ee2917d40 0x555ee29111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:51.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.384+0000 7f8685366b80 1 --2- 172.21.15.2:0/655977808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ee29132c0 0x555ee29080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:51.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.384+0000 7f8685366b80 1 --2- 172.21.15.2:0/655977808 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ee2917420 0x555ee2910c90 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:51.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.384+0000 7f8685366b80 1 -- 172.21.15.2:0/655977808 >> 172.21.15.2:0/655977808 conn(0x555ee283feb0 msgr2=0x555ee2911890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:51.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.384+0000 7f8685366b80 1 -- 172.21.15.2:0/655977808 shutdown_connections 2024-05-14T16:55:51.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.384+0000 7f8685366b80 1 -- 172.21.15.2:0/655977808 wait complete. 2024-05-14T16:55:51.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.384+0000 7f8685366b80 1 -- start start 2024-05-14T16:55:51.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.384+0000 7f8685366b80 1 Processor -- start 2024-05-14T16:55:51.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.384+0000 7f8685366b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ee29132c0 0x555ee291f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:51.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.384+0000 7f8685366b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ee2917420 0x555ee291fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:51.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.384+0000 7f8685366b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ee2917d40 0x555ee2920190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:51.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.385+0000 7f8685366b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f86741ac3b0 con 0x555ee2917d40 2024-05-14T16:55:51.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.385+0000 7f8685366b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f86741ad820 con 0x555ee29132c0 2024-05-14T16:55:51.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.385+0000 7f8685366b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f86741aca80 con 0x555ee2917420 2024-05-14T16:55:51.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.385+0000 7f8685366b80 10 client.0 _ll_register_callbacks cb 0x555ee27fba50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:55:51.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.385+0000 7f867906d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ee2917d40 0x555ee2920190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:51.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.385+0000 7f867906d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ee2917d40 0x555ee2920190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34896/0 (socket says 172.21.15.2:34896) 2024-05-14T16:55:51.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.385+0000 7f867906d700 1 -- 172.21.15.2:0/2456211181 learned_addr learned my addr 172.21.15.2:0/2456211181 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:55:51.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.385+0000 7f8673fff700 1 --2- 172.21.15.2:0/2456211181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ee2917420 0x555ee291fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:51.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.385+0000 7f867886c700 1 --2- 172.21.15.2:0/2456211181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ee29132c0 0x555ee291f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:51.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.385+0000 7f867906d700 1 -- 172.21.15.2:0/2456211181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ee2917420 msgr2=0x555ee291fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:51.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.385+0000 7f867906d700 1 --2- 172.21.15.2:0/2456211181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ee2917420 0x555ee291fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:51.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.385+0000 7f867906d700 1 -- 172.21.15.2:0/2456211181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ee29132c0 msgr2=0x555ee291f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:55:51.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.385+0000 7f867906d700 1 --2- 172.21.15.2:0/2456211181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ee29132c0 0x555ee291f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:51.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.385+0000 7f867906d700 1 -- 172.21.15.2:0/2456211181 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ee2996bf0 con 0x555ee2917d40 2024-05-14T16:55:51.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.385+0000 7f867886c700 1 --2- 172.21.15.2:0/2456211181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ee29132c0 0x555ee291f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:55:51.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.385+0000 7f867906d700 1 --2- 172.21.15.2:0/2456211181 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ee2917d40 0x555ee2920190 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f866c00ba60 tx=0x7f866c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f6b342e10b1533c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:55:51.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.385+0000 7f8653fff700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:55:51.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.385+0000 7f8653fff700 1 -- 172.21.15.2:0/2456211181 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866c003e00 con 0x555ee2917d40 2024-05-14T16:55:51.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.385+0000 7f8653fff700 1 -- 172.21.15.2:0/2456211181 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f866c004830 con 0x555ee2917d40 2024-05-14T16:55:51.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.385+0000 7f8685366b80 1 -- 172.21.15.2:0/2456211181 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f865c08be50 con 0x555ee2917d40 2024-05-14T16:55:51.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.386+0000 7f8653fff700 1 -- 172.21.15.2:0/2456211181 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f866c004a30 con 0x555ee2917d40 2024-05-14T16:55:51.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.386+0000 7f8685366b80 10 client.15450 fetch_fsmap learned FSMap version 28 2024-05-14T16:55:51.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.386+0000 7f8685366b80 1 -- 172.21.15.2:0/2456211181 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555ee2997e80 con 0x555ee2917d40 2024-05-14T16:55:51.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.386+0000 7f8653fff700 1 -- 172.21.15.2:0/2456211181 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f865c08be50 con 0x555ee2917d40 2024-05-14T16:55:51.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.386+0000 7f8653fff700 1 -- 172.21.15.2:0/2456211181 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 28) ==== 8350+0+0 (secure 0 0 0) 0x7f866c01b070 con 0x555ee2917d40 2024-05-14T16:55:51.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.386+0000 7f8685366b80 10 client.15450 fetch_fsmap finished waiting for FSMap version 28 2024-05-14T16:55:51.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.386+0000 7f8685366b80 10 client.15450 resolve_mds: validated gid 24479 aka daemon mds.b 2024-05-14T16:55:51.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.386+0000 7f8685366b80 20 client.15450 populate_metadata read hostname 'smithi002' 2024-05-14T16:55:51.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.386+0000 7f8685366b80 1 --2- 172.21.15.2:0/2456211181 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x555ee2998640 0x555ee299aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:55:51.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.386+0000 7f8685366b80 4 client.15450 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "822ccb9f", "release": true, "await": true}] 2024-05-14T16:55:51.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.386+0000 7f8685366b80 1 -- 172.21.15.2:0/2456211181 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "quiesce db", "set_id": "822ccb9f", "release": true, "await": true}) -- 0x555ee286c5a0 con 0x555ee2998640 2024-05-14T16:55:51.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.386+0000 7f867886c700 1 --2- 172.21.15.2:0/2456211181 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x555ee2998640 0x555ee299aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:55:51.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.387+0000 7f867886c700 1 --2- 172.21.15.2:0/2456211181 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x555ee2998640 0x555ee299aa60 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=817ac0b23e1919cf server_cookie=c89657bfea5ffbbe in_seq=0 out_seq=0 2024-05-14T16:55:51.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.387+0000 7f8653fff700 10 client.15450 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:55:51.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.408+0000 7f8653fff700 1 -- 172.21.15.2:0/2456211181 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x555ee286c5a0 con 0x555ee2998640 2024-05-14T16:55:51.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.408+0000 7f8653fff700 10 client.15450 handle_command_reply: tid=0 2024-05-14T16:55:51.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.408+0000 7f8685366b80 1 client.15450 shutdown 2024-05-14T16:55:51.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.408+0000 7f8685366b80 1 -- 172.21.15.2:0/2456211181 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ee2917d40 msgr2=0x555ee2920190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:51.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.408+0000 7f8685366b80 1 --2- 172.21.15.2:0/2456211181 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ee2917d40 0x555ee2920190 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f866c00ba60 tx=0x7f866c00bd70 comp rx=0 tx=0).stop 2024-05-14T16:55:51.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.408+0000 7f8685366b80 1 -- 172.21.15.2:0/2456211181 shutdown_connections 2024-05-14T16:55:51.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.408+0000 7f8685366b80 1 --2- 172.21.15.2:0/2456211181 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x555ee2998640 0x555ee299aa60 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:51.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.408+0000 7f8685366b80 1 --2- 172.21.15.2:0/2456211181 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ee2917d40 0x555ee2920190 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:51.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.408+0000 7f8685366b80 1 --2- 172.21.15.2:0/2456211181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ee29132c0 0x555ee291f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:51.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.408+0000 7f8685366b80 1 --2- 172.21.15.2:0/2456211181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555ee2917420 0x555ee291fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:51.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.408+0000 7f8653fff700 0 client.15450 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:55:51.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.408+0000 7f8685366b80 1 -- 172.21.15.2:0/2456211181 >> 172.21.15.2:0/2456211181 conn(0x555ee283feb0 msgr2=0x555ee29120c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:51.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.409+0000 7f8685366b80 1 -- 172.21.15.2:0/2456211181 shutdown_connections 2024-05-14T16:55:51.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.409+0000 7f8685366b80 1 -- 172.21.15.2:0/2456211181 wait complete. 2024-05-14T16:55:51.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.409+0000 7f8685366b80 20 client.15450 trim_cache size 0 max 16384 2024-05-14T16:55:51.410 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T16:55:51.410 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 28, 2024-05-14T16:55:51.410 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24479, 2024-05-14T16:55:51.410 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 4, 2024-05-14T16:55:51.410 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-05-14T16:55:51.410 INFO:teuthology.orchestra.run.smithi002.stdout: "822ccb9f": { 2024-05-14T16:55:51.410 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 4, 2024-05-14T16:55:51.410 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-05-14T16:55:51.410 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-05-14T16:55:51.410 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-05-14T16:55:51.411 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-05-14T16:55:51.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T16:55:51.411 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-05-14T16:55:51.411 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 135.2, 2024-05-14T16:55:51.411 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-05-14T16:55:51.411 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-05-14T16:55:51.411 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-05-14T16:55:51.411 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-05-14T16:55:51.411 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-05-14T16:55:51.412 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-05-14T16:55:51.412 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:55:51.412 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:55:51.412 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:55:51.412 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:55:51.412 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:55:51.412 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T16:55:51.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.409+0000 7f867aad0700 1 -- 172.21.15.2:0/1760210834 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f86600b2b10 msgr2=0x7f86600b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:51.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.409+0000 7f867aad0700 1 --2- 172.21.15.2:0/1760210834 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f86600b2b10 0x7f86600b4fd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f86640045b0 tx=0x7f866400b3c0 comp rx=0 tx=0).stop 2024-05-14T16:55:51.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.409+0000 7f867aad0700 1 -- 172.21.15.2:0/1760210834 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8674103fc0 msgr2=0x7f867419a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:55:51.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.409+0000 7f867aad0700 1 --2- 172.21.15.2:0/1760210834 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8674103fc0 0x7f867419a1d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f865c00d010 tx=0x7f865c007a20 comp rx=0 tx=0).stop 2024-05-14T16:55:51.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.409+0000 7f867aad0700 1 -- 172.21.15.2:0/1760210834 shutdown_connections 2024-05-14T16:55:51.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.409+0000 7f867aad0700 1 --2- 172.21.15.2:0/1760210834 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f86600b2b10 0x7f86600b4fd0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:51.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.409+0000 7f867aad0700 1 --2- 172.21.15.2:0/1760210834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86741095e0 0x7f867419e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:51.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.409+0000 7f867aad0700 1 --2- 172.21.15.2:0/1760210834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8674103600 0x7f8674199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:51.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.409+0000 7f867aad0700 1 --2- 172.21.15.2:0/1760210834 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8674103fc0 0x7f867419a1d0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:55:51.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.409+0000 7f867aad0700 1 -- 172.21.15.2:0/1760210834 >> 172.21.15.2:0/1760210834 conn(0x7f867407a700 msgr2=0x7f86741022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:55:51.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.409+0000 7f867aad0700 1 -- 172.21.15.2:0/1760210834 shutdown_connections 2024-05-14T16:55:51.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:55:51.409+0000 7f867aad0700 1 -- 172.21.15.2:0/1760210834 wait complete. 2024-05-14T16:55:51.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 35.0 s, 246.4 tps, lat 937.315 ms stddev 3078.042 2024-05-14T16:55:51.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:51 smithi123 ceph-mon[35971]: pgmap v38: 97 pgs: 97 active+clean; 958 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 14 MiB/s wr, 369 op/s 2024-05-14T16:55:51.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:51 smithi002 ceph-mon[31538]: pgmap v38: 97 pgs: 97 active+clean; 958 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 14 MiB/s wr, 369 op/s 2024-05-14T16:55:52.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:51 smithi151 ceph-mon[36066]: pgmap v38: 97 pgs: 97 active+clean; 958 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 14 MiB/s wr, 369 op/s 2024-05-14T16:55:52.172 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '822ccb9f', total seconds elapsed: 21.2 2024-05-14T16:55:54.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:54 smithi151 ceph-mon[36066]: pgmap v39: 97 pgs: 97 active+clean; 959 MiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 15 MiB/s wr, 449 op/s 2024-05-14T16:55:54.611 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:54 smithi123 ceph-mon[35971]: pgmap v39: 97 pgs: 97 active+clean; 959 MiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 15 MiB/s wr, 449 op/s 2024-05-14T16:55:54.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:54 smithi002 ceph-mon[31538]: pgmap v39: 97 pgs: 97 active+clean; 959 MiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 15 MiB/s wr, 449 op/s 2024-05-14T16:55:56.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:56 smithi151 ceph-mon[36066]: pgmap v40: 97 pgs: 97 active+clean; 959 MiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.9 MiB/s wr, 92 op/s 2024-05-14T16:55:56.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:56 smithi123 ceph-mon[35971]: pgmap v40: 97 pgs: 97 active+clean; 959 MiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.9 MiB/s wr, 92 op/s 2024-05-14T16:55:56.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:56 smithi002 ceph-mon[31538]: pgmap v40: 97 pgs: 97 active+clean; 959 MiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.9 MiB/s wr, 92 op/s 2024-05-14T16:55:56.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 40.0 s, 4056.8 tps, lat 24.623 ms stddev 24.113 2024-05-14T16:55:58.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:55:58 smithi151 ceph-mon[36066]: pgmap v41: 97 pgs: 97 active+clean; 972 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.2 MiB/s wr, 406 op/s 2024-05-14T16:55:58.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:55:58 smithi123 ceph-mon[35971]: pgmap v41: 97 pgs: 97 active+clean; 972 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.2 MiB/s wr, 406 op/s 2024-05-14T16:55:58.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:55:58 smithi002 ceph-mon[31538]: pgmap v41: 97 pgs: 97 active+clean; 972 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.2 MiB/s wr, 406 op/s 2024-05-14T16:56:00.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:56:00 smithi151 ceph-mon[36066]: pgmap v42: 97 pgs: 97 active+clean; 978 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.9 MiB/s wr, 412 op/s 2024-05-14T16:56:00.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:56:00 smithi123 ceph-mon[35971]: pgmap v42: 97 pgs: 97 active+clean; 978 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.9 MiB/s wr, 412 op/s 2024-05-14T16:56:00.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:56:00 smithi002 ceph-mon[31538]: pgmap v42: 97 pgs: 97 active+clean; 978 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.9 MiB/s wr, 412 op/s 2024-05-14T16:56:01.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 45.0 s, 3362.6 tps, lat 28.788 ms stddev 35.857 2024-05-14T16:56:02.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:56:02 smithi151 ceph-mon[36066]: pgmap v43: 97 pgs: 97 active+clean; 978 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.9 MiB/s wr, 412 op/s 2024-05-14T16:56:02.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:56:02 smithi123 ceph-mon[35971]: pgmap v43: 97 pgs: 97 active+clean; 978 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.9 MiB/s wr, 412 op/s 2024-05-14T16:56:02.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:56:02 smithi002 ceph-mon[31538]: pgmap v43: 97 pgs: 97 active+clean; 978 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.9 MiB/s wr, 412 op/s 2024-05-14T16:56:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.782+0000 7fc2124a0640 1 -- 172.21.15.2:0/3826040530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc20c101910 msgr2=0x7fc20c10f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.782+0000 7fc2124a0640 1 --2- 172.21.15.2:0/3826040530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc20c101910 0x7fc20c10f8e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc204009cd0 tx=0x7fc20402a8e0 comp rx=0 tx=0).stop 2024-05-14T16:56:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.782+0000 7fc2124a0640 1 -- 172.21.15.2:0/3826040530 shutdown_connections 2024-05-14T16:56:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.782+0000 7fc2124a0640 1 --2- 172.21.15.2:0/3826040530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc20c101910 0x7fc20c10f8e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.782+0000 7fc2124a0640 1 --2- 172.21.15.2:0/3826040530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc20c100f50 0x7fc20c1013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.782+0000 7fc2124a0640 1 --2- 172.21.15.2:0/3826040530 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc20c107a30 0x7fc20c107e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.782+0000 7fc2124a0640 1 -- 172.21.15.2:0/3826040530 >> 172.21.15.2:0/3826040530 conn(0x7fc20c077ee0 msgr2=0x7fc20c0ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.782+0000 7fc2124a0640 1 -- 172.21.15.2:0/3826040530 shutdown_connections 2024-05-14T16:56:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.782+0000 7fc2124a0640 1 -- 172.21.15.2:0/3826040530 wait complete. 2024-05-14T16:56:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.783+0000 7fc2124a0640 1 Processor -- start 2024-05-14T16:56:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.783+0000 7fc2124a0640 1 -- start start 2024-05-14T16:56:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.783+0000 7fc2124a0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc20c100f50 0x7fc20c199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.783+0000 7fc2124a0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc20c101910 0x7fc20c199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.783+0000 7fc2124a0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc20c107a30 0x7fc20c19daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.783+0000 7fc2124a0640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c112df0 con 0x7fc20c100f50 2024-05-14T16:56:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.783+0000 7fc2124a0640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c112c70 con 0x7fc20c101910 2024-05-14T16:56:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.783+0000 7fc2124a0640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c112f70 con 0x7fc20c107a30 2024-05-14T16:56:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.783+0000 7fc20bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc20c100f50 0x7fc20c199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.783+0000 7fc20bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc20c100f50 0x7fc20c199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50836/0 (socket says 172.21.15.2:50836) 2024-05-14T16:56:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.783+0000 7fc20bfff640 1 -- 172.21.15.2:0/2432999876 learned_addr learned my addr 172.21.15.2:0/2432999876 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:56:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.783+0000 7fc203fff640 1 --2- 172.21.15.2:0/2432999876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc20c101910 0x7fc20c199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.783+0000 7fc210a3d640 1 --2- 172.21.15.2:0/2432999876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc20c107a30 0x7fc20c19daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.784+0000 7fc203fff640 1 -- 172.21.15.2:0/2432999876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc20c107a30 msgr2=0x7fc20c19daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.784+0000 7fc203fff640 1 --2- 172.21.15.2:0/2432999876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc20c107a30 0x7fc20c19daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.784+0000 7fc203fff640 1 -- 172.21.15.2:0/2432999876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc20c100f50 msgr2=0x7fc20c199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.784+0000 7fc203fff640 1 --2- 172.21.15.2:0/2432999876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc20c100f50 0x7fc20c199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.784+0000 7fc203fff640 1 -- 172.21.15.2:0/2432999876 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc20c19e220 con 0x7fc20c101910 2024-05-14T16:56:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.784+0000 7fc20bfff640 1 --2- 172.21.15.2:0/2432999876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc20c100f50 0x7fc20c199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:56:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.784+0000 7fc203fff640 1 --2- 172.21.15.2:0/2432999876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc20c101910 0x7fc20c199770 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc1f400e9f0 tx=0x7fc1f400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.784+0000 7fc209ffb640 1 -- 172.21.15.2:0/2432999876 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f400ccc0 con 0x7fc20c101910 2024-05-14T16:56:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.784+0000 7fc209ffb640 1 -- 172.21.15.2:0/2432999876 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1f40108e0 con 0x7fc20c101910 2024-05-14T16:56:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.784+0000 7fc2124a0640 1 -- 172.21.15.2:0/2432999876 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc20c19e510 con 0x7fc20c101910 2024-05-14T16:56:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.784+0000 7fc209ffb640 1 -- 172.21.15.2:0/2432999876 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4018880 con 0x7fc20c101910 2024-05-14T16:56:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.784+0000 7fc2124a0640 1 -- 172.21.15.2:0/2432999876 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc20c1a5da0 con 0x7fc20c101910 2024-05-14T16:56:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.787+0000 7fc209ffb640 1 -- 172.21.15.2:0/2432999876 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 149884+0+0 (secure 0 0 0) 0x7fc1f4010430 con 0x7fc20c101910 2024-05-14T16:56:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.787+0000 7fc2151be740 1 Processor -- start 2024-05-14T16:56:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.787+0000 7fc2151be740 1 -- start start 2024-05-14T16:56:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.788+0000 7fc2151be740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3f08ee8f0 0x55f3f08eecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.788+0000 7fc2151be740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3f08ef210 0x55f3f08f9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.788+0000 7fc2151be740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3f08f9ad0 0x55f3f08fbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.788+0000 7fc2151be740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f3f0861c30 con 0x55f3f08ee8f0 2024-05-14T16:56:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.788+0000 7fc2151be740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f3f08517d0 con 0x55f3f08ef210 2024-05-14T16:56:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.788+0000 7fc2151be740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f3f0855810 con 0x55f3f08f9ad0 2024-05-14T16:56:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.788+0000 7fc20bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3f08ef210 0x55f3f08f9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.788+0000 7fc20bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3f08ef210 0x55f3f08f9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:57352/0 (socket says 172.21.15.2:57352) 2024-05-14T16:56:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.788+0000 7fc20bfff640 1 -- 172.21.15.2:0/245973465 learned_addr learned my addr 172.21.15.2:0/245973465 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:56:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.788+0000 7fc210a3d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3f08ee8f0 0x55f3f08eecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.788+0000 7fc203fff640 1 --2- 172.21.15.2:0/245973465 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3f08f9ad0 0x55f3f08fbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.788+0000 7fc210a3d640 1 -- 172.21.15.2:0/245973465 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3f08f9ad0 msgr2=0x55f3f08fbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.788+0000 7fc210a3d640 1 --2- 172.21.15.2:0/245973465 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3f08f9ad0 0x55f3f08fbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.788+0000 7fc210a3d640 1 -- 172.21.15.2:0/245973465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3f08ef210 msgr2=0x55f3f08f9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.788+0000 7fc210a3d640 1 --2- 172.21.15.2:0/245973465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3f08ef210 0x55f3f08f9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.788+0000 7fc210a3d640 1 -- 172.21.15.2:0/245973465 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3f0566210 con 0x55f3f08ee8f0 2024-05-14T16:56:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.788+0000 7fc20bfff640 1 --2- 172.21.15.2:0/245973465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3f08ef210 0x55f3f08f9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:56:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.789+0000 7fc210a3d640 1 --2- 172.21.15.2:0/245973465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3f08ee8f0 0x55f3f08eecd0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc204009ca0 tx=0x7fc204009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e8e63f012785a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.789+0000 7fc209ffb640 1 --2- 172.21.15.2:0/2432999876 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc1f80b2b30 0x7fc1f80b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.789+0000 7fc20bfff640 1 --2- 172.21.15.2:0/2432999876 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc1f80b2b30 0x7fc1f80b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.789+0000 7fc209ffb640 1 -- 172.21.15.2:0/2432999876 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fc1f40e2bc0 con 0x7fc20c101910 2024-05-14T16:56:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.789+0000 7fc2027fc640 1 -- 172.21.15.2:0/245973465 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc204049070 con 0x55f3f08ee8f0 2024-05-14T16:56:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.789+0000 7fc2027fc640 1 -- 172.21.15.2:0/245973465 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc204036040 con 0x55f3f08ee8f0 2024-05-14T16:56:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.789+0000 7fc2027fc640 1 -- 172.21.15.2:0/245973465 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc204004040 con 0x55f3f08ee8f0 2024-05-14T16:56:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.790+0000 7fc20bfff640 1 --2- 172.21.15.2:0/2432999876 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc1f80b2b30 0x7fc1f80b4ff0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc009fd0 tx=0x7fc1fc005f90 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.790+0000 7fc2151be740 1 -- 172.21.15.2:0/245973465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3f08ee8f0 msgr2=0x55f3f08eecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.790+0000 7fc2151be740 1 --2- 172.21.15.2:0/245973465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3f08ee8f0 0x55f3f08eecd0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc204009ca0 tx=0x7fc204009a20 comp rx=0 tx=0).stop 2024-05-14T16:56:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.790+0000 7fc2151be740 1 -- 172.21.15.2:0/245973465 shutdown_connections 2024-05-14T16:56:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.790+0000 7fc2151be740 1 --2- 172.21.15.2:0/245973465 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3f08f9ad0 0x55f3f08fbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.790+0000 7fc2151be740 1 --2- 172.21.15.2:0/245973465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3f08ef210 0x55f3f08f9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.790+0000 7fc2151be740 1 --2- 172.21.15.2:0/245973465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3f08ee8f0 0x55f3f08eecd0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.790+0000 7fc2151be740 1 -- 172.21.15.2:0/245973465 >> 172.21.15.2:0/245973465 conn(0x55f3f08eae00 msgr2=0x55f3f08eb1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.790+0000 7fc2151be740 1 -- 172.21.15.2:0/245973465 shutdown_connections 2024-05-14T16:56:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.790+0000 7fc2151be740 1 -- 172.21.15.2:0/245973465 wait complete. 2024-05-14T16:56:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.790+0000 7fc2151be740 1 -- start start 2024-05-14T16:56:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.791+0000 7fc2151be740 1 Processor -- start 2024-05-14T16:56:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.791+0000 7fc2151be740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3f08ee8f0 0x55f3f08f9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.791+0000 7fc2151be740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3f08ef210 0x55f3f08f6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.791+0000 7fc2151be740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3f08f9ad0 0x55f3f08f6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.791+0000 7fc2151be740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f3f0537790 con 0x55f3f08f9ad0 2024-05-14T16:56:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.791+0000 7fc2151be740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f3f080d890 con 0x55f3f08ee8f0 2024-05-14T16:56:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.791+0000 7fc2151be740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f3f08fc740 con 0x55f3f08ef210 2024-05-14T16:56:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.791+0000 7fc2151be740 10 client.0 _ll_register_callbacks cb 0x55f3f0810d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:56:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.791+0000 7fc20bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3f08ee8f0 0x55f3f08f9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.791+0000 7fc210a3d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3f08f9ad0 0x55f3f08f6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.791+0000 7fc20bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3f08ee8f0 0x55f3f08f9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:57354/0 (socket says 172.21.15.2:57354) 2024-05-14T16:56:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.791+0000 7fc20bfff640 1 -- 172.21.15.2:0/3638436918 learned_addr learned my addr 172.21.15.2:0/3638436918 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:56:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.791+0000 7fc203fff640 1 --2- 172.21.15.2:0/3638436918 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3f08ef210 0x55f3f08f6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.791+0000 7fc20bfff640 1 -- 172.21.15.2:0/3638436918 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3f08ef210 msgr2=0x55f3f08f6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.791+0000 7fc20bfff640 1 --2- 172.21.15.2:0/3638436918 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3f08ef210 0x55f3f08f6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.792+0000 7fc20bfff640 1 -- 172.21.15.2:0/3638436918 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3f08f9ad0 msgr2=0x55f3f08f6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.792+0000 7fc20bfff640 1 --2- 172.21.15.2:0/3638436918 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3f08f9ad0 0x55f3f08f6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.792+0000 7fc20bfff640 1 -- 172.21.15.2:0/3638436918 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3f0632620 con 0x55f3f08ee8f0 2024-05-14T16:56:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.792+0000 7fc20bfff640 1 --2- 172.21.15.2:0/3638436918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3f08ee8f0 0x55f3f08f9440 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc0124e0 tx=0x7fc1fc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6805f6821edf7a86 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.792+0000 7fc1eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T16:56:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.792+0000 7fc1eb7fe640 1 -- 172.21.15.2:0/3638436918 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc011b50 con 0x55f3f08ee8f0 2024-05-14T16:56:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.792+0000 7fc1eb7fe640 1 -- 172.21.15.2:0/3638436918 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fc1fc011cf0 con 0x55f3f08ee8f0 2024-05-14T16:56:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.792+0000 7fc2151be740 1 -- 172.21.15.2:0/3638436918 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc204036040 con 0x55f3f08ee8f0 2024-05-14T16:56:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.792+0000 7fc1eb7fe640 1 -- 172.21.15.2:0/3638436918 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc1fc029440 con 0x55f3f08ee8f0 2024-05-14T16:56:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.792+0000 7fc2151be740 10 client.25199 fetch_fsmap learned FSMap version 29 2024-05-14T16:56:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.792+0000 7fc2151be740 1 -- 172.21.15.2:0/3638436918 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f3f0908250 con 0x55f3f08ee8f0 2024-05-14T16:56:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.792+0000 7fc1eb7fe640 1 -- 172.21.15.2:0/3638436918 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc204036040 con 0x55f3f08ee8f0 2024-05-14T16:56:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.793+0000 7fc1eb7fe640 1 -- 172.21.15.2:0/3638436918 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc1fc027070 con 0x55f3f08ee8f0 2024-05-14T16:56:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.793+0000 7fc2151be740 10 client.25199 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T16:56:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.793+0000 7fc2151be740 10 client.25199 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:56:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.793+0000 7fc2151be740 20 client.25199 populate_metadata read hostname 'smithi002' 2024-05-14T16:56:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.793+0000 7fc2151be740 1 --2- 172.21.15.2:0/3638436918 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55f3f09098c0 0x55f3f090bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.793+0000 7fc2151be740 4 client.25199 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:56:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.793+0000 7fc2151be740 1 -- 172.21.15.2:0/3638436918 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3f0863b80 con 0x55f3f09098c0 2024-05-14T16:56:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.793+0000 7fc203fff640 1 --2- 172.21.15.2:0/3638436918 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55f3f09098c0 0x55f3f090bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.793+0000 7fc1eb7fe640 10 client.25199 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:56:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.794+0000 7fc203fff640 1 --2- 172.21.15.2:0/3638436918 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55f3f09098c0 0x55f3f090bd40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=372f641a33ed3b11 server_cookie=ffcc599982ef8d0a in_seq=0 out_seq=0 2024-05-14T16:56:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.794+0000 7fc1eb7fe640 1 -- 172.21.15.2:0/3638436918 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f3f0863b80 con 0x55f3f09098c0 2024-05-14T16:56:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.794+0000 7fc1eb7fe640 10 client.25199 handle_command_reply: tid=0 2024-05-14T16:56:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.794+0000 7fc2151be740 1 client.25199 shutdown 2024-05-14T16:56:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.795+0000 7fc2151be740 1 -- 172.21.15.2:0/3638436918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3f08ee8f0 msgr2=0x55f3f08f9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.795+0000 7fc2151be740 1 --2- 172.21.15.2:0/3638436918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3f08ee8f0 0x55f3f08f9440 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc0124e0 tx=0x7fc1fc004730 comp rx=0 tx=0).stop 2024-05-14T16:56:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.795+0000 7fc2151be740 1 -- 172.21.15.2:0/3638436918 shutdown_connections 2024-05-14T16:56:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.795+0000 7fc2151be740 1 --2- 172.21.15.2:0/3638436918 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55f3f09098c0 0x55f3f090bd40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.795+0000 7fc2151be740 1 --2- 172.21.15.2:0/3638436918 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3f08f9ad0 0x55f3f08f6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.795+0000 7fc2151be740 1 --2- 172.21.15.2:0/3638436918 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3f08ef210 0x55f3f08f6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.795+0000 7fc2151be740 1 --2- 172.21.15.2:0/3638436918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3f08ee8f0 0x55f3f08f9440 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.795+0000 7fc2151be740 1 -- 172.21.15.2:0/3638436918 >> 172.21.15.2:0/3638436918 conn(0x55f3f08eae00 msgr2=0x55f3f08ea100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.795+0000 7fc1eb7fe640 0 client.25199 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:56:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.795+0000 7fc2151be740 1 -- 172.21.15.2:0/3638436918 shutdown_connections 2024-05-14T16:56:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.795+0000 7fc2151be740 1 -- 172.21.15.2:0/3638436918 wait complete. 2024-05-14T16:56:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.795+0000 7fc2151be740 20 client.25199 trim_cache size 0 max 16384 2024-05-14T16:56:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.812+0000 7fc2151be740 1 Processor -- start 2024-05-14T16:56:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.812+0000 7fc2151be740 1 -- start start 2024-05-14T16:56:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.812+0000 7fc2151be740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3f08ee8f0 0x55f3f08e5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.812+0000 7fc2151be740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3f08ef210 0x55f3f08f5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.812+0000 7fc2151be740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3f08f9ad0 0x55f3f0911610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.812+0000 7fc2151be740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c1aebc0 con 0x55f3f08ee8f0 2024-05-14T16:56:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.812+0000 7fc2151be740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c1ac430 con 0x55f3f08f9ad0 2024-05-14T16:56:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.812+0000 7fc2151be740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c1aa280 con 0x55f3f08ef210 2024-05-14T16:56:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.812+0000 7fc20bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3f08ee8f0 0x55f3f08e5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.812+0000 7fc203fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3f08ef210 0x55f3f08f5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.812+0000 7fc20bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3f08ee8f0 0x55f3f08e5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50872/0 (socket says 172.21.15.2:50872) 2024-05-14T16:56:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.812+0000 7fc20bfff640 1 -- 172.21.15.2:0/365584024 learned_addr learned my addr 172.21.15.2:0/365584024 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:56:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.812+0000 7fc203fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3f08ef210 0x55f3f08f5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:36460/0 (socket says 172.21.15.2:36460) 2024-05-14T16:56:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.812+0000 7fc210a3d640 1 --2- 172.21.15.2:0/365584024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3f08f9ad0 0x55f3f0911610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.812+0000 7fc20bfff640 1 -- 172.21.15.2:0/365584024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3f08ef210 msgr2=0x55f3f08f5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.812+0000 7fc20bfff640 1 --2- 172.21.15.2:0/365584024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3f08ef210 0x55f3f08f5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.812+0000 7fc20bfff640 1 -- 172.21.15.2:0/365584024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3f08f9ad0 msgr2=0x55f3f0911610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.812+0000 7fc20bfff640 1 --2- 172.21.15.2:0/365584024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3f08f9ad0 0x55f3f0911610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.812+0000 7fc20bfff640 1 -- 172.21.15.2:0/365584024 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3f08e9f50 con 0x55f3f08ee8f0 2024-05-14T16:56:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.812+0000 7fc203fff640 1 --2- 172.21.15.2:0/365584024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3f08ef210 0x55f3f08f5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:56:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.812+0000 7fc210a3d640 1 --2- 172.21.15.2:0/365584024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3f08f9ad0 0x55f3f0911610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:56:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.813+0000 7fc20bfff640 1 --2- 172.21.15.2:0/365584024 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3f08ee8f0 0x55f3f08e5f90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc01f630 tx=0x7fc1fc0092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4774c7db6a46352 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.813+0000 7fc201ffb640 1 -- 172.21.15.2:0/365584024 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc036030 con 0x55f3f08ee8f0 2024-05-14T16:56:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.813+0000 7fc201ffb640 1 -- 172.21.15.2:0/365584024 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1fc01f960 con 0x55f3f08ee8f0 2024-05-14T16:56:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.813+0000 7fc201ffb640 1 -- 172.21.15.2:0/365584024 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc029a60 con 0x55f3f08ee8f0 2024-05-14T16:56:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.814+0000 7fc2151be740 1 -- 172.21.15.2:0/365584024 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3f08ee8f0 msgr2=0x55f3f08e5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.814+0000 7fc2151be740 1 --2- 172.21.15.2:0/365584024 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3f08ee8f0 0x55f3f08e5f90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc01f630 tx=0x7fc1fc0092a0 comp rx=0 tx=0).stop 2024-05-14T16:56:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.814+0000 7fc2151be740 1 -- 172.21.15.2:0/365584024 shutdown_connections 2024-05-14T16:56:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.814+0000 7fc2151be740 1 --2- 172.21.15.2:0/365584024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3f08f9ad0 0x55f3f0911610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.814+0000 7fc2151be740 1 --2- 172.21.15.2:0/365584024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3f08ef210 0x55f3f08f5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.814+0000 7fc2151be740 1 --2- 172.21.15.2:0/365584024 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3f08ee8f0 0x55f3f08e5f90 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.814+0000 7fc2151be740 1 -- 172.21.15.2:0/365584024 >> 172.21.15.2:0/365584024 conn(0x55f3f08eae00 msgr2=0x55f3f0904600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.814+0000 7fc2151be740 1 -- 172.21.15.2:0/365584024 shutdown_connections 2024-05-14T16:56:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.814+0000 7fc2151be740 1 -- 172.21.15.2:0/365584024 wait complete. 2024-05-14T16:56:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.814+0000 7fc2151be740 1 -- start start 2024-05-14T16:56:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.814+0000 7fc2151be740 1 Processor -- start 2024-05-14T16:56:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.815+0000 7fc2151be740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3f08ee8f0 0x55f3f0901c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.815+0000 7fc2151be740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3f08ef210 0x55f3f08feff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.815+0000 7fc2151be740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3f08f9ad0 0x55f3f08ff530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.815+0000 7fc2151be740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c1aa8f0 con 0x55f3f08f9ad0 2024-05-14T16:56:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.815+0000 7fc2151be740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c1abd60 con 0x55f3f08ee8f0 2024-05-14T16:56:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.815+0000 7fc2151be740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c1aafc0 con 0x55f3f08ef210 2024-05-14T16:56:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.815+0000 7fc2151be740 10 client.0 _ll_register_callbacks cb 0x7fc1fc009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:56:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.815+0000 7fc210a3d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3f08f9ad0 0x55f3f08ff530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.815+0000 7fc20bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3f08ee8f0 0x55f3f0901c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.815+0000 7fc210a3d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3f08f9ad0 0x55f3f08ff530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50882/0 (socket says 172.21.15.2:50882) 2024-05-14T16:56:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.815+0000 7fc210a3d640 1 -- 172.21.15.2:0/3147889232 learned_addr learned my addr 172.21.15.2:0/3147889232 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:56:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.815+0000 7fc203fff640 1 --2- 172.21.15.2:0/3147889232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3f08ef210 0x55f3f08feff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.815+0000 7fc210a3d640 1 -- 172.21.15.2:0/3147889232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3f08ef210 msgr2=0x55f3f08feff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.815+0000 7fc210a3d640 1 --2- 172.21.15.2:0/3147889232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3f08ef210 0x55f3f08feff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.815+0000 7fc210a3d640 1 -- 172.21.15.2:0/3147889232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3f08ee8f0 msgr2=0x55f3f0901c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.815+0000 7fc210a3d640 1 --2- 172.21.15.2:0/3147889232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3f08ee8f0 0x55f3f0901c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.815+0000 7fc210a3d640 1 -- 172.21.15.2:0/3147889232 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3f08ffaa0 con 0x55f3f08f9ad0 2024-05-14T16:56:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.815+0000 7fc20bfff640 1 --2- 172.21.15.2:0/3147889232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3f08ee8f0 0x55f3f0901c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:56:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.815+0000 7fc203fff640 1 --2- 172.21.15.2:0/3147889232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3f08ef210 0x55f3f08feff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:56:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.815+0000 7fc210a3d640 1 --2- 172.21.15.2:0/3147889232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3f08f9ad0 0x55f3f08ff530 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc204009ca0 tx=0x7fc204031a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852f2dd0a41743ed server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.815+0000 7fc1eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:56:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.815+0000 7fc1eb7fe640 1 -- 172.21.15.2:0/3147889232 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc204049040 con 0x55f3f08f9ad0 2024-05-14T16:56:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.815+0000 7fc1eb7fe640 1 -- 172.21.15.2:0/3147889232 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fc204031bf0 con 0x55f3f08f9ad0 2024-05-14T16:56:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.815+0000 7fc2151be740 1 -- 172.21.15.2:0/3147889232 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1fc01f960 con 0x55f3f08f9ad0 2024-05-14T16:56:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.816+0000 7fc1eb7fe640 1 -- 172.21.15.2:0/3147889232 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc204009540 con 0x55f3f08f9ad0 2024-05-14T16:56:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.816+0000 7fc2151be740 10 client.15468 fetch_fsmap learned FSMap version 29 2024-05-14T16:56:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.816+0000 7fc2151be740 1 -- 172.21.15.2:0/3147889232 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f3f0973f60 con 0x55f3f08f9ad0 2024-05-14T16:56:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.816+0000 7fc1eb7fe640 1 -- 172.21.15.2:0/3147889232 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1fc01f960 con 0x55f3f08f9ad0 2024-05-14T16:56:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.816+0000 7fc1eb7fe640 1 -- 172.21.15.2:0/3147889232 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc204030070 con 0x55f3f08f9ad0 2024-05-14T16:56:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.816+0000 7fc2151be740 10 client.15468 fetch_fsmap finished waiting for FSMap version 29 2024-05-14T16:56:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.816+0000 7fc2151be740 10 client.15468 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:56:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.816+0000 7fc2151be740 20 client.15468 populate_metadata read hostname 'smithi002' 2024-05-14T16:56:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.816+0000 7fc2151be740 1 --2- 172.21.15.2:0/3147889232 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55f3f09742a0 0x55f3f0976690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.816+0000 7fc2151be740 4 client.15468 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:56:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.816+0000 7fc2151be740 1 -- 172.21.15.2:0/3147889232 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f3f085d360 con 0x55f3f09742a0 2024-05-14T16:56:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.816+0000 7fc20bfff640 1 --2- 172.21.15.2:0/3147889232 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55f3f09742a0 0x55f3f0976690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.817+0000 7fc20bfff640 1 --2- 172.21.15.2:0/3147889232 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55f3f09742a0 0x55f3f0976690 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44a5a1e499ba0690 server_cookie=82bbf4fc78b45822 in_seq=0 out_seq=0 2024-05-14T16:56:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:02.817+0000 7fc1eb7fe640 10 client.15468 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:56:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:03.457+0000 7fc1eb7fe640 1 -- 172.21.15.2:0/3147889232 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f3f085d360 con 0x55f3f09742a0 2024-05-14T16:56:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:03.457+0000 7fc1eb7fe640 10 client.15468 handle_command_reply: tid=0 2024-05-14T16:56:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:03.457+0000 7fc2151be740 1 client.15468 shutdown 2024-05-14T16:56:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:03.458+0000 7fc2151be740 1 -- 172.21.15.2:0/3147889232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3f08f9ad0 msgr2=0x55f3f08ff530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:03.458+0000 7fc2151be740 1 --2- 172.21.15.2:0/3147889232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3f08f9ad0 0x55f3f08ff530 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc204009ca0 tx=0x7fc204031a80 comp rx=0 tx=0).stop 2024-05-14T16:56:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:03.458+0000 7fc2151be740 1 -- 172.21.15.2:0/3147889232 shutdown_connections 2024-05-14T16:56:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:03.458+0000 7fc2151be740 1 --2- 172.21.15.2:0/3147889232 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55f3f09742a0 0x55f3f0976690 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:03.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:03.458+0000 7fc2151be740 1 --2- 172.21.15.2:0/3147889232 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f3f08f9ad0 0x55f3f08ff530 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:03.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:03.458+0000 7fc2151be740 1 --2- 172.21.15.2:0/3147889232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f3f08ef210 0x55f3f08feff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:03.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:03.458+0000 7fc2151be740 1 --2- 172.21.15.2:0/3147889232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3f08ee8f0 0x55f3f0901c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:03.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:03.458+0000 7fc2151be740 1 -- 172.21.15.2:0/3147889232 >> 172.21.15.2:0/3147889232 conn(0x55f3f08eae00 msgr2=0x55f3f0904600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:03.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:03.458+0000 7fc1eb7fe640 0 client.15468 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:56:03.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:03.458+0000 7fc2151be740 1 -- 172.21.15.2:0/3147889232 shutdown_connections 2024-05-14T16:56:03.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:03.458+0000 7fc2151be740 1 -- 172.21.15.2:0/3147889232 wait complete. 2024-05-14T16:56:03.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:03.458+0000 7fc2151be740 20 client.15468 trim_cache size 0 max 16384 2024-05-14T16:56:03.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T16:56:03.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T16:56:03.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T16:56:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:03.458+0000 7fc2124a0640 1 -- 172.21.15.2:0/2432999876 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc1f80b2b30 msgr2=0x7fc1f80b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:03.458+0000 7fc2124a0640 1 --2- 172.21.15.2:0/2432999876 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc1f80b2b30 0x7fc1f80b4ff0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc009fd0 tx=0x7fc1fc005f90 comp rx=0 tx=0).stop 2024-05-14T16:56:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:03.458+0000 7fc2124a0640 1 -- 172.21.15.2:0/2432999876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc20c101910 msgr2=0x7fc20c199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:03.458+0000 7fc2124a0640 1 --2- 172.21.15.2:0/2432999876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc20c101910 0x7fc20c199770 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc1f400e9f0 tx=0x7fc1f400eeb0 comp rx=0 tx=0).stop 2024-05-14T16:56:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:03.459+0000 7fc2124a0640 1 -- 172.21.15.2:0/2432999876 shutdown_connections 2024-05-14T16:56:03.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:03.459+0000 7fc2124a0640 1 --2- 172.21.15.2:0/2432999876 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc1f80b2b30 0x7fc1f80b4ff0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:03.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:03.459+0000 7fc2124a0640 1 --2- 172.21.15.2:0/2432999876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc20c107a30 0x7fc20c19daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:03.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:03.459+0000 7fc2124a0640 1 --2- 172.21.15.2:0/2432999876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc20c101910 0x7fc20c199770 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:03.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:03.459+0000 7fc2124a0640 1 --2- 172.21.15.2:0/2432999876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc20c100f50 0x7fc20c199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:03.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:03.459+0000 7fc2124a0640 1 -- 172.21.15.2:0/2432999876 >> 172.21.15.2:0/2432999876 conn(0x7fc20c077ee0 msgr2=0x7fc20c105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:03.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:03.459+0000 7fc2124a0640 1 -- 172.21.15.2:0/2432999876 shutdown_connections 2024-05-14T16:56:03.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:03.459+0000 7fc2124a0640 1 -- 172.21.15.2:0/2432999876 wait complete. 2024-05-14T16:56:03.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:56:03 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:56:03.644 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:56:03 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:56:03.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:56:03 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:56:04.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:56:04 smithi151 ceph-mon[36066]: pgmap v44: 97 pgs: 97 active+clean; 980 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.8 MiB/s wr, 767 op/s 2024-05-14T16:56:04.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:56:04 smithi123 ceph-mon[35971]: pgmap v44: 97 pgs: 97 active+clean; 980 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.8 MiB/s wr, 767 op/s 2024-05-14T16:56:04.660 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:56:04 smithi002 ceph-mon[31538]: pgmap v44: 97 pgs: 97 active+clean; 980 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.8 MiB/s wr, 767 op/s 2024-05-14T16:56:06.595 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:56:06 smithi002 ceph-mon[31538]: pgmap v45: 97 pgs: 97 active+clean; 980 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.6 MiB/s wr, 687 op/s 2024-05-14T16:56:06.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:56:06 smithi151 ceph-mon[36066]: pgmap v45: 97 pgs: 97 active+clean; 980 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.6 MiB/s wr, 687 op/s 2024-05-14T16:56:06.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:56:06 smithi123 ceph-mon[35971]: pgmap v45: 97 pgs: 97 active+clean; 980 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.6 MiB/s wr, 687 op/s 2024-05-14T16:56:06.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 50.0 s, 1771.4 tps, lat 58.273 ms stddev 112.519 2024-05-14T16:56:08.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:56:08 smithi151 ceph-mon[36066]: pgmap v46: 97 pgs: 97 active+clean; 1010 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 39 MiB/s wr, 1.66k op/s 2024-05-14T16:56:08.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:56:08 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:56:08.661 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:56:08 smithi123 ceph-mon[35971]: pgmap v46: 97 pgs: 97 active+clean; 1010 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 39 MiB/s wr, 1.66k op/s 2024-05-14T16:56:08.661 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:56:08 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:56:08.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:56:08 smithi002 ceph-mon[31538]: pgmap v46: 97 pgs: 97 active+clean; 1010 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 39 MiB/s wr, 1.66k op/s 2024-05-14T16:56:08.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:56:08 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:56:10.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:56:10 smithi151 ceph-mon[36066]: pgmap v47: 97 pgs: 97 active+clean; 1012 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 39 MiB/s wr, 1.42k op/s 2024-05-14T16:56:10.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:56:10 smithi123 ceph-mon[35971]: pgmap v47: 97 pgs: 97 active+clean; 1012 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 39 MiB/s wr, 1.42k op/s 2024-05-14T16:56:10.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:56:10 smithi002 ceph-mon[31538]: pgmap v47: 97 pgs: 97 active+clean; 1012 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 39 MiB/s wr, 1.42k op/s 2024-05-14T16:56:11.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 55.0 s, 3916.8 tps, lat 25.487 ms stddev 25.701 2024-05-14T16:56:12.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:56:12 smithi151 ceph-mon[36066]: pgmap v48: 97 pgs: 97 active+clean; 1012 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 38 MiB/s wr, 1.40k op/s 2024-05-14T16:56:12.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:56:12 smithi123 ceph-mon[35971]: pgmap v48: 97 pgs: 97 active+clean; 1012 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 38 MiB/s wr, 1.40k op/s 2024-05-14T16:56:12.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:56:12 smithi002 ceph-mon[31538]: pgmap v48: 97 pgs: 97 active+clean; 1012 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 38 MiB/s wr, 1.40k op/s 2024-05-14T16:56:14.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:56:14 smithi151 ceph-mon[36066]: pgmap v49: 97 pgs: 97 active+clean; 1.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 42 MiB/s wr, 1.74k op/s 2024-05-14T16:56:14.633 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:56:14 smithi151 ceph-mon[36066]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-05-14T16:56:14.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:56:14 smithi123 ceph-mon[35971]: pgmap v49: 97 pgs: 97 active+clean; 1.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 42 MiB/s wr, 1.74k op/s 2024-05-14T16:56:14.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:56:14 smithi123 ceph-mon[35971]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-05-14T16:56:14.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:56:14 smithi002 ceph-mon[31538]: pgmap v49: 97 pgs: 97 active+clean; 1.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 42 MiB/s wr, 1.74k op/s 2024-05-14T16:56:14.782 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:56:14 smithi002 ceph-mon[31538]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-05-14T16:56:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.595+0000 7fcbf2245640 1 -- 172.21.15.2:0/3490329290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcbec108a40 msgr2=0x7fcbec108e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.595+0000 7fcbf2245640 1 --2- 172.21.15.2:0/3490329290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcbec108a40 0x7fcbec108e20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009930 tx=0x7fcbdc02a420 comp rx=0 tx=0).stop 2024-05-14T16:56:16.625 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:56:16 smithi151 ceph-mon[36066]: pgmap v50: 97 pgs: 97 active+clean; 1.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 39 MiB/s wr, 1.38k op/s 2024-05-14T16:56:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.596+0000 7fcbf2245640 1 -- 172.21.15.2:0/3490329290 shutdown_connections 2024-05-14T16:56:16.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.596+0000 7fcbf2245640 1 --2- 172.21.15.2:0/3490329290 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcbec1033f0 0x7fcbec10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.596+0000 7fcbf2245640 1 --2- 172.21.15.2:0/3490329290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbec102a30 0x7fcbec102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.596+0000 7fcbf2245640 1 --2- 172.21.15.2:0/3490329290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcbec108a40 0x7fcbec108e20 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.596+0000 7fcbf2245640 1 -- 172.21.15.2:0/3490329290 >> 172.21.15.2:0/3490329290 conn(0x7fcbec0fe500 msgr2=0x7fcbec100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.596+0000 7fcbf2245640 1 -- 172.21.15.2:0/3490329290 shutdown_connections 2024-05-14T16:56:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.596+0000 7fcbf2245640 1 -- 172.21.15.2:0/3490329290 wait complete. 2024-05-14T16:56:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.597+0000 7fcbf2245640 1 Processor -- start 2024-05-14T16:56:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.597+0000 7fcbf2245640 1 -- start start 2024-05-14T16:56:16.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.597+0000 7fcbf2245640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcbec102a30 0x7fcbec199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:16.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.597+0000 7fcbf2245640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcbec1033f0 0x7fcbec199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.597+0000 7fcbf2245640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbec108a40 0x7fcbec19dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.597+0000 7fcbf2245640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcbec112d50 con 0x7fcbec1033f0 2024-05-14T16:56:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.597+0000 7fcbf2245640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcbec112bd0 con 0x7fcbec108a40 2024-05-14T16:56:16.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.597+0000 7fcbf2245640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcbec112ed0 con 0x7fcbec102a30 2024-05-14T16:56:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.597+0000 7fcbeaffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcbec1033f0 0x7fcbec199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:16.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.597+0000 7fcbeb7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcbec102a30 0x7fcbec199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.597+0000 7fcbeb7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcbec102a30 0x7fcbec199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:32858/0 (socket says 172.21.15.2:32858) 2024-05-14T16:56:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.597+0000 7fcbeb7fe640 1 -- 172.21.15.2:0/3503916839 learned_addr learned my addr 172.21.15.2:0/3503916839 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:56:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.597+0000 7fcbeaffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcbec1033f0 0x7fcbec199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32854/0 (socket says 172.21.15.2:32854) 2024-05-14T16:56:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.597+0000 7fcbebfff640 1 --2- 172.21.15.2:0/3503916839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbec108a40 0x7fcbec19dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:16.919 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.598+0000 7fcbeaffd640 1 -- 172.21.15.2:0/3503916839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcbec102a30 msgr2=0x7fcbec199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:16.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.598+0000 7fcbeaffd640 1 --2- 172.21.15.2:0/3503916839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcbec102a30 0x7fcbec199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:16.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.598+0000 7fcbeaffd640 1 -- 172.21.15.2:0/3503916839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbec108a40 msgr2=0x7fcbec19dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.598+0000 7fcbeaffd640 1 --2- 172.21.15.2:0/3503916839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbec108a40 0x7fcbec19dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.598+0000 7fcbeaffd640 1 -- 172.21.15.2:0/3503916839 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbec19e1f0 con 0x7fcbec1033f0 2024-05-14T16:56:16.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.598+0000 7fcbeaffd640 1 --2- 172.21.15.2:0/3503916839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcbec1033f0 0x7fcbec199790 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcbe000e9f0 tx=0x7fcbe000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.598+0000 7fcbe8ff9640 1 -- 172.21.15.2:0/3503916839 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe000ccc0 con 0x7fcbec1033f0 2024-05-14T16:56:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.598+0000 7fcbe8ff9640 1 -- 172.21.15.2:0/3503916839 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbe00108e0 con 0x7fcbec1033f0 2024-05-14T16:56:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.598+0000 7fcbe8ff9640 1 -- 172.21.15.2:0/3503916839 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe0018880 con 0x7fcbec1033f0 2024-05-14T16:56:16.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.598+0000 7fcbf2245640 1 -- 172.21.15.2:0/3503916839 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbec19e4e0 con 0x7fcbec1033f0 2024-05-14T16:56:16.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.598+0000 7fcbf2245640 1 -- 172.21.15.2:0/3503916839 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbec1a5d70 con 0x7fcbec1033f0 2024-05-14T16:56:17.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.602+0000 7fcbf4bfd740 1 Processor -- start 2024-05-14T16:56:17.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.602+0000 7fcbe8ff9640 1 -- 172.21.15.2:0/3503916839 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fcbe0010430 con 0x7fcbec1033f0 2024-05-14T16:56:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.602+0000 7fcbf4bfd740 1 -- start start 2024-05-14T16:56:17.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.602+0000 7fcbf4bfd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b1a8d57b0 0x555b1a8d5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.602+0000 7fcbf4bfd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b1a8d60d0 0x555b1a8e49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.602+0000 7fcbf4bfd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1a8e4ee0 0x555b1a8e72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.602+0000 7fcbf4bfd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x555b1a848c30 con 0x555b1a8d60d0 2024-05-14T16:56:17.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.602+0000 7fcbf4bfd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555b1a8387d0 con 0x555b1a8e4ee0 2024-05-14T16:56:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.602+0000 7fcbf4bfd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555b1a83c810 con 0x555b1a8d57b0 2024-05-14T16:56:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.602+0000 7fcbeb7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b1a8d60d0 0x555b1a8e49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.602+0000 7fcbeaffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1a8e4ee0 0x555b1a8e72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.602+0000 7fcbeb7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b1a8d60d0 0x555b1a8e49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32860/0 (socket says 172.21.15.2:32860) 2024-05-14T16:56:17.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.602+0000 7fcbebfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b1a8d57b0 0x555b1a8d5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.602+0000 7fcbeb7fe640 1 -- 172.21.15.2:0/2245438128 learned_addr learned my addr 172.21.15.2:0/2245438128 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:56:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.603+0000 7fcbeb7fe640 1 -- 172.21.15.2:0/2245438128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b1a8d57b0 msgr2=0x555b1a8d5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.603+0000 7fcbeb7fe640 1 --2- 172.21.15.2:0/2245438128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b1a8d57b0 0x555b1a8d5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.603+0000 7fcbeb7fe640 1 -- 172.21.15.2:0/2245438128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1a8e4ee0 msgr2=0x555b1a8e72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.603+0000 7fcbeb7fe640 1 --2- 172.21.15.2:0/2245438128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1a8e4ee0 0x555b1a8e72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.603+0000 7fcbeb7fe640 1 -- 172.21.15.2:0/2245438128 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b1a54d210 con 0x555b1a8d60d0 2024-05-14T16:56:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.603+0000 7fcbeb7fe640 1 --2- 172.21.15.2:0/2245438128 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b1a8d60d0 0x555b1a8e49a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009900 tx=0x7fcbdc004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf09d093dec103e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:17.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.604+0000 7fcbd1ffb640 1 -- 172.21.15.2:0/2245438128 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc051070 con 0x555b1a8d60d0 2024-05-14T16:56:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.604+0000 7fcbd1ffb640 1 -- 172.21.15.2:0/2245438128 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbdc0054d0 con 0x555b1a8d60d0 2024-05-14T16:56:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.604+0000 7fcbd1ffb640 1 -- 172.21.15.2:0/2245438128 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc035040 con 0x555b1a8d60d0 2024-05-14T16:56:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.604+0000 7fcbe8ff9640 1 --2- 172.21.15.2:0/3503916839 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcbd40b2a90 0x7fcbd40b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.605+0000 7fcbebfff640 1 --2- 172.21.15.2:0/3503916839 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcbd40b2a90 0x7fcbd40b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.605+0000 7fcbe8ff9640 1 -- 172.21.15.2:0/3503916839 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fcbe00e2900 con 0x7fcbec1033f0 2024-05-14T16:56:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.605+0000 7fcbf4bfd740 1 -- 172.21.15.2:0/2245438128 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b1a8d60d0 msgr2=0x555b1a8e49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.605+0000 7fcbf4bfd740 1 --2- 172.21.15.2:0/2245438128 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b1a8d60d0 0x555b1a8e49a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009900 tx=0x7fcbdc004180 comp rx=0 tx=0).stop 2024-05-14T16:56:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.605+0000 7fcbf4bfd740 1 -- 172.21.15.2:0/2245438128 shutdown_connections 2024-05-14T16:56:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.605+0000 7fcbf4bfd740 1 --2- 172.21.15.2:0/2245438128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1a8e4ee0 0x555b1a8e72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.605+0000 7fcbf4bfd740 1 --2- 172.21.15.2:0/2245438128 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b1a8d60d0 0x555b1a8e49a0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:17.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.605+0000 7fcbf4bfd740 1 --2- 172.21.15.2:0/2245438128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b1a8d57b0 0x555b1a8d5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.605+0000 7fcbf4bfd740 1 -- 172.21.15.2:0/2245438128 >> 172.21.15.2:0/2245438128 conn(0x555b1a8d1e00 msgr2=0x555b1a8d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:17.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.605+0000 7fcbf4bfd740 1 -- 172.21.15.2:0/2245438128 shutdown_connections 2024-05-14T16:56:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.605+0000 7fcbf4bfd740 1 -- 172.21.15.2:0/2245438128 wait complete. 2024-05-14T16:56:17.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.606+0000 7fcbebfff640 1 --2- 172.21.15.2:0/3503916839 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcbd40b2a90 0x7fcbd40b4f50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcbe400a530 tx=0x7fcbe400d040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:17.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.606+0000 7fcbf4bfd740 1 -- start start 2024-05-14T16:56:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.606+0000 7fcbf4bfd740 1 Processor -- start 2024-05-14T16:56:17.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.606+0000 7fcbf4bfd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b1a8d57b0 0x555b1a8ddd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.606+0000 7fcbf4bfd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1a8d60d0 0x555b1a8de290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.606+0000 7fcbf4bfd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b1a8e4ee0 0x555b1a8de7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.606+0000 7fcbf4bfd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x555b1a51e790 con 0x555b1a8e4ee0 2024-05-14T16:56:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.606+0000 7fcbf4bfd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555b1a7f4890 con 0x555b1a8d60d0 2024-05-14T16:56:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.606+0000 7fcbf4bfd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555b1a8d8800 con 0x555b1a8d57b0 2024-05-14T16:56:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.606+0000 7fcbf4bfd740 10 client.0 _ll_register_callbacks cb 0x555b1a7f7d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:56:17.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.606+0000 7fcbebfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b1a8e4ee0 0x555b1a8de7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.606+0000 7fcbebfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b1a8e4ee0 0x555b1a8de7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32872/0 (socket says 172.21.15.2:32872) 2024-05-14T16:56:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.606+0000 7fcbebfff640 1 -- 172.21.15.2:0/1428595507 learned_addr learned my addr 172.21.15.2:0/1428595507 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:56:17.664 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.606+0000 7fcbeb7fe640 1 --2- 172.21.15.2:0/1428595507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b1a8d57b0 0x555b1a8ddd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.607+0000 7fcbeaffd640 1 --2- 172.21.15.2:0/1428595507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1a8d60d0 0x555b1a8de290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:17.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.607+0000 7fcbebfff640 1 -- 172.21.15.2:0/1428595507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b1a8d57b0 msgr2=0x555b1a8ddd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.607+0000 7fcbebfff640 1 --2- 172.21.15.2:0/1428595507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b1a8d57b0 0x555b1a8ddd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:17.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.607+0000 7fcbebfff640 1 -- 172.21.15.2:0/1428595507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1a8d60d0 msgr2=0x555b1a8de290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.607+0000 7fcbebfff640 1 --2- 172.21.15.2:0/1428595507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1a8d60d0 0x555b1a8de290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:17.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.607+0000 7fcbebfff640 1 -- 172.21.15.2:0/1428595507 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b1a619620 con 0x555b1a8e4ee0 2024-05-14T16:56:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.607+0000 7fcbeaffd640 1 --2- 172.21.15.2:0/1428595507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1a8d60d0 0x555b1a8de290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:56:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.607+0000 7fcbeb7fe640 1 --2- 172.21.15.2:0/1428595507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b1a8d57b0 0x555b1a8ddd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:56:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.607+0000 7fcbebfff640 1 --2- 172.21.15.2:0/1428595507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b1a8e4ee0 0x555b1a8de7d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4007c90 tx=0x7fcbe4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a1d1444adc8f073 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:17.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.607+0000 7fcbcaffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:56:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.608+0000 7fcbcaffd640 1 -- 172.21.15.2:0/1428595507 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe4021460 con 0x555b1a8e4ee0 2024-05-14T16:56:17.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.608+0000 7fcbf4bfd740 1 -- 172.21.15.2:0/1428595507 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbdc0054d0 con 0x555b1a8e4ee0 2024-05-14T16:56:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.608+0000 7fcbcaffd640 1 -- 172.21.15.2:0/1428595507 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fcbe4021600 con 0x555b1a8e4ee0 2024-05-14T16:56:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.609+0000 7fcbcaffd640 1 -- 172.21.15.2:0/1428595507 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcbe402f050 con 0x555b1a8e4ee0 2024-05-14T16:56:17.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.609+0000 7fcbf4bfd740 10 client.15480 fetch_fsmap learned FSMap version 30 2024-05-14T16:56:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.609+0000 7fcbf4bfd740 1 -- 172.21.15.2:0/1428595507 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555b1a8ef2a0 con 0x555b1a8e4ee0 2024-05-14T16:56:17.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.609+0000 7fcbcaffd640 1 -- 172.21.15.2:0/1428595507 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbdc0054d0 con 0x555b1a8e4ee0 2024-05-14T16:56:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.610+0000 7fcbcaffd640 1 -- 172.21.15.2:0/1428595507 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fcbe402a810 con 0x555b1a8e4ee0 2024-05-14T16:56:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.610+0000 7fcbf4bfd740 10 client.15480 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T16:56:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.610+0000 7fcbf4bfd740 10 client.15480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:56:17.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.610+0000 7fcbf4bfd740 20 client.15480 populate_metadata read hostname 'smithi002' 2024-05-14T16:56:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.610+0000 7fcbf4bfd740 1 --2- 172.21.15.2:0/1428595507 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x555b1a8f07b0 0x555b1a8f2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.610+0000 7fcbf4bfd740 4 client.15480 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:56:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.610+0000 7fcbf4bfd740 1 -- 172.21.15.2:0/1428595507 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b1a84ab80 con 0x555b1a8f07b0 2024-05-14T16:56:18.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.610+0000 7fcbeb7fe640 1 --2- 172.21.15.2:0/1428595507 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x555b1a8f07b0 0x555b1a8f2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:18.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.610+0000 7fcbcaffd640 10 client.15480 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:56:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.610+0000 7fcbeb7fe640 1 --2- 172.21.15.2:0/1428595507 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x555b1a8f07b0 0x555b1a8f2bc0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72988ba0af8d3baf server_cookie=1fb73e026fb39668 in_seq=0 out_seq=0 2024-05-14T16:56:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.611+0000 7fcbcaffd640 1 -- 172.21.15.2:0/1428595507 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555b1a84ab80 con 0x555b1a8f07b0 2024-05-14T16:56:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.611+0000 7fcbcaffd640 10 client.15480 handle_command_reply: tid=0 2024-05-14T16:56:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.611+0000 7fcbf4bfd740 1 client.15480 shutdown 2024-05-14T16:56:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.611+0000 7fcbf4bfd740 1 -- 172.21.15.2:0/1428595507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b1a8e4ee0 msgr2=0x555b1a8de7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.611+0000 7fcbf4bfd740 1 --2- 172.21.15.2:0/1428595507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b1a8e4ee0 0x555b1a8de7d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4007c90 tx=0x7fcbe4004a40 comp rx=0 tx=0).stop 2024-05-14T16:56:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.611+0000 7fcbf4bfd740 1 -- 172.21.15.2:0/1428595507 shutdown_connections 2024-05-14T16:56:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.611+0000 7fcbf4bfd740 1 --2- 172.21.15.2:0/1428595507 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x555b1a8f07b0 0x555b1a8f2bc0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.611+0000 7fcbf4bfd740 1 --2- 172.21.15.2:0/1428595507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b1a8e4ee0 0x555b1a8de7d0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.611+0000 7fcbf4bfd740 1 --2- 172.21.15.2:0/1428595507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1a8d60d0 0x555b1a8de290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.611+0000 7fcbcaffd640 0 client.15480 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:56:18.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.611+0000 7fcbf4bfd740 1 --2- 172.21.15.2:0/1428595507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b1a8d57b0 0x555b1a8ddd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:18.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.611+0000 7fcbf4bfd740 1 -- 172.21.15.2:0/1428595507 >> 172.21.15.2:0/1428595507 conn(0x555b1a8d1e00 msgr2=0x555b1a8d1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.611+0000 7fcbf4bfd740 1 -- 172.21.15.2:0/1428595507 shutdown_connections 2024-05-14T16:56:18.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.611+0000 7fcbf4bfd740 1 -- 172.21.15.2:0/1428595507 wait complete. 2024-05-14T16:56:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.612+0000 7fcbf4bfd740 20 client.15480 trim_cache size 0 max 16384 2024-05-14T16:56:18.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.639+0000 7fcbf4bfd740 1 Processor -- start 2024-05-14T16:56:18.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.639+0000 7fcbf4bfd740 1 -- start start 2024-05-14T16:56:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.639+0000 7fcbf4bfd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1a8d57b0 0x555b1a8e2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.640+0000 7fcbf4bfd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b1a8d60d0 0x555b1a8f84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.640+0000 7fcbf4bfd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b1a8e4ee0 0x555b1a8f8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.640+0000 7fcbeaffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b1a8d60d0 0x555b1a8f84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.640+0000 7fcbf4bfd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcbec1aee30 con 0x555b1a8e4ee0 2024-05-14T16:56:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.640+0000 7fcbeaffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b1a8d60d0 0x555b1a8f84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:32898/0 (socket says 172.21.15.2:32898) 2024-05-14T16:56:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.640+0000 7fcbeaffd640 1 -- 172.21.15.2:0/2188354594 learned_addr learned my addr 172.21.15.2:0/2188354594 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:56:18.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.640+0000 7fcbebfff640 1 --2- 172.21.15.2:0/2188354594 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b1a8e4ee0 0x555b1a8f8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.640+0000 7fcbf4bfd740 1 -- 172.21.15.2:0/2188354594 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcbec1ac670 con 0x555b1a8d57b0 2024-05-14T16:56:18.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.640+0000 7fcbf4bfd740 1 -- 172.21.15.2:0/2188354594 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcbec1aa4c0 con 0x555b1a8d60d0 2024-05-14T16:56:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.640+0000 7fcbeb7fe640 1 --2- 172.21.15.2:0/2188354594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1a8d57b0 0x555b1a8e2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:18.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.640+0000 7fcbeaffd640 1 -- 172.21.15.2:0/2188354594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1a8d57b0 msgr2=0x555b1a8e2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:18.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.640+0000 7fcbeaffd640 1 --2- 172.21.15.2:0/2188354594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1a8d57b0 0x555b1a8e2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:18.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.640+0000 7fcbeaffd640 1 -- 172.21.15.2:0/2188354594 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b1a8e4ee0 msgr2=0x555b1a8f8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:18.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.640+0000 7fcbeaffd640 1 --2- 172.21.15.2:0/2188354594 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b1a8e4ee0 0x555b1a8f8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.640+0000 7fcbeaffd640 1 -- 172.21.15.2:0/2188354594 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b1a8dc2a0 con 0x555b1a8d60d0 2024-05-14T16:56:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.640+0000 7fcbeb7fe640 1 --2- 172.21.15.2:0/2188354594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1a8d57b0 0x555b1a8e2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:56:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.641+0000 7fcbeaffd640 1 --2- 172.21.15.2:0/2188354594 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b1a8d60d0 0x555b1a8f84d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcbe00a5ce0 tx=0x7fcbe0009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=779b9e813519d8de server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.641+0000 7fcbd17fa640 1 -- 172.21.15.2:0/2188354594 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe00cae60 con 0x555b1a8d60d0 2024-05-14T16:56:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.641+0000 7fcbd17fa640 1 -- 172.21.15.2:0/2188354594 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbe0091040 con 0x555b1a8d60d0 2024-05-14T16:56:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.641+0000 7fcbd17fa640 1 -- 172.21.15.2:0/2188354594 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe00b2a10 con 0x555b1a8d60d0 2024-05-14T16:56:18.439 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:56:16 smithi123 ceph-mon[35971]: pgmap v50: 97 pgs: 97 active+clean; 1.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 39 MiB/s wr, 1.38k op/s 2024-05-14T16:56:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.642+0000 7fcbf4bfd740 1 -- 172.21.15.2:0/2188354594 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b1a8d60d0 msgr2=0x555b1a8f84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:18.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.642+0000 7fcbf4bfd740 1 --2- 172.21.15.2:0/2188354594 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b1a8d60d0 0x555b1a8f84d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcbe00a5ce0 tx=0x7fcbe0009700 comp rx=0 tx=0).stop 2024-05-14T16:56:18.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.642+0000 7fcbf4bfd740 1 -- 172.21.15.2:0/2188354594 shutdown_connections 2024-05-14T16:56:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.642+0000 7fcbf4bfd740 1 --2- 172.21.15.2:0/2188354594 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b1a8e4ee0 0x555b1a8f8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.642+0000 7fcbf4bfd740 1 --2- 172.21.15.2:0/2188354594 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b1a8d60d0 0x555b1a8f84d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.642+0000 7fcbf4bfd740 1 --2- 172.21.15.2:0/2188354594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1a8d57b0 0x555b1a8e2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:18.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.642+0000 7fcbf4bfd740 1 -- 172.21.15.2:0/2188354594 >> 172.21.15.2:0/2188354594 conn(0x555b1a8d1e00 msgr2=0x555b1a8d9310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.642+0000 7fcbf4bfd740 1 -- 172.21.15.2:0/2188354594 shutdown_connections 2024-05-14T16:56:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.642+0000 7fcbf4bfd740 1 -- 172.21.15.2:0/2188354594 wait complete. 2024-05-14T16:56:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.642+0000 7fcbf4bfd740 1 -- start start 2024-05-14T16:56:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.643+0000 7fcbf4bfd740 1 Processor -- start 2024-05-14T16:56:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.643+0000 7fcbf4bfd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1a8d57b0 0x555b1a8ecfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.643+0000 7fcbf4bfd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b1a8d60d0 0x555b1a8e8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:18.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.643+0000 7fcbf4bfd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b1a8e4ee0 0x555b1a8e8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:18.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.643+0000 7fcbf4bfd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcbec1aab30 con 0x555b1a8e4ee0 2024-05-14T16:56:18.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.643+0000 7fcbf4bfd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcbec1abfa0 con 0x555b1a8d57b0 2024-05-14T16:56:18.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.643+0000 7fcbf4bfd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcbec1ab200 con 0x555b1a8d60d0 2024-05-14T16:56:18.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.643+0000 7fcbf4bfd740 10 client.0 _ll_register_callbacks cb 0x7fcbe4007940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:56:18.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.644+0000 7fcbeaffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b1a8d60d0 0x555b1a8e8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:18.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.644+0000 7fcbeaffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b1a8d60d0 0x555b1a8e8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:32914/0 (socket says 172.21.15.2:32914) 2024-05-14T16:56:18.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.644+0000 7fcbeaffd640 1 -- 172.21.15.2:0/2568254756 learned_addr learned my addr 172.21.15.2:0/2568254756 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:56:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.644+0000 7fcbebfff640 1 --2- 172.21.15.2:0/2568254756 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b1a8e4ee0 0x555b1a8e8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:18.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.644+0000 7fcbeb7fe640 1 --2- 172.21.15.2:0/2568254756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1a8d57b0 0x555b1a8ecfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.644+0000 7fcbeaffd640 1 -- 172.21.15.2:0/2568254756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1a8d57b0 msgr2=0x555b1a8ecfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.644+0000 7fcbeaffd640 1 --2- 172.21.15.2:0/2568254756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1a8d57b0 0x555b1a8ecfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.644+0000 7fcbeaffd640 1 -- 172.21.15.2:0/2568254756 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b1a8e4ee0 msgr2=0x555b1a8e8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:18.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.644+0000 7fcbeaffd640 1 --2- 172.21.15.2:0/2568254756 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b1a8e4ee0 0x555b1a8e8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:18.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.644+0000 7fcbeaffd640 1 -- 172.21.15.2:0/2568254756 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b1a8e8ea0 con 0x555b1a8d60d0 2024-05-14T16:56:18.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.645+0000 7fcbeaffd640 1 --2- 172.21.15.2:0/2568254756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b1a8d60d0 0x555b1a8e8320 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcbe00a9960 tx=0x7fcbe0010cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7d8b183077c22a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.645+0000 7fcbcaffd640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-14T16:56:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.645+0000 7fcbcaffd640 1 -- 172.21.15.2:0/2568254756 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe00c2020 con 0x555b1a8d60d0 2024-05-14T16:56:18.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.645+0000 7fcbcaffd640 1 -- 172.21.15.2:0/2568254756 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fcbe00e02a0 con 0x555b1a8d60d0 2024-05-14T16:56:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.645+0000 7fcbf4bfd740 1 -- 172.21.15.2:0/2568254756 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbe0091040 con 0x555b1a8d60d0 2024-05-14T16:56:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.645+0000 7fcbcaffd640 1 -- 172.21.15.2:0/2568254756 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcbe00cae60 con 0x555b1a8d60d0 2024-05-14T16:56:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.645+0000 7fcbf4bfd740 10 client.25216 fetch_fsmap learned FSMap version 30 2024-05-14T16:56:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.645+0000 7fcbf4bfd740 1 -- 172.21.15.2:0/2568254756 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555b1a95aea0 con 0x555b1a8d60d0 2024-05-14T16:56:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.646+0000 7fcbcaffd640 1 -- 172.21.15.2:0/2568254756 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbe0091040 con 0x555b1a8d60d0 2024-05-14T16:56:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.646+0000 7fcbcaffd640 1 -- 172.21.15.2:0/2568254756 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fcbe00a55d0 con 0x555b1a8d60d0 2024-05-14T16:56:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.646+0000 7fcbf4bfd740 10 client.25216 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T16:56:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.646+0000 7fcbf4bfd740 10 client.25216 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:56:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.646+0000 7fcbf4bfd740 20 client.25216 populate_metadata read hostname 'smithi002' 2024-05-14T16:56:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.646+0000 7fcbf4bfd740 1 --2- 172.21.15.2:0/2568254756 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x555b1a95b670 0x555b1a95da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:18.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.646+0000 7fcbf4bfd740 4 client.25216 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:56:18.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.646+0000 7fcbf4bfd740 1 -- 172.21.15.2:0/2568254756 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555b1a844360 con 0x555b1a95b670 2024-05-14T16:56:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.646+0000 7fcbeb7fe640 1 --2- 172.21.15.2:0/2568254756 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x555b1a95b670 0x555b1a95da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.646+0000 7fcbeb7fe640 1 --2- 172.21.15.2:0/2568254756 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x555b1a95b670 0x555b1a95da90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e93bbab57bd29a57 server_cookie=cc1985d58ee0520f in_seq=0 out_seq=0 2024-05-14T16:56:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.646+0000 7fcbcaffd640 10 client.25216 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:56:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.665+0000 7fcbcaffd640 1 -- 172.21.15.2:0/2568254756 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555b1a844360 con 0x555b1a95b670 2024-05-14T16:56:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.665+0000 7fcbcaffd640 10 client.25216 handle_command_reply: tid=0 2024-05-14T16:56:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.665+0000 7fcbf4bfd740 1 client.25216 shutdown 2024-05-14T16:56:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.665+0000 7fcbf4bfd740 1 -- 172.21.15.2:0/2568254756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b1a8d60d0 msgr2=0x555b1a8e8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.665+0000 7fcbf4bfd740 1 --2- 172.21.15.2:0/2568254756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b1a8d60d0 0x555b1a8e8320 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcbe00a9960 tx=0x7fcbe0010cc0 comp rx=0 tx=0).stop 2024-05-14T16:56:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.665+0000 7fcbf4bfd740 1 -- 172.21.15.2:0/2568254756 shutdown_connections 2024-05-14T16:56:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.665+0000 7fcbf4bfd740 1 --2- 172.21.15.2:0/2568254756 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x555b1a95b670 0x555b1a95da90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.665+0000 7fcbf4bfd740 1 --2- 172.21.15.2:0/2568254756 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555b1a8e4ee0 0x555b1a8e8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:19.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.665+0000 7fcbf4bfd740 1 --2- 172.21.15.2:0/2568254756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b1a8d60d0 0x555b1a8e8320 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.665+0000 7fcbcaffd640 0 client.25216 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:56:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.665+0000 7fcbf4bfd740 1 --2- 172.21.15.2:0/2568254756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b1a8d57b0 0x555b1a8ecfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:19.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.665+0000 7fcbf4bfd740 1 -- 172.21.15.2:0/2568254756 >> 172.21.15.2:0/2568254756 conn(0x555b1a8d1e00 msgr2=0x555b1a8d8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:19.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.666+0000 7fcbf4bfd740 1 -- 172.21.15.2:0/2568254756 shutdown_connections 2024-05-14T16:56:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.666+0000 7fcbf4bfd740 1 -- 172.21.15.2:0/2568254756 wait complete. 2024-05-14T16:56:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.666+0000 7fcbf4bfd740 20 client.25216 trim_cache size 0 max 16384 2024-05-14T16:56:19.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T16:56:19.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T16:56:19.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T16:56:19.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.666+0000 7fcbf2245640 1 -- 172.21.15.2:0/3503916839 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcbd40b2a90 msgr2=0x7fcbd40b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:19.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.666+0000 7fcbf2245640 1 --2- 172.21.15.2:0/3503916839 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcbd40b2a90 0x7fcbd40b4f50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcbe400a530 tx=0x7fcbe400d040 comp rx=0 tx=0).stop 2024-05-14T16:56:19.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.666+0000 7fcbf2245640 1 -- 172.21.15.2:0/3503916839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcbec1033f0 msgr2=0x7fcbec199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.666+0000 7fcbf2245640 1 --2- 172.21.15.2:0/3503916839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcbec1033f0 0x7fcbec199790 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcbe000e9f0 tx=0x7fcbe000eeb0 comp rx=0 tx=0).stop 2024-05-14T16:56:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.666+0000 7fcbf2245640 1 -- 172.21.15.2:0/3503916839 shutdown_connections 2024-05-14T16:56:19.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.666+0000 7fcbf2245640 1 --2- 172.21.15.2:0/3503916839 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcbd40b2a90 0x7fcbd40b4f50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:19.263 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.666+0000 7fcbf2245640 1 --2- 172.21.15.2:0/3503916839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbec108a40 0x7fcbec19dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:19.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.666+0000 7fcbf2245640 1 --2- 172.21.15.2:0/3503916839 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcbec1033f0 0x7fcbec199790 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.666+0000 7fcbf2245640 1 --2- 172.21.15.2:0/3503916839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcbec102a30 0x7fcbec199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:19.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.666+0000 7fcbf2245640 1 -- 172.21.15.2:0/3503916839 >> 172.21.15.2:0/3503916839 conn(0x7fcbec0fe500 msgr2=0x7fcbec0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.667+0000 7fcbf2245640 1 -- 172.21.15.2:0/3503916839 shutdown_connections 2024-05-14T16:56:19.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:16.667+0000 7fcbf2245640 1 -- 172.21.15.2:0/3503916839 wait complete. 2024-05-14T16:56:19.338 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:56:16 smithi002 ceph-mon[31538]: pgmap v50: 97 pgs: 97 active+clean; 1.0 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 39 MiB/s wr, 1.38k op/s 2024-05-14T16:56:19.344 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 60.0 s, 3726.6 tps, lat 26.822 ms stddev 27.422 2024-05-14T16:56:19.350 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:56:18 smithi002 ceph-mon[31538]: pgmap v51: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 44 MiB/s wr, 1.75k op/s 2024-05-14T16:56:19.356 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:56:18 smithi123 ceph-mon[35971]: pgmap v51: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 44 MiB/s wr, 1.75k op/s 2024-05-14T16:56:19.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:56:18 smithi151 ceph-mon[36066]: pgmap v51: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 44 MiB/s wr, 1.75k op/s 2024-05-14T16:56:20.402 INFO:tasks.quiescer.fs.[cephfs]:Total duration (53.4) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-14T16:56:20.493 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.7 2024-05-14T16:56:20.500 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:56:20.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:56:20 smithi151 ceph-mon[36066]: pgmap v52: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 12 MiB/s wr, 803 op/s 2024-05-14T16:56:20.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:56:20 smithi123 ceph-mon[35971]: pgmap v52: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 12 MiB/s wr, 803 op/s 2024-05-14T16:56:20.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:56:20 smithi002 ceph-mon[31538]: pgmap v52: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 12 MiB/s wr, 803 op/s 2024-05-14T16:56:20.910 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:56:21.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 65.0 s, 3673.4 tps, lat 27.224 ms stddev 28.467 2024-05-14T16:56:22.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:56:22 smithi151 ceph-mon[36066]: pgmap v53: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 9.0 MiB/s wr, 732 op/s 2024-05-14T16:56:22.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:56:22 smithi123 ceph-mon[35971]: pgmap v53: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 9.0 MiB/s wr, 732 op/s 2024-05-14T16:56:22.687 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:56:22 smithi002 ceph-mon[31538]: pgmap v53: 97 pgs: 97 active+clean; 1.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 9.0 MiB/s wr, 732 op/s 2024-05-14T16:56:23.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.020+0000 7f80c9489700 1 -- 172.21.15.2:0/3310079263 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f80c4102430 msgr2=0x7f80c41028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:23.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.020+0000 7f80c9489700 1 --2- 172.21.15.2:0/3310079263 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f80c4102430 0x7f80c41028b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f80b80098f0 tx=0x7f80b8009c00 comp rx=0 tx=0).stop 2024-05-14T16:56:23.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.020+0000 7f80c9489700 1 -- 172.21.15.2:0/3310079263 shutdown_connections 2024-05-14T16:56:23.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.020+0000 7f80c9489700 1 --2- 172.21.15.2:0/3310079263 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80c4102df0 0x7f80c410f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:23.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.020+0000 7f80c9489700 1 --2- 172.21.15.2:0/3310079263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80c4108510 0x7f80c41088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:23.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.020+0000 7f80c9489700 1 --2- 172.21.15.2:0/3310079263 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f80c4102430 0x7f80c41028b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:23.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.020+0000 7f80c9489700 1 -- 172.21.15.2:0/3310079263 >> 172.21.15.2:0/3310079263 conn(0x7f80c40793b0 msgr2=0x7f80c40797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:23.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.020+0000 7f80c9489700 1 -- 172.21.15.2:0/3310079263 shutdown_connections 2024-05-14T16:56:23.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.020+0000 7f80c9489700 1 -- 172.21.15.2:0/3310079263 wait complete. 2024-05-14T16:56:23.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.021+0000 7f80c9489700 1 Processor -- start 2024-05-14T16:56:23.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.021+0000 7f80c9489700 1 -- start start 2024-05-14T16:56:23.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.022+0000 7f80c9489700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80c4102430 0x7f80c4104070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:23.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.022+0000 7f80c9489700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f80c4102df0 0x7f80c41045b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:23.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.022+0000 7f80c9489700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80c4108510 0x7f80c4104af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:23.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.022+0000 7f80c9489700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f80c4112660 con 0x7f80c4108510 2024-05-14T16:56:23.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.022+0000 7f80c9489700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f80c41124e0 con 0x7f80c4102430 2024-05-14T16:56:23.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.022+0000 7f80c9489700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f80c41127e0 con 0x7f80c4102df0 2024-05-14T16:56:23.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.022+0000 7f80c37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80c4108510 0x7f80c4104af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:23.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.022+0000 7f80c37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80c4108510 0x7f80c4104af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32920/0 (socket says 172.21.15.2:32920) 2024-05-14T16:56:23.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.022+0000 7f80c37fe700 1 -- 172.21.15.2:0/3122784627 learned_addr learned my addr 172.21.15.2:0/3122784627 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:56:23.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.022+0000 7f80c27fc700 1 --2- 172.21.15.2:0/3122784627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f80c4102df0 0x7f80c41045b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:23.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.023+0000 7f80c2ffd700 1 --2- 172.21.15.2:0/3122784627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80c4102430 0x7f80c4104070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:23.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.023+0000 7f80c37fe700 1 -- 172.21.15.2:0/3122784627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f80c4102df0 msgr2=0x7f80c41045b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:23.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.023+0000 7f80c37fe700 1 --2- 172.21.15.2:0/3122784627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f80c4102df0 0x7f80c41045b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:23.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.023+0000 7f80c37fe700 1 -- 172.21.15.2:0/3122784627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80c4102430 msgr2=0x7f80c4104070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:23.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.023+0000 7f80c37fe700 1 --2- 172.21.15.2:0/3122784627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80c4102430 0x7f80c4104070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:23.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.023+0000 7f80c37fe700 1 -- 172.21.15.2:0/3122784627 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c41a81e0 con 0x7f80c4108510 2024-05-14T16:56:23.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.024+0000 7f80c37fe700 1 --2- 172.21.15.2:0/3122784627 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80c4108510 0x7f80c4104af0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f80bc00d980 tx=0x7f80bc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:23.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.024+0000 7f80abfff700 1 -- 172.21.15.2:0/3122784627 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80bc017070 con 0x7f80c4108510 2024-05-14T16:56:23.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.024+0000 7f80abfff700 1 -- 172.21.15.2:0/3122784627 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80bc005020 con 0x7f80c4108510 2024-05-14T16:56:23.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.025+0000 7f80abfff700 1 -- 172.21.15.2:0/3122784627 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80bc00fab0 con 0x7f80c4108510 2024-05-14T16:56:23.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.026+0000 7f80c9489700 1 -- 172.21.15.2:0/3122784627 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80c41a84d0 con 0x7f80c4108510 2024-05-14T16:56:23.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.026+0000 7f80c9489700 1 -- 172.21.15.2:0/3122784627 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80c41a8990 con 0x7f80c4108510 2024-05-14T16:56:23.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.028+0000 7f80c9489700 1 -- 172.21.15.2:0/3122784627 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80c4103b50 con 0x7f80c4108510 2024-05-14T16:56:23.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.030+0000 7f80abfff700 1 -- 172.21.15.2:0/3122784627 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f80bc00fc50 con 0x7f80c4108510 2024-05-14T16:56:23.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.031+0000 7f80abfff700 1 --2- 172.21.15.2:0/3122784627 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f80ac0b2a50 0x7f80ac0b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:23.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.031+0000 7f80abfff700 1 -- 172.21.15.2:0/3122784627 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f80bc0e1f20 con 0x7f80c4108510 2024-05-14T16:56:23.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.031+0000 7f80c2ffd700 1 --2- 172.21.15.2:0/3122784627 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f80ac0b2a50 0x7f80ac0b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:23.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.032+0000 7f80c2ffd700 1 --2- 172.21.15.2:0/3122784627 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f80ac0b2a50 0x7f80ac0b4f10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f80b4003e50 tx=0x7f80b40091b0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:23.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.038+0000 7f80abfff700 1 -- 172.21.15.2:0/3122784627 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f80bc0a99b0 con 0x7f80c4108510 2024-05-14T16:56:23.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.485+0000 7f80c9489700 1 -- 172.21.15.2:0/3122784627 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80c406a230 con 0x7f80c4108510 2024-05-14T16:56:23.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.486+0000 7f80abfff700 1 -- 172.21.15.2:0/3122784627 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11857 (secure 0 0 0) 0x7f80bc0a92b0 con 0x7f80c4108510 2024-05-14T16:56:23.647 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:56:23.656 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2275696923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2275696923},{"type":"v1","addr":"172.21.15.2:6835","nonce":2275696923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/716718761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":716718761},{"type":"v1","addr":"172.21.15.2:6839","nonce":716718761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/3644815179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3644815179},{"type":"v1","addr":"172.21.15.2:6841","nonce":3644815179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1880336731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1880336731},{"type":"v1","addr":"172.21.15.151:6833","nonce":1880336731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2415605548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2415605548},{"type":"v1","addr":"172.21.15.2:6837","nonce":2415605548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2635167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2635167},{"type":"v1","addr":"172.21.15.123:6833","nonce":2635167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:51:34.106919+0000","modified":"2024-05-14T16:56:06.904708+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24455,"mds_2":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24415":{"gid":24415,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/762044339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":762044339},{"type":"v1","addr":"172.21.15.151:6835","nonce":762044339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/132043151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":132043151},{"type":"v1","addr":"172.21.15.151:6837","nonce":132043151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/4107834375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":4107834375},{"type":"v1","addr":"172.21.15.151:6839","nonce":4107834375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.123:6835/1198670013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1198670013},{"type":"v1","addr":"172.21.15.123:6835","nonce":1198670013}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/333426645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":333426645},{"type":"v1","addr":"172.21.15.123:6837","nonce":333426645}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/580275877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":580275877},{"type":"v1","addr":"172.21.15.123:6839","nonce":580275877}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24455,24479,24431]},"id":1}]} 2024-05-14T16:56:23.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.490+0000 7f80c9489700 1 -- 172.21.15.2:0/3122784627 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f80ac0b2a50 msgr2=0x7f80ac0b4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:23.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.490+0000 7f80c9489700 1 --2- 172.21.15.2:0/3122784627 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f80ac0b2a50 0x7f80ac0b4f10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f80b4003e50 tx=0x7f80b40091b0 comp rx=0 tx=0).stop 2024-05-14T16:56:23.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.490+0000 7f80c9489700 1 -- 172.21.15.2:0/3122784627 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80c4108510 msgr2=0x7f80c4104af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:23.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.490+0000 7f80c9489700 1 --2- 172.21.15.2:0/3122784627 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80c4108510 0x7f80c4104af0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f80bc00d980 tx=0x7f80bc00dc90 comp rx=0 tx=0).stop 2024-05-14T16:56:23.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.490+0000 7f80c9489700 1 -- 172.21.15.2:0/3122784627 shutdown_connections 2024-05-14T16:56:23.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.490+0000 7f80c9489700 1 --2- 172.21.15.2:0/3122784627 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f80ac0b2a50 0x7f80ac0b4f10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:23.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.490+0000 7f80c9489700 1 --2- 172.21.15.2:0/3122784627 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80c4108510 0x7f80c4104af0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:23.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.490+0000 7f80c9489700 1 --2- 172.21.15.2:0/3122784627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80c4102430 0x7f80c4104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:23.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.490+0000 7f80c9489700 1 --2- 172.21.15.2:0/3122784627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f80c4102df0 0x7f80c41045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:23.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.490+0000 7f80c9489700 1 -- 172.21.15.2:0/3122784627 >> 172.21.15.2:0/3122784627 conn(0x7f80c40793b0 msgr2=0x7f80c406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:23.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.491+0000 7f80c9489700 1 -- 172.21.15.2:0/3122784627 shutdown_connections 2024-05-14T16:56:23.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:23.491+0000 7f80c9489700 1 -- 172.21.15.2:0/3122784627 wait complete. 2024-05-14T16:56:23.830 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 30 2024-05-14T16:56:24.178 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 146.7 --await-for 45.0' 2024-05-14T16:56:24.189 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 146.7 --await-for 45.0 2024-05-14T16:56:24.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:56:24 smithi002 ceph-mon[31538]: pgmap v54: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 13 MiB/s wr, 1.09k op/s 2024-05-14T16:56:24.514 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:56:24 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/3122784627' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:56:24.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:56:24 smithi151 ceph-mon[36066]: pgmap v54: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 13 MiB/s wr, 1.09k op/s 2024-05-14T16:56:24.611 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:56:24 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/3122784627' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:56:25.668 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:56:24 smithi123 ceph-mon[35971]: pgmap v54: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 13 MiB/s wr, 1.09k op/s 2024-05-14T16:56:25.702 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:56:24 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/3122784627' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:56:25.711 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:56:26.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:56:26 smithi002 ceph-mon[31538]: pgmap v55: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.3 MiB/s wr, 757 op/s 2024-05-14T16:56:26.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:56:26 smithi151 ceph-mon[36066]: pgmap v55: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.3 MiB/s wr, 757 op/s 2024-05-14T16:56:26.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:56:26 smithi123 ceph-mon[35971]: pgmap v55: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.3 MiB/s wr, 757 op/s 2024-05-14T16:56:26.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.725+0000 7fca7198d700 1 -- 172.21.15.2:0/3780801649 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca6c108900 msgr2=0x7fca6c110480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:26.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.725+0000 7fca7198d700 1 --2- 172.21.15.2:0/3780801649 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca6c108900 0x7fca6c110480 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fca6400b0d0 tx=0x7fca6400b3e0 comp rx=0 tx=0).stop 2024-05-14T16:56:26.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.726+0000 7fca7198d700 1 -- 172.21.15.2:0/3780801649 shutdown_connections 2024-05-14T16:56:26.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.726+0000 7fca7198d700 1 --2- 172.21.15.2:0/3780801649 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca6c108900 0x7fca6c110480 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:26.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.726+0000 7fca7198d700 1 --2- 172.21.15.2:0/3780801649 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca6c06b480 0x7fca6c06b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:26.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.726+0000 7fca7198d700 1 --2- 172.21.15.2:0/3780801649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca6c06be30 0x7fca6c1081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:26.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.726+0000 7fca7198d700 1 -- 172.21.15.2:0/3780801649 >> 172.21.15.2:0/3780801649 conn(0x7fca6c0793e0 msgr2=0x7fca6c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:26.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.726+0000 7fca7198d700 1 -- 172.21.15.2:0/3780801649 shutdown_connections 2024-05-14T16:56:26.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.726+0000 7fca7198d700 1 -- 172.21.15.2:0/3780801649 wait complete. 2024-05-14T16:56:26.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.727+0000 7fca7198d700 1 Processor -- start 2024-05-14T16:56:26.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.727+0000 7fca7198d700 1 -- start start 2024-05-14T16:56:26.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.728+0000 7fca7198d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca6c06b480 0x7fca6c19f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:26.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.728+0000 7fca7198d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca6c06be30 0x7fca6c19fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:26.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.728+0000 7fca7198d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca6c108900 0x7fca6c1998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:26.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.728+0000 7fca7198d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c113750 con 0x7fca6c06b480 2024-05-14T16:56:26.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.728+0000 7fca7198d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c1135d0 con 0x7fca6c108900 2024-05-14T16:56:26.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.728+0000 7fca7198d700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c1138d0 con 0x7fca6c06be30 2024-05-14T16:56:26.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.728+0000 7fca6affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca6c06b480 0x7fca6c19f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:26.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.728+0000 7fca6b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca6c108900 0x7fca6c1998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:27.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.728+0000 7fca6affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca6c06b480 0x7fca6c19f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39668/0 (socket says 172.21.15.2:39668) 2024-05-14T16:56:27.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.728+0000 7fca6affd700 1 -- 172.21.15.2:0/1748219405 learned_addr learned my addr 172.21.15.2:0/1748219405 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:56:27.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.728+0000 7fca6a7fc700 1 --2- 172.21.15.2:0/1748219405 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca6c06be30 0x7fca6c19fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:27.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.728+0000 7fca6b7fe700 1 -- 172.21.15.2:0/1748219405 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca6c06be30 msgr2=0x7fca6c19fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:27.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.728+0000 7fca6b7fe700 1 --2- 172.21.15.2:0/1748219405 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca6c06be30 0x7fca6c19fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:27.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.728+0000 7fca6b7fe700 1 -- 172.21.15.2:0/1748219405 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca6c06b480 msgr2=0x7fca6c19f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:27.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.728+0000 7fca6b7fe700 1 --2- 172.21.15.2:0/1748219405 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca6c06b480 0x7fca6c19f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:27.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.728+0000 7fca6b7fe700 1 -- 172.21.15.2:0/1748219405 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca6c19a050 con 0x7fca6c108900 2024-05-14T16:56:27.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.728+0000 7fca6b7fe700 1 --2- 172.21.15.2:0/1748219405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca6c108900 0x7fca6c1998f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fca6400b770 tx=0x7fca64007980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:27.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.729+0000 7fca7098b700 1 -- 172.21.15.2:0/1748219405 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6401f070 con 0x7fca6c108900 2024-05-14T16:56:27.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.729+0000 7fca7098b700 1 -- 172.21.15.2:0/1748219405 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca64007dc0 con 0x7fca6c108900 2024-05-14T16:56:27.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.729+0000 7fca7098b700 1 -- 172.21.15.2:0/1748219405 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6401b430 con 0x7fca6c108900 2024-05-14T16:56:27.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.729+0000 7fca7198d700 1 -- 172.21.15.2:0/1748219405 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca6c19a2e0 con 0x7fca6c108900 2024-05-14T16:56:27.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.729+0000 7fca7198d700 1 -- 172.21.15.2:0/1748219405 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca6c19a6b0 con 0x7fca6c108900 2024-05-14T16:56:27.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.734+0000 7fca7098b700 1 -- 172.21.15.2:0/1748219405 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fca64022070 con 0x7fca6c108900 2024-05-14T16:56:27.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.736+0000 7fca7098b700 1 --2- 172.21.15.2:0/1748219405 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fca580b2c60 0x7fca580b5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:27.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.736+0000 7fca6affd700 1 --2- 172.21.15.2:0/1748219405 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fca580b2c60 0x7fca580b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:27.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.736+0000 7fca7098b700 1 -- 172.21.15.2:0/1748219405 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fca640e3040 con 0x7fca6c108900 2024-05-14T16:56:27.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.737+0000 7fca6affd700 1 --2- 172.21.15.2:0/1748219405 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fca580b2c60 0x7fca580b5120 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fca6c19aa30 tx=0x7fca5c009260 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:27.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.737+0000 7fca7c223b80 1 Processor -- start 2024-05-14T16:56:27.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.737+0000 7fca7c223b80 1 -- start start 2024-05-14T16:56:27.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.738+0000 7fca7c223b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b4f35b320 0x560b4f35b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:27.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.738+0000 7fca7c223b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560b4f35bc40 0x560b4f35f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:27.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.738+0000 7fca7c223b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b4f35fae0 0x560b4f35ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:27.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.738+0000 7fca7c223b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x560b4f2b4f00 con 0x560b4f35fae0 2024-05-14T16:56:27.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.738+0000 7fca7c223b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560b4f2a4aa0 con 0x560b4f35bc40 2024-05-14T16:56:27.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.738+0000 7fca7c223b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560b4f2a8ae0 con 0x560b4f35b320 2024-05-14T16:56:27.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.738+0000 7fca6b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b4f35fae0 0x560b4f35ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:27.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.738+0000 7fca6affd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b4f35b320 0x560b4f35b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:27.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.738+0000 7fca6a7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560b4f35bc40 0x560b4f35f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:27.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.738+0000 7fca6b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b4f35fae0 0x560b4f35ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39672/0 (socket says 172.21.15.2:39672) 2024-05-14T16:56:27.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.738+0000 7fca6b7fe700 1 -- 172.21.15.2:0/1417671915 learned_addr learned my addr 172.21.15.2:0/1417671915 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:56:27.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.738+0000 7fca6affd700 1 -- 172.21.15.2:0/1417671915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560b4f35bc40 msgr2=0x560b4f35f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:27.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.738+0000 7fca6affd700 1 --2- 172.21.15.2:0/1417671915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560b4f35bc40 0x560b4f35f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:27.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.738+0000 7fca6affd700 1 -- 172.21.15.2:0/1417671915 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b4f35fae0 msgr2=0x560b4f35ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:27.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.738+0000 7fca6affd700 1 --2- 172.21.15.2:0/1417671915 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b4f35fae0 0x560b4f35ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:27.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.738+0000 7fca6affd700 1 -- 172.21.15.2:0/1417671915 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b4f24e140 con 0x560b4f35b320 2024-05-14T16:56:27.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.738+0000 7fca6affd700 1 --2- 172.21.15.2:0/1417671915 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b4f35b320 0x560b4f35b700 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fca5c009c20 tx=0x7fca5c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2866104ae7d7c36b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:27.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.739+0000 7fca61ffb700 1 -- 172.21.15.2:0/1417671915 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c01ce10 con 0x560b4f35b320 2024-05-14T16:56:27.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.739+0000 7fca61ffb700 1 -- 172.21.15.2:0/1417671915 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca5c028930 con 0x560b4f35b320 2024-05-14T16:56:27.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.739+0000 7fca61ffb700 1 -- 172.21.15.2:0/1417671915 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c01dae0 con 0x560b4f35b320 2024-05-14T16:56:27.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.740+0000 7fca7c223b80 1 -- 172.21.15.2:0/1417671915 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b4f35b320 msgr2=0x560b4f35b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:27.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.740+0000 7fca7c223b80 1 --2- 172.21.15.2:0/1417671915 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b4f35b320 0x560b4f35b700 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fca5c009c20 tx=0x7fca5c014000 comp rx=0 tx=0).stop 2024-05-14T16:56:27.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.740+0000 7fca7c223b80 1 -- 172.21.15.2:0/1417671915 shutdown_connections 2024-05-14T16:56:27.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.740+0000 7fca7c223b80 1 --2- 172.21.15.2:0/1417671915 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b4f35fae0 0x560b4f35ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:27.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.740+0000 7fca7c223b80 1 --2- 172.21.15.2:0/1417671915 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b4f35b320 0x560b4f35b700 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:27.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.740+0000 7fca7c223b80 1 --2- 172.21.15.2:0/1417671915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560b4f35bc40 0x560b4f35f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:27.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.740+0000 7fca7c223b80 1 -- 172.21.15.2:0/1417671915 >> 172.21.15.2:0/1417671915 conn(0x560b4f283f40 msgr2=0x560b4f351530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:27.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.740+0000 7fca7c223b80 1 -- 172.21.15.2:0/1417671915 shutdown_connections 2024-05-14T16:56:27.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.740+0000 7fca7c223b80 1 -- 172.21.15.2:0/1417671915 wait complete. 2024-05-14T16:56:27.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.741+0000 7fca7c223b80 1 -- start start 2024-05-14T16:56:27.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.741+0000 7fca7c223b80 1 Processor -- start 2024-05-14T16:56:27.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.741+0000 7fca7c223b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560b4f35b320 0x560b4f367000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:27.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.741+0000 7fca7c223b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b4f35bc40 0x560b4f3643a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:27.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.741+0000 7fca7c223b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b4f35fae0 0x560b4f3648e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:27.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.741+0000 7fca7c223b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x560b4f1ab130 con 0x560b4f35fae0 2024-05-14T16:56:27.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.741+0000 7fca7c223b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560b4f20cf30 con 0x560b4f35b320 2024-05-14T16:56:27.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.741+0000 7fca7c223b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560b4f367d10 con 0x560b4f35bc40 2024-05-14T16:56:27.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.741+0000 7fca7c223b80 10 client.0 _ll_register_callbacks cb 0x560b4f23fae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:56:27.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.741+0000 7fca6b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b4f35fae0 0x560b4f3648e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:27.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.742+0000 7fca6a7fc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b4f35bc40 0x560b4f3643a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:27.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.742+0000 7fca6affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560b4f35b320 0x560b4f367000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:27.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.742+0000 7fca6b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b4f35fae0 0x560b4f3648e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39684/0 (socket says 172.21.15.2:39684) 2024-05-14T16:56:27.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.742+0000 7fca6b7fe700 1 -- 172.21.15.2:0/542515065 learned_addr learned my addr 172.21.15.2:0/542515065 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:56:27.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.742+0000 7fca6affd700 1 -- 172.21.15.2:0/542515065 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b4f35bc40 msgr2=0x560b4f3643a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:27.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.742+0000 7fca6affd700 1 --2- 172.21.15.2:0/542515065 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b4f35bc40 0x560b4f3643a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:27.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.742+0000 7fca6affd700 1 -- 172.21.15.2:0/542515065 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b4f35fae0 msgr2=0x560b4f3648e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:27.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.742+0000 7fca6affd700 1 --2- 172.21.15.2:0/542515065 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b4f35fae0 0x560b4f3648e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:27.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.742+0000 7fca6affd700 1 -- 172.21.15.2:0/542515065 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b4f1f4a60 con 0x560b4f35b320 2024-05-14T16:56:27.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.742+0000 7fca6b7fe700 1 --2- 172.21.15.2:0/542515065 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b4f35fae0 0x560b4f3648e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:56:27.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.742+0000 7fca6affd700 1 --2- 172.21.15.2:0/542515065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560b4f35b320 0x560b4f367000 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fca5c011ab0 tx=0x7fca5c0286b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b43df0c0cc1d20dd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:27.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.742+0000 7fca4affd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T16:56:27.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.742+0000 7fca4affd700 1 -- 172.21.15.2:0/542515065 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c005040 con 0x560b4f35b320 2024-05-14T16:56:27.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.743+0000 7fca4affd700 1 -- 172.21.15.2:0/542515065 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fca5c0051e0 con 0x560b4f35b320 2024-05-14T16:56:27.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.743+0000 7fca7c223b80 1 -- 172.21.15.2:0/542515065 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca5c028930 con 0x560b4f35b320 2024-05-14T16:56:27.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.743+0000 7fca4affd700 1 -- 172.21.15.2:0/542515065 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fca5c0053e0 con 0x560b4f35b320 2024-05-14T16:56:27.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.745+0000 7fca7c223b80 10 client.25238 fetch_fsmap learned FSMap version 30 2024-05-14T16:56:27.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.745+0000 7fca7c223b80 1 -- 172.21.15.2:0/542515065 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560b4f25e8f0 con 0x560b4f35b320 2024-05-14T16:56:27.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.746+0000 7fca4affd700 1 -- 172.21.15.2:0/542515065 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca5c005580 con 0x560b4f35b320 2024-05-14T16:56:27.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.746+0000 7fca4affd700 1 -- 172.21.15.2:0/542515065 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fca5c0049b0 con 0x560b4f35b320 2024-05-14T16:56:27.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.746+0000 7fca7c223b80 10 client.25238 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T16:56:28.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.746+0000 7fca7c223b80 10 client.25238 resolve_mds: validated gid 24479 aka daemon mds.b 2024-05-14T16:56:28.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.746+0000 7fca7c223b80 20 client.25238 populate_metadata read hostname 'smithi002' 2024-05-14T16:56:28.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.746+0000 7fca7c223b80 1 --2- 172.21.15.2:0/542515065 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x560b4f374770 0x560b4f376b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:28.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.746+0000 7fca7c223b80 4 client.25238 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:56:28.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.746+0000 7fca7c223b80 1 -- 172.21.15.2:0/542515065 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fca5c028930 con 0x560b4f374770 2024-05-14T16:56:28.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.746+0000 7fca6a7fc700 1 --2- 172.21.15.2:0/542515065 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x560b4f374770 0x560b4f376b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:28.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.747+0000 7fca4affd700 10 client.25238 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:56:28.087 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 70.0 s, 3300.2 tps, lat 30.314 ms stddev 29.975 2024-05-14T16:56:28.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.747+0000 7fca6a7fc700 1 --2- 172.21.15.2:0/542515065 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x560b4f374770 0x560b4f376b60 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa412d92112b9e08 server_cookie=60fac8b25d32c52c in_seq=0 out_seq=0 2024-05-14T16:56:28.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.748+0000 7fca4affd700 1 -- 172.21.15.2:0/542515065 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fca5c028930 con 0x560b4f374770 2024-05-14T16:56:28.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.748+0000 7fca4affd700 10 client.25238 handle_command_reply: tid=0 2024-05-14T16:56:28.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.748+0000 7fca7c223b80 1 client.25238 shutdown 2024-05-14T16:56:28.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.748+0000 7fca7c223b80 1 -- 172.21.15.2:0/542515065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560b4f35b320 msgr2=0x560b4f367000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:28.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.748+0000 7fca7c223b80 1 --2- 172.21.15.2:0/542515065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560b4f35b320 0x560b4f367000 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fca5c011ab0 tx=0x7fca5c0286b0 comp rx=0 tx=0).stop 2024-05-14T16:56:28.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.749+0000 7fca7c223b80 1 -- 172.21.15.2:0/542515065 shutdown_connections 2024-05-14T16:56:28.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.749+0000 7fca7c223b80 1 --2- 172.21.15.2:0/542515065 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x560b4f374770 0x560b4f376b60 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:28.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.749+0000 7fca7c223b80 1 --2- 172.21.15.2:0/542515065 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b4f35fae0 0x560b4f3648e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:28.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.749+0000 7fca7c223b80 1 --2- 172.21.15.2:0/542515065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560b4f35b320 0x560b4f367000 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:28.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.749+0000 7fca7c223b80 1 --2- 172.21.15.2:0/542515065 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b4f35bc40 0x560b4f3643a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:28.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.749+0000 7fca7c223b80 1 -- 172.21.15.2:0/542515065 >> 172.21.15.2:0/542515065 conn(0x560b4f283f40 msgr2=0x560b4f360f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:28.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.749+0000 7fca4affd700 0 client.25238 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:56:28.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.749+0000 7fca7c223b80 1 -- 172.21.15.2:0/542515065 shutdown_connections 2024-05-14T16:56:28.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.749+0000 7fca7c223b80 1 -- 172.21.15.2:0/542515065 wait complete. 2024-05-14T16:56:28.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.749+0000 7fca7c223b80 20 client.25238 trim_cache size 0 max 16384 2024-05-14T16:56:28.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.780+0000 7fca7c223b80 1 Processor -- start 2024-05-14T16:56:28.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.781+0000 7fca7c223b80 1 -- start start 2024-05-14T16:56:28.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.781+0000 7fca7c223b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b4f35b320 0x560b4f370c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:28.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.781+0000 7fca7c223b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b4f35bc40 0x560b4f360ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:28.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.781+0000 7fca7c223b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560b4f35fae0 0x560b4f35f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:28.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.782+0000 7fca7c223b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c1ae360 con 0x560b4f35b320 2024-05-14T16:56:28.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.782+0000 7fca7c223b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c1abb60 con 0x560b4f35fae0 2024-05-14T16:56:28.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.782+0000 7fca7c223b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c1a99d0 con 0x560b4f35bc40 2024-05-14T16:56:28.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.782+0000 7fca6affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b4f35b320 0x560b4f370c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:28.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.782+0000 7fca6affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b4f35b320 0x560b4f370c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39688/0 (socket says 172.21.15.2:39688) 2024-05-14T16:56:28.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.782+0000 7fca6affd700 1 -- 172.21.15.2:0/3617514581 learned_addr learned my addr 172.21.15.2:0/3617514581 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:56:28.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.783+0000 7fca6b7fe700 1 --2- 172.21.15.2:0/3617514581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560b4f35fae0 0x560b4f35f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:28.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.783+0000 7fca6a7fc700 1 --2- 172.21.15.2:0/3617514581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b4f35bc40 0x560b4f360ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:28.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.784+0000 7fca6affd700 1 -- 172.21.15.2:0/3617514581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b4f35bc40 msgr2=0x560b4f360ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:28.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.784+0000 7fca6affd700 1 --2- 172.21.15.2:0/3617514581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b4f35bc40 0x560b4f360ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:28.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.784+0000 7fca6affd700 1 -- 172.21.15.2:0/3617514581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560b4f35fae0 msgr2=0x560b4f35f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:28.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.784+0000 7fca6affd700 1 --2- 172.21.15.2:0/3617514581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560b4f35fae0 0x560b4f35f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:28.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.784+0000 7fca6affd700 1 -- 172.21.15.2:0/3617514581 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b4f367bf0 con 0x560b4f35b320 2024-05-14T16:56:28.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.786+0000 7fca6affd700 1 --2- 172.21.15.2:0/3617514581 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b4f35b320 0x560b4f370c20 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fca5c01c470 tx=0x7fca5c005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67810f8a76440ff3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:28.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.787+0000 7fca617fa700 1 -- 172.21.15.2:0/3617514581 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c047070 con 0x560b4f35b320 2024-05-14T16:56:28.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.787+0000 7fca617fa700 1 -- 172.21.15.2:0/3617514581 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca5c028930 con 0x560b4f35b320 2024-05-14T16:56:28.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.788+0000 7fca7c223b80 1 -- 172.21.15.2:0/3617514581 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b4f35b320 msgr2=0x560b4f370c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:28.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.788+0000 7fca7c223b80 1 --2- 172.21.15.2:0/3617514581 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b4f35b320 0x560b4f370c20 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fca5c01c470 tx=0x7fca5c005ce0 comp rx=0 tx=0).stop 2024-05-14T16:56:28.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.788+0000 7fca7c223b80 1 -- 172.21.15.2:0/3617514581 shutdown_connections 2024-05-14T16:56:28.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.788+0000 7fca7c223b80 1 --2- 172.21.15.2:0/3617514581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560b4f35fae0 0x560b4f35f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:28.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.788+0000 7fca7c223b80 1 --2- 172.21.15.2:0/3617514581 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b4f35b320 0x560b4f370c20 secure :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fca5c01c470 tx=0x7fca5c005ce0 comp rx=0 tx=0).stop 2024-05-14T16:56:28.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.788+0000 7fca7c223b80 1 --2- 172.21.15.2:0/3617514581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b4f35bc40 0x560b4f360ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:28.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.788+0000 7fca7c223b80 1 -- 172.21.15.2:0/3617514581 >> 172.21.15.2:0/3617514581 conn(0x560b4f283f40 msgr2=0x560b4f35e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:28.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.793+0000 7fca7c223b80 1 -- 172.21.15.2:0/3617514581 shutdown_connections 2024-05-14T16:56:28.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.794+0000 7fca7c223b80 1 -- 172.21.15.2:0/3617514581 wait complete. 2024-05-14T16:56:28.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.794+0000 7fca7c223b80 1 -- start start 2024-05-14T16:56:28.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.794+0000 7fca7c223b80 1 Processor -- start 2024-05-14T16:56:28.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.795+0000 7fca7c223b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b4f35bc40 0x560b4f35cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:28.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.795+0000 7fca7c223b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b4f35fae0 0x560b4f35d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:28.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.795+0000 7fca7c223b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560b4f35d5d0 0x560b4f36cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:28.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.795+0000 7fca7c223b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c1aa020 con 0x560b4f35fae0 2024-05-14T16:56:28.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.795+0000 7fca7c223b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c1ab490 con 0x560b4f35d5d0 2024-05-14T16:56:28.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.795+0000 7fca7c223b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c1aa6f0 con 0x560b4f35bc40 2024-05-14T16:56:28.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.795+0000 7fca7c223b80 10 client.0 _ll_register_callbacks cb 0x560b4f23fae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:56:28.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.796+0000 7fca6affd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b4f35bc40 0x560b4f35cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:28.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.796+0000 7fca6a7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b4f35fae0 0x560b4f35d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:28.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.796+0000 7fca6a7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b4f35fae0 0x560b4f35d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39690/0 (socket says 172.21.15.2:39690) 2024-05-14T16:56:28.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.796+0000 7fca6a7fc700 1 -- 172.21.15.2:0/2646770097 learned_addr learned my addr 172.21.15.2:0/2646770097 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:56:28.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.796+0000 7fca6a7fc700 1 -- 172.21.15.2:0/2646770097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b4f35bc40 msgr2=0x560b4f35cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:28.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.796+0000 7fca6a7fc700 1 --2- 172.21.15.2:0/2646770097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b4f35bc40 0x560b4f35cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:28.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.796+0000 7fca6a7fc700 1 -- 172.21.15.2:0/2646770097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560b4f35d5d0 msgr2=0x560b4f36cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:28.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.796+0000 7fca6a7fc700 1 --2- 172.21.15.2:0/2646770097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560b4f35d5d0 0x560b4f36cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:28.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.796+0000 7fca6a7fc700 1 -- 172.21.15.2:0/2646770097 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b4f36d6b0 con 0x560b4f35fae0 2024-05-14T16:56:28.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.797+0000 7fca6a7fc700 1 --2- 172.21.15.2:0/2646770097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b4f35fae0 0x560b4f35d090 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x560b4f365970 tx=0x7fca540107c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=878bcfb739b7067a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:28.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.798+0000 7fca4affd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:56:28.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.798+0000 7fca4affd700 1 -- 172.21.15.2:0/2646770097 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca54004460 con 0x560b4f35fae0 2024-05-14T16:56:28.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.798+0000 7fca7c223b80 1 -- 172.21.15.2:0/2646770097 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca5c028930 con 0x560b4f35fae0 2024-05-14T16:56:28.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.798+0000 7fca4affd700 1 -- 172.21.15.2:0/2646770097 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fca5401f400 con 0x560b4f35fae0 2024-05-14T16:56:28.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.798+0000 7fca4affd700 1 -- 172.21.15.2:0/2646770097 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fca5402e050 con 0x560b4f35fae0 2024-05-14T16:56:28.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.798+0000 7fca7c223b80 10 client.15510 fetch_fsmap learned FSMap version 30 2024-05-14T16:56:28.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.798+0000 7fca7c223b80 1 -- 172.21.15.2:0/2646770097 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560b4f3e4270 con 0x560b4f35fae0 2024-05-14T16:56:28.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.799+0000 7fca4affd700 1 -- 172.21.15.2:0/2646770097 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca5c028930 con 0x560b4f35fae0 2024-05-14T16:56:28.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.799+0000 7fca4affd700 1 -- 172.21.15.2:0/2646770097 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fca540273e0 con 0x560b4f35fae0 2024-05-14T16:56:28.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.799+0000 7fca7c223b80 10 client.15510 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T16:56:28.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.799+0000 7fca7c223b80 10 client.15510 resolve_mds: validated gid 24479 aka daemon mds.b 2024-05-14T16:56:28.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.799+0000 7fca7c223b80 20 client.15510 populate_metadata read hostname 'smithi002' 2024-05-14T16:56:28.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.799+0000 7fca7c223b80 1 --2- 172.21.15.2:0/2646770097 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x560b4f3e4ab0 0x560b4f3e6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:28.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.799+0000 7fca7c223b80 4 client.15510 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.7, "await_for": 45.0}] 2024-05-14T16:56:28.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.799+0000 7fca7c223b80 1 -- 172.21.15.2:0/2646770097 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.7, "await_for": 45.0}) -- 0x560b4f2b6e50 con 0x560b4f3e4ab0 2024-05-14T16:56:28.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.801+0000 7fca6affd700 1 --2- 172.21.15.2:0/2646770097 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x560b4f3e4ab0 0x560b4f3e6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:28.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.804+0000 7fca6affd700 1 --2- 172.21.15.2:0/2646770097 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x560b4f3e4ab0 0x560b4f3e6ed0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ed93c76d4bd232a server_cookie=950a58ce6071bb9d in_seq=0 out_seq=0 2024-05-14T16:56:28.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:26.804+0000 7fca4affd700 10 client.15510 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:56:29.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:27.041+0000 7fca4affd700 1 -- 172.21.15.2:0/2646770097 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x560b4f2b6e50 con 0x560b4f3e4ab0 2024-05-14T16:56:29.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:27.041+0000 7fca4affd700 10 client.15510 handle_command_reply: tid=0 2024-05-14T16:56:29.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:27.041+0000 7fca7c223b80 1 client.15510 shutdown 2024-05-14T16:56:29.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:27.042+0000 7fca7c223b80 1 -- 172.21.15.2:0/2646770097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b4f35fae0 msgr2=0x560b4f35d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:29.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:27.042+0000 7fca7c223b80 1 --2- 172.21.15.2:0/2646770097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b4f35fae0 0x560b4f35d090 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x560b4f365970 tx=0x7fca540107c0 comp rx=0 tx=0).stop 2024-05-14T16:56:29.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:27.042+0000 7fca7c223b80 1 -- 172.21.15.2:0/2646770097 shutdown_connections 2024-05-14T16:56:29.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:27.042+0000 7fca7c223b80 1 --2- 172.21.15.2:0/2646770097 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x560b4f3e4ab0 0x560b4f3e6ed0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:29.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:27.042+0000 7fca7c223b80 1 --2- 172.21.15.2:0/2646770097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560b4f35d5d0 0x560b4f36cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:29.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:27.042+0000 7fca7c223b80 1 --2- 172.21.15.2:0/2646770097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b4f35bc40 0x560b4f35cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:29.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:27.042+0000 7fca7c223b80 1 --2- 172.21.15.2:0/2646770097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b4f35fae0 0x560b4f35d090 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:29.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:27.042+0000 7fca7c223b80 1 -- 172.21.15.2:0/2646770097 >> 172.21.15.2:0/2646770097 conn(0x560b4f283f40 msgr2=0x560b4f366230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:29.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:27.042+0000 7fca4affd700 0 client.15510 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:56:29.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:27.042+0000 7fca7c223b80 1 -- 172.21.15.2:0/2646770097 shutdown_connections 2024-05-14T16:56:29.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:27.042+0000 7fca7c223b80 1 -- 172.21.15.2:0/2646770097 wait complete. 2024-05-14T16:56:29.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:27.042+0000 7fca7c223b80 20 client.15510 trim_cache size 0 max 16384 2024-05-14T16:56:29.182 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T16:56:29.190 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 30, 2024-05-14T16:56:29.199 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24479, 2024-05-14T16:56:29.207 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 6, 2024-05-14T16:56:29.215 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-05-14T16:56:29.224 INFO:teuthology.orchestra.run.smithi002.stdout: "a7bd3c7e": { 2024-05-14T16:56:29.231 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 6, 2024-05-14T16:56:29.239 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-05-14T16:56:29.246 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-05-14T16:56:29.254 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-05-14T16:56:29.287 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-05-14T16:56:29.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T16:56:29.304 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-05-14T16:56:29.354 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 146.7, 2024-05-14T16:56:29.398 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-05-14T16:56:29.405 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-05-14T16:56:29.414 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-05-14T16:56:29.420 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-05-14T16:56:29.427 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-05-14T16:56:29.435 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-05-14T16:56:29.443 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:56:29.451 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:56:29.459 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:56:29.467 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:56:29.475 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:56:29.484 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T16:56:29.495 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:56:28 smithi123 ceph-mon[35971]: pgmap v56: 97 pgs: 97 active+clean; 1.1 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 1.03k op/s 2024-05-14T16:56:29.518 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:56:28 smithi151 ceph-mon[36066]: pgmap v56: 97 pgs: 97 active+clean; 1.1 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 1.03k op/s 2024-05-14T16:56:29.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:27.043+0000 7fca7198d700 1 -- 172.21.15.2:0/1748219405 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fca580b2c60 msgr2=0x7fca580b5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:29.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:27.043+0000 7fca7198d700 1 --2- 172.21.15.2:0/1748219405 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fca580b2c60 0x7fca580b5120 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fca6c19aa30 tx=0x7fca5c009260 comp rx=0 tx=0).stop 2024-05-14T16:56:29.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:27.043+0000 7fca7198d700 1 -- 172.21.15.2:0/1748219405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca6c108900 msgr2=0x7fca6c1998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:29.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:27.043+0000 7fca7198d700 1 --2- 172.21.15.2:0/1748219405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca6c108900 0x7fca6c1998f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fca6400b770 tx=0x7fca64007980 comp rx=0 tx=0).stop 2024-05-14T16:56:29.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:27.043+0000 7fca7198d700 1 -- 172.21.15.2:0/1748219405 shutdown_connections 2024-05-14T16:56:29.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:27.043+0000 7fca7198d700 1 --2- 172.21.15.2:0/1748219405 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fca580b2c60 0x7fca580b5120 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:29.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:27.043+0000 7fca7198d700 1 --2- 172.21.15.2:0/1748219405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca6c108900 0x7fca6c1998f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:29.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:27.043+0000 7fca7198d700 1 --2- 172.21.15.2:0/1748219405 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca6c06b480 0x7fca6c19f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:29.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:27.043+0000 7fca7198d700 1 --2- 172.21.15.2:0/1748219405 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca6c06be30 0x7fca6c19fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:29.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:27.043+0000 7fca7198d700 1 -- 172.21.15.2:0/1748219405 >> 172.21.15.2:0/1748219405 conn(0x7fca6c0793e0 msgr2=0x7fca6c1004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:29.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:27.043+0000 7fca7198d700 1 -- 172.21.15.2:0/1748219405 shutdown_connections 2024-05-14T16:56:29.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:27.043+0000 7fca7198d700 1 -- 172.21.15.2:0/1748219405 wait complete. 2024-05-14T16:56:29.636 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:56:28 smithi002 ceph-mon[31538]: pgmap v56: 97 pgs: 97 active+clean; 1.1 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 1.03k op/s 2024-05-14T16:56:29.644 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a7bd3c7e', quiesce took 9.2 seconds. Will release after: 17.5 2024-05-14T16:56:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.774+0000 7fe148220640 1 -- 172.21.15.2:0/3469369025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe140102660 msgr2=0x7fe14010e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.774+0000 7fe148220640 1 --2- 172.21.15.2:0/3469369025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe140102660 0x7fe14010e770 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe138009ca0 tx=0x7fe13802a8c0 comp rx=0 tx=0).stop 2024-05-14T16:56:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.774+0000 7fe148220640 1 -- 172.21.15.2:0/3469369025 shutdown_connections 2024-05-14T16:56:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.774+0000 7fe148220640 1 --2- 172.21.15.2:0/3469369025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe140102660 0x7fe14010e770 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.774+0000 7fe148220640 1 --2- 172.21.15.2:0/3469369025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe140101ca0 0x7fe140102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.774+0000 7fe148220640 1 --2- 172.21.15.2:0/3469369025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe1401068f0 0x7fe140106cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.774+0000 7fe148220640 1 -- 172.21.15.2:0/3469369025 >> 172.21.15.2:0/3469369025 conn(0x7fe140077750 msgr2=0x7fe140077b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.774+0000 7fe148220640 1 -- 172.21.15.2:0/3469369025 shutdown_connections 2024-05-14T16:56:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.774+0000 7fe148220640 1 -- 172.21.15.2:0/3469369025 wait complete. 2024-05-14T16:56:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.774+0000 7fe148220640 1 Processor -- start 2024-05-14T16:56:29.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.775+0000 7fe148220640 1 -- start start 2024-05-14T16:56:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.775+0000 7fe148220640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe140101ca0 0x7fe140199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.775+0000 7fe148220640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe140102660 0x7fe140199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.775+0000 7fe148220640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1401068f0 0x7fe14019d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:29.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.775+0000 7fe148220640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe140111bd0 con 0x7fe140102660 2024-05-14T16:56:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.775+0000 7fe148220640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe140111a50 con 0x7fe1401068f0 2024-05-14T16:56:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.775+0000 7fe148220640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe140111d50 con 0x7fe140101ca0 2024-05-14T16:56:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.775+0000 7fe1457bb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe140102660 0x7fe140199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.775+0000 7fe1457bb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe140102660 0x7fe140199540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39710/0 (socket says 172.21.15.2:39710) 2024-05-14T16:56:30.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.775+0000 7fe1457bb640 1 -- 172.21.15.2:0/3515290603 learned_addr learned my addr 172.21.15.2:0/3515290603 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:56:30.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.775+0000 7fe145fbc640 1 --2- 172.21.15.2:0/3515290603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe140101ca0 0x7fe140199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.775+0000 7fe1467bd640 1 --2- 172.21.15.2:0/3515290603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1401068f0 0x7fe14019d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.775+0000 7fe1457bb640 1 -- 172.21.15.2:0/3515290603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe140101ca0 msgr2=0x7fe140199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.775+0000 7fe1457bb640 1 --2- 172.21.15.2:0/3515290603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe140101ca0 0x7fe140199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.775+0000 7fe1457bb640 1 -- 172.21.15.2:0/3515290603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1401068f0 msgr2=0x7fe14019d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:30.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.775+0000 7fe1457bb640 1 --2- 172.21.15.2:0/3515290603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1401068f0 0x7fe14019d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:30.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.775+0000 7fe1457bb640 1 -- 172.21.15.2:0/3515290603 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe14019dff0 con 0x7fe140102660 2024-05-14T16:56:30.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.775+0000 7fe1467bd640 1 --2- 172.21.15.2:0/3515290603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1401068f0 0x7fe14019d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:56:30.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.775+0000 7fe1457bb640 1 --2- 172.21.15.2:0/3515290603 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe140102660 0x7fe140199540 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe13000c920 tx=0x7fe13000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:30.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.776+0000 7fe136ffd640 1 -- 172.21.15.2:0/3515290603 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe130007c70 con 0x7fe140102660 2024-05-14T16:56:30.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.776+0000 7fe136ffd640 1 -- 172.21.15.2:0/3515290603 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe130007e10 con 0x7fe140102660 2024-05-14T16:56:30.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.776+0000 7fe136ffd640 1 -- 172.21.15.2:0/3515290603 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe130018820 con 0x7fe140102660 2024-05-14T16:56:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.776+0000 7fe148220640 1 -- 172.21.15.2:0/3515290603 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe14019e2e0 con 0x7fe140102660 2024-05-14T16:56:30.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.776+0000 7fe148220640 1 -- 172.21.15.2:0/3515290603 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1401a5b70 con 0x7fe140102660 2024-05-14T16:56:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.778+0000 7fe14b013740 1 Processor -- start 2024-05-14T16:56:30.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.779+0000 7fe14b013740 1 -- start start 2024-05-14T16:56:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.779+0000 7fe136ffd640 1 -- 172.21.15.2:0/3515290603 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fe13000f450 con 0x7fe140102660 2024-05-14T16:56:30.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.779+0000 7fe14b013740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a347ef5830 0x55a347ef5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:30.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.779+0000 7fe14b013740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a347ef6150 0x55a347f049a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:30.219 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.779+0000 7fe14b013740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a347f04ee0 0x55a347f072d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.779+0000 7fe14b013740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55a347e68c30 con 0x55a347ef6150 2024-05-14T16:56:30.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.779+0000 7fe14b013740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a347e587d0 con 0x55a347f04ee0 2024-05-14T16:56:30.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.779+0000 7fe14b013740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a347e5c810 con 0x55a347ef5830 2024-05-14T16:56:30.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.779+0000 7fe145fbc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a347ef6150 0x55a347f049a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:30.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.779+0000 7fe145fbc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a347ef6150 0x55a347f049a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39724/0 (socket says 172.21.15.2:39724) 2024-05-14T16:56:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.779+0000 7fe145fbc640 1 -- 172.21.15.2:0/3039421202 learned_addr learned my addr 172.21.15.2:0/3039421202 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:56:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.779+0000 7fe1467bd640 1 --2- 172.21.15.2:0/3039421202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a347ef5830 0x55a347ef5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.779+0000 7fe1457bb640 1 --2- 172.21.15.2:0/3039421202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a347f04ee0 0x55a347f072d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:30.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.779+0000 7fe145fbc640 1 -- 172.21.15.2:0/3039421202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a347ef5830 msgr2=0x55a347ef5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:30.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.779+0000 7fe145fbc640 1 --2- 172.21.15.2:0/3039421202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a347ef5830 0x55a347ef5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.779+0000 7fe145fbc640 1 -- 172.21.15.2:0/3039421202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a347f04ee0 msgr2=0x55a347f072d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.779+0000 7fe145fbc640 1 --2- 172.21.15.2:0/3039421202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a347f04ee0 0x55a347f072d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:30.334 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.779+0000 7fe145fbc640 1 -- 172.21.15.2:0/3039421202 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a347b6d210 con 0x55a347ef6150 2024-05-14T16:56:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.779+0000 7fe1457bb640 1 --2- 172.21.15.2:0/3039421202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a347f04ee0 0x55a347f072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:56:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.780+0000 7fe145fbc640 1 --2- 172.21.15.2:0/3039421202 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a347ef6150 0x55a347f049a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe13c00b760 tx=0x7fe13c00bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa9cd8feb4773f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:30.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.780+0000 7fe12bfff640 1 -- 172.21.15.2:0/3039421202 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c003dd0 con 0x55a347ef6150 2024-05-14T16:56:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.780+0000 7fe12bfff640 1 -- 172.21.15.2:0/3039421202 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe13c015070 con 0x55a347ef6150 2024-05-14T16:56:30.402 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.780+0000 7fe12bfff640 1 -- 172.21.15.2:0/3039421202 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c004d90 con 0x55a347ef6150 2024-05-14T16:56:30.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.780+0000 7fe136ffd640 1 --2- 172.21.15.2:0/3515290603 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fe12c0b2a90 0x7fe12c0b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:30.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.780+0000 7fe1467bd640 1 --2- 172.21.15.2:0/3515290603 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fe12c0b2a90 0x7fe12c0b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.780+0000 7fe136ffd640 1 -- 172.21.15.2:0/3515290603 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fe1300e26d0 con 0x7fe140102660 2024-05-14T16:56:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.781+0000 7fe14b013740 1 -- 172.21.15.2:0/3039421202 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a347ef6150 msgr2=0x55a347f049a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.781+0000 7fe1467bd640 1 --2- 172.21.15.2:0/3515290603 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fe12c0b2a90 0x7fe12c0b4f50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe138004af0 tx=0x7fe138005d80 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.781+0000 7fe14b013740 1 --2- 172.21.15.2:0/3039421202 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a347ef6150 0x55a347f049a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe13c00b760 tx=0x7fe13c00bc20 comp rx=0 tx=0).stop 2024-05-14T16:56:30.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.781+0000 7fe14b013740 1 -- 172.21.15.2:0/3039421202 shutdown_connections 2024-05-14T16:56:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.781+0000 7fe14b013740 1 --2- 172.21.15.2:0/3039421202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a347f04ee0 0x55a347f072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.781+0000 7fe14b013740 1 --2- 172.21.15.2:0/3039421202 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a347ef6150 0x55a347f049a0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.781+0000 7fe14b013740 1 --2- 172.21.15.2:0/3039421202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a347ef5830 0x55a347ef5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.781+0000 7fe14b013740 1 -- 172.21.15.2:0/3039421202 >> 172.21.15.2:0/3039421202 conn(0x55a347ef1e00 msgr2=0x55a347ef21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.781+0000 7fe14b013740 1 -- 172.21.15.2:0/3039421202 shutdown_connections 2024-05-14T16:56:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.781+0000 7fe14b013740 1 -- 172.21.15.2:0/3039421202 wait complete. 2024-05-14T16:56:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.781+0000 7fe14b013740 1 -- start start 2024-05-14T16:56:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.781+0000 7fe14b013740 1 Processor -- start 2024-05-14T16:56:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.782+0000 7fe14b013740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a347ef5830 0x55a347f04850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.782+0000 7fe14b013740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a347ef6150 0x55a347f01b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.782+0000 7fe14b013740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a347f04ee0 0x55a347f020a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.782+0000 7fe14b013740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55a347b3e790 con 0x55a347ef5830 2024-05-14T16:56:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.782+0000 7fe14b013740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a347e14890 con 0x55a347f04ee0 2024-05-14T16:56:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.782+0000 7fe14b013740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a347f07b50 con 0x55a347ef6150 2024-05-14T16:56:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.782+0000 7fe14b013740 10 client.0 _ll_register_callbacks cb 0x55a347e17d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:56:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.782+0000 7fe145fbc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a347ef5830 0x55a347f04850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.782+0000 7fe145fbc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a347ef5830 0x55a347f04850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39736/0 (socket says 172.21.15.2:39736) 2024-05-14T16:56:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.782+0000 7fe145fbc640 1 -- 172.21.15.2:0/560654582 learned_addr learned my addr 172.21.15.2:0/560654582 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:56:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.782+0000 7fe1467bd640 1 --2- 172.21.15.2:0/560654582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a347f04ee0 0x55a347f020a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.782+0000 7fe1457bb640 1 --2- 172.21.15.2:0/560654582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a347ef6150 0x55a347f01b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.782+0000 7fe145fbc640 1 -- 172.21.15.2:0/560654582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a347ef6150 msgr2=0x55a347f01b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.782+0000 7fe145fbc640 1 --2- 172.21.15.2:0/560654582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a347ef6150 0x55a347f01b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.782+0000 7fe145fbc640 1 -- 172.21.15.2:0/560654582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a347f04ee0 msgr2=0x55a347f020a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:30.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.782+0000 7fe145fbc640 1 --2- 172.21.15.2:0/560654582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a347f04ee0 0x55a347f020a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:30.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.782+0000 7fe145fbc640 1 -- 172.21.15.2:0/560654582 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a347c39620 con 0x55a347ef5830 2024-05-14T16:56:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.782+0000 7fe1467bd640 1 --2- 172.21.15.2:0/560654582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a347f04ee0 0x55a347f020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:56:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.782+0000 7fe145fbc640 1 --2- 172.21.15.2:0/560654582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a347ef5830 0x55a347f04850 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe13c006010 tx=0x7fe13c004800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a862baf732392a4f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:30.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.783+0000 7fe1297fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:56:30.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.783+0000 7fe1297fa640 1 -- 172.21.15.2:0/560654582 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c028060 con 0x55a347ef5830 2024-05-14T16:56:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.783+0000 7fe1297fa640 1 -- 172.21.15.2:0/560654582 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fe13c011640 con 0x55a347ef5830 2024-05-14T16:56:30.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.783+0000 7fe14b013740 1 -- 172.21.15.2:0/560654582 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe13c015070 con 0x55a347ef5830 2024-05-14T16:56:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.783+0000 7fe1297fa640 1 -- 172.21.15.2:0/560654582 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe13c011840 con 0x55a347ef5830 2024-05-14T16:56:30.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.783+0000 7fe14b013740 10 client.15522 fetch_fsmap learned FSMap version 30 2024-05-14T16:56:30.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.783+0000 7fe14b013740 1 -- 172.21.15.2:0/560654582 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a347f135f0 con 0x55a347ef5830 2024-05-14T16:56:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.783+0000 7fe1297fa640 1 -- 172.21.15.2:0/560654582 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe13c015070 con 0x55a347ef5830 2024-05-14T16:56:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.783+0000 7fe1297fa640 1 -- 172.21.15.2:0/560654582 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fe13c022070 con 0x55a347ef5830 2024-05-14T16:56:30.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.783+0000 7fe14b013740 10 client.15522 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T16:56:30.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.783+0000 7fe14b013740 10 client.15522 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:56:30.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.783+0000 7fe14b013740 20 client.15522 populate_metadata read hostname 'smithi002' 2024-05-14T16:56:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.783+0000 7fe14b013740 1 --2- 172.21.15.2:0/560654582 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55a347f14a50 0x55a347f16e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.783+0000 7fe14b013740 4 client.15522 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:56:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.783+0000 7fe14b013740 1 -- 172.21.15.2:0/560654582 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a347e6ab80 con 0x55a347f14a50 2024-05-14T16:56:30.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.784+0000 7fe1457bb640 1 --2- 172.21.15.2:0/560654582 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55a347f14a50 0x55a347f16e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.784+0000 7fe1457bb640 1 --2- 172.21.15.2:0/560654582 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55a347f14a50 0x55a347f16e40 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eee425094cea586d server_cookie=b82e02a848abc852 in_seq=0 out_seq=0 2024-05-14T16:56:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.784+0000 7fe1297fa640 10 client.15522 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:56:30.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.785+0000 7fe1297fa640 1 -- 172.21.15.2:0/560654582 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a347e6ab80 con 0x55a347f14a50 2024-05-14T16:56:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.785+0000 7fe1297fa640 10 client.15522 handle_command_reply: tid=0 2024-05-14T16:56:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.785+0000 7fe14b013740 1 client.15522 shutdown 2024-05-14T16:56:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.785+0000 7fe14b013740 1 -- 172.21.15.2:0/560654582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a347ef5830 msgr2=0x55a347f04850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.785+0000 7fe14b013740 1 --2- 172.21.15.2:0/560654582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a347ef5830 0x55a347f04850 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe13c006010 tx=0x7fe13c004800 comp rx=0 tx=0).stop 2024-05-14T16:56:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.785+0000 7fe14b013740 1 -- 172.21.15.2:0/560654582 shutdown_connections 2024-05-14T16:56:30.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.785+0000 7fe14b013740 1 --2- 172.21.15.2:0/560654582 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55a347f14a50 0x55a347f16e40 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.785+0000 7fe14b013740 1 --2- 172.21.15.2:0/560654582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a347f04ee0 0x55a347f020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:31.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.785+0000 7fe14b013740 1 --2- 172.21.15.2:0/560654582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a347ef6150 0x55a347f01b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.785+0000 7fe14b013740 1 --2- 172.21.15.2:0/560654582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a347ef5830 0x55a347f04850 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:31.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.785+0000 7fe14b013740 1 -- 172.21.15.2:0/560654582 >> 172.21.15.2:0/560654582 conn(0x55a347ef1e00 msgr2=0x55a347ef1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.785+0000 7fe1297fa640 0 client.15522 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:56:31.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.785+0000 7fe14b013740 1 -- 172.21.15.2:0/560654582 shutdown_connections 2024-05-14T16:56:31.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.785+0000 7fe14b013740 1 -- 172.21.15.2:0/560654582 wait complete. 2024-05-14T16:56:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.785+0000 7fe14b013740 20 client.15522 trim_cache size 0 max 16384 2024-05-14T16:56:31.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.800+0000 7fe14b013740 1 Processor -- start 2024-05-14T16:56:31.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.801+0000 7fe14b013740 1 -- start start 2024-05-14T16:56:31.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.801+0000 7fe14b013740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a347ef5830 0x55a347f04380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.801+0000 7fe14b013740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a347ef6150 0x55a347f1c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.801+0000 7fe14b013740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a347f04ee0 0x55a347f1c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.801+0000 7fe14b013740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe1401aeaf0 con 0x55a347ef6150 2024-05-14T16:56:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.801+0000 7fe14b013740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe1401ac330 con 0x55a347f04ee0 2024-05-14T16:56:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.801+0000 7fe14b013740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe1401aa180 con 0x55a347ef5830 2024-05-14T16:56:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.801+0000 7fe1457bb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a347ef6150 0x55a347f1c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:31.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.801+0000 7fe1457bb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a347ef6150 0x55a347f1c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39742/0 (socket says 172.21.15.2:39742) 2024-05-14T16:56:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.801+0000 7fe1457bb640 1 -- 172.21.15.2:0/2731134869 learned_addr learned my addr 172.21.15.2:0/2731134869 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:56:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.801+0000 7fe145fbc640 1 --2- 172.21.15.2:0/2731134869 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a347ef5830 0x55a347f04380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:31.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.801+0000 7fe1457bb640 1 -- 172.21.15.2:0/2731134869 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a347ef5830 msgr2=0x55a347f04380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.801+0000 7fe1457bb640 1 --2- 172.21.15.2:0/2731134869 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a347ef5830 0x55a347f04380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:31.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.801+0000 7fe1457bb640 1 -- 172.21.15.2:0/2731134869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a347f04ee0 msgr2=0x55a347f1c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:31.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.801+0000 7fe1467bd640 1 --2- 172.21.15.2:0/2731134869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a347f04ee0 0x55a347f1c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:31.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.801+0000 7fe1457bb640 1 --2- 172.21.15.2:0/2731134869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a347f04ee0 0x55a347f1c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:31.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.801+0000 7fe1457bb640 1 -- 172.21.15.2:0/2731134869 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a347ef81e0 con 0x55a347ef6150 2024-05-14T16:56:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.801+0000 7fe1457bb640 1 --2- 172.21.15.2:0/2731134869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a347ef6150 0x55a347f1c490 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe1300e0450 tx=0x7fe1300e4c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=119984eea5674f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:31.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.801+0000 7fe12b7fe640 1 -- 172.21.15.2:0/2731134869 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1300aa0e0 con 0x55a347ef6150 2024-05-14T16:56:31.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.802+0000 7fe12b7fe640 1 -- 172.21.15.2:0/2731134869 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1300a99e0 con 0x55a347ef6150 2024-05-14T16:56:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.802+0000 7fe12b7fe640 1 -- 172.21.15.2:0/2731134869 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1300a5c70 con 0x55a347ef6150 2024-05-14T16:56:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.802+0000 7fe14b013740 1 -- 172.21.15.2:0/2731134869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a347ef6150 msgr2=0x55a347f1c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.802+0000 7fe14b013740 1 --2- 172.21.15.2:0/2731134869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a347ef6150 0x55a347f1c490 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe1300e0450 tx=0x7fe1300e4c50 comp rx=0 tx=0).stop 2024-05-14T16:56:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.802+0000 7fe14b013740 1 -- 172.21.15.2:0/2731134869 shutdown_connections 2024-05-14T16:56:31.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.802+0000 7fe14b013740 1 --2- 172.21.15.2:0/2731134869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a347f04ee0 0x55a347f1c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:31.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.802+0000 7fe14b013740 1 --2- 172.21.15.2:0/2731134869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a347ef6150 0x55a347f1c490 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:31.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.802+0000 7fe14b013740 1 --2- 172.21.15.2:0/2731134869 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a347ef5830 0x55a347f04380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:31.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.802+0000 7fe14b013740 1 -- 172.21.15.2:0/2731134869 >> 172.21.15.2:0/2731134869 conn(0x55a347ef1e00 msgr2=0x55a347ef7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.802+0000 7fe14b013740 1 -- 172.21.15.2:0/2731134869 shutdown_connections 2024-05-14T16:56:31.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.802+0000 7fe14b013740 1 -- 172.21.15.2:0/2731134869 wait complete. 2024-05-14T16:56:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.803+0000 7fe14b013740 1 -- start start 2024-05-14T16:56:31.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.803+0000 7fe14b013740 1 Processor -- start 2024-05-14T16:56:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.803+0000 7fe14b013740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a347ef5830 0x55a347f0cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.803+0000 7fe14b013740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a347ef6150 0x55a347f0d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.803+0000 7fe14b013740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a347f04ee0 0x55a347f094c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.803+0000 7fe14b013740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe1401aa7f0 con 0x55a347f04ee0 2024-05-14T16:56:31.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.803+0000 7fe14b013740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe1401abc60 con 0x55a347ef6150 2024-05-14T16:56:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.803+0000 7fe14b013740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe1401aaec0 con 0x55a347ef5830 2024-05-14T16:56:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.803+0000 7fe14b013740 10 client.0 _ll_register_callbacks cb 0x7fe13c0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:56:31.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.803+0000 7fe1467bd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a347f04ee0 0x55a347f094c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:31.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.803+0000 7fe1467bd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a347f04ee0 0x55a347f094c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39748/0 (socket says 172.21.15.2:39748) 2024-05-14T16:56:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.803+0000 7fe1467bd640 1 -- 172.21.15.2:0/247396422 learned_addr learned my addr 172.21.15.2:0/247396422 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:56:31.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.803+0000 7fe145fbc640 1 --2- 172.21.15.2:0/247396422 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a347ef5830 0x55a347f0cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.803+0000 7fe1457bb640 1 --2- 172.21.15.2:0/247396422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a347ef6150 0x55a347f0d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.804+0000 7fe1467bd640 1 -- 172.21.15.2:0/247396422 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a347ef5830 msgr2=0x55a347f0cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.804+0000 7fe1467bd640 1 --2- 172.21.15.2:0/247396422 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a347ef5830 0x55a347f0cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.804+0000 7fe1467bd640 1 -- 172.21.15.2:0/247396422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a347ef6150 msgr2=0x55a347f0d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.804+0000 7fe1467bd640 1 --2- 172.21.15.2:0/247396422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a347ef6150 0x55a347f0d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:31.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.804+0000 7fe1467bd640 1 -- 172.21.15.2:0/247396422 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a347f09cc0 con 0x55a347f04ee0 2024-05-14T16:56:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.804+0000 7fe1457bb640 1 --2- 172.21.15.2:0/247396422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a347ef6150 0x55a347f0d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:56:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.804+0000 7fe145fbc640 1 --2- 172.21.15.2:0/247396422 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a347ef5830 0x55a347f0cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:56:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.804+0000 7fe1467bd640 1 --2- 172.21.15.2:0/247396422 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a347f04ee0 0x55a347f094c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe138031640 tx=0x7fe13802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b2d7a12209d1e61 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.804+0000 7fe1297fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:56:31.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.804+0000 7fe1297fa640 1 -- 172.21.15.2:0/247396422 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe138056060 con 0x55a347f04ee0 2024-05-14T16:56:31.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.804+0000 7fe1297fa640 1 -- 172.21.15.2:0/247396422 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fe138041030 con 0x55a347f04ee0 2024-05-14T16:56:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.804+0000 7fe14b013740 1 -- 172.21.15.2:0/247396422 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1300a99e0 con 0x55a347f04ee0 2024-05-14T16:56:31.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.804+0000 7fe1297fa640 1 -- 172.21.15.2:0/247396422 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe13802feb0 con 0x55a347f04ee0 2024-05-14T16:56:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.804+0000 7fe14b013740 10 client.15528 fetch_fsmap learned FSMap version 30 2024-05-14T16:56:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.804+0000 7fe14b013740 1 -- 172.21.15.2:0/247396422 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a347f7fc70 con 0x55a347f04ee0 2024-05-14T16:56:31.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.804+0000 7fe1297fa640 1 -- 172.21.15.2:0/247396422 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1300a99e0 con 0x55a347f04ee0 2024-05-14T16:56:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.804+0000 7fe1297fa640 1 -- 172.21.15.2:0/247396422 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fe138048650 con 0x55a347f04ee0 2024-05-14T16:56:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.804+0000 7fe14b013740 10 client.15528 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T16:56:31.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.804+0000 7fe14b013740 10 client.15528 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:56:31.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.804+0000 7fe14b013740 20 client.15528 populate_metadata read hostname 'smithi002' 2024-05-14T16:56:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.805+0000 7fe14b013740 1 --2- 172.21.15.2:0/247396422 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55a347f80480 0x55a347f828a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.805+0000 7fe14b013740 4 client.15528 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:56:31.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.805+0000 7fe14b013740 1 -- 172.21.15.2:0/247396422 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a347e64360 con 0x55a347f80480 2024-05-14T16:56:31.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.805+0000 7fe145fbc640 1 --2- 172.21.15.2:0/247396422 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55a347f80480 0x55a347f828a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:31.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.805+0000 7fe1297fa640 10 client.15528 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:56:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.805+0000 7fe145fbc640 1 --2- 172.21.15.2:0/247396422 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55a347f80480 0x55a347f828a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a58d3afaa003f2ea server_cookie=ed4676b1a51ee954 in_seq=0 out_seq=0 2024-05-14T16:56:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.825+0000 7fe1297fa640 1 -- 172.21.15.2:0/247396422 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a347e64360 con 0x55a347f80480 2024-05-14T16:56:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.825+0000 7fe1297fa640 10 client.15528 handle_command_reply: tid=0 2024-05-14T16:56:31.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.825+0000 7fe14b013740 1 client.15528 shutdown 2024-05-14T16:56:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.825+0000 7fe14b013740 1 -- 172.21.15.2:0/247396422 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a347f04ee0 msgr2=0x55a347f094c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.825+0000 7fe14b013740 1 --2- 172.21.15.2:0/247396422 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a347f04ee0 0x55a347f094c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe138031640 tx=0x7fe13802a8c0 comp rx=0 tx=0).stop 2024-05-14T16:56:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.825+0000 7fe14b013740 1 -- 172.21.15.2:0/247396422 shutdown_connections 2024-05-14T16:56:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.825+0000 7fe14b013740 1 --2- 172.21.15.2:0/247396422 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55a347f80480 0x55a347f828a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.825+0000 7fe14b013740 1 --2- 172.21.15.2:0/247396422 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a347f04ee0 0x55a347f094c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.825+0000 7fe14b013740 1 --2- 172.21.15.2:0/247396422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a347ef6150 0x55a347f0d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.825+0000 7fe1297fa640 0 client.15528 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:56:32.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.825+0000 7fe14b013740 1 --2- 172.21.15.2:0/247396422 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a347ef5830 0x55a347f0cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:32.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.825+0000 7fe14b013740 1 -- 172.21.15.2:0/247396422 >> 172.21.15.2:0/247396422 conn(0x55a347ef1e00 msgr2=0x55a347ef0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:32.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.825+0000 7fe14b013740 1 -- 172.21.15.2:0/247396422 shutdown_connections 2024-05-14T16:56:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.825+0000 7fe14b013740 1 -- 172.21.15.2:0/247396422 wait complete. 2024-05-14T16:56:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.825+0000 7fe14b013740 20 client.15528 trim_cache size 0 max 16384 2024-05-14T16:56:32.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T16:56:32.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T16:56:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T16:56:32.173 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:56:30 smithi123 ceph-mon[35971]: pgmap v57: 97 pgs: 97 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.3 MiB/s wr, 684 op/s 2024-05-14T16:56:32.181 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:56:30 smithi151 ceph-mon[36066]: pgmap v57: 97 pgs: 97 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.3 MiB/s wr, 684 op/s 2024-05-14T16:56:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.826+0000 7fe148220640 1 -- 172.21.15.2:0/3515290603 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fe12c0b2a90 msgr2=0x7fe12c0b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:32.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.826+0000 7fe148220640 1 --2- 172.21.15.2:0/3515290603 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fe12c0b2a90 0x7fe12c0b4f50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe138004af0 tx=0x7fe138005d80 comp rx=0 tx=0).stop 2024-05-14T16:56:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.826+0000 7fe148220640 1 -- 172.21.15.2:0/3515290603 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe140102660 msgr2=0x7fe140199540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.826+0000 7fe148220640 1 --2- 172.21.15.2:0/3515290603 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe140102660 0x7fe140199540 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe13000c920 tx=0x7fe13000cde0 comp rx=0 tx=0).stop 2024-05-14T16:56:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.826+0000 7fe148220640 1 -- 172.21.15.2:0/3515290603 shutdown_connections 2024-05-14T16:56:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.826+0000 7fe148220640 1 --2- 172.21.15.2:0/3515290603 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fe12c0b2a90 0x7fe12c0b4f50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.826+0000 7fe148220640 1 --2- 172.21.15.2:0/3515290603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1401068f0 0x7fe14019d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:32.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.826+0000 7fe148220640 1 --2- 172.21.15.2:0/3515290603 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe140102660 0x7fe140199540 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.826+0000 7fe148220640 1 --2- 172.21.15.2:0/3515290603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe140101ca0 0x7fe140199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.826+0000 7fe148220640 1 -- 172.21.15.2:0/3515290603 >> 172.21.15.2:0/3515290603 conn(0x7fe140077750 msgr2=0x7fe14006b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.826+0000 7fe148220640 1 -- 172.21.15.2:0/3515290603 shutdown_connections 2024-05-14T16:56:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:29.826+0000 7fe148220640 1 -- 172.21.15.2:0/3515290603 wait complete. 2024-05-14T16:56:32.350 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:56:30 smithi002 ceph-mon[31538]: pgmap v57: 97 pgs: 97 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.3 MiB/s wr, 684 op/s 2024-05-14T16:56:32.357 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 75.0 s, 100.9 tps, lat 47.124 ms stddev 53.505 2024-05-14T16:56:32.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:56:32 smithi151 ceph-mon[36066]: pgmap v58: 97 pgs: 97 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.7 MiB/s wr, 653 op/s 2024-05-14T16:56:32.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:56:32 smithi123 ceph-mon[35971]: pgmap v58: 97 pgs: 97 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.7 MiB/s wr, 653 op/s 2024-05-14T16:56:32.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:56:32 smithi002 ceph-mon[31538]: pgmap v58: 97 pgs: 97 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.7 MiB/s wr, 653 op/s 2024-05-14T16:56:34.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:56:34 smithi151 ceph-mon[36066]: pgmap v59: 97 pgs: 97 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.7 MiB/s wr, 662 op/s 2024-05-14T16:56:34.662 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:56:34 smithi123 ceph-mon[35971]: pgmap v59: 97 pgs: 97 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.7 MiB/s wr, 662 op/s 2024-05-14T16:56:34.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:56:34 smithi002 ceph-mon[31538]: pgmap v59: 97 pgs: 97 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.7 MiB/s wr, 662 op/s 2024-05-14T16:56:36.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:56:36 smithi123 ceph-mon[35971]: pgmap v60: 97 pgs: 97 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.9 MiB/s wr, 304 op/s 2024-05-14T16:56:36.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:56:36 smithi002 ceph-mon[31538]: pgmap v60: 97 pgs: 97 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.9 MiB/s wr, 304 op/s 2024-05-14T16:56:36.786 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 80.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:56:36.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:56:36 smithi151 ceph-mon[36066]: pgmap v60: 97 pgs: 97 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.9 MiB/s wr, 304 op/s 2024-05-14T16:56:38.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:56:38 smithi123 ceph-mon[35971]: pgmap v61: 97 pgs: 97 active+clean; 1.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 776 op/s 2024-05-14T16:56:38.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:56:38 smithi002 ceph-mon[31538]: pgmap v61: 97 pgs: 97 active+clean; 1.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 776 op/s 2024-05-14T16:56:38.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:56:38 smithi151 ceph-mon[36066]: pgmap v61: 97 pgs: 97 active+clean; 1.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 776 op/s 2024-05-14T16:56:40.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:56:40 smithi123 ceph-mon[35971]: pgmap v62: 97 pgs: 97 active+clean; 1.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 499 op/s 2024-05-14T16:56:40.845 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:56:40 smithi002 ceph-mon[31538]: pgmap v62: 97 pgs: 97 active+clean; 1.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 499 op/s 2024-05-14T16:56:40.868 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:56:40 smithi151 ceph-mon[36066]: pgmap v62: 97 pgs: 97 active+clean; 1.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 499 op/s 2024-05-14T16:56:41.750 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 85.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:56:42.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:56:42 smithi123 ceph-mon[35971]: pgmap v63: 97 pgs: 97 active+clean; 1.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 480 op/s 2024-05-14T16:56:42.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:56:42 smithi002 ceph-mon[31538]: pgmap v63: 97 pgs: 97 active+clean; 1.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 480 op/s 2024-05-14T16:56:42.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:56:42 smithi151 ceph-mon[36066]: pgmap v63: 97 pgs: 97 active+clean; 1.2 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 480 op/s 2024-05-14T16:56:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.927+0000 7fde7b359640 1 -- 172.21.15.2:0/3467479281 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fde74101ca0 msgr2=0x7fde74102120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.927+0000 7fde7b359640 1 --2- 172.21.15.2:0/3467479281 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fde74101ca0 0x7fde74102120 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fde64009900 tx=0x7fde6402a410 comp rx=0 tx=0).stop 2024-05-14T16:56:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.927+0000 7fde7b359640 1 -- 172.21.15.2:0/3467479281 shutdown_connections 2024-05-14T16:56:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.927+0000 7fde7b359640 1 --2- 172.21.15.2:0/3467479281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde74102660 0x7fde7410e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.927+0000 7fde7b359640 1 --2- 172.21.15.2:0/3467479281 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fde74101ca0 0x7fde74102120 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.927+0000 7fde7b359640 1 --2- 172.21.15.2:0/3467479281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde741068f0 0x7fde74106cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.927+0000 7fde7b359640 1 -- 172.21.15.2:0/3467479281 >> 172.21.15.2:0/3467479281 conn(0x7fde74077750 msgr2=0x7fde74077b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.927+0000 7fde7b359640 1 -- 172.21.15.2:0/3467479281 shutdown_connections 2024-05-14T16:56:42.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.927+0000 7fde7b359640 1 -- 172.21.15.2:0/3467479281 wait complete. 2024-05-14T16:56:42.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.928+0000 7fde7b359640 1 Processor -- start 2024-05-14T16:56:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.928+0000 7fde7b359640 1 -- start start 2024-05-14T16:56:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.928+0000 7fde7b359640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde74101ca0 0x7fde74199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.928+0000 7fde7b359640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde74102660 0x7fde74199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.928+0000 7fde7b359640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fde741068f0 0x7fde7419d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.928+0000 7fde7b359640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fde74111bd0 con 0x7fde741068f0 2024-05-14T16:56:43.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.928+0000 7fde7b359640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fde74111a50 con 0x7fde74102660 2024-05-14T16:56:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.928+0000 7fde7b359640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fde74111d50 con 0x7fde74101ca0 2024-05-14T16:56:43.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.928+0000 7fde798f6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fde741068f0 0x7fde7419d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:43.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.928+0000 7fde798f6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fde741068f0 0x7fde7419d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49786/0 (socket says 172.21.15.2:49786) 2024-05-14T16:56:43.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.928+0000 7fde798f6640 1 -- 172.21.15.2:0/3894239367 learned_addr learned my addr 172.21.15.2:0/3894239367 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:56:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.929+0000 7fde788f4640 1 --2- 172.21.15.2:0/3894239367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde74102660 0x7fde74199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:43.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.929+0000 7fde790f5640 1 --2- 172.21.15.2:0/3894239367 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde74101ca0 0x7fde74199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.929+0000 7fde798f6640 1 -- 172.21.15.2:0/3894239367 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde74101ca0 msgr2=0x7fde74199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.929+0000 7fde798f6640 1 --2- 172.21.15.2:0/3894239367 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde74101ca0 0x7fde74199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.929+0000 7fde798f6640 1 -- 172.21.15.2:0/3894239367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde74102660 msgr2=0x7fde74199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:43.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.929+0000 7fde798f6640 1 --2- 172.21.15.2:0/3894239367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde74102660 0x7fde74199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.929+0000 7fde798f6640 1 -- 172.21.15.2:0/3894239367 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde7419e110 con 0x7fde741068f0 2024-05-14T16:56:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.929+0000 7fde798f6640 1 --2- 172.21.15.2:0/3894239367 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fde741068f0 0x7fde7419d9e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fde6c009fd0 tx=0x7fde6c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.929+0000 7fde6a7fc640 1 -- 172.21.15.2:0/3894239367 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c017070 con 0x7fde741068f0 2024-05-14T16:56:43.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.929+0000 7fde6a7fc640 1 -- 172.21.15.2:0/3894239367 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde6c003e20 con 0x7fde741068f0 2024-05-14T16:56:43.219 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.929+0000 7fde7b359640 1 -- 172.21.15.2:0/3894239367 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde7419e400 con 0x7fde741068f0 2024-05-14T16:56:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.929+0000 7fde6a7fc640 1 -- 172.21.15.2:0/3894239367 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c010460 con 0x7fde741068f0 2024-05-14T16:56:43.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.929+0000 7fde7b359640 1 -- 172.21.15.2:0/3894239367 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde741a5c90 con 0x7fde741068f0 2024-05-14T16:56:43.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.932+0000 7fde6a7fc640 1 -- 172.21.15.2:0/3894239367 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fde6c003970 con 0x7fde741068f0 2024-05-14T16:56:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.932+0000 7fde7e070740 1 Processor -- start 2024-05-14T16:56:43.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.932+0000 7fde7e070740 1 -- start start 2024-05-14T16:56:43.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.933+0000 7fde7e070740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645997d08f0 0x5645997d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.933+0000 7fde7e070740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645997d1210 0x5645997db590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.933+0000 7fde7e070740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645997dbad0 0x5645997ddec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:43.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.933+0000 7fde7e070740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x564599743c30 con 0x5645997d08f0 2024-05-14T16:56:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.933+0000 7fde7e070740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5645997337d0 con 0x5645997d1210 2024-05-14T16:56:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.933+0000 7fde7e070740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564599737810 con 0x5645997dbad0 2024-05-14T16:56:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.933+0000 7fde788f4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645997d08f0 0x5645997d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.933+0000 7fde788f4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645997d08f0 0x5645997d0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49800/0 (socket says 172.21.15.2:49800) 2024-05-14T16:56:43.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.933+0000 7fde788f4640 1 -- 172.21.15.2:0/3886475122 learned_addr learned my addr 172.21.15.2:0/3886475122 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:56:43.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.933+0000 7fde790f5640 1 --2- 172.21.15.2:0/3886475122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645997d1210 0x5645997db590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.933+0000 7fde788f4640 1 -- 172.21.15.2:0/3886475122 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645997dbad0 msgr2=0x5645997ddec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:43.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.933+0000 7fde788f4640 1 --2- 172.21.15.2:0/3886475122 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645997dbad0 0x5645997ddec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.933+0000 7fde788f4640 1 -- 172.21.15.2:0/3886475122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645997d1210 msgr2=0x5645997db590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:43.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.933+0000 7fde788f4640 1 --2- 172.21.15.2:0/3886475122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645997d1210 0x5645997db590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:43.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.933+0000 7fde788f4640 1 -- 172.21.15.2:0/3886475122 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564599448210 con 0x5645997d08f0 2024-05-14T16:56:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.933+0000 7fde6a7fc640 1 --2- 172.21.15.2:0/3894239367 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fde600b2a10 0x7fde600b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:43.550 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.933+0000 7fde790f5640 1 --2- 172.21.15.2:0/3894239367 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fde600b2a10 0x7fde600b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:43.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.933+0000 7fde788f4640 1 --2- 172.21.15.2:0/3886475122 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645997d08f0 0x5645997d0cd0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fde6402a930 tx=0x7fde64032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1605cbe35504bfc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:43.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.934+0000 7fde6a7fc640 1 -- 172.21.15.2:0/3894239367 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fde6c0e2520 con 0x7fde741068f0 2024-05-14T16:56:43.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.934+0000 7fde5f7fe640 1 -- 172.21.15.2:0/3886475122 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6403f070 con 0x5645997d08f0 2024-05-14T16:56:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.934+0000 7fde5f7fe640 1 -- 172.21.15.2:0/3886475122 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde64032e40 con 0x5645997d08f0 2024-05-14T16:56:43.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.934+0000 7fde5f7fe640 1 -- 172.21.15.2:0/3886475122 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde64035070 con 0x5645997d08f0 2024-05-14T16:56:43.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.934+0000 7fde7e070740 1 -- 172.21.15.2:0/3886475122 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645997d08f0 msgr2=0x5645997d0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.934+0000 7fde790f5640 1 --2- 172.21.15.2:0/3894239367 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fde600b2a10 0x7fde600b4ed0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x5645997d1e90 tx=0x7fde7000b4d0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.934+0000 7fde7e070740 1 --2- 172.21.15.2:0/3886475122 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645997d08f0 0x5645997d0cd0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fde6402a930 tx=0x7fde64032d50 comp rx=0 tx=0).stop 2024-05-14T16:56:43.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.934+0000 7fde7e070740 1 -- 172.21.15.2:0/3886475122 shutdown_connections 2024-05-14T16:56:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.934+0000 7fde7e070740 1 --2- 172.21.15.2:0/3886475122 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645997dbad0 0x5645997ddec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.934+0000 7fde7e070740 1 --2- 172.21.15.2:0/3886475122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645997d1210 0x5645997db590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.934+0000 7fde7e070740 1 --2- 172.21.15.2:0/3886475122 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645997d08f0 0x5645997d0cd0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.934+0000 7fde7e070740 1 -- 172.21.15.2:0/3886475122 >> 172.21.15.2:0/3886475122 conn(0x5645997cce00 msgr2=0x5645997cd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.935+0000 7fde7e070740 1 -- 172.21.15.2:0/3886475122 shutdown_connections 2024-05-14T16:56:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.935+0000 7fde7e070740 1 -- 172.21.15.2:0/3886475122 wait complete. 2024-05-14T16:56:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.935+0000 7fde7e070740 1 -- start start 2024-05-14T16:56:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.935+0000 7fde7e070740 1 Processor -- start 2024-05-14T16:56:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.935+0000 7fde7e070740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645997d08f0 0x5645997db440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.935+0000 7fde7e070740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645997d1210 0x5645997d8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.935+0000 7fde7e070740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645997dbad0 0x5645997d8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.935+0000 7fde7e070740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x564599419790 con 0x5645997d08f0 2024-05-14T16:56:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.936+0000 7fde7e070740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5645996ef890 con 0x5645997d1210 2024-05-14T16:56:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.936+0000 7fde7e070740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5645997de740 con 0x5645997dbad0 2024-05-14T16:56:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.936+0000 7fde7e070740 10 client.0 _ll_register_callbacks cb 0x5645996f2d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:56:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.936+0000 7fde790f5640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645997d08f0 0x5645997db440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.936+0000 7fde790f5640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645997d08f0 0x5645997db440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49808/0 (socket says 172.21.15.2:49808) 2024-05-14T16:56:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.936+0000 7fde790f5640 1 -- 172.21.15.2:0/59173430 learned_addr learned my addr 172.21.15.2:0/59173430 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:56:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.936+0000 7fde798f6640 1 --2- 172.21.15.2:0/59173430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645997dbad0 0x5645997d8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.936+0000 7fde788f4640 1 --2- 172.21.15.2:0/59173430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645997d1210 0x5645997d8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:43.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.936+0000 7fde790f5640 1 -- 172.21.15.2:0/59173430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645997dbad0 msgr2=0x5645997d8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.936+0000 7fde790f5640 1 --2- 172.21.15.2:0/59173430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645997dbad0 0x5645997d8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:43.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.936+0000 7fde790f5640 1 -- 172.21.15.2:0/59173430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645997d1210 msgr2=0x5645997d8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.936+0000 7fde790f5640 1 --2- 172.21.15.2:0/59173430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645997d1210 0x5645997d8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:43.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.936+0000 7fde790f5640 1 -- 172.21.15.2:0/59173430 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564599514620 con 0x5645997d08f0 2024-05-14T16:56:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.936+0000 7fde790f5640 1 --2- 172.21.15.2:0/59173430 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645997d08f0 0x5645997db440 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fde7001d720 tx=0x7fde700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c9532dafc29200d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:43.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.936+0000 7fde5cff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:56:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.936+0000 7fde5cff9640 1 -- 172.21.15.2:0/59173430 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7001e820 con 0x5645997d08f0 2024-05-14T16:56:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.936+0000 7fde5cff9640 1 -- 172.21.15.2:0/59173430 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fde7001e9c0 con 0x5645997d08f0 2024-05-14T16:56:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.936+0000 7fde7e070740 1 -- 172.21.15.2:0/59173430 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564599745b80 con 0x5645997d08f0 2024-05-14T16:56:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.937+0000 7fde5cff9640 1 -- 172.21.15.2:0/59173430 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fde7001ebc0 con 0x5645997d08f0 2024-05-14T16:56:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.937+0000 7fde7e070740 10 client.15540 fetch_fsmap learned FSMap version 30 2024-05-14T16:56:43.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.937+0000 7fde7e070740 1 -- 172.21.15.2:0/59173430 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5645997ea250 con 0x5645997d08f0 2024-05-14T16:56:43.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.937+0000 7fde5cff9640 1 -- 172.21.15.2:0/59173430 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564599745b80 con 0x5645997d08f0 2024-05-14T16:56:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.937+0000 7fde5cff9640 1 -- 172.21.15.2:0/59173430 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fde70032400 con 0x5645997d08f0 2024-05-14T16:56:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.937+0000 7fde7e070740 10 client.15540 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T16:56:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.937+0000 7fde7e070740 10 client.15540 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:56:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.937+0000 7fde7e070740 20 client.15540 populate_metadata read hostname 'smithi002' 2024-05-14T16:56:44.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.937+0000 7fde7e070740 1 --2- 172.21.15.2:0/59173430 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5645997eb760 0x5645997edb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.937+0000 7fde7e070740 4 client.15540 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:56:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.937+0000 7fde7e070740 1 -- 172.21.15.2:0/59173430 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56459973f360 con 0x5645997eb760 2024-05-14T16:56:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.937+0000 7fde788f4640 1 --2- 172.21.15.2:0/59173430 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5645997eb760 0x5645997edb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:44.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.938+0000 7fde788f4640 1 --2- 172.21.15.2:0/59173430 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5645997eb760 0x5645997edb90 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66365aa6dc1b18b7 server_cookie=a897e94af84d8ace in_seq=0 out_seq=0 2024-05-14T16:56:44.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.938+0000 7fde5cff9640 10 client.15540 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:56:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.938+0000 7fde5cff9640 1 -- 172.21.15.2:0/59173430 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56459973f360 con 0x5645997eb760 2024-05-14T16:56:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.939+0000 7fde5cff9640 10 client.15540 handle_command_reply: tid=0 2024-05-14T16:56:44.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.939+0000 7fde7e070740 1 client.15540 shutdown 2024-05-14T16:56:44.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.939+0000 7fde7e070740 1 -- 172.21.15.2:0/59173430 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645997d08f0 msgr2=0x5645997db440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.939+0000 7fde7e070740 1 --2- 172.21.15.2:0/59173430 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645997d08f0 0x5645997db440 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fde7001d720 tx=0x7fde700046c0 comp rx=0 tx=0).stop 2024-05-14T16:56:44.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.939+0000 7fde7e070740 1 -- 172.21.15.2:0/59173430 shutdown_connections 2024-05-14T16:56:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.939+0000 7fde7e070740 1 --2- 172.21.15.2:0/59173430 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5645997eb760 0x5645997edb90 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:44.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.939+0000 7fde7e070740 1 --2- 172.21.15.2:0/59173430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645997dbad0 0x5645997d8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.939+0000 7fde7e070740 1 --2- 172.21.15.2:0/59173430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645997d1210 0x5645997d8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.939+0000 7fde7e070740 1 --2- 172.21.15.2:0/59173430 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645997d08f0 0x5645997db440 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.939+0000 7fde5cff9640 0 client.15540 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:56:44.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.939+0000 7fde7e070740 1 -- 172.21.15.2:0/59173430 >> 172.21.15.2:0/59173430 conn(0x5645997cce00 msgr2=0x5645997cc100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.939+0000 7fde7e070740 1 -- 172.21.15.2:0/59173430 shutdown_connections 2024-05-14T16:56:44.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.939+0000 7fde7e070740 1 -- 172.21.15.2:0/59173430 wait complete. 2024-05-14T16:56:44.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.939+0000 7fde7e070740 20 client.15540 trim_cache size 0 max 16384 2024-05-14T16:56:44.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.955+0000 7fde7e070740 1 Processor -- start 2024-05-14T16:56:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.955+0000 7fde7e070740 1 -- start start 2024-05-14T16:56:44.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.955+0000 7fde7e070740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645997d08f0 0x5645997d3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.955+0000 7fde7e070740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645997d1210 0x5645997daf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:44.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.955+0000 7fde7e070740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645997dbad0 0x5645997f34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.955+0000 7fde7e070740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fde741b0cd0 con 0x5645997d1210 2024-05-14T16:56:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.955+0000 7fde7e070740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fde741ae510 con 0x5645997dbad0 2024-05-14T16:56:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.955+0000 7fde7e070740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fde741ac360 con 0x5645997d08f0 2024-05-14T16:56:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.955+0000 7fde788f4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645997d1210 0x5645997daf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:44.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.955+0000 7fde788f4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645997d1210 0x5645997daf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49820/0 (socket says 172.21.15.2:49820) 2024-05-14T16:56:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.955+0000 7fde788f4640 1 -- 172.21.15.2:0/277722471 learned_addr learned my addr 172.21.15.2:0/277722471 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:56:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.955+0000 7fde798f6640 1 --2- 172.21.15.2:0/277722471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645997dbad0 0x5645997f34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:44.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.955+0000 7fde790f5640 1 --2- 172.21.15.2:0/277722471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645997d08f0 0x5645997d3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.955+0000 7fde788f4640 1 -- 172.21.15.2:0/277722471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645997d08f0 msgr2=0x5645997d3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.955+0000 7fde788f4640 1 --2- 172.21.15.2:0/277722471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645997d08f0 0x5645997d3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.955+0000 7fde788f4640 1 -- 172.21.15.2:0/277722471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645997dbad0 msgr2=0x5645997f34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.955+0000 7fde788f4640 1 --2- 172.21.15.2:0/277722471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645997dbad0 0x5645997f34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.956+0000 7fde788f4640 1 -- 172.21.15.2:0/277722471 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645997ee210 con 0x5645997d1210 2024-05-14T16:56:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.956+0000 7fde798f6640 1 --2- 172.21.15.2:0/277722471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645997dbad0 0x5645997f34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:56:44.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.956+0000 7fde788f4640 1 --2- 172.21.15.2:0/277722471 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645997d1210 0x5645997daf70 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fde64009900 tx=0x7fde64031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ae4ecb4ab7b2ed server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:44.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.956+0000 7fde5effd640 1 -- 172.21.15.2:0/277722471 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6403f040 con 0x5645997d1210 2024-05-14T16:56:44.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.956+0000 7fde5effd640 1 -- 172.21.15.2:0/277722471 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde6404e920 con 0x5645997d1210 2024-05-14T16:56:44.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.956+0000 7fde5effd640 1 -- 172.21.15.2:0/277722471 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde64035040 con 0x5645997d1210 2024-05-14T16:56:44.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.956+0000 7fde7e070740 1 -- 172.21.15.2:0/277722471 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645997d1210 msgr2=0x5645997daf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:44.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.956+0000 7fde7e070740 1 --2- 172.21.15.2:0/277722471 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645997d1210 0x5645997daf70 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fde64009900 tx=0x7fde64031400 comp rx=0 tx=0).stop 2024-05-14T16:56:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.956+0000 7fde7e070740 1 -- 172.21.15.2:0/277722471 shutdown_connections 2024-05-14T16:56:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.956+0000 7fde7e070740 1 --2- 172.21.15.2:0/277722471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645997dbad0 0x5645997f34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:44.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.956+0000 7fde7e070740 1 --2- 172.21.15.2:0/277722471 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645997d1210 0x5645997daf70 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.956+0000 7fde7e070740 1 --2- 172.21.15.2:0/277722471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645997d08f0 0x5645997d3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:44.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.956+0000 7fde7e070740 1 -- 172.21.15.2:0/277722471 >> 172.21.15.2:0/277722471 conn(0x5645997cce00 msgr2=0x5645997cc100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:44.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.957+0000 7fde7e070740 1 -- 172.21.15.2:0/277722471 shutdown_connections 2024-05-14T16:56:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.957+0000 7fde7e070740 1 -- 172.21.15.2:0/277722471 wait complete. 2024-05-14T16:56:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.957+0000 7fde7e070740 1 -- start start 2024-05-14T16:56:44.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.957+0000 7fde7e070740 1 Processor -- start 2024-05-14T16:56:44.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.957+0000 7fde7e070740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645997d08f0 0x5645997dfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.957+0000 7fde7e070740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645997d1210 0x5645997e0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.958+0000 7fde7e070740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645997dbad0 0x56459984f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.958+0000 7fde7e070740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fde741ac9d0 con 0x5645997d08f0 2024-05-14T16:56:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.958+0000 7fde7e070740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fde741ade40 con 0x5645997dbad0 2024-05-14T16:56:44.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.958+0000 7fde7e070740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fde741ad0a0 con 0x5645997d1210 2024-05-14T16:56:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.958+0000 7fde7e070740 10 client.0 _ll_register_callbacks cb 0x7fde7001d3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:56:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.958+0000 7fde790f5640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645997d08f0 0x5645997dfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.958+0000 7fde790f5640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645997d08f0 0x5645997dfdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49822/0 (socket says 172.21.15.2:49822) 2024-05-14T16:56:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.958+0000 7fde790f5640 1 -- 172.21.15.2:0/3822662752 learned_addr learned my addr 172.21.15.2:0/3822662752 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:56:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.958+0000 7fde788f4640 1 --2- 172.21.15.2:0/3822662752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645997d1210 0x5645997e0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:44.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.958+0000 7fde798f6640 1 --2- 172.21.15.2:0/3822662752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645997dbad0 0x56459984f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.958+0000 7fde790f5640 1 -- 172.21.15.2:0/3822662752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645997d1210 msgr2=0x5645997e0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.958+0000 7fde790f5640 1 --2- 172.21.15.2:0/3822662752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645997d1210 0x5645997e0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:45.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.958+0000 7fde790f5640 1 -- 172.21.15.2:0/3822662752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645997dbad0 msgr2=0x56459984f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:45.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.958+0000 7fde790f5640 1 --2- 172.21.15.2:0/3822662752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645997dbad0 0x56459984f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:45.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.958+0000 7fde790f5640 1 -- 172.21.15.2:0/3822662752 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56459984f650 con 0x5645997d08f0 2024-05-14T16:56:45.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.958+0000 7fde790f5640 1 --2- 172.21.15.2:0/3822662752 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645997d08f0 0x5645997dfdd0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fde70017040 tx=0x7fde7000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e06136989e5ac4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:45.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.958+0000 7fde5cff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:56:45.090 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.958+0000 7fde5cff9640 1 -- 172.21.15.2:0/3822662752 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde70042060 con 0x5645997d08f0 2024-05-14T16:56:45.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.958+0000 7fde5cff9640 1 -- 172.21.15.2:0/3822662752 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fde7001e620 con 0x5645997d08f0 2024-05-14T16:56:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.958+0000 7fde7e070740 1 -- 172.21.15.2:0/3822662752 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde6404e920 con 0x5645997d08f0 2024-05-14T16:56:45.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.958+0000 7fde5cff9640 1 -- 172.21.15.2:0/3822662752 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fde7000b9d0 con 0x5645997d08f0 2024-05-14T16:56:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.958+0000 7fde7e070740 10 client.15546 fetch_fsmap learned FSMap version 30 2024-05-14T16:56:45.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.958+0000 7fde7e070740 1 -- 172.21.15.2:0/3822662752 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564599856a70 con 0x5645997d08f0 2024-05-14T16:56:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.959+0000 7fde5cff9640 1 -- 172.21.15.2:0/3822662752 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde6404e920 con 0x5645997d08f0 2024-05-14T16:56:45.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.959+0000 7fde5cff9640 1 -- 172.21.15.2:0/3822662752 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fde70004730 con 0x5645997d08f0 2024-05-14T16:56:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.959+0000 7fde7e070740 10 client.15546 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T16:56:45.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.959+0000 7fde7e070740 10 client.15546 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:56:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.959+0000 7fde7e070740 20 client.15546 populate_metadata read hostname 'smithi002' 2024-05-14T16:56:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.959+0000 7fde7e070740 1 --2- 172.21.15.2:0/3822662752 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5645998571d0 0x5645998595f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.959+0000 7fde7e070740 4 client.15546 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:56:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.959+0000 7fde7e070740 1 -- 172.21.15.2:0/3822662752 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5645996b95f0 con 0x5645998571d0 2024-05-14T16:56:45.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.959+0000 7fde788f4640 1 --2- 172.21.15.2:0/3822662752 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5645998571d0 0x5645998595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.959+0000 7fde5cff9640 10 client.15546 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:56:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.959+0000 7fde788f4640 1 --2- 172.21.15.2:0/3822662752 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5645998571d0 0x5645998595f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9106c680638b82f3 server_cookie=e062b3c5de89bfe3 in_seq=0 out_seq=0 2024-05-14T16:56:45.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.977+0000 7fde5cff9640 1 -- 172.21.15.2:0/3822662752 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5645996b95f0 con 0x5645998571d0 2024-05-14T16:56:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.977+0000 7fde5cff9640 10 client.15546 handle_command_reply: tid=0 2024-05-14T16:56:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.977+0000 7fde7e070740 1 client.15546 shutdown 2024-05-14T16:56:45.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.977+0000 7fde7e070740 1 -- 172.21.15.2:0/3822662752 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645997d08f0 msgr2=0x5645997dfdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.977+0000 7fde7e070740 1 --2- 172.21.15.2:0/3822662752 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645997d08f0 0x5645997dfdd0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fde70017040 tx=0x7fde7000bf10 comp rx=0 tx=0).stop 2024-05-14T16:56:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.977+0000 7fde7e070740 1 -- 172.21.15.2:0/3822662752 shutdown_connections 2024-05-14T16:56:45.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.977+0000 7fde7e070740 1 --2- 172.21.15.2:0/3822662752 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5645998571d0 0x5645998595f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:45.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.977+0000 7fde7e070740 1 --2- 172.21.15.2:0/3822662752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645997dbad0 0x56459984f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.977+0000 7fde5cff9640 0 client.15546 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:56:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.977+0000 7fde7e070740 1 --2- 172.21.15.2:0/3822662752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645997d1210 0x5645997e0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.977+0000 7fde7e070740 1 --2- 172.21.15.2:0/3822662752 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645997d08f0 0x5645997dfdd0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.977+0000 7fde7e070740 1 -- 172.21.15.2:0/3822662752 >> 172.21.15.2:0/3822662752 conn(0x5645997cce00 msgr2=0x5645997d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.977+0000 7fde7e070740 1 -- 172.21.15.2:0/3822662752 shutdown_connections 2024-05-14T16:56:45.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.977+0000 7fde7e070740 1 -- 172.21.15.2:0/3822662752 wait complete. 2024-05-14T16:56:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.977+0000 7fde7e070740 20 client.15546 trim_cache size 0 max 16384 2024-05-14T16:56:45.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T16:56:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T16:56:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T16:56:45.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.978+0000 7fde7b359640 1 -- 172.21.15.2:0/3894239367 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fde600b2a10 msgr2=0x7fde600b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.978+0000 7fde7b359640 1 --2- 172.21.15.2:0/3894239367 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fde600b2a10 0x7fde600b4ed0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x5645997d1e90 tx=0x7fde7000b4d0 comp rx=0 tx=0).stop 2024-05-14T16:56:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.978+0000 7fde7b359640 1 -- 172.21.15.2:0/3894239367 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fde741068f0 msgr2=0x7fde7419d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.978+0000 7fde7b359640 1 --2- 172.21.15.2:0/3894239367 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fde741068f0 0x7fde7419d9e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fde6c009fd0 tx=0x7fde6c00b260 comp rx=0 tx=0).stop 2024-05-14T16:56:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.978+0000 7fde7b359640 1 -- 172.21.15.2:0/3894239367 shutdown_connections 2024-05-14T16:56:45.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.978+0000 7fde7b359640 1 --2- 172.21.15.2:0/3894239367 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fde600b2a10 0x7fde600b4ed0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.978+0000 7fde7b359640 1 --2- 172.21.15.2:0/3894239367 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fde741068f0 0x7fde7419d9e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:45.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.978+0000 7fde7b359640 1 --2- 172.21.15.2:0/3894239367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde74102660 0x7fde74199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.978+0000 7fde7b359640 1 --2- 172.21.15.2:0/3894239367 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde74101ca0 0x7fde74199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.978+0000 7fde7b359640 1 -- 172.21.15.2:0/3894239367 >> 172.21.15.2:0/3894239367 conn(0x7fde74077750 msgr2=0x7fde7406b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.978+0000 7fde7b359640 1 -- 172.21.15.2:0/3894239367 shutdown_connections 2024-05-14T16:56:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:42.978+0000 7fde7b359640 1 -- 172.21.15.2:0/3894239367 wait complete. 2024-05-14T16:56:45.724 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:56:44 smithi151 ceph-mon[36066]: pgmap v64: 97 pgs: 97 active+clean; 1.2 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 10 MiB/s wr, 485 op/s 2024-05-14T16:56:45.732 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:56:44 smithi123 ceph-mon[35971]: pgmap v64: 97 pgs: 97 active+clean; 1.2 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 10 MiB/s wr, 485 op/s 2024-05-14T16:56:45.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:56:44 smithi002 ceph-mon[31538]: pgmap v64: 97 pgs: 97 active+clean; 1.2 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 10 MiB/s wr, 485 op/s 2024-05-14T16:56:46.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:56:46 smithi123 ceph-mon[35971]: pgmap v65: 97 pgs: 97 active+clean; 1.2 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 476 op/s 2024-05-14T16:56:46.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:56:46 smithi002 ceph-mon[31538]: pgmap v65: 97 pgs: 97 active+clean; 1.2 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 476 op/s 2024-05-14T16:56:46.751 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 90.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:56:46.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:56:46 smithi151 ceph-mon[36066]: pgmap v65: 97 pgs: 97 active+clean; 1.2 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 476 op/s 2024-05-14T16:56:47.152 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:56:47.462 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:56:48.626 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:56:48 smithi002 ceph-mon[31538]: pgmap v66: 97 pgs: 97 active+clean; 1.3 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 18 MiB/s wr, 783 op/s 2024-05-14T16:56:48.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:56:48 smithi123 ceph-mon[35971]: pgmap v66: 97 pgs: 97 active+clean; 1.3 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 18 MiB/s wr, 783 op/s 2024-05-14T16:56:48.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:56:48 smithi151 ceph-mon[36066]: pgmap v66: 97 pgs: 97 active+clean; 1.3 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 18 MiB/s wr, 783 op/s 2024-05-14T16:56:49.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.561+0000 7f3cd0fd1700 1 -- 172.21.15.2:0/1507847429 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ccc102470 msgr2=0x7f3ccc1028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:49.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.561+0000 7f3cd0fd1700 1 --2- 172.21.15.2:0/1507847429 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ccc102470 0x7f3ccc1028f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0009990 tx=0x7f3cc0009ca0 comp rx=0 tx=0).stop 2024-05-14T16:56:49.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.561+0000 7f3cd0fd1700 1 -- 172.21.15.2:0/1507847429 shutdown_connections 2024-05-14T16:56:49.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.561+0000 7f3cd0fd1700 1 --2- 172.21.15.2:0/1507847429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ccc102e30 0x7f3ccc10f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:49.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.561+0000 7f3cd0fd1700 1 --2- 172.21.15.2:0/1507847429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ccc1084b0 0x7f3ccc108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:49.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.561+0000 7f3cd0fd1700 1 --2- 172.21.15.2:0/1507847429 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ccc102470 0x7f3ccc1028f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:49.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.561+0000 7f3cd0fd1700 1 -- 172.21.15.2:0/1507847429 >> 172.21.15.2:0/1507847429 conn(0x7f3ccc0793f0 msgr2=0x7f3ccc079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:49.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.561+0000 7f3cd0fd1700 1 -- 172.21.15.2:0/1507847429 shutdown_connections 2024-05-14T16:56:49.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.561+0000 7f3cd0fd1700 1 -- 172.21.15.2:0/1507847429 wait complete. 2024-05-14T16:56:49.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.562+0000 7f3cd0fd1700 1 Processor -- start 2024-05-14T16:56:49.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.562+0000 7f3cd0fd1700 1 -- start start 2024-05-14T16:56:49.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.562+0000 7f3cd0fd1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ccc102470 0x7f3ccc104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:49.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.562+0000 7f3cd0fd1700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ccc102e30 0x7f3ccc1045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:49.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.562+0000 7f3cd0fd1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ccc1084b0 0x7f3ccc104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:49.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.562+0000 7f3cd0fd1700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3ccc1125c0 con 0x7f3ccc1084b0 2024-05-14T16:56:49.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.562+0000 7f3cd0fd1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3ccc112440 con 0x7f3ccc102470 2024-05-14T16:56:49.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.562+0000 7f3cd0fd1700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3ccc112740 con 0x7f3ccc102e30 2024-05-14T16:56:49.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.562+0000 7f3ccad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ccc1084b0 0x7f3ccc104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:49.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.562+0000 7f3ccad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ccc1084b0 0x7f3ccc104b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44906/0 (socket says 172.21.15.2:44906) 2024-05-14T16:56:49.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.562+0000 7f3ccad9d700 1 -- 172.21.15.2:0/4078687445 learned_addr learned my addr 172.21.15.2:0/4078687445 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:56:49.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.562+0000 7f3cc9d9b700 1 --2- 172.21.15.2:0/4078687445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ccc102e30 0x7f3ccc1045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:49.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.562+0000 7f3cca59c700 1 --2- 172.21.15.2:0/4078687445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ccc102470 0x7f3ccc104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:49.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.562+0000 7f3ccad9d700 1 -- 172.21.15.2:0/4078687445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ccc102e30 msgr2=0x7f3ccc1045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:49.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.562+0000 7f3ccad9d700 1 --2- 172.21.15.2:0/4078687445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ccc102e30 0x7f3ccc1045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:49.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.562+0000 7f3ccad9d700 1 -- 172.21.15.2:0/4078687445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ccc102470 msgr2=0x7f3ccc104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:49.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.562+0000 7f3ccad9d700 1 --2- 172.21.15.2:0/4078687445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ccc102470 0x7f3ccc104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:49.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.562+0000 7f3ccad9d700 1 -- 172.21.15.2:0/4078687445 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ccc1a81f0 con 0x7f3ccc1084b0 2024-05-14T16:56:49.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.562+0000 7f3cca59c700 1 --2- 172.21.15.2:0/4078687445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ccc102470 0x7f3ccc104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:56:49.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.562+0000 7f3ccad9d700 1 --2- 172.21.15.2:0/4078687445 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ccc1084b0 0x7f3ccc104b00 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4007ee0 tx=0x7f3cc400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:49.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.563+0000 7f3cbb7fe700 1 -- 172.21.15.2:0/4078687445 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc400dbd0 con 0x7f3ccc1084b0 2024-05-14T16:56:49.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.563+0000 7f3cbb7fe700 1 -- 172.21.15.2:0/4078687445 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cc400dd70 con 0x7f3ccc1084b0 2024-05-14T16:56:50.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.563+0000 7f3cd0fd1700 1 -- 172.21.15.2:0/4078687445 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ccc1a84e0 con 0x7f3ccc1084b0 2024-05-14T16:56:50.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.563+0000 7f3cbb7fe700 1 -- 172.21.15.2:0/4078687445 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4005990 con 0x7f3ccc1084b0 2024-05-14T16:56:50.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.563+0000 7f3cd0fd1700 1 -- 172.21.15.2:0/4078687445 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ccc1a89a0 con 0x7f3ccc1084b0 2024-05-14T16:56:50.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.564+0000 7f3cd0fd1700 1 -- 172.21.15.2:0/4078687445 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ccc06a230 con 0x7f3ccc1084b0 2024-05-14T16:56:50.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.565+0000 7f3cbb7fe700 1 -- 172.21.15.2:0/4078687445 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f3cc401a020 con 0x7f3ccc1084b0 2024-05-14T16:56:50.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.566+0000 7f3cbb7fe700 1 --2- 172.21.15.2:0/4078687445 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f3cb40b29f0 0x7f3cb40b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:50.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.566+0000 7f3cca59c700 1 --2- 172.21.15.2:0/4078687445 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f3cb40b29f0 0x7f3cb40b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:50.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.566+0000 7f3cbb7fe700 1 -- 172.21.15.2:0/4078687445 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f3cc40e2500 con 0x7f3ccc1084b0 2024-05-14T16:56:50.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.567+0000 7f3cca59c700 1 --2- 172.21.15.2:0/4078687445 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f3cb40b29f0 0x7f3cb40b4eb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc005e10 tx=0x7f3cbc005d60 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:50.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.569+0000 7f3cbb7fe700 1 -- 172.21.15.2:0/4078687445 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3cc4010070 con 0x7f3ccc1084b0 2024-05-14T16:56:50.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.838+0000 7f3cd0fd1700 1 -- 172.21.15.2:0/4078687445 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ccc02e560 con 0x7f3ccc1084b0 2024-05-14T16:56:50.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.838+0000 7f3cbb7fe700 1 -- 172.21.15.2:0/4078687445 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11857 (secure 0 0 0) 0x7f3cc40a9eb0 con 0x7f3ccc1084b0 2024-05-14T16:56:50.227 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:56:50.233 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2275696923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2275696923},{"type":"v1","addr":"172.21.15.2:6835","nonce":2275696923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/716718761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":716718761},{"type":"v1","addr":"172.21.15.2:6839","nonce":716718761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/3644815179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3644815179},{"type":"v1","addr":"172.21.15.2:6841","nonce":3644815179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1880336731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1880336731},{"type":"v1","addr":"172.21.15.151:6833","nonce":1880336731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2415605548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2415605548},{"type":"v1","addr":"172.21.15.2:6837","nonce":2415605548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2635167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2635167},{"type":"v1","addr":"172.21.15.123:6833","nonce":2635167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:51:34.106919+0000","modified":"2024-05-14T16:56:06.904708+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24455,"mds_2":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24415":{"gid":24415,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/762044339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":762044339},{"type":"v1","addr":"172.21.15.151:6835","nonce":762044339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/132043151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":132043151},{"type":"v1","addr":"172.21.15.151:6837","nonce":132043151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/4107834375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":4107834375},{"type":"v1","addr":"172.21.15.151:6839","nonce":4107834375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.123:6835/1198670013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1198670013},{"type":"v1","addr":"172.21.15.123:6835","nonce":1198670013}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/333426645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":333426645},{"type":"v1","addr":"172.21.15.123:6837","nonce":333426645}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/580275877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":580275877},{"type":"v1","addr":"172.21.15.123:6839","nonce":580275877}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24455,24479,24431]},"id":1}]} 2024-05-14T16:56:50.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.841+0000 7f3cd0fd1700 1 -- 172.21.15.2:0/4078687445 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f3cb40b29f0 msgr2=0x7f3cb40b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:50.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.841+0000 7f3cd0fd1700 1 --2- 172.21.15.2:0/4078687445 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f3cb40b29f0 0x7f3cb40b4eb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc005e10 tx=0x7f3cbc005d60 comp rx=0 tx=0).stop 2024-05-14T16:56:50.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.841+0000 7f3cd0fd1700 1 -- 172.21.15.2:0/4078687445 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ccc1084b0 msgr2=0x7f3ccc104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:50.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.841+0000 7f3cd0fd1700 1 --2- 172.21.15.2:0/4078687445 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ccc1084b0 0x7f3ccc104b00 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4007ee0 tx=0x7f3cc400d3b0 comp rx=0 tx=0).stop 2024-05-14T16:56:50.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.841+0000 7f3cd0fd1700 1 -- 172.21.15.2:0/4078687445 shutdown_connections 2024-05-14T16:56:50.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.841+0000 7f3cd0fd1700 1 --2- 172.21.15.2:0/4078687445 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f3cb40b29f0 0x7f3cb40b4eb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:50.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.841+0000 7f3cd0fd1700 1 --2- 172.21.15.2:0/4078687445 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ccc1084b0 0x7f3ccc104b00 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:50.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.841+0000 7f3cd0fd1700 1 --2- 172.21.15.2:0/4078687445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ccc102470 0x7f3ccc104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:50.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.841+0000 7f3cd0fd1700 1 --2- 172.21.15.2:0/4078687445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ccc102e30 0x7f3ccc1045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:50.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.841+0000 7f3cd0fd1700 1 -- 172.21.15.2:0/4078687445 >> 172.21.15.2:0/4078687445 conn(0x7f3ccc0793f0 msgr2=0x7f3ccc06c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:50.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.841+0000 7f3cd0fd1700 1 -- 172.21.15.2:0/4078687445 shutdown_connections 2024-05-14T16:56:50.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:49.841+0000 7f3cd0fd1700 1 -- 172.21.15.2:0/4078687445 wait complete. 2024-05-14T16:56:50.424 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 30 2024-05-14T16:56:50.447 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:56:50 smithi002 ceph-mon[31538]: pgmap v67: 97 pgs: 97 active+clean; 1.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 586 op/s 2024-05-14T16:56:50.454 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:56:50 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:56:50.459 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:56:50 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/4078687445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:56:50.473 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id a7bd3c7e --release --await' 2024-05-14T16:56:50.480 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id a7bd3c7e --release --await 2024-05-14T16:56:50.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:56:50 smithi123 ceph-mon[35971]: pgmap v67: 97 pgs: 97 active+clean; 1.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 586 op/s 2024-05-14T16:56:50.675 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:56:50 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:56:50.683 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:56:50 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/4078687445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:56:50.764 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:56:50.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:56:50 smithi151 ceph-mon[36066]: pgmap v67: 97 pgs: 97 active+clean; 1.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 586 op/s 2024-05-14T16:56:50.860 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:56:50 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:56:50.941 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:56:50 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/4078687445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:56:51.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:56:51 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:56:51.513 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:56:51 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:56:51.520 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:56:51 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:56:51.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:56:51 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:56:51.686 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:56:51 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:56:51.693 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:56:51 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:56:51.749 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 95.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:56:51.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:56:51 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:56:51.860 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:56:51 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:56:51.867 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:56:51 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:56:52.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:56:52 smithi002 ceph-mon[31538]: pgmap v68: 97 pgs: 97 active+clean; 1.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 586 op/s 2024-05-14T16:56:52.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.616+0000 7fa89b96f700 1 -- 172.21.15.2:0/161066361 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa894102440 msgr2=0x7fa8941028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:52.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.616+0000 7fa89b96f700 1 --2- 172.21.15.2:0/161066361 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa894102440 0x7fa8941028c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa884009990 tx=0x7fa884009ca0 comp rx=0 tx=0).stop 2024-05-14T16:56:52.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.616+0000 7fa89b96f700 1 -- 172.21.15.2:0/161066361 shutdown_connections 2024-05-14T16:56:52.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.616+0000 7fa89b96f700 1 --2- 172.21.15.2:0/161066361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa894102e00 0x7fa89410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:52.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.616+0000 7fa89b96f700 1 --2- 172.21.15.2:0/161066361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa894108520 0x7fa894108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:52.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.616+0000 7fa89b96f700 1 --2- 172.21.15.2:0/161066361 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa894102440 0x7fa8941028c0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:52.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.617+0000 7fa89b96f700 1 -- 172.21.15.2:0/161066361 >> 172.21.15.2:0/161066361 conn(0x7fa8940793f0 msgr2=0x7fa894079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:52.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.617+0000 7fa89b96f700 1 -- 172.21.15.2:0/161066361 shutdown_connections 2024-05-14T16:56:52.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.617+0000 7fa89b96f700 1 -- 172.21.15.2:0/161066361 wait complete. 2024-05-14T16:56:52.741 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:56:52 smithi123 ceph-mon[35971]: pgmap v68: 97 pgs: 97 active+clean; 1.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 586 op/s 2024-05-14T16:56:52.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.617+0000 7fa89b96f700 1 Processor -- start 2024-05-14T16:56:52.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.617+0000 7fa89b96f700 1 -- start start 2024-05-14T16:56:52.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.618+0000 7fa89b96f700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa894102440 0x7fa89419fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:52.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.618+0000 7fa89b96f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa894102e00 0x7fa89419ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:52.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.618+0000 7fa89b96f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa894108520 0x7fa894199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:52.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.618+0000 7fa89b96f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa894112650 con 0x7fa894102e00 2024-05-14T16:56:52.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.618+0000 7fa89b96f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa8941124d0 con 0x7fa894108520 2024-05-14T16:56:52.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.618+0000 7fa89b96f700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa8941127d0 con 0x7fa894102440 2024-05-14T16:56:52.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.618+0000 7fa898f0a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa894102e00 0x7fa89419ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:52.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.618+0000 7fa898f0a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa894102e00 0x7fa89419ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44926/0 (socket says 172.21.15.2:44926) 2024-05-14T16:56:52.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.618+0000 7fa898f0a700 1 -- 172.21.15.2:0/3956913028 learned_addr learned my addr 172.21.15.2:0/3956913028 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:56:52.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.618+0000 7fa899f0c700 1 --2- 172.21.15.2:0/3956913028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa894108520 0x7fa894199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:52.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.618+0000 7fa89970b700 1 --2- 172.21.15.2:0/3956913028 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa894102440 0x7fa89419fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:52.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.618+0000 7fa898f0a700 1 -- 172.21.15.2:0/3956913028 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa894102440 msgr2=0x7fa89419fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:52.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.618+0000 7fa898f0a700 1 --2- 172.21.15.2:0/3956913028 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa894102440 0x7fa89419fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:52.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.618+0000 7fa898f0a700 1 -- 172.21.15.2:0/3956913028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa894108520 msgr2=0x7fa894199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:52.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.618+0000 7fa898f0a700 1 --2- 172.21.15.2:0/3956913028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa894108520 0x7fa894199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:52.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.618+0000 7fa898f0a700 1 -- 172.21.15.2:0/3956913028 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa89419a400 con 0x7fa894102e00 2024-05-14T16:56:52.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.618+0000 7fa898f0a700 1 --2- 172.21.15.2:0/3956913028 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa894102e00 0x7fa89419ff90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa884000c00 tx=0x7fa8840047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:52.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.619+0000 7fa88a7fc700 1 -- 172.21.15.2:0/3956913028 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa884027070 con 0x7fa894102e00 2024-05-14T16:56:52.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.619+0000 7fa88a7fc700 1 -- 172.21.15.2:0/3956913028 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8840040e0 con 0x7fa894102e00 2024-05-14T16:56:52.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.619+0000 7fa89b96f700 1 -- 172.21.15.2:0/3956913028 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa89419a690 con 0x7fa894102e00 2024-05-14T16:56:52.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.619+0000 7fa88a7fc700 1 -- 172.21.15.2:0/3956913028 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88400f460 con 0x7fa894102e00 2024-05-14T16:56:52.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.619+0000 7fa89b96f700 1 -- 172.21.15.2:0/3956913028 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8941a6800 con 0x7fa894102e00 2024-05-14T16:56:52.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.621+0000 7fa88a7fc700 1 -- 172.21.15.2:0/3956913028 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fa88402a070 con 0x7fa894102e00 2024-05-14T16:56:52.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.622+0000 7fa88a7fc700 1 --2- 172.21.15.2:0/3956913028 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fa8800b29f0 0x7fa8800b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:52.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.622+0000 7fa88a7fc700 1 -- 172.21.15.2:0/3956913028 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fa8840e35d0 con 0x7fa894102e00 2024-05-14T16:56:52.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.622+0000 7fa89970b700 1 --2- 172.21.15.2:0/3956913028 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fa8800b29f0 0x7fa8800b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:52.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.623+0000 7fa8a6205b80 1 Processor -- start 2024-05-14T16:56:53.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.623+0000 7fa8a6205b80 1 -- start start 2024-05-14T16:56:53.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.623+0000 7fa89970b700 1 --2- 172.21.15.2:0/3956913028 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fa8800b29f0 0x7fa8800b4eb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa890005fd0 tx=0x7fa89000a560 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:53.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.623+0000 7fa8a6205b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558faf42420 0x5558faf42800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:53.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.623+0000 7fa8a6205b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558faf42d40 0x5558faf3dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:53.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.623+0000 7fa8a6205b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558faf3e2c0 0x5558faf3e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:53.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.623+0000 7fa8a6205b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5558fae9be70 con 0x5558faf42d40 2024-05-14T16:56:53.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.623+0000 7fa8a6205b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5558fae8ba10 con 0x5558faf3e2c0 2024-05-14T16:56:53.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.623+0000 7fa8a6205b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5558fae8fa50 con 0x5558faf42420 2024-05-14T16:56:53.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.623+0000 7fa898f0a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558faf42d40 0x5558faf3dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:53.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.623+0000 7fa899f0c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558faf3e2c0 0x5558faf3e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:53.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.623+0000 7fa898f0a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558faf42d40 0x5558faf3dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44942/0 (socket says 172.21.15.2:44942) 2024-05-14T16:56:53.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.623+0000 7fa898f0a700 1 -- 172.21.15.2:0/4280505313 learned_addr learned my addr 172.21.15.2:0/4280505313 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:56:53.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.623+0000 7fa89970b700 1 --2- 172.21.15.2:0/4280505313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558faf42420 0x5558faf42800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:53.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.624+0000 7fa898f0a700 1 -- 172.21.15.2:0/4280505313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558faf42420 msgr2=0x5558faf42800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:53.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.624+0000 7fa898f0a700 1 --2- 172.21.15.2:0/4280505313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558faf42420 0x5558faf42800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:53.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.624+0000 7fa898f0a700 1 -- 172.21.15.2:0/4280505313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558faf3e2c0 msgr2=0x5558faf3e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:53.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.624+0000 7fa898f0a700 1 --2- 172.21.15.2:0/4280505313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558faf3e2c0 0x5558faf3e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:53.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.624+0000 7fa898f0a700 1 -- 172.21.15.2:0/4280505313 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558fae350b0 con 0x5558faf42d40 2024-05-14T16:56:53.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.624+0000 7fa899f0c700 1 --2- 172.21.15.2:0/4280505313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558faf3e2c0 0x5558faf3e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:56:53.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.624+0000 7fa898f0a700 1 --2- 172.21.15.2:0/4280505313 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558faf42d40 0x5558faf3dca0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa8840aa670 tx=0x7fa8840aa3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b5b4d2500df8f97 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:53.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.624+0000 7fa87f7fe700 1 -- 172.21.15.2:0/4280505313 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8840aa480 con 0x5558faf42d40 2024-05-14T16:56:53.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.624+0000 7fa87f7fe700 1 -- 172.21.15.2:0/4280505313 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8840ab070 con 0x5558faf42d40 2024-05-14T16:56:53.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.624+0000 7fa87f7fe700 1 -- 172.21.15.2:0/4280505313 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8840b10d0 con 0x5558faf42d40 2024-05-14T16:56:53.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.625+0000 7fa8a6205b80 1 -- 172.21.15.2:0/4280505313 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558faf42d40 msgr2=0x5558faf3dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:53.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.625+0000 7fa8a6205b80 1 --2- 172.21.15.2:0/4280505313 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558faf42d40 0x5558faf3dca0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa8840aa670 tx=0x7fa8840aa3e0 comp rx=0 tx=0).stop 2024-05-14T16:56:53.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.625+0000 7fa8a6205b80 1 -- 172.21.15.2:0/4280505313 shutdown_connections 2024-05-14T16:56:53.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.625+0000 7fa8a6205b80 1 --2- 172.21.15.2:0/4280505313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558faf3e2c0 0x5558faf3e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:53.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.625+0000 7fa8a6205b80 1 --2- 172.21.15.2:0/4280505313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558faf42420 0x5558faf42800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:53.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.625+0000 7fa8a6205b80 1 --2- 172.21.15.2:0/4280505313 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558faf42d40 0x5558faf3dca0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:53.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.625+0000 7fa8a6205b80 1 -- 172.21.15.2:0/4280505313 >> 172.21.15.2:0/4280505313 conn(0x5558fae6aeb0 msgr2=0x5558faf384a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:53.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.625+0000 7fa8a6205b80 1 -- 172.21.15.2:0/4280505313 shutdown_connections 2024-05-14T16:56:53.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.625+0000 7fa8a6205b80 1 -- 172.21.15.2:0/4280505313 wait complete. 2024-05-14T16:56:53.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.625+0000 7fa8a6205b80 1 -- start start 2024-05-14T16:56:53.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.625+0000 7fa8a6205b80 1 Processor -- start 2024-05-14T16:56:53.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.626+0000 7fa8a6205b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558faf3e2c0 0x5558faf49b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:53.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.626+0000 7fa8a6205b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558faf42420 0x5558faf46ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:53.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.626+0000 7fa8a6205b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558faf42d40 0x5558faf47430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:53.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.626+0000 7fa8a6205b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5558fad920a0 con 0x5558faf3e2c0 2024-05-14T16:56:53.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.626+0000 7fa8a6205b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5558fadf3ea0 con 0x5558faf42d40 2024-05-14T16:56:53.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.626+0000 7fa8a6205b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5558faf4a640 con 0x5558faf42420 2024-05-14T16:56:53.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.626+0000 7fa8a6205b80 10 client.0 _ll_register_callbacks cb 0x5558fae26a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:56:53.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.626+0000 7fa89970b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558faf3e2c0 0x5558faf49b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:53.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.626+0000 7fa89970b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558faf3e2c0 0x5558faf49b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44946/0 (socket says 172.21.15.2:44946) 2024-05-14T16:56:53.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.626+0000 7fa89970b700 1 -- 172.21.15.2:0/4075235263 learned_addr learned my addr 172.21.15.2:0/4075235263 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:56:53.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.626+0000 7fa898f0a700 1 --2- 172.21.15.2:0/4075235263 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558faf42420 0x5558faf46ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:53.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.626+0000 7fa899f0c700 1 --2- 172.21.15.2:0/4075235263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558faf42d40 0x5558faf47430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:53.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.626+0000 7fa89970b700 1 -- 172.21.15.2:0/4075235263 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558faf42420 msgr2=0x5558faf46ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:53.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.626+0000 7fa89970b700 1 --2- 172.21.15.2:0/4075235263 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558faf42420 0x5558faf46ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:53.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.626+0000 7fa89970b700 1 -- 172.21.15.2:0/4075235263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558faf42d40 msgr2=0x5558faf47430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:53.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.626+0000 7fa89970b700 1 --2- 172.21.15.2:0/4075235263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558faf42d40 0x5558faf47430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:53.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.626+0000 7fa89970b700 1 -- 172.21.15.2:0/4075235263 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558faddb9d0 con 0x5558faf3e2c0 2024-05-14T16:56:53.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.626+0000 7fa899f0c700 1 --2- 172.21.15.2:0/4075235263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558faf42d40 0x5558faf47430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:56:53.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.626+0000 7fa89970b700 1 --2- 172.21.15.2:0/4075235263 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558faf3e2c0 0x5558faf49b50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa8900109e0 tx=0x7fa890004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc4b2583e1ee327d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:53.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.626+0000 7fa87cff9700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:56:53.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.627+0000 7fa87cff9700 1 -- 172.21.15.2:0/4075235263 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa890004a30 con 0x5558faf3e2c0 2024-05-14T16:56:53.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.627+0000 7fa87cff9700 1 -- 172.21.15.2:0/4075235263 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fa890004bd0 con 0x5558faf3e2c0 2024-05-14T16:56:53.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.627+0000 7fa8a6205b80 1 -- 172.21.15.2:0/4075235263 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8840ab070 con 0x5558faf3e2c0 2024-05-14T16:56:53.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.627+0000 7fa87cff9700 1 -- 172.21.15.2:0/4075235263 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa890004dd0 con 0x5558faf3e2c0 2024-05-14T16:56:53.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.627+0000 7fa8a6205b80 10 client.15564 fetch_fsmap learned FSMap version 30 2024-05-14T16:56:53.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.627+0000 7fa8a6205b80 1 -- 172.21.15.2:0/4075235263 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5558fae45860 con 0x5558faf3e2c0 2024-05-14T16:56:53.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.627+0000 7fa87cff9700 1 -- 172.21.15.2:0/4075235263 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8840ab070 con 0x5558faf3e2c0 2024-05-14T16:56:53.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.627+0000 7fa87cff9700 1 -- 172.21.15.2:0/4075235263 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fa8900053d0 con 0x5558faf3e2c0 2024-05-14T16:56:53.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.627+0000 7fa8a6205b80 10 client.15564 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T16:56:53.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.627+0000 7fa8a6205b80 10 client.15564 resolve_mds: validated gid 24479 aka daemon mds.b 2024-05-14T16:56:53.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.627+0000 7fa8a6205b80 20 client.15564 populate_metadata read hostname 'smithi002' 2024-05-14T16:56:53.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.627+0000 7fa8a6205b80 1 --2- 172.21.15.2:0/4075235263 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5558faf57240 0x5558faf59660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:53.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.627+0000 7fa8a6205b80 4 client.15564 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:56:53.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.627+0000 7fa8a6205b80 1 -- 172.21.15.2:0/4075235263 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558fae9ddc0 con 0x5558faf57240 2024-05-14T16:56:53.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.628+0000 7fa898f0a700 1 --2- 172.21.15.2:0/4075235263 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5558faf57240 0x5558faf59660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:53.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.628+0000 7fa898f0a700 1 --2- 172.21.15.2:0/4075235263 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5558faf57240 0x5558faf59660 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42717e6b488096ae server_cookie=d41fce0417a1033b in_seq=0 out_seq=0 2024-05-14T16:56:53.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.628+0000 7fa87cff9700 10 client.15564 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:56:53.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.629+0000 7fa87cff9700 1 -- 172.21.15.2:0/4075235263 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5558fae9ddc0 con 0x5558faf57240 2024-05-14T16:56:53.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.629+0000 7fa87cff9700 10 client.15564 handle_command_reply: tid=0 2024-05-14T16:56:53.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.629+0000 7fa8a6205b80 1 client.15564 shutdown 2024-05-14T16:56:53.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.629+0000 7fa8a6205b80 1 -- 172.21.15.2:0/4075235263 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558faf3e2c0 msgr2=0x5558faf49b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:53.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.629+0000 7fa8a6205b80 1 --2- 172.21.15.2:0/4075235263 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558faf3e2c0 0x5558faf49b50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa8900109e0 tx=0x7fa890004750 comp rx=0 tx=0).stop 2024-05-14T16:56:53.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.629+0000 7fa8a6205b80 1 -- 172.21.15.2:0/4075235263 shutdown_connections 2024-05-14T16:56:53.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.629+0000 7fa8a6205b80 1 --2- 172.21.15.2:0/4075235263 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5558faf57240 0x5558faf59660 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:53.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.629+0000 7fa8a6205b80 1 --2- 172.21.15.2:0/4075235263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558faf42d40 0x5558faf47430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:53.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.629+0000 7fa8a6205b80 1 --2- 172.21.15.2:0/4075235263 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558faf3e2c0 0x5558faf49b50 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:53.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.629+0000 7fa8a6205b80 1 --2- 172.21.15.2:0/4075235263 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558faf42420 0x5558faf46ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:53.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.629+0000 7fa8a6205b80 1 -- 172.21.15.2:0/4075235263 >> 172.21.15.2:0/4075235263 conn(0x5558fae6aeb0 msgr2=0x5558faf32fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:53.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.629+0000 7fa87cff9700 0 client.15564 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:56:53.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.629+0000 7fa8a6205b80 1 -- 172.21.15.2:0/4075235263 shutdown_connections 2024-05-14T16:56:53.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.629+0000 7fa8a6205b80 1 -- 172.21.15.2:0/4075235263 wait complete. 2024-05-14T16:56:53.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.629+0000 7fa8a6205b80 20 client.15564 trim_cache size 0 max 16384 2024-05-14T16:56:53.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.649+0000 7fa8a6205b80 1 Processor -- start 2024-05-14T16:56:53.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.649+0000 7fa8a6205b80 1 -- start start 2024-05-14T16:56:53.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.649+0000 7fa8a6205b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558faf3e2c0 0x5558faf330d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:53.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.649+0000 7fa8a6205b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558faf42420 0x5558faf4fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:53.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.649+0000 7fa8a6205b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558faf42d40 0x5558faf417f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:53.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.650+0000 7fa8a6205b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa8941b06f0 con 0x5558faf3e2c0 2024-05-14T16:56:53.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.650+0000 7fa8a6205b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa8941adef0 con 0x5558faf42420 2024-05-14T16:56:53.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.650+0000 7fa8a6205b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa8941abd60 con 0x5558faf42d40 2024-05-14T16:56:53.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.650+0000 7fa89970b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558faf3e2c0 0x5558faf330d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:53.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.650+0000 7fa89970b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558faf3e2c0 0x5558faf330d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44950/0 (socket says 172.21.15.2:44950) 2024-05-14T16:56:53.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.650+0000 7fa89970b700 1 -- 172.21.15.2:0/4026174057 learned_addr learned my addr 172.21.15.2:0/4026174057 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:56:53.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.650+0000 7fa898f0a700 1 --2- 172.21.15.2:0/4026174057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558faf42420 0x5558faf4fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:53.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.650+0000 7fa89970b700 1 -- 172.21.15.2:0/4026174057 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558faf42d40 msgr2=0x5558faf417f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:53.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.650+0000 7fa89970b700 1 --2- 172.21.15.2:0/4026174057 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558faf42d40 0x5558faf417f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:53.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.650+0000 7fa89970b700 1 -- 172.21.15.2:0/4026174057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558faf42420 msgr2=0x5558faf4fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:53.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.650+0000 7fa89970b700 1 --2- 172.21.15.2:0/4026174057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558faf42420 0x5558faf4fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:53.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.650+0000 7fa89970b700 1 -- 172.21.15.2:0/4026174057 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558faf54b90 con 0x5558faf3e2c0 2024-05-14T16:56:53.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.650+0000 7fa898f0a700 1 --2- 172.21.15.2:0/4026174057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558faf42420 0x5558faf4fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:56:53.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.650+0000 7fa89970b700 1 --2- 172.21.15.2:0/4026174057 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558faf3e2c0 0x5558faf330d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa890005950 tx=0x7fa89000e8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=315a7d17922fbe0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:53.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.650+0000 7fa87effd700 1 -- 172.21.15.2:0/4026174057 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa890033040 con 0x5558faf3e2c0 2024-05-14T16:56:53.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.650+0000 7fa87effd700 1 -- 172.21.15.2:0/4026174057 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa890005420 con 0x5558faf3e2c0 2024-05-14T16:56:53.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.650+0000 7fa87effd700 1 -- 172.21.15.2:0/4026174057 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89001ddb0 con 0x5558faf3e2c0 2024-05-14T16:56:53.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.651+0000 7fa8a6205b80 1 -- 172.21.15.2:0/4026174057 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558faf3e2c0 msgr2=0x5558faf330d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:53.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.651+0000 7fa8a6205b80 1 --2- 172.21.15.2:0/4026174057 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558faf3e2c0 0x5558faf330d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa890005950 tx=0x7fa89000e8f0 comp rx=0 tx=0).stop 2024-05-14T16:56:53.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.651+0000 7fa8a6205b80 1 -- 172.21.15.2:0/4026174057 shutdown_connections 2024-05-14T16:56:53.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.651+0000 7fa8a6205b80 1 --2- 172.21.15.2:0/4026174057 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558faf42d40 0x5558faf417f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:54.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.651+0000 7fa8a6205b80 1 --2- 172.21.15.2:0/4026174057 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558faf3e2c0 0x5558faf330d0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:54.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.651+0000 7fa8a6205b80 1 --2- 172.21.15.2:0/4026174057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558faf42420 0x5558faf4fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:54.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.651+0000 7fa8a6205b80 1 -- 172.21.15.2:0/4026174057 >> 172.21.15.2:0/4026174057 conn(0x5558fae6aeb0 msgr2=0x5558faf410e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:54.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.651+0000 7fa8a6205b80 1 -- 172.21.15.2:0/4026174057 shutdown_connections 2024-05-14T16:56:54.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.651+0000 7fa8a6205b80 1 -- 172.21.15.2:0/4026174057 wait complete. 2024-05-14T16:56:54.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.651+0000 7fa8a6205b80 1 -- start start 2024-05-14T16:56:54.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.651+0000 7fa8a6205b80 1 Processor -- start 2024-05-14T16:56:54.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.652+0000 7fa8a6205b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558faf3e2c0 0x5558fafbfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:54.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.652+0000 7fa8a6205b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558faf42420 0x5558fafc0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:54.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.652+0000 7fa8a6205b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558faf42d40 0x5558fafba140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:54.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.652+0000 7fa8a6205b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa8941ac3b0 con 0x5558faf42d40 2024-05-14T16:56:54.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.652+0000 7fa8a6205b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa8941ad820 con 0x5558faf42420 2024-05-14T16:56:54.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.652+0000 7fa8a6205b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa8941aca80 con 0x5558faf3e2c0 2024-05-14T16:56:54.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.652+0000 7fa8a6205b80 10 client.0 _ll_register_callbacks cb 0x5558fae26a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:56:54.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.652+0000 7fa899f0c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558faf42d40 0x5558fafba140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:54.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.652+0000 7fa899f0c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558faf42d40 0x5558fafba140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44962/0 (socket says 172.21.15.2:44962) 2024-05-14T16:56:54.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.652+0000 7fa899f0c700 1 -- 172.21.15.2:0/3286396856 learned_addr learned my addr 172.21.15.2:0/3286396856 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:56:54.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.652+0000 7fa89970b700 1 --2- 172.21.15.2:0/3286396856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558faf3e2c0 0x5558fafbfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:54.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.652+0000 7fa898f0a700 1 --2- 172.21.15.2:0/3286396856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558faf42420 0x5558fafc0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:54.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.652+0000 7fa899f0c700 1 -- 172.21.15.2:0/3286396856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558faf3e2c0 msgr2=0x5558fafbfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:54.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.652+0000 7fa899f0c700 1 --2- 172.21.15.2:0/3286396856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558faf3e2c0 0x5558fafbfd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:54.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.652+0000 7fa899f0c700 1 -- 172.21.15.2:0/3286396856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558faf42420 msgr2=0x5558fafc0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:54.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.652+0000 7fa899f0c700 1 --2- 172.21.15.2:0/3286396856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558faf42420 0x5558fafc0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:54.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.652+0000 7fa899f0c700 1 -- 172.21.15.2:0/3286396856 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558fafba840 con 0x5558faf42d40 2024-05-14T16:56:54.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.652+0000 7fa898f0a700 1 --2- 172.21.15.2:0/3286396856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558faf42420 0x5558fafc0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:56:54.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.653+0000 7fa899f0c700 1 --2- 172.21.15.2:0/3286396856 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558faf42d40 0x5558fafba140 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa88c004c20 tx=0x7fa88c013230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8c0175513afd21 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:54.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.653+0000 7fa87cff9700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:56:54.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.653+0000 7fa87cff9700 1 -- 172.21.15.2:0/3286396856 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88c013a90 con 0x5558faf42d40 2024-05-14T16:56:54.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.653+0000 7fa87cff9700 1 -- 172.21.15.2:0/3286396856 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fa88c013c30 con 0x5558faf42d40 2024-05-14T16:56:54.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.653+0000 7fa8a6205b80 1 -- 172.21.15.2:0/3286396856 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa890005420 con 0x5558faf42d40 2024-05-14T16:56:54.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.653+0000 7fa87cff9700 1 -- 172.21.15.2:0/3286396856 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa88c027050 con 0x5558faf42d40 2024-05-14T16:56:54.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.653+0000 7fa8a6205b80 10 client.15570 fetch_fsmap learned FSMap version 30 2024-05-14T16:56:54.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.653+0000 7fa8a6205b80 1 -- 172.21.15.2:0/3286396856 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5558fafc7560 con 0x5558faf42d40 2024-05-14T16:56:54.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.653+0000 7fa87cff9700 1 -- 172.21.15.2:0/3286396856 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa890005420 con 0x5558faf42d40 2024-05-14T16:56:54.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.653+0000 7fa87cff9700 1 -- 172.21.15.2:0/3286396856 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fa88c017570 con 0x5558faf42d40 2024-05-14T16:56:54.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.653+0000 7fa8a6205b80 10 client.15570 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T16:56:54.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.653+0000 7fa8a6205b80 10 client.15570 resolve_mds: validated gid 24479 aka daemon mds.b 2024-05-14T16:56:54.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.653+0000 7fa8a6205b80 20 client.15570 populate_metadata read hostname 'smithi002' 2024-05-14T16:56:54.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.653+0000 7fa8a6205b80 1 --2- 172.21.15.2:0/3286396856 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5558fafc7d50 0x5558fafca170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:54.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.654+0000 7fa8a6205b80 4 client.15570 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a7bd3c7e", "release": true, "await": true}] 2024-05-14T16:56:54.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.654+0000 7fa8a6205b80 1 -- 172.21.15.2:0/3286396856 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a7bd3c7e", "release": true, "await": true}) -- 0x5558fae975a0 con 0x5558fafc7d50 2024-05-14T16:56:54.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.654+0000 7fa89970b700 1 --2- 172.21.15.2:0/3286396856 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5558fafc7d50 0x5558fafca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:54.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.654+0000 7fa89970b700 1 --2- 172.21.15.2:0/3286396856 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5558fafc7d50 0x5558fafca170 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f54f615487ded92 server_cookie=761af0106ca397a9 in_seq=0 out_seq=0 2024-05-14T16:56:54.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.654+0000 7fa87cff9700 10 client.15570 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:56:54.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.668+0000 7fa87cff9700 1 -- 172.21.15.2:0/3286396856 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5558fae975a0 con 0x5558fafc7d50 2024-05-14T16:56:54.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.668+0000 7fa87cff9700 10 client.15570 handle_command_reply: tid=0 2024-05-14T16:56:54.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.668+0000 7fa8a6205b80 1 client.15570 shutdown 2024-05-14T16:56:54.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.668+0000 7fa8a6205b80 1 -- 172.21.15.2:0/3286396856 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558faf42d40 msgr2=0x5558fafba140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:54.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.668+0000 7fa8a6205b80 1 --2- 172.21.15.2:0/3286396856 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558faf42d40 0x5558fafba140 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa88c004c20 tx=0x7fa88c013230 comp rx=0 tx=0).stop 2024-05-14T16:56:54.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.668+0000 7fa8a6205b80 1 -- 172.21.15.2:0/3286396856 shutdown_connections 2024-05-14T16:56:54.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.668+0000 7fa8a6205b80 1 --2- 172.21.15.2:0/3286396856 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5558fafc7d50 0x5558fafca170 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:54.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.668+0000 7fa8a6205b80 1 --2- 172.21.15.2:0/3286396856 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5558faf42d40 0x5558fafba140 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:54.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.668+0000 7fa8a6205b80 1 --2- 172.21.15.2:0/3286396856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558faf3e2c0 0x5558fafbfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:54.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.668+0000 7fa8a6205b80 1 --2- 172.21.15.2:0/3286396856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558faf42420 0x5558fafc0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:54.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.668+0000 7fa8a6205b80 1 -- 172.21.15.2:0/3286396856 >> 172.21.15.2:0/3286396856 conn(0x5558fae6aeb0 msgr2=0x5558faf41970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:54.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.668+0000 7fa87cff9700 0 client.15570 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:56:54.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.669+0000 7fa8a6205b80 1 -- 172.21.15.2:0/3286396856 shutdown_connections 2024-05-14T16:56:54.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.669+0000 7fa8a6205b80 1 -- 172.21.15.2:0/3286396856 wait complete. 2024-05-14T16:56:54.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.669+0000 7fa8a6205b80 20 client.15570 trim_cache size 0 max 16384 2024-05-14T16:56:54.514 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T16:56:54.521 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 30, 2024-05-14T16:56:54.528 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24479, 2024-05-14T16:56:54.534 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 8, 2024-05-14T16:56:54.540 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-05-14T16:56:54.546 INFO:teuthology.orchestra.run.smithi002.stdout: "a7bd3c7e": { 2024-05-14T16:56:54.553 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 8, 2024-05-14T16:56:54.561 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-05-14T16:56:54.568 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-05-14T16:56:54.599 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-05-14T16:56:54.608 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-05-14T16:56:54.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T16:56:54.622 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-05-14T16:56:54.629 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 146.7, 2024-05-14T16:56:54.634 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-05-14T16:56:54.640 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-05-14T16:56:54.646 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-05-14T16:56:54.651 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-05-14T16:56:54.656 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-05-14T16:56:54.662 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-05-14T16:56:54.669 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:56:54.674 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:56:54.680 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:56:54.685 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:56:54.690 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:56:54.696 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T16:56:54.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.669+0000 7fa89b96f700 1 -- 172.21.15.2:0/3956913028 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fa8800b29f0 msgr2=0x7fa8800b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:54.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.669+0000 7fa89b96f700 1 --2- 172.21.15.2:0/3956913028 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fa8800b29f0 0x7fa8800b4eb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa890005fd0 tx=0x7fa89000a560 comp rx=0 tx=0).stop 2024-05-14T16:56:54.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.669+0000 7fa89b96f700 1 -- 172.21.15.2:0/3956913028 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa894102e00 msgr2=0x7fa89419ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:54.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.669+0000 7fa89b96f700 1 --2- 172.21.15.2:0/3956913028 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa894102e00 0x7fa89419ff90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa884000c00 tx=0x7fa8840047b0 comp rx=0 tx=0).stop 2024-05-14T16:56:54.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.669+0000 7fa89b96f700 1 -- 172.21.15.2:0/3956913028 shutdown_connections 2024-05-14T16:56:54.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.669+0000 7fa89b96f700 1 --2- 172.21.15.2:0/3956913028 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fa8800b29f0 0x7fa8800b4eb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:54.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.669+0000 7fa89b96f700 1 --2- 172.21.15.2:0/3956913028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa894108520 0x7fa894199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:54.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.669+0000 7fa89b96f700 1 --2- 172.21.15.2:0/3956913028 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa894102440 0x7fa89419fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:54.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.669+0000 7fa89b96f700 1 --2- 172.21.15.2:0/3956913028 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa894102e00 0x7fa89419ff90 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:54.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.669+0000 7fa89b96f700 1 -- 172.21.15.2:0/3956913028 >> 172.21.15.2:0/3956913028 conn(0x7fa8940793f0 msgr2=0x7fa89406b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:54.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.669+0000 7fa89b96f700 1 -- 172.21.15.2:0/3956913028 shutdown_connections 2024-05-14T16:56:54.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:52.669+0000 7fa89b96f700 1 -- 172.21.15.2:0/3956913028 wait complete. 2024-05-14T16:56:54.807 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a7bd3c7e', total seconds elapsed: 34.3 2024-05-14T16:56:54.812 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.7 2024-05-14T16:56:54.818 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:56:54.824 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:56:54 smithi002 ceph-mon[31538]: pgmap v69: 97 pgs: 97 active+clean; 1.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 680 op/s 2024-05-14T16:56:54.830 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:56:54 smithi123 ceph-mon[35971]: pgmap v69: 97 pgs: 97 active+clean; 1.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 680 op/s 2024-05-14T16:56:54.868 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:56:52 smithi151 ceph-mon[36066]: pgmap v68: 97 pgs: 97 active+clean; 1.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 586 op/s 2024-05-14T16:56:54.874 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:56:54 smithi151 ceph-mon[36066]: pgmap v69: 97 pgs: 97 active+clean; 1.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 MiB/s wr, 680 op/s 2024-05-14T16:56:55.275 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:56:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.113+0000 7f6dc8885640 1 -- 172.21.15.2:0/1221333999 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6dc410e3a0 msgr2=0x7f6dc4115f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.113+0000 7f6dc8885640 1 --2- 172.21.15.2:0/1221333999 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6dc410e3a0 0x7f6dc4115f70 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6db4009900 tx=0x7f6db402a410 comp rx=0 tx=0).stop 2024-05-14T16:56:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.113+0000 7f6dc8885640 1 -- 172.21.15.2:0/1221333999 shutdown_connections 2024-05-14T16:56:56.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.113+0000 7f6dc8885640 1 --2- 172.21.15.2:0/1221333999 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6dc410e3a0 0x7f6dc4115f70 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:56.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.113+0000 7f6dc8885640 1 --2- 172.21.15.2:0/1221333999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dc4109c80 0x7f6dc410dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.113+0000 7f6dc8885640 1 --2- 172.21.15.2:0/1221333999 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6dc41092d0 0x7f6dc41096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.113+0000 7f6dc8885640 1 -- 172.21.15.2:0/1221333999 >> 172.21.15.2:0/1221333999 conn(0x7f6dc4077000 msgr2=0x7f6dc4079420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:56.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:56:56 smithi123 ceph-mon[35971]: pgmap v70: 97 pgs: 97 active+clean; 1.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 675 op/s 2024-05-14T16:56:56.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.116+0000 7f6dc8885640 1 -- 172.21.15.2:0/1221333999 shutdown_connections 2024-05-14T16:56:56.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.116+0000 7f6dc8885640 1 -- 172.21.15.2:0/1221333999 wait complete. 2024-05-14T16:56:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.118+0000 7f6dc8885640 1 Processor -- start 2024-05-14T16:56:56.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.119+0000 7f6dc8885640 1 -- start start 2024-05-14T16:56:56.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.119+0000 7f6dc8885640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6dc41092d0 0x7f6dc41a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:56.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.119+0000 7f6dc8885640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dc410e3a0 0x7f6dc41a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.119+0000 7f6dc8885640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6dc419f670 0x7f6dc419fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:56.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.120+0000 7f6dc8885640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc41193b0 con 0x7f6dc41092d0 2024-05-14T16:56:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.120+0000 7f6dc8885640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc4119230 con 0x7f6dc410e3a0 2024-05-14T16:56:56.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.120+0000 7f6dc8885640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc4119530 con 0x7f6dc419f670 2024-05-14T16:56:56.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.120+0000 7f6dc2d9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6dc419f670 0x7f6dc419fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.120+0000 7f6dc2d9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6dc419f670 0x7f6dc419fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:46170/0 (socket says 172.21.15.2:46170) 2024-05-14T16:56:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.120+0000 7f6dc2d9d640 1 -- 172.21.15.2:0/619573635 learned_addr learned my addr 172.21.15.2:0/619573635 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:56:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.121+0000 7f6dc1d9b640 1 --2- 172.21.15.2:0/619573635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dc410e3a0 0x7f6dc41a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.121+0000 7f6dc259c640 1 --2- 172.21.15.2:0/619573635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6dc41092d0 0x7f6dc41a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:57.014 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.121+0000 7f6dc1d9b640 1 -- 172.21.15.2:0/619573635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6dc419f670 msgr2=0x7f6dc419fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.121+0000 7f6dc1d9b640 1 --2- 172.21.15.2:0/619573635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6dc419f670 0x7f6dc419fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.121+0000 7f6dc1d9b640 1 -- 172.21.15.2:0/619573635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6dc41092d0 msgr2=0x7f6dc41a5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.121+0000 7f6dc1d9b640 1 --2- 172.21.15.2:0/619573635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6dc41092d0 0x7f6dc41a5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.121+0000 7f6dc1d9b640 1 -- 172.21.15.2:0/619573635 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc41a0190 con 0x7f6dc410e3a0 2024-05-14T16:56:57.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.122+0000 7f6dc1d9b640 1 --2- 172.21.15.2:0/619573635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dc410e3a0 0x7f6dc41a59b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6db8009fd0 tx=0x7f6db800b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:57.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.122+0000 7f6db37fe640 1 -- 172.21.15.2:0/619573635 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db8015020 con 0x7f6dc410e3a0 2024-05-14T16:56:57.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.122+0000 7f6db37fe640 1 -- 172.21.15.2:0/619573635 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6db8011ce0 con 0x7f6dc410e3a0 2024-05-14T16:56:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.122+0000 7f6dc8885640 1 -- 172.21.15.2:0/619573635 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dc41a0480 con 0x7f6dc410e3a0 2024-05-14T16:56:57.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.122+0000 7f6db37fe640 1 -- 172.21.15.2:0/619573635 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db8015020 con 0x7f6dc410e3a0 2024-05-14T16:56:57.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.123+0000 7f6dc8885640 1 -- 172.21.15.2:0/619573635 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dc41ac630 con 0x7f6dc410e3a0 2024-05-14T16:56:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.126+0000 7f6dcb5a0740 1 Processor -- start 2024-05-14T16:56:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.127+0000 7f6dcb5a0740 1 -- start start 2024-05-14T16:56:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.127+0000 7f6dcb5a0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596632c88f0 0x5596632c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.127+0000 7f6dcb5a0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596632c9210 0x5596632d3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.127+0000 7f6dcb5a0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5596632d3ad0 0x5596632d5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:57.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.127+0000 7f6dcb5a0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55966323bc30 con 0x5596632d3ad0 2024-05-14T16:56:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.127+0000 7f6dcb5a0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55966322b7d0 con 0x5596632c9210 2024-05-14T16:56:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.127+0000 7f6dcb5a0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55966322f810 con 0x5596632c88f0 2024-05-14T16:56:57.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.127+0000 7f6dc259c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596632c9210 0x5596632d3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.127+0000 7f6dc259c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596632c9210 0x5596632d3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:36412/0 (socket says 172.21.15.2:36412) 2024-05-14T16:56:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.127+0000 7f6dc259c640 1 -- 172.21.15.2:0/1299320526 learned_addr learned my addr 172.21.15.2:0/1299320526 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:56:57.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.128+0000 7f6dc259c640 1 -- 172.21.15.2:0/1299320526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596632c88f0 msgr2=0x5596632c8cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T16:56:57.236 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.128+0000 7f6dc259c640 1 --2- 172.21.15.2:0/1299320526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596632c88f0 0x5596632c8cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:57.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.128+0000 7f6dc259c640 1 -- 172.21.15.2:0/1299320526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5596632d3ad0 msgr2=0x5596632d5ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T16:56:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.128+0000 7f6dc259c640 1 --2- 172.21.15.2:0/1299320526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5596632d3ad0 0x5596632d5ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.128+0000 7f6dc259c640 1 -- 172.21.15.2:0/1299320526 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559662f40210 con 0x5596632c9210 2024-05-14T16:56:57.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.128+0000 7f6dc259c640 1 --2- 172.21.15.2:0/1299320526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596632c9210 0x5596632d3590 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc00ae90 tx=0x7f6dbc00f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea1d203113e82371 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.128+0000 7f6db0ff9640 1 -- 172.21.15.2:0/1299320526 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc011850 con 0x5596632c9210 2024-05-14T16:56:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.128+0000 7f6db0ff9640 1 -- 172.21.15.2:0/1299320526 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dbc01a480 con 0x5596632c9210 2024-05-14T16:56:57.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.129+0000 7f6db37fe640 1 -- 172.21.15.2:0/619573635 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f6db8018020 con 0x7f6dc410e3a0 2024-05-14T16:56:57.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.129+0000 7f6db0ff9640 1 -- 172.21.15.2:0/1299320526 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc010630 con 0x5596632c9210 2024-05-14T16:56:57.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.129+0000 7f6dcb5a0740 1 -- 172.21.15.2:0/1299320526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596632c9210 msgr2=0x5596632d3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:57.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.129+0000 7f6dcb5a0740 1 --2- 172.21.15.2:0/1299320526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596632c9210 0x5596632d3590 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc00ae90 tx=0x7f6dbc00f730 comp rx=0 tx=0).stop 2024-05-14T16:56:57.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.130+0000 7f6db37fe640 1 --2- 172.21.15.2:0/619573635 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f6dac0b2c00 0x7f6dac0b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:57.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.130+0000 7f6db37fe640 1 -- 172.21.15.2:0/619573635 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f6db80e1960 con 0x7f6dc410e3a0 2024-05-14T16:56:57.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.130+0000 7f6dcb5a0740 1 -- 172.21.15.2:0/1299320526 shutdown_connections 2024-05-14T16:56:57.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.130+0000 7f6dcb5a0740 1 --2- 172.21.15.2:0/1299320526 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5596632d3ad0 0x5596632d5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:57.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.130+0000 7f6dcb5a0740 1 --2- 172.21.15.2:0/1299320526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596632c9210 0x5596632d3590 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:57.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.130+0000 7f6dcb5a0740 1 --2- 172.21.15.2:0/1299320526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596632c88f0 0x5596632c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:57.403 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.131+0000 7f6dcb5a0740 1 -- 172.21.15.2:0/1299320526 >> 172.21.15.2:0/1299320526 conn(0x5596632c4e00 msgr2=0x5596632c51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.131+0000 7f6dc2d9d640 1 --2- 172.21.15.2:0/619573635 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f6dac0b2c00 0x7f6dac0b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:57.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.131+0000 7f6dcb5a0740 1 -- 172.21.15.2:0/1299320526 shutdown_connections 2024-05-14T16:56:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.132+0000 7f6dcb5a0740 1 -- 172.21.15.2:0/1299320526 wait complete. 2024-05-14T16:56:57.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.132+0000 7f6dcb5a0740 1 -- start start 2024-05-14T16:56:57.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.132+0000 7f6dcb5a0740 1 Processor -- start 2024-05-14T16:56:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.133+0000 7f6dc2d9d640 1 --2- 172.21.15.2:0/619573635 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f6dac0b2c00 0x7f6dac0b50c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6db40096c0 tx=0x7f6db4005a40 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.133+0000 7f6dcb5a0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596632c88f0 0x5596632d34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:57.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.133+0000 7f6dcb5a0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596632c9210 0x5596632d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:57.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.133+0000 7f6dcb5a0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5596632d3ad0 0x5596632d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:57.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.133+0000 7f6dcb5a0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x559662f11790 con 0x5596632d3ad0 2024-05-14T16:56:57.549 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.133+0000 7f6dcb5a0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5596631e7890 con 0x5596632c88f0 2024-05-14T16:56:57.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.133+0000 7f6dcb5a0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5596632d67f0 con 0x5596632c9210 2024-05-14T16:56:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.133+0000 7f6dcb5a0740 10 client.0 _ll_register_callbacks cb 0x5596631ead60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:56:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.133+0000 7f6dc259c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596632c88f0 0x5596632d34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:57.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.133+0000 7f6dc259c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596632c88f0 0x5596632d34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:36428/0 (socket says 172.21.15.2:36428) 2024-05-14T16:56:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.133+0000 7f6dc259c640 1 -- 172.21.15.2:0/3191397800 learned_addr learned my addr 172.21.15.2:0/3191397800 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:56:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.133+0000 7f6dc1d9b640 1 --2- 172.21.15.2:0/3191397800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596632c9210 0x5596632d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.134+0000 7f6dc259c640 1 -- 172.21.15.2:0/3191397800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596632c9210 msgr2=0x5596632d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.134+0000 7f6dc259c640 1 --2- 172.21.15.2:0/3191397800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596632c9210 0x5596632d07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.134+0000 7f6dc259c640 1 -- 172.21.15.2:0/3191397800 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5596632d3ad0 msgr2=0x5596632d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.134+0000 7f6dc259c640 1 --2- 172.21.15.2:0/3191397800 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5596632d3ad0 0x5596632d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.134+0000 7f6dc259c640 1 -- 172.21.15.2:0/3191397800 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55966300c620 con 0x5596632c88f0 2024-05-14T16:56:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.134+0000 7f6dc259c640 1 --2- 172.21.15.2:0/3191397800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596632c88f0 0x5596632d34f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc01a9d0 tx=0x7f6dbc01aa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5eb97c224c24f13c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:57.727 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.135+0000 7f6da9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T16:56:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.135+0000 7f6da9ffb640 1 -- 172.21.15.2:0/3191397800 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc01ae60 con 0x5596632c88f0 2024-05-14T16:56:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.135+0000 7f6da9ffb640 1 -- 172.21.15.2:0/3191397800 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f6dbc004b20 con 0x5596632c88f0 2024-05-14T16:56:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.135+0000 7f6dcb5a0740 1 -- 172.21.15.2:0/3191397800 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dbc01a480 con 0x5596632c88f0 2024-05-14T16:56:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.135+0000 7f6da9ffb640 1 -- 172.21.15.2:0/3191397800 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6dbc004d60 con 0x5596632c88f0 2024-05-14T16:56:57.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.135+0000 7f6dcb5a0740 10 client.25292 fetch_fsmap learned FSMap version 30 2024-05-14T16:56:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.135+0000 7f6dcb5a0740 1 -- 172.21.15.2:0/3191397800 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5596632e2100 con 0x5596632c88f0 2024-05-14T16:56:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.135+0000 7f6da9ffb640 1 -- 172.21.15.2:0/3191397800 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dbc01a480 con 0x5596632c88f0 2024-05-14T16:56:57.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.136+0000 7f6da9ffb640 1 -- 172.21.15.2:0/3191397800 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f6dbc0054b0 con 0x5596632c88f0 2024-05-14T16:56:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.136+0000 7f6dcb5a0740 10 client.25292 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T16:56:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.136+0000 7f6dcb5a0740 10 client.25292 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:56:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.136+0000 7f6dcb5a0740 20 client.25292 populate_metadata read hostname 'smithi002' 2024-05-14T16:56:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.136+0000 7f6dcb5a0740 1 --2- 172.21.15.2:0/3191397800 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5596632e3880 0x5596632e5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.136+0000 7f6dcb5a0740 4 client.25292 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:56:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.136+0000 7f6dcb5a0740 1 -- 172.21.15.2:0/3191397800 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55966323db80 con 0x5596632e3880 2024-05-14T16:56:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.136+0000 7f6dc1d9b640 1 --2- 172.21.15.2:0/3191397800 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5596632e3880 0x5596632e5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.136+0000 7f6dc1d9b640 1 --2- 172.21.15.2:0/3191397800 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5596632e3880 0x5596632e5ce0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3255a0b5ddfe4d8a server_cookie=b9daadcd202931bc in_seq=0 out_seq=0 2024-05-14T16:56:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.136+0000 7f6da9ffb640 10 client.25292 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:56:57.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.137+0000 7f6da9ffb640 1 -- 172.21.15.2:0/3191397800 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55966323db80 con 0x5596632e3880 2024-05-14T16:56:57.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.137+0000 7f6da9ffb640 10 client.25292 handle_command_reply: tid=0 2024-05-14T16:56:57.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.137+0000 7f6dcb5a0740 1 client.25292 shutdown 2024-05-14T16:56:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.137+0000 7f6dcb5a0740 1 -- 172.21.15.2:0/3191397800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596632c88f0 msgr2=0x5596632d34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.137+0000 7f6dcb5a0740 1 --2- 172.21.15.2:0/3191397800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596632c88f0 0x5596632d34f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc01a9d0 tx=0x7f6dbc01aa00 comp rx=0 tx=0).stop 2024-05-14T16:56:57.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.137+0000 7f6dcb5a0740 1 -- 172.21.15.2:0/3191397800 shutdown_connections 2024-05-14T16:56:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.137+0000 7f6dcb5a0740 1 --2- 172.21.15.2:0/3191397800 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5596632e3880 0x5596632e5ce0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.137+0000 7f6dcb5a0740 1 --2- 172.21.15.2:0/3191397800 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5596632d3ad0 0x5596632d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.137+0000 7f6dcb5a0740 1 --2- 172.21.15.2:0/3191397800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596632c9210 0x5596632d07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:58.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.137+0000 7f6dcb5a0740 1 --2- 172.21.15.2:0/3191397800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596632c88f0 0x5596632d34f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.137+0000 7f6dcb5a0740 1 -- 172.21.15.2:0/3191397800 >> 172.21.15.2:0/3191397800 conn(0x5596632c4e00 msgr2=0x5596632c4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.137+0000 7f6da9ffb640 0 client.25292 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:56:58.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.137+0000 7f6dcb5a0740 1 -- 172.21.15.2:0/3191397800 shutdown_connections 2024-05-14T16:56:58.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.139+0000 7f6dcb5a0740 1 -- 172.21.15.2:0/3191397800 wait complete. 2024-05-14T16:56:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.139+0000 7f6dcb5a0740 20 client.25292 trim_cache size 0 max 16384 2024-05-14T16:56:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.162+0000 7f6dcb5a0740 1 Processor -- start 2024-05-14T16:56:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.162+0000 7f6dcb5a0740 1 -- start start 2024-05-14T16:56:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.162+0000 7f6dcb5a0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5596632c88f0 0x5596632df6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:58.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.162+0000 7f6dcb5a0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596632c9210 0x5596632eb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:58.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.162+0000 7f6dcb5a0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596632d3ad0 0x5596632ebb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.162+0000 7f6dcb5a0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc41b7670 con 0x5596632c88f0 2024-05-14T16:56:58.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.162+0000 7f6dcb5a0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc41b4eb0 con 0x5596632c9210 2024-05-14T16:56:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.162+0000 7f6dcb5a0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc41b2d00 con 0x5596632d3ad0 2024-05-14T16:56:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.162+0000 7f6dc2d9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596632d3ad0 0x5596632ebb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.162+0000 7f6dc2d9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596632d3ad0 0x5596632ebb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:46190/0 (socket says 172.21.15.2:46190) 2024-05-14T16:56:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.162+0000 7f6dc2d9d640 1 -- 172.21.15.2:0/3457151609 learned_addr learned my addr 172.21.15.2:0/3457151609 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:56:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.163+0000 7f6dc1d9b640 1 --2- 172.21.15.2:0/3457151609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596632c9210 0x5596632eb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.163+0000 7f6dc2d9d640 1 -- 172.21.15.2:0/3457151609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596632c9210 msgr2=0x5596632eb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.163+0000 7f6dc2d9d640 1 --2- 172.21.15.2:0/3457151609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596632c9210 0x5596632eb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.163+0000 7f6dc2d9d640 1 -- 172.21.15.2:0/3457151609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5596632c88f0 msgr2=0x5596632df6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.163+0000 7f6dc2d9d640 1 --2- 172.21.15.2:0/3457151609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5596632c88f0 0x5596632df6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.163+0000 7f6dc2d9d640 1 -- 172.21.15.2:0/3457151609 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596632ced30 con 0x5596632d3ad0 2024-05-14T16:56:58.334 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.163+0000 7f6dc2d9d640 1 --2- 172.21.15.2:0/3457151609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596632d3ad0 0x5596632ebb30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6db4004610 tx=0x7f6db402a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cba59e129dcbece server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.164+0000 7f6dabfff640 1 -- 172.21.15.2:0/3457151609 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db404d020 con 0x5596632d3ad0 2024-05-14T16:56:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.164+0000 7f6dabfff640 1 -- 172.21.15.2:0/3457151609 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6db40314a0 con 0x5596632d3ad0 2024-05-14T16:56:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.164+0000 7f6dabfff640 1 -- 172.21.15.2:0/3457151609 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db4035040 con 0x5596632d3ad0 2024-05-14T16:56:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.165+0000 7f6dcb5a0740 1 -- 172.21.15.2:0/3457151609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596632d3ad0 msgr2=0x5596632ebb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.165+0000 7f6dcb5a0740 1 --2- 172.21.15.2:0/3457151609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596632d3ad0 0x5596632ebb30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6db4004610 tx=0x7f6db402a410 comp rx=0 tx=0).stop 2024-05-14T16:56:58.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.165+0000 7f6dcb5a0740 1 -- 172.21.15.2:0/3457151609 shutdown_connections 2024-05-14T16:56:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.165+0000 7f6dcb5a0740 1 --2- 172.21.15.2:0/3457151609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596632d3ad0 0x5596632ebb30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.165+0000 7f6dcb5a0740 1 --2- 172.21.15.2:0/3457151609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596632c9210 0x5596632eb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:58.396 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.165+0000 7f6dcb5a0740 1 --2- 172.21.15.2:0/3457151609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5596632c88f0 0x5596632df6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:58.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.165+0000 7f6dcb5a0740 1 -- 172.21.15.2:0/3457151609 >> 172.21.15.2:0/3457151609 conn(0x5596632c4e00 msgr2=0x5596632c4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:58.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.165+0000 7f6dcb5a0740 1 -- 172.21.15.2:0/3457151609 shutdown_connections 2024-05-14T16:56:58.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.166+0000 7f6dcb5a0740 1 -- 172.21.15.2:0/3457151609 wait complete. 2024-05-14T16:56:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.166+0000 7f6dcb5a0740 1 -- start start 2024-05-14T16:56:58.488 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.166+0000 7f6dcb5a0740 1 Processor -- start 2024-05-14T16:56:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.167+0000 7f6dcb5a0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5596632c88f0 0x5596632dbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:58.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.167+0000 7f6dcb5a0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596632c9210 0x5596632d91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.167+0000 7f6dcb5a0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596632d3ad0 0x5596632d96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.167+0000 7f6dcb5a0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc41b3370 con 0x5596632c88f0 2024-05-14T16:56:58.549 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.167+0000 7f6dcb5a0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc41b47e0 con 0x5596632c9210 2024-05-14T16:56:58.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.167+0000 7f6dcb5a0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc41b3a40 con 0x5596632d3ad0 2024-05-14T16:56:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.167+0000 7f6dcb5a0740 10 client.0 _ll_register_callbacks cb 0x7f6dbc01a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:56:58.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.167+0000 7f6dc1d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596632c9210 0x5596632d91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:58.578 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.167+0000 7f6dc1d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596632c9210 0x5596632d91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:36446/0 (socket says 172.21.15.2:36446) 2024-05-14T16:56:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.167+0000 7f6dc1d9b640 1 -- 172.21.15.2:0/276395260 learned_addr learned my addr 172.21.15.2:0/276395260 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:56:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.167+0000 7f6dc2d9d640 1 --2- 172.21.15.2:0/276395260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596632d3ad0 0x5596632d96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.168+0000 7f6dc259c640 1 --2- 172.21.15.2:0/276395260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5596632c88f0 0x5596632dbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.168+0000 7f6dc1d9b640 1 -- 172.21.15.2:0/276395260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596632d3ad0 msgr2=0x5596632d96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.168+0000 7f6dc1d9b640 1 --2- 172.21.15.2:0/276395260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596632d3ad0 0x5596632d96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.168+0000 7f6dc1d9b640 1 -- 172.21.15.2:0/276395260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5596632c88f0 msgr2=0x5596632dbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:56:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.168+0000 7f6dc1d9b640 1 --2- 172.21.15.2:0/276395260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5596632c88f0 0x5596632dbd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.168+0000 7f6dc1d9b640 1 -- 172.21.15.2:0/276395260 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596632d9ca0 con 0x5596632c9210 2024-05-14T16:56:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.168+0000 7f6dc259c640 1 --2- 172.21.15.2:0/276395260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5596632c88f0 0x5596632dbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:56:58.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.168+0000 7f6dc1d9b640 1 --2- 172.21.15.2:0/276395260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596632c9210 0x5596632d91a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6db80a5ff0 tx=0x7f6db8012f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e44bbee04095890f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.168+0000 7f6da9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T16:56:58.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.168+0000 7f6da9ffb640 1 -- 172.21.15.2:0/276395260 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db806c070 con 0x5596632c9210 2024-05-14T16:56:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.169+0000 7f6da9ffb640 1 -- 172.21.15.2:0/276395260 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f6db80a5370 con 0x5596632c9210 2024-05-14T16:56:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.169+0000 7f6dcb5a0740 1 -- 172.21.15.2:0/276395260 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6db40314a0 con 0x5596632c9210 2024-05-14T16:56:58.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.169+0000 7f6da9ffb640 1 -- 172.21.15.2:0/276395260 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6db80b1be0 con 0x5596632c9210 2024-05-14T16:56:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.169+0000 7f6dcb5a0740 10 client.25298 fetch_fsmap learned FSMap version 30 2024-05-14T16:56:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.169+0000 7f6dcb5a0740 1 -- 172.21.15.2:0/276395260 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55966335f000 con 0x5596632c9210 2024-05-14T16:56:58.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.169+0000 7f6da9ffb640 1 -- 172.21.15.2:0/276395260 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6db40314a0 con 0x5596632c9210 2024-05-14T16:56:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.170+0000 7f6da9ffb640 1 -- 172.21.15.2:0/276395260 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f6db80c3050 con 0x5596632c9210 2024-05-14T16:56:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.170+0000 7f6dcb5a0740 10 client.25298 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T16:56:58.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.170+0000 7f6dcb5a0740 10 client.25298 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:56:58.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.170+0000 7f6dcb5a0740 20 client.25298 populate_metadata read hostname 'smithi002' 2024-05-14T16:56:58.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.170+0000 7f6dcb5a0740 1 --2- 172.21.15.2:0/276395260 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55966335f740 0x559663361b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:58.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.170+0000 7f6dcb5a0740 4 client.25298 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:56:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.170+0000 7f6dcb5a0740 1 -- 172.21.15.2:0/276395260 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559663237360 con 0x55966335f740 2024-05-14T16:56:58.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.171+0000 7f6dc259c640 1 --2- 172.21.15.2:0/276395260 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55966335f740 0x559663361b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.172+0000 7f6dc259c640 1 --2- 172.21.15.2:0/276395260 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55966335f740 0x559663361b60 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=604d4b22e3cc0a7f server_cookie=c16ae35e9b426bb4 in_seq=0 out_seq=0 2024-05-14T16:56:58.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.172+0000 7f6da9ffb640 10 client.25298 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:56:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.184+0000 7f6da9ffb640 1 -- 172.21.15.2:0/276395260 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559663237360 con 0x55966335f740 2024-05-14T16:56:58.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.184+0000 7f6da9ffb640 10 client.25298 handle_command_reply: tid=0 2024-05-14T16:56:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.184+0000 7f6dcb5a0740 1 client.25298 shutdown 2024-05-14T16:56:58.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.184+0000 7f6dcb5a0740 1 -- 172.21.15.2:0/276395260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596632c9210 msgr2=0x5596632d91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.184+0000 7f6dcb5a0740 1 --2- 172.21.15.2:0/276395260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596632c9210 0x5596632d91a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6db80a5ff0 tx=0x7f6db8012f20 comp rx=0 tx=0).stop 2024-05-14T16:56:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.184+0000 7f6dcb5a0740 1 -- 172.21.15.2:0/276395260 shutdown_connections 2024-05-14T16:56:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.184+0000 7f6dcb5a0740 1 --2- 172.21.15.2:0/276395260 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55966335f740 0x559663361b60 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.184+0000 7f6dcb5a0740 1 --2- 172.21.15.2:0/276395260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5596632d3ad0 0x5596632d96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:59.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.184+0000 7f6dcb5a0740 1 --2- 172.21.15.2:0/276395260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596632c9210 0x5596632d91a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:59.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.184+0000 7f6dcb5a0740 1 --2- 172.21.15.2:0/276395260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5596632c88f0 0x5596632dbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:59.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.184+0000 7f6dcb5a0740 1 -- 172.21.15.2:0/276395260 >> 172.21.15.2:0/276395260 conn(0x5596632c4e00 msgr2=0x5596632d2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:59.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.184+0000 7f6da9ffb640 0 client.25298 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:56:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.184+0000 7f6dcb5a0740 1 -- 172.21.15.2:0/276395260 shutdown_connections 2024-05-14T16:56:59.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.184+0000 7f6dcb5a0740 1 -- 172.21.15.2:0/276395260 wait complete. 2024-05-14T16:56:59.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.185+0000 7f6dcb5a0740 20 client.25298 trim_cache size 0 max 16384 2024-05-14T16:56:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T16:56:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T16:56:59.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T16:56:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.186+0000 7f6dc8885640 1 -- 172.21.15.2:0/619573635 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f6dac0b2c00 msgr2=0x7f6dac0b50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.186+0000 7f6dc8885640 1 --2- 172.21.15.2:0/619573635 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f6dac0b2c00 0x7f6dac0b50c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6db40096c0 tx=0x7f6db4005a40 comp rx=0 tx=0).stop 2024-05-14T16:56:59.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.186+0000 7f6dc8885640 1 -- 172.21.15.2:0/619573635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dc410e3a0 msgr2=0x7f6dc41a59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.186+0000 7f6dc8885640 1 --2- 172.21.15.2:0/619573635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dc410e3a0 0x7f6dc41a59b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6db8009fd0 tx=0x7f6db800b870 comp rx=0 tx=0).stop 2024-05-14T16:56:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.186+0000 7f6dc8885640 1 -- 172.21.15.2:0/619573635 shutdown_connections 2024-05-14T16:56:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.186+0000 7f6dc8885640 1 --2- 172.21.15.2:0/619573635 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f6dac0b2c00 0x7f6dac0b50c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.186+0000 7f6dc8885640 1 --2- 172.21.15.2:0/619573635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6dc419f670 0x7f6dc419fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:59.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.186+0000 7f6dc8885640 1 --2- 172.21.15.2:0/619573635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dc410e3a0 0x7f6dc41a59b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.186+0000 7f6dc8885640 1 --2- 172.21.15.2:0/619573635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6dc41092d0 0x7f6dc41a5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.186+0000 7f6dc8885640 1 -- 172.21.15.2:0/619573635 >> 172.21.15.2:0/619573635 conn(0x7f6dc4077000 msgr2=0x7f6dc4079350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.186+0000 7f6dc8885640 1 -- 172.21.15.2:0/619573635 shutdown_connections 2024-05-14T16:56:59.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:56:56.186+0000 7f6dc8885640 1 -- 172.21.15.2:0/619573635 wait complete. 2024-05-14T16:56:59.307 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:56:56 smithi002 ceph-mon[31538]: pgmap v70: 97 pgs: 97 active+clean; 1.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 675 op/s 2024-05-14T16:56:59.315 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:56:58 smithi123 ceph-mon[35971]: pgmap v71: 97 pgs: 97 active+clean; 1.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 801 op/s 2024-05-14T16:56:59.323 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:56:58 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/958016074' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:56:59.344 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 100.0 s, 2909.6 tps, lat 204.646 ms stddev 2125.981 2024-05-14T16:56:59.351 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:56:56 smithi151 ceph-mon[36066]: pgmap v70: 97 pgs: 97 active+clean; 1.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 16 MiB/s wr, 675 op/s 2024-05-14T16:56:59.358 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:56:58 smithi151 ceph-mon[36066]: pgmap v71: 97 pgs: 97 active+clean; 1.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 801 op/s 2024-05-14T16:56:59.368 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:56:58 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/958016074' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:56:59.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.461+0000 7f64155ee700 1 -- 172.21.15.2:0/3266757352 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f641010a540 msgr2=0x7f641010a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:59.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.461+0000 7f64155ee700 1 --2- 172.21.15.2:0/3266757352 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f641010a540 0x7f641010a920 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f64000098f0 tx=0x7f6400009c00 comp rx=0 tx=0).stop 2024-05-14T16:56:59.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.461+0000 7f64155ee700 1 -- 172.21.15.2:0/3266757352 shutdown_connections 2024-05-14T16:56:59.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.461+0000 7f64155ee700 1 --2- 172.21.15.2:0/3266757352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6410116690 0x7f6410118a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:59.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.461+0000 7f64155ee700 1 --2- 172.21.15.2:0/3266757352 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f641010a540 0x7f641010a920 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:59.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.461+0000 7f64155ee700 1 --2- 172.21.15.2:0/3266757352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f641010ae60 0x7f6410116150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:59.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.461+0000 7f64155ee700 1 -- 172.21.15.2:0/3266757352 >> 172.21.15.2:0/3266757352 conn(0x7f641006e6b0 msgr2=0x7f641006eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:59.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.462+0000 7f64155ee700 1 -- 172.21.15.2:0/3266757352 shutdown_connections 2024-05-14T16:56:59.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.462+0000 7f64155ee700 1 -- 172.21.15.2:0/3266757352 wait complete. 2024-05-14T16:56:59.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.463+0000 7f64155ee700 1 Processor -- start 2024-05-14T16:56:59.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.463+0000 7f64155ee700 1 -- start start 2024-05-14T16:56:59.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.463+0000 7f64155ee700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f641010a540 0x7f641010bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:59.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.463+0000 7f64155ee700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f641010ae60 0x7f641010c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:59.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.463+0000 7f64155ee700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6410116690 0x7f641010ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:59.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.463+0000 7f64155ee700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f641011c000 con 0x7f641010a540 2024-05-14T16:56:59.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.463+0000 7f64155ee700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f641011be80 con 0x7f6410116690 2024-05-14T16:56:59.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.463+0000 7f64155ee700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f641011c180 con 0x7f641010ae60 2024-05-14T16:56:59.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.464+0000 7f640effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f641010a540 0x7f641010bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:59.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.464+0000 7f640effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f641010a540 0x7f641010bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43958/0 (socket says 172.21.15.2:43958) 2024-05-14T16:56:59.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.464+0000 7f640effd700 1 -- 172.21.15.2:0/958016074 learned_addr learned my addr 172.21.15.2:0/958016074 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:56:59.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.464+0000 7f640e7fc700 1 --2- 172.21.15.2:0/958016074 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f641010ae60 0x7f641010c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:59.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.464+0000 7f640f7fe700 1 --2- 172.21.15.2:0/958016074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6410116690 0x7f641010ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:59.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.464+0000 7f640effd700 1 -- 172.21.15.2:0/958016074 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f641010ae60 msgr2=0x7f641010c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:59.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.464+0000 7f640effd700 1 --2- 172.21.15.2:0/958016074 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f641010ae60 0x7f641010c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:59.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.464+0000 7f640effd700 1 -- 172.21.15.2:0/958016074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6410116690 msgr2=0x7f641010ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:59.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.464+0000 7f640effd700 1 --2- 172.21.15.2:0/958016074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6410116690 0x7f641010ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:59.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.464+0000 7f640effd700 1 -- 172.21.15.2:0/958016074 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64101b4ea0 con 0x7f641010a540 2024-05-14T16:56:59.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.465+0000 7f640f7fe700 1 --2- 172.21.15.2:0/958016074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6410116690 0x7f641010ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:56:59.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.465+0000 7f640effd700 1 --2- 172.21.15.2:0/958016074 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f641010a540 0x7f641010bfb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6400009fd0 tx=0x7f640000fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:59.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.465+0000 7f63f7fff700 1 -- 172.21.15.2:0/958016074 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64000051f0 con 0x7f641010a540 2024-05-14T16:56:59.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.465+0000 7f64155ee700 1 -- 172.21.15.2:0/958016074 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64101b50d0 con 0x7f641010a540 2024-05-14T16:56:59.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.465+0000 7f64155ee700 1 -- 172.21.15.2:0/958016074 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64101b5610 con 0x7f641010a540 2024-05-14T16:56:59.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.465+0000 7f63f7fff700 1 -- 172.21.15.2:0/958016074 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f640000b560 con 0x7f641010a540 2024-05-14T16:56:59.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.467+0000 7f63f7fff700 1 -- 172.21.15.2:0/958016074 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6400003b40 con 0x7f641010a540 2024-05-14T16:56:59.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.467+0000 7f64155ee700 1 -- 172.21.15.2:0/958016074 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63fc005330 con 0x7f641010a540 2024-05-14T16:56:59.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.470+0000 7f63f7fff700 1 -- 172.21.15.2:0/958016074 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f6400005390 con 0x7f641010a540 2024-05-14T16:56:59.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.471+0000 7f63f7fff700 1 --2- 172.21.15.2:0/958016074 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f63f80b2980 0x7f63f80b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:56:59.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.471+0000 7f63f7fff700 1 -- 172.21.15.2:0/958016074 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f64000e3860 con 0x7f641010a540 2024-05-14T16:56:59.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.472+0000 7f640e7fc700 1 --2- 172.21.15.2:0/958016074 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f63f80b2980 0x7f63f80b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:56:59.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.473+0000 7f640e7fc700 1 --2- 172.21.15.2:0/958016074 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f63f80b2980 0x7f63f80b4e40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f641010d770 tx=0x7f640800b040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:56:59.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.477+0000 7f63f7fff700 1 -- 172.21.15.2:0/958016074 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6400013030 con 0x7f641010a540 2024-05-14T16:56:59.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.918+0000 7f64155ee700 1 -- 172.21.15.2:0/958016074 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63fc005620 con 0x7f641010a540 2024-05-14T16:56:59.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.919+0000 7f63f7fff700 1 -- 172.21.15.2:0/958016074 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11857 (secure 0 0 0) 0x7f64000ab2a0 con 0x7f641010a540 2024-05-14T16:56:59.874 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:56:59.879 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2275696923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2275696923},{"type":"v1","addr":"172.21.15.2:6835","nonce":2275696923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/716718761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":716718761},{"type":"v1","addr":"172.21.15.2:6839","nonce":716718761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/3644815179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3644815179},{"type":"v1","addr":"172.21.15.2:6841","nonce":3644815179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1880336731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1880336731},{"type":"v1","addr":"172.21.15.151:6833","nonce":1880336731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2415605548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2415605548},{"type":"v1","addr":"172.21.15.2:6837","nonce":2415605548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2635167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2635167},{"type":"v1","addr":"172.21.15.123:6833","nonce":2635167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:51:34.106919+0000","modified":"2024-05-14T16:56:06.904708+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24455,"mds_2":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24415":{"gid":24415,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/762044339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":762044339},{"type":"v1","addr":"172.21.15.151:6835","nonce":762044339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/132043151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":132043151},{"type":"v1","addr":"172.21.15.151:6837","nonce":132043151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/4107834375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":4107834375},{"type":"v1","addr":"172.21.15.151:6839","nonce":4107834375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.123:6835/1198670013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1198670013},{"type":"v1","addr":"172.21.15.123:6835","nonce":1198670013}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/333426645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":333426645},{"type":"v1","addr":"172.21.15.123:6837","nonce":333426645}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/580275877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":580275877},{"type":"v1","addr":"172.21.15.123:6839","nonce":580275877}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24455,24479,24431]},"id":1}]} 2024-05-14T16:56:59.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.923+0000 7f64155ee700 1 -- 172.21.15.2:0/958016074 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f63f80b2980 msgr2=0x7f63f80b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:59.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.923+0000 7f64155ee700 1 --2- 172.21.15.2:0/958016074 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f63f80b2980 0x7f63f80b4e40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f641010d770 tx=0x7f640800b040 comp rx=0 tx=0).stop 2024-05-14T16:56:59.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.923+0000 7f64155ee700 1 -- 172.21.15.2:0/958016074 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f641010a540 msgr2=0x7f641010bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:56:59.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.923+0000 7f64155ee700 1 --2- 172.21.15.2:0/958016074 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f641010a540 0x7f641010bfb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6400009fd0 tx=0x7f640000fb00 comp rx=0 tx=0).stop 2024-05-14T16:56:59.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.923+0000 7f64155ee700 1 -- 172.21.15.2:0/958016074 shutdown_connections 2024-05-14T16:56:59.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.923+0000 7f64155ee700 1 --2- 172.21.15.2:0/958016074 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f63f80b2980 0x7f63f80b4e40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:59.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.923+0000 7f64155ee700 1 --2- 172.21.15.2:0/958016074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6410116690 0x7f641010ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:59.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.923+0000 7f64155ee700 1 --2- 172.21.15.2:0/958016074 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f641010a540 0x7f641010bfb0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:59.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.923+0000 7f64155ee700 1 --2- 172.21.15.2:0/958016074 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f641010ae60 0x7f641010c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:56:59.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.923+0000 7f64155ee700 1 -- 172.21.15.2:0/958016074 >> 172.21.15.2:0/958016074 conn(0x7f641006e6b0 msgr2=0x7f6410072320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:56:59.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.924+0000 7f64155ee700 1 -- 172.21.15.2:0/958016074 shutdown_connections 2024-05-14T16:56:59.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:56:57.924+0000 7f64155ee700 1 -- 172.21.15.2:0/958016074 wait complete. 2024-05-14T16:56:59.987 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 30 2024-05-14T16:57:00.034 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:56:58 smithi002 ceph-mon[31538]: pgmap v71: 97 pgs: 97 active+clean; 1.4 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 801 op/s 2024-05-14T16:57:00.041 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:56:58 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/958016074' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:57:00.066 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 146.7 --await-for 45.0' 2024-05-14T16:57:00.073 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 146.7 --await-for 45.0 2024-05-14T16:57:00.469 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:57:00.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:57:00 smithi123 ceph-mon[35971]: pgmap v72: 97 pgs: 97 active+clean; 1.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 660 op/s 2024-05-14T16:57:00.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:57:00 smithi002 ceph-mon[31538]: pgmap v72: 97 pgs: 97 active+clean; 1.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 660 op/s 2024-05-14T16:57:00.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:57:00 smithi151 ceph-mon[36066]: pgmap v72: 97 pgs: 97 active+clean; 1.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 660 op/s 2024-05-14T16:57:01.747 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 105.0 s, 3152.5 tps, lat 31.548 ms stddev 33.580 2024-05-14T16:57:02.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:57:02 smithi123 ceph-mon[35971]: pgmap v73: 97 pgs: 97 active+clean; 1.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.5 MiB/s wr, 385 op/s 2024-05-14T16:57:02.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:57:02 smithi002 ceph-mon[31538]: pgmap v73: 97 pgs: 97 active+clean; 1.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.5 MiB/s wr, 385 op/s 2024-05-14T16:57:02.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:57:02 smithi151 ceph-mon[36066]: pgmap v73: 97 pgs: 97 active+clean; 1.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.5 MiB/s wr, 385 op/s 2024-05-14T16:57:02.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.894+0000 7f5e9d8bc700 1 -- 172.21.15.2:0/2941421002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e98076200 msgr2=0x7f5e98113e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:02.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.894+0000 7f5e9d8bc700 1 --2- 172.21.15.2:0/2941421002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e98076200 0x7f5e98113e70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c009990 tx=0x7f5e8c009ca0 comp rx=0 tx=0).stop 2024-05-14T16:57:02.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.895+0000 7f5e9d8bc700 1 -- 172.21.15.2:0/2941421002 shutdown_connections 2024-05-14T16:57:02.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.895+0000 7f5e9d8bc700 1 --2- 172.21.15.2:0/2941421002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e981143b0 0x7f5e981167a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:02.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.895+0000 7f5e9d8bc700 1 --2- 172.21.15.2:0/2941421002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e980758e0 0x7f5e98075cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:02.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.895+0000 7f5e9d8bc700 1 --2- 172.21.15.2:0/2941421002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e98076200 0x7f5e98113e70 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:02.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.895+0000 7f5e9d8bc700 1 -- 172.21.15.2:0/2941421002 >> 172.21.15.2:0/2941421002 conn(0x7f5e980feef0 msgr2=0x7f5e98101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:02.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.895+0000 7f5e9d8bc700 1 -- 172.21.15.2:0/2941421002 shutdown_connections 2024-05-14T16:57:03.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.895+0000 7f5e9d8bc700 1 -- 172.21.15.2:0/2941421002 wait complete. 2024-05-14T16:57:03.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.896+0000 7f5e9d8bc700 1 Processor -- start 2024-05-14T16:57:03.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.896+0000 7f5e9d8bc700 1 -- start start 2024-05-14T16:57:03.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.896+0000 7f5e9d8bc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e980758e0 0x7f5e98113960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:03.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.896+0000 7f5e9d8bc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e98076200 0x7f5e9810c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:03.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.896+0000 7f5e9d8bc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e981143b0 0x7f5e9810cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:03.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.896+0000 7f5e9d8bc700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5e98119c70 con 0x7f5e981143b0 2024-05-14T16:57:03.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.896+0000 7f5e9d8bc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5e98119af0 con 0x7f5e98076200 2024-05-14T16:57:03.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.896+0000 7f5e9d8bc700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5e98119df0 con 0x7f5e980758e0 2024-05-14T16:57:03.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.897+0000 7f5e967fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e98076200 0x7f5e9810c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:03.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.897+0000 7f5e977fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e981143b0 0x7f5e9810cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:03.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.897+0000 7f5e977fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e981143b0 0x7f5e9810cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43974/0 (socket says 172.21.15.2:43974) 2024-05-14T16:57:03.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.897+0000 7f5e977fe700 1 -- 172.21.15.2:0/1370546020 learned_addr learned my addr 172.21.15.2:0/1370546020 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:57:03.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.897+0000 7f5e977fe700 1 -- 172.21.15.2:0/1370546020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e980758e0 msgr2=0x7f5e98113960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:03.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.897+0000 7f5e96ffd700 1 --2- 172.21.15.2:0/1370546020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e980758e0 0x7f5e98113960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:03.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.898+0000 7f5e977fe700 1 --2- 172.21.15.2:0/1370546020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e980758e0 0x7f5e98113960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:03.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.898+0000 7f5e977fe700 1 -- 172.21.15.2:0/1370546020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e98076200 msgr2=0x7f5e9810c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:03.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.898+0000 7f5e977fe700 1 --2- 172.21.15.2:0/1370546020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e98076200 0x7f5e9810c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:03.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.898+0000 7f5e977fe700 1 -- 172.21.15.2:0/1370546020 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e9810d520 con 0x7f5e981143b0 2024-05-14T16:57:03.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.898+0000 7f5e967fc700 1 --2- 172.21.15.2:0/1370546020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e98076200 0x7f5e9810c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:57:03.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.898+0000 7f5e96ffd700 1 --2- 172.21.15.2:0/1370546020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e980758e0 0x7f5e98113960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:57:03.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.898+0000 7f5e977fe700 1 --2- 172.21.15.2:0/1370546020 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e981143b0 0x7f5e9810cf30 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5e9000b7b0 tx=0x7f5e9000bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:03.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.899+0000 7f5e9c8ba700 1 -- 172.21.15.2:0/1370546020 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e90014070 con 0x7f5e981143b0 2024-05-14T16:57:03.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.899+0000 7f5e9c8ba700 1 -- 172.21.15.2:0/1370546020 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e90010cb0 con 0x7f5e981143b0 2024-05-14T16:57:03.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.899+0000 7f5e9c8ba700 1 -- 172.21.15.2:0/1370546020 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e90017d60 con 0x7f5e981143b0 2024-05-14T16:57:03.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.899+0000 7f5e9d8bc700 1 -- 172.21.15.2:0/1370546020 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e9810d810 con 0x7f5e981143b0 2024-05-14T16:57:03.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.899+0000 7f5e9d8bc700 1 -- 172.21.15.2:0/1370546020 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e98111780 con 0x7f5e981143b0 2024-05-14T16:57:03.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.904+0000 7f5e9c8ba700 1 -- 172.21.15.2:0/1370546020 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f5e90017070 con 0x7f5e981143b0 2024-05-14T16:57:03.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.905+0000 7f5ea8152b80 1 Processor -- start 2024-05-14T16:57:03.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.905+0000 7f5ea8152b80 1 -- start start 2024-05-14T16:57:03.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.906+0000 7f5e9c8ba700 1 --2- 172.21.15.2:0/1370546020 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f5e800b2b20 0x7f5e800b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:03.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.906+0000 7f5e967fc700 1 --2- 172.21.15.2:0/1370546020 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f5e800b2b20 0x7f5e800b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:03.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.906+0000 7f5e9c8ba700 1 -- 172.21.15.2:0/1370546020 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f5e900e1ee0 con 0x7f5e981143b0 2024-05-14T16:57:03.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.906+0000 7f5ea8152b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c510fb5690 0x55c510fb5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:03.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.906+0000 7f5ea8152b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c510fb5fb0 0x55c510fb0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:03.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.906+0000 7f5ea8152b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c510fb14b0 0x55c510fb1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:03.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.906+0000 7f5ea8152b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c510f0ef00 con 0x55c510fb5fb0 2024-05-14T16:57:03.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.906+0000 7f5ea8152b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c510efeaa0 con 0x55c510fb5690 2024-05-14T16:57:03.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.906+0000 7f5ea8152b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c510f02ae0 con 0x55c510fb14b0 2024-05-14T16:57:03.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.906+0000 7f5e977fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c510fb14b0 0x55c510fb1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:03.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.907+0000 7f5e96ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c510fb5690 0x55c510fb5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:03.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.907+0000 7f5e977fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c510fb14b0 0x55c510fb1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:46250/0 (socket says 172.21.15.2:46250) 2024-05-14T16:57:03.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.907+0000 7f5e977fe700 1 -- 172.21.15.2:0/904808873 learned_addr learned my addr 172.21.15.2:0/904808873 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:57:03.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.907+0000 7f5e967fc700 1 --2- 172.21.15.2:0/904808873 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c510fb5fb0 0x55c510fb0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:03.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.907+0000 7f5e977fe700 1 -- 172.21.15.2:0/904808873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c510fb5690 msgr2=0x55c510fb5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:03.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.907+0000 7f5e977fe700 1 --2- 172.21.15.2:0/904808873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c510fb5690 0x55c510fb5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:03.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.907+0000 7f5e977fe700 1 -- 172.21.15.2:0/904808873 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c510fb5fb0 msgr2=0x55c510fb0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:03.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.907+0000 7f5e977fe700 1 --2- 172.21.15.2:0/904808873 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c510fb5fb0 0x55c510fb0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:03.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.907+0000 7f5e977fe700 1 -- 172.21.15.2:0/904808873 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c510ea8140 con 0x55c510fb14b0 2024-05-14T16:57:03.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.908+0000 7f5e977fe700 1 --2- 172.21.15.2:0/904808873 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c510fb14b0 0x55c510fb1910 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5e900a9210 tx=0x7f5e900a89f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5728af2c036786b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:03.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.908+0000 7f5e7dffb700 1 -- 172.21.15.2:0/904808873 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e900a9980 con 0x55c510fb14b0 2024-05-14T16:57:03.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.908+0000 7f5e7dffb700 1 -- 172.21.15.2:0/904808873 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e900a9280 con 0x55c510fb14b0 2024-05-14T16:57:03.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.908+0000 7f5e7dffb700 1 -- 172.21.15.2:0/904808873 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e900b1bc0 con 0x55c510fb14b0 2024-05-14T16:57:03.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.909+0000 7f5ea8152b80 1 -- 172.21.15.2:0/904808873 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c510fb14b0 msgr2=0x55c510fb1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:03.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.909+0000 7f5ea8152b80 1 --2- 172.21.15.2:0/904808873 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c510fb14b0 0x55c510fb1910 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5e900a9210 tx=0x7f5e900a89f0 comp rx=0 tx=0).stop 2024-05-14T16:57:03.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.909+0000 7f5ea8152b80 1 -- 172.21.15.2:0/904808873 shutdown_connections 2024-05-14T16:57:03.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.909+0000 7f5ea8152b80 1 --2- 172.21.15.2:0/904808873 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c510fb14b0 0x55c510fb1910 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:03.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.910+0000 7f5ea8152b80 1 --2- 172.21.15.2:0/904808873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c510fb5690 0x55c510fb5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:03.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.910+0000 7f5ea8152b80 1 --2- 172.21.15.2:0/904808873 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c510fb5fb0 0x55c510fb0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:03.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.910+0000 7f5ea8152b80 1 -- 172.21.15.2:0/904808873 >> 172.21.15.2:0/904808873 conn(0x55c510eddf40 msgr2=0x55c510fab6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:03.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.910+0000 7f5e967fc700 1 --2- 172.21.15.2:0/1370546020 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f5e800b2b20 0x7f5e800b4fe0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5e9810e240 tx=0x7f5e8c003f10 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:03.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.910+0000 7f5ea8152b80 1 -- 172.21.15.2:0/904808873 shutdown_connections 2024-05-14T16:57:03.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.910+0000 7f5ea8152b80 1 -- 172.21.15.2:0/904808873 wait complete. 2024-05-14T16:57:03.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.911+0000 7f5ea8152b80 1 -- start start 2024-05-14T16:57:03.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.911+0000 7f5ea8152b80 1 Processor -- start 2024-05-14T16:57:03.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.912+0000 7f5ea8152b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c510fb14b0 0x55c510fbcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:03.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.912+0000 7f5ea8152b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c510fb5690 0x55c510fba150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:03.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.912+0000 7f5ea8152b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c510fb5fb0 0x55c510fba690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:03.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.912+0000 7f5e967fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c510fb5690 0x55c510fba150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:03.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.912+0000 7f5e967fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c510fb5690 0x55c510fba150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43986/0 (socket says 172.21.15.2:43986) 2024-05-14T16:57:03.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.912+0000 7f5e967fc700 1 -- 172.21.15.2:0/3411693955 learned_addr learned my addr 172.21.15.2:0/3411693955 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:57:03.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.912+0000 7f5ea8152b80 1 -- 172.21.15.2:0/3411693955 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c510e05130 con 0x55c510fb5690 2024-05-14T16:57:03.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.912+0000 7f5ea8152b80 1 -- 172.21.15.2:0/3411693955 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c510e66f30 con 0x55c510fb5fb0 2024-05-14T16:57:03.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.913+0000 7f5ea8152b80 1 -- 172.21.15.2:0/3411693955 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c510fbdbd0 con 0x55c510fb14b0 2024-05-14T16:57:03.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.913+0000 7f5e977fe700 1 --2- 172.21.15.2:0/3411693955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c510fb5fb0 0x55c510fba690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:03.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.913+0000 7f5ea8152b80 10 client.0 _ll_register_callbacks cb 0x55c510e99ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:57:03.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.913+0000 7f5e96ffd700 1 --2- 172.21.15.2:0/3411693955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c510fb14b0 0x55c510fbcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:03.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.913+0000 7f5e977fe700 1 -- 172.21.15.2:0/3411693955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c510fb14b0 msgr2=0x55c510fbcdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:03.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.913+0000 7f5e977fe700 1 --2- 172.21.15.2:0/3411693955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c510fb14b0 0x55c510fbcdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:03.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.913+0000 7f5e977fe700 1 -- 172.21.15.2:0/3411693955 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c510fb5690 msgr2=0x55c510fba150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:03.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.913+0000 7f5e977fe700 1 --2- 172.21.15.2:0/3411693955 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c510fb5690 0x55c510fba150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:03.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.913+0000 7f5e977fe700 1 -- 172.21.15.2:0/3411693955 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c510e4ea60 con 0x55c510fb5fb0 2024-05-14T16:57:03.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.913+0000 7f5e96ffd700 1 --2- 172.21.15.2:0/3411693955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c510fb14b0 0x55c510fbcdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:57:03.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.914+0000 7f5e977fe700 1 --2- 172.21.15.2:0/3411693955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c510fb5fb0 0x55c510fba690 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5e900a9210 tx=0x7f5e900a5740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=331f9922727fa336 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:03.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.915+0000 7f5e76ffd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T16:57:03.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.915+0000 7f5e76ffd700 1 -- 172.21.15.2:0/3411693955 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9006c020 con 0x55c510fb5fb0 2024-05-14T16:57:03.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.915+0000 7f5e76ffd700 1 -- 172.21.15.2:0/3411693955 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f5e900b1dd0 con 0x55c510fb5fb0 2024-05-14T16:57:04.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.915+0000 7f5ea8152b80 1 -- 172.21.15.2:0/3411693955 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e900a9280 con 0x55c510fb5fb0 2024-05-14T16:57:04.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.916+0000 7f5e76ffd700 1 -- 172.21.15.2:0/3411693955 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5e900b2050 con 0x55c510fb5fb0 2024-05-14T16:57:04.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.917+0000 7f5ea8152b80 10 client.25310 fetch_fsmap learned FSMap version 30 2024-05-14T16:57:04.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.917+0000 7f5ea8152b80 1 -- 172.21.15.2:0/3411693955 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c510eb88f0 con 0x55c510fb5fb0 2024-05-14T16:57:04.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.917+0000 7f5e76ffd700 1 -- 172.21.15.2:0/3411693955 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e900a9280 con 0x55c510fb5fb0 2024-05-14T16:57:04.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.918+0000 7f5e76ffd700 1 -- 172.21.15.2:0/3411693955 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f5e900a84e0 con 0x55c510fb5fb0 2024-05-14T16:57:04.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.918+0000 7f5ea8152b80 10 client.25310 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T16:57:04.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.918+0000 7f5ea8152b80 10 client.25310 resolve_mds: validated gid 24479 aka daemon mds.b 2024-05-14T16:57:04.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.918+0000 7f5ea8152b80 20 client.25310 populate_metadata read hostname 'smithi002' 2024-05-14T16:57:04.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.918+0000 7f5ea8152b80 1 --2- 172.21.15.2:0/3411693955 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55c510fca550 0x55c510fcc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:04.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.918+0000 7f5ea8152b80 4 client.25310 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:57:04.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.918+0000 7f5ea8152b80 1 -- 172.21.15.2:0/3411693955 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c510f10e50 con 0x55c510fca550 2024-05-14T16:57:04.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.920+0000 7f5e96ffd700 1 --2- 172.21.15.2:0/3411693955 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55c510fca550 0x55c510fcc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:04.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.921+0000 7f5e96ffd700 1 --2- 172.21.15.2:0/3411693955 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55c510fca550 0x55c510fcc940 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=423669607ba821e9 server_cookie=fbeb020313fa340e in_seq=0 out_seq=0 2024-05-14T16:57:04.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.921+0000 7f5e76ffd700 10 client.25310 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:57:04.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.922+0000 7f5e76ffd700 1 -- 172.21.15.2:0/3411693955 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c510f10e50 con 0x55c510fca550 2024-05-14T16:57:04.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.922+0000 7f5e76ffd700 10 client.25310 handle_command_reply: tid=0 2024-05-14T16:57:04.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.923+0000 7f5ea8152b80 1 client.25310 shutdown 2024-05-14T16:57:04.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.924+0000 7f5ea8152b80 1 -- 172.21.15.2:0/3411693955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c510fb5fb0 msgr2=0x55c510fba690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:04.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.924+0000 7f5ea8152b80 1 --2- 172.21.15.2:0/3411693955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c510fb5fb0 0x55c510fba690 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5e900a9210 tx=0x7f5e900a5740 comp rx=0 tx=0).stop 2024-05-14T16:57:04.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.924+0000 7f5ea8152b80 1 -- 172.21.15.2:0/3411693955 shutdown_connections 2024-05-14T16:57:04.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.924+0000 7f5ea8152b80 1 --2- 172.21.15.2:0/3411693955 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55c510fca550 0x55c510fcc940 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:04.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.924+0000 7f5ea8152b80 1 --2- 172.21.15.2:0/3411693955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c510fb5fb0 0x55c510fba690 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:04.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.924+0000 7f5ea8152b80 1 --2- 172.21.15.2:0/3411693955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c510fb14b0 0x55c510fbcdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:04.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.924+0000 7f5ea8152b80 1 --2- 172.21.15.2:0/3411693955 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c510fb5690 0x55c510fba150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:04.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.924+0000 7f5ea8152b80 1 -- 172.21.15.2:0/3411693955 >> 172.21.15.2:0/3411693955 conn(0x55c510eddf40 msgr2=0x55c510fb6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:04.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.925+0000 7f5e76ffd700 0 client.25310 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:57:04.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.925+0000 7f5ea8152b80 1 -- 172.21.15.2:0/3411693955 shutdown_connections 2024-05-14T16:57:04.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.926+0000 7f5ea8152b80 1 -- 172.21.15.2:0/3411693955 wait complete. 2024-05-14T16:57:04.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.926+0000 7f5ea8152b80 20 client.25310 trim_cache size 0 max 16384 2024-05-14T16:57:04.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.955+0000 7f5ea8152b80 1 Processor -- start 2024-05-14T16:57:04.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.955+0000 7f5ea8152b80 1 -- start start 2024-05-14T16:57:04.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.956+0000 7f5ea8152b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c510fb14b0 0x55c510fa6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:04.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.956+0000 7f5ea8152b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c510fb5690 0x55c510fcf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:04.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.956+0000 7f5ea8152b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c510fb5fb0 0x55c510fcfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:04.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.956+0000 7f5ea8152b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5e981bf540 con 0x55c510fb5fb0 2024-05-14T16:57:04.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.956+0000 7f5ea8152b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5e981bcd40 con 0x55c510fb5690 2024-05-14T16:57:04.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.956+0000 7f5ea8152b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5e981babb0 con 0x55c510fb14b0 2024-05-14T16:57:04.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.956+0000 7f5e96ffd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c510fb14b0 0x55c510fa6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:04.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.956+0000 7f5e96ffd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c510fb14b0 0x55c510fa6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:46270/0 (socket says 172.21.15.2:46270) 2024-05-14T16:57:04.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.956+0000 7f5e96ffd700 1 -- 172.21.15.2:0/67341498 learned_addr learned my addr 172.21.15.2:0/67341498 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:57:04.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.956+0000 7f5e977fe700 1 --2- 172.21.15.2:0/67341498 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c510fb5fb0 0x55c510fcfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:04.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.956+0000 7f5e967fc700 1 --2- 172.21.15.2:0/67341498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c510fb5690 0x55c510fcf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:04.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.956+0000 7f5e96ffd700 1 -- 172.21.15.2:0/67341498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c510fb5690 msgr2=0x55c510fcf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:04.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.956+0000 7f5e96ffd700 1 --2- 172.21.15.2:0/67341498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c510fb5690 0x55c510fcf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:04.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.956+0000 7f5e96ffd700 1 -- 172.21.15.2:0/67341498 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c510fb5fb0 msgr2=0x55c510fcfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:04.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.956+0000 7f5e96ffd700 1 --2- 172.21.15.2:0/67341498 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c510fb5fb0 0x55c510fcfac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:04.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.956+0000 7f5e96ffd700 1 -- 172.21.15.2:0/67341498 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c510fd0000 con 0x55c510fb14b0 2024-05-14T16:57:04.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.957+0000 7f5e96ffd700 1 --2- 172.21.15.2:0/67341498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c510fb14b0 0x55c510fa6060 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5e8800b320 tx=0x7f5e88008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4c08f86291ecb83 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:04.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.957+0000 7f5e7d7fa700 1 -- 172.21.15.2:0/67341498 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e88015b40 con 0x55c510fb14b0 2024-05-14T16:57:04.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.957+0000 7f5e7d7fa700 1 -- 172.21.15.2:0/67341498 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e88015ce0 con 0x55c510fb14b0 2024-05-14T16:57:04.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.957+0000 7f5e7d7fa700 1 -- 172.21.15.2:0/67341498 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e88013870 con 0x55c510fb14b0 2024-05-14T16:57:04.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.958+0000 7f5ea8152b80 1 -- 172.21.15.2:0/67341498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c510fb14b0 msgr2=0x55c510fa6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:04.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.958+0000 7f5ea8152b80 1 --2- 172.21.15.2:0/67341498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c510fb14b0 0x55c510fa6060 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5e8800b320 tx=0x7f5e88008040 comp rx=0 tx=0).stop 2024-05-14T16:57:04.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.958+0000 7f5ea8152b80 1 -- 172.21.15.2:0/67341498 shutdown_connections 2024-05-14T16:57:04.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.958+0000 7f5ea8152b80 1 --2- 172.21.15.2:0/67341498 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c510fb5fb0 0x55c510fcfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:04.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.958+0000 7f5ea8152b80 1 --2- 172.21.15.2:0/67341498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c510fb14b0 0x55c510fa6060 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:04.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.958+0000 7f5ea8152b80 1 --2- 172.21.15.2:0/67341498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c510fb5690 0x55c510fcf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:04.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.959+0000 7f5ea8152b80 1 -- 172.21.15.2:0/67341498 >> 172.21.15.2:0/67341498 conn(0x55c510eddf40 msgr2=0x55c510fa5950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:04.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.965+0000 7f5ea8152b80 1 -- 172.21.15.2:0/67341498 shutdown_connections 2024-05-14T16:57:04.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.966+0000 7f5ea8152b80 1 -- 172.21.15.2:0/67341498 wait complete. 2024-05-14T16:57:04.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.966+0000 7f5ea8152b80 1 -- start start 2024-05-14T16:57:04.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.967+0000 7f5ea8152b80 1 Processor -- start 2024-05-14T16:57:04.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.967+0000 7f5ea8152b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c510fb14b0 0x55c510fcb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:04.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.967+0000 7f5ea8152b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c510fb5690 0x55c510fcbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:04.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.967+0000 7f5ea8152b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c510fb5fb0 0x55c51102da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:04.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.967+0000 7f5ea8152b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5e981bb200 con 0x55c510fb14b0 2024-05-14T16:57:04.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.968+0000 7f5ea8152b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5e981bc670 con 0x55c510fb5fb0 2024-05-14T16:57:04.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.968+0000 7f5ea8152b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5e981bb8d0 con 0x55c510fb5690 2024-05-14T16:57:04.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.968+0000 7f5e977fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c510fb5fb0 0x55c51102da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:04.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.968+0000 7f5e977fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c510fb5fb0 0x55c51102da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:36500/0 (socket says 172.21.15.2:36500) 2024-05-14T16:57:04.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.968+0000 7f5e977fe700 1 -- 172.21.15.2:0/1631559847 learned_addr learned my addr 172.21.15.2:0/1631559847 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:57:04.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.968+0000 7f5ea8152b80 10 client.0 _ll_register_callbacks cb 0x55c510fabd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:57:04.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.969+0000 7f5e96ffd700 1 --2- 172.21.15.2:0/1631559847 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c510fb14b0 0x55c510fcb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:04.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.969+0000 7f5e96ffd700 1 -- 172.21.15.2:0/1631559847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c510fb5690 msgr2=0x55c510fcbd30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T16:57:04.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.969+0000 7f5e96ffd700 1 --2- 172.21.15.2:0/1631559847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c510fb5690 0x55c510fcbd30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:04.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.969+0000 7f5e96ffd700 1 -- 172.21.15.2:0/1631559847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c510fb5fb0 msgr2=0x55c51102da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:04.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.970+0000 7f5e96ffd700 1 --2- 172.21.15.2:0/1631559847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c510fb5fb0 0x55c51102da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:04.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.970+0000 7f5e96ffd700 1 -- 172.21.15.2:0/1631559847 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c51102e070 con 0x55c510fb14b0 2024-05-14T16:57:04.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.970+0000 7f5e96ffd700 1 --2- 172.21.15.2:0/1631559847 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c510fb14b0 0x55c510fcb7f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5e88006f00 tx=0x7f5e88005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42084f64d708aae1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:04.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.970+0000 7f5e76ffd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:57:04.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.970+0000 7f5e76ffd700 1 -- 172.21.15.2:0/1631559847 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8801b710 con 0x55c510fb14b0 2024-05-14T16:57:04.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.971+0000 7f5e76ffd700 1 -- 172.21.15.2:0/1631559847 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f5e8801b8b0 con 0x55c510fb14b0 2024-05-14T16:57:04.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.971+0000 7f5ea8152b80 1 -- 172.21.15.2:0/1631559847 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e88015ce0 con 0x55c510fb14b0 2024-05-14T16:57:04.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.974+0000 7f5e76ffd700 1 -- 172.21.15.2:0/1631559847 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5e8801bab0 con 0x55c510fb14b0 2024-05-14T16:57:04.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.974+0000 7f5ea8152b80 10 client.15600 fetch_fsmap learned FSMap version 30 2024-05-14T16:57:04.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.974+0000 7f5ea8152b80 1 -- 172.21.15.2:0/1631559847 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c5110353e0 con 0x55c510fb14b0 2024-05-14T16:57:04.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.975+0000 7f5e76ffd700 1 -- 172.21.15.2:0/1631559847 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e88015ce0 con 0x55c510fb14b0 2024-05-14T16:57:04.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.976+0000 7f5e76ffd700 1 -- 172.21.15.2:0/1631559847 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f5e88006150 con 0x55c510fb14b0 2024-05-14T16:57:04.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.976+0000 7f5ea8152b80 10 client.15600 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T16:57:05.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.976+0000 7f5ea8152b80 10 client.15600 resolve_mds: validated gid 24479 aka daemon mds.b 2024-05-14T16:57:05.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.976+0000 7f5ea8152b80 20 client.15600 populate_metadata read hostname 'smithi002' 2024-05-14T16:57:05.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.977+0000 7f5ea8152b80 1 --2- 172.21.15.2:0/1631559847 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55c5110359e0 0x55c511037e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:05.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.977+0000 7f5ea8152b80 4 client.15600 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.7, "await_for": 45.0}] 2024-05-14T16:57:05.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.977+0000 7f5ea8152b80 1 -- 172.21.15.2:0/1631559847 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.7, "await_for": 45.0}) -- 0x55c510f0a630 con 0x55c5110359e0 2024-05-14T16:57:05.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.978+0000 7f5e967fc700 1 --2- 172.21.15.2:0/1631559847 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55c5110359e0 0x55c511037e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:05.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.980+0000 7f5e76ffd700 10 client.15600 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:57:05.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:02.980+0000 7f5e967fc700 1 --2- 172.21.15.2:0/1631559847 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55c5110359e0 0x55c511037e00 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b681b547a144c590 server_cookie=25b855483af0bff in_seq=0 out_seq=0 2024-05-14T16:57:05.106 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:57:04 smithi123 ceph-mon[35971]: pgmap v74: 97 pgs: 97 active+clean; 1.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 MiB/s wr, 1.75k op/s 2024-05-14T16:57:05.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:57:04 smithi151 ceph-mon[36066]: pgmap v74: 97 pgs: 97 active+clean; 1.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 MiB/s wr, 1.75k op/s 2024-05-14T16:57:05.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:03.177+0000 7f5e76ffd700 1 -- 172.21.15.2:0/1631559847 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c510f0a630 con 0x55c5110359e0 2024-05-14T16:57:05.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:03.177+0000 7f5e76ffd700 10 client.15600 handle_command_reply: tid=0 2024-05-14T16:57:05.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:03.177+0000 7f5ea8152b80 1 client.15600 shutdown 2024-05-14T16:57:05.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:03.177+0000 7f5ea8152b80 1 -- 172.21.15.2:0/1631559847 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c510fb14b0 msgr2=0x55c510fcb7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:05.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:03.177+0000 7f5ea8152b80 1 --2- 172.21.15.2:0/1631559847 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c510fb14b0 0x55c510fcb7f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5e88006f00 tx=0x7f5e88005db0 comp rx=0 tx=0).stop 2024-05-14T16:57:05.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:03.177+0000 7f5ea8152b80 1 -- 172.21.15.2:0/1631559847 shutdown_connections 2024-05-14T16:57:05.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:03.177+0000 7f5ea8152b80 1 --2- 172.21.15.2:0/1631559847 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55c5110359e0 0x55c511037e00 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:05.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:03.177+0000 7f5ea8152b80 1 --2- 172.21.15.2:0/1631559847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c510fb5fb0 0x55c51102da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:05.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:03.177+0000 7f5ea8152b80 1 --2- 172.21.15.2:0/1631559847 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c510fb14b0 0x55c510fcb7f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:05.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:03.177+0000 7f5ea8152b80 1 --2- 172.21.15.2:0/1631559847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c510fb5690 0x55c510fcbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:05.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:03.177+0000 7f5ea8152b80 1 -- 172.21.15.2:0/1631559847 >> 172.21.15.2:0/1631559847 conn(0x55c510eddf40 msgr2=0x55c510fb8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:05.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:03.177+0000 7f5e76ffd700 0 client.15600 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:57:05.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:03.177+0000 7f5ea8152b80 1 -- 172.21.15.2:0/1631559847 shutdown_connections 2024-05-14T16:57:05.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:03.177+0000 7f5ea8152b80 1 -- 172.21.15.2:0/1631559847 wait complete. 2024-05-14T16:57:05.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:03.177+0000 7f5ea8152b80 20 client.15600 trim_cache size 0 max 16384 2024-05-14T16:57:05.318 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T16:57:05.326 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 30, 2024-05-14T16:57:05.334 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24479, 2024-05-14T16:57:05.342 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 10, 2024-05-14T16:57:05.351 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-05-14T16:57:05.359 INFO:teuthology.orchestra.run.smithi002.stdout: "ab7ccb52": { 2024-05-14T16:57:05.368 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 10, 2024-05-14T16:57:05.389 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-05-14T16:57:05.408 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-05-14T16:57:05.417 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-05-14T16:57:05.426 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-05-14T16:57:05.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T16:57:05.443 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-05-14T16:57:05.452 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 146.7, 2024-05-14T16:57:05.460 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-05-14T16:57:05.485 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-05-14T16:57:05.537 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-05-14T16:57:05.546 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-05-14T16:57:05.554 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-05-14T16:57:05.568 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-05-14T16:57:05.577 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:57:05.585 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:57:05.594 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:57:05.603 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:57:05.615 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:57:05.634 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T16:57:05.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:03.178+0000 7f5e9d8bc700 1 -- 172.21.15.2:0/1370546020 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f5e800b2b20 msgr2=0x7f5e800b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:05.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:03.178+0000 7f5e9d8bc700 1 --2- 172.21.15.2:0/1370546020 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f5e800b2b20 0x7f5e800b4fe0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5e9810e240 tx=0x7f5e8c003f10 comp rx=0 tx=0).stop 2024-05-14T16:57:05.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:03.178+0000 7f5e9d8bc700 1 -- 172.21.15.2:0/1370546020 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e981143b0 msgr2=0x7f5e9810cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:05.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:03.178+0000 7f5e9d8bc700 1 --2- 172.21.15.2:0/1370546020 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e981143b0 0x7f5e9810cf30 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5e9000b7b0 tx=0x7f5e9000bac0 comp rx=0 tx=0).stop 2024-05-14T16:57:05.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:03.178+0000 7f5e9d8bc700 1 -- 172.21.15.2:0/1370546020 shutdown_connections 2024-05-14T16:57:05.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:03.178+0000 7f5e9d8bc700 1 --2- 172.21.15.2:0/1370546020 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f5e800b2b20 0x7f5e800b4fe0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:05.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:03.178+0000 7f5e9d8bc700 1 --2- 172.21.15.2:0/1370546020 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e981143b0 0x7f5e9810cf30 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:05.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:03.178+0000 7f5e9d8bc700 1 --2- 172.21.15.2:0/1370546020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e980758e0 0x7f5e98113960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:05.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:03.178+0000 7f5e9d8bc700 1 --2- 172.21.15.2:0/1370546020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e98076200 0x7f5e9810c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:05.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:03.178+0000 7f5e9d8bc700 1 -- 172.21.15.2:0/1370546020 >> 172.21.15.2:0/1370546020 conn(0x7f5e980feef0 msgr2=0x7f5e980ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:05.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:03.178+0000 7f5e9d8bc700 1 -- 172.21.15.2:0/1370546020 shutdown_connections 2024-05-14T16:57:05.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:03.178+0000 7f5e9d8bc700 1 -- 172.21.15.2:0/1370546020 wait complete. 2024-05-14T16:57:05.779 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ab7ccb52', quiesce took 11.0 seconds. Will release after: 15.7 2024-05-14T16:57:05.786 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:57:04 smithi002 ceph-mon[31538]: pgmap v74: 97 pgs: 97 active+clean; 1.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 MiB/s wr, 1.75k op/s 2024-05-14T16:57:06.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:57:06 smithi002 ceph-mon[31538]: pgmap v75: 97 pgs: 97 active+clean; 1.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 MiB/s wr, 1.66k op/s 2024-05-14T16:57:06.750 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 110.0 s, 388.4 tps, lat 72.779 ms stddev 133.209 2024-05-14T16:57:06.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:57:06 smithi151 ceph-mon[36066]: pgmap v75: 97 pgs: 97 active+clean; 1.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 MiB/s wr, 1.66k op/s 2024-05-14T16:57:06.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:57:06 smithi123 ceph-mon[35971]: pgmap v75: 97 pgs: 97 active+clean; 1.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 MiB/s wr, 1.66k op/s 2024-05-14T16:57:08.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:57:08 smithi002 ceph-mon[31538]: pgmap v76: 97 pgs: 97 active+clean; 1.4 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 MiB/s wr, 1.71k op/s 2024-05-14T16:57:08.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:57:08 smithi151 ceph-mon[36066]: pgmap v76: 97 pgs: 97 active+clean; 1.4 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 MiB/s wr, 1.71k op/s 2024-05-14T16:57:08.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:57:08 smithi123 ceph-mon[35971]: pgmap v76: 97 pgs: 97 active+clean; 1.4 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 MiB/s wr, 1.71k op/s 2024-05-14T16:57:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.294+0000 7f6516209640 1 -- 172.21.15.2:0/3820757927 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6510101910 msgr2=0x7f651010f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.294+0000 7f6516209640 1 --2- 172.21.15.2:0/3820757927 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6510101910 0x7f651010f8e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f6508009cd0 tx=0x7f650802a8e0 comp rx=0 tx=0).stop 2024-05-14T16:57:09.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.294+0000 7f6516209640 1 -- 172.21.15.2:0/3820757927 shutdown_connections 2024-05-14T16:57:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.294+0000 7f6516209640 1 --2- 172.21.15.2:0/3820757927 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6510101910 0x7f651010f8e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.294+0000 7f6516209640 1 --2- 172.21.15.2:0/3820757927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6510100f50 0x7f65101013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:09.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.294+0000 7f6516209640 1 --2- 172.21.15.2:0/3820757927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6510107a30 0x7f6510107e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:09.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.294+0000 7f6516209640 1 -- 172.21.15.2:0/3820757927 >> 172.21.15.2:0/3820757927 conn(0x7f6510077ed0 msgr2=0x7f65100ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.294+0000 7f6516209640 1 -- 172.21.15.2:0/3820757927 shutdown_connections 2024-05-14T16:57:09.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.294+0000 7f6516209640 1 -- 172.21.15.2:0/3820757927 wait complete. 2024-05-14T16:57:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.295+0000 7f6516209640 1 Processor -- start 2024-05-14T16:57:09.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.295+0000 7f6516209640 1 -- start start 2024-05-14T16:57:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.295+0000 7f6516209640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6510100f50 0x7f6510199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.295+0000 7f6516209640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6510101910 0x7f6510199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:09.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.295+0000 7f6516209640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6510107a30 0x7f651019da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.295+0000 7f6516209640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6510112db0 con 0x7f6510107a30 2024-05-14T16:57:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.295+0000 7f6516209640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6510112c30 con 0x7f6510101910 2024-05-14T16:57:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.295+0000 7f6516209640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6510112f30 con 0x7f6510100f50 2024-05-14T16:57:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.295+0000 7f650ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6510107a30 0x7f651019da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.295+0000 7f650ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6510107a30 0x7f651019da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32974/0 (socket says 172.21.15.2:32974) 2024-05-14T16:57:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.295+0000 7f650ffff640 1 -- 172.21.15.2:0/366843873 learned_addr learned my addr 172.21.15.2:0/366843873 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:57:09.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.295+0000 7f6507fff640 1 --2- 172.21.15.2:0/366843873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6510101910 0x7f6510199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:09.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.295+0000 7f650f7fe640 1 --2- 172.21.15.2:0/366843873 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6510100f50 0x7f6510199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.295+0000 7f650ffff640 1 -- 172.21.15.2:0/366843873 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6510100f50 msgr2=0x7f6510199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.295+0000 7f650ffff640 1 --2- 172.21.15.2:0/366843873 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6510100f50 0x7f6510199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:09.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.295+0000 7f650ffff640 1 -- 172.21.15.2:0/366843873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6510101910 msgr2=0x7f6510199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.295+0000 7f650ffff640 1 --2- 172.21.15.2:0/366843873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6510101910 0x7f6510199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.295+0000 7f650ffff640 1 -- 172.21.15.2:0/366843873 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f651019e140 con 0x7f6510107a30 2024-05-14T16:57:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.296+0000 7f650ffff640 1 --2- 172.21.15.2:0/366843873 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6510107a30 0x7f651019da10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6508005cc0 tx=0x7f6508003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.296+0000 7f650d7fa640 1 -- 172.21.15.2:0/366843873 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6508004240 con 0x7f6510107a30 2024-05-14T16:57:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.296+0000 7f6516209640 1 -- 172.21.15.2:0/366843873 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f651019e3d0 con 0x7f6510107a30 2024-05-14T16:57:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.296+0000 7f650d7fa640 1 -- 172.21.15.2:0/366843873 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65080305a0 con 0x7f6510107a30 2024-05-14T16:57:09.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.296+0000 7f650d7fa640 1 -- 172.21.15.2:0/366843873 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65080466c0 con 0x7f6510107a30 2024-05-14T16:57:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.296+0000 7f6516209640 1 -- 172.21.15.2:0/366843873 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65101a5cc0 con 0x7f6510107a30 2024-05-14T16:57:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.298+0000 7f6519004740 1 Processor -- start 2024-05-14T16:57:09.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.298+0000 7f650d7fa640 1 -- 172.21.15.2:0/366843873 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f6508030a50 con 0x7f6510107a30 2024-05-14T16:57:09.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.299+0000 7f6519004740 1 -- start start 2024-05-14T16:57:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.299+0000 7f6519004740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a131d448f0 0x55a131d44cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:09.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.299+0000 7f6519004740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a131d45210 0x55a131d4f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.299+0000 7f6519004740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a131d4fad0 0x55a131d51ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.299+0000 7f6519004740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55a131cb7c30 con 0x55a131d448f0 2024-05-14T16:57:09.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.299+0000 7f6519004740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a131ca77d0 con 0x55a131d45210 2024-05-14T16:57:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.299+0000 7f6519004740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a131cab810 con 0x55a131d4fad0 2024-05-14T16:57:09.919 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.299+0000 7f6507fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a131d448f0 0x55a131d44cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.299+0000 7f6507fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a131d448f0 0x55a131d44cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32976/0 (socket says 172.21.15.2:32976) 2024-05-14T16:57:09.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.299+0000 7f6507fff640 1 -- 172.21.15.2:0/2929084585 learned_addr learned my addr 172.21.15.2:0/2929084585 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:57:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.299+0000 7f650f7fe640 1 --2- 172.21.15.2:0/2929084585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a131d45210 0x55a131d4f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.299+0000 7f6507fff640 1 --2- 172.21.15.2:0/2929084585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a131d4fad0 0x55a131d51ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:09.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.299+0000 7f6507fff640 1 -- 172.21.15.2:0/2929084585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a131d4fad0 msgr2=0x55a131d51ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:09.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.299+0000 7f6507fff640 1 --2- 172.21.15.2:0/2929084585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a131d4fad0 0x55a131d51ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.299+0000 7f6507fff640 1 -- 172.21.15.2:0/2929084585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a131d45210 msgr2=0x55a131d4f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.299+0000 7f6507fff640 1 --2- 172.21.15.2:0/2929084585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a131d45210 0x55a131d4f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:10.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.299+0000 7f6507fff640 1 -- 172.21.15.2:0/2929084585 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1319bc210 con 0x55a131d448f0 2024-05-14T16:57:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.300+0000 7f650d7fa640 1 --2- 172.21.15.2:0/366843873 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f64fc0b2a10 0x7f64fc0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:10.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.300+0000 7f6507fff640 1 --2- 172.21.15.2:0/2929084585 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a131d448f0 0x55a131d44cd0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f64f8013560 tx=0x7f64f800a800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6302e9f91cc84cc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.300+0000 7f650f7fe640 1 --2- 172.21.15.2:0/366843873 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f64fc0b2a10 0x7f64fc0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.300+0000 7f6505ffb640 1 -- 172.21.15.2:0/2929084585 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f8019030 con 0x55a131d448f0 2024-05-14T16:57:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.300+0000 7f650d7fa640 1 -- 172.21.15.2:0/366843873 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f650802e030 con 0x7f6510107a30 2024-05-14T16:57:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.300+0000 7f6505ffb640 1 -- 172.21.15.2:0/2929084585 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64f80110d0 con 0x55a131d448f0 2024-05-14T16:57:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.300+0000 7f6505ffb640 1 -- 172.21.15.2:0/2929084585 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f800f7e0 con 0x55a131d448f0 2024-05-14T16:57:10.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.301+0000 7f650f7fe640 1 --2- 172.21.15.2:0/366843873 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f64fc0b2a10 0x7f64fc0b4ed0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x55a131d45e90 tx=0x7f650000c430 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.300+0000 7f6519004740 1 -- 172.21.15.2:0/2929084585 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a131d448f0 msgr2=0x55a131d44cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.301+0000 7f6519004740 1 --2- 172.21.15.2:0/2929084585 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a131d448f0 0x55a131d44cd0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f64f8013560 tx=0x7f64f800a800 comp rx=0 tx=0).stop 2024-05-14T16:57:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.301+0000 7f6519004740 1 -- 172.21.15.2:0/2929084585 shutdown_connections 2024-05-14T16:57:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.301+0000 7f6519004740 1 --2- 172.21.15.2:0/2929084585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a131d4fad0 0x55a131d51ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.301+0000 7f6519004740 1 --2- 172.21.15.2:0/2929084585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a131d45210 0x55a131d4f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.301+0000 7f6519004740 1 --2- 172.21.15.2:0/2929084585 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a131d448f0 0x55a131d44cd0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.301+0000 7f6519004740 1 -- 172.21.15.2:0/2929084585 >> 172.21.15.2:0/2929084585 conn(0x55a131d40e00 msgr2=0x55a131d411e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.301+0000 7f6519004740 1 -- 172.21.15.2:0/2929084585 shutdown_connections 2024-05-14T16:57:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.301+0000 7f6519004740 1 -- 172.21.15.2:0/2929084585 wait complete. 2024-05-14T16:57:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.301+0000 7f6519004740 1 -- start start 2024-05-14T16:57:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.301+0000 7f6519004740 1 Processor -- start 2024-05-14T16:57:10.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.302+0000 7f6519004740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a131d448f0 0x55a131d4f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:10.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.302+0000 7f6519004740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a131d45210 0x55a131d4c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.302+0000 7f6519004740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a131d4fad0 0x55a131d4cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:10.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.302+0000 7f6519004740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55a13198d790 con 0x55a131d4fad0 2024-05-14T16:57:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.302+0000 7f6519004740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a131c63890 con 0x55a131d45210 2024-05-14T16:57:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.302+0000 7f6519004740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a131d527a0 con 0x55a131d448f0 2024-05-14T16:57:10.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.302+0000 7f6519004740 10 client.0 _ll_register_callbacks cb 0x55a131c66d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:57:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.302+0000 7f650ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a131d4fad0 0x55a131d4cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.302+0000 7f650ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a131d4fad0 0x55a131d4cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32990/0 (socket says 172.21.15.2:32990) 2024-05-14T16:57:10.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.302+0000 7f650ffff640 1 -- 172.21.15.2:0/3260237362 learned_addr learned my addr 172.21.15.2:0/3260237362 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:57:10.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.302+0000 7f6507fff640 1 --2- 172.21.15.2:0/3260237362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a131d45210 0x55a131d4c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.302+0000 7f650f7fe640 1 --2- 172.21.15.2:0/3260237362 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a131d448f0 0x55a131d4f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.302+0000 7f650ffff640 1 -- 172.21.15.2:0/3260237362 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a131d448f0 msgr2=0x55a131d4f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.302+0000 7f650ffff640 1 --2- 172.21.15.2:0/3260237362 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a131d448f0 0x55a131d4f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:10.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.302+0000 7f650ffff640 1 -- 172.21.15.2:0/3260237362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a131d45210 msgr2=0x55a131d4c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.302+0000 7f650ffff640 1 --2- 172.21.15.2:0/3260237362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a131d45210 0x55a131d4c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.302+0000 7f650ffff640 1 -- 172.21.15.2:0/3260237362 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a131a88620 con 0x55a131d4fad0 2024-05-14T16:57:10.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.302+0000 7f6507fff640 1 --2- 172.21.15.2:0/3260237362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a131d45210 0x55a131d4c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:57:10.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.302+0000 7f650ffff640 1 --2- 172.21.15.2:0/3260237362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a131d4fad0 0x55a131d4cc20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6508101580 tx=0x7f65080098f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ee83e3d66fd3049 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:10.488 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.302+0000 7f64eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:57:10.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.302+0000 7f64eeffd640 1 -- 172.21.15.2:0/3260237362 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65080e3020 con 0x55a131d4fad0 2024-05-14T16:57:10.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.302+0000 7f64eeffd640 1 -- 172.21.15.2:0/3260237362 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f65080c9e80 con 0x55a131d4fad0 2024-05-14T16:57:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.302+0000 7f6519004740 1 -- 172.21.15.2:0/3260237362 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64f80110d0 con 0x55a131d4fad0 2024-05-14T16:57:10.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.303+0000 7f64eeffd640 1 -- 172.21.15.2:0/3260237362 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f65080cac20 con 0x55a131d4fad0 2024-05-14T16:57:10.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.303+0000 7f6519004740 10 client.15612 fetch_fsmap learned FSMap version 30 2024-05-14T16:57:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.303+0000 7f6519004740 1 -- 172.21.15.2:0/3260237362 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a131d5e1e0 con 0x55a131d4fad0 2024-05-14T16:57:10.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.303+0000 7f64eeffd640 1 -- 172.21.15.2:0/3260237362 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64f80110d0 con 0x55a131d4fad0 2024-05-14T16:57:10.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.303+0000 7f64eeffd640 1 -- 172.21.15.2:0/3260237362 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f650804e060 con 0x55a131d4fad0 2024-05-14T16:57:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.303+0000 7f6519004740 10 client.15612 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T16:57:10.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.303+0000 7f6519004740 10 client.15612 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:57:10.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.303+0000 7f6519004740 20 client.15612 populate_metadata read hostname 'smithi002' 2024-05-14T16:57:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.303+0000 7f6519004740 1 --2- 172.21.15.2:0/3260237362 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55a131d5f710 0x55a131d61b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.303+0000 7f6519004740 4 client.15612 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:57:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.303+0000 7f6519004740 1 -- 172.21.15.2:0/3260237362 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a131cb9b80 con 0x55a131d5f710 2024-05-14T16:57:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.303+0000 7f650f7fe640 1 --2- 172.21.15.2:0/3260237362 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55a131d5f710 0x55a131d61b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:10.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.304+0000 7f650f7fe640 1 --2- 172.21.15.2:0/3260237362 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55a131d5f710 0x55a131d61b50 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2844e8b6e7019681 server_cookie=8ab7786d7bab31fc in_seq=0 out_seq=0 2024-05-14T16:57:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.304+0000 7f64eeffd640 10 client.15612 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:57:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.304+0000 7f64eeffd640 1 -- 172.21.15.2:0/3260237362 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a131cb9b80 con 0x55a131d5f710 2024-05-14T16:57:10.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.304+0000 7f64eeffd640 10 client.15612 handle_command_reply: tid=0 2024-05-14T16:57:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.305+0000 7f6519004740 1 client.15612 shutdown 2024-05-14T16:57:10.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.305+0000 7f6519004740 1 -- 172.21.15.2:0/3260237362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a131d4fad0 msgr2=0x55a131d4cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:10.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.305+0000 7f6519004740 1 --2- 172.21.15.2:0/3260237362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a131d4fad0 0x55a131d4cc20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6508101580 tx=0x7f65080098f0 comp rx=0 tx=0).stop 2024-05-14T16:57:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.305+0000 7f6519004740 1 -- 172.21.15.2:0/3260237362 shutdown_connections 2024-05-14T16:57:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.305+0000 7f6519004740 1 --2- 172.21.15.2:0/3260237362 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55a131d5f710 0x55a131d61b50 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.305+0000 7f6519004740 1 --2- 172.21.15.2:0/3260237362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a131d4fad0 0x55a131d4cc20 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.305+0000 7f6519004740 1 --2- 172.21.15.2:0/3260237362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a131d45210 0x55a131d4c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:10.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.305+0000 7f64eeffd640 0 client.15612 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:57:10.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.305+0000 7f6519004740 1 --2- 172.21.15.2:0/3260237362 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a131d448f0 0x55a131d4f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:10.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.305+0000 7f6519004740 1 -- 172.21.15.2:0/3260237362 >> 172.21.15.2:0/3260237362 conn(0x55a131d40e00 msgr2=0x55a131d40100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:10.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.305+0000 7f6519004740 1 -- 172.21.15.2:0/3260237362 shutdown_connections 2024-05-14T16:57:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.305+0000 7f6519004740 1 -- 172.21.15.2:0/3260237362 wait complete. 2024-05-14T16:57:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.305+0000 7f6519004740 20 client.15612 trim_cache size 0 max 16384 2024-05-14T16:57:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.320+0000 7f6519004740 1 Processor -- start 2024-05-14T16:57:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.320+0000 7f6519004740 1 -- start start 2024-05-14T16:57:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.320+0000 7f6519004740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a131d448f0 0x55a131d46a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:10.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.320+0000 7f6519004740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a131d45210 0x55a131d4ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:10.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.320+0000 7f6519004740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a131d4fad0 0x55a131d67420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:10.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.320+0000 7f6519004740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f65101b0d30 con 0x55a131d448f0 2024-05-14T16:57:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.320+0000 7f6519004740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f65101ae570 con 0x55a131d45210 2024-05-14T16:57:11.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.320+0000 7f6519004740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f65101ac3c0 con 0x55a131d4fad0 2024-05-14T16:57:11.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.321+0000 7f650f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a131d448f0 0x55a131d46a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:11.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.321+0000 7f650f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a131d448f0 0x55a131d46a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32994/0 (socket says 172.21.15.2:32994) 2024-05-14T16:57:11.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.321+0000 7f650f7fe640 1 -- 172.21.15.2:0/1697031680 learned_addr learned my addr 172.21.15.2:0/1697031680 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:57:11.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.321+0000 7f6507fff640 1 --2- 172.21.15.2:0/1697031680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a131d45210 0x55a131d4ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:11.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.321+0000 7f650f7fe640 1 -- 172.21.15.2:0/1697031680 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a131d4fad0 msgr2=0x55a131d67420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.321+0000 7f650ffff640 1 --2- 172.21.15.2:0/1697031680 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a131d4fad0 0x55a131d67420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:11.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.321+0000 7f650f7fe640 1 --2- 172.21.15.2:0/1697031680 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a131d4fad0 0x55a131d67420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.321+0000 7f650f7fe640 1 -- 172.21.15.2:0/1697031680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a131d45210 msgr2=0x55a131d4ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:11.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.321+0000 7f650f7fe640 1 --2- 172.21.15.2:0/1697031680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a131d45210 0x55a131d4ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.321+0000 7f650f7fe640 1 -- 172.21.15.2:0/1697031680 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a131d52220 con 0x55a131d448f0 2024-05-14T16:57:11.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.321+0000 7f6507fff640 1 --2- 172.21.15.2:0/1697031680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a131d45210 0x55a131d4ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:57:11.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.321+0000 7f650ffff640 1 --2- 172.21.15.2:0/1697031680 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a131d4fad0 0x55a131d67420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:57:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.321+0000 7f650f7fe640 1 --2- 172.21.15.2:0/1697031680 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a131d448f0 0x55a131d46a00 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6500012fd0 tx=0x7f650000a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=787bb955239c170c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:11.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.321+0000 7f65057fa640 1 -- 172.21.15.2:0/1697031680 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6500027640 con 0x55a131d448f0 2024-05-14T16:57:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.321+0000 7f65057fa640 1 -- 172.21.15.2:0/1697031680 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65000277e0 con 0x55a131d448f0 2024-05-14T16:57:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.321+0000 7f65057fa640 1 -- 172.21.15.2:0/1697031680 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650000e610 con 0x55a131d448f0 2024-05-14T16:57:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.322+0000 7f6519004740 1 -- 172.21.15.2:0/1697031680 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a131d448f0 msgr2=0x55a131d46a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.322+0000 7f6519004740 1 --2- 172.21.15.2:0/1697031680 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a131d448f0 0x55a131d46a00 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6500012fd0 tx=0x7f650000a000 comp rx=0 tx=0).stop 2024-05-14T16:57:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.322+0000 7f6519004740 1 -- 172.21.15.2:0/1697031680 shutdown_connections 2024-05-14T16:57:11.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.322+0000 7f6519004740 1 --2- 172.21.15.2:0/1697031680 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a131d4fad0 0x55a131d67420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.322+0000 7f6519004740 1 --2- 172.21.15.2:0/1697031680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a131d45210 0x55a131d4ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.322+0000 7f6519004740 1 --2- 172.21.15.2:0/1697031680 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a131d448f0 0x55a131d46a00 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:11.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.322+0000 7f6519004740 1 -- 172.21.15.2:0/1697031680 >> 172.21.15.2:0/1697031680 conn(0x55a131d40e00 msgr2=0x55a131d462f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:11.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.322+0000 7f6519004740 1 -- 172.21.15.2:0/1697031680 shutdown_connections 2024-05-14T16:57:11.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.322+0000 7f6519004740 1 -- 172.21.15.2:0/1697031680 wait complete. 2024-05-14T16:57:11.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.322+0000 7f6519004740 1 -- start start 2024-05-14T16:57:11.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.322+0000 7f6519004740 1 Processor -- start 2024-05-14T16:57:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.323+0000 7f6519004740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a131d448f0 0x55a131d53600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.323+0000 7f6519004740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a131d45210 0x55a131d53b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.323+0000 7f6519004740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a131d4fad0 0x55a131d54080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.323+0000 7f6519004740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f65101aca30 con 0x55a131d45210 2024-05-14T16:57:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.323+0000 7f6519004740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f65101adea0 con 0x55a131d448f0 2024-05-14T16:57:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.323+0000 7f6519004740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f65101ad100 con 0x55a131d4fad0 2024-05-14T16:57:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.323+0000 7f6519004740 10 client.0 _ll_register_callbacks cb 0x7f65080ca420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:57:11.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.323+0000 7f6507fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a131d45210 0x55a131d53b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.323+0000 7f6507fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a131d45210 0x55a131d53b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32998/0 (socket says 172.21.15.2:32998) 2024-05-14T16:57:11.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.323+0000 7f6507fff640 1 -- 172.21.15.2:0/1909108481 learned_addr learned my addr 172.21.15.2:0/1909108481 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:57:11.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.323+0000 7f650f7fe640 1 --2- 172.21.15.2:0/1909108481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a131d448f0 0x55a131d53600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.323+0000 7f650ffff640 1 --2- 172.21.15.2:0/1909108481 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a131d4fad0 0x55a131d54080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.323+0000 7f6507fff640 1 -- 172.21.15.2:0/1909108481 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a131d4fad0 msgr2=0x55a131d54080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.323+0000 7f6507fff640 1 --2- 172.21.15.2:0/1909108481 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a131d4fad0 0x55a131d54080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.323+0000 7f6507fff640 1 -- 172.21.15.2:0/1909108481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a131d448f0 msgr2=0x55a131d53600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:12.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.323+0000 7f6507fff640 1 --2- 172.21.15.2:0/1909108481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a131d448f0 0x55a131d53600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:12.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.323+0000 7f6507fff640 1 -- 172.21.15.2:0/1909108481 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a131dc8fd0 con 0x55a131d45210 2024-05-14T16:57:12.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.323+0000 7f6507fff640 1 --2- 172.21.15.2:0/1909108481 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a131d45210 0x55a131d53b40 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f64f8011f10 tx=0x7f64f800a870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d136731556d9c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:12.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.323+0000 7f64eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:57:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.324+0000 7f64eeffd640 1 -- 172.21.15.2:0/1909108481 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f8019040 con 0x55a131d45210 2024-05-14T16:57:12.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.324+0000 7f64eeffd640 1 -- 172.21.15.2:0/1909108481 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f64f8011a40 con 0x55a131d45210 2024-05-14T16:57:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.324+0000 7f6519004740 1 -- 172.21.15.2:0/1909108481 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65000277e0 con 0x55a131d45210 2024-05-14T16:57:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.324+0000 7f64eeffd640 1 -- 172.21.15.2:0/1909108481 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f64f8011c80 con 0x55a131d45210 2024-05-14T16:57:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.324+0000 7f6519004740 10 client.15618 fetch_fsmap learned FSMap version 30 2024-05-14T16:57:12.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.324+0000 7f6519004740 1 -- 172.21.15.2:0/1909108481 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a131dca220 con 0x55a131d45210 2024-05-14T16:57:12.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.324+0000 7f64eeffd640 1 -- 172.21.15.2:0/1909108481 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65000277e0 con 0x55a131d45210 2024-05-14T16:57:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.324+0000 7f64eeffd640 1 -- 172.21.15.2:0/1909108481 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f64f8004990 con 0x55a131d45210 2024-05-14T16:57:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.324+0000 7f6519004740 10 client.15618 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T16:57:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.324+0000 7f6519004740 10 client.15618 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:57:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.324+0000 7f6519004740 20 client.15618 populate_metadata read hostname 'smithi002' 2024-05-14T16:57:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.324+0000 7f6519004740 1 --2- 172.21.15.2:0/1909108481 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55a131dca690 0x55a131dccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:12.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.324+0000 7f6519004740 4 client.15618 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:57:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.324+0000 7f6519004740 1 -- 172.21.15.2:0/1909108481 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a131cb3360 con 0x55a131dca690 2024-05-14T16:57:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.325+0000 7f650f7fe640 1 --2- 172.21.15.2:0/1909108481 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55a131dca690 0x55a131dccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:12.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.325+0000 7f650f7fe640 1 --2- 172.21.15.2:0/1909108481 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55a131dca690 0x55a131dccab0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=411988586227bacf server_cookie=b3cc497482a69ecb in_seq=0 out_seq=0 2024-05-14T16:57:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.325+0000 7f64eeffd640 10 client.15618 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:57:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.339+0000 7f64eeffd640 1 -- 172.21.15.2:0/1909108481 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a131cb3360 con 0x55a131dca690 2024-05-14T16:57:12.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.339+0000 7f64eeffd640 10 client.15618 handle_command_reply: tid=0 2024-05-14T16:57:12.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.339+0000 7f6519004740 1 client.15618 shutdown 2024-05-14T16:57:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.340+0000 7f6519004740 1 -- 172.21.15.2:0/1909108481 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a131d45210 msgr2=0x55a131d53b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.340+0000 7f6519004740 1 --2- 172.21.15.2:0/1909108481 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a131d45210 0x55a131d53b40 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f64f8011f10 tx=0x7f64f800a870 comp rx=0 tx=0).stop 2024-05-14T16:57:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.340+0000 7f6519004740 1 -- 172.21.15.2:0/1909108481 shutdown_connections 2024-05-14T16:57:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.340+0000 7f6519004740 1 --2- 172.21.15.2:0/1909108481 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55a131dca690 0x55a131dccab0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:12.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.340+0000 7f6519004740 1 --2- 172.21.15.2:0/1909108481 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a131d4fad0 0x55a131d54080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.340+0000 7f6519004740 1 --2- 172.21.15.2:0/1909108481 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a131d45210 0x55a131d53b40 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.340+0000 7f64eeffd640 0 client.15618 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:57:12.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.340+0000 7f6519004740 1 --2- 172.21.15.2:0/1909108481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a131d448f0 0x55a131d53600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:13.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.340+0000 7f6519004740 1 -- 172.21.15.2:0/1909108481 >> 172.21.15.2:0/1909108481 conn(0x55a131d40e00 msgr2=0x55a131d46b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.340+0000 7f6519004740 1 -- 172.21.15.2:0/1909108481 shutdown_connections 2024-05-14T16:57:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.340+0000 7f6519004740 1 -- 172.21.15.2:0/1909108481 wait complete. 2024-05-14T16:57:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.340+0000 7f6519004740 20 client.15618 trim_cache size 0 max 16384 2024-05-14T16:57:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T16:57:13.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T16:57:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T16:57:13.103 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:57:10 smithi123 ceph-mon[35971]: pgmap v77: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 19 MiB/s wr, 1.62k op/s 2024-05-14T16:57:13.110 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:57:12 smithi123 ceph-mon[35971]: pgmap v78: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 MiB/s wr, 1.45k op/s 2024-05-14T16:57:13.117 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:57:10 smithi151 ceph-mon[36066]: pgmap v77: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 19 MiB/s wr, 1.62k op/s 2024-05-14T16:57:13.147 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:57:12 smithi151 ceph-mon[36066]: pgmap v78: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 MiB/s wr, 1.45k op/s 2024-05-14T16:57:13.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.340+0000 7f6516209640 1 -- 172.21.15.2:0/366843873 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f64fc0b2a10 msgr2=0x7f64fc0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.340+0000 7f6516209640 1 --2- 172.21.15.2:0/366843873 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f64fc0b2a10 0x7f64fc0b4ed0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x55a131d45e90 tx=0x7f650000c430 comp rx=0 tx=0).stop 2024-05-14T16:57:13.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.340+0000 7f6516209640 1 -- 172.21.15.2:0/366843873 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6510107a30 msgr2=0x7f651019da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:13.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.340+0000 7f6516209640 1 --2- 172.21.15.2:0/366843873 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6510107a30 0x7f651019da10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6508005cc0 tx=0x7f6508003ba0 comp rx=0 tx=0).stop 2024-05-14T16:57:13.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.340+0000 7f6516209640 1 -- 172.21.15.2:0/366843873 shutdown_connections 2024-05-14T16:57:13.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.340+0000 7f6516209640 1 --2- 172.21.15.2:0/366843873 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f64fc0b2a10 0x7f64fc0b4ed0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:13.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.340+0000 7f6516209640 1 --2- 172.21.15.2:0/366843873 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6510107a30 0x7f651019da10 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.340+0000 7f6516209640 1 --2- 172.21.15.2:0/366843873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6510101910 0x7f6510199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.340+0000 7f6516209640 1 --2- 172.21.15.2:0/366843873 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6510100f50 0x7f6510199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:13.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.340+0000 7f6516209640 1 -- 172.21.15.2:0/366843873 >> 172.21.15.2:0/366843873 conn(0x7f6510077ed0 msgr2=0x7f6510105860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:13.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.341+0000 7f6516209640 1 -- 172.21.15.2:0/366843873 shutdown_connections 2024-05-14T16:57:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:09.341+0000 7f6516209640 1 -- 172.21.15.2:0/366843873 wait complete. 2024-05-14T16:57:13.316 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:57:10 smithi002 ceph-mon[31538]: pgmap v77: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 19 MiB/s wr, 1.62k op/s 2024-05-14T16:57:13.323 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 115.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:57:13.330 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:57:12 smithi002 ceph-mon[31538]: pgmap v78: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 MiB/s wr, 1.45k op/s 2024-05-14T16:57:14.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:57:14 smithi002 ceph-mon[31538]: pgmap v79: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 18 MiB/s wr, 1.46k op/s 2024-05-14T16:57:14.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:57:14 smithi151 ceph-mon[36066]: pgmap v79: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 18 MiB/s wr, 1.46k op/s 2024-05-14T16:57:14.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:57:14 smithi123 ceph-mon[35971]: pgmap v79: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 18 MiB/s wr, 1.46k op/s 2024-05-14T16:57:16.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:57:16 smithi002 ceph-mon[31538]: pgmap v80: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-05-14T16:57:16.768 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 120.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:57:16.855 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:57:16 smithi151 ceph-mon[36066]: pgmap v80: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-05-14T16:57:16.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:57:16 smithi123 ceph-mon[35971]: pgmap v80: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-05-14T16:57:18.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:57:18 smithi002 ceph-mon[31538]: pgmap v81: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 1.1 MiB/s wr, 96 op/s 2024-05-14T16:57:18.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:57:18 smithi151 ceph-mon[36066]: pgmap v81: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 1.1 MiB/s wr, 96 op/s 2024-05-14T16:57:18.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:57:18 smithi123 ceph-mon[35971]: pgmap v81: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 1.1 MiB/s wr, 96 op/s 2024-05-14T16:57:20.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:57:20 smithi002 ceph-mon[31538]: pgmap v82: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 455 KiB/s wr, 47 op/s 2024-05-14T16:57:20.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:57:20 smithi151 ceph-mon[36066]: pgmap v82: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 455 KiB/s wr, 47 op/s 2024-05-14T16:57:20.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:57:20 smithi123 ceph-mon[35971]: pgmap v82: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 455 KiB/s wr, 47 op/s 2024-05-14T16:57:21.487 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:57:21.751 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 125.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:57:21.760 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:57:22.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.434+0000 7fe4cb146640 1 -- 172.21.15.2:0/3383705285 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4c4074d50 msgr2=0x7fe4c4075130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.434+0000 7fe4cb146640 1 --2- 172.21.15.2:0/3383705285 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4c4074d50 0x7fe4c4075130 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8009900 tx=0x7fe4b802a480 comp rx=0 tx=0).stop 2024-05-14T16:57:22.488 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.435+0000 7fe4cb146640 1 -- 172.21.15.2:0/3383705285 shutdown_connections 2024-05-14T16:57:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.435+0000 7fe4cb146640 1 --2- 172.21.15.2:0/3383705285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4c410f340 0x7fe4c4111730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.435+0000 7fe4cb146640 1 --2- 172.21.15.2:0/3383705285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4c4075670 0x7fe4c410ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:22.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.435+0000 7fe4cb146640 1 --2- 172.21.15.2:0/3383705285 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4c4074d50 0x7fe4c4075130 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:22.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.435+0000 7fe4cb146640 1 -- 172.21.15.2:0/3383705285 >> 172.21.15.2:0/3383705285 conn(0x7fe4c4079ee0 msgr2=0x7fe4c407a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:22.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.435+0000 7fe4cb146640 1 -- 172.21.15.2:0/3383705285 shutdown_connections 2024-05-14T16:57:22.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.435+0000 7fe4cb146640 1 -- 172.21.15.2:0/3383705285 wait complete. 2024-05-14T16:57:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.435+0000 7fe4cb146640 1 Processor -- start 2024-05-14T16:57:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.435+0000 7fe4cb146640 1 -- start start 2024-05-14T16:57:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.436+0000 7fe4cb146640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4c4074d50 0x7fe4c419b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.436+0000 7fe4cb146640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4c4075670 0x7fe4c419b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:22.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.436+0000 7fe4cb146640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4c410f340 0x7fe4c419fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:22.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.436+0000 7fe4cb146640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c4114f20 con 0x7fe4c410f340 2024-05-14T16:57:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.436+0000 7fe4cb146640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c4114da0 con 0x7fe4c4074d50 2024-05-14T16:57:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.436+0000 7fe4cb146640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c41150a0 con 0x7fe4c4075670 2024-05-14T16:57:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.436+0000 7fe4c96e3640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4c410f340 0x7fe4c419fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.436+0000 7fe4c96e3640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4c410f340 0x7fe4c419fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59396/0 (socket says 172.21.15.2:59396) 2024-05-14T16:57:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.436+0000 7fe4c96e3640 1 -- 172.21.15.2:0/39079624 learned_addr learned my addr 172.21.15.2:0/39079624 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:57:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.436+0000 7fe4c8ee2640 1 --2- 172.21.15.2:0/39079624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4c4074d50 0x7fe4c419b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.436+0000 7fe4c3fff640 1 --2- 172.21.15.2:0/39079624 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4c4075670 0x7fe4c419b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.436+0000 7fe4c96e3640 1 -- 172.21.15.2:0/39079624 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4c4075670 msgr2=0x7fe4c419b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:22.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.436+0000 7fe4c96e3640 1 --2- 172.21.15.2:0/39079624 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4c4075670 0x7fe4c419b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.436+0000 7fe4c96e3640 1 -- 172.21.15.2:0/39079624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4c4074d50 msgr2=0x7fe4c419b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:22.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.436+0000 7fe4c96e3640 1 --2- 172.21.15.2:0/39079624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4c4074d50 0x7fe4c419b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:22.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.436+0000 7fe4c96e3640 1 -- 172.21.15.2:0/39079624 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c41a0360 con 0x7fe4c410f340 2024-05-14T16:57:22.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.436+0000 7fe4c96e3640 1 --2- 172.21.15.2:0/39079624 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4c410f340 0x7fe4c419fbe0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc00cab0 tx=0x7fe4bc00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:22.942 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:57:22 smithi123 ceph-mon[35971]: pgmap v83: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 KiB/s wr, 11 op/s 2024-05-14T16:57:22.949 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:57:22 smithi151 ceph-mon[36066]: pgmap v83: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 KiB/s wr, 11 op/s 2024-05-14T16:57:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.438+0000 7fe4c1ffb640 1 -- 172.21.15.2:0/39079624 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc00adc0 con 0x7fe4c410f340 2024-05-14T16:57:22.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.438+0000 7fe4cb146640 1 -- 172.21.15.2:0/39079624 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4c41a0650 con 0x7fe4c410f340 2024-05-14T16:57:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.438+0000 7fe4c1ffb640 1 -- 172.21.15.2:0/39079624 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4bc00f900 con 0x7fe4c410f340 2024-05-14T16:57:22.993 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.438+0000 7fe4c1ffb640 1 -- 172.21.15.2:0/39079624 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc0189a0 con 0x7fe4c410f340 2024-05-14T16:57:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.439+0000 7fe4cb146640 1 -- 172.21.15.2:0/39079624 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4c41a7e90 con 0x7fe4c410f340 2024-05-14T16:57:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.441+0000 7fe4c1ffb640 1 -- 172.21.15.2:0/39079624 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fe4bc00f450 con 0x7fe4c410f340 2024-05-14T16:57:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.441+0000 7fe4cde65740 1 Processor -- start 2024-05-14T16:57:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.441+0000 7fe4cde65740 1 -- start start 2024-05-14T16:57:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.441+0000 7fe4cde65740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645ffdc18f0 0x5645ffdc1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.441+0000 7fe4cde65740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645ffdc2210 0x5645ffdcc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:23.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.441+0000 7fe4cde65740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645ffdccad0 0x5645ffdceec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:23.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.441+0000 7fe4cde65740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5645ffd34c30 con 0x5645ffdccad0 2024-05-14T16:57:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.441+0000 7fe4cde65740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5645ffd247d0 con 0x5645ffdc2210 2024-05-14T16:57:23.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.441+0000 7fe4cde65740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5645ffd28810 con 0x5645ffdc18f0 2024-05-14T16:57:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.441+0000 7fe4c3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645ffdccad0 0x5645ffdceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.442+0000 7fe4c3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645ffdccad0 0x5645ffdceec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59410/0 (socket says 172.21.15.2:59410) 2024-05-14T16:57:23.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.442+0000 7fe4c3fff640 1 -- 172.21.15.2:0/3154916600 learned_addr learned my addr 172.21.15.2:0/3154916600 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:57:23.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.442+0000 7fe4c3fff640 1 --2- 172.21.15.2:0/3154916600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645ffdc18f0 0x5645ffdc1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.442+0000 7fe4c8ee2640 1 --2- 172.21.15.2:0/3154916600 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645ffdc2210 0x5645ffdcc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.442+0000 7fe4c1ffb640 1 --2- 172.21.15.2:0/39079624 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fe4b40b2a10 0x7fe4b40b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:23.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.442+0000 7fe4c3fff640 1 -- 172.21.15.2:0/3154916600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645ffdc18f0 msgr2=0x5645ffdc1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.442+0000 7fe4c3fff640 1 --2- 172.21.15.2:0/3154916600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645ffdc18f0 0x5645ffdc1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:23.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.442+0000 7fe4c3fff640 1 -- 172.21.15.2:0/3154916600 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645ffdc2210 msgr2=0x5645ffdcc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:23.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.442+0000 7fe4c3fff640 1 --2- 172.21.15.2:0/3154916600 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645ffdc2210 0x5645ffdcc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.442+0000 7fe4c3fff640 1 -- 172.21.15.2:0/3154916600 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645ffa39210 con 0x5645ffdccad0 2024-05-14T16:57:23.284 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.442+0000 7fe4c8ee2640 1 --2- 172.21.15.2:0/3154916600 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645ffdc2210 0x5645ffdcc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:57:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.442+0000 7fe4c96e3640 1 --2- 172.21.15.2:0/39079624 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fe4b40b2a10 0x7fe4b40b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.442+0000 7fe4c1ffb640 1 -- 172.21.15.2:0/39079624 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fe4bc0e2800 con 0x7fe4c410f340 2024-05-14T16:57:23.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.442+0000 7fe4c3fff640 1 --2- 172.21.15.2:0/3154916600 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645ffdccad0 0x5645ffdceec0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0012540 tx=0x7fe4b000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c73550a55db11b85 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:23.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.442+0000 7fe4aeffd640 1 -- 172.21.15.2:0/3154916600 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b00106b0 con 0x5645ffdccad0 2024-05-14T16:57:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.442+0000 7fe4aeffd640 1 -- 172.21.15.2:0/3154916600 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4b0010d00 con 0x5645ffdccad0 2024-05-14T16:57:23.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.442+0000 7fe4aeffd640 1 -- 172.21.15.2:0/3154916600 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b001dcb0 con 0x5645ffdccad0 2024-05-14T16:57:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.443+0000 7fe4cde65740 1 -- 172.21.15.2:0/3154916600 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645ffdccad0 msgr2=0x5645ffdceec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:23.383 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.443+0000 7fe4c96e3640 1 --2- 172.21.15.2:0/39079624 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fe4b40b2a10 0x7fe4b40b4ed0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc08ffd0 tx=0x7fe4bc007f60 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.443+0000 7fe4cde65740 1 --2- 172.21.15.2:0/3154916600 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645ffdccad0 0x5645ffdceec0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0012540 tx=0x7fe4b000b870 comp rx=0 tx=0).stop 2024-05-14T16:57:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.443+0000 7fe4cde65740 1 -- 172.21.15.2:0/3154916600 shutdown_connections 2024-05-14T16:57:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.443+0000 7fe4cde65740 1 --2- 172.21.15.2:0/3154916600 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645ffdccad0 0x5645ffdceec0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.443+0000 7fe4cde65740 1 --2- 172.21.15.2:0/3154916600 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645ffdc2210 0x5645ffdcc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.443+0000 7fe4cde65740 1 --2- 172.21.15.2:0/3154916600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645ffdc18f0 0x5645ffdc1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:23.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.443+0000 7fe4cde65740 1 -- 172.21.15.2:0/3154916600 >> 172.21.15.2:0/3154916600 conn(0x5645ffdbde00 msgr2=0x5645ffdbe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:23.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.443+0000 7fe4cde65740 1 -- 172.21.15.2:0/3154916600 shutdown_connections 2024-05-14T16:57:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.443+0000 7fe4cde65740 1 -- 172.21.15.2:0/3154916600 wait complete. 2024-05-14T16:57:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.443+0000 7fe4cde65740 1 -- start start 2024-05-14T16:57:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.443+0000 7fe4cde65740 1 Processor -- start 2024-05-14T16:57:23.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.444+0000 7fe4cde65740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645ffdc18f0 0x5645ffdcc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.444+0000 7fe4cde65740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645ffdc2210 0x5645ffdc96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.444+0000 7fe4cde65740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645ffdccad0 0x5645ffdc9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.444+0000 7fe4cde65740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5645ffa0a790 con 0x5645ffdccad0 2024-05-14T16:57:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.444+0000 7fe4cde65740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5645ffce0890 con 0x5645ffdc2210 2024-05-14T16:57:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.444+0000 7fe4cde65740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5645ffdcf7a0 con 0x5645ffdc18f0 2024-05-14T16:57:23.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.444+0000 7fe4cde65740 10 client.0 _ll_register_callbacks cb 0x5645ffce3d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:57:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.444+0000 7fe4c3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645ffdccad0 0x5645ffdc9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:23.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.444+0000 7fe4c3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645ffdccad0 0x5645ffdc9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59424/0 (socket says 172.21.15.2:59424) 2024-05-14T16:57:23.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.444+0000 7fe4c3fff640 1 -- 172.21.15.2:0/3701226549 learned_addr learned my addr 172.21.15.2:0/3701226549 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:57:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.444+0000 7fe4c8ee2640 1 --2- 172.21.15.2:0/3701226549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645ffdc2210 0x5645ffdc96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.444+0000 7fe4c3fff640 1 --2- 172.21.15.2:0/3701226549 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645ffdc18f0 0x5645ffdcc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.444+0000 7fe4c3fff640 1 -- 172.21.15.2:0/3701226549 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645ffdc18f0 msgr2=0x5645ffdcc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:23.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.444+0000 7fe4c3fff640 1 --2- 172.21.15.2:0/3701226549 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645ffdc18f0 0x5645ffdcc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:23.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.444+0000 7fe4c3fff640 1 -- 172.21.15.2:0/3701226549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645ffdc2210 msgr2=0x5645ffdc96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:23.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.444+0000 7fe4c3fff640 1 --2- 172.21.15.2:0/3701226549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645ffdc2210 0x5645ffdc96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.444+0000 7fe4c3fff640 1 -- 172.21.15.2:0/3701226549 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645ffb05620 con 0x5645ffdccad0 2024-05-14T16:57:23.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.444+0000 7fe4c8ee2640 1 --2- 172.21.15.2:0/3701226549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645ffdc2210 0x5645ffdc96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:57:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.445+0000 7fe4c3fff640 1 --2- 172.21.15.2:0/3701226549 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645ffdccad0 0x5645ffdc9c00 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0013fd0 tx=0x7fe4b001d8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff21d71c10e8d6f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:23.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.445+0000 7fe4a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:57:23.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.445+0000 7fe4a3fff640 1 -- 172.21.15.2:0/3701226549 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b0005250 con 0x5645ffdccad0 2024-05-14T16:57:23.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.445+0000 7fe4a3fff640 1 -- 172.21.15.2:0/3701226549 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fe4b00053f0 con 0x5645ffdccad0 2024-05-14T16:57:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.445+0000 7fe4cde65740 1 -- 172.21.15.2:0/3701226549 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4b0010d00 con 0x5645ffdccad0 2024-05-14T16:57:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.445+0000 7fe4a3fff640 1 -- 172.21.15.2:0/3701226549 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe4b00055f0 con 0x5645ffdccad0 2024-05-14T16:57:23.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.445+0000 7fe4cde65740 10 client.15630 fetch_fsmap learned FSMap version 30 2024-05-14T16:57:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.445+0000 7fe4cde65740 1 -- 172.21.15.2:0/3701226549 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5645ffddb1a0 con 0x5645ffdccad0 2024-05-14T16:57:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.445+0000 7fe4a3fff640 1 -- 172.21.15.2:0/3701226549 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4b0010d00 con 0x5645ffdccad0 2024-05-14T16:57:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.445+0000 7fe4a3fff640 1 -- 172.21.15.2:0/3701226549 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fe4b001c070 con 0x5645ffdccad0 2024-05-14T16:57:23.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.445+0000 7fe4cde65740 10 client.15630 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T16:57:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.445+0000 7fe4cde65740 10 client.15630 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:57:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.445+0000 7fe4cde65740 20 client.15630 populate_metadata read hostname 'smithi002' 2024-05-14T16:57:23.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.445+0000 7fe4cde65740 1 --2- 172.21.15.2:0/3701226549 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5645ffddc550 0x5645ffdde940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:23.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.445+0000 7fe4cde65740 4 client.15630 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:57:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.445+0000 7fe4cde65740 1 -- 172.21.15.2:0/3701226549 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645ffd36b80 con 0x5645ffddc550 2024-05-14T16:57:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.446+0000 7fe4c8ee2640 1 --2- 172.21.15.2:0/3701226549 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5645ffddc550 0x5645ffdde940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.446+0000 7fe4c8ee2640 1 --2- 172.21.15.2:0/3701226549 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5645ffddc550 0x5645ffdde940 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45d453d4d6474726 server_cookie=6685f5192f2655a2 in_seq=0 out_seq=0 2024-05-14T16:57:23.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.446+0000 7fe4a3fff640 10 client.15630 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:57:23.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.447+0000 7fe4a3fff640 1 -- 172.21.15.2:0/3701226549 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5645ffd36b80 con 0x5645ffddc550 2024-05-14T16:57:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.447+0000 7fe4a3fff640 10 client.15630 handle_command_reply: tid=0 2024-05-14T16:57:23.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.447+0000 7fe4cde65740 1 client.15630 shutdown 2024-05-14T16:57:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.447+0000 7fe4cde65740 1 -- 172.21.15.2:0/3701226549 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645ffdccad0 msgr2=0x5645ffdc9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.447+0000 7fe4cde65740 1 --2- 172.21.15.2:0/3701226549 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645ffdccad0 0x5645ffdc9c00 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0013fd0 tx=0x7fe4b001d8f0 comp rx=0 tx=0).stop 2024-05-14T16:57:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.447+0000 7fe4cde65740 1 -- 172.21.15.2:0/3701226549 shutdown_connections 2024-05-14T16:57:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.447+0000 7fe4cde65740 1 --2- 172.21.15.2:0/3701226549 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5645ffddc550 0x5645ffdde940 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.447+0000 7fe4cde65740 1 --2- 172.21.15.2:0/3701226549 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645ffdccad0 0x5645ffdc9c00 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:24.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.447+0000 7fe4cde65740 1 --2- 172.21.15.2:0/3701226549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645ffdc2210 0x5645ffdc96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.447+0000 7fe4cde65740 1 --2- 172.21.15.2:0/3701226549 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645ffdc18f0 0x5645ffdcc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:24.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.447+0000 7fe4cde65740 1 -- 172.21.15.2:0/3701226549 >> 172.21.15.2:0/3701226549 conn(0x5645ffdbde00 msgr2=0x5645ffdbd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:24.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.447+0000 7fe4a3fff640 0 client.15630 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:57:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.447+0000 7fe4cde65740 1 -- 172.21.15.2:0/3701226549 shutdown_connections 2024-05-14T16:57:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.447+0000 7fe4cde65740 1 -- 172.21.15.2:0/3701226549 wait complete. 2024-05-14T16:57:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.447+0000 7fe4cde65740 20 client.15630 trim_cache size 0 max 16384 2024-05-14T16:57:24.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.462+0000 7fe4cde65740 1 Processor -- start 2024-05-14T16:57:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.463+0000 7fe4cde65740 1 -- start start 2024-05-14T16:57:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.463+0000 7fe4cde65740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645ffdc18f0 0x5645ffdd8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:24.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.463+0000 7fe4cde65740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645ffdc2210 0x5645ffde40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:24.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.463+0000 7fe4cde65740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645ffdccad0 0x5645ffde4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:24.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.463+0000 7fe4cde65740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c41b3120 con 0x5645ffdc2210 2024-05-14T16:57:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.463+0000 7fe4cde65740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c41b0960 con 0x5645ffdc18f0 2024-05-14T16:57:24.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.463+0000 7fe4cde65740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c41ae7b0 con 0x5645ffdccad0 2024-05-14T16:57:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.463+0000 7fe4c8ee2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645ffdc2210 0x5645ffde40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.463+0000 7fe4c8ee2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645ffdc2210 0x5645ffde40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59432/0 (socket says 172.21.15.2:59432) 2024-05-14T16:57:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.463+0000 7fe4c8ee2640 1 -- 172.21.15.2:0/2900613372 learned_addr learned my addr 172.21.15.2:0/2900613372 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:57:24.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.463+0000 7fe4c3fff640 1 --2- 172.21.15.2:0/2900613372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645ffdc18f0 0x5645ffdd8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.463+0000 7fe4c3fff640 1 --2- 172.21.15.2:0/2900613372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645ffdccad0 0x5645ffde4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:24.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.463+0000 7fe4c8ee2640 1 -- 172.21.15.2:0/2900613372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645ffdccad0 msgr2=0x5645ffde4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:24.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.463+0000 7fe4c8ee2640 1 --2- 172.21.15.2:0/2900613372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645ffdccad0 0x5645ffde4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:24.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.463+0000 7fe4c8ee2640 1 -- 172.21.15.2:0/2900613372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645ffdc18f0 msgr2=0x5645ffdd8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.463+0000 7fe4c8ee2640 1 --2- 172.21.15.2:0/2900613372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645ffdc18f0 0x5645ffdd8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:24.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.463+0000 7fe4c8ee2640 1 -- 172.21.15.2:0/2900613372 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645ffdc8670 con 0x5645ffdc2210 2024-05-14T16:57:24.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.463+0000 7fe4c3fff640 1 --2- 172.21.15.2:0/2900613372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645ffdccad0 0x5645ffde4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:57:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.463+0000 7fe4c8ee2640 1 --2- 172.21.15.2:0/2900613372 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645ffdc2210 0x5645ffde40d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8009340 tx=0x7fe4b80092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9975832d7e5076d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.464+0000 7fe4ae7fc640 1 -- 172.21.15.2:0/2900613372 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8049070 con 0x5645ffdc2210 2024-05-14T16:57:24.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.464+0000 7fe4ae7fc640 1 -- 172.21.15.2:0/2900613372 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4b80054f0 con 0x5645ffdc2210 2024-05-14T16:57:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.464+0000 7fe4ae7fc640 1 -- 172.21.15.2:0/2900613372 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8035040 con 0x5645ffdc2210 2024-05-14T16:57:24.384 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.465+0000 7fe4cde65740 1 -- 172.21.15.2:0/2900613372 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645ffdc2210 msgr2=0x5645ffde40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.465+0000 7fe4cde65740 1 --2- 172.21.15.2:0/2900613372 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645ffdc2210 0x5645ffde40d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8009340 tx=0x7fe4b80092d0 comp rx=0 tx=0).stop 2024-05-14T16:57:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.465+0000 7fe4cde65740 1 -- 172.21.15.2:0/2900613372 shutdown_connections 2024-05-14T16:57:24.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.465+0000 7fe4cde65740 1 --2- 172.21.15.2:0/2900613372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645ffdccad0 0x5645ffde4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:24.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.465+0000 7fe4cde65740 1 --2- 172.21.15.2:0/2900613372 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645ffdc2210 0x5645ffde40d0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:24.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.465+0000 7fe4cde65740 1 --2- 172.21.15.2:0/2900613372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645ffdc18f0 0x5645ffdd8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:24.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.465+0000 7fe4cde65740 1 -- 172.21.15.2:0/2900613372 >> 172.21.15.2:0/2900613372 conn(0x5645ffdbde00 msgr2=0x5645ffdbd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.465+0000 7fe4cde65740 1 -- 172.21.15.2:0/2900613372 shutdown_connections 2024-05-14T16:57:24.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.465+0000 7fe4cde65740 1 -- 172.21.15.2:0/2900613372 wait complete. 2024-05-14T16:57:24.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.465+0000 7fe4cde65740 1 -- start start 2024-05-14T16:57:24.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.465+0000 7fe4cde65740 1 Processor -- start 2024-05-14T16:57:24.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.466+0000 7fe4cde65740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645ffdc18f0 0x5645ffdd5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.466+0000 7fe4cde65740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645ffdc2210 0x5645ffdd23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.466+0000 7fe4cde65740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645ffdccad0 0x5645ffdd2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:24.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.466+0000 7fe4cde65740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c41aee20 con 0x5645ffdc2210 2024-05-14T16:57:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.466+0000 7fe4cde65740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c41b0290 con 0x5645ffdccad0 2024-05-14T16:57:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.466+0000 7fe4cde65740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c41af4f0 con 0x5645ffdc18f0 2024-05-14T16:57:24.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.466+0000 7fe4cde65740 10 client.0 _ll_register_callbacks cb 0x7fe4b0010eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:57:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.466+0000 7fe4c8ee2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645ffdc2210 0x5645ffdd23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:24.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.466+0000 7fe4c8ee2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645ffdc2210 0x5645ffdd23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59446/0 (socket says 172.21.15.2:59446) 2024-05-14T16:57:24.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.466+0000 7fe4c8ee2640 1 -- 172.21.15.2:0/2168374713 learned_addr learned my addr 172.21.15.2:0/2168374713 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:57:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.466+0000 7fe4c3fff640 1 --2- 172.21.15.2:0/2168374713 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645ffdc18f0 0x5645ffdd5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:24.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.466+0000 7fe4c3fff640 1 --2- 172.21.15.2:0/2168374713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645ffdccad0 0x5645ffdd2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.466+0000 7fe4c8ee2640 1 -- 172.21.15.2:0/2168374713 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645ffdc18f0 msgr2=0x5645ffdd5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.466+0000 7fe4c8ee2640 1 --2- 172.21.15.2:0/2168374713 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645ffdc18f0 0x5645ffdd5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.466+0000 7fe4c8ee2640 1 -- 172.21.15.2:0/2168374713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645ffdccad0 msgr2=0x5645ffdd2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.466+0000 7fe4c8ee2640 1 --2- 172.21.15.2:0/2168374713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645ffdccad0 0x5645ffdd2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.466+0000 7fe4c8ee2640 1 -- 172.21.15.2:0/2168374713 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645ffdc2040 con 0x5645ffdc2210 2024-05-14T16:57:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.466+0000 7fe4c3fff640 1 --2- 172.21.15.2:0/2168374713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645ffdccad0 0x5645ffdd2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:57:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.466+0000 7fe4c8ee2640 1 --2- 172.21.15.2:0/2168374713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645ffdc2210 0x5645ffdd23f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8009230 tx=0x7fe4b80056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a727ee1140603c90 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:24.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.466+0000 7fe4a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:57:24.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.467+0000 7fe4a3fff640 1 -- 172.21.15.2:0/2168374713 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8035070 con 0x5645ffdc2210 2024-05-14T16:57:24.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.467+0000 7fe4a3fff640 1 -- 172.21.15.2:0/2168374713 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fe4b80443d0 con 0x5645ffdc2210 2024-05-14T16:57:24.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.467+0000 7fe4cde65740 1 -- 172.21.15.2:0/2168374713 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4b80054f0 con 0x5645ffdc2210 2024-05-14T16:57:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.467+0000 7fe4a3fff640 1 -- 172.21.15.2:0/2168374713 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe4b8049040 con 0x5645ffdc2210 2024-05-14T16:57:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.467+0000 7fe4cde65740 10 client.15636 fetch_fsmap learned FSMap version 30 2024-05-14T16:57:24.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.467+0000 7fe4cde65740 1 -- 172.21.15.2:0/2168374713 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5645ffe46b80 con 0x5645ffdc2210 2024-05-14T16:57:24.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.467+0000 7fe4a3fff640 1 -- 172.21.15.2:0/2168374713 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4b80054f0 con 0x5645ffdc2210 2024-05-14T16:57:24.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.467+0000 7fe4a3fff640 1 -- 172.21.15.2:0/2168374713 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fe4b804c070 con 0x5645ffdc2210 2024-05-14T16:57:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.467+0000 7fe4cde65740 10 client.15636 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T16:57:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.467+0000 7fe4cde65740 10 client.15636 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:57:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.467+0000 7fe4cde65740 20 client.15636 populate_metadata read hostname 'smithi002' 2024-05-14T16:57:24.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.467+0000 7fe4cde65740 1 --2- 172.21.15.2:0/2168374713 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5645ffe47290 0x5645ffe496b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:24.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.467+0000 7fe4cde65740 4 client.15636 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:57:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.467+0000 7fe4cde65740 1 -- 172.21.15.2:0/2168374713 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5645ffd30360 con 0x5645ffe47290 2024-05-14T16:57:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.468+0000 7fe4c3fff640 1 --2- 172.21.15.2:0/2168374713 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5645ffe47290 0x5645ffe496b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.468+0000 7fe4c3fff640 1 --2- 172.21.15.2:0/2168374713 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5645ffe47290 0x5645ffe496b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fadb9c92e4a434e server_cookie=3c5b54192e701d15 in_seq=0 out_seq=0 2024-05-14T16:57:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.468+0000 7fe4a3fff640 10 client.15636 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:57:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.481+0000 7fe4a3fff640 1 -- 172.21.15.2:0/2168374713 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5645ffd30360 con 0x5645ffe47290 2024-05-14T16:57:24.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.481+0000 7fe4a3fff640 10 client.15636 handle_command_reply: tid=0 2024-05-14T16:57:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.481+0000 7fe4cde65740 1 client.15636 shutdown 2024-05-14T16:57:24.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.481+0000 7fe4cde65740 1 -- 172.21.15.2:0/2168374713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645ffdc2210 msgr2=0x5645ffdd23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.481+0000 7fe4cde65740 1 --2- 172.21.15.2:0/2168374713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645ffdc2210 0x5645ffdd23f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8009230 tx=0x7fe4b80056e0 comp rx=0 tx=0).stop 2024-05-14T16:57:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.481+0000 7fe4cde65740 1 -- 172.21.15.2:0/2168374713 shutdown_connections 2024-05-14T16:57:25.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.481+0000 7fe4cde65740 1 --2- 172.21.15.2:0/2168374713 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5645ffe47290 0x5645ffe496b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.481+0000 7fe4cde65740 1 --2- 172.21.15.2:0/2168374713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645ffdccad0 0x5645ffdd2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.481+0000 7fe4a3fff640 0 client.15636 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:57:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.481+0000 7fe4cde65740 1 --2- 172.21.15.2:0/2168374713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5645ffdc2210 0x5645ffdd23f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.481+0000 7fe4cde65740 1 --2- 172.21.15.2:0/2168374713 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5645ffdc18f0 0x5645ffdd5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.481+0000 7fe4cde65740 1 -- 172.21.15.2:0/2168374713 >> 172.21.15.2:0/2168374713 conn(0x5645ffdbde00 msgr2=0x5645ffdd7660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:25.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.481+0000 7fe4cde65740 1 -- 172.21.15.2:0/2168374713 shutdown_connections 2024-05-14T16:57:25.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.481+0000 7fe4cde65740 1 -- 172.21.15.2:0/2168374713 wait complete. 2024-05-14T16:57:25.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.481+0000 7fe4cde65740 20 client.15636 trim_cache size 0 max 16384 2024-05-14T16:57:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T16:57:25.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T16:57:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T16:57:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.482+0000 7fe4cb146640 1 -- 172.21.15.2:0/39079624 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fe4b40b2a10 msgr2=0x7fe4b40b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.482+0000 7fe4cb146640 1 --2- 172.21.15.2:0/39079624 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fe4b40b2a10 0x7fe4b40b4ed0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc08ffd0 tx=0x7fe4bc007f60 comp rx=0 tx=0).stop 2024-05-14T16:57:25.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.482+0000 7fe4cb146640 1 -- 172.21.15.2:0/39079624 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4c410f340 msgr2=0x7fe4c419fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:25.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.482+0000 7fe4cb146640 1 --2- 172.21.15.2:0/39079624 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4c410f340 0x7fe4c419fbe0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc00cab0 tx=0x7fe4bc00cf70 comp rx=0 tx=0).stop 2024-05-14T16:57:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.482+0000 7fe4cb146640 1 -- 172.21.15.2:0/39079624 shutdown_connections 2024-05-14T16:57:25.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.482+0000 7fe4cb146640 1 --2- 172.21.15.2:0/39079624 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fe4b40b2a10 0x7fe4b40b4ed0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.482+0000 7fe4cb146640 1 --2- 172.21.15.2:0/39079624 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4c410f340 0x7fe4c419fbe0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.482+0000 7fe4cb146640 1 --2- 172.21.15.2:0/39079624 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4c4075670 0x7fe4c419b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.482+0000 7fe4cb146640 1 --2- 172.21.15.2:0/39079624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4c4074d50 0x7fe4c419b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:25.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.482+0000 7fe4cb146640 1 -- 172.21.15.2:0/39079624 >> 172.21.15.2:0/39079624 conn(0x7fe4c4079ee0 msgr2=0x7fe4c40ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:25.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.482+0000 7fe4cb146640 1 -- 172.21.15.2:0/39079624 shutdown_connections 2024-05-14T16:57:25.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:22.482+0000 7fe4cb146640 1 -- 172.21.15.2:0/39079624 wait complete. 2024-05-14T16:57:25.387 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:57:22 smithi002 ceph-mon[31538]: pgmap v83: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 KiB/s wr, 11 op/s 2024-05-14T16:57:25.398 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:57:24 smithi151 ceph-mon[36066]: pgmap v84: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-05-14T16:57:25.435 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:57:24 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/3441936035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:57:25.443 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:57:24 smithi123 ceph-mon[35971]: pgmap v84: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-05-14T16:57:25.451 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:57:24 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/3441936035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:57:25.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.489+0000 7fae5665a700 1 -- 172.21.15.2:0/1386682675 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae501088f0 msgr2=0x7fae50110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:25.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.489+0000 7fae5665a700 1 --2- 172.21.15.2:0/1386682675 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae501088f0 0x7fae50110470 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fae4800b0d0 tx=0x7fae4800b3e0 comp rx=0 tx=0).stop 2024-05-14T16:57:25.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.489+0000 7fae5665a700 1 -- 172.21.15.2:0/1386682675 shutdown_connections 2024-05-14T16:57:25.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.489+0000 7fae5665a700 1 --2- 172.21.15.2:0/1386682675 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae501088f0 0x7fae50110470 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:25.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.489+0000 7fae5665a700 1 --2- 172.21.15.2:0/1386682675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae501037f0 0x7fae50103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:25.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.489+0000 7fae5665a700 1 --2- 172.21.15.2:0/1386682675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae501041a0 0x7fae501081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:25.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.489+0000 7fae5665a700 1 -- 172.21.15.2:0/1386682675 >> 172.21.15.2:0/1386682675 conn(0x7fae5007a6a0 msgr2=0x7fae5007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:25.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.489+0000 7fae5665a700 1 -- 172.21.15.2:0/1386682675 shutdown_connections 2024-05-14T16:57:25.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.489+0000 7fae5665a700 1 -- 172.21.15.2:0/1386682675 wait complete. 2024-05-14T16:57:25.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.490+0000 7fae5665a700 1 Processor -- start 2024-05-14T16:57:25.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.490+0000 7fae5665a700 1 -- start start 2024-05-14T16:57:25.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.490+0000 7fae5665a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae501037f0 0x7fae50075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:25.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.490+0000 7fae5665a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae501041a0 0x7fae50075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:25.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.490+0000 7fae5665a700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae501088f0 0x7fae500762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:25.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.490+0000 7fae5665a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fae501136c0 con 0x7fae501041a0 2024-05-14T16:57:25.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.490+0000 7fae5665a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fae50113540 con 0x7fae501037f0 2024-05-14T16:57:25.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.490+0000 7fae5665a700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fae50113840 con 0x7fae501088f0 2024-05-14T16:57:25.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.490+0000 7fae47fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae501041a0 0x7fae50075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:25.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.490+0000 7fae47fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae501041a0 0x7fae50075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59462/0 (socket says 172.21.15.2:59462) 2024-05-14T16:57:25.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.490+0000 7fae47fff700 1 -- 172.21.15.2:0/3441936035 learned_addr learned my addr 172.21.15.2:0/3441936035 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:57:25.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.490+0000 7fae4ffff700 1 --2- 172.21.15.2:0/3441936035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae501037f0 0x7fae50075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:25.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.490+0000 7fae54bf7700 1 --2- 172.21.15.2:0/3441936035 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae501088f0 0x7fae500762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:25.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.491+0000 7fae47fff700 1 -- 172.21.15.2:0/3441936035 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae501088f0 msgr2=0x7fae500762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:25.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.491+0000 7fae47fff700 1 --2- 172.21.15.2:0/3441936035 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae501088f0 0x7fae500762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:25.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.491+0000 7fae47fff700 1 -- 172.21.15.2:0/3441936035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae501037f0 msgr2=0x7fae50075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:25.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.491+0000 7fae47fff700 1 --2- 172.21.15.2:0/3441936035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae501037f0 0x7fae50075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:25.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.491+0000 7fae47fff700 1 -- 172.21.15.2:0/3441936035 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae501a7fe0 con 0x7fae501041a0 2024-05-14T16:57:25.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.491+0000 7fae47fff700 1 --2- 172.21.15.2:0/3441936035 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae501041a0 0x7fae50075d80 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fae3800d950 tx=0x7fae3800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:25.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.491+0000 7fae4dffb700 1 -- 172.21.15.2:0/3441936035 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae38016070 con 0x7fae501041a0 2024-05-14T16:57:25.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.491+0000 7fae4dffb700 1 -- 172.21.15.2:0/3441936035 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae3800fd40 con 0x7fae501041a0 2024-05-14T16:57:25.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.491+0000 7fae5665a700 1 -- 172.21.15.2:0/3441936035 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae501a82d0 con 0x7fae501041a0 2024-05-14T16:57:25.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.491+0000 7fae4dffb700 1 -- 172.21.15.2:0/3441936035 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3800e3b0 con 0x7fae501041a0 2024-05-14T16:57:25.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.491+0000 7fae5665a700 1 -- 172.21.15.2:0/3441936035 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae501a8810 con 0x7fae501041a0 2024-05-14T16:57:25.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.492+0000 7fae5665a700 1 -- 172.21.15.2:0/3441936035 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae5006a230 con 0x7fae501041a0 2024-05-14T16:57:25.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.493+0000 7fae4dffb700 1 -- 172.21.15.2:0/3441936035 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fae38019070 con 0x7fae501041a0 2024-05-14T16:57:25.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.494+0000 7fae4dffb700 1 --2- 172.21.15.2:0/3441936035 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fae3c0b29f0 0x7fae3c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:25.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.494+0000 7fae4dffb700 1 -- 172.21.15.2:0/3441936035 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fae380e1fb0 con 0x7fae501041a0 2024-05-14T16:57:25.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.494+0000 7fae4ffff700 1 --2- 172.21.15.2:0/3441936035 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fae3c0b29f0 0x7fae3c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:25.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.495+0000 7fae4ffff700 1 --2- 172.21.15.2:0/3441936035 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fae3c0b29f0 0x7fae3c0b4eb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fae4000ac50 tx=0x7fae4000a380 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:25.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.497+0000 7fae4dffb700 1 -- 172.21.15.2:0/3441936035 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fae380a99c0 con 0x7fae501041a0 2024-05-14T16:57:25.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.808+0000 7fae5665a700 1 -- 172.21.15.2:0/3441936035 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae5002e720 con 0x7fae501041a0 2024-05-14T16:57:25.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.810+0000 7fae4dffb700 1 -- 172.21.15.2:0/3441936035 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11857 (secure 0 0 0) 0x7fae380a92c0 con 0x7fae501041a0 2024-05-14T16:57:25.932 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:57:25.939 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2275696923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2275696923},{"type":"v1","addr":"172.21.15.2:6835","nonce":2275696923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/716718761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":716718761},{"type":"v1","addr":"172.21.15.2:6839","nonce":716718761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/3644815179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3644815179},{"type":"v1","addr":"172.21.15.2:6841","nonce":3644815179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1880336731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1880336731},{"type":"v1","addr":"172.21.15.151:6833","nonce":1880336731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2415605548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2415605548},{"type":"v1","addr":"172.21.15.2:6837","nonce":2415605548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2635167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2635167},{"type":"v1","addr":"172.21.15.123:6833","nonce":2635167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:51:34.106919+0000","modified":"2024-05-14T16:56:06.904708+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24455,"mds_2":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24415":{"gid":24415,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/762044339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":762044339},{"type":"v1","addr":"172.21.15.151:6835","nonce":762044339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/132043151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":132043151},{"type":"v1","addr":"172.21.15.151:6837","nonce":132043151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/4107834375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":4107834375},{"type":"v1","addr":"172.21.15.151:6839","nonce":4107834375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.123:6835/1198670013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1198670013},{"type":"v1","addr":"172.21.15.123:6835","nonce":1198670013}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/333426645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":333426645},{"type":"v1","addr":"172.21.15.123:6837","nonce":333426645}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/580275877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":580275877},{"type":"v1","addr":"172.21.15.123:6839","nonce":580275877}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24455,24479,24431]},"id":1}]} 2024-05-14T16:57:25.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.814+0000 7fae5665a700 1 -- 172.21.15.2:0/3441936035 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fae3c0b29f0 msgr2=0x7fae3c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:25.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.814+0000 7fae5665a700 1 --2- 172.21.15.2:0/3441936035 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fae3c0b29f0 0x7fae3c0b4eb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fae4000ac50 tx=0x7fae4000a380 comp rx=0 tx=0).stop 2024-05-14T16:57:25.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.814+0000 7fae5665a700 1 -- 172.21.15.2:0/3441936035 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae501041a0 msgr2=0x7fae50075d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:25.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.814+0000 7fae5665a700 1 --2- 172.21.15.2:0/3441936035 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae501041a0 0x7fae50075d80 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fae3800d950 tx=0x7fae3800dc60 comp rx=0 tx=0).stop 2024-05-14T16:57:25.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.814+0000 7fae5665a700 1 -- 172.21.15.2:0/3441936035 shutdown_connections 2024-05-14T16:57:26.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.814+0000 7fae5665a700 1 --2- 172.21.15.2:0/3441936035 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fae3c0b29f0 0x7fae3c0b4eb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:26.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.814+0000 7fae5665a700 1 --2- 172.21.15.2:0/3441936035 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae501088f0 0x7fae500762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:26.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.814+0000 7fae5665a700 1 --2- 172.21.15.2:0/3441936035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae501037f0 0x7fae50075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:26.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.814+0000 7fae5665a700 1 --2- 172.21.15.2:0/3441936035 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae501041a0 0x7fae50075d80 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:26.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.815+0000 7fae5665a700 1 -- 172.21.15.2:0/3441936035 >> 172.21.15.2:0/3441936035 conn(0x7fae5007a6a0 msgr2=0x7fae501015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:26.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.815+0000 7fae5665a700 1 -- 172.21.15.2:0/3441936035 shutdown_connections 2024-05-14T16:57:26.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:23.815+0000 7fae5665a700 1 -- 172.21.15.2:0/3441936035 wait complete. 2024-05-14T16:57:26.049 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 30 2024-05-14T16:57:26.056 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id ab7ccb52 --release --await' 2024-05-14T16:57:26.096 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id ab7ccb52 --release --await 2024-05-14T16:57:26.104 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:57:24 smithi002 ceph-mon[31538]: pgmap v84: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-05-14T16:57:26.110 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:57:24 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/3441936035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:57:26.370 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:57:26.730 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:57:26 smithi002 ceph-mon[31538]: pgmap v85: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 25 KiB/s wr, 8 op/s 2024-05-14T16:57:26.751 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 130.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:57:26.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:57:26 smithi151 ceph-mon[36066]: pgmap v85: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 25 KiB/s wr, 8 op/s 2024-05-14T16:57:26.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:57:26 smithi123 ceph-mon[35971]: pgmap v85: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 25 KiB/s wr, 8 op/s 2024-05-14T16:57:28.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.205+0000 7fbd5e7a2700 1 -- 172.21.15.2:0/8589617 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd58103fa0 msgr2=0x7fbd58110450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:28.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.205+0000 7fbd5e7a2700 1 --2- 172.21.15.2:0/8589617 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd58103fa0 0x7fbd58110450 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000b0d0 tx=0x7fbd5000b3e0 comp rx=0 tx=0).stop 2024-05-14T16:57:28.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.205+0000 7fbd5e7a2700 1 -- 172.21.15.2:0/8589617 shutdown_connections 2024-05-14T16:57:28.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.205+0000 7fbd5e7a2700 1 --2- 172.21.15.2:0/8589617 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd58103fa0 0x7fbd58110450 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:28.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.205+0000 7fbd5e7a2700 1 --2- 172.21.15.2:0/8589617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd58109680 0x7fbd58109a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:28.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.205+0000 7fbd5e7a2700 1 --2- 172.21.15.2:0/8589617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd581035e0 0x7fbd58103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:28.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.205+0000 7fbd5e7a2700 1 -- 172.21.15.2:0/8589617 >> 172.21.15.2:0/8589617 conn(0x7fbd5807a6f0 msgr2=0x7fbd5807ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:28.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.205+0000 7fbd5e7a2700 1 -- 172.21.15.2:0/8589617 shutdown_connections 2024-05-14T16:57:28.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.205+0000 7fbd5e7a2700 1 -- 172.21.15.2:0/8589617 wait complete. 2024-05-14T16:57:28.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.206+0000 7fbd5e7a2700 1 Processor -- start 2024-05-14T16:57:28.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.206+0000 7fbd5e7a2700 1 -- start start 2024-05-14T16:57:28.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.207+0000 7fbd5e7a2700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd581035e0 0x7fbd5819f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:28.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.207+0000 7fbd5e7a2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd58103fa0 0x7fbd5819fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:28.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.207+0000 7fbd5e7a2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd58109680 0x7fbd58199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:28.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.207+0000 7fbd5e7a2700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbd58113830 con 0x7fbd58109680 2024-05-14T16:57:28.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.207+0000 7fbd5e7a2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbd581136b0 con 0x7fbd58103fa0 2024-05-14T16:57:28.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.207+0000 7fbd5e7a2700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbd581139b0 con 0x7fbd581035e0 2024-05-14T16:57:28.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.207+0000 7fbd5cd3f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd58109680 0x7fbd58199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:28.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.207+0000 7fbd5cd3f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd58109680 0x7fbd58199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49088/0 (socket says 172.21.15.2:49088) 2024-05-14T16:57:28.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.207+0000 7fbd5cd3f700 1 -- 172.21.15.2:0/3886099924 learned_addr learned my addr 172.21.15.2:0/3886099924 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:57:28.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.207+0000 7fbd577fe700 1 --2- 172.21.15.2:0/3886099924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd58103fa0 0x7fbd5819fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:28.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.207+0000 7fbd57fff700 1 --2- 172.21.15.2:0/3886099924 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd581035e0 0x7fbd5819f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:28.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.207+0000 7fbd5cd3f700 1 -- 172.21.15.2:0/3886099924 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd581035e0 msgr2=0x7fbd5819f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:28.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.207+0000 7fbd5cd3f700 1 --2- 172.21.15.2:0/3886099924 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd581035e0 0x7fbd5819f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:28.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.207+0000 7fbd5cd3f700 1 -- 172.21.15.2:0/3886099924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd58103fa0 msgr2=0x7fbd5819fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:28.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.207+0000 7fbd5cd3f700 1 --2- 172.21.15.2:0/3886099924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd58103fa0 0x7fbd5819fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:28.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.207+0000 7fbd5cd3f700 1 -- 172.21.15.2:0/3886099924 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd5819a230 con 0x7fbd58109680 2024-05-14T16:57:28.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.207+0000 7fbd5cd3f700 1 --2- 172.21.15.2:0/3886099924 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd58109680 0x7fbd58199ad0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000d010 tx=0x7fbd500079a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:28.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.207+0000 7fbd557fa700 1 -- 172.21.15.2:0/3886099924 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd5001f070 con 0x7fbd58109680 2024-05-14T16:57:28.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.208+0000 7fbd5e7a2700 1 -- 172.21.15.2:0/3886099924 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd5819a4c0 con 0x7fbd58109680 2024-05-14T16:57:28.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.208+0000 7fbd557fa700 1 -- 172.21.15.2:0/3886099924 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd50007e10 con 0x7fbd58109680 2024-05-14T16:57:28.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.208+0000 7fbd557fa700 1 -- 172.21.15.2:0/3886099924 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd5001b480 con 0x7fbd58109680 2024-05-14T16:57:28.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.208+0000 7fbd5e7a2700 1 -- 172.21.15.2:0/3886099924 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd581a67e0 con 0x7fbd58109680 2024-05-14T16:57:28.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.210+0000 7fbd557fa700 1 -- 172.21.15.2:0/3886099924 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fbd50022070 con 0x7fbd58109680 2024-05-14T16:57:28.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.211+0000 7fbd557fa700 1 --2- 172.21.15.2:0/3886099924 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fbd400b29f0 0x7fbd400b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:28.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.211+0000 7fbd57fff700 1 --2- 172.21.15.2:0/3886099924 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fbd400b29f0 0x7fbd400b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:28.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.211+0000 7fbd557fa700 1 -- 172.21.15.2:0/3886099924 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fbd5000f020 con 0x7fbd58109680 2024-05-14T16:57:28.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.211+0000 7fbd69038b80 1 Processor -- start 2024-05-14T16:57:28.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.211+0000 7fbd57fff700 1 --2- 172.21.15.2:0/3886099924 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fbd400b29f0 0x7fbd400b4eb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbd48009a10 tx=0x7fbd48009190 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:28.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.211+0000 7fbd69038b80 1 -- start start 2024-05-14T16:57:28.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.212+0000 7fbd69038b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28836f420 0x55b28836f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:28.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.212+0000 7fbd69038b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b28836fd40 0x55b28836aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:28.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.212+0000 7fbd69038b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b28836b2c0 0x55b28836b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:28.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.212+0000 7fbd69038b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b2882c8e70 con 0x55b28836fd40 2024-05-14T16:57:28.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.212+0000 7fbd69038b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b2882b8a10 con 0x55b28836b2c0 2024-05-14T16:57:28.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.212+0000 7fbd69038b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b2882bca50 con 0x55b28836f420 2024-05-14T16:57:28.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.212+0000 7fbd577fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b28836fd40 0x55b28836aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:28.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.212+0000 7fbd577fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b28836fd40 0x55b28836aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49098/0 (socket says 172.21.15.2:49098) 2024-05-14T16:57:28.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.212+0000 7fbd577fe700 1 -- 172.21.15.2:0/2125658892 learned_addr learned my addr 172.21.15.2:0/2125658892 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:57:28.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.212+0000 7fbd57fff700 1 --2- 172.21.15.2:0/2125658892 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28836f420 0x55b28836f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:28.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.212+0000 7fbd5cd3f700 1 --2- 172.21.15.2:0/2125658892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b28836b2c0 0x55b28836b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:28.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.212+0000 7fbd577fe700 1 -- 172.21.15.2:0/2125658892 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28836f420 msgr2=0x55b28836f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:28.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.212+0000 7fbd577fe700 1 --2- 172.21.15.2:0/2125658892 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28836f420 0x55b28836f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:28.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.212+0000 7fbd577fe700 1 -- 172.21.15.2:0/2125658892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b28836b2c0 msgr2=0x55b28836b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:28.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.212+0000 7fbd577fe700 1 --2- 172.21.15.2:0/2125658892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b28836b2c0 0x55b28836b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:28.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.212+0000 7fbd577fe700 1 -- 172.21.15.2:0/2125658892 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2882620b0 con 0x55b28836fd40 2024-05-14T16:57:28.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.212+0000 7fbd57fff700 1 --2- 172.21.15.2:0/2125658892 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28836f420 0x55b28836f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:57:28.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.212+0000 7fbd577fe700 1 --2- 172.21.15.2:0/2125658892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b28836fd40 0x55b28836aca0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00bd30 tx=0x7fbd4c0106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=614a7094b7223477 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:28.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.212+0000 7fbd3e7fc700 1 -- 172.21.15.2:0/2125658892 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c004030 con 0x55b28836fd40 2024-05-14T16:57:28.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.213+0000 7fbd3e7fc700 1 -- 172.21.15.2:0/2125658892 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd4c003680 con 0x55b28836fd40 2024-05-14T16:57:28.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.213+0000 7fbd3e7fc700 1 -- 172.21.15.2:0/2125658892 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c0116a0 con 0x55b28836fd40 2024-05-14T16:57:28.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.213+0000 7fbd69038b80 1 -- 172.21.15.2:0/2125658892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b28836fd40 msgr2=0x55b28836aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:28.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.213+0000 7fbd69038b80 1 --2- 172.21.15.2:0/2125658892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b28836fd40 0x55b28836aca0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00bd30 tx=0x7fbd4c0106b0 comp rx=0 tx=0).stop 2024-05-14T16:57:28.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.213+0000 7fbd69038b80 1 -- 172.21.15.2:0/2125658892 shutdown_connections 2024-05-14T16:57:29.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.213+0000 7fbd69038b80 1 --2- 172.21.15.2:0/2125658892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b28836b2c0 0x55b28836b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:29.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.213+0000 7fbd69038b80 1 --2- 172.21.15.2:0/2125658892 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28836f420 0x55b28836f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:29.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.213+0000 7fbd69038b80 1 --2- 172.21.15.2:0/2125658892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b28836fd40 0x55b28836aca0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:29.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.213+0000 7fbd69038b80 1 -- 172.21.15.2:0/2125658892 >> 172.21.15.2:0/2125658892 conn(0x55b288297eb0 msgr2=0x55b2883654a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:29.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.213+0000 7fbd69038b80 1 -- 172.21.15.2:0/2125658892 shutdown_connections 2024-05-14T16:57:29.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.213+0000 7fbd69038b80 1 -- 172.21.15.2:0/2125658892 wait complete. 2024-05-14T16:57:29.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.214+0000 7fbd69038b80 1 -- start start 2024-05-14T16:57:29.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.214+0000 7fbd69038b80 1 Processor -- start 2024-05-14T16:57:29.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.214+0000 7fbd69038b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b28836b2c0 0x55b288376b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:29.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.214+0000 7fbd69038b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28836f420 0x55b288373ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:29.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.214+0000 7fbd69038b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b28836fd40 0x55b288374430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:29.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.214+0000 7fbd69038b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b2881bf0a0 con 0x55b28836fd40 2024-05-14T16:57:29.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.214+0000 7fbd69038b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b288220ea0 con 0x55b28836b2c0 2024-05-14T16:57:29.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.214+0000 7fbd69038b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b288377880 con 0x55b28836f420 2024-05-14T16:57:29.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.214+0000 7fbd69038b80 10 client.0 _ll_register_callbacks cb 0x55b288253a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:57:29.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.214+0000 7fbd5cd3f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b28836fd40 0x55b288374430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:29.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.214+0000 7fbd5cd3f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b28836fd40 0x55b288374430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49104/0 (socket says 172.21.15.2:49104) 2024-05-14T16:57:29.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.214+0000 7fbd5cd3f700 1 -- 172.21.15.2:0/3553524503 learned_addr learned my addr 172.21.15.2:0/3553524503 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:57:29.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.214+0000 7fbd57fff700 1 --2- 172.21.15.2:0/3553524503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b28836b2c0 0x55b288376b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:29.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.215+0000 7fbd577fe700 1 --2- 172.21.15.2:0/3553524503 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28836f420 0x55b288373ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:29.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.215+0000 7fbd5cd3f700 1 -- 172.21.15.2:0/3553524503 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28836f420 msgr2=0x55b288373ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:29.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.215+0000 7fbd5cd3f700 1 --2- 172.21.15.2:0/3553524503 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28836f420 0x55b288373ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:29.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.215+0000 7fbd5cd3f700 1 -- 172.21.15.2:0/3553524503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b28836b2c0 msgr2=0x55b288376b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:29.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.215+0000 7fbd5cd3f700 1 --2- 172.21.15.2:0/3553524503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b28836b2c0 0x55b288376b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:29.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.215+0000 7fbd5cd3f700 1 -- 172.21.15.2:0/3553524503 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2882089d0 con 0x55b28836fd40 2024-05-14T16:57:29.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.215+0000 7fbd577fe700 1 --2- 172.21.15.2:0/3553524503 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28836f420 0x55b288373ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:57:29.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.215+0000 7fbd5cd3f700 1 --2- 172.21.15.2:0/3553524503 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b28836fd40 0x55b288374430 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fbd500e1500 tx=0x7fbd500aa4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4701b32a652fa4d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:29.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.215+0000 7fbd377fe700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:57:29.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.215+0000 7fbd377fe700 1 -- 172.21.15.2:0/3553524503 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd500b3230 con 0x55b28836fd40 2024-05-14T16:57:29.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.215+0000 7fbd377fe700 1 -- 172.21.15.2:0/3553524503 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fbd500aad80 con 0x55b28836fd40 2024-05-14T16:57:29.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.215+0000 7fbd69038b80 1 -- 172.21.15.2:0/3553524503 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd4c003680 con 0x55b28836fd40 2024-05-14T16:57:29.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.215+0000 7fbd377fe700 1 -- 172.21.15.2:0/3553524503 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbd500aa630 con 0x55b28836fd40 2024-05-14T16:57:29.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.215+0000 7fbd69038b80 10 client.15654 fetch_fsmap learned FSMap version 30 2024-05-14T16:57:29.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.215+0000 7fbd69038b80 1 -- 172.21.15.2:0/3553524503 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b288272860 con 0x55b28836fd40 2024-05-14T16:57:29.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.215+0000 7fbd377fe700 1 -- 172.21.15.2:0/3553524503 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd4c003680 con 0x55b28836fd40 2024-05-14T16:57:29.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.216+0000 7fbd377fe700 1 -- 172.21.15.2:0/3553524503 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fbd50011070 con 0x55b28836fd40 2024-05-14T16:57:29.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.216+0000 7fbd69038b80 10 client.15654 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T16:57:29.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.216+0000 7fbd69038b80 10 client.15654 resolve_mds: validated gid 24479 aka daemon mds.b 2024-05-14T16:57:29.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.216+0000 7fbd69038b80 20 client.15654 populate_metadata read hostname 'smithi002' 2024-05-14T16:57:29.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.216+0000 7fbd69038b80 1 --2- 172.21.15.2:0/3553524503 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55b2883843c0 0x55b2883867e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:29.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.216+0000 7fbd69038b80 4 client.15654 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:57:29.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.216+0000 7fbd69038b80 1 -- 172.21.15.2:0/3553524503 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2882cadc0 con 0x55b2883843c0 2024-05-14T16:57:29.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.216+0000 7fbd57fff700 1 --2- 172.21.15.2:0/3553524503 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55b2883843c0 0x55b2883867e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:29.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.217+0000 7fbd377fe700 10 client.15654 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:57:29.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.217+0000 7fbd57fff700 1 --2- 172.21.15.2:0/3553524503 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55b2883843c0 0x55b2883867e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=718be3efe8157017 server_cookie=278b39d7acdc837 in_seq=0 out_seq=0 2024-05-14T16:57:29.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.217+0000 7fbd377fe700 1 -- 172.21.15.2:0/3553524503 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b2882cadc0 con 0x55b2883843c0 2024-05-14T16:57:29.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.217+0000 7fbd377fe700 10 client.15654 handle_command_reply: tid=0 2024-05-14T16:57:29.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.217+0000 7fbd69038b80 1 client.15654 shutdown 2024-05-14T16:57:29.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.218+0000 7fbd69038b80 1 -- 172.21.15.2:0/3553524503 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b28836fd40 msgr2=0x55b288374430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:29.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.218+0000 7fbd69038b80 1 --2- 172.21.15.2:0/3553524503 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b28836fd40 0x55b288374430 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fbd500e1500 tx=0x7fbd500aa4e0 comp rx=0 tx=0).stop 2024-05-14T16:57:29.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.218+0000 7fbd69038b80 1 -- 172.21.15.2:0/3553524503 shutdown_connections 2024-05-14T16:57:29.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.218+0000 7fbd69038b80 1 --2- 172.21.15.2:0/3553524503 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55b2883843c0 0x55b2883867e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:29.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.218+0000 7fbd69038b80 1 --2- 172.21.15.2:0/3553524503 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b28836fd40 0x55b288374430 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:29.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.218+0000 7fbd69038b80 1 --2- 172.21.15.2:0/3553524503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b28836b2c0 0x55b288376b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:29.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.218+0000 7fbd377fe700 0 client.15654 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:57:29.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.218+0000 7fbd69038b80 1 --2- 172.21.15.2:0/3553524503 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28836f420 0x55b288373ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:29.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.218+0000 7fbd69038b80 1 -- 172.21.15.2:0/3553524503 >> 172.21.15.2:0/3553524503 conn(0x55b288297eb0 msgr2=0x55b28835ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:29.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.218+0000 7fbd69038b80 1 -- 172.21.15.2:0/3553524503 shutdown_connections 2024-05-14T16:57:29.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.218+0000 7fbd69038b80 1 -- 172.21.15.2:0/3553524503 wait complete. 2024-05-14T16:57:29.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.218+0000 7fbd69038b80 20 client.15654 trim_cache size 0 max 16384 2024-05-14T16:57:29.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.238+0000 7fbd69038b80 1 Processor -- start 2024-05-14T16:57:29.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.238+0000 7fbd69038b80 1 -- start start 2024-05-14T16:57:29.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.238+0000 7fbd69038b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b28836b2c0 0x55b2883600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:29.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.238+0000 7fbd69038b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b28836f420 0x55b28836e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:29.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.238+0000 7fbd69038b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28836fd40 0x55b288368c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:29.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.238+0000 7fbd69038b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbd581b0600 con 0x55b28836b2c0 2024-05-14T16:57:29.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.238+0000 7fbd69038b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbd581ade00 con 0x55b28836f420 2024-05-14T16:57:29.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.238+0000 7fbd69038b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbd581abc70 con 0x55b28836fd40 2024-05-14T16:57:29.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.238+0000 7fbd57fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b28836b2c0 0x55b2883600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:29.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.238+0000 7fbd57fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b28836b2c0 0x55b2883600d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49118/0 (socket says 172.21.15.2:49118) 2024-05-14T16:57:29.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.238+0000 7fbd57fff700 1 -- 172.21.15.2:0/3025144522 learned_addr learned my addr 172.21.15.2:0/3025144522 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:57:29.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.239+0000 7fbd577fe700 1 --2- 172.21.15.2:0/3025144522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b28836f420 0x55b28836e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:29.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.239+0000 7fbd5cd3f700 1 --2- 172.21.15.2:0/3025144522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28836fd40 0x55b288368c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:29.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.239+0000 7fbd57fff700 1 -- 172.21.15.2:0/3025144522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28836fd40 msgr2=0x55b288368c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:29.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.239+0000 7fbd57fff700 1 --2- 172.21.15.2:0/3025144522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28836fd40 0x55b288368c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:29.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.239+0000 7fbd57fff700 1 -- 172.21.15.2:0/3025144522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b28836f420 msgr2=0x55b28836e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:29.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.239+0000 7fbd57fff700 1 --2- 172.21.15.2:0/3025144522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b28836f420 0x55b28836e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:29.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.239+0000 7fbd57fff700 1 -- 172.21.15.2:0/3025144522 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b288386e60 con 0x55b28836b2c0 2024-05-14T16:57:29.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.239+0000 7fbd577fe700 1 --2- 172.21.15.2:0/3025144522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b28836f420 0x55b28836e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:57:29.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.239+0000 7fbd57fff700 1 --2- 172.21.15.2:0/3025144522 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b28836b2c0 0x55b2883600d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fbd480097f0 tx=0x7fbd48013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf244eced509532 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:29.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.239+0000 7fbd3dffb700 1 -- 172.21.15.2:0/3025144522 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4801b040 con 0x55b28836b2c0 2024-05-14T16:57:29.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.239+0000 7fbd3dffb700 1 -- 172.21.15.2:0/3025144522 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd48029770 con 0x55b28836b2c0 2024-05-14T16:57:29.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.239+0000 7fbd3dffb700 1 -- 172.21.15.2:0/3025144522 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd48015c20 con 0x55b28836b2c0 2024-05-14T16:57:29.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.240+0000 7fbd69038b80 1 -- 172.21.15.2:0/3025144522 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b28836b2c0 msgr2=0x55b2883600d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:29.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.240+0000 7fbd69038b80 1 --2- 172.21.15.2:0/3025144522 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b28836b2c0 0x55b2883600d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fbd480097f0 tx=0x7fbd48013000 comp rx=0 tx=0).stop 2024-05-14T16:57:29.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.240+0000 7fbd69038b80 1 -- 172.21.15.2:0/3025144522 shutdown_connections 2024-05-14T16:57:30.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.240+0000 7fbd69038b80 1 --2- 172.21.15.2:0/3025144522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28836fd40 0x55b288368c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:30.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.240+0000 7fbd69038b80 1 --2- 172.21.15.2:0/3025144522 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b28836b2c0 0x55b2883600d0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:30.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.240+0000 7fbd69038b80 1 --2- 172.21.15.2:0/3025144522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b28836f420 0x55b28836e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:30.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.240+0000 7fbd69038b80 1 -- 172.21.15.2:0/3025144522 >> 172.21.15.2:0/3025144522 conn(0x55b288297eb0 msgr2=0x55b28836e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:30.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.240+0000 7fbd69038b80 1 -- 172.21.15.2:0/3025144522 shutdown_connections 2024-05-14T16:57:30.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.240+0000 7fbd69038b80 1 -- 172.21.15.2:0/3025144522 wait complete. 2024-05-14T16:57:30.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.240+0000 7fbd69038b80 1 -- start start 2024-05-14T16:57:30.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.240+0000 7fbd69038b80 1 Processor -- start 2024-05-14T16:57:30.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.241+0000 7fbd69038b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28836b2c0 0x55b288377ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:30.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.241+0000 7fbd69038b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b28836f420 0x55b288378010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:30.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.241+0000 7fbd69038b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b28836fd40 0x55b288378550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:30.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.241+0000 7fbd69038b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbd581ac2c0 con 0x55b28836f420 2024-05-14T16:57:30.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.241+0000 7fbd69038b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbd581ad730 con 0x55b28836fd40 2024-05-14T16:57:30.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.241+0000 7fbd69038b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbd581ac990 con 0x55b28836b2c0 2024-05-14T16:57:30.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.241+0000 7fbd69038b80 10 client.0 _ll_register_callbacks cb 0x55b288253a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:57:30.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.241+0000 7fbd577fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b28836f420 0x55b288378010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:30.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.241+0000 7fbd577fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b28836f420 0x55b288378010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49134/0 (socket says 172.21.15.2:49134) 2024-05-14T16:57:30.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.241+0000 7fbd577fe700 1 -- 172.21.15.2:0/4076503683 learned_addr learned my addr 172.21.15.2:0/4076503683 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:57:30.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.241+0000 7fbd57fff700 1 --2- 172.21.15.2:0/4076503683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28836b2c0 0x55b288377ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:30.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.241+0000 7fbd5cd3f700 1 --2- 172.21.15.2:0/4076503683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b28836fd40 0x55b288378550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:30.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.241+0000 7fbd577fe700 1 -- 172.21.15.2:0/4076503683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28836b2c0 msgr2=0x55b288377ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:30.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.241+0000 7fbd577fe700 1 --2- 172.21.15.2:0/4076503683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28836b2c0 0x55b288377ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:30.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.241+0000 7fbd577fe700 1 -- 172.21.15.2:0/4076503683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b28836fd40 msgr2=0x55b288378550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:30.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.241+0000 7fbd577fe700 1 --2- 172.21.15.2:0/4076503683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b28836fd40 0x55b288378550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:30.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.241+0000 7fbd577fe700 1 -- 172.21.15.2:0/4076503683 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2883eef80 con 0x55b28836f420 2024-05-14T16:57:30.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.241+0000 7fbd577fe700 1 --2- 172.21.15.2:0/4076503683 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b28836f420 0x55b288378010 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c0059d0 tx=0x7fbd4c010ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98f01f39156599e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:30.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.241+0000 7fbd377fe700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:57:30.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.242+0000 7fbd377fe700 1 -- 172.21.15.2:0/4076503683 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c005450 con 0x55b28836f420 2024-05-14T16:57:30.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.242+0000 7fbd377fe700 1 -- 172.21.15.2:0/4076503683 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fbd4c0055f0 con 0x55b28836f420 2024-05-14T16:57:30.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.242+0000 7fbd69038b80 1 -- 172.21.15.2:0/4076503683 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd48029770 con 0x55b28836f420 2024-05-14T16:57:30.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.242+0000 7fbd377fe700 1 -- 172.21.15.2:0/4076503683 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbd4c003680 con 0x55b28836f420 2024-05-14T16:57:30.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.242+0000 7fbd69038b80 10 client.15660 fetch_fsmap learned FSMap version 30 2024-05-14T16:57:30.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.242+0000 7fbd69038b80 1 -- 172.21.15.2:0/4076503683 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b2883f02d0 con 0x55b28836f420 2024-05-14T16:57:30.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.242+0000 7fbd377fe700 1 -- 172.21.15.2:0/4076503683 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd48029770 con 0x55b28836f420 2024-05-14T16:57:30.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.242+0000 7fbd377fe700 1 -- 172.21.15.2:0/4076503683 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fbd4c011930 con 0x55b28836f420 2024-05-14T16:57:30.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.242+0000 7fbd69038b80 10 client.15660 fetch_fsmap finished waiting for FSMap version 30 2024-05-14T16:57:30.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.242+0000 7fbd69038b80 10 client.15660 resolve_mds: validated gid 24479 aka daemon mds.b 2024-05-14T16:57:30.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.242+0000 7fbd69038b80 20 client.15660 populate_metadata read hostname 'smithi002' 2024-05-14T16:57:30.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.242+0000 7fbd69038b80 1 --2- 172.21.15.2:0/4076503683 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55b2883f0840 0x55b2883f2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:30.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.242+0000 7fbd69038b80 4 client.15660 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ab7ccb52", "release": true, "await": true}] 2024-05-14T16:57:30.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.242+0000 7fbd69038b80 1 -- 172.21.15.2:0/4076503683 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ab7ccb52", "release": true, "await": true}) -- 0x55b2882c45a0 con 0x55b2883f0840 2024-05-14T16:57:30.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.242+0000 7fbd57fff700 1 --2- 172.21.15.2:0/4076503683 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55b2883f0840 0x55b2883f2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:30.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.243+0000 7fbd57fff700 1 --2- 172.21.15.2:0/4076503683 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55b2883f0840 0x55b2883f2c60 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21ba9fc9bfb6f2a4 server_cookie=d8ce7b84d4e83253 in_seq=0 out_seq=0 2024-05-14T16:57:30.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.243+0000 7fbd377fe700 10 client.15660 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:57:30.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.264+0000 7fbd377fe700 1 -- 172.21.15.2:0/4076503683 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b2882c45a0 con 0x55b2883f0840 2024-05-14T16:57:30.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.264+0000 7fbd377fe700 10 client.15660 handle_command_reply: tid=0 2024-05-14T16:57:30.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.264+0000 7fbd69038b80 1 client.15660 shutdown 2024-05-14T16:57:30.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.265+0000 7fbd69038b80 1 -- 172.21.15.2:0/4076503683 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b28836f420 msgr2=0x55b288378010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:30.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.265+0000 7fbd69038b80 1 --2- 172.21.15.2:0/4076503683 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b28836f420 0x55b288378010 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c0059d0 tx=0x7fbd4c010ef0 comp rx=0 tx=0).stop 2024-05-14T16:57:30.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.265+0000 7fbd69038b80 1 -- 172.21.15.2:0/4076503683 shutdown_connections 2024-05-14T16:57:30.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.265+0000 7fbd69038b80 1 --2- 172.21.15.2:0/4076503683 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55b2883f0840 0x55b2883f2c60 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:30.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.265+0000 7fbd69038b80 1 --2- 172.21.15.2:0/4076503683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b28836fd40 0x55b288378550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:30.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.265+0000 7fbd69038b80 1 --2- 172.21.15.2:0/4076503683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b28836b2c0 0x55b288377ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:30.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.265+0000 7fbd69038b80 1 --2- 172.21.15.2:0/4076503683 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b28836f420 0x55b288378010 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:30.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.265+0000 7fbd377fe700 0 client.15660 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:57:30.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.265+0000 7fbd69038b80 1 -- 172.21.15.2:0/4076503683 >> 172.21.15.2:0/4076503683 conn(0x55b288297eb0 msgr2=0x55b2883754f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:30.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.265+0000 7fbd69038b80 1 -- 172.21.15.2:0/4076503683 shutdown_connections 2024-05-14T16:57:30.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.265+0000 7fbd69038b80 1 -- 172.21.15.2:0/4076503683 wait complete. 2024-05-14T16:57:30.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.265+0000 7fbd69038b80 20 client.15660 trim_cache size 0 max 16384 2024-05-14T16:57:30.621 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T16:57:30.629 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 30, 2024-05-14T16:57:30.637 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24479, 2024-05-14T16:57:30.661 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 12, 2024-05-14T16:57:30.670 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-05-14T16:57:30.677 INFO:teuthology.orchestra.run.smithi002.stdout: "ab7ccb52": { 2024-05-14T16:57:30.684 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 12, 2024-05-14T16:57:30.691 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-05-14T16:57:30.711 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-05-14T16:57:30.728 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-05-14T16:57:30.735 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-05-14T16:57:30.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T16:57:30.751 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-05-14T16:57:30.759 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 146.7, 2024-05-14T16:57:30.768 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-05-14T16:57:30.777 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-05-14T16:57:30.785 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-05-14T16:57:30.794 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-05-14T16:57:30.803 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-05-14T16:57:30.812 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-05-14T16:57:30.847 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:57:30.856 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:57:30.864 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:57:30.872 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:57:30.880 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:57:30.889 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T16:57:30.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.266+0000 7fbd5e7a2700 1 -- 172.21.15.2:0/3886099924 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fbd400b29f0 msgr2=0x7fbd400b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:30.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.266+0000 7fbd5e7a2700 1 --2- 172.21.15.2:0/3886099924 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fbd400b29f0 0x7fbd400b4eb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbd48009a10 tx=0x7fbd48009190 comp rx=0 tx=0).stop 2024-05-14T16:57:30.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.266+0000 7fbd5e7a2700 1 -- 172.21.15.2:0/3886099924 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd58109680 msgr2=0x7fbd58199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:30.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.266+0000 7fbd5e7a2700 1 --2- 172.21.15.2:0/3886099924 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd58109680 0x7fbd58199ad0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000d010 tx=0x7fbd500079a0 comp rx=0 tx=0).stop 2024-05-14T16:57:30.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.266+0000 7fbd5e7a2700 1 -- 172.21.15.2:0/3886099924 shutdown_connections 2024-05-14T16:57:30.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.266+0000 7fbd5e7a2700 1 --2- 172.21.15.2:0/3886099924 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fbd400b29f0 0x7fbd400b4eb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:30.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.266+0000 7fbd5e7a2700 1 --2- 172.21.15.2:0/3886099924 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd58109680 0x7fbd58199ad0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:30.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.266+0000 7fbd5e7a2700 1 --2- 172.21.15.2:0/3886099924 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd581035e0 0x7fbd5819f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:30.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.266+0000 7fbd5e7a2700 1 --2- 172.21.15.2:0/3886099924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd58103fa0 0x7fbd5819fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:30.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.266+0000 7fbd5e7a2700 1 -- 172.21.15.2:0/3886099924 >> 172.21.15.2:0/3886099924 conn(0x7fbd5807a6f0 msgr2=0x7fbd581021a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:30.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.266+0000 7fbd5e7a2700 1 -- 172.21.15.2:0/3886099924 shutdown_connections 2024-05-14T16:57:31.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:57:28.266+0000 7fbd5e7a2700 1 -- 172.21.15.2:0/3886099924 wait complete. 2024-05-14T16:57:31.037 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:57:28 smithi002 ceph-mon[31538]: pgmap v86: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 707 KiB/s wr, 88 op/s 2024-05-14T16:57:31.047 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:57:30 smithi002 ceph-mon[31538]: pgmap v87: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-05-14T16:57:31.056 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ab7ccb52', total seconds elapsed: 36.2 2024-05-14T16:57:31.065 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:57:28 smithi123 ceph-mon[35971]: pgmap v86: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 707 KiB/s wr, 88 op/s 2024-05-14T16:57:31.092 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:57:30 smithi123 ceph-mon[35971]: pgmap v87: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-05-14T16:57:31.116 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:57:28 smithi151 ceph-mon[36066]: pgmap v86: 97 pgs: 97 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 707 KiB/s wr, 88 op/s 2024-05-14T16:57:31.124 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:57:30 smithi151 ceph-mon[36066]: pgmap v87: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-05-14T16:57:31.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 135.0 s, 2462.2 tps, lat 232.599 ms stddev 2262.397 2024-05-14T16:57:32.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:57:32 smithi002 ceph-mon[31538]: pgmap v88: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2024-05-14T16:57:32.750 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:57:32 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:57:32.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:57:32 smithi151 ceph-mon[36066]: pgmap v88: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2024-05-14T16:57:32.861 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:57:32 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:57:32.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:57:32 smithi123 ceph-mon[35971]: pgmap v88: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2024-05-14T16:57:32.916 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:57:32 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:57:34.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:57:34 smithi002 ceph-mon[31538]: pgmap v89: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 3.7 MiB/s wr, 402 op/s 2024-05-14T16:57:34.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:57:34 smithi151 ceph-mon[36066]: pgmap v89: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 3.7 MiB/s wr, 402 op/s 2024-05-14T16:57:34.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:57:34 smithi123 ceph-mon[35971]: pgmap v89: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 3.7 MiB/s wr, 402 op/s 2024-05-14T16:57:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.620+0000 7ff98f30a640 1 -- 172.21.15.2:0/266577361 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff988104eb0 msgr2=0x7ff988105290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.620+0000 7ff98f30a640 1 --2- 172.21.15.2:0/266577361 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff988104eb0 0x7ff988105290 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff984009900 tx=0x7ff98402a410 comp rx=0 tx=0).stop 2024-05-14T16:57:35.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.620+0000 7ff98f30a640 1 -- 172.21.15.2:0/266577361 shutdown_connections 2024-05-14T16:57:35.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.620+0000 7ff98f30a640 1 --2- 172.21.15.2:0/266577361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff988109f80 0x7ff988111b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:35.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.620+0000 7ff98f30a640 1 --2- 172.21.15.2:0/266577361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff988105860 0x7ff9881098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.620+0000 7ff98f30a640 1 --2- 172.21.15.2:0/266577361 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff988104eb0 0x7ff988105290 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:35.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.620+0000 7ff98f30a640 1 -- 172.21.15.2:0/266577361 >> 172.21.15.2:0/266577361 conn(0x7ff988100710 msgr2=0x7ff988102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:35.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.620+0000 7ff98f30a640 1 -- 172.21.15.2:0/266577361 shutdown_connections 2024-05-14T16:57:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.620+0000 7ff98f30a640 1 -- 172.21.15.2:0/266577361 wait complete. 2024-05-14T16:57:35.696 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.620+0000 7ff98f30a640 1 Processor -- start 2024-05-14T16:57:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.621+0000 7ff98f30a640 1 -- start start 2024-05-14T16:57:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.621+0000 7ff98f30a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff988104eb0 0x7ff98819b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.621+0000 7ff98f30a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff988109f80 0x7ff98819b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.621+0000 7ff98f30a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff98819fc60 0x7ff9881a0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:35.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.621+0000 7ff98f30a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff988114ee0 con 0x7ff988109f80 2024-05-14T16:57:35.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.621+0000 7ff98f30a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff988114d60 con 0x7ff98819fc60 2024-05-14T16:57:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.621+0000 7ff98f30a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff988115060 con 0x7ff988104eb0 2024-05-14T16:57:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.621+0000 7ff98c8a5640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff988109f80 0x7ff98819b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.621+0000 7ff98c8a5640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff988109f80 0x7ff98819b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35822/0 (socket says 172.21.15.2:35822) 2024-05-14T16:57:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.621+0000 7ff98c8a5640 1 -- 172.21.15.2:0/3970534427 learned_addr learned my addr 172.21.15.2:0/3970534427 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:57:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.621+0000 7ff98d0a6640 1 --2- 172.21.15.2:0/3970534427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff988104eb0 0x7ff98819b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.621+0000 7ff98d8a7640 1 --2- 172.21.15.2:0/3970534427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff98819fc60 0x7ff9881a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.621+0000 7ff98c8a5640 1 -- 172.21.15.2:0/3970534427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff988104eb0 msgr2=0x7ff98819b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.621+0000 7ff98c8a5640 1 --2- 172.21.15.2:0/3970534427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff988104eb0 0x7ff98819b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.621+0000 7ff98c8a5640 1 -- 172.21.15.2:0/3970534427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff98819fc60 msgr2=0x7ff9881a0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.621+0000 7ff98c8a5640 1 --2- 172.21.15.2:0/3970534427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff98819fc60 0x7ff9881a0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:36.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.621+0000 7ff98c8a5640 1 -- 172.21.15.2:0/3970534427 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9881a0750 con 0x7ff988109f80 2024-05-14T16:57:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.621+0000 7ff98c8a5640 1 --2- 172.21.15.2:0/3970534427 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff988109f80 0x7ff98819b8e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff97800d770 tx=0x7ff97800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.622+0000 7ff97e7fc640 1 -- 172.21.15.2:0/3970534427 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff978017070 con 0x7ff988109f80 2024-05-14T16:57:36.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.622+0000 7ff98f30a640 1 -- 172.21.15.2:0/3970534427 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9881a0a40 con 0x7ff988109f80 2024-05-14T16:57:36.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.622+0000 7ff97e7fc640 1 -- 172.21.15.2:0/3970534427 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff978009c30 con 0x7ff988109f80 2024-05-14T16:57:36.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.622+0000 7ff97e7fc640 1 -- 172.21.15.2:0/3970534427 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff978021060 con 0x7ff988109f80 2024-05-14T16:57:36.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.622+0000 7ff98f30a640 1 -- 172.21.15.2:0/3970534427 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9881a8220 con 0x7ff988109f80 2024-05-14T16:57:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.626+0000 7ff97e7fc640 1 -- 172.21.15.2:0/3970534427 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7ff97801a070 con 0x7ff988109f80 2024-05-14T16:57:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.627+0000 7ff97e7fc640 1 --2- 172.21.15.2:0/3970534427 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ff9740b2a90 0x7ff9740b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.627+0000 7ff97e7fc640 1 -- 172.21.15.2:0/3970534427 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7ff9780e18e0 con 0x7ff988109f80 2024-05-14T16:57:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.627+0000 7ff98d0a6640 1 --2- 172.21.15.2:0/3970534427 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ff9740b2a90 0x7ff9740b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.628+0000 7ff992025740 1 Processor -- start 2024-05-14T16:57:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.628+0000 7ff992025740 1 -- start start 2024-05-14T16:57:36.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.628+0000 7ff992025740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0b251f8f0 0x55c0b251fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.628+0000 7ff992025740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0b2520210 0x55c0b252a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.628+0000 7ff992025740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b252aad0 0x55c0b252cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.628+0000 7ff992025740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c0b2492c30 con 0x55c0b252aad0 2024-05-14T16:57:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.628+0000 7ff992025740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c0b24827d0 con 0x55c0b251f8f0 2024-05-14T16:57:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.628+0000 7ff992025740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c0b2486810 con 0x55c0b2520210 2024-05-14T16:57:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.628+0000 7ff98d0a6640 1 --2- 172.21.15.2:0/3970534427 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ff9740b2a90 0x7ff9740b4f50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff984000c00 tx=0x7ff984005d50 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.629+0000 7ff98d0a6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0b251f8f0 0x55c0b251fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.629+0000 7ff98d8a7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b252aad0 0x55c0b252cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.629+0000 7ff98d8a7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b252aad0 0x55c0b252cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35824/0 (socket says 172.21.15.2:35824) 2024-05-14T16:57:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.629+0000 7ff98d8a7640 1 -- 172.21.15.2:0/2269906427 learned_addr learned my addr 172.21.15.2:0/2269906427 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:57:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.629+0000 7ff98c8a5640 1 --2- 172.21.15.2:0/2269906427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0b2520210 0x55c0b252a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.629+0000 7ff98d0a6640 1 -- 172.21.15.2:0/2269906427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0b2520210 msgr2=0x55c0b252a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:36.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.629+0000 7ff98d0a6640 1 --2- 172.21.15.2:0/2269906427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0b2520210 0x55c0b252a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.629+0000 7ff98d0a6640 1 -- 172.21.15.2:0/2269906427 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b252aad0 msgr2=0x55c0b252cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:36.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.629+0000 7ff98d0a6640 1 --2- 172.21.15.2:0/2269906427 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b252aad0 0x55c0b252cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.629+0000 7ff98d0a6640 1 -- 172.21.15.2:0/2269906427 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0b2197210 con 0x55c0b251f8f0 2024-05-14T16:57:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.630+0000 7ff98d0a6640 1 --2- 172.21.15.2:0/2269906427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0b251f8f0 0x55c0b251fcd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff984009230 tx=0x7ff98403f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d8c1cc11d3868ee server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.630+0000 7ff98d8a7640 1 --2- 172.21.15.2:0/2269906427 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b252aad0 0x55c0b252cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:57:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.630+0000 7ff9737fe640 1 -- 172.21.15.2:0/2269906427 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff984035020 con 0x55c0b251f8f0 2024-05-14T16:57:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.631+0000 7ff9737fe640 1 -- 172.21.15.2:0/2269906427 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff984004c10 con 0x55c0b251f8f0 2024-05-14T16:57:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.631+0000 7ff9737fe640 1 -- 172.21.15.2:0/2269906427 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff984031be0 con 0x55c0b251f8f0 2024-05-14T16:57:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.631+0000 7ff992025740 1 -- 172.21.15.2:0/2269906427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0b251f8f0 msgr2=0x55c0b251fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.631+0000 7ff992025740 1 --2- 172.21.15.2:0/2269906427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0b251f8f0 0x55c0b251fcd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff984009230 tx=0x7ff98403f000 comp rx=0 tx=0).stop 2024-05-14T16:57:36.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.631+0000 7ff992025740 1 -- 172.21.15.2:0/2269906427 shutdown_connections 2024-05-14T16:57:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.631+0000 7ff992025740 1 --2- 172.21.15.2:0/2269906427 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b252aad0 0x55c0b252cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.631+0000 7ff992025740 1 --2- 172.21.15.2:0/2269906427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0b2520210 0x55c0b252a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.631+0000 7ff992025740 1 --2- 172.21.15.2:0/2269906427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0b251f8f0 0x55c0b251fcd0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.631+0000 7ff992025740 1 -- 172.21.15.2:0/2269906427 >> 172.21.15.2:0/2269906427 conn(0x55c0b251be00 msgr2=0x55c0b251c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.632+0000 7ff992025740 1 -- 172.21.15.2:0/2269906427 shutdown_connections 2024-05-14T16:57:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.632+0000 7ff992025740 1 -- 172.21.15.2:0/2269906427 wait complete. 2024-05-14T16:57:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.632+0000 7ff992025740 1 -- start start 2024-05-14T16:57:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.632+0000 7ff992025740 1 Processor -- start 2024-05-14T16:57:36.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.633+0000 7ff992025740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b251f8f0 0x55c0b252a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.633+0000 7ff992025740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0b2520210 0x55c0b25277e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.633+0000 7ff992025740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0b252aad0 0x55c0b2527df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:36.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.633+0000 7ff992025740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c0b2168790 con 0x55c0b251f8f0 2024-05-14T16:57:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.633+0000 7ff992025740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c0b243e890 con 0x55c0b252aad0 2024-05-14T16:57:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.633+0000 7ff992025740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c0b252d740 con 0x55c0b2520210 2024-05-14T16:57:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.633+0000 7ff992025740 10 client.0 _ll_register_callbacks cb 0x55c0b2441d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:57:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.633+0000 7ff98d8a7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0b252aad0 0x55c0b2527df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.633+0000 7ff98d8a7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0b252aad0 0x55c0b2527df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:42100/0 (socket says 172.21.15.2:42100) 2024-05-14T16:57:36.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.633+0000 7ff98d8a7640 1 -- 172.21.15.2:0/1700445932 learned_addr learned my addr 172.21.15.2:0/1700445932 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:57:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.633+0000 7ff98d0a6640 1 --2- 172.21.15.2:0/1700445932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b251f8f0 0x55c0b252a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.633+0000 7ff98c8a5640 1 --2- 172.21.15.2:0/1700445932 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0b2520210 0x55c0b25277e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.633+0000 7ff98c8a5640 1 -- 172.21.15.2:0/1700445932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0b252aad0 msgr2=0x55c0b2527df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:36.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.633+0000 7ff98c8a5640 1 --2- 172.21.15.2:0/1700445932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0b252aad0 0x55c0b2527df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.633+0000 7ff98c8a5640 1 -- 172.21.15.2:0/1700445932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b251f8f0 msgr2=0x55c0b252a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.633+0000 7ff98c8a5640 1 --2- 172.21.15.2:0/1700445932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b251f8f0 0x55c0b252a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.633+0000 7ff98c8a5640 1 -- 172.21.15.2:0/1700445932 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0b2263620 con 0x55c0b2520210 2024-05-14T16:57:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.634+0000 7ff98c8a5640 1 --2- 172.21.15.2:0/1700445932 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0b2520210 0x55c0b25277e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff9780df060 tx=0x7ff9780a4560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb4e601f5fdb48a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.634+0000 7ff970ff9640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-14T16:57:36.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.634+0000 7ff970ff9640 1 -- 172.21.15.2:0/1700445932 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9780acfa0 con 0x55c0b2520210 2024-05-14T16:57:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.634+0000 7ff970ff9640 1 -- 172.21.15.2:0/1700445932 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7ff9780a7f00 con 0x55c0b2520210 2024-05-14T16:57:36.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.634+0000 7ff992025740 1 -- 172.21.15.2:0/1700445932 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff984004c10 con 0x55c0b2520210 2024-05-14T16:57:36.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.634+0000 7ff970ff9640 1 -- 172.21.15.2:0/1700445932 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff9780b1a50 con 0x55c0b2520210 2024-05-14T16:57:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.634+0000 7ff992025740 10 client.25366 fetch_fsmap learned FSMap version 31 2024-05-14T16:57:36.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.634+0000 7ff992025740 1 -- 172.21.15.2:0/1700445932 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c0b2539270 con 0x55c0b2520210 2024-05-14T16:57:36.734 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.635+0000 7ff970ff9640 1 -- 172.21.15.2:0/1700445932 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff984004c10 con 0x55c0b2520210 2024-05-14T16:57:36.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.635+0000 7ff970ff9640 1 -- 172.21.15.2:0/1700445932 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7ff9780a7a90 con 0x55c0b2520210 2024-05-14T16:57:36.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.635+0000 7ff992025740 10 client.25366 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T16:57:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.635+0000 7ff992025740 10 client.25366 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:57:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.635+0000 7ff992025740 20 client.25366 populate_metadata read hostname 'smithi002' 2024-05-14T16:57:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.635+0000 7ff992025740 1 --2- 172.21.15.2:0/1700445932 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55c0b253a780 0x55c0b253cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:36.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.635+0000 7ff992025740 4 client.25366 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:57:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.635+0000 7ff992025740 1 -- 172.21.15.2:0/1700445932 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0b2494b80 con 0x55c0b253a780 2024-05-14T16:57:36.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.635+0000 7ff98d0a6640 1 --2- 172.21.15.2:0/1700445932 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55c0b253a780 0x55c0b253cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:36.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.636+0000 7ff970ff9640 10 client.25366 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:57:36.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.636+0000 7ff98d0a6640 1 --2- 172.21.15.2:0/1700445932 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55c0b253a780 0x55c0b253cbb0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=914c1e63f390973 server_cookie=b57f6897295c4802 in_seq=0 out_seq=0 2024-05-14T16:57:36.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.636+0000 7ff970ff9640 1 -- 172.21.15.2:0/1700445932 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c0b2494b80 con 0x55c0b253a780 2024-05-14T16:57:36.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.636+0000 7ff970ff9640 10 client.25366 handle_command_reply: tid=0 2024-05-14T16:57:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.636+0000 7ff992025740 1 client.25366 shutdown 2024-05-14T16:57:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.637+0000 7ff992025740 1 -- 172.21.15.2:0/1700445932 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0b2520210 msgr2=0x55c0b25277e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.637+0000 7ff992025740 1 --2- 172.21.15.2:0/1700445932 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0b2520210 0x55c0b25277e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff9780df060 tx=0x7ff9780a4560 comp rx=0 tx=0).stop 2024-05-14T16:57:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.637+0000 7ff992025740 1 -- 172.21.15.2:0/1700445932 shutdown_connections 2024-05-14T16:57:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.637+0000 7ff992025740 1 --2- 172.21.15.2:0/1700445932 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55c0b253a780 0x55c0b253cbb0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:36.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.637+0000 7ff970ff9640 0 client.25366 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:57:36.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.637+0000 7ff992025740 1 --2- 172.21.15.2:0/1700445932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0b252aad0 0x55c0b2527df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:36.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.637+0000 7ff992025740 1 --2- 172.21.15.2:0/1700445932 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0b2520210 0x55c0b25277e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.637+0000 7ff992025740 1 --2- 172.21.15.2:0/1700445932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b251f8f0 0x55c0b252a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:37.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.637+0000 7ff992025740 1 -- 172.21.15.2:0/1700445932 >> 172.21.15.2:0/1700445932 conn(0x55c0b251be00 msgr2=0x55c0b251b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.637+0000 7ff992025740 1 -- 172.21.15.2:0/1700445932 shutdown_connections 2024-05-14T16:57:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.637+0000 7ff992025740 1 -- 172.21.15.2:0/1700445932 wait complete. 2024-05-14T16:57:37.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.638+0000 7ff992025740 20 client.25366 trim_cache size 0 max 16384 2024-05-14T16:57:37.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.667+0000 7ff992025740 1 Processor -- start 2024-05-14T16:57:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.667+0000 7ff992025740 1 -- start start 2024-05-14T16:57:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.667+0000 7ff992025740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0b251f8f0 0x55c0b2536610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.667+0000 7ff992025740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0b2520210 0x55c0b2542480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.668+0000 7ff992025740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b252aad0 0x55c0b25429c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:37.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.668+0000 7ff98c8a5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0b2520210 0x55c0b2542480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:37.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.668+0000 7ff98c8a5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0b2520210 0x55c0b2542480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:42112/0 (socket says 172.21.15.2:42112) 2024-05-14T16:57:37.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.668+0000 7ff98c8a5640 1 -- 172.21.15.2:0/26704003 learned_addr learned my addr 172.21.15.2:0/26704003 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:57:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.668+0000 7ff98d8a7640 1 --2- 172.21.15.2:0/26704003 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b252aad0 0x55c0b25429c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.668+0000 7ff992025740 1 -- 172.21.15.2:0/26704003 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff9881aed50 con 0x55c0b252aad0 2024-05-14T16:57:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.668+0000 7ff992025740 1 -- 172.21.15.2:0/26704003 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff9881ac5c0 con 0x55c0b2520210 2024-05-14T16:57:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.668+0000 7ff992025740 1 -- 172.21.15.2:0/26704003 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff9881aa410 con 0x55c0b251f8f0 2024-05-14T16:57:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.669+0000 7ff98d8a7640 1 -- 172.21.15.2:0/26704003 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0b251f8f0 msgr2=0x55c0b2536610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:37.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.669+0000 7ff98d8a7640 1 --2- 172.21.15.2:0/26704003 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0b251f8f0 0x55c0b2536610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:37.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.669+0000 7ff98d8a7640 1 -- 172.21.15.2:0/26704003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0b2520210 msgr2=0x55c0b2542480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.669+0000 7ff98d8a7640 1 --2- 172.21.15.2:0/26704003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0b2520210 0x55c0b2542480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.669+0000 7ff98d8a7640 1 -- 172.21.15.2:0/26704003 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0b252cec0 con 0x55c0b252aad0 2024-05-14T16:57:37.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.669+0000 7ff98d8a7640 1 --2- 172.21.15.2:0/26704003 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b252aad0 0x55c0b25429c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff980005f10 tx=0x7ff980013a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f231bc3975bc4a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.669+0000 7ff972ffd640 1 -- 172.21.15.2:0/26704003 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff980021070 con 0x55c0b252aad0 2024-05-14T16:57:37.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.669+0000 7ff972ffd640 1 -- 172.21.15.2:0/26704003 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff980011a30 con 0x55c0b252aad0 2024-05-14T16:57:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.669+0000 7ff972ffd640 1 -- 172.21.15.2:0/26704003 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98001cbc0 con 0x55c0b252aad0 2024-05-14T16:57:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.671+0000 7ff992025740 1 -- 172.21.15.2:0/26704003 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b252aad0 msgr2=0x55c0b25429c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:37.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.671+0000 7ff992025740 1 --2- 172.21.15.2:0/26704003 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b252aad0 0x55c0b25429c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff980005f10 tx=0x7ff980013a80 comp rx=0 tx=0).stop 2024-05-14T16:57:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.671+0000 7ff992025740 1 -- 172.21.15.2:0/26704003 shutdown_connections 2024-05-14T16:57:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.671+0000 7ff992025740 1 --2- 172.21.15.2:0/26704003 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b252aad0 0x55c0b25429c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.671+0000 7ff992025740 1 --2- 172.21.15.2:0/26704003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0b2520210 0x55c0b2542480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.671+0000 7ff992025740 1 --2- 172.21.15.2:0/26704003 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0b251f8f0 0x55c0b2536610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:37.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.671+0000 7ff992025740 1 -- 172.21.15.2:0/26704003 >> 172.21.15.2:0/26704003 conn(0x55c0b251be00 msgr2=0x55c0b251b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.672+0000 7ff992025740 1 -- 172.21.15.2:0/26704003 shutdown_connections 2024-05-14T16:57:37.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.672+0000 7ff992025740 1 -- 172.21.15.2:0/26704003 wait complete. 2024-05-14T16:57:37.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.672+0000 7ff992025740 1 -- start start 2024-05-14T16:57:37.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.672+0000 7ff992025740 1 Processor -- start 2024-05-14T16:57:37.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.673+0000 7ff992025740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0b251f8f0 0x55c0b2530bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:37.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.673+0000 7ff992025740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0b2520210 0x55c0b2531100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:37.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.673+0000 7ff992025740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b252aad0 0x55c0b25a4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:37.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.673+0000 7ff98c8a5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0b2520210 0x55c0b2531100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:37.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.674+0000 7ff98c8a5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0b2520210 0x55c0b2531100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:42122/0 (socket says 172.21.15.2:42122) 2024-05-14T16:57:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.674+0000 7ff98d0a6640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0b251f8f0 0x55c0b2530bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.674+0000 7ff98d0a6640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0b251f8f0 0x55c0b2530bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:58064/0 (socket says 172.21.15.2:58064) 2024-05-14T16:57:37.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.674+0000 7ff98c8a5640 1 -- 172.21.15.2:0/552075540 learned_addr learned my addr 172.21.15.2:0/552075540 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:57:37.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.674+0000 7ff992025740 1 -- 172.21.15.2:0/552075540 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff9881aaa80 con 0x55c0b252aad0 2024-05-14T16:57:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.674+0000 7ff992025740 1 -- 172.21.15.2:0/552075540 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff9881abef0 con 0x55c0b2520210 2024-05-14T16:57:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.674+0000 7ff992025740 1 -- 172.21.15.2:0/552075540 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff9881ab150 con 0x55c0b251f8f0 2024-05-14T16:57:37.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.674+0000 7ff992025740 10 client.0 _ll_register_callbacks cb 0x7ff9780a84a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:57:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.674+0000 7ff98d8a7640 1 --2- 172.21.15.2:0/552075540 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b252aad0 0x55c0b25a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:37.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.674+0000 7ff98c8a5640 1 -- 172.21.15.2:0/552075540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0b251f8f0 msgr2=0x55c0b2530bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.674+0000 7ff98c8a5640 1 --2- 172.21.15.2:0/552075540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0b251f8f0 0x55c0b2530bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.675+0000 7ff98c8a5640 1 -- 172.21.15.2:0/552075540 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b252aad0 msgr2=0x55c0b25a4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:37.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.675+0000 7ff98c8a5640 1 --2- 172.21.15.2:0/552075540 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b252aad0 0x55c0b25a4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.675+0000 7ff98c8a5640 1 -- 172.21.15.2:0/552075540 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0b25a4580 con 0x55c0b2520210 2024-05-14T16:57:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.675+0000 7ff98c8a5640 1 --2- 172.21.15.2:0/552075540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0b2520210 0x55c0b2531100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff9780e1c00 tx=0x7ff97800df10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28ea4781b6509d33 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.675+0000 7ff970ff9640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T16:57:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.675+0000 7ff970ff9640 1 -- 172.21.15.2:0/552075540 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff978089060 con 0x55c0b2520210 2024-05-14T16:57:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.675+0000 7ff970ff9640 1 -- 172.21.15.2:0/552075540 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7ff9780c3280 con 0x55c0b2520210 2024-05-14T16:57:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.675+0000 7ff992025740 1 -- 172.21.15.2:0/552075540 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff980011a30 con 0x55c0b2520210 2024-05-14T16:57:37.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.676+0000 7ff970ff9640 1 -- 172.21.15.2:0/552075540 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff9780acfa0 con 0x55c0b2520210 2024-05-14T16:57:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.676+0000 7ff992025740 10 client.25385 fetch_fsmap learned FSMap version 31 2024-05-14T16:57:37.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.676+0000 7ff992025740 1 -- 172.21.15.2:0/552075540 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c0b25a57c0 con 0x55c0b2520210 2024-05-14T16:57:37.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.677+0000 7ff970ff9640 1 -- 172.21.15.2:0/552075540 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff980011a30 con 0x55c0b2520210 2024-05-14T16:57:37.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.677+0000 7ff970ff9640 1 -- 172.21.15.2:0/552075540 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7ff9780cb0b0 con 0x55c0b2520210 2024-05-14T16:57:37.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.677+0000 7ff992025740 10 client.25385 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T16:57:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.677+0000 7ff992025740 10 client.25385 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:57:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.677+0000 7ff992025740 20 client.25385 populate_metadata read hostname 'smithi002' 2024-05-14T16:57:37.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.677+0000 7ff992025740 1 --2- 172.21.15.2:0/552075540 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55c0b253a780 0x55c0b25a7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.677+0000 7ff992025740 4 client.25385 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:57:37.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.677+0000 7ff992025740 1 -- 172.21.15.2:0/552075540 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c0b248e360 con 0x55c0b253a780 2024-05-14T16:57:37.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.677+0000 7ff98d0a6640 1 --2- 172.21.15.2:0/552075540 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55c0b253a780 0x55c0b25a7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.678+0000 7ff98d0a6640 1 --2- 172.21.15.2:0/552075540 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55c0b253a780 0x55c0b25a7da0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=378153be393d6ccd server_cookie=d0d71a3cf09e3357 in_seq=0 out_seq=0 2024-05-14T16:57:37.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.678+0000 7ff970ff9640 10 client.25385 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:57:37.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.688+0000 7ff970ff9640 1 -- 172.21.15.2:0/552075540 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c0b248e360 con 0x55c0b253a780 2024-05-14T16:57:37.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.688+0000 7ff970ff9640 10 client.25385 handle_command_reply: tid=0 2024-05-14T16:57:37.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.688+0000 7ff992025740 1 client.25385 shutdown 2024-05-14T16:57:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.689+0000 7ff992025740 1 -- 172.21.15.2:0/552075540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0b2520210 msgr2=0x55c0b2531100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.689+0000 7ff992025740 1 --2- 172.21.15.2:0/552075540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0b2520210 0x55c0b2531100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff9780e1c00 tx=0x7ff97800df10 comp rx=0 tx=0).stop 2024-05-14T16:57:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.689+0000 7ff992025740 1 -- 172.21.15.2:0/552075540 shutdown_connections 2024-05-14T16:57:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.689+0000 7ff992025740 1 --2- 172.21.15.2:0/552075540 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55c0b253a780 0x55c0b25a7da0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.689+0000 7ff992025740 1 --2- 172.21.15.2:0/552075540 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c0b252aad0 0x55c0b25a4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.689+0000 7ff992025740 1 --2- 172.21.15.2:0/552075540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0b2520210 0x55c0b2531100 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.689+0000 7ff992025740 1 --2- 172.21.15.2:0/552075540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0b251f8f0 0x55c0b2530bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.689+0000 7ff992025740 1 -- 172.21.15.2:0/552075540 >> 172.21.15.2:0/552075540 conn(0x55c0b251be00 msgr2=0x55c0b2535600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.689+0000 7ff970ff9640 0 client.25385 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:57:37.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.689+0000 7ff992025740 1 -- 172.21.15.2:0/552075540 shutdown_connections 2024-05-14T16:57:37.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.689+0000 7ff992025740 1 -- 172.21.15.2:0/552075540 wait complete. 2024-05-14T16:57:37.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.689+0000 7ff992025740 20 client.25385 trim_cache size 0 max 16384 2024-05-14T16:57:37.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T16:57:37.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T16:57:37.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T16:57:37.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.691+0000 7ff98f30a640 1 -- 172.21.15.2:0/3970534427 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ff9740b2a90 msgr2=0x7ff9740b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.691+0000 7ff98f30a640 1 --2- 172.21.15.2:0/3970534427 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ff9740b2a90 0x7ff9740b4f50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff984000c00 tx=0x7ff984005d50 comp rx=0 tx=0).stop 2024-05-14T16:57:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.691+0000 7ff98f30a640 1 -- 172.21.15.2:0/3970534427 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff988109f80 msgr2=0x7ff98819b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.691+0000 7ff98f30a640 1 --2- 172.21.15.2:0/3970534427 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff988109f80 0x7ff98819b8e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff97800d770 tx=0x7ff97800dc30 comp rx=0 tx=0).stop 2024-05-14T16:57:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.691+0000 7ff98f30a640 1 -- 172.21.15.2:0/3970534427 shutdown_connections 2024-05-14T16:57:38.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.691+0000 7ff98f30a640 1 --2- 172.21.15.2:0/3970534427 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ff9740b2a90 0x7ff9740b4f50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.691+0000 7ff98f30a640 1 --2- 172.21.15.2:0/3970534427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff98819fc60 0x7ff9881a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:38.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.691+0000 7ff98f30a640 1 --2- 172.21.15.2:0/3970534427 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff988109f80 0x7ff98819b8e0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:38.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.691+0000 7ff98f30a640 1 --2- 172.21.15.2:0/3970534427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff988104eb0 0x7ff98819b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:38.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.691+0000 7ff98f30a640 1 -- 172.21.15.2:0/3970534427 >> 172.21.15.2:0/3970534427 conn(0x7ff988100710 msgr2=0x7ff988102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.691+0000 7ff98f30a640 1 -- 172.21.15.2:0/3970534427 shutdown_connections 2024-05-14T16:57:38.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:35.691+0000 7ff98f30a640 1 -- 172.21.15.2:0/3970534427 wait complete. 2024-05-14T16:57:38.123 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:57:36 smithi002 ceph-mon[31538]: pgmap v90: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 3.6 MiB/s wr, 398 op/s 2024-05-14T16:57:38.132 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 140.0 s, 4144.8 tps, lat 24.164 ms stddev 24.380 2024-05-14T16:57:38.141 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:57:36 smithi123 ceph-mon[35971]: pgmap v90: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 3.6 MiB/s wr, 398 op/s 2024-05-14T16:57:38.150 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:57:36 smithi151 ceph-mon[36066]: pgmap v90: 97 pgs: 97 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 3.6 MiB/s wr, 398 op/s 2024-05-14T16:57:38.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:57:38 smithi002 ceph-mon[31538]: pgmap v91: 97 pgs: 97 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 6.5 MiB/s wr, 586 op/s 2024-05-14T16:57:38.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:57:38 smithi151 ceph-mon[36066]: pgmap v91: 97 pgs: 97 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 6.5 MiB/s wr, 586 op/s 2024-05-14T16:57:38.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:57:38 smithi123 ceph-mon[35971]: pgmap v91: 97 pgs: 97 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 6.5 MiB/s wr, 586 op/s 2024-05-14T16:57:40.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:57:40 smithi002 ceph-mon[31538]: pgmap v92: 97 pgs: 97 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 7.7 MiB/s wr, 653 op/s 2024-05-14T16:57:40.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:57:40 smithi151 ceph-mon[36066]: pgmap v92: 97 pgs: 97 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 7.7 MiB/s wr, 653 op/s 2024-05-14T16:57:40.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:57:40 smithi123 ceph-mon[35971]: pgmap v92: 97 pgs: 97 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 7.7 MiB/s wr, 653 op/s 2024-05-14T16:57:41.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 145.0 s, 3773.6 tps, lat 26.520 ms stddev 28.631 2024-05-14T16:57:42.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:57:42 smithi002 ceph-mon[31538]: pgmap v93: 97 pgs: 97 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 7.2 MiB/s wr, 595 op/s 2024-05-14T16:57:43.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:57:42 smithi123 ceph-mon[35971]: pgmap v93: 97 pgs: 97 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 7.2 MiB/s wr, 595 op/s 2024-05-14T16:57:43.320 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:57:42 smithi151 ceph-mon[36066]: pgmap v93: 97 pgs: 97 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 7.2 MiB/s wr, 595 op/s 2024-05-14T16:57:44.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:57:44 smithi002 ceph-mon[31538]: pgmap v94: 97 pgs: 97 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 10 MiB/s wr, 914 op/s 2024-05-14T16:57:44.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:57:44 smithi151 ceph-mon[36066]: pgmap v94: 97 pgs: 97 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 10 MiB/s wr, 914 op/s 2024-05-14T16:57:44.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:57:44 smithi123 ceph-mon[35971]: pgmap v94: 97 pgs: 97 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 10 MiB/s wr, 914 op/s 2024-05-14T16:57:46.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:57:46 smithi002 ceph-mon[31538]: pgmap v95: 97 pgs: 97 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.9 MiB/s wr, 654 op/s 2024-05-14T16:57:46.753 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 150.0 s, 3939.4 tps, lat 25.354 ms stddev 27.155 2024-05-14T16:57:46.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:57:46 smithi151 ceph-mon[36066]: pgmap v95: 97 pgs: 97 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.9 MiB/s wr, 654 op/s 2024-05-14T16:57:46.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:57:46 smithi123 ceph-mon[35971]: pgmap v95: 97 pgs: 97 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.9 MiB/s wr, 654 op/s 2024-05-14T16:57:48.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:57:48 smithi002 ceph-mon[31538]: pgmap v96: 97 pgs: 97 active+clean; 1.6 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 856 op/s 2024-05-14T16:57:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.833+0000 7fd34f072640 1 -- 172.21.15.2:0/1367903554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd348111240 msgr2=0x7fd348109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.833+0000 7fd34f072640 1 --2- 172.21.15.2:0/1367903554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd348111240 0x7fd348109a50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd33c01a940 tx=0x7fd33c03b4e0 comp rx=0 tx=0).stop 2024-05-14T16:57:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.833+0000 7fd34f072640 1 -- 172.21.15.2:0/1367903554 shutdown_connections 2024-05-14T16:57:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.833+0000 7fd34f072640 1 --2- 172.21.15.2:0/1367903554 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd34811c230 0x7fd34811e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.833+0000 7fd34f072640 1 --2- 172.21.15.2:0/1367903554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd348111620 0x7fd348109f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:48.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.833+0000 7fd34f072640 1 --2- 172.21.15.2:0/1367903554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd348111240 0x7fd348109a50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.833+0000 7fd34f072640 1 -- 172.21.15.2:0/1367903554 >> 172.21.15.2:0/1367903554 conn(0x7fd34806db20 msgr2=0x7fd34806df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:48.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.833+0000 7fd34f072640 1 -- 172.21.15.2:0/1367903554 shutdown_connections 2024-05-14T16:57:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.833+0000 7fd34f072640 1 -- 172.21.15.2:0/1367903554 wait complete. 2024-05-14T16:57:48.902 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:57:48 smithi151 ceph-mon[36066]: pgmap v96: 97 pgs: 97 active+clean; 1.6 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 856 op/s 2024-05-14T16:57:48.913 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:57:48 smithi123 ceph-mon[35971]: pgmap v96: 97 pgs: 97 active+clean; 1.6 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 856 op/s 2024-05-14T16:57:48.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.835+0000 7fd34f072640 1 Processor -- start 2024-05-14T16:57:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.835+0000 7fd34f072640 1 -- start start 2024-05-14T16:57:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.836+0000 7fd34f072640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd348111240 0x7fd34811b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.836+0000 7fd34f072640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd34811c230 0x7fd348117a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.836+0000 7fd34f072640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd348117f80 0x7fd348118430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:48.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.836+0000 7fd34f072640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd3481143a0 con 0x7fd348117f80 2024-05-14T16:57:49.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.836+0000 7fd34f072640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd348114220 con 0x7fd348111240 2024-05-14T16:57:49.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.836+0000 7fd34f072640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd348114520 con 0x7fd34811c230 2024-05-14T16:57:49.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.836+0000 7fd34ce0e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd348111240 0x7fd34811b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:49.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.836+0000 7fd34ce0e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd348111240 0x7fd34811b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:57526/0 (socket says 172.21.15.2:57526) 2024-05-14T16:57:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.837+0000 7fd34ce0e640 1 -- 172.21.15.2:0/253980958 learned_addr learned my addr 172.21.15.2:0/253980958 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:57:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.837+0000 7fd34d60f640 1 --2- 172.21.15.2:0/253980958 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd348117f80 0x7fd348118430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.837+0000 7fd347fff640 1 --2- 172.21.15.2:0/253980958 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd34811c230 0x7fd348117a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:49.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.837+0000 7fd34ce0e640 1 -- 172.21.15.2:0/253980958 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd34811c230 msgr2=0x7fd348117a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.837+0000 7fd34ce0e640 1 --2- 172.21.15.2:0/253980958 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd34811c230 0x7fd348117a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:49.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.837+0000 7fd34ce0e640 1 -- 172.21.15.2:0/253980958 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd348117f80 msgr2=0x7fd348118430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.837+0000 7fd34ce0e640 1 --2- 172.21.15.2:0/253980958 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd348117f80 0x7fd348118430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.837+0000 7fd34ce0e640 1 -- 172.21.15.2:0/253980958 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3481ac050 con 0x7fd348111240 2024-05-14T16:57:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.838+0000 7fd34ce0e640 1 --2- 172.21.15.2:0/253980958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd348111240 0x7fd34811b9b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd33c01a910 tx=0x7fd33c004a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:49.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.839+0000 7fd345ffb640 1 -- 172.21.15.2:0/253980958 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c04f070 con 0x7fd348111240 2024-05-14T16:57:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.839+0000 7fd34f072640 1 -- 172.21.15.2:0/253980958 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3481ac2e0 con 0x7fd348111240 2024-05-14T16:57:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.839+0000 7fd34f072640 1 -- 172.21.15.2:0/253980958 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3481ac7d0 con 0x7fd348111240 2024-05-14T16:57:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.839+0000 7fd345ffb640 1 -- 172.21.15.2:0/253980958 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd33c040bd0 con 0x7fd348111240 2024-05-14T16:57:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.839+0000 7fd345ffb640 1 -- 172.21.15.2:0/253980958 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c046030 con 0x7fd348111240 2024-05-14T16:57:49.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.846+0000 7fd351d8c740 1 Processor -- start 2024-05-14T16:57:49.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.847+0000 7fd345ffb640 1 -- 172.21.15.2:0/253980958 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fd33c052070 con 0x7fd348111240 2024-05-14T16:57:49.236 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.847+0000 7fd351d8c740 1 -- start start 2024-05-14T16:57:49.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.848+0000 7fd351d8c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa4bd818f0 0x55fa4bd81cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:49.250 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.848+0000 7fd351d8c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa4bd82210 0x55fa4bd8c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:49.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.848+0000 7fd351d8c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa4bd8cad0 0x55fa4bd8eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:49.263 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.848+0000 7fd351d8c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55fa4bcf4c30 con 0x55fa4bd8cad0 2024-05-14T16:57:49.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.848+0000 7fd351d8c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fa4bce47d0 con 0x55fa4bd82210 2024-05-14T16:57:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.848+0000 7fd351d8c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55fa4bce8810 con 0x55fa4bd818f0 2024-05-14T16:57:49.283 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.848+0000 7fd34ce0e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa4bd82210 0x55fa4bd8c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.848+0000 7fd34d60f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa4bd818f0 0x55fa4bd81cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:49.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.848+0000 7fd34ce0e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa4bd82210 0x55fa4bd8c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:57532/0 (socket says 172.21.15.2:57532) 2024-05-14T16:57:49.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.848+0000 7fd34ce0e640 1 -- 172.21.15.2:0/2446172710 learned_addr learned my addr 172.21.15.2:0/2446172710 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:57:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.848+0000 7fd347fff640 1 --2- 172.21.15.2:0/2446172710 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa4bd8cad0 0x55fa4bd8eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.849+0000 7fd34d60f640 1 -- 172.21.15.2:0/2446172710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa4bd82210 msgr2=0x55fa4bd8c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.849+0000 7fd34d60f640 1 --2- 172.21.15.2:0/2446172710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa4bd82210 0x55fa4bd8c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.849+0000 7fd34d60f640 1 -- 172.21.15.2:0/2446172710 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa4bd8cad0 msgr2=0x55fa4bd8eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:49.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.849+0000 7fd34d60f640 1 --2- 172.21.15.2:0/2446172710 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa4bd8cad0 0x55fa4bd8eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.849+0000 7fd34d60f640 1 -- 172.21.15.2:0/2446172710 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa4b9f9210 con 0x55fa4bd818f0 2024-05-14T16:57:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.849+0000 7fd34d60f640 1 --2- 172.21.15.2:0/2446172710 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa4bd818f0 0x55fa4bd81cd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd340009e30 tx=0x7fd34000c7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10c9e89f2c66edd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.850+0000 7fd32effd640 1 -- 172.21.15.2:0/2446172710 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34001a070 con 0x55fa4bd818f0 2024-05-14T16:57:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.850+0000 7fd345ffb640 1 --2- 172.21.15.2:0/253980958 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fd3300b2c20 0x7fd3300b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:49.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.850+0000 7fd32effd640 1 -- 172.21.15.2:0/2446172710 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3400158c0 con 0x55fa4bd818f0 2024-05-14T16:57:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.850+0000 7fd345ffb640 1 -- 172.21.15.2:0/253980958 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fd33c1139c0 con 0x7fd348111240 2024-05-14T16:57:49.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.850+0000 7fd32effd640 1 -- 172.21.15.2:0/2446172710 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34000a820 con 0x55fa4bd818f0 2024-05-14T16:57:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.851+0000 7fd34ce0e640 1 --2- 172.21.15.2:0/253980958 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fd3300b2c20 0x7fd3300b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.852+0000 7fd351d8c740 1 -- 172.21.15.2:0/2446172710 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa4bd818f0 msgr2=0x55fa4bd81cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:49.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.852+0000 7fd351d8c740 1 --2- 172.21.15.2:0/2446172710 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa4bd818f0 0x55fa4bd81cd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd340009e30 tx=0x7fd34000c7f0 comp rx=0 tx=0).stop 2024-05-14T16:57:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.852+0000 7fd34ce0e640 1 --2- 172.21.15.2:0/253980958 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fd3300b2c20 0x7fd3300b50e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd33c0d7ee0 tx=0x7fd33c119530 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.852+0000 7fd351d8c740 1 -- 172.21.15.2:0/2446172710 shutdown_connections 2024-05-14T16:57:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.852+0000 7fd351d8c740 1 --2- 172.21.15.2:0/2446172710 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa4bd8cad0 0x55fa4bd8eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.852+0000 7fd351d8c740 1 --2- 172.21.15.2:0/2446172710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa4bd82210 0x55fa4bd8c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.852+0000 7fd351d8c740 1 --2- 172.21.15.2:0/2446172710 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa4bd818f0 0x55fa4bd81cd0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:49.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.852+0000 7fd351d8c740 1 -- 172.21.15.2:0/2446172710 >> 172.21.15.2:0/2446172710 conn(0x55fa4bd7de00 msgr2=0x55fa4bd7e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.852+0000 7fd351d8c740 1 -- 172.21.15.2:0/2446172710 shutdown_connections 2024-05-14T16:57:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.852+0000 7fd351d8c740 1 -- 172.21.15.2:0/2446172710 wait complete. 2024-05-14T16:57:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.853+0000 7fd351d8c740 1 -- start start 2024-05-14T16:57:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.853+0000 7fd351d8c740 1 Processor -- start 2024-05-14T16:57:49.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.853+0000 7fd351d8c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa4bd818f0 0x55fa4bd8c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.853+0000 7fd351d8c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa4bd82210 0x55fa4bd897a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.853+0000 7fd351d8c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa4bd8cad0 0x55fa4bd89db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.853+0000 7fd351d8c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55fa4b9ca790 con 0x55fa4bd82210 2024-05-14T16:57:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.853+0000 7fd351d8c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fa4bca0890 con 0x55fa4bd8cad0 2024-05-14T16:57:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.853+0000 7fd351d8c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55fa4bd8f7a0 con 0x55fa4bd818f0 2024-05-14T16:57:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.853+0000 7fd351d8c740 10 client.0 _ll_register_callbacks cb 0x55fa4bca3d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:57:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.854+0000 7fd347fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa4bd82210 0x55fa4bd897a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.854+0000 7fd347fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa4bd82210 0x55fa4bd897a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56736/0 (socket says 172.21.15.2:56736) 2024-05-14T16:57:49.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.854+0000 7fd347fff640 1 -- 172.21.15.2:0/1421261147 learned_addr learned my addr 172.21.15.2:0/1421261147 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:57:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.854+0000 7fd34d60f640 1 --2- 172.21.15.2:0/1421261147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa4bd818f0 0x55fa4bd8c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:49.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.854+0000 7fd34d60f640 1 --2- 172.21.15.2:0/1421261147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa4bd8cad0 0x55fa4bd89db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:49.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.854+0000 7fd347fff640 1 -- 172.21.15.2:0/1421261147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa4bd818f0 msgr2=0x55fa4bd8c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.854+0000 7fd347fff640 1 --2- 172.21.15.2:0/1421261147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa4bd818f0 0x55fa4bd8c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:49.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.854+0000 7fd347fff640 1 -- 172.21.15.2:0/1421261147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa4bd8cad0 msgr2=0x55fa4bd89db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:49.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.854+0000 7fd347fff640 1 --2- 172.21.15.2:0/1421261147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa4bd8cad0 0x55fa4bd89db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.854+0000 7fd347fff640 1 -- 172.21.15.2:0/1421261147 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa4bac5620 con 0x55fa4bd82210 2024-05-14T16:57:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.854+0000 7fd34d60f640 1 --2- 172.21.15.2:0/1421261147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa4bd818f0 0x55fa4bd8c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:57:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.854+0000 7fd347fff640 1 --2- 172.21.15.2:0/1421261147 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa4bd82210 0x55fa4bd897a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd3380104d0 tx=0x7fd338010990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=979808371e28ebf0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:49.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.854+0000 7fd327fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:57:49.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.854+0000 7fd327fff640 1 -- 172.21.15.2:0/1421261147 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33801a020 con 0x55fa4bd82210 2024-05-14T16:57:49.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.854+0000 7fd351d8c740 1 -- 172.21.15.2:0/1421261147 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3400158c0 con 0x55fa4bd82210 2024-05-14T16:57:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.854+0000 7fd327fff640 1 -- 172.21.15.2:0/1421261147 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fd338016750 con 0x55fa4bd82210 2024-05-14T16:57:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.855+0000 7fd327fff640 1 -- 172.21.15.2:0/1421261147 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd338025050 con 0x55fa4bd82210 2024-05-14T16:57:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.855+0000 7fd351d8c740 10 client.15687 fetch_fsmap learned FSMap version 31 2024-05-14T16:57:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.855+0000 7fd351d8c740 1 -- 172.21.15.2:0/1421261147 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fa4bd9b370 con 0x55fa4bd82210 2024-05-14T16:57:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.855+0000 7fd327fff640 1 -- 172.21.15.2:0/1421261147 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3400158c0 con 0x55fa4bd82210 2024-05-14T16:57:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.855+0000 7fd327fff640 1 -- 172.21.15.2:0/1421261147 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fd338014a30 con 0x55fa4bd82210 2024-05-14T16:57:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.855+0000 7fd351d8c740 10 client.15687 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T16:57:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.855+0000 7fd351d8c740 10 client.15687 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:57:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.855+0000 7fd351d8c740 20 client.15687 populate_metadata read hostname 'smithi002' 2024-05-14T16:57:49.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.855+0000 7fd351d8c740 1 --2- 172.21.15.2:0/1421261147 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55fa4bd9c990 0x55fa4bd9edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:49.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.855+0000 7fd351d8c740 4 client.15687 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:57:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.855+0000 7fd351d8c740 1 -- 172.21.15.2:0/1421261147 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa4bcf6b80 con 0x55fa4bd9c990 2024-05-14T16:57:49.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.856+0000 7fd34d60f640 1 --2- 172.21.15.2:0/1421261147 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55fa4bd9c990 0x55fa4bd9edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:49.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.857+0000 7fd34d60f640 1 --2- 172.21.15.2:0/1421261147 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55fa4bd9c990 0x55fa4bd9edd0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5a61471de7bbb01 server_cookie=2ef49b2e3e1e18c2 in_seq=0 out_seq=0 2024-05-14T16:57:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.857+0000 7fd327fff640 10 client.15687 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:57:49.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.857+0000 7fd327fff640 1 -- 172.21.15.2:0/1421261147 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fa4bcf6b80 con 0x55fa4bd9c990 2024-05-14T16:57:49.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.857+0000 7fd327fff640 10 client.15687 handle_command_reply: tid=0 2024-05-14T16:57:49.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.857+0000 7fd351d8c740 1 client.15687 shutdown 2024-05-14T16:57:49.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.857+0000 7fd351d8c740 1 -- 172.21.15.2:0/1421261147 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa4bd82210 msgr2=0x55fa4bd897a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.857+0000 7fd351d8c740 1 --2- 172.21.15.2:0/1421261147 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa4bd82210 0x55fa4bd897a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd3380104d0 tx=0x7fd338010990 comp rx=0 tx=0).stop 2024-05-14T16:57:49.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.858+0000 7fd351d8c740 1 -- 172.21.15.2:0/1421261147 shutdown_connections 2024-05-14T16:57:49.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.858+0000 7fd351d8c740 1 --2- 172.21.15.2:0/1421261147 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55fa4bd9c990 0x55fa4bd9edd0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.858+0000 7fd351d8c740 1 --2- 172.21.15.2:0/1421261147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa4bd8cad0 0x55fa4bd89db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:50.011 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.858+0000 7fd327fff640 0 client.15687 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:57:50.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.858+0000 7fd351d8c740 1 --2- 172.21.15.2:0/1421261147 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa4bd82210 0x55fa4bd897a0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:50.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.858+0000 7fd351d8c740 1 --2- 172.21.15.2:0/1421261147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa4bd818f0 0x55fa4bd8c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:50.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.858+0000 7fd351d8c740 1 -- 172.21.15.2:0/1421261147 >> 172.21.15.2:0/1421261147 conn(0x55fa4bd7de00 msgr2=0x55fa4bd7d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.858+0000 7fd351d8c740 1 -- 172.21.15.2:0/1421261147 shutdown_connections 2024-05-14T16:57:50.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.858+0000 7fd351d8c740 1 -- 172.21.15.2:0/1421261147 wait complete. 2024-05-14T16:57:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.858+0000 7fd351d8c740 20 client.15687 trim_cache size 0 max 16384 2024-05-14T16:57:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.882+0000 7fd351d8c740 1 Processor -- start 2024-05-14T16:57:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.882+0000 7fd351d8c740 1 -- start start 2024-05-14T16:57:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.883+0000 7fd351d8c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa4bd818f0 0x55fa4bd84c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.883+0000 7fd351d8c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa4bd82210 0x55fa4bd986d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.883+0000 7fd351d8c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa4bd8cad0 0x55fa4bd8bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.883+0000 7fd34d60f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa4bd818f0 0x55fa4bd84c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.883+0000 7fd34d60f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa4bd818f0 0x55fa4bd84c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:52294/0 (socket says 172.21.15.2:52294) 2024-05-14T16:57:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.883+0000 7fd34d60f640 1 -- 172.21.15.2:0/2842716398 learned_addr learned my addr 172.21.15.2:0/2842716398 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:57:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.883+0000 7fd351d8c740 1 -- 172.21.15.2:0/2842716398 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd334010ff0 con 0x55fa4bd82210 2024-05-14T16:57:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.883+0000 7fd351d8c740 1 -- 172.21.15.2:0/2842716398 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd33400e860 con 0x55fa4bd8cad0 2024-05-14T16:57:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.883+0000 7fd351d8c740 1 -- 172.21.15.2:0/2842716398 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd33400c6b0 con 0x55fa4bd818f0 2024-05-14T16:57:50.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.883+0000 7fd34d60f640 1 --2- 172.21.15.2:0/2842716398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa4bd8cad0 0x55fa4bd8bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.884+0000 7fd34d60f640 1 -- 172.21.15.2:0/2842716398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa4bd8cad0 msgr2=0x55fa4bd8bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.884+0000 7fd34d60f640 1 --2- 172.21.15.2:0/2842716398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa4bd8cad0 0x55fa4bd8bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:50.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.884+0000 7fd34d60f640 1 -- 172.21.15.2:0/2842716398 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa4bd82210 msgr2=0x55fa4bd986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.884+0000 7fd34d60f640 1 --2- 172.21.15.2:0/2842716398 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa4bd82210 0x55fa4bd986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:50.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.884+0000 7fd34d60f640 1 -- 172.21.15.2:0/2842716398 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa4bd8a120 con 0x55fa4bd818f0 2024-05-14T16:57:50.330 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.884+0000 7fd34d60f640 1 --2- 172.21.15.2:0/2842716398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa4bd818f0 0x55fa4bd84c40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd34000ccf0 tx=0x7fd34000c7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d80eecc9177ebb36 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.884+0000 7fd32e7fc640 1 -- 172.21.15.2:0/2842716398 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34001a040 con 0x55fa4bd818f0 2024-05-14T16:57:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.884+0000 7fd32e7fc640 1 -- 172.21.15.2:0/2842716398 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd340015830 con 0x55fa4bd818f0 2024-05-14T16:57:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.884+0000 7fd32e7fc640 1 -- 172.21.15.2:0/2842716398 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd340025410 con 0x55fa4bd818f0 2024-05-14T16:57:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.885+0000 7fd351d8c740 1 -- 172.21.15.2:0/2842716398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa4bd818f0 msgr2=0x55fa4bd84c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.885+0000 7fd351d8c740 1 --2- 172.21.15.2:0/2842716398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa4bd818f0 0x55fa4bd84c40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd34000ccf0 tx=0x7fd34000c7f0 comp rx=0 tx=0).stop 2024-05-14T16:57:50.378 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.885+0000 7fd351d8c740 1 -- 172.21.15.2:0/2842716398 shutdown_connections 2024-05-14T16:57:50.385 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.885+0000 7fd351d8c740 1 --2- 172.21.15.2:0/2842716398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa4bd8cad0 0x55fa4bd8bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.885+0000 7fd351d8c740 1 --2- 172.21.15.2:0/2842716398 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa4bd82210 0x55fa4bd986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:50.401 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.885+0000 7fd351d8c740 1 --2- 172.21.15.2:0/2842716398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa4bd818f0 0x55fa4bd84c40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:50.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.885+0000 7fd351d8c740 1 -- 172.21.15.2:0/2842716398 >> 172.21.15.2:0/2842716398 conn(0x55fa4bd7de00 msgr2=0x55fa4bd84530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.885+0000 7fd351d8c740 1 -- 172.21.15.2:0/2842716398 shutdown_connections 2024-05-14T16:57:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.885+0000 7fd351d8c740 1 -- 172.21.15.2:0/2842716398 wait complete. 2024-05-14T16:57:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.886+0000 7fd351d8c740 1 -- start start 2024-05-14T16:57:50.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.886+0000 7fd351d8c740 1 Processor -- start 2024-05-14T16:57:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.886+0000 7fd351d8c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa4bd818f0 0x55fa4bdff590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.886+0000 7fd351d8c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa4bd82210 0x55fa4bdffad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:50.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.886+0000 7fd351d8c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa4bd8cad0 0x55fa4bd8fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:50.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.886+0000 7fd351d8c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd33400cd20 con 0x55fa4bd82210 2024-05-14T16:57:50.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.886+0000 7fd351d8c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd33400e190 con 0x55fa4bd8cad0 2024-05-14T16:57:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.886+0000 7fd351d8c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd33400d3f0 con 0x55fa4bd818f0 2024-05-14T16:57:50.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.886+0000 7fd351d8c740 10 client.0 _ll_register_callbacks cb 0x7fd33800bd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:57:50.550 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.886+0000 7fd347fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa4bd82210 0x55fa4bdffad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:50.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.886+0000 7fd347fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa4bd82210 0x55fa4bdffad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56762/0 (socket says 172.21.15.2:56762) 2024-05-14T16:57:50.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.886+0000 7fd347fff640 1 -- 172.21.15.2:0/518445082 learned_addr learned my addr 172.21.15.2:0/518445082 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:57:50.578 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.887+0000 7fd34d60f640 1 --2- 172.21.15.2:0/518445082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa4bd8cad0 0x55fa4bd8fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:50.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.887+0000 7fd347fff640 1 -- 172.21.15.2:0/518445082 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa4bd818f0 msgr2=0x55fa4bdff590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:50.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.887+0000 7fd347fff640 1 --2- 172.21.15.2:0/518445082 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa4bd818f0 0x55fa4bdff590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:50.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.887+0000 7fd347fff640 1 -- 172.21.15.2:0/518445082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa4bd8cad0 msgr2=0x55fa4bd8fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:57:50.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.887+0000 7fd347fff640 1 --2- 172.21.15.2:0/518445082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa4bd8cad0 0x55fa4bd8fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:50.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.887+0000 7fd347fff640 1 -- 172.21.15.2:0/518445082 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa4bd90510 con 0x55fa4bd82210 2024-05-14T16:57:50.661 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.887+0000 7fd34d60f640 1 --2- 172.21.15.2:0/518445082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa4bd8cad0 0x55fa4bd8fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:57:50.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.887+0000 7fd347fff640 1 --2- 172.21.15.2:0/518445082 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa4bd82210 0x55fa4bdffad0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd338014e70 tx=0x7fd338012670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e589eba1ed8b8c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:57:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.887+0000 7fd327fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:57:50.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.887+0000 7fd327fff640 1 -- 172.21.15.2:0/518445082 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33801a050 con 0x55fa4bd82210 2024-05-14T16:57:50.693 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.887+0000 7fd327fff640 1 -- 172.21.15.2:0/518445082 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fd338012a00 con 0x55fa4bd82210 2024-05-14T16:57:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.887+0000 7fd351d8c740 1 -- 172.21.15.2:0/518445082 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd340015830 con 0x55fa4bd82210 2024-05-14T16:57:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.887+0000 7fd327fff640 1 -- 172.21.15.2:0/518445082 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd338010e90 con 0x55fa4bd82210 2024-05-14T16:57:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.887+0000 7fd351d8c740 10 client.15690 fetch_fsmap learned FSMap version 31 2024-05-14T16:57:50.727 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.887+0000 7fd351d8c740 1 -- 172.21.15.2:0/518445082 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fa4be06d40 con 0x55fa4bd82210 2024-05-14T16:57:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.887+0000 7fd327fff640 1 -- 172.21.15.2:0/518445082 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd340015830 con 0x55fa4bd82210 2024-05-14T16:57:50.744 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.887+0000 7fd327fff640 1 -- 172.21.15.2:0/518445082 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fd338014430 con 0x55fa4bd82210 2024-05-14T16:57:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.888+0000 7fd351d8c740 10 client.15690 fetch_fsmap finished waiting for FSMap version 31 2024-05-14T16:57:50.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.888+0000 7fd351d8c740 10 client.15690 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:57:50.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.888+0000 7fd351d8c740 20 client.15690 populate_metadata read hostname 'smithi002' 2024-05-14T16:57:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.888+0000 7fd351d8c740 1 --2- 172.21.15.2:0/518445082 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55fa4be074c0 0x55fa4be098e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:57:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.888+0000 7fd351d8c740 4 client.15690 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:57:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.888+0000 7fd351d8c740 1 -- 172.21.15.2:0/518445082 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fa4bcf0360 con 0x55fa4be074c0 2024-05-14T16:57:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.889+0000 7fd34d60f640 1 --2- 172.21.15.2:0/518445082 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55fa4be074c0 0x55fa4be098e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:57:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.889+0000 7fd34d60f640 1 --2- 172.21.15.2:0/518445082 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55fa4be074c0 0x55fa4be098e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1655bdc855115fb server_cookie=57aef455df9c0366 in_seq=0 out_seq=0 2024-05-14T16:57:50.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.889+0000 7fd327fff640 10 client.15690 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:57:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.902+0000 7fd327fff640 1 -- 172.21.15.2:0/518445082 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55fa4bcf0360 con 0x55fa4be074c0 2024-05-14T16:57:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.902+0000 7fd327fff640 10 client.15690 handle_command_reply: tid=0 2024-05-14T16:57:50.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.902+0000 7fd351d8c740 1 client.15690 shutdown 2024-05-14T16:57:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.902+0000 7fd351d8c740 1 -- 172.21.15.2:0/518445082 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa4bd82210 msgr2=0x55fa4bdffad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.902+0000 7fd351d8c740 1 --2- 172.21.15.2:0/518445082 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa4bd82210 0x55fa4bdffad0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd338014e70 tx=0x7fd338012670 comp rx=0 tx=0).stop 2024-05-14T16:57:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.902+0000 7fd351d8c740 1 -- 172.21.15.2:0/518445082 shutdown_connections 2024-05-14T16:57:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.902+0000 7fd351d8c740 1 --2- 172.21.15.2:0/518445082 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55fa4be074c0 0x55fa4be098e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.902+0000 7fd351d8c740 1 --2- 172.21.15.2:0/518445082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa4bd8cad0 0x55fa4bd8fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.902+0000 7fd327fff640 0 client.15690 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:57:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.902+0000 7fd351d8c740 1 --2- 172.21.15.2:0/518445082 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fa4bd82210 0x55fa4bdffad0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:50.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.902+0000 7fd351d8c740 1 --2- 172.21.15.2:0/518445082 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fa4bd818f0 0x55fa4bdff590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.902+0000 7fd351d8c740 1 -- 172.21.15.2:0/518445082 >> 172.21.15.2:0/518445082 conn(0x55fa4bd7de00 msgr2=0x55fa4bd7d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.903+0000 7fd351d8c740 1 -- 172.21.15.2:0/518445082 shutdown_connections 2024-05-14T16:57:51.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.903+0000 7fd351d8c740 1 -- 172.21.15.2:0/518445082 wait complete. 2024-05-14T16:57:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.903+0000 7fd351d8c740 20 client.15690 trim_cache size 0 max 16384 2024-05-14T16:57:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T16:57:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T16:57:51.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T16:57:51.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.903+0000 7fd34f072640 1 -- 172.21.15.2:0/253980958 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fd3300b2c20 msgr2=0x7fd3300b50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.903+0000 7fd34f072640 1 --2- 172.21.15.2:0/253980958 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fd3300b2c20 0x7fd3300b50e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd33c0d7ee0 tx=0x7fd33c119530 comp rx=0 tx=0).stop 2024-05-14T16:57:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.903+0000 7fd34f072640 1 -- 172.21.15.2:0/253980958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd348111240 msgr2=0x7fd34811b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:57:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.903+0000 7fd34f072640 1 --2- 172.21.15.2:0/253980958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd348111240 0x7fd34811b9b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd33c01a910 tx=0x7fd33c004a50 comp rx=0 tx=0).stop 2024-05-14T16:57:51.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.903+0000 7fd34f072640 1 -- 172.21.15.2:0/253980958 shutdown_connections 2024-05-14T16:57:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.903+0000 7fd34f072640 1 --2- 172.21.15.2:0/253980958 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fd3300b2c20 0x7fd3300b50e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.903+0000 7fd34f072640 1 --2- 172.21.15.2:0/253980958 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd348117f80 0x7fd348118430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:51.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.903+0000 7fd34f072640 1 --2- 172.21.15.2:0/253980958 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd34811c230 0x7fd348117a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.903+0000 7fd34f072640 1 --2- 172.21.15.2:0/253980958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd348111240 0x7fd34811b9b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:57:51.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.903+0000 7fd34f072640 1 -- 172.21.15.2:0/253980958 >> 172.21.15.2:0/253980958 conn(0x7fd34806db20 msgr2=0x7fd34811c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:57:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.904+0000 7fd34f072640 1 -- 172.21.15.2:0/253980958 shutdown_connections 2024-05-14T16:57:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:57:48.904+0000 7fd34f072640 1 -- 172.21.15.2:0/253980958 wait complete. 2024-05-14T16:57:51.174 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:57:50 smithi002 ceph-mon[31538]: pgmap v97: 97 pgs: 97 active+clean; 1.6 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.6 MiB/s wr, 788 op/s 2024-05-14T16:57:51.185 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:57:50 smithi123 ceph-mon[35971]: pgmap v97: 97 pgs: 97 active+clean; 1.6 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.6 MiB/s wr, 788 op/s 2024-05-14T16:57:51.193 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:57:50 smithi151 ceph-mon[36066]: pgmap v97: 97 pgs: 97 active+clean; 1.6 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.6 MiB/s wr, 788 op/s 2024-05-14T16:57:51.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:57:51 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:57:51.753 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 155.0 s, 3787.8 tps, lat 26.379 ms stddev 27.165 2024-05-14T16:57:51.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:57:51 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:57:51.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:57:51 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:57:52.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:57:52 smithi002 ceph-mon[31538]: pgmap v98: 97 pgs: 97 active+clean; 1.6 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.8 MiB/s wr, 641 op/s 2024-05-14T16:57:52.751 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:57:52 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:57:52.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:57:52 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:57:52.763 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:57:52 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:57:52.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:57:52 smithi151 ceph-mon[36066]: pgmap v98: 97 pgs: 97 active+clean; 1.6 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.8 MiB/s wr, 641 op/s 2024-05-14T16:57:52.860 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:57:52 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:57:52.866 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:57:52 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:57:52.884 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:57:52 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:57:52.948 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:57:52 smithi123 ceph-mon[35971]: pgmap v98: 97 pgs: 97 active+clean; 1.6 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.8 MiB/s wr, 641 op/s 2024-05-14T16:57:52.960 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:57:52 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:57:52.997 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:57:52 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:57:53.335 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:57:52 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:57:54.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:57:54 smithi151 ceph-mon[36066]: pgmap v99: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 968 op/s 2024-05-14T16:57:54.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:57:54 smithi123 ceph-mon[35971]: pgmap v99: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 968 op/s 2024-05-14T16:57:54.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:57:54 smithi002 ceph-mon[31538]: pgmap v99: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 968 op/s 2024-05-14T16:57:56.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 160.0 s, 3740.0 tps, lat 26.744 ms stddev 27.333 2024-05-14T16:57:56.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:57:56 smithi151 ceph-mon[36066]: pgmap v100: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.8 MiB/s wr, 649 op/s 2024-05-14T16:57:56.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:57:56 smithi123 ceph-mon[35971]: pgmap v100: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.8 MiB/s wr, 649 op/s 2024-05-14T16:57:56.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:57:56 smithi002 ceph-mon[31538]: pgmap v100: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.8 MiB/s wr, 649 op/s 2024-05-14T16:57:58.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:57:58 smithi151 ceph-mon[36066]: pgmap v101: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 845 op/s 2024-05-14T16:57:58.861 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:57:58 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:57:58.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:57:58 smithi123 ceph-mon[35971]: pgmap v101: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 845 op/s 2024-05-14T16:57:58.947 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:57:58 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:57:58.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:57:58 smithi002 ceph-mon[31538]: pgmap v101: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 845 op/s 2024-05-14T16:57:59.003 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:57:58 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:58:00.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:58:00 smithi151 ceph-mon[36066]: pgmap v102: 97 pgs: 97 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 10 MiB/s wr, 794 op/s 2024-05-14T16:58:00.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:58:00 smithi123 ceph-mon[35971]: pgmap v102: 97 pgs: 97 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 10 MiB/s wr, 794 op/s 2024-05-14T16:58:00.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:58:00 smithi002 ceph-mon[31538]: pgmap v102: 97 pgs: 97 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 10 MiB/s wr, 794 op/s 2024-05-14T16:58:01.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 165.0 s, 3257.6 tps, lat 29.148 ms stddev 35.890 2024-05-14T16:58:02.011 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.009+0000 7f5a3de2a640 1 -- 172.21.15.2:0/3342130165 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a38102180 msgr2=0x7f5a3810eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:02.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.009+0000 7f5a3de2a640 1 --2- 172.21.15.2:0/3342130165 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a38102180 0x7f5a3810eed0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c009900 tx=0x7f5a2c02a410 comp rx=0 tx=0).stop 2024-05-14T16:58:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.009+0000 7f5a3de2a640 1 -- 172.21.15.2:0/3342130165 shutdown_connections 2024-05-14T16:58:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.009+0000 7f5a3de2a640 1 --2- 172.21.15.2:0/3342130165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a3810f410 0x7f5a38111800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.009+0000 7f5a3de2a640 1 --2- 172.21.15.2:0/3342130165 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a38102180 0x7f5a3810eed0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:02.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.009+0000 7f5a3de2a640 1 --2- 172.21.15.2:0/3342130165 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a38101860 0x7f5a38101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:02.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.009+0000 7f5a3de2a640 1 -- 172.21.15.2:0/3342130165 >> 172.21.15.2:0/3342130165 conn(0x7f5a38078ad0 msgr2=0x7f5a38078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.009+0000 7f5a3de2a640 1 -- 172.21.15.2:0/3342130165 shutdown_connections 2024-05-14T16:58:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.009+0000 7f5a3de2a640 1 -- 172.21.15.2:0/3342130165 wait complete. 2024-05-14T16:58:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.010+0000 7f5a3de2a640 1 Processor -- start 2024-05-14T16:58:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.010+0000 7f5a3de2a640 1 -- start start 2024-05-14T16:58:02.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.010+0000 7f5a3de2a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a38101860 0x7f5a3819b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.010+0000 7f5a3de2a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a38102180 0x7f5a3819b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.010+0000 7f5a3de2a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a3810f410 0x7f5a3819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.010+0000 7f5a3de2a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5a38114ef0 con 0x7f5a3810f410 2024-05-14T16:58:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.010+0000 7f5a3de2a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5a38114d70 con 0x7f5a38102180 2024-05-14T16:58:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.010+0000 7f5a3de2a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5a38115070 con 0x7f5a38101860 2024-05-14T16:58:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.011+0000 7f5a37fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a3810f410 0x7f5a3819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.011+0000 7f5a377fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a38101860 0x7f5a3819b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.011+0000 7f5a36ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a38102180 0x7f5a3819b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:02.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.011+0000 7f5a37fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a3810f410 0x7f5a3819fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55798/0 (socket says 172.21.15.2:55798) 2024-05-14T16:58:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.011+0000 7f5a37fff640 1 -- 172.21.15.2:0/2261041319 learned_addr learned my addr 172.21.15.2:0/2261041319 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:58:02.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.011+0000 7f5a377fe640 1 --2- 172.21.15.2:0/2261041319 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a38101860 0x7f5a3819b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:43494/0 (socket says 172.21.15.2:43494) 2024-05-14T16:58:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.011+0000 7f5a36ffd640 1 -- 172.21.15.2:0/2261041319 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a38101860 msgr2=0x7f5a3819b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.011+0000 7f5a36ffd640 1 --2- 172.21.15.2:0/2261041319 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a38101860 0x7f5a3819b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:02.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.011+0000 7f5a36ffd640 1 -- 172.21.15.2:0/2261041319 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a3810f410 msgr2=0x7f5a3819fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.011+0000 7f5a36ffd640 1 --2- 172.21.15.2:0/2261041319 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a3810f410 0x7f5a3819fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.011+0000 7f5a36ffd640 1 -- 172.21.15.2:0/2261041319 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a381a03f0 con 0x7f5a38102180 2024-05-14T16:58:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.011+0000 7f5a37fff640 1 --2- 172.21.15.2:0/2261041319 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a3810f410 0x7f5a3819fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:58:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.011+0000 7f5a36ffd640 1 --2- 172.21.15.2:0/2261041319 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a38102180 0x7f5a3819b990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c02a930 tx=0x7f5a2c005e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.011+0000 7f5a34ff9640 1 -- 172.21.15.2:0/2261041319 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c03f030 con 0x7f5a38102180 2024-05-14T16:58:02.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.011+0000 7f5a34ff9640 1 -- 172.21.15.2:0/2261041319 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a2c004490 con 0x7f5a38102180 2024-05-14T16:58:02.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.011+0000 7f5a34ff9640 1 -- 172.21.15.2:0/2261041319 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c035070 con 0x7f5a38102180 2024-05-14T16:58:02.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.011+0000 7f5a3de2a640 1 -- 172.21.15.2:0/2261041319 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a381a0680 con 0x7f5a38102180 2024-05-14T16:58:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.011+0000 7f5a3de2a640 1 -- 172.21.15.2:0/2261041319 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a381a7f70 con 0x7f5a38102180 2024-05-14T16:58:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.014+0000 7f5a34ff9640 1 -- 172.21.15.2:0/2261041319 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f5a2c042030 con 0x7f5a38102180 2024-05-14T16:58:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.015+0000 7f5a40c26740 1 Processor -- start 2024-05-14T16:58:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.015+0000 7f5a40c26740 1 -- start start 2024-05-14T16:58:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.015+0000 7f5a40c26740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649037ce8f0 0x5649037cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.015+0000 7f5a40c26740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649037cf210 0x5649037d9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:02.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.015+0000 7f5a40c26740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5649037d9ad0 0x5649037dbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.015+0000 7f5a40c26740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x564903741c30 con 0x5649037ce8f0 2024-05-14T16:58:02.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.015+0000 7f5a40c26740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5649037317d0 con 0x5649037cf210 2024-05-14T16:58:02.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.015+0000 7f5a40c26740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564903735810 con 0x5649037d9ad0 2024-05-14T16:58:02.549 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.015+0000 7f5a37fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649037ce8f0 0x5649037cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.015+0000 7f5a377fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649037cf210 0x5649037d9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.015+0000 7f5a377fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649037cf210 0x5649037d9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:33776/0 (socket says 172.21.15.2:33776) 2024-05-14T16:58:02.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.015+0000 7f5a377fe640 1 -- 172.21.15.2:0/3724060609 learned_addr learned my addr 172.21.15.2:0/3724060609 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:58:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.016+0000 7f5a34ff9640 1 --2- 172.21.15.2:0/2261041319 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f5a200b2a10 0x7f5a200b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.016+0000 7f5a36ffd640 1 --2- 172.21.15.2:0/3724060609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5649037d9ad0 0x5649037dbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.016+0000 7f5a37fff640 1 --2- 172.21.15.2:0/2261041319 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f5a200b2a10 0x7f5a200b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.016+0000 7f5a34ff9640 1 -- 172.21.15.2:0/2261041319 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f5a2c102880 con 0x7f5a38102180 2024-05-14T16:58:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.016+0000 7f5a377fe640 1 -- 172.21.15.2:0/3724060609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5649037d9ad0 msgr2=0x5649037dbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.016+0000 7f5a377fe640 1 --2- 172.21.15.2:0/3724060609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5649037d9ad0 0x5649037dbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.016+0000 7f5a377fe640 1 -- 172.21.15.2:0/3724060609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649037ce8f0 msgr2=0x5649037cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.016+0000 7f5a377fe640 1 --2- 172.21.15.2:0/3724060609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649037ce8f0 0x5649037cecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.016+0000 7f5a377fe640 1 -- 172.21.15.2:0/3724060609 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564903446210 con 0x5649037cf210 2024-05-14T16:58:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.016+0000 7f5a36ffd640 1 --2- 172.21.15.2:0/3724060609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5649037d9ad0 0x5649037dbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:58:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.016+0000 7f5a37fff640 1 --2- 172.21.15.2:0/3724060609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649037ce8f0 0x5649037cecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:58:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.016+0000 7f5a37fff640 1 --2- 172.21.15.2:0/2261041319 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f5a200b2a10 0x7f5a200b4ed0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5a30009230 tx=0x7f5a30004ab0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.016+0000 7f5a377fe640 1 --2- 172.21.15.2:0/3724060609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649037cf210 0x5649037d9590 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5a28009d20 tx=0x7f5a28011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b06a660570f9103d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.017+0000 7f5a1dffb640 1 -- 172.21.15.2:0/3724060609 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2801e070 con 0x5649037cf210 2024-05-14T16:58:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.017+0000 7f5a1dffb640 1 -- 172.21.15.2:0/3724060609 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a28009e10 con 0x5649037cf210 2024-05-14T16:58:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.017+0000 7f5a1dffb640 1 -- 172.21.15.2:0/3724060609 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2800ea10 con 0x5649037cf210 2024-05-14T16:58:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.018+0000 7f5a40c26740 1 -- 172.21.15.2:0/3724060609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649037cf210 msgr2=0x5649037d9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:02.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.018+0000 7f5a40c26740 1 --2- 172.21.15.2:0/3724060609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649037cf210 0x5649037d9590 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5a28009d20 tx=0x7f5a28011810 comp rx=0 tx=0).stop 2024-05-14T16:58:02.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.018+0000 7f5a40c26740 1 -- 172.21.15.2:0/3724060609 shutdown_connections 2024-05-14T16:58:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.018+0000 7f5a40c26740 1 --2- 172.21.15.2:0/3724060609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5649037d9ad0 0x5649037dbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.018+0000 7f5a40c26740 1 --2- 172.21.15.2:0/3724060609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649037cf210 0x5649037d9590 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.018+0000 7f5a40c26740 1 --2- 172.21.15.2:0/3724060609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649037ce8f0 0x5649037cecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.018+0000 7f5a40c26740 1 -- 172.21.15.2:0/3724060609 >> 172.21.15.2:0/3724060609 conn(0x5649037cae00 msgr2=0x5649037cb1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.018+0000 7f5a40c26740 1 -- 172.21.15.2:0/3724060609 shutdown_connections 2024-05-14T16:58:02.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.018+0000 7f5a40c26740 1 -- 172.21.15.2:0/3724060609 wait complete. 2024-05-14T16:58:02.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.018+0000 7f5a40c26740 1 -- start start 2024-05-14T16:58:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.018+0000 7f5a40c26740 1 Processor -- start 2024-05-14T16:58:02.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.019+0000 7f5a40c26740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649037ce8f0 0x5649037d9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.019+0000 7f5a40c26740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5649037cf210 0x5649037d6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.019+0000 7f5a40c26740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649037d9ad0 0x5649037d6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.019+0000 7f5a40c26740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x564903417790 con 0x5649037ce8f0 2024-05-14T16:58:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.019+0000 7f5a40c26740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5649036ed890 con 0x5649037d9ad0 2024-05-14T16:58:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.019+0000 7f5a40c26740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5649037dc7a0 con 0x5649037cf210 2024-05-14T16:58:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.019+0000 7f5a40c26740 10 client.0 _ll_register_callbacks cb 0x5649036f0d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:58:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.019+0000 7f5a377fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649037ce8f0 0x5649037d9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:02.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.019+0000 7f5a377fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649037ce8f0 0x5649037d9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55810/0 (socket says 172.21.15.2:55810) 2024-05-14T16:58:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.019+0000 7f5a377fe640 1 -- 172.21.15.2:0/2823630413 learned_addr learned my addr 172.21.15.2:0/2823630413 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:58:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.019+0000 7f5a37fff640 1 --2- 172.21.15.2:0/2823630413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649037d9ad0 0x5649037d6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.019+0000 7f5a36ffd640 1 --2- 172.21.15.2:0/2823630413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5649037cf210 0x5649037d6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.020+0000 7f5a37fff640 1 -- 172.21.15.2:0/2823630413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5649037cf210 msgr2=0x5649037d6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.020+0000 7f5a37fff640 1 --2- 172.21.15.2:0/2823630413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5649037cf210 0x5649037d6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.020+0000 7f5a37fff640 1 -- 172.21.15.2:0/2823630413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649037ce8f0 msgr2=0x5649037d9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.020+0000 7f5a37fff640 1 --2- 172.21.15.2:0/2823630413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649037ce8f0 0x5649037d9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.020+0000 7f5a37fff640 1 -- 172.21.15.2:0/2823630413 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564903512620 con 0x5649037d9ad0 2024-05-14T16:58:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.020+0000 7f5a377fe640 1 --2- 172.21.15.2:0/2823630413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649037ce8f0 0x5649037d9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:58:03.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.020+0000 7f5a37fff640 1 --2- 172.21.15.2:0/2823630413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649037d9ad0 0x5649037d6c90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5a30012040 tx=0x7f5a30004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d21debeb0c62daf2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:03.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.020+0000 7f5a16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T16:58:03.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.020+0000 7f5a16ffd640 1 -- 172.21.15.2:0/2823630413 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a30010ac0 con 0x5649037d9ad0 2024-05-14T16:58:03.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.020+0000 7f5a16ffd640 1 -- 172.21.15.2:0/2823630413 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f5a30010c60 con 0x5649037d9ad0 2024-05-14T16:58:03.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.020+0000 7f5a40c26740 1 -- 172.21.15.2:0/2823630413 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a28009e10 con 0x5649037d9ad0 2024-05-14T16:58:03.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.021+0000 7f5a16ffd640 1 -- 172.21.15.2:0/2823630413 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5a30010e60 con 0x5649037d9ad0 2024-05-14T16:58:03.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.021+0000 7f5a40c26740 10 client.25415 fetch_fsmap learned FSMap version 32 2024-05-14T16:58:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.021+0000 7f5a40c26740 1 -- 172.21.15.2:0/2823630413 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5649037e8190 con 0x5649037d9ad0 2024-05-14T16:58:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.021+0000 7f5a16ffd640 1 -- 172.21.15.2:0/2823630413 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a28009e10 con 0x5649037d9ad0 2024-05-14T16:58:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.021+0000 7f5a16ffd640 1 -- 172.21.15.2:0/2823630413 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7f5a30005790 con 0x5649037d9ad0 2024-05-14T16:58:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.021+0000 7f5a40c26740 10 client.25415 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T16:58:03.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.021+0000 7f5a40c26740 10 client.25415 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:58:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.021+0000 7f5a40c26740 20 client.25415 populate_metadata read hostname 'smithi002' 2024-05-14T16:58:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.021+0000 7f5a40c26740 1 --2- 172.21.15.2:0/2823630413 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5649037e96a0 0x5649037ebb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.021+0000 7f5a40c26740 4 client.25415 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:58:03.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.021+0000 7f5a40c26740 1 -- 172.21.15.2:0/2823630413 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564903743b80 con 0x5649037e96a0 2024-05-14T16:58:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.021+0000 7f5a377fe640 1 --2- 172.21.15.2:0/2823630413 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5649037e96a0 0x5649037ebb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.022+0000 7f5a377fe640 1 --2- 172.21.15.2:0/2823630413 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5649037e96a0 0x5649037ebb30 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42dd8741b040e89e server_cookie=eb11fd26a7c1a3e3 in_seq=0 out_seq=0 2024-05-14T16:58:03.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.022+0000 7f5a16ffd640 10 client.25415 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:58:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.022+0000 7f5a16ffd640 1 -- 172.21.15.2:0/2823630413 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564903743b80 con 0x5649037e96a0 2024-05-14T16:58:03.283 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.022+0000 7f5a16ffd640 10 client.25415 handle_command_reply: tid=0 2024-05-14T16:58:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.022+0000 7f5a40c26740 1 client.25415 shutdown 2024-05-14T16:58:03.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.023+0000 7f5a40c26740 1 -- 172.21.15.2:0/2823630413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649037d9ad0 msgr2=0x5649037d6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.023+0000 7f5a40c26740 1 --2- 172.21.15.2:0/2823630413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649037d9ad0 0x5649037d6c90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5a30012040 tx=0x7f5a30004a40 comp rx=0 tx=0).stop 2024-05-14T16:58:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.023+0000 7f5a40c26740 1 -- 172.21.15.2:0/2823630413 shutdown_connections 2024-05-14T16:58:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.023+0000 7f5a40c26740 1 --2- 172.21.15.2:0/2823630413 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5649037e96a0 0x5649037ebb30 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.023+0000 7f5a40c26740 1 --2- 172.21.15.2:0/2823630413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649037d9ad0 0x5649037d6c90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:03.397 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.023+0000 7f5a40c26740 1 --2- 172.21.15.2:0/2823630413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5649037cf210 0x5649037d6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:03.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.023+0000 7f5a16ffd640 0 client.25415 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:58:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.023+0000 7f5a40c26740 1 --2- 172.21.15.2:0/2823630413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649037ce8f0 0x5649037d9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.023+0000 7f5a40c26740 1 -- 172.21.15.2:0/2823630413 >> 172.21.15.2:0/2823630413 conn(0x5649037cae00 msgr2=0x5649037ca100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:03.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.023+0000 7f5a40c26740 1 -- 172.21.15.2:0/2823630413 shutdown_connections 2024-05-14T16:58:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.023+0000 7f5a40c26740 1 -- 172.21.15.2:0/2823630413 wait complete. 2024-05-14T16:58:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.023+0000 7f5a40c26740 20 client.25415 trim_cache size 0 max 16384 2024-05-14T16:58:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.045+0000 7f5a40c26740 1 Processor -- start 2024-05-14T16:58:04.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.045+0000 7f5a40c26740 1 -- start start 2024-05-14T16:58:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.045+0000 7f5a40c26740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649037ce8f0 0x5649037e5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:04.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.045+0000 7f5a40c26740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649037cf210 0x5649037d1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.045+0000 7f5a40c26740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5649037d9ad0 0x5649037d8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:04.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.045+0000 7f5a40c26740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5a381b3190 con 0x5649037ce8f0 2024-05-14T16:58:04.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.045+0000 7f5a40c26740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5a381b0a30 con 0x5649037cf210 2024-05-14T16:58:04.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.045+0000 7f5a40c26740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5a381ae880 con 0x5649037d9ad0 2024-05-14T16:58:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.046+0000 7f5a377fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649037ce8f0 0x5649037e5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:04.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.046+0000 7f5a37fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5649037d9ad0 0x5649037d8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.046+0000 7f5a36ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649037cf210 0x5649037d1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.046+0000 7f5a377fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649037ce8f0 0x5649037e5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55818/0 (socket says 172.21.15.2:55818) 2024-05-14T16:58:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.046+0000 7f5a37fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5649037d9ad0 0x5649037d8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:43530/0 (socket says 172.21.15.2:43530) 2024-05-14T16:58:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.046+0000 7f5a377fe640 1 -- 172.21.15.2:0/1948403049 learned_addr learned my addr 172.21.15.2:0/1948403049 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:58:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.046+0000 7f5a36ffd640 1 -- 172.21.15.2:0/1948403049 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5649037d9ad0 msgr2=0x5649037d8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:04.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.046+0000 7f5a36ffd640 1 --2- 172.21.15.2:0/1948403049 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5649037d9ad0 0x5649037d8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.046+0000 7f5a36ffd640 1 -- 172.21.15.2:0/1948403049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649037ce8f0 msgr2=0x5649037e5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.046+0000 7f5a36ffd640 1 --2- 172.21.15.2:0/1948403049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649037ce8f0 0x5649037e5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.046+0000 7f5a36ffd640 1 -- 172.21.15.2:0/1948403049 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649037c6830 con 0x5649037cf210 2024-05-14T16:58:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.046+0000 7f5a36ffd640 1 --2- 172.21.15.2:0/1948403049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649037cf210 0x5649037d1c40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c100000 tx=0x7f5a2c036b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27a62b6bfe417d30 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.046+0000 7f5a1d7fa640 1 -- 172.21.15.2:0/1948403049 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c049070 con 0x5649037cf210 2024-05-14T16:58:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.047+0000 7f5a1d7fa640 1 -- 172.21.15.2:0/1948403049 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a2c0d2330 con 0x5649037cf210 2024-05-14T16:58:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.047+0000 7f5a1d7fa640 1 -- 172.21.15.2:0/1948403049 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c0cdf40 con 0x5649037cf210 2024-05-14T16:58:04.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.047+0000 7f5a40c26740 1 -- 172.21.15.2:0/1948403049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649037cf210 msgr2=0x5649037d1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:04.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.047+0000 7f5a40c26740 1 --2- 172.21.15.2:0/1948403049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649037cf210 0x5649037d1c40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c100000 tx=0x7f5a2c036b60 comp rx=0 tx=0).stop 2024-05-14T16:58:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.047+0000 7f5a40c26740 1 -- 172.21.15.2:0/1948403049 shutdown_connections 2024-05-14T16:58:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.047+0000 7f5a40c26740 1 --2- 172.21.15.2:0/1948403049 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5649037d9ad0 0x5649037d8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:04.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.047+0000 7f5a40c26740 1 --2- 172.21.15.2:0/1948403049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649037cf210 0x5649037d1c40 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:04.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.047+0000 7f5a40c26740 1 --2- 172.21.15.2:0/1948403049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649037ce8f0 0x5649037e5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.047+0000 7f5a40c26740 1 -- 172.21.15.2:0/1948403049 >> 172.21.15.2:0/1948403049 conn(0x5649037cae00 msgr2=0x5649037d1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.048+0000 7f5a40c26740 1 -- 172.21.15.2:0/1948403049 shutdown_connections 2024-05-14T16:58:04.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.048+0000 7f5a40c26740 1 -- 172.21.15.2:0/1948403049 wait complete. 2024-05-14T16:58:04.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.048+0000 7f5a40c26740 1 -- start start 2024-05-14T16:58:04.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.048+0000 7f5a40c26740 1 Processor -- start 2024-05-14T16:58:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.048+0000 7f5a40c26740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649037ce8f0 0x5649037e1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:04.634 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.048+0000 7f5a40c26740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5649037cf210 0x5649037dd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:04.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.048+0000 7f5a40c26740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649037d9ad0 0x5649037dd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.049+0000 7f5a40c26740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5a381aeef0 con 0x5649037d9ad0 2024-05-14T16:58:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.049+0000 7f5a40c26740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5a381b0360 con 0x5649037ce8f0 2024-05-14T16:58:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.049+0000 7f5a40c26740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5a381af5c0 con 0x5649037cf210 2024-05-14T16:58:04.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.049+0000 7f5a37fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649037d9ad0 0x5649037dd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:04.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.049+0000 7f5a37fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649037d9ad0 0x5649037dd950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55824/0 (socket says 172.21.15.2:55824) 2024-05-14T16:58:04.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.049+0000 7f5a37fff640 1 -- 172.21.15.2:0/3684315085 learned_addr learned my addr 172.21.15.2:0/3684315085 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:58:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.049+0000 7f5a36ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5649037cf210 0x5649037dd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:04.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.049+0000 7f5a40c26740 10 client.0 _ll_register_callbacks cb 0x7f5a30013b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:58:04.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.049+0000 7f5a377fe640 1 --2- 172.21.15.2:0/3684315085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649037ce8f0 0x5649037e1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:04.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.049+0000 7f5a36ffd640 1 -- 172.21.15.2:0/3684315085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649037ce8f0 msgr2=0x5649037e1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:04.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.049+0000 7f5a36ffd640 1 --2- 172.21.15.2:0/3684315085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649037ce8f0 0x5649037e1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:04.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.049+0000 7f5a36ffd640 1 -- 172.21.15.2:0/3684315085 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649037d9ad0 msgr2=0x5649037dd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:04.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.049+0000 7f5a36ffd640 1 --2- 172.21.15.2:0/3684315085 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649037d9ad0 0x5649037dd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:04.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.049+0000 7f5a36ffd640 1 -- 172.21.15.2:0/3684315085 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649037de180 con 0x5649037cf210 2024-05-14T16:58:04.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.049+0000 7f5a37fff640 1 --2- 172.21.15.2:0/3684315085 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649037d9ad0 0x5649037dd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:58:04.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.049+0000 7f5a36ffd640 1 --2- 172.21.15.2:0/3684315085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5649037cf210 0x5649037dd3b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c100030 tx=0x7f5a2c1004b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34093835426e58a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:04.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.049+0000 7f5a16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-14T16:58:04.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.049+0000 7f5a16ffd640 1 -- 172.21.15.2:0/3684315085 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c049040 con 0x5649037cf210 2024-05-14T16:58:04.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.049+0000 7f5a16ffd640 1 -- 172.21.15.2:0/3684315085 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f5a2c0c6230 con 0x5649037cf210 2024-05-14T16:58:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.050+0000 7f5a40c26740 1 -- 172.21.15.2:0/3684315085 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56490373d360 con 0x5649037cf210 2024-05-14T16:58:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.050+0000 7f5a16ffd640 1 -- 172.21.15.2:0/3684315085 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5a2c0c9750 con 0x5649037cf210 2024-05-14T16:58:04.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.050+0000 7f5a40c26740 10 client.25396 fetch_fsmap learned FSMap version 32 2024-05-14T16:58:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.050+0000 7f5a40c26740 1 -- 172.21.15.2:0/3684315085 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564903854090 con 0x5649037cf210 2024-05-14T16:58:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.050+0000 7f5a16ffd640 1 -- 172.21.15.2:0/3684315085 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56490373d360 con 0x5649037cf210 2024-05-14T16:58:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.050+0000 7f5a16ffd640 1 -- 172.21.15.2:0/3684315085 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7f5a2c0c9ea0 con 0x5649037cf210 2024-05-14T16:58:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.050+0000 7f5a40c26740 10 client.25396 fetch_fsmap finished waiting for FSMap version 32 2024-05-14T16:58:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.050+0000 7f5a40c26740 10 client.25396 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:58:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.050+0000 7f5a40c26740 20 client.25396 populate_metadata read hostname 'smithi002' 2024-05-14T16:58:05.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.050+0000 7f5a40c26740 1 --2- 172.21.15.2:0/3684315085 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564903854760 0x564903856b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.050+0000 7f5a40c26740 4 client.25396 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:58:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.050+0000 7f5a40c26740 1 -- 172.21.15.2:0/3684315085 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5649036b75f0 con 0x564903854760 2024-05-14T16:58:05.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.051+0000 7f5a377fe640 1 --2- 172.21.15.2:0/3684315085 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564903854760 0x564903856b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.051+0000 7f5a377fe640 1 --2- 172.21.15.2:0/3684315085 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564903854760 0x564903856b80 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60c9ffa2d835b7ff server_cookie=846bf95087cc9625 in_seq=0 out_seq=0 2024-05-14T16:58:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.051+0000 7f5a16ffd640 10 client.25396 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:58:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.065+0000 7f5a16ffd640 1 -- 172.21.15.2:0/3684315085 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5649036b75f0 con 0x564903854760 2024-05-14T16:58:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.065+0000 7f5a16ffd640 10 client.25396 handle_command_reply: tid=0 2024-05-14T16:58:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.065+0000 7f5a40c26740 1 client.25396 shutdown 2024-05-14T16:58:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.065+0000 7f5a40c26740 1 -- 172.21.15.2:0/3684315085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5649037cf210 msgr2=0x5649037dd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:05.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.065+0000 7f5a40c26740 1 --2- 172.21.15.2:0/3684315085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5649037cf210 0x5649037dd3b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c100030 tx=0x7f5a2c1004b0 comp rx=0 tx=0).stop 2024-05-14T16:58:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.065+0000 7f5a40c26740 1 -- 172.21.15.2:0/3684315085 shutdown_connections 2024-05-14T16:58:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.065+0000 7f5a40c26740 1 --2- 172.21.15.2:0/3684315085 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564903854760 0x564903856b80 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:05.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.065+0000 7f5a16ffd640 0 client.25396 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:58:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.065+0000 7f5a40c26740 1 --2- 172.21.15.2:0/3684315085 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5649037d9ad0 0x5649037dd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:05.219 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.065+0000 7f5a40c26740 1 --2- 172.21.15.2:0/3684315085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5649037cf210 0x5649037dd3b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.065+0000 7f5a40c26740 1 --2- 172.21.15.2:0/3684315085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649037ce8f0 0x5649037e1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.065+0000 7f5a40c26740 1 -- 172.21.15.2:0/3684315085 >> 172.21.15.2:0/3684315085 conn(0x5649037cae00 msgr2=0x5649037d1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.065+0000 7f5a40c26740 1 -- 172.21.15.2:0/3684315085 shutdown_connections 2024-05-14T16:58:05.252 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.065+0000 7f5a40c26740 1 -- 172.21.15.2:0/3684315085 wait complete. 2024-05-14T16:58:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.065+0000 7f5a40c26740 20 client.25396 trim_cache size 0 max 16384 2024-05-14T16:58:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T16:58:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T16:58:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T16:58:05.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.066+0000 7f5a3de2a640 1 -- 172.21.15.2:0/2261041319 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f5a200b2a10 msgr2=0x7f5a200b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.066+0000 7f5a3de2a640 1 --2- 172.21.15.2:0/2261041319 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f5a200b2a10 0x7f5a200b4ed0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5a30009230 tx=0x7f5a30004ab0 comp rx=0 tx=0).stop 2024-05-14T16:58:05.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.066+0000 7f5a3de2a640 1 -- 172.21.15.2:0/2261041319 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a38102180 msgr2=0x7f5a3819b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.066+0000 7f5a3de2a640 1 --2- 172.21.15.2:0/2261041319 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a38102180 0x7f5a3819b990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c02a930 tx=0x7f5a2c005e60 comp rx=0 tx=0).stop 2024-05-14T16:58:05.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.066+0000 7f5a3de2a640 1 -- 172.21.15.2:0/2261041319 shutdown_connections 2024-05-14T16:58:05.396 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.066+0000 7f5a3de2a640 1 --2- 172.21.15.2:0/2261041319 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f5a200b2a10 0x7f5a200b4ed0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.066+0000 7f5a3de2a640 1 --2- 172.21.15.2:0/2261041319 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a3810f410 0x7f5a3819fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:05.419 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.066+0000 7f5a3de2a640 1 --2- 172.21.15.2:0/2261041319 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a38102180 0x7f5a3819b990 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.066+0000 7f5a3de2a640 1 --2- 172.21.15.2:0/2261041319 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a38101860 0x7f5a3819b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.066+0000 7f5a3de2a640 1 -- 172.21.15.2:0/2261041319 >> 172.21.15.2:0/2261041319 conn(0x7f5a38078ad0 msgr2=0x7f5a38110a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.066+0000 7f5a3de2a640 1 -- 172.21.15.2:0/2261041319 shutdown_connections 2024-05-14T16:58:05.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:02.066+0000 7f5a3de2a640 1 -- 172.21.15.2:0/2261041319 wait complete. 2024-05-14T16:58:05.458 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:58:02 smithi002 ceph-mon[31538]: pgmap v103: 97 pgs: 97 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.4 MiB/s wr, 673 op/s 2024-05-14T16:58:05.465 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:58:04 smithi002 ceph-mon[31538]: pgmap v104: 97 pgs: 97 active+clean; 1.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 13 MiB/s wr, 1.06k op/s 2024-05-14T16:58:05.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:58:02 smithi123 ceph-mon[35971]: pgmap v103: 97 pgs: 97 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.4 MiB/s wr, 673 op/s 2024-05-14T16:58:05.481 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:58:04 smithi123 ceph-mon[35971]: pgmap v104: 97 pgs: 97 active+clean; 1.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 13 MiB/s wr, 1.06k op/s 2024-05-14T16:58:05.488 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:58:02 smithi151 ceph-mon[36066]: pgmap v103: 97 pgs: 97 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.4 MiB/s wr, 673 op/s 2024-05-14T16:58:05.495 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:58:04 smithi151 ceph-mon[36066]: pgmap v104: 97 pgs: 97 active+clean; 1.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 13 MiB/s wr, 1.06k op/s 2024-05-14T16:58:06.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 170.0 s, 1142.8 tps, lat 91.875 ms stddev 190.846 2024-05-14T16:58:06.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:58:06 smithi151 ceph-mon[36066]: pgmap v105: 97 pgs: 97 active+clean; 1.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.5 MiB/s wr, 738 op/s 2024-05-14T16:58:06.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:58:06 smithi123 ceph-mon[35971]: pgmap v105: 97 pgs: 97 active+clean; 1.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.5 MiB/s wr, 738 op/s 2024-05-14T16:58:06.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:58:06 smithi002 ceph-mon[31538]: pgmap v105: 97 pgs: 97 active+clean; 1.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.5 MiB/s wr, 738 op/s 2024-05-14T16:58:08.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:58:08 smithi151 ceph-mon[36066]: pgmap v106: 97 pgs: 97 active+clean; 1.8 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 MiB/s wr, 1.44k op/s 2024-05-14T16:58:08.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:58:08 smithi123 ceph-mon[35971]: pgmap v106: 97 pgs: 97 active+clean; 1.8 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 MiB/s wr, 1.44k op/s 2024-05-14T16:58:08.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:58:08 smithi002 ceph-mon[31538]: pgmap v106: 97 pgs: 97 active+clean; 1.8 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 MiB/s wr, 1.44k op/s 2024-05-14T16:58:09.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:58:09 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:58:09.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:58:09 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:58:09.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:58:09 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:58:10.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:58:10 smithi151 ceph-mon[36066]: pgmap v107: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 MiB/s wr, 1.72k op/s 2024-05-14T16:58:10.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:58:10 smithi123 ceph-mon[35971]: pgmap v107: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 MiB/s wr, 1.72k op/s 2024-05-14T16:58:10.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:58:10 smithi002 ceph-mon[31538]: pgmap v107: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 MiB/s wr, 1.72k op/s 2024-05-14T16:58:11.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 175.0 s, 3948.8 tps, lat 25.340 ms stddev 28.679 2024-05-14T16:58:12.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:58:12 smithi151 ceph-mon[36066]: pgmap v108: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 34 MiB/s wr, 1.57k op/s 2024-05-14T16:58:12.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:58:12 smithi123 ceph-mon[35971]: pgmap v108: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 34 MiB/s wr, 1.57k op/s 2024-05-14T16:58:12.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:58:12 smithi002 ceph-mon[31538]: pgmap v108: 97 pgs: 97 active+clean; 1.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 34 MiB/s wr, 1.57k op/s 2024-05-14T16:58:13.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:58:13 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:58:13.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:58:13 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:58:13.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:58:13 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:58:14.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:58:14 smithi151 ceph-mon[36066]: pgmap v109: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 38 MiB/s wr, 1.89k op/s 2024-05-14T16:58:14.874 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:58:14 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:58:14.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:58:14 smithi123 ceph-mon[35971]: pgmap v109: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 38 MiB/s wr, 1.89k op/s 2024-05-14T16:58:14.918 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:58:14 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:58:14.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:58:14 smithi002 ceph-mon[31538]: pgmap v109: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 38 MiB/s wr, 1.89k op/s 2024-05-14T16:58:15.003 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:58:14 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:58:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.197+0000 7f7e46fda640 1 -- 172.21.15.2:0/1079100643 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e40100b20 msgr2=0x7f7e40100fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:15.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.197+0000 7f7e46fda640 1 --2- 172.21.15.2:0/1079100643 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e40100b20 0x7f7e40100fa0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c009930 tx=0x7f7e2c02a440 comp rx=0 tx=0).stop 2024-05-14T16:58:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.198+0000 7f7e46fda640 1 -- 172.21.15.2:0/1079100643 shutdown_connections 2024-05-14T16:58:15.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.198+0000 7f7e46fda640 1 --2- 172.21.15.2:0/1079100643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e401014e0 0x7f7e4010f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.198+0000 7f7e46fda640 1 --2- 172.21.15.2:0/1079100643 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e40100b20 0x7f7e40100fa0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.198+0000 7f7e46fda640 1 --2- 172.21.15.2:0/1079100643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e40107b70 0x7f7e40107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:15.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.198+0000 7f7e46fda640 1 -- 172.21.15.2:0/1079100643 >> 172.21.15.2:0/1079100643 conn(0x7f7e40077a50 msgr2=0x7f7e400fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.198+0000 7f7e46fda640 1 -- 172.21.15.2:0/1079100643 shutdown_connections 2024-05-14T16:58:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.198+0000 7f7e46fda640 1 -- 172.21.15.2:0/1079100643 wait complete. 2024-05-14T16:58:15.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.199+0000 7f7e46fda640 1 Processor -- start 2024-05-14T16:58:15.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.199+0000 7f7e46fda640 1 -- start start 2024-05-14T16:58:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.199+0000 7f7e46fda640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e40100b20 0x7f7e4010c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.199+0000 7f7e46fda640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e401014e0 0x7f7e4010c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:15.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.199+0000 7f7e46fda640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e40107b70 0x7f7e401a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:15.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.199+0000 7f7e46fda640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7e401038d0 con 0x7f7e401014e0 2024-05-14T16:58:15.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.199+0000 7f7e46fda640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7e40103750 con 0x7f7e40107b70 2024-05-14T16:58:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.199+0000 7f7e46fda640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7e40103a50 con 0x7f7e40100b20 2024-05-14T16:58:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.199+0000 7f7e44d76640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e40100b20 0x7f7e4010c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:15.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.200+0000 7f7e44d76640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e40100b20 0x7f7e4010c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:40152/0 (socket says 172.21.15.2:40152) 2024-05-14T16:58:15.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.200+0000 7f7e44d76640 1 -- 172.21.15.2:0/3748744678 learned_addr learned my addr 172.21.15.2:0/3748744678 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:58:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.200+0000 7f7e45577640 1 --2- 172.21.15.2:0/3748744678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e40107b70 0x7f7e401a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.200+0000 7f7e3ffff640 1 --2- 172.21.15.2:0/3748744678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e401014e0 0x7f7e4010c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.200+0000 7f7e44d76640 1 -- 172.21.15.2:0/3748744678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e40107b70 msgr2=0x7f7e401a5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:15.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.200+0000 7f7e44d76640 1 --2- 172.21.15.2:0/3748744678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e40107b70 0x7f7e401a5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.200+0000 7f7e44d76640 1 -- 172.21.15.2:0/3748744678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e401014e0 msgr2=0x7f7e4010c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.200+0000 7f7e44d76640 1 --2- 172.21.15.2:0/3748744678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e401014e0 0x7f7e4010c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:15.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.200+0000 7f7e44d76640 1 -- 172.21.15.2:0/3748744678 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e401a58f0 con 0x7f7e40100b20 2024-05-14T16:58:15.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.200+0000 7f7e3ffff640 1 --2- 172.21.15.2:0/3748744678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e401014e0 0x7f7e4010c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:58:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.201+0000 7f7e44d76640 1 --2- 172.21.15.2:0/3748744678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e40100b20 0x7f7e4010c1e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7e3400d7a0 tx=0x7f7e3400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.201+0000 7f7e3dffb640 1 -- 172.21.15.2:0/3748744678 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e34017070 con 0x7f7e40100b20 2024-05-14T16:58:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.201+0000 7f7e3dffb640 1 -- 172.21.15.2:0/3748744678 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e34009c30 con 0x7f7e40100b20 2024-05-14T16:58:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.201+0000 7f7e46fda640 1 -- 172.21.15.2:0/3748744678 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e401a5be0 con 0x7f7e40100b20 2024-05-14T16:58:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.201+0000 7f7e3dffb640 1 -- 172.21.15.2:0/3748744678 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e34003cf0 con 0x7f7e40100b20 2024-05-14T16:58:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.202+0000 7f7e46fda640 1 -- 172.21.15.2:0/3748744678 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e40115d00 con 0x7f7e40100b20 2024-05-14T16:58:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.206+0000 7f7e3dffb640 1 -- 172.21.15.2:0/3748744678 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f7e34010460 con 0x7f7e40100b20 2024-05-14T16:58:15.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.207+0000 7f7e49dcc740 1 Processor -- start 2024-05-14T16:58:15.727 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.207+0000 7f7e49dcc740 1 -- start start 2024-05-14T16:58:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.207+0000 7f7e3dffb640 1 --2- 172.21.15.2:0/3748744678 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f7e300b2c20 0x7f7e300b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.207+0000 7f7e49dcc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4cab718f0 0x55e4cab71cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:15.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.207+0000 7f7e49dcc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4cab72210 0x55e4cab7c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:15.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.207+0000 7f7e49dcc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4cab7cad0 0x55e4cab7eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.207+0000 7f7e49dcc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e4caae4c30 con 0x55e4cab718f0 2024-05-14T16:58:15.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.207+0000 7f7e49dcc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e4caad47d0 con 0x55e4cab72210 2024-05-14T16:58:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.207+0000 7f7e49dcc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e4caad8810 con 0x55e4cab7cad0 2024-05-14T16:58:15.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.207+0000 7f7e3dffb640 1 -- 172.21.15.2:0/3748744678 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f7e340e1f90 con 0x7f7e40100b20 2024-05-14T16:58:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.208+0000 7f7e44d76640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4cab718f0 0x55e4cab71cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.208+0000 7f7e44d76640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4cab718f0 0x55e4cab71cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60108/0 (socket says 172.21.15.2:60108) 2024-05-14T16:58:15.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.208+0000 7f7e44d76640 1 -- 172.21.15.2:0/761979901 learned_addr learned my addr 172.21.15.2:0/761979901 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:58:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.208+0000 7f7e45577640 1 --2- 172.21.15.2:0/761979901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4cab7cad0 0x55e4cab7eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.208+0000 7f7e45577640 1 --2- 172.21.15.2:0/3748744678 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f7e300b2c20 0x7f7e300b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.208+0000 7f7e3ffff640 1 --2- 172.21.15.2:0/761979901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4cab72210 0x55e4cab7c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.209+0000 7f7e44d76640 1 -- 172.21.15.2:0/761979901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4cab7cad0 msgr2=0x55e4cab7eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.209+0000 7f7e44d76640 1 --2- 172.21.15.2:0/761979901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4cab7cad0 0x55e4cab7eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:15.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.209+0000 7f7e44d76640 1 -- 172.21.15.2:0/761979901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4cab72210 msgr2=0x55e4cab7c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:15.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.209+0000 7f7e44d76640 1 --2- 172.21.15.2:0/761979901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4cab72210 0x55e4cab7c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:15.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.209+0000 7f7e44d76640 1 -- 172.21.15.2:0/761979901 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4ca7e9210 con 0x55e4cab718f0 2024-05-14T16:58:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.209+0000 7f7e45577640 1 --2- 172.21.15.2:0/3748744678 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f7e300b2c20 0x7f7e300b50e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7e380077f0 tx=0x7f7e380078a0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:15.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.209+0000 7f7e44d76640 1 --2- 172.21.15.2:0/761979901 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4cab718f0 0x55e4cab71cd0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7e340a9e50 tx=0x7f7e34014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25fa862c2d13f6f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:15.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.210+0000 7f7e2affd640 1 -- 172.21.15.2:0/761979901 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e34020040 con 0x55e4cab718f0 2024-05-14T16:58:15.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.210+0000 7f7e2affd640 1 -- 172.21.15.2:0/761979901 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e340a9c60 con 0x55e4cab718f0 2024-05-14T16:58:15.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.210+0000 7f7e2affd640 1 -- 172.21.15.2:0/761979901 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e340ae0d0 con 0x55e4cab718f0 2024-05-14T16:58:16.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.211+0000 7f7e49dcc740 1 -- 172.21.15.2:0/761979901 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4cab718f0 msgr2=0x55e4cab71cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.211+0000 7f7e49dcc740 1 --2- 172.21.15.2:0/761979901 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4cab718f0 0x55e4cab71cd0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7e340a9e50 tx=0x7f7e34014000 comp rx=0 tx=0).stop 2024-05-14T16:58:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.212+0000 7f7e49dcc740 1 -- 172.21.15.2:0/761979901 shutdown_connections 2024-05-14T16:58:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.212+0000 7f7e49dcc740 1 --2- 172.21.15.2:0/761979901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4cab7cad0 0x55e4cab7eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.212+0000 7f7e49dcc740 1 --2- 172.21.15.2:0/761979901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4cab72210 0x55e4cab7c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.212+0000 7f7e49dcc740 1 --2- 172.21.15.2:0/761979901 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4cab718f0 0x55e4cab71cd0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.212+0000 7f7e49dcc740 1 -- 172.21.15.2:0/761979901 >> 172.21.15.2:0/761979901 conn(0x55e4cab6de00 msgr2=0x55e4cab6e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.212+0000 7f7e49dcc740 1 -- 172.21.15.2:0/761979901 shutdown_connections 2024-05-14T16:58:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.212+0000 7f7e49dcc740 1 -- 172.21.15.2:0/761979901 wait complete. 2024-05-14T16:58:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.212+0000 7f7e49dcc740 1 -- start start 2024-05-14T16:58:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.212+0000 7f7e49dcc740 1 Processor -- start 2024-05-14T16:58:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.213+0000 7f7e49dcc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4cab718f0 0x55e4cab7c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.213+0000 7f7e49dcc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4cab72210 0x55e4cab79770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.213+0000 7f7e49dcc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4cab7cad0 0x55e4cab79d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:16.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.213+0000 7f7e49dcc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e4ca7ba790 con 0x55e4cab7cad0 2024-05-14T16:58:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.213+0000 7f7e49dcc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e4caa90890 con 0x55e4cab72210 2024-05-14T16:58:16.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.213+0000 7f7e49dcc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e4cab7f740 con 0x55e4cab718f0 2024-05-14T16:58:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.213+0000 7f7e49dcc740 10 client.0 _ll_register_callbacks cb 0x55e4caa93d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:58:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.214+0000 7f7e3ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4cab72210 0x55e4cab79770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.214+0000 7f7e44d76640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4cab718f0 0x55e4cab7c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.214+0000 7f7e3ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4cab72210 0x55e4cab79770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:34402/0 (socket says 172.21.15.2:34402) 2024-05-14T16:58:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.214+0000 7f7e3ffff640 1 -- 172.21.15.2:0/2721215460 learned_addr learned my addr 172.21.15.2:0/2721215460 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:58:16.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.214+0000 7f7e45577640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4cab7cad0 0x55e4cab79d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:16.250 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.214+0000 7f7e44d76640 1 -- 172.21.15.2:0/2721215460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4cab72210 msgr2=0x55e4cab79770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:16.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.214+0000 7f7e44d76640 1 --2- 172.21.15.2:0/2721215460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4cab72210 0x55e4cab79770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.214+0000 7f7e44d76640 1 -- 172.21.15.2:0/2721215460 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4cab7cad0 msgr2=0x55e4cab79d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:16.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.214+0000 7f7e44d76640 1 --2- 172.21.15.2:0/2721215460 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4cab7cad0 0x55e4cab79d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.214+0000 7f7e44d76640 1 -- 172.21.15.2:0/2721215460 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4ca8b5620 con 0x55e4cab718f0 2024-05-14T16:58:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.215+0000 7f7e44d76640 1 --2- 172.21.15.2:0/2721215460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4cab718f0 0x55e4cab7c480 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x55e4cab72780 tx=0x7f7e340ae300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b59fb2773893438 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.215+0000 7f7e1ffff640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-14T16:58:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.217+0000 7f7e1ffff640 1 -- 172.21.15.2:0/2721215460 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e34020070 con 0x55e4cab718f0 2024-05-14T16:58:16.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.217+0000 7f7e1ffff640 1 -- 172.21.15.2:0/2721215460 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f7e340a6740 con 0x55e4cab718f0 2024-05-14T16:58:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.217+0000 7f7e49dcc740 1 -- 172.21.15.2:0/2721215460 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e340a9c60 con 0x55e4cab718f0 2024-05-14T16:58:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.217+0000 7f7e1ffff640 1 -- 172.21.15.2:0/2721215460 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7e340dfb40 con 0x55e4cab718f0 2024-05-14T16:58:16.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.217+0000 7f7e49dcc740 10 client.25408 fetch_fsmap learned FSMap version 35 2024-05-14T16:58:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.217+0000 7f7e49dcc740 1 -- 172.21.15.2:0/2721215460 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e4cab8b1e0 con 0x55e4cab718f0 2024-05-14T16:58:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.219+0000 7f7e1ffff640 1 -- 172.21.15.2:0/2721215460 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e340a9c60 con 0x55e4cab718f0 2024-05-14T16:58:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.219+0000 7f7e1ffff640 1 -- 172.21.15.2:0/2721215460 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f7e340a8f30 con 0x55e4cab718f0 2024-05-14T16:58:16.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.219+0000 7f7e49dcc740 10 client.25408 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T16:58:16.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.219+0000 7f7e49dcc740 10 client.25408 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:58:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.219+0000 7f7e49dcc740 20 client.25408 populate_metadata read hostname 'smithi002' 2024-05-14T16:58:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.219+0000 7f7e49dcc740 1 --2- 172.21.15.2:0/2721215460 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55e4cab8c890 0x55e4cab8ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.219+0000 7f7e49dcc740 4 client.25408 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:58:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.219+0000 7f7e49dcc740 1 -- 172.21.15.2:0/2721215460 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4caae6b80 con 0x55e4cab8c890 2024-05-14T16:58:16.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.219+0000 7f7e3ffff640 1 --2- 172.21.15.2:0/2721215460 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55e4cab8c890 0x55e4cab8ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.220+0000 7f7e1ffff640 10 client.25408 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:58:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.220+0000 7f7e3ffff640 1 --2- 172.21.15.2:0/2721215460 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55e4cab8c890 0x55e4cab8ecd0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c005e74e87155da6 server_cookie=aa25c85e84770289 in_seq=0 out_seq=0 2024-05-14T16:58:16.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.221+0000 7f7e1ffff640 1 -- 172.21.15.2:0/2721215460 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e4caae6b80 con 0x55e4cab8c890 2024-05-14T16:58:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.221+0000 7f7e1ffff640 10 client.25408 handle_command_reply: tid=0 2024-05-14T16:58:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.221+0000 7f7e49dcc740 1 client.25408 shutdown 2024-05-14T16:58:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.221+0000 7f7e49dcc740 1 -- 172.21.15.2:0/2721215460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4cab718f0 msgr2=0x55e4cab7c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.221+0000 7f7e49dcc740 1 --2- 172.21.15.2:0/2721215460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4cab718f0 0x55e4cab7c480 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x55e4cab72780 tx=0x7f7e340ae300 comp rx=0 tx=0).stop 2024-05-14T16:58:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.222+0000 7f7e49dcc740 1 -- 172.21.15.2:0/2721215460 shutdown_connections 2024-05-14T16:58:16.578 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.222+0000 7f7e49dcc740 1 --2- 172.21.15.2:0/2721215460 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55e4cab8c890 0x55e4cab8ecd0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.222+0000 7f7e49dcc740 1 --2- 172.21.15.2:0/2721215460 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4cab7cad0 0x55e4cab79d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.222+0000 7f7e49dcc740 1 --2- 172.21.15.2:0/2721215460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4cab72210 0x55e4cab79770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:16.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.222+0000 7f7e49dcc740 1 --2- 172.21.15.2:0/2721215460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4cab718f0 0x55e4cab7c480 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.222+0000 7f7e1ffff640 0 client.25408 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:58:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.222+0000 7f7e49dcc740 1 -- 172.21.15.2:0/2721215460 >> 172.21.15.2:0/2721215460 conn(0x55e4cab6de00 msgr2=0x55e4cab6d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:16.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.222+0000 7f7e49dcc740 1 -- 172.21.15.2:0/2721215460 shutdown_connections 2024-05-14T16:58:16.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.222+0000 7f7e49dcc740 1 -- 172.21.15.2:0/2721215460 wait complete. 2024-05-14T16:58:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.222+0000 7f7e49dcc740 20 client.25408 trim_cache size 0 max 16384 2024-05-14T16:58:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.247+0000 7f7e49dcc740 1 Processor -- start 2024-05-14T16:58:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.247+0000 7f7e49dcc740 1 -- start start 2024-05-14T16:58:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.247+0000 7f7e49dcc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4cab718f0 0x55e4cab88610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.247+0000 7f7e49dcc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4cab72210 0x55e4cab74c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.247+0000 7f7e49dcc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4cab7cad0 0x55e4cab7bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:16.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.248+0000 7f7e49dcc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7e24011050 con 0x55e4cab718f0 2024-05-14T16:58:16.735 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.248+0000 7f7e49dcc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2400e920 con 0x55e4cab72210 2024-05-14T16:58:16.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.248+0000 7f7e49dcc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2400c770 con 0x55e4cab7cad0 2024-05-14T16:58:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.248+0000 7f7e44d76640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4cab718f0 0x55e4cab88610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.248+0000 7f7e44d76640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4cab718f0 0x55e4cab88610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60124/0 (socket says 172.21.15.2:60124) 2024-05-14T16:58:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.248+0000 7f7e44d76640 1 -- 172.21.15.2:0/3944779388 learned_addr learned my addr 172.21.15.2:0/3944779388 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:58:16.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.248+0000 7f7e45577640 1 --2- 172.21.15.2:0/3944779388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4cab7cad0 0x55e4cab7bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.248+0000 7f7e44d76640 1 -- 172.21.15.2:0/3944779388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4cab7cad0 msgr2=0x55e4cab7bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.248+0000 7f7e44d76640 1 --2- 172.21.15.2:0/3944779388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4cab7cad0 0x55e4cab7bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.248+0000 7f7e44d76640 1 -- 172.21.15.2:0/3944779388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4cab72210 msgr2=0x55e4cab74c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.248+0000 7f7e44d76640 1 --2- 172.21.15.2:0/3944779388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4cab72210 0x55e4cab74c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:16.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.248+0000 7f7e44d76640 1 -- 172.21.15.2:0/3944779388 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4cab72040 con 0x55e4cab718f0 2024-05-14T16:58:16.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.248+0000 7f7e44d76640 1 --2- 172.21.15.2:0/3944779388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4cab718f0 0x55e4cab88610 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7e340aa0e0 tx=0x7f7e340aa2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a83f1384e93b6d37 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.249+0000 7f7e2a7fc640 1 -- 172.21.15.2:0/3944779388 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e34020070 con 0x55e4cab718f0 2024-05-14T16:58:16.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.249+0000 7f7e2a7fc640 1 -- 172.21.15.2:0/3944779388 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e340a9c60 con 0x55e4cab718f0 2024-05-14T16:58:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.249+0000 7f7e2a7fc640 1 -- 172.21.15.2:0/3944779388 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e340ca0f0 con 0x55e4cab718f0 2024-05-14T16:58:16.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.250+0000 7f7e49dcc740 1 -- 172.21.15.2:0/3944779388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4cab718f0 msgr2=0x55e4cab88610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.250+0000 7f7e49dcc740 1 --2- 172.21.15.2:0/3944779388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4cab718f0 0x55e4cab88610 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7e340aa0e0 tx=0x7f7e340aa2d0 comp rx=0 tx=0).stop 2024-05-14T16:58:16.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.251+0000 7f7e49dcc740 1 -- 172.21.15.2:0/3944779388 shutdown_connections 2024-05-14T16:58:16.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.251+0000 7f7e49dcc740 1 --2- 172.21.15.2:0/3944779388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4cab7cad0 0x55e4cab7bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.251+0000 7f7e49dcc740 1 --2- 172.21.15.2:0/3944779388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4cab72210 0x55e4cab74c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.251+0000 7f7e49dcc740 1 --2- 172.21.15.2:0/3944779388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4cab718f0 0x55e4cab88610 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:16.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.251+0000 7f7e49dcc740 1 -- 172.21.15.2:0/3944779388 >> 172.21.15.2:0/3944779388 conn(0x55e4cab6de00 msgr2=0x55e4cab74530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.251+0000 7f7e49dcc740 1 -- 172.21.15.2:0/3944779388 shutdown_connections 2024-05-14T16:58:16.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.251+0000 7f7e49dcc740 1 -- 172.21.15.2:0/3944779388 wait complete. 2024-05-14T16:58:16.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.251+0000 7f7e49dcc740 1 -- start start 2024-05-14T16:58:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.251+0000 7f7e49dcc740 1 Processor -- start 2024-05-14T16:58:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.252+0000 7f7e49dcc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4cab718f0 0x55e4cab7fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.252+0000 7f7e49dcc740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4cab72210 0x55e4cab80160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.252+0000 7f7e49dcc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4cab7cad0 0x55e4cabf5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:17.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.253+0000 7f7e49dcc740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2400cde0 con 0x55e4cab72210 2024-05-14T16:58:17.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.253+0000 7f7e49dcc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2400e250 con 0x55e4cab7cad0 2024-05-14T16:58:17.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.253+0000 7f7e49dcc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2400d4b0 con 0x55e4cab718f0 2024-05-14T16:58:17.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.253+0000 7f7e49dcc740 10 client.0 _ll_register_callbacks cb 0x7f7e340df650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:58:17.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.253+0000 7f7e44d76640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4cab718f0 0x55e4cab7fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.253+0000 7f7e45577640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4cab7cad0 0x55e4cabf5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:17.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.253+0000 7f7e45577640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4cab7cad0 0x55e4cabf5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:34430/0 (socket says 172.21.15.2:34430) 2024-05-14T16:58:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.253+0000 7f7e45577640 1 -- 172.21.15.2:0/4036391972 learned_addr learned my addr 172.21.15.2:0/4036391972 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:58:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.253+0000 7f7e44d76640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4cab718f0 0x55e4cab7fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:40176/0 (socket says 172.21.15.2:40176) 2024-05-14T16:58:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.253+0000 7f7e3ffff640 1 --2- 172.21.15.2:0/4036391972 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4cab72210 0x55e4cab80160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.253+0000 7f7e45577640 1 -- 172.21.15.2:0/4036391972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4cab718f0 msgr2=0x55e4cab7fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.253+0000 7f7e45577640 1 --2- 172.21.15.2:0/4036391972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4cab718f0 0x55e4cab7fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.253+0000 7f7e45577640 1 -- 172.21.15.2:0/4036391972 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4cab72210 msgr2=0x55e4cab80160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.253+0000 7f7e45577640 1 --2- 172.21.15.2:0/4036391972 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4cab72210 0x55e4cab80160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:17.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.253+0000 7f7e45577640 1 -- 172.21.15.2:0/4036391972 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4cab7a080 con 0x55e4cab7cad0 2024-05-14T16:58:17.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.253+0000 7f7e45577640 1 --2- 172.21.15.2:0/4036391972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4cab7cad0 0x55e4cabf5140 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7e38003ba0 tx=0x7f7e3801b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b84f92a67f0e61e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.254+0000 7f7e1ffff640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T16:58:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.254+0000 7f7e1ffff640 1 -- 172.21.15.2:0/4036391972 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e380046d0 con 0x55e4cab7cad0 2024-05-14T16:58:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.254+0000 7f7e49dcc740 1 -- 172.21.15.2:0/4036391972 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e340a9c60 con 0x55e4cab7cad0 2024-05-14T16:58:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.254+0000 7f7e1ffff640 1 -- 172.21.15.2:0/4036391972 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f7e38004b20 con 0x55e4cab7cad0 2024-05-14T16:58:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.254+0000 7f7e1ffff640 1 -- 172.21.15.2:0/4036391972 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7e38004d20 con 0x55e4cab7cad0 2024-05-14T16:58:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.254+0000 7f7e49dcc740 10 client.25433 fetch_fsmap learned FSMap version 35 2024-05-14T16:58:17.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.254+0000 7f7e49dcc740 1 -- 172.21.15.2:0/4036391972 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e4cabf6820 con 0x55e4cab7cad0 2024-05-14T16:58:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.254+0000 7f7e1ffff640 1 -- 172.21.15.2:0/4036391972 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e340a9c60 con 0x55e4cab7cad0 2024-05-14T16:58:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.255+0000 7f7e1ffff640 1 -- 172.21.15.2:0/4036391972 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f7e3802b470 con 0x55e4cab7cad0 2024-05-14T16:58:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.255+0000 7f7e49dcc740 10 client.25433 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T16:58:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.255+0000 7f7e49dcc740 10 client.25433 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:58:17.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.255+0000 7f7e49dcc740 20 client.25433 populate_metadata read hostname 'smithi002' 2024-05-14T16:58:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.255+0000 7f7e49dcc740 1 --2- 172.21.15.2:0/4036391972 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55e4cabf6fe0 0x55e4cabf9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:17.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.255+0000 7f7e49dcc740 4 client.25433 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:58:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.255+0000 7f7e49dcc740 1 -- 172.21.15.2:0/4036391972 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e4caae0360 con 0x55e4cabf6fe0 2024-05-14T16:58:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.255+0000 7f7e44d76640 1 --2- 172.21.15.2:0/4036391972 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55e4cabf6fe0 0x55e4cabf9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.255+0000 7f7e1ffff640 10 client.25433 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:58:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.255+0000 7f7e44d76640 1 --2- 172.21.15.2:0/4036391972 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55e4cabf6fe0 0x55e4cabf9400 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4a7572219f90072 server_cookie=f7c1ac70393d3a4b in_seq=0 out_seq=0 2024-05-14T16:58:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.285+0000 7f7e1ffff640 1 -- 172.21.15.2:0/4036391972 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e4caae0360 con 0x55e4cabf6fe0 2024-05-14T16:58:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.285+0000 7f7e1ffff640 10 client.25433 handle_command_reply: tid=0 2024-05-14T16:58:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.285+0000 7f7e49dcc740 1 client.25433 shutdown 2024-05-14T16:58:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.288+0000 7f7e49dcc740 1 -- 172.21.15.2:0/4036391972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4cab7cad0 msgr2=0x55e4cabf5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.288+0000 7f7e49dcc740 1 --2- 172.21.15.2:0/4036391972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4cab7cad0 0x55e4cabf5140 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7e38003ba0 tx=0x7f7e3801b000 comp rx=0 tx=0).stop 2024-05-14T16:58:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.288+0000 7f7e49dcc740 1 -- 172.21.15.2:0/4036391972 shutdown_connections 2024-05-14T16:58:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.288+0000 7f7e49dcc740 1 --2- 172.21.15.2:0/4036391972 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55e4cabf6fe0 0x55e4cabf9400 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.288+0000 7f7e49dcc740 1 --2- 172.21.15.2:0/4036391972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4cab7cad0 0x55e4cabf5140 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.289+0000 7f7e49dcc740 1 --2- 172.21.15.2:0/4036391972 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4cab72210 0x55e4cab80160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.289+0000 7f7e1ffff640 0 client.25433 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:58:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.289+0000 7f7e49dcc740 1 --2- 172.21.15.2:0/4036391972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e4cab718f0 0x55e4cab7fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:17.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.289+0000 7f7e49dcc740 1 -- 172.21.15.2:0/4036391972 >> 172.21.15.2:0/4036391972 conn(0x55e4cab6de00 msgr2=0x55e4cab6d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.290+0000 7f7e49dcc740 1 -- 172.21.15.2:0/4036391972 shutdown_connections 2024-05-14T16:58:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.290+0000 7f7e49dcc740 1 -- 172.21.15.2:0/4036391972 wait complete. 2024-05-14T16:58:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.293+0000 7f7e49dcc740 20 client.25433 trim_cache size 0 max 16384 2024-05-14T16:58:17.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T16:58:17.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T16:58:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T16:58:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.300+0000 7f7e46fda640 1 -- 172.21.15.2:0/3748744678 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f7e300b2c20 msgr2=0x7f7e300b50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.300+0000 7f7e46fda640 1 --2- 172.21.15.2:0/3748744678 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f7e300b2c20 0x7f7e300b50e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7e380077f0 tx=0x7f7e380078a0 comp rx=0 tx=0).stop 2024-05-14T16:58:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.300+0000 7f7e46fda640 1 -- 172.21.15.2:0/3748744678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e40100b20 msgr2=0x7f7e4010c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.300+0000 7f7e46fda640 1 --2- 172.21.15.2:0/3748744678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e40100b20 0x7f7e4010c1e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7e3400d7a0 tx=0x7f7e3400dc60 comp rx=0 tx=0).stop 2024-05-14T16:58:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.301+0000 7f7e46fda640 1 -- 172.21.15.2:0/3748744678 shutdown_connections 2024-05-14T16:58:17.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.301+0000 7f7e46fda640 1 --2- 172.21.15.2:0/3748744678 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f7e300b2c20 0x7f7e300b50e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.301+0000 7f7e46fda640 1 --2- 172.21.15.2:0/3748744678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e40107b70 0x7f7e401a5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:17.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.301+0000 7f7e46fda640 1 --2- 172.21.15.2:0/3748744678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7e401014e0 0x7f7e4010c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.301+0000 7f7e46fda640 1 --2- 172.21.15.2:0/3748744678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e40100b20 0x7f7e4010c1e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.301+0000 7f7e46fda640 1 -- 172.21.15.2:0/3748744678 >> 172.21.15.2:0/3748744678 conn(0x7f7e40077a50 msgr2=0x7f7e40106340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:17.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.302+0000 7f7e46fda640 1 -- 172.21.15.2:0/3748744678 shutdown_connections 2024-05-14T16:58:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:15.302+0000 7f7e46fda640 1 -- 172.21.15.2:0/3748744678 wait complete. 2024-05-14T16:58:17.683 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 180.0 s, 3606.0 tps, lat 27.725 ms stddev 28.276 2024-05-14T16:58:17.692 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:58:16 smithi002 ceph-mon[31538]: pgmap v110: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 34 MiB/s wr, 1.50k op/s 2024-05-14T16:58:17.702 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:58:16 smithi123 ceph-mon[35971]: pgmap v110: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 34 MiB/s wr, 1.50k op/s 2024-05-14T16:58:17.710 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:58:16 smithi151 ceph-mon[36066]: pgmap v110: 97 pgs: 97 active+clean; 2.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 34 MiB/s wr, 1.50k op/s 2024-05-14T16:58:18.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:58:18 smithi151 ceph-mon[36066]: pgmap v111: 97 pgs: 97 active+clean; 2.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 36 MiB/s wr, 1.68k op/s 2024-05-14T16:58:19.877 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:58:18 smithi002 ceph-mon[31538]: pgmap v111: 97 pgs: 97 active+clean; 2.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 36 MiB/s wr, 1.68k op/s 2024-05-14T16:58:19.953 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:58:18 smithi123 ceph-mon[35971]: pgmap v111: 97 pgs: 97 active+clean; 2.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 36 MiB/s wr, 1.68k op/s 2024-05-14T16:58:20.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:58:20 smithi151 ceph-mon[36066]: pgmap v112: 97 pgs: 97 active+clean; 2.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 20 MiB/s wr, 1.13k op/s 2024-05-14T16:58:20.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:58:20 smithi123 ceph-mon[35971]: pgmap v112: 97 pgs: 97 active+clean; 2.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 20 MiB/s wr, 1.13k op/s 2024-05-14T16:58:20.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:58:20 smithi002 ceph-mon[31538]: pgmap v112: 97 pgs: 97 active+clean; 2.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 20 MiB/s wr, 1.13k op/s 2024-05-14T16:58:21.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 185.0 s, 3347.6 tps, lat 29.715 ms stddev 33.568 2024-05-14T16:58:22.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:58:22 smithi151 ceph-mon[36066]: pgmap v113: 97 pgs: 97 active+clean; 2.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.0 MiB/s wr, 652 op/s 2024-05-14T16:58:22.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:58:22 smithi123 ceph-mon[35971]: pgmap v113: 97 pgs: 97 active+clean; 2.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.0 MiB/s wr, 652 op/s 2024-05-14T16:58:22.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:58:22 smithi002 ceph-mon[31538]: pgmap v113: 97 pgs: 97 active+clean; 2.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.0 MiB/s wr, 652 op/s 2024-05-14T16:58:24.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:58:24 smithi151 ceph-mon[36066]: pgmap v114: 97 pgs: 97 active+clean; 2.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 12 MiB/s wr, 960 op/s 2024-05-14T16:58:24.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:58:24 smithi123 ceph-mon[35971]: pgmap v114: 97 pgs: 97 active+clean; 2.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 12 MiB/s wr, 960 op/s 2024-05-14T16:58:24.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:58:24 smithi002 ceph-mon[31538]: pgmap v114: 97 pgs: 97 active+clean; 2.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 12 MiB/s wr, 960 op/s 2024-05-14T16:58:26.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 190.0 s, 3786.4 tps, lat 26.549 ms stddev 26.909 2024-05-14T16:58:26.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:58:26 smithi151 ceph-mon[36066]: pgmap v115: 97 pgs: 97 active+clean; 2.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.8 MiB/s wr, 633 op/s 2024-05-14T16:58:26.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:58:26 smithi123 ceph-mon[35971]: pgmap v115: 97 pgs: 97 active+clean; 2.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.8 MiB/s wr, 633 op/s 2024-05-14T16:58:26.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:58:26 smithi002 ceph-mon[31538]: pgmap v115: 97 pgs: 97 active+clean; 2.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.8 MiB/s wr, 633 op/s 2024-05-14T16:58:28.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.442+0000 7f7ce470a640 1 -- 172.21.15.2:0/3109484266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cdc109a50 msgr2=0x7f7cdc109e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:28.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.442+0000 7f7ce470a640 1 --2- 172.21.15.2:0/3109484266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cdc109a50 0x7f7cdc109e30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4009c10 tx=0x7f7cd402a770 comp rx=0 tx=0).stop 2024-05-14T16:58:28.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.442+0000 7f7ce470a640 1 -- 172.21.15.2:0/3109484266 shutdown_connections 2024-05-14T16:58:28.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.442+0000 7f7ce470a640 1 --2- 172.21.15.2:0/3109484266 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7cdc115ba0 0x7f7cdc117f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.442+0000 7f7ce470a640 1 --2- 172.21.15.2:0/3109484266 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cdc10a370 0x7f7cdc115660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:28.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.442+0000 7f7ce470a640 1 --2- 172.21.15.2:0/3109484266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cdc109a50 0x7f7cdc109e30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:28.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.442+0000 7f7ce470a640 1 -- 172.21.15.2:0/3109484266 >> 172.21.15.2:0/3109484266 conn(0x7f7cdc06db20 msgr2=0x7f7cdc06df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.442+0000 7f7ce470a640 1 -- 172.21.15.2:0/3109484266 shutdown_connections 2024-05-14T16:58:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.442+0000 7f7ce470a640 1 -- 172.21.15.2:0/3109484266 wait complete. 2024-05-14T16:58:28.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.443+0000 7f7ce470a640 1 Processor -- start 2024-05-14T16:58:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.443+0000 7f7ce470a640 1 -- start start 2024-05-14T16:58:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.444+0000 7f7ce470a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cdc109a50 0x7f7cdc1a76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.444+0000 7f7ce470a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cdc10a370 0x7f7cdc1a7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.444+0000 7f7ce470a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7cdc115ba0 0x7f7cdc1a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.444+0000 7f7ce470a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7cdc11b4c0 con 0x7f7cdc115ba0 2024-05-14T16:58:28.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.444+0000 7f7ce470a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7cdc11b340 con 0x7f7cdc10a370 2024-05-14T16:58:28.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.444+0000 7f7ce470a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7cdc11b640 con 0x7f7cdc109a50 2024-05-14T16:58:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.444+0000 7f7ce24a6640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cdc109a50 0x7f7cdc1a76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.444+0000 7f7ce24a6640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cdc109a50 0x7f7cdc1a76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:54544/0 (socket says 172.21.15.2:54544) 2024-05-14T16:58:28.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.444+0000 7f7ce24a6640 1 -- 172.21.15.2:0/724210030 learned_addr learned my addr 172.21.15.2:0/724210030 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:58:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.444+0000 7f7ce1ca5640 1 --2- 172.21.15.2:0/724210030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cdc10a370 0x7f7cdc1a7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.444+0000 7f7ce24a6640 1 -- 172.21.15.2:0/724210030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cdc10a370 msgr2=0x7f7cdc1a7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.444+0000 7f7ce24a6640 1 --2- 172.21.15.2:0/724210030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cdc10a370 0x7f7cdc1a7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.444+0000 7f7ce24a6640 1 -- 172.21.15.2:0/724210030 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7cdc115ba0 msgr2=0x7f7cdc1a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.444+0000 7f7ce24a6640 1 --2- 172.21.15.2:0/724210030 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7cdc115ba0 0x7f7cdc1a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.444+0000 7f7ce24a6640 1 -- 172.21.15.2:0/724210030 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cdc1a2020 con 0x7f7cdc109a50 2024-05-14T16:58:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.445+0000 7f7ce24a6640 1 --2- 172.21.15.2:0/724210030 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cdc109a50 0x7f7cdc1a76f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4000f80 tx=0x7f7cd4003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.446+0000 7f7cd37fe640 1 -- 172.21.15.2:0/724210030 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd4041020 con 0x7f7cdc109a50 2024-05-14T16:58:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.446+0000 7f7ce470a640 1 -- 172.21.15.2:0/724210030 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cdc1a22b0 con 0x7f7cdc109a50 2024-05-14T16:58:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.446+0000 7f7ce470a640 1 -- 172.21.15.2:0/724210030 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cdc1ae5b0 con 0x7f7cdc109a50 2024-05-14T16:58:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.446+0000 7f7cd37fe640 1 -- 172.21.15.2:0/724210030 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cd40318f0 con 0x7f7cdc109a50 2024-05-14T16:58:28.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.446+0000 7f7cd37fe640 1 -- 172.21.15.2:0/724210030 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd4036040 con 0x7f7cdc109a50 2024-05-14T16:58:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.450+0000 7f7cd37fe640 1 -- 172.21.15.2:0/724210030 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f7cd4044020 con 0x7f7cdc109a50 2024-05-14T16:58:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.450+0000 7f7ce7427740 1 Processor -- start 2024-05-14T16:58:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.451+0000 7f7ce7427740 1 -- start start 2024-05-14T16:58:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.451+0000 7f7ce7427740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b59543de30 0x55b59543c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.451+0000 7f7ce7427740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b59543ccf0 0x55b59543d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:28.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.451+0000 7f7ce7427740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b595450170 0x55b595452560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:28.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.451+0000 7f7ce7427740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b5953afc30 con 0x55b59543ccf0 2024-05-14T16:58:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.451+0000 7f7ce7427740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b59539f7d0 con 0x55b59543de30 2024-05-14T16:58:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.451+0000 7f7ce7427740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b5953a3810 con 0x55b595450170 2024-05-14T16:58:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.451+0000 7f7ce1ca5640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b595450170 0x55b595452560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:28.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.451+0000 7f7ce1ca5640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b595450170 0x55b595452560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:54558/0 (socket says 172.21.15.2:54558) 2024-05-14T16:58:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.451+0000 7f7ce1ca5640 1 -- 172.21.15.2:0/955768625 learned_addr learned my addr 172.21.15.2:0/955768625 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:58:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.452+0000 7f7ce1ca5640 1 -- 172.21.15.2:0/955768625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b59543de30 msgr2=0x55b59543c7b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T16:58:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.452+0000 7f7ce1ca5640 1 --2- 172.21.15.2:0/955768625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b59543de30 0x55b59543c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.452+0000 7f7ce1ca5640 1 -- 172.21.15.2:0/955768625 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b59543ccf0 msgr2=0x55b59543d170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T16:58:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.452+0000 7f7ce1ca5640 1 --2- 172.21.15.2:0/955768625 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b59543ccf0 0x55b59543d170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:29.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.452+0000 7f7ce1ca5640 1 -- 172.21.15.2:0/955768625 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5950b4210 con 0x55b595450170 2024-05-14T16:58:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.452+0000 7f7cd37fe640 1 --2- 172.21.15.2:0/724210030 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f7cc80b2b50 0x7f7cc80b5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.453+0000 7f7cd37fe640 1 -- 172.21.15.2:0/724210030 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f7cd41030f0 con 0x7f7cdc109a50 2024-05-14T16:58:29.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.453+0000 7f7ce1ca5640 1 --2- 172.21.15.2:0/955768625 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b595450170 0x55b595452560 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8009bf0 tx=0x7f7cd80117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cad031a929b7ef0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:29.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.453+0000 7f7ce2ca7640 1 --2- 172.21.15.2:0/724210030 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f7cc80b2b50 0x7f7cc80b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.454+0000 7f7cd0ff9640 1 -- 172.21.15.2:0/955768625 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd8012450 con 0x55b595450170 2024-05-14T16:58:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.454+0000 7f7cd0ff9640 1 -- 172.21.15.2:0/955768625 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cd8012aa0 con 0x55b595450170 2024-05-14T16:58:29.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.454+0000 7f7cd0ff9640 1 -- 172.21.15.2:0/955768625 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd800fac0 con 0x55b595450170 2024-05-14T16:58:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.455+0000 7f7ce7427740 1 -- 172.21.15.2:0/955768625 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b595450170 msgr2=0x55b595452560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.455+0000 7f7ce7427740 1 --2- 172.21.15.2:0/955768625 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b595450170 0x55b595452560 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8009bf0 tx=0x7f7cd80117b0 comp rx=0 tx=0).stop 2024-05-14T16:58:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.455+0000 7f7ce2ca7640 1 --2- 172.21.15.2:0/724210030 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f7cc80b2b50 0x7f7cc80b5010 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc005ab0 tx=0x7f7ccc0095a0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.455+0000 7f7ce7427740 1 -- 172.21.15.2:0/955768625 shutdown_connections 2024-05-14T16:58:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.455+0000 7f7ce7427740 1 --2- 172.21.15.2:0/955768625 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b595450170 0x55b595452560 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.455+0000 7f7ce7427740 1 --2- 172.21.15.2:0/955768625 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b59543ccf0 0x55b59543d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.455+0000 7f7ce7427740 1 --2- 172.21.15.2:0/955768625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b59543de30 0x55b59543c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.455+0000 7f7ce7427740 1 -- 172.21.15.2:0/955768625 >> 172.21.15.2:0/955768625 conn(0x55b595438e00 msgr2=0x55b5954391e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:29.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.456+0000 7f7ce7427740 1 -- 172.21.15.2:0/955768625 shutdown_connections 2024-05-14T16:58:29.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.456+0000 7f7ce7427740 1 -- 172.21.15.2:0/955768625 wait complete. 2024-05-14T16:58:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.456+0000 7f7ce7427740 1 -- start start 2024-05-14T16:58:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.456+0000 7f7ce7427740 1 Processor -- start 2024-05-14T16:58:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.456+0000 7f7ce7427740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b59543ccf0 0x55b595443010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.457+0000 7f7ce7427740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b59543de30 0x55b595440300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.457+0000 7f7ce7427740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b595450170 0x55b595440a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.457+0000 7f7ce7427740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b595085790 con 0x55b59543ccf0 2024-05-14T16:58:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.457+0000 7f7ce7427740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b59535b890 con 0x55b59543de30 2024-05-14T16:58:29.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.457+0000 7f7ce7427740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b595452fc0 con 0x55b595450170 2024-05-14T16:58:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.457+0000 7f7ce7427740 10 client.0 _ll_register_callbacks cb 0x55b59535ed60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:58:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.457+0000 7f7ce24a6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b59543ccf0 0x55b595443010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.457+0000 7f7ce24a6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b59543ccf0 0x55b595443010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57264/0 (socket says 172.21.15.2:57264) 2024-05-14T16:58:29.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.457+0000 7f7ce24a6640 1 -- 172.21.15.2:0/4039294464 learned_addr learned my addr 172.21.15.2:0/4039294464 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:58:29.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.457+0000 7f7ce1ca5640 1 --2- 172.21.15.2:0/4039294464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b59543de30 0x55b595440300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:29.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.457+0000 7f7ce2ca7640 1 --2- 172.21.15.2:0/4039294464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b595450170 0x55b595440a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.457+0000 7f7ce1ca5640 1 -- 172.21.15.2:0/4039294464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b595450170 msgr2=0x55b595440a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.457+0000 7f7ce1ca5640 1 --2- 172.21.15.2:0/4039294464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b595450170 0x55b595440a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:29.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.457+0000 7f7ce1ca5640 1 -- 172.21.15.2:0/4039294464 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b59543ccf0 msgr2=0x55b595443010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:29.330 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.457+0000 7f7ce1ca5640 1 --2- 172.21.15.2:0/4039294464 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b59543ccf0 0x55b595443010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.457+0000 7f7ce1ca5640 1 -- 172.21.15.2:0/4039294464 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b595180620 con 0x55b59543de30 2024-05-14T16:58:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.457+0000 7f7ce2ca7640 1 --2- 172.21.15.2:0/4039294464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b595450170 0x55b595440a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:58:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.457+0000 7f7ce24a6640 1 --2- 172.21.15.2:0/4039294464 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b59543ccf0 0x55b595443010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:58:29.395 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.457+0000 7f7ce1ca5640 1 --2- 172.21.15.2:0/4039294464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b59543de30 0x55b595440300 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7cd801c040 tx=0x7f7cd8012e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3cf13e9510fb829 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.457+0000 7f7cc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T16:58:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.458+0000 7f7cc5ffb640 1 -- 172.21.15.2:0/4039294464 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd8028070 con 0x55b59543de30 2024-05-14T16:58:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.458+0000 7f7cc5ffb640 1 -- 172.21.15.2:0/4039294464 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f7cd8012860 con 0x55b59543de30 2024-05-14T16:58:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.458+0000 7f7ce7427740 1 -- 172.21.15.2:0/4039294464 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cd8012aa0 con 0x55b59543de30 2024-05-14T16:58:29.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.458+0000 7f7cc5ffb640 1 -- 172.21.15.2:0/4039294464 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7cd8031050 con 0x55b59543de30 2024-05-14T16:58:29.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.458+0000 7f7ce7427740 10 client.25442 fetch_fsmap learned FSMap version 35 2024-05-14T16:58:29.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.458+0000 7f7ce7427740 1 -- 172.21.15.2:0/4039294464 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b59545eba0 con 0x55b59543de30 2024-05-14T16:58:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.459+0000 7f7cc5ffb640 1 -- 172.21.15.2:0/4039294464 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cd8012aa0 con 0x55b59543de30 2024-05-14T16:58:29.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.459+0000 7f7cc5ffb640 1 -- 172.21.15.2:0/4039294464 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f7cd80046c0 con 0x55b59543de30 2024-05-14T16:58:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.459+0000 7f7ce7427740 10 client.25442 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T16:58:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.459+0000 7f7ce7427740 10 client.25442 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:58:29.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.459+0000 7f7ce7427740 20 client.25442 populate_metadata read hostname 'smithi002' 2024-05-14T16:58:29.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.459+0000 7f7ce7427740 1 --2- 172.21.15.2:0/4039294464 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55b59545ffb0 0x55b5954623a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.460+0000 7f7ce7427740 4 client.25442 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:58:29.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.460+0000 7f7ce7427740 1 -- 172.21.15.2:0/4039294464 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5953b1b80 con 0x55b59545ffb0 2024-05-14T16:58:29.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.460+0000 7f7ce24a6640 1 --2- 172.21.15.2:0/4039294464 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55b59545ffb0 0x55b5954623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.460+0000 7f7cc5ffb640 10 client.25442 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:58:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.460+0000 7f7ce24a6640 1 --2- 172.21.15.2:0/4039294464 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55b59545ffb0 0x55b5954623a0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c77a0d651e3968a server_cookie=26993ed393730e87 in_seq=0 out_seq=0 2024-05-14T16:58:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.461+0000 7f7cc5ffb640 1 -- 172.21.15.2:0/4039294464 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b5953b1b80 con 0x55b59545ffb0 2024-05-14T16:58:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.461+0000 7f7cc5ffb640 10 client.25442 handle_command_reply: tid=0 2024-05-14T16:58:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.461+0000 7f7ce7427740 1 client.25442 shutdown 2024-05-14T16:58:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.461+0000 7f7ce7427740 1 -- 172.21.15.2:0/4039294464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b59543de30 msgr2=0x55b595440300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.461+0000 7f7ce7427740 1 --2- 172.21.15.2:0/4039294464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b59543de30 0x55b595440300 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7cd801c040 tx=0x7f7cd8012e80 comp rx=0 tx=0).stop 2024-05-14T16:58:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.461+0000 7f7ce7427740 1 -- 172.21.15.2:0/4039294464 shutdown_connections 2024-05-14T16:58:29.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.461+0000 7f7ce7427740 1 --2- 172.21.15.2:0/4039294464 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55b59545ffb0 0x55b5954623a0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:29.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.461+0000 7f7ce7427740 1 --2- 172.21.15.2:0/4039294464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b595450170 0x55b595440a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:29.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.461+0000 7f7ce7427740 1 --2- 172.21.15.2:0/4039294464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b59543de30 0x55b595440300 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:29.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.461+0000 7f7ce7427740 1 --2- 172.21.15.2:0/4039294464 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b59543ccf0 0x55b595443010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.461+0000 7f7cc5ffb640 0 client.25442 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:58:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.461+0000 7f7ce7427740 1 -- 172.21.15.2:0/4039294464 >> 172.21.15.2:0/4039294464 conn(0x55b595438e00 msgr2=0x55b59543ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.461+0000 7f7ce7427740 1 -- 172.21.15.2:0/4039294464 shutdown_connections 2024-05-14T16:58:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.462+0000 7f7ce7427740 1 -- 172.21.15.2:0/4039294464 wait complete. 2024-05-14T16:58:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.462+0000 7f7ce7427740 20 client.25442 trim_cache size 0 max 16384 2024-05-14T16:58:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.487+0000 7f7ce7427740 1 Processor -- start 2024-05-14T16:58:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.487+0000 7f7ce7427740 1 -- start start 2024-05-14T16:58:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.487+0000 7f7ce7427740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b59543ccf0 0x55b5954402c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.487+0000 7f7ce7427740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b59543de30 0x55b59545bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:29.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.487+0000 7f7ce7427740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b595450170 0x55b595467a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.487+0000 7f7ce7427740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc0011050 con 0x55b59543de30 2024-05-14T16:58:29.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.487+0000 7f7ce7427740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc000e920 con 0x55b59543ccf0 2024-05-14T16:58:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.487+0000 7f7ce7427740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc000c770 con 0x55b595450170 2024-05-14T16:58:29.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.487+0000 7f7ce1ca5640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b59543de30 0x55b59545bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.487+0000 7f7ce1ca5640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b59543de30 0x55b59545bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57266/0 (socket says 172.21.15.2:57266) 2024-05-14T16:58:29.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.487+0000 7f7ce1ca5640 1 -- 172.21.15.2:0/3531575691 learned_addr learned my addr 172.21.15.2:0/3531575691 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:58:29.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.487+0000 7f7ce24a6640 1 --2- 172.21.15.2:0/3531575691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b59543ccf0 0x55b5954402c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:29.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.487+0000 7f7ce2ca7640 1 --2- 172.21.15.2:0/3531575691 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b595450170 0x55b595467a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:29.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.488+0000 7f7ce24a6640 1 -- 172.21.15.2:0/3531575691 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b595450170 msgr2=0x55b595467a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:29.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.488+0000 7f7ce24a6640 1 --2- 172.21.15.2:0/3531575691 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b595450170 0x55b595467a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.488+0000 7f7ce24a6640 1 -- 172.21.15.2:0/3531575691 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b59543de30 msgr2=0x55b59545bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:30.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.488+0000 7f7ce24a6640 1 --2- 172.21.15.2:0/3531575691 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b59543de30 0x55b59545bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:30.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.488+0000 7f7ce24a6640 1 -- 172.21.15.2:0/3531575691 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b595434830 con 0x55b59543ccf0 2024-05-14T16:58:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.488+0000 7f7ce24a6640 1 --2- 172.21.15.2:0/3531575691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b59543ccf0 0x55b5954402c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4100e70 tx=0x7f7cd402a770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=603f499f34f94fee server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.488+0000 7f7cc7fff640 1 -- 172.21.15.2:0/3531575691 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd40e4070 con 0x55b59543ccf0 2024-05-14T16:58:30.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.488+0000 7f7cc7fff640 1 -- 172.21.15.2:0/3531575691 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cd40d3260 con 0x55b59543ccf0 2024-05-14T16:58:30.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.488+0000 7f7cc7fff640 1 -- 172.21.15.2:0/3531575691 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd4004660 con 0x55b59543ccf0 2024-05-14T16:58:30.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.489+0000 7f7ce7427740 1 -- 172.21.15.2:0/3531575691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b59543ccf0 msgr2=0x55b5954402c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:30.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.489+0000 7f7ce7427740 1 --2- 172.21.15.2:0/3531575691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b59543ccf0 0x55b5954402c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4100e70 tx=0x7f7cd402a770 comp rx=0 tx=0).stop 2024-05-14T16:58:30.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.489+0000 7f7ce7427740 1 -- 172.21.15.2:0/3531575691 shutdown_connections 2024-05-14T16:58:30.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.489+0000 7f7ce7427740 1 --2- 172.21.15.2:0/3531575691 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b595450170 0x55b595467a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:30.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.489+0000 7f7ce7427740 1 --2- 172.21.15.2:0/3531575691 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b59543de30 0x55b59545bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.489+0000 7f7ce7427740 1 --2- 172.21.15.2:0/3531575691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b59543ccf0 0x55b5954402c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:30.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.489+0000 7f7ce7427740 1 -- 172.21.15.2:0/3531575691 >> 172.21.15.2:0/3531575691 conn(0x55b595438e00 msgr2=0x55b59543fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.489+0000 7f7ce7427740 1 -- 172.21.15.2:0/3531575691 shutdown_connections 2024-05-14T16:58:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.489+0000 7f7ce7427740 1 -- 172.21.15.2:0/3531575691 wait complete. 2024-05-14T16:58:30.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.490+0000 7f7ce7427740 1 -- start start 2024-05-14T16:58:30.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.490+0000 7f7ce7427740 1 Processor -- start 2024-05-14T16:58:30.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.490+0000 7f7ce7427740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b59543ccf0 0x55b5954582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.490+0000 7f7ce7427740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b59543de30 0x55b595458820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.490+0000 7f7ce7427740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b595450170 0x55b59544e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:30.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.490+0000 7f7ce7427740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc000cde0 con 0x55b59543de30 2024-05-14T16:58:30.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.490+0000 7f7ce7427740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc000e250 con 0x55b59543ccf0 2024-05-14T16:58:30.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.490+0000 7f7ce7427740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc000d4b0 con 0x55b595450170 2024-05-14T16:58:30.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.490+0000 7f7ce7427740 10 client.0 _ll_register_callbacks cb 0x7f7cd80097d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:58:30.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.490+0000 7f7ce1ca5640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b59543de30 0x55b595458820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:30.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.490+0000 7f7ce24a6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b59543ccf0 0x55b5954582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.490+0000 7f7ce24a6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b59543ccf0 0x55b5954582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:39058/0 (socket says 172.21.15.2:39058) 2024-05-14T16:58:30.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.490+0000 7f7ce1ca5640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b59543de30 0x55b595458820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57274/0 (socket says 172.21.15.2:57274) 2024-05-14T16:58:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.490+0000 7f7ce24a6640 1 -- 172.21.15.2:0/1319701738 learned_addr learned my addr 172.21.15.2:0/1319701738 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:58:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.490+0000 7f7ce2ca7640 1 --2- 172.21.15.2:0/1319701738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b595450170 0x55b59544e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.491+0000 7f7ce24a6640 1 -- 172.21.15.2:0/1319701738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b595450170 msgr2=0x55b59544e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:30.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.491+0000 7f7ce24a6640 1 --2- 172.21.15.2:0/1319701738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b595450170 0x55b59544e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:30.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.491+0000 7f7ce24a6640 1 -- 172.21.15.2:0/1319701738 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b59543de30 msgr2=0x55b595458820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.491+0000 7f7ce24a6640 1 --2- 172.21.15.2:0/1319701738 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b59543de30 0x55b595458820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.491+0000 7f7ce24a6640 1 -- 172.21.15.2:0/1319701738 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b59544e740 con 0x55b59543ccf0 2024-05-14T16:58:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.491+0000 7f7ce2ca7640 1 --2- 172.21.15.2:0/1319701738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b595450170 0x55b59544e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:58:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.491+0000 7f7ce24a6640 1 --2- 172.21.15.2:0/1319701738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b59543ccf0 0x55b5954582e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7cd41030c0 tx=0x7f7cd40ca050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f010cbe5fd48234 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.491+0000 7f7cc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T16:58:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.491+0000 7f7cc5ffb640 1 -- 172.21.15.2:0/1319701738 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd40e8020 con 0x55b59543ccf0 2024-05-14T16:58:30.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.491+0000 7f7cc5ffb640 1 -- 172.21.15.2:0/1319701738 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f7cd40eb380 con 0x55b59543ccf0 2024-05-14T16:58:30.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.491+0000 7f7ce7427740 1 -- 172.21.15.2:0/1319701738 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cd40d3260 con 0x55b59543ccf0 2024-05-14T16:58:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.492+0000 7f7cc5ffb640 1 -- 172.21.15.2:0/1319701738 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7cd40e4040 con 0x55b59543ccf0 2024-05-14T16:58:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.493+0000 7f7ce7427740 10 client.25448 fetch_fsmap learned FSMap version 35 2024-05-14T16:58:30.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.493+0000 7f7ce7427740 1 -- 172.21.15.2:0/1319701738 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b595450c00 con 0x55b59543ccf0 2024-05-14T16:58:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.493+0000 7f7cc5ffb640 1 -- 172.21.15.2:0/1319701738 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cd40d3260 con 0x55b59543ccf0 2024-05-14T16:58:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.494+0000 7f7cc5ffb640 1 -- 172.21.15.2:0/1319701738 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f7cd40eb020 con 0x55b59543ccf0 2024-05-14T16:58:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.494+0000 7f7ce7427740 10 client.25448 fetch_fsmap finished waiting for FSMap version 35 2024-05-14T16:58:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.494+0000 7f7ce7427740 10 client.25448 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:58:30.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.494+0000 7f7ce7427740 20 client.25448 populate_metadata read hostname 'smithi002' 2024-05-14T16:58:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.494+0000 7f7ce7427740 1 --2- 172.21.15.2:0/1319701738 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55b5954ca5b0 0x55b5954cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:30.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.494+0000 7f7ce7427740 4 client.25448 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:58:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.494+0000 7f7ce7427740 1 -- 172.21.15.2:0/1319701738 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b5953ab360 con 0x55b5954ca5b0 2024-05-14T16:58:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.495+0000 7f7ce1ca5640 1 --2- 172.21.15.2:0/1319701738 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55b5954ca5b0 0x55b5954cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.495+0000 7f7cc5ffb640 10 client.25448 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:58:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.495+0000 7f7ce1ca5640 1 --2- 172.21.15.2:0/1319701738 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55b5954ca5b0 0x55b5954cc9a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88fe7189cda8864f server_cookie=a52eda7239563194 in_seq=0 out_seq=0 2024-05-14T16:58:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.508+0000 7f7cc5ffb640 1 -- 172.21.15.2:0/1319701738 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b5953ab360 con 0x55b5954ca5b0 2024-05-14T16:58:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.508+0000 7f7cc5ffb640 10 client.25448 handle_command_reply: tid=0 2024-05-14T16:58:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.509+0000 7f7ce7427740 1 client.25448 shutdown 2024-05-14T16:58:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.509+0000 7f7ce7427740 1 -- 172.21.15.2:0/1319701738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b59543ccf0 msgr2=0x55b5954582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.509+0000 7f7ce7427740 1 --2- 172.21.15.2:0/1319701738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b59543ccf0 0x55b5954582e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7cd41030c0 tx=0x7f7cd40ca050 comp rx=0 tx=0).stop 2024-05-14T16:58:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.509+0000 7f7ce7427740 1 -- 172.21.15.2:0/1319701738 shutdown_connections 2024-05-14T16:58:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.509+0000 7f7ce7427740 1 --2- 172.21.15.2:0/1319701738 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55b5954ca5b0 0x55b5954cc9a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:30.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.509+0000 7f7ce7427740 1 --2- 172.21.15.2:0/1319701738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b595450170 0x55b59544e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.509+0000 7f7ce7427740 1 --2- 172.21.15.2:0/1319701738 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b59543de30 0x55b595458820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.509+0000 7f7cc5ffb640 0 client.25448 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:58:30.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.509+0000 7f7ce7427740 1 --2- 172.21.15.2:0/1319701738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b59543ccf0 0x55b5954582e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.509+0000 7f7ce7427740 1 -- 172.21.15.2:0/1319701738 >> 172.21.15.2:0/1319701738 conn(0x55b595438e00 msgr2=0x55b59543e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.509+0000 7f7ce7427740 1 -- 172.21.15.2:0/1319701738 shutdown_connections 2024-05-14T16:58:30.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.509+0000 7f7ce7427740 1 -- 172.21.15.2:0/1319701738 wait complete. 2024-05-14T16:58:30.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.509+0000 7f7ce7427740 20 client.25448 trim_cache size 0 max 16384 2024-05-14T16:58:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T16:58:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T16:58:30.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T16:58:30.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.510+0000 7f7ce470a640 1 -- 172.21.15.2:0/724210030 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f7cc80b2b50 msgr2=0x7f7cc80b5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:30.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.510+0000 7f7ce470a640 1 --2- 172.21.15.2:0/724210030 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f7cc80b2b50 0x7f7cc80b5010 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc005ab0 tx=0x7f7ccc0095a0 comp rx=0 tx=0).stop 2024-05-14T16:58:30.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.511+0000 7f7ce470a640 1 -- 172.21.15.2:0/724210030 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cdc109a50 msgr2=0x7f7cdc1a76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:30.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.511+0000 7f7ce470a640 1 --2- 172.21.15.2:0/724210030 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cdc109a50 0x7f7cdc1a76f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4000f80 tx=0x7f7cd4003ba0 comp rx=0 tx=0).stop 2024-05-14T16:58:30.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.511+0000 7f7ce470a640 1 -- 172.21.15.2:0/724210030 shutdown_connections 2024-05-14T16:58:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.511+0000 7f7ce470a640 1 --2- 172.21.15.2:0/724210030 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f7cc80b2b50 0x7f7cc80b5010 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:30.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.511+0000 7f7ce470a640 1 --2- 172.21.15.2:0/724210030 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7cdc115ba0 0x7f7cdc1a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.511+0000 7f7ce470a640 1 --2- 172.21.15.2:0/724210030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7cdc10a370 0x7f7cdc1a7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.511+0000 7f7ce470a640 1 --2- 172.21.15.2:0/724210030 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cdc109a50 0x7f7cdc1a76f0 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4000f80 tx=0x7f7cd4003ba0 comp rx=0 tx=0).stop 2024-05-14T16:58:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.511+0000 7f7ce470a640 1 -- 172.21.15.2:0/724210030 >> 172.21.15.2:0/724210030 conn(0x7f7cdc06db20 msgr2=0x7f7cdc071770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.511+0000 7f7ce470a640 1 -- 172.21.15.2:0/724210030 shutdown_connections 2024-05-14T16:58:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:28.511+0000 7f7ce470a640 1 -- 172.21.15.2:0/724210030 wait complete. 2024-05-14T16:58:30.906 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:58:28 smithi002 ceph-mon[31538]: pgmap v116: 97 pgs: 97 active+clean; 2.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.8 MiB/s wr, 817 op/s 2024-05-14T16:58:30.915 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:58:28 smithi123 ceph-mon[35971]: pgmap v116: 97 pgs: 97 active+clean; 2.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.8 MiB/s wr, 817 op/s 2024-05-14T16:58:30.924 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:58:30 smithi123 ceph-mon[35971]: pgmap v117: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 10 MiB/s wr, 796 op/s 2024-05-14T16:58:30.933 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:58:28 smithi151 ceph-mon[36066]: pgmap v116: 97 pgs: 97 active+clean; 2.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.8 MiB/s wr, 817 op/s 2024-05-14T16:58:30.942 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:58:30 smithi151 ceph-mon[36066]: pgmap v117: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 10 MiB/s wr, 796 op/s 2024-05-14T16:58:30.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:58:30 smithi002 ceph-mon[31538]: pgmap v117: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 10 MiB/s wr, 796 op/s 2024-05-14T16:58:31.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 195.0 s, 3185.2 tps, lat 31.214 ms stddev 33.841 2024-05-14T16:58:32.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:58:32 smithi151 ceph-mon[36066]: pgmap v118: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.5 MiB/s wr, 643 op/s 2024-05-14T16:58:32.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:58:32 smithi123 ceph-mon[35971]: pgmap v118: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.5 MiB/s wr, 643 op/s 2024-05-14T16:58:32.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:58:32 smithi002 ceph-mon[31538]: pgmap v118: 97 pgs: 97 active+clean; 2.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.5 MiB/s wr, 643 op/s 2024-05-14T16:58:34.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:58:34 smithi151 ceph-mon[36066]: pgmap v119: 97 pgs: 97 active+clean; 2.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 14 MiB/s wr, 992 op/s 2024-05-14T16:58:34.945 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:58:34 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:58:34.953 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:58:34 smithi123 ceph-mon[35971]: pgmap v119: 97 pgs: 97 active+clean; 2.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 14 MiB/s wr, 992 op/s 2024-05-14T16:58:34.960 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:58:34 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:58:34.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:58:34 smithi002 ceph-mon[31538]: pgmap v119: 97 pgs: 97 active+clean; 2.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 14 MiB/s wr, 992 op/s 2024-05-14T16:58:34.999 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:58:34 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T16:58:36.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 200.0 s, 3614.8 tps, lat 27.782 ms stddev 29.680 2024-05-14T16:58:36.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:58:36 smithi151 ceph-mon[36066]: pgmap v120: 97 pgs: 97 active+clean; 2.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.8 MiB/s wr, 685 op/s 2024-05-14T16:58:36.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:58:36 smithi123 ceph-mon[35971]: pgmap v120: 97 pgs: 97 active+clean; 2.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.8 MiB/s wr, 685 op/s 2024-05-14T16:58:36.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:58:36 smithi002 ceph-mon[31538]: pgmap v120: 97 pgs: 97 active+clean; 2.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.8 MiB/s wr, 685 op/s 2024-05-14T16:58:38.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:58:38 smithi151 ceph-mon[36066]: pgmap v121: 97 pgs: 97 active+clean; 2.1 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 11 MiB/s wr, 837 op/s 2024-05-14T16:58:38.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:58:38 smithi123 ceph-mon[35971]: pgmap v121: 97 pgs: 97 active+clean; 2.1 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 11 MiB/s wr, 837 op/s 2024-05-14T16:58:38.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:58:38 smithi002 ceph-mon[31538]: pgmap v121: 97 pgs: 97 active+clean; 2.1 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 11 MiB/s wr, 837 op/s 2024-05-14T16:58:40.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:58:40 smithi151 ceph-mon[36066]: pgmap v122: 97 pgs: 97 active+clean; 2.1 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 822 op/s 2024-05-14T16:58:40.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:58:40 smithi123 ceph-mon[35971]: pgmap v122: 97 pgs: 97 active+clean; 2.1 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 822 op/s 2024-05-14T16:58:40.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:58:40 smithi002 ceph-mon[31538]: pgmap v122: 97 pgs: 97 active+clean; 2.1 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 822 op/s 2024-05-14T16:58:41.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.649+0000 7f99515da640 1 -- 172.21.15.2:0/2346144221 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f994c109c60 msgr2=0x7f994c10dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.649+0000 7f99515da640 1 --2- 172.21.15.2:0/2346144221 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f994c109c60 0x7f994c10dca0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9934009ca0 tx=0x7f993402a8c0 comp rx=0 tx=0).stop 2024-05-14T16:58:41.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.649+0000 7f99515da640 1 -- 172.21.15.2:0/2346144221 shutdown_connections 2024-05-14T16:58:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.649+0000 7f99515da640 1 --2- 172.21.15.2:0/2346144221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f994c10e380 0x7f994c115f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:41.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.649+0000 7f99515da640 1 --2- 172.21.15.2:0/2346144221 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f994c109c60 0x7f994c10dca0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.649+0000 7f99515da640 1 --2- 172.21.15.2:0/2346144221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f994c1092b0 0x7f994c109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.649+0000 7f99515da640 1 -- 172.21.15.2:0/2346144221 >> 172.21.15.2:0/2346144221 conn(0x7f994c076fe0 msgr2=0x7f994c079400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:41.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.649+0000 7f99515da640 1 -- 172.21.15.2:0/2346144221 shutdown_connections 2024-05-14T16:58:41.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.650+0000 7f99515da640 1 -- 172.21.15.2:0/2346144221 wait complete. 2024-05-14T16:58:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.650+0000 7f99515da640 1 Processor -- start 2024-05-14T16:58:41.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.651+0000 7f99515da640 1 -- start start 2024-05-14T16:58:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.651+0000 7f99515da640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f994c1092b0 0x7f994c1a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:41.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.651+0000 7f99515da640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f994c10e380 0x7f994c1a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.651+0000 7f99515da640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f994c19f610 0x7f994c19fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.651+0000 7f99515da640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f994c1192f0 con 0x7f994c1092b0 2024-05-14T16:58:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.651+0000 7f99515da640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f994c119170 con 0x7f994c10e380 2024-05-14T16:58:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.651+0000 7f99515da640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f994c119470 con 0x7f994c19f610 2024-05-14T16:58:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.651+0000 7f994affd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f994c1092b0 0x7f994c1a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.651+0000 7f994affd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f994c1092b0 0x7f994c1a5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58846/0 (socket says 172.21.15.2:58846) 2024-05-14T16:58:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.651+0000 7f994affd640 1 -- 172.21.15.2:0/3630622635 learned_addr learned my addr 172.21.15.2:0/3630622635 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:58:41.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.652+0000 7f99427fc640 1 --2- 172.21.15.2:0/3630622635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f994c10e380 0x7f994c1a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.652+0000 7f994b7fe640 1 --2- 172.21.15.2:0/3630622635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f994c19f610 0x7f994c19fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.652+0000 7f994affd640 1 -- 172.21.15.2:0/3630622635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f994c19f610 msgr2=0x7f994c19fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.652+0000 7f994affd640 1 --2- 172.21.15.2:0/3630622635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f994c19f610 0x7f994c19fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.652+0000 7f994affd640 1 -- 172.21.15.2:0/3630622635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f994c10e380 msgr2=0x7f994c1a59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.652+0000 7f994affd640 1 --2- 172.21.15.2:0/3630622635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f994c10e380 0x7f994c1a59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:42.033 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.652+0000 7f994affd640 1 -- 172.21.15.2:0/3630622635 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994c1a0130 con 0x7f994c1092b0 2024-05-14T16:58:42.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.652+0000 7f994affd640 1 --2- 172.21.15.2:0/3630622635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f994c1092b0 0x7f994c1a5460 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f993c00e9f0 tx=0x7f993c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:42.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.652+0000 7f9948ff9640 1 -- 172.21.15.2:0/3630622635 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c00cd80 con 0x7f994c1092b0 2024-05-14T16:58:42.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.652+0000 7f99515da640 1 -- 172.21.15.2:0/3630622635 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f994c1a0420 con 0x7f994c1092b0 2024-05-14T16:58:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.652+0000 7f9948ff9640 1 -- 172.21.15.2:0/3630622635 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f993c0108e0 con 0x7f994c1092b0 2024-05-14T16:58:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.652+0000 7f9948ff9640 1 -- 172.21.15.2:0/3630622635 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c018980 con 0x7f994c1092b0 2024-05-14T16:58:42.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.652+0000 7f99515da640 1 -- 172.21.15.2:0/3630622635 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f994c1ac670 con 0x7f994c1092b0 2024-05-14T16:58:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.655+0000 7f99543c6740 1 Processor -- start 2024-05-14T16:58:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.655+0000 7f9948ff9640 1 -- 172.21.15.2:0/3630622635 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f993c018b20 con 0x7f994c1092b0 2024-05-14T16:58:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.657+0000 7f99543c6740 1 -- start start 2024-05-14T16:58:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.657+0000 7f9948ff9640 1 --2- 172.21.15.2:0/3630622635 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f99380b2a90 0x7f99380b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:42.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.657+0000 7f994b7fe640 1 --2- 172.21.15.2:0/3630622635 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f99380b2a90 0x7f99380b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.657+0000 7f99543c6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f566c608f0 0x55f566c60cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.657+0000 7f99543c6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f566c61210 0x55f566c6b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.657+0000 7f99543c6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f566c6bad0 0x55f566c6dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:42.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.657+0000 7f99543c6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f566bd3c30 con 0x55f566c6bad0 2024-05-14T16:58:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.657+0000 7f99543c6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f566bc37d0 con 0x55f566c608f0 2024-05-14T16:58:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.657+0000 7f99543c6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f566bc7810 con 0x55f566c61210 2024-05-14T16:58:42.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.657+0000 7f9948ff9640 1 -- 172.21.15.2:0/3630622635 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f993c0e2a70 con 0x7f994c1092b0 2024-05-14T16:58:42.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.657+0000 7f994affd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f566c608f0 0x55f566c60cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:42.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.657+0000 7f99427fc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f566c61210 0x55f566c6b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:42.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.657+0000 7f994b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f566c6bad0 0x55f566c6dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.658+0000 7f994affd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f566c608f0 0x55f566c60cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:43668/0 (socket says 172.21.15.2:43668) 2024-05-14T16:58:42.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.658+0000 7f99427fc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f566c61210 0x55f566c6b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:54710/0 (socket says 172.21.15.2:54710) 2024-05-14T16:58:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.658+0000 7f994affd640 1 -- 172.21.15.2:0/532479829 learned_addr learned my addr 172.21.15.2:0/532479829 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:58:42.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.658+0000 7f994b7fe640 1 --2- 172.21.15.2:0/3630622635 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f99380b2a90 0x7f99380b4f50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9944009b60 tx=0x7f9944009f60 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:42.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.658+0000 7f994b7fe640 1 -- 172.21.15.2:0/532479829 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f566c61210 msgr2=0x55f566c6b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.658+0000 7f994b7fe640 1 --2- 172.21.15.2:0/532479829 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f566c61210 0x55f566c6b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.658+0000 7f994b7fe640 1 -- 172.21.15.2:0/532479829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f566c608f0 msgr2=0x55f566c60cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.658+0000 7f994b7fe640 1 --2- 172.21.15.2:0/532479829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f566c608f0 0x55f566c60cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.658+0000 7f994b7fe640 1 -- 172.21.15.2:0/532479829 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5668d8210 con 0x55f566c6bad0 2024-05-14T16:58:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.658+0000 7f994b7fe640 1 --2- 172.21.15.2:0/532479829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f566c6bad0 0x55f566c6dec0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f99440095f0 tx=0x7f994401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=700490f2b77fb8be server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.658+0000 7f99417fa640 1 -- 172.21.15.2:0/532479829 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99440142d0 con 0x55f566c6bad0 2024-05-14T16:58:42.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.658+0000 7f99417fa640 1 -- 172.21.15.2:0/532479829 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9944011e80 con 0x55f566c6bad0 2024-05-14T16:58:42.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.658+0000 7f99417fa640 1 -- 172.21.15.2:0/532479829 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994401f880 con 0x55f566c6bad0 2024-05-14T16:58:42.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.659+0000 7f99543c6740 1 -- 172.21.15.2:0/532479829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f566c6bad0 msgr2=0x55f566c6dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.659+0000 7f99543c6740 1 --2- 172.21.15.2:0/532479829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f566c6bad0 0x55f566c6dec0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f99440095f0 tx=0x7f994401c000 comp rx=0 tx=0).stop 2024-05-14T16:58:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.659+0000 7f99543c6740 1 -- 172.21.15.2:0/532479829 shutdown_connections 2024-05-14T16:58:42.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.659+0000 7f99543c6740 1 --2- 172.21.15.2:0/532479829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f566c6bad0 0x55f566c6dec0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:42.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.659+0000 7f99543c6740 1 --2- 172.21.15.2:0/532479829 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f566c61210 0x55f566c6b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.659+0000 7f99543c6740 1 --2- 172.21.15.2:0/532479829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f566c608f0 0x55f566c60cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:42.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.659+0000 7f99543c6740 1 -- 172.21.15.2:0/532479829 >> 172.21.15.2:0/532479829 conn(0x55f566c5ce00 msgr2=0x55f566c5d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:42.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.659+0000 7f99543c6740 1 -- 172.21.15.2:0/532479829 shutdown_connections 2024-05-14T16:58:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.659+0000 7f99543c6740 1 -- 172.21.15.2:0/532479829 wait complete. 2024-05-14T16:58:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.659+0000 7f99543c6740 1 -- start start 2024-05-14T16:58:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.659+0000 7f99543c6740 1 Processor -- start 2024-05-14T16:58:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.660+0000 7f99543c6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f566c608f0 0x55f566c6b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.660+0000 7f99543c6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f566c61210 0x55f566c68830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.660+0000 7f99543c6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f566c6bad0 0x55f566c68f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.660+0000 7f99543c6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f5668a9790 con 0x55f566c6bad0 2024-05-14T16:58:42.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.660+0000 7f99543c6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f566b7f890 con 0x55f566c608f0 2024-05-14T16:58:42.618 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.660+0000 7f99543c6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f566c6e7a0 con 0x55f566c61210 2024-05-14T16:58:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.660+0000 7f994affd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f566c608f0 0x55f566c6b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.660+0000 7f99543c6740 10 client.0 _ll_register_callbacks cb 0x55f566b82d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:58:42.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.660+0000 7f994affd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f566c608f0 0x55f566c6b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:43674/0 (socket says 172.21.15.2:43674) 2024-05-14T16:58:42.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.660+0000 7f994affd640 1 -- 172.21.15.2:0/1317684507 learned_addr learned my addr 172.21.15.2:0/1317684507 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:58:42.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.660+0000 7f99427fc640 1 --2- 172.21.15.2:0/1317684507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f566c61210 0x55f566c68830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:42.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.660+0000 7f994affd640 1 -- 172.21.15.2:0/1317684507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f566c61210 msgr2=0x55f566c68830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.660+0000 7f994affd640 1 --2- 172.21.15.2:0/1317684507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f566c61210 0x55f566c68830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:42.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.660+0000 7f994affd640 1 -- 172.21.15.2:0/1317684507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f566c6bad0 msgr2=0x55f566c68f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:42.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.660+0000 7f994affd640 1 --2- 172.21.15.2:0/1317684507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f566c6bad0 0x55f566c68f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:42.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.660+0000 7f994affd640 1 -- 172.21.15.2:0/1317684507 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5669a4620 con 0x55f566c608f0 2024-05-14T16:58:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.661+0000 7f994affd640 1 --2- 172.21.15.2:0/1317684507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f566c608f0 0x55f566c6b540 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f993c00e9c0 tx=0x7f993c0a9970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ee9996079041f59 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.661+0000 7f99327fc640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T16:58:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.661+0000 7f99327fc640 1 -- 172.21.15.2:0/1317684507 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c0ae0d0 con 0x55f566c608f0 2024-05-14T16:58:42.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.661+0000 7f99327fc640 1 -- 172.21.15.2:0/1317684507 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f993c0e0410 con 0x55f566c608f0 2024-05-14T16:58:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.661+0000 7f99543c6740 1 -- 172.21.15.2:0/1317684507 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f566bd5b80 con 0x55f566c608f0 2024-05-14T16:58:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.661+0000 7f99327fc640 1 -- 172.21.15.2:0/1317684507 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f993c0c0070 con 0x55f566c608f0 2024-05-14T16:58:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.662+0000 7f99543c6740 10 client.25460 fetch_fsmap learned FSMap version 36 2024-05-14T16:58:42.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.662+0000 7f99543c6740 1 -- 172.21.15.2:0/1317684507 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f566c7a2c0 con 0x55f566c608f0 2024-05-14T16:58:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.662+0000 7f99327fc640 1 -- 172.21.15.2:0/1317684507 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f566bd5b80 con 0x55f566c608f0 2024-05-14T16:58:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.662+0000 7f99327fc640 1 -- 172.21.15.2:0/1317684507 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 36) ==== 8342+0+0 (secure 0 0 0) 0x7f993c0b27f0 con 0x55f566c608f0 2024-05-14T16:58:42.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.662+0000 7f99543c6740 10 client.25460 fetch_fsmap finished waiting for FSMap version 36 2024-05-14T16:58:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.662+0000 7f99543c6740 10 client.25460 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:58:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.662+0000 7f99543c6740 20 client.25460 populate_metadata read hostname 'smithi002' 2024-05-14T16:58:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.662+0000 7f99543c6740 1 --2- 172.21.15.2:0/1317684507 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55f566c7b730 0x55f566c7dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.662+0000 7f99543c6740 4 client.25460 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:58:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.662+0000 7f99543c6740 1 -- 172.21.15.2:0/1317684507 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f566bcf360 con 0x55f566c7b730 2024-05-14T16:58:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.663+0000 7f99427fc640 1 --2- 172.21.15.2:0/1317684507 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55f566c7b730 0x55f566c7dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.664+0000 7f99427fc640 1 --2- 172.21.15.2:0/1317684507 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55f566c7b730 0x55f566c7dbc0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=525564f7ff5b72bd server_cookie=e6a8a1e37799f4ff in_seq=0 out_seq=0 2024-05-14T16:58:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.664+0000 7f99327fc640 10 client.25460 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:58:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.665+0000 7f99327fc640 1 -- 172.21.15.2:0/1317684507 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f566bcf360 con 0x55f566c7b730 2024-05-14T16:58:42.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.665+0000 7f99327fc640 10 client.25460 handle_command_reply: tid=0 2024-05-14T16:58:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.665+0000 7f99543c6740 1 client.25460 shutdown 2024-05-14T16:58:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.665+0000 7f99543c6740 1 -- 172.21.15.2:0/1317684507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f566c608f0 msgr2=0x55f566c6b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.665+0000 7f99543c6740 1 --2- 172.21.15.2:0/1317684507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f566c608f0 0x55f566c6b540 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f993c00e9c0 tx=0x7f993c0a9970 comp rx=0 tx=0).stop 2024-05-14T16:58:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.665+0000 7f99543c6740 1 -- 172.21.15.2:0/1317684507 shutdown_connections 2024-05-14T16:58:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.665+0000 7f99543c6740 1 --2- 172.21.15.2:0/1317684507 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55f566c7b730 0x55f566c7dbc0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.665+0000 7f99327fc640 0 client.25460 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:58:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.665+0000 7f99543c6740 1 --2- 172.21.15.2:0/1317684507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f566c6bad0 0x55f566c68f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.665+0000 7f99543c6740 1 --2- 172.21.15.2:0/1317684507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f566c61210 0x55f566c68830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.665+0000 7f99543c6740 1 --2- 172.21.15.2:0/1317684507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f566c608f0 0x55f566c6b540 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.665+0000 7f99543c6740 1 -- 172.21.15.2:0/1317684507 >> 172.21.15.2:0/1317684507 conn(0x55f566c5ce00 msgr2=0x55f566c5c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:43.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.665+0000 7f99543c6740 1 -- 172.21.15.2:0/1317684507 shutdown_connections 2024-05-14T16:58:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.665+0000 7f99543c6740 1 -- 172.21.15.2:0/1317684507 wait complete. 2024-05-14T16:58:43.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.665+0000 7f99543c6740 20 client.25460 trim_cache size 0 max 16384 2024-05-14T16:58:43.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.687+0000 7f99543c6740 1 Processor -- start 2024-05-14T16:58:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.687+0000 7f99543c6740 1 -- start start 2024-05-14T16:58:43.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.687+0000 7f99543c6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f566c608f0 0x55f566c57f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:43.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.688+0000 7f99543c6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f566c61210 0x55f566c77670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.688+0000 7f99543c6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f566c6bad0 0x55f566c6b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:43.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.688+0000 7f99543c6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f994c1b7570 con 0x55f566c6bad0 2024-05-14T16:58:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.688+0000 7f99543c6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f994c1b4e40 con 0x55f566c608f0 2024-05-14T16:58:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.688+0000 7f99543c6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f994c1b2c90 con 0x55f566c61210 2024-05-14T16:58:43.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.688+0000 7f994affd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f566c608f0 0x55f566c57f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:43.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.688+0000 7f99427fc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f566c61210 0x55f566c77670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.688+0000 7f994b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f566c6bad0 0x55f566c6b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.688+0000 7f994affd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f566c608f0 0x55f566c57f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:43678/0 (socket says 172.21.15.2:43678) 2024-05-14T16:58:43.237 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.688+0000 7f994affd640 1 -- 172.21.15.2:0/4047285138 learned_addr learned my addr 172.21.15.2:0/4047285138 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:58:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.688+0000 7f99427fc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f566c61210 0x55f566c77670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:54732/0 (socket says 172.21.15.2:54732) 2024-05-14T16:58:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.688+0000 7f994affd640 1 -- 172.21.15.2:0/4047285138 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f566c61210 msgr2=0x55f566c77670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:43.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.688+0000 7f994affd640 1 --2- 172.21.15.2:0/4047285138 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f566c61210 0x55f566c77670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:43.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.688+0000 7f994affd640 1 -- 172.21.15.2:0/4047285138 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f566c6bad0 msgr2=0x55f566c6b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:43.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.688+0000 7f994affd640 1 --2- 172.21.15.2:0/4047285138 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f566c6bad0 0x55f566c6b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:43.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.688+0000 7f994affd640 1 -- 172.21.15.2:0/4047285138 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f566c61040 con 0x55f566c608f0 2024-05-14T16:58:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.688+0000 7f994b7fe640 1 --2- 172.21.15.2:0/4047285138 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f566c6bad0 0x55f566c6b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:58:43.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.688+0000 7f994affd640 1 --2- 172.21.15.2:0/4047285138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f566c608f0 0x55f566c57f90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f993c0a9cb0 tx=0x7f993c018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7f977b1165f9a30 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:43.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.688+0000 7f9940ff9640 1 -- 172.21.15.2:0/4047285138 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c0ae0d0 con 0x55f566c608f0 2024-05-14T16:58:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.688+0000 7f9940ff9640 1 -- 172.21.15.2:0/4047285138 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f993c08b020 con 0x55f566c608f0 2024-05-14T16:58:43.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.688+0000 7f9940ff9640 1 -- 172.21.15.2:0/4047285138 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c0a5710 con 0x55f566c608f0 2024-05-14T16:58:43.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.689+0000 7f99543c6740 1 -- 172.21.15.2:0/4047285138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f566c608f0 msgr2=0x55f566c57f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.689+0000 7f99543c6740 1 --2- 172.21.15.2:0/4047285138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f566c608f0 0x55f566c57f90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f993c0a9cb0 tx=0x7f993c018420 comp rx=0 tx=0).stop 2024-05-14T16:58:43.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.689+0000 7f99543c6740 1 -- 172.21.15.2:0/4047285138 shutdown_connections 2024-05-14T16:58:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.689+0000 7f99543c6740 1 --2- 172.21.15.2:0/4047285138 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f566c6bad0 0x55f566c6b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.689+0000 7f99543c6740 1 --2- 172.21.15.2:0/4047285138 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f566c61210 0x55f566c77670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.689+0000 7f99543c6740 1 --2- 172.21.15.2:0/4047285138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f566c608f0 0x55f566c57f90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:43.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.689+0000 7f99543c6740 1 -- 172.21.15.2:0/4047285138 >> 172.21.15.2:0/4047285138 conn(0x55f566c5ce00 msgr2=0x55f566c63dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:43.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.689+0000 7f99543c6740 1 -- 172.21.15.2:0/4047285138 shutdown_connections 2024-05-14T16:58:43.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.689+0000 7f99543c6740 1 -- 172.21.15.2:0/4047285138 wait complete. 2024-05-14T16:58:43.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.690+0000 7f99543c6740 1 -- start start 2024-05-14T16:58:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.690+0000 7f99543c6740 1 Processor -- start 2024-05-14T16:58:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.691+0000 7f99543c6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f566c608f0 0x55f566c73ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:43.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.691+0000 7f99543c6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f566c61210 0x55f566c73ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.691+0000 7f99543c6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f566c6bad0 0x55f566c6eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:43.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.691+0000 7f994affd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f566c608f0 0x55f566c73ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:43.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.691+0000 7f994affd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f566c608f0 0x55f566c73ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:43688/0 (socket says 172.21.15.2:43688) 2024-05-14T16:58:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.691+0000 7f994affd640 1 -- 172.21.15.2:0/2710595541 learned_addr learned my addr 172.21.15.2:0/2710595541 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:58:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.691+0000 7f99543c6740 1 -- 172.21.15.2:0/2710595541 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f994c1b3300 con 0x55f566c6bad0 2024-05-14T16:58:43.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.691+0000 7f99427fc640 1 --2- 172.21.15.2:0/2710595541 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f566c61210 0x55f566c73ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:43.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.691+0000 7f99543c6740 1 -- 172.21.15.2:0/2710595541 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f994c1b4770 con 0x55f566c608f0 2024-05-14T16:58:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.691+0000 7f99543c6740 1 -- 172.21.15.2:0/2710595541 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f994c1b39d0 con 0x55f566c61210 2024-05-14T16:58:43.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.691+0000 7f994b7fe640 1 --2- 172.21.15.2:0/2710595541 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f566c6bad0 0x55f566c6eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.691+0000 7f99543c6740 10 client.0 _ll_register_callbacks cb 0x7f993c0a9490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:58:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.692+0000 7f994affd640 1 -- 172.21.15.2:0/2710595541 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f566c61210 msgr2=0x55f566c73ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.692+0000 7f994affd640 1 --2- 172.21.15.2:0/2710595541 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f566c61210 0x55f566c73ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:43.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.692+0000 7f994affd640 1 -- 172.21.15.2:0/2710595541 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f566c6bad0 msgr2=0x55f566c6eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.692+0000 7f994affd640 1 --2- 172.21.15.2:0/2710595541 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f566c6bad0 0x55f566c6eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.692+0000 7f994affd640 1 -- 172.21.15.2:0/2710595541 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f566c6f820 con 0x55f566c608f0 2024-05-14T16:58:43.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.692+0000 7f994affd640 1 --2- 172.21.15.2:0/2710595541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f566c608f0 0x55f566c73ab0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f993c0a62c0 tx=0x7f993c01f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f05e85e659db70d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.692+0000 7f99327fc640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T16:58:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.692+0000 7f99327fc640 1 -- 172.21.15.2:0/2710595541 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c0ae0d0 con 0x55f566c608f0 2024-05-14T16:58:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.692+0000 7f99327fc640 1 -- 172.21.15.2:0/2710595541 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f993c0a6310 con 0x55f566c608f0 2024-05-14T16:58:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.692+0000 7f99543c6740 1 -- 172.21.15.2:0/2710595541 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f993c08b020 con 0x55f566c608f0 2024-05-14T16:58:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.692+0000 7f99327fc640 1 -- 172.21.15.2:0/2710595541 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f993c010640 con 0x55f566c608f0 2024-05-14T16:58:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.692+0000 7f99543c6740 10 client.25466 fetch_fsmap learned FSMap version 36 2024-05-14T16:58:43.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.692+0000 7f99543c6740 1 -- 172.21.15.2:0/2710595541 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f566ce5c10 con 0x55f566c608f0 2024-05-14T16:58:43.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.693+0000 7f99327fc640 1 -- 172.21.15.2:0/2710595541 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f993c08b020 con 0x55f566c608f0 2024-05-14T16:58:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.693+0000 7f99327fc640 1 -- 172.21.15.2:0/2710595541 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 36) ==== 8342+0+0 (secure 0 0 0) 0x7f993c0af260 con 0x55f566c608f0 2024-05-14T16:58:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.693+0000 7f99543c6740 10 client.25466 fetch_fsmap finished waiting for FSMap version 36 2024-05-14T16:58:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.693+0000 7f99543c6740 10 client.25466 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:58:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.693+0000 7f99543c6740 20 client.25466 populate_metadata read hostname 'smithi002' 2024-05-14T16:58:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.693+0000 7f99543c6740 1 --2- 172.21.15.2:0/2710595541 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55f566ce5fa0 0x55f566ce8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.693+0000 7f99543c6740 4 client.25466 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:58:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.693+0000 7f99543c6740 1 -- 172.21.15.2:0/2710595541 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f566b495f0 con 0x55f566ce5fa0 2024-05-14T16:58:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.693+0000 7f99427fc640 1 --2- 172.21.15.2:0/2710595541 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55f566ce5fa0 0x55f566ce8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.694+0000 7f99427fc640 1 --2- 172.21.15.2:0/2710595541 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55f566ce5fa0 0x55f566ce8390 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3db79deacce3a989 server_cookie=9d92407d634949f3 in_seq=0 out_seq=0 2024-05-14T16:58:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.694+0000 7f99327fc640 10 client.25466 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:58:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.705+0000 7f99327fc640 1 -- 172.21.15.2:0/2710595541 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f566b495f0 con 0x55f566ce5fa0 2024-05-14T16:58:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.705+0000 7f99327fc640 10 client.25466 handle_command_reply: tid=0 2024-05-14T16:58:43.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.706+0000 7f99543c6740 1 client.25466 shutdown 2024-05-14T16:58:43.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.706+0000 7f99543c6740 1 -- 172.21.15.2:0/2710595541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f566c608f0 msgr2=0x55f566c73ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.706+0000 7f99543c6740 1 --2- 172.21.15.2:0/2710595541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f566c608f0 0x55f566c73ab0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f993c0a62c0 tx=0x7f993c01f000 comp rx=0 tx=0).stop 2024-05-14T16:58:43.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.706+0000 7f99543c6740 1 -- 172.21.15.2:0/2710595541 shutdown_connections 2024-05-14T16:58:43.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.706+0000 7f99543c6740 1 --2- 172.21.15.2:0/2710595541 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55f566ce5fa0 0x55f566ce8390 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.707+0000 7f99543c6740 1 --2- 172.21.15.2:0/2710595541 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f566c6bad0 0x55f566c6eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:43.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.707+0000 7f99543c6740 1 --2- 172.21.15.2:0/2710595541 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f566c61210 0x55f566c73ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.707+0000 7f99543c6740 1 --2- 172.21.15.2:0/2710595541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f566c608f0 0x55f566c73ab0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:43.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.707+0000 7f99543c6740 1 -- 172.21.15.2:0/2710595541 >> 172.21.15.2:0/2710595541 conn(0x55f566c5ce00 msgr2=0x55f566c63dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.707+0000 7f99327fc640 0 client.25466 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:58:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.707+0000 7f99543c6740 1 -- 172.21.15.2:0/2710595541 shutdown_connections 2024-05-14T16:58:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.707+0000 7f99543c6740 1 -- 172.21.15.2:0/2710595541 wait complete. 2024-05-14T16:58:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.707+0000 7f99543c6740 20 client.25466 trim_cache size 0 max 16384 2024-05-14T16:58:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T16:58:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T16:58:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T16:58:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.707+0000 7f99515da640 1 -- 172.21.15.2:0/3630622635 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f99380b2a90 msgr2=0x7f99380b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:44.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.707+0000 7f99515da640 1 --2- 172.21.15.2:0/3630622635 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f99380b2a90 0x7f99380b4f50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9944009b60 tx=0x7f9944009f60 comp rx=0 tx=0).stop 2024-05-14T16:58:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.707+0000 7f99515da640 1 -- 172.21.15.2:0/3630622635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f994c1092b0 msgr2=0x7f994c1a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.707+0000 7f99515da640 1 --2- 172.21.15.2:0/3630622635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f994c1092b0 0x7f994c1a5460 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f993c00e9f0 tx=0x7f993c00eeb0 comp rx=0 tx=0).stop 2024-05-14T16:58:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.707+0000 7f99515da640 1 -- 172.21.15.2:0/3630622635 shutdown_connections 2024-05-14T16:58:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.707+0000 7f99515da640 1 --2- 172.21.15.2:0/3630622635 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f99380b2a90 0x7f99380b4f50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.707+0000 7f99515da640 1 --2- 172.21.15.2:0/3630622635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f994c19f610 0x7f994c19fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:44.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.707+0000 7f99515da640 1 --2- 172.21.15.2:0/3630622635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f994c10e380 0x7f994c1a59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:44.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.707+0000 7f99515da640 1 --2- 172.21.15.2:0/3630622635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f994c1092b0 0x7f994c1a5460 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.707+0000 7f99515da640 1 -- 172.21.15.2:0/3630622635 >> 172.21.15.2:0/3630622635 conn(0x7f994c076fe0 msgr2=0x7f994c079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.708+0000 7f99515da640 1 -- 172.21.15.2:0/3630622635 shutdown_connections 2024-05-14T16:58:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:41.708+0000 7f99515da640 1 -- 172.21.15.2:0/3630622635 wait complete. 2024-05-14T16:58:44.188 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 205.0 s, 3475.0 tps, lat 28.734 ms stddev 30.403 2024-05-14T16:58:44.195 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:58:42 smithi002 ceph-mon[31538]: pgmap v123: 97 pgs: 97 active+clean; 2.1 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.2 MiB/s wr, 670 op/s 2024-05-14T16:58:44.202 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:58:42 smithi151 ceph-mon[36066]: pgmap v123: 97 pgs: 97 active+clean; 2.1 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.2 MiB/s wr, 670 op/s 2024-05-14T16:58:44.210 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:58:42 smithi123 ceph-mon[35971]: pgmap v123: 97 pgs: 97 active+clean; 2.1 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.2 MiB/s wr, 670 op/s 2024-05-14T16:58:44.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:58:44 smithi123 ceph-mon[35971]: pgmap v124: 97 pgs: 97 active+clean; 2.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 987 op/s 2024-05-14T16:58:45.345 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:58:44 smithi151 ceph-mon[36066]: pgmap v124: 97 pgs: 97 active+clean; 2.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 987 op/s 2024-05-14T16:58:45.355 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:58:44 smithi002 ceph-mon[31538]: pgmap v124: 97 pgs: 97 active+clean; 2.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 987 op/s 2024-05-14T16:58:45.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:58:45 smithi123 ceph-mon[35971]: pgmap v125: 97 pgs: 97 active+clean; 2.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.7 MiB/s wr, 638 op/s 2024-05-14T16:58:45.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:58:45 smithi002 ceph-mon[31538]: pgmap v125: 97 pgs: 97 active+clean; 2.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.7 MiB/s wr, 638 op/s 2024-05-14T16:58:46.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:58:45 smithi151 ceph-mon[36066]: pgmap v125: 97 pgs: 97 active+clean; 2.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.7 MiB/s wr, 638 op/s 2024-05-14T16:58:46.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 210.0 s, 3097.4 tps, lat 32.235 ms stddev 34.105 2024-05-14T16:58:48.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:58:48 smithi151 ceph-mon[36066]: pgmap v126: 97 pgs: 97 active+clean; 2.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.7 MiB/s wr, 757 op/s 2024-05-14T16:58:48.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:58:48 smithi123 ceph-mon[35971]: pgmap v126: 97 pgs: 97 active+clean; 2.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.7 MiB/s wr, 757 op/s 2024-05-14T16:58:48.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:58:48 smithi002 ceph-mon[31538]: pgmap v126: 97 pgs: 97 active+clean; 2.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.7 MiB/s wr, 757 op/s 2024-05-14T16:58:50.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:58:50 smithi151 ceph-mon[36066]: pgmap v127: 97 pgs: 97 active+clean; 2.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.1 MiB/s wr, 787 op/s 2024-05-14T16:58:50.664 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:58:50 smithi123 ceph-mon[35971]: pgmap v127: 97 pgs: 97 active+clean; 2.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.1 MiB/s wr, 787 op/s 2024-05-14T16:58:50.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:58:50 smithi002 ceph-mon[31538]: pgmap v127: 97 pgs: 97 active+clean; 2.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.1 MiB/s wr, 787 op/s 2024-05-14T16:58:51.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 215.0 s, 3646.6 tps, lat 27.407 ms stddev 28.588 2024-05-14T16:58:52.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:58:52 smithi151 ceph-mon[36066]: pgmap v128: 97 pgs: 97 active+clean; 2.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.3 MiB/s wr, 619 op/s 2024-05-14T16:58:52.611 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:58:52 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:58:52.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:58:52 smithi123 ceph-mon[35971]: pgmap v128: 97 pgs: 97 active+clean; 2.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.3 MiB/s wr, 619 op/s 2024-05-14T16:58:52.667 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:58:52 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:58:52.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:58:52 smithi002 ceph-mon[31538]: pgmap v128: 97 pgs: 97 active+clean; 2.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.3 MiB/s wr, 619 op/s 2024-05-14T16:58:52.750 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:58:52 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:58:53.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:58:53 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:58:53.621 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:58:53 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:58:53.675 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:58:53 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:58:53.683 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:58:53 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:58:53.691 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:58:53 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:58:53.699 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:58:53 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:58:53.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:58:53 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:58:53.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:58:53 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:58:53.764 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:58:53 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:58:54.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:58:54 smithi151 ceph-mon[36066]: pgmap v129: 97 pgs: 97 active+clean; 2.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 12 MiB/s wr, 936 op/s 2024-05-14T16:58:54.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:58:54 smithi123 ceph-mon[35971]: pgmap v129: 97 pgs: 97 active+clean; 2.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 12 MiB/s wr, 936 op/s 2024-05-14T16:58:54.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:58:54 smithi002 ceph-mon[31538]: pgmap v129: 97 pgs: 97 active+clean; 2.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 12 MiB/s wr, 936 op/s 2024-05-14T16:58:54.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.844+0000 7f943b2e1640 1 -- 172.21.15.2:0/3170303006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94340778a0 msgr2=0x7f94341133b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.844+0000 7f943b2e1640 1 --2- 172.21.15.2:0/3170303006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94340778a0 0x7f94341133b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9424009930 tx=0x7f942402a440 comp rx=0 tx=0).stop 2024-05-14T16:58:54.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.844+0000 7f943b2e1640 1 -- 172.21.15.2:0/3170303006 shutdown_connections 2024-05-14T16:58:54.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.844+0000 7f943b2e1640 1 --2- 172.21.15.2:0/3170303006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94341138f0 0x7f9434115ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:54.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.844+0000 7f943b2e1640 1 --2- 172.21.15.2:0/3170303006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94340778a0 0x7f94341133b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:54.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.844+0000 7f943b2e1640 1 --2- 172.21.15.2:0/3170303006 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9434076f80 0x7f9434077360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:54.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.844+0000 7f943b2e1640 1 -- 172.21.15.2:0/3170303006 >> 172.21.15.2:0/3170303006 conn(0x7f9434100780 msgr2=0x7f9434102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:54.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.844+0000 7f943b2e1640 1 -- 172.21.15.2:0/3170303006 shutdown_connections 2024-05-14T16:58:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.844+0000 7f943b2e1640 1 -- 172.21.15.2:0/3170303006 wait complete. 2024-05-14T16:58:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.844+0000 7f943b2e1640 1 Processor -- start 2024-05-14T16:58:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.845+0000 7f943b2e1640 1 -- start start 2024-05-14T16:58:54.993 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.845+0000 7f943b2e1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9434076f80 0x7f94341a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.845+0000 7f943b2e1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94340778a0 0x7f94341a5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:55.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.845+0000 7f943b2e1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94341138f0 0x7f943419f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.845+0000 7f943b2e1640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9434119300 con 0x7f94340778a0 2024-05-14T16:58:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.845+0000 7f943b2e1640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9434119180 con 0x7f9434076f80 2024-05-14T16:58:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.845+0000 7f943b2e1640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9434119480 con 0x7f94341138f0 2024-05-14T16:58:55.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.845+0000 7f943887c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94340778a0 0x7f94341a5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:55.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.845+0000 7f943887c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94340778a0 0x7f94341a5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41982/0 (socket says 172.21.15.2:41982) 2024-05-14T16:58:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.845+0000 7f943887c640 1 -- 172.21.15.2:0/3300134934 learned_addr learned my addr 172.21.15.2:0/3300134934 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:58:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.845+0000 7f943987e640 1 --2- 172.21.15.2:0/3300134934 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94341138f0 0x7f943419f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:55.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.845+0000 7f943907d640 1 --2- 172.21.15.2:0/3300134934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9434076f80 0x7f94341a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:55.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.845+0000 7f943887c640 1 -- 172.21.15.2:0/3300134934 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94341138f0 msgr2=0x7f943419f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.845+0000 7f943887c640 1 --2- 172.21.15.2:0/3300134934 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94341138f0 0x7f943419f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.845+0000 7f943887c640 1 -- 172.21.15.2:0/3300134934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9434076f80 msgr2=0x7f94341a55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:55.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.845+0000 7f943887c640 1 --2- 172.21.15.2:0/3300134934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9434076f80 0x7f94341a55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:55.514 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.845+0000 7f943887c640 1 -- 172.21.15.2:0/3300134934 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f943419ff60 con 0x7f94340778a0 2024-05-14T16:58:55.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.846+0000 7f943887c640 1 --2- 172.21.15.2:0/3300134934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94340778a0 0x7f94341a5b20 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9424005940 tx=0x7f94240040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:55.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.847+0000 7f942a7fc640 1 -- 172.21.15.2:0/3300134934 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942403f030 con 0x7f94340778a0 2024-05-14T16:58:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.847+0000 7f942a7fc640 1 -- 172.21.15.2:0/3300134934 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94240044c0 con 0x7f94340778a0 2024-05-14T16:58:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.847+0000 7f943b2e1640 1 -- 172.21.15.2:0/3300134934 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94341a01f0 con 0x7f94340778a0 2024-05-14T16:58:55.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.847+0000 7f943b2e1640 1 -- 172.21.15.2:0/3300134934 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94341ac380 con 0x7f94340778a0 2024-05-14T16:58:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.848+0000 7f942a7fc640 1 -- 172.21.15.2:0/3300134934 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942403f030 con 0x7f94340778a0 2024-05-14T16:58:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.851+0000 7f943e004740 1 Processor -- start 2024-05-14T16:58:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.851+0000 7f943e004740 1 -- start start 2024-05-14T16:58:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.852+0000 7f943e004740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561a7f1088f0 0x561a7f108cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.852+0000 7f943e004740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a7f109210 0x561a7f113590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.852+0000 7f943e004740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a7f113ad0 0x561a7f115ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.852+0000 7f943e004740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x561a7f07bc30 con 0x561a7f1088f0 2024-05-14T16:58:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.852+0000 7f943e004740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561a7f06b7d0 con 0x561a7f113ad0 2024-05-14T16:58:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.852+0000 7f943e004740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561a7f06f810 con 0x561a7f109210 2024-05-14T16:58:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.852+0000 7f943907d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a7f109210 0x561a7f113590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.852+0000 7f943987e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561a7f1088f0 0x561a7f108cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:55.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.852+0000 7f943987e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561a7f1088f0 0x561a7f108cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41988/0 (socket says 172.21.15.2:41988) 2024-05-14T16:58:55.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.852+0000 7f943987e640 1 -- 172.21.15.2:0/2484923636 learned_addr learned my addr 172.21.15.2:0/2484923636 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:58:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.853+0000 7f943907d640 1 -- 172.21.15.2:0/2484923636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a7f113ad0 msgr2=0x561a7f115ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T16:58:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.853+0000 7f943907d640 1 --2- 172.21.15.2:0/2484923636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a7f113ad0 0x561a7f115ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.853+0000 7f943907d640 1 -- 172.21.15.2:0/2484923636 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561a7f1088f0 msgr2=0x561a7f108cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.853+0000 7f943907d640 1 --2- 172.21.15.2:0/2484923636 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561a7f1088f0 0x561a7f108cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:55.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.853+0000 7f943907d640 1 -- 172.21.15.2:0/2484923636 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a7ed80210 con 0x561a7f109210 2024-05-14T16:58:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.853+0000 7f942a7fc640 1 -- 172.21.15.2:0/3300134934 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f9424042030 con 0x7f94340778a0 2024-05-14T16:58:55.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.853+0000 7f943907d640 1 --2- 172.21.15.2:0/2484923636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a7f109210 0x561a7f113590 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9430009c00 tx=0x7f9430011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc606e30ec464f4b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.853+0000 7f941f7fe640 1 -- 172.21.15.2:0/2484923636 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9430009db0 con 0x561a7f109210 2024-05-14T16:58:55.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.853+0000 7f941f7fe640 1 -- 172.21.15.2:0/2484923636 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94300054d0 con 0x561a7f109210 2024-05-14T16:58:55.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.853+0000 7f941f7fe640 1 -- 172.21.15.2:0/2484923636 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9430019410 con 0x561a7f109210 2024-05-14T16:58:55.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.854+0000 7f942a7fc640 1 --2- 172.21.15.2:0/3300134934 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f94200b2c00 0x7f94200b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.854+0000 7f943987e640 1 --2- 172.21.15.2:0/3300134934 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f94200b2c00 0x7f94200b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.854+0000 7f943e004740 1 -- 172.21.15.2:0/2484923636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a7f109210 msgr2=0x561a7f113590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:55.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.854+0000 7f943e004740 1 --2- 172.21.15.2:0/2484923636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a7f109210 0x561a7f113590 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9430009c00 tx=0x7f9430011810 comp rx=0 tx=0).stop 2024-05-14T16:58:55.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.854+0000 7f942a7fc640 1 -- 172.21.15.2:0/3300134934 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f9424102510 con 0x7f94340778a0 2024-05-14T16:58:55.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.854+0000 7f943e004740 1 -- 172.21.15.2:0/2484923636 shutdown_connections 2024-05-14T16:58:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.854+0000 7f943e004740 1 --2- 172.21.15.2:0/2484923636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a7f113ad0 0x561a7f115ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:56.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.854+0000 7f943e004740 1 --2- 172.21.15.2:0/2484923636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a7f109210 0x561a7f113590 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.854+0000 7f943e004740 1 --2- 172.21.15.2:0/2484923636 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561a7f1088f0 0x561a7f108cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.854+0000 7f943e004740 1 -- 172.21.15.2:0/2484923636 >> 172.21.15.2:0/2484923636 conn(0x561a7f104e00 msgr2=0x561a7f1051e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.854+0000 7f943e004740 1 -- 172.21.15.2:0/2484923636 shutdown_connections 2024-05-14T16:58:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.854+0000 7f943e004740 1 -- 172.21.15.2:0/2484923636 wait complete. 2024-05-14T16:58:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.855+0000 7f943987e640 1 --2- 172.21.15.2:0/3300134934 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f94200b2c00 0x7f94200b50c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f942c005fd0 tx=0x7f942c009500 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:56.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.855+0000 7f943e004740 1 -- start start 2024-05-14T16:58:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.855+0000 7f943e004740 1 Processor -- start 2024-05-14T16:58:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.855+0000 7f943e004740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a7f1088f0 0x561a7f113400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.855+0000 7f943e004740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561a7f109210 0x561a7f1106f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.855+0000 7f943e004740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a7f113ad0 0x561a7f110c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.855+0000 7f943e004740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x561a7ed51790 con 0x561a7f109210 2024-05-14T16:58:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.855+0000 7f943e004740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561a7f027890 con 0x561a7f113ad0 2024-05-14T16:58:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.855+0000 7f943e004740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561a7f1167a0 con 0x561a7f1088f0 2024-05-14T16:58:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.855+0000 7f943e004740 10 client.0 _ll_register_callbacks cb 0x561a7f02ad60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:58:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.855+0000 7f943907d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a7f1088f0 0x561a7f113400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.855+0000 7f943907d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a7f1088f0 0x561a7f113400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:53894/0 (socket says 172.21.15.2:53894) 2024-05-14T16:58:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.855+0000 7f943907d640 1 -- 172.21.15.2:0/286949046 learned_addr learned my addr 172.21.15.2:0/286949046 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:58:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.855+0000 7f943887c640 1 --2- 172.21.15.2:0/286949046 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561a7f109210 0x561a7f1106f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.855+0000 7f943987e640 1 --2- 172.21.15.2:0/286949046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a7f113ad0 0x561a7f110c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:56.252 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.856+0000 7f943907d640 1 -- 172.21.15.2:0/286949046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a7f113ad0 msgr2=0x561a7f110c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.856+0000 7f943907d640 1 --2- 172.21.15.2:0/286949046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a7f113ad0 0x561a7f110c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:56.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.856+0000 7f943907d640 1 -- 172.21.15.2:0/286949046 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561a7f109210 msgr2=0x561a7f1106f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:56.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.856+0000 7f943907d640 1 --2- 172.21.15.2:0/286949046 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561a7f109210 0x561a7f1106f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:56.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.856+0000 7f943907d640 1 -- 172.21.15.2:0/286949046 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a7ee4c620 con 0x561a7f1088f0 2024-05-14T16:58:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.856+0000 7f943907d640 1 --2- 172.21.15.2:0/286949046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a7f1088f0 0x561a7f113400 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f943001b040 tx=0x7f9430005450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7334202179842c99 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.856+0000 7f941cff9640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-14T16:58:56.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.856+0000 7f941cff9640 1 -- 172.21.15.2:0/286949046 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9430028070 con 0x561a7f1088f0 2024-05-14T16:58:56.334 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.856+0000 7f941cff9640 1 -- 172.21.15.2:0/286949046 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f94300046c0 con 0x561a7f1088f0 2024-05-14T16:58:56.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.857+0000 7f943e004740 1 -- 172.21.15.2:0/286949046 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94300054d0 con 0x561a7f1088f0 2024-05-14T16:58:56.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.857+0000 7f941cff9640 1 -- 172.21.15.2:0/286949046 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9430034050 con 0x561a7f1088f0 2024-05-14T16:58:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.857+0000 7f943e004740 10 client.25462 fetch_fsmap learned FSMap version 36 2024-05-14T16:58:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.857+0000 7f943e004740 1 -- 172.21.15.2:0/286949046 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561a7f1221e0 con 0x561a7f1088f0 2024-05-14T16:58:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.858+0000 7f941cff9640 1 -- 172.21.15.2:0/286949046 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94300054d0 con 0x561a7f1088f0 2024-05-14T16:58:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.858+0000 7f941cff9640 1 -- 172.21.15.2:0/286949046 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 36) ==== 8342+0+0 (secure 0 0 0) 0x7f9430022570 con 0x561a7f1088f0 2024-05-14T16:58:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.858+0000 7f943e004740 10 client.25462 fetch_fsmap finished waiting for FSMap version 36 2024-05-14T16:58:56.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.858+0000 7f943e004740 10 client.25462 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:58:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.858+0000 7f943e004740 20 client.25462 populate_metadata read hostname 'smithi002' 2024-05-14T16:58:56.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.858+0000 7f943e004740 1 --2- 172.21.15.2:0/286949046 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x561a7f1235f0 0x561a7f1259e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:56.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.858+0000 7f943e004740 4 client.25462 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:58:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.858+0000 7f943e004740 1 -- 172.21.15.2:0/286949046 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a7f07db80 con 0x561a7f1235f0 2024-05-14T16:58:56.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.859+0000 7f943887c640 1 --2- 172.21.15.2:0/286949046 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x561a7f1235f0 0x561a7f1259e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.860+0000 7f943887c640 1 --2- 172.21.15.2:0/286949046 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x561a7f1235f0 0x561a7f1259e0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=129ad4681b72f2cc server_cookie=796259a06a4f5f3 in_seq=0 out_seq=0 2024-05-14T16:58:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.860+0000 7f941cff9640 10 client.25462 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:58:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.860+0000 7f941cff9640 1 -- 172.21.15.2:0/286949046 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561a7f07db80 con 0x561a7f1235f0 2024-05-14T16:58:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.860+0000 7f941cff9640 10 client.25462 handle_command_reply: tid=0 2024-05-14T16:58:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.860+0000 7f943e004740 1 client.25462 shutdown 2024-05-14T16:58:56.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.861+0000 7f943e004740 1 -- 172.21.15.2:0/286949046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a7f1088f0 msgr2=0x561a7f113400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.861+0000 7f943e004740 1 --2- 172.21.15.2:0/286949046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a7f1088f0 0x561a7f113400 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f943001b040 tx=0x7f9430005450 comp rx=0 tx=0).stop 2024-05-14T16:58:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.861+0000 7f943e004740 1 -- 172.21.15.2:0/286949046 shutdown_connections 2024-05-14T16:58:56.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.861+0000 7f943e004740 1 --2- 172.21.15.2:0/286949046 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x561a7f1235f0 0x561a7f1259e0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.861+0000 7f943e004740 1 --2- 172.21.15.2:0/286949046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a7f113ad0 0x561a7f110c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:56.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.861+0000 7f943e004740 1 --2- 172.21.15.2:0/286949046 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561a7f109210 0x561a7f1106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:56.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.861+0000 7f943e004740 1 --2- 172.21.15.2:0/286949046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a7f1088f0 0x561a7f113400 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.861+0000 7f941cff9640 0 client.25462 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:58:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.861+0000 7f943e004740 1 -- 172.21.15.2:0/286949046 >> 172.21.15.2:0/286949046 conn(0x561a7f104e00 msgr2=0x561a7f104100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.861+0000 7f943e004740 1 -- 172.21.15.2:0/286949046 shutdown_connections 2024-05-14T16:58:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.861+0000 7f943e004740 1 -- 172.21.15.2:0/286949046 wait complete. 2024-05-14T16:58:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.861+0000 7f943e004740 20 client.25462 trim_cache size 0 max 16384 2024-05-14T16:58:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.887+0000 7f943e004740 1 Processor -- start 2024-05-14T16:58:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.887+0000 7f943e004740 1 -- start start 2024-05-14T16:58:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.888+0000 7f943e004740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a7f1088f0 0x561a7f10bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:56.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.888+0000 7f943e004740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a7f109210 0x561a7f103f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.888+0000 7f943e004740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561a7f113ad0 0x561a7f112f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.888+0000 7f943e004740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f94341b7670 con 0x561a7f113ad0 2024-05-14T16:58:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.888+0000 7f943e004740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f94341b4f40 con 0x561a7f1088f0 2024-05-14T16:58:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.888+0000 7f943e004740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f94341b2d90 con 0x561a7f109210 2024-05-14T16:58:56.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.888+0000 7f943907d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a7f1088f0 0x561a7f10bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:56.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.888+0000 7f943907d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a7f1088f0 0x561a7f10bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:53442/0 (socket says 172.21.15.2:53442) 2024-05-14T16:58:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.888+0000 7f943907d640 1 -- 172.21.15.2:0/1193871010 learned_addr learned my addr 172.21.15.2:0/1193871010 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:58:56.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.888+0000 7f943887c640 1 --2- 172.21.15.2:0/1193871010 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a7f109210 0x561a7f103f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.888+0000 7f943987e640 1 --2- 172.21.15.2:0/1193871010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561a7f113ad0 0x561a7f112f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.888+0000 7f943907d640 1 -- 172.21.15.2:0/1193871010 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a7f109210 msgr2=0x561a7f103f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:56.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.888+0000 7f943907d640 1 --2- 172.21.15.2:0/1193871010 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a7f109210 0x561a7f103f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.889+0000 7f943907d640 1 -- 172.21.15.2:0/1193871010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561a7f113ad0 msgr2=0x561a7f112f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.889+0000 7f943907d640 1 --2- 172.21.15.2:0/1193871010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561a7f113ad0 0x561a7f112f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.889+0000 7f943907d640 1 -- 172.21.15.2:0/1193871010 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a7f109040 con 0x561a7f1088f0 2024-05-14T16:58:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.889+0000 7f943907d640 1 --2- 172.21.15.2:0/1193871010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a7f1088f0 0x561a7f10bdc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9430009c00 tx=0x7f9430009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8942518c1ce88ece server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.889+0000 7f941effd640 1 -- 172.21.15.2:0/1193871010 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9430028070 con 0x561a7f1088f0 2024-05-14T16:58:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.889+0000 7f941effd640 1 -- 172.21.15.2:0/1193871010 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9430004ad0 con 0x561a7f1088f0 2024-05-14T16:58:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.889+0000 7f941effd640 1 -- 172.21.15.2:0/1193871010 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9430004510 con 0x561a7f1088f0 2024-05-14T16:58:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.890+0000 7f943e004740 1 -- 172.21.15.2:0/1193871010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a7f1088f0 msgr2=0x561a7f10bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.890+0000 7f943e004740 1 --2- 172.21.15.2:0/1193871010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a7f1088f0 0x561a7f10bdc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9430009c00 tx=0x7f9430009780 comp rx=0 tx=0).stop 2024-05-14T16:58:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.890+0000 7f943e004740 1 -- 172.21.15.2:0/1193871010 shutdown_connections 2024-05-14T16:58:57.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.890+0000 7f943e004740 1 --2- 172.21.15.2:0/1193871010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561a7f113ad0 0x561a7f112f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.890+0000 7f943e004740 1 --2- 172.21.15.2:0/1193871010 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a7f109210 0x561a7f103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:57.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.890+0000 7f943e004740 1 --2- 172.21.15.2:0/1193871010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a7f1088f0 0x561a7f10bdc0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.890+0000 7f943e004740 1 -- 172.21.15.2:0/1193871010 >> 172.21.15.2:0/1193871010 conn(0x561a7f104e00 msgr2=0x561a7f103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:57.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.890+0000 7f943e004740 1 -- 172.21.15.2:0/1193871010 shutdown_connections 2024-05-14T16:58:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.890+0000 7f943e004740 1 -- 172.21.15.2:0/1193871010 wait complete. 2024-05-14T16:58:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.891+0000 7f943e004740 1 -- start start 2024-05-14T16:58:57.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.891+0000 7f943e004740 1 Processor -- start 2024-05-14T16:58:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.891+0000 7f943e004740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561a7f1088f0 0x561a7f11bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.891+0000 7f943e004740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a7f109210 0x561a7f11c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.891+0000 7f943e004740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a7f113ad0 0x561a7f117170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.891+0000 7f943e004740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f94341b3400 con 0x561a7f1088f0 2024-05-14T16:58:57.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.891+0000 7f943e004740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f94341b4870 con 0x561a7f113ad0 2024-05-14T16:58:57.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.891+0000 7f943e004740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f94341b3ad0 con 0x561a7f109210 2024-05-14T16:58:57.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.891+0000 7f943e004740 10 client.0 _ll_register_callbacks cb 0x7f9430005680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:58:57.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.891+0000 7f943907d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561a7f1088f0 0x561a7f11bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:57.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.891+0000 7f943887c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a7f109210 0x561a7f11c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:57.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.891+0000 7f943907d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561a7f1088f0 0x561a7f11bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42030/0 (socket says 172.21.15.2:42030) 2024-05-14T16:58:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.891+0000 7f943907d640 1 -- 172.21.15.2:0/3562995558 learned_addr learned my addr 172.21.15.2:0/3562995558 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:58:57.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.891+0000 7f943987e640 1 --2- 172.21.15.2:0/3562995558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a7f113ad0 0x561a7f117170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:57.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.892+0000 7f943887c640 1 -- 172.21.15.2:0/3562995558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a7f113ad0 msgr2=0x561a7f117170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.892+0000 7f943887c640 1 --2- 172.21.15.2:0/3562995558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a7f113ad0 0x561a7f117170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.892+0000 7f943887c640 1 -- 172.21.15.2:0/3562995558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561a7f1088f0 msgr2=0x561a7f11bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:58:57.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.892+0000 7f943887c640 1 --2- 172.21.15.2:0/3562995558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561a7f1088f0 0x561a7f11bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:57.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.892+0000 7f943887c640 1 -- 172.21.15.2:0/3562995558 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a7f117970 con 0x561a7f109210 2024-05-14T16:58:57.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.892+0000 7f943907d640 1 --2- 172.21.15.2:0/3562995558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561a7f1088f0 0x561a7f11bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:58:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.892+0000 7f943987e640 1 --2- 172.21.15.2:0/3562995558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a7f113ad0 0x561a7f117170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:58:57.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.892+0000 7f941cff9640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-14T16:58:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.892+0000 7f943887c640 1 --2- 172.21.15.2:0/3562995558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a7f109210 0x561a7f11c0a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f94240ffc90 tx=0x7f94240d2040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e5ccc27234c4f40 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:58:57.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.892+0000 7f941cff9640 1 -- 172.21.15.2:0/3562995558 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9424035040 con 0x561a7f109210 2024-05-14T16:58:57.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.893+0000 7f941cff9640 1 -- 172.21.15.2:0/3562995558 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f94240c5e10 con 0x561a7f109210 2024-05-14T16:58:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.893+0000 7f943e004740 1 -- 172.21.15.2:0/3562995558 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9430004ad0 con 0x561a7f109210 2024-05-14T16:58:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.893+0000 7f941cff9640 1 -- 172.21.15.2:0/3562995558 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9424035040 con 0x561a7f109210 2024-05-14T16:58:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.893+0000 7f943e004740 10 client.25468 fetch_fsmap learned FSMap version 36 2024-05-14T16:58:57.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.893+0000 7f943e004740 1 -- 172.21.15.2:0/3562995558 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561a7f18d9c0 con 0x561a7f109210 2024-05-14T16:58:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.894+0000 7f941cff9640 1 -- 172.21.15.2:0/3562995558 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9430004ad0 con 0x561a7f109210 2024-05-14T16:58:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.894+0000 7f941cff9640 1 -- 172.21.15.2:0/3562995558 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 36) ==== 8342+0+0 (secure 0 0 0) 0x7f94240d21f0 con 0x561a7f109210 2024-05-14T16:58:57.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.894+0000 7f943e004740 10 client.25468 fetch_fsmap finished waiting for FSMap version 36 2024-05-14T16:58:57.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.894+0000 7f943e004740 10 client.25468 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:58:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.894+0000 7f943e004740 20 client.25468 populate_metadata read hostname 'smithi002' 2024-05-14T16:58:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.894+0000 7f943e004740 1 --2- 172.21.15.2:0/3562995558 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x561a7f18e120 0x561a7f190540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:58:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.894+0000 7f943e004740 4 client.25468 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:58:57.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.894+0000 7f943e004740 1 -- 172.21.15.2:0/3562995558 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561a7f077360 con 0x561a7f18e120 2024-05-14T16:58:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.894+0000 7f943907d640 1 --2- 172.21.15.2:0/3562995558 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x561a7f18e120 0x561a7f190540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:58:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.896+0000 7f943907d640 1 --2- 172.21.15.2:0/3562995558 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x561a7f18e120 0x561a7f190540 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c36a2087cb684bb server_cookie=26b6ad11ab5ea691 in_seq=0 out_seq=0 2024-05-14T16:58:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.896+0000 7f941cff9640 10 client.25468 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:58:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.908+0000 7f941cff9640 1 -- 172.21.15.2:0/3562995558 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561a7f077360 con 0x561a7f18e120 2024-05-14T16:58:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.908+0000 7f941cff9640 10 client.25468 handle_command_reply: tid=0 2024-05-14T16:58:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.908+0000 7f943e004740 1 client.25468 shutdown 2024-05-14T16:58:57.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.909+0000 7f943e004740 1 -- 172.21.15.2:0/3562995558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a7f109210 msgr2=0x561a7f11c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:57.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.909+0000 7f943e004740 1 --2- 172.21.15.2:0/3562995558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a7f109210 0x561a7f11c0a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f94240ffc90 tx=0x7f94240d2040 comp rx=0 tx=0).stop 2024-05-14T16:58:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.909+0000 7f943e004740 1 -- 172.21.15.2:0/3562995558 shutdown_connections 2024-05-14T16:58:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.909+0000 7f943e004740 1 --2- 172.21.15.2:0/3562995558 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x561a7f18e120 0x561a7f190540 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:57.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.909+0000 7f943e004740 1 --2- 172.21.15.2:0/3562995558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a7f113ad0 0x561a7f117170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:57.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.909+0000 7f943e004740 1 --2- 172.21.15.2:0/3562995558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a7f109210 0x561a7f11c0a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:57.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.909+0000 7f943e004740 1 --2- 172.21.15.2:0/3562995558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561a7f1088f0 0x561a7f11bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:57.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.909+0000 7f943e004740 1 -- 172.21.15.2:0/3562995558 >> 172.21.15.2:0/3562995558 conn(0x561a7f104e00 msgr2=0x561a7f10b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:57.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.909+0000 7f941cff9640 0 client.25468 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:58:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.910+0000 7f943e004740 1 -- 172.21.15.2:0/3562995558 shutdown_connections 2024-05-14T16:58:57.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.910+0000 7f943e004740 1 -- 172.21.15.2:0/3562995558 wait complete. 2024-05-14T16:58:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.910+0000 7f943e004740 20 client.25468 trim_cache size 0 max 16384 2024-05-14T16:58:57.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T16:58:57.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T16:58:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T16:58:57.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.911+0000 7f943b2e1640 1 -- 172.21.15.2:0/3300134934 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f94200b2c00 msgr2=0x7f94200b50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.911+0000 7f943b2e1640 1 --2- 172.21.15.2:0/3300134934 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f94200b2c00 0x7f94200b50c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f942c005fd0 tx=0x7f942c009500 comp rx=0 tx=0).stop 2024-05-14T16:58:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.911+0000 7f943b2e1640 1 -- 172.21.15.2:0/3300134934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94340778a0 msgr2=0x7f94341a5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:58:57.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.911+0000 7f943b2e1640 1 --2- 172.21.15.2:0/3300134934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94340778a0 0x7f94341a5b20 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9424005940 tx=0x7f94240040c0 comp rx=0 tx=0).stop 2024-05-14T16:58:58.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.911+0000 7f943b2e1640 1 -- 172.21.15.2:0/3300134934 shutdown_connections 2024-05-14T16:58:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.911+0000 7f943b2e1640 1 --2- 172.21.15.2:0/3300134934 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f94200b2c00 0x7f94200b50c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:58.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.911+0000 7f943b2e1640 1 --2- 172.21.15.2:0/3300134934 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94341138f0 0x7f943419f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:58.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.911+0000 7f943b2e1640 1 --2- 172.21.15.2:0/3300134934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94340778a0 0x7f94341a5b20 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:58.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.911+0000 7f943b2e1640 1 --2- 172.21.15.2:0/3300134934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9434076f80 0x7f94341a55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:58:58.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.911+0000 7f943b2e1640 1 -- 172.21.15.2:0/3300134934 >> 172.21.15.2:0/3300134934 conn(0x7f9434100780 msgr2=0x7f9434102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:58:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.912+0000 7f943b2e1640 1 -- 172.21.15.2:0/3300134934 shutdown_connections 2024-05-14T16:58:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:58:54.912+0000 7f943b2e1640 1 -- 172.21.15.2:0/3300134934 wait complete. 2024-05-14T16:58:58.159 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:58:56 smithi002 ceph-mon[31538]: pgmap v130: 97 pgs: 97 active+clean; 2.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.4 MiB/s wr, 619 op/s 2024-05-14T16:58:58.168 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 220.0 s, 3496.0 tps, lat 28.719 ms stddev 29.462 2024-05-14T16:58:58.178 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:58:56 smithi123 ceph-mon[35971]: pgmap v130: 97 pgs: 97 active+clean; 2.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.4 MiB/s wr, 619 op/s 2024-05-14T16:58:58.187 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:58:56 smithi151 ceph-mon[36066]: pgmap v130: 97 pgs: 97 active+clean; 2.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.4 MiB/s wr, 619 op/s 2024-05-14T16:58:58.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:58:58 smithi151 ceph-mon[36066]: pgmap v131: 97 pgs: 97 active+clean; 2.2 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.8 MiB/s wr, 759 op/s 2024-05-14T16:58:58.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:58:58 smithi123 ceph-mon[35971]: pgmap v131: 97 pgs: 97 active+clean; 2.2 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.8 MiB/s wr, 759 op/s 2024-05-14T16:58:58.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:58:58 smithi002 ceph-mon[31538]: pgmap v131: 97 pgs: 97 active+clean; 2.2 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.8 MiB/s wr, 759 op/s 2024-05-14T16:59:00.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:59:00 smithi151 ceph-mon[36066]: pgmap v132: 97 pgs: 97 active+clean; 2.2 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 10 MiB/s wr, 830 op/s 2024-05-14T16:59:00.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:59:00 smithi123 ceph-mon[35971]: pgmap v132: 97 pgs: 97 active+clean; 2.2 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 10 MiB/s wr, 830 op/s 2024-05-14T16:59:00.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:59:00 smithi002 ceph-mon[31538]: pgmap v132: 97 pgs: 97 active+clean; 2.2 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 10 MiB/s wr, 830 op/s 2024-05-14T16:59:01.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 225.0 s, 2234.6 tps, lat 43.394 ms stddev 70.740 2024-05-14T16:59:02.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:59:02 smithi151 ceph-mon[36066]: pgmap v133: 97 pgs: 97 active+clean; 2.2 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.3 MiB/s wr, 648 op/s 2024-05-14T16:59:02.662 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:59:02 smithi123 ceph-mon[35971]: pgmap v133: 97 pgs: 97 active+clean; 2.2 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.3 MiB/s wr, 648 op/s 2024-05-14T16:59:02.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:59:02 smithi002 ceph-mon[31538]: pgmap v133: 97 pgs: 97 active+clean; 2.2 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.3 MiB/s wr, 648 op/s 2024-05-14T16:59:04.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:59:04 smithi151 ceph-mon[36066]: pgmap v134: 97 pgs: 97 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 13 MiB/s wr, 973 op/s 2024-05-14T16:59:04.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:59:04 smithi123 ceph-mon[35971]: pgmap v134: 97 pgs: 97 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 13 MiB/s wr, 973 op/s 2024-05-14T16:59:04.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:59:04 smithi002 ceph-mon[31538]: pgmap v134: 97 pgs: 97 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 13 MiB/s wr, 973 op/s 2024-05-14T16:59:06.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:59:06 smithi123 ceph-mon[35971]: pgmap v135: 97 pgs: 97 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.5 MiB/s wr, 656 op/s 2024-05-14T16:59:06.799 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:59:06 smithi002 ceph-mon[31538]: pgmap v135: 97 pgs: 97 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.5 MiB/s wr, 656 op/s 2024-05-14T16:59:06.815 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 230.0 s, 328.4 tps, lat 303.476 ms stddev 186.295 2024-05-14T16:59:06.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:59:06 smithi151 ceph-mon[36066]: pgmap v135: 97 pgs: 97 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.5 MiB/s wr, 656 op/s 2024-05-14T16:59:08.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.056+0000 7fb587bc1640 1 -- 172.21.15.2:0/1556068667 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb580109f90 msgr2=0x7fb58010a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.056+0000 7fb587bc1640 1 --2- 172.21.15.2:0/1556068667 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb580109f90 0x7fb58010a3f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb578009ca0 tx=0x7fb57802a8f0 comp rx=0 tx=0).stop 2024-05-14T16:59:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.057+0000 7fb587bc1640 1 -- 172.21.15.2:0/1556068667 shutdown_connections 2024-05-14T16:59:08.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.057+0000 7fb587bc1640 1 --2- 172.21.15.2:0/1556068667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb580117ae0 0x7fb580119ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:08.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.057+0000 7fb587bc1640 1 --2- 172.21.15.2:0/1556068667 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb580109f90 0x7fb58010a3f0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.057+0000 7fb587bc1640 1 --2- 172.21.15.2:0/1556068667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb58010d150 0x7fb580109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:08.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.057+0000 7fb587bc1640 1 -- 172.21.15.2:0/1556068667 >> 172.21.15.2:0/1556068667 conn(0x7fb58006db20 msgr2=0x7fb58006df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.058+0000 7fb587bc1640 1 -- 172.21.15.2:0/1556068667 shutdown_connections 2024-05-14T16:59:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.059+0000 7fb587bc1640 1 -- 172.21.15.2:0/1556068667 wait complete. 2024-05-14T16:59:08.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.061+0000 7fb587bc1640 1 Processor -- start 2024-05-14T16:59:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.061+0000 7fb587bc1640 1 -- start start 2024-05-14T16:59:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.061+0000 7fb587bc1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb580109f90 0x7fb5801172b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.061+0000 7fb587bc1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb58010d150 0x7fb580114350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.062+0000 7fb587bc1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb580117ae0 0x7fb580114890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.062+0000 7fb58595d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb580109f90 0x7fb5801172b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.062+0000 7fb58595d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb580109f90 0x7fb5801172b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:50304/0 (socket says 172.21.15.2:50304) 2024-05-14T16:59:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.062+0000 7fb58615e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb580117ae0 0x7fb580114890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:08.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.062+0000 7fb58615e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb580117ae0 0x7fb580114890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:49346/0 (socket says 172.21.15.2:49346) 2024-05-14T16:59:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.062+0000 7fb58615e640 1 -- 172.21.15.2:0/4209356402 learned_addr learned my addr 172.21.15.2:0/4209356402 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:59:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.062+0000 7fb587bc1640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb58011d830 con 0x7fb58010d150 2024-05-14T16:59:08.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.062+0000 7fb587bc1640 1 -- 172.21.15.2:0/4209356402 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb58011d6b0 con 0x7fb580109f90 2024-05-14T16:59:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.062+0000 7fb587bc1640 1 -- 172.21.15.2:0/4209356402 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb58011d9b0 con 0x7fb580117ae0 2024-05-14T16:59:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.064+0000 7fb58615e640 1 -- 172.21.15.2:0/4209356402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb580109f90 msgr2=0x7fb5801172b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:08.397 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.064+0000 7fb58615e640 1 --2- 172.21.15.2:0/4209356402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb580109f90 0x7fb5801172b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:08.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.064+0000 7fb58615e640 1 -- 172.21.15.2:0/4209356402 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb58010d150 msgr2=0x7fb580114350 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T16:59:08.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.064+0000 7fb58615e640 1 --2- 172.21.15.2:0/4209356402 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb58010d150 0x7fb580114350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:08.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.064+0000 7fb58615e640 1 -- 172.21.15.2:0/4209356402 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb580115120 con 0x7fb580117ae0 2024-05-14T16:59:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.067+0000 7fb58615e640 1 --2- 172.21.15.2:0/4209356402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb580117ae0 0x7fb580114890 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb57000b3c0 tx=0x7fb57000b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.068+0000 7fb576ffd640 1 -- 172.21.15.2:0/4209356402 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb570011830 con 0x7fb580117ae0 2024-05-14T16:59:08.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.072+0000 7fb587bc1640 1 -- 172.21.15.2:0/4209356402 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5801b4570 con 0x7fb580117ae0 2024-05-14T16:59:08.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.072+0000 7fb587bc1640 1 -- 172.21.15.2:0/4209356402 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5801b4ac0 con 0x7fb580117ae0 2024-05-14T16:59:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.073+0000 7fb576ffd640 1 -- 172.21.15.2:0/4209356402 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb570016070 con 0x7fb580117ae0 2024-05-14T16:59:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.075+0000 7fb576ffd640 1 -- 172.21.15.2:0/4209356402 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb570010de0 con 0x7fb580117ae0 2024-05-14T16:59:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.079+0000 7fb58a9ba740 1 Processor -- start 2024-05-14T16:59:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.079+0000 7fb576ffd640 1 -- 172.21.15.2:0/4209356402 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fb57001b4c0 con 0x7fb580117ae0 2024-05-14T16:59:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.079+0000 7fb58a9ba740 1 -- start start 2024-05-14T16:59:08.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.080+0000 7fb58a9ba740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d8b1898f0 0x564d8b189cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:08.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.080+0000 7fb58a9ba740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d8b18a210 0x564d8b194590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.080+0000 7fb58a9ba740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d8b194ad0 0x564d8b196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.080+0000 7fb58a9ba740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x564d8b0fcc30 con 0x564d8b1898f0 2024-05-14T16:59:08.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.080+0000 7fb58515c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d8b1898f0 0x564d8b189cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.080+0000 7fb58515c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d8b1898f0 0x564d8b189cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35708/0 (socket says 172.21.15.2:35708) 2024-05-14T16:59:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.080+0000 7fb58515c640 1 -- 172.21.15.2:0/2022141175 learned_addr learned my addr 172.21.15.2:0/2022141175 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:59:08.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.080+0000 7fb58a9ba740 1 -- 172.21.15.2:0/2022141175 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564d8b0ec7d0 con 0x564d8b18a210 2024-05-14T16:59:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.080+0000 7fb58a9ba740 1 -- 172.21.15.2:0/2022141175 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564d8b0f0810 con 0x564d8b194ad0 2024-05-14T16:59:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.082+0000 7fb58595d640 1 --2- 172.21.15.2:0/2022141175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d8b18a210 0x564d8b194590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:08.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.082+0000 7fb576ffd640 1 --2- 172.21.15.2:0/4209356402 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb56c0b2ae0 0x7fb56c0b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:08.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.083+0000 7fb576ffd640 1 -- 172.21.15.2:0/4209356402 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fb5700e1ab0 con 0x7fb580117ae0 2024-05-14T16:59:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.083+0000 7fb58615e640 1 --2- 172.21.15.2:0/4209356402 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb56c0b2ae0 0x7fb56c0b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.083+0000 7fb58515c640 1 --2- 172.21.15.2:0/2022141175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d8b194ad0 0x564d8b196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.083+0000 7fb58515c640 1 -- 172.21.15.2:0/2022141175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d8b194ad0 msgr2=0x564d8b196ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:08.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.083+0000 7fb58515c640 1 --2- 172.21.15.2:0/2022141175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d8b194ad0 0x564d8b196ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:08.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.083+0000 7fb58515c640 1 -- 172.21.15.2:0/2022141175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d8b18a210 msgr2=0x564d8b194590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:08.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.083+0000 7fb58515c640 1 --2- 172.21.15.2:0/2022141175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d8b18a210 0x564d8b194590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.083+0000 7fb58515c640 1 -- 172.21.15.2:0/2022141175 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d8ae01210 con 0x564d8b1898f0 2024-05-14T16:59:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.085+0000 7fb58615e640 1 --2- 172.21.15.2:0/4209356402 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb56c0b2ae0 0x7fb56c0b4fa0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb5700a8bd0 tx=0x7fb57000bf30 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.086+0000 7fb58515c640 1 --2- 172.21.15.2:0/2022141175 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d8b1898f0 0x564d8b189cd0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb578035000 tx=0x7fb5780095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ca72d9ed4c134e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.086+0000 7fb56bfff640 1 -- 172.21.15.2:0/2022141175 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb578032e10 con 0x564d8b1898f0 2024-05-14T16:59:08.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.089+0000 7fb56bfff640 1 -- 172.21.15.2:0/2022141175 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb578045020 con 0x564d8b1898f0 2024-05-14T16:59:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.089+0000 7fb56bfff640 1 -- 172.21.15.2:0/2022141175 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb578004390 con 0x564d8b1898f0 2024-05-14T16:59:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.090+0000 7fb58a9ba740 1 -- 172.21.15.2:0/2022141175 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d8b1898f0 msgr2=0x564d8b189cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.090+0000 7fb58a9ba740 1 --2- 172.21.15.2:0/2022141175 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d8b1898f0 0x564d8b189cd0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb578035000 tx=0x7fb5780095a0 comp rx=0 tx=0).stop 2024-05-14T16:59:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.092+0000 7fb58a9ba740 1 -- 172.21.15.2:0/2022141175 shutdown_connections 2024-05-14T16:59:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.092+0000 7fb58a9ba740 1 --2- 172.21.15.2:0/2022141175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d8b194ad0 0x564d8b196ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.092+0000 7fb58a9ba740 1 --2- 172.21.15.2:0/2022141175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d8b18a210 0x564d8b194590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:08.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.092+0000 7fb58a9ba740 1 --2- 172.21.15.2:0/2022141175 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d8b1898f0 0x564d8b189cd0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.092+0000 7fb58a9ba740 1 -- 172.21.15.2:0/2022141175 >> 172.21.15.2:0/2022141175 conn(0x564d8b185e00 msgr2=0x564d8b1861e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:08.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.094+0000 7fb58a9ba740 1 -- 172.21.15.2:0/2022141175 shutdown_connections 2024-05-14T16:59:08.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.094+0000 7fb58a9ba740 1 -- 172.21.15.2:0/2022141175 wait complete. 2024-05-14T16:59:08.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.094+0000 7fb58a9ba740 1 -- start start 2024-05-14T16:59:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.095+0000 7fb58a9ba740 1 Processor -- start 2024-05-14T16:59:08.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.095+0000 7fb58a9ba740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d8b1898f0 0x564d8b194460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:08.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.096+0000 7fb58a9ba740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d8b18a210 0x564d8b191750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.096+0000 7fb58a9ba740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d8b194ad0 0x564d8b191cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.096+0000 7fb58a9ba740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x564d8add2790 con 0x564d8b1898f0 2024-05-14T16:59:08.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.096+0000 7fb58a9ba740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564d8b0a8890 con 0x564d8b194ad0 2024-05-14T16:59:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.096+0000 7fb58a9ba740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564d8b197740 con 0x564d8b18a210 2024-05-14T16:59:09.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.096+0000 7fb58a9ba740 10 client.0 _ll_register_callbacks cb 0x564d8b0abd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:59:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.097+0000 7fb58595d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d8b18a210 0x564d8b191750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.097+0000 7fb58595d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d8b18a210 0x564d8b191750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:49364/0 (socket says 172.21.15.2:49364) 2024-05-14T16:59:09.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.097+0000 7fb58595d640 1 -- 172.21.15.2:0/3277245023 learned_addr learned my addr 172.21.15.2:0/3277245023 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:59:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.099+0000 7fb58595d640 1 -- 172.21.15.2:0/3277245023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d8b194ad0 msgr2=0x564d8b191cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T16:59:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.099+0000 7fb58595d640 1 --2- 172.21.15.2:0/3277245023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d8b194ad0 0x564d8b191cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.099+0000 7fb58595d640 1 -- 172.21.15.2:0/3277245023 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d8b1898f0 msgr2=0x564d8b194460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:09.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.099+0000 7fb58595d640 1 --2- 172.21.15.2:0/3277245023 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d8b1898f0 0x564d8b194460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.100+0000 7fb58595d640 1 -- 172.21.15.2:0/3277245023 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d8aecd620 con 0x564d8b18a210 2024-05-14T16:59:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.100+0000 7fb58595d640 1 --2- 172.21.15.2:0/3277245023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d8b18a210 0x564d8b191750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00bfb0 tx=0x7fb57c00f670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d350bd6de649750b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.100+0000 7fb5697fa640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-14T16:59:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.102+0000 7fb5697fa640 1 -- 172.21.15.2:0/3277245023 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c018020 con 0x564d8b18a210 2024-05-14T16:59:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.103+0000 7fb58a9ba740 1 -- 172.21.15.2:0/3277245023 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb578045020 con 0x564d8b18a210 2024-05-14T16:59:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.103+0000 7fb5697fa640 1 -- 172.21.15.2:0/3277245023 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fb57c014870 con 0x564d8b18a210 2024-05-14T16:59:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.103+0000 7fb5697fa640 1 -- 172.21.15.2:0/3277245023 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb57c023050 con 0x564d8b18a210 2024-05-14T16:59:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.105+0000 7fb58a9ba740 10 client.25477 fetch_fsmap learned FSMap version 36 2024-05-14T16:59:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.105+0000 7fb58a9ba740 1 -- 172.21.15.2:0/3277245023 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564d8b1a30a0 con 0x564d8b18a210 2024-05-14T16:59:09.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.106+0000 7fb5697fa640 1 -- 172.21.15.2:0/3277245023 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb578045020 con 0x564d8b18a210 2024-05-14T16:59:09.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.107+0000 7fb5697fa640 1 -- 172.21.15.2:0/3277245023 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 36) ==== 8342+0+0 (secure 0 0 0) 0x7fb57c013ad0 con 0x564d8b18a210 2024-05-14T16:59:09.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.108+0000 7fb58a9ba740 10 client.25477 fetch_fsmap finished waiting for FSMap version 36 2024-05-14T16:59:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.108+0000 7fb58a9ba740 10 client.25477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:59:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.108+0000 7fb58a9ba740 20 client.25477 populate_metadata read hostname 'smithi002' 2024-05-14T16:59:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.108+0000 7fb58a9ba740 1 --2- 172.21.15.2:0/3277245023 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564d8b1a4620 0x564d8b1a6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.108+0000 7fb58a9ba740 4 client.25477 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:59:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.108+0000 7fb58a9ba740 1 -- 172.21.15.2:0/3277245023 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d8b0feb80 con 0x564d8b1a4620 2024-05-14T16:59:09.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.115+0000 7fb58515c640 1 --2- 172.21.15.2:0/3277245023 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564d8b1a4620 0x564d8b1a6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.121+0000 7fb5697fa640 10 client.25477 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:59:09.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.121+0000 7fb58515c640 1 --2- 172.21.15.2:0/3277245023 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564d8b1a4620 0x564d8b1a6ac0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4cce5b17cc6f00b server_cookie=d29b31e84b69cf3a in_seq=0 out_seq=0 2024-05-14T16:59:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.122+0000 7fb5697fa640 1 -- 172.21.15.2:0/3277245023 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564d8b0feb80 con 0x564d8b1a4620 2024-05-14T16:59:09.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.122+0000 7fb5697fa640 10 client.25477 handle_command_reply: tid=0 2024-05-14T16:59:09.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.122+0000 7fb58a9ba740 1 client.25477 shutdown 2024-05-14T16:59:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.123+0000 7fb58a9ba740 1 -- 172.21.15.2:0/3277245023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d8b18a210 msgr2=0x564d8b191750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.123+0000 7fb58a9ba740 1 --2- 172.21.15.2:0/3277245023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d8b18a210 0x564d8b191750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00bfb0 tx=0x7fb57c00f670 comp rx=0 tx=0).stop 2024-05-14T16:59:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.123+0000 7fb58a9ba740 1 -- 172.21.15.2:0/3277245023 shutdown_connections 2024-05-14T16:59:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.123+0000 7fb58a9ba740 1 --2- 172.21.15.2:0/3277245023 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564d8b1a4620 0x564d8b1a6ac0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.123+0000 7fb58a9ba740 1 --2- 172.21.15.2:0/3277245023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d8b194ad0 0x564d8b191cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.123+0000 7fb58a9ba740 1 --2- 172.21.15.2:0/3277245023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d8b18a210 0x564d8b191750 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:09.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.123+0000 7fb58a9ba740 1 --2- 172.21.15.2:0/3277245023 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d8b1898f0 0x564d8b194460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.123+0000 7fb58a9ba740 1 -- 172.21.15.2:0/3277245023 >> 172.21.15.2:0/3277245023 conn(0x564d8b185e00 msgr2=0x564d8b185100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:09.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.123+0000 7fb5697fa640 0 client.25477 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:59:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.124+0000 7fb58a9ba740 1 -- 172.21.15.2:0/3277245023 shutdown_connections 2024-05-14T16:59:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.124+0000 7fb58a9ba740 1 -- 172.21.15.2:0/3277245023 wait complete. 2024-05-14T16:59:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.125+0000 7fb58a9ba740 20 client.25477 trim_cache size 0 max 16384 2024-05-14T16:59:09.503 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.153+0000 7fb58a9ba740 1 Processor -- start 2024-05-14T16:59:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.153+0000 7fb58a9ba740 1 -- start start 2024-05-14T16:59:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.153+0000 7fb58a9ba740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d8b1898f0 0x564d8b18cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.153+0000 7fb58a9ba740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d8b18a210 0x564d8b1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.153+0000 7fb58a9ba740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d8b194ad0 0x564d8b193f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.153+0000 7fb58a9ba740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb564010f10 con 0x564d8b1898f0 2024-05-14T16:59:09.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.153+0000 7fb58a9ba740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb56400e7e0 con 0x564d8b18a210 2024-05-14T16:59:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.153+0000 7fb58a9ba740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb56400c6e0 con 0x564d8b194ad0 2024-05-14T16:59:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.153+0000 7fb58515c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d8b1898f0 0x564d8b18cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.153+0000 7fb58515c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d8b1898f0 0x564d8b18cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35738/0 (socket says 172.21.15.2:35738) 2024-05-14T16:59:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.153+0000 7fb58515c640 1 -- 172.21.15.2:0/10250879 learned_addr learned my addr 172.21.15.2:0/10250879 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:59:09.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.154+0000 7fb58515c640 1 --2- 172.21.15.2:0/10250879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d8b194ad0 0x564d8b193f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.154+0000 7fb58595d640 1 --2- 172.21.15.2:0/10250879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d8b18a210 0x564d8b1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.154+0000 7fb58515c640 1 -- 172.21.15.2:0/10250879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d8b194ad0 msgr2=0x564d8b193f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.154+0000 7fb58515c640 1 --2- 172.21.15.2:0/10250879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d8b194ad0 0x564d8b193f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.154+0000 7fb58515c640 1 -- 172.21.15.2:0/10250879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d8b18a210 msgr2=0x564d8b1a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:09.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.154+0000 7fb58515c640 1 --2- 172.21.15.2:0/10250879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d8b18a210 0x564d8b1a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.154+0000 7fb58515c640 1 -- 172.21.15.2:0/10250879 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d8b190700 con 0x564d8b1898f0 2024-05-14T16:59:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.154+0000 7fb58515c640 1 --2- 172.21.15.2:0/10250879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d8b1898f0 0x564d8b18cc40 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb578047040 tx=0x7fb578009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b407b5277bf059c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.154+0000 7fb56b7fe640 1 -- 172.21.15.2:0/10250879 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb578058070 con 0x564d8b1898f0 2024-05-14T16:59:09.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.154+0000 7fb56b7fe640 1 -- 172.21.15.2:0/10250879 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb578045040 con 0x564d8b1898f0 2024-05-14T16:59:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.155+0000 7fb56b7fe640 1 -- 172.21.15.2:0/10250879 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57804f070 con 0x564d8b1898f0 2024-05-14T16:59:09.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.155+0000 7fb58a9ba740 1 -- 172.21.15.2:0/10250879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d8b1898f0 msgr2=0x564d8b18cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.155+0000 7fb58a9ba740 1 --2- 172.21.15.2:0/10250879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d8b1898f0 0x564d8b18cc40 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb578047040 tx=0x7fb578009860 comp rx=0 tx=0).stop 2024-05-14T16:59:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.156+0000 7fb58a9ba740 1 -- 172.21.15.2:0/10250879 shutdown_connections 2024-05-14T16:59:09.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.156+0000 7fb58a9ba740 1 --2- 172.21.15.2:0/10250879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d8b194ad0 0x564d8b193f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.156+0000 7fb58a9ba740 1 --2- 172.21.15.2:0/10250879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d8b18a210 0x564d8b1a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.156+0000 7fb58a9ba740 1 --2- 172.21.15.2:0/10250879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d8b1898f0 0x564d8b18cc40 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:09.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.156+0000 7fb58a9ba740 1 -- 172.21.15.2:0/10250879 >> 172.21.15.2:0/10250879 conn(0x564d8b185e00 msgr2=0x564d8b18c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.157+0000 7fb58a9ba740 1 -- 172.21.15.2:0/10250879 shutdown_connections 2024-05-14T16:59:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.157+0000 7fb58a9ba740 1 -- 172.21.15.2:0/10250879 wait complete. 2024-05-14T16:59:09.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.158+0000 7fb58a9ba740 1 -- start start 2024-05-14T16:59:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.158+0000 7fb58a9ba740 1 Processor -- start 2024-05-14T16:59:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.158+0000 7fb58a9ba740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d8b1898f0 0x564d8b197da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.158+0000 7fb58a9ba740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d8b18a210 0x564d8b1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.158+0000 7fb58a9ba740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d8b194ad0 0x564d8b206f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:09.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.158+0000 7fb58a9ba740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb56400cca0 con 0x564d8b194ad0 2024-05-14T16:59:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.158+0000 7fb58a9ba740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb56400e110 con 0x564d8b18a210 2024-05-14T16:59:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.158+0000 7fb58a9ba740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb56400d370 con 0x564d8b1898f0 2024-05-14T16:59:09.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.158+0000 7fb58a9ba740 10 client.0 _ll_register_callbacks cb 0x7fb57c00bb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:59:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.158+0000 7fb58515c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d8b194ad0 0x564d8b206f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:09.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.158+0000 7fb58515c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d8b194ad0 0x564d8b206f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35748/0 (socket says 172.21.15.2:35748) 2024-05-14T16:59:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.159+0000 7fb58515c640 1 -- 172.21.15.2:0/2568273120 learned_addr learned my addr 172.21.15.2:0/2568273120 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:59:09.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.159+0000 7fb58595d640 1 --2- 172.21.15.2:0/2568273120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d8b18a210 0x564d8b1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:09.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.159+0000 7fb58515c640 1 --2- 172.21.15.2:0/2568273120 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d8b1898f0 0x564d8b197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.159+0000 7fb58515c640 1 -- 172.21.15.2:0/2568273120 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d8b1898f0 msgr2=0x564d8b197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:09.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.159+0000 7fb58515c640 1 --2- 172.21.15.2:0/2568273120 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d8b1898f0 0x564d8b197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:09.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.159+0000 7fb58515c640 1 -- 172.21.15.2:0/2568273120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d8b18a210 msgr2=0x564d8b1982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:09.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.159+0000 7fb58515c640 1 --2- 172.21.15.2:0/2568273120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d8b18a210 0x564d8b1982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.159+0000 7fb58515c640 1 -- 172.21.15.2:0/2568273120 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d8b207580 con 0x564d8b194ad0 2024-05-14T16:59:10.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.159+0000 7fb58515c640 1 --2- 172.21.15.2:0/2568273120 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d8b194ad0 0x564d8b206f40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb578056040 tx=0x7fb57802f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e85aec7bb14f4eac server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.159+0000 7fb5697fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:59:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.159+0000 7fb5697fa640 1 -- 172.21.15.2:0/2568273120 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb578058070 con 0x564d8b194ad0 2024-05-14T16:59:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.159+0000 7fb58a9ba740 1 -- 172.21.15.2:0/2568273120 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb578045040 con 0x564d8b194ad0 2024-05-14T16:59:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.159+0000 7fb5697fa640 1 -- 172.21.15.2:0/2568273120 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fb578031680 con 0x564d8b194ad0 2024-05-14T16:59:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.160+0000 7fb5697fa640 1 -- 172.21.15.2:0/2568273120 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb57804f040 con 0x564d8b194ad0 2024-05-14T16:59:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.160+0000 7fb58a9ba740 10 client.15783 fetch_fsmap learned FSMap version 36 2024-05-14T16:59:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.160+0000 7fb58a9ba740 1 -- 172.21.15.2:0/2568273120 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564d8b20e900 con 0x564d8b194ad0 2024-05-14T16:59:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.160+0000 7fb5697fa640 1 -- 172.21.15.2:0/2568273120 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb578045040 con 0x564d8b194ad0 2024-05-14T16:59:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.160+0000 7fb5697fa640 1 -- 172.21.15.2:0/2568273120 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 36) ==== 8342+0+0 (secure 0 0 0) 0x7fb5780305f0 con 0x564d8b194ad0 2024-05-14T16:59:10.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.160+0000 7fb58a9ba740 10 client.15783 fetch_fsmap finished waiting for FSMap version 36 2024-05-14T16:59:10.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.160+0000 7fb58a9ba740 10 client.15783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:59:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.160+0000 7fb58a9ba740 20 client.15783 populate_metadata read hostname 'smithi002' 2024-05-14T16:59:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.160+0000 7fb58a9ba740 1 --2- 172.21.15.2:0/2568273120 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564d8b20eb70 0x564d8b210f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:10.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.160+0000 7fb58a9ba740 4 client.15783 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:59:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.160+0000 7fb58a9ba740 1 -- 172.21.15.2:0/2568273120 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564d8b0f8360 con 0x564d8b20eb70 2024-05-14T16:59:10.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.160+0000 7fb58595d640 1 --2- 172.21.15.2:0/2568273120 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564d8b20eb70 0x564d8b210f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:10.180 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.161+0000 7fb5697fa640 10 client.15783 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:59:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.161+0000 7fb58595d640 1 --2- 172.21.15.2:0/2568273120 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564d8b20eb70 0x564d8b210f60 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e69e6aaa3a6391d5 server_cookie=ab020006749a11fb in_seq=0 out_seq=0 2024-05-14T16:59:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.175+0000 7fb5697fa640 1 -- 172.21.15.2:0/2568273120 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564d8b0f8360 con 0x564d8b20eb70 2024-05-14T16:59:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.175+0000 7fb5697fa640 10 client.15783 handle_command_reply: tid=0 2024-05-14T16:59:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.175+0000 7fb58a9ba740 1 client.15783 shutdown 2024-05-14T16:59:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.176+0000 7fb58a9ba740 1 -- 172.21.15.2:0/2568273120 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d8b194ad0 msgr2=0x564d8b206f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.176+0000 7fb58a9ba740 1 --2- 172.21.15.2:0/2568273120 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d8b194ad0 0x564d8b206f40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb578056040 tx=0x7fb57802f640 comp rx=0 tx=0).stop 2024-05-14T16:59:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.176+0000 7fb58a9ba740 1 -- 172.21.15.2:0/2568273120 shutdown_connections 2024-05-14T16:59:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.176+0000 7fb58a9ba740 1 --2- 172.21.15.2:0/2568273120 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564d8b20eb70 0x564d8b210f60 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:10.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.176+0000 7fb58a9ba740 1 --2- 172.21.15.2:0/2568273120 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564d8b194ad0 0x564d8b206f40 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.176+0000 7fb58a9ba740 1 --2- 172.21.15.2:0/2568273120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564d8b18a210 0x564d8b1982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:10.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.176+0000 7fb5697fa640 0 client.15783 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:59:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.176+0000 7fb58a9ba740 1 --2- 172.21.15.2:0/2568273120 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564d8b1898f0 0x564d8b197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.176+0000 7fb58a9ba740 1 -- 172.21.15.2:0/2568273120 >> 172.21.15.2:0/2568273120 conn(0x564d8b185e00 msgr2=0x564d8b18cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.176+0000 7fb58a9ba740 1 -- 172.21.15.2:0/2568273120 shutdown_connections 2024-05-14T16:59:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.176+0000 7fb58a9ba740 1 -- 172.21.15.2:0/2568273120 wait complete. 2024-05-14T16:59:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.176+0000 7fb58a9ba740 20 client.15783 trim_cache size 0 max 16384 2024-05-14T16:59:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T16:59:10.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T16:59:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T16:59:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.177+0000 7fb587bc1640 1 -- 172.21.15.2:0/4209356402 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb56c0b2ae0 msgr2=0x7fb56c0b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:10.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.177+0000 7fb587bc1640 1 --2- 172.21.15.2:0/4209356402 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb56c0b2ae0 0x7fb56c0b4fa0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb5700a8bd0 tx=0x7fb57000bf30 comp rx=0 tx=0).stop 2024-05-14T16:59:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.177+0000 7fb587bc1640 1 -- 172.21.15.2:0/4209356402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb580117ae0 msgr2=0x7fb580114890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:10.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.177+0000 7fb587bc1640 1 --2- 172.21.15.2:0/4209356402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb580117ae0 0x7fb580114890 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb57000b3c0 tx=0x7fb57000b880 comp rx=0 tx=0).stop 2024-05-14T16:59:10.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.177+0000 7fb587bc1640 1 -- 172.21.15.2:0/4209356402 shutdown_connections 2024-05-14T16:59:10.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.177+0000 7fb587bc1640 1 --2- 172.21.15.2:0/4209356402 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb56c0b2ae0 0x7fb56c0b4fa0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:10.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.177+0000 7fb587bc1640 1 --2- 172.21.15.2:0/4209356402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb580117ae0 0x7fb580114890 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:10.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.177+0000 7fb587bc1640 1 --2- 172.21.15.2:0/4209356402 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb58010d150 0x7fb580114350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:10.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.177+0000 7fb587bc1640 1 --2- 172.21.15.2:0/4209356402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb580109f90 0x7fb5801172b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:10.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.177+0000 7fb587bc1640 1 -- 172.21.15.2:0/4209356402 >> 172.21.15.2:0/4209356402 conn(0x7fb58006db20 msgr2=0x7fb580117ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:10.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.177+0000 7fb587bc1640 1 -- 172.21.15.2:0/4209356402 shutdown_connections 2024-05-14T16:59:10.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:08.177+0000 7fb587bc1640 1 -- 172.21.15.2:0/4209356402 wait complete. 2024-05-14T16:59:10.599 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:59:08 smithi002 ceph-mon[31538]: pgmap v136: 97 pgs: 97 active+clean; 2.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 13 MiB/s wr, 876 op/s 2024-05-14T16:59:10.613 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.8 2024-05-14T16:59:10.621 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:59:10.658 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:59:08 smithi151 ceph-mon[36066]: pgmap v136: 97 pgs: 97 active+clean; 2.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 13 MiB/s wr, 876 op/s 2024-05-14T16:59:10.666 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:59:08 smithi123 ceph-mon[35971]: pgmap v136: 97 pgs: 97 active+clean; 2.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 13 MiB/s wr, 876 op/s 2024-05-14T16:59:10.675 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:59:10 smithi123 ceph-mon[35971]: pgmap v137: 97 pgs: 97 active+clean; 2.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 20 MiB/s wr, 1.09k op/s 2024-05-14T16:59:10.704 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:59:10 smithi002 ceph-mon[31538]: pgmap v137: 97 pgs: 97 active+clean; 2.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 20 MiB/s wr, 1.09k op/s 2024-05-14T16:59:10.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:59:10 smithi151 ceph-mon[36066]: pgmap v137: 97 pgs: 97 active+clean; 2.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 20 MiB/s wr, 1.09k op/s 2024-05-14T16:59:11.072 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:59:11.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 235.0 s, 2378.2 tps, lat 43.445 ms stddev 76.228 2024-05-14T16:59:12.497 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:59:12 smithi002 ceph-mon[31538]: pgmap v138: 97 pgs: 97 active+clean; 2.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 17 MiB/s wr, 900 op/s 2024-05-14T16:59:12.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:59:12 smithi123 ceph-mon[35971]: pgmap v138: 97 pgs: 97 active+clean; 2.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 17 MiB/s wr, 900 op/s 2024-05-14T16:59:12.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:59:12 smithi151 ceph-mon[36066]: pgmap v138: 97 pgs: 97 active+clean; 2.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 17 MiB/s wr, 900 op/s 2024-05-14T16:59:13.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.456+0000 7f26e4ff6700 1 -- 172.21.15.2:0/2552783367 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26e01076d0 msgr2=0x7f26e010f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:13.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.456+0000 7f26e4ff6700 1 --2- 172.21.15.2:0/2552783367 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26e01076d0 0x7f26e010f2a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f26d800b0d0 tx=0x7f26d800b3e0 comp rx=0 tx=0).stop 2024-05-14T16:59:13.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.456+0000 7f26e4ff6700 1 -- 172.21.15.2:0/2552783367 shutdown_connections 2024-05-14T16:59:13.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.456+0000 7f26e4ff6700 1 --2- 172.21.15.2:0/2552783367 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26e01076d0 0x7f26e010f2a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:13.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.456+0000 7f26e4ff6700 1 --2- 172.21.15.2:0/2552783367 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26e01025f0 0x7f26e01029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:13.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.456+0000 7f26e4ff6700 1 --2- 172.21.15.2:0/2552783367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26e0102fa0 0x7f26e0106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:13.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.456+0000 7f26e4ff6700 1 -- 172.21.15.2:0/2552783367 >> 172.21.15.2:0/2552783367 conn(0x7f26e00fdec0 msgr2=0x7f26e01002e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:13.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.456+0000 7f26e4ff6700 1 -- 172.21.15.2:0/2552783367 shutdown_connections 2024-05-14T16:59:13.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.456+0000 7f26e4ff6700 1 -- 172.21.15.2:0/2552783367 wait complete. 2024-05-14T16:59:13.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.458+0000 7f26e4ff6700 1 Processor -- start 2024-05-14T16:59:13.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.458+0000 7f26e4ff6700 1 -- start start 2024-05-14T16:59:13.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.459+0000 7f26e4ff6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26e01025f0 0x7f26e019d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:13.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.459+0000 7f26e4ff6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26e0102fa0 0x7f26e019db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:13.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.459+0000 7f26e4ff6700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26e01076d0 0x7f26e01977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:13.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.459+0000 7f26e4ff6700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f26e0111520 con 0x7f26e0102fa0 2024-05-14T16:59:13.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.459+0000 7f26e4ff6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f26e01113a0 con 0x7f26e01025f0 2024-05-14T16:59:13.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.459+0000 7f26e4ff6700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f26e01116a0 con 0x7f26e01076d0 2024-05-14T16:59:13.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.459+0000 7f26ded9d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26e01076d0 0x7f26e01977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:13.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.459+0000 7f26ded9d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26e01076d0 0x7f26e01977e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:49400/0 (socket says 172.21.15.2:49400) 2024-05-14T16:59:13.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.459+0000 7f26ded9d700 1 -- 172.21.15.2:0/2061806641 learned_addr learned my addr 172.21.15.2:0/2061806641 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:59:13.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.459+0000 7f26de59c700 1 --2- 172.21.15.2:0/2061806641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26e01025f0 0x7f26e019d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:13.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.459+0000 7f26ddd9b700 1 --2- 172.21.15.2:0/2061806641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26e0102fa0 0x7f26e019db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:13.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.460+0000 7f26ded9d700 1 -- 172.21.15.2:0/2061806641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26e01025f0 msgr2=0x7f26e019d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:13.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.460+0000 7f26ded9d700 1 --2- 172.21.15.2:0/2061806641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26e01025f0 0x7f26e019d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:13.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.460+0000 7f26ded9d700 1 -- 172.21.15.2:0/2061806641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26e0102fa0 msgr2=0x7f26e019db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:13.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.460+0000 7f26ded9d700 1 --2- 172.21.15.2:0/2061806641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26e0102fa0 0x7f26e019db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:13.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.460+0000 7f26ded9d700 1 -- 172.21.15.2:0/2061806641 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e0197fc0 con 0x7f26e01076d0 2024-05-14T16:59:13.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.460+0000 7f26ded9d700 1 --2- 172.21.15.2:0/2061806641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26e01076d0 0x7f26e01977e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f26d800b770 tx=0x7f26d8007980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:13.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.460+0000 7f26cf7fe700 1 -- 172.21.15.2:0/2061806641 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d801f070 con 0x7f26e01076d0 2024-05-14T16:59:13.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.460+0000 7f26e4ff6700 1 -- 172.21.15.2:0/2061806641 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26e0198250 con 0x7f26e01076d0 2024-05-14T16:59:13.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.460+0000 7f26cf7fe700 1 -- 172.21.15.2:0/2061806641 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26d8004c50 con 0x7f26e01076d0 2024-05-14T16:59:13.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.460+0000 7f26e4ff6700 1 -- 172.21.15.2:0/2061806641 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26e01a8880 con 0x7f26e01076d0 2024-05-14T16:59:13.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.461+0000 7f26cf7fe700 1 -- 172.21.15.2:0/2061806641 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d801b690 con 0x7f26e01076d0 2024-05-14T16:59:13.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.462+0000 7f26e4ff6700 1 -- 172.21.15.2:0/2061806641 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26c0005330 con 0x7f26e01076d0 2024-05-14T16:59:13.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.463+0000 7f26cf7fe700 1 -- 172.21.15.2:0/2061806641 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f26d8022070 con 0x7f26e01076d0 2024-05-14T16:59:13.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.466+0000 7f26cf7fe700 1 --2- 172.21.15.2:0/2061806641 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f26c80b2a70 0x7f26c80b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:13.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.466+0000 7f26de59c700 1 --2- 172.21.15.2:0/2061806641 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f26c80b2a70 0x7f26c80b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:14.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.466+0000 7f26cf7fe700 1 -- 172.21.15.2:0/2061806641 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f26d800f020 con 0x7f26e01076d0 2024-05-14T16:59:14.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.467+0000 7f26de59c700 1 --2- 172.21.15.2:0/2061806641 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f26c80b2a70 0x7f26c80b4f30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f26d000db40 tx=0x7f26d000f040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:14.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.469+0000 7f26cf7fe700 1 -- 172.21.15.2:0/2061806641 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f26d8029020 con 0x7f26e01076d0 2024-05-14T16:59:14.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.896+0000 7f26e4ff6700 1 -- 172.21.15.2:0/2061806641 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26c0005620 con 0x7f26e01076d0 2024-05-14T16:59:14.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.897+0000 7f26cf7fe700 1 -- 172.21.15.2:0/2061806641 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11851 (secure 0 0 0) 0x7f26d80ab020 con 0x7f26e01076d0 2024-05-14T16:59:14.060 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:59:14.071 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2275696923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2275696923},{"type":"v1","addr":"172.21.15.2:6835","nonce":2275696923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/716718761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":716718761},{"type":"v1","addr":"172.21.15.2:6839","nonce":716718761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/3644815179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3644815179},{"type":"v1","addr":"172.21.15.2:6841","nonce":3644815179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1880336731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1880336731},{"type":"v1","addr":"172.21.15.151:6833","nonce":1880336731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2415605548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2415605548},{"type":"v1","addr":"172.21.15.2:6837","nonce":2415605548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2635167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2635167},{"type":"v1","addr":"172.21.15.123:6833","nonce":2635167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:51:34.106919+0000","modified":"2024-05-14T16:58:33.451604+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24455,"mds_2":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24415":{"gid":24415,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/762044339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":762044339},{"type":"v1","addr":"172.21.15.151:6835","nonce":762044339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/132043151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":132043151},{"type":"v1","addr":"172.21.15.151:6837","nonce":132043151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/4107834375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":4107834375},{"type":"v1","addr":"172.21.15.151:6839","nonce":4107834375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.123:6835/1198670013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1198670013},{"type":"v1","addr":"172.21.15.123:6835","nonce":1198670013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/333426645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":333426645},{"type":"v1","addr":"172.21.15.123:6837","nonce":333426645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/580275877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":580275877},{"type":"v1","addr":"172.21.15.123:6839","nonce":580275877}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24455,24479,24431]},"id":1}]} 2024-05-14T16:59:14.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.901+0000 7f26e4ff6700 1 -- 172.21.15.2:0/2061806641 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f26c80b2a70 msgr2=0x7f26c80b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:14.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.901+0000 7f26e4ff6700 1 --2- 172.21.15.2:0/2061806641 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f26c80b2a70 0x7f26c80b4f30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f26d000db40 tx=0x7f26d000f040 comp rx=0 tx=0).stop 2024-05-14T16:59:14.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.901+0000 7f26e4ff6700 1 -- 172.21.15.2:0/2061806641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26e01076d0 msgr2=0x7f26e01977e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:14.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.901+0000 7f26e4ff6700 1 --2- 172.21.15.2:0/2061806641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26e01076d0 0x7f26e01977e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f26d800b770 tx=0x7f26d8007980 comp rx=0 tx=0).stop 2024-05-14T16:59:14.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.902+0000 7f26e4ff6700 1 -- 172.21.15.2:0/2061806641 shutdown_connections 2024-05-14T16:59:14.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.902+0000 7f26e4ff6700 1 --2- 172.21.15.2:0/2061806641 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f26c80b2a70 0x7f26c80b4f30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:14.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.902+0000 7f26e4ff6700 1 --2- 172.21.15.2:0/2061806641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26e01076d0 0x7f26e01977e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:14.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.902+0000 7f26e4ff6700 1 --2- 172.21.15.2:0/2061806641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26e01025f0 0x7f26e019d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:14.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.902+0000 7f26e4ff6700 1 --2- 172.21.15.2:0/2061806641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26e0102fa0 0x7f26e019db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:14.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.902+0000 7f26e4ff6700 1 -- 172.21.15.2:0/2061806641 >> 172.21.15.2:0/2061806641 conn(0x7f26e00fdec0 msgr2=0x7f26e0100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:14.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.903+0000 7f26e4ff6700 1 -- 172.21.15.2:0/2061806641 shutdown_connections 2024-05-14T16:59:14.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:13.903+0000 7f26e4ff6700 1 -- 172.21.15.2:0/2061806641 wait complete. 2024-05-14T16:59:14.231 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 36 2024-05-14T16:59:14.563 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:59:14 smithi002 ceph-mon[31538]: pgmap v139: 97 pgs: 97 active+clean; 2.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 39 MiB/s wr, 1.66k op/s 2024-05-14T16:59:14.571 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:59:14 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/2061806641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:59:14.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:59:14 smithi123 ceph-mon[35971]: pgmap v139: 97 pgs: 97 active+clean; 2.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 39 MiB/s wr, 1.66k op/s 2024-05-14T16:59:14.666 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:59:14 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/2061806641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:59:14.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:59:14 smithi151 ceph-mon[36066]: pgmap v139: 97 pgs: 97 active+clean; 2.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 39 MiB/s wr, 1.66k op/s 2024-05-14T16:59:14.865 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:59:14 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/2061806641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:59:15.428 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 146.8 --await-for 45.0' 2024-05-14T16:59:15.467 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 146.8 --await-for 45.0 2024-05-14T16:59:15.860 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:59:16.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:59:16 smithi123 ceph-mon[35971]: pgmap v140: 97 pgs: 97 active+clean; 2.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 34 MiB/s wr, 1.33k op/s 2024-05-14T16:59:16.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:59:16 smithi002 ceph-mon[31538]: pgmap v140: 97 pgs: 97 active+clean; 2.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 34 MiB/s wr, 1.33k op/s 2024-05-14T16:59:16.754 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 240.0 s, 3148.8 tps, lat 31.602 ms stddev 34.689 2024-05-14T16:59:16.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:59:16 smithi151 ceph-mon[36066]: pgmap v140: 97 pgs: 97 active+clean; 2.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 34 MiB/s wr, 1.33k op/s 2024-05-14T16:59:17.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.835+0000 7fc45d85e700 1 -- 172.21.15.2:0/1240828800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc458114230 msgr2=0x7fc458116640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:17.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.835+0000 7fc45d85e700 1 --2- 172.21.15.2:0/1240828800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc458114230 0x7fc458116640 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc450009990 tx=0x7fc450009ca0 comp rx=0 tx=0).stop 2024-05-14T16:59:17.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.835+0000 7fc45d85e700 1 -- 172.21.15.2:0/1240828800 shutdown_connections 2024-05-14T16:59:17.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.835+0000 7fc45d85e700 1 --2- 172.21.15.2:0/1240828800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc458114230 0x7fc458116640 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:17.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.835+0000 7fc45d85e700 1 --2- 172.21.15.2:0/1240828800 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4580775a0 0x7fc458075770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:17.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.835+0000 7fc45d85e700 1 --2- 172.21.15.2:0/1240828800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc458075cb0 0x7fc458076130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:17.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.835+0000 7fc45d85e700 1 -- 172.21.15.2:0/1240828800 >> 172.21.15.2:0/1240828800 conn(0x7fc45807a6f0 msgr2=0x7fc45807ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:17.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.836+0000 7fc45d85e700 1 -- 172.21.15.2:0/1240828800 shutdown_connections 2024-05-14T16:59:17.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.836+0000 7fc45d85e700 1 -- 172.21.15.2:0/1240828800 wait complete. 2024-05-14T16:59:18.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.836+0000 7fc45d85e700 1 Processor -- start 2024-05-14T16:59:18.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.837+0000 7fc45d85e700 1 -- start start 2024-05-14T16:59:18.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.837+0000 7fc45d85e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc458075cb0 0x7fc4581a0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:18.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.837+0000 7fc45d85e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4580775a0 0x7fc4581a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:18.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.837+0000 7fc45d85e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc458114230 0x7fc4581a4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:18.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.837+0000 7fc45d85e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc458119da0 con 0x7fc458075cb0 2024-05-14T16:59:18.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.837+0000 7fc45d85e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc458119c20 con 0x7fc458114230 2024-05-14T16:59:18.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.837+0000 7fc45d85e700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc458119f20 con 0x7fc4580775a0 2024-05-14T16:59:18.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.838+0000 7fc44e7fc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4580775a0 0x7fc4581a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:18.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.838+0000 7fc4577fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc458114230 0x7fc4581a4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:18.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.838+0000 7fc456ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc458075cb0 0x7fc4581a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:18.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.838+0000 7fc456ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc458075cb0 0x7fc4581a0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42944/0 (socket says 172.21.15.2:42944) 2024-05-14T16:59:18.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.838+0000 7fc456ffd700 1 -- 172.21.15.2:0/3654299495 learned_addr learned my addr 172.21.15.2:0/3654299495 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:59:18.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.838+0000 7fc44e7fc700 1 -- 172.21.15.2:0/3654299495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc458114230 msgr2=0x7fc4581a4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:18.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.838+0000 7fc44e7fc700 1 --2- 172.21.15.2:0/3654299495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc458114230 0x7fc4581a4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:18.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.838+0000 7fc44e7fc700 1 -- 172.21.15.2:0/3654299495 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc458075cb0 msgr2=0x7fc4581a0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:18.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.838+0000 7fc44e7fc700 1 --2- 172.21.15.2:0/3654299495 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc458075cb0 0x7fc4581a0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:18.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.838+0000 7fc44e7fc700 1 -- 172.21.15.2:0/3654299495 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4581a5160 con 0x7fc4580775a0 2024-05-14T16:59:18.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.838+0000 7fc456ffd700 1 --2- 172.21.15.2:0/3654299495 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc458075cb0 0x7fc4581a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:59:18.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.839+0000 7fc44e7fc700 1 --2- 172.21.15.2:0/3654299495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4580775a0 0x7fc4581a0690 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc44000c970 tx=0x7fc44000cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:18.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.839+0000 7fc45c85c700 1 -- 172.21.15.2:0/3654299495 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc440004a30 con 0x7fc4580775a0 2024-05-14T16:59:18.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.839+0000 7fc45c85c700 1 -- 172.21.15.2:0/3654299495 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4400054d0 con 0x7fc4580775a0 2024-05-14T16:59:18.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.839+0000 7fc45c85c700 1 -- 172.21.15.2:0/3654299495 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44000eb40 con 0x7fc4580775a0 2024-05-14T16:59:18.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.839+0000 7fc45d85e700 1 -- 172.21.15.2:0/3654299495 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4581a5450 con 0x7fc4580775a0 2024-05-14T16:59:18.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.839+0000 7fc45d85e700 1 -- 172.21.15.2:0/3654299495 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc45806cf60 con 0x7fc4580775a0 2024-05-14T16:59:18.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.845+0000 7fc4680f4b80 1 Processor -- start 2024-05-14T16:59:18.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.845+0000 7fc4680f4b80 1 -- start start 2024-05-14T16:59:18.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.846+0000 7fc4680f4b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca07323e30 0x55ca07324210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:18.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.846+0000 7fc4680f4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca07324750 0x55ca0731e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:18.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.846+0000 7fc4680f4b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca0731eb60 0x55ca0731efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:18.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.846+0000 7fc4680f4b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ca0727ce20 con 0x55ca0731eb60 2024-05-14T16:59:18.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.846+0000 7fc4680f4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ca0726c9c0 con 0x55ca07324750 2024-05-14T16:59:18.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.846+0000 7fc4680f4b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ca07270a00 con 0x55ca07323e30 2024-05-14T16:59:18.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.846+0000 7fc45c85c700 1 -- 172.21.15.2:0/3654299495 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fc44000ed00 con 0x7fc4580775a0 2024-05-14T16:59:18.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.846+0000 7fc456ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca07324750 0x55ca0731e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:18.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.846+0000 7fc456ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca07324750 0x55ca0731e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:60826/0 (socket says 172.21.15.2:60826) 2024-05-14T16:59:18.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.846+0000 7fc456ffd700 1 -- 172.21.15.2:0/3910082444 learned_addr learned my addr 172.21.15.2:0/3910082444 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:59:18.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.846+0000 7fc4577fe700 1 --2- 172.21.15.2:0/3910082444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca07323e30 0x55ca07324210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:18.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.846+0000 7fc456ffd700 1 -- 172.21.15.2:0/3910082444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca07323e30 msgr2=0x55ca07324210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:18.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.847+0000 7fc456ffd700 1 --2- 172.21.15.2:0/3910082444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca07323e30 0x55ca07324210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:18.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.847+0000 7fc456ffd700 1 -- 172.21.15.2:0/3910082444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca0731eb60 msgr2=0x55ca0731efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:18.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.847+0000 7fc456ffd700 1 --2- 172.21.15.2:0/3910082444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca0731eb60 0x55ca0731efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:18.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.847+0000 7fc456ffd700 1 -- 172.21.15.2:0/3910082444 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca071bca60 con 0x55ca07324750 2024-05-14T16:59:18.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.847+0000 7fc456ffd700 1 --2- 172.21.15.2:0/3910082444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca07324750 0x55ca0731e540 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc448007850 tx=0x7fc4480092b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edfd7460fdd50d4b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:18.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.848+0000 7fc45c85c700 1 --2- 172.21.15.2:0/3654299495 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc4440b2c60 0x7fc4440b5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:18.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.848+0000 7fc44dffb700 1 -- 172.21.15.2:0/3910082444 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc448019020 con 0x55ca07324750 2024-05-14T16:59:18.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.848+0000 7fc45c85c700 1 -- 172.21.15.2:0/3654299495 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fc4400e1f40 con 0x7fc4580775a0 2024-05-14T16:59:18.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.848+0000 7fc44dffb700 1 -- 172.21.15.2:0/3910082444 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc448015d00 con 0x55ca07324750 2024-05-14T16:59:18.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.848+0000 7fc44dffb700 1 -- 172.21.15.2:0/3910082444 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc448014d80 con 0x55ca07324750 2024-05-14T16:59:18.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.849+0000 7fc4577fe700 1 --2- 172.21.15.2:0/3654299495 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc4440b2c60 0x7fc4440b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:18.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.849+0000 7fc4680f4b80 1 -- 172.21.15.2:0/3910082444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca07324750 msgr2=0x55ca0731e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:18.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.849+0000 7fc4680f4b80 1 --2- 172.21.15.2:0/3910082444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca07324750 0x55ca0731e540 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc448007850 tx=0x7fc4480092b0 comp rx=0 tx=0).stop 2024-05-14T16:59:18.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.850+0000 7fc4577fe700 1 --2- 172.21.15.2:0/3654299495 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc4440b2c60 0x7fc4440b5120 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc4500098e0 tx=0x7fc450005ab0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:18.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.850+0000 7fc4680f4b80 1 -- 172.21.15.2:0/3910082444 shutdown_connections 2024-05-14T16:59:18.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.850+0000 7fc4680f4b80 1 --2- 172.21.15.2:0/3910082444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca0731eb60 0x55ca0731efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:18.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.850+0000 7fc4680f4b80 1 --2- 172.21.15.2:0/3910082444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca07323e30 0x55ca07324210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:18.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.850+0000 7fc4680f4b80 1 --2- 172.21.15.2:0/3910082444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca07324750 0x55ca0731e540 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:18.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.850+0000 7fc4680f4b80 1 -- 172.21.15.2:0/3910082444 >> 172.21.15.2:0/3910082444 conn(0x55ca072493e0 msgr2=0x55ca07318d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:18.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.850+0000 7fc4680f4b80 1 -- 172.21.15.2:0/3910082444 shutdown_connections 2024-05-14T16:59:18.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.850+0000 7fc4680f4b80 1 -- 172.21.15.2:0/3910082444 wait complete. 2024-05-14T16:59:18.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.850+0000 7fc4680f4b80 1 -- start start 2024-05-14T16:59:18.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.850+0000 7fc4680f4b80 1 Processor -- start 2024-05-14T16:59:18.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.851+0000 7fc4680f4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca0731eb60 0x55ca0732a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:18.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.851+0000 7fc4680f4b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca07323e30 0x55ca073279e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:18.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.851+0000 7fc4680f4b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca07324750 0x55ca07327f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:18.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.851+0000 7fc4680f4b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ca07173130 con 0x55ca07324750 2024-05-14T16:59:18.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.851+0000 7fc4680f4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ca0720e950 con 0x55ca0731eb60 2024-05-14T16:59:18.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.851+0000 7fc4680f4b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ca0732b190 con 0x55ca07323e30 2024-05-14T16:59:18.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.851+0000 7fc4680f4b80 10 client.0 _ll_register_callbacks cb 0x55ca07203710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:59:18.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.851+0000 7fc456ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca0731eb60 0x55ca0732a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:18.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.851+0000 7fc456ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca0731eb60 0x55ca0732a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:60830/0 (socket says 172.21.15.2:60830) 2024-05-14T16:59:18.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.851+0000 7fc456ffd700 1 -- 172.21.15.2:0/1853107429 learned_addr learned my addr 172.21.15.2:0/1853107429 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:59:18.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.852+0000 7fc456ffd700 1 -- 172.21.15.2:0/1853107429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca07323e30 msgr2=0x55ca073279e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T16:59:18.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.852+0000 7fc456ffd700 1 --2- 172.21.15.2:0/1853107429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca07323e30 0x55ca073279e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:18.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.852+0000 7fc456ffd700 1 -- 172.21.15.2:0/1853107429 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca07324750 msgr2=0x55ca07327f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:18.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.852+0000 7fc456ffd700 1 --2- 172.21.15.2:0/1853107429 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca07324750 0x55ca07327f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:18.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.852+0000 7fc456ffd700 1 -- 172.21.15.2:0/1853107429 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca0720b820 con 0x55ca0731eb60 2024-05-14T16:59:18.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.852+0000 7fc456ffd700 1 --2- 172.21.15.2:0/1853107429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca0731eb60 0x55ca0732a5f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc448007820 tx=0x7fc448004db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27a569329aa5ec10 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:18.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.852+0000 7fc436ffd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T16:59:18.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.852+0000 7fc436ffd700 1 -- 172.21.15.2:0/1853107429 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc448019040 con 0x55ca0731eb60 2024-05-14T16:59:18.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.852+0000 7fc4680f4b80 1 -- 172.21.15.2:0/1853107429 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc448015d00 con 0x55ca0731eb60 2024-05-14T16:59:18.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.852+0000 7fc436ffd700 1 -- 172.21.15.2:0/1853107429 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fc448015a60 con 0x55ca0731eb60 2024-05-14T16:59:18.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.853+0000 7fc436ffd700 1 -- 172.21.15.2:0/1853107429 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc448014980 con 0x55ca0731eb60 2024-05-14T16:59:18.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.853+0000 7fc4680f4b80 10 client.25508 fetch_fsmap learned FSMap version 36 2024-05-14T16:59:18.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.853+0000 7fc4680f4b80 1 -- 172.21.15.2:0/1853107429 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ca07335a40 con 0x55ca0731eb60 2024-05-14T16:59:18.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.853+0000 7fc436ffd700 1 -- 172.21.15.2:0/1853107429 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc448015d00 con 0x55ca0731eb60 2024-05-14T16:59:18.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.854+0000 7fc436ffd700 1 -- 172.21.15.2:0/1853107429 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 36) ==== 8342+0+0 (secure 0 0 0) 0x7fc4480057c0 con 0x55ca0731eb60 2024-05-14T16:59:18.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.854+0000 7fc4680f4b80 10 client.25508 fetch_fsmap finished waiting for FSMap version 36 2024-05-14T16:59:18.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.854+0000 7fc4680f4b80 10 client.25508 resolve_mds: validated gid 24479 aka daemon mds.b 2024-05-14T16:59:18.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.854+0000 7fc4680f4b80 20 client.25508 populate_metadata read hostname 'smithi002' 2024-05-14T16:59:19.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.854+0000 7fc4680f4b80 1 --2- 172.21.15.2:0/1853107429 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ca07337fc0 0x55ca0733a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:19.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.854+0000 7fc4680f4b80 4 client.25508 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:59:19.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.854+0000 7fc4680f4b80 1 -- 172.21.15.2:0/1853107429 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca072ea4f0 con 0x55ca07337fc0 2024-05-14T16:59:19.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.854+0000 7fc44e7fc700 1 --2- 172.21.15.2:0/1853107429 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ca07337fc0 0x55ca0733a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:19.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.854+0000 7fc44e7fc700 1 --2- 172.21.15.2:0/1853107429 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ca07337fc0 0x55ca0733a3b0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2648983e370be46 server_cookie=ac2982c481ed1816 in_seq=0 out_seq=0 2024-05-14T16:59:19.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.854+0000 7fc436ffd700 10 client.25508 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:59:19.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.855+0000 7fc436ffd700 1 -- 172.21.15.2:0/1853107429 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ca072ea4f0 con 0x55ca07337fc0 2024-05-14T16:59:19.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.855+0000 7fc436ffd700 10 client.25508 handle_command_reply: tid=0 2024-05-14T16:59:19.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.855+0000 7fc4680f4b80 1 client.25508 shutdown 2024-05-14T16:59:19.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.855+0000 7fc4680f4b80 1 -- 172.21.15.2:0/1853107429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca0731eb60 msgr2=0x55ca0732a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:19.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.855+0000 7fc4680f4b80 1 --2- 172.21.15.2:0/1853107429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca0731eb60 0x55ca0732a5f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc448007820 tx=0x7fc448004db0 comp rx=0 tx=0).stop 2024-05-14T16:59:19.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.855+0000 7fc4680f4b80 1 -- 172.21.15.2:0/1853107429 shutdown_connections 2024-05-14T16:59:19.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.855+0000 7fc4680f4b80 1 --2- 172.21.15.2:0/1853107429 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ca07337fc0 0x55ca0733a3b0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:19.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.855+0000 7fc4680f4b80 1 --2- 172.21.15.2:0/1853107429 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca07324750 0x55ca07327f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:19.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.855+0000 7fc4680f4b80 1 --2- 172.21.15.2:0/1853107429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca0731eb60 0x55ca0732a5f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:19.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.855+0000 7fc4680f4b80 1 --2- 172.21.15.2:0/1853107429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca07323e30 0x55ca073279e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:19.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.855+0000 7fc4680f4b80 1 -- 172.21.15.2:0/1853107429 >> 172.21.15.2:0/1853107429 conn(0x55ca072493e0 msgr2=0x55ca07313860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:19.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.855+0000 7fc436ffd700 0 client.25508 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:59:19.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.855+0000 7fc4680f4b80 1 -- 172.21.15.2:0/1853107429 shutdown_connections 2024-05-14T16:59:19.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.856+0000 7fc4680f4b80 1 -- 172.21.15.2:0/1853107429 wait complete. 2024-05-14T16:59:19.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.856+0000 7fc4680f4b80 20 client.25508 trim_cache size 0 max 16384 2024-05-14T16:59:19.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.892+0000 7fc4680f4b80 1 Processor -- start 2024-05-14T16:59:19.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.892+0000 7fc4680f4b80 1 -- start start 2024-05-14T16:59:19.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.892+0000 7fc4680f4b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca0731eb60 0x55ca0731e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:19.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.892+0000 7fc4680f4b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca07323e30 0x55ca07313970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:19.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.892+0000 7fc4680f4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca07324750 0x55ca073279a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:19.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.892+0000 7fc4680f4b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc4581bf550 con 0x55ca07323e30 2024-05-14T16:59:19.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.892+0000 7fc4680f4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc458074870 con 0x55ca07324750 2024-05-14T16:59:19.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.892+0000 7fc4680f4b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc4580726e0 con 0x55ca0731eb60 2024-05-14T16:59:19.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.892+0000 7fc44e7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca07323e30 0x55ca07313970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:19.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.892+0000 7fc44e7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca07323e30 0x55ca07313970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42970/0 (socket says 172.21.15.2:42970) 2024-05-14T16:59:19.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.892+0000 7fc44e7fc700 1 -- 172.21.15.2:0/1422315806 learned_addr learned my addr 172.21.15.2:0/1422315806 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:59:19.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.892+0000 7fc4577fe700 1 --2- 172.21.15.2:0/1422315806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca07324750 0x55ca073279a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:19.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.892+0000 7fc456ffd700 1 --2- 172.21.15.2:0/1422315806 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca0731eb60 0x55ca0731e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:19.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.893+0000 7fc4577fe700 1 -- 172.21.15.2:0/1422315806 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca0731eb60 msgr2=0x55ca0731e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:19.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.893+0000 7fc4577fe700 1 --2- 172.21.15.2:0/1422315806 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca0731eb60 0x55ca0731e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:19.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.893+0000 7fc4577fe700 1 -- 172.21.15.2:0/1422315806 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca07323e30 msgr2=0x55ca07313970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:19.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.893+0000 7fc4577fe700 1 --2- 172.21.15.2:0/1422315806 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca07323e30 0x55ca07313970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:19.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.893+0000 7fc4577fe700 1 -- 172.21.15.2:0/1422315806 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca07313eb0 con 0x55ca07324750 2024-05-14T16:59:19.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.893+0000 7fc44e7fc700 1 --2- 172.21.15.2:0/1422315806 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca07323e30 0x55ca07313970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:59:19.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.893+0000 7fc4577fe700 1 --2- 172.21.15.2:0/1422315806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca07324750 0x55ca073279a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc450006010 tx=0x7fc450019070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30681b3cc408cb96 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:19.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.893+0000 7fc44d7fa700 1 -- 172.21.15.2:0/1422315806 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4500050d0 con 0x55ca07324750 2024-05-14T16:59:19.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.893+0000 7fc44d7fa700 1 -- 172.21.15.2:0/1422315806 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc45000b7d0 con 0x55ca07324750 2024-05-14T16:59:19.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.893+0000 7fc44d7fa700 1 -- 172.21.15.2:0/1422315806 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc450023750 con 0x55ca07324750 2024-05-14T16:59:19.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.894+0000 7fc4680f4b80 1 -- 172.21.15.2:0/1422315806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca07324750 msgr2=0x55ca073279a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:19.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.894+0000 7fc4680f4b80 1 --2- 172.21.15.2:0/1422315806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca07324750 0x55ca073279a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc450006010 tx=0x7fc450019070 comp rx=0 tx=0).stop 2024-05-14T16:59:19.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.894+0000 7fc4680f4b80 1 -- 172.21.15.2:0/1422315806 shutdown_connections 2024-05-14T16:59:19.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.894+0000 7fc4680f4b80 1 --2- 172.21.15.2:0/1422315806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca07324750 0x55ca073279a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:19.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.894+0000 7fc4680f4b80 1 --2- 172.21.15.2:0/1422315806 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca0731eb60 0x55ca0731e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:19.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.894+0000 7fc4680f4b80 1 --2- 172.21.15.2:0/1422315806 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca07323e30 0x55ca07313970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:19.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.894+0000 7fc4680f4b80 1 -- 172.21.15.2:0/1422315806 >> 172.21.15.2:0/1422315806 conn(0x55ca072493e0 msgr2=0x55ca07327290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:19.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.894+0000 7fc4680f4b80 1 -- 172.21.15.2:0/1422315806 shutdown_connections 2024-05-14T16:59:19.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.895+0000 7fc4680f4b80 1 -- 172.21.15.2:0/1422315806 wait complete. 2024-05-14T16:59:19.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.895+0000 7fc4680f4b80 1 -- start start 2024-05-14T16:59:19.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.895+0000 7fc4680f4b80 1 Processor -- start 2024-05-14T16:59:19.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.896+0000 7fc4680f4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca0731eb60 0x55ca0733d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:19.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.896+0000 7fc4680f4b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca07323e30 0x55ca0733b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:19.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.896+0000 7fc4680f4b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca07324750 0x55ca0733bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:19.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.896+0000 7fc4680f4b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc458072d30 con 0x55ca07324750 2024-05-14T16:59:19.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.896+0000 7fc4680f4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc4580741a0 con 0x55ca0731eb60 2024-05-14T16:59:19.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.896+0000 7fc4680f4b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc458073400 con 0x55ca07323e30 2024-05-14T16:59:19.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.896+0000 7fc4680f4b80 10 client.0 _ll_register_callbacks cb 0x55ca07203710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:59:19.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.896+0000 7fc44e7fc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca07323e30 0x55ca0733b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:19.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.896+0000 7fc44e7fc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca07323e30 0x55ca0733b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:58892/0 (socket says 172.21.15.2:58892) 2024-05-14T16:59:19.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.896+0000 7fc44e7fc700 1 -- 172.21.15.2:0/778700011 learned_addr learned my addr 172.21.15.2:0/778700011 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:59:19.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.896+0000 7fc4577fe700 1 --2- 172.21.15.2:0/778700011 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca07324750 0x55ca0733bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:19.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.896+0000 7fc456ffd700 1 --2- 172.21.15.2:0/778700011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca0731eb60 0x55ca0733d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:19.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.896+0000 7fc456ffd700 1 -- 172.21.15.2:0/778700011 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca07323e30 msgr2=0x55ca0733b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:19.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.896+0000 7fc456ffd700 1 --2- 172.21.15.2:0/778700011 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca07323e30 0x55ca0733b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:19.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.896+0000 7fc456ffd700 1 -- 172.21.15.2:0/778700011 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca07324750 msgr2=0x55ca0733bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:19.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.896+0000 7fc456ffd700 1 --2- 172.21.15.2:0/778700011 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca07324750 0x55ca0733bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:19.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.896+0000 7fc456ffd700 1 -- 172.21.15.2:0/778700011 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca0733c710 con 0x55ca0731eb60 2024-05-14T16:59:19.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.897+0000 7fc456ffd700 1 --2- 172.21.15.2:0/778700011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca0731eb60 0x55ca0733d5c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc448018040 tx=0x7fc448004b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5264fa297d68730 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:19.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.898+0000 7fc436ffd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T16:59:19.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.898+0000 7fc436ffd700 1 -- 172.21.15.2:0/778700011 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc448033070 con 0x55ca0731eb60 2024-05-14T16:59:19.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.898+0000 7fc436ffd700 1 -- 172.21.15.2:0/778700011 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fc448014d20 con 0x55ca0731eb60 2024-05-14T16:59:19.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.898+0000 7fc4680f4b80 1 -- 172.21.15.2:0/778700011 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc45000b7d0 con 0x55ca0731eb60 2024-05-14T16:59:19.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.899+0000 7fc436ffd700 1 -- 172.21.15.2:0/778700011 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc448019070 con 0x55ca0731eb60 2024-05-14T16:59:19.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.899+0000 7fc4680f4b80 10 client.25514 fetch_fsmap learned FSMap version 36 2024-05-14T16:59:19.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.899+0000 7fc4680f4b80 1 -- 172.21.15.2:0/778700011 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ca073a3560 con 0x55ca0731eb60 2024-05-14T16:59:19.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.899+0000 7fc436ffd700 1 -- 172.21.15.2:0/778700011 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc448003a00 con 0x55ca0731eb60 2024-05-14T16:59:19.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.899+0000 7fc436ffd700 1 -- 172.21.15.2:0/778700011 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 36) ==== 8342+0+0 (secure 0 0 0) 0x7fc448015980 con 0x55ca0731eb60 2024-05-14T16:59:19.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.899+0000 7fc4680f4b80 10 client.25514 fetch_fsmap finished waiting for FSMap version 36 2024-05-14T16:59:19.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.899+0000 7fc4680f4b80 10 client.25514 resolve_mds: validated gid 24479 aka daemon mds.b 2024-05-14T16:59:19.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.899+0000 7fc4680f4b80 20 client.25514 populate_metadata read hostname 'smithi002' 2024-05-14T16:59:19.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.899+0000 7fc4680f4b80 1 --2- 172.21.15.2:0/778700011 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ca073a3c20 0x55ca073a6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:19.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.899+0000 7fc4680f4b80 4 client.25514 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.8, "await_for": 45.0}] 2024-05-14T16:59:20.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.899+0000 7fc4680f4b80 1 -- 172.21.15.2:0/778700011 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.8, "await_for": 45.0}) -- 0x7fc45000b7d0 con 0x55ca073a3c20 2024-05-14T16:59:20.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.900+0000 7fc44e7fc700 1 --2- 172.21.15.2:0/778700011 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ca073a3c20 0x55ca073a6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:20.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.900+0000 7fc44e7fc700 1 --2- 172.21.15.2:0/778700011 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ca073a3c20 0x55ca073a6040 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e209d8dd60d580e0 server_cookie=c2783fda3f725afb in_seq=0 out_seq=0 2024-05-14T16:59:20.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:17.900+0000 7fc436ffd700 10 client.25514 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:59:20.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:18.125+0000 7fc436ffd700 1 -- 172.21.15.2:0/778700011 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7fc45000b7d0 con 0x55ca073a3c20 2024-05-14T16:59:20.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:18.125+0000 7fc436ffd700 10 client.25514 handle_command_reply: tid=0 2024-05-14T16:59:20.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:18.125+0000 7fc4680f4b80 1 client.25514 shutdown 2024-05-14T16:59:20.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:18.126+0000 7fc4680f4b80 1 -- 172.21.15.2:0/778700011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca0731eb60 msgr2=0x55ca0733d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:20.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:18.126+0000 7fc4680f4b80 1 --2- 172.21.15.2:0/778700011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca0731eb60 0x55ca0733d5c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc448018040 tx=0x7fc448004b00 comp rx=0 tx=0).stop 2024-05-14T16:59:20.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:18.126+0000 7fc4680f4b80 1 -- 172.21.15.2:0/778700011 shutdown_connections 2024-05-14T16:59:20.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:18.126+0000 7fc4680f4b80 1 --2- 172.21.15.2:0/778700011 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ca073a3c20 0x55ca073a6040 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:20.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:18.126+0000 7fc4680f4b80 1 --2- 172.21.15.2:0/778700011 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ca07324750 0x55ca0733bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:20.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:18.126+0000 7fc4680f4b80 1 --2- 172.21.15.2:0/778700011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca0731eb60 0x55ca0733d5c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:20.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:18.126+0000 7fc436ffd700 0 client.25514 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:59:20.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:18.126+0000 7fc4680f4b80 1 --2- 172.21.15.2:0/778700011 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ca07323e30 0x55ca0733b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:20.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:18.126+0000 7fc4680f4b80 1 -- 172.21.15.2:0/778700011 >> 172.21.15.2:0/778700011 conn(0x55ca072493e0 msgr2=0x55ca07327a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:20.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:18.126+0000 7fc4680f4b80 1 -- 172.21.15.2:0/778700011 shutdown_connections 2024-05-14T16:59:20.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:18.126+0000 7fc4680f4b80 1 -- 172.21.15.2:0/778700011 wait complete. 2024-05-14T16:59:20.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:18.126+0000 7fc4680f4b80 20 client.25514 trim_cache size 0 max 16384 2024-05-14T16:59:20.303 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T16:59:20.308 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 36, 2024-05-14T16:59:20.315 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24479, 2024-05-14T16:59:20.320 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 14, 2024-05-14T16:59:20.325 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-05-14T16:59:20.329 INFO:teuthology.orchestra.run.smithi002.stdout: "38e5e2d2": { 2024-05-14T16:59:20.333 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 14, 2024-05-14T16:59:20.338 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-05-14T16:59:20.344 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-05-14T16:59:20.350 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-05-14T16:59:20.356 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-05-14T16:59:20.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T16:59:20.385 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-05-14T16:59:20.392 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 146.8, 2024-05-14T16:59:20.419 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-05-14T16:59:20.425 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-05-14T16:59:20.431 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-05-14T16:59:20.465 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-05-14T16:59:20.471 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-05-14T16:59:20.477 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-05-14T16:59:20.487 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:59:20.493 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:59:20.500 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:59:20.505 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:59:20.512 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:59:20.521 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T16:59:20.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:18.127+0000 7fc45d85e700 1 -- 172.21.15.2:0/3654299495 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc4440b2c60 msgr2=0x7fc4440b5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:20.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:18.127+0000 7fc45d85e700 1 --2- 172.21.15.2:0/3654299495 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc4440b2c60 0x7fc4440b5120 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc4500098e0 tx=0x7fc450005ab0 comp rx=0 tx=0).stop 2024-05-14T16:59:20.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:18.127+0000 7fc45d85e700 1 -- 172.21.15.2:0/3654299495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4580775a0 msgr2=0x7fc4581a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:20.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:18.127+0000 7fc45d85e700 1 --2- 172.21.15.2:0/3654299495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4580775a0 0x7fc4581a0690 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc44000c970 tx=0x7fc44000cc80 comp rx=0 tx=0).stop 2024-05-14T16:59:20.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:18.127+0000 7fc45d85e700 1 -- 172.21.15.2:0/3654299495 shutdown_connections 2024-05-14T16:59:20.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:18.127+0000 7fc45d85e700 1 --2- 172.21.15.2:0/3654299495 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc4440b2c60 0x7fc4440b5120 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:20.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:18.127+0000 7fc45d85e700 1 --2- 172.21.15.2:0/3654299495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc458114230 0x7fc4581a4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:20.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:18.127+0000 7fc45d85e700 1 --2- 172.21.15.2:0/3654299495 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc458075cb0 0x7fc4581a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:20.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:18.127+0000 7fc45d85e700 1 --2- 172.21.15.2:0/3654299495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4580775a0 0x7fc4581a0690 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:20.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:18.127+0000 7fc45d85e700 1 -- 172.21.15.2:0/3654299495 >> 172.21.15.2:0/3654299495 conn(0x7fc45807a6f0 msgr2=0x7fc458112e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:20.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:18.127+0000 7fc45d85e700 1 -- 172.21.15.2:0/3654299495 shutdown_connections 2024-05-14T16:59:20.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:18.127+0000 7fc45d85e700 1 -- 172.21.15.2:0/3654299495 wait complete. 2024-05-14T16:59:20.702 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:59:18 smithi002 ceph-mon[31538]: pgmap v141: 97 pgs: 97 active+clean; 2.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 MiB/s wr, 1.46k op/s 2024-05-14T16:59:20.712 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '38e5e2d2', quiesce took 10.1 seconds. Will release after: 16.700000000000003 2024-05-14T16:59:20.725 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:59:18 smithi151 ceph-mon[36066]: pgmap v141: 97 pgs: 97 active+clean; 2.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 MiB/s wr, 1.46k op/s 2024-05-14T16:59:20.733 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:59:18 smithi123 ceph-mon[35971]: pgmap v141: 97 pgs: 97 active+clean; 2.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 MiB/s wr, 1.46k op/s 2024-05-14T16:59:20.741 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:59:20 smithi123 ceph-mon[35971]: pgmap v142: 97 pgs: 97 active+clean; 2.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 MiB/s wr, 1.41k op/s 2024-05-14T16:59:20.750 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:59:20 smithi002 ceph-mon[31538]: pgmap v142: 97 pgs: 97 active+clean; 2.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 MiB/s wr, 1.41k op/s 2024-05-14T16:59:20.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:59:20 smithi151 ceph-mon[36066]: pgmap v142: 97 pgs: 97 active+clean; 2.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 MiB/s wr, 1.41k op/s 2024-05-14T16:59:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.283+0000 7f8a028fc640 1 -- 172.21.15.2:0/1062493841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89fc10f340 msgr2=0x7f89fc111730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:21.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.283+0000 7f8a028fc640 1 --2- 172.21.15.2:0/1062493841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89fc10f340 0x7f89fc111730 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f89f4009ca0 tx=0x7f89f402a950 comp rx=0 tx=0).stop 2024-05-14T16:59:21.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.283+0000 7f8a028fc640 1 -- 172.21.15.2:0/1062493841 shutdown_connections 2024-05-14T16:59:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.283+0000 7f8a028fc640 1 --2- 172.21.15.2:0/1062493841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89fc10f340 0x7f89fc111730 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:21.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.283+0000 7f8a028fc640 1 --2- 172.21.15.2:0/1062493841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89fc075670 0x7f89fc10ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.283+0000 7f8a028fc640 1 --2- 172.21.15.2:0/1062493841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89fc074d50 0x7f89fc075130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.283+0000 7f8a028fc640 1 -- 172.21.15.2:0/1062493841 >> 172.21.15.2:0/1062493841 conn(0x7f89fc079ee0 msgr2=0x7f89fc07a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.283+0000 7f8a028fc640 1 -- 172.21.15.2:0/1062493841 shutdown_connections 2024-05-14T16:59:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.283+0000 7f8a028fc640 1 -- 172.21.15.2:0/1062493841 wait complete. 2024-05-14T16:59:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.284+0000 7f8a028fc640 1 Processor -- start 2024-05-14T16:59:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.284+0000 7f8a028fc640 1 -- start start 2024-05-14T16:59:21.419 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.284+0000 7f8a028fc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89fc074d50 0x7f89fc19b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:21.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.284+0000 7f8a028fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89fc075670 0x7f89fc19b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.284+0000 7f8a028fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89fc10f340 0x7f89fc19fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.284+0000 7f8a028fc640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc114f20 con 0x7f89fc10f340 2024-05-14T16:59:21.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.284+0000 7f8a028fc640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc114da0 con 0x7f89fc075670 2024-05-14T16:59:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.284+0000 7f8a028fc640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc1150a0 con 0x7f89fc074d50 2024-05-14T16:59:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.285+0000 7f8a00e99640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89fc10f340 0x7f89fc19fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.285+0000 7f8a00e99640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89fc10f340 0x7f89fc19fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42990/0 (socket says 172.21.15.2:42990) 2024-05-14T16:59:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.285+0000 7f8a00e99640 1 -- 172.21.15.2:0/3275796192 learned_addr learned my addr 172.21.15.2:0/3275796192 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:59:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.285+0000 7f89f3fff640 1 --2- 172.21.15.2:0/3275796192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89fc075670 0x7f89fc19b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:21.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.285+0000 7f89fbfff640 1 --2- 172.21.15.2:0/3275796192 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89fc074d50 0x7f89fc19b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.285+0000 7f8a00e99640 1 -- 172.21.15.2:0/3275796192 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89fc074d50 msgr2=0x7f89fc19b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.285+0000 7f8a00e99640 1 --2- 172.21.15.2:0/3275796192 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89fc074d50 0x7f89fc19b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.285+0000 7f8a00e99640 1 -- 172.21.15.2:0/3275796192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89fc075670 msgr2=0x7f89fc19b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:21.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.285+0000 7f8a00e99640 1 --2- 172.21.15.2:0/3275796192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89fc075670 0x7f89fc19b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:21.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.285+0000 7f8a00e99640 1 -- 172.21.15.2:0/3275796192 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89fc1a02d0 con 0x7f89fc10f340 2024-05-14T16:59:21.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.285+0000 7f8a00e99640 1 --2- 172.21.15.2:0/3275796192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89fc10f340 0x7f89fc19fb50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f89f4004af0 tx=0x7f89f4031f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:21.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.285+0000 7f89f9ffb640 1 -- 172.21.15.2:0/3275796192 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f4034040 con 0x7f89fc10f340 2024-05-14T16:59:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.285+0000 7f89f9ffb640 1 -- 172.21.15.2:0/3275796192 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89f40308f0 con 0x7f89fc10f340 2024-05-14T16:59:21.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.285+0000 7f8a028fc640 1 -- 172.21.15.2:0/3275796192 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89fc1a0560 con 0x7f89fc10f340 2024-05-14T16:59:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.285+0000 7f89f9ffb640 1 -- 172.21.15.2:0/3275796192 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f4004600 con 0x7f89fc10f340 2024-05-14T16:59:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.285+0000 7f8a028fc640 1 -- 172.21.15.2:0/3275796192 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89fc1a7e00 con 0x7f89fc10f340 2024-05-14T16:59:21.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.289+0000 7f8a05615740 1 Processor -- start 2024-05-14T16:59:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.289+0000 7f89f9ffb640 1 -- 172.21.15.2:0/3275796192 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f89f4037020 con 0x7f89fc10f340 2024-05-14T16:59:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.289+0000 7f8a05615740 1 -- start start 2024-05-14T16:59:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.290+0000 7f8a05615740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56257b8458f0 0x56257b845cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.290+0000 7f8a05615740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56257b846210 0x56257b850590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.290+0000 7f8a05615740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56257b850ad0 0x56257b852ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.290+0000 7f8a05615740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56257b7b8c30 con 0x56257b8458f0 2024-05-14T16:59:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.290+0000 7f8a05615740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56257b7a87d0 con 0x56257b850ad0 2024-05-14T16:59:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.290+0000 7f8a05615740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56257b7ac810 con 0x56257b846210 2024-05-14T16:59:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.290+0000 7f89f3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56257b8458f0 0x56257b845cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:21.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.290+0000 7f89f3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56257b8458f0 0x56257b845cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43004/0 (socket says 172.21.15.2:43004) 2024-05-14T16:59:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.290+0000 7f89f3fff640 1 -- 172.21.15.2:0/4145016639 learned_addr learned my addr 172.21.15.2:0/4145016639 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:59:21.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.290+0000 7f89fbfff640 1 --2- 172.21.15.2:0/4145016639 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56257b846210 0x56257b850590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:21.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.290+0000 7f89f3fff640 1 --2- 172.21.15.2:0/4145016639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56257b850ad0 0x56257b852ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:21.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.290+0000 7f89f3fff640 1 -- 172.21.15.2:0/4145016639 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56257b846210 msgr2=0x56257b850590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.290+0000 7f89f9ffb640 1 --2- 172.21.15.2:0/3275796192 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f89e80b2a90 0x7f89e80b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.290+0000 7f89f3fff640 1 --2- 172.21.15.2:0/4145016639 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56257b846210 0x56257b850590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.290+0000 7f89f3fff640 1 -- 172.21.15.2:0/4145016639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56257b850ad0 msgr2=0x56257b852ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.290+0000 7f89f3fff640 1 --2- 172.21.15.2:0/4145016639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56257b850ad0 0x56257b852ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:21.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.290+0000 7f89f3fff640 1 -- 172.21.15.2:0/4145016639 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56257b4bd210 con 0x56257b8458f0 2024-05-14T16:59:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.290+0000 7f8a00e99640 1 --2- 172.21.15.2:0/3275796192 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f89e80b2a90 0x7f89e80b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:21.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.290+0000 7f89f3fff640 1 --2- 172.21.15.2:0/4145016639 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56257b8458f0 0x56257b845cd0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f89e40058c0 tx=0x7f89e4005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc9358b5a900395b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.291+0000 7f89f9ffb640 1 -- 172.21.15.2:0/3275796192 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f89f41030d0 con 0x7f89fc10f340 2024-05-14T16:59:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.291+0000 7f89f27fc640 1 -- 172.21.15.2:0/4145016639 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e4004050 con 0x56257b8458f0 2024-05-14T16:59:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.291+0000 7f89f27fc640 1 -- 172.21.15.2:0/4145016639 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89e40041f0 con 0x56257b8458f0 2024-05-14T16:59:22.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.291+0000 7f89f27fc640 1 -- 172.21.15.2:0/4145016639 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e400f760 con 0x56257b8458f0 2024-05-14T16:59:22.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.291+0000 7f8a05615740 1 -- 172.21.15.2:0/4145016639 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56257b8458f0 msgr2=0x56257b845cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.291+0000 7f8a00e99640 1 --2- 172.21.15.2:0/3275796192 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f89e80b2a90 0x7f89e80b4f50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f89f40c6570 tx=0x7f89f40c6490 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:22.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.291+0000 7f8a05615740 1 --2- 172.21.15.2:0/4145016639 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56257b8458f0 0x56257b845cd0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f89e40058c0 tx=0x7f89e4005850 comp rx=0 tx=0).stop 2024-05-14T16:59:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.292+0000 7f8a05615740 1 -- 172.21.15.2:0/4145016639 shutdown_connections 2024-05-14T16:59:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.292+0000 7f8a05615740 1 --2- 172.21.15.2:0/4145016639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56257b850ad0 0x56257b852ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:22.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.292+0000 7f8a05615740 1 --2- 172.21.15.2:0/4145016639 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56257b846210 0x56257b850590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:22.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.292+0000 7f8a05615740 1 --2- 172.21.15.2:0/4145016639 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56257b8458f0 0x56257b845cd0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:22.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.292+0000 7f8a05615740 1 -- 172.21.15.2:0/4145016639 >> 172.21.15.2:0/4145016639 conn(0x56257b841e00 msgr2=0x56257b8421e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.292+0000 7f8a05615740 1 -- 172.21.15.2:0/4145016639 shutdown_connections 2024-05-14T16:59:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.292+0000 7f8a05615740 1 -- 172.21.15.2:0/4145016639 wait complete. 2024-05-14T16:59:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.292+0000 7f8a05615740 1 -- start start 2024-05-14T16:59:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.292+0000 7f8a05615740 1 Processor -- start 2024-05-14T16:59:22.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.293+0000 7f8a05615740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56257b8458f0 0x56257b8503e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:22.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.293+0000 7f8a05615740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56257b846210 0x56257b84d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.293+0000 7f8a05615740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56257b850ad0 0x56257b84dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.293+0000 7f8a05615740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56257b48e790 con 0x56257b8458f0 2024-05-14T16:59:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.293+0000 7f8a05615740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56257b764890 con 0x56257b850ad0 2024-05-14T16:59:22.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.293+0000 7f8a05615740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56257b853740 con 0x56257b846210 2024-05-14T16:59:22.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.293+0000 7f8a05615740 10 client.0 _ll_register_callbacks cb 0x56257b767d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:59:22.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.293+0000 7f89fbfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56257b846210 0x56257b84d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:22.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.293+0000 7f89fbfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56257b846210 0x56257b84d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:58940/0 (socket says 172.21.15.2:58940) 2024-05-14T16:59:22.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.293+0000 7f89fbfff640 1 -- 172.21.15.2:0/4279822042 learned_addr learned my addr 172.21.15.2:0/4279822042 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:59:22.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.293+0000 7f89f3fff640 1 --2- 172.21.15.2:0/4279822042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56257b850ad0 0x56257b84dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:22.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.293+0000 7f89f3fff640 1 --2- 172.21.15.2:0/4279822042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56257b8458f0 0x56257b8503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:22.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.293+0000 7f89fbfff640 1 -- 172.21.15.2:0/4279822042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56257b850ad0 msgr2=0x56257b84dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:22.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.293+0000 7f89fbfff640 1 --2- 172.21.15.2:0/4279822042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56257b850ad0 0x56257b84dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.293+0000 7f89fbfff640 1 -- 172.21.15.2:0/4279822042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56257b8458f0 msgr2=0x56257b8503e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.293+0000 7f89fbfff640 1 --2- 172.21.15.2:0/4279822042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56257b8458f0 0x56257b8503e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.293+0000 7f89fbfff640 1 -- 172.21.15.2:0/4279822042 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56257b589620 con 0x56257b846210 2024-05-14T16:59:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.293+0000 7f89f3fff640 1 --2- 172.21.15.2:0/4279822042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56257b850ad0 0x56257b84dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:59:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.293+0000 7f89fbfff640 1 --2- 172.21.15.2:0/4279822042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56257b846210 0x56257b84d6d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f89ec00bad0 tx=0x7f89ec00bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=307e913cc8e8e7ad server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.293+0000 7f89db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-14T16:59:22.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.294+0000 7f89db7fe640 1 -- 172.21.15.2:0/4279822042 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ec005020 con 0x56257b846210 2024-05-14T16:59:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.294+0000 7f8a05615740 1 -- 172.21.15.2:0/4279822042 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89e40041f0 con 0x56257b846210 2024-05-14T16:59:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.294+0000 7f89db7fe640 1 -- 172.21.15.2:0/4279822042 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f89ec0051c0 con 0x56257b846210 2024-05-14T16:59:22.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.294+0000 7f89db7fe640 1 -- 172.21.15.2:0/4279822042 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f89ec005400 con 0x56257b846210 2024-05-14T16:59:22.400 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.294+0000 7f8a05615740 10 client.25513 fetch_fsmap learned FSMap version 36 2024-05-14T16:59:22.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.294+0000 7f8a05615740 1 -- 172.21.15.2:0/4279822042 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56257b85f170 con 0x56257b846210 2024-05-14T16:59:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.294+0000 7f89db7fe640 1 -- 172.21.15.2:0/4279822042 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89e40041f0 con 0x56257b846210 2024-05-14T16:59:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.294+0000 7f89db7fe640 1 -- 172.21.15.2:0/4279822042 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 36) ==== 8342+0+0 (secure 0 0 0) 0x7f89ec01b600 con 0x56257b846210 2024-05-14T16:59:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.294+0000 7f8a05615740 10 client.25513 fetch_fsmap finished waiting for FSMap version 36 2024-05-14T16:59:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.294+0000 7f8a05615740 10 client.25513 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:59:22.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.294+0000 7f8a05615740 20 client.25513 populate_metadata read hostname 'smithi002' 2024-05-14T16:59:22.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.294+0000 7f8a05615740 1 --2- 172.21.15.2:0/4279822042 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56257b860630 0x56257b862a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:22.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.294+0000 7f8a05615740 4 client.25513 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:59:22.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.294+0000 7f8a05615740 1 -- 172.21.15.2:0/4279822042 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56257b7bab80 con 0x56257b860630 2024-05-14T16:59:22.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.295+0000 7f89f3fff640 1 --2- 172.21.15.2:0/4279822042 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56257b860630 0x56257b862a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:22.550 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.295+0000 7f89f3fff640 1 --2- 172.21.15.2:0/4279822042 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56257b860630 0x56257b862a20 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8ac3cea04f92ada server_cookie=54034300743bfc6b in_seq=0 out_seq=0 2024-05-14T16:59:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.295+0000 7f89db7fe640 10 client.25513 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:59:22.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.296+0000 7f89db7fe640 1 -- 172.21.15.2:0/4279822042 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56257b7bab80 con 0x56257b860630 2024-05-14T16:59:22.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.296+0000 7f89db7fe640 10 client.25513 handle_command_reply: tid=0 2024-05-14T16:59:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.296+0000 7f8a05615740 1 client.25513 shutdown 2024-05-14T16:59:22.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.296+0000 7f8a05615740 1 -- 172.21.15.2:0/4279822042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56257b846210 msgr2=0x56257b84d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.296+0000 7f8a05615740 1 --2- 172.21.15.2:0/4279822042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56257b846210 0x56257b84d6d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f89ec00bad0 tx=0x7f89ec00bed0 comp rx=0 tx=0).stop 2024-05-14T16:59:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.296+0000 7f8a05615740 1 -- 172.21.15.2:0/4279822042 shutdown_connections 2024-05-14T16:59:22.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.296+0000 7f8a05615740 1 --2- 172.21.15.2:0/4279822042 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56257b860630 0x56257b862a20 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:22.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.296+0000 7f8a05615740 1 --2- 172.21.15.2:0/4279822042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56257b850ad0 0x56257b84dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.296+0000 7f8a05615740 1 --2- 172.21.15.2:0/4279822042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56257b846210 0x56257b84d6d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.296+0000 7f8a05615740 1 --2- 172.21.15.2:0/4279822042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56257b8458f0 0x56257b8503e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.296+0000 7f89db7fe640 0 client.25513 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:59:22.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.296+0000 7f8a05615740 1 -- 172.21.15.2:0/4279822042 >> 172.21.15.2:0/4279822042 conn(0x56257b841e00 msgr2=0x56257b841100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.296+0000 7f8a05615740 1 -- 172.21.15.2:0/4279822042 shutdown_connections 2024-05-14T16:59:22.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.296+0000 7f8a05615740 1 -- 172.21.15.2:0/4279822042 wait complete. 2024-05-14T16:59:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.296+0000 7f8a05615740 20 client.25513 trim_cache size 0 max 16384 2024-05-14T16:59:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.312+0000 7f8a05615740 1 Processor -- start 2024-05-14T16:59:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.312+0000 7f8a05615740 1 -- start start 2024-05-14T16:59:22.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.312+0000 7f8a05615740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56257b8458f0 0x56257b8682f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.312+0000 7f8a05615740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56257b846210 0x56257b868830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.312+0000 7f8a05615740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56257b850ad0 0x56257b84a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.312+0000 7f8a05615740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc1b3010 con 0x56257b8458f0 2024-05-14T16:59:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.312+0000 7f8a05615740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc1b08e0 con 0x56257b846210 2024-05-14T16:59:22.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.312+0000 7f8a05615740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc1ae730 con 0x56257b850ad0 2024-05-14T16:59:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.312+0000 7f89f3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56257b8458f0 0x56257b8682f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.312+0000 7f89f3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56257b8458f0 0x56257b8682f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43014/0 (socket says 172.21.15.2:43014) 2024-05-14T16:59:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.312+0000 7f89f3fff640 1 -- 172.21.15.2:0/3685079930 learned_addr learned my addr 172.21.15.2:0/3685079930 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:59:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.312+0000 7f89fbfff640 1 --2- 172.21.15.2:0/3685079930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56257b846210 0x56257b868830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:22.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.312+0000 7f89f3fff640 1 -- 172.21.15.2:0/3685079930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56257b850ad0 msgr2=0x56257b84a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.312+0000 7f89f3fff640 1 --2- 172.21.15.2:0/3685079930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56257b850ad0 0x56257b84a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.312+0000 7f89f3fff640 1 -- 172.21.15.2:0/3685079930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56257b846210 msgr2=0x56257b868830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.312+0000 7f89f3fff640 1 --2- 172.21.15.2:0/3685079930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56257b846210 0x56257b868830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:23.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.312+0000 7f89f3fff640 1 -- 172.21.15.2:0/3685079930 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56257b83c510 con 0x56257b8458f0 2024-05-14T16:59:23.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.312+0000 7f89f3fff640 1 --2- 172.21.15.2:0/3685079930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56257b8458f0 0x56257b8682f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f89e4005bc0 tx=0x7f89e4003d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c31ed7b4af4f104f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.313+0000 7f89f1ffb640 1 -- 172.21.15.2:0/3685079930 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e40041d0 con 0x56257b8458f0 2024-05-14T16:59:23.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.313+0000 7f89f1ffb640 1 -- 172.21.15.2:0/3685079930 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89e40054f0 con 0x56257b8458f0 2024-05-14T16:59:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.313+0000 7f89f1ffb640 1 -- 172.21.15.2:0/3685079930 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e4003af0 con 0x56257b8458f0 2024-05-14T16:59:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.313+0000 7f8a05615740 1 -- 172.21.15.2:0/3685079930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56257b8458f0 msgr2=0x56257b8682f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.313+0000 7f8a05615740 1 --2- 172.21.15.2:0/3685079930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56257b8458f0 0x56257b8682f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f89e4005bc0 tx=0x7f89e4003d20 comp rx=0 tx=0).stop 2024-05-14T16:59:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.313+0000 7f8a05615740 1 -- 172.21.15.2:0/3685079930 shutdown_connections 2024-05-14T16:59:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.313+0000 7f8a05615740 1 --2- 172.21.15.2:0/3685079930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56257b850ad0 0x56257b84a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:23.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.313+0000 7f8a05615740 1 --2- 172.21.15.2:0/3685079930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56257b846210 0x56257b868830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.313+0000 7f8a05615740 1 --2- 172.21.15.2:0/3685079930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56257b8458f0 0x56257b8682f0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.313+0000 7f8a05615740 1 -- 172.21.15.2:0/3685079930 >> 172.21.15.2:0/3685079930 conn(0x56257b841e00 msgr2=0x56257b85b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.313+0000 7f8a05615740 1 -- 172.21.15.2:0/3685079930 shutdown_connections 2024-05-14T16:59:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.313+0000 7f8a05615740 1 -- 172.21.15.2:0/3685079930 wait complete. 2024-05-14T16:59:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.314+0000 7f8a05615740 1 -- start start 2024-05-14T16:59:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.314+0000 7f8a05615740 1 Processor -- start 2024-05-14T16:59:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.314+0000 7f8a05615740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56257b8458f0 0x56257b858b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.314+0000 7f8a05615740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56257b846210 0x56257b859050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:23.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.314+0000 7f8a05615740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56257b850ad0 0x56257b856160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.314+0000 7f8a05615740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc1aeda0 con 0x56257b850ad0 2024-05-14T16:59:23.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.314+0000 7f8a05615740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc1b0210 con 0x56257b846210 2024-05-14T16:59:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.314+0000 7f8a05615740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc1af470 con 0x56257b8458f0 2024-05-14T16:59:23.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.314+0000 7f8a05615740 10 client.0 _ll_register_callbacks cb 0x7f89ec004480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:59:23.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.314+0000 7f89f3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56257b850ad0 0x56257b856160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.314+0000 7f89f3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56257b850ad0 0x56257b856160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43016/0 (socket says 172.21.15.2:43016) 2024-05-14T16:59:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.314+0000 7f89f3fff640 1 -- 172.21.15.2:0/2766965173 learned_addr learned my addr 172.21.15.2:0/2766965173 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:59:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.314+0000 7f89fbfff640 1 --2- 172.21.15.2:0/2766965173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56257b846210 0x56257b859050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.315+0000 7f89f3fff640 1 --2- 172.21.15.2:0/2766965173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56257b8458f0 0x56257b858b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.315+0000 7f89fbfff640 1 -- 172.21.15.2:0/2766965173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56257b8458f0 msgr2=0x56257b858b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:23.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.315+0000 7f89fbfff640 1 --2- 172.21.15.2:0/2766965173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56257b8458f0 0x56257b858b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:23.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.315+0000 7f89fbfff640 1 -- 172.21.15.2:0/2766965173 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56257b850ad0 msgr2=0x56257b856160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.315+0000 7f89fbfff640 1 --2- 172.21.15.2:0/2766965173 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56257b850ad0 0x56257b856160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:23.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.315+0000 7f89fbfff640 1 -- 172.21.15.2:0/2766965173 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56257b856960 con 0x56257b846210 2024-05-14T16:59:23.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.315+0000 7f89f3fff640 1 --2- 172.21.15.2:0/2766965173 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56257b850ad0 0x56257b856160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:59:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.315+0000 7f89fbfff640 1 --2- 172.21.15.2:0/2766965173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56257b846210 0x56257b859050 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f89ec01b9d0 tx=0x7f89ec003fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f211194b38e6bcc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:23.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.315+0000 7f89db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T16:59:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.315+0000 7f89db7fe640 1 -- 172.21.15.2:0/2766965173 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ec02a070 con 0x56257b846210 2024-05-14T16:59:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.315+0000 7f8a05615740 1 -- 172.21.15.2:0/2766965173 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89e40054f0 con 0x56257b846210 2024-05-14T16:59:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.315+0000 7f89db7fe640 1 -- 172.21.15.2:0/2766965173 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f89ec005020 con 0x56257b846210 2024-05-14T16:59:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.315+0000 7f89db7fe640 1 -- 172.21.15.2:0/2766965173 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f89ec005220 con 0x56257b846210 2024-05-14T16:59:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.315+0000 7f8a05615740 10 client.25532 fetch_fsmap learned FSMap version 36 2024-05-14T16:59:23.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.315+0000 7f8a05615740 1 -- 172.21.15.2:0/2766965173 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56257b8caaf0 con 0x56257b846210 2024-05-14T16:59:23.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.316+0000 7f89db7fe640 1 -- 172.21.15.2:0/2766965173 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89e40054f0 con 0x56257b846210 2024-05-14T16:59:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.316+0000 7f89db7fe640 1 -- 172.21.15.2:0/2766965173 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 36) ==== 8342+0+0 (secure 0 0 0) 0x7f89ec004cc0 con 0x56257b846210 2024-05-14T16:59:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.316+0000 7f8a05615740 10 client.25532 fetch_fsmap finished waiting for FSMap version 36 2024-05-14T16:59:23.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.316+0000 7f8a05615740 10 client.25532 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:59:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.316+0000 7f8a05615740 20 client.25532 populate_metadata read hostname 'smithi002' 2024-05-14T16:59:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.316+0000 7f8a05615740 1 --2- 172.21.15.2:0/2766965173 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56257b8cb320 0x56257b8cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.316+0000 7f8a05615740 4 client.25532 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:59:23.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.316+0000 7f8a05615740 1 -- 172.21.15.2:0/2766965173 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56257b7b4360 con 0x56257b8cb320 2024-05-14T16:59:23.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.316+0000 7f89f3fff640 1 --2- 172.21.15.2:0/2766965173 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56257b8cb320 0x56257b8cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:23.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.316+0000 7f89db7fe640 10 client.25532 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:59:23.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.316+0000 7f89f3fff640 1 --2- 172.21.15.2:0/2766965173 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56257b8cb320 0x56257b8cd740 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c94ece438472f96 server_cookie=3e0dfb7b97bbe8cc in_seq=0 out_seq=0 2024-05-14T16:59:23.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.332+0000 7f89db7fe640 1 -- 172.21.15.2:0/2766965173 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56257b7b4360 con 0x56257b8cb320 2024-05-14T16:59:23.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.332+0000 7f89db7fe640 10 client.25532 handle_command_reply: tid=0 2024-05-14T16:59:23.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.332+0000 7f8a05615740 1 client.25532 shutdown 2024-05-14T16:59:23.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.332+0000 7f8a05615740 1 -- 172.21.15.2:0/2766965173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56257b846210 msgr2=0x56257b859050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.332+0000 7f8a05615740 1 --2- 172.21.15.2:0/2766965173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56257b846210 0x56257b859050 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f89ec01b9d0 tx=0x7f89ec003fb0 comp rx=0 tx=0).stop 2024-05-14T16:59:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.332+0000 7f8a05615740 1 -- 172.21.15.2:0/2766965173 shutdown_connections 2024-05-14T16:59:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.332+0000 7f8a05615740 1 --2- 172.21.15.2:0/2766965173 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56257b8cb320 0x56257b8cd740 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:23.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.332+0000 7f8a05615740 1 --2- 172.21.15.2:0/2766965173 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56257b850ad0 0x56257b856160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:23.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.332+0000 7f8a05615740 1 --2- 172.21.15.2:0/2766965173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56257b846210 0x56257b859050 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:23.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.332+0000 7f89db7fe640 0 client.25532 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:59:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.332+0000 7f8a05615740 1 --2- 172.21.15.2:0/2766965173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56257b8458f0 0x56257b858b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.332+0000 7f8a05615740 1 -- 172.21.15.2:0/2766965173 >> 172.21.15.2:0/2766965173 conn(0x56257b841e00 msgr2=0x56257b85be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.332+0000 7f8a05615740 1 -- 172.21.15.2:0/2766965173 shutdown_connections 2024-05-14T16:59:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.332+0000 7f8a05615740 1 -- 172.21.15.2:0/2766965173 wait complete. 2024-05-14T16:59:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.332+0000 7f8a05615740 20 client.25532 trim_cache size 0 max 16384 2024-05-14T16:59:23.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T16:59:23.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T16:59:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T16:59:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.333+0000 7f8a028fc640 1 -- 172.21.15.2:0/3275796192 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f89e80b2a90 msgr2=0x7f89e80b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:23.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.333+0000 7f8a028fc640 1 --2- 172.21.15.2:0/3275796192 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f89e80b2a90 0x7f89e80b4f50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f89f40c6570 tx=0x7f89f40c6490 comp rx=0 tx=0).stop 2024-05-14T16:59:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.333+0000 7f8a028fc640 1 -- 172.21.15.2:0/3275796192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89fc10f340 msgr2=0x7f89fc19fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.333+0000 7f8a028fc640 1 --2- 172.21.15.2:0/3275796192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89fc10f340 0x7f89fc19fb50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f89f4004af0 tx=0x7f89f4031f50 comp rx=0 tx=0).stop 2024-05-14T16:59:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.333+0000 7f8a028fc640 1 -- 172.21.15.2:0/3275796192 shutdown_connections 2024-05-14T16:59:23.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.333+0000 7f8a028fc640 1 --2- 172.21.15.2:0/3275796192 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f89e80b2a90 0x7f89e80b4f50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:23.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.333+0000 7f8a028fc640 1 --2- 172.21.15.2:0/3275796192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89fc10f340 0x7f89fc19fb50 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:23.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.333+0000 7f8a028fc640 1 --2- 172.21.15.2:0/3275796192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89fc075670 0x7f89fc19b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:23.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.333+0000 7f8a028fc640 1 --2- 172.21.15.2:0/3275796192 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89fc074d50 0x7f89fc19b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.333+0000 7f8a028fc640 1 -- 172.21.15.2:0/3275796192 >> 172.21.15.2:0/3275796192 conn(0x7f89fc079ee0 msgr2=0x7f89fc0ff910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:23.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.333+0000 7f8a028fc640 1 -- 172.21.15.2:0/3275796192 shutdown_connections 2024-05-14T16:59:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:21.333+0000 7f8a028fc640 1 -- 172.21.15.2:0/3275796192 wait complete. 2024-05-14T16:59:24.025 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:59:22 smithi151 ceph-mon[36066]: pgmap v143: 97 pgs: 97 active+clean; 2.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 24 MiB/s wr, 1.05k op/s 2024-05-14T16:59:24.037 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:59:22 smithi123 ceph-mon[35971]: pgmap v143: 97 pgs: 97 active+clean; 2.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 24 MiB/s wr, 1.05k op/s 2024-05-14T16:59:24.064 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 245.0 s, 692.7 tps, lat 35.012 ms stddev 38.672 2024-05-14T16:59:24.072 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:59:22 smithi002 ceph-mon[31538]: pgmap v143: 97 pgs: 97 active+clean; 2.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 24 MiB/s wr, 1.05k op/s 2024-05-14T16:59:24.656 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:59:24 smithi151 ceph-mon[36066]: pgmap v144: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 25 MiB/s wr, 1.10k op/s 2024-05-14T16:59:24.670 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:59:24 smithi123 ceph-mon[35971]: pgmap v144: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 25 MiB/s wr, 1.10k op/s 2024-05-14T16:59:24.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:59:24 smithi002 ceph-mon[31538]: pgmap v144: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 25 MiB/s wr, 1.10k op/s 2024-05-14T16:59:26.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:59:26 smithi123 ceph-mon[35971]: pgmap v145: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 3.3 MiB/s wr, 348 op/s 2024-05-14T16:59:26.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:59:26 smithi002 ceph-mon[31538]: pgmap v145: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 3.3 MiB/s wr, 348 op/s 2024-05-14T16:59:26.751 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 250.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:59:26.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:59:26 smithi151 ceph-mon[36066]: pgmap v145: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 3.3 MiB/s wr, 348 op/s 2024-05-14T16:59:28.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:59:28 smithi123 ceph-mon[35971]: pgmap v146: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 3.4 MiB/s wr, 361 op/s 2024-05-14T16:59:28.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:59:28 smithi002 ceph-mon[31538]: pgmap v146: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 3.4 MiB/s wr, 361 op/s 2024-05-14T16:59:28.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:59:28 smithi151 ceph-mon[36066]: pgmap v146: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 3.4 MiB/s wr, 361 op/s 2024-05-14T16:59:30.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:59:30 smithi123 ceph-mon[35971]: pgmap v147: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 2.8 MiB/s wr, 280 op/s 2024-05-14T16:59:30.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:59:30 smithi002 ceph-mon[31538]: pgmap v147: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 2.8 MiB/s wr, 280 op/s 2024-05-14T16:59:30.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:59:30 smithi151 ceph-mon[36066]: pgmap v147: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 2.8 MiB/s wr, 280 op/s 2024-05-14T16:59:31.758 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 255.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:59:32.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:59:32 smithi123 ceph-mon[35971]: pgmap v148: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 1.1 MiB/s wr, 105 op/s 2024-05-14T16:59:32.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:59:32 smithi002 ceph-mon[31538]: pgmap v148: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 1.1 MiB/s wr, 105 op/s 2024-05-14T16:59:32.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:59:32 smithi151 ceph-mon[36066]: pgmap v148: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 1.1 MiB/s wr, 105 op/s 2024-05-14T16:59:34.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.435+0000 7f9538d1c640 1 -- 172.21.15.2:0/3150084677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9534103420 msgr2=0x7f953410f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.435+0000 7f9538d1c640 1 --2- 172.21.15.2:0/3150084677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9534103420 0x7f953410f930 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f952c009cd0 tx=0x7f952c02a8e0 comp rx=0 tx=0).stop 2024-05-14T16:59:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.435+0000 7f9538d1c640 1 -- 172.21.15.2:0/3150084677 shutdown_connections 2024-05-14T16:59:34.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.435+0000 7f9538d1c640 1 --2- 172.21.15.2:0/3150084677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9534103420 0x7f953410f930 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.435+0000 7f9538d1c640 1 --2- 172.21.15.2:0/3150084677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9534102a60 0x7f9534102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.435+0000 7f9538d1c640 1 --2- 172.21.15.2:0/3150084677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9534108a70 0x7f9534108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:34.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.435+0000 7f9538d1c640 1 -- 172.21.15.2:0/3150084677 >> 172.21.15.2:0/3150084677 conn(0x7f9534079ee0 msgr2=0x7f953407a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.435+0000 7f9538d1c640 1 -- 172.21.15.2:0/3150084677 shutdown_connections 2024-05-14T16:59:34.503 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.435+0000 7f9538d1c640 1 -- 172.21.15.2:0/3150084677 wait complete. 2024-05-14T16:59:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.436+0000 7f9538d1c640 1 Processor -- start 2024-05-14T16:59:34.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.436+0000 7f9538d1c640 1 -- start start 2024-05-14T16:59:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.436+0000 7f9538d1c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9534102a60 0x7f9534074d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.436+0000 7f9538d1c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9534103420 0x7f9534075290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:34.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.436+0000 7f9538d1c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9534108a70 0x7f95340757d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:34.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.436+0000 7f9538d1c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9534112d80 con 0x7f9534108a70 2024-05-14T16:59:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.436+0000 7f9538d1c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9534112c00 con 0x7f9534103420 2024-05-14T16:59:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.436+0000 7f9538d1c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9534112f00 con 0x7f9534102a60 2024-05-14T16:59:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.436+0000 7f9532d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9534108a70 0x7f95340757d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:34.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.436+0000 7f9532d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9534108a70 0x7f95340757d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44160/0 (socket says 172.21.15.2:44160) 2024-05-14T16:59:34.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.436+0000 7f9532d9d640 1 -- 172.21.15.2:0/819036711 learned_addr learned my addr 172.21.15.2:0/819036711 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:59:34.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.436+0000 7f9531d9b640 1 --2- 172.21.15.2:0/819036711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9534103420 0x7f9534075290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:34.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.436+0000 7f953259c640 1 --2- 172.21.15.2:0/819036711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9534102a60 0x7f9534074d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.437+0000 7f9532d9d640 1 -- 172.21.15.2:0/819036711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9534102a60 msgr2=0x7f9534074d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.437+0000 7f9532d9d640 1 --2- 172.21.15.2:0/819036711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9534102a60 0x7f9534074d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.437+0000 7f9532d9d640 1 -- 172.21.15.2:0/819036711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9534103420 msgr2=0x7f9534075290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:34.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.437+0000 7f9532d9d640 1 --2- 172.21.15.2:0/819036711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9534103420 0x7f9534075290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.437+0000 7f9532d9d640 1 -- 172.21.15.2:0/819036711 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95341a7660 con 0x7f9534108a70 2024-05-14T16:59:34.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.437+0000 7f9531d9b640 1 --2- 172.21.15.2:0/819036711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9534103420 0x7f9534075290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:59:34.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.437+0000 7f9532d9d640 1 --2- 172.21.15.2:0/819036711 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9534108a70 0x7f95340757d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f952c005e30 tx=0x7f952c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.437+0000 7f952b7fe640 1 -- 172.21.15.2:0/819036711 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c004240 con 0x7f9534108a70 2024-05-14T16:59:34.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.437+0000 7f952b7fe640 1 -- 172.21.15.2:0/819036711 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f952c031d00 con 0x7f9534108a70 2024-05-14T16:59:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.437+0000 7f9538d1c640 1 -- 172.21.15.2:0/819036711 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95341a78f0 con 0x7f9534108a70 2024-05-14T16:59:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.437+0000 7f952b7fe640 1 -- 172.21.15.2:0/819036711 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c030e30 con 0x7f9534108a70 2024-05-14T16:59:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.437+0000 7f9538d1c640 1 -- 172.21.15.2:0/819036711 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95341a7e40 con 0x7f9534108a70 2024-05-14T16:59:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.440+0000 7f953ba31740 1 Processor -- start 2024-05-14T16:59:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.441+0000 7f952b7fe640 1 -- 172.21.15.2:0/819036711 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f952c0305a0 con 0x7f9534108a70 2024-05-14T16:59:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.441+0000 7f953ba31740 1 -- start start 2024-05-14T16:59:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.441+0000 7f953ba31740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d071830 0x55ef7d071c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.441+0000 7f953ba31740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef7d072150 0x55ef7d084da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.441+0000 7f953ba31740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef7d0852e0 0x55ef7d0876d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.441+0000 7f953ba31740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ef7cfe4c30 con 0x55ef7d0852e0 2024-05-14T16:59:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.441+0000 7f953ba31740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ef7cfd47d0 con 0x55ef7d071830 2024-05-14T16:59:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.441+0000 7f953ba31740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ef7cfd8810 con 0x55ef7d072150 2024-05-14T16:59:34.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.441+0000 7f9531d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef7d0852e0 0x55ef7d0876d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.441+0000 7f9531d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef7d0852e0 0x55ef7d0876d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44176/0 (socket says 172.21.15.2:44176) 2024-05-14T16:59:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.441+0000 7f9531d9b640 1 -- 172.21.15.2:0/3242617094 learned_addr learned my addr 172.21.15.2:0/3242617094 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:59:34.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.442+0000 7f953259c640 1 --2- 172.21.15.2:0/3242617094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef7d072150 0x55ef7d084da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.442+0000 7f9531d9b640 1 --2- 172.21.15.2:0/3242617094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d071830 0x55ef7d071c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.442+0000 7f952b7fe640 1 --2- 172.21.15.2:0/819036711 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f95200b2a40 0x7f95200b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:34.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.442+0000 7f9531d9b640 1 -- 172.21.15.2:0/3242617094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef7d072150 msgr2=0x55ef7d084da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.442+0000 7f9531d9b640 1 --2- 172.21.15.2:0/3242617094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef7d072150 0x55ef7d084da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:34.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.442+0000 7f9531d9b640 1 -- 172.21.15.2:0/3242617094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d071830 msgr2=0x55ef7d071c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.442+0000 7f9531d9b640 1 --2- 172.21.15.2:0/3242617094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d071830 0x55ef7d071c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.442+0000 7f9531d9b640 1 -- 172.21.15.2:0/3242617094 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef7cce9210 con 0x55ef7d0852e0 2024-05-14T16:59:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.442+0000 7f9532d9d640 1 --2- 172.21.15.2:0/819036711 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f95200b2a40 0x7f95200b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:34.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.442+0000 7f952b7fe640 1 -- 172.21.15.2:0/819036711 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f952c02e030 con 0x7f9534108a70 2024-05-14T16:59:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.442+0000 7f9531d9b640 1 --2- 172.21.15.2:0/3242617094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef7d0852e0 0x55ef7d0876d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f951c012980 tx=0x7f951c00bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f846067b99b332c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:35.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.442+0000 7f9528ff9640 1 -- 172.21.15.2:0/3242617094 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c004220 con 0x55ef7d0852e0 2024-05-14T16:59:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.442+0000 7f9528ff9640 1 -- 172.21.15.2:0/3242617094 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f951c0043c0 con 0x55ef7d0852e0 2024-05-14T16:59:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.442+0000 7f9528ff9640 1 -- 172.21.15.2:0/3242617094 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c003d50 con 0x55ef7d0852e0 2024-05-14T16:59:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.443+0000 7f953ba31740 1 -- 172.21.15.2:0/3242617094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef7d0852e0 msgr2=0x55ef7d0876d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.443+0000 7f9532d9d640 1 --2- 172.21.15.2:0/819036711 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f95200b2a40 0x7f95200b4f00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f952c101170 tx=0x7f952c0905f0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:35.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.443+0000 7f953ba31740 1 --2- 172.21.15.2:0/3242617094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef7d0852e0 0x55ef7d0876d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f951c012980 tx=0x7f951c00bc40 comp rx=0 tx=0).stop 2024-05-14T16:59:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.443+0000 7f953ba31740 1 -- 172.21.15.2:0/3242617094 shutdown_connections 2024-05-14T16:59:35.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.443+0000 7f953ba31740 1 --2- 172.21.15.2:0/3242617094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef7d0852e0 0x55ef7d0876d0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.443+0000 7f953ba31740 1 --2- 172.21.15.2:0/3242617094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef7d072150 0x55ef7d084da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.443+0000 7f953ba31740 1 --2- 172.21.15.2:0/3242617094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d071830 0x55ef7d071c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.443+0000 7f953ba31740 1 -- 172.21.15.2:0/3242617094 >> 172.21.15.2:0/3242617094 conn(0x55ef7d06de00 msgr2=0x55ef7d06e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.443+0000 7f953ba31740 1 -- 172.21.15.2:0/3242617094 shutdown_connections 2024-05-14T16:59:35.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.443+0000 7f953ba31740 1 -- 172.21.15.2:0/3242617094 wait complete. 2024-05-14T16:59:35.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.443+0000 7f953ba31740 1 -- start start 2024-05-14T16:59:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.444+0000 7f953ba31740 1 Processor -- start 2024-05-14T16:59:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.444+0000 7f953ba31740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef7d071830 0x55ef7d084c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.444+0000 7f953ba31740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef7d072150 0x55ef7d081f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.444+0000 7f953ba31740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d0852e0 0x55ef7d0824e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.444+0000 7f953ba31740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ef7ccba790 con 0x55ef7d072150 2024-05-14T16:59:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.444+0000 7f953ba31740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ef7cf90890 con 0x55ef7d0852e0 2024-05-14T16:59:35.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.444+0000 7f953ba31740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ef7d087f50 con 0x55ef7d071830 2024-05-14T16:59:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.444+0000 7f953ba31740 10 client.0 _ll_register_callbacks cb 0x55ef7cf93d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:59:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.444+0000 7f953259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef7d072150 0x55ef7d081f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.444+0000 7f953259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef7d072150 0x55ef7d081f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44188/0 (socket says 172.21.15.2:44188) 2024-05-14T16:59:35.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.444+0000 7f953259c640 1 -- 172.21.15.2:0/3063354452 learned_addr learned my addr 172.21.15.2:0/3063354452 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:59:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.444+0000 7f9531d9b640 1 --2- 172.21.15.2:0/3063354452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d0852e0 0x55ef7d0824e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.444+0000 7f9531d9b640 1 --2- 172.21.15.2:0/3063354452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef7d071830 0x55ef7d084c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.444+0000 7f953259c640 1 -- 172.21.15.2:0/3063354452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef7d071830 msgr2=0x55ef7d084c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.444+0000 7f953259c640 1 --2- 172.21.15.2:0/3063354452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef7d071830 0x55ef7d084c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.444+0000 7f953259c640 1 -- 172.21.15.2:0/3063354452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d0852e0 msgr2=0x55ef7d0824e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:35.284 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.444+0000 7f953259c640 1 --2- 172.21.15.2:0/3063354452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d0852e0 0x55ef7d0824e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:35.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.444+0000 7f953259c640 1 -- 172.21.15.2:0/3063354452 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef7cdb5620 con 0x55ef7d072150 2024-05-14T16:59:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.444+0000 7f9531d9b640 1 --2- 172.21.15.2:0/3063354452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef7d071830 0x55ef7d084c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T16:59:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.445+0000 7f953259c640 1 --2- 172.21.15.2:0/3063354452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef7d072150 0x55ef7d081f60 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f952400b940 tx=0x7f952400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d0f3f28f3f6bee server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.445+0000 7f9519ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:59:35.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.445+0000 7f9519ffb640 1 -- 172.21.15.2:0/3063354452 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9524003ea0 con 0x55ef7d072150 2024-05-14T16:59:35.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.445+0000 7f9519ffb640 1 -- 172.21.15.2:0/3063354452 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f952401b640 con 0x55ef7d072150 2024-05-14T16:59:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.445+0000 7f953ba31740 1 -- 172.21.15.2:0/3063354452 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f951c0043c0 con 0x55ef7d072150 2024-05-14T16:59:35.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.445+0000 7f9519ffb640 1 -- 172.21.15.2:0/3063354452 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f952401b840 con 0x55ef7d072150 2024-05-14T16:59:35.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.445+0000 7f953ba31740 10 client.15831 fetch_fsmap learned FSMap version 36 2024-05-14T16:59:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.445+0000 7f953ba31740 1 -- 172.21.15.2:0/3063354452 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ef7d093aa0 con 0x55ef7d072150 2024-05-14T16:59:35.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.445+0000 7f9519ffb640 1 -- 172.21.15.2:0/3063354452 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f951c0043c0 con 0x55ef7d072150 2024-05-14T16:59:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.445+0000 7f9519ffb640 1 -- 172.21.15.2:0/3063354452 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 36) ==== 8342+0+0 (secure 0 0 0) 0x7f9524019020 con 0x55ef7d072150 2024-05-14T16:59:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.445+0000 7f953ba31740 10 client.15831 fetch_fsmap finished waiting for FSMap version 36 2024-05-14T16:59:35.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.445+0000 7f953ba31740 10 client.15831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:59:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.445+0000 7f953ba31740 20 client.15831 populate_metadata read hostname 'smithi002' 2024-05-14T16:59:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.445+0000 7f953ba31740 1 --2- 172.21.15.2:0/3063354452 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ef7d094fb0 0x55ef7d0973a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:35.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.445+0000 7f953ba31740 4 client.15831 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:59:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.445+0000 7f953ba31740 1 -- 172.21.15.2:0/3063354452 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef7cfe6b80 con 0x55ef7d094fb0 2024-05-14T16:59:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.446+0000 7f9531d9b640 1 --2- 172.21.15.2:0/3063354452 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ef7d094fb0 0x55ef7d0973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.446+0000 7f9519ffb640 10 client.15831 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:59:35.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.446+0000 7f9531d9b640 1 --2- 172.21.15.2:0/3063354452 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ef7d094fb0 0x55ef7d0973a0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfe02bf46d4e484e server_cookie=d410b155d795aee0 in_seq=0 out_seq=0 2024-05-14T16:59:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.447+0000 7f9519ffb640 1 -- 172.21.15.2:0/3063354452 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ef7cfe6b80 con 0x55ef7d094fb0 2024-05-14T16:59:35.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.447+0000 7f9519ffb640 10 client.15831 handle_command_reply: tid=0 2024-05-14T16:59:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.447+0000 7f953ba31740 1 client.15831 shutdown 2024-05-14T16:59:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.447+0000 7f953ba31740 1 -- 172.21.15.2:0/3063354452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef7d072150 msgr2=0x55ef7d081f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.447+0000 7f953ba31740 1 --2- 172.21.15.2:0/3063354452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef7d072150 0x55ef7d081f60 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f952400b940 tx=0x7f952400be00 comp rx=0 tx=0).stop 2024-05-14T16:59:35.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.447+0000 7f953ba31740 1 -- 172.21.15.2:0/3063354452 shutdown_connections 2024-05-14T16:59:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.447+0000 7f953ba31740 1 --2- 172.21.15.2:0/3063354452 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ef7d094fb0 0x55ef7d0973a0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.447+0000 7f953ba31740 1 --2- 172.21.15.2:0/3063354452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d0852e0 0x55ef7d0824e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.447+0000 7f953ba31740 1 --2- 172.21.15.2:0/3063354452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef7d072150 0x55ef7d081f60 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:35.550 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.447+0000 7f9519ffb640 0 client.15831 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:59:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.447+0000 7f953ba31740 1 --2- 172.21.15.2:0/3063354452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef7d071830 0x55ef7d084c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.447+0000 7f953ba31740 1 -- 172.21.15.2:0/3063354452 >> 172.21.15.2:0/3063354452 conn(0x55ef7d06de00 msgr2=0x55ef7d06d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.447+0000 7f953ba31740 1 -- 172.21.15.2:0/3063354452 shutdown_connections 2024-05-14T16:59:35.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.447+0000 7f953ba31740 1 -- 172.21.15.2:0/3063354452 wait complete. 2024-05-14T16:59:35.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.448+0000 7f953ba31740 20 client.15831 trim_cache size 0 max 16384 2024-05-14T16:59:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.463+0000 7f953ba31740 1 Processor -- start 2024-05-14T16:59:35.634 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.463+0000 7f953ba31740 1 -- start start 2024-05-14T16:59:35.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.463+0000 7f953ba31740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d071830 0x55ef7d090e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:35.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.463+0000 7f953ba31740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef7d072150 0x55ef7d0847a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:35.661 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.463+0000 7f953ba31740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef7d0852e0 0x55ef7d09cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.463+0000 7f953ba31740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f95341aeb90 con 0x55ef7d072150 2024-05-14T16:59:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.463+0000 7f953ba31740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f95341ac460 con 0x55ef7d071830 2024-05-14T16:59:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.463+0000 7f953ba31740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f95341aa2b0 con 0x55ef7d0852e0 2024-05-14T16:59:35.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.463+0000 7f953259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef7d072150 0x55ef7d0847a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:35.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.463+0000 7f953259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef7d072150 0x55ef7d0847a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44192/0 (socket says 172.21.15.2:44192) 2024-05-14T16:59:35.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.463+0000 7f953259c640 1 -- 172.21.15.2:0/3604826534 learned_addr learned my addr 172.21.15.2:0/3604826534 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:59:35.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.463+0000 7f9531d9b640 1 --2- 172.21.15.2:0/3604826534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef7d0852e0 0x55ef7d09cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:35.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.463+0000 7f9531d9b640 1 --2- 172.21.15.2:0/3604826534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d071830 0x55ef7d090e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:35.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.464+0000 7f953259c640 1 -- 172.21.15.2:0/3604826534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef7d0852e0 msgr2=0x55ef7d09cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:35.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.464+0000 7f953259c640 1 --2- 172.21.15.2:0/3604826534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef7d0852e0 0x55ef7d09cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:35.788 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.464+0000 7f953259c640 1 -- 172.21.15.2:0/3604826534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d071830 msgr2=0x55ef7d090e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.464+0000 7f953259c640 1 --2- 172.21.15.2:0/3604826534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d071830 0x55ef7d090e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:35.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.464+0000 7f953259c640 1 -- 172.21.15.2:0/3604826534 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef7d069830 con 0x55ef7d072150 2024-05-14T16:59:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.464+0000 7f953259c640 1 --2- 172.21.15.2:0/3604826534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef7d072150 0x55ef7d0847a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f9524004340 tx=0x7f9524025ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aef99b234e8cc36f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.464+0000 7f951bfff640 1 -- 172.21.15.2:0/3604826534 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952401b640 con 0x55ef7d072150 2024-05-14T16:59:35.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.464+0000 7f951bfff640 1 -- 172.21.15.2:0/3604826534 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9524003ea0 con 0x55ef7d072150 2024-05-14T16:59:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.464+0000 7f951bfff640 1 -- 172.21.15.2:0/3604826534 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9524005030 con 0x55ef7d072150 2024-05-14T16:59:35.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.465+0000 7f953ba31740 1 -- 172.21.15.2:0/3604826534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef7d072150 msgr2=0x55ef7d0847a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.465+0000 7f953ba31740 1 --2- 172.21.15.2:0/3604826534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef7d072150 0x55ef7d0847a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f9524004340 tx=0x7f9524025ed0 comp rx=0 tx=0).stop 2024-05-14T16:59:35.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.465+0000 7f953ba31740 1 -- 172.21.15.2:0/3604826534 shutdown_connections 2024-05-14T16:59:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.465+0000 7f953ba31740 1 --2- 172.21.15.2:0/3604826534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef7d0852e0 0x55ef7d09cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:35.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.465+0000 7f953ba31740 1 --2- 172.21.15.2:0/3604826534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef7d072150 0x55ef7d0847a0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:35.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.465+0000 7f953ba31740 1 --2- 172.21.15.2:0/3604826534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d071830 0x55ef7d090e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:35.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.465+0000 7f953ba31740 1 -- 172.21.15.2:0/3604826534 >> 172.21.15.2:0/3604826534 conn(0x55ef7d06de00 msgr2=0x55ef7d07d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:35.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.465+0000 7f953ba31740 1 -- 172.21.15.2:0/3604826534 shutdown_connections 2024-05-14T16:59:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.465+0000 7f953ba31740 1 -- 172.21.15.2:0/3604826534 wait complete. 2024-05-14T16:59:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.465+0000 7f953ba31740 1 -- start start 2024-05-14T16:59:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.465+0000 7f953ba31740 1 Processor -- start 2024-05-14T16:59:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.465+0000 7f953ba31740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef7d071830 0x55ef7d089180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.465+0000 7f953ba31740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef7d072150 0x55ef7d0896c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.465+0000 7f953ba31740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d0852e0 0x55ef7d0fe840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.465+0000 7f953ba31740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f95341aa920 con 0x55ef7d072150 2024-05-14T16:59:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.465+0000 7f953ba31740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f95341abd90 con 0x55ef7d0852e0 2024-05-14T16:59:36.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.465+0000 7f953ba31740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f95341aaff0 con 0x55ef7d071830 2024-05-14T16:59:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.465+0000 7f953ba31740 10 client.0 _ll_register_callbacks cb 0x7f9524004540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:59:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.465+0000 7f953259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef7d072150 0x55ef7d0896c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.465+0000 7f953259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef7d072150 0x55ef7d0896c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44200/0 (socket says 172.21.15.2:44200) 2024-05-14T16:59:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.465+0000 7f953259c640 1 -- 172.21.15.2:0/2939081255 learned_addr learned my addr 172.21.15.2:0/2939081255 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:59:36.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.466+0000 7f9531d9b640 1 --2- 172.21.15.2:0/2939081255 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef7d071830 0x55ef7d089180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:36.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.466+0000 7f9531d9b640 1 --2- 172.21.15.2:0/2939081255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d0852e0 0x55ef7d0fe840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:36.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.466+0000 7f953259c640 1 -- 172.21.15.2:0/2939081255 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef7d071830 msgr2=0x55ef7d089180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:36.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.466+0000 7f953259c640 1 --2- 172.21.15.2:0/2939081255 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef7d071830 0x55ef7d089180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.466+0000 7f953259c640 1 -- 172.21.15.2:0/2939081255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d0852e0 msgr2=0x55ef7d0fe840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:36.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.466+0000 7f953259c640 1 --2- 172.21.15.2:0/2939081255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d0852e0 0x55ef7d0fe840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.466+0000 7f953259c640 1 -- 172.21.15.2:0/2939081255 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef7d0fed80 con 0x55ef7d072150 2024-05-14T16:59:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.466+0000 7f9531d9b640 1 --2- 172.21.15.2:0/2939081255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d0852e0 0x55ef7d0fe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:59:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.466+0000 7f953259c640 1 --2- 172.21.15.2:0/2939081255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef7d072150 0x55ef7d0896c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9524024cf0 tx=0x7f952400b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c5dd28fb5dff2f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.466+0000 7f9519ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:59:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.466+0000 7f9519ffb640 1 -- 172.21.15.2:0/2939081255 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952401b640 con 0x55ef7d072150 2024-05-14T16:59:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.466+0000 7f9519ffb640 1 -- 172.21.15.2:0/2939081255 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f95240246b0 con 0x55ef7d072150 2024-05-14T16:59:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.466+0000 7f953ba31740 1 -- 172.21.15.2:0/2939081255 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9524003ea0 con 0x55ef7d072150 2024-05-14T16:59:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.466+0000 7f9519ffb640 1 -- 172.21.15.2:0/2939081255 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f95240248b0 con 0x55ef7d072150 2024-05-14T16:59:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.466+0000 7f953ba31740 10 client.15837 fetch_fsmap learned FSMap version 36 2024-05-14T16:59:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.466+0000 7f953ba31740 1 -- 172.21.15.2:0/2939081255 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ef7d0fff10 con 0x55ef7d072150 2024-05-14T16:59:36.177 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.466+0000 7f9519ffb640 1 -- 172.21.15.2:0/2939081255 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9524003ea0 con 0x55ef7d072150 2024-05-14T16:59:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.467+0000 7f9519ffb640 1 -- 172.21.15.2:0/2939081255 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 36) ==== 8342+0+0 (secure 0 0 0) 0x7f9524019070 con 0x55ef7d072150 2024-05-14T16:59:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.467+0000 7f953ba31740 10 client.15837 fetch_fsmap finished waiting for FSMap version 36 2024-05-14T16:59:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.467+0000 7f953ba31740 10 client.15837 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:59:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.467+0000 7f953ba31740 20 client.15837 populate_metadata read hostname 'smithi002' 2024-05-14T16:59:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.467+0000 7f953ba31740 1 --2- 172.21.15.2:0/2939081255 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ef7d100540 0x55ef7d102960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.467+0000 7f953ba31740 4 client.15837 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:59:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.467+0000 7f953ba31740 1 -- 172.21.15.2:0/2939081255 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ef7cfe0360 con 0x55ef7d100540 2024-05-14T16:59:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.467+0000 7f9531d9b640 1 --2- 172.21.15.2:0/2939081255 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ef7d100540 0x55ef7d102960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.467+0000 7f9519ffb640 10 client.15837 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:59:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.467+0000 7f9531d9b640 1 --2- 172.21.15.2:0/2939081255 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ef7d100540 0x55ef7d102960 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26a6c49502cdbd04 server_cookie=d3371c60c177402f in_seq=0 out_seq=0 2024-05-14T16:59:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.480+0000 7f9519ffb640 1 -- 172.21.15.2:0/2939081255 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ef7cfe0360 con 0x55ef7d100540 2024-05-14T16:59:36.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.480+0000 7f9519ffb640 10 client.15837 handle_command_reply: tid=0 2024-05-14T16:59:36.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.480+0000 7f953ba31740 1 client.15837 shutdown 2024-05-14T16:59:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.480+0000 7f953ba31740 1 -- 172.21.15.2:0/2939081255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef7d072150 msgr2=0x55ef7d0896c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:36.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.480+0000 7f953ba31740 1 --2- 172.21.15.2:0/2939081255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef7d072150 0x55ef7d0896c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9524024cf0 tx=0x7f952400b410 comp rx=0 tx=0).stop 2024-05-14T16:59:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.480+0000 7f953ba31740 1 -- 172.21.15.2:0/2939081255 shutdown_connections 2024-05-14T16:59:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.480+0000 7f953ba31740 1 --2- 172.21.15.2:0/2939081255 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ef7d100540 0x55ef7d102960 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.480+0000 7f953ba31740 1 --2- 172.21.15.2:0/2939081255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d0852e0 0x55ef7d0fe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.480+0000 7f9519ffb640 0 client.15837 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:59:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.480+0000 7f953ba31740 1 --2- 172.21.15.2:0/2939081255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef7d072150 0x55ef7d0896c0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.480+0000 7f953ba31740 1 --2- 172.21.15.2:0/2939081255 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef7d071830 0x55ef7d089180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.480+0000 7f953ba31740 1 -- 172.21.15.2:0/2939081255 >> 172.21.15.2:0/2939081255 conn(0x55ef7d06de00 msgr2=0x55ef7d073280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.480+0000 7f953ba31740 1 -- 172.21.15.2:0/2939081255 shutdown_connections 2024-05-14T16:59:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.480+0000 7f953ba31740 1 -- 172.21.15.2:0/2939081255 wait complete. 2024-05-14T16:59:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.480+0000 7f953ba31740 20 client.15837 trim_cache size 0 max 16384 2024-05-14T16:59:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T16:59:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T16:59:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T16:59:36.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.481+0000 7f9538d1c640 1 -- 172.21.15.2:0/819036711 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f95200b2a40 msgr2=0x7f95200b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.481+0000 7f9538d1c640 1 --2- 172.21.15.2:0/819036711 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f95200b2a40 0x7f95200b4f00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f952c101170 tx=0x7f952c0905f0 comp rx=0 tx=0).stop 2024-05-14T16:59:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.481+0000 7f9538d1c640 1 -- 172.21.15.2:0/819036711 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9534108a70 msgr2=0x7f95340757d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.481+0000 7f9538d1c640 1 --2- 172.21.15.2:0/819036711 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9534108a70 0x7f95340757d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f952c005e30 tx=0x7f952c003ba0 comp rx=0 tx=0).stop 2024-05-14T16:59:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.481+0000 7f9538d1c640 1 -- 172.21.15.2:0/819036711 shutdown_connections 2024-05-14T16:59:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.481+0000 7f9538d1c640 1 --2- 172.21.15.2:0/819036711 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f95200b2a40 0x7f95200b4f00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.481+0000 7f9538d1c640 1 --2- 172.21.15.2:0/819036711 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9534108a70 0x7f95340757d0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.481+0000 7f9538d1c640 1 --2- 172.21.15.2:0/819036711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9534103420 0x7f9534075290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.481+0000 7f9538d1c640 1 --2- 172.21.15.2:0/819036711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9534102a60 0x7f9534074d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.481+0000 7f9538d1c640 1 -- 172.21.15.2:0/819036711 >> 172.21.15.2:0/819036711 conn(0x7f9534079ee0 msgr2=0x7f95341002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.481+0000 7f9538d1c640 1 -- 172.21.15.2:0/819036711 shutdown_connections 2024-05-14T16:59:36.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:34.481+0000 7f9538d1c640 1 -- 172.21.15.2:0/819036711 wait complete. 2024-05-14T16:59:36.575 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:59:34 smithi002 ceph-mon[31538]: pgmap v149: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 1.6 MiB/s wr, 172 op/s 2024-05-14T16:59:36.583 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:59:34 smithi151 ceph-mon[36066]: pgmap v149: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 1.6 MiB/s wr, 172 op/s 2024-05-14T16:59:36.590 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:59:34 smithi123 ceph-mon[35971]: pgmap v149: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 1.6 MiB/s wr, 172 op/s 2024-05-14T16:59:36.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:59:36 smithi123 ceph-mon[35971]: pgmap v150: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-05-14T16:59:36.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:59:36 smithi002 ceph-mon[31538]: pgmap v150: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-05-14T16:59:36.751 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 260.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:59:36.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:59:36 smithi151 ceph-mon[36066]: pgmap v150: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-05-14T16:59:37.425 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T16:59:37.699 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:59:38.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:59:38 smithi123 ceph-mon[35971]: pgmap v151: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 1.2 MiB/s wr, 127 op/s 2024-05-14T16:59:38.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:59:38 smithi002 ceph-mon[31538]: pgmap v151: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 1.2 MiB/s wr, 127 op/s 2024-05-14T16:59:38.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:59:38 smithi151 ceph-mon[36066]: pgmap v151: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 1.2 MiB/s wr, 127 op/s 2024-05-14T16:59:39.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.411+0000 7f4f1e58a700 1 -- 172.21.15.2:0/3562875816 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f18108520 msgr2=0x7f4f18108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:39.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.411+0000 7f4f1e58a700 1 --2- 172.21.15.2:0/3562875816 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f18108520 0x7f4f18108900 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4f08009990 tx=0x7f4f08009ca0 comp rx=0 tx=0).stop 2024-05-14T16:59:39.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.411+0000 7f4f1e58a700 1 -- 172.21.15.2:0/3562875816 shutdown_connections 2024-05-14T16:59:39.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.411+0000 7f4f1e58a700 1 --2- 172.21.15.2:0/3562875816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f18102e00 0x7f4f1810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:39.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.411+0000 7f4f1e58a700 1 --2- 172.21.15.2:0/3562875816 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f18108520 0x7f4f18108900 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:39.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.411+0000 7f4f1e58a700 1 --2- 172.21.15.2:0/3562875816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f18102440 0x7f4f181028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:39.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.411+0000 7f4f1e58a700 1 -- 172.21.15.2:0/3562875816 >> 172.21.15.2:0/3562875816 conn(0x7f4f180793f0 msgr2=0x7f4f18079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:39.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.411+0000 7f4f1e58a700 1 -- 172.21.15.2:0/3562875816 shutdown_connections 2024-05-14T16:59:39.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.411+0000 7f4f1e58a700 1 -- 172.21.15.2:0/3562875816 wait complete. 2024-05-14T16:59:39.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.412+0000 7f4f1e58a700 1 Processor -- start 2024-05-14T16:59:39.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.412+0000 7f4f1e58a700 1 -- start start 2024-05-14T16:59:39.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.412+0000 7f4f1e58a700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f18102440 0x7f4f1819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:39.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.412+0000 7f4f1e58a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f18102e00 0x7f4f1819af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:39.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.412+0000 7f4f1e58a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f18108520 0x7f4f181a5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:39.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.412+0000 7f4f1e58a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4f181125d0 con 0x7f4f18102e00 2024-05-14T16:59:39.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.412+0000 7f4f1e58a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4f18112450 con 0x7f4f18108520 2024-05-14T16:59:39.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.412+0000 7f4f1e58a700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4f18112750 con 0x7f4f18102440 2024-05-14T16:59:39.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.412+0000 7f4f177fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f18102e00 0x7f4f1819af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:39.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.412+0000 7f4f177fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f18102e00 0x7f4f1819af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44210/0 (socket says 172.21.15.2:44210) 2024-05-14T16:59:39.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.412+0000 7f4f177fe700 1 -- 172.21.15.2:0/4009675463 learned_addr learned my addr 172.21.15.2:0/4009675463 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:59:39.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.412+0000 7f4f17fff700 1 --2- 172.21.15.2:0/4009675463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f18102440 0x7f4f1819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:39.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.412+0000 7f4f1cb27700 1 --2- 172.21.15.2:0/4009675463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f18108520 0x7f4f181a5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:39.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.413+0000 7f4f177fe700 1 -- 172.21.15.2:0/4009675463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f18102440 msgr2=0x7f4f1819a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:39.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.413+0000 7f4f177fe700 1 --2- 172.21.15.2:0/4009675463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f18102440 0x7f4f1819a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:39.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.413+0000 7f4f177fe700 1 -- 172.21.15.2:0/4009675463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f18108520 msgr2=0x7f4f181a5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:39.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.413+0000 7f4f177fe700 1 --2- 172.21.15.2:0/4009675463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f18108520 0x7f4f181a5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:39.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.413+0000 7f4f177fe700 1 -- 172.21.15.2:0/4009675463 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f181a5fb0 con 0x7f4f18102e00 2024-05-14T16:59:39.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.413+0000 7f4f177fe700 1 --2- 172.21.15.2:0/4009675463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f18102e00 0x7f4f1819af30 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c00d980 tx=0x7f4f0c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:39.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.413+0000 7f4f157fa700 1 -- 172.21.15.2:0/4009675463 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c016070 con 0x7f4f18102e00 2024-05-14T16:59:39.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.413+0000 7f4f1e58a700 1 -- 172.21.15.2:0/4009675463 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f181a6240 con 0x7f4f18102e00 2024-05-14T16:59:39.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.413+0000 7f4f157fa700 1 -- 172.21.15.2:0/4009675463 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f0c004830 con 0x7f4f18102e00 2024-05-14T16:59:39.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.413+0000 7f4f157fa700 1 -- 172.21.15.2:0/4009675463 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c005540 con 0x7f4f18102e00 2024-05-14T16:59:39.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.413+0000 7f4f1e58a700 1 -- 172.21.15.2:0/4009675463 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f181a6750 con 0x7f4f18102e00 2024-05-14T16:59:39.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.414+0000 7f4f1e58a700 1 -- 172.21.15.2:0/4009675463 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f1810a9b0 con 0x7f4f18102e00 2024-05-14T16:59:39.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.416+0000 7f4f157fa700 1 -- 172.21.15.2:0/4009675463 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f4f0c004220 con 0x7f4f18102e00 2024-05-14T16:59:39.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.417+0000 7f4f157fa700 1 --2- 172.21.15.2:0/4009675463 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f4f000b29f0 0x7f4f000b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:39.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.418+0000 7f4f157fa700 1 -- 172.21.15.2:0/4009675463 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f4f0c0e28f0 con 0x7f4f18102e00 2024-05-14T16:59:39.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.418+0000 7f4f17fff700 1 --2- 172.21.15.2:0/4009675463 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f4f000b29f0 0x7f4f000b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:39.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.418+0000 7f4f17fff700 1 --2- 172.21.15.2:0/4009675463 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f4f000b29f0 0x7f4f000b4eb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4f0800b5c0 tx=0x7f4f08005960 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:39.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.421+0000 7f4f157fa700 1 -- 172.21.15.2:0/4009675463 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4f0c01f020 con 0x7f4f18102e00 2024-05-14T16:59:39.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.691+0000 7f4f1e58a700 1 -- 172.21.15.2:0/4009675463 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f1806a230 con 0x7f4f18102e00 2024-05-14T16:59:39.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.691+0000 7f4f157fa700 1 -- 172.21.15.2:0/4009675463 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11851 (secure 0 0 0) 0x7f4f0c0aa2a0 con 0x7f4f18102e00 2024-05-14T16:59:39.864 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T16:59:39.874 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2275696923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2275696923},{"type":"v1","addr":"172.21.15.2:6835","nonce":2275696923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/716718761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":716718761},{"type":"v1","addr":"172.21.15.2:6839","nonce":716718761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/3644815179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3644815179},{"type":"v1","addr":"172.21.15.2:6841","nonce":3644815179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1880336731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1880336731},{"type":"v1","addr":"172.21.15.151:6833","nonce":1880336731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2415605548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2415605548},{"type":"v1","addr":"172.21.15.2:6837","nonce":2415605548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2635167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2635167},{"type":"v1","addr":"172.21.15.123:6833","nonce":2635167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:51:34.106919+0000","modified":"2024-05-14T16:58:33.451604+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24455,"mds_2":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24415":{"gid":24415,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/762044339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":762044339},{"type":"v1","addr":"172.21.15.151:6835","nonce":762044339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/132043151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":132043151},{"type":"v1","addr":"172.21.15.151:6837","nonce":132043151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/4107834375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":4107834375},{"type":"v1","addr":"172.21.15.151:6839","nonce":4107834375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.123:6835/1198670013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1198670013},{"type":"v1","addr":"172.21.15.123:6835","nonce":1198670013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/333426645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":333426645},{"type":"v1","addr":"172.21.15.123:6837","nonce":333426645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/580275877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":580275877},{"type":"v1","addr":"172.21.15.123:6839","nonce":580275877}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24455,24479,24431]},"id":1}]} 2024-05-14T16:59:39.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.694+0000 7f4f1e58a700 1 -- 172.21.15.2:0/4009675463 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f4f000b29f0 msgr2=0x7f4f000b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:39.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.694+0000 7f4f1e58a700 1 --2- 172.21.15.2:0/4009675463 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f4f000b29f0 0x7f4f000b4eb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4f0800b5c0 tx=0x7f4f08005960 comp rx=0 tx=0).stop 2024-05-14T16:59:39.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.694+0000 7f4f1e58a700 1 -- 172.21.15.2:0/4009675463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f18102e00 msgr2=0x7f4f1819af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:39.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.694+0000 7f4f1e58a700 1 --2- 172.21.15.2:0/4009675463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f18102e00 0x7f4f1819af30 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c00d980 tx=0x7f4f0c00dc90 comp rx=0 tx=0).stop 2024-05-14T16:59:39.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.694+0000 7f4f1e58a700 1 -- 172.21.15.2:0/4009675463 shutdown_connections 2024-05-14T16:59:39.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.694+0000 7f4f1e58a700 1 --2- 172.21.15.2:0/4009675463 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f4f000b29f0 0x7f4f000b4eb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:39.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.694+0000 7f4f1e58a700 1 --2- 172.21.15.2:0/4009675463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f18108520 0x7f4f181a5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:39.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.694+0000 7f4f1e58a700 1 --2- 172.21.15.2:0/4009675463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f18102440 0x7f4f1819a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:39.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.694+0000 7f4f1e58a700 1 --2- 172.21.15.2:0/4009675463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f18102e00 0x7f4f1819af30 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:39.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.694+0000 7f4f1e58a700 1 -- 172.21.15.2:0/4009675463 >> 172.21.15.2:0/4009675463 conn(0x7f4f180793f0 msgr2=0x7f4f181012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:40.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.694+0000 7f4f1e58a700 1 -- 172.21.15.2:0/4009675463 shutdown_connections 2024-05-14T16:59:40.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:39.694+0000 7f4f1e58a700 1 -- 172.21.15.2:0/4009675463 wait complete. 2024-05-14T16:59:40.019 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 36 2024-05-14T16:59:40.412 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id 38e5e2d2 --release --await' 2024-05-14T16:59:40.425 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id 38e5e2d2 --release --await 2024-05-14T16:59:40.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:59:40 smithi123 ceph-mon[35971]: pgmap v152: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 1.1 MiB/s wr, 118 op/s 2024-05-14T16:59:40.677 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:59:40 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/4009675463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:59:40.692 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T16:59:40.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:59:40 smithi002 ceph-mon[31538]: pgmap v152: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 1.1 MiB/s wr, 118 op/s 2024-05-14T16:59:40.753 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:59:40 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/4009675463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:59:40.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:59:40 smithi151 ceph-mon[36066]: pgmap v152: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 1.1 MiB/s wr, 118 op/s 2024-05-14T16:59:40.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:59:40 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/4009675463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T16:59:41.751 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 265.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T16:59:42.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.357+0000 7faaf4147700 1 -- 172.21.15.2:0/2882929488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaec1041a0 msgr2=0x7faaec1081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:42.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.357+0000 7faaf4147700 1 --2- 172.21.15.2:0/2882929488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaec1041a0 0x7faaec1081c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7faadc0099c0 tx=0x7faadc009cd0 comp rx=0 tx=0).stop 2024-05-14T16:59:42.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.358+0000 7faaf4147700 1 -- 172.21.15.2:0/2882929488 shutdown_connections 2024-05-14T16:59:42.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.358+0000 7faaf4147700 1 --2- 172.21.15.2:0/2882929488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faaec1088f0 0x7faaec110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:42.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.358+0000 7faaf4147700 1 --2- 172.21.15.2:0/2882929488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faaec1037f0 0x7faaec103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:42.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.358+0000 7faaf4147700 1 --2- 172.21.15.2:0/2882929488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaec1041a0 0x7faaec1081c0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:42.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.358+0000 7faaf4147700 1 -- 172.21.15.2:0/2882929488 >> 172.21.15.2:0/2882929488 conn(0x7faaec07a6a0 msgr2=0x7faaec07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:42.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.358+0000 7faaf4147700 1 -- 172.21.15.2:0/2882929488 shutdown_connections 2024-05-14T16:59:42.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.358+0000 7faaf4147700 1 -- 172.21.15.2:0/2882929488 wait complete. 2024-05-14T16:59:42.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.358+0000 7faaf4147700 1 Processor -- start 2024-05-14T16:59:42.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.359+0000 7faaf4147700 1 -- start start 2024-05-14T16:59:42.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.359+0000 7faaf4147700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faaec1037f0 0x7faaec19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:42.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.359+0000 7faaf4147700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faaec1041a0 0x7faaec19fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:42.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.359+0000 7faaf4147700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaec1088f0 0x7faaec199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:42.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.359+0000 7faaf4147700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7faaec1136d0 con 0x7faaec1088f0 2024-05-14T16:59:42.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.359+0000 7faaf4147700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faaec113550 con 0x7faaec1037f0 2024-05-14T16:59:42.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.359+0000 7faaf4147700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faaec113850 con 0x7faaec1041a0 2024-05-14T16:59:42.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.359+0000 7faaf26e4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaec1088f0 0x7faaec199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:42.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.359+0000 7faaf26e4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaec1088f0 0x7faaec199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44214/0 (socket says 172.21.15.2:44214) 2024-05-14T16:59:42.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.359+0000 7faaf26e4700 1 -- 172.21.15.2:0/1194252214 learned_addr learned my addr 172.21.15.2:0/1194252214 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:59:42.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.359+0000 7faaf16e2700 1 --2- 172.21.15.2:0/1194252214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faaec1041a0 0x7faaec19fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:42.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.359+0000 7faaf1ee3700 1 --2- 172.21.15.2:0/1194252214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faaec1037f0 0x7faaec19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:42.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.359+0000 7faaf26e4700 1 -- 172.21.15.2:0/1194252214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faaec1041a0 msgr2=0x7faaec19fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:42.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.359+0000 7faaf26e4700 1 --2- 172.21.15.2:0/1194252214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faaec1041a0 0x7faaec19fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:42.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.359+0000 7faaf26e4700 1 -- 172.21.15.2:0/1194252214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faaec1037f0 msgr2=0x7faaec19f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:42.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.359+0000 7faaf26e4700 1 --2- 172.21.15.2:0/1194252214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faaec1037f0 0x7faaec19f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:42.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.359+0000 7faaf26e4700 1 -- 172.21.15.2:0/1194252214 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaec19a0f0 con 0x7faaec1088f0 2024-05-14T16:59:42.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.360+0000 7faaf26e4700 1 --2- 172.21.15.2:0/1194252214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaec1088f0 0x7faaec199990 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7faae400ef30 tx=0x7faae400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:42.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.360+0000 7faae2ffd700 1 -- 172.21.15.2:0/1194252214 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae40092e0 con 0x7faaec1088f0 2024-05-14T16:59:42.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.360+0000 7faae2ffd700 1 -- 172.21.15.2:0/1194252214 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faae4004bb0 con 0x7faaec1088f0 2024-05-14T16:59:42.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.360+0000 7faaf4147700 1 -- 172.21.15.2:0/1194252214 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaec19a380 con 0x7faaec1088f0 2024-05-14T16:59:42.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.360+0000 7faae2ffd700 1 -- 172.21.15.2:0/1194252214 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae4004020 con 0x7faaec1088f0 2024-05-14T16:59:42.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.360+0000 7faaf4147700 1 -- 172.21.15.2:0/1194252214 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaec1a66a0 con 0x7faaec1088f0 2024-05-14T16:59:42.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.362+0000 7faae2ffd700 1 -- 172.21.15.2:0/1194252214 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7faae401b030 con 0x7faaec1088f0 2024-05-14T16:59:42.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.363+0000 7faae2ffd700 1 --2- 172.21.15.2:0/1194252214 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7faad80b2a70 0x7faad80b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:42.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.363+0000 7faae2ffd700 1 -- 172.21.15.2:0/1194252214 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7faae40e1fe0 con 0x7faaec1088f0 2024-05-14T16:59:42.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.363+0000 7faaf1ee3700 1 --2- 172.21.15.2:0/1194252214 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7faad80b2a70 0x7faad80b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:42.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.363+0000 7faafe9ddb80 1 Processor -- start 2024-05-14T16:59:42.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.364+0000 7faaf1ee3700 1 --2- 172.21.15.2:0/1194252214 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7faad80b2a70 0x7faad80b4f30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faaec19aad0 tx=0x7faae8009260 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:42.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.364+0000 7faafe9ddb80 1 -- start start 2024-05-14T16:59:42.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.364+0000 7faafe9ddb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f19d52420 0x555f19d52800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:42.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.364+0000 7faafe9ddb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f19d52d40 0x555f19d4dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:42.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.364+0000 7faafe9ddb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f19d4e2c0 0x555f19d4e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:42.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.364+0000 7faafe9ddb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x555f19cabe70 con 0x555f19d52d40 2024-05-14T16:59:42.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.364+0000 7faafe9ddb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555f19c9ba10 con 0x555f19d4e2c0 2024-05-14T16:59:42.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.364+0000 7faafe9ddb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555f19c9fa50 con 0x555f19d52420 2024-05-14T16:59:42.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.364+0000 7faaf16e2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f19d52d40 0x555f19d4dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:42.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.364+0000 7faaf16e2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f19d52d40 0x555f19d4dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44218/0 (socket says 172.21.15.2:44218) 2024-05-14T16:59:42.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.364+0000 7faaf16e2700 1 -- 172.21.15.2:0/1132083163 learned_addr learned my addr 172.21.15.2:0/1132083163 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:59:42.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.364+0000 7faaf1ee3700 1 --2- 172.21.15.2:0/1132083163 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f19d52420 0x555f19d52800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:42.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.364+0000 7faaf26e4700 1 --2- 172.21.15.2:0/1132083163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f19d4e2c0 0x555f19d4e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:42.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.364+0000 7faaf16e2700 1 -- 172.21.15.2:0/1132083163 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f19d52420 msgr2=0x555f19d52800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:42.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.364+0000 7faaf16e2700 1 --2- 172.21.15.2:0/1132083163 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f19d52420 0x555f19d52800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:42.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.364+0000 7faaf16e2700 1 -- 172.21.15.2:0/1132083163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f19d4e2c0 msgr2=0x555f19d4e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:42.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.364+0000 7faaf16e2700 1 --2- 172.21.15.2:0/1132083163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f19d4e2c0 0x555f19d4e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:43.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.364+0000 7faaf16e2700 1 -- 172.21.15.2:0/1132083163 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f19c450b0 con 0x555f19d52d40 2024-05-14T16:59:43.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.365+0000 7faaf16e2700 1 --2- 172.21.15.2:0/1132083163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f19d52d40 0x555f19d4dca0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7faadc0099c0 tx=0x7faadc0036c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e27708e260037bf server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:43.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.365+0000 7faad7fff700 1 -- 172.21.15.2:0/1132083163 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc028070 con 0x555f19d52d40 2024-05-14T16:59:43.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.365+0000 7faad7fff700 1 -- 172.21.15.2:0/1132083163 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faadc0095d0 con 0x555f19d52d40 2024-05-14T16:59:43.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.365+0000 7faad7fff700 1 -- 172.21.15.2:0/1132083163 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc00b9f0 con 0x555f19d52d40 2024-05-14T16:59:43.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.366+0000 7faafe9ddb80 1 -- 172.21.15.2:0/1132083163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f19d52d40 msgr2=0x555f19d4dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:43.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.366+0000 7faafe9ddb80 1 --2- 172.21.15.2:0/1132083163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f19d52d40 0x555f19d4dca0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7faadc0099c0 tx=0x7faadc0036c0 comp rx=0 tx=0).stop 2024-05-14T16:59:43.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.366+0000 7faafe9ddb80 1 -- 172.21.15.2:0/1132083163 shutdown_connections 2024-05-14T16:59:43.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.366+0000 7faafe9ddb80 1 --2- 172.21.15.2:0/1132083163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f19d4e2c0 0x555f19d4e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:43.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.366+0000 7faafe9ddb80 1 --2- 172.21.15.2:0/1132083163 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f19d52420 0x555f19d52800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:43.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.366+0000 7faafe9ddb80 1 --2- 172.21.15.2:0/1132083163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f19d52d40 0x555f19d4dca0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:43.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.366+0000 7faafe9ddb80 1 -- 172.21.15.2:0/1132083163 >> 172.21.15.2:0/1132083163 conn(0x555f19c7aeb0 msgr2=0x555f19d484a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:43.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.366+0000 7faafe9ddb80 1 -- 172.21.15.2:0/1132083163 shutdown_connections 2024-05-14T16:59:43.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.366+0000 7faafe9ddb80 1 -- 172.21.15.2:0/1132083163 wait complete. 2024-05-14T16:59:43.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.366+0000 7faafe9ddb80 1 -- start start 2024-05-14T16:59:43.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.366+0000 7faafe9ddb80 1 Processor -- start 2024-05-14T16:59:43.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.367+0000 7faafe9ddb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f19d4e2c0 0x555f19d59ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:43.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.367+0000 7faafe9ddb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f19d52420 0x555f19d56ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:43.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.367+0000 7faafe9ddb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f19d52d40 0x555f19d57400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:43.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.367+0000 7faafe9ddb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x555f19ba20a0 con 0x555f19d4e2c0 2024-05-14T16:59:43.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.367+0000 7faafe9ddb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555f19c03ea0 con 0x555f19d52420 2024-05-14T16:59:43.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.367+0000 7faafe9ddb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555f19d5a820 con 0x555f19d52d40 2024-05-14T16:59:43.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.367+0000 7faafe9ddb80 10 client.0 _ll_register_callbacks cb 0x555f19c36a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:59:43.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.367+0000 7faaf1ee3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f19d4e2c0 0x555f19d59ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:43.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.367+0000 7faaf1ee3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f19d4e2c0 0x555f19d59ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44226/0 (socket says 172.21.15.2:44226) 2024-05-14T16:59:43.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.367+0000 7faaf1ee3700 1 -- 172.21.15.2:0/877731895 learned_addr learned my addr 172.21.15.2:0/877731895 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:59:43.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.367+0000 7faaf16e2700 1 --2- 172.21.15.2:0/877731895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f19d52420 0x555f19d56ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:43.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.367+0000 7faaf26e4700 1 --2- 172.21.15.2:0/877731895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f19d52d40 0x555f19d57400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:43.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.367+0000 7faaf1ee3700 1 -- 172.21.15.2:0/877731895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f19d52d40 msgr2=0x555f19d57400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:43.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.367+0000 7faaf1ee3700 1 --2- 172.21.15.2:0/877731895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f19d52d40 0x555f19d57400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:43.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.367+0000 7faaf1ee3700 1 -- 172.21.15.2:0/877731895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f19d52420 msgr2=0x555f19d56ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:43.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.367+0000 7faaf1ee3700 1 --2- 172.21.15.2:0/877731895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f19d52420 0x555f19d56ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:43.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.367+0000 7faaf1ee3700 1 -- 172.21.15.2:0/877731895 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f19beb9d0 con 0x555f19d4e2c0 2024-05-14T16:59:43.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.367+0000 7faaf1ee3700 1 --2- 172.21.15.2:0/877731895 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f19d4e2c0 0x555f19d59ad0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7faae80045b0 tx=0x7faae80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b1232838356a96 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:43.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.367+0000 7faad57fa700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:59:43.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.367+0000 7faad57fa700 1 -- 172.21.15.2:0/877731895 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae8004830 con 0x555f19d4e2c0 2024-05-14T16:59:43.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.367+0000 7faafe9ddb80 1 -- 172.21.15.2:0/877731895 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faadc0095d0 con 0x555f19d4e2c0 2024-05-14T16:59:43.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.367+0000 7faad57fa700 1 -- 172.21.15.2:0/877731895 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7faae80049d0 con 0x555f19d4e2c0 2024-05-14T16:59:43.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.368+0000 7faad57fa700 1 -- 172.21.15.2:0/877731895 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7faae8004bd0 con 0x555f19d4e2c0 2024-05-14T16:59:43.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.368+0000 7faafe9ddb80 10 client.15855 fetch_fsmap learned FSMap version 36 2024-05-14T16:59:43.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.368+0000 7faafe9ddb80 1 -- 172.21.15.2:0/877731895 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x555f19c55860 con 0x555f19d4e2c0 2024-05-14T16:59:43.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.368+0000 7faad57fa700 1 -- 172.21.15.2:0/877731895 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faadc0095d0 con 0x555f19d4e2c0 2024-05-14T16:59:43.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.368+0000 7faad57fa700 1 -- 172.21.15.2:0/877731895 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 36) ==== 8342+0+0 (secure 0 0 0) 0x7faae801d610 con 0x555f19d4e2c0 2024-05-14T16:59:43.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.368+0000 7faafe9ddb80 10 client.15855 fetch_fsmap finished waiting for FSMap version 36 2024-05-14T16:59:43.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.368+0000 7faafe9ddb80 10 client.15855 resolve_mds: validated gid 24479 aka daemon mds.b 2024-05-14T16:59:43.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.368+0000 7faafe9ddb80 20 client.15855 populate_metadata read hostname 'smithi002' 2024-05-14T16:59:43.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.368+0000 7faafe9ddb80 1 --2- 172.21.15.2:0/877731895 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x555f19d67350 0x555f19d69740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:43.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.368+0000 7faafe9ddb80 4 client.15855 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:59:43.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.368+0000 7faafe9ddb80 1 -- 172.21.15.2:0/877731895 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f19caddc0 con 0x555f19d67350 2024-05-14T16:59:43.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.368+0000 7faaf16e2700 1 --2- 172.21.15.2:0/877731895 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x555f19d67350 0x555f19d69740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:43.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.369+0000 7faaf16e2700 1 --2- 172.21.15.2:0/877731895 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x555f19d67350 0x555f19d69740 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=249a185f6f0492f3 server_cookie=3d4e05c4a5823c21 in_seq=0 out_seq=0 2024-05-14T16:59:43.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.369+0000 7faad57fa700 10 client.15855 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:59:43.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.370+0000 7faad57fa700 1 -- 172.21.15.2:0/877731895 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555f19caddc0 con 0x555f19d67350 2024-05-14T16:59:43.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.370+0000 7faad57fa700 10 client.15855 handle_command_reply: tid=0 2024-05-14T16:59:43.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.370+0000 7faafe9ddb80 1 client.15855 shutdown 2024-05-14T16:59:43.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.370+0000 7faafe9ddb80 1 -- 172.21.15.2:0/877731895 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f19d4e2c0 msgr2=0x555f19d59ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:43.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.370+0000 7faafe9ddb80 1 --2- 172.21.15.2:0/877731895 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f19d4e2c0 0x555f19d59ad0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7faae80045b0 tx=0x7faae80047c0 comp rx=0 tx=0).stop 2024-05-14T16:59:43.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.370+0000 7faafe9ddb80 1 -- 172.21.15.2:0/877731895 shutdown_connections 2024-05-14T16:59:43.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.370+0000 7faafe9ddb80 1 --2- 172.21.15.2:0/877731895 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x555f19d67350 0x555f19d69740 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:43.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.370+0000 7faad57fa700 0 client.15855 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:59:43.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.370+0000 7faafe9ddb80 1 --2- 172.21.15.2:0/877731895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f19d52d40 0x555f19d57400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:43.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.370+0000 7faafe9ddb80 1 --2- 172.21.15.2:0/877731895 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f19d4e2c0 0x555f19d59ad0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:43.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.370+0000 7faafe9ddb80 1 --2- 172.21.15.2:0/877731895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f19d52420 0x555f19d56ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:43.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.370+0000 7faafe9ddb80 1 -- 172.21.15.2:0/877731895 >> 172.21.15.2:0/877731895 conn(0x555f19c7aeb0 msgr2=0x555f19d42fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:43.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.370+0000 7faafe9ddb80 1 -- 172.21.15.2:0/877731895 shutdown_connections 2024-05-14T16:59:43.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.370+0000 7faafe9ddb80 1 -- 172.21.15.2:0/877731895 wait complete. 2024-05-14T16:59:43.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.370+0000 7faafe9ddb80 20 client.15855 trim_cache size 0 max 16384 2024-05-14T16:59:43.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.396+0000 7faafe9ddb80 1 Processor -- start 2024-05-14T16:59:43.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.397+0000 7faafe9ddb80 1 -- start start 2024-05-14T16:59:43.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.397+0000 7faafe9ddb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f19d52420 0x555f19d430d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:43.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.397+0000 7faafe9ddb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f19d52d40 0x555f19d6f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:43.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.397+0000 7faafe9ddb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f19d67350 0x555f19d6f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:43.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.397+0000 7faafe9ddb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7faaec1b05d0 con 0x555f19d67350 2024-05-14T16:59:43.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.397+0000 7faafe9ddb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faaec1ade60 con 0x555f19d52420 2024-05-14T16:59:43.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.397+0000 7faafe9ddb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faaec1abcd0 con 0x555f19d52d40 2024-05-14T16:59:43.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.397+0000 7faaf26e4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f19d67350 0x555f19d6f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:43.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.397+0000 7faaf26e4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f19d67350 0x555f19d6f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44236/0 (socket says 172.21.15.2:44236) 2024-05-14T16:59:43.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.397+0000 7faaf26e4700 1 -- 172.21.15.2:0/3841204317 learned_addr learned my addr 172.21.15.2:0/3841204317 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:59:43.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.397+0000 7faaf1ee3700 1 --2- 172.21.15.2:0/3841204317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f19d52420 0x555f19d430d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:43.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.397+0000 7faaf26e4700 1 -- 172.21.15.2:0/3841204317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f19d52d40 msgr2=0x555f19d6f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:43.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.397+0000 7faaf26e4700 1 --2- 172.21.15.2:0/3841204317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f19d52d40 0x555f19d6f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:43.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.397+0000 7faaf26e4700 1 -- 172.21.15.2:0/3841204317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f19d52420 msgr2=0x555f19d430d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:43.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.397+0000 7faaf26e4700 1 --2- 172.21.15.2:0/3841204317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f19d52420 0x555f19d430d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:43.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.397+0000 7faaf26e4700 1 -- 172.21.15.2:0/3841204317 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f19d59710 con 0x555f19d67350 2024-05-14T16:59:43.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.397+0000 7faaf26e4700 1 --2- 172.21.15.2:0/3841204317 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f19d67350 0x555f19d6f570 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7faae400ef00 tx=0x7faae40a8de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bef4cbfe77290421 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:44.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.397+0000 7faad77fe700 1 -- 172.21.15.2:0/3841204317 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae40a9380 con 0x555f19d67350 2024-05-14T16:59:44.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.397+0000 7faad77fe700 1 -- 172.21.15.2:0/3841204317 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faae40ad730 con 0x555f19d67350 2024-05-14T16:59:44.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.397+0000 7faad77fe700 1 -- 172.21.15.2:0/3841204317 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae40b1f20 con 0x555f19d67350 2024-05-14T16:59:44.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.398+0000 7faafe9ddb80 1 -- 172.21.15.2:0/3841204317 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f19d67350 msgr2=0x555f19d6f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:44.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.398+0000 7faafe9ddb80 1 --2- 172.21.15.2:0/3841204317 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f19d67350 0x555f19d6f570 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7faae400ef00 tx=0x7faae40a8de0 comp rx=0 tx=0).stop 2024-05-14T16:59:44.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.398+0000 7faafe9ddb80 1 -- 172.21.15.2:0/3841204317 shutdown_connections 2024-05-14T16:59:44.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.398+0000 7faafe9ddb80 1 --2- 172.21.15.2:0/3841204317 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f19d67350 0x555f19d6f570 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:44.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.398+0000 7faafe9ddb80 1 --2- 172.21.15.2:0/3841204317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f19d52420 0x555f19d430d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:44.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.398+0000 7faafe9ddb80 1 --2- 172.21.15.2:0/3841204317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f19d52d40 0x555f19d6f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:44.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.398+0000 7faafe9ddb80 1 -- 172.21.15.2:0/3841204317 >> 172.21.15.2:0/3841204317 conn(0x555f19c7aeb0 msgr2=0x555f19d51970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:44.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.398+0000 7faafe9ddb80 1 -- 172.21.15.2:0/3841204317 shutdown_connections 2024-05-14T16:59:44.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.398+0000 7faafe9ddb80 1 -- 172.21.15.2:0/3841204317 wait complete. 2024-05-14T16:59:44.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.399+0000 7faafe9ddb80 1 -- start start 2024-05-14T16:59:44.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.399+0000 7faafe9ddb80 1 Processor -- start 2024-05-14T16:59:44.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.399+0000 7faafe9ddb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f19d52420 0x555f19d5ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:44.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.399+0000 7faafe9ddb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f19d52d40 0x555f19d5d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:44.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.399+0000 7faafe9ddb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f19d67350 0x555f19dd1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:44.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.399+0000 7faafe9ddb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7faaec1ac320 con 0x555f19d67350 2024-05-14T16:59:44.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.399+0000 7faafe9ddb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faaec1ad790 con 0x555f19d52d40 2024-05-14T16:59:44.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.399+0000 7faafe9ddb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faaec1ac9f0 con 0x555f19d52420 2024-05-14T16:59:44.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.399+0000 7faafe9ddb80 10 client.0 _ll_register_callbacks cb 0x555f19c36a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:59:44.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.399+0000 7faaf26e4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f19d67350 0x555f19dd1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:44.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.399+0000 7faaf26e4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f19d67350 0x555f19dd1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44246/0 (socket says 172.21.15.2:44246) 2024-05-14T16:59:44.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.399+0000 7faaf26e4700 1 -- 172.21.15.2:0/2640669841 learned_addr learned my addr 172.21.15.2:0/2640669841 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:59:44.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.399+0000 7faaf1ee3700 1 --2- 172.21.15.2:0/2640669841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f19d52420 0x555f19d5ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:44.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.400+0000 7faaf16e2700 1 --2- 172.21.15.2:0/2640669841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f19d52d40 0x555f19d5d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:44.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.400+0000 7faaf26e4700 1 -- 172.21.15.2:0/2640669841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f19d52420 msgr2=0x555f19d5ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:44.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.400+0000 7faaf26e4700 1 --2- 172.21.15.2:0/2640669841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f19d52420 0x555f19d5ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:44.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.400+0000 7faaf26e4700 1 -- 172.21.15.2:0/2640669841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f19d52d40 msgr2=0x555f19d5d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:44.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.400+0000 7faaf26e4700 1 --2- 172.21.15.2:0/2640669841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f19d52d40 0x555f19d5d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:44.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.400+0000 7faaf26e4700 1 -- 172.21.15.2:0/2640669841 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f19dd21b0 con 0x555f19d67350 2024-05-14T16:59:44.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.400+0000 7faaf26e4700 1 --2- 172.21.15.2:0/2640669841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f19d67350 0x555f19dd1bf0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7faae4004fd0 tx=0x7faae40df780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cd8496f041a6ceb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:44.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.400+0000 7faad57fa700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:59:44.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.400+0000 7faad57fa700 1 -- 172.21.15.2:0/2640669841 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae40b2150 con 0x555f19d67350 2024-05-14T16:59:44.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.400+0000 7faad57fa700 1 -- 172.21.15.2:0/2640669841 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7faae40a5970 con 0x555f19d67350 2024-05-14T16:59:44.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.400+0000 7faafe9ddb80 1 -- 172.21.15.2:0/2640669841 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faae40ad730 con 0x555f19d67350 2024-05-14T16:59:44.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.400+0000 7faad57fa700 1 -- 172.21.15.2:0/2640669841 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7faae40a9380 con 0x555f19d67350 2024-05-14T16:59:44.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.400+0000 7faafe9ddb80 10 client.15861 fetch_fsmap learned FSMap version 36 2024-05-14T16:59:44.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.400+0000 7faafe9ddb80 1 -- 172.21.15.2:0/2640669841 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x555f19dd34d0 con 0x555f19d67350 2024-05-14T16:59:44.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.400+0000 7faad57fa700 1 -- 172.21.15.2:0/2640669841 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faae40ad730 con 0x555f19d67350 2024-05-14T16:59:44.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.401+0000 7faad57fa700 1 -- 172.21.15.2:0/2640669841 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 36) ==== 8342+0+0 (secure 0 0 0) 0x7faae40a8050 con 0x555f19d67350 2024-05-14T16:59:44.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.401+0000 7faafe9ddb80 10 client.15861 fetch_fsmap finished waiting for FSMap version 36 2024-05-14T16:59:44.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.401+0000 7faafe9ddb80 10 client.15861 resolve_mds: validated gid 24479 aka daemon mds.b 2024-05-14T16:59:44.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.401+0000 7faafe9ddb80 20 client.15861 populate_metadata read hostname 'smithi002' 2024-05-14T16:59:44.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.401+0000 7faafe9ddb80 1 --2- 172.21.15.2:0/2640669841 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x555f19dd3aa0 0x555f19dd5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:44.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.401+0000 7faafe9ddb80 4 client.15861 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "38e5e2d2", "release": true, "await": true}] 2024-05-14T16:59:44.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.401+0000 7faafe9ddb80 1 -- 172.21.15.2:0/2640669841 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "quiesce db", "set_id": "38e5e2d2", "release": true, "await": true}) -- 0x555f19ca75a0 con 0x555f19dd3aa0 2024-05-14T16:59:44.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.401+0000 7faaf1ee3700 1 --2- 172.21.15.2:0/2640669841 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x555f19dd3aa0 0x555f19dd5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:44.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.401+0000 7faaf1ee3700 1 --2- 172.21.15.2:0/2640669841 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x555f19dd3aa0 0x555f19dd5ec0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4825c2da2dd581f server_cookie=b1e0697a40daf08f in_seq=0 out_seq=0 2024-05-14T16:59:44.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.401+0000 7faad57fa700 10 client.15861 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:59:44.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.417+0000 7faad57fa700 1 -- 172.21.15.2:0/2640669841 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555f19ca75a0 con 0x555f19dd3aa0 2024-05-14T16:59:44.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.417+0000 7faad57fa700 10 client.15861 handle_command_reply: tid=0 2024-05-14T16:59:44.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.417+0000 7faafe9ddb80 1 client.15861 shutdown 2024-05-14T16:59:44.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.417+0000 7faafe9ddb80 1 -- 172.21.15.2:0/2640669841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f19d67350 msgr2=0x555f19dd1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:44.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.417+0000 7faafe9ddb80 1 --2- 172.21.15.2:0/2640669841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f19d67350 0x555f19dd1bf0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7faae4004fd0 tx=0x7faae40df780 comp rx=0 tx=0).stop 2024-05-14T16:59:44.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.417+0000 7faafe9ddb80 1 -- 172.21.15.2:0/2640669841 shutdown_connections 2024-05-14T16:59:44.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.417+0000 7faafe9ddb80 1 --2- 172.21.15.2:0/2640669841 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x555f19dd3aa0 0x555f19dd5ec0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:44.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.417+0000 7faafe9ddb80 1 --2- 172.21.15.2:0/2640669841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f19d67350 0x555f19dd1bf0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:44.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.417+0000 7faafe9ddb80 1 --2- 172.21.15.2:0/2640669841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f19d52420 0x555f19d5ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:44.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.417+0000 7faad57fa700 0 client.15861 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:59:44.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.417+0000 7faafe9ddb80 1 --2- 172.21.15.2:0/2640669841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f19d52d40 0x555f19d5d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:44.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.417+0000 7faafe9ddb80 1 -- 172.21.15.2:0/2640669841 >> 172.21.15.2:0/2640669841 conn(0x555f19c7aeb0 msgr2=0x555f19d583d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:44.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.417+0000 7faafe9ddb80 1 -- 172.21.15.2:0/2640669841 shutdown_connections 2024-05-14T16:59:44.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.418+0000 7faafe9ddb80 1 -- 172.21.15.2:0/2640669841 wait complete. 2024-05-14T16:59:44.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.418+0000 7faafe9ddb80 20 client.15861 trim_cache size 0 max 16384 2024-05-14T16:59:44.665 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T16:59:44.674 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 36, 2024-05-14T16:59:44.681 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24479, 2024-05-14T16:59:44.726 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 16, 2024-05-14T16:59:44.736 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-05-14T16:59:44.743 INFO:teuthology.orchestra.run.smithi002.stdout: "38e5e2d2": { 2024-05-14T16:59:44.750 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 16, 2024-05-14T16:59:44.755 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-05-14T16:59:44.761 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-05-14T16:59:44.769 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-05-14T16:59:44.775 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-05-14T16:59:44.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T16:59:44.790 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-05-14T16:59:44.797 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 146.8, 2024-05-14T16:59:44.822 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-05-14T16:59:44.830 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-05-14T16:59:44.836 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-05-14T16:59:44.848 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-05-14T16:59:44.855 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-05-14T16:59:44.862 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-05-14T16:59:44.911 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:59:44.926 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:59:44.935 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:59:44.941 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:59:44.948 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T16:59:44.955 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T16:59:44.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.418+0000 7faaf4147700 1 -- 172.21.15.2:0/1194252214 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7faad80b2a70 msgr2=0x7faad80b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:44.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.418+0000 7faaf4147700 1 --2- 172.21.15.2:0/1194252214 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7faad80b2a70 0x7faad80b4f30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faaec19aad0 tx=0x7faae8009260 comp rx=0 tx=0).stop 2024-05-14T16:59:44.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.418+0000 7faaf4147700 1 -- 172.21.15.2:0/1194252214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaec1088f0 msgr2=0x7faaec199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:44.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.418+0000 7faaf4147700 1 --2- 172.21.15.2:0/1194252214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaec1088f0 0x7faaec199990 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7faae400ef30 tx=0x7faae400c3a0 comp rx=0 tx=0).stop 2024-05-14T16:59:45.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.418+0000 7faaf4147700 1 -- 172.21.15.2:0/1194252214 shutdown_connections 2024-05-14T16:59:45.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.418+0000 7faaf4147700 1 --2- 172.21.15.2:0/1194252214 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7faad80b2a70 0x7faad80b4f30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:45.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.418+0000 7faaf4147700 1 --2- 172.21.15.2:0/1194252214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faaec1088f0 0x7faaec199990 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:45.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.418+0000 7faaf4147700 1 --2- 172.21.15.2:0/1194252214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faaec1037f0 0x7faaec19f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:45.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.418+0000 7faaf4147700 1 --2- 172.21.15.2:0/1194252214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faaec1041a0 0x7faaec19fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:45.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.418+0000 7faaf4147700 1 -- 172.21.15.2:0/1194252214 >> 172.21.15.2:0/1194252214 conn(0x7faaec07a6a0 msgr2=0x7faaec109a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:45.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.418+0000 7faaf4147700 1 -- 172.21.15.2:0/1194252214 shutdown_connections 2024-05-14T16:59:45.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T16:59:42.418+0000 7faaf4147700 1 -- 172.21.15.2:0/1194252214 wait complete. 2024-05-14T16:59:45.132 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:59:42 smithi002 ceph-mon[31538]: pgmap v153: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 614 KiB/s wr, 74 op/s 2024-05-14T16:59:45.144 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '38e5e2d2', total seconds elapsed: 34.5 2024-05-14T16:59:45.189 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:59:44 smithi002 ceph-mon[31538]: pgmap v154: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.4 MiB/s wr, 79 op/s 2024-05-14T16:59:45.203 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:59:42 smithi123 ceph-mon[35971]: pgmap v153: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 614 KiB/s wr, 74 op/s 2024-05-14T16:59:45.212 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:59:44 smithi123 ceph-mon[35971]: pgmap v154: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.4 MiB/s wr, 79 op/s 2024-05-14T16:59:45.221 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:59:42 smithi151 ceph-mon[36066]: pgmap v153: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 614 KiB/s wr, 74 op/s 2024-05-14T16:59:45.230 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:59:44 smithi151 ceph-mon[36066]: pgmap v154: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.4 MiB/s wr, 79 op/s 2024-05-14T16:59:46.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:59:46 smithi002 ceph-mon[31538]: pgmap v155: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 851 KiB/s wr, 12 op/s 2024-05-14T16:59:46.752 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 270.0 s, 2377.2 tps, lat 242.598 ms stddev 2242.637 2024-05-14T16:59:46.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:59:46 smithi151 ceph-mon[36066]: pgmap v155: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 851 KiB/s wr, 12 op/s 2024-05-14T16:59:46.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:59:46 smithi123 ceph-mon[35971]: pgmap v155: 97 pgs: 97 active+clean; 2.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 851 KiB/s wr, 12 op/s 2024-05-14T16:59:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.613+0000 7fcb0054e640 1 -- 172.21.15.2:0/4076123826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcaf8108a40 msgr2=0x7fcaf8108e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:47.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.613+0000 7fcb0054e640 1 --2- 172.21.15.2:0/4076123826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcaf8108a40 0x7fcaf8108e20 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fcae8009930 tx=0x7fcae802a440 comp rx=0 tx=0).stop 2024-05-14T16:59:47.633 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.613+0000 7fcb0054e640 1 -- 172.21.15.2:0/4076123826 shutdown_connections 2024-05-14T16:59:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.613+0000 7fcb0054e640 1 --2- 172.21.15.2:0/4076123826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcaf81033f0 0x7fcaf810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.613+0000 7fcb0054e640 1 --2- 172.21.15.2:0/4076123826 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcaf8102a30 0x7fcaf8102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.613+0000 7fcb0054e640 1 --2- 172.21.15.2:0/4076123826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcaf8108a40 0x7fcaf8108e20 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.613+0000 7fcb0054e640 1 -- 172.21.15.2:0/4076123826 >> 172.21.15.2:0/4076123826 conn(0x7fcaf80fe500 msgr2=0x7fcaf8100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.613+0000 7fcb0054e640 1 -- 172.21.15.2:0/4076123826 shutdown_connections 2024-05-14T16:59:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.613+0000 7fcb0054e640 1 -- 172.21.15.2:0/4076123826 wait complete. 2024-05-14T16:59:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.614+0000 7fcb0054e640 1 Processor -- start 2024-05-14T16:59:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.614+0000 7fcb0054e640 1 -- start start 2024-05-14T16:59:47.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.614+0000 7fcb0054e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcaf8102a30 0x7fcaf8076950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:47.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.614+0000 7fcb0054e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcaf81033f0 0x7fcaf8074d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:47.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.614+0000 7fcb0054e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcaf8108a40 0x7fcaf8075390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:47.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.614+0000 7fcb0054e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf8112cd0 con 0x7fcaf8102a30 2024-05-14T16:59:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.614+0000 7fcb0054e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf8112b50 con 0x7fcaf81033f0 2024-05-14T16:59:47.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.614+0000 7fcb0054e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf8112e50 con 0x7fcaf8108a40 2024-05-14T16:59:47.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.615+0000 7fcafe2ea640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcaf8102a30 0x7fcaf8076950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:47.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.615+0000 7fcafe2ea640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcaf8102a30 0x7fcaf8076950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40156/0 (socket says 172.21.15.2:40156) 2024-05-14T16:59:47.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.615+0000 7fcafe2ea640 1 -- 172.21.15.2:0/3403258202 learned_addr learned my addr 172.21.15.2:0/3403258202 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:59:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.615+0000 7fcafeaeb640 1 --2- 172.21.15.2:0/3403258202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcaf8108a40 0x7fcaf8075390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:47.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.615+0000 7fcafdae9640 1 --2- 172.21.15.2:0/3403258202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcaf81033f0 0x7fcaf8074d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:47.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.615+0000 7fcafeaeb640 1 -- 172.21.15.2:0/3403258202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcaf81033f0 msgr2=0x7fcaf8074d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:47.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.615+0000 7fcafeaeb640 1 --2- 172.21.15.2:0/3403258202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcaf81033f0 0x7fcaf8074d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.615+0000 7fcafeaeb640 1 -- 172.21.15.2:0/3403258202 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcaf8102a30 msgr2=0x7fcaf8076950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.615+0000 7fcafeaeb640 1 --2- 172.21.15.2:0/3403258202 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcaf8102a30 0x7fcaf8076950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.615+0000 7fcafeaeb640 1 -- 172.21.15.2:0/3403258202 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf81abbf0 con 0x7fcaf8108a40 2024-05-14T16:59:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.615+0000 7fcafe2ea640 1 --2- 172.21.15.2:0/3403258202 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcaf8102a30 0x7fcaf8076950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:59:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.615+0000 7fcafeaeb640 1 --2- 172.21.15.2:0/3403258202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcaf8108a40 0x7fcaf8075390 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0009fd0 tx=0x7fcaf000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:47.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.615+0000 7fcaf77fe640 1 -- 172.21.15.2:0/3403258202 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf000ce20 con 0x7fcaf8108a40 2024-05-14T16:59:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.616+0000 7fcb0054e640 1 -- 172.21.15.2:0/3403258202 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaf81abe20 con 0x7fcaf8108a40 2024-05-14T16:59:48.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.616+0000 7fcb0054e640 1 -- 172.21.15.2:0/3403258202 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaf81ac2f0 con 0x7fcaf8108a40 2024-05-14T16:59:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.616+0000 7fcaf77fe640 1 -- 172.21.15.2:0/3403258202 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaf0005020 con 0x7fcaf8108a40 2024-05-14T16:59:48.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.617+0000 7fcaf77fe640 1 -- 172.21.15.2:0/3403258202 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0010690 con 0x7fcaf8108a40 2024-05-14T16:59:48.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.620+0000 7fcaf77fe640 1 -- 172.21.15.2:0/3403258202 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fcaf0004a40 con 0x7fcaf8108a40 2024-05-14T16:59:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.620+0000 7fcb0325f740 1 Processor -- start 2024-05-14T16:59:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.621+0000 7fcb0325f740 1 -- start start 2024-05-14T16:59:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.621+0000 7fcb0325f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a03982c8f0 0x55a03982ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.621+0000 7fcb0325f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a03982d210 0x55a039837590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.621+0000 7fcb0325f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a039837ad0 0x55a039839ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.621+0000 7fcb0325f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55a03979fc30 con 0x55a03982d210 2024-05-14T16:59:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.621+0000 7fcb0325f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a03978f7d0 con 0x55a039837ad0 2024-05-14T16:59:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.621+0000 7fcb0325f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a039793810 con 0x55a03982c8f0 2024-05-14T16:59:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.621+0000 7fcafdae9640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a039837ad0 0x55a039839ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.621+0000 7fcafdae9640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a039837ad0 0x55a039839ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:43864/0 (socket says 172.21.15.2:43864) 2024-05-14T16:59:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.621+0000 7fcafdae9640 1 -- 172.21.15.2:0/1804818470 learned_addr learned my addr 172.21.15.2:0/1804818470 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:59:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.621+0000 7fcafe2ea640 1 --2- 172.21.15.2:0/1804818470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a03982d210 0x55a039837590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.622+0000 7fcafdae9640 1 --2- 172.21.15.2:0/1804818470 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a03982c8f0 0x55a03982ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.622+0000 7fcafe2ea640 1 -- 172.21.15.2:0/1804818470 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a03982c8f0 msgr2=0x55a03982ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.622+0000 7fcafe2ea640 1 --2- 172.21.15.2:0/1804818470 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a03982c8f0 0x55a03982ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.622+0000 7fcafe2ea640 1 -- 172.21.15.2:0/1804818470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a039837ad0 msgr2=0x55a039839ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.622+0000 7fcafe2ea640 1 --2- 172.21.15.2:0/1804818470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a039837ad0 0x55a039839ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:48.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.622+0000 7fcafe2ea640 1 -- 172.21.15.2:0/1804818470 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0394a4210 con 0x55a03982d210 2024-05-14T16:59:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.622+0000 7fcafdae9640 1 --2- 172.21.15.2:0/1804818470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a039837ad0 0x55a039839ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:59:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.622+0000 7fcafe2ea640 1 --2- 172.21.15.2:0/1804818470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a03982d210 0x55a039837590 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fcae8004770 tx=0x7fcae8032ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eda508fe255bd65 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.623+0000 7fcaf4bfe640 1 -- 172.21.15.2:0/1804818470 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae8035040 con 0x55a03982d210 2024-05-14T16:59:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.623+0000 7fcaf77fe640 1 --2- 172.21.15.2:0/3403258202 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcae00b2a90 0x7fcae00b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:48.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.623+0000 7fcaf77fe640 1 -- 172.21.15.2:0/3403258202 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fcaf00e2a70 con 0x7fcaf8108a40 2024-05-14T16:59:48.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.623+0000 7fcaf4bfe640 1 -- 172.21.15.2:0/1804818470 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcae80317a0 con 0x55a03982d210 2024-05-14T16:59:48.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.623+0000 7fcaf4bfe640 1 -- 172.21.15.2:0/1804818470 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae80047a0 con 0x55a03982d210 2024-05-14T16:59:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.624+0000 7fcb0325f740 1 -- 172.21.15.2:0/1804818470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a03982d210 msgr2=0x55a039837590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:48.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.624+0000 7fcb0325f740 1 --2- 172.21.15.2:0/1804818470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a03982d210 0x55a039837590 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fcae8004770 tx=0x7fcae8032ed0 comp rx=0 tx=0).stop 2024-05-14T16:59:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.624+0000 7fcafdae9640 1 --2- 172.21.15.2:0/3403258202 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcae00b2a90 0x7fcae00b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:48.396 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.625+0000 7fcb0325f740 1 -- 172.21.15.2:0/1804818470 shutdown_connections 2024-05-14T16:59:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.625+0000 7fcb0325f740 1 --2- 172.21.15.2:0/1804818470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a039837ad0 0x55a039839ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.625+0000 7fcb0325f740 1 --2- 172.21.15.2:0/1804818470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a03982d210 0x55a039837590 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.625+0000 7fcb0325f740 1 --2- 172.21.15.2:0/1804818470 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a03982c8f0 0x55a03982ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.625+0000 7fcb0325f740 1 -- 172.21.15.2:0/1804818470 >> 172.21.15.2:0/1804818470 conn(0x55a039828e00 msgr2=0x55a0398291e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:48.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.625+0000 7fcb0325f740 1 -- 172.21.15.2:0/1804818470 shutdown_connections 2024-05-14T16:59:48.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.625+0000 7fcb0325f740 1 -- 172.21.15.2:0/1804818470 wait complete. 2024-05-14T16:59:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.625+0000 7fcafdae9640 1 --2- 172.21.15.2:0/3403258202 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcae00b2a90 0x7fcae00b4f50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55a03982a220 tx=0x7fcaec0046c0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.625+0000 7fcb0325f740 1 -- start start 2024-05-14T16:59:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.626+0000 7fcb0325f740 1 Processor -- start 2024-05-14T16:59:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.626+0000 7fcb0325f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a03982c8f0 0x55a0398373e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.626+0000 7fcb0325f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a03982d210 0x55a0398346d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:48.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.626+0000 7fcafe2ea640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a03982c8f0 0x55a0398373e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:48.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.627+0000 7fcafdae9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a03982d210 0x55a0398346d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.627+0000 7fcafdae9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a03982d210 0x55a0398346d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40166/0 (socket says 172.21.15.2:40166) 2024-05-14T16:59:48.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.627+0000 7fcafe2ea640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a03982c8f0 0x55a0398373e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:52166/0 (socket says 172.21.15.2:52166) 2024-05-14T16:59:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.627+0000 7fcafdae9640 1 -- 172.21.15.2:0/2087018444 learned_addr learned my addr 172.21.15.2:0/2087018444 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:59:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.627+0000 7fcb0325f740 1 --2- 172.21.15.2:0/2087018444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a039837ad0 0x55a039834c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.627+0000 7fcb0325f740 1 -- 172.21.15.2:0/2087018444 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55a039475790 con 0x55a03982d210 2024-05-14T16:59:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.627+0000 7fcb0325f740 1 -- 172.21.15.2:0/2087018444 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a03974b890 con 0x55a039837ad0 2024-05-14T16:59:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.627+0000 7fcb0325f740 1 -- 172.21.15.2:0/2087018444 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a03983a740 con 0x55a03982c8f0 2024-05-14T16:59:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.627+0000 7fcb0325f740 10 client.0 _ll_register_callbacks cb 0x55a03974ed60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:59:48.634 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.627+0000 7fcafeaeb640 1 --2- 172.21.15.2:0/2087018444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a039837ad0 0x55a039834c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.627+0000 7fcafdae9640 1 -- 172.21.15.2:0/2087018444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a03982c8f0 msgr2=0x55a0398373e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:48.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.627+0000 7fcafdae9640 1 --2- 172.21.15.2:0/2087018444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a03982c8f0 0x55a0398373e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.627+0000 7fcafdae9640 1 -- 172.21.15.2:0/2087018444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a039837ad0 msgr2=0x55a039834c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.627+0000 7fcafdae9640 1 --2- 172.21.15.2:0/2087018444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a039837ad0 0x55a039834c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.627+0000 7fcafdae9640 1 -- 172.21.15.2:0/2087018444 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a039570620 con 0x55a03982d210 2024-05-14T16:59:48.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.627+0000 7fcafe2ea640 1 --2- 172.21.15.2:0/2087018444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a03982c8f0 0x55a0398373e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:59:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.627+0000 7fcafeaeb640 1 --2- 172.21.15.2:0/2087018444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a039837ad0 0x55a039834c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:59:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.628+0000 7fcafdae9640 1 --2- 172.21.15.2:0/2087018444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a03982d210 0x55a0398346d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fcaec011c10 tx=0x7fcaec005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d9b80eea1eef8b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.628+0000 7fcaddffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:59:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.628+0000 7fcaddffb640 1 -- 172.21.15.2:0/2087018444 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec011c40 con 0x55a03982d210 2024-05-14T16:59:48.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.628+0000 7fcaddffb640 1 -- 172.21.15.2:0/2087018444 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fcaec01a050 con 0x55a03982d210 2024-05-14T16:59:48.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.628+0000 7fcb0325f740 1 -- 172.21.15.2:0/2087018444 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcae80317a0 con 0x55a03982d210 2024-05-14T16:59:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.628+0000 7fcaddffb640 1 -- 172.21.15.2:0/2087018444 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fcaec011de0 con 0x55a03982d210 2024-05-14T16:59:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.628+0000 7fcb0325f740 10 client.15873 fetch_fsmap learned FSMap version 36 2024-05-14T16:59:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.628+0000 7fcb0325f740 1 -- 172.21.15.2:0/2087018444 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a039846180 con 0x55a03982d210 2024-05-14T16:59:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.628+0000 7fcaddffb640 1 -- 172.21.15.2:0/2087018444 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaec004c50 con 0x55a03982d210 2024-05-14T16:59:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.630+0000 7fcaddffb640 1 -- 172.21.15.2:0/2087018444 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 36) ==== 8342+0+0 (secure 0 0 0) 0x7fcaec01d070 con 0x55a03982d210 2024-05-14T16:59:48.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.630+0000 7fcb0325f740 10 client.15873 fetch_fsmap finished waiting for FSMap version 36 2024-05-14T16:59:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.630+0000 7fcb0325f740 10 client.15873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:59:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.630+0000 7fcb0325f740 20 client.15873 populate_metadata read hostname 'smithi002' 2024-05-14T16:59:48.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.630+0000 7fcb0325f740 1 --2- 172.21.15.2:0/2087018444 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55a039847700 0x55a039849b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.630+0000 7fcb0325f740 4 client.15873 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T16:59:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.630+0000 7fcb0325f740 1 -- 172.21.15.2:0/2087018444 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcae80317a0 con 0x55a039847700 2024-05-14T16:59:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.630+0000 7fcafe2ea640 1 --2- 172.21.15.2:0/2087018444 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55a039847700 0x55a039849b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:48.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.631+0000 7fcaddffb640 10 client.15873 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:59:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.631+0000 7fcafe2ea640 1 --2- 172.21.15.2:0/2087018444 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55a039847700 0x55a039849b60 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10538a90f80e9350 server_cookie=2959ceb368bc481a in_seq=0 out_seq=0 2024-05-14T16:59:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.632+0000 7fcaddffb640 1 -- 172.21.15.2:0/2087018444 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fcae80317a0 con 0x55a039847700 2024-05-14T16:59:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.632+0000 7fcaddffb640 10 client.15873 handle_command_reply: tid=0 2024-05-14T16:59:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.632+0000 7fcb0325f740 1 client.15873 shutdown 2024-05-14T16:59:49.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.632+0000 7fcb0325f740 1 -- 172.21.15.2:0/2087018444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a03982d210 msgr2=0x55a0398346d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.632+0000 7fcb0325f740 1 --2- 172.21.15.2:0/2087018444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a03982d210 0x55a0398346d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fcaec011c10 tx=0x7fcaec005f90 comp rx=0 tx=0).stop 2024-05-14T16:59:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.632+0000 7fcb0325f740 1 -- 172.21.15.2:0/2087018444 shutdown_connections 2024-05-14T16:59:49.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.632+0000 7fcb0325f740 1 --2- 172.21.15.2:0/2087018444 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55a039847700 0x55a039849b60 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:49.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.632+0000 7fcb0325f740 1 --2- 172.21.15.2:0/2087018444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a039837ad0 0x55a039834c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:49.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.632+0000 7fcaddffb640 0 client.15873 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:59:49.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.633+0000 7fcb0325f740 1 --2- 172.21.15.2:0/2087018444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a03982d210 0x55a0398346d0 secure :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fcaec011c10 tx=0x7fcaec005f90 comp rx=0 tx=0).stop 2024-05-14T16:59:49.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.633+0000 7fcb0325f740 1 --2- 172.21.15.2:0/2087018444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a03982c8f0 0x55a0398373e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:49.090 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.633+0000 7fcb0325f740 1 -- 172.21.15.2:0/2087018444 >> 172.21.15.2:0/2087018444 conn(0x55a039828e00 msgr2=0x55a039828100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.633+0000 7fcb0325f740 1 -- 172.21.15.2:0/2087018444 shutdown_connections 2024-05-14T16:59:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.633+0000 7fcb0325f740 1 -- 172.21.15.2:0/2087018444 wait complete. 2024-05-14T16:59:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.633+0000 7fcb0325f740 20 client.15873 trim_cache size 0 max 16384 2024-05-14T16:59:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.658+0000 7fcb0325f740 1 Processor -- start 2024-05-14T16:59:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.658+0000 7fcb0325f740 1 -- start start 2024-05-14T16:59:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.658+0000 7fcb0325f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a03982c8f0 0x55a039843640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.658+0000 7fcb0325f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a039837ad0 0x55a03984f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.658+0000 7fcb0325f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a03984f970 0x55a039831950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.659+0000 7fcafe2ea640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a03982c8f0 0x55a039843640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.659+0000 7fcafe2ea640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a03982c8f0 0x55a039843640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:43884/0 (socket says 172.21.15.2:43884) 2024-05-14T16:59:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.659+0000 7fcafe2ea640 1 -- 172.21.15.2:0/1818092809 learned_addr learned my addr 172.21.15.2:0/1818092809 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:59:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.659+0000 7fcb0325f740 1 -- 172.21.15.2:0/1818092809 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcae4010f40 con 0x55a039837ad0 2024-05-14T16:59:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.659+0000 7fcb0325f740 1 -- 172.21.15.2:0/1818092809 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcae400e810 con 0x55a03982c8f0 2024-05-14T16:59:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.659+0000 7fcb0325f740 1 -- 172.21.15.2:0/1818092809 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcae400c660 con 0x55a03984f970 2024-05-14T16:59:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.659+0000 7fcafeaeb640 1 --2- 172.21.15.2:0/1818092809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a03984f970 0x55a039831950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:49.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.659+0000 7fcafe2ea640 1 -- 172.21.15.2:0/1818092809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a03984f970 msgr2=0x55a039831950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:49.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.659+0000 7fcafe2ea640 1 --2- 172.21.15.2:0/1818092809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a03984f970 0x55a039831950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.659+0000 7fcafe2ea640 1 -- 172.21.15.2:0/1818092809 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a039837ad0 msgr2=0x55a03984f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:49.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.659+0000 7fcafdae9640 1 --2- 172.21.15.2:0/1818092809 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a039837ad0 0x55a03984f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.659+0000 7fcafe2ea640 1 --2- 172.21.15.2:0/1818092809 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a039837ad0 0x55a03984f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:49.283 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.659+0000 7fcafe2ea640 1 -- 172.21.15.2:0/1818092809 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a039833680 con 0x55a03982c8f0 2024-05-14T16:59:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.659+0000 7fcafdae9640 1 --2- 172.21.15.2:0/1818092809 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a039837ad0 0x55a03984f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T16:59:49.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.660+0000 7fcafe2ea640 1 --2- 172.21.15.2:0/1818092809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a03982c8f0 0x55a039843640 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcae8005dc0 tx=0x7fcae8031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e438a296f3ae5d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.660+0000 7fcadffff640 1 -- 172.21.15.2:0/1818092809 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae8078070 con 0x55a03982c8f0 2024-05-14T16:59:49.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.660+0000 7fcadffff640 1 -- 172.21.15.2:0/1818092809 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcae8051930 con 0x55a03982c8f0 2024-05-14T16:59:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.660+0000 7fcadffff640 1 -- 172.21.15.2:0/1818092809 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae8035040 con 0x55a03982c8f0 2024-05-14T16:59:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.661+0000 7fcb0325f740 1 -- 172.21.15.2:0/1818092809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a03982c8f0 msgr2=0x55a039843640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.661+0000 7fcb0325f740 1 --2- 172.21.15.2:0/1818092809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a03982c8f0 0x55a039843640 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcae8005dc0 tx=0x7fcae8031400 comp rx=0 tx=0).stop 2024-05-14T16:59:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.662+0000 7fcb0325f740 1 -- 172.21.15.2:0/1818092809 shutdown_connections 2024-05-14T16:59:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.662+0000 7fcb0325f740 1 --2- 172.21.15.2:0/1818092809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a03984f970 0x55a039831950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.662+0000 7fcb0325f740 1 --2- 172.21.15.2:0/1818092809 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a039837ad0 0x55a03984f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:49.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.662+0000 7fcb0325f740 1 --2- 172.21.15.2:0/1818092809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a03982c8f0 0x55a039843640 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.662+0000 7fcb0325f740 1 -- 172.21.15.2:0/1818092809 >> 172.21.15.2:0/1818092809 conn(0x55a039828e00 msgr2=0x55a039828100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.662+0000 7fcb0325f740 1 -- 172.21.15.2:0/1818092809 shutdown_connections 2024-05-14T16:59:49.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.662+0000 7fcb0325f740 1 -- 172.21.15.2:0/1818092809 wait complete. 2024-05-14T16:59:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.663+0000 7fcb0325f740 1 -- start start 2024-05-14T16:59:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.663+0000 7fcb0325f740 1 Processor -- start 2024-05-14T16:59:49.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.663+0000 7fcb0325f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a03982c8f0 0x55a03983dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.663+0000 7fcb0325f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a039837ad0 0x55a03983e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.663+0000 7fcb0325f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a03984f970 0x55a0398b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.663+0000 7fcb0325f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcae400ccd0 con 0x55a03982c8f0 2024-05-14T16:59:49.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.663+0000 7fcb0325f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcae400e140 con 0x55a03984f970 2024-05-14T16:59:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.663+0000 7fcb0325f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcae400d3a0 con 0x55a039837ad0 2024-05-14T16:59:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.663+0000 7fcb0325f740 10 client.0 _ll_register_callbacks cb 0x7fcaec011aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T16:59:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.663+0000 7fcafe2ea640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a03982c8f0 0x55a03983dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.663+0000 7fcafdae9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a039837ad0 0x55a03983e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.663+0000 7fcafe2ea640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a03982c8f0 0x55a03983dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40180/0 (socket says 172.21.15.2:40180) 2024-05-14T16:59:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.663+0000 7fcafe2ea640 1 -- 172.21.15.2:0/3930200055 learned_addr learned my addr 172.21.15.2:0/3930200055 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T16:59:49.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.663+0000 7fcafeaeb640 1 --2- 172.21.15.2:0/3930200055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a03984f970 0x55a0398b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.664+0000 7fcafe2ea640 1 -- 172.21.15.2:0/3930200055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a039837ad0 msgr2=0x55a03983e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.664+0000 7fcafe2ea640 1 --2- 172.21.15.2:0/3930200055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a039837ad0 0x55a03983e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.664+0000 7fcafe2ea640 1 -- 172.21.15.2:0/3930200055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a03984f970 msgr2=0x55a0398b0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T16:59:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.664+0000 7fcafe2ea640 1 --2- 172.21.15.2:0/3930200055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a03984f970 0x55a0398b0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.664+0000 7fcafe2ea640 1 -- 172.21.15.2:0/3930200055 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a03983a230 con 0x55a03982c8f0 2024-05-14T16:59:49.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.664+0000 7fcafdae9640 1 --2- 172.21.15.2:0/3930200055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a039837ad0 0x55a03983e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:59:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.664+0000 7fcafeaeb640 1 --2- 172.21.15.2:0/3930200055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a03984f970 0x55a0398b0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T16:59:49.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.664+0000 7fcafe2ea640 1 --2- 172.21.15.2:0/3930200055 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a03982c8f0 0x55a03983dce0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fcae8009a50 tx=0x7fcae8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d9a183748f181b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T16:59:49.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.664+0000 7fcaddffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T16:59:49.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.664+0000 7fcaddffb640 1 -- 172.21.15.2:0/3930200055 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae8078030 con 0x55a03982c8f0 2024-05-14T16:59:49.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.664+0000 7fcaddffb640 1 -- 172.21.15.2:0/3930200055 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fcae8051690 con 0x55a03982c8f0 2024-05-14T16:59:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.664+0000 7fcb0325f740 1 -- 172.21.15.2:0/3930200055 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcae8051930 con 0x55a03982c8f0 2024-05-14T16:59:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.664+0000 7fcaddffb640 1 -- 172.21.15.2:0/3930200055 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fcae8035070 con 0x55a03982c8f0 2024-05-14T16:59:49.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.665+0000 7fcb0325f740 10 client.15876 fetch_fsmap learned FSMap version 36 2024-05-14T16:59:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.665+0000 7fcb0325f740 1 -- 172.21.15.2:0/3930200055 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a0398b24e0 con 0x55a03982c8f0 2024-05-14T16:59:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.666+0000 7fcaddffb640 1 -- 172.21.15.2:0/3930200055 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcae8051930 con 0x55a03982c8f0 2024-05-14T16:59:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.666+0000 7fcaddffb640 1 -- 172.21.15.2:0/3930200055 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 36) ==== 8342+0+0 (secure 0 0 0) 0x7fcae8003700 con 0x55a03982c8f0 2024-05-14T16:59:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.666+0000 7fcb0325f740 10 client.15876 fetch_fsmap finished waiting for FSMap version 36 2024-05-14T16:59:49.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.666+0000 7fcb0325f740 10 client.15876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T16:59:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.666+0000 7fcb0325f740 20 client.15876 populate_metadata read hostname 'smithi002' 2024-05-14T16:59:49.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.666+0000 7fcb0325f740 1 --2- 172.21.15.2:0/3930200055 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55a0398b2760 0x55a0398b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T16:59:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.666+0000 7fcb0325f740 4 client.15876 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T16:59:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.666+0000 7fcb0325f740 1 -- 172.21.15.2:0/3930200055 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a0397a1b80 con 0x55a0398b2760 2024-05-14T16:59:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.666+0000 7fcafdae9640 1 --2- 172.21.15.2:0/3930200055 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55a0398b2760 0x55a0398b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T16:59:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.667+0000 7fcafdae9640 1 --2- 172.21.15.2:0/3930200055 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55a0398b2760 0x55a0398b4b50 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=436efdc3c9f1b0c7 server_cookie=b647ae3edee56360 in_seq=0 out_seq=0 2024-05-14T16:59:49.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.667+0000 7fcaddffb640 10 client.15876 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T16:59:49.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.679+0000 7fcaddffb640 1 -- 172.21.15.2:0/3930200055 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a0397a1b80 con 0x55a0398b2760 2024-05-14T16:59:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.679+0000 7fcaddffb640 10 client.15876 handle_command_reply: tid=0 2024-05-14T16:59:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.679+0000 7fcb0325f740 1 client.15876 shutdown 2024-05-14T16:59:49.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.679+0000 7fcb0325f740 1 -- 172.21.15.2:0/3930200055 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a03982c8f0 msgr2=0x55a03983dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:49.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.679+0000 7fcb0325f740 1 --2- 172.21.15.2:0/3930200055 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a03982c8f0 0x55a03983dce0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fcae8009a50 tx=0x7fcae8005fd0 comp rx=0 tx=0).stop 2024-05-14T16:59:49.943 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.679+0000 7fcb0325f740 1 -- 172.21.15.2:0/3930200055 shutdown_connections 2024-05-14T16:59:49.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.679+0000 7fcb0325f740 1 --2- 172.21.15.2:0/3930200055 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55a0398b2760 0x55a0398b4b50 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:49.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.679+0000 7fcb0325f740 1 --2- 172.21.15.2:0/3930200055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a03984f970 0x55a0398b0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:49.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.679+0000 7fcb0325f740 1 --2- 172.21.15.2:0/3930200055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a039837ad0 0x55a03983e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:49.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.679+0000 7fcb0325f740 1 --2- 172.21.15.2:0/3930200055 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a03982c8f0 0x55a03983dce0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.679+0000 7fcb0325f740 1 -- 172.21.15.2:0/3930200055 >> 172.21.15.2:0/3930200055 conn(0x55a039828e00 msgr2=0x55a03984e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:50.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.679+0000 7fcaddffb640 0 client.15876 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T16:59:50.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.680+0000 7fcb0325f740 1 -- 172.21.15.2:0/3930200055 shutdown_connections 2024-05-14T16:59:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.680+0000 7fcb0325f740 1 -- 172.21.15.2:0/3930200055 wait complete. 2024-05-14T16:59:50.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.680+0000 7fcb0325f740 20 client.15876 trim_cache size 0 max 16384 2024-05-14T16:59:50.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T16:59:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T16:59:50.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T16:59:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.681+0000 7fcb0054e640 1 -- 172.21.15.2:0/3403258202 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcae00b2a90 msgr2=0x7fcae00b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:50.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.681+0000 7fcb0054e640 1 --2- 172.21.15.2:0/3403258202 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcae00b2a90 0x7fcae00b4f50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55a03982a220 tx=0x7fcaec0046c0 comp rx=0 tx=0).stop 2024-05-14T16:59:50.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.681+0000 7fcb0054e640 1 -- 172.21.15.2:0/3403258202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcaf8108a40 msgr2=0x7fcaf8075390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T16:59:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.681+0000 7fcb0054e640 1 --2- 172.21.15.2:0/3403258202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcaf8108a40 0x7fcaf8075390 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0009fd0 tx=0x7fcaf000ef10 comp rx=0 tx=0).stop 2024-05-14T16:59:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.682+0000 7fcb0054e640 1 -- 172.21.15.2:0/3403258202 shutdown_connections 2024-05-14T16:59:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.682+0000 7fcb0054e640 1 --2- 172.21.15.2:0/3403258202 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcae00b2a90 0x7fcae00b4f50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.682+0000 7fcb0054e640 1 --2- 172.21.15.2:0/3403258202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcaf8108a40 0x7fcaf8075390 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.682+0000 7fcb0054e640 1 --2- 172.21.15.2:0/3403258202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcaf81033f0 0x7fcaf8074d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.682+0000 7fcb0054e640 1 --2- 172.21.15.2:0/3403258202 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcaf8102a30 0x7fcaf8076950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T16:59:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.682+0000 7fcb0054e640 1 -- 172.21.15.2:0/3403258202 >> 172.21.15.2:0/3403258202 conn(0x7fcaf80fe500 msgr2=0x7fcaf80fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T16:59:50.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.682+0000 7fcb0054e640 1 -- 172.21.15.2:0/3403258202 shutdown_connections 2024-05-14T16:59:50.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T16:59:47.682+0000 7fcb0054e640 1 -- 172.21.15.2:0/3403258202 wait complete. 2024-05-14T16:59:50.190 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:59:48 smithi002 ceph-mon[31538]: pgmap v156: 97 pgs: 97 active+clean; 2.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 11 MiB/s wr, 183 op/s 2024-05-14T16:59:50.196 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:59:48 smithi123 ceph-mon[35971]: pgmap v156: 97 pgs: 97 active+clean; 2.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 11 MiB/s wr, 183 op/s 2024-05-14T16:59:50.202 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:59:48 smithi151 ceph-mon[36066]: pgmap v156: 97 pgs: 97 active+clean; 2.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 11 MiB/s wr, 183 op/s 2024-05-14T16:59:50.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:59:50 smithi002 ceph-mon[31538]: pgmap v157: 97 pgs: 97 active+clean; 2.7 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 MiB/s wr, 404 op/s 2024-05-14T16:59:50.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:59:50 smithi151 ceph-mon[36066]: pgmap v157: 97 pgs: 97 active+clean; 2.7 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 MiB/s wr, 404 op/s 2024-05-14T16:59:50.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:59:50 smithi123 ceph-mon[35971]: pgmap v157: 97 pgs: 97 active+clean; 2.7 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 MiB/s wr, 404 op/s 2024-05-14T16:59:51.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 275.0 s, 3731.8 tps, lat 26.816 ms stddev 30.842 2024-05-14T16:59:52.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:59:52 smithi002 ceph-mon[31538]: pgmap v158: 97 pgs: 97 active+clean; 2.7 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 MiB/s wr, 400 op/s 2024-05-14T16:59:52.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:59:52 smithi151 ceph-mon[36066]: pgmap v158: 97 pgs: 97 active+clean; 2.7 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 MiB/s wr, 400 op/s 2024-05-14T16:59:52.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:59:52 smithi123 ceph-mon[35971]: pgmap v158: 97 pgs: 97 active+clean; 2.7 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 MiB/s wr, 400 op/s 2024-05-14T16:59:54.711 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:59:54 smithi123 ceph-mon[35971]: pgmap v159: 97 pgs: 97 active+clean; 2.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 MiB/s wr, 737 op/s 2024-05-14T16:59:54.730 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:59:54 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:59:54.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:59:54 smithi002 ceph-mon[31538]: pgmap v159: 97 pgs: 97 active+clean; 2.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 MiB/s wr, 737 op/s 2024-05-14T16:59:54.751 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:59:54 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:59:54.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:59:54 smithi151 ceph-mon[36066]: pgmap v159: 97 pgs: 97 active+clean; 2.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 MiB/s wr, 737 op/s 2024-05-14T16:59:54.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:59:54 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T16:59:55.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:59:55 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:55.750 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:59:55 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:59:55.758 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:59:55 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:59:55.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:59:55 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:55.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:59:55 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:59:55.872 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:59:55 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:59:55.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:59:55 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T16:59:55.918 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:59:55 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T16:59:55.927 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:59:55 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T16:59:56.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 280.0 s, 3751.0 tps, lat 26.700 ms stddev 30.638 2024-05-14T16:59:56.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:59:56 smithi151 ceph-mon[36066]: pgmap v160: 97 pgs: 97 active+clean; 2.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 MiB/s wr, 733 op/s 2024-05-14T16:59:56.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:59:56 smithi123 ceph-mon[35971]: pgmap v160: 97 pgs: 97 active+clean; 2.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 MiB/s wr, 733 op/s 2024-05-14T16:59:56.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:59:56 smithi002 ceph-mon[31538]: pgmap v160: 97 pgs: 97 active+clean; 2.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 MiB/s wr, 733 op/s 2024-05-14T16:59:58.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 16:59:58 smithi151 ceph-mon[36066]: pgmap v161: 97 pgs: 97 active+clean; 2.9 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 47 MiB/s wr, 886 op/s 2024-05-14T16:59:58.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 16:59:58 smithi123 ceph-mon[35971]: pgmap v161: 97 pgs: 97 active+clean; 2.9 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 47 MiB/s wr, 886 op/s 2024-05-14T16:59:58.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 16:59:58 smithi002 ceph-mon[31538]: pgmap v161: 97 pgs: 97 active+clean; 2.9 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 47 MiB/s wr, 886 op/s 2024-05-14T17:00:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.797+0000 7f73f61a4640 1 -- 172.21.15.2:0/4073756894 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73f0102340 msgr2=0x7f73f010e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.797+0000 7f73f61a4640 1 --2- 172.21.15.2:0/4073756894 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73f0102340 0x7f73f010e800 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f73e8009cd0 tx=0x7f73e802a8e0 comp rx=0 tx=0).stop 2024-05-14T17:00:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.797+0000 7f73f61a4640 1 -- 172.21.15.2:0/4073756894 shutdown_connections 2024-05-14T17:00:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.797+0000 7f73f61a4640 1 --2- 172.21.15.2:0/4073756894 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73f0102340 0x7f73f010e800 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.797+0000 7f73f61a4640 1 --2- 172.21.15.2:0/4073756894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73f0101980 0x7f73f0101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.797+0000 7f73f61a4640 1 --2- 172.21.15.2:0/4073756894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73f0107940 0x7f73f0107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.797+0000 7f73f61a4640 1 -- 172.21.15.2:0/4073756894 >> 172.21.15.2:0/4073756894 conn(0x7f73f0078ad0 msgr2=0x7f73f0078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.797+0000 7f73f61a4640 1 -- 172.21.15.2:0/4073756894 shutdown_connections 2024-05-14T17:00:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.797+0000 7f73f61a4640 1 -- 172.21.15.2:0/4073756894 wait complete. 2024-05-14T17:00:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.798+0000 7f73f61a4640 1 Processor -- start 2024-05-14T17:00:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.798+0000 7f73f61a4640 1 -- start start 2024-05-14T17:00:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.798+0000 7f73f61a4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73f0101980 0x7f73f0198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.798+0000 7f73f61a4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73f0102340 0x7f73f0198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.798+0000 7f73f61a4640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73f0107940 0x7f73f019c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.798+0000 7f73f61a4640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f73f0111c00 con 0x7f73f0101980 2024-05-14T17:00:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.798+0000 7f73f61a4640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f73f0111a80 con 0x7f73f0102340 2024-05-14T17:00:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.798+0000 7f73f61a4640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f73f0111d80 con 0x7f73f0107940 2024-05-14T17:00:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.798+0000 7f73ef7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73f0101980 0x7f73f0198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.798+0000 7f73ef7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73f0101980 0x7f73f0198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46406/0 (socket says 172.21.15.2:46406) 2024-05-14T17:00:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.798+0000 7f73ef7fe640 1 -- 172.21.15.2:0/313254666 learned_addr learned my addr 172.21.15.2:0/313254666 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:00:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.798+0000 7f73e7fff640 1 --2- 172.21.15.2:0/313254666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73f0102340 0x7f73f0198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.798+0000 7f73effff640 1 --2- 172.21.15.2:0/313254666 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73f0107940 0x7f73f019c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:01.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.798+0000 7f73ef7fe640 1 -- 172.21.15.2:0/313254666 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73f0107940 msgr2=0x7f73f019c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:01.090 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.798+0000 7f73ef7fe640 1 --2- 172.21.15.2:0/313254666 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73f0107940 0x7f73f019c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:01.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.798+0000 7f73ef7fe640 1 -- 172.21.15.2:0/313254666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73f0102340 msgr2=0x7f73f0198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.798+0000 7f73ef7fe640 1 --2- 172.21.15.2:0/313254666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73f0102340 0x7f73f0198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.798+0000 7f73ef7fe640 1 -- 172.21.15.2:0/313254666 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f019d0a0 con 0x7f73f0101980 2024-05-14T17:00:01.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.798+0000 7f73e7fff640 1 --2- 172.21.15.2:0/313254666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73f0102340 0x7f73f0198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:00:01.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.799+0000 7f73ef7fe640 1 --2- 172.21.15.2:0/313254666 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73f0101980 0x7f73f0198100 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f73e000e9f0 tx=0x7f73e000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:01.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.799+0000 7f73ed7fa640 1 -- 172.21.15.2:0/313254666 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e000cd80 con 0x7f73f0101980 2024-05-14T17:00:01.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.799+0000 7f73ed7fa640 1 -- 172.21.15.2:0/313254666 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73e00108e0 con 0x7f73f0101980 2024-05-14T17:00:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.799+0000 7f73ed7fa640 1 -- 172.21.15.2:0/313254666 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e0018980 con 0x7f73f0101980 2024-05-14T17:00:01.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.799+0000 7f73f61a4640 1 -- 172.21.15.2:0/313254666 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73f019d390 con 0x7f73f0101980 2024-05-14T17:00:01.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.799+0000 7f73f61a4640 1 -- 172.21.15.2:0/313254666 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73f01a4c20 con 0x7f73f0101980 2024-05-14T17:00:01.178 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:00:00 smithi123 ceph-mon[35971]: pgmap v162: 97 pgs: 97 active+clean; 2.9 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 46 MiB/s wr, 917 op/s 2024-05-14T17:00:01.211 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:00:00 smithi123 ceph-mon[35971]: overall HEALTH_OK 2024-05-14T17:00:01.218 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:00:00 smithi151 ceph-mon[36066]: pgmap v162: 97 pgs: 97 active+clean; 2.9 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 46 MiB/s wr, 917 op/s 2024-05-14T17:00:01.225 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:00:00 smithi151 ceph-mon[36066]: overall HEALTH_OK 2024-05-14T17:00:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.802+0000 7f73ed7fa640 1 -- 172.21.15.2:0/313254666 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f73e0010430 con 0x7f73f0101980 2024-05-14T17:00:01.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.802+0000 7f73f8f99740 1 Processor -- start 2024-05-14T17:00:01.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.802+0000 7f73f8f99740 1 -- start start 2024-05-14T17:00:01.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.802+0000 7f73f8f99740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55acd7e387b0 0x55acd7e38b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:01.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.802+0000 7f73f8f99740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acd7e390d0 0x55acd7e457a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:01.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.802+0000 7f73f8f99740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acd7e45ce0 0x55acd7e480d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.802+0000 7f73f8f99740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55acd7dabc30 con 0x55acd7e387b0 2024-05-14T17:00:01.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.802+0000 7f73f8f99740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55acd7d9b7d0 con 0x55acd7e45ce0 2024-05-14T17:00:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.802+0000 7f73f8f99740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55acd7d9f810 con 0x55acd7e390d0 2024-05-14T17:00:01.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.802+0000 7f73effff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55acd7e387b0 0x55acd7e38b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:01.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.802+0000 7f73effff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55acd7e387b0 0x55acd7e38b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46414/0 (socket says 172.21.15.2:46414) 2024-05-14T17:00:01.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.802+0000 7f73effff640 1 -- 172.21.15.2:0/3049968157 learned_addr learned my addr 172.21.15.2:0/3049968157 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:00:01.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.802+0000 7f73e7fff640 1 --2- 172.21.15.2:0/3049968157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acd7e45ce0 0x55acd7e480d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:01.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.802+0000 7f73ef7fe640 1 --2- 172.21.15.2:0/3049968157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acd7e390d0 0x55acd7e457a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.803+0000 7f73effff640 1 -- 172.21.15.2:0/3049968157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acd7e390d0 msgr2=0x55acd7e457a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.803+0000 7f73effff640 1 --2- 172.21.15.2:0/3049968157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acd7e390d0 0x55acd7e457a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.803+0000 7f73effff640 1 -- 172.21.15.2:0/3049968157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acd7e45ce0 msgr2=0x55acd7e480d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.803+0000 7f73effff640 1 --2- 172.21.15.2:0/3049968157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acd7e45ce0 0x55acd7e480d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.803+0000 7f73effff640 1 -- 172.21.15.2:0/3049968157 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acd7ab0210 con 0x55acd7e387b0 2024-05-14T17:00:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.803+0000 7f73e7fff640 1 --2- 172.21.15.2:0/3049968157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acd7e45ce0 0x55acd7e480d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:00:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.803+0000 7f73effff640 1 --2- 172.21.15.2:0/3049968157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55acd7e387b0 0x55acd7e38b90 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f73e8009cd0 tx=0x7f73e8009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b4dbebe6d16654a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.803+0000 7f73ed7fa640 1 --2- 172.21.15.2:0/313254666 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f73dc0b2a90 0x7f73dc0b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.803+0000 7f73e5ffb640 1 -- 172.21.15.2:0/3049968157 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e8049070 con 0x55acd7e387b0 2024-05-14T17:00:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.803+0000 7f73ef7fe640 1 --2- 172.21.15.2:0/313254666 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f73dc0b2a90 0x7f73dc0b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.803+0000 7f73e5ffb640 1 -- 172.21.15.2:0/3049968157 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73e8036040 con 0x55acd7e387b0 2024-05-14T17:00:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.803+0000 7f73e5ffb640 1 -- 172.21.15.2:0/3049968157 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e8004040 con 0x55acd7e387b0 2024-05-14T17:00:01.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.803+0000 7f73ed7fa640 1 -- 172.21.15.2:0/313254666 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f73e00e2820 con 0x7f73f0101980 2024-05-14T17:00:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.804+0000 7f73f8f99740 1 -- 172.21.15.2:0/3049968157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55acd7e387b0 msgr2=0x55acd7e38b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.804+0000 7f73ef7fe640 1 --2- 172.21.15.2:0/313254666 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f73dc0b2a90 0x7f73dc0b4f50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f73e00e0050 tx=0x7f73e0000bc0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.804+0000 7f73f8f99740 1 --2- 172.21.15.2:0/3049968157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55acd7e387b0 0x55acd7e38b90 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f73e8009cd0 tx=0x7f73e8009a20 comp rx=0 tx=0).stop 2024-05-14T17:00:01.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.804+0000 7f73f8f99740 1 -- 172.21.15.2:0/3049968157 shutdown_connections 2024-05-14T17:00:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.804+0000 7f73f8f99740 1 --2- 172.21.15.2:0/3049968157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acd7e45ce0 0x55acd7e480d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.804+0000 7f73f8f99740 1 --2- 172.21.15.2:0/3049968157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acd7e390d0 0x55acd7e457a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.804+0000 7f73f8f99740 1 --2- 172.21.15.2:0/3049968157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55acd7e387b0 0x55acd7e38b90 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.804+0000 7f73f8f99740 1 -- 172.21.15.2:0/3049968157 >> 172.21.15.2:0/3049968157 conn(0x55acd7e34e00 msgr2=0x55acd7e351e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.804+0000 7f73f8f99740 1 -- 172.21.15.2:0/3049968157 shutdown_connections 2024-05-14T17:00:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.804+0000 7f73f8f99740 1 -- 172.21.15.2:0/3049968157 wait complete. 2024-05-14T17:00:01.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.805+0000 7f73f8f99740 1 -- start start 2024-05-14T17:00:01.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.805+0000 7f73f8f99740 1 Processor -- start 2024-05-14T17:00:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.805+0000 7f73f8f99740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acd7e387b0 0x55acd7e45680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:01.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.805+0000 7f73f8f99740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55acd7e390d0 0x55acd7e42970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:01.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.805+0000 7f73f8f99740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acd7e45ce0 0x55acd7e42f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.805+0000 7f73f8f99740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55acd7a81790 con 0x55acd7e390d0 2024-05-14T17:00:01.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.805+0000 7f73f8f99740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55acd7d57890 con 0x55acd7e45ce0 2024-05-14T17:00:01.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.805+0000 7f73f8f99740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55acd7e488f0 con 0x55acd7e387b0 2024-05-14T17:00:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.805+0000 7f73f8f99740 10 client.0 _ll_register_callbacks cb 0x55acd7d5ad60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:00:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.805+0000 7f73e7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55acd7e390d0 0x55acd7e42970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.805+0000 7f73e7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55acd7e390d0 0x55acd7e42970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46422/0 (socket says 172.21.15.2:46422) 2024-05-14T17:00:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.805+0000 7f73e7fff640 1 -- 172.21.15.2:0/1428805563 learned_addr learned my addr 172.21.15.2:0/1428805563 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:00:01.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.806+0000 7f73effff640 1 --2- 172.21.15.2:0/1428805563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acd7e387b0 0x55acd7e45680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.806+0000 7f73effff640 1 --2- 172.21.15.2:0/1428805563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acd7e45ce0 0x55acd7e42f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.806+0000 7f73effff640 1 -- 172.21.15.2:0/1428805563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acd7e45ce0 msgr2=0x55acd7e42f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:01.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.806+0000 7f73effff640 1 --2- 172.21.15.2:0/1428805563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acd7e45ce0 0x55acd7e42f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.806+0000 7f73effff640 1 -- 172.21.15.2:0/1428805563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55acd7e390d0 msgr2=0x55acd7e42970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:01.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.806+0000 7f73effff640 1 --2- 172.21.15.2:0/1428805563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55acd7e390d0 0x55acd7e42970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:01.734 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.806+0000 7f73effff640 1 -- 172.21.15.2:0/1428805563 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acd7b7c620 con 0x55acd7e387b0 2024-05-14T17:00:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.806+0000 7f73e7fff640 1 --2- 172.21.15.2:0/1428805563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55acd7e390d0 0x55acd7e42970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:00:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.806+0000 7f73effff640 1 --2- 172.21.15.2:0/1428805563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acd7e387b0 0x55acd7e45680 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f73e8006010 tx=0x7f73e8003c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e44b61f290ed23e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.806+0000 7f73ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-14T17:00:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.806+0000 7f73ceffd640 1 -- 172.21.15.2:0/1428805563 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e8049040 con 0x55acd7e387b0 2024-05-14T17:00:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.806+0000 7f73f8f99740 1 -- 172.21.15.2:0/1428805563 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73e8036040 con 0x55acd7e387b0 2024-05-14T17:00:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.807+0000 7f73ceffd640 1 -- 172.21.15.2:0/1428805563 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f73e8004200 con 0x55acd7e387b0 2024-05-14T17:00:01.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.807+0000 7f73ceffd640 1 -- 172.21.15.2:0/1428805563 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f73e8004400 con 0x55acd7e387b0 2024-05-14T17:00:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.807+0000 7f73f8f99740 10 client.25582 fetch_fsmap learned FSMap version 36 2024-05-14T17:00:01.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.807+0000 7f73f8f99740 1 -- 172.21.15.2:0/1428805563 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55acd7e543d0 con 0x55acd7e387b0 2024-05-14T17:00:01.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.807+0000 7f73ceffd640 1 -- 172.21.15.2:0/1428805563 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73e8036040 con 0x55acd7e387b0 2024-05-14T17:00:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.807+0000 7f73ceffd640 1 -- 172.21.15.2:0/1428805563 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 36) ==== 8342+0+0 (secure 0 0 0) 0x7f73e804f070 con 0x55acd7e387b0 2024-05-14T17:00:01.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.807+0000 7f73f8f99740 10 client.25582 fetch_fsmap finished waiting for FSMap version 36 2024-05-14T17:00:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.807+0000 7f73f8f99740 10 client.25582 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:00:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.807+0000 7f73f8f99740 20 client.25582 populate_metadata read hostname 'smithi002' 2024-05-14T17:00:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.807+0000 7f73f8f99740 1 --2- 172.21.15.2:0/1428805563 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55acd7e55720 0x55acd7e57b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.807+0000 7f73f8f99740 4 client.25582 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:00:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.807+0000 7f73f8f99740 1 -- 172.21.15.2:0/1428805563 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55acd7dadb80 con 0x55acd7e55720 2024-05-14T17:00:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.807+0000 7f73e7fff640 1 --2- 172.21.15.2:0/1428805563 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55acd7e55720 0x55acd7e57b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.808+0000 7f73e7fff640 1 --2- 172.21.15.2:0/1428805563 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55acd7e55720 0x55acd7e57b10 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e2d485a651d9a44 server_cookie=27780e1dd7edd8cf in_seq=0 out_seq=0 2024-05-14T17:00:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.808+0000 7f73ceffd640 10 client.25582 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:00:01.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.808+0000 7f73ceffd640 1 -- 172.21.15.2:0/1428805563 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55acd7dadb80 con 0x55acd7e55720 2024-05-14T17:00:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.808+0000 7f73ceffd640 10 client.25582 handle_command_reply: tid=0 2024-05-14T17:00:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.809+0000 7f73f8f99740 1 client.25582 shutdown 2024-05-14T17:00:01.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.809+0000 7f73f8f99740 1 -- 172.21.15.2:0/1428805563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acd7e387b0 msgr2=0x55acd7e45680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:01.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.809+0000 7f73f8f99740 1 --2- 172.21.15.2:0/1428805563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acd7e387b0 0x55acd7e45680 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f73e8006010 tx=0x7f73e8003c30 comp rx=0 tx=0).stop 2024-05-14T17:00:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.809+0000 7f73f8f99740 1 -- 172.21.15.2:0/1428805563 shutdown_connections 2024-05-14T17:00:02.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.809+0000 7f73f8f99740 1 --2- 172.21.15.2:0/1428805563 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55acd7e55720 0x55acd7e57b10 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:02.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.809+0000 7f73f8f99740 1 --2- 172.21.15.2:0/1428805563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acd7e45ce0 0x55acd7e42f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:02.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.809+0000 7f73f8f99740 1 --2- 172.21.15.2:0/1428805563 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55acd7e390d0 0x55acd7e42970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:02.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.809+0000 7f73f8f99740 1 --2- 172.21.15.2:0/1428805563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acd7e387b0 0x55acd7e45680 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:02.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.809+0000 7f73f8f99740 1 -- 172.21.15.2:0/1428805563 >> 172.21.15.2:0/1428805563 conn(0x55acd7e34e00 msgr2=0x55acd7e34100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:02.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.809+0000 7f73ceffd640 0 client.25582 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:00:02.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.809+0000 7f73f8f99740 1 -- 172.21.15.2:0/1428805563 shutdown_connections 2024-05-14T17:00:02.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.809+0000 7f73f8f99740 1 -- 172.21.15.2:0/1428805563 wait complete. 2024-05-14T17:00:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.809+0000 7f73f8f99740 20 client.25582 trim_cache size 0 max 16384 2024-05-14T17:00:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.827+0000 7f73f8f99740 1 Processor -- start 2024-05-14T17:00:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.827+0000 7f73f8f99740 1 -- start start 2024-05-14T17:00:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.827+0000 7f73f8f99740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acd7e387b0 0x55acd7e41ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:02.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.827+0000 7f73f8f99740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acd7e390d0 0x55acd7e3de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:02.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.827+0000 7f73f8f99740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55acd7e45ce0 0x55acd7e451b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.827+0000 7f73f8f99740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f73f01aeb90 con 0x55acd7e45ce0 2024-05-14T17:00:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.827+0000 7f73f8f99740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f73f01ac460 con 0x55acd7e387b0 2024-05-14T17:00:02.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.827+0000 7f73f8f99740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f73f01aa2b0 con 0x55acd7e390d0 2024-05-14T17:00:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.827+0000 7f73effff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55acd7e45ce0 0x55acd7e451b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.827+0000 7f73e7fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acd7e390d0 0x55acd7e3de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.827+0000 7f73effff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55acd7e45ce0 0x55acd7e451b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46424/0 (socket says 172.21.15.2:46424) 2024-05-14T17:00:02.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.827+0000 7f73e7fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acd7e390d0 0x55acd7e3de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:44428/0 (socket says 172.21.15.2:44428) 2024-05-14T17:00:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.827+0000 7f73effff640 1 -- 172.21.15.2:0/967636846 learned_addr learned my addr 172.21.15.2:0/967636846 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:00:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.827+0000 7f73effff640 1 --2- 172.21.15.2:0/967636846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acd7e387b0 0x55acd7e41ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.828+0000 7f73effff640 1 -- 172.21.15.2:0/967636846 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acd7e390d0 msgr2=0x55acd7e3de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:02.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.828+0000 7f73effff640 1 --2- 172.21.15.2:0/967636846 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acd7e390d0 0x55acd7e3de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.828+0000 7f73effff640 1 -- 172.21.15.2:0/967636846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acd7e387b0 msgr2=0x55acd7e41ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.828+0000 7f73effff640 1 --2- 172.21.15.2:0/967636846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acd7e387b0 0x55acd7e41ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.828+0000 7f73effff640 1 -- 172.21.15.2:0/967636846 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acd7e35b90 con 0x55acd7e45ce0 2024-05-14T17:00:02.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.828+0000 7f73effff640 1 --2- 172.21.15.2:0/967636846 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55acd7e45ce0 0x55acd7e451b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f73e8009620 tx=0x7f73e8009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae97098dc1a8d23 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:02.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.828+0000 7f73e57fa640 1 -- 172.21.15.2:0/967636846 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e8004730 con 0x55acd7e45ce0 2024-05-14T17:00:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.828+0000 7f73e57fa640 1 -- 172.21.15.2:0/967636846 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73e8036070 con 0x55acd7e45ce0 2024-05-14T17:00:02.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.828+0000 7f73e57fa640 1 -- 172.21.15.2:0/967636846 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e805b450 con 0x55acd7e45ce0 2024-05-14T17:00:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.829+0000 7f73f8f99740 1 -- 172.21.15.2:0/967636846 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55acd7e45ce0 msgr2=0x55acd7e451b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.829+0000 7f73f8f99740 1 --2- 172.21.15.2:0/967636846 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55acd7e45ce0 0x55acd7e451b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f73e8009620 tx=0x7f73e8009a20 comp rx=0 tx=0).stop 2024-05-14T17:00:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.829+0000 7f73f8f99740 1 -- 172.21.15.2:0/967636846 shutdown_connections 2024-05-14T17:00:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.829+0000 7f73f8f99740 1 --2- 172.21.15.2:0/967636846 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55acd7e45ce0 0x55acd7e451b0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.829+0000 7f73f8f99740 1 --2- 172.21.15.2:0/967636846 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acd7e390d0 0x55acd7e3de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.829+0000 7f73f8f99740 1 --2- 172.21.15.2:0/967636846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acd7e387b0 0x55acd7e41ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.829+0000 7f73f8f99740 1 -- 172.21.15.2:0/967636846 >> 172.21.15.2:0/967636846 conn(0x55acd7e34e00 msgr2=0x55acd7e3d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.829+0000 7f73f8f99740 1 -- 172.21.15.2:0/967636846 shutdown_connections 2024-05-14T17:00:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.829+0000 7f73f8f99740 1 -- 172.21.15.2:0/967636846 wait complete. 2024-05-14T17:00:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.829+0000 7f73f8f99740 1 -- start start 2024-05-14T17:00:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.830+0000 7f73f8f99740 1 Processor -- start 2024-05-14T17:00:02.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.830+0000 7f73f8f99740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55acd7e387b0 0x55acd7e57460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:02.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.830+0000 7f73f8f99740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acd7e390d0 0x55acd7e579a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:02.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.830+0000 7f73f8f99740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acd7e45ce0 0x55acd7e57ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.830+0000 7f73f8f99740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f73f01aa920 con 0x55acd7e387b0 2024-05-14T17:00:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.830+0000 7f73f8f99740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f73f01abd90 con 0x55acd7e390d0 2024-05-14T17:00:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.830+0000 7f73f8f99740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f73f01aaff0 con 0x55acd7e45ce0 2024-05-14T17:00:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.830+0000 7f73f8f99740 10 client.0 _ll_register_callbacks cb 0x7f73e8003e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:00:02.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.830+0000 7f73effff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55acd7e387b0 0x55acd7e57460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.830+0000 7f73e7fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acd7e390d0 0x55acd7e579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:02.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.830+0000 7f73e7fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acd7e390d0 0x55acd7e579a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:32868/0 (socket says 172.21.15.2:32868) 2024-05-14T17:00:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.830+0000 7f73e7fff640 1 -- 172.21.15.2:0/453422371 learned_addr learned my addr 172.21.15.2:0/453422371 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:00:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.830+0000 7f73effff640 1 --2- 172.21.15.2:0/453422371 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acd7e45ce0 0x55acd7e57ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.830+0000 7f73e7fff640 1 -- 172.21.15.2:0/453422371 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acd7e45ce0 msgr2=0x55acd7e57ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:03.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.830+0000 7f73e7fff640 1 --2- 172.21.15.2:0/453422371 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acd7e45ce0 0x55acd7e57ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:03.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.830+0000 7f73e7fff640 1 -- 172.21.15.2:0/453422371 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55acd7e387b0 msgr2=0x55acd7e57460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:03.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.830+0000 7f73e7fff640 1 --2- 172.21.15.2:0/453422371 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55acd7e387b0 0x55acd7e57460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.830+0000 7f73e7fff640 1 -- 172.21.15.2:0/453422371 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acd7ebdd90 con 0x55acd7e390d0 2024-05-14T17:00:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.831+0000 7f73e7fff640 1 --2- 172.21.15.2:0/453422371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acd7e390d0 0x55acd7e579a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x55acd7dabcf0 tx=0x7f73d80207d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb0366836c744d3e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.831+0000 7f73ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T17:00:03.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.831+0000 7f73ceffd640 1 -- 172.21.15.2:0/453422371 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d80120d0 con 0x55acd7e390d0 2024-05-14T17:00:03.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.831+0000 7f73ceffd640 1 -- 172.21.15.2:0/453422371 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f73d8005020 con 0x55acd7e390d0 2024-05-14T17:00:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.831+0000 7f73f8f99740 1 -- 172.21.15.2:0/453422371 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73e8036070 con 0x55acd7e390d0 2024-05-14T17:00:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.831+0000 7f73ceffd640 1 -- 172.21.15.2:0/453422371 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f73d8032050 con 0x55acd7e390d0 2024-05-14T17:00:03.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.831+0000 7f73f8f99740 10 client.25607 fetch_fsmap learned FSMap version 36 2024-05-14T17:00:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.831+0000 7f73f8f99740 1 -- 172.21.15.2:0/453422371 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55acd7ebf080 con 0x55acd7e390d0 2024-05-14T17:00:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.831+0000 7f73ceffd640 1 -- 172.21.15.2:0/453422371 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73e8036070 con 0x55acd7e390d0 2024-05-14T17:00:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.831+0000 7f73ceffd640 1 -- 172.21.15.2:0/453422371 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 36) ==== 8342+0+0 (secure 0 0 0) 0x7f73d8004ca0 con 0x55acd7e390d0 2024-05-14T17:00:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.831+0000 7f73f8f99740 10 client.25607 fetch_fsmap finished waiting for FSMap version 36 2024-05-14T17:00:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.831+0000 7f73f8f99740 10 client.25607 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:00:03.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.831+0000 7f73f8f99740 20 client.25607 populate_metadata read hostname 'smithi002' 2024-05-14T17:00:03.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.831+0000 7f73f8f99740 1 --2- 172.21.15.2:0/453422371 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55acd7ebf810 0x55acd7ec1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:03.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.831+0000 7f73f8f99740 4 client.25607 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:00:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.831+0000 7f73f8f99740 1 -- 172.21.15.2:0/453422371 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55acd7da7360 con 0x55acd7ebf810 2024-05-14T17:00:03.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.832+0000 7f73effff640 1 --2- 172.21.15.2:0/453422371 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55acd7ebf810 0x55acd7ec1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.832+0000 7f73effff640 1 --2- 172.21.15.2:0/453422371 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55acd7ebf810 0x55acd7ec1c30 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af546cfde2303c75 server_cookie=43e0a35f928a6454 in_seq=0 out_seq=0 2024-05-14T17:00:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.832+0000 7f73ceffd640 10 client.25607 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:00:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.844+0000 7f73ceffd640 1 -- 172.21.15.2:0/453422371 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55acd7da7360 con 0x55acd7ebf810 2024-05-14T17:00:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.844+0000 7f73ceffd640 10 client.25607 handle_command_reply: tid=0 2024-05-14T17:00:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.844+0000 7f73f8f99740 1 client.25607 shutdown 2024-05-14T17:00:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.844+0000 7f73f8f99740 1 -- 172.21.15.2:0/453422371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acd7e390d0 msgr2=0x55acd7e579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.844+0000 7f73f8f99740 1 --2- 172.21.15.2:0/453422371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acd7e390d0 0x55acd7e579a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x55acd7dabcf0 tx=0x7f73d80207d0 comp rx=0 tx=0).stop 2024-05-14T17:00:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.844+0000 7f73f8f99740 1 -- 172.21.15.2:0/453422371 shutdown_connections 2024-05-14T17:00:03.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.844+0000 7f73f8f99740 1 --2- 172.21.15.2:0/453422371 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55acd7ebf810 0x55acd7ec1c30 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:03.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.844+0000 7f73f8f99740 1 --2- 172.21.15.2:0/453422371 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acd7e45ce0 0x55acd7e57ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:03.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.844+0000 7f73ceffd640 0 client.25607 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:00:03.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.844+0000 7f73f8f99740 1 --2- 172.21.15.2:0/453422371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55acd7e390d0 0x55acd7e579a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.844+0000 7f73f8f99740 1 --2- 172.21.15.2:0/453422371 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55acd7e387b0 0x55acd7e57460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:03.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.844+0000 7f73f8f99740 1 -- 172.21.15.2:0/453422371 >> 172.21.15.2:0/453422371 conn(0x55acd7e34e00 msgr2=0x55acd7e3dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:03.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.844+0000 7f73f8f99740 1 -- 172.21.15.2:0/453422371 shutdown_connections 2024-05-14T17:00:03.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.844+0000 7f73f8f99740 1 -- 172.21.15.2:0/453422371 wait complete. 2024-05-14T17:00:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.844+0000 7f73f8f99740 20 client.25607 trim_cache size 0 max 16384 2024-05-14T17:00:03.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:00:03.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:00:03.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:00:03.772 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:00:02 smithi123 ceph-mon[35971]: pgmap v163: 97 pgs: 97 active+clean; 2.9 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 MiB/s wr, 692 op/s 2024-05-14T17:00:03.787 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:00:02 smithi151 ceph-mon[36066]: pgmap v163: 97 pgs: 97 active+clean; 2.9 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 MiB/s wr, 692 op/s 2024-05-14T17:00:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.845+0000 7f73f61a4640 1 -- 172.21.15.2:0/313254666 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f73dc0b2a90 msgr2=0x7f73dc0b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.845+0000 7f73f61a4640 1 --2- 172.21.15.2:0/313254666 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f73dc0b2a90 0x7f73dc0b4f50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f73e00e0050 tx=0x7f73e0000bc0 comp rx=0 tx=0).stop 2024-05-14T17:00:03.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.845+0000 7f73f61a4640 1 -- 172.21.15.2:0/313254666 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73f0101980 msgr2=0x7f73f0198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.845+0000 7f73f61a4640 1 --2- 172.21.15.2:0/313254666 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73f0101980 0x7f73f0198100 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f73e000e9f0 tx=0x7f73e000eeb0 comp rx=0 tx=0).stop 2024-05-14T17:00:03.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.845+0000 7f73f61a4640 1 -- 172.21.15.2:0/313254666 shutdown_connections 2024-05-14T17:00:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.845+0000 7f73f61a4640 1 --2- 172.21.15.2:0/313254666 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f73dc0b2a90 0x7f73dc0b4f50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.845+0000 7f73f61a4640 1 --2- 172.21.15.2:0/313254666 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73f0107940 0x7f73f019c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:03.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.845+0000 7f73f61a4640 1 --2- 172.21.15.2:0/313254666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73f0102340 0x7f73f0198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:03.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.845+0000 7f73f61a4640 1 --2- 172.21.15.2:0/313254666 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73f0101980 0x7f73f0198100 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.845+0000 7f73f61a4640 1 -- 172.21.15.2:0/313254666 >> 172.21.15.2:0/313254666 conn(0x7f73f0078ad0 msgr2=0x7f73f0104540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:03.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.845+0000 7f73f61a4640 1 -- 172.21.15.2:0/313254666 shutdown_connections 2024-05-14T17:00:03.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:00.845+0000 7f73f61a4640 1 -- 172.21.15.2:0/313254666 wait complete. 2024-05-14T17:00:03.916 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:00:00 smithi002 ceph-mon[31538]: pgmap v162: 97 pgs: 97 active+clean; 2.9 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 46 MiB/s wr, 917 op/s 2024-05-14T17:00:03.922 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:00:00 smithi002 ceph-mon[31538]: overall HEALTH_OK 2024-05-14T17:00:03.929 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 285.0 s, 2778.8 tps, lat 35.359 ms stddev 60.382 2024-05-14T17:00:03.936 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:00:02 smithi002 ceph-mon[31538]: pgmap v163: 97 pgs: 97 active+clean; 2.9 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 MiB/s wr, 692 op/s 2024-05-14T17:00:04.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:00:04 smithi151 ceph-mon[36066]: pgmap v164: 97 pgs: 97 active+clean; 3.0 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 47 MiB/s wr, 1.15k op/s 2024-05-14T17:00:04.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:00:04 smithi123 ceph-mon[35971]: pgmap v164: 97 pgs: 97 active+clean; 3.0 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 47 MiB/s wr, 1.15k op/s 2024-05-14T17:00:04.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:00:04 smithi002 ceph-mon[31538]: pgmap v164: 97 pgs: 97 active+clean; 3.0 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 47 MiB/s wr, 1.15k op/s 2024-05-14T17:00:06.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 290.0 s, 401.4 tps, lat 244.338 ms stddev 143.808 2024-05-14T17:00:06.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:00:06 smithi151 ceph-mon[36066]: pgmap v165: 97 pgs: 97 active+clean; 3.0 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 26 MiB/s wr, 817 op/s 2024-05-14T17:00:06.859 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:00:06 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:00:06.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:00:06 smithi123 ceph-mon[35971]: pgmap v165: 97 pgs: 97 active+clean; 3.0 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 26 MiB/s wr, 817 op/s 2024-05-14T17:00:06.916 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:00:06 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:00:06.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:00:06 smithi002 ceph-mon[31538]: pgmap v165: 97 pgs: 97 active+clean; 3.0 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 26 MiB/s wr, 817 op/s 2024-05-14T17:00:07.000 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:00:06 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:00:08.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:00:08 smithi151 ceph-mon[36066]: pgmap v166: 97 pgs: 97 active+clean; 3.0 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 28 MiB/s wr, 938 op/s 2024-05-14T17:00:08.932 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:00:08 smithi123 ceph-mon[35971]: pgmap v166: 97 pgs: 97 active+clean; 3.0 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 28 MiB/s wr, 938 op/s 2024-05-14T17:00:08.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:00:08 smithi002 ceph-mon[31538]: pgmap v166: 97 pgs: 97 active+clean; 3.0 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 28 MiB/s wr, 938 op/s 2024-05-14T17:00:10.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:00:10 smithi151 ceph-mon[36066]: pgmap v167: 97 pgs: 97 active+clean; 3.0 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 24 MiB/s wr, 988 op/s 2024-05-14T17:00:10.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:00:10 smithi123 ceph-mon[35971]: pgmap v167: 97 pgs: 97 active+clean; 3.0 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 24 MiB/s wr, 988 op/s 2024-05-14T17:00:10.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:00:10 smithi002 ceph-mon[31538]: pgmap v167: 97 pgs: 97 active+clean; 3.0 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 24 MiB/s wr, 988 op/s 2024-05-14T17:00:11.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 295.0 s, 2224.0 tps, lat 46.543 ms stddev 126.469 2024-05-14T17:00:12.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:00:12 smithi151 ceph-mon[36066]: pgmap v168: 97 pgs: 97 active+clean; 3.0 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 15 MiB/s wr, 785 op/s 2024-05-14T17:00:12.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:00:12 smithi123 ceph-mon[35971]: pgmap v168: 97 pgs: 97 active+clean; 3.0 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 15 MiB/s wr, 785 op/s 2024-05-14T17:00:12.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:00:12 smithi002 ceph-mon[31538]: pgmap v168: 97 pgs: 97 active+clean; 3.0 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 15 MiB/s wr, 785 op/s 2024-05-14T17:00:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:13.992+0000 7f7945a3f640 1 -- 172.21.15.2:0/2481279060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7940109c80 msgr2=0x7f794010dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:13.992+0000 7f7945a3f640 1 --2- 172.21.15.2:0/2481279060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7940109c80 0x7f794010dcc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7934009900 tx=0x7f793402a3f0 comp rx=0 tx=0).stop 2024-05-14T17:00:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:13.992+0000 7f7945a3f640 1 -- 172.21.15.2:0/2481279060 shutdown_connections 2024-05-14T17:00:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:13.992+0000 7f7945a3f640 1 --2- 172.21.15.2:0/2481279060 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f794010e3a0 0x7f7940115f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:13.992+0000 7f7945a3f640 1 --2- 172.21.15.2:0/2481279060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7940109c80 0x7f794010dcc0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:13.992+0000 7f7945a3f640 1 --2- 172.21.15.2:0/2481279060 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79401092d0 0x7f79401096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:13.992+0000 7f7945a3f640 1 -- 172.21.15.2:0/2481279060 >> 172.21.15.2:0/2481279060 conn(0x7f7940076fd0 msgr2=0x7f79400793f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:13.992+0000 7f7945a3f640 1 -- 172.21.15.2:0/2481279060 shutdown_connections 2024-05-14T17:00:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:13.992+0000 7f7945a3f640 1 -- 172.21.15.2:0/2481279060 wait complete. 2024-05-14T17:00:14.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:13.993+0000 7f7945a3f640 1 Processor -- start 2024-05-14T17:00:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:13.993+0000 7f7945a3f640 1 -- start start 2024-05-14T17:00:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:13.994+0000 7f7945a3f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79401092d0 0x7f79401a56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:13.994+0000 7f7945a3f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f794010e3a0 0x7f79401a5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:13.994+0000 7f7945a3f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f794019f840 0x7f794019fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:13.994+0000 7f7945a3f640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7940119310 con 0x7f794019f840 2024-05-14T17:00:14.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:13.994+0000 7f793effd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79401092d0 0x7f79401a56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:14.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:13.994+0000 7f793effd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79401092d0 0x7f79401a56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:49446/0 (socket says 172.21.15.2:49446) 2024-05-14T17:00:14.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:13.994+0000 7f793effd640 1 -- 172.21.15.2:0/3969109541 learned_addr learned my addr 172.21.15.2:0/3969109541 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:00:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:13.994+0000 7f793f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f794019f840 0x7f794019fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:14.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:13.995+0000 7f7945a3f640 1 -- 172.21.15.2:0/3969109541 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7940119190 con 0x7f79401092d0 2024-05-14T17:00:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:13.995+0000 7f7945a3f640 1 -- 172.21.15.2:0/3969109541 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7940119490 con 0x7f794010e3a0 2024-05-14T17:00:14.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:13.994+0000 7f793e7fc640 1 --2- 172.21.15.2:0/3969109541 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f794010e3a0 0x7f79401a5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:14.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:13.995+0000 7f793e7fc640 1 -- 172.21.15.2:0/3969109541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79401092d0 msgr2=0x7f79401a56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:14.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:13.995+0000 7f793e7fc640 1 --2- 172.21.15.2:0/3969109541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79401092d0 0x7f79401a56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:14.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:13.995+0000 7f793e7fc640 1 -- 172.21.15.2:0/3969109541 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f794019f840 msgr2=0x7f794019fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:14.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:13.995+0000 7f793e7fc640 1 --2- 172.21.15.2:0/3969109541 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f794019f840 0x7f794019fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:14.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:13.995+0000 7f793e7fc640 1 -- 172.21.15.2:0/3969109541 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79401a03d0 con 0x7f794010e3a0 2024-05-14T17:00:14.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:13.995+0000 7f793effd640 1 --2- 172.21.15.2:0/3969109541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79401092d0 0x7f79401a56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:00:14.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:13.996+0000 7f793f7fe640 1 --2- 172.21.15.2:0/3969109541 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f794019f840 0x7f794019fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:00:14.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:13.996+0000 7f793e7fc640 1 --2- 172.21.15.2:0/3969109541 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f794010e3a0 0x7f79401a5c10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f794010a1d0 tx=0x7f7934003c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:13.996+0000 7f7944a3d640 1 -- 172.21.15.2:0/3969109541 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793403f020 con 0x7f794010e3a0 2024-05-14T17:00:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:13.996+0000 7f7944a3d640 1 -- 172.21.15.2:0/3969109541 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7934031b50 con 0x7f794010e3a0 2024-05-14T17:00:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:13.996+0000 7f7945a3f640 1 -- 172.21.15.2:0/3969109541 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79401a0660 con 0x7f794010e3a0 2024-05-14T17:00:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:13.996+0000 7f7944a3d640 1 -- 172.21.15.2:0/3969109541 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7934035070 con 0x7f794010e3a0 2024-05-14T17:00:14.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:13.996+0000 7f7945a3f640 1 -- 172.21.15.2:0/3969109541 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79401ac810 con 0x7f794010e3a0 2024-05-14T17:00:14.691 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.000+0000 7f7944a3d640 1 -- 172.21.15.2:0/3969109541 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f7934042020 con 0x7f794010e3a0 2024-05-14T17:00:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.000+0000 7f7948830740 1 Processor -- start 2024-05-14T17:00:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.002+0000 7f7948830740 1 -- start start 2024-05-14T17:00:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.002+0000 7f7948830740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d3690d8f0 0x563d3690dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.002+0000 7f7948830740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d3690e210 0x563d36918590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:14.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.002+0000 7f7948830740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d36918ad0 0x563d3691aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.002+0000 7f7948830740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563d36880c30 con 0x563d3690e210 2024-05-14T17:00:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.002+0000 7f7948830740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563d368707d0 con 0x563d36918ad0 2024-05-14T17:00:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.002+0000 7f7948830740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563d36874810 con 0x563d3690d8f0 2024-05-14T17:00:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.003+0000 7f793effd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d3690e210 0x563d36918590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.003+0000 7f793f7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d3690d8f0 0x563d3690dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:14.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.003+0000 7f793e7fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d36918ad0 0x563d3691aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.003+0000 7f793f7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d3690d8f0 0x563d3690dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:60548/0 (socket says 172.21.15.2:60548) 2024-05-14T17:00:14.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.003+0000 7f793f7fe640 1 -- 172.21.15.2:0/3146059126 learned_addr learned my addr 172.21.15.2:0/3146059126 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:00:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.003+0000 7f7944a3d640 1 --2- 172.21.15.2:0/3969109541 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f79280b2ae0 0x7f79280b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.003+0000 7f793e7fc640 1 -- 172.21.15.2:0/3146059126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d3690d8f0 msgr2=0x563d3690dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.003+0000 7f793e7fc640 1 --2- 172.21.15.2:0/3146059126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d3690d8f0 0x563d3690dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:15.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.003+0000 7f793e7fc640 1 -- 172.21.15.2:0/3146059126 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d3690e210 msgr2=0x563d36918590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.003+0000 7f793e7fc640 1 --2- 172.21.15.2:0/3146059126 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d3690e210 0x563d36918590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:15.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.003+0000 7f793e7fc640 1 -- 172.21.15.2:0/3146059126 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d36585210 con 0x563d36918ad0 2024-05-14T17:00:15.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.003+0000 7f793f7fe640 1 --2- 172.21.15.2:0/3146059126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d3690d8f0 0x563d3690dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:00:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.003+0000 7f793f7fe640 1 --2- 172.21.15.2:0/3969109541 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f79280b2ae0 0x7f79280b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:15.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.003+0000 7f7944a3d640 1 -- 172.21.15.2:0/3969109541 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f7934102d70 con 0x7f794010e3a0 2024-05-14T17:00:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.003+0000 7f793e7fc640 1 --2- 172.21.15.2:0/3146059126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d36918ad0 0x563d3691aec0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f79341004f0 tx=0x7f79340bf7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4dda0b7d14ea22f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.004+0000 7f7925ffb640 1 -- 172.21.15.2:0/3146059126 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79340cad10 con 0x563d36918ad0 2024-05-14T17:00:15.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.004+0000 7f7925ffb640 1 -- 172.21.15.2:0/3146059126 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f793403b460 con 0x563d36918ad0 2024-05-14T17:00:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.004+0000 7f7925ffb640 1 -- 172.21.15.2:0/3146059126 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79340ca6b0 con 0x563d36918ad0 2024-05-14T17:00:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.004+0000 7f793f7fe640 1 --2- 172.21.15.2:0/3969109541 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f79280b2ae0 0x7f79280b4fa0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f79380115e0 tx=0x7f7938004a40 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.005+0000 7f7948830740 1 -- 172.21.15.2:0/3146059126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d36918ad0 msgr2=0x563d3691aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:15.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.005+0000 7f7948830740 1 --2- 172.21.15.2:0/3146059126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d36918ad0 0x563d3691aec0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f79341004f0 tx=0x7f79340bf7c0 comp rx=0 tx=0).stop 2024-05-14T17:00:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.005+0000 7f7948830740 1 -- 172.21.15.2:0/3146059126 shutdown_connections 2024-05-14T17:00:15.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.005+0000 7f7948830740 1 --2- 172.21.15.2:0/3146059126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d36918ad0 0x563d3691aec0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:15.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.005+0000 7f7948830740 1 --2- 172.21.15.2:0/3146059126 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d3690e210 0x563d36918590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:15.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.005+0000 7f7948830740 1 --2- 172.21.15.2:0/3146059126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d3690d8f0 0x563d3690dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.005+0000 7f7948830740 1 -- 172.21.15.2:0/3146059126 >> 172.21.15.2:0/3146059126 conn(0x563d36909e00 msgr2=0x563d3690a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:15.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.005+0000 7f7948830740 1 -- 172.21.15.2:0/3146059126 shutdown_connections 2024-05-14T17:00:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.005+0000 7f7948830740 1 -- 172.21.15.2:0/3146059126 wait complete. 2024-05-14T17:00:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.006+0000 7f7948830740 1 -- start start 2024-05-14T17:00:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.006+0000 7f7948830740 1 Processor -- start 2024-05-14T17:00:15.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.006+0000 7f7948830740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d3690d8f0 0x563d36918440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.006+0000 7f7948830740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d3690e210 0x563d36915730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:15.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.006+0000 7f7948830740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d36918ad0 0x563d36915c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:15.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.006+0000 7f7948830740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563d36556790 con 0x563d36918ad0 2024-05-14T17:00:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.006+0000 7f7948830740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563d3682c890 con 0x563d3690e210 2024-05-14T17:00:15.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.006+0000 7f7948830740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563d3691b4c0 con 0x563d3690d8f0 2024-05-14T17:00:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.006+0000 7f7948830740 10 client.0 _ll_register_callbacks cb 0x563d3682fd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:00:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.006+0000 7f793effd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d3690d8f0 0x563d36918440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:15.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.006+0000 7f793f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d36918ad0 0x563d36915c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.006+0000 7f793effd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d3690d8f0 0x563d36918440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:60550/0 (socket says 172.21.15.2:60550) 2024-05-14T17:00:15.419 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.006+0000 7f793effd640 1 -- 172.21.15.2:0/454782940 learned_addr learned my addr 172.21.15.2:0/454782940 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:00:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.006+0000 7f793e7fc640 1 --2- 172.21.15.2:0/454782940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d3690e210 0x563d36915730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.007+0000 7f793e7fc640 1 -- 172.21.15.2:0/454782940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d3690d8f0 msgr2=0x563d36918440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:15.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.007+0000 7f793e7fc640 1 --2- 172.21.15.2:0/454782940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d3690d8f0 0x563d36918440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:15.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.007+0000 7f793e7fc640 1 -- 172.21.15.2:0/454782940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d36918ad0 msgr2=0x563d36915c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.007+0000 7f793e7fc640 1 --2- 172.21.15.2:0/454782940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d36918ad0 0x563d36915c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.007+0000 7f793e7fc640 1 -- 172.21.15.2:0/454782940 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d36651620 con 0x563d3690e210 2024-05-14T17:00:15.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.007+0000 7f793f7fe640 1 --2- 172.21.15.2:0/454782940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d36918ad0 0x563d36915c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:00:15.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.007+0000 7f793effd640 1 --2- 172.21.15.2:0/454782940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d3690d8f0 0x563d36918440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:00:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.007+0000 7f793e7fc640 1 --2- 172.21.15.2:0/454782940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d3690e210 0x563d36915730 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f79340caed0 tx=0x7f7934031f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d62fd9bc664934d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.007+0000 7f791effd640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T17:00:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.008+0000 7f791effd640 1 -- 172.21.15.2:0/454782940 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79340ca890 con 0x563d3690e210 2024-05-14T17:00:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.008+0000 7f791effd640 1 -- 172.21.15.2:0/454782940 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f79340c71d0 con 0x563d3690e210 2024-05-14T17:00:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.009+0000 7f7948830740 1 -- 172.21.15.2:0/454782940 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563d36882b80 con 0x563d3690e210 2024-05-14T17:00:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.010+0000 7f791effd640 1 -- 172.21.15.2:0/454782940 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f79340ca6b0 con 0x563d3690e210 2024-05-14T17:00:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.010+0000 7f7948830740 10 client.25619 fetch_fsmap learned FSMap version 37 2024-05-14T17:00:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.010+0000 7f7948830740 1 -- 172.21.15.2:0/454782940 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563d36926ea0 con 0x563d3690e210 2024-05-14T17:00:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.010+0000 7f791effd640 1 -- 172.21.15.2:0/454782940 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563d36882b80 con 0x563d3690e210 2024-05-14T17:00:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.011+0000 7f791effd640 1 -- 172.21.15.2:0/454782940 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f79341000d0 con 0x563d3690e210 2024-05-14T17:00:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.011+0000 7f7948830740 10 client.25619 fetch_fsmap finished waiting for FSMap version 37 2024-05-14T17:00:15.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.011+0000 7f7948830740 10 client.25619 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:00:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.011+0000 7f7948830740 20 client.25619 populate_metadata read hostname 'smithi002' 2024-05-14T17:00:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.011+0000 7f7948830740 1 --2- 172.21.15.2:0/454782940 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x563d369281f0 0x563d3692a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:15.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.011+0000 7f7948830740 4 client.25619 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:00:15.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.011+0000 7f7948830740 1 -- 172.21.15.2:0/454782940 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d3687c360 con 0x563d369281f0 2024-05-14T17:00:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.011+0000 7f793effd640 1 --2- 172.21.15.2:0/454782940 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x563d369281f0 0x563d3692a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.012+0000 7f791effd640 10 client.25619 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:00:15.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.012+0000 7f793effd640 1 --2- 172.21.15.2:0/454782940 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x563d369281f0 0x563d3692a5e0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1add2ba2f1436353 server_cookie=9cb0de18ccc96a7f in_seq=0 out_seq=0 2024-05-14T17:00:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.013+0000 7f791effd640 1 -- 172.21.15.2:0/454782940 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563d3687c360 con 0x563d369281f0 2024-05-14T17:00:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.013+0000 7f791effd640 10 client.25619 handle_command_reply: tid=0 2024-05-14T17:00:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.015+0000 7f7948830740 1 client.25619 shutdown 2024-05-14T17:00:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.015+0000 7f7948830740 1 -- 172.21.15.2:0/454782940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d3690e210 msgr2=0x563d36915730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.015+0000 7f7948830740 1 --2- 172.21.15.2:0/454782940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d3690e210 0x563d36915730 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f79340caed0 tx=0x7f7934031f90 comp rx=0 tx=0).stop 2024-05-14T17:00:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.015+0000 7f7948830740 1 -- 172.21.15.2:0/454782940 shutdown_connections 2024-05-14T17:00:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.015+0000 7f7948830740 1 --2- 172.21.15.2:0/454782940 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x563d369281f0 0x563d3692a5e0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:15.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.015+0000 7f7948830740 1 --2- 172.21.15.2:0/454782940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d36918ad0 0x563d36915c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:15.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.015+0000 7f7948830740 1 --2- 172.21.15.2:0/454782940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d3690e210 0x563d36915730 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:15.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.015+0000 7f7948830740 1 --2- 172.21.15.2:0/454782940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d3690d8f0 0x563d36918440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.015+0000 7f791effd640 0 client.25619 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:00:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.015+0000 7f7948830740 1 -- 172.21.15.2:0/454782940 >> 172.21.15.2:0/454782940 conn(0x563d36909e00 msgr2=0x563d36909100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.016+0000 7f7948830740 1 -- 172.21.15.2:0/454782940 shutdown_connections 2024-05-14T17:00:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.017+0000 7f7948830740 1 -- 172.21.15.2:0/454782940 wait complete. 2024-05-14T17:00:15.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.017+0000 7f7948830740 20 client.25619 trim_cache size 0 max 16384 2024-05-14T17:00:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.042+0000 7f7948830740 1 Processor -- start 2024-05-14T17:00:15.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.043+0000 7f7948830740 1 -- start start 2024-05-14T17:00:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.043+0000 7f7948830740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d3690d8f0 0x563d36917f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.043+0000 7f7948830740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d3690e210 0x563d3692fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.043+0000 7f7948830740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d36918ad0 0x563d369302b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.043+0000 7f7948830740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f79401b79a0 con 0x563d3690e210 2024-05-14T17:00:16.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.043+0000 7f7948830740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f79401b5240 con 0x563d3690d8f0 2024-05-14T17:00:16.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.043+0000 7f7948830740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f79401b3090 con 0x563d36918ad0 2024-05-14T17:00:16.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.043+0000 7f793e7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d3690e210 0x563d3692fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.043+0000 7f793effd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d3690d8f0 0x563d36917f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:16.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.043+0000 7f793f7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d36918ad0 0x563d369302b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.043+0000 7f793e7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d3690e210 0x563d3692fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33586/0 (socket says 172.21.15.2:33586) 2024-05-14T17:00:16.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.043+0000 7f793effd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d3690d8f0 0x563d36917f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:49492/0 (socket says 172.21.15.2:49492) 2024-05-14T17:00:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.043+0000 7f793e7fc640 1 -- 172.21.15.2:0/181752872 learned_addr learned my addr 172.21.15.2:0/181752872 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:00:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.044+0000 7f793f7fe640 1 -- 172.21.15.2:0/181752872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d3690d8f0 msgr2=0x563d36917f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.044+0000 7f793f7fe640 1 --2- 172.21.15.2:0/181752872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d3690d8f0 0x563d36917f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:16.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.044+0000 7f793f7fe640 1 -- 172.21.15.2:0/181752872 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d3690e210 msgr2=0x563d3692fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:16.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.044+0000 7f793f7fe640 1 --2- 172.21.15.2:0/181752872 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d3690e210 0x563d3692fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:16.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.044+0000 7f793f7fe640 1 -- 172.21.15.2:0/181752872 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d36909c30 con 0x563d36918ad0 2024-05-14T17:00:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.044+0000 7f793f7fe640 1 --2- 172.21.15.2:0/181752872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d36918ad0 0x563d369302b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f793800a560 tx=0x7f7938014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6726c170a66701b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.044+0000 7f79257fa640 1 -- 172.21.15.2:0/181752872 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7938012cb0 con 0x563d36918ad0 2024-05-14T17:00:16.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.045+0000 7f79257fa640 1 -- 172.21.15.2:0/181752872 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7938016070 con 0x563d36918ad0 2024-05-14T17:00:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.045+0000 7f79257fa640 1 -- 172.21.15.2:0/181752872 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793801d6a0 con 0x563d36918ad0 2024-05-14T17:00:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.045+0000 7f7948830740 1 -- 172.21.15.2:0/181752872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d36918ad0 msgr2=0x563d369302b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.045+0000 7f7948830740 1 --2- 172.21.15.2:0/181752872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d36918ad0 0x563d369302b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f793800a560 tx=0x7f7938014000 comp rx=0 tx=0).stop 2024-05-14T17:00:16.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.045+0000 7f7948830740 1 -- 172.21.15.2:0/181752872 shutdown_connections 2024-05-14T17:00:16.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.045+0000 7f7948830740 1 --2- 172.21.15.2:0/181752872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d36918ad0 0x563d369302b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.045+0000 7f7948830740 1 --2- 172.21.15.2:0/181752872 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d3690e210 0x563d3692fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.045+0000 7f7948830740 1 --2- 172.21.15.2:0/181752872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d3690d8f0 0x563d36917f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.045+0000 7f7948830740 1 -- 172.21.15.2:0/181752872 >> 172.21.15.2:0/181752872 conn(0x563d36909e00 msgr2=0x563d36904f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.046+0000 7f7948830740 1 -- 172.21.15.2:0/181752872 shutdown_connections 2024-05-14T17:00:16.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.046+0000 7f7948830740 1 -- 172.21.15.2:0/181752872 wait complete. 2024-05-14T17:00:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.046+0000 7f7948830740 1 -- start start 2024-05-14T17:00:16.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.046+0000 7f7948830740 1 Processor -- start 2024-05-14T17:00:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.046+0000 7f7948830740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d3690d8f0 0x563d3698b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.046+0000 7f7948830740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d3690e210 0x563d3698ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.046+0000 7f7948830740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d36918ad0 0x563d3698bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.046+0000 7f7948830740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f79401b3700 con 0x563d3690d8f0 2024-05-14T17:00:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.046+0000 7f7948830740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f79401b4b70 con 0x563d3690e210 2024-05-14T17:00:16.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.046+0000 7f7948830740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f79401b3dd0 con 0x563d36918ad0 2024-05-14T17:00:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.047+0000 7f7948830740 10 client.0 _ll_register_callbacks cb 0x7f79340c6e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:00:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.047+0000 7f793f7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d36918ad0 0x563d3698bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.047+0000 7f793f7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d36918ad0 0x563d3698bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:60580/0 (socket says 172.21.15.2:60580) 2024-05-14T17:00:16.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.047+0000 7f793f7fe640 1 -- 172.21.15.2:0/4239992510 learned_addr learned my addr 172.21.15.2:0/4239992510 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:00:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.047+0000 7f793e7fc640 1 --2- 172.21.15.2:0/4239992510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d3690e210 0x563d3698ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:16.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.047+0000 7f793f7fe640 1 -- 172.21.15.2:0/4239992510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d3690e210 msgr2=0x563d3698ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.047+0000 7f793f7fe640 1 --2- 172.21.15.2:0/4239992510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d3690e210 0x563d3698ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.047+0000 7f793f7fe640 1 -- 172.21.15.2:0/4239992510 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d3690d8f0 msgr2=0x563d3698b500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T17:00:16.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.048+0000 7f793f7fe640 1 --2- 172.21.15.2:0/4239992510 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d3690d8f0 0x563d3698b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.048+0000 7f793f7fe640 1 -- 172.21.15.2:0/4239992510 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d3691ea50 con 0x563d36918ad0 2024-05-14T17:00:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.048+0000 7f793e7fc640 1 --2- 172.21.15.2:0/4239992510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d3690e210 0x563d3698ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:00:16.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.048+0000 7f793f7fe640 1 --2- 172.21.15.2:0/4239992510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d36918ad0 0x563d3698bf80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f793800afd0 tx=0x7f793801a930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=110d16d6a079340 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.048+0000 7f791effd640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-14T17:00:16.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.048+0000 7f791effd640 1 -- 172.21.15.2:0/4239992510 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7938035060 con 0x563d36918ad0 2024-05-14T17:00:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.048+0000 7f791effd640 1 -- 172.21.15.2:0/4239992510 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f7938023040 con 0x563d36918ad0 2024-05-14T17:00:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.049+0000 7f7948830740 1 -- 172.21.15.2:0/4239992510 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7938016070 con 0x563d36918ad0 2024-05-14T17:00:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.049+0000 7f791effd640 1 -- 172.21.15.2:0/4239992510 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7938030480 con 0x563d36918ad0 2024-05-14T17:00:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.049+0000 7f7948830740 10 client.25603 fetch_fsmap learned FSMap version 37 2024-05-14T17:00:16.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.049+0000 7f7948830740 1 -- 172.21.15.2:0/4239992510 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563d369a3bd0 con 0x563d36918ad0 2024-05-14T17:00:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.049+0000 7f791effd640 1 -- 172.21.15.2:0/4239992510 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7938016070 con 0x563d36918ad0 2024-05-14T17:00:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.050+0000 7f791effd640 1 -- 172.21.15.2:0/4239992510 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 37) ==== 8346+0+0 (secure 0 0 0) 0x7f7938027420 con 0x563d36918ad0 2024-05-14T17:00:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.050+0000 7f7948830740 10 client.25603 fetch_fsmap finished waiting for FSMap version 37 2024-05-14T17:00:17.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.050+0000 7f7948830740 10 client.25603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:00:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.050+0000 7f7948830740 20 client.25603 populate_metadata read hostname 'smithi002' 2024-05-14T17:00:17.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.050+0000 7f7948830740 1 --2- 172.21.15.2:0/4239992510 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x563d369a4250 0x563d369a6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:17.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.050+0000 7f7948830740 4 client.25603 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:00:17.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.050+0000 7f7948830740 1 -- 172.21.15.2:0/4239992510 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563d367f65f0 con 0x563d369a4250 2024-05-14T17:00:17.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.051+0000 7f793effd640 1 --2- 172.21.15.2:0/4239992510 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x563d369a4250 0x563d369a6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:17.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.051+0000 7f793effd640 1 --2- 172.21.15.2:0/4239992510 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x563d369a4250 0x563d369a6670 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41aa248e0be48500 server_cookie=93ec6b2ebd326bdd in_seq=0 out_seq=0 2024-05-14T17:00:17.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.051+0000 7f791effd640 10 client.25603 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:00:17.072 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:00:14 smithi123 ceph-mon[35971]: pgmap v169: 97 pgs: 97 active+clean; 3.3 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 46 MiB/s wr, 2.47k op/s 2024-05-14T17:00:17.081 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:00:16 smithi123 ceph-mon[35971]: pgmap v170: 97 pgs: 97 active+clean; 3.3 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 36 MiB/s wr, 2.01k op/s 2024-05-14T17:00:17.090 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:00:14 smithi151 ceph-mon[36066]: pgmap v169: 97 pgs: 97 active+clean; 3.3 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 46 MiB/s wr, 2.47k op/s 2024-05-14T17:00:17.099 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:00:16 smithi151 ceph-mon[36066]: pgmap v170: 97 pgs: 97 active+clean; 3.3 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 36 MiB/s wr, 2.01k op/s 2024-05-14T17:00:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.112+0000 7f791effd640 1 -- 172.21.15.2:0/4239992510 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563d367f65f0 con 0x563d369a4250 2024-05-14T17:00:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.112+0000 7f791effd640 10 client.25603 handle_command_reply: tid=0 2024-05-14T17:00:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.112+0000 7f7948830740 1 client.25603 shutdown 2024-05-14T17:00:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.113+0000 7f7948830740 1 -- 172.21.15.2:0/4239992510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d36918ad0 msgr2=0x563d3698bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.113+0000 7f7948830740 1 --2- 172.21.15.2:0/4239992510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d36918ad0 0x563d3698bf80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f793800afd0 tx=0x7f793801a930 comp rx=0 tx=0).stop 2024-05-14T17:00:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.113+0000 7f7948830740 1 -- 172.21.15.2:0/4239992510 shutdown_connections 2024-05-14T17:00:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.113+0000 7f7948830740 1 --2- 172.21.15.2:0/4239992510 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x563d369a4250 0x563d369a6670 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.113+0000 7f7948830740 1 --2- 172.21.15.2:0/4239992510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563d36918ad0 0x563d3698bf80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:17.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.113+0000 7f7948830740 1 --2- 172.21.15.2:0/4239992510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d3690e210 0x563d3698ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:17.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.113+0000 7f791effd640 0 client.25603 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:00:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.113+0000 7f7948830740 1 --2- 172.21.15.2:0/4239992510 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563d3690d8f0 0x563d3698b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.113+0000 7f7948830740 1 -- 172.21.15.2:0/4239992510 >> 172.21.15.2:0/4239992510 conn(0x563d36909e00 msgr2=0x563d36908870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.113+0000 7f7948830740 1 -- 172.21.15.2:0/4239992510 shutdown_connections 2024-05-14T17:00:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.113+0000 7f7948830740 1 -- 172.21.15.2:0/4239992510 wait complete. 2024-05-14T17:00:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.114+0000 7f7948830740 20 client.25603 trim_cache size 0 max 16384 2024-05-14T17:00:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:00:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:00:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:00:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.115+0000 7f7945a3f640 1 -- 172.21.15.2:0/3969109541 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f79280b2ae0 msgr2=0x7f79280b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:17.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.115+0000 7f7945a3f640 1 --2- 172.21.15.2:0/3969109541 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f79280b2ae0 0x7f79280b4fa0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f79380115e0 tx=0x7f7938004a40 comp rx=0 tx=0).stop 2024-05-14T17:00:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.115+0000 7f7945a3f640 1 -- 172.21.15.2:0/3969109541 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f794010e3a0 msgr2=0x7f79401a5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:17.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.115+0000 7f7945a3f640 1 --2- 172.21.15.2:0/3969109541 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f794010e3a0 0x7f79401a5c10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f794010a1d0 tx=0x7f7934003c00 comp rx=0 tx=0).stop 2024-05-14T17:00:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.116+0000 7f7945a3f640 1 -- 172.21.15.2:0/3969109541 shutdown_connections 2024-05-14T17:00:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.116+0000 7f7945a3f640 1 --2- 172.21.15.2:0/3969109541 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f79280b2ae0 0x7f79280b4fa0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:17.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.116+0000 7f7945a3f640 1 --2- 172.21.15.2:0/3969109541 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f794019f840 0x7f794019fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.116+0000 7f7945a3f640 1 --2- 172.21.15.2:0/3969109541 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f794010e3a0 0x7f79401a5c10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:17.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.116+0000 7f7945a3f640 1 --2- 172.21.15.2:0/3969109541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79401092d0 0x7f79401a56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.116+0000 7f7945a3f640 1 -- 172.21.15.2:0/3969109541 >> 172.21.15.2:0/3969109541 conn(0x7f7940076fd0 msgr2=0x7f79400779c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.117+0000 7f7945a3f640 1 -- 172.21.15.2:0/3969109541 shutdown_connections 2024-05-14T17:00:17.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:14.117+0000 7f7945a3f640 1 -- 172.21.15.2:0/3969109541 wait complete. 2024-05-14T17:00:17.611 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:00:14 smithi002 ceph-mon[31538]: pgmap v169: 97 pgs: 97 active+clean; 3.3 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 46 MiB/s wr, 2.47k op/s 2024-05-14T17:00:17.620 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 300.0 s, 3920.0 tps, lat 25.487 ms stddev 28.605 2024-05-14T17:00:17.629 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:00:16 smithi002 ceph-mon[31538]: pgmap v170: 97 pgs: 97 active+clean; 3.3 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 36 MiB/s wr, 2.01k op/s 2024-05-14T17:00:18.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:00:18 smithi151 ceph-mon[36066]: pgmap v171: 97 pgs: 97 active+clean; 3.3 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 39 MiB/s wr, 2.17k op/s 2024-05-14T17:00:19.354 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:00:18 smithi002 ceph-mon[31538]: pgmap v171: 97 pgs: 97 active+clean; 3.3 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 39 MiB/s wr, 2.17k op/s 2024-05-14T17:00:19.417 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:00:18 smithi123 ceph-mon[35971]: pgmap v171: 97 pgs: 97 active+clean; 3.3 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 39 MiB/s wr, 2.17k op/s 2024-05-14T17:00:20.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:00:20 smithi151 ceph-mon[36066]: pgmap v172: 97 pgs: 97 active+clean; 3.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 40 MiB/s wr, 2.23k op/s 2024-05-14T17:00:20.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:00:20 smithi123 ceph-mon[35971]: pgmap v172: 97 pgs: 97 active+clean; 3.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 40 MiB/s wr, 2.23k op/s 2024-05-14T17:00:20.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:00:20 smithi002 ceph-mon[31538]: pgmap v172: 97 pgs: 97 active+clean; 3.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 40 MiB/s wr, 2.23k op/s 2024-05-14T17:00:21.747 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 305.0 s, 3574.8 tps, lat 28.060 ms stddev 30.637 2024-05-14T17:00:22.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:00:22 smithi123 ceph-mon[35971]: pgmap v173: 97 pgs: 97 active+clean; 3.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 37 MiB/s wr, 2.03k op/s 2024-05-14T17:00:22.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:00:22 smithi002 ceph-mon[31538]: pgmap v173: 97 pgs: 97 active+clean; 3.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 37 MiB/s wr, 2.03k op/s 2024-05-14T17:00:23.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:00:22 smithi151 ceph-mon[36066]: pgmap v173: 97 pgs: 97 active+clean; 3.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 37 MiB/s wr, 2.03k op/s 2024-05-14T17:00:24.906 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:00:24 smithi151 ceph-mon[36066]: pgmap v174: 97 pgs: 97 active+clean; 3.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 43 MiB/s wr, 2.36k op/s 2024-05-14T17:00:24.919 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:00:24 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:00:24.928 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:00:24 smithi123 ceph-mon[35971]: pgmap v174: 97 pgs: 97 active+clean; 3.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 43 MiB/s wr, 2.36k op/s 2024-05-14T17:00:24.936 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:00:24 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:00:24.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:00:24 smithi002 ceph-mon[31538]: pgmap v174: 97 pgs: 97 active+clean; 3.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 43 MiB/s wr, 2.36k op/s 2024-05-14T17:00:25.000 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:00:24 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:00:26.748 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 310.0 s, 2769.5 tps, lat 26.827 ms stddev 28.001 2024-05-14T17:00:26.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:00:26 smithi123 ceph-mon[35971]: pgmap v175: 97 pgs: 97 active+clean; 3.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 11 MiB/s wr, 675 op/s 2024-05-14T17:00:26.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:00:26 smithi002 ceph-mon[31538]: pgmap v175: 97 pgs: 97 active+clean; 3.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 11 MiB/s wr, 675 op/s 2024-05-14T17:00:27.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:00:26 smithi151 ceph-mon[36066]: pgmap v175: 97 pgs: 97 active+clean; 3.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 11 MiB/s wr, 675 op/s 2024-05-14T17:00:27.219 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.217+0000 7fc6f690a640 1 -- 172.21.15.2:0/2522686712 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6f0101bc0 msgr2=0x7fc6f0102020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.217+0000 7fc6f690a640 1 --2- 172.21.15.2:0/2522686712 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6f0101bc0 0x7fc6f0102020 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4009900 tx=0x7fc6e402a410 comp rx=0 tx=0).stop 2024-05-14T17:00:27.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.217+0000 7fc6f690a640 1 -- 172.21.15.2:0/2522686712 shutdown_connections 2024-05-14T17:00:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.217+0000 7fc6f690a640 1 --2- 172.21.15.2:0/2522686712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6f0102560 0x7fc6f010e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.217+0000 7fc6f690a640 1 --2- 172.21.15.2:0/2522686712 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6f0101bc0 0x7fc6f0102020 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:27.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.217+0000 7fc6f690a640 1 --2- 172.21.15.2:0/2522686712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6f0107a10 0x7fc6f0107df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.217+0000 7fc6f690a640 1 -- 172.21.15.2:0/2522686712 >> 172.21.15.2:0/2522686712 conn(0x7fc6f00788c0 msgr2=0x7fc6f0078cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.217+0000 7fc6f690a640 1 -- 172.21.15.2:0/2522686712 shutdown_connections 2024-05-14T17:00:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.217+0000 7fc6f690a640 1 -- 172.21.15.2:0/2522686712 wait complete. 2024-05-14T17:00:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.218+0000 7fc6f690a640 1 Processor -- start 2024-05-14T17:00:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.218+0000 7fc6f690a640 1 -- start start 2024-05-14T17:00:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.218+0000 7fc6f690a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6f0101bc0 0x7fc6f0199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.218+0000 7fc6f690a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6f0102560 0x7fc6f01996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:27.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.218+0000 7fc6f690a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6f0107a10 0x7fc6f019da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.218+0000 7fc6f690a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f0111cc0 con 0x7fc6f0101bc0 2024-05-14T17:00:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.218+0000 7fc6f690a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f0111b40 con 0x7fc6f0107a10 2024-05-14T17:00:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.218+0000 7fc6f690a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f0111e40 con 0x7fc6f0102560 2024-05-14T17:00:27.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.218+0000 7fc6effff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6f0101bc0 0x7fc6f0199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.218+0000 7fc6effff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6f0101bc0 0x7fc6f0199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60876/0 (socket says 172.21.15.2:60876) 2024-05-14T17:00:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.218+0000 7fc6effff640 1 -- 172.21.15.2:0/1611955202 learned_addr learned my addr 172.21.15.2:0/1611955202 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:00:27.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.218+0000 7fc6f4ea7640 1 --2- 172.21.15.2:0/1611955202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6f0107a10 0x7fc6f019da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:27.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.218+0000 7fc6ef7fe640 1 --2- 172.21.15.2:0/1611955202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6f0102560 0x7fc6f01996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:27.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.218+0000 7fc6effff640 1 -- 172.21.15.2:0/1611955202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6f0102560 msgr2=0x7fc6f01996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.218+0000 7fc6effff640 1 --2- 172.21.15.2:0/1611955202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6f0102560 0x7fc6f01996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.218+0000 7fc6effff640 1 -- 172.21.15.2:0/1611955202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6f0107a10 msgr2=0x7fc6f019da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.218+0000 7fc6effff640 1 --2- 172.21.15.2:0/1611955202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6f0107a10 0x7fc6f019da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:27.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.218+0000 7fc6effff640 1 -- 172.21.15.2:0/1611955202 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6f019e160 con 0x7fc6f0101bc0 2024-05-14T17:00:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.219+0000 7fc6effff640 1 --2- 172.21.15.2:0/1611955202 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6f0101bc0 0x7fc6f0199170 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc6e000e9f0 tx=0x7fc6e000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:27.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.219+0000 7fc6ed7fa640 1 -- 172.21.15.2:0/1611955202 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e000cd80 con 0x7fc6f0101bc0 2024-05-14T17:00:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.219+0000 7fc6ed7fa640 1 -- 172.21.15.2:0/1611955202 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6e00108e0 con 0x7fc6f0101bc0 2024-05-14T17:00:27.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.219+0000 7fc6f690a640 1 -- 172.21.15.2:0/1611955202 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6f019e450 con 0x7fc6f0101bc0 2024-05-14T17:00:27.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.219+0000 7fc6ed7fa640 1 -- 172.21.15.2:0/1611955202 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0018980 con 0x7fc6f0101bc0 2024-05-14T17:00:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.219+0000 7fc6f690a640 1 -- 172.21.15.2:0/1611955202 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6f01a5ce0 con 0x7fc6f0101bc0 2024-05-14T17:00:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.222+0000 7fc6ed7fa640 1 -- 172.21.15.2:0/1611955202 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fc6e0010430 con 0x7fc6f0101bc0 2024-05-14T17:00:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.222+0000 7fc6f9628740 1 Processor -- start 2024-05-14T17:00:27.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.222+0000 7fc6f9628740 1 -- start start 2024-05-14T17:00:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.222+0000 7fc6f9628740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630b093d7f0 0x5630b093dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.222+0000 7fc6f9628740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630b093e110 0x5630b094c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:27.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.222+0000 7fc6f9628740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630b094cee0 0x5630b094f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:27.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.222+0000 7fc6f9628740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5630b08b0c30 con 0x5630b093d7f0 2024-05-14T17:00:27.744 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.222+0000 7fc6f9628740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5630b08a07d0 con 0x5630b093e110 2024-05-14T17:00:27.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.222+0000 7fc6f9628740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5630b08a4810 con 0x5630b094cee0 2024-05-14T17:00:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.222+0000 7fc6f4ea7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630b093d7f0 0x5630b093dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:27.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.222+0000 7fc6f4ea7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630b093d7f0 0x5630b093dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60882/0 (socket says 172.21.15.2:60882) 2024-05-14T17:00:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.222+0000 7fc6f4ea7640 1 -- 172.21.15.2:0/450702770 learned_addr learned my addr 172.21.15.2:0/450702770 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:00:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.222+0000 7fc6ef7fe640 1 --2- 172.21.15.2:0/450702770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630b094cee0 0x5630b094f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:27.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.222+0000 7fc6effff640 1 --2- 172.21.15.2:0/450702770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630b093e110 0x5630b094c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:27.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.222+0000 7fc6ef7fe640 1 -- 172.21.15.2:0/450702770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630b093e110 msgr2=0x5630b094c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:27.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.222+0000 7fc6ef7fe640 1 --2- 172.21.15.2:0/450702770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630b093e110 0x5630b094c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.222+0000 7fc6ef7fe640 1 -- 172.21.15.2:0/450702770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630b093d7f0 msgr2=0x5630b093dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.222+0000 7fc6ef7fe640 1 --2- 172.21.15.2:0/450702770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630b093d7f0 0x5630b093dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.222+0000 7fc6ef7fe640 1 -- 172.21.15.2:0/450702770 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630b05b5210 con 0x5630b094cee0 2024-05-14T17:00:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.223+0000 7fc6ed7fa640 1 --2- 172.21.15.2:0/1611955202 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc6d80b2a10 0x7fc6d80b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.223+0000 7fc6ef7fe640 1 --2- 172.21.15.2:0/450702770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630b094cee0 0x5630b094f2d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc6e402aa00 tx=0x7fc6e4004180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a6191fa8de9cbcf server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.223+0000 7fc6f4ea7640 1 --2- 172.21.15.2:0/1611955202 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc6d80b2a10 0x7fc6d80b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.223+0000 7fc6ed7fa640 1 -- 172.21.15.2:0/1611955202 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fc6e00e2880 con 0x7fc6f0101bc0 2024-05-14T17:00:27.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.223+0000 7fc6d67fc640 1 -- 172.21.15.2:0/450702770 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4035040 con 0x5630b094cee0 2024-05-14T17:00:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.223+0000 7fc6d67fc640 1 -- 172.21.15.2:0/450702770 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6e40054f0 con 0x5630b094cee0 2024-05-14T17:00:28.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.223+0000 7fc6d67fc640 1 -- 172.21.15.2:0/450702770 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4003850 con 0x5630b094cee0 2024-05-14T17:00:28.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.224+0000 7fc6f9628740 1 -- 172.21.15.2:0/450702770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630b094cee0 msgr2=0x5630b094f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.224+0000 7fc6f9628740 1 --2- 172.21.15.2:0/450702770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630b094cee0 0x5630b094f2d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc6e402aa00 tx=0x7fc6e4004180 comp rx=0 tx=0).stop 2024-05-14T17:00:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.224+0000 7fc6f4ea7640 1 --2- 172.21.15.2:0/1611955202 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc6d80b2a10 0x7fc6d80b4ed0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8005f40 tx=0x7fc6e80096a0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:28.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.224+0000 7fc6f9628740 1 -- 172.21.15.2:0/450702770 shutdown_connections 2024-05-14T17:00:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.224+0000 7fc6f9628740 1 --2- 172.21.15.2:0/450702770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630b094cee0 0x5630b094f2d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.224+0000 7fc6f9628740 1 --2- 172.21.15.2:0/450702770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630b093e110 0x5630b094c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.224+0000 7fc6f9628740 1 --2- 172.21.15.2:0/450702770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630b093d7f0 0x5630b093dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.224+0000 7fc6f9628740 1 -- 172.21.15.2:0/450702770 >> 172.21.15.2:0/450702770 conn(0x5630b0939e00 msgr2=0x5630b093a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:28.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.224+0000 7fc6f9628740 1 -- 172.21.15.2:0/450702770 shutdown_connections 2024-05-14T17:00:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.224+0000 7fc6f9628740 1 -- 172.21.15.2:0/450702770 wait complete. 2024-05-14T17:00:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.224+0000 7fc6f9628740 1 -- start start 2024-05-14T17:00:28.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.224+0000 7fc6f9628740 1 Processor -- start 2024-05-14T17:00:28.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.225+0000 7fc6f9628740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630b093d7f0 0x5630b094c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.225+0000 7fc6f9628740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630b093e110 0x5630b0949b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:28.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.225+0000 7fc6f9628740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630b094cee0 0x5630b094a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.225+0000 7fc6f9628740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5630b0586790 con 0x5630b093e110 2024-05-14T17:00:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.225+0000 7fc6f9628740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5630b085c890 con 0x5630b093d7f0 2024-05-14T17:00:28.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.225+0000 7fc6f9628740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5630b094fb50 con 0x5630b094cee0 2024-05-14T17:00:28.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.225+0000 7fc6f9628740 10 client.0 _ll_register_callbacks cb 0x5630b085fd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:00:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.225+0000 7fc6ef7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630b093e110 0x5630b0949b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:28.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.225+0000 7fc6ef7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630b093e110 0x5630b0949b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60896/0 (socket says 172.21.15.2:60896) 2024-05-14T17:00:28.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.225+0000 7fc6ef7fe640 1 -- 172.21.15.2:0/2017767168 learned_addr learned my addr 172.21.15.2:0/2017767168 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:00:28.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.225+0000 7fc6f4ea7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630b094cee0 0x5630b094a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.225+0000 7fc6effff640 1 --2- 172.21.15.2:0/2017767168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630b093d7f0 0x5630b094c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:28.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.225+0000 7fc6ef7fe640 1 -- 172.21.15.2:0/2017767168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630b094cee0 msgr2=0x5630b094a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:28.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.225+0000 7fc6ef7fe640 1 --2- 172.21.15.2:0/2017767168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630b094cee0 0x5630b094a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:28.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.225+0000 7fc6ef7fe640 1 -- 172.21.15.2:0/2017767168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630b093d7f0 msgr2=0x5630b094c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.225+0000 7fc6ef7fe640 1 --2- 172.21.15.2:0/2017767168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630b093d7f0 0x5630b094c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.225+0000 7fc6ef7fe640 1 -- 172.21.15.2:0/2017767168 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630b0681620 con 0x5630b093e110 2024-05-14T17:00:28.418 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.225+0000 7fc6f4ea7640 1 --2- 172.21.15.2:0/2017767168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630b094cee0 0x5630b094a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:00:28.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.225+0000 7fc6effff640 1 --2- 172.21.15.2:0/2017767168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630b093d7f0 0x5630b094c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:00:28.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.225+0000 7fc6ef7fe640 1 --2- 172.21.15.2:0/2017767168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630b093e110 0x5630b0949b40 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc6e402a930 tx=0x7fc6e40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=884df6cdcbd6d742 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:28.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.225+0000 7fc6cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:00:28.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.225+0000 7fc6cf7fe640 1 -- 172.21.15.2:0/2017767168 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4052060 con 0x5630b093e110 2024-05-14T17:00:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.225+0000 7fc6cf7fe640 1 -- 172.21.15.2:0/2017767168 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fc6e40048d0 con 0x5630b093e110 2024-05-14T17:00:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.225+0000 7fc6f9628740 1 -- 172.21.15.2:0/2017767168 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6e40054f0 con 0x5630b093e110 2024-05-14T17:00:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.226+0000 7fc6cf7fe640 1 -- 172.21.15.2:0/2017767168 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fc6e4035050 con 0x5630b093e110 2024-05-14T17:00:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.226+0000 7fc6f9628740 10 client.15921 fetch_fsmap learned FSMap version 38 2024-05-14T17:00:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.226+0000 7fc6f9628740 1 -- 172.21.15.2:0/2017767168 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5630b095b5f0 con 0x5630b093e110 2024-05-14T17:00:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.226+0000 7fc6cf7fe640 1 -- 172.21.15.2:0/2017767168 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6e40054f0 con 0x5630b093e110 2024-05-14T17:00:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.226+0000 7fc6cf7fe640 1 -- 172.21.15.2:0/2017767168 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) ==== 8342+0+0 (secure 0 0 0) 0x7fc6e4003a10 con 0x5630b093e110 2024-05-14T17:00:28.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.226+0000 7fc6f9628740 10 client.15921 fetch_fsmap finished waiting for FSMap version 38 2024-05-14T17:00:28.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.226+0000 7fc6f9628740 10 client.15921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:00:28.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.226+0000 7fc6f9628740 20 client.15921 populate_metadata read hostname 'smithi002' 2024-05-14T17:00:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.226+0000 7fc6f9628740 1 --2- 172.21.15.2:0/2017767168 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5630b095c940 0x5630b095ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:28.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.226+0000 7fc6f9628740 4 client.15921 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:00:28.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.226+0000 7fc6f9628740 1 -- 172.21.15.2:0/2017767168 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630b08b2b80 con 0x5630b095c940 2024-05-14T17:00:28.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.226+0000 7fc6effff640 1 --2- 172.21.15.2:0/2017767168 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5630b095c940 0x5630b095ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.227+0000 7fc6effff640 1 --2- 172.21.15.2:0/2017767168 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5630b095c940 0x5630b095ed30 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cec210b592a91989 server_cookie=fc4bdfef418a1ea9 in_seq=0 out_seq=0 2024-05-14T17:00:28.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.227+0000 7fc6cf7fe640 10 client.15921 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:00:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.228+0000 7fc6cf7fe640 1 -- 172.21.15.2:0/2017767168 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5630b08b2b80 con 0x5630b095c940 2024-05-14T17:00:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.228+0000 7fc6cf7fe640 10 client.15921 handle_command_reply: tid=0 2024-05-14T17:00:28.734 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.228+0000 7fc6f9628740 1 client.15921 shutdown 2024-05-14T17:00:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.228+0000 7fc6f9628740 1 -- 172.21.15.2:0/2017767168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630b093e110 msgr2=0x5630b0949b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.228+0000 7fc6f9628740 1 --2- 172.21.15.2:0/2017767168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630b093e110 0x5630b0949b40 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc6e402a930 tx=0x7fc6e40045e0 comp rx=0 tx=0).stop 2024-05-14T17:00:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.228+0000 7fc6f9628740 1 -- 172.21.15.2:0/2017767168 shutdown_connections 2024-05-14T17:00:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.228+0000 7fc6f9628740 1 --2- 172.21.15.2:0/2017767168 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5630b095c940 0x5630b095ed30 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.228+0000 7fc6f9628740 1 --2- 172.21.15.2:0/2017767168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630b094cee0 0x5630b094a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.228+0000 7fc6f9628740 1 --2- 172.21.15.2:0/2017767168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630b093e110 0x5630b0949b40 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.228+0000 7fc6cf7fe640 0 client.15921 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:00:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.228+0000 7fc6f9628740 1 --2- 172.21.15.2:0/2017767168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630b093d7f0 0x5630b094c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.228+0000 7fc6f9628740 1 -- 172.21.15.2:0/2017767168 >> 172.21.15.2:0/2017767168 conn(0x5630b0939e00 msgr2=0x5630b093fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:28.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.228+0000 7fc6f9628740 1 -- 172.21.15.2:0/2017767168 shutdown_connections 2024-05-14T17:00:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.228+0000 7fc6f9628740 1 -- 172.21.15.2:0/2017767168 wait complete. 2024-05-14T17:00:28.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.228+0000 7fc6f9628740 20 client.15921 trim_cache size 0 max 16384 2024-05-14T17:00:28.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.244+0000 7fc6f9628740 1 Processor -- start 2024-05-14T17:00:28.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.244+0000 7fc6f9628740 1 -- start start 2024-05-14T17:00:28.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.244+0000 7fc6f9628740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630b093d7f0 0x5630b094c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.244+0000 7fc6f9628740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630b093e110 0x5630b0961d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.244+0000 7fc6f9628740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630b094cee0 0x5630b0962290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:28.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.244+0000 7fc6f9628740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f01aeb40 con 0x5630b093e110 2024-05-14T17:00:29.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.244+0000 7fc6f9628740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f01ac3e0 con 0x5630b094cee0 2024-05-14T17:00:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.244+0000 7fc6f9628740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f01aa250 con 0x5630b093d7f0 2024-05-14T17:00:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.244+0000 7fc6ef7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630b093e110 0x5630b0961d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.244+0000 7fc6ef7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630b093e110 0x5630b0961d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60902/0 (socket says 172.21.15.2:60902) 2024-05-14T17:00:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.244+0000 7fc6ef7fe640 1 -- 172.21.15.2:0/2146689944 learned_addr learned my addr 172.21.15.2:0/2146689944 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:00:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.245+0000 7fc6effff640 1 --2- 172.21.15.2:0/2146689944 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630b093d7f0 0x5630b094c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.245+0000 7fc6f4ea7640 1 --2- 172.21.15.2:0/2146689944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630b094cee0 0x5630b0962290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.245+0000 7fc6ef7fe640 1 -- 172.21.15.2:0/2146689944 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630b093d7f0 msgr2=0x5630b094c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.245+0000 7fc6ef7fe640 1 --2- 172.21.15.2:0/2146689944 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630b093d7f0 0x5630b094c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.245+0000 7fc6ef7fe640 1 -- 172.21.15.2:0/2146689944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630b094cee0 msgr2=0x5630b0962290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:29.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.245+0000 7fc6ef7fe640 1 --2- 172.21.15.2:0/2146689944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630b094cee0 0x5630b0962290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:29.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.245+0000 7fc6ef7fe640 1 -- 172.21.15.2:0/2146689944 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630b094a410 con 0x5630b093e110 2024-05-14T17:00:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.245+0000 7fc6ef7fe640 1 --2- 172.21.15.2:0/2146689944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630b093e110 0x5630b0961d50 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4000c00 tx=0x7fc6e4001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b99a4c959be85c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.245+0000 7fc6d5ffb640 1 -- 172.21.15.2:0/2146689944 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4052070 con 0x5630b093e110 2024-05-14T17:00:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.245+0000 7fc6d5ffb640 1 -- 172.21.15.2:0/2146689944 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6e4003aa0 con 0x5630b093e110 2024-05-14T17:00:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.245+0000 7fc6d5ffb640 1 -- 172.21.15.2:0/2146689944 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4057a80 con 0x5630b093e110 2024-05-14T17:00:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.246+0000 7fc6f9628740 1 -- 172.21.15.2:0/2146689944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630b093e110 msgr2=0x5630b0961d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.246+0000 7fc6f9628740 1 --2- 172.21.15.2:0/2146689944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630b093e110 0x5630b0961d50 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4000c00 tx=0x7fc6e4001e70 comp rx=0 tx=0).stop 2024-05-14T17:00:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.246+0000 7fc6f9628740 1 -- 172.21.15.2:0/2146689944 shutdown_connections 2024-05-14T17:00:29.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.246+0000 7fc6f9628740 1 --2- 172.21.15.2:0/2146689944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630b094cee0 0x5630b0962290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:29.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.246+0000 7fc6f9628740 1 --2- 172.21.15.2:0/2146689944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630b093e110 0x5630b0961d50 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.246+0000 7fc6f9628740 1 --2- 172.21.15.2:0/2146689944 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630b093d7f0 0x5630b094c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.246+0000 7fc6f9628740 1 -- 172.21.15.2:0/2146689944 >> 172.21.15.2:0/2146689944 conn(0x5630b0939e00 msgr2=0x5630b09451d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.246+0000 7fc6f9628740 1 -- 172.21.15.2:0/2146689944 shutdown_connections 2024-05-14T17:00:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.246+0000 7fc6f9628740 1 -- 172.21.15.2:0/2146689944 wait complete. 2024-05-14T17:00:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.246+0000 7fc6f9628740 1 -- start start 2024-05-14T17:00:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.246+0000 7fc6f9628740 1 Processor -- start 2024-05-14T17:00:29.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.246+0000 7fc6f9628740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630b093d7f0 0x5630b0962450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.246+0000 7fc6f9628740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630b093e110 0x5630b095f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.246+0000 7fc6f9628740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630b094cee0 0x5630b095fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.246+0000 7fc6f9628740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f01aa8a0 con 0x5630b093d7f0 2024-05-14T17:00:29.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.246+0000 7fc6f9628740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f01abd10 con 0x5630b093e110 2024-05-14T17:00:29.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.246+0000 7fc6f9628740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f01aaf70 con 0x5630b094cee0 2024-05-14T17:00:29.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.247+0000 7fc6f9628740 10 client.0 _ll_register_callbacks cb 0x7fc6e40040d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:00:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.247+0000 7fc6effff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630b093d7f0 0x5630b0962450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:29.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.247+0000 7fc6effff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630b093d7f0 0x5630b0962450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60916/0 (socket says 172.21.15.2:60916) 2024-05-14T17:00:29.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.247+0000 7fc6effff640 1 -- 172.21.15.2:0/3107543138 learned_addr learned my addr 172.21.15.2:0/3107543138 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:00:29.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.247+0000 7fc6f4ea7640 1 --2- 172.21.15.2:0/3107543138 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630b094cee0 0x5630b095fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:29.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.247+0000 7fc6ef7fe640 1 --2- 172.21.15.2:0/3107543138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630b093e110 0x5630b095f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:29.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.247+0000 7fc6effff640 1 -- 172.21.15.2:0/3107543138 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630b094cee0 msgr2=0x5630b095fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.247+0000 7fc6effff640 1 --2- 172.21.15.2:0/3107543138 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630b094cee0 0x5630b095fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.247+0000 7fc6effff640 1 -- 172.21.15.2:0/3107543138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630b093e110 msgr2=0x5630b095f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:29.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.247+0000 7fc6effff640 1 --2- 172.21.15.2:0/3107543138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630b093e110 0x5630b095f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:29.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.247+0000 7fc6effff640 1 -- 172.21.15.2:0/3107543138 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630b09c5e60 con 0x5630b093d7f0 2024-05-14T17:00:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.247+0000 7fc6effff640 1 --2- 172.21.15.2:0/3107543138 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630b093d7f0 0x5630b0962450 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc6e00a5cc0 tx=0x7fc6e00e8550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9257524c7c5d5144 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.247+0000 7fc6cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:00:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.247+0000 7fc6cf7fe640 1 -- 172.21.15.2:0/3107543138 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e00a53d0 con 0x5630b093d7f0 2024-05-14T17:00:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.247+0000 7fc6cf7fe640 1 -- 172.21.15.2:0/3107543138 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fc6e00a5ed0 con 0x5630b093d7f0 2024-05-14T17:00:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.247+0000 7fc6f9628740 1 -- 172.21.15.2:0/3107543138 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6e4003aa0 con 0x5630b093d7f0 2024-05-14T17:00:29.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.247+0000 7fc6cf7fe640 1 -- 172.21.15.2:0/3107543138 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fc6e00e8320 con 0x5630b093d7f0 2024-05-14T17:00:29.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.247+0000 7fc6f9628740 10 client.15927 fetch_fsmap learned FSMap version 38 2024-05-14T17:00:29.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.247+0000 7fc6f9628740 1 -- 172.21.15.2:0/3107543138 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5630b09c7070 con 0x5630b093d7f0 2024-05-14T17:00:29.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.247+0000 7fc6cf7fe640 1 -- 172.21.15.2:0/3107543138 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6e4003aa0 con 0x5630b093d7f0 2024-05-14T17:00:29.734 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.248+0000 7fc6cf7fe640 1 -- 172.21.15.2:0/3107543138 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) ==== 8342+0+0 (secure 0 0 0) 0x7fc6e0018b20 con 0x5630b093d7f0 2024-05-14T17:00:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.248+0000 7fc6f9628740 10 client.15927 fetch_fsmap finished waiting for FSMap version 38 2024-05-14T17:00:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.248+0000 7fc6f9628740 10 client.15927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:00:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.248+0000 7fc6f9628740 20 client.15927 populate_metadata read hostname 'smithi002' 2024-05-14T17:00:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.248+0000 7fc6f9628740 1 --2- 172.21.15.2:0/3107543138 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5630b09c7740 0x5630b09c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.248+0000 7fc6f9628740 4 client.15927 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:00:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.248+0000 7fc6f9628740 1 -- 172.21.15.2:0/3107543138 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5630b08ac360 con 0x5630b09c7740 2024-05-14T17:00:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.248+0000 7fc6ef7fe640 1 --2- 172.21.15.2:0/3107543138 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5630b09c7740 0x5630b09c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.248+0000 7fc6cf7fe640 10 client.15927 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:00:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.248+0000 7fc6ef7fe640 1 --2- 172.21.15.2:0/3107543138 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5630b09c7740 0x5630b09c9b30 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5b925228588cf30 server_cookie=8d8f63cd4b875ea7 in_seq=0 out_seq=0 2024-05-14T17:00:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.263+0000 7fc6cf7fe640 1 -- 172.21.15.2:0/3107543138 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5630b08ac360 con 0x5630b09c7740 2024-05-14T17:00:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.263+0000 7fc6cf7fe640 10 client.15927 handle_command_reply: tid=0 2024-05-14T17:00:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.263+0000 7fc6f9628740 1 client.15927 shutdown 2024-05-14T17:00:29.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.263+0000 7fc6f9628740 1 -- 172.21.15.2:0/3107543138 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630b093d7f0 msgr2=0x5630b0962450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.263+0000 7fc6f9628740 1 --2- 172.21.15.2:0/3107543138 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630b093d7f0 0x5630b0962450 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc6e00a5cc0 tx=0x7fc6e00e8550 comp rx=0 tx=0).stop 2024-05-14T17:00:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.263+0000 7fc6f9628740 1 -- 172.21.15.2:0/3107543138 shutdown_connections 2024-05-14T17:00:29.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.263+0000 7fc6f9628740 1 --2- 172.21.15.2:0/3107543138 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5630b09c7740 0x5630b09c9b30 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:29.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.263+0000 7fc6f9628740 1 --2- 172.21.15.2:0/3107543138 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5630b094cee0 0x5630b095fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.263+0000 7fc6cf7fe640 0 client.15927 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:00:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.263+0000 7fc6f9628740 1 --2- 172.21.15.2:0/3107543138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630b093e110 0x5630b095f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.263+0000 7fc6f9628740 1 --2- 172.21.15.2:0/3107543138 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630b093d7f0 0x5630b0962450 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.263+0000 7fc6f9628740 1 -- 172.21.15.2:0/3107543138 >> 172.21.15.2:0/3107543138 conn(0x5630b0939e00 msgr2=0x5630b093f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:30.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.263+0000 7fc6f9628740 1 -- 172.21.15.2:0/3107543138 shutdown_connections 2024-05-14T17:00:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.263+0000 7fc6f9628740 1 -- 172.21.15.2:0/3107543138 wait complete. 2024-05-14T17:00:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.263+0000 7fc6f9628740 20 client.15927 trim_cache size 0 max 16384 2024-05-14T17:00:30.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:00:30.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:00:30.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:00:30.108 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:00:28 smithi151 ceph-mon[36066]: pgmap v176: 97 pgs: 97 active+clean; 3.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 12 MiB/s wr, 772 op/s 2024-05-14T17:00:30.118 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:00:28 smithi123 ceph-mon[35971]: pgmap v176: 97 pgs: 97 active+clean; 3.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 12 MiB/s wr, 772 op/s 2024-05-14T17:00:30.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.264+0000 7fc6f690a640 1 -- 172.21.15.2:0/1611955202 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc6d80b2a10 msgr2=0x7fc6d80b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:30.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.264+0000 7fc6f690a640 1 --2- 172.21.15.2:0/1611955202 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc6d80b2a10 0x7fc6d80b4ed0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8005f40 tx=0x7fc6e80096a0 comp rx=0 tx=0).stop 2024-05-14T17:00:30.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.264+0000 7fc6f690a640 1 -- 172.21.15.2:0/1611955202 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6f0101bc0 msgr2=0x7fc6f0199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:30.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.264+0000 7fc6f690a640 1 --2- 172.21.15.2:0/1611955202 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6f0101bc0 0x7fc6f0199170 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc6e000e9f0 tx=0x7fc6e000eeb0 comp rx=0 tx=0).stop 2024-05-14T17:00:30.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.264+0000 7fc6f690a640 1 -- 172.21.15.2:0/1611955202 shutdown_connections 2024-05-14T17:00:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.264+0000 7fc6f690a640 1 --2- 172.21.15.2:0/1611955202 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc6d80b2a10 0x7fc6d80b4ed0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:30.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.264+0000 7fc6f690a640 1 --2- 172.21.15.2:0/1611955202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6f0107a10 0x7fc6f019da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:30.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.264+0000 7fc6f690a640 1 --2- 172.21.15.2:0/1611955202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6f0102560 0x7fc6f01996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:30.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.264+0000 7fc6f690a640 1 --2- 172.21.15.2:0/1611955202 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6f0101bc0 0x7fc6f0199170 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:30.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.264+0000 7fc6f690a640 1 -- 172.21.15.2:0/1611955202 >> 172.21.15.2:0/1611955202 conn(0x7fc6f00788c0 msgr2=0x7fc6f006b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:30.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.264+0000 7fc6f690a640 1 -- 172.21.15.2:0/1611955202 shutdown_connections 2024-05-14T17:00:30.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:27.264+0000 7fc6f690a640 1 -- 172.21.15.2:0/1611955202 wait complete. 2024-05-14T17:00:30.249 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:00:28 smithi002 ceph-mon[31538]: pgmap v176: 97 pgs: 97 active+clean; 3.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 12 MiB/s wr, 772 op/s 2024-05-14T17:00:30.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:00:30 smithi123 ceph-mon[35971]: pgmap v177: 97 pgs: 97 active+clean; 3.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 11 MiB/s wr, 781 op/s 2024-05-14T17:00:30.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:00:30 smithi002 ceph-mon[31538]: pgmap v177: 97 pgs: 97 active+clean; 3.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 11 MiB/s wr, 781 op/s 2024-05-14T17:00:31.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:00:30 smithi151 ceph-mon[36066]: pgmap v177: 97 pgs: 97 active+clean; 3.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 11 MiB/s wr, 781 op/s 2024-05-14T17:00:31.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 315.0 s, 868.2 tps, lat 144.576 ms stddev 765.672 2024-05-14T17:00:32.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:00:32 smithi123 ceph-mon[35971]: pgmap v178: 97 pgs: 97 active+clean; 3.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.7 MiB/s wr, 599 op/s 2024-05-14T17:00:32.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:00:32 smithi002 ceph-mon[31538]: pgmap v178: 97 pgs: 97 active+clean; 3.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.7 MiB/s wr, 599 op/s 2024-05-14T17:00:33.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:00:32 smithi151 ceph-mon[36066]: pgmap v178: 97 pgs: 97 active+clean; 3.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.7 MiB/s wr, 599 op/s 2024-05-14T17:00:33.937 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:00:33 smithi151 ceph-mon[36066]: pgmap v179: 97 pgs: 97 active+clean; 3.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 11 MiB/s wr, 787 op/s 2024-05-14T17:00:33.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:00:33 smithi002 ceph-mon[31538]: pgmap v179: 97 pgs: 97 active+clean; 3.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 11 MiB/s wr, 787 op/s 2024-05-14T17:00:34.159 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:00:33 smithi123 ceph-mon[35971]: pgmap v179: 97 pgs: 97 active+clean; 3.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 11 MiB/s wr, 787 op/s 2024-05-14T17:00:34.961 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 13.7 2024-05-14T17:00:34.971 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:00:35.371 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:00:36.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:00:36 smithi123 ceph-mon[35971]: pgmap v180: 97 pgs: 97 active+clean; 3.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.7 MiB/s wr, 457 op/s 2024-05-14T17:00:36.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:00:36 smithi002 ceph-mon[31538]: pgmap v180: 97 pgs: 97 active+clean; 3.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.7 MiB/s wr, 457 op/s 2024-05-14T17:00:36.751 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 320.0 s, 3284.0 tps, lat 30.215 ms stddev 33.519 2024-05-14T17:00:36.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:00:36 smithi151 ceph-mon[36066]: pgmap v180: 97 pgs: 97 active+clean; 3.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.7 MiB/s wr, 457 op/s 2024-05-14T17:00:37.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.424+0000 7f7ec759e700 1 -- 172.21.15.2:0/4186116579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ec81025d0 msgr2=0x7f7ec81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:37.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.424+0000 7f7ec759e700 1 --2- 172.21.15.2:0/4186116579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ec81025d0 0x7f7ec81029b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7eb80099c0 tx=0x7f7eb8009cd0 comp rx=0 tx=0).stop 2024-05-14T17:00:37.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.425+0000 7f7ec759e700 1 -- 172.21.15.2:0/4186116579 shutdown_connections 2024-05-14T17:00:37.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.425+0000 7f7ec759e700 1 --2- 172.21.15.2:0/4186116579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ec81076b0 0x7f7ec810f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:37.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.425+0000 7f7ec759e700 1 --2- 172.21.15.2:0/4186116579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ec81025d0 0x7f7ec81029b0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:37.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.425+0000 7f7ec759e700 1 --2- 172.21.15.2:0/4186116579 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ec8102f80 0x7f7ec8106fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:37.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.425+0000 7f7ec759e700 1 -- 172.21.15.2:0/4186116579 >> 172.21.15.2:0/4186116579 conn(0x7f7ec80fde60 msgr2=0x7f7ec8100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:37.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.425+0000 7f7ec759e700 1 -- 172.21.15.2:0/4186116579 shutdown_connections 2024-05-14T17:00:37.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.425+0000 7f7ec759e700 1 -- 172.21.15.2:0/4186116579 wait complete. 2024-05-14T17:00:37.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.426+0000 7f7ec759e700 1 Processor -- start 2024-05-14T17:00:37.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.426+0000 7f7ec759e700 1 -- start start 2024-05-14T17:00:37.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.426+0000 7f7ec759e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ec81025d0 0x7f7ec8198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:37.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.426+0000 7f7ec759e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ec8102f80 0x7f7ec8198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:37.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.426+0000 7f7ec759e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ec81076b0 0x7f7ec8199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:37.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.427+0000 7f7ec759e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec8112560 con 0x7f7ec8102f80 2024-05-14T17:00:37.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.427+0000 7f7ec759e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec81123e0 con 0x7f7ec81076b0 2024-05-14T17:00:37.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.427+0000 7f7ec759e700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec81126e0 con 0x7f7ec81025d0 2024-05-14T17:00:37.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.427+0000 7f7ec6d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ec81076b0 0x7f7ec8199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:37.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.427+0000 7f7ec6d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ec81076b0 0x7f7ec8199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:50198/0 (socket says 172.21.15.2:50198) 2024-05-14T17:00:37.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.427+0000 7f7ec6d9d700 1 -- 172.21.15.2:0/899089169 learned_addr learned my addr 172.21.15.2:0/899089169 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:00:37.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.427+0000 7f7ec6d9d700 1 -- 172.21.15.2:0/899089169 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ec81025d0 msgr2=0x7f7ec8198790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T17:00:37.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.427+0000 7f7ec6d9d700 1 --2- 172.21.15.2:0/899089169 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ec81025d0 0x7f7ec8198790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:37.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.427+0000 7f7ec6d9d700 1 -- 172.21.15.2:0/899089169 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ec8102f80 msgr2=0x7f7ec8198cd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T17:00:37.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.427+0000 7f7ec6d9d700 1 --2- 172.21.15.2:0/899089169 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ec8102f80 0x7f7ec8198cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:37.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.427+0000 7f7ec6d9d700 1 -- 172.21.15.2:0/899089169 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ec81a8000 con 0x7f7ec81076b0 2024-05-14T17:00:37.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.428+0000 7f7ec6d9d700 1 --2- 172.21.15.2:0/899089169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ec81076b0 0x7f7ec8199210 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7ec000ef30 tx=0x7f7ec000c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:37.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.428+0000 7f7eb77fe700 1 -- 172.21.15.2:0/899089169 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec00092e0 con 0x7f7ec81076b0 2024-05-14T17:00:37.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.428+0000 7f7eb77fe700 1 -- 172.21.15.2:0/899089169 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ec0004bb0 con 0x7f7ec81076b0 2024-05-14T17:00:37.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.428+0000 7f7ec759e700 1 -- 172.21.15.2:0/899089169 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ec81a8290 con 0x7f7ec81076b0 2024-05-14T17:00:37.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.429+0000 7f7ec759e700 1 -- 172.21.15.2:0/899089169 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ec81a87d0 con 0x7f7ec81076b0 2024-05-14T17:00:37.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.429+0000 7f7eb77fe700 1 -- 172.21.15.2:0/899089169 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec0004280 con 0x7f7ec81076b0 2024-05-14T17:00:37.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.430+0000 7f7ec759e700 1 -- 172.21.15.2:0/899089169 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ec806a230 con 0x7f7ec81076b0 2024-05-14T17:00:37.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.432+0000 7f7eb77fe700 1 -- 172.21.15.2:0/899089169 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f7ec001b030 con 0x7f7ec81076b0 2024-05-14T17:00:37.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.433+0000 7f7eb77fe700 1 --2- 172.21.15.2:0/899089169 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f7eb00b2af0 0x7f7eb00b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:37.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.433+0000 7f7eb77fe700 1 -- 172.21.15.2:0/899089169 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f7ec00e21f0 con 0x7f7ec81076b0 2024-05-14T17:00:37.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.433+0000 7f7ec659c700 1 --2- 172.21.15.2:0/899089169 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f7eb00b2af0 0x7f7eb00b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:37.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.433+0000 7f7ec659c700 1 --2- 172.21.15.2:0/899089169 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f7eb00b2af0 0x7f7eb00b4fb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7eb80098e0 tx=0x7f7eb8000bc0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:37.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.438+0000 7f7eb77fe700 1 -- 172.21.15.2:0/899089169 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7ec0022020 con 0x7f7ec81076b0 2024-05-14T17:00:37.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.860+0000 7f7ec759e700 1 -- 172.21.15.2:0/899089169 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ec802e560 con 0x7f7ec81076b0 2024-05-14T17:00:37.867 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:00:37.874 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2275696923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2275696923},{"type":"v1","addr":"172.21.15.2:6835","nonce":2275696923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/716718761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":716718761},{"type":"v1","addr":"172.21.15.2:6839","nonce":716718761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/3644815179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3644815179},{"type":"v1","addr":"172.21.15.2:6841","nonce":3644815179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1880336731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1880336731},{"type":"v1","addr":"172.21.15.151:6833","nonce":1880336731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2415605548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2415605548},{"type":"v1","addr":"172.21.15.2:6837","nonce":2415605548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2635167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2635167},{"type":"v1","addr":"172.21.15.123:6833","nonce":2635167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:51:34.106919+0000","modified":"2024-05-14T17:00:23.454968+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24455,"mds_2":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24415":{"gid":24415,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/762044339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":762044339},{"type":"v1","addr":"172.21.15.151:6835","nonce":762044339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/132043151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":132043151},{"type":"v1","addr":"172.21.15.151:6837","nonce":132043151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/4107834375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":4107834375},{"type":"v1","addr":"172.21.15.151:6839","nonce":4107834375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.123:6835/1198670013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1198670013},{"type":"v1","addr":"172.21.15.123:6835","nonce":1198670013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/333426645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":333426645},{"type":"v1","addr":"172.21.15.123:6837","nonce":333426645}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/580275877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":580275877},{"type":"v1","addr":"172.21.15.123:6839","nonce":580275877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24431,24455]},"id":1}]} 2024-05-14T17:00:37.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.861+0000 7f7eb77fe700 1 -- 172.21.15.2:0/899089169 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11851 (secure 0 0 0) 0x7f7ec0010040 con 0x7f7ec81076b0 2024-05-14T17:00:37.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.865+0000 7f7ec759e700 1 -- 172.21.15.2:0/899089169 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f7eb00b2af0 msgr2=0x7f7eb00b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:37.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.865+0000 7f7ec759e700 1 --2- 172.21.15.2:0/899089169 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f7eb00b2af0 0x7f7eb00b4fb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7eb80098e0 tx=0x7f7eb8000bc0 comp rx=0 tx=0).stop 2024-05-14T17:00:37.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.865+0000 7f7ec759e700 1 -- 172.21.15.2:0/899089169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ec81076b0 msgr2=0x7f7ec8199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:37.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.865+0000 7f7ec759e700 1 --2- 172.21.15.2:0/899089169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ec81076b0 0x7f7ec8199210 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7ec000ef30 tx=0x7f7ec000c450 comp rx=0 tx=0).stop 2024-05-14T17:00:37.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.865+0000 7f7ec759e700 1 -- 172.21.15.2:0/899089169 shutdown_connections 2024-05-14T17:00:37.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.865+0000 7f7ec759e700 1 --2- 172.21.15.2:0/899089169 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f7eb00b2af0 0x7f7eb00b4fb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:37.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.865+0000 7f7ec759e700 1 --2- 172.21.15.2:0/899089169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ec81076b0 0x7f7ec8199210 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:37.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.865+0000 7f7ec759e700 1 --2- 172.21.15.2:0/899089169 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ec81025d0 0x7f7ec8198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:37.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.865+0000 7f7ec759e700 1 --2- 172.21.15.2:0/899089169 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ec8102f80 0x7f7ec8198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:37.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.865+0000 7f7ec759e700 1 -- 172.21.15.2:0/899089169 >> 172.21.15.2:0/899089169 conn(0x7f7ec80fde60 msgr2=0x7f7ec8106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:37.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.865+0000 7f7ec759e700 1 -- 172.21.15.2:0/899089169 shutdown_connections 2024-05-14T17:00:37.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:37.865+0000 7f7ec759e700 1 -- 172.21.15.2:0/899089169 wait complete. 2024-05-14T17:00:38.009 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 38 2024-05-14T17:00:38.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:00:38 smithi002 ceph-mon[31538]: pgmap v181: 97 pgs: 97 active+clean; 3.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.9 MiB/s wr, 566 op/s 2024-05-14T17:00:38.500 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:00:38 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/899089169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:00:38.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:00:38 smithi123 ceph-mon[35971]: pgmap v181: 97 pgs: 97 active+clean; 3.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.9 MiB/s wr, 566 op/s 2024-05-14T17:00:38.669 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:00:38 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/899089169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:00:38.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:00:38 smithi151 ceph-mon[36066]: pgmap v181: 97 pgs: 97 active+clean; 3.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.9 MiB/s wr, 566 op/s 2024-05-14T17:00:38.860 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:00:38 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/899089169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:00:39.114 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 133.7 --await-for 45.0' 2024-05-14T17:00:39.123 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 133.7 --await-for 45.0 2024-05-14T17:00:39.549 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:00:40.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.409+0000 7fa16aecf640 1 -- 172.21.15.2:0/527154404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa164074d50 msgr2=0x7fa164075130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.409+0000 7fa16aecf640 1 --2- 172.21.15.2:0/527154404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa164074d50 0x7fa164075130 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa158009890 tx=0x7fa15802a410 comp rx=0 tx=0).stop 2024-05-14T17:00:40.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.409+0000 7fa16aecf640 1 -- 172.21.15.2:0/527154404 shutdown_connections 2024-05-14T17:00:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.409+0000 7fa16aecf640 1 --2- 172.21.15.2:0/527154404 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa164113b10 0x7fa164115f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:40.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.409+0000 7fa16aecf640 1 --2- 172.21.15.2:0/527154404 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa164075670 0x7fa1641135d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.409+0000 7fa16aecf640 1 --2- 172.21.15.2:0/527154404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa164074d50 0x7fa164075130 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.409+0000 7fa16aecf640 1 -- 172.21.15.2:0/527154404 >> 172.21.15.2:0/527154404 conn(0x7fa164079ee0 msgr2=0x7fa16407a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.409+0000 7fa16aecf640 1 -- 172.21.15.2:0/527154404 shutdown_connections 2024-05-14T17:00:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.409+0000 7fa16aecf640 1 -- 172.21.15.2:0/527154404 wait complete. 2024-05-14T17:00:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.410+0000 7fa16aecf640 1 Processor -- start 2024-05-14T17:00:40.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.410+0000 7fa16aecf640 1 -- start start 2024-05-14T17:00:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.411+0000 7fa16aecf640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa164074d50 0x7fa1641a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.411+0000 7fa16aecf640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa164075670 0x7fa1641a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.411+0000 7fa16aecf640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa164113b10 0x7fa16419f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.411+0000 7fa16aecf640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa164119370 con 0x7fa164074d50 2024-05-14T17:00:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.411+0000 7fa16946c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa164113b10 0x7fa16419f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.411+0000 7fa16946c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa164113b10 0x7fa16419f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:45230/0 (socket says 172.21.15.2:45230) 2024-05-14T17:00:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.411+0000 7fa16946c640 1 -- 172.21.15.2:0/1508556171 learned_addr learned my addr 172.21.15.2:0/1508556171 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:00:40.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.411+0000 7fa168c6b640 1 --2- 172.21.15.2:0/1508556171 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa164074d50 0x7fa1641a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.411+0000 7fa163fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa164075670 0x7fa1641a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.411+0000 7fa16aecf640 1 -- 172.21.15.2:0/1508556171 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa1641191f0 con 0x7fa164075670 2024-05-14T17:00:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.411+0000 7fa16aecf640 1 -- 172.21.15.2:0/1508556171 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa1641194f0 con 0x7fa164113b10 2024-05-14T17:00:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.412+0000 7fa16946c640 1 -- 172.21.15.2:0/1508556171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa164075670 msgr2=0x7fa1641a5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:40.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.412+0000 7fa16946c640 1 --2- 172.21.15.2:0/1508556171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa164075670 0x7fa1641a5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.412+0000 7fa16946c640 1 -- 172.21.15.2:0/1508556171 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa164074d50 msgr2=0x7fa1641a55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.412+0000 7fa16946c640 1 --2- 172.21.15.2:0/1508556171 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa164074d50 0x7fa1641a55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:40.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.412+0000 7fa16946c640 1 -- 172.21.15.2:0/1508556171 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa16419ff30 con 0x7fa164113b10 2024-05-14T17:00:40.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.412+0000 7fa16946c640 1 --2- 172.21.15.2:0/1508556171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa164113b10 0x7fa16419f670 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00efc0 tx=0x7fa15c00c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.413+0000 7fa161ffb640 1 -- 172.21.15.2:0/1508556171 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15c02b020 con 0x7fa164113b10 2024-05-14T17:00:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.413+0000 7fa161ffb640 1 -- 172.21.15.2:0/1508556171 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa15c003b70 con 0x7fa164113b10 2024-05-14T17:00:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.413+0000 7fa161ffb640 1 -- 172.21.15.2:0/1508556171 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15c027680 con 0x7fa164113b10 2024-05-14T17:00:40.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.413+0000 7fa16aecf640 1 -- 172.21.15.2:0/1508556171 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1641a01c0 con 0x7fa164113b10 2024-05-14T17:00:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.413+0000 7fa16aecf640 1 -- 172.21.15.2:0/1508556171 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa164106da0 con 0x7fa164113b10 2024-05-14T17:00:40.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.416+0000 7fa161ffb640 1 -- 172.21.15.2:0/1508556171 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fa15c004020 con 0x7fa164113b10 2024-05-14T17:00:40.939 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.416+0000 7fa16dbe1740 1 Processor -- start 2024-05-14T17:00:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.417+0000 7fa16dbe1740 1 -- start start 2024-05-14T17:00:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.417+0000 7fa16dbe1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bbab4a78f0 0x55bbab4a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:40.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.417+0000 7fa16dbe1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bbab4a8210 0x55bbab4b2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:40.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.417+0000 7fa16dbe1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bbab4b2ad0 0x55bbab4b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:40.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.417+0000 7fa16dbe1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55bbab41ac30 con 0x55bbab4a8210 2024-05-14T17:00:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.417+0000 7fa16dbe1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bbab40a7d0 con 0x55bbab4b2ad0 2024-05-14T17:00:41.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.417+0000 7fa16dbe1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55bbab40e810 con 0x55bbab4a78f0 2024-05-14T17:00:41.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.417+0000 7fa168c6b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bbab4a8210 0x55bbab4b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.417+0000 7fa163fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bbab4a78f0 0x55bbab4a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.417+0000 7fa168c6b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bbab4a8210 0x55bbab4b2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59434/0 (socket says 172.21.15.2:59434) 2024-05-14T17:00:41.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.417+0000 7fa168c6b640 1 -- 172.21.15.2:0/1291678285 learned_addr learned my addr 172.21.15.2:0/1291678285 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:00:41.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.417+0000 7fa163fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bbab4a78f0 0x55bbab4a7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:45240/0 (socket says 172.21.15.2:45240) 2024-05-14T17:00:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.417+0000 7fa163fff640 1 --2- 172.21.15.2:0/1291678285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bbab4b2ad0 0x55bbab4b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.417+0000 7fa161ffb640 1 --2- 172.21.15.2:0/1508556171 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fa1540b2a90 0x7fa1540b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:41.062 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.417+0000 7fa163fff640 1 -- 172.21.15.2:0/1291678285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bbab4b2ad0 msgr2=0x55bbab4b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.417+0000 7fa163fff640 1 --2- 172.21.15.2:0/1291678285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bbab4b2ad0 0x55bbab4b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.417+0000 7fa16946c640 1 --2- 172.21.15.2:0/1508556171 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fa1540b2a90 0x7fa1540b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.417+0000 7fa163fff640 1 -- 172.21.15.2:0/1291678285 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bbab4a8210 msgr2=0x55bbab4b2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.417+0000 7fa163fff640 1 --2- 172.21.15.2:0/1291678285 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bbab4a8210 0x55bbab4b2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.417+0000 7fa163fff640 1 -- 172.21.15.2:0/1291678285 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbab11f210 con 0x55bbab4a78f0 2024-05-14T17:00:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.417+0000 7fa168c6b640 1 --2- 172.21.15.2:0/1291678285 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bbab4a8210 0x55bbab4b2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:00:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.417+0000 7fa161ffb640 1 -- 172.21.15.2:0/1508556171 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fa15c0f39a0 con 0x7fa164113b10 2024-05-14T17:00:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.418+0000 7fa163fff640 1 --2- 172.21.15.2:0/1291678285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bbab4a78f0 0x55bbab4a7cd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa150009fd0 tx=0x7fa150004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ac9e629af2473af server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.418+0000 7fa14effd640 1 -- 172.21.15.2:0/1291678285 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa150004050 con 0x55bbab4a78f0 2024-05-14T17:00:41.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.418+0000 7fa16946c640 1 --2- 172.21.15.2:0/1508556171 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fa1540b2a90 0x7fa1540b4f50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa15c0b7f50 tx=0x7fa15c0ba6f0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:41.178 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.418+0000 7fa14effd640 1 -- 172.21.15.2:0/1291678285 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa15000fc40 con 0x55bbab4a78f0 2024-05-14T17:00:41.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.418+0000 7fa14effd640 1 -- 172.21.15.2:0/1291678285 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa150003850 con 0x55bbab4a78f0 2024-05-14T17:00:41.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.419+0000 7fa16dbe1740 1 -- 172.21.15.2:0/1291678285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bbab4a78f0 msgr2=0x55bbab4a7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.419+0000 7fa16dbe1740 1 --2- 172.21.15.2:0/1291678285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bbab4a78f0 0x55bbab4a7cd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa150009fd0 tx=0x7fa150004730 comp rx=0 tx=0).stop 2024-05-14T17:00:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.419+0000 7fa16dbe1740 1 -- 172.21.15.2:0/1291678285 shutdown_connections 2024-05-14T17:00:41.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.419+0000 7fa16dbe1740 1 --2- 172.21.15.2:0/1291678285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bbab4b2ad0 0x55bbab4b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:41.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.419+0000 7fa16dbe1740 1 --2- 172.21.15.2:0/1291678285 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bbab4a8210 0x55bbab4b2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.419+0000 7fa16dbe1740 1 --2- 172.21.15.2:0/1291678285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bbab4a78f0 0x55bbab4a7cd0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.419+0000 7fa16dbe1740 1 -- 172.21.15.2:0/1291678285 >> 172.21.15.2:0/1291678285 conn(0x55bbab4a3e00 msgr2=0x55bbab4a41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.419+0000 7fa16dbe1740 1 -- 172.21.15.2:0/1291678285 shutdown_connections 2024-05-14T17:00:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.419+0000 7fa16dbe1740 1 -- 172.21.15.2:0/1291678285 wait complete. 2024-05-14T17:00:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.420+0000 7fa16dbe1740 1 -- start start 2024-05-14T17:00:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.420+0000 7fa16dbe1740 1 Processor -- start 2024-05-14T17:00:41.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.420+0000 7fa16dbe1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bbab4a78f0 0x55bbab4b24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.420+0000 7fa16dbe1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bbab4a8210 0x55bbab4af7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.420+0000 7fa16dbe1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bbab4b2ad0 0x55bbab4afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.420+0000 7fa16dbe1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55bbab0f0790 con 0x55bbab4a8210 2024-05-14T17:00:41.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.420+0000 7fa16dbe1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bbab3c6890 con 0x55bbab4a78f0 2024-05-14T17:00:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.420+0000 7fa16dbe1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55bbab4b57a0 con 0x55bbab4b2ad0 2024-05-14T17:00:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.421+0000 7fa16dbe1740 10 client.0 _ll_register_callbacks cb 0x55bbab3c9d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:00:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.421+0000 7fa168c6b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bbab4a8210 0x55bbab4af7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:41.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.421+0000 7fa163fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bbab4a78f0 0x55bbab4b24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:41.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.421+0000 7fa163fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bbab4a78f0 0x55bbab4b24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:50236/0 (socket says 172.21.15.2:50236) 2024-05-14T17:00:41.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.421+0000 7fa168c6b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bbab4a8210 0x55bbab4af7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59442/0 (socket says 172.21.15.2:59442) 2024-05-14T17:00:41.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.421+0000 7fa163fff640 1 -- 172.21.15.2:0/1848773811 learned_addr learned my addr 172.21.15.2:0/1848773811 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:00:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.421+0000 7fa163fff640 1 --2- 172.21.15.2:0/1848773811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bbab4b2ad0 0x55bbab4afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:41.514 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.421+0000 7fa168c6b640 1 -- 172.21.15.2:0/1848773811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bbab4b2ad0 msgr2=0x55bbab4afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:41.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.421+0000 7fa168c6b640 1 --2- 172.21.15.2:0/1848773811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bbab4b2ad0 0x55bbab4afdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.421+0000 7fa168c6b640 1 -- 172.21.15.2:0/1848773811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bbab4a78f0 msgr2=0x55bbab4b24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:41.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.421+0000 7fa168c6b640 1 --2- 172.21.15.2:0/1848773811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bbab4a78f0 0x55bbab4b24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:41.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.421+0000 7fa168c6b640 1 -- 172.21.15.2:0/1848773811 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbab1eb620 con 0x55bbab4a8210 2024-05-14T17:00:41.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.421+0000 7fa163fff640 1 --2- 172.21.15.2:0/1848773811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bbab4a78f0 0x55bbab4b24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:00:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.421+0000 7fa168c6b640 1 --2- 172.21.15.2:0/1848773811 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bbab4a8210 0x55bbab4af7b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa15802aa00 tx=0x7fa158030bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c35d753b4c19f77 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:41.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.421+0000 7fa143fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:00:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.421+0000 7fa143fff640 1 -- 172.21.15.2:0/1848773811 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa158030e30 con 0x55bbab4a8210 2024-05-14T17:00:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.421+0000 7fa143fff640 1 -- 172.21.15.2:0/1848773811 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fa158005040 con 0x55bbab4a8210 2024-05-14T17:00:41.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.421+0000 7fa16dbe1740 1 -- 172.21.15.2:0/1848773811 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa15000fc40 con 0x55bbab4a8210 2024-05-14T17:00:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.422+0000 7fa143fff640 1 -- 172.21.15.2:0/1848773811 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fa158005240 con 0x55bbab4a8210 2024-05-14T17:00:41.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.422+0000 7fa16dbe1740 10 client.15942 fetch_fsmap learned FSMap version 38 2024-05-14T17:00:41.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.422+0000 7fa16dbe1740 1 -- 172.21.15.2:0/1848773811 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55bbab4c1240 con 0x55bbab4a8210 2024-05-14T17:00:41.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.422+0000 7fa143fff640 1 -- 172.21.15.2:0/1848773811 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa15000fc40 con 0x55bbab4a8210 2024-05-14T17:00:41.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.422+0000 7fa143fff640 1 -- 172.21.15.2:0/1848773811 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) ==== 8342+0+0 (secure 0 0 0) 0x7fa158046460 con 0x55bbab4a8210 2024-05-14T17:00:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.422+0000 7fa16dbe1740 10 client.15942 fetch_fsmap finished waiting for FSMap version 38 2024-05-14T17:00:41.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.422+0000 7fa16dbe1740 10 client.15942 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:00:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.422+0000 7fa16dbe1740 20 client.15942 populate_metadata read hostname 'smithi002' 2024-05-14T17:00:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.422+0000 7fa16dbe1740 1 --2- 172.21.15.2:0/1848773811 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55bbab4c2700 0x55bbab4c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:41.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.422+0000 7fa16dbe1740 4 client.15942 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:00:41.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.422+0000 7fa16dbe1740 1 -- 172.21.15.2:0/1848773811 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbab41cb80 con 0x55bbab4c2700 2024-05-14T17:00:41.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.422+0000 7fa163fff640 1 --2- 172.21.15.2:0/1848773811 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55bbab4c2700 0x55bbab4c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.423+0000 7fa143fff640 10 client.15942 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:00:41.788 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.423+0000 7fa163fff640 1 --2- 172.21.15.2:0/1848773811 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55bbab4c2700 0x55bbab4c4af0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8319bd0a112f14e9 server_cookie=7af9b52c53e5eac0 in_seq=0 out_seq=0 2024-05-14T17:00:41.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.423+0000 7fa143fff640 1 -- 172.21.15.2:0/1848773811 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bbab41cb80 con 0x55bbab4c2700 2024-05-14T17:00:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.423+0000 7fa143fff640 10 client.15942 handle_command_reply: tid=0 2024-05-14T17:00:41.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.424+0000 7fa16dbe1740 1 client.15942 shutdown 2024-05-14T17:00:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.424+0000 7fa16dbe1740 1 -- 172.21.15.2:0/1848773811 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bbab4a8210 msgr2=0x55bbab4af7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:41.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.424+0000 7fa16dbe1740 1 --2- 172.21.15.2:0/1848773811 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bbab4a8210 0x55bbab4af7b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa15802aa00 tx=0x7fa158030bb0 comp rx=0 tx=0).stop 2024-05-14T17:00:41.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.424+0000 7fa16dbe1740 1 -- 172.21.15.2:0/1848773811 shutdown_connections 2024-05-14T17:00:41.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.424+0000 7fa16dbe1740 1 --2- 172.21.15.2:0/1848773811 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55bbab4c2700 0x55bbab4c4af0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:41.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.424+0000 7fa16dbe1740 1 --2- 172.21.15.2:0/1848773811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bbab4b2ad0 0x55bbab4afdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.424+0000 7fa143fff640 0 client.15942 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:00:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.424+0000 7fa16dbe1740 1 --2- 172.21.15.2:0/1848773811 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bbab4a8210 0x55bbab4af7b0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.424+0000 7fa16dbe1740 1 --2- 172.21.15.2:0/1848773811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bbab4a78f0 0x55bbab4b24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.424+0000 7fa16dbe1740 1 -- 172.21.15.2:0/1848773811 >> 172.21.15.2:0/1848773811 conn(0x55bbab4a3e00 msgr2=0x55bbab4a3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.424+0000 7fa16dbe1740 1 -- 172.21.15.2:0/1848773811 shutdown_connections 2024-05-14T17:00:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.426+0000 7fa16dbe1740 1 -- 172.21.15.2:0/1848773811 wait complete. 2024-05-14T17:00:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.426+0000 7fa16dbe1740 20 client.15942 trim_cache size 0 max 16384 2024-05-14T17:00:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.451+0000 7fa16dbe1740 1 Processor -- start 2024-05-14T17:00:41.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.452+0000 7fa16dbe1740 1 -- start start 2024-05-14T17:00:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.452+0000 7fa16dbe1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bbab4a78f0 0x55bbab49ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.452+0000 7fa16dbe1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bbab4a8210 0x55bbab4be670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.452+0000 7fa16dbe1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bbab4b2ad0 0x55bbab4b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.452+0000 7fa16dbe1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa1641aed30 con 0x55bbab4a78f0 2024-05-14T17:00:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.452+0000 7fa16dbe1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa1641ac5d0 con 0x55bbab4a8210 2024-05-14T17:00:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.452+0000 7fa16dbe1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa16410d680 con 0x55bbab4b2ad0 2024-05-14T17:00:42.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.452+0000 7fa163fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bbab4a78f0 0x55bbab49ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:42.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.452+0000 7fa163fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bbab4a78f0 0x55bbab49ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59458/0 (socket says 172.21.15.2:59458) 2024-05-14T17:00:42.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.452+0000 7fa163fff640 1 -- 172.21.15.2:0/2484782906 learned_addr learned my addr 172.21.15.2:0/2484782906 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:00:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.452+0000 7fa168c6b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bbab4a8210 0x55bbab4be670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.452+0000 7fa163fff640 1 --2- 172.21.15.2:0/2484782906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bbab4b2ad0 0x55bbab4b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.453+0000 7fa163fff640 1 -- 172.21.15.2:0/2484782906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bbab4b2ad0 msgr2=0x55bbab4b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.453+0000 7fa163fff640 1 --2- 172.21.15.2:0/2484782906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bbab4b2ad0 0x55bbab4b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.453+0000 7fa163fff640 1 -- 172.21.15.2:0/2484782906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bbab4a8210 msgr2=0x55bbab4be670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.453+0000 7fa163fff640 1 --2- 172.21.15.2:0/2484782906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bbab4a8210 0x55bbab4be670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.453+0000 7fa163fff640 1 -- 172.21.15.2:0/2484782906 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbab4c4e60 con 0x55bbab4a78f0 2024-05-14T17:00:42.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.453+0000 7fa163fff640 1 --2- 172.21.15.2:0/2484782906 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bbab4a78f0 0x55bbab49ef90 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa150010010 tx=0x7fa150009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1ed82fe98b6c5f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:42.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.453+0000 7fa14e7fc640 1 -- 172.21.15.2:0/2484782906 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15000fa90 con 0x55bbab4a78f0 2024-05-14T17:00:42.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.453+0000 7fa14e7fc640 1 -- 172.21.15.2:0/2484782906 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa15000fc30 con 0x55bbab4a78f0 2024-05-14T17:00:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.453+0000 7fa14e7fc640 1 -- 172.21.15.2:0/2484782906 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa150003be0 con 0x55bbab4a78f0 2024-05-14T17:00:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.454+0000 7fa16dbe1740 1 -- 172.21.15.2:0/2484782906 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bbab4a78f0 msgr2=0x55bbab49ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.454+0000 7fa16dbe1740 1 --2- 172.21.15.2:0/2484782906 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bbab4a78f0 0x55bbab49ef90 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa150010010 tx=0x7fa150009ac0 comp rx=0 tx=0).stop 2024-05-14T17:00:42.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.454+0000 7fa16dbe1740 1 -- 172.21.15.2:0/2484782906 shutdown_connections 2024-05-14T17:00:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.454+0000 7fa16dbe1740 1 --2- 172.21.15.2:0/2484782906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bbab4b2ad0 0x55bbab4b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.454+0000 7fa16dbe1740 1 --2- 172.21.15.2:0/2484782906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bbab4a8210 0x55bbab4be670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.454+0000 7fa16dbe1740 1 --2- 172.21.15.2:0/2484782906 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bbab4a78f0 0x55bbab49ef90 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.454+0000 7fa16dbe1740 1 -- 172.21.15.2:0/2484782906 >> 172.21.15.2:0/2484782906 conn(0x55bbab4a3e00 msgr2=0x55bbab4a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.454+0000 7fa16dbe1740 1 -- 172.21.15.2:0/2484782906 shutdown_connections 2024-05-14T17:00:42.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.454+0000 7fa16dbe1740 1 -- 172.21.15.2:0/2484782906 wait complete. 2024-05-14T17:00:42.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.455+0000 7fa16dbe1740 1 -- start start 2024-05-14T17:00:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.455+0000 7fa16dbe1740 1 Processor -- start 2024-05-14T17:00:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.455+0000 7fa16dbe1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bbab4a78f0 0x55bbab4bacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:42.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.455+0000 7fa16dbe1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bbab4a8210 0x55bbab4ac0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.455+0000 7fa16dbe1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bbab4b2ad0 0x55bbab4ac620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.455+0000 7fa16dbe1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa16410dcf0 con 0x55bbab4a78f0 2024-05-14T17:00:42.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.455+0000 7fa16dbe1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa16410f160 con 0x55bbab4b2ad0 2024-05-14T17:00:42.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.455+0000 7fa16dbe1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa16410e3c0 con 0x55bbab4a8210 2024-05-14T17:00:42.334 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.455+0000 7fa163fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bbab4a78f0 0x55bbab4bacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:42.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.455+0000 7fa163fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bbab4a78f0 0x55bbab4bacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59460/0 (socket says 172.21.15.2:59460) 2024-05-14T17:00:42.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.455+0000 7fa16dbe1740 10 client.0 _ll_register_callbacks cb 0x7fa1580308a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:00:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.455+0000 7fa163fff640 1 -- 172.21.15.2:0/1718631271 learned_addr learned my addr 172.21.15.2:0/1718631271 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:00:42.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.455+0000 7fa168c6b640 1 --2- 172.21.15.2:0/1718631271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bbab4a8210 0x55bbab4ac0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:42.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.456+0000 7fa168c6b640 1 -- 172.21.15.2:0/1718631271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bbab4b2ad0 msgr2=0x55bbab4ac620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.456+0000 7fa168c6b640 1 --2- 172.21.15.2:0/1718631271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bbab4b2ad0 0x55bbab4ac620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.456+0000 7fa168c6b640 1 -- 172.21.15.2:0/1718631271 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bbab4a78f0 msgr2=0x55bbab4bacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.456+0000 7fa168c6b640 1 --2- 172.21.15.2:0/1718631271 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bbab4a78f0 0x55bbab4bacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.456+0000 7fa168c6b640 1 -- 172.21.15.2:0/1718631271 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbab4acb60 con 0x55bbab4a8210 2024-05-14T17:00:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.456+0000 7fa168c6b640 1 --2- 172.21.15.2:0/1718631271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bbab4a8210 0x55bbab4ac0e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa1580468b0 tx=0x7fa158005a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8de5826e76189c89 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:42.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.456+0000 7fa143fff640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-14T17:00:42.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.456+0000 7fa143fff640 1 -- 172.21.15.2:0/1718631271 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa158053070 con 0x55bbab4a8210 2024-05-14T17:00:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.456+0000 7fa143fff640 1 -- 172.21.15.2:0/1718631271 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fa158030e30 con 0x55bbab4a8210 2024-05-14T17:00:42.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.456+0000 7fa16dbe1740 1 -- 172.21.15.2:0/1718631271 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa15000fc30 con 0x55bbab4a8210 2024-05-14T17:00:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.457+0000 7fa143fff640 1 -- 172.21.15.2:0/1718631271 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fa158009450 con 0x55bbab4a8210 2024-05-14T17:00:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.457+0000 7fa16dbe1740 10 client.25642 fetch_fsmap learned FSMap version 38 2024-05-14T17:00:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.457+0000 7fa16dbe1740 1 -- 172.21.15.2:0/1718631271 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55bbab52cc70 con 0x55bbab4a8210 2024-05-14T17:00:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.457+0000 7fa143fff640 1 -- 172.21.15.2:0/1718631271 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa158046aa0 con 0x55bbab4a8210 2024-05-14T17:00:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.457+0000 7fa143fff640 1 -- 172.21.15.2:0/1718631271 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 38) ==== 8342+0+0 (secure 0 0 0) 0x7fa158003860 con 0x55bbab4a8210 2024-05-14T17:00:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.457+0000 7fa16dbe1740 10 client.25642 fetch_fsmap finished waiting for FSMap version 38 2024-05-14T17:00:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.457+0000 7fa16dbe1740 10 client.25642 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:00:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.457+0000 7fa16dbe1740 20 client.25642 populate_metadata read hostname 'smithi002' 2024-05-14T17:00:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.458+0000 7fa16dbe1740 1 --2- 172.21.15.2:0/1718631271 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55bbab4c2700 0x55bbab52f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.458+0000 7fa16dbe1740 4 client.25642 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:00:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.458+0000 7fa16dbe1740 1 -- 172.21.15.2:0/1718631271 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fa15000fc30 con 0x55bbab4c2700 2024-05-14T17:00:42.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.458+0000 7fa163fff640 1 --2- 172.21.15.2:0/1718631271 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55bbab4c2700 0x55bbab52f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.458+0000 7fa143fff640 10 client.25642 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:00:42.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.459+0000 7fa163fff640 1 --2- 172.21.15.2:0/1718631271 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55bbab4c2700 0x55bbab52f400 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3367a9f2e8193fa server_cookie=44eae7c9a0846a09 in_seq=0 out_seq=0 2024-05-14T17:00:42.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.470+0000 7fa143fff640 1 -- 172.21.15.2:0/1718631271 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7fa15000fc30 con 0x55bbab4c2700 2024-05-14T17:00:42.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.470+0000 7fa143fff640 10 client.25642 handle_command_reply: tid=0 2024-05-14T17:00:42.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.470+0000 7fa16dbe1740 1 client.25642 shutdown 2024-05-14T17:00:42.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.470+0000 7fa16dbe1740 1 -- 172.21.15.2:0/1718631271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bbab4a8210 msgr2=0x55bbab4ac0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.470+0000 7fa16dbe1740 1 --2- 172.21.15.2:0/1718631271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bbab4a8210 0x55bbab4ac0e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa1580468b0 tx=0x7fa158005a90 comp rx=0 tx=0).stop 2024-05-14T17:00:42.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.470+0000 7fa16dbe1740 1 -- 172.21.15.2:0/1718631271 shutdown_connections 2024-05-14T17:00:42.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.470+0000 7fa16dbe1740 1 --2- 172.21.15.2:0/1718631271 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55bbab4c2700 0x55bbab52f400 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.470+0000 7fa16dbe1740 1 --2- 172.21.15.2:0/1718631271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bbab4b2ad0 0x55bbab4ac620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:42.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.470+0000 7fa16dbe1740 1 --2- 172.21.15.2:0/1718631271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bbab4a8210 0x55bbab4ac0e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.470+0000 7fa16dbe1740 1 --2- 172.21.15.2:0/1718631271 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bbab4a78f0 0x55bbab4bacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.470+0000 7fa143fff640 0 client.25642 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:00:42.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.470+0000 7fa16dbe1740 1 -- 172.21.15.2:0/1718631271 >> 172.21.15.2:0/1718631271 conn(0x55bbab4a3e00 msgr2=0x55bbab4a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.470+0000 7fa16dbe1740 1 -- 172.21.15.2:0/1718631271 shutdown_connections 2024-05-14T17:00:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.470+0000 7fa16dbe1740 1 -- 172.21.15.2:0/1718631271 wait complete. 2024-05-14T17:00:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.470+0000 7fa16dbe1740 20 client.25642 trim_cache size 0 max 16384 2024-05-14T17:00:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:00:42.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:00:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:00:42.832 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:00:40 smithi151 ceph-mon[36066]: pgmap v182: 97 pgs: 97 active+clean; 3.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 8.4 MiB/s wr, 665 op/s 2024-05-14T17:00:42.843 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:00:40 smithi123 ceph-mon[35971]: pgmap v182: 97 pgs: 97 active+clean; 3.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 8.4 MiB/s wr, 665 op/s 2024-05-14T17:00:42.851 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:00:42 smithi123 ceph-mon[35971]: pgmap v183: 97 pgs: 97 active+clean; 3.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.2 MiB/s wr, 492 op/s 2024-05-14T17:00:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.471+0000 7fa16aecf640 1 -- 172.21.15.2:0/1508556171 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fa1540b2a90 msgr2=0x7fa1540b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.471+0000 7fa16aecf640 1 --2- 172.21.15.2:0/1508556171 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fa1540b2a90 0x7fa1540b4f50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa15c0b7f50 tx=0x7fa15c0ba6f0 comp rx=0 tx=0).stop 2024-05-14T17:00:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.471+0000 7fa16aecf640 1 -- 172.21.15.2:0/1508556171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa164113b10 msgr2=0x7fa16419f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.471+0000 7fa16aecf640 1 --2- 172.21.15.2:0/1508556171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa164113b10 0x7fa16419f670 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00efc0 tx=0x7fa15c00c800 comp rx=0 tx=0).stop 2024-05-14T17:00:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.471+0000 7fa16aecf640 1 -- 172.21.15.2:0/1508556171 shutdown_connections 2024-05-14T17:00:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.471+0000 7fa16aecf640 1 --2- 172.21.15.2:0/1508556171 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fa1540b2a90 0x7fa1540b4f50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.471+0000 7fa16aecf640 1 --2- 172.21.15.2:0/1508556171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa164113b10 0x7fa16419f670 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:42.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.471+0000 7fa16aecf640 1 --2- 172.21.15.2:0/1508556171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa164075670 0x7fa1641a5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.471+0000 7fa16aecf640 1 --2- 172.21.15.2:0/1508556171 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa164074d50 0x7fa1641a55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.471+0000 7fa16aecf640 1 -- 172.21.15.2:0/1508556171 >> 172.21.15.2:0/1508556171 conn(0x7fa164079ee0 msgr2=0x7fa1640ff930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:42.993 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.471+0000 7fa16aecf640 1 -- 172.21.15.2:0/1508556171 shutdown_connections 2024-05-14T17:00:43.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:40.471+0000 7fa16aecf640 1 -- 172.21.15.2:0/1508556171 wait complete. 2024-05-14T17:00:43.009 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:00:40 smithi002 ceph-mon[31538]: pgmap v182: 97 pgs: 97 active+clean; 3.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 8.4 MiB/s wr, 665 op/s 2024-05-14T17:00:43.017 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 325.0 s, 3207.0 tps, lat 31.435 ms stddev 35.688 2024-05-14T17:00:43.026 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:00:42 smithi151 ceph-mon[36066]: pgmap v183: 97 pgs: 97 active+clean; 3.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.2 MiB/s wr, 492 op/s 2024-05-14T17:00:43.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.158+0000 7fef86766700 1 -- 172.21.15.2:0/2354925633 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef80105fd0 msgr2=0x7fef80106480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:43.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.158+0000 7fef86766700 1 --2- 172.21.15.2:0/2354925633 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef80105fd0 0x7fef80106480 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fef7800b0a0 tx=0x7fef7800b3b0 comp rx=0 tx=0).stop 2024-05-14T17:00:43.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.159+0000 7fef86766700 1 -- 172.21.15.2:0/2354925633 shutdown_connections 2024-05-14T17:00:43.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.159+0000 7fef86766700 1 --2- 172.21.15.2:0/2354925633 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef80105fd0 0x7fef80106480 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:43.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.159+0000 7fef86766700 1 --2- 172.21.15.2:0/2354925633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef80108560 0x7fef8006bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:43.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.159+0000 7fef86766700 1 --2- 172.21.15.2:0/2354925633 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef801089d0 0x7fef8006c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:43.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.159+0000 7fef86766700 1 -- 172.21.15.2:0/2354925633 >> 172.21.15.2:0/2354925633 conn(0x7fef800793f0 msgr2=0x7fef80079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:43.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.159+0000 7fef86766700 1 -- 172.21.15.2:0/2354925633 shutdown_connections 2024-05-14T17:00:43.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.159+0000 7fef86766700 1 -- 172.21.15.2:0/2354925633 wait complete. 2024-05-14T17:00:43.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.160+0000 7fef86766700 1 Processor -- start 2024-05-14T17:00:43.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.160+0000 7fef86766700 1 -- start start 2024-05-14T17:00:43.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.160+0000 7fef86766700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef80105fd0 0x7fef800749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:43.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.160+0000 7fef86766700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef80108560 0x7fef80074f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:43.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.160+0000 7fef86766700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef801089d0 0x7fef8006eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:43.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.160+0000 7fef86766700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fef8006d820 con 0x7fef80108560 2024-05-14T17:00:43.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.160+0000 7fef86766700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fef8006d6a0 con 0x7fef801089d0 2024-05-14T17:00:43.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.160+0000 7fef86766700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fef8006d9a0 con 0x7fef80105fd0 2024-05-14T17:00:43.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.161+0000 7fef7ffff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef80105fd0 0x7fef800749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:43.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.161+0000 7fef84d03700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef801089d0 0x7fef8006eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:43.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.161+0000 7fef84d03700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef801089d0 0x7fef8006eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:50284/0 (socket says 172.21.15.2:50284) 2024-05-14T17:00:43.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.161+0000 7fef84d03700 1 -- 172.21.15.2:0/192915036 learned_addr learned my addr 172.21.15.2:0/192915036 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:00:43.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.161+0000 7fef84d03700 1 -- 172.21.15.2:0/192915036 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef80105fd0 msgr2=0x7fef800749d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:43.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.161+0000 7fef84d03700 1 --2- 172.21.15.2:0/192915036 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef80105fd0 0x7fef800749d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:43.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.161+0000 7fef84d03700 1 -- 172.21.15.2:0/192915036 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef80108560 msgr2=0x7fef80074f10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T17:00:43.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.161+0000 7fef84d03700 1 --2- 172.21.15.2:0/192915036 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef80108560 0x7fef80074f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:43.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.161+0000 7fef84d03700 1 -- 172.21.15.2:0/192915036 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef8006f2b0 con 0x7fef801089d0 2024-05-14T17:00:43.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.161+0000 7fef84d03700 1 --2- 172.21.15.2:0/192915036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef801089d0 0x7fef8006eaf0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fef780062a0 tx=0x7fef7800bef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:43.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.162+0000 7fef7d7fa700 1 -- 172.21.15.2:0/192915036 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef78021070 con 0x7fef801089d0 2024-05-14T17:00:43.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.162+0000 7fef7d7fa700 1 -- 172.21.15.2:0/192915036 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef78007d00 con 0x7fef801089d0 2024-05-14T17:00:43.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.162+0000 7fef7d7fa700 1 -- 172.21.15.2:0/192915036 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7801d450 con 0x7fef801089d0 2024-05-14T17:00:43.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.162+0000 7fef86766700 1 -- 172.21.15.2:0/192915036 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef8006f540 con 0x7fef801089d0 2024-05-14T17:00:43.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.163+0000 7fef86766700 1 -- 172.21.15.2:0/192915036 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef801b5180 con 0x7fef801089d0 2024-05-14T17:00:43.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.166+0000 7fef7d7fa700 1 -- 172.21.15.2:0/192915036 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fef78003a00 con 0x7fef801089d0 2024-05-14T17:00:43.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.167+0000 7fef7d7fa700 1 --2- 172.21.15.2:0/192915036 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fef680b2be0 0x7fef680b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:43.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.167+0000 7fef7ffff700 1 --2- 172.21.15.2:0/192915036 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fef680b2be0 0x7fef680b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:43.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.167+0000 7fef7d7fa700 1 -- 172.21.15.2:0/192915036 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fef780e3a80 con 0x7fef801089d0 2024-05-14T17:00:43.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.168+0000 7fef7ffff700 1 --2- 172.21.15.2:0/192915036 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fef680b2be0 0x7fef680b50a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fef74009c60 tx=0x7fef74009260 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:43.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.168+0000 7fef90ffcb80 1 Processor -- start 2024-05-14T17:00:43.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.169+0000 7fef90ffcb80 1 -- start start 2024-05-14T17:00:43.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.169+0000 7fef90ffcb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614e6a78e30 0x5614e6a79210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:43.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.169+0000 7fef90ffcb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614e6a79750 0x5614e6a73540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:43.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.169+0000 7fef90ffcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614e6a73b60 0x5614e6a73fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:43.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.169+0000 7fef90ffcb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5614e69d1e20 con 0x5614e6a78e30 2024-05-14T17:00:43.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.169+0000 7fef90ffcb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5614e69c19c0 con 0x5614e6a73b60 2024-05-14T17:00:43.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.169+0000 7fef90ffcb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5614e69c5a00 con 0x5614e6a79750 2024-05-14T17:00:43.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.170+0000 7fef84d03700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614e6a73b60 0x5614e6a73fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:43.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.170+0000 7fef84d03700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614e6a73b60 0x5614e6a73fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:50290/0 (socket says 172.21.15.2:50290) 2024-05-14T17:00:43.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.170+0000 7fef84d03700 1 -- 172.21.15.2:0/4171494720 learned_addr learned my addr 172.21.15.2:0/4171494720 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:00:43.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.170+0000 7fef84d03700 1 -- 172.21.15.2:0/4171494720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614e6a79750 msgr2=0x5614e6a73540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:43.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.170+0000 7fef84d03700 1 --2- 172.21.15.2:0/4171494720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614e6a79750 0x5614e6a73540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:43.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.170+0000 7fef84d03700 1 -- 172.21.15.2:0/4171494720 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614e6a78e30 msgr2=0x5614e6a79210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:43.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.170+0000 7fef84d03700 1 --2- 172.21.15.2:0/4171494720 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614e6a78e30 0x5614e6a79210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:43.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.170+0000 7fef84d03700 1 -- 172.21.15.2:0/4171494720 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614e6911a60 con 0x5614e6a73b60 2024-05-14T17:00:43.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.170+0000 7fef84d03700 1 --2- 172.21.15.2:0/4171494720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614e6a73b60 0x5614e6a73fe0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fef780a6ea0 tx=0x7fef78009040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7684005c4e59320d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:43.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.171+0000 7fef667fc700 1 -- 172.21.15.2:0/4171494720 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7802a030 con 0x5614e6a73b60 2024-05-14T17:00:43.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.171+0000 7fef667fc700 1 -- 172.21.15.2:0/4171494720 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef780af1d0 con 0x5614e6a73b60 2024-05-14T17:00:43.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.171+0000 7fef667fc700 1 -- 172.21.15.2:0/4171494720 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef78003fd0 con 0x5614e6a73b60 2024-05-14T17:00:43.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.172+0000 7fef90ffcb80 1 -- 172.21.15.2:0/4171494720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614e6a73b60 msgr2=0x5614e6a73fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:43.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.172+0000 7fef90ffcb80 1 --2- 172.21.15.2:0/4171494720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614e6a73b60 0x5614e6a73fe0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fef780a6ea0 tx=0x7fef78009040 comp rx=0 tx=0).stop 2024-05-14T17:00:43.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.173+0000 7fef90ffcb80 1 -- 172.21.15.2:0/4171494720 shutdown_connections 2024-05-14T17:00:43.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.173+0000 7fef90ffcb80 1 --2- 172.21.15.2:0/4171494720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614e6a73b60 0x5614e6a73fe0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:43.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.173+0000 7fef90ffcb80 1 --2- 172.21.15.2:0/4171494720 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614e6a78e30 0x5614e6a79210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:43.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.173+0000 7fef90ffcb80 1 --2- 172.21.15.2:0/4171494720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614e6a79750 0x5614e6a73540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:43.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.173+0000 7fef90ffcb80 1 -- 172.21.15.2:0/4171494720 >> 172.21.15.2:0/4171494720 conn(0x5614e699e3e0 msgr2=0x5614e6a6dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:43.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.173+0000 7fef90ffcb80 1 -- 172.21.15.2:0/4171494720 shutdown_connections 2024-05-14T17:00:43.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.173+0000 7fef90ffcb80 1 -- 172.21.15.2:0/4171494720 wait complete. 2024-05-14T17:00:43.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.174+0000 7fef90ffcb80 1 -- start start 2024-05-14T17:00:43.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.174+0000 7fef90ffcb80 1 Processor -- start 2024-05-14T17:00:43.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.175+0000 7fef90ffcb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614e6a73b60 0x5614e6a7f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:43.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.175+0000 7fef90ffcb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614e6a78e30 0x5614e6a7c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:43.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.175+0000 7fef90ffcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614e6a79750 0x5614e6a7ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:43.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.175+0000 7fef90ffcb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5614e68c8130 con 0x5614e6a73b60 2024-05-14T17:00:43.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.175+0000 7fef90ffcb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5614e6963950 con 0x5614e6a79750 2024-05-14T17:00:43.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.175+0000 7fef90ffcb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5614e6a7ff50 con 0x5614e6a78e30 2024-05-14T17:00:43.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.175+0000 7fef90ffcb80 10 client.0 _ll_register_callbacks cb 0x5614e6958710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:00:43.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.175+0000 7fef7f7fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614e6a78e30 0x5614e6a7c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:43.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.175+0000 7fef84d03700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614e6a79750 0x5614e6a7ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:43.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.175+0000 7fef7f7fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614e6a78e30 0x5614e6a7c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:45300/0 (socket says 172.21.15.2:45300) 2024-05-14T17:00:43.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.175+0000 7fef7f7fe700 1 -- 172.21.15.2:0/3594165971 learned_addr learned my addr 172.21.15.2:0/3594165971 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:00:43.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.176+0000 7fef7ffff700 1 --2- 172.21.15.2:0/3594165971 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614e6a73b60 0x5614e6a7f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:44.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.176+0000 7fef7f7fe700 1 -- 172.21.15.2:0/3594165971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614e6a79750 msgr2=0x5614e6a7ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:44.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.176+0000 7fef7f7fe700 1 --2- 172.21.15.2:0/3594165971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614e6a79750 0x5614e6a7ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:44.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.176+0000 7fef7f7fe700 1 -- 172.21.15.2:0/3594165971 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614e6a73b60 msgr2=0x5614e6a7f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:44.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.176+0000 7fef7f7fe700 1 --2- 172.21.15.2:0/3594165971 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614e6a73b60 0x5614e6a7f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:44.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.176+0000 7fef7f7fe700 1 -- 172.21.15.2:0/3594165971 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614e6960820 con 0x5614e6a78e30 2024-05-14T17:00:44.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.176+0000 7fef7ffff700 1 --2- 172.21.15.2:0/3594165971 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614e6a73b60 0x5614e6a7f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:00:44.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.176+0000 7fef7f7fe700 1 --2- 172.21.15.2:0/3594165971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614e6a78e30 0x5614e6a7c900 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fef8007a670 tx=0x7fef7000e630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5eaef7d0e764fd22 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:44.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.177+0000 7fef5f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-14T17:00:44.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.177+0000 7fef5f7fe700 1 -- 172.21.15.2:0/3594165971 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef70004220 con 0x5614e6a78e30 2024-05-14T17:00:44.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.177+0000 7fef90ffcb80 1 -- 172.21.15.2:0/3594165971 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef780af1d0 con 0x5614e6a78e30 2024-05-14T17:00:44.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.177+0000 7fef5f7fe700 1 -- 172.21.15.2:0/3594165971 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fef700043c0 con 0x5614e6a78e30 2024-05-14T17:00:44.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.177+0000 7fef5f7fe700 1 -- 172.21.15.2:0/3594165971 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fef7000ee90 con 0x5614e6a78e30 2024-05-14T17:00:44.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.177+0000 7fef90ffcb80 10 client.25651 fetch_fsmap learned FSMap version 38 2024-05-14T17:00:44.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.177+0000 7fef90ffcb80 1 -- 172.21.15.2:0/3594165971 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5614e6a8a720 con 0x5614e6a78e30 2024-05-14T17:00:44.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.178+0000 7fef5f7fe700 1 -- 172.21.15.2:0/3594165971 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef780af1d0 con 0x5614e6a78e30 2024-05-14T17:00:44.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.179+0000 7fef5f7fe700 1 -- 172.21.15.2:0/3594165971 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 38) ==== 8342+0+0 (secure 0 0 0) 0x7fef70018840 con 0x5614e6a78e30 2024-05-14T17:00:44.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.179+0000 7fef90ffcb80 10 client.25651 fetch_fsmap finished waiting for FSMap version 38 2024-05-14T17:00:45.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.179+0000 7fef90ffcb80 10 client.25651 resolve_mds: validated gid 24479 aka daemon mds.b 2024-05-14T17:00:45.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.179+0000 7fef90ffcb80 20 client.25651 populate_metadata read hostname 'smithi002' 2024-05-14T17:00:45.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.179+0000 7fef90ffcb80 1 --2- 172.21.15.2:0/3594165971 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5614e6a8d060 0x5614e6a8f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:45.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.179+0000 7fef90ffcb80 4 client.25651 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:00:45.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.179+0000 7fef90ffcb80 1 -- 172.21.15.2:0/3594165971 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614e6a3f4f0 con 0x5614e6a8d060 2024-05-14T17:00:45.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.180+0000 7fef7ffff700 1 --2- 172.21.15.2:0/3594165971 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5614e6a8d060 0x5614e6a8f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:45.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.182+0000 7fef7ffff700 1 --2- 172.21.15.2:0/3594165971 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5614e6a8d060 0x5614e6a8f4e0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=976eed48897f1755 server_cookie=b566c83b082ed951 in_seq=0 out_seq=0 2024-05-14T17:00:45.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.182+0000 7fef5f7fe700 10 client.25651 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:00:45.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.183+0000 7fef5f7fe700 1 -- 172.21.15.2:0/3594165971 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5614e6a3f4f0 con 0x5614e6a8d060 2024-05-14T17:00:45.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.183+0000 7fef5f7fe700 10 client.25651 handle_command_reply: tid=0 2024-05-14T17:00:45.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.183+0000 7fef90ffcb80 1 client.25651 shutdown 2024-05-14T17:00:45.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.183+0000 7fef90ffcb80 1 -- 172.21.15.2:0/3594165971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614e6a78e30 msgr2=0x5614e6a7c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:45.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.183+0000 7fef90ffcb80 1 --2- 172.21.15.2:0/3594165971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614e6a78e30 0x5614e6a7c900 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fef8007a670 tx=0x7fef7000e630 comp rx=0 tx=0).stop 2024-05-14T17:00:45.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.183+0000 7fef90ffcb80 1 -- 172.21.15.2:0/3594165971 shutdown_connections 2024-05-14T17:00:45.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.183+0000 7fef90ffcb80 1 --2- 172.21.15.2:0/3594165971 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5614e6a8d060 0x5614e6a8f4e0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:45.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.183+0000 7fef90ffcb80 1 --2- 172.21.15.2:0/3594165971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614e6a79750 0x5614e6a7ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:45.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.183+0000 7fef90ffcb80 1 --2- 172.21.15.2:0/3594165971 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614e6a73b60 0x5614e6a7f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:45.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.183+0000 7fef90ffcb80 1 --2- 172.21.15.2:0/3594165971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614e6a78e30 0x5614e6a7c900 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:45.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.183+0000 7fef90ffcb80 1 -- 172.21.15.2:0/3594165971 >> 172.21.15.2:0/3594165971 conn(0x5614e699e3e0 msgr2=0x5614e6a68860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:45.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.184+0000 7fef5f7fe700 0 client.25651 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:00:45.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.184+0000 7fef90ffcb80 1 -- 172.21.15.2:0/3594165971 shutdown_connections 2024-05-14T17:00:45.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.184+0000 7fef90ffcb80 1 -- 172.21.15.2:0/3594165971 wait complete. 2024-05-14T17:00:45.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.184+0000 7fef90ffcb80 20 client.25651 trim_cache size 0 max 16384 2024-05-14T17:00:45.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.216+0000 7fef90ffcb80 1 Processor -- start 2024-05-14T17:00:45.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.216+0000 7fef90ffcb80 1 -- start start 2024-05-14T17:00:45.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.217+0000 7fef90ffcb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614e6a73b60 0x5614e6a68970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:45.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.217+0000 7fef90ffcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614e6a78e30 0x5614e6a88b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:45.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.217+0000 7fef90ffcb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614e6a79750 0x5614e6a7a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:45.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.217+0000 7fef7ffff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614e6a73b60 0x5614e6a68970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:45.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.217+0000 7fef7ffff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614e6a73b60 0x5614e6a68970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:45306/0 (socket says 172.21.15.2:45306) 2024-05-14T17:00:45.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.218+0000 7fef7f7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614e6a78e30 0x5614e6a88b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:45.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.218+0000 7fef7f7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614e6a78e30 0x5614e6a88b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:50298/0 (socket says 172.21.15.2:50298) 2024-05-14T17:00:45.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.218+0000 7fef84d03700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614e6a79750 0x5614e6a7a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:45.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.218+0000 7fef84d03700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614e6a79750 0x5614e6a7a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59496/0 (socket says 172.21.15.2:59496) 2024-05-14T17:00:45.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.218+0000 7fef7ffff700 1 -- 172.21.15.2:0/2160900090 learned_addr learned my addr 172.21.15.2:0/2160900090 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:00:45.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.218+0000 7fef90ffcb80 1 -- 172.21.15.2:0/2160900090 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fef801beee0 con 0x5614e6a79750 2024-05-14T17:00:45.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.218+0000 7fef90ffcb80 1 -- 172.21.15.2:0/2160900090 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fef801bc740 con 0x5614e6a78e30 2024-05-14T17:00:45.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.218+0000 7fef90ffcb80 1 -- 172.21.15.2:0/2160900090 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fef801ba640 con 0x5614e6a73b60 2024-05-14T17:00:45.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.219+0000 7fef84d03700 1 -- 172.21.15.2:0/2160900090 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614e6a73b60 msgr2=0x5614e6a68970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:45.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.219+0000 7fef84d03700 1 --2- 172.21.15.2:0/2160900090 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614e6a73b60 0x5614e6a68970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:45.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.219+0000 7fef84d03700 1 -- 172.21.15.2:0/2160900090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614e6a78e30 msgr2=0x5614e6a88b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:45.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.219+0000 7fef84d03700 1 --2- 172.21.15.2:0/2160900090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614e6a78e30 0x5614e6a88b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:45.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.219+0000 7fef84d03700 1 -- 172.21.15.2:0/2160900090 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614e6a7ae70 con 0x5614e6a79750 2024-05-14T17:00:45.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.219+0000 7fef7f7fe700 1 --2- 172.21.15.2:0/2160900090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614e6a78e30 0x5614e6a88b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:00:45.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.219+0000 7fef7ffff700 1 --2- 172.21.15.2:0/2160900090 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614e6a73b60 0x5614e6a68970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:00:45.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.219+0000 7fef84d03700 1 --2- 172.21.15.2:0/2160900090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614e6a79750 0x5614e6a7a3c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x5614e6a7fe40 tx=0x7fef780cd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154c6477bf7eca30 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:45.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.219+0000 7fef65ffb700 1 -- 172.21.15.2:0/2160900090 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7802a040 con 0x5614e6a79750 2024-05-14T17:00:45.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.220+0000 7fef65ffb700 1 -- 172.21.15.2:0/2160900090 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef780b3bf0 con 0x5614e6a79750 2024-05-14T17:00:45.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.220+0000 7fef65ffb700 1 -- 172.21.15.2:0/2160900090 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef780a70b0 con 0x5614e6a79750 2024-05-14T17:00:45.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.221+0000 7fef90ffcb80 1 -- 172.21.15.2:0/2160900090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614e6a79750 msgr2=0x5614e6a7a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:45.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.221+0000 7fef90ffcb80 1 --2- 172.21.15.2:0/2160900090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614e6a79750 0x5614e6a7a3c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x5614e6a7fe40 tx=0x7fef780cd040 comp rx=0 tx=0).stop 2024-05-14T17:00:45.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.223+0000 7fef90ffcb80 1 -- 172.21.15.2:0/2160900090 shutdown_connections 2024-05-14T17:00:45.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.223+0000 7fef90ffcb80 1 --2- 172.21.15.2:0/2160900090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614e6a79750 0x5614e6a7a3c0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:45.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.223+0000 7fef90ffcb80 1 --2- 172.21.15.2:0/2160900090 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614e6a73b60 0x5614e6a68970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:45.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.223+0000 7fef90ffcb80 1 --2- 172.21.15.2:0/2160900090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614e6a78e30 0x5614e6a88b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:45.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.223+0000 7fef90ffcb80 1 -- 172.21.15.2:0/2160900090 >> 172.21.15.2:0/2160900090 conn(0x5614e699e3e0 msgr2=0x5614e6a780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:45.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.224+0000 7fef90ffcb80 1 -- 172.21.15.2:0/2160900090 shutdown_connections 2024-05-14T17:00:45.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.224+0000 7fef90ffcb80 1 -- 172.21.15.2:0/2160900090 wait complete. 2024-05-14T17:00:45.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.225+0000 7fef90ffcb80 1 -- start start 2024-05-14T17:00:45.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.225+0000 7fef90ffcb80 1 Processor -- start 2024-05-14T17:00:45.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.226+0000 7fef90ffcb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614e6a73b60 0x5614e6a851c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:45.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.226+0000 7fef90ffcb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614e6a78e30 0x5614e6a81ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:45.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.226+0000 7fef90ffcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614e6a79750 0x5614e6a82410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:45.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.226+0000 7fef90ffcb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fef801bac00 con 0x5614e6a78e30 2024-05-14T17:00:45.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.226+0000 7fef90ffcb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fef801bc070 con 0x5614e6a79750 2024-05-14T17:00:45.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.226+0000 7fef90ffcb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fef801bb2d0 con 0x5614e6a73b60 2024-05-14T17:00:45.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.226+0000 7fef90ffcb80 10 client.0 _ll_register_callbacks cb 0x5614e6a6ff40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:00:45.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.230+0000 7fef7ffff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614e6a73b60 0x5614e6a851c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:45.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.231+0000 7fef84d03700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614e6a79750 0x5614e6a82410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:45.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.232+0000 7fef84d03700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614e6a79750 0x5614e6a82410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:50310/0 (socket says 172.21.15.2:50310) 2024-05-14T17:00:45.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.232+0000 7fef84d03700 1 -- 172.21.15.2:0/482143704 learned_addr learned my addr 172.21.15.2:0/482143704 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:00:45.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.232+0000 7fef7ffff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614e6a73b60 0x5614e6a851c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:45318/0 (socket says 172.21.15.2:45318) 2024-05-14T17:00:45.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.232+0000 7fef7f7fe700 1 --2- 172.21.15.2:0/482143704 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614e6a78e30 0x5614e6a81ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:45.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.235+0000 7fef7ffff700 1 -- 172.21.15.2:0/482143704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614e6a79750 msgr2=0x5614e6a82410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:45.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.235+0000 7fef7ffff700 1 --2- 172.21.15.2:0/482143704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614e6a79750 0x5614e6a82410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:45.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.235+0000 7fef7ffff700 1 -- 172.21.15.2:0/482143704 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614e6a78e30 msgr2=0x5614e6a81ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:45.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.235+0000 7fef7ffff700 1 --2- 172.21.15.2:0/482143704 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614e6a78e30 0x5614e6a81ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:45.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.235+0000 7fef7ffff700 1 -- 172.21.15.2:0/482143704 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614e6a82950 con 0x5614e6a73b60 2024-05-14T17:00:45.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.236+0000 7fef7ffff700 1 --2- 172.21.15.2:0/482143704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614e6a73b60 0x5614e6a851c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fef7400a010 tx=0x7fef740047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad9b4acf262e4c1a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:45.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.236+0000 7fef5f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-14T17:00:45.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.238+0000 7fef5f7fe700 1 -- 172.21.15.2:0/482143704 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef74003d20 con 0x5614e6a73b60 2024-05-14T17:00:45.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.238+0000 7fef5f7fe700 1 -- 172.21.15.2:0/482143704 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fef74025290 con 0x5614e6a73b60 2024-05-14T17:00:45.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.240+0000 7fef90ffcb80 1 -- 172.21.15.2:0/482143704 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef780b3bf0 con 0x5614e6a73b60 2024-05-14T17:00:45.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.243+0000 7fef5f7fe700 1 -- 172.21.15.2:0/482143704 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fef74025490 con 0x5614e6a73b60 2024-05-14T17:00:45.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.243+0000 7fef90ffcb80 10 client.25657 fetch_fsmap learned FSMap version 38 2024-05-14T17:00:45.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.243+0000 7fef90ffcb80 1 -- 172.21.15.2:0/482143704 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5614e6af8720 con 0x5614e6a73b60 2024-05-14T17:00:45.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.247+0000 7fef5f7fe700 1 -- 172.21.15.2:0/482143704 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef780b3bf0 con 0x5614e6a73b60 2024-05-14T17:00:46.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.247+0000 7fef5f7fe700 1 -- 172.21.15.2:0/482143704 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 38) ==== 8342+0+0 (secure 0 0 0) 0x7fef74037b40 con 0x5614e6a73b60 2024-05-14T17:00:46.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.248+0000 7fef90ffcb80 10 client.25657 fetch_fsmap finished waiting for FSMap version 38 2024-05-14T17:00:46.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.248+0000 7fef90ffcb80 10 client.25657 resolve_mds: validated gid 24479 aka daemon mds.b 2024-05-14T17:00:46.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.248+0000 7fef90ffcb80 20 client.25657 populate_metadata read hostname 'smithi002' 2024-05-14T17:00:46.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.248+0000 7fef90ffcb80 1 --2- 172.21.15.2:0/482143704 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5614e6af8dd0 0x5614e6afb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:46.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.248+0000 7fef90ffcb80 4 client.25657 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.7, "await_for": 45.0}] 2024-05-14T17:00:46.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.248+0000 7fef90ffcb80 1 -- 172.21.15.2:0/482143704 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.7, "await_for": 45.0}) -- 0x5614e69d3d70 con 0x5614e6af8dd0 2024-05-14T17:00:46.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.249+0000 7fef7f7fe700 1 --2- 172.21.15.2:0/482143704 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5614e6af8dd0 0x5614e6afb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:46.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.250+0000 7fef7f7fe700 1 --2- 172.21.15.2:0/482143704 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5614e6af8dd0 0x5614e6afb1f0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f68a314bfb45bf2 server_cookie=80ac8c9d7dd70a5 in_seq=0 out_seq=0 2024-05-14T17:00:46.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.252+0000 7fef5f7fe700 10 client.25657 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:00:46.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.584+0000 7fef5f7fe700 1 -- 172.21.15.2:0/482143704 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5614e69d3d70 con 0x5614e6af8dd0 2024-05-14T17:00:46.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.584+0000 7fef5f7fe700 10 client.25657 handle_command_reply: tid=0 2024-05-14T17:00:46.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.584+0000 7fef90ffcb80 1 client.25657 shutdown 2024-05-14T17:00:46.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.584+0000 7fef90ffcb80 1 -- 172.21.15.2:0/482143704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614e6a73b60 msgr2=0x5614e6a851c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:46.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.584+0000 7fef90ffcb80 1 --2- 172.21.15.2:0/482143704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614e6a73b60 0x5614e6a851c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fef7400a010 tx=0x7fef740047c0 comp rx=0 tx=0).stop 2024-05-14T17:00:46.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.584+0000 7fef90ffcb80 1 -- 172.21.15.2:0/482143704 shutdown_connections 2024-05-14T17:00:46.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.584+0000 7fef90ffcb80 1 --2- 172.21.15.2:0/482143704 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5614e6af8dd0 0x5614e6afb1f0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:46.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.584+0000 7fef90ffcb80 1 --2- 172.21.15.2:0/482143704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614e6a79750 0x5614e6a82410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:46.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.584+0000 7fef90ffcb80 1 --2- 172.21.15.2:0/482143704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5614e6a73b60 0x5614e6a851c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:46.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.584+0000 7fef90ffcb80 1 --2- 172.21.15.2:0/482143704 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614e6a78e30 0x5614e6a81ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:46.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.584+0000 7fef5f7fe700 0 client.25657 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:00:46.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.584+0000 7fef90ffcb80 1 -- 172.21.15.2:0/482143704 >> 172.21.15.2:0/482143704 conn(0x5614e699e3e0 msgr2=0x5614e6a7a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:46.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.584+0000 7fef90ffcb80 1 -- 172.21.15.2:0/482143704 shutdown_connections 2024-05-14T17:00:46.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.584+0000 7fef90ffcb80 1 -- 172.21.15.2:0/482143704 wait complete. 2024-05-14T17:00:46.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.584+0000 7fef90ffcb80 20 client.25657 trim_cache size 0 max 16384 2024-05-14T17:00:46.266 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T17:00:46.272 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 38, 2024-05-14T17:00:46.288 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24479, 2024-05-14T17:00:46.295 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 18, 2024-05-14T17:00:46.305 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-05-14T17:00:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: "bf03fc11": { 2024-05-14T17:00:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 18, 2024-05-14T17:00:46.353 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-05-14T17:00:46.362 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-05-14T17:00:46.386 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-05-14T17:00:46.392 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-05-14T17:00:46.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:00:46.405 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-05-14T17:00:46.412 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 133.7, 2024-05-14T17:00:46.418 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-05-14T17:00:46.425 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-05-14T17:00:46.434 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-05-14T17:00:46.440 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-05-14T17:00:46.447 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-05-14T17:00:46.471 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-05-14T17:00:46.479 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:00:46.485 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:00:46.491 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:00:46.498 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:00:46.534 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:00:46.541 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T17:00:46.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.585+0000 7fef86766700 1 -- 172.21.15.2:0/192915036 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fef680b2be0 msgr2=0x7fef680b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:46.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.585+0000 7fef86766700 1 --2- 172.21.15.2:0/192915036 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fef680b2be0 0x7fef680b50a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fef74009c60 tx=0x7fef74009260 comp rx=0 tx=0).stop 2024-05-14T17:00:46.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.585+0000 7fef86766700 1 -- 172.21.15.2:0/192915036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef801089d0 msgr2=0x7fef8006eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:46.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.585+0000 7fef86766700 1 --2- 172.21.15.2:0/192915036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef801089d0 0x7fef8006eaf0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fef780062a0 tx=0x7fef7800bef0 comp rx=0 tx=0).stop 2024-05-14T17:00:46.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.585+0000 7fef86766700 1 -- 172.21.15.2:0/192915036 shutdown_connections 2024-05-14T17:00:46.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.585+0000 7fef86766700 1 --2- 172.21.15.2:0/192915036 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fef680b2be0 0x7fef680b50a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:46.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.585+0000 7fef86766700 1 --2- 172.21.15.2:0/192915036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef801089d0 0x7fef8006eaf0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:46.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.585+0000 7fef86766700 1 --2- 172.21.15.2:0/192915036 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef80105fd0 0x7fef800749d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:46.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.585+0000 7fef86766700 1 --2- 172.21.15.2:0/192915036 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef80108560 0x7fef80074f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:46.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.585+0000 7fef86766700 1 -- 172.21.15.2:0/192915036 >> 172.21.15.2:0/192915036 conn(0x7fef800793f0 msgr2=0x7fef800ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:46.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.585+0000 7fef86766700 1 -- 172.21.15.2:0/192915036 shutdown_connections 2024-05-14T17:00:46.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:42.585+0000 7fef86766700 1 -- 172.21.15.2:0/192915036 wait complete. 2024-05-14T17:00:46.648 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:00:42 smithi002 ceph-mon[31538]: pgmap v183: 97 pgs: 97 active+clean; 3.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.2 MiB/s wr, 492 op/s 2024-05-14T17:00:46.656 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:00:44 smithi002 ceph-mon[31538]: pgmap v184: 97 pgs: 97 active+clean; 3.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 9.8 MiB/s wr, 797 op/s 2024-05-14T17:00:46.664 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'bf03fc11', quiesce took 11.7 seconds. Will release after: 2.0 2024-05-14T17:00:46.695 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:00:44 smithi151 ceph-mon[36066]: pgmap v184: 97 pgs: 97 active+clean; 3.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 9.8 MiB/s wr, 797 op/s 2024-05-14T17:00:46.703 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:00:44 smithi123 ceph-mon[35971]: pgmap v184: 97 pgs: 97 active+clean; 3.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 9.8 MiB/s wr, 797 op/s 2024-05-14T17:00:46.715 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:00:46 smithi123 ceph-mon[35971]: pgmap v185: 97 pgs: 97 active+clean; 3.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.3 MiB/s wr, 609 op/s 2024-05-14T17:00:46.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:00:46 smithi002 ceph-mon[31538]: pgmap v185: 97 pgs: 97 active+clean; 3.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.3 MiB/s wr, 609 op/s 2024-05-14T17:00:46.752 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 330.0 s, 314.6 tps, lat 31.204 ms stddev 32.187 2024-05-14T17:00:46.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:00:46 smithi151 ceph-mon[36066]: pgmap v185: 97 pgs: 97 active+clean; 3.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.3 MiB/s wr, 609 op/s 2024-05-14T17:00:48.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:00:48 smithi123 ceph-mon[35971]: pgmap v186: 97 pgs: 97 active+clean; 3.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 7.7 MiB/s wr, 661 op/s 2024-05-14T17:00:48.695 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:00:48.737 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:00:48 smithi002 ceph-mon[31538]: pgmap v186: 97 pgs: 97 active+clean; 3.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 7.7 MiB/s wr, 661 op/s 2024-05-14T17:00:48.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:00:48 smithi151 ceph-mon[36066]: pgmap v186: 97 pgs: 97 active+clean; 3.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 7.7 MiB/s wr, 661 op/s 2024-05-14T17:00:48.977 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:00:50.651 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:00:50 smithi002 ceph-mon[31538]: pgmap v187: 97 pgs: 97 active+clean; 3.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.8 MiB/s wr, 581 op/s 2024-05-14T17:00:50.692 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:00:50 smithi123 ceph-mon[35971]: pgmap v187: 97 pgs: 97 active+clean; 3.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.8 MiB/s wr, 581 op/s 2024-05-14T17:00:50.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.757+0000 7fbd4c9d1700 1 -- 172.21.15.2:0/4047121584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd48108910 msgr2=0x7fbd48110490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:50.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.757+0000 7fbd4c9d1700 1 --2- 172.21.15.2:0/4047121584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd48108910 0x7fbd48110490 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fbd4000b0d0 tx=0x7fbd4000b3e0 comp rx=0 tx=0).stop 2024-05-14T17:00:50.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.757+0000 7fbd4c9d1700 1 -- 172.21.15.2:0/4047121584 shutdown_connections 2024-05-14T17:00:50.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.757+0000 7fbd4c9d1700 1 --2- 172.21.15.2:0/4047121584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd48108910 0x7fbd48110490 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:50.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.757+0000 7fbd4c9d1700 1 --2- 172.21.15.2:0/4047121584 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd48103820 0x7fbd48103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:50.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.757+0000 7fbd4c9d1700 1 --2- 172.21.15.2:0/4047121584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd481041d0 0x7fbd481081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:50.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.757+0000 7fbd4c9d1700 1 -- 172.21.15.2:0/4047121584 >> 172.21.15.2:0/4047121584 conn(0x7fbd4807a6b0 msgr2=0x7fbd4807aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:50.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.757+0000 7fbd4c9d1700 1 -- 172.21.15.2:0/4047121584 shutdown_connections 2024-05-14T17:00:50.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.757+0000 7fbd4c9d1700 1 -- 172.21.15.2:0/4047121584 wait complete. 2024-05-14T17:00:50.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.758+0000 7fbd4c9d1700 1 Processor -- start 2024-05-14T17:00:50.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.758+0000 7fbd4c9d1700 1 -- start start 2024-05-14T17:00:50.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.758+0000 7fbd4c9d1700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd48103820 0x7fbd4819f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:50.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.758+0000 7fbd4c9d1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd481041d0 0x7fbd4819fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:50.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.758+0000 7fbd4c9d1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd48108910 0x7fbd48199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:50.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.758+0000 7fbd4c9d1700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbd481136e0 con 0x7fbd481041d0 2024-05-14T17:00:50.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.758+0000 7fbd4c9d1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbd48113560 con 0x7fbd48108910 2024-05-14T17:00:50.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.758+0000 7fbd4c9d1700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbd48113860 con 0x7fbd48103820 2024-05-14T17:00:50.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.758+0000 7fbd45d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd481041d0 0x7fbd4819fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:50.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.758+0000 7fbd45d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd481041d0 0x7fbd4819fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55740/0 (socket says 172.21.15.2:55740) 2024-05-14T17:00:50.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.758+0000 7fbd45d9b700 1 -- 172.21.15.2:0/277314630 learned_addr learned my addr 172.21.15.2:0/277314630 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:00:50.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.758+0000 7fbd46d9d700 1 --2- 172.21.15.2:0/277314630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd48108910 0x7fbd48199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:50.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.758+0000 7fbd4659c700 1 --2- 172.21.15.2:0/277314630 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd48103820 0x7fbd4819f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:50.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.759+0000 7fbd45d9b700 1 -- 172.21.15.2:0/277314630 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd48103820 msgr2=0x7fbd4819f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:50.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.759+0000 7fbd45d9b700 1 --2- 172.21.15.2:0/277314630 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd48103820 0x7fbd4819f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:50.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.759+0000 7fbd45d9b700 1 -- 172.21.15.2:0/277314630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd48108910 msgr2=0x7fbd48199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:51.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.759+0000 7fbd45d9b700 1 --2- 172.21.15.2:0/277314630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd48108910 0x7fbd48199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:51.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.759+0000 7fbd45d9b700 1 -- 172.21.15.2:0/277314630 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd4819a0a0 con 0x7fbd481041d0 2024-05-14T17:00:51.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.759+0000 7fbd4659c700 1 --2- 172.21.15.2:0/277314630 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd48103820 0x7fbd4819f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:00:51.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.759+0000 7fbd45d9b700 1 --2- 172.21.15.2:0/277314630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd481041d0 0x7fbd4819fc70 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c00d980 tx=0x7fbd3c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:51.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.759+0000 7fbd377fe700 1 -- 172.21.15.2:0/277314630 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c016070 con 0x7fbd481041d0 2024-05-14T17:00:51.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.759+0000 7fbd377fe700 1 -- 172.21.15.2:0/277314630 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd3c004830 con 0x7fbd481041d0 2024-05-14T17:00:51.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.759+0000 7fbd377fe700 1 -- 172.21.15.2:0/277314630 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c0054c0 con 0x7fbd481041d0 2024-05-14T17:00:51.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.759+0000 7fbd4c9d1700 1 -- 172.21.15.2:0/277314630 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd4819a390 con 0x7fbd481041d0 2024-05-14T17:00:51.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.759+0000 7fbd4c9d1700 1 -- 172.21.15.2:0/277314630 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd481a6600 con 0x7fbd481041d0 2024-05-14T17:00:51.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.760+0000 7fbd4c9d1700 1 -- 172.21.15.2:0/277314630 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd4806a230 con 0x7fbd481041d0 2024-05-14T17:00:51.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.761+0000 7fbd377fe700 1 -- 172.21.15.2:0/277314630 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fbd3c004220 con 0x7fbd481041d0 2024-05-14T17:00:51.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.762+0000 7fbd377fe700 1 --2- 172.21.15.2:0/277314630 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fbd300b2a70 0x7fbd300b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:51.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.763+0000 7fbd4659c700 1 --2- 172.21.15.2:0/277314630 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fbd300b2a70 0x7fbd300b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:51.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.763+0000 7fbd377fe700 1 -- 172.21.15.2:0/277314630 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7fbd3c0e28d0 con 0x7fbd481041d0 2024-05-14T17:00:51.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.763+0000 7fbd4659c700 1 --2- 172.21.15.2:0/277314630 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fbd300b2a70 0x7fbd300b4f30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbd38005950 tx=0x7fbd380058e0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:51.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:50.766+0000 7fbd377fe700 1 -- 172.21.15.2:0/277314630 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbd3c01f020 con 0x7fbd481041d0 2024-05-14T17:00:51.178 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:00:50 smithi151 ceph-mon[36066]: pgmap v187: 97 pgs: 97 active+clean; 3.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.8 MiB/s wr, 581 op/s 2024-05-14T17:00:51.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:51.027+0000 7fbd4c9d1700 1 -- 172.21.15.2:0/277314630 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd4802e560 con 0x7fbd481041d0 2024-05-14T17:00:51.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:51.028+0000 7fbd377fe700 1 -- 172.21.15.2:0/277314630 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11851 (secure 0 0 0) 0x7fbd3c0aa280 con 0x7fbd481041d0 2024-05-14T17:00:51.197 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:00:51.203 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2275696923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2275696923},{"type":"v1","addr":"172.21.15.2:6835","nonce":2275696923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/716718761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":716718761},{"type":"v1","addr":"172.21.15.2:6839","nonce":716718761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/3644815179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3644815179},{"type":"v1","addr":"172.21.15.2:6841","nonce":3644815179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1880336731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1880336731},{"type":"v1","addr":"172.21.15.151:6833","nonce":1880336731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2415605548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2415605548},{"type":"v1","addr":"172.21.15.2:6837","nonce":2415605548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2635167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2635167},{"type":"v1","addr":"172.21.15.123:6833","nonce":2635167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:51:34.106919+0000","modified":"2024-05-14T17:00:23.454968+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24455,"mds_2":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24415":{"gid":24415,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/762044339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":762044339},{"type":"v1","addr":"172.21.15.151:6835","nonce":762044339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/132043151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":132043151},{"type":"v1","addr":"172.21.15.151:6837","nonce":132043151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/4107834375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":4107834375},{"type":"v1","addr":"172.21.15.151:6839","nonce":4107834375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.123:6835/1198670013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1198670013},{"type":"v1","addr":"172.21.15.123:6835","nonce":1198670013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/333426645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":333426645},{"type":"v1","addr":"172.21.15.123:6837","nonce":333426645}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/580275877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":580275877},{"type":"v1","addr":"172.21.15.123:6839","nonce":580275877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24431,24455]},"id":1}]} 2024-05-14T17:00:51.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:51.031+0000 7fbd4c9d1700 1 -- 172.21.15.2:0/277314630 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fbd300b2a70 msgr2=0x7fbd300b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:51.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:51.031+0000 7fbd4c9d1700 1 --2- 172.21.15.2:0/277314630 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fbd300b2a70 0x7fbd300b4f30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbd38005950 tx=0x7fbd380058e0 comp rx=0 tx=0).stop 2024-05-14T17:00:51.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:51.031+0000 7fbd4c9d1700 1 -- 172.21.15.2:0/277314630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd481041d0 msgr2=0x7fbd4819fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:51.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:51.031+0000 7fbd4c9d1700 1 --2- 172.21.15.2:0/277314630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd481041d0 0x7fbd4819fc70 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c00d980 tx=0x7fbd3c00dc90 comp rx=0 tx=0).stop 2024-05-14T17:00:51.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:51.031+0000 7fbd4c9d1700 1 -- 172.21.15.2:0/277314630 shutdown_connections 2024-05-14T17:00:51.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:51.031+0000 7fbd4c9d1700 1 --2- 172.21.15.2:0/277314630 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fbd300b2a70 0x7fbd300b4f30 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:51.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:51.031+0000 7fbd4c9d1700 1 --2- 172.21.15.2:0/277314630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd48108910 0x7fbd48199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:51.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:51.031+0000 7fbd4c9d1700 1 --2- 172.21.15.2:0/277314630 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd48103820 0x7fbd4819f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:51.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:51.031+0000 7fbd4c9d1700 1 --2- 172.21.15.2:0/277314630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd481041d0 0x7fbd4819fc70 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:51.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:51.031+0000 7fbd4c9d1700 1 -- 172.21.15.2:0/277314630 >> 172.21.15.2:0/277314630 conn(0x7fbd4807a6b0 msgr2=0x7fbd48101610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:51.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:51.031+0000 7fbd4c9d1700 1 -- 172.21.15.2:0/277314630 shutdown_connections 2024-05-14T17:00:51.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:51.031+0000 7fbd4c9d1700 1 -- 172.21.15.2:0/277314630 wait complete. 2024-05-14T17:00:51.326 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 38 2024-05-14T17:00:51.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:00:51 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/277314630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:00:51.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:00:51 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/277314630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:00:51.755 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 335.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:00:51.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:00:51 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/277314630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:00:51.907 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id bf03fc11 --release --await' 2024-05-14T17:00:51.944 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id bf03fc11 --release --await 2024-05-14T17:00:52.216 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:00:52.590 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:00:52 smithi002 ceph-mon[31538]: pgmap v188: 97 pgs: 97 active+clean; 3.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 4.3 MiB/s wr, 386 op/s 2024-05-14T17:00:52.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:00:52 smithi123 ceph-mon[35971]: pgmap v188: 97 pgs: 97 active+clean; 3.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 4.3 MiB/s wr, 386 op/s 2024-05-14T17:00:52.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:00:52 smithi151 ceph-mon[36066]: pgmap v188: 97 pgs: 97 active+clean; 3.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 4.3 MiB/s wr, 386 op/s 2024-05-14T17:00:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.568+0000 7f15333da640 1 -- 172.21.15.2:0/1867934687 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f152c074e80 msgr2=0x7f152c075260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.568+0000 7f15333da640 1 --2- 172.21.15.2:0/1867934687 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f152c074e80 0x7f152c075260 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1528009900 tx=0x7f152802a480 comp rx=0 tx=0).stop 2024-05-14T17:00:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.568+0000 7f15333da640 1 -- 172.21.15.2:0/1867934687 shutdown_connections 2024-05-14T17:00:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.568+0000 7f15333da640 1 --2- 172.21.15.2:0/1867934687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f152c10f270 0x7f152c1116c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:53.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.568+0000 7f15333da640 1 --2- 172.21.15.2:0/1867934687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f152c0757a0 0x7f152c10ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.568+0000 7f15333da640 1 --2- 172.21.15.2:0/1867934687 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f152c074e80 0x7f152c075260 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.568+0000 7f15333da640 1 -- 172.21.15.2:0/1867934687 >> 172.21.15.2:0/1867934687 conn(0x7f152c0fe500 msgr2=0x7f152c100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.568+0000 7f15333da640 1 -- 172.21.15.2:0/1867934687 shutdown_connections 2024-05-14T17:00:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.568+0000 7f15333da640 1 -- 172.21.15.2:0/1867934687 wait complete. 2024-05-14T17:00:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.569+0000 7f15333da640 1 Processor -- start 2024-05-14T17:00:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.569+0000 7f15333da640 1 -- start start 2024-05-14T17:00:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.569+0000 7f15333da640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f152c074e80 0x7f152c19b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:53.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.569+0000 7f15333da640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f152c0757a0 0x7f152c19b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:53.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.569+0000 7f15333da640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f152c10f270 0x7f152c19fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.569+0000 7f15333da640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f152c114ec0 con 0x7f152c074e80 2024-05-14T17:00:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.569+0000 7f15333da640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f152c114d40 con 0x7f152c10f270 2024-05-14T17:00:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.569+0000 7f15333da640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f152c115040 con 0x7f152c0757a0 2024-05-14T17:00:53.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.569+0000 7f1531176640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f152c074e80 0x7f152c19b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.569+0000 7f1531176640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f152c074e80 0x7f152c19b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55750/0 (socket says 172.21.15.2:55750) 2024-05-14T17:00:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.569+0000 7f1531176640 1 -- 172.21.15.2:0/3218439981 learned_addr learned my addr 172.21.15.2:0/3218439981 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:00:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.569+0000 7f1531977640 1 --2- 172.21.15.2:0/3218439981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f152c10f270 0x7f152c19fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:53.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.569+0000 7f1531176640 1 -- 172.21.15.2:0/3218439981 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f152c0757a0 msgr2=0x7f152c19b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:53.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.569+0000 7f1530975640 1 --2- 172.21.15.2:0/3218439981 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f152c0757a0 0x7f152c19b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.569+0000 7f1531176640 1 --2- 172.21.15.2:0/3218439981 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f152c0757a0 0x7f152c19b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.569+0000 7f1531176640 1 -- 172.21.15.2:0/3218439981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f152c10f270 msgr2=0x7f152c19fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:53.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.569+0000 7f1531176640 1 --2- 172.21.15.2:0/3218439981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f152c10f270 0x7f152c19fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.569+0000 7f1531176640 1 -- 172.21.15.2:0/3218439981 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f152c1a0320 con 0x7f152c074e80 2024-05-14T17:00:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.569+0000 7f1531977640 1 --2- 172.21.15.2:0/3218439981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f152c10f270 0x7f152c19fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:00:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.570+0000 7f1531176640 1 --2- 172.21.15.2:0/3218439981 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f152c074e80 0x7f152c19b380 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f1528006010 tx=0x7f15280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:53.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.570+0000 7f15227fc640 1 -- 172.21.15.2:0/3218439981 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f152803f030 con 0x7f152c074e80 2024-05-14T17:00:53.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.570+0000 7f15333da640 1 -- 172.21.15.2:0/3218439981 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f152c1a05b0 con 0x7f152c074e80 2024-05-14T17:00:53.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.570+0000 7f15227fc640 1 -- 172.21.15.2:0/3218439981 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1528003800 con 0x7f152c074e80 2024-05-14T17:00:53.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.570+0000 7f15333da640 1 -- 172.21.15.2:0/3218439981 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f152c1a7ea0 con 0x7f152c074e80 2024-05-14T17:00:53.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.570+0000 7f15227fc640 1 -- 172.21.15.2:0/3218439981 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1528049060 con 0x7f152c074e80 2024-05-14T17:00:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.573+0000 7f15227fc640 1 -- 172.21.15.2:0/3218439981 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f1528003cb0 con 0x7f152c074e80 2024-05-14T17:00:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.573+0000 7f15361cf740 1 Processor -- start 2024-05-14T17:00:53.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.573+0000 7f15361cf740 1 -- start start 2024-05-14T17:00:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.573+0000 7f15361cf740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef6fc918f0 0x55ef6fc91cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.573+0000 7f15361cf740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef6fc92210 0x55ef6fc9c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:53.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.573+0000 7f15361cf740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef6fc9cad0 0x55ef6fc9eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.573+0000 7f15361cf740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ef6fc04c30 con 0x55ef6fc918f0 2024-05-14T17:00:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.573+0000 7f15361cf740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ef6fbf47d0 con 0x55ef6fc9cad0 2024-05-14T17:00:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.573+0000 7f15361cf740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ef6fbf8810 con 0x55ef6fc92210 2024-05-14T17:00:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.573+0000 7f1531977640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef6fc918f0 0x55ef6fc91cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:54.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.573+0000 7f1531977640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef6fc918f0 0x55ef6fc91cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55762/0 (socket says 172.21.15.2:55762) 2024-05-14T17:00:54.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.573+0000 7f1531977640 1 -- 172.21.15.2:0/3881281965 learned_addr learned my addr 172.21.15.2:0/3881281965 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:00:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.573+0000 7f1530975640 1 --2- 172.21.15.2:0/3881281965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef6fc9cad0 0x55ef6fc9eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:54.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.573+0000 7f1531176640 1 --2- 172.21.15.2:0/3881281965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef6fc92210 0x55ef6fc9c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:54.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.574+0000 7f15227fc640 1 --2- 172.21.15.2:0/3218439981 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f15180b2a10 0x7f15180b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:54.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.574+0000 7f1531977640 1 -- 172.21.15.2:0/3881281965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef6fc92210 msgr2=0x55ef6fc9c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.574+0000 7f1531977640 1 --2- 172.21.15.2:0/3881281965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef6fc92210 0x55ef6fc9c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:54.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.574+0000 7f1531977640 1 -- 172.21.15.2:0/3881281965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef6fc9cad0 msgr2=0x55ef6fc9eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.574+0000 7f1531977640 1 --2- 172.21.15.2:0/3881281965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef6fc9cad0 0x55ef6fc9eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.574+0000 7f1531977640 1 -- 172.21.15.2:0/3881281965 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef6f909210 con 0x55ef6fc918f0 2024-05-14T17:00:54.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.574+0000 7f15227fc640 1 -- 172.21.15.2:0/3218439981 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f1528103360 con 0x7f152c074e80 2024-05-14T17:00:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.574+0000 7f1531977640 1 --2- 172.21.15.2:0/3881281965 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef6fc918f0 0x55ef6fc91cd0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f1524007e60 tx=0x7f152400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a08e594cc0daa77e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.574+0000 7f1531977640 1 --2- 172.21.15.2:0/3218439981 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f15180b2a10 0x7f15180b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:54.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.574+0000 7f15177fe640 1 -- 172.21.15.2:0/3881281965 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1524020070 con 0x55ef6fc918f0 2024-05-14T17:00:54.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.574+0000 7f15177fe640 1 -- 172.21.15.2:0/3881281965 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f152401ae30 con 0x55ef6fc918f0 2024-05-14T17:00:54.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.574+0000 7f15177fe640 1 -- 172.21.15.2:0/3881281965 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1524014070 con 0x55ef6fc918f0 2024-05-14T17:00:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.575+0000 7f15361cf740 1 -- 172.21.15.2:0/3881281965 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef6fc918f0 msgr2=0x55ef6fc91cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.575+0000 7f1531977640 1 --2- 172.21.15.2:0/3218439981 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f15180b2a10 0x7f15180b4ed0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f152401d000 tx=0x7f1524020000 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.575+0000 7f15361cf740 1 --2- 172.21.15.2:0/3881281965 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef6fc918f0 0x55ef6fc91cd0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f1524007e60 tx=0x7f152400e720 comp rx=0 tx=0).stop 2024-05-14T17:00:54.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.575+0000 7f15361cf740 1 -- 172.21.15.2:0/3881281965 shutdown_connections 2024-05-14T17:00:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.575+0000 7f15361cf740 1 --2- 172.21.15.2:0/3881281965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef6fc9cad0 0x55ef6fc9eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.575+0000 7f15361cf740 1 --2- 172.21.15.2:0/3881281965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef6fc92210 0x55ef6fc9c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.575+0000 7f15361cf740 1 --2- 172.21.15.2:0/3881281965 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef6fc918f0 0x55ef6fc91cd0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:54.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.575+0000 7f15361cf740 1 -- 172.21.15.2:0/3881281965 >> 172.21.15.2:0/3881281965 conn(0x55ef6fc8de00 msgr2=0x55ef6fc8e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.575+0000 7f15361cf740 1 -- 172.21.15.2:0/3881281965 shutdown_connections 2024-05-14T17:00:55.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.575+0000 7f15361cf740 1 -- 172.21.15.2:0/3881281965 wait complete. 2024-05-14T17:00:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.575+0000 7f15361cf740 1 -- start start 2024-05-14T17:00:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.575+0000 7f15361cf740 1 Processor -- start 2024-05-14T17:00:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.576+0000 7f15361cf740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef6fc918f0 0x55ef6fc9c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.576+0000 7f15361cf740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef6fc92210 0x55ef6fc99700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:55.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.576+0000 7f15361cf740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef6fc9cad0 0x55ef6fc99c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:55.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.576+0000 7f15361cf740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ef6f8da790 con 0x55ef6fc9cad0 2024-05-14T17:00:55.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.576+0000 7f15361cf740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ef6fbb0890 con 0x55ef6fc918f0 2024-05-14T17:00:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.576+0000 7f15361cf740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ef6fc9f7a0 con 0x55ef6fc92210 2024-05-14T17:00:55.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.576+0000 7f15361cf740 10 client.0 _ll_register_callbacks cb 0x55ef6fbb3d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:00:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.576+0000 7f1531977640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef6fc9cad0 0x55ef6fc99c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:55.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.576+0000 7f1531977640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef6fc9cad0 0x55ef6fc99c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55774/0 (socket says 172.21.15.2:55774) 2024-05-14T17:00:55.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.576+0000 7f1531977640 1 -- 172.21.15.2:0/1296890128 learned_addr learned my addr 172.21.15.2:0/1296890128 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:00:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.576+0000 7f1531176640 1 --2- 172.21.15.2:0/1296890128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef6fc918f0 0x55ef6fc9c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.576+0000 7f1530975640 1 --2- 172.21.15.2:0/1296890128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef6fc92210 0x55ef6fc99700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.576+0000 7f1531977640 1 -- 172.21.15.2:0/1296890128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef6fc92210 msgr2=0x55ef6fc99700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.576+0000 7f1531977640 1 --2- 172.21.15.2:0/1296890128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef6fc92210 0x55ef6fc99700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.576+0000 7f1531977640 1 -- 172.21.15.2:0/1296890128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef6fc918f0 msgr2=0x55ef6fc9c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.576+0000 7f1531977640 1 --2- 172.21.15.2:0/1296890128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef6fc918f0 0x55ef6fc9c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.576+0000 7f1531977640 1 -- 172.21.15.2:0/1296890128 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef6f9d5620 con 0x55ef6fc9cad0 2024-05-14T17:00:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.576+0000 7f1530975640 1 --2- 172.21.15.2:0/1296890128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef6fc92210 0x55ef6fc99700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:00:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.576+0000 7f1531977640 1 --2- 172.21.15.2:0/1296890128 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef6fc9cad0 0x55ef6fc99c40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1524012000 tx=0x7f15240072a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04f1081ec500d9e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.576+0000 7f1514ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:00:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.577+0000 7f1514ff9640 1 -- 172.21.15.2:0/1296890128 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1524014050 con 0x55ef6fc9cad0 2024-05-14T17:00:55.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.577+0000 7f1514ff9640 1 -- 172.21.15.2:0/1296890128 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f1524003b80 con 0x55ef6fc9cad0 2024-05-14T17:00:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.577+0000 7f15361cf740 1 -- 172.21.15.2:0/1296890128 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f152401ae30 con 0x55ef6fc9cad0 2024-05-14T17:00:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.577+0000 7f1514ff9640 1 -- 172.21.15.2:0/1296890128 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f1524003d80 con 0x55ef6fc9cad0 2024-05-14T17:00:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.577+0000 7f15361cf740 10 client.15975 fetch_fsmap learned FSMap version 38 2024-05-14T17:00:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.577+0000 7f15361cf740 1 -- 172.21.15.2:0/1296890128 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ef6fcab1a0 con 0x55ef6fc9cad0 2024-05-14T17:00:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.577+0000 7f1514ff9640 1 -- 172.21.15.2:0/1296890128 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f152401ae30 con 0x55ef6fc9cad0 2024-05-14T17:00:55.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.577+0000 7f1514ff9640 1 -- 172.21.15.2:0/1296890128 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) ==== 8342+0+0 (secure 0 0 0) 0x7f152402c070 con 0x55ef6fc9cad0 2024-05-14T17:00:55.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.577+0000 7f15361cf740 10 client.15975 fetch_fsmap finished waiting for FSMap version 38 2024-05-14T17:00:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.577+0000 7f15361cf740 10 client.15975 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:00:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.577+0000 7f15361cf740 20 client.15975 populate_metadata read hostname 'smithi002' 2024-05-14T17:00:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.577+0000 7f15361cf740 1 --2- 172.21.15.2:0/1296890128 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ef6fcac550 0x55ef6fcae940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:55.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.577+0000 7f15361cf740 4 client.15975 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:00:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.577+0000 7f15361cf740 1 -- 172.21.15.2:0/1296890128 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef6fc06b80 con 0x55ef6fcac550 2024-05-14T17:00:55.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.578+0000 7f1531176640 1 --2- 172.21.15.2:0/1296890128 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ef6fcac550 0x55ef6fcae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.578+0000 7f1531176640 1 --2- 172.21.15.2:0/1296890128 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ef6fcac550 0x55ef6fcae940 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a61acdbcbb09a58f server_cookie=65f536e473fa7c61 in_seq=0 out_seq=0 2024-05-14T17:00:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.578+0000 7f1514ff9640 10 client.15975 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:00:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.579+0000 7f1514ff9640 1 -- 172.21.15.2:0/1296890128 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ef6fc06b80 con 0x55ef6fcac550 2024-05-14T17:00:55.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.579+0000 7f1514ff9640 10 client.15975 handle_command_reply: tid=0 2024-05-14T17:00:55.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.579+0000 7f15361cf740 1 client.15975 shutdown 2024-05-14T17:00:55.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.579+0000 7f15361cf740 1 -- 172.21.15.2:0/1296890128 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef6fc9cad0 msgr2=0x55ef6fc99c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:55.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.579+0000 7f15361cf740 1 --2- 172.21.15.2:0/1296890128 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef6fc9cad0 0x55ef6fc99c40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1524012000 tx=0x7f15240072a0 comp rx=0 tx=0).stop 2024-05-14T17:00:55.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.579+0000 7f15361cf740 1 -- 172.21.15.2:0/1296890128 shutdown_connections 2024-05-14T17:00:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.579+0000 7f15361cf740 1 --2- 172.21.15.2:0/1296890128 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ef6fcac550 0x55ef6fcae940 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:55.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.579+0000 7f15361cf740 1 --2- 172.21.15.2:0/1296890128 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef6fc9cad0 0x55ef6fc99c40 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:55.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.579+0000 7f15361cf740 1 --2- 172.21.15.2:0/1296890128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef6fc92210 0x55ef6fc99700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:55.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.579+0000 7f15361cf740 1 --2- 172.21.15.2:0/1296890128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef6fc918f0 0x55ef6fc9c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.579+0000 7f1514ff9640 0 client.15975 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:00:56.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.579+0000 7f15361cf740 1 -- 172.21.15.2:0/1296890128 >> 172.21.15.2:0/1296890128 conn(0x55ef6fc8de00 msgr2=0x55ef6fc8d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.579+0000 7f15361cf740 1 -- 172.21.15.2:0/1296890128 shutdown_connections 2024-05-14T17:00:56.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.579+0000 7f15361cf740 1 -- 172.21.15.2:0/1296890128 wait complete. 2024-05-14T17:00:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.579+0000 7f15361cf740 20 client.15975 trim_cache size 0 max 16384 2024-05-14T17:00:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.594+0000 7f15361cf740 1 Processor -- start 2024-05-14T17:00:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.594+0000 7f15361cf740 1 -- start start 2024-05-14T17:00:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.594+0000 7f15361cf740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef6fc918f0 0x55ef6fcb4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.594+0000 7f15361cf740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef6fc92210 0x55ef6fcb4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.594+0000 7f15361cf740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef6fc9cad0 0x55ef6fc96960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.594+0000 7f15361cf740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f152c1b3040 con 0x55ef6fc92210 2024-05-14T17:00:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.594+0000 7f15361cf740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f152c1b08e0 con 0x55ef6fc918f0 2024-05-14T17:00:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.594+0000 7f15361cf740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f152c1ae730 con 0x55ef6fc9cad0 2024-05-14T17:00:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.595+0000 7f1530975640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef6fc92210 0x55ef6fcb4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.595+0000 7f1530975640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef6fc92210 0x55ef6fcb4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55778/0 (socket says 172.21.15.2:55778) 2024-05-14T17:00:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.595+0000 7f1530975640 1 -- 172.21.15.2:0/994166099 learned_addr learned my addr 172.21.15.2:0/994166099 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:00:56.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.595+0000 7f1531176640 1 --2- 172.21.15.2:0/994166099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef6fc918f0 0x55ef6fcb4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.595+0000 7f1530975640 1 -- 172.21.15.2:0/994166099 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef6fc9cad0 msgr2=0x55ef6fc96960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:56.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.595+0000 7f1530975640 1 --2- 172.21.15.2:0/994166099 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef6fc9cad0 0x55ef6fc96960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.595+0000 7f1530975640 1 -- 172.21.15.2:0/994166099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef6fc918f0 msgr2=0x55ef6fcb4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.595+0000 7f1530975640 1 --2- 172.21.15.2:0/994166099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef6fc918f0 0x55ef6fcb4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.595+0000 7f1530975640 1 -- 172.21.15.2:0/994166099 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef6fc8e740 con 0x55ef6fc92210 2024-05-14T17:00:56.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.595+0000 7f1530975640 1 --2- 172.21.15.2:0/994166099 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef6fc92210 0x55ef6fcb4660 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f151c00b940 tx=0x7f151c00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b4446061193166b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:56.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.595+0000 7f1516ffd640 1 -- 172.21.15.2:0/994166099 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c010c60 con 0x55ef6fc92210 2024-05-14T17:00:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.595+0000 7f1516ffd640 1 -- 172.21.15.2:0/994166099 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f151c010e00 con 0x55ef6fc92210 2024-05-14T17:00:56.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.595+0000 7f1516ffd640 1 -- 172.21.15.2:0/994166099 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c00e810 con 0x55ef6fc92210 2024-05-14T17:00:56.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.596+0000 7f15361cf740 1 -- 172.21.15.2:0/994166099 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef6fc92210 msgr2=0x55ef6fcb4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:56.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.596+0000 7f15361cf740 1 --2- 172.21.15.2:0/994166099 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef6fc92210 0x55ef6fcb4660 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f151c00b940 tx=0x7f151c00be00 comp rx=0 tx=0).stop 2024-05-14T17:00:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.596+0000 7f15361cf740 1 -- 172.21.15.2:0/994166099 shutdown_connections 2024-05-14T17:00:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.596+0000 7f15361cf740 1 --2- 172.21.15.2:0/994166099 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef6fc9cad0 0x55ef6fc96960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:56.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.596+0000 7f15361cf740 1 --2- 172.21.15.2:0/994166099 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef6fc92210 0x55ef6fcb4660 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:56.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.596+0000 7f15361cf740 1 --2- 172.21.15.2:0/994166099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef6fc918f0 0x55ef6fcb4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:56.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.596+0000 7f15361cf740 1 -- 172.21.15.2:0/994166099 >> 172.21.15.2:0/994166099 conn(0x55ef6fc8de00 msgr2=0x55ef6fca7660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.596+0000 7f15361cf740 1 -- 172.21.15.2:0/994166099 shutdown_connections 2024-05-14T17:00:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.596+0000 7f15361cf740 1 -- 172.21.15.2:0/994166099 wait complete. 2024-05-14T17:00:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.596+0000 7f15361cf740 1 -- start start 2024-05-14T17:00:56.378 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.596+0000 7f15361cf740 1 Processor -- start 2024-05-14T17:00:56.384 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.597+0000 7f15361cf740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef6fc918f0 0x55ef6fca2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:56.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.597+0000 7f15361cf740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef6fc92210 0x55ef6fca2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.597+0000 7f15361cf740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef6fc9cad0 0x55ef6fca32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.597+0000 7f15361cf740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f152c1aeda0 con 0x55ef6fc92210 2024-05-14T17:00:56.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.597+0000 7f15361cf740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f152c1b0210 con 0x55ef6fc9cad0 2024-05-14T17:00:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.597+0000 7f15361cf740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f152c1af470 con 0x55ef6fc918f0 2024-05-14T17:00:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.597+0000 7f15361cf740 10 client.0 _ll_register_callbacks cb 0x7f1524007ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:00:56.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.597+0000 7f1530975640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef6fc92210 0x55ef6fca2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:56.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.597+0000 7f1530975640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef6fc92210 0x55ef6fca2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55784/0 (socket says 172.21.15.2:55784) 2024-05-14T17:00:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.597+0000 7f1530975640 1 -- 172.21.15.2:0/4282557171 learned_addr learned my addr 172.21.15.2:0/4282557171 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:00:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.597+0000 7f1531176640 1 --2- 172.21.15.2:0/4282557171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef6fc918f0 0x55ef6fca2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.597+0000 7f1530975640 1 -- 172.21.15.2:0/4282557171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef6fc918f0 msgr2=0x55ef6fca2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.597+0000 7f1530975640 1 --2- 172.21.15.2:0/4282557171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef6fc918f0 0x55ef6fca2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:56.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.597+0000 7f1531977640 1 --2- 172.21.15.2:0/4282557171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef6fc9cad0 0x55ef6fca32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.597+0000 7f1530975640 1 -- 172.21.15.2:0/4282557171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef6fc9cad0 msgr2=0x55ef6fca32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.597+0000 7f1530975640 1 --2- 172.21.15.2:0/4282557171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef6fc9cad0 0x55ef6fca32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.597+0000 7f1530975640 1 -- 172.21.15.2:0/4282557171 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef6fd15f80 con 0x55ef6fc92210 2024-05-14T17:00:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.597+0000 7f1530975640 1 --2- 172.21.15.2:0/4282557171 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef6fc92210 0x55ef6fca2d90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f151c000c00 tx=0x7f151c019e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e2a7cd7762bc3fb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:56.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.597+0000 7f1514ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:00:56.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.597+0000 7f1514ff9640 1 -- 172.21.15.2:0/4282557171 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c028020 con 0x55ef6fc92210 2024-05-14T17:00:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.597+0000 7f15361cf740 1 -- 172.21.15.2:0/4282557171 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f151c010e00 con 0x55ef6fc92210 2024-05-14T17:00:56.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.597+0000 7f1514ff9640 1 -- 172.21.15.2:0/4282557171 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f151c019800 con 0x55ef6fc92210 2024-05-14T17:00:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.598+0000 7f1514ff9640 1 -- 172.21.15.2:0/4282557171 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f151c033050 con 0x55ef6fc92210 2024-05-14T17:00:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.598+0000 7f15361cf740 10 client.15981 fetch_fsmap learned FSMap version 38 2024-05-14T17:00:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.598+0000 7f15361cf740 1 -- 172.21.15.2:0/4282557171 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ef6fd17290 con 0x55ef6fc92210 2024-05-14T17:00:56.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.598+0000 7f1514ff9640 1 -- 172.21.15.2:0/4282557171 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f151c010e00 con 0x55ef6fc92210 2024-05-14T17:00:56.618 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.598+0000 7f1514ff9640 1 -- 172.21.15.2:0/4282557171 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) ==== 8342+0+0 (secure 0 0 0) 0x7f151c018070 con 0x55ef6fc92210 2024-05-14T17:00:56.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.598+0000 7f15361cf740 10 client.15981 fetch_fsmap finished waiting for FSMap version 38 2024-05-14T17:00:56.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.598+0000 7f15361cf740 10 client.15981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:00:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.598+0000 7f15361cf740 20 client.15981 populate_metadata read hostname 'smithi002' 2024-05-14T17:00:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.598+0000 7f15361cf740 1 --2- 172.21.15.2:0/4282557171 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ef6fd17a40 0x55ef6fd19e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.598+0000 7f15361cf740 4 client.15981 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:00:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.598+0000 7f15361cf740 1 -- 172.21.15.2:0/4282557171 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ef6fc00360 con 0x55ef6fd17a40 2024-05-14T17:00:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.598+0000 7f1531176640 1 --2- 172.21.15.2:0/4282557171 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ef6fd17a40 0x55ef6fd19e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.599+0000 7f1531176640 1 --2- 172.21.15.2:0/4282557171 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ef6fd17a40 0x55ef6fd19e60 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d086d8c3480fcbc server_cookie=d9e9f538b6207d7e in_seq=0 out_seq=0 2024-05-14T17:00:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.599+0000 7f1514ff9640 10 client.15981 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:00:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.620+0000 7f1514ff9640 1 -- 172.21.15.2:0/4282557171 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ef6fc00360 con 0x55ef6fd17a40 2024-05-14T17:00:56.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.620+0000 7f1514ff9640 10 client.15981 handle_command_reply: tid=0 2024-05-14T17:00:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.620+0000 7f15361cf740 1 client.15981 shutdown 2024-05-14T17:00:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.620+0000 7f15361cf740 1 -- 172.21.15.2:0/4282557171 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef6fc92210 msgr2=0x55ef6fca2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.620+0000 7f15361cf740 1 --2- 172.21.15.2:0/4282557171 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef6fc92210 0x55ef6fca2d90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f151c000c00 tx=0x7f151c019e50 comp rx=0 tx=0).stop 2024-05-14T17:00:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.620+0000 7f15361cf740 1 -- 172.21.15.2:0/4282557171 shutdown_connections 2024-05-14T17:00:56.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.620+0000 7f15361cf740 1 --2- 172.21.15.2:0/4282557171 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ef6fd17a40 0x55ef6fd19e60 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:56.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.620+0000 7f15361cf740 1 --2- 172.21.15.2:0/4282557171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef6fc9cad0 0x55ef6fca32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:56.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.620+0000 7f15361cf740 1 --2- 172.21.15.2:0/4282557171 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef6fc92210 0x55ef6fca2d90 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.620+0000 7f1514ff9640 0 client.15981 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:00:56.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.620+0000 7f15361cf740 1 --2- 172.21.15.2:0/4282557171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef6fc918f0 0x55ef6fca2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.620+0000 7f15361cf740 1 -- 172.21.15.2:0/4282557171 >> 172.21.15.2:0/4282557171 conn(0x55ef6fc8de00 msgr2=0x55ef6fca7ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.620+0000 7f15361cf740 1 -- 172.21.15.2:0/4282557171 shutdown_connections 2024-05-14T17:00:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.620+0000 7f15361cf740 1 -- 172.21.15.2:0/4282557171 wait complete. 2024-05-14T17:00:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.620+0000 7f15361cf740 20 client.15981 trim_cache size 0 max 16384 2024-05-14T17:00:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:00:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:00:56.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:00:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.621+0000 7f15333da640 1 -- 172.21.15.2:0/3218439981 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f15180b2a10 msgr2=0x7f15180b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:56.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.621+0000 7f15333da640 1 --2- 172.21.15.2:0/3218439981 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f15180b2a10 0x7f15180b4ed0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f152401d000 tx=0x7f1524020000 comp rx=0 tx=0).stop 2024-05-14T17:00:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.621+0000 7f15333da640 1 -- 172.21.15.2:0/3218439981 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f152c074e80 msgr2=0x7f152c19b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:56.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.621+0000 7f15333da640 1 --2- 172.21.15.2:0/3218439981 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f152c074e80 0x7f152c19b380 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f1528006010 tx=0x7f15280042f0 comp rx=0 tx=0).stop 2024-05-14T17:00:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.621+0000 7f15333da640 1 -- 172.21.15.2:0/3218439981 shutdown_connections 2024-05-14T17:00:57.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.621+0000 7f15333da640 1 --2- 172.21.15.2:0/3218439981 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f15180b2a10 0x7f15180b4ed0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.621+0000 7f15333da640 1 --2- 172.21.15.2:0/3218439981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f152c10f270 0x7f152c19fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.621+0000 7f15333da640 1 --2- 172.21.15.2:0/3218439981 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f152c0757a0 0x7f152c19b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:57.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.621+0000 7f15333da640 1 --2- 172.21.15.2:0/3218439981 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f152c074e80 0x7f152c19b380 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:57.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.621+0000 7f15333da640 1 -- 172.21.15.2:0/3218439981 >> 172.21.15.2:0/3218439981 conn(0x7f152c0fe500 msgr2=0x7f152c0ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:57.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.621+0000 7f15333da640 1 -- 172.21.15.2:0/3218439981 shutdown_connections 2024-05-14T17:00:57.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:00:53.621+0000 7f15333da640 1 -- 172.21.15.2:0/3218439981 wait complete. 2024-05-14T17:00:57.112 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:00:54 smithi123 ceph-mon[35971]: pgmap v189: 97 pgs: 97 active+clean; 3.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.3 MiB/s wr, 392 op/s 2024-05-14T17:00:57.128 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:00:55 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:00:57.136 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:00:56 smithi123 ceph-mon[35971]: pgmap v190: 97 pgs: 97 active+clean; 3.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 799 KiB/s wr, 87 op/s 2024-05-14T17:00:57.150 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:00:56 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:00:57.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:00:56 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:00:57.164 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:00:56 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:00:57.171 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:00:54 smithi151 ceph-mon[36066]: pgmap v189: 97 pgs: 97 active+clean; 3.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.3 MiB/s wr, 392 op/s 2024-05-14T17:00:57.177 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:00:55 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:00:57.183 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:00:56 smithi151 ceph-mon[36066]: pgmap v190: 97 pgs: 97 active+clean; 3.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 799 KiB/s wr, 87 op/s 2024-05-14T17:00:57.189 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:00:56 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:00:57.223 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:00:56 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:00:57.260 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:00:56 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:00:57.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:53.997+0000 7f4917091700 1 -- 172.21.15.2:0/407354876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4910102e20 msgr2=0x7f491010f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:57.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:53.997+0000 7f4917091700 1 --2- 172.21.15.2:0/407354876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4910102e20 0x7f491010f330 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4908009990 tx=0x7f4908009ca0 comp rx=0 tx=0).stop 2024-05-14T17:00:57.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:53.997+0000 7f4917091700 1 -- 172.21.15.2:0/407354876 shutdown_connections 2024-05-14T17:00:57.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:53.997+0000 7f4917091700 1 --2- 172.21.15.2:0/407354876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4910102e20 0x7f491010f330 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:57.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:53.997+0000 7f4917091700 1 --2- 172.21.15.2:0/407354876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49101084f0 0x7f49101088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:57.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:53.997+0000 7f4917091700 1 --2- 172.21.15.2:0/407354876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4910102460 0x7f49101028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:57.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:53.997+0000 7f4917091700 1 -- 172.21.15.2:0/407354876 >> 172.21.15.2:0/407354876 conn(0x7f49100794e0 msgr2=0x7f49100798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:57.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:53.997+0000 7f4917091700 1 -- 172.21.15.2:0/407354876 shutdown_connections 2024-05-14T17:00:57.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:53.997+0000 7f4917091700 1 -- 172.21.15.2:0/407354876 wait complete. 2024-05-14T17:00:57.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:53.998+0000 7f4917091700 1 Processor -- start 2024-05-14T17:00:57.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:53.998+0000 7f4917091700 1 -- start start 2024-05-14T17:00:57.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:53.998+0000 7f4917091700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4910102460 0x7f49100ff490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:57.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:53.998+0000 7f4917091700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4910102e20 0x7f49100ff9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:57.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:53.998+0000 7f4917091700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49101084f0 0x7f49100fff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:57.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:53.998+0000 7f4917091700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4910112750 con 0x7f4910102460 2024-05-14T17:00:57.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:53.998+0000 7f4917091700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f49101125d0 con 0x7f4910102e20 2024-05-14T17:00:57.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:53.998+0000 7f4917091700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f49101128d0 con 0x7f49101084f0 2024-05-14T17:00:57.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:53.998+0000 7f4914e2d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4910102460 0x7f49100ff490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:57.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:53.998+0000 7f4914e2d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4910102460 0x7f49100ff490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59428/0 (socket says 172.21.15.2:59428) 2024-05-14T17:00:57.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:53.998+0000 7f4914e2d700 1 -- 172.21.15.2:0/1222554944 learned_addr learned my addr 172.21.15.2:0/1222554944 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:00:57.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:53.998+0000 7f490ffff700 1 --2- 172.21.15.2:0/1222554944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4910102e20 0x7f49100ff9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:57.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:53.998+0000 7f491562e700 1 --2- 172.21.15.2:0/1222554944 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49101084f0 0x7f49100fff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:57.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:53.999+0000 7f4914e2d700 1 -- 172.21.15.2:0/1222554944 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49101084f0 msgr2=0x7f49100fff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:57.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:53.999+0000 7f4914e2d700 1 --2- 172.21.15.2:0/1222554944 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49101084f0 0x7f49100fff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:57.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:53.999+0000 7f4914e2d700 1 -- 172.21.15.2:0/1222554944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4910102e20 msgr2=0x7f49100ff9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:57.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:53.999+0000 7f4914e2d700 1 --2- 172.21.15.2:0/1222554944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4910102e20 0x7f49100ff9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:57.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:53.999+0000 7f4914e2d700 1 -- 172.21.15.2:0/1222554944 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49101a83a0 con 0x7f4910102460 2024-05-14T17:00:57.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:53.999+0000 7f491562e700 1 --2- 172.21.15.2:0/1222554944 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49101084f0 0x7f49100fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:00:57.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:53.999+0000 7f4914e2d700 1 --2- 172.21.15.2:0/1222554944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4910102460 0x7f49100ff490 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f490000ba50 tx=0x7f490000bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:57.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:53.999+0000 7f490dffb700 1 -- 172.21.15.2:0/1222554944 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4900019070 con 0x7f4910102460 2024-05-14T17:00:57.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:53.999+0000 7f490dffb700 1 -- 172.21.15.2:0/1222554944 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4900014440 con 0x7f4910102460 2024-05-14T17:00:57.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:53.999+0000 7f490dffb700 1 -- 172.21.15.2:0/1222554944 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490000ce60 con 0x7f4910102460 2024-05-14T17:00:57.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:53.999+0000 7f4917091700 1 -- 172.21.15.2:0/1222554944 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49101a8690 con 0x7f4910102460 2024-05-14T17:00:57.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:53.999+0000 7f4917091700 1 -- 172.21.15.2:0/1222554944 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49101a8b80 con 0x7f4910102460 2024-05-14T17:00:57.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.002+0000 7f490dffb700 1 -- 172.21.15.2:0/1222554944 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f490001c070 con 0x7f4910102460 2024-05-14T17:00:57.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.003+0000 7f490dffb700 1 --2- 172.21.15.2:0/1222554944 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f48fc0b29f0 0x7f48fc0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:57.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.003+0000 7f490dffb700 1 -- 172.21.15.2:0/1222554944 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f49000e1900 con 0x7f4910102460 2024-05-14T17:00:57.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.003+0000 7f490ffff700 1 --2- 172.21.15.2:0/1222554944 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f48fc0b29f0 0x7f48fc0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:57.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.004+0000 7f4921927b80 1 Processor -- start 2024-05-14T17:00:57.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.004+0000 7f4921927b80 1 -- start start 2024-05-14T17:00:57.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.004+0000 7f490ffff700 1 --2- 172.21.15.2:0/1222554944 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f48fc0b29f0 0x7f48fc0b4eb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4910079ce0 tx=0x7f48f8009190 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:57.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.004+0000 7f4921927b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564168fb9da0 0x564168fba180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:57.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.004+0000 7f4921927b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564168fba6c0 0x564168fb44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:57.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.004+0000 7f4921927b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564168fb4ad0 0x564168fb4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:57.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.004+0000 7f4921927b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x564168f12d90 con 0x564168fb9da0 2024-05-14T17:00:57.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.004+0000 7f4921927b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564168f02930 con 0x564168fb4ad0 2024-05-14T17:00:57.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.004+0000 7f4921927b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564168f06970 con 0x564168fba6c0 2024-05-14T17:00:57.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.004+0000 7f4914e2d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564168fb9da0 0x564168fba180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:57.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.004+0000 7f4914e2d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564168fb9da0 0x564168fba180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59434/0 (socket says 172.21.15.2:59434) 2024-05-14T17:00:57.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.004+0000 7f4914e2d700 1 -- 172.21.15.2:0/4252513471 learned_addr learned my addr 172.21.15.2:0/4252513471 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:00:57.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.004+0000 7f490ffff700 1 --2- 172.21.15.2:0/4252513471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564168fba6c0 0x564168fb44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:57.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.004+0000 7f491562e700 1 --2- 172.21.15.2:0/4252513471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564168fb4ad0 0x564168fb4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:57.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.004+0000 7f4914e2d700 1 -- 172.21.15.2:0/4252513471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564168fba6c0 msgr2=0x564168fb44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:57.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.004+0000 7f4914e2d700 1 --2- 172.21.15.2:0/4252513471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564168fba6c0 0x564168fb44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:57.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.004+0000 7f4914e2d700 1 -- 172.21.15.2:0/4252513471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564168fb4ad0 msgr2=0x564168fb4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:57.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.004+0000 7f4914e2d700 1 --2- 172.21.15.2:0/4252513471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564168fb4ad0 0x564168fb4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:57.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.004+0000 7f4914e2d700 1 -- 172.21.15.2:0/4252513471 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564168e529d0 con 0x564168fb9da0 2024-05-14T17:00:57.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.004+0000 7f491562e700 1 --2- 172.21.15.2:0/4252513471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564168fb4ad0 0x564168fb4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:00:57.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.005+0000 7f4914e2d700 1 --2- 172.21.15.2:0/4252513471 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564168fb9da0 0x564168fba180 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4900016000 tx=0x7f49000a8850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f61a047673f49c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:57.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.005+0000 7f4906ffd700 1 -- 172.21.15.2:0/4252513471 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49000ad070 con 0x564168fb9da0 2024-05-14T17:00:57.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.005+0000 7f4906ffd700 1 -- 172.21.15.2:0/4252513471 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49000b1b20 con 0x564168fb9da0 2024-05-14T17:00:57.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.005+0000 7f4906ffd700 1 -- 172.21.15.2:0/4252513471 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49000a9120 con 0x564168fb9da0 2024-05-14T17:00:57.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.005+0000 7f4921927b80 1 -- 172.21.15.2:0/4252513471 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564168fb9da0 msgr2=0x564168fba180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:57.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.005+0000 7f4921927b80 1 --2- 172.21.15.2:0/4252513471 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564168fb9da0 0x564168fba180 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4900016000 tx=0x7f49000a8850 comp rx=0 tx=0).stop 2024-05-14T17:00:57.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.006+0000 7f4921927b80 1 -- 172.21.15.2:0/4252513471 shutdown_connections 2024-05-14T17:00:57.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.006+0000 7f4921927b80 1 --2- 172.21.15.2:0/4252513471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564168fb4ad0 0x564168fb4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:57.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.006+0000 7f4921927b80 1 --2- 172.21.15.2:0/4252513471 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564168fb9da0 0x564168fba180 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:58.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.006+0000 7f4921927b80 1 --2- 172.21.15.2:0/4252513471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564168fba6c0 0x564168fb44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:58.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.006+0000 7f4921927b80 1 -- 172.21.15.2:0/4252513471 >> 172.21.15.2:0/4252513471 conn(0x564168edf350 msgr2=0x564168faece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:58.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.006+0000 7f4921927b80 1 -- 172.21.15.2:0/4252513471 shutdown_connections 2024-05-14T17:00:58.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.006+0000 7f4921927b80 1 -- 172.21.15.2:0/4252513471 wait complete. 2024-05-14T17:00:58.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.006+0000 7f4921927b80 1 -- start start 2024-05-14T17:00:58.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.006+0000 7f4921927b80 1 Processor -- start 2024-05-14T17:00:58.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.006+0000 7f4921927b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564168fb4ad0 0x564168fc03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:58.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.006+0000 7f4921927b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564168fb9da0 0x564168fbd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:58.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.006+0000 7f4921927b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564168fba6c0 0x564168fbdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:58.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.007+0000 7f4921927b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x564168e090a0 con 0x564168fb4ad0 2024-05-14T17:00:58.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.007+0000 7f4921927b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564168ea48c0 con 0x564168fba6c0 2024-05-14T17:00:58.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.007+0000 7f4921927b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564168fc1100 con 0x564168fb9da0 2024-05-14T17:00:58.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.007+0000 7f4921927b80 10 client.0 _ll_register_callbacks cb 0x564168e99680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:00:58.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.007+0000 7f4914e2d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564168fb4ad0 0x564168fc03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:58.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.007+0000 7f4914e2d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564168fb4ad0 0x564168fc03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59446/0 (socket says 172.21.15.2:59446) 2024-05-14T17:00:58.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.007+0000 7f4914e2d700 1 -- 172.21.15.2:0/652811273 learned_addr learned my addr 172.21.15.2:0/652811273 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:00:58.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.007+0000 7f491562e700 1 --2- 172.21.15.2:0/652811273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564168fba6c0 0x564168fbdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:58.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.007+0000 7f490ffff700 1 --2- 172.21.15.2:0/652811273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564168fb9da0 0x564168fbd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:58.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.007+0000 7f4914e2d700 1 -- 172.21.15.2:0/652811273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564168fb9da0 msgr2=0x564168fbd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:58.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.007+0000 7f4914e2d700 1 --2- 172.21.15.2:0/652811273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564168fb9da0 0x564168fbd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:58.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.007+0000 7f4914e2d700 1 -- 172.21.15.2:0/652811273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564168fba6c0 msgr2=0x564168fbdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:58.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.007+0000 7f4914e2d700 1 --2- 172.21.15.2:0/652811273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564168fba6c0 0x564168fbdcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:58.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.007+0000 7f4914e2d700 1 -- 172.21.15.2:0/652811273 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564168ea1790 con 0x564168fb4ad0 2024-05-14T17:00:58.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.007+0000 7f490ffff700 1 --2- 172.21.15.2:0/652811273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564168fb9da0 0x564168fbd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:00:58.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.007+0000 7f491562e700 1 --2- 172.21.15.2:0/652811273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564168fba6c0 0x564168fbdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:00:58.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.007+0000 7f4914e2d700 1 --2- 172.21.15.2:0/652811273 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564168fb4ad0 0x564168fc03c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f490000ba20 tx=0x7f49000099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ecd4f63b701d5c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:58.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.007+0000 7f48effff700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:00:58.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.007+0000 7f48effff700 1 -- 172.21.15.2:0/652811273 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49000c33b0 con 0x564168fb4ad0 2024-05-14T17:00:58.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.007+0000 7f48effff700 1 -- 172.21.15.2:0/652811273 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f49000a9130 con 0x564168fb4ad0 2024-05-14T17:00:58.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.007+0000 7f4921927b80 1 -- 172.21.15.2:0/652811273 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49000b1b20 con 0x564168fb4ad0 2024-05-14T17:00:58.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.007+0000 7f48effff700 1 -- 172.21.15.2:0/652811273 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f49000b1690 con 0x564168fb4ad0 2024-05-14T17:00:58.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.008+0000 7f4921927b80 10 client.15993 fetch_fsmap learned FSMap version 38 2024-05-14T17:00:58.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.008+0000 7f4921927b80 1 -- 172.21.15.2:0/652811273 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x564168fcb800 con 0x564168fb4ad0 2024-05-14T17:00:58.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.008+0000 7f48effff700 1 -- 172.21.15.2:0/652811273 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49000b1b20 con 0x564168fb4ad0 2024-05-14T17:00:58.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.008+0000 7f48effff700 1 -- 172.21.15.2:0/652811273 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) ==== 8342+0+0 (secure 0 0 0) 0x7f49000a4630 con 0x564168fb4ad0 2024-05-14T17:00:58.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.008+0000 7f4921927b80 10 client.15993 fetch_fsmap finished waiting for FSMap version 38 2024-05-14T17:00:58.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.008+0000 7f4921927b80 10 client.15993 resolve_mds: validated gid 24479 aka daemon mds.b 2024-05-14T17:00:58.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.008+0000 7f4921927b80 20 client.15993 populate_metadata read hostname 'smithi002' 2024-05-14T17:00:58.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.008+0000 7f4921927b80 1 --2- 172.21.15.2:0/652811273 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564168fcdcd0 0x564168fd00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:58.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.008+0000 7f4921927b80 4 client.15993 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:00:58.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.008+0000 7f4921927b80 1 -- 172.21.15.2:0/652811273 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564168f80460 con 0x564168fcdcd0 2024-05-14T17:00:58.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.008+0000 7f490ffff700 1 --2- 172.21.15.2:0/652811273 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564168fcdcd0 0x564168fd00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:58.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.009+0000 7f48effff700 10 client.15993 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:00:58.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.009+0000 7f490ffff700 1 --2- 172.21.15.2:0/652811273 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564168fcdcd0 0x564168fd00c0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83fedde093385755 server_cookie=9c10b439290ff0c0 in_seq=0 out_seq=0 2024-05-14T17:00:58.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.009+0000 7f48effff700 1 -- 172.21.15.2:0/652811273 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564168f80460 con 0x564168fcdcd0 2024-05-14T17:00:58.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.009+0000 7f48effff700 10 client.15993 handle_command_reply: tid=0 2024-05-14T17:00:58.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.009+0000 7f4921927b80 1 client.15993 shutdown 2024-05-14T17:00:58.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.009+0000 7f4921927b80 1 -- 172.21.15.2:0/652811273 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564168fb4ad0 msgr2=0x564168fc03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:58.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.009+0000 7f4921927b80 1 --2- 172.21.15.2:0/652811273 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564168fb4ad0 0x564168fc03c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f490000ba20 tx=0x7f49000099c0 comp rx=0 tx=0).stop 2024-05-14T17:00:58.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.009+0000 7f4921927b80 1 -- 172.21.15.2:0/652811273 shutdown_connections 2024-05-14T17:00:58.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.009+0000 7f4921927b80 1 --2- 172.21.15.2:0/652811273 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564168fcdcd0 0x564168fd00c0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:58.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.010+0000 7f4921927b80 1 --2- 172.21.15.2:0/652811273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564168fba6c0 0x564168fbdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:58.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.010+0000 7f4921927b80 1 --2- 172.21.15.2:0/652811273 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564168fb4ad0 0x564168fc03c0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:58.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.010+0000 7f4921927b80 1 --2- 172.21.15.2:0/652811273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564168fb9da0 0x564168fbd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:58.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.010+0000 7f4921927b80 1 -- 172.21.15.2:0/652811273 >> 172.21.15.2:0/652811273 conn(0x564168edf350 msgr2=0x564168fa97d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:58.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.010+0000 7f48effff700 0 client.15993 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:00:58.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.010+0000 7f4921927b80 1 -- 172.21.15.2:0/652811273 shutdown_connections 2024-05-14T17:00:58.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.010+0000 7f4921927b80 1 -- 172.21.15.2:0/652811273 wait complete. 2024-05-14T17:00:58.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.010+0000 7f4921927b80 20 client.15993 trim_cache size 0 max 16384 2024-05-14T17:00:58.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.030+0000 7f4921927b80 1 Processor -- start 2024-05-14T17:00:58.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.031+0000 7f4921927b80 1 -- start start 2024-05-14T17:00:58.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.031+0000 7f4921927b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564168fb4ad0 0x564168fa98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:58.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.031+0000 7f4921927b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564168fb9da0 0x564168fbf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:58.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.031+0000 7f4921927b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564168fba6c0 0x564168fbf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:58.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.031+0000 7f4921927b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f49101b29d0 con 0x564168fba6c0 2024-05-14T17:00:58.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.031+0000 7f4921927b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f49101b0230 con 0x564168fb4ad0 2024-05-14T17:00:58.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.031+0000 7f4921927b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f49101ae0a0 con 0x564168fb9da0 2024-05-14T17:00:58.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.031+0000 7f491562e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564168fba6c0 0x564168fbf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:58.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.031+0000 7f491562e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564168fba6c0 0x564168fbf9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59454/0 (socket says 172.21.15.2:59454) 2024-05-14T17:00:58.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.031+0000 7f491562e700 1 -- 172.21.15.2:0/4200598828 learned_addr learned my addr 172.21.15.2:0/4200598828 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:00:58.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.031+0000 7f4914e2d700 1 --2- 172.21.15.2:0/4200598828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564168fb4ad0 0x564168fa98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:58.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.031+0000 7f490ffff700 1 --2- 172.21.15.2:0/4200598828 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564168fb9da0 0x564168fbf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:58.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.031+0000 7f491562e700 1 -- 172.21.15.2:0/4200598828 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564168fb9da0 msgr2=0x564168fbf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:58.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.031+0000 7f491562e700 1 --2- 172.21.15.2:0/4200598828 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564168fb9da0 0x564168fbf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:58.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.031+0000 7f491562e700 1 -- 172.21.15.2:0/4200598828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564168fb4ad0 msgr2=0x564168fa98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:58.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.031+0000 7f491562e700 1 --2- 172.21.15.2:0/4200598828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564168fb4ad0 0x564168fa98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:58.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.031+0000 7f491562e700 1 -- 172.21.15.2:0/4200598828 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564168fa9e20 con 0x564168fba6c0 2024-05-14T17:00:58.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.031+0000 7f490ffff700 1 --2- 172.21.15.2:0/4200598828 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564168fb9da0 0x564168fbf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:00:58.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.031+0000 7f4914e2d700 1 --2- 172.21.15.2:0/4200598828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564168fb4ad0 0x564168fa98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:00:58.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.031+0000 7f491562e700 1 --2- 172.21.15.2:0/4200598828 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564168fba6c0 0x564168fbf9b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4908009510 tx=0x7f4908009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c079da00fdca9b90 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:58.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.032+0000 7f49067fc700 1 -- 172.21.15.2:0/4200598828 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4908029070 con 0x564168fba6c0 2024-05-14T17:00:58.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.032+0000 7f49067fc700 1 -- 172.21.15.2:0/4200598828 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4908005070 con 0x564168fba6c0 2024-05-14T17:00:58.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.032+0000 7f49067fc700 1 -- 172.21.15.2:0/4200598828 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490800bbd0 con 0x564168fba6c0 2024-05-14T17:00:58.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.032+0000 7f4921927b80 1 -- 172.21.15.2:0/4200598828 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564168fba6c0 msgr2=0x564168fbf9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:58.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.032+0000 7f4921927b80 1 --2- 172.21.15.2:0/4200598828 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564168fba6c0 0x564168fbf9b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4908009510 tx=0x7f4908009540 comp rx=0 tx=0).stop 2024-05-14T17:00:58.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.032+0000 7f4921927b80 1 -- 172.21.15.2:0/4200598828 shutdown_connections 2024-05-14T17:00:58.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.032+0000 7f4921927b80 1 --2- 172.21.15.2:0/4200598828 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564168fba6c0 0x564168fbf9b0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:58.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.032+0000 7f4921927b80 1 --2- 172.21.15.2:0/4200598828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564168fb4ad0 0x564168fa98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:58.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.032+0000 7f4921927b80 1 --2- 172.21.15.2:0/4200598828 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564168fb9da0 0x564168fbf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:58.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.032+0000 7f4921927b80 1 -- 172.21.15.2:0/4200598828 >> 172.21.15.2:0/4200598828 conn(0x564168edf350 msgr2=0x564168fbed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:58.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.032+0000 7f4921927b80 1 -- 172.21.15.2:0/4200598828 shutdown_connections 2024-05-14T17:00:58.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.032+0000 7f4921927b80 1 -- 172.21.15.2:0/4200598828 wait complete. 2024-05-14T17:00:58.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.033+0000 7f4921927b80 1 -- start start 2024-05-14T17:00:58.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.033+0000 7f4921927b80 1 Processor -- start 2024-05-14T17:00:58.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.033+0000 7f4921927b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564168fb4ad0 0x564168fd1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:58.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.033+0000 7f4921927b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564168fb9da0 0x564168fd1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:58.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.033+0000 7f4921927b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564168fba6c0 0x564168fd2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:58.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.033+0000 7f4921927b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f49101ae6f0 con 0x564168fb4ad0 2024-05-14T17:00:59.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.033+0000 7f4921927b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f49101afb60 con 0x564168fba6c0 2024-05-14T17:00:59.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.033+0000 7f4921927b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f49101aedc0 con 0x564168fb9da0 2024-05-14T17:00:59.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.033+0000 7f4921927b80 10 client.0 _ll_register_callbacks cb 0x564168e99680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:00:59.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.033+0000 7f4914e2d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564168fb4ad0 0x564168fd1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:59.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.033+0000 7f4914e2d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564168fb4ad0 0x564168fd1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59464/0 (socket says 172.21.15.2:59464) 2024-05-14T17:00:59.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.033+0000 7f4914e2d700 1 -- 172.21.15.2:0/1617772035 learned_addr learned my addr 172.21.15.2:0/1617772035 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:00:59.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.033+0000 7f491562e700 1 --2- 172.21.15.2:0/1617772035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564168fba6c0 0x564168fd2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:59.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.033+0000 7f490ffff700 1 --2- 172.21.15.2:0/1617772035 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564168fb9da0 0x564168fd1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:59.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.033+0000 7f4914e2d700 1 -- 172.21.15.2:0/1617772035 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564168fb9da0 msgr2=0x564168fd1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:59.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.034+0000 7f4914e2d700 1 --2- 172.21.15.2:0/1617772035 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564168fb9da0 0x564168fd1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:59.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.034+0000 7f4914e2d700 1 -- 172.21.15.2:0/1617772035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564168fba6c0 msgr2=0x564168fd2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:00:59.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.034+0000 7f4914e2d700 1 --2- 172.21.15.2:0/1617772035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564168fba6c0 0x564168fd2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:59.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.034+0000 7f4914e2d700 1 -- 172.21.15.2:0/1617772035 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564169038390 con 0x564168fb4ad0 2024-05-14T17:00:59.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.034+0000 7f491562e700 1 --2- 172.21.15.2:0/1617772035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564168fba6c0 0x564168fd2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:00:59.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.034+0000 7f4914e2d700 1 --2- 172.21.15.2:0/1617772035 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564168fb4ad0 0x564168fd1a80 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f49000a5260 tx=0x7f49000a4510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6858bfab93bee40 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:00:59.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.034+0000 7f48effff700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:00:59.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.034+0000 7f48effff700 1 -- 172.21.15.2:0/1617772035 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49000c33b0 con 0x564168fb4ad0 2024-05-14T17:00:59.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.034+0000 7f48effff700 1 -- 172.21.15.2:0/1617772035 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8469+0+0 (secure 0 0 0) 0x7f49000a7ab0 con 0x564168fb4ad0 2024-05-14T17:00:59.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.034+0000 7f4921927b80 1 -- 172.21.15.2:0/1617772035 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4908005070 con 0x564168fb4ad0 2024-05-14T17:00:59.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.034+0000 7f48effff700 1 -- 172.21.15.2:0/1617772035 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f49000b1b20 con 0x564168fb4ad0 2024-05-14T17:00:59.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.034+0000 7f4921927b80 10 client.15999 fetch_fsmap learned FSMap version 38 2024-05-14T17:00:59.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.034+0000 7f4921927b80 1 -- 172.21.15.2:0/1617772035 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x564169038390 con 0x564168fb4ad0 2024-05-14T17:00:59.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.034+0000 7f48effff700 1 -- 172.21.15.2:0/1617772035 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4908005070 con 0x564168fb4ad0 2024-05-14T17:00:59.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.034+0000 7f48effff700 1 -- 172.21.15.2:0/1617772035 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) ==== 8342+0+0 (secure 0 0 0) 0x7f490008e070 con 0x564168fb4ad0 2024-05-14T17:00:59.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.034+0000 7f4921927b80 10 client.15999 fetch_fsmap finished waiting for FSMap version 38 2024-05-14T17:00:59.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.034+0000 7f4921927b80 10 client.15999 resolve_mds: validated gid 24479 aka daemon mds.b 2024-05-14T17:00:59.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.034+0000 7f4921927b80 20 client.15999 populate_metadata read hostname 'smithi002' 2024-05-14T17:00:59.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.034+0000 7f4921927b80 1 --2- 172.21.15.2:0/1617772035 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564169039870 0x56416903bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:00:59.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.034+0000 7f4921927b80 4 client.15999 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "bf03fc11", "release": true, "await": true}] 2024-05-14T17:00:59.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.034+0000 7f4921927b80 1 -- 172.21.15.2:0/1617772035 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "quiesce db", "set_id": "bf03fc11", "release": true, "await": true}) -- 0x564168f14ce0 con 0x564169039870 2024-05-14T17:00:59.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.035+0000 7f490ffff700 1 --2- 172.21.15.2:0/1617772035 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564169039870 0x56416903bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:00:59.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.035+0000 7f48effff700 10 client.15999 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:00:59.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.035+0000 7f490ffff700 1 --2- 172.21.15.2:0/1617772035 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564169039870 0x56416903bc60 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61d7d54c3a8b3893 server_cookie=30c93d8314a68635 in_seq=0 out_seq=0 2024-05-14T17:00:59.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.091+0000 7f48effff700 1 -- 172.21.15.2:0/1617772035 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564168f14ce0 con 0x564169039870 2024-05-14T17:00:59.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.091+0000 7f48effff700 10 client.15999 handle_command_reply: tid=0 2024-05-14T17:00:59.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.091+0000 7f4921927b80 1 client.15999 shutdown 2024-05-14T17:00:59.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.092+0000 7f4921927b80 1 -- 172.21.15.2:0/1617772035 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564168fb4ad0 msgr2=0x564168fd1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:59.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.092+0000 7f4921927b80 1 --2- 172.21.15.2:0/1617772035 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564168fb4ad0 0x564168fd1a80 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f49000a5260 tx=0x7f49000a4510 comp rx=0 tx=0).stop 2024-05-14T17:00:59.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.092+0000 7f4921927b80 1 -- 172.21.15.2:0/1617772035 shutdown_connections 2024-05-14T17:00:59.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.092+0000 7f4921927b80 1 --2- 172.21.15.2:0/1617772035 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564169039870 0x56416903bc60 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:59.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.092+0000 7f4921927b80 1 --2- 172.21.15.2:0/1617772035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564168fba6c0 0x564168fd2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:59.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.092+0000 7f4921927b80 1 --2- 172.21.15.2:0/1617772035 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564168fb4ad0 0x564168fd1a80 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:59.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.092+0000 7f4921927b80 1 --2- 172.21.15.2:0/1617772035 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564168fb9da0 0x564168fd1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:59.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.092+0000 7f4921927b80 1 -- 172.21.15.2:0/1617772035 >> 172.21.15.2:0/1617772035 conn(0x564168edf350 msgr2=0x564168fbf5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:59.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.092+0000 7f48effff700 0 client.15999 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:00:59.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.092+0000 7f4921927b80 1 -- 172.21.15.2:0/1617772035 shutdown_connections 2024-05-14T17:00:59.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.092+0000 7f4921927b80 1 -- 172.21.15.2:0/1617772035 wait complete. 2024-05-14T17:00:59.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.092+0000 7f4921927b80 20 client.15999 trim_cache size 0 max 16384 2024-05-14T17:00:59.534 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T17:00:59.540 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 38, 2024-05-14T17:00:59.547 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24479, 2024-05-14T17:00:59.554 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 20, 2024-05-14T17:00:59.562 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-05-14T17:00:59.568 INFO:teuthology.orchestra.run.smithi002.stdout: "bf03fc11": { 2024-05-14T17:00:59.582 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 20, 2024-05-14T17:00:59.602 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-05-14T17:00:59.609 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-05-14T17:00:59.622 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-05-14T17:00:59.628 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-05-14T17:00:59.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:00:59.643 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-05-14T17:00:59.649 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 133.7, 2024-05-14T17:00:59.655 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-05-14T17:00:59.662 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-05-14T17:00:59.669 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-05-14T17:00:59.676 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-05-14T17:00:59.683 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-05-14T17:00:59.690 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-05-14T17:00:59.696 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:00:59.703 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:00:59.710 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:00:59.720 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:00:59.746 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:00:59.754 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T17:00:59.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.093+0000 7f4917091700 1 -- 172.21.15.2:0/1222554944 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f48fc0b29f0 msgr2=0x7f48fc0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:59.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.093+0000 7f4917091700 1 --2- 172.21.15.2:0/1222554944 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f48fc0b29f0 0x7f48fc0b4eb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4910079ce0 tx=0x7f48f8009190 comp rx=0 tx=0).stop 2024-05-14T17:00:59.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.093+0000 7f4917091700 1 -- 172.21.15.2:0/1222554944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4910102460 msgr2=0x7f49100ff490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:00:59.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.093+0000 7f4917091700 1 --2- 172.21.15.2:0/1222554944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4910102460 0x7f49100ff490 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f490000ba50 tx=0x7f490000bd60 comp rx=0 tx=0).stop 2024-05-14T17:00:59.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.093+0000 7f4917091700 1 -- 172.21.15.2:0/1222554944 shutdown_connections 2024-05-14T17:00:59.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.093+0000 7f4917091700 1 --2- 172.21.15.2:0/1222554944 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f48fc0b29f0 0x7f48fc0b4eb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:59.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.093+0000 7f4917091700 1 --2- 172.21.15.2:0/1222554944 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49101084f0 0x7f49100fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:59.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.093+0000 7f4917091700 1 --2- 172.21.15.2:0/1222554944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4910102460 0x7f49100ff490 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:59.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.093+0000 7f4917091700 1 --2- 172.21.15.2:0/1222554944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4910102e20 0x7f49100ff9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:00:59.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.093+0000 7f4917091700 1 -- 172.21.15.2:0/1222554944 >> 172.21.15.2:0/1222554944 conn(0x7f49100794e0 msgr2=0x7f491006c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:00:59.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.094+0000 7f4917091700 1 -- 172.21.15.2:0/1222554944 shutdown_connections 2024-05-14T17:00:59.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:00:54.094+0000 7f4917091700 1 -- 172.21.15.2:0/1222554944 wait complete. 2024-05-14T17:00:59.854 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:00:54 smithi002 ceph-mon[31538]: pgmap v189: 97 pgs: 97 active+clean; 3.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.3 MiB/s wr, 392 op/s 2024-05-14T17:00:59.863 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:00:55 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:00:59.879 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:00:56 smithi002 ceph-mon[31538]: pgmap v190: 97 pgs: 97 active+clean; 3.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 799 KiB/s wr, 87 op/s 2024-05-14T17:00:59.921 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:00:56 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:00:59.929 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:00:56 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:00:59.947 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:00:56 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:00:59.956 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 340.0 s, 1753.5 tps, lat 165.585 ms stddev 1257.193 2024-05-14T17:00:59.965 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:00:58 smithi002 ceph-mon[31538]: pgmap v191: 97 pgs: 97 active+clean; 3.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 1.4 MiB/s wr, 143 op/s 2024-05-14T17:00:59.974 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'bf03fc11', total seconds elapsed: 25.0 2024-05-14T17:00:59.983 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:00:58 smithi123 ceph-mon[35971]: pgmap v191: 97 pgs: 97 active+clean; 3.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 1.4 MiB/s wr, 143 op/s 2024-05-14T17:00:59.992 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:00:58 smithi151 ceph-mon[36066]: pgmap v191: 97 pgs: 97 active+clean; 3.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 1.4 MiB/s wr, 143 op/s 2024-05-14T17:01:00.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:01:00 smithi002 ceph-mon[31538]: pgmap v192: 97 pgs: 97 active+clean; 3.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 3.1 MiB/s wr, 253 op/s 2024-05-14T17:01:00.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:01:00 smithi151 ceph-mon[36066]: pgmap v192: 97 pgs: 97 active+clean; 3.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 3.1 MiB/s wr, 253 op/s 2024-05-14T17:01:00.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:01:00 smithi123 ceph-mon[35971]: pgmap v192: 97 pgs: 97 active+clean; 3.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 3.1 MiB/s wr, 253 op/s 2024-05-14T17:01:01.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 345.0 s, 2135.4 tps, lat 45.174 ms stddev 68.675 2024-05-14T17:01:02.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:01:02 smithi002 ceph-mon[31538]: pgmap v193: 97 pgs: 97 active+clean; 3.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 2.8 MiB/s wr, 223 op/s 2024-05-14T17:01:02.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:01:02 smithi151 ceph-mon[36066]: pgmap v193: 97 pgs: 97 active+clean; 3.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 2.8 MiB/s wr, 223 op/s 2024-05-14T17:01:02.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:01:02 smithi123 ceph-mon[35971]: pgmap v193: 97 pgs: 97 active+clean; 3.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 2.8 MiB/s wr, 223 op/s 2024-05-14T17:01:04.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:01:04 smithi002 ceph-mon[31538]: pgmap v194: 97 pgs: 97 active+clean; 3.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.8 MiB/s wr, 520 op/s 2024-05-14T17:01:04.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:01:04 smithi151 ceph-mon[36066]: pgmap v194: 97 pgs: 97 active+clean; 3.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.8 MiB/s wr, 520 op/s 2024-05-14T17:01:04.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:01:04 smithi123 ceph-mon[35971]: pgmap v194: 97 pgs: 97 active+clean; 3.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.8 MiB/s wr, 520 op/s 2024-05-14T17:01:05.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:01:05 smithi002 ceph-mon[31538]: osdmap e82: 12 total, 12 up, 12 in 2024-05-14T17:01:05.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:01:05 smithi151 ceph-mon[36066]: osdmap e82: 12 total, 12 up, 12 in 2024-05-14T17:01:05.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:01:05 smithi123 ceph-mon[35971]: osdmap e82: 12 total, 12 up, 12 in 2024-05-14T17:01:06.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.733+0000 7fc760807640 1 -- 172.21.15.2:0/2783612890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc75c109c80 msgr2=0x7fc75c10dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:06.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.733+0000 7fc760807640 1 --2- 172.21.15.2:0/2783612890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc75c109c80 0x7fc75c10dcc0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc750009900 tx=0x7fc75002a410 comp rx=0 tx=0).stop 2024-05-14T17:01:06.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.733+0000 7fc760807640 1 -- 172.21.15.2:0/2783612890 shutdown_connections 2024-05-14T17:01:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.733+0000 7fc760807640 1 --2- 172.21.15.2:0/2783612890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc75c10e3a0 0x7fc75c115f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.733+0000 7fc760807640 1 --2- 172.21.15.2:0/2783612890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc75c109c80 0x7fc75c10dcc0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.733+0000 7fc760807640 1 --2- 172.21.15.2:0/2783612890 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc75c1092d0 0x7fc75c1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.733+0000 7fc760807640 1 -- 172.21.15.2:0/2783612890 >> 172.21.15.2:0/2783612890 conn(0x7fc75c076fd0 msgr2=0x7fc75c0793f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:06.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.733+0000 7fc760807640 1 -- 172.21.15.2:0/2783612890 shutdown_connections 2024-05-14T17:01:06.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.733+0000 7fc760807640 1 -- 172.21.15.2:0/2783612890 wait complete. 2024-05-14T17:01:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.734+0000 7fc760807640 1 Processor -- start 2024-05-14T17:01:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.734+0000 7fc760807640 1 -- start start 2024-05-14T17:01:06.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.734+0000 7fc760807640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc75c1092d0 0x7fc75c1a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.734+0000 7fc760807640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc75c10e3a0 0x7fc75c1a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:06.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.734+0000 7fc760807640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc75c19f650 0x7fc75c19fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.734+0000 7fc760807640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c119310 con 0x7fc75c19f650 2024-05-14T17:01:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.734+0000 7fc760807640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c119190 con 0x7fc75c1092d0 2024-05-14T17:01:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.734+0000 7fc760807640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c119490 con 0x7fc75c10e3a0 2024-05-14T17:01:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.734+0000 7fc75a59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc75c1092d0 0x7fc75c1a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.734+0000 7fc75ad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc75c19f650 0x7fc75c19fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:06.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.734+0000 7fc75a59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc75c1092d0 0x7fc75c1a5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:47392/0 (socket says 172.21.15.2:47392) 2024-05-14T17:01:06.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.734+0000 7fc75a59c640 1 -- 172.21.15.2:0/1945467046 learned_addr learned my addr 172.21.15.2:0/1945467046 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:01:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.734+0000 7fc759d9b640 1 --2- 172.21.15.2:0/1945467046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc75c10e3a0 0x7fc75c1a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.734+0000 7fc75a59c640 1 -- 172.21.15.2:0/1945467046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc75c10e3a0 msgr2=0x7fc75c1a5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:07.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.734+0000 7fc75a59c640 1 --2- 172.21.15.2:0/1945467046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc75c10e3a0 0x7fc75c1a5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.735+0000 7fc75a59c640 1 -- 172.21.15.2:0/1945467046 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc75c19f650 msgr2=0x7fc75c19fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.735+0000 7fc75a59c640 1 --2- 172.21.15.2:0/1945467046 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc75c19f650 0x7fc75c19fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.735+0000 7fc75a59c640 1 -- 172.21.15.2:0/1945467046 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc75c1a0170 con 0x7fc75c1092d0 2024-05-14T17:01:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.735+0000 7fc75ad9d640 1 --2- 172.21.15.2:0/1945467046 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc75c19f650 0x7fc75c19fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:01:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.735+0000 7fc75a59c640 1 --2- 172.21.15.2:0/1945467046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc75c1092d0 0x7fc75c1a5450 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc74c00ea60 tx=0x7fc74c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.735+0000 7fc74b7fe640 1 -- 172.21.15.2:0/1945467046 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c00cde0 con 0x7fc75c1092d0 2024-05-14T17:01:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.735+0000 7fc760807640 1 -- 172.21.15.2:0/1945467046 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc75c1a0460 con 0x7fc75c1092d0 2024-05-14T17:01:07.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.735+0000 7fc74b7fe640 1 -- 172.21.15.2:0/1945467046 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc74c0035f0 con 0x7fc75c1092d0 2024-05-14T17:01:07.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.735+0000 7fc74b7fe640 1 -- 172.21.15.2:0/1945467046 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c010690 con 0x7fc75c1092d0 2024-05-14T17:01:07.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.735+0000 7fc760807640 1 -- 172.21.15.2:0/1945467046 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc75c1ac6b0 con 0x7fc75c1092d0 2024-05-14T17:01:07.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.738+0000 7fc74b7fe640 1 -- 172.21.15.2:0/1945467046 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fc74c0037a0 con 0x7fc75c1092d0 2024-05-14T17:01:07.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.739+0000 7fc74b7fe640 1 --2- 172.21.15.2:0/1945467046 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc7440b2a10 0x7fc7440b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.739+0000 7fc7635f7740 1 Processor -- start 2024-05-14T17:01:07.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.739+0000 7fc759d9b640 1 --2- 172.21.15.2:0/1945467046 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc7440b2a10 0x7fc7440b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.739+0000 7fc74b7fe640 1 -- 172.21.15.2:0/1945467046 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8557+0+0 (secure 0 0 0) 0x7fc74c0e28c0 con 0x7fc75c1092d0 2024-05-14T17:01:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.739+0000 7fc7635f7740 1 -- start start 2024-05-14T17:01:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.739+0000 7fc7635f7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633bee318f0 0x5633bee31cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.739+0000 7fc7635f7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5633bee32210 0x5633bee3c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:07.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.739+0000 7fc7635f7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bee3cad0 0x5633bee3eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.739+0000 7fc7635f7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5633beda4c30 con 0x5633bee32210 2024-05-14T17:01:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.739+0000 7fc7635f7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5633bed947d0 con 0x5633bee318f0 2024-05-14T17:01:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.739+0000 7fc7635f7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5633bed98810 con 0x5633bee3cad0 2024-05-14T17:01:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.740+0000 7fc759d9b640 1 --2- 172.21.15.2:0/1945467046 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc7440b2a10 0x7fc7440b4ed0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc750031440 tx=0x7fc750005a20 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.740+0000 7fc75a59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633bee318f0 0x5633bee31cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.740+0000 7fc759d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5633bee32210 0x5633bee3c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:07.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.740+0000 7fc75a59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633bee318f0 0x5633bee31cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:47398/0 (socket says 172.21.15.2:47398) 2024-05-14T17:01:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.740+0000 7fc75a59c640 1 -- 172.21.15.2:0/3277497353 learned_addr learned my addr 172.21.15.2:0/3277497353 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:01:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.740+0000 7fc75ad9d640 1 --2- 172.21.15.2:0/3277497353 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bee3cad0 0x5633bee3eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.740+0000 7fc75a59c640 1 -- 172.21.15.2:0/3277497353 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bee3cad0 msgr2=0x5633bee3eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:07.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.740+0000 7fc75a59c640 1 --2- 172.21.15.2:0/3277497353 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bee3cad0 0x5633bee3eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.740+0000 7fc75a59c640 1 -- 172.21.15.2:0/3277497353 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5633bee32210 msgr2=0x5633bee3c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:07.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.740+0000 7fc75a59c640 1 --2- 172.21.15.2:0/3277497353 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5633bee32210 0x5633bee3c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:07.384 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.740+0000 7fc75a59c640 1 -- 172.21.15.2:0/3277497353 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633beaa9210 con 0x5633bee318f0 2024-05-14T17:01:07.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.740+0000 7fc75a59c640 1 --2- 172.21.15.2:0/3277497353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633bee318f0 0x5633bee31cd0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc74c0a9880 tx=0x7fc74c0a9590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=700db77168d7d67e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:07.418 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.740+0000 7fc748ff9640 1 -- 172.21.15.2:0/3277497353 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c0b2330 con 0x5633bee318f0 2024-05-14T17:01:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.740+0000 7fc748ff9640 1 -- 172.21.15.2:0/3277497353 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc74c0aa280 con 0x5633bee318f0 2024-05-14T17:01:07.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.740+0000 7fc748ff9640 1 -- 172.21.15.2:0/3277497353 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c004000 con 0x5633bee318f0 2024-05-14T17:01:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.741+0000 7fc7635f7740 1 -- 172.21.15.2:0/3277497353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633bee318f0 msgr2=0x5633bee31cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.741+0000 7fc7635f7740 1 --2- 172.21.15.2:0/3277497353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633bee318f0 0x5633bee31cd0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc74c0a9880 tx=0x7fc74c0a9590 comp rx=0 tx=0).stop 2024-05-14T17:01:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.741+0000 7fc7635f7740 1 -- 172.21.15.2:0/3277497353 shutdown_connections 2024-05-14T17:01:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.741+0000 7fc7635f7740 1 --2- 172.21.15.2:0/3277497353 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bee3cad0 0x5633bee3eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:07.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.741+0000 7fc7635f7740 1 --2- 172.21.15.2:0/3277497353 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5633bee32210 0x5633bee3c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.741+0000 7fc7635f7740 1 --2- 172.21.15.2:0/3277497353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633bee318f0 0x5633bee31cd0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.741+0000 7fc7635f7740 1 -- 172.21.15.2:0/3277497353 >> 172.21.15.2:0/3277497353 conn(0x5633bee2de00 msgr2=0x5633bee2e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.741+0000 7fc7635f7740 1 -- 172.21.15.2:0/3277497353 shutdown_connections 2024-05-14T17:01:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.742+0000 7fc7635f7740 1 -- 172.21.15.2:0/3277497353 wait complete. 2024-05-14T17:01:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.742+0000 7fc7635f7740 1 -- start start 2024-05-14T17:01:07.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.742+0000 7fc7635f7740 1 Processor -- start 2024-05-14T17:01:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.742+0000 7fc7635f7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5633bee318f0 0x5633bee3c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.742+0000 7fc7635f7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633bee32210 0x5633bee397b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.742+0000 7fc7635f7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bee3cad0 0x5633bee39dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.743+0000 7fc75a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5633bee318f0 0x5633bee3c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:07.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.743+0000 7fc7635f7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5633bea7a790 con 0x5633bee318f0 2024-05-14T17:01:07.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.743+0000 7fc7635f7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5633bed50890 con 0x5633bee32210 2024-05-14T17:01:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.743+0000 7fc7635f7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5633bee3f7a0 con 0x5633bee3cad0 2024-05-14T17:01:07.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.743+0000 7fc75a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5633bee318f0 0x5633bee3c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44890/0 (socket says 172.21.15.2:44890) 2024-05-14T17:01:07.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.743+0000 7fc75a59c640 1 -- 172.21.15.2:0/2448383872 learned_addr learned my addr 172.21.15.2:0/2448383872 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:01:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.743+0000 7fc759d9b640 1 --2- 172.21.15.2:0/2448383872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633bee32210 0x5633bee397b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.743+0000 7fc75ad9d640 1 --2- 172.21.15.2:0/2448383872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bee3cad0 0x5633bee39dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.743+0000 7fc75a59c640 1 -- 172.21.15.2:0/2448383872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bee3cad0 msgr2=0x5633bee39dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.743+0000 7fc75a59c640 1 --2- 172.21.15.2:0/2448383872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bee3cad0 0x5633bee39dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:07.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.743+0000 7fc75a59c640 1 -- 172.21.15.2:0/2448383872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633bee32210 msgr2=0x5633bee397b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.743+0000 7fc75a59c640 1 --2- 172.21.15.2:0/2448383872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633bee32210 0x5633bee397b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.743+0000 7fc75a59c640 1 -- 172.21.15.2:0/2448383872 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633beb75620 con 0x5633bee318f0 2024-05-14T17:01:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.743+0000 7fc75ad9d640 1 --2- 172.21.15.2:0/2448383872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bee3cad0 0x5633bee39dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:01:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.743+0000 7fc7635f7740 10 client.0 _ll_register_callbacks cb 0x5633bed53d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:01:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.743+0000 7fc7635f7740 1 -- 172.21.15.2:0/2448383872 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc74c0aa280 con 0x5633bee318f0 2024-05-14T17:01:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.743+0000 7fc75a59c640 1 --2- 172.21.15.2:0/2448383872 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5633bee318f0 0x5633bee3c4c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc74c00c760 tx=0x7fc74c0a9600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e311ae83af8dc75 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.743+0000 7fc741ffb640 10 client.16011 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:01:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.743+0000 7fc741ffb640 1 -- 172.21.15.2:0/2448383872 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c0c8020 con 0x5633bee318f0 2024-05-14T17:01:07.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.743+0000 7fc741ffb640 1 -- 172.21.15.2:0/2448383872 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8557+0+0 (secure 0 0 0) 0x7fc74c004000 con 0x5633bee318f0 2024-05-14T17:01:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.743+0000 7fc741ffb640 1 -- 172.21.15.2:0/2448383872 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fc74c0c4050 con 0x5633bee318f0 2024-05-14T17:01:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.743+0000 7fc7635f7740 10 client.16011 fetch_fsmap learned FSMap version 38 2024-05-14T17:01:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.743+0000 7fc7635f7740 1 -- 172.21.15.2:0/2448383872 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({fsmap=38}) -- 0x5633bee4afb0 con 0x5633bee318f0 2024-05-14T17:01:07.847 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:01:06 smithi002 ceph-mon[31538]: pgmap v196: 97 pgs: 97 active+clean; 3.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.9 MiB/s wr, 616 op/s 2024-05-14T17:01:07.861 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:01:06 smithi123 ceph-mon[35971]: pgmap v196: 97 pgs: 97 active+clean; 3.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.9 MiB/s wr, 616 op/s 2024-05-14T17:01:07.868 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:01:06 smithi151 ceph-mon[36066]: pgmap v196: 97 pgs: 97 active+clean; 3.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.9 MiB/s wr, 616 op/s 2024-05-14T17:01:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.744+0000 7fc741ffb640 1 -- 172.21.15.2:0/2448383872 <== mon.0 v2:172.21.15.2:3300/0 4 ==== fsmap(e 38) ==== 8342+0+0 (secure 0 0 0) 0x7fc74c0a8de0 con 0x5633bee318f0 2024-05-14T17:01:07.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.744+0000 7fc7635f7740 10 client.16011 fetch_fsmap finished waiting for FSMap version 38 2024-05-14T17:01:07.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.744+0000 7fc7635f7740 10 client.16011 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:01:07.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.744+0000 7fc7635f7740 20 client.16011 populate_metadata read hostname 'smithi002' 2024-05-14T17:01:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.744+0000 7fc7635f7740 1 --2- 172.21.15.2:0/2448383872 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5633bee4c5c0 0x5633bee4ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:07.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.744+0000 7fc7635f7740 4 client.16011 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:01:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.744+0000 7fc7635f7740 1 -- 172.21.15.2:0/2448383872 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633beda6b80 con 0x5633bee4c5c0 2024-05-14T17:01:07.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.744+0000 7fc759d9b640 1 --2- 172.21.15.2:0/2448383872 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5633bee4c5c0 0x5633bee4ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:07.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.745+0000 7fc759d9b640 1 --2- 172.21.15.2:0/2448383872 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5633bee4c5c0 0x5633bee4ea80 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=593fe070fc5215cf server_cookie=d16c74aa89106017 in_seq=0 out_seq=0 2024-05-14T17:01:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.745+0000 7fc741ffb640 10 client.16011 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:01:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.745+0000 7fc741ffb640 1 -- 172.21.15.2:0/2448383872 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5633beda6b80 con 0x5633bee4c5c0 2024-05-14T17:01:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.745+0000 7fc741ffb640 10 client.16011 handle_command_reply: tid=0 2024-05-14T17:01:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.745+0000 7fc7635f7740 1 client.16011 shutdown 2024-05-14T17:01:08.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.746+0000 7fc7635f7740 1 -- 172.21.15.2:0/2448383872 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5633bee318f0 msgr2=0x5633bee3c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.746+0000 7fc7635f7740 1 --2- 172.21.15.2:0/2448383872 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5633bee318f0 0x5633bee3c4c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc74c00c760 tx=0x7fc74c0a9600 comp rx=0 tx=0).stop 2024-05-14T17:01:08.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.746+0000 7fc7635f7740 1 -- 172.21.15.2:0/2448383872 shutdown_connections 2024-05-14T17:01:08.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.746+0000 7fc7635f7740 1 --2- 172.21.15.2:0/2448383872 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5633bee4c5c0 0x5633bee4ea80 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:08.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.746+0000 7fc7635f7740 1 --2- 172.21.15.2:0/2448383872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bee3cad0 0x5633bee39dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.746+0000 7fc741ffb640 0 client.16011 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:01:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.746+0000 7fc7635f7740 1 --2- 172.21.15.2:0/2448383872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633bee32210 0x5633bee397b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:08.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.746+0000 7fc7635f7740 1 --2- 172.21.15.2:0/2448383872 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5633bee318f0 0x5633bee3c4c0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.746+0000 7fc7635f7740 1 -- 172.21.15.2:0/2448383872 >> 172.21.15.2:0/2448383872 conn(0x5633bee2de00 msgr2=0x5633bee2d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:08.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.746+0000 7fc7635f7740 1 -- 172.21.15.2:0/2448383872 shutdown_connections 2024-05-14T17:01:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.747+0000 7fc7635f7740 1 -- 172.21.15.2:0/2448383872 wait complete. 2024-05-14T17:01:08.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.747+0000 7fc7635f7740 20 client.16011 trim_cache size 0 max 16384 2024-05-14T17:01:08.156 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 350.0 s, 328.4 tps, lat 304.780 ms stddev 183.875 2024-05-14T17:01:08.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.779+0000 7fc7635f7740 1 Processor -- start 2024-05-14T17:01:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.779+0000 7fc7635f7740 1 -- start start 2024-05-14T17:01:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.780+0000 7fc7635f7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bee318f0 0x5633bee48670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:08.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.780+0000 7fc7635f7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633bee32210 0x5633bee3bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.780+0000 7fc7635f7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5633bee3cad0 0x5633bee54390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.780+0000 7fc7635f7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c1b76a0 con 0x5633bee3cad0 2024-05-14T17:01:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.780+0000 7fc7635f7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c1b4f40 con 0x5633bee32210 2024-05-14T17:01:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.780+0000 7fc7635f7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c1b2db0 con 0x5633bee318f0 2024-05-14T17:01:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.780+0000 7fc75ad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5633bee3cad0 0x5633bee54390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:08.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.780+0000 7fc75ad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5633bee3cad0 0x5633bee54390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44892/0 (socket says 172.21.15.2:44892) 2024-05-14T17:01:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.780+0000 7fc759d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633bee32210 0x5633bee3bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:08.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.780+0000 7fc75ad9d640 1 -- 172.21.15.2:0/2036828867 learned_addr learned my addr 172.21.15.2:0/2036828867 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:01:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.780+0000 7fc75a59c640 1 --2- 172.21.15.2:0/2036828867 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bee318f0 0x5633bee48670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:08.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.780+0000 7fc759d9b640 1 -- 172.21.15.2:0/2036828867 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bee318f0 msgr2=0x5633bee48670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.780+0000 7fc759d9b640 1 --2- 172.21.15.2:0/2036828867 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bee318f0 0x5633bee48670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.780+0000 7fc759d9b640 1 -- 172.21.15.2:0/2036828867 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5633bee3cad0 msgr2=0x5633bee54390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.780+0000 7fc759d9b640 1 --2- 172.21.15.2:0/2036828867 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5633bee3cad0 0x5633bee54390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.780+0000 7fc759d9b640 1 -- 172.21.15.2:0/2036828867 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633bee3a130 con 0x5633bee32210 2024-05-14T17:01:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.780+0000 7fc75ad9d640 1 --2- 172.21.15.2:0/2036828867 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5633bee3cad0 0x5633bee54390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:01:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.781+0000 7fc759d9b640 1 --2- 172.21.15.2:0/2036828867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633bee32210 0x5633bee3bff0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc75002a930 tx=0x7fc750040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c94608758495f169 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.781+0000 7fc743fff640 1 -- 172.21.15.2:0/2036828867 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750035040 con 0x5633bee32210 2024-05-14T17:01:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.781+0000 7fc743fff640 1 -- 172.21.15.2:0/2036828867 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7500537f0 con 0x5633bee32210 2024-05-14T17:01:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.781+0000 7fc743fff640 1 -- 172.21.15.2:0/2036828867 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750004e70 con 0x5633bee32210 2024-05-14T17:01:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.782+0000 7fc7635f7740 1 -- 172.21.15.2:0/2036828867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633bee32210 msgr2=0x5633bee3bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.782+0000 7fc7635f7740 1 --2- 172.21.15.2:0/2036828867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633bee32210 0x5633bee3bff0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc75002a930 tx=0x7fc750040000 comp rx=0 tx=0).stop 2024-05-14T17:01:08.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.782+0000 7fc7635f7740 1 -- 172.21.15.2:0/2036828867 shutdown_connections 2024-05-14T17:01:08.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.782+0000 7fc7635f7740 1 --2- 172.21.15.2:0/2036828867 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5633bee3cad0 0x5633bee54390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.782+0000 7fc7635f7740 1 --2- 172.21.15.2:0/2036828867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633bee32210 0x5633bee3bff0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.782+0000 7fc7635f7740 1 --2- 172.21.15.2:0/2036828867 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bee318f0 0x5633bee48670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.782+0000 7fc7635f7740 1 -- 172.21.15.2:0/2036828867 >> 172.21.15.2:0/2036828867 conn(0x5633bee2de00 msgr2=0x5633bee33b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.782+0000 7fc7635f7740 1 -- 172.21.15.2:0/2036828867 shutdown_connections 2024-05-14T17:01:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.782+0000 7fc7635f7740 1 -- 172.21.15.2:0/2036828867 wait complete. 2024-05-14T17:01:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.782+0000 7fc7635f7740 1 -- start start 2024-05-14T17:01:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.782+0000 7fc7635f7740 1 Processor -- start 2024-05-14T17:01:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.783+0000 7fc7635f7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5633bee318f0 0x5633bee44f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.783+0000 7fc7635f7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633bee32210 0x5633bee40280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.783+0000 7fc7635f7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bee3cad0 0x5633bee407c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.783+0000 7fc7635f7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c1b3400 con 0x5633bee318f0 2024-05-14T17:01:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.783+0000 7fc7635f7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c1b4870 con 0x5633bee32210 2024-05-14T17:01:08.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.783+0000 7fc7635f7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c1b3ad0 con 0x5633bee3cad0 2024-05-14T17:01:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.783+0000 7fc7635f7740 10 client.0 _ll_register_callbacks cb 0x7fc74c0a9950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:01:08.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.783+0000 7fc75a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5633bee318f0 0x5633bee44f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:08.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.783+0000 7fc75a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5633bee318f0 0x5633bee44f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44908/0 (socket says 172.21.15.2:44908) 2024-05-14T17:01:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.783+0000 7fc75a59c640 1 -- 172.21.15.2:0/2134643722 learned_addr learned my addr 172.21.15.2:0/2134643722 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:01:08.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.783+0000 7fc759d9b640 1 --2- 172.21.15.2:0/2134643722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633bee32210 0x5633bee40280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:08.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.783+0000 7fc75ad9d640 1 --2- 172.21.15.2:0/2134643722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bee3cad0 0x5633bee407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:08.664 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.783+0000 7fc75a59c640 1 -- 172.21.15.2:0/2134643722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bee3cad0 msgr2=0x5633bee407c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.783+0000 7fc75a59c640 1 --2- 172.21.15.2:0/2134643722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bee3cad0 0x5633bee407c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:08.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.783+0000 7fc75a59c640 1 -- 172.21.15.2:0/2134643722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633bee32210 msgr2=0x5633bee40280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:08.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.783+0000 7fc75a59c640 1 --2- 172.21.15.2:0/2134643722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633bee32210 0x5633bee40280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:08.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.783+0000 7fc75a59c640 1 -- 172.21.15.2:0/2134643722 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633bee40fd0 con 0x5633bee318f0 2024-05-14T17:01:08.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.783+0000 7fc759d9b640 1 --2- 172.21.15.2:0/2134643722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633bee32210 0x5633bee40280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:01:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.783+0000 7fc75a59c640 1 --2- 172.21.15.2:0/2134643722 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5633bee318f0 0x5633bee44f10 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc74c0a5c00 tx=0x7fc74c0a9590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ecebee7339935c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.783+0000 7fc741ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:01:08.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.784+0000 7fc741ffb640 1 -- 172.21.15.2:0/2134643722 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c0c8050 con 0x5633bee318f0 2024-05-14T17:01:08.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.784+0000 7fc741ffb640 1 -- 172.21.15.2:0/2134643722 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8557+0+0 (secure 0 0 0) 0x7fc74c004000 con 0x5633bee318f0 2024-05-14T17:01:08.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.784+0000 7fc7635f7740 1 -- 172.21.15.2:0/2134643722 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7500537f0 con 0x5633bee318f0 2024-05-14T17:01:08.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.784+0000 7fc741ffb640 1 -- 172.21.15.2:0/2134643722 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fc74c0a5310 con 0x5633bee318f0 2024-05-14T17:01:08.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.784+0000 7fc7635f7740 10 client.16017 fetch_fsmap learned FSMap version 38 2024-05-14T17:01:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.784+0000 7fc7635f7740 1 -- 172.21.15.2:0/2134643722 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5633bee40fd0 con 0x5633bee318f0 2024-05-14T17:01:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.784+0000 7fc741ffb640 1 -- 172.21.15.2:0/2134643722 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7500537f0 con 0x5633bee318f0 2024-05-14T17:01:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.784+0000 7fc741ffb640 1 -- 172.21.15.2:0/2134643722 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) ==== 8342+0+0 (secure 0 0 0) 0x7fc74c0a8850 con 0x5633bee318f0 2024-05-14T17:01:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.784+0000 7fc7635f7740 10 client.16017 fetch_fsmap finished waiting for FSMap version 38 2024-05-14T17:01:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.784+0000 7fc7635f7740 10 client.16017 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:01:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.784+0000 7fc7635f7740 20 client.16017 populate_metadata read hostname 'smithi002' 2024-05-14T17:01:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.784+0000 7fc7635f7740 1 --2- 172.21.15.2:0/2134643722 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5633bee4c5c0 0x5633beeb9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:08.919 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.784+0000 7fc7635f7740 4 client.16017 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:01:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.784+0000 7fc7635f7740 1 -- 172.21.15.2:0/2134643722 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5633beda0360 con 0x5633bee4c5c0 2024-05-14T17:01:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.784+0000 7fc759d9b640 1 --2- 172.21.15.2:0/2134643722 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5633bee4c5c0 0x5633beeb9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.785+0000 7fc741ffb640 10 client.16017 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:01:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.785+0000 7fc759d9b640 1 --2- 172.21.15.2:0/2134643722 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5633bee4c5c0 0x5633beeb9290 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6e009d861cf18cc server_cookie=7b072e203cafbd56 in_seq=0 out_seq=0 2024-05-14T17:01:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.796+0000 7fc741ffb640 1 -- 172.21.15.2:0/2134643722 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5633beda0360 con 0x5633bee4c5c0 2024-05-14T17:01:08.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.796+0000 7fc741ffb640 10 client.16017 handle_command_reply: tid=0 2024-05-14T17:01:08.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.796+0000 7fc7635f7740 1 client.16017 shutdown 2024-05-14T17:01:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.797+0000 7fc7635f7740 1 -- 172.21.15.2:0/2134643722 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5633bee318f0 msgr2=0x5633bee44f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:08.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.797+0000 7fc7635f7740 1 --2- 172.21.15.2:0/2134643722 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5633bee318f0 0x5633bee44f10 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc74c0a5c00 tx=0x7fc74c0a9590 comp rx=0 tx=0).stop 2024-05-14T17:01:09.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.797+0000 7fc7635f7740 1 -- 172.21.15.2:0/2134643722 shutdown_connections 2024-05-14T17:01:09.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.797+0000 7fc7635f7740 1 --2- 172.21.15.2:0/2134643722 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5633bee4c5c0 0x5633beeb9290 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.797+0000 7fc7635f7740 1 --2- 172.21.15.2:0/2134643722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bee3cad0 0x5633bee407c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.797+0000 7fc7635f7740 1 --2- 172.21.15.2:0/2134643722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633bee32210 0x5633bee40280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:09.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.797+0000 7fc7635f7740 1 --2- 172.21.15.2:0/2134643722 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5633bee318f0 0x5633bee44f10 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.797+0000 7fc7635f7740 1 -- 172.21.15.2:0/2134643722 >> 172.21.15.2:0/2134643722 conn(0x5633bee2de00 msgr2=0x5633bee332a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.797+0000 7fc741ffb640 0 client.16017 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:01:09.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.797+0000 7fc7635f7740 1 -- 172.21.15.2:0/2134643722 shutdown_connections 2024-05-14T17:01:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.797+0000 7fc7635f7740 1 -- 172.21.15.2:0/2134643722 wait complete. 2024-05-14T17:01:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.797+0000 7fc7635f7740 20 client.16017 trim_cache size 0 max 16384 2024-05-14T17:01:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:01:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:01:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:01:09.142 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:01:08 smithi123 ceph-mon[35971]: pgmap v197: 97 pgs: 97 active+clean; 3.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.2 MiB/s wr, 549 op/s 2024-05-14T17:01:09.148 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:01:08 smithi151 ceph-mon[36066]: pgmap v197: 97 pgs: 97 active+clean; 3.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.2 MiB/s wr, 549 op/s 2024-05-14T17:01:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.797+0000 7fc760807640 1 -- 172.21.15.2:0/1945467046 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc7440b2a10 msgr2=0x7fc7440b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.798+0000 7fc760807640 1 --2- 172.21.15.2:0/1945467046 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc7440b2a10 0x7fc7440b4ed0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc750031440 tx=0x7fc750005a20 comp rx=0 tx=0).stop 2024-05-14T17:01:09.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.798+0000 7fc760807640 1 -- 172.21.15.2:0/1945467046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc75c1092d0 msgr2=0x7fc75c1a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:09.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.798+0000 7fc760807640 1 --2- 172.21.15.2:0/1945467046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc75c1092d0 0x7fc75c1a5450 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc74c00ea60 tx=0x7fc74c00ef20 comp rx=0 tx=0).stop 2024-05-14T17:01:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.798+0000 7fc760807640 1 -- 172.21.15.2:0/1945467046 shutdown_connections 2024-05-14T17:01:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.798+0000 7fc760807640 1 --2- 172.21.15.2:0/1945467046 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc7440b2a10 0x7fc7440b4ed0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.798+0000 7fc760807640 1 --2- 172.21.15.2:0/1945467046 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc75c19f650 0x7fc75c19fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:09.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.798+0000 7fc760807640 1 --2- 172.21.15.2:0/1945467046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc75c10e3a0 0x7fc75c1a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:09.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.798+0000 7fc760807640 1 --2- 172.21.15.2:0/1945467046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc75c1092d0 0x7fc75c1a5450 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.798+0000 7fc760807640 1 -- 172.21.15.2:0/1945467046 >> 172.21.15.2:0/1945467046 conn(0x7fc75c076fd0 msgr2=0x7fc75c079380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.798+0000 7fc760807640 1 -- 172.21.15.2:0/1945467046 shutdown_connections 2024-05-14T17:01:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:06.798+0000 7fc760807640 1 -- 172.21.15.2:0/1945467046 wait complete. 2024-05-14T17:01:09.294 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:01:08 smithi002 ceph-mon[31538]: pgmap v197: 97 pgs: 97 active+clean; 3.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.2 MiB/s wr, 549 op/s 2024-05-14T17:01:10.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:01:10 smithi002 ceph-mon[31538]: pgmap v198: 97 pgs: 97 active+clean; 3.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 MiB/s wr, 1.15k op/s 2024-05-14T17:01:10.752 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:01:10 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:01:10.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:01:10 smithi151 ceph-mon[36066]: pgmap v198: 97 pgs: 97 active+clean; 3.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 MiB/s wr, 1.15k op/s 2024-05-14T17:01:10.860 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:01:10 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:01:10.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:01:10 smithi123 ceph-mon[35971]: pgmap v198: 97 pgs: 97 active+clean; 3.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 MiB/s wr, 1.15k op/s 2024-05-14T17:01:10.915 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:01:10 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:01:11.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 355.0 s, 2337.4 tps, lat 44.271 ms stddev 141.738 2024-05-14T17:01:12.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:01:12 smithi002 ceph-mon[31538]: pgmap v199: 97 pgs: 97 active+clean; 3.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 MiB/s wr, 1.15k op/s 2024-05-14T17:01:12.765 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:01:12 smithi002 ceph-mon[31538]: osdmap e83: 12 total, 12 up, 12 in 2024-05-14T17:01:12.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:01:12 smithi151 ceph-mon[36066]: pgmap v199: 97 pgs: 97 active+clean; 3.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 MiB/s wr, 1.15k op/s 2024-05-14T17:01:12.860 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:01:12 smithi151 ceph-mon[36066]: osdmap e83: 12 total, 12 up, 12 in 2024-05-14T17:01:12.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:01:12 smithi123 ceph-mon[35971]: pgmap v199: 97 pgs: 97 active+clean; 3.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 MiB/s wr, 1.15k op/s 2024-05-14T17:01:12.916 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:01:12 smithi123 ceph-mon[35971]: osdmap e83: 12 total, 12 up, 12 in 2024-05-14T17:01:14.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:01:14 smithi002 ceph-mon[31538]: pgmap v201: 97 pgs: 97 active+clean; 3.8 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 39 MiB/s wr, 1.98k op/s 2024-05-14T17:01:14.760 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:01:14 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:01:14.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:01:14 smithi151 ceph-mon[36066]: pgmap v201: 97 pgs: 97 active+clean; 3.8 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 39 MiB/s wr, 1.98k op/s 2024-05-14T17:01:14.861 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:01:14 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:01:14.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:01:14 smithi123 ceph-mon[35971]: pgmap v201: 97 pgs: 97 active+clean; 3.8 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 39 MiB/s wr, 1.98k op/s 2024-05-14T17:01:14.917 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:01:14 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:01:16.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:01:16 smithi002 ceph-mon[31538]: pgmap v202: 97 pgs: 97 active+clean; 3.8 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 34 MiB/s wr, 1.76k op/s 2024-05-14T17:01:16.752 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 360.0 s, 3773.0 tps, lat 26.453 ms stddev 28.555 2024-05-14T17:01:16.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:01:16 smithi151 ceph-mon[36066]: pgmap v202: 97 pgs: 97 active+clean; 3.8 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 34 MiB/s wr, 1.76k op/s 2024-05-14T17:01:16.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:01:16 smithi123 ceph-mon[35971]: pgmap v202: 97 pgs: 97 active+clean; 3.8 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 34 MiB/s wr, 1.76k op/s 2024-05-14T17:01:18.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:01:18 smithi002 ceph-mon[31538]: pgmap v203: 97 pgs: 97 active+clean; 3.8 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 34 MiB/s wr, 1.76k op/s 2024-05-14T17:01:18.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:01:18 smithi151 ceph-mon[36066]: pgmap v203: 97 pgs: 97 active+clean; 3.8 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 34 MiB/s wr, 1.76k op/s 2024-05-14T17:01:18.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:01:18 smithi123 ceph-mon[35971]: pgmap v203: 97 pgs: 97 active+clean; 3.8 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 34 MiB/s wr, 1.76k op/s 2024-05-14T17:01:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.932+0000 7f8a4182a640 1 -- 172.21.15.2:0/433518349 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a3c102340 msgr2=0x7f8a3c10e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:19.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.932+0000 7f8a4182a640 1 --2- 172.21.15.2:0/433518349 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a3c102340 0x7f8a3c10e800 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8a34009cd0 tx=0x7f8a3402a8f0 comp rx=0 tx=0).stop 2024-05-14T17:01:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.932+0000 7f8a4182a640 1 -- 172.21.15.2:0/433518349 shutdown_connections 2024-05-14T17:01:19.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.932+0000 7f8a4182a640 1 --2- 172.21.15.2:0/433518349 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a3c102340 0x7f8a3c10e800 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.932+0000 7f8a4182a640 1 --2- 172.21.15.2:0/433518349 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a3c101980 0x7f8a3c101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.932+0000 7f8a4182a640 1 --2- 172.21.15.2:0/433518349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a3c107940 0x7f8a3c107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.932+0000 7f8a4182a640 1 -- 172.21.15.2:0/433518349 >> 172.21.15.2:0/433518349 conn(0x7f8a3c078ad0 msgr2=0x7f8a3c078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.932+0000 7f8a4182a640 1 -- 172.21.15.2:0/433518349 shutdown_connections 2024-05-14T17:01:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.932+0000 7f8a4182a640 1 -- 172.21.15.2:0/433518349 wait complete. 2024-05-14T17:01:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.933+0000 7f8a4182a640 1 Processor -- start 2024-05-14T17:01:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.933+0000 7f8a4182a640 1 -- start start 2024-05-14T17:01:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.933+0000 7f8a4182a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a3c101980 0x7f8a3c1980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.934+0000 7f8a4182a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a3c102340 0x7f8a3c198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:20.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.934+0000 7f8a4182a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a3c107940 0x7f8a3c19c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.934+0000 7f8a4182a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c111c00 con 0x7f8a3c101980 2024-05-14T17:01:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.934+0000 7f8a4182a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c111a80 con 0x7f8a3c107940 2024-05-14T17:01:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.934+0000 7f8a4182a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c111d80 con 0x7f8a3c102340 2024-05-14T17:01:20.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.934+0000 7f8a3affd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a3c101980 0x7f8a3c1980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:20.219 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.934+0000 7f8a3a7fc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a3c102340 0x7f8a3c198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.934+0000 7f8a3a7fc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a3c102340 0x7f8a3c198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:47630/0 (socket says 172.21.15.2:47630) 2024-05-14T17:01:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.934+0000 7f8a3affd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a3c101980 0x7f8a3c1980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:42372/0 (socket says 172.21.15.2:42372) 2024-05-14T17:01:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.934+0000 7f8a3a7fc640 1 -- 172.21.15.2:0/2041687171 learned_addr learned my addr 172.21.15.2:0/2041687171 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:01:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.935+0000 7f8a3b7fe640 1 --2- 172.21.15.2:0/2041687171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a3c107940 0x7f8a3c19c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.935+0000 7f8a3affd640 1 -- 172.21.15.2:0/2041687171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a3c102340 msgr2=0x7f8a3c198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.935+0000 7f8a3affd640 1 --2- 172.21.15.2:0/2041687171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a3c102340 0x7f8a3c198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:20.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.935+0000 7f8a3affd640 1 -- 172.21.15.2:0/2041687171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a3c107940 msgr2=0x7f8a3c19c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.935+0000 7f8a3affd640 1 --2- 172.21.15.2:0/2041687171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a3c107940 0x7f8a3c19c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:20.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.935+0000 7f8a3affd640 1 -- 172.21.15.2:0/2041687171 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a3c19d030 con 0x7f8a3c101980 2024-05-14T17:01:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.935+0000 7f8a3affd640 1 --2- 172.21.15.2:0/2041687171 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a3c101980 0x7f8a3c1980e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c00d6d0 tx=0x7f8a2c00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:20.382 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.936+0000 7f8a40828640 1 -- 172.21.15.2:0/2041687171 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c017070 con 0x7f8a3c101980 2024-05-14T17:01:20.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.936+0000 7f8a4182a640 1 -- 172.21.15.2:0/2041687171 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a3c19d320 con 0x7f8a3c101980 2024-05-14T17:01:20.419 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.936+0000 7f8a40828640 1 -- 172.21.15.2:0/2041687171 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a2c009820 con 0x7f8a3c101980 2024-05-14T17:01:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.936+0000 7f8a40828640 1 -- 172.21.15.2:0/2041687171 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c004c40 con 0x7f8a3c101980 2024-05-14T17:01:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.936+0000 7f8a4182a640 1 -- 172.21.15.2:0/2041687171 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a3c06a100 con 0x7f8a3c101980 2024-05-14T17:01:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.939+0000 7f8a44622740 1 Processor -- start 2024-05-14T17:01:20.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.940+0000 7f8a40828640 1 -- 172.21.15.2:0/2041687171 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f8a2c005220 con 0x7f8a3c101980 2024-05-14T17:01:20.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.940+0000 7f8a44622740 1 -- start start 2024-05-14T17:01:20.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.941+0000 7f8a44622740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b87182990 0x557b87182d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:20.503 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.941+0000 7f8a44622740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557b8717e400 0x557b8717e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.941+0000 7f8a44622740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b8717edc0 0x557b8718a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:20.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.941+0000 7f8a44622740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x557b870e7c30 con 0x557b8717e400 2024-05-14T17:01:20.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.941+0000 7f8a44622740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557b870d77d0 con 0x557b87182990 2024-05-14T17:01:20.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.941+0000 7f8a44622740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557b870db810 con 0x557b8717edc0 2024-05-14T17:01:20.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.941+0000 7f8a3a7fc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b8717edc0 0x557b8718a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:20.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.941+0000 7f8a3b7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b87182990 0x557b87182d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.941+0000 7f8a3a7fc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b8717edc0 0x557b8718a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:47644/0 (socket says 172.21.15.2:47644) 2024-05-14T17:01:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.941+0000 7f8a3a7fc640 1 -- 172.21.15.2:0/3692999964 learned_addr learned my addr 172.21.15.2:0/3692999964 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:01:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.941+0000 7f8a3a7fc640 1 -- 172.21.15.2:0/3692999964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b87182990 msgr2=0x557b87182d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.941+0000 7f8a3a7fc640 1 --2- 172.21.15.2:0/3692999964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b87182990 0x557b87182d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.941+0000 7f8a3a7fc640 1 -- 172.21.15.2:0/3692999964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557b8717e400 msgr2=0x557b8717e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.941+0000 7f8a3a7fc640 1 --2- 172.21.15.2:0/3692999964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557b8717e400 0x557b8717e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.941+0000 7f8a3a7fc640 1 -- 172.21.15.2:0/3692999964 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b86dec210 con 0x557b8717edc0 2024-05-14T17:01:20.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.942+0000 7f8a3a7fc640 1 --2- 172.21.15.2:0/3692999964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b8717edc0 0x557b8718a8e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8a3000bd60 tx=0x7f8a300107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f62641863a9a9301 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.942+0000 7f8a21ffb640 1 -- 172.21.15.2:0/3692999964 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a30011640 con 0x557b8717edc0 2024-05-14T17:01:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.942+0000 7f8a21ffb640 1 -- 172.21.15.2:0/3692999964 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a30011c90 con 0x557b8717edc0 2024-05-14T17:01:20.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.942+0000 7f8a21ffb640 1 -- 172.21.15.2:0/3692999964 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3001bc40 con 0x557b8717edc0 2024-05-14T17:01:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.943+0000 7f8a40828640 1 --2- 172.21.15.2:0/2041687171 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f8a240b2a10 0x7f8a240b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.943+0000 7f8a44622740 1 -- 172.21.15.2:0/3692999964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b8717edc0 msgr2=0x557b8718a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.943+0000 7f8a44622740 1 --2- 172.21.15.2:0/3692999964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b8717edc0 0x557b8718a8e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8a3000bd60 tx=0x7f8a300107a0 comp rx=0 tx=0).stop 2024-05-14T17:01:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.944+0000 7f8a3b7fe640 1 --2- 172.21.15.2:0/2041687171 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f8a240b2a10 0x7f8a240b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.944+0000 7f8a40828640 1 -- 172.21.15.2:0/2041687171 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7f8a2c0e20a0 con 0x7f8a3c101980 2024-05-14T17:01:20.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.944+0000 7f8a44622740 1 -- 172.21.15.2:0/3692999964 shutdown_connections 2024-05-14T17:01:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.944+0000 7f8a44622740 1 --2- 172.21.15.2:0/3692999964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b8717edc0 0x557b8718a8e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.944+0000 7f8a44622740 1 --2- 172.21.15.2:0/3692999964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557b8717e400 0x557b8717e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.944+0000 7f8a44622740 1 --2- 172.21.15.2:0/3692999964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b87182990 0x557b87182d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.944+0000 7f8a44622740 1 -- 172.21.15.2:0/3692999964 >> 172.21.15.2:0/3692999964 conn(0x557b871708b0 msgr2=0x557b87170c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:20.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.944+0000 7f8a44622740 1 -- 172.21.15.2:0/3692999964 shutdown_connections 2024-05-14T17:01:20.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.945+0000 7f8a44622740 1 -- 172.21.15.2:0/3692999964 wait complete. 2024-05-14T17:01:20.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.945+0000 7f8a3b7fe640 1 --2- 172.21.15.2:0/2041687171 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f8a240b2a10 0x7f8a240b4ed0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8a34006010 tx=0x7f8a3403d040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.945+0000 7f8a44622740 1 -- start start 2024-05-14T17:01:20.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.945+0000 7f8a44622740 1 Processor -- start 2024-05-14T17:01:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.946+0000 7f8a44622740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557b8717e400 0x557b8718a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.946+0000 7f8a44622740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b8717edc0 0x557b87187b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.946+0000 7f8a44622740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b87182990 0x557b871880b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.946+0000 7f8a44622740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x557b86dbd790 con 0x557b8717e400 2024-05-14T17:01:20.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.946+0000 7f8a44622740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557b87093890 con 0x557b87182990 2024-05-14T17:01:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.946+0000 7f8a44622740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557b8718af80 con 0x557b8717edc0 2024-05-14T17:01:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.946+0000 7f8a44622740 10 client.0 _ll_register_callbacks cb 0x557b87096d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:01:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.947+0000 7f8a3a7fc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b8717edc0 0x557b87187b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.947+0000 7f8a3a7fc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b8717edc0 0x557b87187b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:47656/0 (socket says 172.21.15.2:47656) 2024-05-14T17:01:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.947+0000 7f8a3a7fc640 1 -- 172.21.15.2:0/280964259 learned_addr learned my addr 172.21.15.2:0/280964259 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:01:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.947+0000 7f8a3b7fe640 1 --2- 172.21.15.2:0/280964259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b87182990 0x557b871880b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.947+0000 7f8a3affd640 1 --2- 172.21.15.2:0/280964259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557b8717e400 0x557b8718a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:20.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.947+0000 7f8a3a7fc640 1 -- 172.21.15.2:0/280964259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b87182990 msgr2=0x557b871880b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:20.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.947+0000 7f8a3a7fc640 1 --2- 172.21.15.2:0/280964259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b87182990 0x557b871880b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:20.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.947+0000 7f8a3a7fc640 1 -- 172.21.15.2:0/280964259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557b8717e400 msgr2=0x557b8718a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.947+0000 7f8a3a7fc640 1 --2- 172.21.15.2:0/280964259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557b8717e400 0x557b8718a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.947+0000 7f8a3a7fc640 1 -- 172.21.15.2:0/280964259 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b86eb8620 con 0x557b8717edc0 2024-05-14T17:01:20.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.947+0000 7f8a3affd640 1 --2- 172.21.15.2:0/280964259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557b8717e400 0x557b8718a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:01:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.948+0000 7f8a3a7fc640 1 --2- 172.21.15.2:0/280964259 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b8717edc0 0x557b87187b10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8a3001b740 tx=0x7f8a3001b660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70e1fa38445ef9de server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.948+0000 7f8a1affd640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-14T17:01:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.948+0000 7f8a1affd640 1 -- 172.21.15.2:0/280964259 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a30011e40 con 0x557b8717edc0 2024-05-14T17:01:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.948+0000 7f8a44622740 1 -- 172.21.15.2:0/280964259 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a30011c90 con 0x557b8717edc0 2024-05-14T17:01:21.033 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.948+0000 7f8a1affd640 1 -- 172.21.15.2:0/280964259 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7f8a3001bb50 con 0x557b8717edc0 2024-05-14T17:01:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.948+0000 7f8a1affd640 1 -- 172.21.15.2:0/280964259 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f8a3001bd50 con 0x557b8717edc0 2024-05-14T17:01:21.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.948+0000 7f8a44622740 10 client.25708 fetch_fsmap learned FSMap version 40 2024-05-14T17:01:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.948+0000 7f8a44622740 1 -- 172.21.15.2:0/280964259 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x557b87196a60 con 0x557b8717edc0 2024-05-14T17:01:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.948+0000 7f8a1affd640 1 -- 172.21.15.2:0/280964259 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a30011c90 con 0x557b8717edc0 2024-05-14T17:01:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.949+0000 7f8a1affd640 1 -- 172.21.15.2:0/280964259 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7f8a30011880 con 0x557b8717edc0 2024-05-14T17:01:21.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.949+0000 7f8a44622740 10 client.25708 fetch_fsmap finished waiting for FSMap version 40 2024-05-14T17:01:21.139 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.949+0000 7f8a44622740 10 client.25708 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:01:21.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.949+0000 7f8a44622740 20 client.25708 populate_metadata read hostname 'smithi002' 2024-05-14T17:01:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.950+0000 7f8a44622740 1 --2- 172.21.15.2:0/280964259 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x557b87198320 0x557b8719a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:21.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.950+0000 7f8a44622740 4 client.25708 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:01:21.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.950+0000 7f8a44622740 1 -- 172.21.15.2:0/280964259 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b870e9b80 con 0x557b87198320 2024-05-14T17:01:21.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.950+0000 7f8a3affd640 1 --2- 172.21.15.2:0/280964259 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x557b87198320 0x557b8719a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.950+0000 7f8a1affd640 10 client.25708 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:01:21.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.951+0000 7f8a3affd640 1 --2- 172.21.15.2:0/280964259 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x557b87198320 0x557b8719a780 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15559cf9fa13a5cd server_cookie=14108cab44c4c24f in_seq=0 out_seq=0 2024-05-14T17:01:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.952+0000 7f8a1affd640 1 -- 172.21.15.2:0/280964259 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557b870e9b80 con 0x557b87198320 2024-05-14T17:01:21.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.952+0000 7f8a1affd640 10 client.25708 handle_command_reply: tid=0 2024-05-14T17:01:21.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.952+0000 7f8a44622740 1 client.25708 shutdown 2024-05-14T17:01:21.250 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.952+0000 7f8a44622740 1 -- 172.21.15.2:0/280964259 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b8717edc0 msgr2=0x557b87187b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.952+0000 7f8a44622740 1 --2- 172.21.15.2:0/280964259 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b8717edc0 0x557b87187b10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8a3001b740 tx=0x7f8a3001b660 comp rx=0 tx=0).stop 2024-05-14T17:01:21.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.952+0000 7f8a44622740 1 -- 172.21.15.2:0/280964259 shutdown_connections 2024-05-14T17:01:21.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.952+0000 7f8a44622740 1 --2- 172.21.15.2:0/280964259 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x557b87198320 0x557b8719a780 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.952+0000 7f8a44622740 1 --2- 172.21.15.2:0/280964259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b87182990 0x557b871880b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:21.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.952+0000 7f8a44622740 1 --2- 172.21.15.2:0/280964259 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b8717edc0 0x557b87187b10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.952+0000 7f8a44622740 1 --2- 172.21.15.2:0/280964259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557b8717e400 0x557b8718a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:21.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.952+0000 7f8a44622740 1 -- 172.21.15.2:0/280964259 >> 172.21.15.2:0/280964259 conn(0x557b871708b0 msgr2=0x557b8717dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.952+0000 7f8a1affd640 0 client.25708 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:01:21.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.952+0000 7f8a44622740 1 -- 172.21.15.2:0/280964259 shutdown_connections 2024-05-14T17:01:21.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.952+0000 7f8a44622740 1 -- 172.21.15.2:0/280964259 wait complete. 2024-05-14T17:01:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.952+0000 7f8a44622740 20 client.25708 trim_cache size 0 max 16384 2024-05-14T17:01:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.977+0000 7f8a44622740 1 Processor -- start 2024-05-14T17:01:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.977+0000 7f8a44622740 1 -- start start 2024-05-14T17:01:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.977+0000 7f8a44622740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557b8717e400 0x557b87187ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.977+0000 7f8a44622740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b8717edc0 0x557b87193e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.977+0000 7f8a44622740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b87182990 0x557b8718a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.977+0000 7f8a44622740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c1aed60 con 0x557b8717e400 2024-05-14T17:01:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.977+0000 7f8a3b7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b87182990 0x557b8718a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:21.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.977+0000 7f8a3b7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b87182990 0x557b8718a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:36188/0 (socket says 172.21.15.2:36188) 2024-05-14T17:01:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.977+0000 7f8a3b7fe640 1 -- 172.21.15.2:0/4108891784 learned_addr learned my addr 172.21.15.2:0/4108891784 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:01:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.977+0000 7f8a3a7fc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b8717edc0 0x557b87193e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:21.488 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.977+0000 7f8a44622740 1 -- 172.21.15.2:0/4108891784 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c1ac600 con 0x557b87182990 2024-05-14T17:01:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.977+0000 7f8a44622740 1 -- 172.21.15.2:0/4108891784 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c1aa470 con 0x557b8717edc0 2024-05-14T17:01:21.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.977+0000 7f8a3affd640 1 --2- 172.21.15.2:0/4108891784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557b8717e400 0x557b87187ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.978+0000 7f8a3b7fe640 1 -- 172.21.15.2:0/4108891784 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b8717edc0 msgr2=0x557b87193e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.978+0000 7f8a3b7fe640 1 --2- 172.21.15.2:0/4108891784 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b8717edc0 0x557b87193e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.978+0000 7f8a3b7fe640 1 -- 172.21.15.2:0/4108891784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557b8717e400 msgr2=0x557b87187ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.978+0000 7f8a3b7fe640 1 --2- 172.21.15.2:0/4108891784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557b8717e400 0x557b87187ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:21.578 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.978+0000 7f8a3b7fe640 1 -- 172.21.15.2:0/4108891784 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b87186060 con 0x557b87182990 2024-05-14T17:01:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.978+0000 7f8a3b7fe640 1 --2- 172.21.15.2:0/4108891784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b87182990 0x557b8718a300 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8a34035000 tx=0x7f8a34043000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f263969906dd9dfe server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.978+0000 7f8a217fa640 1 -- 172.21.15.2:0/4108891784 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a34030b80 con 0x557b87182990 2024-05-14T17:01:21.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.979+0000 7f8a217fa640 1 -- 172.21.15.2:0/4108891784 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a34036040 con 0x557b87182990 2024-05-14T17:01:21.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.979+0000 7f8a217fa640 1 -- 172.21.15.2:0/4108891784 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a34053480 con 0x557b87182990 2024-05-14T17:01:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.980+0000 7f8a44622740 1 -- 172.21.15.2:0/4108891784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b87182990 msgr2=0x557b8718a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.980+0000 7f8a44622740 1 --2- 172.21.15.2:0/4108891784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b87182990 0x557b8718a300 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8a34035000 tx=0x7f8a34043000 comp rx=0 tx=0).stop 2024-05-14T17:01:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.980+0000 7f8a44622740 1 -- 172.21.15.2:0/4108891784 shutdown_connections 2024-05-14T17:01:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.980+0000 7f8a44622740 1 --2- 172.21.15.2:0/4108891784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b87182990 0x557b8718a300 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:21.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.980+0000 7f8a44622740 1 --2- 172.21.15.2:0/4108891784 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b8717edc0 0x557b87193e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.980+0000 7f8a44622740 1 --2- 172.21.15.2:0/4108891784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557b8717e400 0x557b87187ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.980+0000 7f8a44622740 1 -- 172.21.15.2:0/4108891784 >> 172.21.15.2:0/4108891784 conn(0x557b871708b0 msgr2=0x557b871873a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:21.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.980+0000 7f8a44622740 1 -- 172.21.15.2:0/4108891784 shutdown_connections 2024-05-14T17:01:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.980+0000 7f8a44622740 1 -- 172.21.15.2:0/4108891784 wait complete. 2024-05-14T17:01:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.980+0000 7f8a44622740 1 -- start start 2024-05-14T17:01:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.981+0000 7f8a44622740 1 Processor -- start 2024-05-14T17:01:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.981+0000 7f8a44622740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b8717e400 0x557b871a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.981+0000 7f8a44622740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557b8717edc0 0x557b87190210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.981+0000 7f8a44622740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b87182990 0x557b8719f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.981+0000 7f8a44622740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c1aaac0 con 0x557b8717edc0 2024-05-14T17:01:21.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.981+0000 7f8a44622740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c1abf30 con 0x557b8717e400 2024-05-14T17:01:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.981+0000 7f8a44622740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3c1ab190 con 0x557b87182990 2024-05-14T17:01:21.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.981+0000 7f8a44622740 10 client.0 _ll_register_callbacks cb 0x7f8a30005880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:01:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.982+0000 7f8a3affd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b8717e400 0x557b871a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:21.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.982+0000 7f8a3affd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b8717e400 0x557b871a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:36190/0 (socket says 172.21.15.2:36190) 2024-05-14T17:01:21.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.982+0000 7f8a3affd640 1 -- 172.21.15.2:0/135213601 learned_addr learned my addr 172.21.15.2:0/135213601 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:01:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.982+0000 7f8a3a7fc640 1 --2- 172.21.15.2:0/135213601 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557b8717edc0 0x557b87190210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.982+0000 7f8a3affd640 1 -- 172.21.15.2:0/135213601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b87182990 msgr2=0x557b8719f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.982+0000 7f8a3b7fe640 1 --2- 172.21.15.2:0/135213601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b87182990 0x557b8719f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.982+0000 7f8a3affd640 1 --2- 172.21.15.2:0/135213601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b87182990 0x557b8719f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:22.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.982+0000 7f8a3affd640 1 -- 172.21.15.2:0/135213601 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557b8717edc0 msgr2=0x557b87190210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:22.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.982+0000 7f8a3affd640 1 --2- 172.21.15.2:0/135213601 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557b8717edc0 0x557b87190210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.982+0000 7f8a3affd640 1 -- 172.21.15.2:0/135213601 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b87190c40 con 0x557b8717e400 2024-05-14T17:01:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.983+0000 7f8a3affd640 1 --2- 172.21.15.2:0/135213601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b8717e400 0x557b871a06c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c0a9360 tx=0x7f8a2c0e44f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=447fa0ee1c3f9ba8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.983+0000 7f8a1affd640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T17:01:22.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.983+0000 7f8a1affd640 1 -- 172.21.15.2:0/135213601 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c020040 con 0x557b8717e400 2024-05-14T17:01:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.983+0000 7f8a1affd640 1 -- 172.21.15.2:0/135213601 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7f8a2c0099d0 con 0x557b8717e400 2024-05-14T17:01:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.983+0000 7f8a44622740 1 -- 172.21.15.2:0/135213601 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a34036040 con 0x557b8717e400 2024-05-14T17:01:22.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.983+0000 7f8a1affd640 1 -- 172.21.15.2:0/135213601 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f8a2c0b2180 con 0x557b8717e400 2024-05-14T17:01:22.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.983+0000 7f8a44622740 10 client.25739 fetch_fsmap learned FSMap version 40 2024-05-14T17:01:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.983+0000 7f8a44622740 1 -- 172.21.15.2:0/135213601 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x557b872020b0 con 0x557b8717e400 2024-05-14T17:01:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.983+0000 7f8a1affd640 1 -- 172.21.15.2:0/135213601 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a34036040 con 0x557b8717e400 2024-05-14T17:01:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.984+0000 7f8a1affd640 1 -- 172.21.15.2:0/135213601 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7f8a2c0e40d0 con 0x557b8717e400 2024-05-14T17:01:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.984+0000 7f8a44622740 10 client.25739 fetch_fsmap finished waiting for FSMap version 40 2024-05-14T17:01:22.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.984+0000 7f8a44622740 10 client.25739 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:01:22.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.984+0000 7f8a44622740 20 client.25739 populate_metadata read hostname 'smithi002' 2024-05-14T17:01:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.984+0000 7f8a44622740 1 --2- 172.21.15.2:0/135213601 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x557b87202840 0x557b87204c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.984+0000 7f8a44622740 4 client.25739 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:01:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.984+0000 7f8a44622740 1 -- 172.21.15.2:0/135213601 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557b870e3360 con 0x557b87202840 2024-05-14T17:01:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.984+0000 7f8a3a7fc640 1 --2- 172.21.15.2:0/135213601 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x557b87202840 0x557b87204c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.985+0000 7f8a1affd640 10 client.25739 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:01:22.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.985+0000 7f8a3a7fc640 1 --2- 172.21.15.2:0/135213601 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x557b87202840 0x557b87204c60 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9bb36579cefdb35 server_cookie=2c92f92c17fe3bbd in_seq=0 out_seq=0 2024-05-14T17:01:22.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.999+0000 7f8a1affd640 1 -- 172.21.15.2:0/135213601 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557b870e3360 con 0x557b87202840 2024-05-14T17:01:22.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.999+0000 7f8a1affd640 10 client.25739 handle_command_reply: tid=0 2024-05-14T17:01:22.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:19.999+0000 7f8a44622740 1 client.25739 shutdown 2024-05-14T17:01:22.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:20.000+0000 7f8a44622740 1 -- 172.21.15.2:0/135213601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b8717e400 msgr2=0x557b871a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:20.000+0000 7f8a44622740 1 --2- 172.21.15.2:0/135213601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b8717e400 0x557b871a06c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c0a9360 tx=0x7f8a2c0e44f0 comp rx=0 tx=0).stop 2024-05-14T17:01:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:20.000+0000 7f8a44622740 1 -- 172.21.15.2:0/135213601 shutdown_connections 2024-05-14T17:01:22.283 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:20.000+0000 7f8a44622740 1 --2- 172.21.15.2:0/135213601 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x557b87202840 0x557b87204c60 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:22.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:20.000+0000 7f8a44622740 1 --2- 172.21.15.2:0/135213601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557b87182990 0x557b8719f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:20.000+0000 7f8a44622740 1 --2- 172.21.15.2:0/135213601 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557b8717edc0 0x557b87190210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:20.000+0000 7f8a44622740 1 --2- 172.21.15.2:0/135213601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b8717e400 0x557b871a06c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:20.000+0000 7f8a44622740 1 -- 172.21.15.2:0/135213601 >> 172.21.15.2:0/135213601 conn(0x557b871708b0 msgr2=0x557b87183630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:20.000+0000 7f8a1affd640 0 client.25739 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:01:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:20.001+0000 7f8a44622740 1 -- 172.21.15.2:0/135213601 shutdown_connections 2024-05-14T17:01:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:20.001+0000 7f8a44622740 1 -- 172.21.15.2:0/135213601 wait complete. 2024-05-14T17:01:22.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:20.005+0000 7f8a44622740 20 client.25739 trim_cache size 0 max 16384 2024-05-14T17:01:22.401 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:01:22.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:01:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:01:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:20.008+0000 7f8a4182a640 1 -- 172.21.15.2:0/2041687171 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f8a240b2a10 msgr2=0x7f8a240b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:20.008+0000 7f8a4182a640 1 --2- 172.21.15.2:0/2041687171 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f8a240b2a10 0x7f8a240b4ed0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8a34006010 tx=0x7f8a3403d040 comp rx=0 tx=0).stop 2024-05-14T17:01:22.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:20.008+0000 7f8a4182a640 1 -- 172.21.15.2:0/2041687171 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a3c101980 msgr2=0x7f8a3c1980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:22.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:20.008+0000 7f8a4182a640 1 --2- 172.21.15.2:0/2041687171 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a3c101980 0x7f8a3c1980e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c00d6d0 tx=0x7f8a2c00db90 comp rx=0 tx=0).stop 2024-05-14T17:01:22.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:20.012+0000 7f8a4182a640 1 -- 172.21.15.2:0/2041687171 shutdown_connections 2024-05-14T17:01:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:20.012+0000 7f8a4182a640 1 --2- 172.21.15.2:0/2041687171 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f8a240b2a10 0x7f8a240b4ed0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:20.012+0000 7f8a4182a640 1 --2- 172.21.15.2:0/2041687171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a3c107940 0x7f8a3c19c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:22.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:20.012+0000 7f8a4182a640 1 --2- 172.21.15.2:0/2041687171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a3c102340 0x7f8a3c198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:22.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:20.012+0000 7f8a4182a640 1 --2- 172.21.15.2:0/2041687171 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a3c101980 0x7f8a3c1980e0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:20.012+0000 7f8a4182a640 1 -- 172.21.15.2:0/2041687171 >> 172.21.15.2:0/2041687171 conn(0x7f8a3c078ad0 msgr2=0x7f8a3c1044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:20.015+0000 7f8a4182a640 1 -- 172.21.15.2:0/2041687171 shutdown_connections 2024-05-14T17:01:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:20.015+0000 7f8a4182a640 1 -- 172.21.15.2:0/2041687171 wait complete. 2024-05-14T17:01:22.550 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:01:20 smithi002 ceph-mon[31538]: pgmap v204: 97 pgs: 97 active+clean; 3.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 25 MiB/s wr, 1.41k op/s 2024-05-14T17:01:22.557 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 365.0 s, 3242.8 tps, lat 30.713 ms stddev 33.054 2024-05-14T17:01:22.566 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:01:20 smithi123 ceph-mon[35971]: pgmap v204: 97 pgs: 97 active+clean; 3.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 25 MiB/s wr, 1.41k op/s 2024-05-14T17:01:22.575 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:01:20 smithi151 ceph-mon[36066]: pgmap v204: 97 pgs: 97 active+clean; 3.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 25 MiB/s wr, 1.41k op/s 2024-05-14T17:01:22.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:01:22 smithi002 ceph-mon[31538]: pgmap v205: 97 pgs: 97 active+clean; 3.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 25 MiB/s wr, 1.41k op/s 2024-05-14T17:01:22.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:01:22 smithi151 ceph-mon[36066]: pgmap v205: 97 pgs: 97 active+clean; 3.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 25 MiB/s wr, 1.41k op/s 2024-05-14T17:01:22.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:01:22 smithi123 ceph-mon[35971]: pgmap v205: 97 pgs: 97 active+clean; 3.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 25 MiB/s wr, 1.41k op/s 2024-05-14T17:01:24.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:01:24 smithi002 ceph-mon[31538]: pgmap v206: 97 pgs: 97 active+clean; 3.9 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 26 MiB/s wr, 1.57k op/s 2024-05-14T17:01:24.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:01:24 smithi151 ceph-mon[36066]: pgmap v206: 97 pgs: 97 active+clean; 3.9 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 26 MiB/s wr, 1.57k op/s 2024-05-14T17:01:24.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:01:24 smithi123 ceph-mon[35971]: pgmap v206: 97 pgs: 97 active+clean; 3.9 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 26 MiB/s wr, 1.57k op/s 2024-05-14T17:01:25.426 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 16.4 2024-05-14T17:01:25.434 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:01:25.831 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:01:26.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:01:26 smithi002 ceph-mon[31538]: pgmap v207: 97 pgs: 97 active+clean; 3.9 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.7 MiB/s wr, 663 op/s 2024-05-14T17:01:26.753 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 370.0 s, 3421.0 tps, lat 29.373 ms stddev 32.234 2024-05-14T17:01:26.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:01:26 smithi151 ceph-mon[36066]: pgmap v207: 97 pgs: 97 active+clean; 3.9 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.7 MiB/s wr, 663 op/s 2024-05-14T17:01:26.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:01:26 smithi123 ceph-mon[35971]: pgmap v207: 97 pgs: 97 active+clean; 3.9 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.7 MiB/s wr, 663 op/s 2024-05-14T17:01:27.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:27.821+0000 7f1385ffb700 1 -- 172.21.15.2:0/3747663061 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137000be60 con 0x7f1388111d20 2024-05-14T17:01:27.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:27.821+0000 7f138d9e2700 1 -- 172.21.15.2:0/3747663061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1388111d20 msgr2=0x7f138810a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:27.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:27.821+0000 7f138d9e2700 1 --2- 172.21.15.2:0/3747663061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1388111d20 0x7f138810a540 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f13700098f0 tx=0x7f1370009c00 comp rx=0 tx=0).stop 2024-05-14T17:01:27.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:27.821+0000 7f138d9e2700 1 -- 172.21.15.2:0/3747663061 shutdown_connections 2024-05-14T17:01:27.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:27.821+0000 7f138d9e2700 1 --2- 172.21.15.2:0/3747663061 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f138811cd20 0x7f138811f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:27.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:27.821+0000 7f138d9e2700 1 --2- 172.21.15.2:0/3747663061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1388111d20 0x7f138810a540 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:27.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:27.821+0000 7f138d9e2700 1 --2- 172.21.15.2:0/3747663061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f138810aa80 0x7f138810af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:27.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:27.821+0000 7f138d9e2700 1 -- 172.21.15.2:0/3747663061 >> 172.21.15.2:0/3747663061 conn(0x7f138806e6b0 msgr2=0x7f138806eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:27.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:27.822+0000 7f138d9e2700 1 -- 172.21.15.2:0/3747663061 shutdown_connections 2024-05-14T17:01:27.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:27.822+0000 7f138d9e2700 1 -- 172.21.15.2:0/3747663061 wait complete. 2024-05-14T17:01:27.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:27.823+0000 7f138d9e2700 1 Processor -- start 2024-05-14T17:01:28.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:27.823+0000 7f138d9e2700 1 -- start start 2024-05-14T17:01:28.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:27.823+0000 7f138d9e2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f138810aa80 0x7f13881ae800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:28.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:27.823+0000 7f138d9e2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1388111d20 0x7f13881aed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:28.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:27.823+0000 7f138d9e2700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f138811cd20 0x7f13881a8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:28.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:27.823+0000 7f138d9e2700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f13881226b0 con 0x7f138810aa80 2024-05-14T17:01:28.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:27.823+0000 7f138d9e2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1388122530 con 0x7f1388111d20 2024-05-14T17:01:28.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:27.823+0000 7f138d9e2700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1388122830 con 0x7f138811cd20 2024-05-14T17:01:28.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:27.824+0000 7f1386ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f138810aa80 0x7f13881ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:28.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:27.824+0000 7f1386ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f138810aa80 0x7f13881ae800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57310/0 (socket says 172.21.15.2:57310) 2024-05-14T17:01:28.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:27.824+0000 7f1386ffd700 1 -- 172.21.15.2:0/2496024784 learned_addr learned my addr 172.21.15.2:0/2496024784 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:01:28.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:27.824+0000 7f1386ffd700 1 -- 172.21.15.2:0/2496024784 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f138811cd20 msgr2=0x7f13881a8a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T17:01:28.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:27.824+0000 7f1386ffd700 1 --2- 172.21.15.2:0/2496024784 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f138811cd20 0x7f13881a8a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:28.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:27.824+0000 7f1386ffd700 1 -- 172.21.15.2:0/2496024784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1388111d20 msgr2=0x7f13881aed40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T17:01:28.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:27.824+0000 7f1386ffd700 1 --2- 172.21.15.2:0/2496024784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1388111d20 0x7f13881aed40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:28.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:27.824+0000 7f1386ffd700 1 -- 172.21.15.2:0/2496024784 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13881a9170 con 0x7f138810aa80 2024-05-14T17:01:28.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:27.825+0000 7f1386ffd700 1 --2- 172.21.15.2:0/2496024784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f138810aa80 0x7f13881ae800 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1370009810 tx=0x7f137000fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:28.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:27.825+0000 7f138c9e0700 1 -- 172.21.15.2:0/2496024784 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1370003a90 con 0x7f138810aa80 2024-05-14T17:01:28.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:27.825+0000 7f138d9e2700 1 -- 172.21.15.2:0/2496024784 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13881a9400 con 0x7f138810aa80 2024-05-14T17:01:28.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:27.825+0000 7f138d9e2700 1 -- 172.21.15.2:0/2496024784 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1388112500 con 0x7f138810aa80 2024-05-14T17:01:28.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:27.826+0000 7f138c9e0700 1 -- 172.21.15.2:0/2496024784 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1370003c30 con 0x7f138810aa80 2024-05-14T17:01:28.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:27.826+0000 7f138c9e0700 1 -- 172.21.15.2:0/2496024784 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13700177c0 con 0x7f138810aa80 2024-05-14T17:01:28.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:27.830+0000 7f138d9e2700 1 -- 172.21.15.2:0/2496024784 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1368005330 con 0x7f138810aa80 2024-05-14T17:01:28.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:27.831+0000 7f138c9e0700 1 -- 172.21.15.2:0/2496024784 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f1370017960 con 0x7f138810aa80 2024-05-14T17:01:28.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:27.832+0000 7f138c9e0700 1 --2- 172.21.15.2:0/2496024784 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f13740b2c00 0x7f13740b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:28.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:27.832+0000 7f138c9e0700 1 -- 172.21.15.2:0/2496024784 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7f13700e3210 con 0x7f138810aa80 2024-05-14T17:01:28.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:27.833+0000 7f13867fc700 1 --2- 172.21.15.2:0/2496024784 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f13740b2c00 0x7f13740b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:28.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:27.833+0000 7f13867fc700 1 --2- 172.21.15.2:0/2496024784 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f13740b2c00 0x7f13740b50c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f138806ed60 tx=0x7f1378009250 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:28.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:27.839+0000 7f138c9e0700 1 -- 172.21.15.2:0/2496024784 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1370013030 con 0x7f138810aa80 2024-05-14T17:01:28.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:28.286+0000 7f138d9e2700 1 -- 172.21.15.2:0/2496024784 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13680058f0 con 0x7f138810aa80 2024-05-14T17:01:28.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:28.288+0000 7f138c9e0700 1 -- 172.21.15.2:0/2496024784 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11854 (secure 0 0 0) 0x7f13700aac20 con 0x7f138810aa80 2024-05-14T17:01:28.394 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:01:28.401 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2275696923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2275696923},{"type":"v1","addr":"172.21.15.2:6835","nonce":2275696923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/716718761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":716718761},{"type":"v1","addr":"172.21.15.2:6839","nonce":716718761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/3644815179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3644815179},{"type":"v1","addr":"172.21.15.2:6841","nonce":3644815179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1880336731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1880336731},{"type":"v1","addr":"172.21.15.151:6833","nonce":1880336731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2415605548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2415605548},{"type":"v1","addr":"172.21.15.2:6837","nonce":2415605548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2635167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2635167},{"type":"v1","addr":"172.21.15.123:6833","nonce":2635167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:51:34.106919+0000","modified":"2024-05-14T17:01:13.455170+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24455,"mds_2":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24415":{"gid":24415,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/762044339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":762044339},{"type":"v1","addr":"172.21.15.151:6835","nonce":762044339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/132043151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":132043151},{"type":"v1","addr":"172.21.15.151:6837","nonce":132043151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/4107834375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":4107834375},{"type":"v1","addr":"172.21.15.151:6839","nonce":4107834375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.123:6835/1198670013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1198670013},{"type":"v1","addr":"172.21.15.123:6835","nonce":1198670013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/333426645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":333426645},{"type":"v1","addr":"172.21.15.123:6837","nonce":333426645}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/580275877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":580275877},{"type":"v1","addr":"172.21.15.123:6839","nonce":580275877}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24431,24455,24479]},"id":1}]} 2024-05-14T17:01:28.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:28.292+0000 7f138d9e2700 1 -- 172.21.15.2:0/2496024784 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f13740b2c00 msgr2=0x7f13740b50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:28.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:28.292+0000 7f138d9e2700 1 --2- 172.21.15.2:0/2496024784 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f13740b2c00 0x7f13740b50c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f138806ed60 tx=0x7f1378009250 comp rx=0 tx=0).stop 2024-05-14T17:01:28.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:28.292+0000 7f138d9e2700 1 -- 172.21.15.2:0/2496024784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f138810aa80 msgr2=0x7f13881ae800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:28.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:28.292+0000 7f138d9e2700 1 --2- 172.21.15.2:0/2496024784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f138810aa80 0x7f13881ae800 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1370009810 tx=0x7f137000fc60 comp rx=0 tx=0).stop 2024-05-14T17:01:28.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:28.292+0000 7f138d9e2700 1 -- 172.21.15.2:0/2496024784 shutdown_connections 2024-05-14T17:01:28.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:28.292+0000 7f138d9e2700 1 --2- 172.21.15.2:0/2496024784 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f13740b2c00 0x7f13740b50c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:28.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:28.292+0000 7f138d9e2700 1 --2- 172.21.15.2:0/2496024784 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f138811cd20 0x7f13881a8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:28.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:28.292+0000 7f138d9e2700 1 --2- 172.21.15.2:0/2496024784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f138810aa80 0x7f13881ae800 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:28.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:28.292+0000 7f138d9e2700 1 --2- 172.21.15.2:0/2496024784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1388111d20 0x7f13881aed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:28.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:28.292+0000 7f138d9e2700 1 -- 172.21.15.2:0/2496024784 >> 172.21.15.2:0/2496024784 conn(0x7f138806e6b0 msgr2=0x7f138811af90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:28.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:28.293+0000 7f138d9e2700 1 -- 172.21.15.2:0/2496024784 shutdown_connections 2024-05-14T17:01:28.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:28.293+0000 7f138d9e2700 1 -- 172.21.15.2:0/2496024784 wait complete. 2024-05-14T17:01:28.553 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 40 2024-05-14T17:01:28.635 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:01:28 smithi002 ceph-mon[31538]: pgmap v208: 97 pgs: 97 active+clean; 3.9 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.7 MiB/s wr, 664 op/s 2024-05-14T17:01:28.644 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:01:28 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/2496024784' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:01:28.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:01:28 smithi151 ceph-mon[36066]: pgmap v208: 97 pgs: 97 active+clean; 3.9 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.7 MiB/s wr, 664 op/s 2024-05-14T17:01:28.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:01:28 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/2496024784' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:01:28.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:01:28 smithi123 ceph-mon[35971]: pgmap v208: 97 pgs: 97 active+clean; 3.9 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.7 MiB/s wr, 664 op/s 2024-05-14T17:01:28.917 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:01:28 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/2496024784' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:01:29.111 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 136.4 --await-for 45.0' 2024-05-14T17:01:29.120 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 136.4 --await-for 45.0 2024-05-14T17:01:29.570 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:01:30.724 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:01:30 smithi002 ceph-mon[31538]: pgmap v209: 97 pgs: 97 active+clean; 4.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 15 MiB/s wr, 1.13k op/s 2024-05-14T17:01:30.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:01:30 smithi151 ceph-mon[36066]: pgmap v209: 97 pgs: 97 active+clean; 4.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 15 MiB/s wr, 1.13k op/s 2024-05-14T17:01:30.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:01:30 smithi123 ceph-mon[35971]: pgmap v209: 97 pgs: 97 active+clean; 4.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 15 MiB/s wr, 1.13k op/s 2024-05-14T17:01:31.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.570+0000 7f1232466700 1 -- 172.21.15.2:0/1050842424 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f122c06c670 msgr2=0x7f122c1103e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:31.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.570+0000 7f1232466700 1 --2- 172.21.15.2:0/1050842424 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f122c06c670 0x7f122c1103e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1224009f90 tx=0x7f122400b330 comp rx=0 tx=0).stop 2024-05-14T17:01:31.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.570+0000 7f1232466700 1 -- 172.21.15.2:0/1050842424 shutdown_connections 2024-05-14T17:01:31.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.570+0000 7f1232466700 1 --2- 172.21.15.2:0/1050842424 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f122c06c670 0x7f122c1103e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:31.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.570+0000 7f1232466700 1 --2- 172.21.15.2:0/1050842424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f122c109610 0x7f122c1099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:31.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.570+0000 7f1232466700 1 --2- 172.21.15.2:0/1050842424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f122c06bcb0 0x7f122c06c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:31.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.570+0000 7f1232466700 1 -- 172.21.15.2:0/1050842424 >> 172.21.15.2:0/1050842424 conn(0x7f122c079460 msgr2=0x7f122c079870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:31.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.570+0000 7f1232466700 1 -- 172.21.15.2:0/1050842424 shutdown_connections 2024-05-14T17:01:31.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.570+0000 7f1232466700 1 -- 172.21.15.2:0/1050842424 wait complete. 2024-05-14T17:01:31.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.571+0000 7f1232466700 1 Processor -- start 2024-05-14T17:01:31.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.571+0000 7f1232466700 1 -- start start 2024-05-14T17:01:31.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.571+0000 7f1232466700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f122c06bcb0 0x7f122c1a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:31.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.571+0000 7f1232466700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f122c06c670 0x7f122c1a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:31.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.571+0000 7f1232466700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f122c109610 0x7f122c19ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:31.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.571+0000 7f1232466700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f122c1137c0 con 0x7f122c06bcb0 2024-05-14T17:01:31.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.571+0000 7f1232466700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f122c113640 con 0x7f122c109610 2024-05-14T17:01:31.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.571+0000 7f1232466700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f122c113940 con 0x7f122c06c670 2024-05-14T17:01:31.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.572+0000 7f1223fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f122c06c670 0x7f122c1a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:31.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.572+0000 7f1223fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f122c06c670 0x7f122c1a1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:39760/0 (socket says 172.21.15.2:39760) 2024-05-14T17:01:31.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.572+0000 7f1223fff700 1 -- 172.21.15.2:0/1207491014 learned_addr learned my addr 172.21.15.2:0/1207491014 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:01:31.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.572+0000 7f122bfff700 1 --2- 172.21.15.2:0/1207491014 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f122c06bcb0 0x7f122c1a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:31.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.572+0000 7f1230a03700 1 --2- 172.21.15.2:0/1207491014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f122c109610 0x7f122c19ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:31.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.572+0000 7f1223fff700 1 -- 172.21.15.2:0/1207491014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f122c109610 msgr2=0x7f122c19ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:31.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.572+0000 7f1223fff700 1 --2- 172.21.15.2:0/1207491014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f122c109610 0x7f122c19ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:31.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.572+0000 7f1223fff700 1 -- 172.21.15.2:0/1207491014 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f122c06bcb0 msgr2=0x7f122c1a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:31.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.572+0000 7f1223fff700 1 --2- 172.21.15.2:0/1207491014 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f122c06bcb0 0x7f122c1a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:31.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.572+0000 7f1223fff700 1 -- 172.21.15.2:0/1207491014 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f122c19c1c0 con 0x7f122c06c670 2024-05-14T17:01:31.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.572+0000 7f122bfff700 1 --2- 172.21.15.2:0/1207491014 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f122c06bcb0 0x7f122c1a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:01:31.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.573+0000 7f1223fff700 1 --2- 172.21.15.2:0/1207491014 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f122c06c670 0x7f122c1a1de0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f121400e9a0 tx=0x7f121400ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:31.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.573+0000 7f1229ffb700 1 -- 172.21.15.2:0/1207491014 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121400cbd0 con 0x7f122c06c670 2024-05-14T17:01:31.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.573+0000 7f1229ffb700 1 -- 172.21.15.2:0/1207491014 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f121400cd70 con 0x7f122c06c670 2024-05-14T17:01:32.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.573+0000 7f1229ffb700 1 -- 172.21.15.2:0/1207491014 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1214018680 con 0x7f122c06c670 2024-05-14T17:01:32.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.573+0000 7f1232466700 1 -- 172.21.15.2:0/1207491014 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f122c19c4b0 con 0x7f122c06c670 2024-05-14T17:01:32.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.574+0000 7f1232466700 1 -- 172.21.15.2:0/1207491014 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f122c1a8770 con 0x7f122c06c670 2024-05-14T17:01:32.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.577+0000 7f1229ffb700 1 -- 172.21.15.2:0/1207491014 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f1214010970 con 0x7f122c06c670 2024-05-14T17:01:32.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.579+0000 7f1229ffb700 1 --2- 172.21.15.2:0/1207491014 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f12180b2c60 0x7f12180b5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:32.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.579+0000 7f122bfff700 1 --2- 172.21.15.2:0/1207491014 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f12180b2c60 0x7f12180b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:32.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.579+0000 7f1229ffb700 1 -- 172.21.15.2:0/1207491014 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7f12140e28f0 con 0x7f122c06c670 2024-05-14T17:01:32.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.580+0000 7f122bfff700 1 --2- 172.21.15.2:0/1207491014 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f12180b2c60 0x7f12180b5120 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f122c19cba0 tx=0x7f121c009310 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:32.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.580+0000 7f123ccfcb80 1 Processor -- start 2024-05-14T17:01:32.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.580+0000 7f123ccfcb80 1 -- start start 2024-05-14T17:01:32.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.580+0000 7f123ccfcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b506208730 0x55b506208b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:32.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.580+0000 7f123ccfcb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b506209050 0x55b50620ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:32.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.581+0000 7f123ccfcb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b50620cf60 0x55b50620d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:32.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.581+0000 7f1223fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b506209050 0x55b50620ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:32.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.581+0000 7f1223fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b506209050 0x55b50620ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57334/0 (socket says 172.21.15.2:57334) 2024-05-14T17:01:32.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.581+0000 7f1223fff700 1 -- 172.21.15.2:0/892743211 learned_addr learned my addr 172.21.15.2:0/892743211 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:01:32.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.581+0000 7f123ccfcb80 1 -- 172.21.15.2:0/892743211 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b506155e20 con 0x55b506209050 2024-05-14T17:01:32.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.581+0000 7f123ccfcb80 1 -- 172.21.15.2:0/892743211 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b5061459c0 con 0x55b506208730 2024-05-14T17:01:32.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.581+0000 7f123ccfcb80 1 -- 172.21.15.2:0/892743211 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b506149a00 con 0x55b50620cf60 2024-05-14T17:01:32.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.581+0000 7f122bfff700 1 --2- 172.21.15.2:0/892743211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b506208730 0x55b506208b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:32.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.581+0000 7f1230a03700 1 --2- 172.21.15.2:0/892743211 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b50620cf60 0x55b50620d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:32.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.581+0000 7f1223fff700 1 -- 172.21.15.2:0/892743211 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b50620cf60 msgr2=0x55b50620d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:32.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.581+0000 7f1223fff700 1 --2- 172.21.15.2:0/892743211 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b50620cf60 0x55b50620d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:32.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.581+0000 7f1223fff700 1 -- 172.21.15.2:0/892743211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b506208730 msgr2=0x55b506208b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:32.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.581+0000 7f1223fff700 1 --2- 172.21.15.2:0/892743211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b506208730 0x55b506208b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:32.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.581+0000 7f1223fff700 1 -- 172.21.15.2:0/892743211 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b506095a60 con 0x55b506209050 2024-05-14T17:01:32.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.581+0000 7f1223fff700 1 --2- 172.21.15.2:0/892743211 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b506209050 0x55b50620ca20 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f12140a9870 tx=0x7f12140a95f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=435dec1588f68a6a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:32.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.582+0000 7f12227fc700 1 -- 172.21.15.2:0/892743211 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12140adf20 con 0x55b506209050 2024-05-14T17:01:32.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.582+0000 7f12227fc700 1 -- 172.21.15.2:0/892743211 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12140b29d0 con 0x55b506209050 2024-05-14T17:01:32.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.582+0000 7f12227fc700 1 -- 172.21.15.2:0/892743211 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12140a5740 con 0x55b506209050 2024-05-14T17:01:32.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.583+0000 7f123ccfcb80 1 -- 172.21.15.2:0/892743211 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b506209050 msgr2=0x55b50620ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:32.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.583+0000 7f123ccfcb80 1 --2- 172.21.15.2:0/892743211 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b506209050 0x55b50620ca20 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f12140a9870 tx=0x7f12140a95f0 comp rx=0 tx=0).stop 2024-05-14T17:01:32.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.583+0000 7f123ccfcb80 1 -- 172.21.15.2:0/892743211 shutdown_connections 2024-05-14T17:01:32.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.583+0000 7f123ccfcb80 1 --2- 172.21.15.2:0/892743211 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b50620cf60 0x55b50620d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:32.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.583+0000 7f123ccfcb80 1 --2- 172.21.15.2:0/892743211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b506208730 0x55b506208b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:32.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.583+0000 7f123ccfcb80 1 --2- 172.21.15.2:0/892743211 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b506209050 0x55b50620ca20 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:32.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.583+0000 7f123ccfcb80 1 -- 172.21.15.2:0/892743211 >> 172.21.15.2:0/892743211 conn(0x55b5061223e0 msgr2=0x55b5061f1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:32.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.583+0000 7f123ccfcb80 1 -- 172.21.15.2:0/892743211 shutdown_connections 2024-05-14T17:01:32.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.583+0000 7f123ccfcb80 1 -- 172.21.15.2:0/892743211 wait complete. 2024-05-14T17:01:32.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.584+0000 7f123ccfcb80 1 -- start start 2024-05-14T17:01:32.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.584+0000 7f123ccfcb80 1 Processor -- start 2024-05-14T17:01:32.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.584+0000 7f123ccfcb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b506208730 0x55b506214440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:32.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.584+0000 7f123ccfcb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b506209050 0x55b5062117e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:32.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.584+0000 7f123ccfcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b50620cf60 0x55b506211d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:32.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.585+0000 7f123ccfcb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b50604c130 con 0x55b506208730 2024-05-14T17:01:32.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.585+0000 7f123ccfcb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b5060e7950 con 0x55b50620cf60 2024-05-14T17:01:32.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.585+0000 7f123ccfcb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b506215130 con 0x55b506209050 2024-05-14T17:01:32.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.585+0000 7f123ccfcb80 10 client.0 _ll_register_callbacks cb 0x55b5060dc710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:01:32.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.585+0000 7f1223fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b506209050 0x55b5062117e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:32.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.585+0000 7f1223fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b506209050 0x55b5062117e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:39776/0 (socket says 172.21.15.2:39776) 2024-05-14T17:01:32.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.585+0000 7f1223fff700 1 -- 172.21.15.2:0/158640826 learned_addr learned my addr 172.21.15.2:0/158640826 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:01:32.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.585+0000 7f122bfff700 1 --2- 172.21.15.2:0/158640826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b506208730 0x55b506214440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:32.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.585+0000 7f1223fff700 1 -- 172.21.15.2:0/158640826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b50620cf60 msgr2=0x55b506211d20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T17:01:32.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.585+0000 7f1223fff700 1 --2- 172.21.15.2:0/158640826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b50620cf60 0x55b506211d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:32.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.585+0000 7f1223fff700 1 -- 172.21.15.2:0/158640826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b506208730 msgr2=0x55b506214440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:32.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.585+0000 7f1223fff700 1 --2- 172.21.15.2:0/158640826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b506208730 0x55b506214440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:32.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.585+0000 7f1223fff700 1 -- 172.21.15.2:0/158640826 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5060e4820 con 0x55b506209050 2024-05-14T17:01:32.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.586+0000 7f1223fff700 1 --2- 172.21.15.2:0/158640826 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b506209050 0x55b5062117e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f121400e970 tx=0x7f1214005b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5b99c8722c3af54 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:32.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.586+0000 7f120b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-14T17:01:32.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.586+0000 7f120b7fe700 1 -- 172.21.15.2:0/158640826 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12140c2070 con 0x55b506209050 2024-05-14T17:01:32.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.586+0000 7f123ccfcb80 1 -- 172.21.15.2:0/158640826 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12140b29d0 con 0x55b506209050 2024-05-14T17:01:32.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.586+0000 7f120b7fe700 1 -- 172.21.15.2:0/158640826 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7f12140a8fe0 con 0x55b506209050 2024-05-14T17:01:32.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.586+0000 7f120b7fe700 1 -- 172.21.15.2:0/158640826 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f12140a54e0 con 0x55b506209050 2024-05-14T17:01:32.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.586+0000 7f123ccfcb80 10 client.25726 fetch_fsmap learned FSMap version 40 2024-05-14T17:01:32.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.587+0000 7f123ccfcb80 1 -- 172.21.15.2:0/158640826 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b50621e470 con 0x55b506209050 2024-05-14T17:01:32.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.587+0000 7f120b7fe700 1 -- 172.21.15.2:0/158640826 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12140b29d0 con 0x55b506209050 2024-05-14T17:01:32.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.587+0000 7f120b7fe700 1 -- 172.21.15.2:0/158640826 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7f12140a8910 con 0x55b506209050 2024-05-14T17:01:32.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.587+0000 7f123ccfcb80 10 client.25726 fetch_fsmap finished waiting for FSMap version 40 2024-05-14T17:01:32.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.587+0000 7f123ccfcb80 10 client.25726 resolve_mds: validated gid 24479 aka daemon mds.b 2024-05-14T17:01:32.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.587+0000 7f123ccfcb80 20 client.25726 populate_metadata read hostname 'smithi002' 2024-05-14T17:01:32.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.587+0000 7f123ccfcb80 1 --2- 172.21.15.2:0/158640826 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55b5062209a0 0x55b506222e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:32.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.587+0000 7f123ccfcb80 4 client.25726 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:01:32.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.587+0000 7f123ccfcb80 1 -- 172.21.15.2:0/158640826 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5061c34f0 con 0x55b5062209a0 2024-05-14T17:01:32.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.587+0000 7f122bfff700 1 --2- 172.21.15.2:0/158640826 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55b5062209a0 0x55b506222e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:33.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.588+0000 7f120b7fe700 10 client.25726 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:01:33.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.588+0000 7f122bfff700 1 --2- 172.21.15.2:0/158640826 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55b5062209a0 0x55b506222e00 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42a696ba034a0abb server_cookie=a8d8a4532ab8b360 in_seq=0 out_seq=0 2024-05-14T17:01:33.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.589+0000 7f120b7fe700 1 -- 172.21.15.2:0/158640826 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b5061c34f0 con 0x55b5062209a0 2024-05-14T17:01:33.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.589+0000 7f120b7fe700 10 client.25726 handle_command_reply: tid=0 2024-05-14T17:01:33.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.589+0000 7f123ccfcb80 1 client.25726 shutdown 2024-05-14T17:01:33.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.590+0000 7f123ccfcb80 1 -- 172.21.15.2:0/158640826 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b506209050 msgr2=0x55b5062117e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:33.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.590+0000 7f123ccfcb80 1 --2- 172.21.15.2:0/158640826 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b506209050 0x55b5062117e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f121400e970 tx=0x7f1214005b40 comp rx=0 tx=0).stop 2024-05-14T17:01:33.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.590+0000 7f123ccfcb80 1 -- 172.21.15.2:0/158640826 shutdown_connections 2024-05-14T17:01:33.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.590+0000 7f123ccfcb80 1 --2- 172.21.15.2:0/158640826 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55b5062209a0 0x55b506222e00 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:33.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.590+0000 7f123ccfcb80 1 --2- 172.21.15.2:0/158640826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b50620cf60 0x55b506211d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:33.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.590+0000 7f123ccfcb80 1 --2- 172.21.15.2:0/158640826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b506208730 0x55b506214440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:33.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.590+0000 7f123ccfcb80 1 --2- 172.21.15.2:0/158640826 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b506209050 0x55b5062117e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:33.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.590+0000 7f123ccfcb80 1 -- 172.21.15.2:0/158640826 >> 172.21.15.2:0/158640826 conn(0x55b5061223e0 msgr2=0x55b5061ec860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:33.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.590+0000 7f120b7fe700 0 client.25726 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:01:33.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.590+0000 7f123ccfcb80 1 -- 172.21.15.2:0/158640826 shutdown_connections 2024-05-14T17:01:33.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.590+0000 7f123ccfcb80 1 -- 172.21.15.2:0/158640826 wait complete. 2024-05-14T17:01:33.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.590+0000 7f123ccfcb80 20 client.25726 trim_cache size 0 max 16384 2024-05-14T17:01:33.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.626+0000 7f123ccfcb80 1 Processor -- start 2024-05-14T17:01:33.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.627+0000 7f123ccfcb80 1 -- start start 2024-05-14T17:01:33.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.627+0000 7f123ccfcb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b506208730 0x55b5061f6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:33.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.627+0000 7f123ccfcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b506209050 0x55b5061ec970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:33.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.627+0000 7f123ccfcb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b50620cf60 0x55b50620bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:33.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.627+0000 7f1223fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b506209050 0x55b5061ec970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:33.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.627+0000 7f1230a03700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b50620cf60 0x55b50620bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:33.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.627+0000 7f1230a03700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b50620cf60 0x55b50620bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57348/0 (socket says 172.21.15.2:57348) 2024-05-14T17:01:33.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.627+0000 7f1223fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b506209050 0x55b5061ec970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:41700/0 (socket says 172.21.15.2:41700) 2024-05-14T17:01:33.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.627+0000 7f1230a03700 1 -- 172.21.15.2:0/3527388576 learned_addr learned my addr 172.21.15.2:0/3527388576 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:01:33.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.627+0000 7f123ccfcb80 1 -- 172.21.15.2:0/3527388576 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f122c072aa0 con 0x55b50620cf60 2024-05-14T17:01:33.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.627+0000 7f123ccfcb80 1 -- 172.21.15.2:0/3527388576 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f122c070300 con 0x55b506209050 2024-05-14T17:01:33.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.627+0000 7f123ccfcb80 1 -- 172.21.15.2:0/3527388576 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f122c06e170 con 0x55b506208730 2024-05-14T17:01:33.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.631+0000 7f1223fff700 1 -- 172.21.15.2:0/3527388576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b506208730 msgr2=0x55b5061f6f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T17:01:33.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.631+0000 7f1223fff700 1 --2- 172.21.15.2:0/3527388576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b506208730 0x55b5061f6f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:33.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.631+0000 7f1223fff700 1 -- 172.21.15.2:0/3527388576 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b50620cf60 msgr2=0x55b50620bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:33.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.631+0000 7f1223fff700 1 --2- 172.21.15.2:0/3527388576 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b50620cf60 0x55b50620bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:33.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.631+0000 7f1223fff700 1 -- 172.21.15.2:0/3527388576 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5061f2bb0 con 0x55b506209050 2024-05-14T17:01:33.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.635+0000 7f1223fff700 1 --2- 172.21.15.2:0/3527388576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b506209050 0x55b5061ec970 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f12140a5bf0 tx=0x7f12140a95f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eee4b640176335ae server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:33.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.636+0000 7f1221ffb700 1 -- 172.21.15.2:0/3527388576 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12140c2070 con 0x55b506209050 2024-05-14T17:01:33.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.637+0000 7f1221ffb700 1 -- 172.21.15.2:0/3527388576 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12140b29d0 con 0x55b506209050 2024-05-14T17:01:33.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.637+0000 7f1221ffb700 1 -- 172.21.15.2:0/3527388576 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12140a9070 con 0x55b506209050 2024-05-14T17:01:33.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.638+0000 7f123ccfcb80 1 -- 172.21.15.2:0/3527388576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b506209050 msgr2=0x55b5061ec970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:33.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.638+0000 7f123ccfcb80 1 --2- 172.21.15.2:0/3527388576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b506209050 0x55b5061ec970 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f12140a5bf0 tx=0x7f12140a95f0 comp rx=0 tx=0).stop 2024-05-14T17:01:33.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.639+0000 7f123ccfcb80 1 -- 172.21.15.2:0/3527388576 shutdown_connections 2024-05-14T17:01:33.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.639+0000 7f123ccfcb80 1 --2- 172.21.15.2:0/3527388576 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b50620cf60 0x55b50620bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:33.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.639+0000 7f123ccfcb80 1 --2- 172.21.15.2:0/3527388576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b506208730 0x55b5061f6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:33.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.639+0000 7f123ccfcb80 1 --2- 172.21.15.2:0/3527388576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b506209050 0x55b5061ec970 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:33.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.639+0000 7f123ccfcb80 1 -- 172.21.15.2:0/3527388576 >> 172.21.15.2:0/3527388576 conn(0x55b5061223e0 msgr2=0x55b50621dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:33.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.639+0000 7f123ccfcb80 1 -- 172.21.15.2:0/3527388576 shutdown_connections 2024-05-14T17:01:33.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.640+0000 7f123ccfcb80 1 -- 172.21.15.2:0/3527388576 wait complete. 2024-05-14T17:01:33.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.641+0000 7f123ccfcb80 1 -- start start 2024-05-14T17:01:33.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.641+0000 7f123ccfcb80 1 Processor -- start 2024-05-14T17:01:33.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.642+0000 7f123ccfcb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b506208730 0x55b506221960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:33.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.642+0000 7f123ccfcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b506209050 0x55b506221ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:33.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.642+0000 7f123ccfcb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b50620cf60 0x55b5061fdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:33.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.642+0000 7f122bfff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b506208730 0x55b506221960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:33.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.642+0000 7f122bfff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b506208730 0x55b506221960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:39784/0 (socket says 172.21.15.2:39784) 2024-05-14T17:01:33.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.642+0000 7f122bfff700 1 -- 172.21.15.2:0/1409068991 learned_addr learned my addr 172.21.15.2:0/1409068991 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:01:33.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.642+0000 7f123ccfcb80 1 -- 172.21.15.2:0/1409068991 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f122c06e7c0 con 0x55b50620cf60 2024-05-14T17:01:33.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.642+0000 7f123ccfcb80 1 -- 172.21.15.2:0/1409068991 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f122c06fc30 con 0x55b506209050 2024-05-14T17:01:33.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.643+0000 7f123ccfcb80 1 -- 172.21.15.2:0/1409068991 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f122c06ee90 con 0x55b506208730 2024-05-14T17:01:33.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.643+0000 7f1230a03700 1 --2- 172.21.15.2:0/1409068991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b50620cf60 0x55b5061fdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:33.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.643+0000 7f123ccfcb80 10 client.0 _ll_register_callbacks cb 0x55b5060dc710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:01:33.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.644+0000 7f1223fff700 1 --2- 172.21.15.2:0/1409068991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b506209050 0x55b506221ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:33.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.644+0000 7f122bfff700 1 -- 172.21.15.2:0/1409068991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b506209050 msgr2=0x55b506221ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:33.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.644+0000 7f122bfff700 1 --2- 172.21.15.2:0/1409068991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b506209050 0x55b506221ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:33.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.644+0000 7f122bfff700 1 -- 172.21.15.2:0/1409068991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b50620cf60 msgr2=0x55b5061fdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:33.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.644+0000 7f122bfff700 1 --2- 172.21.15.2:0/1409068991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b50620cf60 0x55b5061fdf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:33.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.644+0000 7f122bfff700 1 -- 172.21.15.2:0/1409068991 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5061fe490 con 0x55b506208730 2024-05-14T17:01:33.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.645+0000 7f1223fff700 1 --2- 172.21.15.2:0/1409068991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b506209050 0x55b506221ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:01:33.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.645+0000 7f122bfff700 1 --2- 172.21.15.2:0/1409068991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b506208730 0x55b506221960 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f121c009960 tx=0x7f121c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9581dd8f2becdee2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:33.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.645+0000 7f120b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-14T17:01:33.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.645+0000 7f120b7fe700 1 -- 172.21.15.2:0/1409068991 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121c02f060 con 0x55b506208730 2024-05-14T17:01:33.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.645+0000 7f120b7fe700 1 -- 172.21.15.2:0/1409068991 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7f121c004440 con 0x55b506208730 2024-05-14T17:01:33.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.646+0000 7f123ccfcb80 1 -- 172.21.15.2:0/1409068991 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12140b29d0 con 0x55b506208730 2024-05-14T17:01:33.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.647+0000 7f120b7fe700 1 -- 172.21.15.2:0/1409068991 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f121c012860 con 0x55b506208730 2024-05-14T17:01:33.990 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:01:32 smithi123 ceph-mon[35971]: pgmap v210: 97 pgs: 97 active+clean; 4.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.9 MiB/s wr, 763 op/s 2024-05-14T17:01:34.001 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:01:32 smithi151 ceph-mon[36066]: pgmap v210: 97 pgs: 97 active+clean; 4.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.9 MiB/s wr, 763 op/s 2024-05-14T17:01:34.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.652+0000 7f123ccfcb80 10 client.25729 fetch_fsmap learned FSMap version 40 2024-05-14T17:01:34.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.652+0000 7f123ccfcb80 1 -- 172.21.15.2:0/1409068991 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b5061ff6a0 con 0x55b506208730 2024-05-14T17:01:34.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.653+0000 7f120b7fe700 1 -- 172.21.15.2:0/1409068991 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12140b29d0 con 0x55b506208730 2024-05-14T17:01:34.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.653+0000 7f120b7fe700 1 -- 172.21.15.2:0/1409068991 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7f121c00f040 con 0x55b506208730 2024-05-14T17:01:34.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.654+0000 7f123ccfcb80 10 client.25729 fetch_fsmap finished waiting for FSMap version 40 2024-05-14T17:01:34.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.654+0000 7f123ccfcb80 10 client.25729 resolve_mds: validated gid 24479 aka daemon mds.b 2024-05-14T17:01:34.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.654+0000 7f123ccfcb80 20 client.25729 populate_metadata read hostname 'smithi002' 2024-05-14T17:01:34.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.654+0000 7f123ccfcb80 1 --2- 172.21.15.2:0/1409068991 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55b5061ffd80 0x55b5062021a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:34.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.654+0000 7f123ccfcb80 4 client.25729 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.4, "await_for": 45.0}] 2024-05-14T17:01:34.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.654+0000 7f123ccfcb80 1 -- 172.21.15.2:0/1409068991 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.4, "await_for": 45.0}) -- 0x55b506157d70 con 0x55b5061ffd80 2024-05-14T17:01:34.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.658+0000 7f1223fff700 1 --2- 172.21.15.2:0/1409068991 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55b5061ffd80 0x55b5062021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:34.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.661+0000 7f1223fff700 1 --2- 172.21.15.2:0/1409068991 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55b5061ffd80 0x55b5062021a0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a86093281743f6d server_cookie=c7d386b70117f35c in_seq=0 out_seq=0 2024-05-14T17:01:34.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.662+0000 7f120b7fe700 10 client.25729 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:01:34.189 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 375.0 s, 2732.8 tps, lat 35.889 ms stddev 38.744 2024-05-14T17:01:34.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.968+0000 7f120b7fe700 1 -- 172.21.15.2:0/1409068991 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b506157d70 con 0x55b5061ffd80 2024-05-14T17:01:34.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.968+0000 7f120b7fe700 10 client.25729 handle_command_reply: tid=0 2024-05-14T17:01:34.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.968+0000 7f123ccfcb80 1 client.25729 shutdown 2024-05-14T17:01:34.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.968+0000 7f123ccfcb80 1 -- 172.21.15.2:0/1409068991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b506208730 msgr2=0x55b506221960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:34.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.968+0000 7f123ccfcb80 1 --2- 172.21.15.2:0/1409068991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b506208730 0x55b506221960 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f121c009960 tx=0x7f121c0047c0 comp rx=0 tx=0).stop 2024-05-14T17:01:34.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.968+0000 7f123ccfcb80 1 -- 172.21.15.2:0/1409068991 shutdown_connections 2024-05-14T17:01:34.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.968+0000 7f123ccfcb80 1 --2- 172.21.15.2:0/1409068991 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55b5061ffd80 0x55b5062021a0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:34.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.968+0000 7f123ccfcb80 1 --2- 172.21.15.2:0/1409068991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b50620cf60 0x55b5061fdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:34.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.968+0000 7f123ccfcb80 1 --2- 172.21.15.2:0/1409068991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b506208730 0x55b506221960 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:34.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.968+0000 7f123ccfcb80 1 --2- 172.21.15.2:0/1409068991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b506209050 0x55b506221ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:34.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.968+0000 7f120b7fe700 0 client.25729 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:01:34.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.968+0000 7f123ccfcb80 1 -- 172.21.15.2:0/1409068991 >> 172.21.15.2:0/1409068991 conn(0x55b5061223e0 msgr2=0x55b50620f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:34.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.968+0000 7f123ccfcb80 1 -- 172.21.15.2:0/1409068991 shutdown_connections 2024-05-14T17:01:34.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.968+0000 7f123ccfcb80 1 -- 172.21.15.2:0/1409068991 wait complete. 2024-05-14T17:01:34.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.968+0000 7f123ccfcb80 20 client.25729 trim_cache size 0 max 16384 2024-05-14T17:01:34.461 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T17:01:34.471 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 40, 2024-05-14T17:01:34.483 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24479, 2024-05-14T17:01:34.496 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 22, 2024-05-14T17:01:34.508 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-05-14T17:01:34.534 INFO:teuthology.orchestra.run.smithi002.stdout: "3b74f8b4": { 2024-05-14T17:01:34.544 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 22, 2024-05-14T17:01:34.577 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-05-14T17:01:34.586 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-05-14T17:01:34.595 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-05-14T17:01:34.605 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-05-14T17:01:34.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:01:34.625 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-05-14T17:01:34.633 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 136.4, 2024-05-14T17:01:34.643 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-05-14T17:01:34.650 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-05-14T17:01:34.658 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-05-14T17:01:34.668 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-05-14T17:01:34.676 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-05-14T17:01:34.686 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-05-14T17:01:34.695 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:01:34.704 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:01:34.767 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:01:34.775 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:01:34.802 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:01:34.813 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T17:01:34.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.968+0000 7f1232466700 1 -- 172.21.15.2:0/1207491014 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f12180b2c60 msgr2=0x7f12180b5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:34.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.968+0000 7f1232466700 1 --2- 172.21.15.2:0/1207491014 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f12180b2c60 0x7f12180b5120 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f122c19cba0 tx=0x7f121c009310 comp rx=0 tx=0).stop 2024-05-14T17:01:34.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.968+0000 7f1232466700 1 -- 172.21.15.2:0/1207491014 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f122c06c670 msgr2=0x7f122c1a1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:34.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.968+0000 7f1232466700 1 --2- 172.21.15.2:0/1207491014 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f122c06c670 0x7f122c1a1de0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f121400e9a0 tx=0x7f121400ed60 comp rx=0 tx=0).stop 2024-05-14T17:01:34.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.968+0000 7f1232466700 1 -- 172.21.15.2:0/1207491014 shutdown_connections 2024-05-14T17:01:34.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.968+0000 7f1232466700 1 --2- 172.21.15.2:0/1207491014 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f12180b2c60 0x7f12180b5120 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:34.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.968+0000 7f1232466700 1 --2- 172.21.15.2:0/1207491014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f122c109610 0x7f122c19ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:34.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.968+0000 7f1232466700 1 --2- 172.21.15.2:0/1207491014 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f122c06bcb0 0x7f122c1a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:34.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.968+0000 7f1232466700 1 --2- 172.21.15.2:0/1207491014 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f122c06c670 0x7f122c1a1de0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:34.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.968+0000 7f1232466700 1 -- 172.21.15.2:0/1207491014 >> 172.21.15.2:0/1207491014 conn(0x7f122c079460 msgr2=0x7f122c10e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:34.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.968+0000 7f1232466700 1 -- 172.21.15.2:0/1207491014 shutdown_connections 2024-05-14T17:01:34.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:31.968+0000 7f1232466700 1 -- 172.21.15.2:0/1207491014 wait complete. 2024-05-14T17:01:34.958 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:01:32 smithi002 ceph-mon[31538]: pgmap v210: 97 pgs: 97 active+clean; 4.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.9 MiB/s wr, 763 op/s 2024-05-14T17:01:34.978 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '3b74f8b4', quiesce took 9.6 seconds. Will release after: 6.799999999999999 2024-05-14T17:01:34.986 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:01:34 smithi123 ceph-mon[35971]: pgmap v211: 97 pgs: 97 active+clean; 4.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 13 MiB/s wr, 1.03k op/s 2024-05-14T17:01:35.015 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:01:34 smithi151 ceph-mon[36066]: pgmap v211: 97 pgs: 97 active+clean; 4.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 13 MiB/s wr, 1.03k op/s 2024-05-14T17:01:35.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.124+0000 7f972cb72640 1 -- 172.21.15.2:0/2069429104 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9728102330 msgr2=0x7f972810e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.124+0000 7f972cb72640 1 --2- 172.21.15.2:0/2069429104 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9728102330 0x7f972810e840 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f9720009cd0 tx=0x7f972002a8e0 comp rx=0 tx=0).stop 2024-05-14T17:01:35.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.124+0000 7f972cb72640 1 -- 172.21.15.2:0/2069429104 shutdown_connections 2024-05-14T17:01:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.124+0000 7f972cb72640 1 --2- 172.21.15.2:0/2069429104 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9728102330 0x7f972810e840 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.124+0000 7f972cb72640 1 --2- 172.21.15.2:0/2069429104 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9728101970 0x7f9728101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.124+0000 7f972cb72640 1 --2- 172.21.15.2:0/2069429104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9728107980 0x7f9728107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:35.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.124+0000 7f972cb72640 1 -- 172.21.15.2:0/2069429104 >> 172.21.15.2:0/2069429104 conn(0x7f9728077cd0 msgr2=0x7f97280780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.124+0000 7f972cb72640 1 -- 172.21.15.2:0/2069429104 shutdown_connections 2024-05-14T17:01:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.124+0000 7f972cb72640 1 -- 172.21.15.2:0/2069429104 wait complete. 2024-05-14T17:01:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.125+0000 7f972cb72640 1 Processor -- start 2024-05-14T17:01:35.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.125+0000 7f972cb72640 1 -- start start 2024-05-14T17:01:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.126+0000 7f972cb72640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9728101970 0x7f9728199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:35.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.126+0000 7f972cb72640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9728102330 0x7f972819a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.126+0000 7f972cb72640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9728107980 0x7f97281a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.126+0000 7f972cb72640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9728111c90 con 0x7f9728102330 2024-05-14T17:01:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.126+0000 7f972cb72640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9728111b10 con 0x7f9728107980 2024-05-14T17:01:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.126+0000 7f972cb72640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9728111e10 con 0x7f9728101970 2024-05-14T17:01:35.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.126+0000 7f971dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9728102330 0x7f972819a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.126+0000 7f971dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9728102330 0x7f972819a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57374/0 (socket says 172.21.15.2:57374) 2024-05-14T17:01:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.126+0000 7f971dd9b640 1 -- 172.21.15.2:0/1333336002 learned_addr learned my addr 172.21.15.2:0/1333336002 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:01:35.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.126+0000 7f972659c640 1 --2- 172.21.15.2:0/1333336002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9728101970 0x7f9728199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:35.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.126+0000 7f9726d9d640 1 --2- 172.21.15.2:0/1333336002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9728107980 0x7f97281a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.126+0000 7f971dd9b640 1 -- 172.21.15.2:0/1333336002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9728101970 msgr2=0x7f9728199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:35.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.126+0000 7f971dd9b640 1 --2- 172.21.15.2:0/1333336002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9728101970 0x7f9728199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:35.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.126+0000 7f971dd9b640 1 -- 172.21.15.2:0/1333336002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9728107980 msgr2=0x7f97281a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.126+0000 7f971dd9b640 1 --2- 172.21.15.2:0/1333336002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9728107980 0x7f97281a5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.126+0000 7f971dd9b640 1 -- 172.21.15.2:0/1333336002 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97281a55b0 con 0x7f9728102330 2024-05-14T17:01:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.126+0000 7f971dd9b640 1 --2- 172.21.15.2:0/1333336002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9728102330 0x7f972819a530 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f971000d950 tx=0x7f971000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:35.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.126+0000 7f971ffff640 1 -- 172.21.15.2:0/1333336002 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9710017070 con 0x7f9728102330 2024-05-14T17:01:35.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.127+0000 7f971ffff640 1 -- 172.21.15.2:0/1333336002 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9710009c30 con 0x7f9728102330 2024-05-14T17:01:35.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.127+0000 7f972cb72640 1 -- 172.21.15.2:0/1333336002 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97281a5780 con 0x7f9728102330 2024-05-14T17:01:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.127+0000 7f971ffff640 1 -- 172.21.15.2:0/1333336002 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971000f620 con 0x7f9728102330 2024-05-14T17:01:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.127+0000 7f972cb72640 1 -- 172.21.15.2:0/1333336002 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97281a5c40 con 0x7f9728102330 2024-05-14T17:01:35.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.129+0000 7f972f885740 1 Processor -- start 2024-05-14T17:01:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.130+0000 7f972f885740 1 -- start start 2024-05-14T17:01:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.130+0000 7f971ffff640 1 -- 172.21.15.2:0/1333336002 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f9710010460 con 0x7f9728102330 2024-05-14T17:01:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.130+0000 7f972f885740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ee34c08f0 0x557ee34c0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.130+0000 7f972f885740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ee34c1210 0x557ee34cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.130+0000 7f972f885740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ee34cbad0 0x557ee34cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.130+0000 7f972f885740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x557ee3433c30 con 0x557ee34c1210 2024-05-14T17:01:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.130+0000 7f972f885740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557ee34237d0 con 0x557ee34cbad0 2024-05-14T17:01:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.130+0000 7f972f885740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557ee3427810 con 0x557ee34c08f0 2024-05-14T17:01:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.130+0000 7f972659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ee34c1210 0x557ee34cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.130+0000 7f972659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ee34c1210 0x557ee34cb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57384/0 (socket says 172.21.15.2:57384) 2024-05-14T17:01:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.130+0000 7f972659c640 1 -- 172.21.15.2:0/3711110609 learned_addr learned my addr 172.21.15.2:0/3711110609 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:01:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.130+0000 7f9726d9d640 1 --2- 172.21.15.2:0/3711110609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ee34c08f0 0x557ee34c0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.130+0000 7f971dd9b640 1 --2- 172.21.15.2:0/3711110609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ee34cbad0 0x557ee34cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.130+0000 7f972659c640 1 -- 172.21.15.2:0/3711110609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ee34c08f0 msgr2=0x557ee34c0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.130+0000 7f972659c640 1 --2- 172.21.15.2:0/3711110609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ee34c08f0 0x557ee34c0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:35.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.130+0000 7f972659c640 1 -- 172.21.15.2:0/3711110609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ee34cbad0 msgr2=0x557ee34cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:35.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.130+0000 7f972659c640 1 --2- 172.21.15.2:0/3711110609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ee34cbad0 0x557ee34cdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:35.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.130+0000 7f972659c640 1 -- 172.21.15.2:0/3711110609 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ee3138210 con 0x557ee34c1210 2024-05-14T17:01:35.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.130+0000 7f9726d9d640 1 --2- 172.21.15.2:0/3711110609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ee34c08f0 0x557ee34c0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:01:35.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.130+0000 7f972659c640 1 --2- 172.21.15.2:0/3711110609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ee34c1210 0x557ee34cb590 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f9718009c70 tx=0x7f9718011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3379ee624665518 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:35.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.131+0000 7f971ffff640 1 --2- 172.21.15.2:0/1333336002 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f97140b2a10 0x7f97140b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:35.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.131+0000 7f971cd99640 1 -- 172.21.15.2:0/3711110609 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9718004230 con 0x557ee34c1210 2024-05-14T17:01:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.131+0000 7f971cd99640 1 -- 172.21.15.2:0/3711110609 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97180043d0 con 0x557ee34c1210 2024-05-14T17:01:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.131+0000 7f971cd99640 1 -- 172.21.15.2:0/3711110609 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97180196d0 con 0x557ee34c1210 2024-05-14T17:01:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.131+0000 7f9726d9d640 1 --2- 172.21.15.2:0/1333336002 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f97140b2a10 0x7f97140b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.131+0000 7f971ffff640 1 -- 172.21.15.2:0/1333336002 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7f97100e2100 con 0x7f9728102330 2024-05-14T17:01:35.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.131+0000 7f972f885740 1 -- 172.21.15.2:0/3711110609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ee34c1210 msgr2=0x557ee34cb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:35.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.131+0000 7f972f885740 1 --2- 172.21.15.2:0/3711110609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ee34c1210 0x557ee34cb590 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f9718009c70 tx=0x7f9718011810 comp rx=0 tx=0).stop 2024-05-14T17:01:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.131+0000 7f9726d9d640 1 --2- 172.21.15.2:0/1333336002 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f97140b2a10 0x7f97140b4ed0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9720004af0 tx=0x7f9720005d80 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:35.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.132+0000 7f972f885740 1 -- 172.21.15.2:0/3711110609 shutdown_connections 2024-05-14T17:01:35.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.132+0000 7f972f885740 1 --2- 172.21.15.2:0/3711110609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ee34cbad0 0x557ee34cdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:35.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.132+0000 7f972f885740 1 --2- 172.21.15.2:0/3711110609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ee34c1210 0x557ee34cb590 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.132+0000 7f972f885740 1 --2- 172.21.15.2:0/3711110609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ee34c08f0 0x557ee34c0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.132+0000 7f972f885740 1 -- 172.21.15.2:0/3711110609 >> 172.21.15.2:0/3711110609 conn(0x557ee34bce00 msgr2=0x557ee34bd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.132+0000 7f972f885740 1 -- 172.21.15.2:0/3711110609 shutdown_connections 2024-05-14T17:01:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.132+0000 7f972f885740 1 -- 172.21.15.2:0/3711110609 wait complete. 2024-05-14T17:01:36.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.132+0000 7f972f885740 1 -- start start 2024-05-14T17:01:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.132+0000 7f972f885740 1 Processor -- start 2024-05-14T17:01:36.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.132+0000 7f972f885740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ee34c08f0 0x557ee34cb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:36.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.132+0000 7f972f885740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ee34c1210 0x557ee34c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:36.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.132+0000 7f972f885740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ee34cbad0 0x557ee34c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:36.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.132+0000 7f972f885740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x557ee3109790 con 0x557ee34cbad0 2024-05-14T17:01:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.132+0000 7f972f885740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557ee33df890 con 0x557ee34c08f0 2024-05-14T17:01:36.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.132+0000 7f972f885740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557ee34ce7a0 con 0x557ee34c1210 2024-05-14T17:01:36.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.132+0000 7f972f885740 10 client.0 _ll_register_callbacks cb 0x557ee33e2d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:01:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.132+0000 7f9726d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ee34cbad0 0x557ee34c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.133+0000 7f9726d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ee34cbad0 0x557ee34c8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57398/0 (socket says 172.21.15.2:57398) 2024-05-14T17:01:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.133+0000 7f9726d9d640 1 -- 172.21.15.2:0/794586987 learned_addr learned my addr 172.21.15.2:0/794586987 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:01:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.133+0000 7f971dd9b640 1 --2- 172.21.15.2:0/794586987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ee34c1210 0x557ee34c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:36.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.133+0000 7f972659c640 1 --2- 172.21.15.2:0/794586987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ee34c08f0 0x557ee34cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.133+0000 7f9726d9d640 1 -- 172.21.15.2:0/794586987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ee34c1210 msgr2=0x557ee34c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:36.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.133+0000 7f9726d9d640 1 --2- 172.21.15.2:0/794586987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ee34c1210 0x557ee34c86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.133+0000 7f9726d9d640 1 -- 172.21.15.2:0/794586987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ee34c08f0 msgr2=0x557ee34cb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.133+0000 7f9726d9d640 1 --2- 172.21.15.2:0/794586987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ee34c08f0 0x557ee34cb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.133+0000 7f9726d9d640 1 -- 172.21.15.2:0/794586987 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ee3204620 con 0x557ee34cbad0 2024-05-14T17:01:36.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.133+0000 7f972659c640 1 --2- 172.21.15.2:0/794586987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ee34c08f0 0x557ee34cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:01:36.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.133+0000 7f971dd9b640 1 --2- 172.21.15.2:0/794586987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ee34c1210 0x557ee34c86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:01:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.133+0000 7f9726d9d640 1 --2- 172.21.15.2:0/794586987 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ee34cbad0 0x557ee34c8c30 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9720009cd0 tx=0x7f972002a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14541faeecbeac server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.133+0000 7f970dffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:01:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.133+0000 7f970dffb640 1 -- 172.21.15.2:0/794586987 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9720056060 con 0x557ee34cbad0 2024-05-14T17:01:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.133+0000 7f972f885740 1 -- 172.21.15.2:0/794586987 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97180043d0 con 0x557ee34cbad0 2024-05-14T17:01:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.133+0000 7f970dffb640 1 -- 172.21.15.2:0/794586987 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7f9720041030 con 0x557ee34cbad0 2024-05-14T17:01:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.133+0000 7f970dffb640 1 -- 172.21.15.2:0/794586987 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f97200305a0 con 0x557ee34cbad0 2024-05-14T17:01:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.133+0000 7f972f885740 10 client.16065 fetch_fsmap learned FSMap version 40 2024-05-14T17:01:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.133+0000 7f972f885740 1 -- 172.21.15.2:0/794586987 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x557ee34da1f0 con 0x557ee34cbad0 2024-05-14T17:01:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.134+0000 7f970dffb640 1 -- 172.21.15.2:0/794586987 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97180043d0 con 0x557ee34cbad0 2024-05-14T17:01:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.134+0000 7f970dffb640 1 -- 172.21.15.2:0/794586987 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7f9720030070 con 0x557ee34cbad0 2024-05-14T17:01:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.134+0000 7f972f885740 10 client.16065 fetch_fsmap finished waiting for FSMap version 40 2024-05-14T17:01:36.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.134+0000 7f972f885740 10 client.16065 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:01:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.134+0000 7f972f885740 20 client.16065 populate_metadata read hostname 'smithi002' 2024-05-14T17:01:36.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.134+0000 7f972f885740 1 --2- 172.21.15.2:0/794586987 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x557ee34db700 0x557ee34ddb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.134+0000 7f972f885740 4 client.16065 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:01:36.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.134+0000 7f972f885740 1 -- 172.21.15.2:0/794586987 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ee3435b80 con 0x557ee34db700 2024-05-14T17:01:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.134+0000 7f972659c640 1 --2- 172.21.15.2:0/794586987 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x557ee34db700 0x557ee34ddb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.135+0000 7f972659c640 1 --2- 172.21.15.2:0/794586987 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x557ee34db700 0x557ee34ddb50 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d84d97463aecf4a server_cookie=2834e8aab25b2d53 in_seq=0 out_seq=0 2024-05-14T17:01:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.135+0000 7f970dffb640 10 client.16065 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:01:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.135+0000 7f970dffb640 1 -- 172.21.15.2:0/794586987 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557ee3435b80 con 0x557ee34db700 2024-05-14T17:01:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.135+0000 7f970dffb640 10 client.16065 handle_command_reply: tid=0 2024-05-14T17:01:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.135+0000 7f972f885740 1 client.16065 shutdown 2024-05-14T17:01:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.135+0000 7f972f885740 1 -- 172.21.15.2:0/794586987 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ee34cbad0 msgr2=0x557ee34c8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:36.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.135+0000 7f972f885740 1 --2- 172.21.15.2:0/794586987 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ee34cbad0 0x557ee34c8c30 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9720009cd0 tx=0x7f972002a8e0 comp rx=0 tx=0).stop 2024-05-14T17:01:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.135+0000 7f972f885740 1 -- 172.21.15.2:0/794586987 shutdown_connections 2024-05-14T17:01:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.135+0000 7f972f885740 1 --2- 172.21.15.2:0/794586987 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x557ee34db700 0x557ee34ddb50 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:36.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.135+0000 7f972f885740 1 --2- 172.21.15.2:0/794586987 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ee34cbad0 0x557ee34c8c30 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:36.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.136+0000 7f970dffb640 0 client.16065 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:01:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.136+0000 7f972f885740 1 --2- 172.21.15.2:0/794586987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ee34c1210 0x557ee34c86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.136+0000 7f972f885740 1 --2- 172.21.15.2:0/794586987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ee34c08f0 0x557ee34cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:36.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.136+0000 7f972f885740 1 -- 172.21.15.2:0/794586987 >> 172.21.15.2:0/794586987 conn(0x557ee34bce00 msgr2=0x557ee34bc100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.136+0000 7f972f885740 1 -- 172.21.15.2:0/794586987 shutdown_connections 2024-05-14T17:01:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.136+0000 7f972f885740 1 -- 172.21.15.2:0/794586987 wait complete. 2024-05-14T17:01:36.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.136+0000 7f972f885740 20 client.16065 trim_cache size 0 max 16384 2024-05-14T17:01:36.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.151+0000 7f972f885740 1 Processor -- start 2024-05-14T17:01:36.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.151+0000 7f972f885740 1 -- start start 2024-05-14T17:01:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.151+0000 7f972f885740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ee34c08f0 0x557ee34e3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:36.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.151+0000 7f972f885740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ee34c1210 0x557ee34e39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:36.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.151+0000 7f972f885740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ee34cbad0 0x557ee34c5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:36.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.151+0000 7f972f885740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f97281b0f30 con 0x557ee34c08f0 2024-05-14T17:01:36.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.151+0000 7f972f885740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f97281ae7d0 con 0x557ee34c1210 2024-05-14T17:01:36.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.151+0000 7f972f885740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f97281ac640 con 0x557ee34cbad0 2024-05-14T17:01:36.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.151+0000 7f972659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ee34c08f0 0x557ee34e3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.151+0000 7f972659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ee34c08f0 0x557ee34e3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57412/0 (socket says 172.21.15.2:57412) 2024-05-14T17:01:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.151+0000 7f972659c640 1 -- 172.21.15.2:0/3368350354 learned_addr learned my addr 172.21.15.2:0/3368350354 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:01:37.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.152+0000 7f9726d9d640 1 --2- 172.21.15.2:0/3368350354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ee34cbad0 0x557ee34c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:37.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.152+0000 7f972659c640 1 -- 172.21.15.2:0/3368350354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ee34cbad0 msgr2=0x557ee34c5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:37.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.152+0000 7f972659c640 1 --2- 172.21.15.2:0/3368350354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ee34cbad0 0x557ee34c5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.152+0000 7f971dd9b640 1 --2- 172.21.15.2:0/3368350354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ee34c1210 0x557ee34e39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.152+0000 7f972659c640 1 -- 172.21.15.2:0/3368350354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ee34c1210 msgr2=0x557ee34e39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:37.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.152+0000 7f972659c640 1 --2- 172.21.15.2:0/3368350354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ee34c1210 0x557ee34e39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:37.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.152+0000 7f972659c640 1 -- 172.21.15.2:0/3368350354 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ee34c1040 con 0x557ee34c08f0 2024-05-14T17:01:37.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.152+0000 7f972659c640 1 --2- 172.21.15.2:0/3368350354 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ee34c08f0 0x557ee34e3460 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f97180058c0 tx=0x7f9718005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50b903dd02768e48 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.152+0000 7f970ffff640 1 -- 172.21.15.2:0/3368350354 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9718032070 con 0x557ee34c08f0 2024-05-14T17:01:37.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.152+0000 7f970ffff640 1 -- 172.21.15.2:0/3368350354 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f971802a070 con 0x557ee34c08f0 2024-05-14T17:01:37.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.152+0000 7f970ffff640 1 -- 172.21.15.2:0/3368350354 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971802ca70 con 0x557ee34c08f0 2024-05-14T17:01:37.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.153+0000 7f972f885740 1 -- 172.21.15.2:0/3368350354 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ee34c08f0 msgr2=0x557ee34e3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.153+0000 7f972f885740 1 --2- 172.21.15.2:0/3368350354 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ee34c08f0 0x557ee34e3460 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f97180058c0 tx=0x7f9718005710 comp rx=0 tx=0).stop 2024-05-14T17:01:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.153+0000 7f972f885740 1 -- 172.21.15.2:0/3368350354 shutdown_connections 2024-05-14T17:01:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.153+0000 7f972f885740 1 --2- 172.21.15.2:0/3368350354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ee34cbad0 0x557ee34c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:37.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.153+0000 7f972f885740 1 --2- 172.21.15.2:0/3368350354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ee34c1210 0x557ee34e39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:37.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.153+0000 7f972f885740 1 --2- 172.21.15.2:0/3368350354 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ee34c08f0 0x557ee34e3460 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.153+0000 7f972f885740 1 -- 172.21.15.2:0/3368350354 >> 172.21.15.2:0/3368350354 conn(0x557ee34bce00 msgr2=0x557ee34d6660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.153+0000 7f972f885740 1 -- 172.21.15.2:0/3368350354 shutdown_connections 2024-05-14T17:01:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.153+0000 7f972f885740 1 -- 172.21.15.2:0/3368350354 wait complete. 2024-05-14T17:01:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.153+0000 7f972f885740 1 -- start start 2024-05-14T17:01:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.153+0000 7f972f885740 1 Processor -- start 2024-05-14T17:01:37.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.153+0000 7f972f885740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ee34c08f0 0x557ee34d1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:37.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.153+0000 7f972f885740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ee34c1210 0x557ee34d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.153+0000 7f972f885740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ee34cbad0 0x557ee3545020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:37.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.153+0000 7f972f885740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f97281acc90 con 0x557ee34c08f0 2024-05-14T17:01:37.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.153+0000 7f972f885740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f97281ae100 con 0x557ee34c1210 2024-05-14T17:01:37.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.153+0000 7f972f885740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f97281ad360 con 0x557ee34cbad0 2024-05-14T17:01:37.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.153+0000 7f972f885740 10 client.0 _ll_register_callbacks cb 0x7f9720031cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:01:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.153+0000 7f972659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ee34c08f0 0x557ee34d1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:37.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.154+0000 7f972659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ee34c08f0 0x557ee34d1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57418/0 (socket says 172.21.15.2:57418) 2024-05-14T17:01:37.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.154+0000 7f972659c640 1 -- 172.21.15.2:0/4126281858 learned_addr learned my addr 172.21.15.2:0/4126281858 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:01:37.514 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.154+0000 7f971dd9b640 1 --2- 172.21.15.2:0/4126281858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ee34c1210 0x557ee34d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:37.551 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.154+0000 7f9726d9d640 1 --2- 172.21.15.2:0/4126281858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ee34cbad0 0x557ee3545020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.154+0000 7f972659c640 1 -- 172.21.15.2:0/4126281858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ee34cbad0 msgr2=0x557ee3545020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:37.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.154+0000 7f972659c640 1 --2- 172.21.15.2:0/4126281858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ee34cbad0 0x557ee3545020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.154+0000 7f972659c640 1 -- 172.21.15.2:0/4126281858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ee34c1210 msgr2=0x557ee34d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.154+0000 7f972659c640 1 --2- 172.21.15.2:0/4126281858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ee34c1210 0x557ee34d1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.154+0000 7f972659c640 1 -- 172.21.15.2:0/4126281858 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ee34d2780 con 0x557ee34c08f0 2024-05-14T17:01:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.154+0000 7f971dd9b640 1 --2- 172.21.15.2:0/4126281858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ee34c1210 0x557ee34d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:01:37.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.154+0000 7f972659c640 1 --2- 172.21.15.2:0/4126281858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ee34c08f0 0x557ee34d1940 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9718000c00 tx=0x7f9718005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aabdb7b0efd9687e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.154+0000 7f970dffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:01:37.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.154+0000 7f970dffb640 1 -- 172.21.15.2:0/4126281858 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9718032070 con 0x557ee34c08f0 2024-05-14T17:01:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.154+0000 7f970dffb640 1 -- 172.21.15.2:0/4126281858 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7f971801f040 con 0x557ee34c08f0 2024-05-14T17:01:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.154+0000 7f972f885740 1 -- 172.21.15.2:0/4126281858 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f971802a070 con 0x557ee34c08f0 2024-05-14T17:01:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.154+0000 7f970dffb640 1 -- 172.21.15.2:0/4126281858 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f971802c320 con 0x557ee34c08f0 2024-05-14T17:01:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.154+0000 7f972f885740 10 client.16071 fetch_fsmap learned FSMap version 40 2024-05-14T17:01:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.154+0000 7f972f885740 1 -- 172.21.15.2:0/4126281858 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x557ee35466a0 con 0x557ee34c08f0 2024-05-14T17:01:37.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.154+0000 7f970dffb640 1 -- 172.21.15.2:0/4126281858 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f971802a070 con 0x557ee34c08f0 2024-05-14T17:01:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.155+0000 7f970dffb640 1 -- 172.21.15.2:0/4126281858 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7f9718018070 con 0x557ee34c08f0 2024-05-14T17:01:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.155+0000 7f972f885740 10 client.16071 fetch_fsmap finished waiting for FSMap version 40 2024-05-14T17:01:37.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.155+0000 7f972f885740 10 client.16071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:01:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.155+0000 7f972f885740 20 client.16071 populate_metadata read hostname 'smithi002' 2024-05-14T17:01:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.155+0000 7f972f885740 1 --2- 172.21.15.2:0/4126281858 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x557ee34db700 0x557ee3548b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.155+0000 7f972f885740 4 client.16071 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:01:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.155+0000 7f972f885740 1 -- 172.21.15.2:0/4126281858 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557ee342f360 con 0x557ee34db700 2024-05-14T17:01:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.155+0000 7f971dd9b640 1 --2- 172.21.15.2:0/4126281858 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x557ee34db700 0x557ee3548b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.155+0000 7f970dffb640 10 client.16071 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:01:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.155+0000 7f971dd9b640 1 --2- 172.21.15.2:0/4126281858 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x557ee34db700 0x557ee3548b00 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c299cc4073fd2299 server_cookie=622653ccfbf7cd2e in_seq=0 out_seq=0 2024-05-14T17:01:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.172+0000 7f970dffb640 1 -- 172.21.15.2:0/4126281858 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557ee342f360 con 0x557ee34db700 2024-05-14T17:01:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.172+0000 7f970dffb640 10 client.16071 handle_command_reply: tid=0 2024-05-14T17:01:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.172+0000 7f972f885740 1 client.16071 shutdown 2024-05-14T17:01:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.172+0000 7f972f885740 1 -- 172.21.15.2:0/4126281858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ee34c08f0 msgr2=0x557ee34d1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:37.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.172+0000 7f972f885740 1 --2- 172.21.15.2:0/4126281858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ee34c08f0 0x557ee34d1940 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9718000c00 tx=0x7f9718005ab0 comp rx=0 tx=0).stop 2024-05-14T17:01:37.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.172+0000 7f972f885740 1 -- 172.21.15.2:0/4126281858 shutdown_connections 2024-05-14T17:01:37.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.172+0000 7f972f885740 1 --2- 172.21.15.2:0/4126281858 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x557ee34db700 0x557ee3548b00 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.172+0000 7f972f885740 1 --2- 172.21.15.2:0/4126281858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ee34cbad0 0x557ee3545020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.172+0000 7f970dffb640 0 client.16071 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:01:38.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.172+0000 7f972f885740 1 --2- 172.21.15.2:0/4126281858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ee34c1210 0x557ee34d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.172+0000 7f972f885740 1 --2- 172.21.15.2:0/4126281858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ee34c08f0 0x557ee34d1940 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.172+0000 7f972f885740 1 -- 172.21.15.2:0/4126281858 >> 172.21.15.2:0/4126281858 conn(0x557ee34bce00 msgr2=0x557ee34d6660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:38.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.172+0000 7f972f885740 1 -- 172.21.15.2:0/4126281858 shutdown_connections 2024-05-14T17:01:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.172+0000 7f972f885740 1 -- 172.21.15.2:0/4126281858 wait complete. 2024-05-14T17:01:38.062 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.172+0000 7f972f885740 20 client.16071 trim_cache size 0 max 16384 2024-05-14T17:01:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:01:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:01:38.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:01:38.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.173+0000 7f972cb72640 1 -- 172.21.15.2:0/1333336002 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f97140b2a10 msgr2=0x7f97140b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.173+0000 7f972cb72640 1 --2- 172.21.15.2:0/1333336002 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f97140b2a10 0x7f97140b4ed0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9720004af0 tx=0x7f9720005d80 comp rx=0 tx=0).stop 2024-05-14T17:01:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.173+0000 7f972cb72640 1 -- 172.21.15.2:0/1333336002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9728102330 msgr2=0x7f972819a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.173+0000 7f972cb72640 1 --2- 172.21.15.2:0/1333336002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9728102330 0x7f972819a530 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f971000d950 tx=0x7f971000de10 comp rx=0 tx=0).stop 2024-05-14T17:01:38.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.173+0000 7f972cb72640 1 -- 172.21.15.2:0/1333336002 shutdown_connections 2024-05-14T17:01:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.173+0000 7f972cb72640 1 --2- 172.21.15.2:0/1333336002 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f97140b2a10 0x7f97140b4ed0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:38.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.173+0000 7f972cb72640 1 --2- 172.21.15.2:0/1333336002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9728107980 0x7f97281a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:38.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.173+0000 7f972cb72640 1 --2- 172.21.15.2:0/1333336002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9728102330 0x7f972819a530 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.173+0000 7f972cb72640 1 --2- 172.21.15.2:0/1333336002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9728101970 0x7f9728199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.173+0000 7f972cb72640 1 -- 172.21.15.2:0/1333336002 >> 172.21.15.2:0/1333336002 conn(0x7f9728077cd0 msgr2=0x7f97280ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.173+0000 7f972cb72640 1 -- 172.21.15.2:0/1333336002 shutdown_connections 2024-05-14T17:01:38.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:33.173+0000 7f972cb72640 1 -- 172.21.15.2:0/1333336002 wait complete. 2024-05-14T17:01:38.246 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:01:34 smithi002 ceph-mon[31538]: pgmap v211: 97 pgs: 97 active+clean; 4.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 13 MiB/s wr, 1.03k op/s 2024-05-14T17:01:38.252 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 380.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:01:38.259 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:01:36 smithi002 ceph-mon[31538]: pgmap v212: 97 pgs: 97 active+clean; 4.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.3 MiB/s wr, 729 op/s 2024-05-14T17:01:38.266 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:01:36 smithi123 ceph-mon[35971]: pgmap v212: 97 pgs: 97 active+clean; 4.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.3 MiB/s wr, 729 op/s 2024-05-14T17:01:38.273 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:01:36 smithi151 ceph-mon[36066]: pgmap v212: 97 pgs: 97 active+clean; 4.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.3 MiB/s wr, 729 op/s 2024-05-14T17:01:38.409 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:01:38 smithi123 ceph-mon[35971]: pgmap v213: 97 pgs: 97 active+clean; 4.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.3 MiB/s wr, 729 op/s 2024-05-14T17:01:38.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:01:38 smithi002 ceph-mon[31538]: pgmap v213: 97 pgs: 97 active+clean; 4.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.3 MiB/s wr, 729 op/s 2024-05-14T17:01:38.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:01:38 smithi151 ceph-mon[36066]: pgmap v213: 97 pgs: 97 active+clean; 4.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.3 MiB/s wr, 729 op/s 2024-05-14T17:01:40.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:01:40 smithi123 ceph-mon[35971]: pgmap v214: 97 pgs: 97 active+clean; 4.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 9.3 MiB/s wr, 737 op/s 2024-05-14T17:01:40.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:01:40 smithi002 ceph-mon[31538]: pgmap v214: 97 pgs: 97 active+clean; 4.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 9.3 MiB/s wr, 737 op/s 2024-05-14T17:01:40.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:01:40 smithi151 ceph-mon[36066]: pgmap v214: 97 pgs: 97 active+clean; 4.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 9.3 MiB/s wr, 737 op/s 2024-05-14T17:01:41.751 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 385.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:01:42.188 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:01:42.574 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:01:42.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:01:42 smithi123 ceph-mon[35971]: pgmap v215: 97 pgs: 97 active+clean; 4.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 3.1 MiB/s wr, 271 op/s 2024-05-14T17:01:42.693 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:01:42 smithi002 ceph-mon[31538]: pgmap v215: 97 pgs: 97 active+clean; 4.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 3.1 MiB/s wr, 271 op/s 2024-05-14T17:01:42.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:01:42 smithi151 ceph-mon[36066]: pgmap v215: 97 pgs: 97 active+clean; 4.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 3.1 MiB/s wr, 271 op/s 2024-05-14T17:01:44.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.330+0000 7f2a0b9f6700 1 -- 172.21.15.2:0/1295276659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a0406bbe0 msgr2=0x7f2a041103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:44.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.330+0000 7f2a0b9f6700 1 --2- 172.21.15.2:0/1295276659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a0406bbe0 0x7f2a041103d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f29fc00b0a0 tx=0x7f29fc00b3b0 comp rx=0 tx=0).stop 2024-05-14T17:01:44.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.331+0000 7f2a0b9f6700 1 -- 172.21.15.2:0/1295276659 shutdown_connections 2024-05-14T17:01:44.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.331+0000 7f2a0b9f6700 1 --2- 172.21.15.2:0/1295276659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a0406bbe0 0x7f2a041103d0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:44.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.331+0000 7f2a0b9f6700 1 --2- 172.21.15.2:0/1295276659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a041095b0 0x7f2a04109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:44.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.331+0000 7f2a0b9f6700 1 --2- 172.21.15.2:0/1295276659 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a0406b220 0x7f2a0406b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:44.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.331+0000 7f2a0b9f6700 1 -- 172.21.15.2:0/1295276659 >> 172.21.15.2:0/1295276659 conn(0x7f2a040793f0 msgr2=0x7f2a04079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:44.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.331+0000 7f2a0b9f6700 1 -- 172.21.15.2:0/1295276659 shutdown_connections 2024-05-14T17:01:44.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.331+0000 7f2a0b9f6700 1 -- 172.21.15.2:0/1295276659 wait complete. 2024-05-14T17:01:44.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.331+0000 7f2a0b9f6700 1 Processor -- start 2024-05-14T17:01:44.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.331+0000 7f2a0b9f6700 1 -- start start 2024-05-14T17:01:44.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.332+0000 7f2a0b9f6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a0406b220 0x7f2a04102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:44.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.332+0000 7f2a0b9f6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a0406bbe0 0x7f2a04102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:44.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.332+0000 7f2a0b9f6700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a041095b0 0x7f2a04102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:44.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.332+0000 7f2a0b9f6700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2a04113720 con 0x7f2a0406bbe0 2024-05-14T17:01:44.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.332+0000 7f2a0b9f6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2a041135a0 con 0x7f2a0406b220 2024-05-14T17:01:44.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.332+0000 7f2a0b9f6700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2a041138a0 con 0x7f2a041095b0 2024-05-14T17:01:44.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.332+0000 7f2a08f91700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a0406bbe0 0x7f2a04102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:44.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.332+0000 7f2a08f91700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a0406bbe0 0x7f2a04102a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52518/0 (socket says 172.21.15.2:52518) 2024-05-14T17:01:44.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.332+0000 7f2a08f91700 1 -- 172.21.15.2:0/1806362379 learned_addr learned my addr 172.21.15.2:0/1806362379 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:01:44.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.332+0000 7f2a09f93700 1 --2- 172.21.15.2:0/1806362379 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a041095b0 0x7f2a04102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:44.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.332+0000 7f2a09792700 1 --2- 172.21.15.2:0/1806362379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a0406b220 0x7f2a04102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:44.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.332+0000 7f2a08f91700 1 -- 172.21.15.2:0/1806362379 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a041095b0 msgr2=0x7f2a04102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:44.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.332+0000 7f2a08f91700 1 --2- 172.21.15.2:0/1806362379 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a041095b0 0x7f2a04102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:44.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.332+0000 7f2a08f91700 1 -- 172.21.15.2:0/1806362379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a0406b220 msgr2=0x7f2a04102510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:44.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.332+0000 7f2a08f91700 1 --2- 172.21.15.2:0/1806362379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a0406b220 0x7f2a04102510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:44.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.332+0000 7f2a08f91700 1 -- 172.21.15.2:0/1806362379 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a041a80f0 con 0x7f2a0406bbe0 2024-05-14T17:01:44.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.332+0000 7f2a09792700 1 --2- 172.21.15.2:0/1806362379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a0406b220 0x7f2a04102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:01:44.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.332+0000 7f2a08f91700 1 --2- 172.21.15.2:0/1806362379 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a0406bbe0 0x7f2a04102a50 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f29f400c9b0 tx=0x7f29f400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:44.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.332+0000 7f29fa7fc700 1 -- 172.21.15.2:0/1806362379 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f4007b00 con 0x7f2a0406bbe0 2024-05-14T17:01:44.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.333+0000 7f2a0b9f6700 1 -- 172.21.15.2:0/1806362379 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a041a83e0 con 0x7f2a0406bbe0 2024-05-14T17:01:44.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.333+0000 7f29fa7fc700 1 -- 172.21.15.2:0/1806362379 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29f4007ca0 con 0x7f2a0406bbe0 2024-05-14T17:01:44.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.333+0000 7f29fa7fc700 1 -- 172.21.15.2:0/1806362379 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f4018660 con 0x7f2a0406bbe0 2024-05-14T17:01:44.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.333+0000 7f2a0b9f6700 1 -- 172.21.15.2:0/1806362379 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a041a8970 con 0x7f2a0406bbe0 2024-05-14T17:01:44.790 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:01:44 smithi123 ceph-mon[35971]: pgmap v216: 97 pgs: 97 active+clean; 4.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 31 MiB/s wr, 1.07k op/s 2024-05-14T17:01:44.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.334+0000 7f2a0b9f6700 1 -- 172.21.15.2:0/1806362379 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a0406a230 con 0x7f2a0406bbe0 2024-05-14T17:01:44.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.335+0000 7f29fa7fc700 1 -- 172.21.15.2:0/1806362379 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f29f400f450 con 0x7f2a0406bbe0 2024-05-14T17:01:44.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.336+0000 7f29fa7fc700 1 --2- 172.21.15.2:0/1806362379 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f29f00b29f0 0x7f29f00b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:44.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.336+0000 7f2a09792700 1 --2- 172.21.15.2:0/1806362379 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f29f00b29f0 0x7f29f00b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:44.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.336+0000 7f29fa7fc700 1 -- 172.21.15.2:0/1806362379 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7f29f40e28b0 con 0x7f2a0406bbe0 2024-05-14T17:01:44.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.336+0000 7f2a09792700 1 --2- 172.21.15.2:0/1806362379 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f29f00b29f0 0x7f29f00b4eb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2a00005fd0 tx=0x7f2a00005db0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:44.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.339+0000 7f29fa7fc700 1 -- 172.21.15.2:0/1806362379 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f29f4014030 con 0x7f2a0406bbe0 2024-05-14T17:01:44.898 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:01:44 smithi002 ceph-mon[31538]: pgmap v216: 97 pgs: 97 active+clean; 4.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 31 MiB/s wr, 1.07k op/s 2024-05-14T17:01:44.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.613+0000 7f2a0b9f6700 1 -- 172.21.15.2:0/1806362379 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a041039f0 con 0x7f2a0406bbe0 2024-05-14T17:01:44.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.614+0000 7f29fa7fc700 1 -- 172.21.15.2:0/1806362379 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11854 (secure 0 0 0) 0x7f29f40aa320 con 0x7f2a0406bbe0 2024-05-14T17:01:44.924 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:01:44.939 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2275696923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2275696923},{"type":"v1","addr":"172.21.15.2:6835","nonce":2275696923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/716718761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":716718761},{"type":"v1","addr":"172.21.15.2:6839","nonce":716718761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/3644815179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3644815179},{"type":"v1","addr":"172.21.15.2:6841","nonce":3644815179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1880336731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1880336731},{"type":"v1","addr":"172.21.15.151:6833","nonce":1880336731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2415605548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2415605548},{"type":"v1","addr":"172.21.15.2:6837","nonce":2415605548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2635167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2635167},{"type":"v1","addr":"172.21.15.123:6833","nonce":2635167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:51:34.106919+0000","modified":"2024-05-14T17:01:13.455170+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24455,"mds_2":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24415":{"gid":24415,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/762044339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":762044339},{"type":"v1","addr":"172.21.15.151:6835","nonce":762044339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/132043151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":132043151},{"type":"v1","addr":"172.21.15.151:6837","nonce":132043151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/4107834375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":4107834375},{"type":"v1","addr":"172.21.15.151:6839","nonce":4107834375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.123:6835/1198670013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1198670013},{"type":"v1","addr":"172.21.15.123:6835","nonce":1198670013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/333426645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":333426645},{"type":"v1","addr":"172.21.15.123:6837","nonce":333426645}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/580275877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":580275877},{"type":"v1","addr":"172.21.15.123:6839","nonce":580275877}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24431,24455,24479]},"id":1}]} 2024-05-14T17:01:44.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.616+0000 7f2a0b9f6700 1 -- 172.21.15.2:0/1806362379 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f29f00b29f0 msgr2=0x7f29f00b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:44.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.616+0000 7f2a0b9f6700 1 --2- 172.21.15.2:0/1806362379 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f29f00b29f0 0x7f29f00b4eb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2a00005fd0 tx=0x7f2a00005db0 comp rx=0 tx=0).stop 2024-05-14T17:01:44.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.616+0000 7f2a0b9f6700 1 -- 172.21.15.2:0/1806362379 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a0406bbe0 msgr2=0x7f2a04102a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:44.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.616+0000 7f2a0b9f6700 1 --2- 172.21.15.2:0/1806362379 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a0406bbe0 0x7f2a04102a50 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f29f400c9b0 tx=0x7f29f400ccc0 comp rx=0 tx=0).stop 2024-05-14T17:01:44.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.616+0000 7f2a0b9f6700 1 -- 172.21.15.2:0/1806362379 shutdown_connections 2024-05-14T17:01:45.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.616+0000 7f2a0b9f6700 1 --2- 172.21.15.2:0/1806362379 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f29f00b29f0 0x7f29f00b4eb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:45.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.616+0000 7f2a0b9f6700 1 --2- 172.21.15.2:0/1806362379 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a041095b0 0x7f2a04102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:45.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.616+0000 7f2a0b9f6700 1 --2- 172.21.15.2:0/1806362379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a0406b220 0x7f2a04102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:45.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.616+0000 7f2a0b9f6700 1 --2- 172.21.15.2:0/1806362379 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a0406bbe0 0x7f2a04102a50 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:45.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.616+0000 7f2a0b9f6700 1 -- 172.21.15.2:0/1806362379 >> 172.21.15.2:0/1806362379 conn(0x7f2a040793f0 msgr2=0x7f2a04101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:45.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.616+0000 7f2a0b9f6700 1 -- 172.21.15.2:0/1806362379 shutdown_connections 2024-05-14T17:01:45.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:44.617+0000 7f2a0b9f6700 1 -- 172.21.15.2:0/1806362379 wait complete. 2024-05-14T17:01:45.109 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 40 2024-05-14T17:01:45.118 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:01:44 smithi151 ceph-mon[36066]: pgmap v216: 97 pgs: 97 active+clean; 4.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 31 MiB/s wr, 1.07k op/s 2024-05-14T17:01:45.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:01:45 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/1806362379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:01:45.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:01:45 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/1806362379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:01:45.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:01:45 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/1806362379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:01:45.867 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id 3b74f8b4 --release --await' 2024-05-14T17:01:45.877 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id 3b74f8b4 --release --await 2024-05-14T17:01:46.147 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:01:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.264+0000 7f72629da640 1 -- 172.21.15.2:0/1494144365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f725c10d050 msgr2=0x7f725c10f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.264+0000 7f72629da640 1 --2- 172.21.15.2:0/1494144365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f725c10d050 0x7f725c10f4d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f7254009ca0 tx=0x7f725402a950 comp rx=0 tx=0).stop 2024-05-14T17:01:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.264+0000 7f72629da640 1 -- 172.21.15.2:0/1494144365 shutdown_connections 2024-05-14T17:01:46.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.264+0000 7f72629da640 1 --2- 172.21.15.2:0/1494144365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f725c10d050 0x7f725c10f4d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.264+0000 7f72629da640 1 --2- 172.21.15.2:0/1494144365 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f725c1045a0 0x7f725c10ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.264+0000 7f72629da640 1 --2- 172.21.15.2:0/1494144365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f725c103c80 0x7f725c104060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.264+0000 7f72629da640 1 -- 172.21.15.2:0/1494144365 >> 172.21.15.2:0/1494144365 conn(0x7f725c078ad0 msgr2=0x7f725c078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.264+0000 7f72629da640 1 -- 172.21.15.2:0/1494144365 shutdown_connections 2024-05-14T17:01:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.264+0000 7f72629da640 1 -- 172.21.15.2:0/1494144365 wait complete. 2024-05-14T17:01:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.265+0000 7f72629da640 1 Processor -- start 2024-05-14T17:01:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.265+0000 7f72629da640 1 -- start start 2024-05-14T17:01:46.418 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.265+0000 7f72629da640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f725c103c80 0x7f725c199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:46.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.265+0000 7f72629da640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f725c1045a0 0x7f725c1997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.265+0000 7f72629da640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f725c10d050 0x7f725c19daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:46.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.265+0000 7f72629da640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f725c112d20 con 0x7f725c103c80 2024-05-14T17:01:46.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.265+0000 7f72629da640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f725c112ba0 con 0x7f725c1045a0 2024-05-14T17:01:46.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.265+0000 7f72629da640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f725c112ea0 con 0x7f725c10d050 2024-05-14T17:01:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.265+0000 7f725bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f725c103c80 0x7f725c199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:46.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.265+0000 7f725bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f725c103c80 0x7f725c199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52544/0 (socket says 172.21.15.2:52544) 2024-05-14T17:01:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.265+0000 7f725bfff640 1 -- 172.21.15.2:0/2809633283 learned_addr learned my addr 172.21.15.2:0/2809633283 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:01:46.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.265+0000 7f7260f77640 1 --2- 172.21.15.2:0/2809633283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f725c10d050 0x7f725c19daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:46.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.265+0000 7f725b7fe640 1 --2- 172.21.15.2:0/2809633283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f725c1045a0 0x7f725c1997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.266+0000 7f725bfff640 1 -- 172.21.15.2:0/2809633283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f725c10d050 msgr2=0x7f725c19daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:46.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.266+0000 7f725bfff640 1 --2- 172.21.15.2:0/2809633283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f725c10d050 0x7f725c19daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.266+0000 7f725bfff640 1 -- 172.21.15.2:0/2809633283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f725c1045a0 msgr2=0x7f725c1997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.266+0000 7f725bfff640 1 --2- 172.21.15.2:0/2809633283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f725c1045a0 0x7f725c1997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.266+0000 7f725bfff640 1 -- 172.21.15.2:0/2809633283 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f725c19e220 con 0x7f725c103c80 2024-05-14T17:01:46.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.266+0000 7f725bfff640 1 --2- 172.21.15.2:0/2809633283 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f725c103c80 0x7f725c199280 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f724c00c890 tx=0x7f724c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.266+0000 7f72597fa640 1 -- 172.21.15.2:0/2809633283 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724c004220 con 0x7f725c103c80 2024-05-14T17:01:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.266+0000 7f72597fa640 1 -- 172.21.15.2:0/2809633283 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f724c0043c0 con 0x7f725c103c80 2024-05-14T17:01:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.266+0000 7f72597fa640 1 -- 172.21.15.2:0/2809633283 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724c004a00 con 0x7f725c103c80 2024-05-14T17:01:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.266+0000 7f72629da640 1 -- 172.21.15.2:0/2809633283 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f725c19e510 con 0x7f725c103c80 2024-05-14T17:01:46.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.266+0000 7f72629da640 1 -- 172.21.15.2:0/2809633283 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f725c1a5da0 con 0x7f725c103c80 2024-05-14T17:01:46.674 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:01:46 smithi123 ceph-mon[35971]: pgmap v217: 97 pgs: 97 active+clean; 4.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 28 MiB/s wr, 807 op/s 2024-05-14T17:01:46.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.269+0000 7f72657c7740 1 Processor -- start 2024-05-14T17:01:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.269+0000 7f72657c7740 1 -- start start 2024-05-14T17:01:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.269+0000 7f72597fa640 1 -- 172.21.15.2:0/2809633283 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f724c0079b0 con 0x7f725c103c80 2024-05-14T17:01:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.269+0000 7f72657c7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55952649e810 0x55952649ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.269+0000 7f72657c7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55952649f130 0x5595264ad9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.269+0000 7f72657c7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595264adee0 0x5595264b02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.269+0000 7f72657c7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x559526411c30 con 0x5595264adee0 2024-05-14T17:01:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.269+0000 7f72657c7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5595264017d0 con 0x55952649f130 2024-05-14T17:01:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.269+0000 7f72657c7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559526405810 con 0x55952649e810 2024-05-14T17:01:46.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.269+0000 7f725b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595264adee0 0x5595264b02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.269+0000 7f725b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595264adee0 0x5595264b02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52558/0 (socket says 172.21.15.2:52558) 2024-05-14T17:01:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.269+0000 7f725b7fe640 1 -- 172.21.15.2:0/984979063 learned_addr learned my addr 172.21.15.2:0/984979063 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:01:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.269+0000 7f725bfff640 1 --2- 172.21.15.2:0/984979063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55952649f130 0x5595264ad9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:46.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.269+0000 7f7260f77640 1 --2- 172.21.15.2:0/984979063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55952649e810 0x55952649ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.270+0000 7f725b7fe640 1 -- 172.21.15.2:0/984979063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55952649e810 msgr2=0x55952649ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:46.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.270+0000 7f725b7fe640 1 --2- 172.21.15.2:0/984979063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55952649e810 0x55952649ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:46.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.270+0000 7f725b7fe640 1 -- 172.21.15.2:0/984979063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55952649f130 msgr2=0x5595264ad9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:46.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.270+0000 7f725b7fe640 1 --2- 172.21.15.2:0/984979063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55952649f130 0x5595264ad9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:46.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.270+0000 7f725b7fe640 1 -- 172.21.15.2:0/984979063 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559526116210 con 0x5595264adee0 2024-05-14T17:01:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.270+0000 7f725bfff640 1 --2- 172.21.15.2:0/984979063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55952649f130 0x5595264ad9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:01:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.270+0000 7f72597fa640 1 --2- 172.21.15.2:0/2809633283 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f72480b2a10 0x7f72480b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:46.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.270+0000 7f725b7fe640 1 --2- 172.21.15.2:0/984979063 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595264adee0 0x5595264b02d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f724400b7f0 tx=0x7f724400bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a4def0e1bdbf2bd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.270+0000 7f7260f77640 1 --2- 172.21.15.2:0/2809633283 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f72480b2a10 0x7f72480b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.270+0000 7f72597fa640 1 -- 172.21.15.2:0/2809633283 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7f724c0e23c0 con 0x7f725c103c80 2024-05-14T17:01:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.270+0000 7f72527fc640 1 -- 172.21.15.2:0/984979063 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724400d950 con 0x5595264adee0 2024-05-14T17:01:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.270+0000 7f72527fc640 1 -- 172.21.15.2:0/984979063 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7244018070 con 0x5595264adee0 2024-05-14T17:01:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.270+0000 7f72527fc640 1 -- 172.21.15.2:0/984979063 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72440123e0 con 0x5595264adee0 2024-05-14T17:01:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.271+0000 7f72657c7740 1 -- 172.21.15.2:0/984979063 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595264adee0 msgr2=0x5595264b02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:47.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.271+0000 7f7260f77640 1 --2- 172.21.15.2:0/2809633283 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f72480b2a10 0x7f72480b4ed0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7254009a60 tx=0x7f7254005c50 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.271+0000 7f72657c7740 1 --2- 172.21.15.2:0/984979063 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595264adee0 0x5595264b02d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f724400b7f0 tx=0x7f724400bcb0 comp rx=0 tx=0).stop 2024-05-14T17:01:47.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.271+0000 7f72657c7740 1 -- 172.21.15.2:0/984979063 shutdown_connections 2024-05-14T17:01:47.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.271+0000 7f72657c7740 1 --2- 172.21.15.2:0/984979063 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5595264adee0 0x5595264b02d0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:47.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.271+0000 7f72657c7740 1 --2- 172.21.15.2:0/984979063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55952649f130 0x5595264ad9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:47.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.271+0000 7f72657c7740 1 --2- 172.21.15.2:0/984979063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55952649e810 0x55952649ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:47.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.271+0000 7f72657c7740 1 -- 172.21.15.2:0/984979063 >> 172.21.15.2:0/984979063 conn(0x55952649ae00 msgr2=0x55952649b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:47.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.271+0000 7f72657c7740 1 -- 172.21.15.2:0/984979063 shutdown_connections 2024-05-14T17:01:47.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.271+0000 7f72657c7740 1 -- 172.21.15.2:0/984979063 wait complete. 2024-05-14T17:01:47.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.271+0000 7f72657c7740 1 -- start start 2024-05-14T17:01:47.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.271+0000 7f72657c7740 1 Processor -- start 2024-05-14T17:01:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.272+0000 7f72657c7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55952649e810 0x5595264ad850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:47.180 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.272+0000 7f72657c7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55952649f130 0x5595264aab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:47.187 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.272+0000 7f72657c7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595264adee0 0x5595264ab0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:47.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.272+0000 7f72657c7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5595260e7790 con 0x55952649e810 2024-05-14T17:01:47.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.272+0000 7f72657c7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5595263bd890 con 0x55952649f130 2024-05-14T17:01:47.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.272+0000 7f72657c7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5595264b0bb0 con 0x5595264adee0 2024-05-14T17:01:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.272+0000 7f72657c7740 10 client.0 _ll_register_callbacks cb 0x5595263c0d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:01:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.272+0000 7f725bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55952649e810 0x5595264ad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:47.252 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.272+0000 7f725bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55952649e810 0x5595264ad850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52574/0 (socket says 172.21.15.2:52574) 2024-05-14T17:01:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.272+0000 7f725bfff640 1 -- 172.21.15.2:0/2376692235 learned_addr learned my addr 172.21.15.2:0/2376692235 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:01:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.272+0000 7f7260f77640 1 --2- 172.21.15.2:0/2376692235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595264adee0 0x5595264ab0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.272+0000 7f725b7fe640 1 --2- 172.21.15.2:0/2376692235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55952649f130 0x5595264aab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:47.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.272+0000 7f725bfff640 1 -- 172.21.15.2:0/2376692235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595264adee0 msgr2=0x5595264ab0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:47.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.272+0000 7f725bfff640 1 --2- 172.21.15.2:0/2376692235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595264adee0 0x5595264ab0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:47.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.272+0000 7f725bfff640 1 -- 172.21.15.2:0/2376692235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55952649f130 msgr2=0x5595264aab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:47.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.272+0000 7f725bfff640 1 --2- 172.21.15.2:0/2376692235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55952649f130 0x5595264aab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:47.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.272+0000 7f725bfff640 1 -- 172.21.15.2:0/2376692235 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595261e2620 con 0x55952649e810 2024-05-14T17:01:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.272+0000 7f725bfff640 1 --2- 172.21.15.2:0/2376692235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55952649e810 0x5595264ad850 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f724c0058c0 tx=0x7f724c0a9530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e33a0d2863a91c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:47.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.272+0000 7f723b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:01:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.272+0000 7f723b7fe640 1 -- 172.21.15.2:0/2376692235 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724c01d040 con 0x55952649e810 2024-05-14T17:01:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.272+0000 7f723b7fe640 1 -- 172.21.15.2:0/2376692235 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7f724c0a5930 con 0x55952649e810 2024-05-14T17:01:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.273+0000 7f72657c7740 1 -- 172.21.15.2:0/2376692235 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7244018070 con 0x55952649e810 2024-05-14T17:01:47.396 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.273+0000 7f723b7fe640 1 -- 172.21.15.2:0/2376692235 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f724c0a91b0 con 0x55952649e810 2024-05-14T17:01:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.273+0000 7f72657c7740 10 client.16089 fetch_fsmap learned FSMap version 40 2024-05-14T17:01:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.273+0000 7f72657c7740 1 -- 172.21.15.2:0/2376692235 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5595264bc5d0 con 0x55952649e810 2024-05-14T17:01:47.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.273+0000 7f723b7fe640 1 -- 172.21.15.2:0/2376692235 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7244018070 con 0x55952649e810 2024-05-14T17:01:47.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.273+0000 7f723b7fe640 1 -- 172.21.15.2:0/2376692235 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7f724c0910c0 con 0x55952649e810 2024-05-14T17:01:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.273+0000 7f72657c7740 10 client.16089 fetch_fsmap finished waiting for FSMap version 40 2024-05-14T17:01:47.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.273+0000 7f72657c7740 10 client.16089 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:01:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.273+0000 7f72657c7740 20 client.16089 populate_metadata read hostname 'smithi002' 2024-05-14T17:01:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.273+0000 7f72657c7740 1 --2- 172.21.15.2:0/2376692235 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5595264bda90 0x5595264bfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.273+0000 7f72657c7740 4 client.16089 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:01:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.273+0000 7f72657c7740 1 -- 172.21.15.2:0/2376692235 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559526413b80 con 0x5595264bda90 2024-05-14T17:01:47.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.273+0000 7f725b7fe640 1 --2- 172.21.15.2:0/2376692235 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5595264bda90 0x5595264bfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.274+0000 7f723b7fe640 10 client.16089 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:01:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.274+0000 7f725b7fe640 1 --2- 172.21.15.2:0/2376692235 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5595264bda90 0x5595264bfee0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c5ddf8dbc88a7cc server_cookie=f64a8b0dd6b82ad1 in_seq=0 out_seq=0 2024-05-14T17:01:47.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.275+0000 7f723b7fe640 1 -- 172.21.15.2:0/2376692235 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559526413b80 con 0x5595264bda90 2024-05-14T17:01:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.275+0000 7f723b7fe640 10 client.16089 handle_command_reply: tid=0 2024-05-14T17:01:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.275+0000 7f72657c7740 1 client.16089 shutdown 2024-05-14T17:01:47.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.275+0000 7f72657c7740 1 -- 172.21.15.2:0/2376692235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55952649e810 msgr2=0x5595264ad850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.275+0000 7f72657c7740 1 --2- 172.21.15.2:0/2376692235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55952649e810 0x5595264ad850 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f724c0058c0 tx=0x7f724c0a9530 comp rx=0 tx=0).stop 2024-05-14T17:01:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.275+0000 7f72657c7740 1 -- 172.21.15.2:0/2376692235 shutdown_connections 2024-05-14T17:01:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.275+0000 7f72657c7740 1 --2- 172.21.15.2:0/2376692235 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5595264bda90 0x5595264bfee0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.275+0000 7f72657c7740 1 --2- 172.21.15.2:0/2376692235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595264adee0 0x5595264ab0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.275+0000 7f72657c7740 1 --2- 172.21.15.2:0/2376692235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55952649f130 0x5595264aab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.275+0000 7f72657c7740 1 --2- 172.21.15.2:0/2376692235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55952649e810 0x5595264ad850 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.275+0000 7f72657c7740 1 -- 172.21.15.2:0/2376692235 >> 172.21.15.2:0/2376692235 conn(0x55952649ae00 msgr2=0x5595264a0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:47.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.275+0000 7f723b7fe640 0 client.16089 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:01:47.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.275+0000 7f72657c7740 1 -- 172.21.15.2:0/2376692235 shutdown_connections 2024-05-14T17:01:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.275+0000 7f72657c7740 1 -- 172.21.15.2:0/2376692235 wait complete. 2024-05-14T17:01:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.276+0000 7f72657c7740 20 client.16089 trim_cache size 0 max 16384 2024-05-14T17:01:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.291+0000 7f72657c7740 1 Processor -- start 2024-05-14T17:01:47.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.291+0000 7f72657c7740 1 -- start start 2024-05-14T17:01:47.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.291+0000 7f72657c7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55952649e810 0x5595264b9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:47.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.291+0000 7f72657c7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55952649f130 0x5595264a6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:47.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.291+0000 7f72657c7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595264adee0 0x5595264ad380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.291+0000 7f72657c7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f725c1aed10 con 0x55952649e810 2024-05-14T17:01:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.291+0000 7f72657c7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f725c1ac5b0 con 0x55952649f130 2024-05-14T17:01:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.291+0000 7f72657c7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f725c1aa440 con 0x5595264adee0 2024-05-14T17:01:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.291+0000 7f725bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55952649e810 0x5595264b9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.291+0000 7f725bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55952649e810 0x5595264b9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52584/0 (socket says 172.21.15.2:52584) 2024-05-14T17:01:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.291+0000 7f725bfff640 1 -- 172.21.15.2:0/3546631092 learned_addr learned my addr 172.21.15.2:0/3546631092 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:01:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.291+0000 7f7260f77640 1 --2- 172.21.15.2:0/3546631092 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595264adee0 0x5595264ad380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.291+0000 7f725b7fe640 1 --2- 172.21.15.2:0/3546631092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55952649f130 0x5595264a6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.291+0000 7f725bfff640 1 -- 172.21.15.2:0/3546631092 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595264adee0 msgr2=0x5595264ad380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:48.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.291+0000 7f725bfff640 1 --2- 172.21.15.2:0/3546631092 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595264adee0 0x5595264ad380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.291+0000 7f725bfff640 1 -- 172.21.15.2:0/3546631092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55952649f130 msgr2=0x5595264a6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:48.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.291+0000 7f725bfff640 1 --2- 172.21.15.2:0/3546631092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55952649f130 0x5595264a6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.291+0000 7f725bfff640 1 -- 172.21.15.2:0/3546631092 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55952649ef60 con 0x55952649e810 2024-05-14T17:01:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.291+0000 7f725b7fe640 1 --2- 172.21.15.2:0/3546631092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55952649f130 0x5595264a6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:01:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.292+0000 7f725bfff640 1 --2- 172.21.15.2:0/3546631092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55952649e810 0x5595264b9a80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f724c0a93a0 tx=0x7f724c0cc0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c0ceece39f31ece server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.292+0000 7f7251ffb640 1 -- 172.21.15.2:0/3546631092 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724c01d040 con 0x55952649e810 2024-05-14T17:01:48.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.292+0000 7f7251ffb640 1 -- 172.21.15.2:0/3546631092 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f724c0a9da0 con 0x55952649e810 2024-05-14T17:01:48.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.292+0000 7f7251ffb640 1 -- 172.21.15.2:0/3546631092 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724c0beaf0 con 0x55952649e810 2024-05-14T17:01:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.292+0000 7f72657c7740 1 -- 172.21.15.2:0/3546631092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55952649e810 msgr2=0x5595264b9a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:48.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.292+0000 7f72657c7740 1 --2- 172.21.15.2:0/3546631092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55952649e810 0x5595264b9a80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f724c0a93a0 tx=0x7f724c0cc0d0 comp rx=0 tx=0).stop 2024-05-14T17:01:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.292+0000 7f72657c7740 1 -- 172.21.15.2:0/3546631092 shutdown_connections 2024-05-14T17:01:48.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.292+0000 7f72657c7740 1 --2- 172.21.15.2:0/3546631092 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595264adee0 0x5595264ad380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.292+0000 7f72657c7740 1 --2- 172.21.15.2:0/3546631092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55952649f130 0x5595264a6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.292+0000 7f72657c7740 1 --2- 172.21.15.2:0/3546631092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55952649e810 0x5595264b9a80 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.292+0000 7f72657c7740 1 -- 172.21.15.2:0/3546631092 >> 172.21.15.2:0/3546631092 conn(0x55952649ae00 msgr2=0x5595264a5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.293+0000 7f72657c7740 1 -- 172.21.15.2:0/3546631092 shutdown_connections 2024-05-14T17:01:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.293+0000 7f72657c7740 1 -- 172.21.15.2:0/3546631092 wait complete. 2024-05-14T17:01:48.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.293+0000 7f72657c7740 1 -- start start 2024-05-14T17:01:48.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.293+0000 7f72657c7740 1 Processor -- start 2024-05-14T17:01:48.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.293+0000 7f72657c7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55952649e810 0x5595264b60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.293+0000 7f72657c7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55952649f130 0x5595264b1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.293+0000 7f72657c7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595264adee0 0x5595264b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:48.382 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.293+0000 7f72657c7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f725c1aaa90 con 0x55952649f130 2024-05-14T17:01:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.293+0000 7f72657c7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f725c1abee0 con 0x55952649e810 2024-05-14T17:01:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.293+0000 7f72657c7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f725c1ab140 con 0x5595264adee0 2024-05-14T17:01:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.293+0000 7f72657c7740 10 client.0 _ll_register_callbacks cb 0x7f724c005760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:01:48.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.293+0000 7f725b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55952649f130 0x5595264b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.293+0000 7f725b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55952649f130 0x5595264b1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52586/0 (socket says 172.21.15.2:52586) 2024-05-14T17:01:48.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.293+0000 7f725b7fe640 1 -- 172.21.15.2:0/3529395180 learned_addr learned my addr 172.21.15.2:0/3529395180 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:01:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.294+0000 7f725bfff640 1 --2- 172.21.15.2:0/3529395180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55952649e810 0x5595264b60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.294+0000 7f725b7fe640 1 -- 172.21.15.2:0/3529395180 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595264adee0 msgr2=0x5595264b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.294+0000 7f725b7fe640 1 --2- 172.21.15.2:0/3529395180 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595264adee0 0x5595264b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.294+0000 7f725b7fe640 1 -- 172.21.15.2:0/3529395180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55952649e810 msgr2=0x5595264b60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:48.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.294+0000 7f725b7fe640 1 --2- 172.21.15.2:0/3529395180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55952649e810 0x5595264b60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:48.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.294+0000 7f725b7fe640 1 -- 172.21.15.2:0/3529395180 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595264b1f60 con 0x55952649f130 2024-05-14T17:01:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.294+0000 7f725bfff640 1 --2- 172.21.15.2:0/3529395180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55952649e810 0x5595264b60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:01:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.294+0000 7f725b7fe640 1 --2- 172.21.15.2:0/3529395180 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55952649f130 0x5595264b1460 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7244000c00 tx=0x7f7244013820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f0816eafc123898 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:48.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.294+0000 7f723b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:01:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.294+0000 7f723b7fe640 1 -- 172.21.15.2:0/3529395180 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724400dd60 con 0x55952649f130 2024-05-14T17:01:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.294+0000 7f723b7fe640 1 -- 172.21.15.2:0/3529395180 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7f72440222f0 con 0x55952649f130 2024-05-14T17:01:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.294+0000 7f72657c7740 1 -- 172.21.15.2:0/3529395180 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f724c0a9da0 con 0x55952649f130 2024-05-14T17:01:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.294+0000 7f723b7fe640 1 -- 172.21.15.2:0/3529395180 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f72440224f0 con 0x55952649f130 2024-05-14T17:01:48.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.294+0000 7f72657c7740 10 client.16095 fetch_fsmap learned FSMap version 40 2024-05-14T17:01:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.294+0000 7f72657c7740 1 -- 172.21.15.2:0/3529395180 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5595265280e0 con 0x55952649f130 2024-05-14T17:01:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.294+0000 7f723b7fe640 1 -- 172.21.15.2:0/3529395180 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f724c0a9da0 con 0x55952649f130 2024-05-14T17:01:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.295+0000 7f723b7fe640 1 -- 172.21.15.2:0/3529395180 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7f7244013ce0 con 0x55952649f130 2024-05-14T17:01:48.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.295+0000 7f72657c7740 10 client.16095 fetch_fsmap finished waiting for FSMap version 40 2024-05-14T17:01:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.295+0000 7f72657c7740 10 client.16095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:01:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.295+0000 7f72657c7740 20 client.16095 populate_metadata read hostname 'smithi002' 2024-05-14T17:01:48.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.295+0000 7f72657c7740 1 --2- 172.21.15.2:0/3529395180 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x559526528710 0x55952652ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.295+0000 7f72657c7740 4 client.16095 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:01:48.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.295+0000 7f72657c7740 1 -- 172.21.15.2:0/3529395180 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55952640d360 con 0x559526528710 2024-05-14T17:01:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.295+0000 7f725bfff640 1 --2- 172.21.15.2:0/3529395180 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x559526528710 0x55952652ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.295+0000 7f723b7fe640 10 client.16095 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:01:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.295+0000 7f725bfff640 1 --2- 172.21.15.2:0/3529395180 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x559526528710 0x55952652ab30 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5732850c0ee9643 server_cookie=8faad0fb70537a1c in_seq=0 out_seq=0 2024-05-14T17:01:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.309+0000 7f723b7fe640 1 -- 172.21.15.2:0/3529395180 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55952640d360 con 0x559526528710 2024-05-14T17:01:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.309+0000 7f723b7fe640 10 client.16095 handle_command_reply: tid=0 2024-05-14T17:01:48.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.309+0000 7f72657c7740 1 client.16095 shutdown 2024-05-14T17:01:48.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.310+0000 7f72657c7740 1 -- 172.21.15.2:0/3529395180 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55952649f130 msgr2=0x5595264b1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.310+0000 7f72657c7740 1 --2- 172.21.15.2:0/3529395180 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55952649f130 0x5595264b1460 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7244000c00 tx=0x7f7244013820 comp rx=0 tx=0).stop 2024-05-14T17:01:48.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.310+0000 7f72657c7740 1 -- 172.21.15.2:0/3529395180 shutdown_connections 2024-05-14T17:01:48.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.310+0000 7f72657c7740 1 --2- 172.21.15.2:0/3529395180 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x559526528710 0x55952652ab30 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.310+0000 7f72657c7740 1 --2- 172.21.15.2:0/3529395180 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595264adee0 0x5595264b19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:48.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.310+0000 7f72657c7740 1 --2- 172.21.15.2:0/3529395180 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55952649f130 0x5595264b1460 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.310+0000 7f72657c7740 1 --2- 172.21.15.2:0/3529395180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55952649e810 0x5595264b60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:48.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.310+0000 7f723b7fe640 0 client.16095 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:01:48.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.310+0000 7f72657c7740 1 -- 172.21.15.2:0/3529395180 >> 172.21.15.2:0/3529395180 conn(0x55952649ae00 msgr2=0x5595264a61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.310+0000 7f72657c7740 1 -- 172.21.15.2:0/3529395180 shutdown_connections 2024-05-14T17:01:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.310+0000 7f72657c7740 1 -- 172.21.15.2:0/3529395180 wait complete. 2024-05-14T17:01:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.310+0000 7f72657c7740 20 client.16095 trim_cache size 0 max 16384 2024-05-14T17:01:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:01:48.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:01:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:01:48.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.310+0000 7f72629da640 1 -- 172.21.15.2:0/2809633283 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f72480b2a10 msgr2=0x7f72480b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.310+0000 7f72629da640 1 --2- 172.21.15.2:0/2809633283 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f72480b2a10 0x7f72480b4ed0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7254009a60 tx=0x7f7254005c50 comp rx=0 tx=0).stop 2024-05-14T17:01:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.310+0000 7f72629da640 1 -- 172.21.15.2:0/2809633283 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f725c103c80 msgr2=0x7f725c199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:49.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.310+0000 7f72629da640 1 --2- 172.21.15.2:0/2809633283 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f725c103c80 0x7f725c199280 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f724c00c890 tx=0x7f724c00cd50 comp rx=0 tx=0).stop 2024-05-14T17:01:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.310+0000 7f72629da640 1 -- 172.21.15.2:0/2809633283 shutdown_connections 2024-05-14T17:01:49.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.310+0000 7f72629da640 1 --2- 172.21.15.2:0/2809633283 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f72480b2a10 0x7f72480b4ed0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.310+0000 7f72629da640 1 --2- 172.21.15.2:0/2809633283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f725c10d050 0x7f725c19daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:49.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.310+0000 7f72629da640 1 --2- 172.21.15.2:0/2809633283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f725c1045a0 0x7f725c1997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:49.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.310+0000 7f72629da640 1 --2- 172.21.15.2:0/2809633283 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f725c103c80 0x7f725c199280 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:49.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.310+0000 7f72629da640 1 -- 172.21.15.2:0/2809633283 >> 172.21.15.2:0/2809633283 conn(0x7f725c078ad0 msgr2=0x7f725c0ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:49.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.311+0000 7f72629da640 1 -- 172.21.15.2:0/2809633283 shutdown_connections 2024-05-14T17:01:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:46.311+0000 7f72629da640 1 -- 172.21.15.2:0/2809633283 wait complete. 2024-05-14T17:01:49.182 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:01:46 smithi002 ceph-mon[31538]: pgmap v217: 97 pgs: 97 active+clean; 4.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 28 MiB/s wr, 807 op/s 2024-05-14T17:01:49.192 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 390.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:01:49.200 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:01:48 smithi123 ceph-mon[35971]: pgmap v218: 97 pgs: 97 active+clean; 4.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 28 MiB/s wr, 807 op/s 2024-05-14T17:01:49.208 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:01:46 smithi151 ceph-mon[36066]: pgmap v217: 97 pgs: 97 active+clean; 4.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 28 MiB/s wr, 807 op/s 2024-05-14T17:01:49.218 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:01:48 smithi151 ceph-mon[36066]: pgmap v218: 97 pgs: 97 active+clean; 4.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 28 MiB/s wr, 807 op/s 2024-05-14T17:01:49.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.392+0000 7f326d63e700 1 -- 172.21.15.2:0/702864920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f32681095e0 msgr2=0x7f32681099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:49.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.392+0000 7f326d63e700 1 --2- 172.21.15.2:0/702864920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f32681095e0 0x7f32681099c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f325800b0a0 tx=0x7f325800b3b0 comp rx=0 tx=0).stop 2024-05-14T17:01:49.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.393+0000 7f326d63e700 1 -- 172.21.15.2:0/702864920 shutdown_connections 2024-05-14T17:01:49.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.393+0000 7f326d63e700 1 --2- 172.21.15.2:0/702864920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f326806c690 0x7f3268110420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:49.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.393+0000 7f326d63e700 1 --2- 172.21.15.2:0/702864920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f32681095e0 0x7f32681099c0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:49.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.393+0000 7f326d63e700 1 --2- 172.21.15.2:0/702864920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f326806bcd0 0x7f326806c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:49.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.393+0000 7f326d63e700 1 -- 172.21.15.2:0/702864920 >> 172.21.15.2:0/702864920 conn(0x7f32680793e0 msgr2=0x7f32680797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:49.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.393+0000 7f326d63e700 1 -- 172.21.15.2:0/702864920 shutdown_connections 2024-05-14T17:01:49.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.393+0000 7f326d63e700 1 -- 172.21.15.2:0/702864920 wait complete. 2024-05-14T17:01:49.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.393+0000 7f326d63e700 1 Processor -- start 2024-05-14T17:01:49.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.393+0000 7f326d63e700 1 -- start start 2024-05-14T17:01:49.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.394+0000 7f326d63e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f326806bcd0 0x7f3268102540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:49.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.394+0000 7f326d63e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f326806c690 0x7f3268102a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:49.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.394+0000 7f326d63e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32681095e0 0x7f3268102fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:49.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.394+0000 7f326d63e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3268113720 con 0x7f326806c690 2024-05-14T17:01:49.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.394+0000 7f326d63e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f32681135a0 con 0x7f326806bcd0 2024-05-14T17:01:49.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.394+0000 7f326d63e700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f32681138a0 con 0x7f32681095e0 2024-05-14T17:01:49.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.394+0000 7f32667fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f326806c690 0x7f3268102a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:49.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.394+0000 7f32667fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f326806c690 0x7f3268102a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52610/0 (socket says 172.21.15.2:52610) 2024-05-14T17:01:49.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.394+0000 7f32667fc700 1 -- 172.21.15.2:0/1948670033 learned_addr learned my addr 172.21.15.2:0/1948670033 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:01:49.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.394+0000 7f32677fe700 1 --2- 172.21.15.2:0/1948670033 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32681095e0 0x7f3268102fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:49.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.394+0000 7f3266ffd700 1 --2- 172.21.15.2:0/1948670033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f326806bcd0 0x7f3268102540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:49.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.394+0000 7f32667fc700 1 -- 172.21.15.2:0/1948670033 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32681095e0 msgr2=0x7f3268102fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:49.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.394+0000 7f32667fc700 1 --2- 172.21.15.2:0/1948670033 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32681095e0 0x7f3268102fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:49.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.394+0000 7f32667fc700 1 -- 172.21.15.2:0/1948670033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f326806bcd0 msgr2=0x7f3268102540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:49.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.394+0000 7f32667fc700 1 --2- 172.21.15.2:0/1948670033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f326806bcd0 0x7f3268102540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:49.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.394+0000 7f32667fc700 1 -- 172.21.15.2:0/1948670033 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32681a8100 con 0x7f326806c690 2024-05-14T17:01:49.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.394+0000 7f32667fc700 1 --2- 172.21.15.2:0/1948670033 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f326806c690 0x7f3268102a80 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f325c00d980 tx=0x7f325c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:49.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.395+0000 7f324ffff700 1 -- 172.21.15.2:0/1948670033 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325c017070 con 0x7f326806c690 2024-05-14T17:01:49.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.395+0000 7f324ffff700 1 -- 172.21.15.2:0/1948670033 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f325c005020 con 0x7f326806c690 2024-05-14T17:01:49.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.395+0000 7f326d63e700 1 -- 172.21.15.2:0/1948670033 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32681a83f0 con 0x7f326806c690 2024-05-14T17:01:49.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.395+0000 7f324ffff700 1 -- 172.21.15.2:0/1948670033 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325c00fa10 con 0x7f326806c690 2024-05-14T17:01:49.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.395+0000 7f326d63e700 1 -- 172.21.15.2:0/1948670033 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32681a88e0 con 0x7f326806c690 2024-05-14T17:01:49.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.397+0000 7f324ffff700 1 -- 172.21.15.2:0/1948670033 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f325c004a30 con 0x7f326806c690 2024-05-14T17:01:49.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.398+0000 7f324ffff700 1 --2- 172.21.15.2:0/1948670033 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f32500b29f0 0x7f32500b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:49.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.398+0000 7f3266ffd700 1 --2- 172.21.15.2:0/1948670033 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f32500b29f0 0x7f32500b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:49.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.398+0000 7f324ffff700 1 -- 172.21.15.2:0/1948670033 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7f325c0e2540 con 0x7f326806c690 2024-05-14T17:01:49.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.398+0000 7f3277ed4b80 1 Processor -- start 2024-05-14T17:01:49.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.399+0000 7f3277ed4b80 1 -- start start 2024-05-14T17:01:49.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.399+0000 7f3266ffd700 1 --2- 172.21.15.2:0/1948670033 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f32500b29f0 0x7f32500b4eb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3258009e60 tx=0x7f325800b7b0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:49.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.399+0000 7f3277ed4b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e85b8c9420 0x55e85b8c9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:49.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.399+0000 7f3277ed4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e85b8c9d40 0x55e85b8c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:49.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.399+0000 7f3277ed4b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e85b8c52c0 0x55e85b8c5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:49.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.399+0000 7f3277ed4b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e85b822e70 con 0x55e85b8c52c0 2024-05-14T17:01:49.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.399+0000 7f3277ed4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e85b812a10 con 0x55e85b8c9d40 2024-05-14T17:01:49.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.399+0000 7f3277ed4b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e85b816a50 con 0x55e85b8c9420 2024-05-14T17:01:49.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.399+0000 7f32677fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e85b8c52c0 0x55e85b8c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:49.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.399+0000 7f32677fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e85b8c52c0 0x55e85b8c5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52622/0 (socket says 172.21.15.2:52622) 2024-05-14T17:01:49.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.399+0000 7f32677fe700 1 -- 172.21.15.2:0/3116976258 learned_addr learned my addr 172.21.15.2:0/3116976258 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:01:49.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.399+0000 7f3266ffd700 1 --2- 172.21.15.2:0/3116976258 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e85b8c9420 0x55e85b8c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:49.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.399+0000 7f32667fc700 1 --2- 172.21.15.2:0/3116976258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e85b8c9d40 0x55e85b8c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:49.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.399+0000 7f32677fe700 1 -- 172.21.15.2:0/3116976258 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e85b8c9420 msgr2=0x55e85b8c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:49.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.399+0000 7f32677fe700 1 --2- 172.21.15.2:0/3116976258 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e85b8c9420 0x55e85b8c9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:49.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.399+0000 7f32677fe700 1 -- 172.21.15.2:0/3116976258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e85b8c9d40 msgr2=0x55e85b8c4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:49.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.399+0000 7f32677fe700 1 --2- 172.21.15.2:0/3116976258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e85b8c9d40 0x55e85b8c4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:49.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.399+0000 7f32677fe700 1 -- 172.21.15.2:0/3116976258 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e85b7bc0b0 con 0x55e85b8c52c0 2024-05-14T17:01:49.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.399+0000 7f32677fe700 1 --2- 172.21.15.2:0/3116976258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e85b8c52c0 0x55e85b8c5720 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f326000ac30 tx=0x7f326000af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3bf8646d72a1cf server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:49.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.400+0000 7f324d7fa700 1 -- 172.21.15.2:0/3116976258 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326000fe90 con 0x55e85b8c52c0 2024-05-14T17:01:49.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.400+0000 7f324d7fa700 1 -- 172.21.15.2:0/3116976258 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3260004ce0 con 0x55e85b8c52c0 2024-05-14T17:01:49.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.400+0000 7f324d7fa700 1 -- 172.21.15.2:0/3116976258 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32600186a0 con 0x55e85b8c52c0 2024-05-14T17:01:49.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.400+0000 7f3277ed4b80 1 -- 172.21.15.2:0/3116976258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e85b8c52c0 msgr2=0x55e85b8c5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:49.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.400+0000 7f3277ed4b80 1 --2- 172.21.15.2:0/3116976258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e85b8c52c0 0x55e85b8c5720 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f326000ac30 tx=0x7f326000af40 comp rx=0 tx=0).stop 2024-05-14T17:01:50.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.400+0000 7f3277ed4b80 1 -- 172.21.15.2:0/3116976258 shutdown_connections 2024-05-14T17:01:50.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.400+0000 7f3277ed4b80 1 --2- 172.21.15.2:0/3116976258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e85b8c52c0 0x55e85b8c5720 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:50.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.400+0000 7f3277ed4b80 1 --2- 172.21.15.2:0/3116976258 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e85b8c9420 0x55e85b8c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:50.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.400+0000 7f3277ed4b80 1 --2- 172.21.15.2:0/3116976258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e85b8c9d40 0x55e85b8c4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:50.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.400+0000 7f3277ed4b80 1 -- 172.21.15.2:0/3116976258 >> 172.21.15.2:0/3116976258 conn(0x55e85b7f1eb0 msgr2=0x55e85b8bf4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:50.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.401+0000 7f3277ed4b80 1 -- 172.21.15.2:0/3116976258 shutdown_connections 2024-05-14T17:01:50.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.401+0000 7f3277ed4b80 1 -- 172.21.15.2:0/3116976258 wait complete. 2024-05-14T17:01:50.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.401+0000 7f3277ed4b80 1 -- start start 2024-05-14T17:01:50.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.401+0000 7f3277ed4b80 1 Processor -- start 2024-05-14T17:01:50.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.401+0000 7f3277ed4b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e85b8c52c0 0x55e85b8d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:50.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.401+0000 7f3277ed4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e85b8c9420 0x55e85b8cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:50.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.401+0000 7f3277ed4b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e85b8c9d40 0x55e85b8ce400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:50.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.401+0000 7f3277ed4b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55e85b7190a0 con 0x55e85b8c9d40 2024-05-14T17:01:50.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.401+0000 7f3277ed4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e85b77aea0 con 0x55e85b8c9420 2024-05-14T17:01:50.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.401+0000 7f3277ed4b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e85b8d1880 con 0x55e85b8c52c0 2024-05-14T17:01:50.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.401+0000 7f3277ed4b80 10 client.0 _ll_register_callbacks cb 0x55e85b7ada50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:01:50.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.401+0000 7f32677fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e85b8c9d40 0x55e85b8ce400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:50.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.401+0000 7f32677fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e85b8c9d40 0x55e85b8ce400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52632/0 (socket says 172.21.15.2:52632) 2024-05-14T17:01:50.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.401+0000 7f32677fe700 1 -- 172.21.15.2:0/309894344 learned_addr learned my addr 172.21.15.2:0/309894344 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:01:50.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.402+0000 7f3266ffd700 1 --2- 172.21.15.2:0/309894344 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e85b8c52c0 0x55e85b8d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:50.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.402+0000 7f32667fc700 1 --2- 172.21.15.2:0/309894344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e85b8c9420 0x55e85b8cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:50.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.402+0000 7f32677fe700 1 -- 172.21.15.2:0/309894344 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e85b8c52c0 msgr2=0x55e85b8d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:50.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.402+0000 7f32677fe700 1 --2- 172.21.15.2:0/309894344 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e85b8c52c0 0x55e85b8d0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:50.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.402+0000 7f32677fe700 1 -- 172.21.15.2:0/309894344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e85b8c9420 msgr2=0x55e85b8cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:50.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.402+0000 7f32677fe700 1 --2- 172.21.15.2:0/309894344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e85b8c9420 0x55e85b8cdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:50.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.402+0000 7f32677fe700 1 -- 172.21.15.2:0/309894344 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e85b7629d0 con 0x55e85b8c9d40 2024-05-14T17:01:50.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.402+0000 7f32677fe700 1 --2- 172.21.15.2:0/309894344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e85b8c9d40 0x55e85b8ce400 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f326000a830 tx=0x7f3260004e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd6a8344e1f9a424 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:50.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.402+0000 7f324aff5700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:01:50.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.402+0000 7f324aff5700 1 -- 172.21.15.2:0/309894344 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3260026020 con 0x55e85b8c9d40 2024-05-14T17:01:50.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.402+0000 7f324aff5700 1 -- 172.21.15.2:0/309894344 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7f3260004950 con 0x55e85b8c9d40 2024-05-14T17:01:50.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.402+0000 7f3277ed4b80 1 -- 172.21.15.2:0/309894344 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3260004ce0 con 0x55e85b8c9d40 2024-05-14T17:01:50.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.403+0000 7f324aff5700 1 -- 172.21.15.2:0/309894344 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f3260031050 con 0x55e85b8c9d40 2024-05-14T17:01:50.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.403+0000 7f3277ed4b80 10 client.16107 fetch_fsmap learned FSMap version 40 2024-05-14T17:01:50.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.403+0000 7f3277ed4b80 1 -- 172.21.15.2:0/309894344 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55e85b7cc860 con 0x55e85b8c9d40 2024-05-14T17:01:50.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.403+0000 7f324aff5700 1 -- 172.21.15.2:0/309894344 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3260004ce0 con 0x55e85b8c9d40 2024-05-14T17:01:50.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.403+0000 7f324aff5700 1 -- 172.21.15.2:0/309894344 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7f3260003ba0 con 0x55e85b8c9d40 2024-05-14T17:01:50.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.403+0000 7f3277ed4b80 10 client.16107 fetch_fsmap finished waiting for FSMap version 40 2024-05-14T17:01:50.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.403+0000 7f3277ed4b80 10 client.16107 resolve_mds: validated gid 24479 aka daemon mds.b 2024-05-14T17:01:50.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.403+0000 7f3277ed4b80 20 client.16107 populate_metadata read hostname 'smithi002' 2024-05-14T17:01:50.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.403+0000 7f3277ed4b80 1 --2- 172.21.15.2:0/309894344 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55e85b8de7b0 0x55e85b8e0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:50.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.403+0000 7f3277ed4b80 4 client.16107 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:01:50.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.403+0000 7f3277ed4b80 1 -- 172.21.15.2:0/309894344 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e85b824dc0 con 0x55e85b8de7b0 2024-05-14T17:01:50.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.403+0000 7f3266ffd700 1 --2- 172.21.15.2:0/309894344 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55e85b8de7b0 0x55e85b8e0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:50.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.404+0000 7f324aff5700 10 client.16107 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:01:50.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.404+0000 7f3266ffd700 1 --2- 172.21.15.2:0/309894344 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55e85b8de7b0 0x55e85b8e0bd0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a99745a16e70322 server_cookie=ff0dc11743ccaf0d in_seq=0 out_seq=0 2024-05-14T17:01:50.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.405+0000 7f324aff5700 1 -- 172.21.15.2:0/309894344 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e85b824dc0 con 0x55e85b8de7b0 2024-05-14T17:01:50.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.405+0000 7f324aff5700 10 client.16107 handle_command_reply: tid=0 2024-05-14T17:01:50.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.405+0000 7f3277ed4b80 1 client.16107 shutdown 2024-05-14T17:01:50.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.405+0000 7f3277ed4b80 1 -- 172.21.15.2:0/309894344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e85b8c9d40 msgr2=0x55e85b8ce400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:50.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.405+0000 7f3277ed4b80 1 --2- 172.21.15.2:0/309894344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e85b8c9d40 0x55e85b8ce400 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f326000a830 tx=0x7f3260004e90 comp rx=0 tx=0).stop 2024-05-14T17:01:50.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.405+0000 7f3277ed4b80 1 -- 172.21.15.2:0/309894344 shutdown_connections 2024-05-14T17:01:50.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.405+0000 7f3277ed4b80 1 --2- 172.21.15.2:0/309894344 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55e85b8de7b0 0x55e85b8e0bd0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:50.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.405+0000 7f3277ed4b80 1 --2- 172.21.15.2:0/309894344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e85b8c9d40 0x55e85b8ce400 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:50.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.405+0000 7f3277ed4b80 1 --2- 172.21.15.2:0/309894344 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e85b8c52c0 0x55e85b8d0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:50.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.405+0000 7f3277ed4b80 1 --2- 172.21.15.2:0/309894344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e85b8c9420 0x55e85b8cdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:50.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.405+0000 7f3277ed4b80 1 -- 172.21.15.2:0/309894344 >> 172.21.15.2:0/309894344 conn(0x55e85b7f1eb0 msgr2=0x55e85b8b9fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:50.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.405+0000 7f324aff5700 0 client.16107 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:01:50.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.405+0000 7f3277ed4b80 1 -- 172.21.15.2:0/309894344 shutdown_connections 2024-05-14T17:01:50.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.405+0000 7f3277ed4b80 1 -- 172.21.15.2:0/309894344 wait complete. 2024-05-14T17:01:50.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.405+0000 7f3277ed4b80 20 client.16107 trim_cache size 0 max 16384 2024-05-14T17:01:50.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.426+0000 7f3277ed4b80 1 Processor -- start 2024-05-14T17:01:50.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.426+0000 7f3277ed4b80 1 -- start start 2024-05-14T17:01:50.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.426+0000 7f3277ed4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e85b8c52c0 0x55e85b8cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:50.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.426+0000 7f3277ed4b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e85b8c9420 0x55e85b8ba0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:50.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.426+0000 7f3277ed4b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e85b8c9d40 0x55e85b8cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:50.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.426+0000 7f3277ed4b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f32681b28f0 con 0x55e85b8c9d40 2024-05-14T17:01:50.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.426+0000 7f3277ed4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f32681b0150 con 0x55e85b8c52c0 2024-05-14T17:01:50.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.426+0000 7f3277ed4b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f32681adfc0 con 0x55e85b8c9420 2024-05-14T17:01:50.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.426+0000 7f32677fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e85b8c9d40 0x55e85b8cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:50.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.426+0000 7f32677fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e85b8c9d40 0x55e85b8cfbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52640/0 (socket says 172.21.15.2:52640) 2024-05-14T17:01:51.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.426+0000 7f32677fe700 1 -- 172.21.15.2:0/2327432288 learned_addr learned my addr 172.21.15.2:0/2327432288 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:01:51.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.426+0000 7f3266ffd700 1 --2- 172.21.15.2:0/2327432288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e85b8c52c0 0x55e85b8cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:51.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.426+0000 7f32667fc700 1 --2- 172.21.15.2:0/2327432288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e85b8c9420 0x55e85b8ba0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:51.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.426+0000 7f32677fe700 1 -- 172.21.15.2:0/2327432288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e85b8c9420 msgr2=0x55e85b8ba0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:51.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.426+0000 7f32677fe700 1 --2- 172.21.15.2:0/2327432288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e85b8c9420 0x55e85b8ba0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:51.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.426+0000 7f32677fe700 1 -- 172.21.15.2:0/2327432288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e85b8c52c0 msgr2=0x55e85b8cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:51.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.426+0000 7f32677fe700 1 --2- 172.21.15.2:0/2327432288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e85b8c52c0 0x55e85b8cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:51.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.426+0000 7f32677fe700 1 -- 172.21.15.2:0/2327432288 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e85b8cc430 con 0x55e85b8c9d40 2024-05-14T17:01:51.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.426+0000 7f32667fc700 1 --2- 172.21.15.2:0/2327432288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e85b8c9420 0x55e85b8ba0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:01:51.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.426+0000 7f32677fe700 1 --2- 172.21.15.2:0/2327432288 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e85b8c9d40 0x55e85b8cfbd0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f326000afb0 tx=0x7f326000a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=747ee2ae1aae5009 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:51.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.427+0000 7f324cff9700 1 -- 172.21.15.2:0/2327432288 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3260026070 con 0x55e85b8c9d40 2024-05-14T17:01:51.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.427+0000 7f324cff9700 1 -- 172.21.15.2:0/2327432288 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3260005470 con 0x55e85b8c9d40 2024-05-14T17:01:51.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.427+0000 7f324cff9700 1 -- 172.21.15.2:0/2327432288 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3260004710 con 0x55e85b8c9d40 2024-05-14T17:01:51.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.427+0000 7f3277ed4b80 1 -- 172.21.15.2:0/2327432288 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e85b8c9d40 msgr2=0x55e85b8cfbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:51.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.427+0000 7f3277ed4b80 1 --2- 172.21.15.2:0/2327432288 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e85b8c9d40 0x55e85b8cfbd0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f326000afb0 tx=0x7f326000a690 comp rx=0 tx=0).stop 2024-05-14T17:01:51.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.427+0000 7f3277ed4b80 1 -- 172.21.15.2:0/2327432288 shutdown_connections 2024-05-14T17:01:51.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.427+0000 7f3277ed4b80 1 --2- 172.21.15.2:0/2327432288 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e85b8c9d40 0x55e85b8cfbd0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:51.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.427+0000 7f3277ed4b80 1 --2- 172.21.15.2:0/2327432288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e85b8c52c0 0x55e85b8cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:51.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.427+0000 7f3277ed4b80 1 --2- 172.21.15.2:0/2327432288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e85b8c9420 0x55e85b8ba0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:51.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.427+0000 7f3277ed4b80 1 -- 172.21.15.2:0/2327432288 >> 172.21.15.2:0/2327432288 conn(0x55e85b7f1eb0 msgr2=0x55e85b8cf4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:51.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.427+0000 7f3277ed4b80 1 -- 172.21.15.2:0/2327432288 shutdown_connections 2024-05-14T17:01:51.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.428+0000 7f3277ed4b80 1 -- 172.21.15.2:0/2327432288 wait complete. 2024-05-14T17:01:51.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.428+0000 7f3277ed4b80 1 -- start start 2024-05-14T17:01:51.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.428+0000 7f3277ed4b80 1 Processor -- start 2024-05-14T17:01:51.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.428+0000 7f3277ed4b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e85b8c52c0 0x55e85b8d6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:51.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.428+0000 7f3277ed4b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e85b8c9420 0x55e85b8d7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:51.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.428+0000 7f3277ed4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e85b8c9d40 0x55e85b8d1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:51.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.428+0000 7f3277ed4b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f32681ae610 con 0x55e85b8c9420 2024-05-14T17:01:51.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.428+0000 7f3277ed4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f32681afa80 con 0x55e85b8c9d40 2024-05-14T17:01:51.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.428+0000 7f3277ed4b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f32681aece0 con 0x55e85b8c52c0 2024-05-14T17:01:51.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.428+0000 7f3277ed4b80 10 client.0 _ll_register_callbacks cb 0x55e85b7ada50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:01:51.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.428+0000 7f32667fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e85b8c9420 0x55e85b8d7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:51.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.428+0000 7f32667fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e85b8c9420 0x55e85b8d7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52642/0 (socket says 172.21.15.2:52642) 2024-05-14T17:01:51.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.428+0000 7f32667fc700 1 -- 172.21.15.2:0/3424264480 learned_addr learned my addr 172.21.15.2:0/3424264480 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:01:51.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.429+0000 7f32677fe700 1 --2- 172.21.15.2:0/3424264480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e85b8c9d40 0x55e85b8d1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:51.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.429+0000 7f3266ffd700 1 --2- 172.21.15.2:0/3424264480 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e85b8c52c0 0x55e85b8d6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:51.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.429+0000 7f32667fc700 1 -- 172.21.15.2:0/3424264480 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e85b8c52c0 msgr2=0x55e85b8d6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:51.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.429+0000 7f32667fc700 1 --2- 172.21.15.2:0/3424264480 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e85b8c52c0 0x55e85b8d6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:51.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.429+0000 7f32667fc700 1 -- 172.21.15.2:0/3424264480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e85b8c9d40 msgr2=0x55e85b8d1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:01:51.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.429+0000 7f32667fc700 1 --2- 172.21.15.2:0/3424264480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e85b8c9d40 0x55e85b8d1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:51.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.429+0000 7f32667fc700 1 -- 172.21.15.2:0/3424264480 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e85b8d2060 con 0x55e85b8c9420 2024-05-14T17:01:51.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.429+0000 7f32677fe700 1 --2- 172.21.15.2:0/3424264480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e85b8c9d40 0x55e85b8d1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:01:51.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.429+0000 7f32667fc700 1 --2- 172.21.15.2:0/3424264480 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e85b8c9420 0x55e85b8d7030 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f325c0a6a30 tx=0x7f325c0057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a6cfcdc48e6fe88 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:51.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.429+0000 7f324aff5700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:01:51.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.429+0000 7f324aff5700 1 -- 172.21.15.2:0/3424264480 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325c0adb70 con 0x55e85b8c9420 2024-05-14T17:01:51.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.429+0000 7f324aff5700 1 -- 172.21.15.2:0/3424264480 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7f325c0a9a20 con 0x55e85b8c9420 2024-05-14T17:01:51.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.429+0000 7f3277ed4b80 1 -- 172.21.15.2:0/3424264480 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3260005470 con 0x55e85b8c9420 2024-05-14T17:01:51.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.429+0000 7f324aff5700 1 -- 172.21.15.2:0/3424264480 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f325c0b2620 con 0x55e85b8c9420 2024-05-14T17:01:51.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.429+0000 7f3277ed4b80 10 client.16113 fetch_fsmap learned FSMap version 40 2024-05-14T17:01:51.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.429+0000 7f3277ed4b80 1 -- 172.21.15.2:0/3424264480 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55e85b949950 con 0x55e85b8c9420 2024-05-14T17:01:51.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.429+0000 7f324aff5700 1 -- 172.21.15.2:0/3424264480 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3260005470 con 0x55e85b8c9420 2024-05-14T17:01:51.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.430+0000 7f324aff5700 1 -- 172.21.15.2:0/3424264480 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7f325c08fce0 con 0x55e85b8c9420 2024-05-14T17:01:51.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.430+0000 7f3277ed4b80 10 client.16113 fetch_fsmap finished waiting for FSMap version 40 2024-05-14T17:01:51.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.430+0000 7f3277ed4b80 10 client.16113 resolve_mds: validated gid 24479 aka daemon mds.b 2024-05-14T17:01:51.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.430+0000 7f3277ed4b80 20 client.16113 populate_metadata read hostname 'smithi002' 2024-05-14T17:01:51.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.430+0000 7f3277ed4b80 1 --2- 172.21.15.2:0/3424264480 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55e85b94a070 0x55e85b94c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:51.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.430+0000 7f3277ed4b80 4 client.16113 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "3b74f8b4", "release": true, "await": true}] 2024-05-14T17:01:51.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.430+0000 7f3277ed4b80 1 -- 172.21.15.2:0/3424264480 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "quiesce db", "set_id": "3b74f8b4", "release": true, "await": true}) -- 0x55e85b81e5a0 con 0x55e85b94a070 2024-05-14T17:01:51.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.430+0000 7f3266ffd700 1 --2- 172.21.15.2:0/3424264480 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55e85b94a070 0x55e85b94c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:51.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.430+0000 7f3266ffd700 1 --2- 172.21.15.2:0/3424264480 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55e85b94a070 0x55e85b94c490 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c63955eea3a924c server_cookie=5cdb7c7e28281dea in_seq=0 out_seq=0 2024-05-14T17:01:51.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.430+0000 7f324aff5700 10 client.16113 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:01:51.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.462+0000 7f324aff5700 1 -- 172.21.15.2:0/3424264480 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e85b81e5a0 con 0x55e85b94a070 2024-05-14T17:01:51.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.462+0000 7f324aff5700 10 client.16113 handle_command_reply: tid=0 2024-05-14T17:01:51.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.462+0000 7f3277ed4b80 1 client.16113 shutdown 2024-05-14T17:01:51.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.462+0000 7f3277ed4b80 1 -- 172.21.15.2:0/3424264480 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e85b8c9420 msgr2=0x55e85b8d7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:51.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.462+0000 7f3277ed4b80 1 --2- 172.21.15.2:0/3424264480 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e85b8c9420 0x55e85b8d7030 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f325c0a6a30 tx=0x7f325c0057a0 comp rx=0 tx=0).stop 2024-05-14T17:01:51.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.462+0000 7f3277ed4b80 1 -- 172.21.15.2:0/3424264480 shutdown_connections 2024-05-14T17:01:51.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.462+0000 7f3277ed4b80 1 --2- 172.21.15.2:0/3424264480 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55e85b94a070 0x55e85b94c490 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:51.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.463+0000 7f3277ed4b80 1 --2- 172.21.15.2:0/3424264480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e85b8c9d40 0x55e85b8d1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:51.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.463+0000 7f3277ed4b80 1 --2- 172.21.15.2:0/3424264480 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e85b8c52c0 0x55e85b8d6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:51.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.463+0000 7f324aff5700 0 client.16113 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:01:51.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.463+0000 7f3277ed4b80 1 --2- 172.21.15.2:0/3424264480 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e85b8c9420 0x55e85b8d7030 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:51.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.463+0000 7f3277ed4b80 1 -- 172.21.15.2:0/3424264480 >> 172.21.15.2:0/3424264480 conn(0x55e85b7f1eb0 msgr2=0x55e85b8cfb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:51.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.463+0000 7f3277ed4b80 1 -- 172.21.15.2:0/3424264480 shutdown_connections 2024-05-14T17:01:51.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.463+0000 7f3277ed4b80 1 -- 172.21.15.2:0/3424264480 wait complete. 2024-05-14T17:01:52.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.463+0000 7f3277ed4b80 20 client.16113 trim_cache size 0 max 16384 2024-05-14T17:01:52.010 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T17:01:52.025 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 40, 2024-05-14T17:01:52.034 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24479, 2024-05-14T17:01:52.043 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 24, 2024-05-14T17:01:52.051 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-05-14T17:01:52.066 INFO:teuthology.orchestra.run.smithi002.stdout: "3b74f8b4": { 2024-05-14T17:01:52.092 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 24, 2024-05-14T17:01:52.128 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-05-14T17:01:52.179 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-05-14T17:01:52.186 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-05-14T17:01:52.192 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-05-14T17:01:52.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:01:52.208 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-05-14T17:01:52.215 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 136.4, 2024-05-14T17:01:52.221 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-05-14T17:01:52.277 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-05-14T17:01:52.284 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-05-14T17:01:52.291 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-05-14T17:01:52.309 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-05-14T17:01:52.317 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-05-14T17:01:52.323 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:01:52.331 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:01:52.341 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:01:52.359 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:01:52.412 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:01:52.420 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T17:01:52.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.463+0000 7f326d63e700 1 -- 172.21.15.2:0/1948670033 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f32500b29f0 msgr2=0x7f32500b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:52.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.463+0000 7f326d63e700 1 --2- 172.21.15.2:0/1948670033 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f32500b29f0 0x7f32500b4eb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3258009e60 tx=0x7f325800b7b0 comp rx=0 tx=0).stop 2024-05-14T17:01:52.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.463+0000 7f326d63e700 1 -- 172.21.15.2:0/1948670033 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f326806c690 msgr2=0x7f3268102a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:52.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.463+0000 7f326d63e700 1 --2- 172.21.15.2:0/1948670033 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f326806c690 0x7f3268102a80 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f325c00d980 tx=0x7f325c00dc90 comp rx=0 tx=0).stop 2024-05-14T17:01:52.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.464+0000 7f326d63e700 1 -- 172.21.15.2:0/1948670033 shutdown_connections 2024-05-14T17:01:52.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.464+0000 7f326d63e700 1 --2- 172.21.15.2:0/1948670033 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f32500b29f0 0x7f32500b4eb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:52.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.464+0000 7f326d63e700 1 --2- 172.21.15.2:0/1948670033 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32681095e0 0x7f3268102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:52.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.464+0000 7f326d63e700 1 --2- 172.21.15.2:0/1948670033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f326806bcd0 0x7f3268102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:52.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.464+0000 7f326d63e700 1 --2- 172.21.15.2:0/1948670033 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f326806c690 0x7f3268102a80 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:52.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.464+0000 7f326d63e700 1 -- 172.21.15.2:0/1948670033 >> 172.21.15.2:0/1948670033 conn(0x7f32680793e0 msgr2=0x7f32681010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:52.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.464+0000 7f326d63e700 1 -- 172.21.15.2:0/1948670033 shutdown_connections 2024-05-14T17:01:52.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:01:48.464+0000 7f326d63e700 1 -- 172.21.15.2:0/1948670033 wait complete. 2024-05-14T17:01:52.536 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:01:48 smithi002 ceph-mon[31538]: pgmap v218: 97 pgs: 97 active+clean; 4.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 28 MiB/s wr, 807 op/s 2024-05-14T17:01:52.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:01:50 smithi002 ceph-mon[31538]: pgmap v219: 97 pgs: 97 active+clean; 4.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 28 MiB/s wr, 814 op/s 2024-05-14T17:01:52.567 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 395.0 s, 2015.9 tps, lat 199.292 ms stddev 1664.910 2024-05-14T17:01:52.613 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '3b74f8b4', total seconds elapsed: 27.2 2024-05-14T17:01:52.626 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:01:50 smithi151 ceph-mon[36066]: pgmap v219: 97 pgs: 97 active+clean; 4.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 28 MiB/s wr, 814 op/s 2024-05-14T17:01:52.633 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:01:50 smithi123 ceph-mon[35971]: pgmap v219: 97 pgs: 97 active+clean; 4.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 28 MiB/s wr, 814 op/s 2024-05-14T17:01:52.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:01:52 smithi123 ceph-mon[35971]: pgmap v220: 97 pgs: 97 active+clean; 4.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 28 MiB/s wr, 806 op/s 2024-05-14T17:01:52.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:01:52 smithi002 ceph-mon[31538]: pgmap v220: 97 pgs: 97 active+clean; 4.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 28 MiB/s wr, 806 op/s 2024-05-14T17:01:52.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:01:52 smithi151 ceph-mon[36066]: pgmap v220: 97 pgs: 97 active+clean; 4.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 28 MiB/s wr, 806 op/s 2024-05-14T17:01:54.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:01:54 smithi002 ceph-mon[31538]: pgmap v221: 97 pgs: 97 active+clean; 4.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 30 MiB/s wr, 1.05k op/s 2024-05-14T17:01:54.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:01:54 smithi151 ceph-mon[36066]: pgmap v221: 97 pgs: 97 active+clean; 4.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 30 MiB/s wr, 1.05k op/s 2024-05-14T17:01:54.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:01:54 smithi123 ceph-mon[35971]: pgmap v221: 97 pgs: 97 active+clean; 4.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 30 MiB/s wr, 1.05k op/s 2024-05-14T17:01:56.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:01:56 smithi002 ceph-mon[31538]: pgmap v222: 97 pgs: 97 active+clean; 4.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 2.5 MiB/s wr, 253 op/s 2024-05-14T17:01:56.768 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:01:56 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:01:56.778 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 400.0 s, 3126.2 tps, lat 31.776 ms stddev 35.250 2024-05-14T17:01:56.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:01:56 smithi151 ceph-mon[36066]: pgmap v222: 97 pgs: 97 active+clean; 4.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 2.5 MiB/s wr, 253 op/s 2024-05-14T17:01:56.860 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:01:56 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:01:56.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:01:56 smithi123 ceph-mon[35971]: pgmap v222: 97 pgs: 97 active+clean; 4.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 2.5 MiB/s wr, 253 op/s 2024-05-14T17:01:57.065 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:01:56 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:01:57.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:01:57 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:01:57.749 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:01:57 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:01:57.759 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:01:57 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:01:57.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:01:57 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:01:57.860 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:01:57 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:01:57.867 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:01:57 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:01:57.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:01:57 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:01:57.915 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:01:57 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:01:57.923 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:01:57 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:01:58.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:01:58 smithi002 ceph-mon[31538]: pgmap v223: 97 pgs: 97 active+clean; 4.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 2.5 MiB/s wr, 253 op/s 2024-05-14T17:01:58.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:01:58 smithi151 ceph-mon[36066]: pgmap v223: 97 pgs: 97 active+clean; 4.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 2.5 MiB/s wr, 253 op/s 2024-05-14T17:01:58.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:01:58 smithi123 ceph-mon[35971]: pgmap v223: 97 pgs: 97 active+clean; 4.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 2.5 MiB/s wr, 253 op/s 2024-05-14T17:01:59.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.453+0000 7fc77ea23640 1 -- 172.21.15.2:0/1479375773 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc778076f80 msgr2=0x7fc778077360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.453+0000 7fc77ea23640 1 --2- 172.21.15.2:0/1479375773 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc778076f80 0x7fc778077360 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc76c009900 tx=0x7fc76c02a400 comp rx=0 tx=0).stop 2024-05-14T17:01:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.455+0000 7fc77ea23640 1 -- 172.21.15.2:0/1479375773 shutdown_connections 2024-05-14T17:01:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.455+0000 7fc77ea23640 1 --2- 172.21.15.2:0/1479375773 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7781138f0 0x7fc778115ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.455+0000 7fc77ea23640 1 --2- 172.21.15.2:0/1479375773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7780778a0 0x7fc7781133b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.455+0000 7fc77ea23640 1 --2- 172.21.15.2:0/1479375773 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc778076f80 0x7fc778077360 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.455+0000 7fc77ea23640 1 -- 172.21.15.2:0/1479375773 >> 172.21.15.2:0/1479375773 conn(0x7fc778100780 msgr2=0x7fc778102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:01:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.456+0000 7fc77ea23640 1 -- 172.21.15.2:0/1479375773 shutdown_connections 2024-05-14T17:01:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.456+0000 7fc77ea23640 1 -- 172.21.15.2:0/1479375773 wait complete. 2024-05-14T17:01:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.457+0000 7fc77ea23640 1 Processor -- start 2024-05-14T17:01:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.457+0000 7fc77ea23640 1 -- start start 2024-05-14T17:01:59.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.457+0000 7fc77ea23640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc778076f80 0x7fc778074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.457+0000 7fc77ea23640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc7780778a0 0x7fc77806d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.457+0000 7fc77ea23640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7781138f0 0x7fc77806d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.457+0000 7fc77ea23640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc7781193c0 con 0x7fc7780778a0 2024-05-14T17:01:59.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.457+0000 7fc77ea23640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc778119240 con 0x7fc778076f80 2024-05-14T17:01:59.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.457+0000 7fc77ea23640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc778119540 con 0x7fc7781138f0 2024-05-14T17:01:59.691 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.458+0000 7fc77cfc0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7781138f0 0x7fc77806d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:59.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.458+0000 7fc77cfc0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7781138f0 0x7fc77806d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:53292/0 (socket says 172.21.15.2:53292) 2024-05-14T17:01:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.458+0000 7fc777fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc778076f80 0x7fc778074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.458+0000 7fc77cfc0640 1 -- 172.21.15.2:0/726655580 learned_addr learned my addr 172.21.15.2:0/726655580 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:01:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.458+0000 7fc777fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc778076f80 0x7fc778074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:34082/0 (socket says 172.21.15.2:34082) 2024-05-14T17:01:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.459+0000 7fc7777fe640 1 --2- 172.21.15.2:0/726655580 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc7780778a0 0x7fc77806d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:59.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.459+0000 7fc77cfc0640 1 -- 172.21.15.2:0/726655580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc778076f80 msgr2=0x7fc778074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:59.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.459+0000 7fc77cfc0640 1 --2- 172.21.15.2:0/726655580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc778076f80 0x7fc778074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:59.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.459+0000 7fc77cfc0640 1 -- 172.21.15.2:0/726655580 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc7780778a0 msgr2=0x7fc77806d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:01:59.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.459+0000 7fc77cfc0640 1 --2- 172.21.15.2:0/726655580 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc7780778a0 0x7fc77806d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:01:59.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.459+0000 7fc77cfc0640 1 -- 172.21.15.2:0/726655580 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc77806de40 con 0x7fc7781138f0 2024-05-14T17:01:59.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.459+0000 7fc7777fe640 1 --2- 172.21.15.2:0/726655580 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc7780778a0 0x7fc77806d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:01:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.459+0000 7fc77cfc0640 1 --2- 172.21.15.2:0/726655580 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7781138f0 0x7fc77806d880 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc77000cab0 tx=0x7fc77000cf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:01:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.459+0000 7fc7757fa640 1 -- 172.21.15.2:0/726655580 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77000ae20 con 0x7fc7781138f0 2024-05-14T17:01:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.460+0000 7fc7757fa640 1 -- 172.21.15.2:0/726655580 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc77000f900 con 0x7fc7781138f0 2024-05-14T17:01:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.460+0000 7fc77ea23640 1 -- 172.21.15.2:0/726655580 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc77806e130 con 0x7fc7781138f0 2024-05-14T17:01:59.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.460+0000 7fc77ea23640 1 -- 172.21.15.2:0/726655580 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc778072070 con 0x7fc7781138f0 2024-05-14T17:01:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.460+0000 7fc7757fa640 1 -- 172.21.15.2:0/726655580 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc770018a50 con 0x7fc7781138f0 2024-05-14T17:01:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.463+0000 7fc78181d740 1 Processor -- start 2024-05-14T17:01:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.464+0000 7fc7757fa640 1 -- 172.21.15.2:0/726655580 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fc7700213f0 con 0x7fc7781138f0 2024-05-14T17:01:59.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.464+0000 7fc78181d740 1 -- start start 2024-05-14T17:01:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.464+0000 7fc78181d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56515ba1e8f0 0x56515ba1ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:59.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.464+0000 7fc78181d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56515ba1f210 0x56515ba29590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.465+0000 7fc78181d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56515ba29ad0 0x56515ba2bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.465+0000 7fc7777fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56515ba1e8f0 0x56515ba1ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:01:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.465+0000 7fc7777fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56515ba1e8f0 0x56515ba1ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:34084/0 (socket says 172.21.15.2:34084) 2024-05-14T17:01:59.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.465+0000 7fc7777fe640 1 -- 172.21.15.2:0/3018452495 learned_addr learned my addr 172.21.15.2:0/3018452495 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:01:59.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.465+0000 7fc78181d740 1 -- 172.21.15.2:0/3018452495 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56515b991c30 con 0x56515ba1f210 2024-05-14T17:01:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.465+0000 7fc78181d740 1 -- 172.21.15.2:0/3018452495 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56515b9817d0 con 0x56515ba1e8f0 2024-05-14T17:01:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.465+0000 7fc7757fa640 1 --2- 172.21.15.2:0/726655580 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc7600b2bd0 0x7fc7600b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:01:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.465+0000 7fc78181d740 1 -- 172.21.15.2:0/3018452495 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56515b985810 con 0x56515ba29ad0 2024-05-14T17:01:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.465+0000 7fc7757fa640 1 -- 172.21.15.2:0/726655580 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7fc7700e2c60 con 0x7fc7781138f0 2024-05-14T17:02:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.465+0000 7fc777fff640 1 --2- 172.21.15.2:0/3018452495 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56515ba1f210 0x56515ba29590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:00.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.466+0000 7fc77cfc0640 1 --2- 172.21.15.2:0/726655580 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc7600b2bd0 0x7fc7600b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:00.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.466+0000 7fc7777fe640 1 --2- 172.21.15.2:0/3018452495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56515ba29ad0 0x56515ba2bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:00.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.466+0000 7fc7777fe640 1 -- 172.21.15.2:0/3018452495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56515ba29ad0 msgr2=0x56515ba2bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.466+0000 7fc7777fe640 1 --2- 172.21.15.2:0/3018452495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56515ba29ad0 0x56515ba2bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:00.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.466+0000 7fc7777fe640 1 -- 172.21.15.2:0/3018452495 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56515ba1f210 msgr2=0x56515ba29590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:00.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.466+0000 7fc7777fe640 1 --2- 172.21.15.2:0/3018452495 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56515ba1f210 0x56515ba29590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:00.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.466+0000 7fc7777fe640 1 -- 172.21.15.2:0/3018452495 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56515b696210 con 0x56515ba1e8f0 2024-05-14T17:02:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.467+0000 7fc7777fe640 1 --2- 172.21.15.2:0/3018452495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56515ba1e8f0 0x56515ba1ecd0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc76800ac20 tx=0x7fc76800a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=533c57f170735c49 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.467+0000 7fc77cfc0640 1 --2- 172.21.15.2:0/726655580 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc7600b2bd0 0x7fc7600b5090 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc7700a9e70 tx=0x7fc7700071c0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.467+0000 7fc75e7fc640 1 -- 172.21.15.2:0/3018452495 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76800cd60 con 0x56515ba1e8f0 2024-05-14T17:02:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.467+0000 7fc75e7fc640 1 -- 172.21.15.2:0/3018452495 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc768019070 con 0x56515ba1e8f0 2024-05-14T17:02:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.467+0000 7fc75e7fc640 1 -- 172.21.15.2:0/3018452495 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7680135e0 con 0x56515ba1e8f0 2024-05-14T17:02:00.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.468+0000 7fc78181d740 1 -- 172.21.15.2:0/3018452495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56515ba1e8f0 msgr2=0x56515ba1ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.468+0000 7fc78181d740 1 --2- 172.21.15.2:0/3018452495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56515ba1e8f0 0x56515ba1ecd0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc76800ac20 tx=0x7fc76800a660 comp rx=0 tx=0).stop 2024-05-14T17:02:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.468+0000 7fc78181d740 1 -- 172.21.15.2:0/3018452495 shutdown_connections 2024-05-14T17:02:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.469+0000 7fc78181d740 1 --2- 172.21.15.2:0/3018452495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56515ba29ad0 0x56515ba2bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.469+0000 7fc78181d740 1 --2- 172.21.15.2:0/3018452495 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56515ba1f210 0x56515ba29590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:00.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.469+0000 7fc78181d740 1 --2- 172.21.15.2:0/3018452495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56515ba1e8f0 0x56515ba1ecd0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.469+0000 7fc78181d740 1 -- 172.21.15.2:0/3018452495 >> 172.21.15.2:0/3018452495 conn(0x56515ba1ae00 msgr2=0x56515ba1b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.469+0000 7fc78181d740 1 -- 172.21.15.2:0/3018452495 shutdown_connections 2024-05-14T17:02:00.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.469+0000 7fc78181d740 1 -- 172.21.15.2:0/3018452495 wait complete. 2024-05-14T17:02:00.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.470+0000 7fc78181d740 1 -- start start 2024-05-14T17:02:00.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.472+0000 7fc78181d740 1 Processor -- start 2024-05-14T17:02:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.473+0000 7fc78181d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56515ba1e8f0 0x56515ba29460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.473+0000 7fc78181d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56515ba1f210 0x56515ba26750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:00.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.473+0000 7fc78181d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56515ba29ad0 0x56515ba26cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:00.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.473+0000 7fc7777fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56515ba1e8f0 0x56515ba29460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:00.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.473+0000 7fc7777fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56515ba1e8f0 0x56515ba29460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:53302/0 (socket says 172.21.15.2:53302) 2024-05-14T17:02:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.473+0000 7fc777fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56515ba1f210 0x56515ba26750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.473+0000 7fc7777fe640 1 -- 172.21.15.2:0/2492326392 learned_addr learned my addr 172.21.15.2:0/2492326392 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:02:00.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.473+0000 7fc78181d740 1 -- 172.21.15.2:0/2492326392 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56515b667790 con 0x56515ba29ad0 2024-05-14T17:02:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.473+0000 7fc78181d740 1 -- 172.21.15.2:0/2492326392 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56515b93d890 con 0x56515ba1f210 2024-05-14T17:02:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.473+0000 7fc78181d740 1 -- 172.21.15.2:0/2492326392 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56515ba2c7a0 con 0x56515ba1e8f0 2024-05-14T17:02:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.473+0000 7fc7777fe640 1 --2- 172.21.15.2:0/2492326392 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56515ba29ad0 0x56515ba26cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.473+0000 7fc78181d740 10 client.0 _ll_register_callbacks cb 0x56515b940d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:02:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.474+0000 7fc777fff640 1 -- 172.21.15.2:0/2492326392 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56515ba1e8f0 msgr2=0x56515ba29460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:00.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.474+0000 7fc777fff640 1 --2- 172.21.15.2:0/2492326392 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56515ba1e8f0 0x56515ba29460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.474+0000 7fc777fff640 1 -- 172.21.15.2:0/2492326392 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56515ba29ad0 msgr2=0x56515ba26cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.474+0000 7fc777fff640 1 --2- 172.21.15.2:0/2492326392 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56515ba29ad0 0x56515ba26cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.474+0000 7fc777fff640 1 -- 172.21.15.2:0/2492326392 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56515b762620 con 0x56515ba1f210 2024-05-14T17:02:00.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.474+0000 7fc78181d740 1 -- 172.21.15.2:0/2492326392 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc768019070 con 0x56515ba1f210 2024-05-14T17:02:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.474+0000 7fc777fff640 1 --2- 172.21.15.2:0/2492326392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56515ba1f210 0x56515ba26750 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc76c02a9f0 tx=0x7fc76c003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3627c2f40080005 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.474+0000 7fc7577fe640 10 client.25796 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T17:02:00.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.474+0000 7fc7577fe640 1 -- 172.21.15.2:0/2492326392 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76c035040 con 0x56515ba1f210 2024-05-14T17:02:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.474+0000 7fc7577fe640 1 -- 172.21.15.2:0/2492326392 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7fc76c003d20 con 0x56515ba1f210 2024-05-14T17:02:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.474+0000 7fc7577fe640 1 -- 172.21.15.2:0/2492326392 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc76c005020 con 0x56515ba1f210 2024-05-14T17:02:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.474+0000 7fc78181d740 10 client.25796 fetch_fsmap learned FSMap version 40 2024-05-14T17:02:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.474+0000 7fc78181d740 1 -- 172.21.15.2:0/2492326392 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=40}) -- 0x56515ba38130 con 0x56515ba1f210 2024-05-14T17:02:00.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.475+0000 7fc7577fe640 1 -- 172.21.15.2:0/2492326392 <== mon.2 v2:172.21.15.123:3300/0 4 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7fc76c0051c0 con 0x56515ba1f210 2024-05-14T17:02:00.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.475+0000 7fc78181d740 10 client.25796 fetch_fsmap finished waiting for FSMap version 40 2024-05-14T17:02:00.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.475+0000 7fc78181d740 10 client.25796 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:02:00.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.475+0000 7fc78181d740 20 client.25796 populate_metadata read hostname 'smithi002' 2024-05-14T17:02:00.661 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.475+0000 7fc78181d740 1 --2- 172.21.15.2:0/2492326392 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56515ba396d0 0x56515ba3bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:00.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.475+0000 7fc78181d740 4 client.25796 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:02:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.475+0000 7fc78181d740 1 -- 172.21.15.2:0/2492326392 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56515b993b80 con 0x56515ba396d0 2024-05-14T17:02:00.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.475+0000 7fc7777fe640 1 --2- 172.21.15.2:0/2492326392 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56515ba396d0 0x56515ba3bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:00.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.475+0000 7fc7777fe640 1 --2- 172.21.15.2:0/2492326392 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56515ba396d0 0x56515ba3bb30 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de9de796272a69d8 server_cookie=23b83b7c99b28f6d in_seq=0 out_seq=0 2024-05-14T17:02:00.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.475+0000 7fc7577fe640 10 client.25796 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:02:00.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.476+0000 7fc7577fe640 1 -- 172.21.15.2:0/2492326392 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56515b993b80 con 0x56515ba396d0 2024-05-14T17:02:00.727 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.476+0000 7fc7577fe640 10 client.25796 handle_command_reply: tid=0 2024-05-14T17:02:00.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.476+0000 7fc78181d740 1 client.25796 shutdown 2024-05-14T17:02:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.477+0000 7fc78181d740 1 -- 172.21.15.2:0/2492326392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56515ba1f210 msgr2=0x56515ba26750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.477+0000 7fc78181d740 1 --2- 172.21.15.2:0/2492326392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56515ba1f210 0x56515ba26750 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc76c02a9f0 tx=0x7fc76c003820 comp rx=0 tx=0).stop 2024-05-14T17:02:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.477+0000 7fc78181d740 1 -- 172.21.15.2:0/2492326392 shutdown_connections 2024-05-14T17:02:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.477+0000 7fc78181d740 1 --2- 172.21.15.2:0/2492326392 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56515ba396d0 0x56515ba3bb30 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.477+0000 7fc78181d740 1 --2- 172.21.15.2:0/2492326392 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56515ba29ad0 0x56515ba26cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.477+0000 7fc78181d740 1 --2- 172.21.15.2:0/2492326392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56515ba1f210 0x56515ba26750 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.477+0000 7fc78181d740 1 --2- 172.21.15.2:0/2492326392 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56515ba1e8f0 0x56515ba29460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.477+0000 7fc7577fe640 0 client.25796 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:02:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.477+0000 7fc78181d740 1 -- 172.21.15.2:0/2492326392 >> 172.21.15.2:0/2492326392 conn(0x56515ba1ae00 msgr2=0x56515ba1a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.477+0000 7fc78181d740 1 -- 172.21.15.2:0/2492326392 shutdown_connections 2024-05-14T17:02:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.477+0000 7fc78181d740 1 -- 172.21.15.2:0/2492326392 wait complete. 2024-05-14T17:02:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.477+0000 7fc78181d740 20 client.25796 trim_cache size 0 max 16384 2024-05-14T17:02:00.920 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:02:00 smithi123 ceph-mon[35971]: pgmap v224: 97 pgs: 97 active+clean; 4.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.5 MiB/s wr, 549 op/s 2024-05-14T17:02:00.927 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:02:00 smithi151 ceph-mon[36066]: pgmap v224: 97 pgs: 97 active+clean; 4.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.5 MiB/s wr, 549 op/s 2024-05-14T17:02:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.499+0000 7fc78181d740 1 Processor -- start 2024-05-14T17:02:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.499+0000 7fc78181d740 1 -- start start 2024-05-14T17:02:00.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.500+0000 7fc78181d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56515ba1e8f0 0x56515ba356a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.500+0000 7fc78181d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56515ba1f210 0x56515ba41400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.500+0000 7fc7777fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56515ba1e8f0 0x56515ba356a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.500+0000 7fc7777fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56515ba1e8f0 0x56515ba356a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:34094/0 (socket says 172.21.15.2:34094) 2024-05-14T17:02:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.500+0000 7fc7777fe640 1 -- 172.21.15.2:0/1999595133 learned_addr learned my addr 172.21.15.2:0/1999595133 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:02:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.500+0000 7fc78181d740 1 --2- 172.21.15.2:0/1999595133 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56515ba29ad0 0x56515ba41940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.500+0000 7fc78181d740 1 -- 172.21.15.2:0/1999595133 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc764010fc0 con 0x56515ba29ad0 2024-05-14T17:02:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.500+0000 7fc78181d740 1 -- 172.21.15.2:0/1999595133 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc76400e860 con 0x56515ba1e8f0 2024-05-14T17:02:01.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.500+0000 7fc78181d740 1 -- 172.21.15.2:0/1999595133 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc76400c6d0 con 0x56515ba1f210 2024-05-14T17:02:01.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.501+0000 7fc777fff640 1 --2- 172.21.15.2:0/1999595133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56515ba1f210 0x56515ba41400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:01.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.501+0000 7fc7777fe640 1 --2- 172.21.15.2:0/1999595133 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56515ba29ad0 0x56515ba41940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.501+0000 7fc7777fe640 1 -- 172.21.15.2:0/1999595133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56515ba1f210 msgr2=0x56515ba41400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:01.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.501+0000 7fc7777fe640 1 --2- 172.21.15.2:0/1999595133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56515ba1f210 0x56515ba41400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.501+0000 7fc7777fe640 1 -- 172.21.15.2:0/1999595133 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56515ba29ad0 msgr2=0x56515ba41940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:01.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.501+0000 7fc7777fe640 1 --2- 172.21.15.2:0/1999595133 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56515ba29ad0 0x56515ba41940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:01.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.501+0000 7fc7777fe640 1 -- 172.21.15.2:0/1999595133 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56515ba16830 con 0x56515ba1e8f0 2024-05-14T17:02:01.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.501+0000 7fc7777fe640 1 --2- 172.21.15.2:0/1999595133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56515ba1e8f0 0x56515ba356a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc768014e10 tx=0x7fc768008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fad30db6812f1c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.501+0000 7fc75dffb640 1 -- 172.21.15.2:0/1999595133 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc768034070 con 0x56515ba1e8f0 2024-05-14T17:02:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.501+0000 7fc75dffb640 1 -- 172.21.15.2:0/1999595133 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc768014b30 con 0x56515ba1e8f0 2024-05-14T17:02:01.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.502+0000 7fc75dffb640 1 -- 172.21.15.2:0/1999595133 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76801f410 con 0x56515ba1e8f0 2024-05-14T17:02:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.503+0000 7fc78181d740 1 -- 172.21.15.2:0/1999595133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56515ba1e8f0 msgr2=0x56515ba356a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.503+0000 7fc78181d740 1 --2- 172.21.15.2:0/1999595133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56515ba1e8f0 0x56515ba356a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc768014e10 tx=0x7fc768008040 comp rx=0 tx=0).stop 2024-05-14T17:02:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.503+0000 7fc78181d740 1 -- 172.21.15.2:0/1999595133 shutdown_connections 2024-05-14T17:02:01.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.503+0000 7fc78181d740 1 --2- 172.21.15.2:0/1999595133 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56515ba29ad0 0x56515ba41940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.503+0000 7fc78181d740 1 --2- 172.21.15.2:0/1999595133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56515ba1f210 0x56515ba41400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.503+0000 7fc78181d740 1 --2- 172.21.15.2:0/1999595133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56515ba1e8f0 0x56515ba356a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:01.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.503+0000 7fc78181d740 1 -- 172.21.15.2:0/1999595133 >> 172.21.15.2:0/1999595133 conn(0x56515ba1ae00 msgr2=0x56515ba1a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:01.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.503+0000 7fc78181d740 1 -- 172.21.15.2:0/1999595133 shutdown_connections 2024-05-14T17:02:01.284 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.503+0000 7fc78181d740 1 -- 172.21.15.2:0/1999595133 wait complete. 2024-05-14T17:02:01.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.503+0000 7fc78181d740 1 -- start start 2024-05-14T17:02:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.503+0000 7fc78181d740 1 Processor -- start 2024-05-14T17:02:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.504+0000 7fc78181d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56515ba1e8f0 0x56515ba31ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:01.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.504+0000 7fc78181d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56515ba1f210 0x56515ba2f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:01.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.504+0000 7fc78181d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56515ba29ad0 0x56515ba2f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:01.330 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.504+0000 7fc7777fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56515ba1e8f0 0x56515ba31ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:01.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.504+0000 7fc7777fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56515ba1e8f0 0x56515ba31ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:34106/0 (socket says 172.21.15.2:34106) 2024-05-14T17:02:01.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.504+0000 7fc7777fe640 1 -- 172.21.15.2:0/194968926 learned_addr learned my addr 172.21.15.2:0/194968926 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:02:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.504+0000 7fc78181d740 1 -- 172.21.15.2:0/194968926 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc76400cd20 con 0x56515ba1f210 2024-05-14T17:02:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.504+0000 7fc78181d740 1 -- 172.21.15.2:0/194968926 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc76400e190 con 0x56515ba1e8f0 2024-05-14T17:02:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.504+0000 7fc78181d740 1 -- 172.21.15.2:0/194968926 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc76400d3f0 con 0x56515ba29ad0 2024-05-14T17:02:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.504+0000 7fc777fff640 1 --2- 172.21.15.2:0/194968926 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56515ba1f210 0x56515ba2f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.504+0000 7fc78181d740 10 client.0 _ll_register_callbacks cb 0x7fc76c0043c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:02:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.504+0000 7fc7777fe640 1 --2- 172.21.15.2:0/194968926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56515ba29ad0 0x56515ba2f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:01.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.504+0000 7fc7777fe640 1 -- 172.21.15.2:0/194968926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56515ba29ad0 msgr2=0x56515ba2f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.504+0000 7fc7777fe640 1 --2- 172.21.15.2:0/194968926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56515ba29ad0 0x56515ba2f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:01.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.504+0000 7fc7777fe640 1 -- 172.21.15.2:0/194968926 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56515ba1f210 msgr2=0x56515ba2f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.504+0000 7fc7777fe640 1 --2- 172.21.15.2:0/194968926 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56515ba1f210 0x56515ba2f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.504+0000 7fc7777fe640 1 -- 172.21.15.2:0/194968926 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56515ba30050 con 0x56515ba1e8f0 2024-05-14T17:02:01.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.504+0000 7fc777fff640 1 --2- 172.21.15.2:0/194968926 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56515ba1f210 0x56515ba2f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:02:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.505+0000 7fc7777fe640 1 --2- 172.21.15.2:0/194968926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56515ba1e8f0 0x56515ba31ee0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc7680046a0 tx=0x7fc76800a9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6a9d5ed95941de4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.505+0000 7fc7577fe640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T17:02:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.505+0000 7fc7577fe640 1 -- 172.21.15.2:0/194968926 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc768034040 con 0x56515ba1e8f0 2024-05-14T17:02:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.505+0000 7fc7577fe640 1 -- 172.21.15.2:0/194968926 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8513+0+0 (secure 0 0 0) 0x7fc76800c9b0 con 0x56515ba1e8f0 2024-05-14T17:02:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.505+0000 7fc78181d740 1 -- 172.21.15.2:0/194968926 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc768014b30 con 0x56515ba1e8f0 2024-05-14T17:02:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.505+0000 7fc7577fe640 1 -- 172.21.15.2:0/194968926 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc76802c380 con 0x56515ba1e8f0 2024-05-14T17:02:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.505+0000 7fc78181d740 10 client.25802 fetch_fsmap learned FSMap version 40 2024-05-14T17:02:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.505+0000 7fc78181d740 1 -- 172.21.15.2:0/194968926 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56515baa3f20 con 0x56515ba1e8f0 2024-05-14T17:02:01.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.506+0000 7fc7577fe640 1 -- 172.21.15.2:0/194968926 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc768014b30 con 0x56515ba1e8f0 2024-05-14T17:02:01.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.507+0000 7fc7577fe640 1 -- 172.21.15.2:0/194968926 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7fc76803a070 con 0x56515ba1e8f0 2024-05-14T17:02:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.507+0000 7fc78181d740 10 client.25802 fetch_fsmap finished waiting for FSMap version 40 2024-05-14T17:02:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.507+0000 7fc78181d740 10 client.25802 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:02:01.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.507+0000 7fc78181d740 20 client.25802 populate_metadata read hostname 'smithi002' 2024-05-14T17:02:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.507+0000 7fc78181d740 1 --2- 172.21.15.2:0/194968926 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56515baa4200 0x56515baa65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.507+0000 7fc78181d740 4 client.25802 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:02:01.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.507+0000 7fc78181d740 1 -- 172.21.15.2:0/194968926 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56515b98d360 con 0x56515baa4200 2024-05-14T17:02:01.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.507+0000 7fc777fff640 1 --2- 172.21.15.2:0/194968926 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56515baa4200 0x56515baa65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:01.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.509+0000 7fc777fff640 1 --2- 172.21.15.2:0/194968926 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56515baa4200 0x56515baa65f0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b1de2154155cf30 server_cookie=62222b8f5a2c713d in_seq=0 out_seq=0 2024-05-14T17:02:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.509+0000 7fc7577fe640 10 client.25802 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:02:01.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.520+0000 7fc7577fe640 1 -- 172.21.15.2:0/194968926 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56515b98d360 con 0x56515baa4200 2024-05-14T17:02:01.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.520+0000 7fc7577fe640 10 client.25802 handle_command_reply: tid=0 2024-05-14T17:02:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.520+0000 7fc78181d740 1 client.25802 shutdown 2024-05-14T17:02:01.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.521+0000 7fc78181d740 1 -- 172.21.15.2:0/194968926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56515ba1e8f0 msgr2=0x56515ba31ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.521+0000 7fc78181d740 1 --2- 172.21.15.2:0/194968926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56515ba1e8f0 0x56515ba31ee0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc7680046a0 tx=0x7fc76800a9b0 comp rx=0 tx=0).stop 2024-05-14T17:02:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.521+0000 7fc78181d740 1 -- 172.21.15.2:0/194968926 shutdown_connections 2024-05-14T17:02:01.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.521+0000 7fc78181d740 1 --2- 172.21.15.2:0/194968926 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56515baa4200 0x56515baa65f0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.521+0000 7fc78181d740 1 --2- 172.21.15.2:0/194968926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56515ba29ad0 0x56515ba2f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.521+0000 7fc78181d740 1 --2- 172.21.15.2:0/194968926 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56515ba1f210 0x56515ba2f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:01.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.521+0000 7fc78181d740 1 --2- 172.21.15.2:0/194968926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56515ba1e8f0 0x56515ba31ee0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.521+0000 7fc7577fe640 0 client.25802 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:02:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.521+0000 7fc78181d740 1 -- 172.21.15.2:0/194968926 >> 172.21.15.2:0/194968926 conn(0x56515ba1ae00 msgr2=0x56515ba34690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.521+0000 7fc78181d740 1 -- 172.21.15.2:0/194968926 shutdown_connections 2024-05-14T17:02:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.521+0000 7fc78181d740 1 -- 172.21.15.2:0/194968926 wait complete. 2024-05-14T17:02:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.522+0000 7fc78181d740 20 client.25802 trim_cache size 0 max 16384 2024-05-14T17:02:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:02:01.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:02:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:02:01.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.522+0000 7fc77ea23640 1 -- 172.21.15.2:0/726655580 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc7600b2bd0 msgr2=0x7fc7600b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.522+0000 7fc77ea23640 1 --2- 172.21.15.2:0/726655580 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc7600b2bd0 0x7fc7600b5090 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc7700a9e70 tx=0x7fc7700071c0 comp rx=0 tx=0).stop 2024-05-14T17:02:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.522+0000 7fc77ea23640 1 -- 172.21.15.2:0/726655580 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7781138f0 msgr2=0x7fc77806d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:01.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.522+0000 7fc77ea23640 1 --2- 172.21.15.2:0/726655580 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7781138f0 0x7fc77806d880 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc77000cab0 tx=0x7fc77000cf70 comp rx=0 tx=0).stop 2024-05-14T17:02:01.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.522+0000 7fc77ea23640 1 -- 172.21.15.2:0/726655580 shutdown_connections 2024-05-14T17:02:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.522+0000 7fc77ea23640 1 --2- 172.21.15.2:0/726655580 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc7600b2bd0 0x7fc7600b5090 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.522+0000 7fc77ea23640 1 --2- 172.21.15.2:0/726655580 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7781138f0 0x7fc77806d880 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:02.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.522+0000 7fc77ea23640 1 --2- 172.21.15.2:0/726655580 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc7780778a0 0x7fc77806d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.522+0000 7fc77ea23640 1 --2- 172.21.15.2:0/726655580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc778076f80 0x7fc778074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:02.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.522+0000 7fc77ea23640 1 -- 172.21.15.2:0/726655580 >> 172.21.15.2:0/726655580 conn(0x7fc778100780 msgr2=0x7fc778102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:02.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.522+0000 7fc77ea23640 1 -- 172.21.15.2:0/726655580 shutdown_connections 2024-05-14T17:02:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:01:59.522+0000 7fc77ea23640 1 -- 172.21.15.2:0/726655580 wait complete. 2024-05-14T17:02:02.100 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:02:00 smithi002 ceph-mon[31538]: pgmap v224: 97 pgs: 97 active+clean; 4.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.5 MiB/s wr, 549 op/s 2024-05-14T17:02:02.106 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 405.0 s, 1885.4 tps, lat 52.468 ms stddev 98.156 2024-05-14T17:02:02.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:02:02 smithi002 ceph-mon[31538]: pgmap v225: 97 pgs: 97 active+clean; 4.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.4 MiB/s wr, 542 op/s 2024-05-14T17:02:02.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:02:02 smithi151 ceph-mon[36066]: pgmap v225: 97 pgs: 97 active+clean; 4.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.4 MiB/s wr, 542 op/s 2024-05-14T17:02:02.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:02:02 smithi123 ceph-mon[35971]: pgmap v225: 97 pgs: 97 active+clean; 4.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.4 MiB/s wr, 542 op/s 2024-05-14T17:02:04.746 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:02:04 smithi002 ceph-mon[31538]: pgmap v226: 97 pgs: 97 active+clean; 4.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 22 MiB/s wr, 1.93k op/s 2024-05-14T17:02:04.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:02:04 smithi151 ceph-mon[36066]: pgmap v226: 97 pgs: 97 active+clean; 4.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 22 MiB/s wr, 1.93k op/s 2024-05-14T17:02:04.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:02:04 smithi123 ceph-mon[35971]: pgmap v226: 97 pgs: 97 active+clean; 4.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 22 MiB/s wr, 1.93k op/s 2024-05-14T17:02:06.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:02:06 smithi002 ceph-mon[31538]: pgmap v227: 97 pgs: 97 active+clean; 4.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 19 MiB/s wr, 1.68k op/s 2024-05-14T17:02:06.758 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 410.0 s, 2844.8 tps, lat 35.633 ms stddev 48.497 2024-05-14T17:02:06.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:02:06 smithi151 ceph-mon[36066]: pgmap v227: 97 pgs: 97 active+clean; 4.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 19 MiB/s wr, 1.68k op/s 2024-05-14T17:02:06.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:02:06 smithi123 ceph-mon[35971]: pgmap v227: 97 pgs: 97 active+clean; 4.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 19 MiB/s wr, 1.68k op/s 2024-05-14T17:02:08.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:02:08 smithi002 ceph-mon[31538]: pgmap v228: 97 pgs: 97 active+clean; 4.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 19 MiB/s wr, 1.68k op/s 2024-05-14T17:02:08.970 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:02:08 smithi123 ceph-mon[35971]: pgmap v228: 97 pgs: 97 active+clean; 4.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 19 MiB/s wr, 1.68k op/s 2024-05-14T17:02:08.978 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:02:08 smithi151 ceph-mon[36066]: pgmap v228: 97 pgs: 97 active+clean; 4.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 19 MiB/s wr, 1.68k op/s 2024-05-14T17:02:10.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:02:10 smithi002 ceph-mon[31538]: pgmap v229: 97 pgs: 97 active+clean; 4.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 22 MiB/s wr, 2.00k op/s 2024-05-14T17:02:10.751 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:02:10 smithi002 ceph-mon[31538]: osdmap e84: 12 total, 12 up, 12 in 2024-05-14T17:02:10.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:02:10 smithi151 ceph-mon[36066]: pgmap v229: 97 pgs: 97 active+clean; 4.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 22 MiB/s wr, 2.00k op/s 2024-05-14T17:02:10.860 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:02:10 smithi151 ceph-mon[36066]: osdmap e84: 12 total, 12 up, 12 in 2024-05-14T17:02:10.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:02:10 smithi123 ceph-mon[35971]: pgmap v229: 97 pgs: 97 active+clean; 4.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 22 MiB/s wr, 2.00k op/s 2024-05-14T17:02:10.916 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:02:10 smithi123 ceph-mon[35971]: osdmap e84: 12 total, 12 up, 12 in 2024-05-14T17:02:11.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 415.0 s, 1877.0 tps, lat 53.061 ms stddev 62.263 2024-05-14T17:02:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.644+0000 7fe0b2ca0640 1 -- 172.21.15.2:0/4199335402 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0ac10a370 msgr2=0x7fe0ac115660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.644+0000 7fe0b2ca0640 1 --2- 172.21.15.2:0/4199335402 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0ac10a370 0x7fe0ac115660 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe098009900 tx=0x7fe09802a480 comp rx=0 tx=0).stop 2024-05-14T17:02:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.644+0000 7fe0b2ca0640 1 -- 172.21.15.2:0/4199335402 shutdown_connections 2024-05-14T17:02:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.644+0000 7fe0b2ca0640 1 --2- 172.21.15.2:0/4199335402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0ac115ba0 0x7fe0ac117f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:12.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.645+0000 7fe0b2ca0640 1 --2- 172.21.15.2:0/4199335402 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0ac10a370 0x7fe0ac115660 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.645+0000 7fe0b2ca0640 1 --2- 172.21.15.2:0/4199335402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0ac109a50 0x7fe0ac109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:12.696 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.645+0000 7fe0b2ca0640 1 -- 172.21.15.2:0/4199335402 >> 172.21.15.2:0/4199335402 conn(0x7fe0ac06db20 msgr2=0x7fe0ac06df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.645+0000 7fe0b2ca0640 1 -- 172.21.15.2:0/4199335402 shutdown_connections 2024-05-14T17:02:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.645+0000 7fe0b2ca0640 1 -- 172.21.15.2:0/4199335402 wait complete. 2024-05-14T17:02:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.645+0000 7fe0b2ca0640 1 Processor -- start 2024-05-14T17:02:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.645+0000 7fe0b2ca0640 1 -- start start 2024-05-14T17:02:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.645+0000 7fe0b2ca0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0ac109a50 0x7fe0ac1a76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:12.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.645+0000 7fe0b2ca0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0ac10a370 0x7fe0ac1a7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.645+0000 7fe0b2ca0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0ac115ba0 0x7fe0ac1a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.645+0000 7fe0b2ca0640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ac11b4a0 con 0x7fe0ac115ba0 2024-05-14T17:02:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.645+0000 7fe0b2ca0640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ac11b320 con 0x7fe0ac10a370 2024-05-14T17:02:12.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.645+0000 7fe0b2ca0640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ac11b620 con 0x7fe0ac109a50 2024-05-14T17:02:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.646+0000 7fe0abfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0ac10a370 0x7fe0ac1a7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.646+0000 7fe0b123d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0ac115ba0 0x7fe0ac1a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.646+0000 7fe0abfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0ac10a370 0x7fe0ac1a7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:53496/0 (socket says 172.21.15.2:53496) 2024-05-14T17:02:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.646+0000 7fe0abfff640 1 -- 172.21.15.2:0/212900569 learned_addr learned my addr 172.21.15.2:0/212900569 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:02:12.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.646+0000 7fe0b0a3c640 1 --2- 172.21.15.2:0/212900569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0ac109a50 0x7fe0ac1a76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:12.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.646+0000 7fe0abfff640 1 -- 172.21.15.2:0/212900569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0ac109a50 msgr2=0x7fe0ac1a76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.646+0000 7fe0abfff640 1 --2- 172.21.15.2:0/212900569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0ac109a50 0x7fe0ac1a76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:12.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.646+0000 7fe0abfff640 1 -- 172.21.15.2:0/212900569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0ac115ba0 msgr2=0x7fe0ac1a18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.646+0000 7fe0abfff640 1 --2- 172.21.15.2:0/212900569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0ac115ba0 0x7fe0ac1a18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.646+0000 7fe0abfff640 1 -- 172.21.15.2:0/212900569 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0ac1a2040 con 0x7fe0ac10a370 2024-05-14T17:02:12.988 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:02:12 smithi123 ceph-mon[35971]: pgmap v231: 97 pgs: 97 active+clean; 4.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 2.04k op/s 2024-05-14T17:02:13.019 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:02:12 smithi151 ceph-mon[36066]: pgmap v231: 97 pgs: 97 active+clean; 4.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 2.04k op/s 2024-05-14T17:02:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.647+0000 7fe0abfff640 1 --2- 172.21.15.2:0/212900569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0ac10a370 0x7fe0ac1a7c00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe098006010 tx=0x7fe0980040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:13.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.647+0000 7fe0a9ffb640 1 -- 172.21.15.2:0/212900569 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09803f030 con 0x7fe0ac10a370 2024-05-14T17:02:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.647+0000 7fe0a9ffb640 1 -- 172.21.15.2:0/212900569 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0980044c0 con 0x7fe0ac10a370 2024-05-14T17:02:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.647+0000 7fe0a9ffb640 1 -- 172.21.15.2:0/212900569 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe098035070 con 0x7fe0ac10a370 2024-05-14T17:02:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.647+0000 7fe0b2ca0640 1 -- 172.21.15.2:0/212900569 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0ac1a22d0 con 0x7fe0ac10a370 2024-05-14T17:02:13.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.648+0000 7fe0b2ca0640 1 -- 172.21.15.2:0/212900569 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0ac10b3f0 con 0x7fe0ac10a370 2024-05-14T17:02:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.650+0000 7fe0b59c4740 1 Processor -- start 2024-05-14T17:02:13.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.650+0000 7fe0a9ffb640 1 -- 172.21.15.2:0/212900569 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fe098042030 con 0x7fe0ac10a370 2024-05-14T17:02:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.650+0000 7fe0b59c4740 1 -- start start 2024-05-14T17:02:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.651+0000 7fe0b59c4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9827d78f0 0x55a9827d7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:13.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.651+0000 7fe0b59c4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9827d8210 0x55a9827e2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.651+0000 7fe0b59c4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a9827e2ad0 0x55a9827e4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:13.180 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.651+0000 7fe0b59c4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55a98274ac30 con 0x55a9827e2ad0 2024-05-14T17:02:13.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.651+0000 7fe0b59c4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a98273a7d0 con 0x55a9827d8210 2024-05-14T17:02:13.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.651+0000 7fe0b59c4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a98273e810 con 0x55a9827d78f0 2024-05-14T17:02:13.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.651+0000 7fe0b0a3c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9827d8210 0x55a9827e2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.651+0000 7fe0abfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a9827e2ad0 0x55a9827e4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:13.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.651+0000 7fe0b0a3c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9827d8210 0x55a9827e2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:53498/0 (socket says 172.21.15.2:53498) 2024-05-14T17:02:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.651+0000 7fe0b0a3c640 1 -- 172.21.15.2:0/1513275479 learned_addr learned my addr 172.21.15.2:0/1513275479 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:02:13.263 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.651+0000 7fe0b123d640 1 --2- 172.21.15.2:0/1513275479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9827d78f0 0x55a9827d7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:13.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.651+0000 7fe0b0a3c640 1 -- 172.21.15.2:0/1513275479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9827d78f0 msgr2=0x55a9827d7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:13.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.651+0000 7fe0b0a3c640 1 --2- 172.21.15.2:0/1513275479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9827d78f0 0x55a9827d7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.651+0000 7fe0b0a3c640 1 -- 172.21.15.2:0/1513275479 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a9827e2ad0 msgr2=0x55a9827e4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:13.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.651+0000 7fe0a9ffb640 1 --2- 172.21.15.2:0/212900569 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fe09c0b2a90 0x7fe09c0b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:13.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.652+0000 7fe0b0a3c640 1 --2- 172.21.15.2:0/1513275479 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a9827e2ad0 0x55a9827e4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:13.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.652+0000 7fe0b0a3c640 1 -- 172.21.15.2:0/1513275479 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a98244f210 con 0x55a9827d8210 2024-05-14T17:02:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.652+0000 7fe0abfff640 1 --2- 172.21.15.2:0/1513275479 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a9827e2ad0 0x55a9827e4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:02:13.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.652+0000 7fe0b123d640 1 --2- 172.21.15.2:0/212900569 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fe09c0b2a90 0x7fe09c0b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:13.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.652+0000 7fe0a9ffb640 1 -- 172.21.15.2:0/212900569 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8557+0+0 (secure 0 0 0) 0x7fe098102950 con 0x7fe0ac10a370 2024-05-14T17:02:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.653+0000 7fe0b123d640 1 --2- 172.21.15.2:0/212900569 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fe09c0b2a90 0x7fe09c0b4f50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4005fd0 tx=0x7fe0a4012040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:13.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.653+0000 7fe0b0a3c640 1 --2- 172.21.15.2:0/1513275479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9827d8210 0x55a9827e2590 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0009c90 tx=0x7fe0a0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4aad30ab35f8e5a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.653+0000 7fe096ffd640 1 -- 172.21.15.2:0/1513275479 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a0005220 con 0x55a9827d8210 2024-05-14T17:02:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.653+0000 7fe096ffd640 1 -- 172.21.15.2:0/1513275479 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0a0004c50 con 0x55a9827d8210 2024-05-14T17:02:13.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.653+0000 7fe096ffd640 1 -- 172.21.15.2:0/1513275479 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a00197a0 con 0x55a9827d8210 2024-05-14T17:02:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.654+0000 7fe0b59c4740 1 -- 172.21.15.2:0/1513275479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9827d8210 msgr2=0x55a9827e2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:13.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.654+0000 7fe0b59c4740 1 --2- 172.21.15.2:0/1513275479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9827d8210 0x55a9827e2590 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0009c90 tx=0x7fe0a0011810 comp rx=0 tx=0).stop 2024-05-14T17:02:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.654+0000 7fe0b59c4740 1 -- 172.21.15.2:0/1513275479 shutdown_connections 2024-05-14T17:02:13.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.654+0000 7fe0b59c4740 1 --2- 172.21.15.2:0/1513275479 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a9827e2ad0 0x55a9827e4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.654+0000 7fe0b59c4740 1 --2- 172.21.15.2:0/1513275479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9827d8210 0x55a9827e2590 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:13.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.654+0000 7fe0b59c4740 1 --2- 172.21.15.2:0/1513275479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9827d78f0 0x55a9827d7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.654+0000 7fe0b59c4740 1 -- 172.21.15.2:0/1513275479 >> 172.21.15.2:0/1513275479 conn(0x55a9827d3e00 msgr2=0x55a9827d41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.654+0000 7fe0b59c4740 1 -- 172.21.15.2:0/1513275479 shutdown_connections 2024-05-14T17:02:13.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.654+0000 7fe0b59c4740 1 -- 172.21.15.2:0/1513275479 wait complete. 2024-05-14T17:02:13.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.655+0000 7fe0b59c4740 1 -- start start 2024-05-14T17:02:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.655+0000 7fe0b59c4740 1 Processor -- start 2024-05-14T17:02:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.655+0000 7fe0b59c4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a9827d78f0 0x55a9827e23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.655+0000 7fe0b59c4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9827d8210 0x55a9827df690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.655+0000 7fe0b59c4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9827e2ad0 0x55a9827dfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.655+0000 7fe0b59c4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55a982420790 con 0x55a9827d78f0 2024-05-14T17:02:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.655+0000 7fe0b59c4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a9826f6890 con 0x55a9827e2ad0 2024-05-14T17:02:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.655+0000 7fe0b59c4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a9827e57f0 con 0x55a9827d8210 2024-05-14T17:02:13.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.656+0000 7fe0b0a3c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a9827d78f0 0x55a9827e23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:13.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.656+0000 7fe0b0a3c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a9827d78f0 0x55a9827e23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60270/0 (socket says 172.21.15.2:60270) 2024-05-14T17:02:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.656+0000 7fe0b59c4740 10 client.0 _ll_register_callbacks cb 0x55a9826f9d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:02:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.656+0000 7fe0b0a3c640 1 -- 172.21.15.2:0/743042892 learned_addr learned my addr 172.21.15.2:0/743042892 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:02:13.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.656+0000 7fe0b123d640 1 --2- 172.21.15.2:0/743042892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9827e2ad0 0x55a9827dfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:13.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.656+0000 7fe0abfff640 1 --2- 172.21.15.2:0/743042892 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9827d8210 0x55a9827df690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:13.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.656+0000 7fe0b0a3c640 1 -- 172.21.15.2:0/743042892 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9827d8210 msgr2=0x55a9827df690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.656+0000 7fe0b0a3c640 1 --2- 172.21.15.2:0/743042892 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9827d8210 0x55a9827df690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.656+0000 7fe0b0a3c640 1 -- 172.21.15.2:0/743042892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9827e2ad0 msgr2=0x55a9827dfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:13.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.656+0000 7fe0b0a3c640 1 --2- 172.21.15.2:0/743042892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9827e2ad0 0x55a9827dfbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:13.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.656+0000 7fe0b0a3c640 1 -- 172.21.15.2:0/743042892 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a98251b620 con 0x55a9827d78f0 2024-05-14T17:02:13.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.656+0000 7fe0b123d640 1 --2- 172.21.15.2:0/743042892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9827e2ad0 0x55a9827dfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:02:13.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.658+0000 7fe0b0a3c640 1 --2- 172.21.15.2:0/743042892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a9827d78f0 0x55a9827e23a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0009560 tx=0x7fe0a0005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3293b886365da84 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:13.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.658+0000 7fe08bfff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:02:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.658+0000 7fe08bfff640 1 -- 172.21.15.2:0/743042892 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a0029070 con 0x55a9827d78f0 2024-05-14T17:02:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.658+0000 7fe08bfff640 1 -- 172.21.15.2:0/743042892 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8557+0+0 (secure 0 0 0) 0x7fe0a00196a0 con 0x55a9827d78f0 2024-05-14T17:02:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.658+0000 7fe0b59c4740 1 -- 172.21.15.2:0/743042892 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0a0004c50 con 0x55a9827d78f0 2024-05-14T17:02:13.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.658+0000 7fe08bfff640 1 -- 172.21.15.2:0/743042892 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fe0a0033050 con 0x55a9827d78f0 2024-05-14T17:02:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.658+0000 7fe0b59c4740 10 client.16140 fetch_fsmap learned FSMap version 40 2024-05-14T17:02:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.659+0000 7fe0b59c4740 1 -- 172.21.15.2:0/743042892 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a9827f11d0 con 0x55a9827d78f0 2024-05-14T17:02:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.659+0000 7fe08bfff640 1 -- 172.21.15.2:0/743042892 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0a0004c50 con 0x55a9827d78f0 2024-05-14T17:02:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.659+0000 7fe08bfff640 1 -- 172.21.15.2:0/743042892 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7fe0a002e4d0 con 0x55a9827d78f0 2024-05-14T17:02:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.659+0000 7fe0b59c4740 10 client.16140 fetch_fsmap finished waiting for FSMap version 40 2024-05-14T17:02:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.659+0000 7fe0b59c4740 10 client.16140 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:02:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.659+0000 7fe0b59c4740 20 client.16140 populate_metadata read hostname 'smithi002' 2024-05-14T17:02:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.659+0000 7fe0b59c4740 1 --2- 172.21.15.2:0/743042892 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55a9827f25d0 0x55a9827f49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.659+0000 7fe0b59c4740 4 client.16140 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:02:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.659+0000 7fe0b59c4740 1 -- 172.21.15.2:0/743042892 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a98274cb80 con 0x55a9827f25d0 2024-05-14T17:02:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.659+0000 7fe0abfff640 1 --2- 172.21.15.2:0/743042892 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55a9827f25d0 0x55a9827f49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.660+0000 7fe0abfff640 1 --2- 172.21.15.2:0/743042892 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55a9827f25d0 0x55a9827f49c0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c844d6431d8d9648 server_cookie=e365a25abd78f90c in_seq=0 out_seq=0 2024-05-14T17:02:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.660+0000 7fe08bfff640 10 client.16140 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:02:14.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.660+0000 7fe08bfff640 1 -- 172.21.15.2:0/743042892 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a98274cb80 con 0x55a9827f25d0 2024-05-14T17:02:14.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.660+0000 7fe08bfff640 10 client.16140 handle_command_reply: tid=0 2024-05-14T17:02:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.660+0000 7fe0b59c4740 1 client.16140 shutdown 2024-05-14T17:02:14.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.661+0000 7fe0b59c4740 1 -- 172.21.15.2:0/743042892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a9827d78f0 msgr2=0x55a9827e23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:14.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.661+0000 7fe0b59c4740 1 --2- 172.21.15.2:0/743042892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a9827d78f0 0x55a9827e23a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0009560 tx=0x7fe0a0005230 comp rx=0 tx=0).stop 2024-05-14T17:02:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.661+0000 7fe0b59c4740 1 -- 172.21.15.2:0/743042892 shutdown_connections 2024-05-14T17:02:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.661+0000 7fe0b59c4740 1 --2- 172.21.15.2:0/743042892 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55a9827f25d0 0x55a9827f49c0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:14.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.661+0000 7fe0b59c4740 1 --2- 172.21.15.2:0/743042892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9827e2ad0 0x55a9827dfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.661+0000 7fe08bfff640 0 client.16140 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:02:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.661+0000 7fe0b59c4740 1 --2- 172.21.15.2:0/743042892 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9827d8210 0x55a9827df690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:14.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.661+0000 7fe0b59c4740 1 --2- 172.21.15.2:0/743042892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a9827d78f0 0x55a9827e23a0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.661+0000 7fe0b59c4740 1 -- 172.21.15.2:0/743042892 >> 172.21.15.2:0/743042892 conn(0x55a9827d3e00 msgr2=0x55a9827d3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:14.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.661+0000 7fe0b59c4740 1 -- 172.21.15.2:0/743042892 shutdown_connections 2024-05-14T17:02:14.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.662+0000 7fe0b59c4740 1 -- 172.21.15.2:0/743042892 wait complete. 2024-05-14T17:02:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.662+0000 7fe0b59c4740 20 client.16140 trim_cache size 0 max 16384 2024-05-14T17:02:14.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.686+0000 7fe0b59c4740 1 Processor -- start 2024-05-14T17:02:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.686+0000 7fe0b59c4740 1 -- start start 2024-05-14T17:02:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.686+0000 7fe0b59c4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a9827d78f0 0x55a9827d2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.686+0000 7fe0b59c4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9827d8210 0x55a9827e1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.686+0000 7fe0b59c4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9827e2ad0 0x55a9827fa150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:14.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.686+0000 7fe0b59c4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ac1b7730 con 0x55a9827d78f0 2024-05-14T17:02:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.686+0000 7fe0b59c4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ac1b4fd0 con 0x55a9827d8210 2024-05-14T17:02:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.686+0000 7fe0b59c4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ac1b2e40 con 0x55a9827e2ad0 2024-05-14T17:02:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.687+0000 7fe0b0a3c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a9827d78f0 0x55a9827d2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.687+0000 7fe0b0a3c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a9827d78f0 0x55a9827d2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60286/0 (socket says 172.21.15.2:60286) 2024-05-14T17:02:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.687+0000 7fe0b0a3c640 1 -- 172.21.15.2:0/2702256111 learned_addr learned my addr 172.21.15.2:0/2702256111 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:02:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.687+0000 7fe0b123d640 1 --2- 172.21.15.2:0/2702256111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9827e2ad0 0x55a9827fa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:14.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.687+0000 7fe0abfff640 1 --2- 172.21.15.2:0/2702256111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9827d8210 0x55a9827e1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:14.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.687+0000 7fe0b0a3c640 1 -- 172.21.15.2:0/2702256111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9827e2ad0 msgr2=0x55a9827fa150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:14.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.687+0000 7fe0b0a3c640 1 --2- 172.21.15.2:0/2702256111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9827e2ad0 0x55a9827fa150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:14.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.687+0000 7fe0b0a3c640 1 -- 172.21.15.2:0/2702256111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9827d8210 msgr2=0x55a9827e1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:14.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.687+0000 7fe0b0a3c640 1 --2- 172.21.15.2:0/2702256111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9827d8210 0x55a9827e1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:14.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.687+0000 7fe0b0a3c640 1 -- 172.21.15.2:0/2702256111 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9827d3c30 con 0x55a9827d78f0 2024-05-14T17:02:14.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.687+0000 7fe0abfff640 1 --2- 172.21.15.2:0/2702256111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9827d8210 0x55a9827e1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:02:14.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.687+0000 7fe0b0a3c640 1 --2- 172.21.15.2:0/2702256111 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a9827d78f0 0x55a9827d2f80 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0009530 tx=0x7fe0a0001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=777f7d80c8150d1c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:14.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.687+0000 7fe0967fc640 1 -- 172.21.15.2:0/2702256111 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a0029050 con 0x55a9827d78f0 2024-05-14T17:02:14.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.687+0000 7fe0967fc640 1 -- 172.21.15.2:0/2702256111 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0a00044e0 con 0x55a9827d78f0 2024-05-14T17:02:14.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.687+0000 7fe0967fc640 1 -- 172.21.15.2:0/2702256111 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a0004e00 con 0x55a9827d78f0 2024-05-14T17:02:14.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.688+0000 7fe0b59c4740 1 -- 172.21.15.2:0/2702256111 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a9827d78f0 msgr2=0x55a9827d2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:14.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.688+0000 7fe0b59c4740 1 --2- 172.21.15.2:0/2702256111 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a9827d78f0 0x55a9827d2f80 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0009530 tx=0x7fe0a0001e50 comp rx=0 tx=0).stop 2024-05-14T17:02:14.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.688+0000 7fe0b59c4740 1 -- 172.21.15.2:0/2702256111 shutdown_connections 2024-05-14T17:02:14.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.688+0000 7fe0b59c4740 1 --2- 172.21.15.2:0/2702256111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9827e2ad0 0x55a9827fa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:14.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.688+0000 7fe0b59c4740 1 --2- 172.21.15.2:0/2702256111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9827d8210 0x55a9827e1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:14.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.688+0000 7fe0b59c4740 1 --2- 172.21.15.2:0/2702256111 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a9827d78f0 0x55a9827d2f80 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:14.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.688+0000 7fe0b59c4740 1 -- 172.21.15.2:0/2702256111 >> 172.21.15.2:0/2702256111 conn(0x55a9827d3e00 msgr2=0x55a9827d2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:14.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.688+0000 7fe0b59c4740 1 -- 172.21.15.2:0/2702256111 shutdown_connections 2024-05-14T17:02:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.688+0000 7fe0b59c4740 1 -- 172.21.15.2:0/2702256111 wait complete. 2024-05-14T17:02:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.688+0000 7fe0b59c4740 1 -- start start 2024-05-14T17:02:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.689+0000 7fe0b59c4740 1 Processor -- start 2024-05-14T17:02:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.689+0000 7fe0b59c4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9827d78f0 0x55a9827eb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.689+0000 7fe0b59c4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a9827d8210 0x55a9827e6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.689+0000 7fe0b59c4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9827e2ad0 0x55a9827e6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.689+0000 7fe0b59c4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ac1b3490 con 0x55a9827d8210 2024-05-14T17:02:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.689+0000 7fe0b59c4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ac1b4900 con 0x55a9827e2ad0 2024-05-14T17:02:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.689+0000 7fe0b59c4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ac1b3b60 con 0x55a9827d78f0 2024-05-14T17:02:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.689+0000 7fe0b59c4740 10 client.0 _ll_register_callbacks cb 0x7fe0a0004a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:02:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.689+0000 7fe0abfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a9827d8210 0x55a9827e6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.689+0000 7fe0b0a3c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9827d78f0 0x55a9827eb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.689+0000 7fe0b123d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9827e2ad0 0x55a9827e6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:14.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.689+0000 7fe0b0a3c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9827d78f0 0x55a9827eb0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:49042/0 (socket says 172.21.15.2:49042) 2024-05-14T17:02:14.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.689+0000 7fe0b0a3c640 1 -- 172.21.15.2:0/3971984149 learned_addr learned my addr 172.21.15.2:0/3971984149 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:02:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.690+0000 7fe0b0a3c640 1 -- 172.21.15.2:0/3971984149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9827e2ad0 msgr2=0x55a9827e6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.690+0000 7fe0b0a3c640 1 --2- 172.21.15.2:0/3971984149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9827e2ad0 0x55a9827e6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.690+0000 7fe0b0a3c640 1 -- 172.21.15.2:0/3971984149 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a9827d8210 msgr2=0x55a9827e6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.690+0000 7fe0b0a3c640 1 --2- 172.21.15.2:0/3971984149 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a9827d8210 0x55a9827e6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:14.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.690+0000 7fe0b0a3c640 1 -- 172.21.15.2:0/3971984149 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a98285bd00 con 0x55a9827d78f0 2024-05-14T17:02:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.690+0000 7fe0b123d640 1 --2- 172.21.15.2:0/3971984149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9827e2ad0 0x55a9827e6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:02:14.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.690+0000 7fe0b0a3c640 1 --2- 172.21.15.2:0/3971984149 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9827d78f0 0x55a9827eb0f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe0a00059f0 tx=0x7fe0a002e870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19bb5369cbd5b6b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:14.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.690+0000 7fe08bfff640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-14T17:02:14.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.691+0000 7fe08bfff640 1 -- 172.21.15.2:0/3971984149 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a0047060 con 0x55a9827d78f0 2024-05-14T17:02:14.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.691+0000 7fe08bfff640 1 -- 172.21.15.2:0/3971984149 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8557+0+0 (secure 0 0 0) 0x7fe0a002e9f0 con 0x55a9827d78f0 2024-05-14T17:02:14.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.691+0000 7fe0b59c4740 1 -- 172.21.15.2:0/3971984149 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0a00044e0 con 0x55a9827d78f0 2024-05-14T17:02:14.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.691+0000 7fe08bfff640 1 -- 172.21.15.2:0/3971984149 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fe0a0029040 con 0x55a9827d78f0 2024-05-14T17:02:14.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.691+0000 7fe0b59c4740 10 client.25804 fetch_fsmap learned FSMap version 40 2024-05-14T17:02:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.691+0000 7fe0b59c4740 1 -- 172.21.15.2:0/3971984149 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a98285ce70 con 0x55a9827d78f0 2024-05-14T17:02:14.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.691+0000 7fe08bfff640 1 -- 172.21.15.2:0/3971984149 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0a00044e0 con 0x55a9827d78f0 2024-05-14T17:02:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.691+0000 7fe08bfff640 1 -- 172.21.15.2:0/3971984149 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7fe0a004bb30 con 0x55a9827d78f0 2024-05-14T17:02:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.692+0000 7fe0b59c4740 10 client.25804 fetch_fsmap finished waiting for FSMap version 40 2024-05-14T17:02:14.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.692+0000 7fe0b59c4740 10 client.25804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:02:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.692+0000 7fe0b59c4740 20 client.25804 populate_metadata read hostname 'smithi002' 2024-05-14T17:02:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.692+0000 7fe0b59c4740 1 --2- 172.21.15.2:0/3971984149 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55a9827f25d0 0x55a98285f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:14.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.692+0000 7fe0b59c4740 4 client.25804 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:02:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.692+0000 7fe0b59c4740 1 -- 172.21.15.2:0/3971984149 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a982746360 con 0x55a9827f25d0 2024-05-14T17:02:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.692+0000 7fe0abfff640 1 --2- 172.21.15.2:0/3971984149 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55a9827f25d0 0x55a98285f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:15.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.692+0000 7fe0abfff640 1 --2- 172.21.15.2:0/3971984149 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55a9827f25d0 0x55a98285f520 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=717af5c2b950d85 server_cookie=2e3565ed2e987034 in_seq=0 out_seq=0 2024-05-14T17:02:15.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.692+0000 7fe08bfff640 10 client.25804 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:02:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.704+0000 7fe08bfff640 1 -- 172.21.15.2:0/3971984149 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a982746360 con 0x55a9827f25d0 2024-05-14T17:02:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.704+0000 7fe08bfff640 10 client.25804 handle_command_reply: tid=0 2024-05-14T17:02:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.704+0000 7fe0b59c4740 1 client.25804 shutdown 2024-05-14T17:02:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.704+0000 7fe0b59c4740 1 -- 172.21.15.2:0/3971984149 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9827d78f0 msgr2=0x55a9827eb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:15.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.704+0000 7fe0b59c4740 1 --2- 172.21.15.2:0/3971984149 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9827d78f0 0x55a9827eb0f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe0a00059f0 tx=0x7fe0a002e870 comp rx=0 tx=0).stop 2024-05-14T17:02:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.704+0000 7fe0b59c4740 1 -- 172.21.15.2:0/3971984149 shutdown_connections 2024-05-14T17:02:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.704+0000 7fe0b59c4740 1 --2- 172.21.15.2:0/3971984149 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55a9827f25d0 0x55a98285f520 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.704+0000 7fe08bfff640 0 client.25804 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:02:15.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.704+0000 7fe0b59c4740 1 --2- 172.21.15.2:0/3971984149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9827e2ad0 0x55a9827e6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:15.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.704+0000 7fe0b59c4740 1 --2- 172.21.15.2:0/3971984149 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a9827d8210 0x55a9827e6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:15.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.704+0000 7fe0b59c4740 1 --2- 172.21.15.2:0/3971984149 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9827d78f0 0x55a9827eb0f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:15.178 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.704+0000 7fe0b59c4740 1 -- 172.21.15.2:0/3971984149 >> 172.21.15.2:0/3971984149 conn(0x55a9827d3e00 msgr2=0x55a9827d3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:15.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.704+0000 7fe0b59c4740 1 -- 172.21.15.2:0/3971984149 shutdown_connections 2024-05-14T17:02:15.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.704+0000 7fe0b59c4740 1 -- 172.21.15.2:0/3971984149 wait complete. 2024-05-14T17:02:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.704+0000 7fe0b59c4740 20 client.25804 trim_cache size 0 max 16384 2024-05-14T17:02:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:02:15.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:02:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:02:15.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.706+0000 7fe0b2ca0640 1 -- 172.21.15.2:0/212900569 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fe09c0b2a90 msgr2=0x7fe09c0b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.706+0000 7fe0b2ca0640 1 --2- 172.21.15.2:0/212900569 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fe09c0b2a90 0x7fe09c0b4f50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4005fd0 tx=0x7fe0a4012040 comp rx=0 tx=0).stop 2024-05-14T17:02:15.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.706+0000 7fe0b2ca0640 1 -- 172.21.15.2:0/212900569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0ac10a370 msgr2=0x7fe0ac1a7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:15.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.706+0000 7fe0b2ca0640 1 --2- 172.21.15.2:0/212900569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0ac10a370 0x7fe0ac1a7c00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe098006010 tx=0x7fe0980040c0 comp rx=0 tx=0).stop 2024-05-14T17:02:15.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.706+0000 7fe0b2ca0640 1 -- 172.21.15.2:0/212900569 shutdown_connections 2024-05-14T17:02:15.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.706+0000 7fe0b2ca0640 1 --2- 172.21.15.2:0/212900569 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fe09c0b2a90 0x7fe09c0b4f50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.706+0000 7fe0b2ca0640 1 --2- 172.21.15.2:0/212900569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0ac115ba0 0x7fe0ac1a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:15.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.706+0000 7fe0b2ca0640 1 --2- 172.21.15.2:0/212900569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0ac10a370 0x7fe0ac1a7c00 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.706+0000 7fe0b2ca0640 1 --2- 172.21.15.2:0/212900569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0ac109a50 0x7fe0ac1a76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.707+0000 7fe0b2ca0640 1 -- 172.21.15.2:0/212900569 >> 172.21.15.2:0/212900569 conn(0x7fe0ac06db20 msgr2=0x7fe0ac072b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.707+0000 7fe0b2ca0640 1 -- 172.21.15.2:0/212900569 shutdown_connections 2024-05-14T17:02:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:12.707+0000 7fe0b2ca0640 1 -- 172.21.15.2:0/212900569 wait complete. 2024-05-14T17:02:15.407 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:02:12 smithi002 ceph-mon[31538]: pgmap v231: 97 pgs: 97 active+clean; 4.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 2.04k op/s 2024-05-14T17:02:15.414 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:02:14 smithi002 ceph-mon[31538]: pgmap v232: 97 pgs: 97 active+clean; 4.0 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.8 MiB/s wr, 695 op/s 2024-05-14T17:02:15.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:02:14 smithi123 ceph-mon[35971]: pgmap v232: 97 pgs: 97 active+clean; 4.0 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.8 MiB/s wr, 695 op/s 2024-05-14T17:02:15.438 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:02:14 smithi151 ceph-mon[36066]: pgmap v232: 97 pgs: 97 active+clean; 4.0 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.8 MiB/s wr, 695 op/s 2024-05-14T17:02:16.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:02:16 smithi002 ceph-mon[31538]: pgmap v233: 97 pgs: 97 active+clean; 4.0 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.8 MiB/s wr, 695 op/s 2024-05-14T17:02:16.750 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 420.0 s, 2794.6 tps, lat 36.130 ms stddev 43.489 2024-05-14T17:02:16.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:02:16 smithi151 ceph-mon[36066]: pgmap v233: 97 pgs: 97 active+clean; 4.0 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.8 MiB/s wr, 695 op/s 2024-05-14T17:02:16.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:02:16 smithi123 ceph-mon[35971]: pgmap v233: 97 pgs: 97 active+clean; 4.0 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.8 MiB/s wr, 695 op/s 2024-05-14T17:02:18.409 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:02:18 smithi123 ceph-mon[35971]: osdmap e85: 12 total, 12 up, 12 in 2024-05-14T17:02:18.422 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:02:18 smithi123 ceph-mon[35971]: pgmap v235: 97 pgs: 97 active+clean; 4.0 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 3.7 MiB/s wr, 400 op/s 2024-05-14T17:02:18.483 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:02:18 smithi002 ceph-mon[31538]: osdmap e85: 12 total, 12 up, 12 in 2024-05-14T17:02:18.492 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:02:18 smithi002 ceph-mon[31538]: pgmap v235: 97 pgs: 97 active+clean; 4.0 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 3.7 MiB/s wr, 400 op/s 2024-05-14T17:02:18.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:02:18 smithi151 ceph-mon[36066]: osdmap e85: 12 total, 12 up, 12 in 2024-05-14T17:02:18.611 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:02:18 smithi151 ceph-mon[36066]: pgmap v235: 97 pgs: 97 active+clean; 4.0 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 3.7 MiB/s wr, 400 op/s 2024-05-14T17:02:20.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:02:20 smithi123 ceph-mon[35971]: pgmap v236: 97 pgs: 97 active+clean; 4.0 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 6.4 MiB/s wr, 675 op/s 2024-05-14T17:02:20.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:02:20 smithi002 ceph-mon[31538]: pgmap v236: 97 pgs: 97 active+clean; 4.0 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 6.4 MiB/s wr, 675 op/s 2024-05-14T17:02:20.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:02:20 smithi151 ceph-mon[36066]: pgmap v236: 97 pgs: 97 active+clean; 4.0 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 6.4 MiB/s wr, 675 op/s 2024-05-14T17:02:21.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 425.0 s, 2606.0 tps, lat 38.212 ms stddev 39.469 2024-05-14T17:02:22.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:02:22 smithi123 ceph-mon[35971]: pgmap v237: 97 pgs: 97 active+clean; 4.0 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 6.3 MiB/s wr, 669 op/s 2024-05-14T17:02:22.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:02:22 smithi002 ceph-mon[31538]: pgmap v237: 97 pgs: 97 active+clean; 4.0 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 6.3 MiB/s wr, 669 op/s 2024-05-14T17:02:22.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:02:22 smithi151 ceph-mon[36066]: pgmap v237: 97 pgs: 97 active+clean; 4.0 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 6.3 MiB/s wr, 669 op/s 2024-05-14T17:02:23.084 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 31.3 2024-05-14T17:02:23.091 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:02:23.486 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:02:24.668 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:02:24 smithi151 ceph-mon[36066]: pgmap v238: 97 pgs: 97 active+clean; 4.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.6 MiB/s wr, 735 op/s 2024-05-14T17:02:24.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:02:24 smithi002 ceph-mon[31538]: pgmap v238: 97 pgs: 97 active+clean; 4.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.6 MiB/s wr, 735 op/s 2024-05-14T17:02:24.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:02:24 smithi123 ceph-mon[35971]: pgmap v238: 97 pgs: 97 active+clean; 4.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.6 MiB/s wr, 735 op/s 2024-05-14T17:02:25.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.832+0000 7f8bac96a640 1 -- 172.21.15.2:0/1111381888 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ba81033f0 msgr2=0x7f8ba810f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:25.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.832+0000 7f8bac96a640 1 --2- 172.21.15.2:0/1111381888 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ba81033f0 0x7f8ba810f900 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0009cd0 tx=0x7f8ba002a8e0 comp rx=0 tx=0).stop 2024-05-14T17:02:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.834+0000 7f8bac96a640 1 -- 172.21.15.2:0/1111381888 shutdown_connections 2024-05-14T17:02:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.834+0000 7f8bac96a640 1 --2- 172.21.15.2:0/1111381888 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ba81033f0 0x7f8ba810f900 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:25.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.834+0000 7f8bac96a640 1 --2- 172.21.15.2:0/1111381888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba8102a30 0x7f8ba8102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.834+0000 7f8bac96a640 1 --2- 172.21.15.2:0/1111381888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ba8108a40 0x7f8ba8108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.834+0000 7f8bac96a640 1 -- 172.21.15.2:0/1111381888 >> 172.21.15.2:0/1111381888 conn(0x7f8ba80fe500 msgr2=0x7f8ba8100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.834+0000 7f8bac96a640 1 -- 172.21.15.2:0/1111381888 shutdown_connections 2024-05-14T17:02:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.834+0000 7f8bac96a640 1 -- 172.21.15.2:0/1111381888 wait complete. 2024-05-14T17:02:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.835+0000 7f8bac96a640 1 Processor -- start 2024-05-14T17:02:25.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.835+0000 7f8bac96a640 1 -- start start 2024-05-14T17:02:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.836+0000 7f8bac96a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ba8102a30 0x7f8ba81991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.836+0000 7f8bac96a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ba81033f0 0x7f8ba8199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:25.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.836+0000 7f8bac96a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba8108a40 0x7f8ba819da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:25.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.836+0000 7f8bac96a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba8112d00 con 0x7f8ba8102a30 2024-05-14T17:02:26.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.836+0000 7f8bac96a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba8112b80 con 0x7f8ba8108a40 2024-05-14T17:02:26.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.836+0000 7f8bac96a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba8112e80 con 0x7f8ba81033f0 2024-05-14T17:02:26.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.836+0000 7f8ba659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ba8102a30 0x7f8ba81991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.836+0000 7f8ba5d9b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ba81033f0 0x7f8ba8199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.836+0000 7f8ba5d9b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ba81033f0 0x7f8ba8199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:54344/0 (socket says 172.21.15.2:54344) 2024-05-14T17:02:26.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.836+0000 7f8ba5d9b640 1 -- 172.21.15.2:0/65138333 learned_addr learned my addr 172.21.15.2:0/65138333 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:02:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.836+0000 7f8ba6d9d640 1 --2- 172.21.15.2:0/65138333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba8108a40 0x7f8ba819da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.836+0000 7f8ba5d9b640 1 -- 172.21.15.2:0/65138333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba8108a40 msgr2=0x7f8ba819da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.836+0000 7f8ba5d9b640 1 --2- 172.21.15.2:0/65138333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba8108a40 0x7f8ba819da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.836+0000 7f8ba5d9b640 1 -- 172.21.15.2:0/65138333 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ba8102a30 msgr2=0x7f8ba81991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.836+0000 7f8ba5d9b640 1 --2- 172.21.15.2:0/65138333 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ba8102a30 0x7f8ba81991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:26.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.836+0000 7f8ba5d9b640 1 -- 172.21.15.2:0/65138333 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba819e180 con 0x7f8ba81033f0 2024-05-14T17:02:26.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.836+0000 7f8ba5d9b640 1 --2- 172.21.15.2:0/65138333 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ba81033f0 0x7f8ba8199720 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c00e9f0 tx=0x7f8b9c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.837+0000 7f8b977fe640 1 -- 172.21.15.2:0/65138333 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c00cd80 con 0x7f8ba81033f0 2024-05-14T17:02:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.837+0000 7f8b977fe640 1 -- 172.21.15.2:0/65138333 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b9c0108e0 con 0x7f8ba81033f0 2024-05-14T17:02:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.837+0000 7f8bac96a640 1 -- 172.21.15.2:0/65138333 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ba819e470 con 0x7f8ba81033f0 2024-05-14T17:02:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.837+0000 7f8b977fe640 1 -- 172.21.15.2:0/65138333 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c018980 con 0x7f8ba81033f0 2024-05-14T17:02:26.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.837+0000 7f8bac96a640 1 -- 172.21.15.2:0/65138333 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ba81a5d00 con 0x7f8ba81033f0 2024-05-14T17:02:26.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.840+0000 7f8baf682740 1 Processor -- start 2024-05-14T17:02:26.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.840+0000 7f8baf682740 1 -- start start 2024-05-14T17:02:26.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.841+0000 7f8baf682740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580eb1df250 0x5580eb1ddc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:26.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.841+0000 7f8baf682740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580eb1de180 0x5580eb1de5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:26.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.841+0000 7f8baf682740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580eb1ed170 0x5580eb1ef560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.841+0000 7f8baf682740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5580eb14cc30 con 0x5580eb1ed170 2024-05-14T17:02:26.418 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.841+0000 7f8baf682740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5580eb13c7d0 con 0x5580eb1de180 2024-05-14T17:02:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.841+0000 7f8baf682740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5580eb140810 con 0x5580eb1df250 2024-05-14T17:02:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.841+0000 7f8ba6d9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580eb1df250 0x5580eb1ddc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.841+0000 7f8ba6d9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580eb1df250 0x5580eb1ddc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:54352/0 (socket says 172.21.15.2:54352) 2024-05-14T17:02:26.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.841+0000 7f8ba6d9d640 1 -- 172.21.15.2:0/1025731774 learned_addr learned my addr 172.21.15.2:0/1025731774 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:02:26.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.841+0000 7f8ba659c640 1 --2- 172.21.15.2:0/1025731774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580eb1de180 0x5580eb1de5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.841+0000 7f8ba659c640 1 -- 172.21.15.2:0/1025731774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580eb1df250 msgr2=0x5580eb1ddc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.841+0000 7f8ba659c640 1 --2- 172.21.15.2:0/1025731774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580eb1df250 0x5580eb1ddc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.841+0000 7f8ba659c640 1 -- 172.21.15.2:0/1025731774 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580eb1ed170 msgr2=0x5580eb1ef560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T17:02:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.841+0000 7f8ba659c640 1 --2- 172.21.15.2:0/1025731774 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580eb1ed170 0x5580eb1ef560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:26.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.841+0000 7f8ba659c640 1 -- 172.21.15.2:0/1025731774 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580eae51210 con 0x5580eb1de180 2024-05-14T17:02:26.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.841+0000 7f8ba6d9d640 1 --2- 172.21.15.2:0/1025731774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580eb1df250 0x5580eb1ddc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:02:26.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.841+0000 7f8b977fe640 1 -- 172.21.15.2:0/65138333 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f8b9c010430 con 0x7f8ba81033f0 2024-05-14T17:02:26.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.841+0000 7f8ba659c640 1 --2- 172.21.15.2:0/1025731774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580eb1de180 0x5580eb1de5e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8b98009c00 tx=0x7f8b98011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c399be3536c97372 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.842+0000 7f8b94ff9640 1 -- 172.21.15.2:0/1025731774 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b98004050 con 0x5580eb1de180 2024-05-14T17:02:26.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.842+0000 7f8b94ff9640 1 -- 172.21.15.2:0/1025731774 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b980054d0 con 0x5580eb1de180 2024-05-14T17:02:26.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.842+0000 7f8b94ff9640 1 -- 172.21.15.2:0/1025731774 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b98004c40 con 0x5580eb1de180 2024-05-14T17:02:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.842+0000 7f8b977fe640 1 --2- 172.21.15.2:0/65138333 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f8b900b2a10 0x7f8b900b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:26.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.843+0000 7f8baf682740 1 -- 172.21.15.2:0/1025731774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580eb1de180 msgr2=0x5580eb1de5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.843+0000 7f8baf682740 1 --2- 172.21.15.2:0/1025731774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580eb1de180 0x5580eb1de5e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8b98009c00 tx=0x7f8b98011810 comp rx=0 tx=0).stop 2024-05-14T17:02:26.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.843+0000 7f8ba6d9d640 1 --2- 172.21.15.2:0/65138333 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f8b900b2a10 0x7f8b900b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.843+0000 7f8b977fe640 1 -- 172.21.15.2:0/65138333 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f8b9c0e2940 con 0x7f8ba81033f0 2024-05-14T17:02:26.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.843+0000 7f8baf682740 1 -- 172.21.15.2:0/1025731774 shutdown_connections 2024-05-14T17:02:26.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.843+0000 7f8baf682740 1 --2- 172.21.15.2:0/1025731774 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580eb1ed170 0x5580eb1ef560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:26.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.843+0000 7f8baf682740 1 --2- 172.21.15.2:0/1025731774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580eb1de180 0x5580eb1de5e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:26.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.843+0000 7f8baf682740 1 --2- 172.21.15.2:0/1025731774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580eb1df250 0x5580eb1ddc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.843+0000 7f8baf682740 1 -- 172.21.15.2:0/1025731774 >> 172.21.15.2:0/1025731774 conn(0x5580eb1da210 msgr2=0x5580eb1da5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:26.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.843+0000 7f8baf682740 1 -- 172.21.15.2:0/1025731774 shutdown_connections 2024-05-14T17:02:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.843+0000 7f8baf682740 1 -- 172.21.15.2:0/1025731774 wait complete. 2024-05-14T17:02:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.843+0000 7f8baf682740 1 -- start start 2024-05-14T17:02:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.843+0000 7f8ba6d9d640 1 --2- 172.21.15.2:0/65138333 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f8b900b2a10 0x7f8b900b4ed0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0005e10 tx=0x7f8ba003d040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.843+0000 7f8baf682740 1 Processor -- start 2024-05-14T17:02:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.844+0000 7f8baf682740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580eb1de180 0x5580eb1e69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:26.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.844+0000 7f8baf682740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580eb1df250 0x5580eb1e6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:26.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.844+0000 7f8baf682740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580eb1ed170 0x5580eb1ec6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:26.993 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.844+0000 7f8baf682740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5580eae22790 con 0x5580eb1df250 2024-05-14T17:02:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.844+0000 7f8baf682740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5580eb0f8890 con 0x5580eb1de180 2024-05-14T17:02:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.844+0000 7f8baf682740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5580eb1e0de0 con 0x5580eb1ed170 2024-05-14T17:02:27.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.844+0000 7f8baf682740 10 client.0 _ll_register_callbacks cb 0x5580eb0fbd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:02:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.844+0000 7f8ba5d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580eb1df250 0x5580eb1e6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:27.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.844+0000 7f8ba5d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580eb1df250 0x5580eb1e6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36046/0 (socket says 172.21.15.2:36046) 2024-05-14T17:02:27.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.844+0000 7f8ba5d9b640 1 -- 172.21.15.2:0/2927578738 learned_addr learned my addr 172.21.15.2:0/2927578738 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:02:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.844+0000 7f8ba6d9d640 1 --2- 172.21.15.2:0/2927578738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580eb1ed170 0x5580eb1ec6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.844+0000 7f8ba659c640 1 --2- 172.21.15.2:0/2927578738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580eb1de180 0x5580eb1e69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.844+0000 7f8ba6d9d640 1 -- 172.21.15.2:0/2927578738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580eb1de180 msgr2=0x5580eb1e69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.844+0000 7f8ba6d9d640 1 --2- 172.21.15.2:0/2927578738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580eb1de180 0x5580eb1e69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.844+0000 7f8ba6d9d640 1 -- 172.21.15.2:0/2927578738 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580eb1df250 msgr2=0x5580eb1e6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.844+0000 7f8ba6d9d640 1 --2- 172.21.15.2:0/2927578738 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580eb1df250 0x5580eb1e6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:27.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.844+0000 7f8ba6d9d640 1 -- 172.21.15.2:0/2927578738 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580eaf1d620 con 0x5580eb1ed170 2024-05-14T17:02:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.844+0000 7f8ba659c640 1 --2- 172.21.15.2:0/2927578738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580eb1de180 0x5580eb1e69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:02:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.844+0000 7f8ba5d9b640 1 --2- 172.21.15.2:0/2927578738 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580eb1df250 0x5580eb1e6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:02:27.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.845+0000 7f8ba6d9d640 1 --2- 172.21.15.2:0/2927578738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580eb1ed170 0x5580eb1ec6c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0035000 tx=0x7f8ba002a8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd766f385f29e5f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:27.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.845+0000 7f8b8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-14T17:02:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.845+0000 7f8b8dffb640 1 -- 172.21.15.2:0/2927578738 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba0056060 con 0x5580eb1ed170 2024-05-14T17:02:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.845+0000 7f8b8dffb640 1 -- 172.21.15.2:0/2927578738 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f8ba0003e00 con 0x5580eb1ed170 2024-05-14T17:02:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.845+0000 7f8baf682740 1 -- 172.21.15.2:0/2927578738 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b980054d0 con 0x5580eb1ed170 2024-05-14T17:02:27.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.845+0000 7f8b8dffb640 1 -- 172.21.15.2:0/2927578738 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f8ba0004750 con 0x5580eb1ed170 2024-05-14T17:02:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.846+0000 7f8baf682740 10 client.25816 fetch_fsmap learned FSMap version 40 2024-05-14T17:02:27.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.846+0000 7f8baf682740 1 -- 172.21.15.2:0/2927578738 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5580eb204290 con 0x5580eb1ed170 2024-05-14T17:02:27.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.846+0000 7f8b8dffb640 1 -- 172.21.15.2:0/2927578738 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b980054d0 con 0x5580eb1ed170 2024-05-14T17:02:27.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.846+0000 7f8b8dffb640 1 -- 172.21.15.2:0/2927578738 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7f8ba0030070 con 0x5580eb1ed170 2024-05-14T17:02:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.847+0000 7f8baf682740 10 client.25816 fetch_fsmap finished waiting for FSMap version 40 2024-05-14T17:02:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.847+0000 7f8baf682740 10 client.25816 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:02:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.847+0000 7f8baf682740 20 client.25816 populate_metadata read hostname 'smithi002' 2024-05-14T17:02:27.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.847+0000 7f8baf682740 1 --2- 172.21.15.2:0/2927578738 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5580eb2057a0 0x5580eb207b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:27.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.847+0000 7f8baf682740 4 client.25816 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:02:27.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.847+0000 7f8baf682740 1 -- 172.21.15.2:0/2927578738 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580eb14eb80 con 0x5580eb2057a0 2024-05-14T17:02:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.848+0000 7f8ba659c640 1 --2- 172.21.15.2:0/2927578738 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5580eb2057a0 0x5580eb207b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.848+0000 7f8ba659c640 1 --2- 172.21.15.2:0/2927578738 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5580eb2057a0 0x5580eb207b90 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4e613a86694637d server_cookie=f62735ab37d014b4 in_seq=0 out_seq=0 2024-05-14T17:02:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.848+0000 7f8b8dffb640 10 client.25816 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:02:27.400 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.849+0000 7f8b8dffb640 1 -- 172.21.15.2:0/2927578738 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5580eb14eb80 con 0x5580eb2057a0 2024-05-14T17:02:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.849+0000 7f8b8dffb640 10 client.25816 handle_command_reply: tid=0 2024-05-14T17:02:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.849+0000 7f8baf682740 1 client.25816 shutdown 2024-05-14T17:02:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.849+0000 7f8baf682740 1 -- 172.21.15.2:0/2927578738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580eb1ed170 msgr2=0x5580eb1ec6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.849+0000 7f8baf682740 1 --2- 172.21.15.2:0/2927578738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580eb1ed170 0x5580eb1ec6c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0035000 tx=0x7f8ba002a8e0 comp rx=0 tx=0).stop 2024-05-14T17:02:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.849+0000 7f8baf682740 1 -- 172.21.15.2:0/2927578738 shutdown_connections 2024-05-14T17:02:27.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.849+0000 7f8baf682740 1 --2- 172.21.15.2:0/2927578738 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5580eb2057a0 0x5580eb207b90 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.849+0000 7f8baf682740 1 --2- 172.21.15.2:0/2927578738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580eb1ed170 0x5580eb1ec6c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:27.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.849+0000 7f8b8dffb640 0 client.25816 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:02:27.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.849+0000 7f8baf682740 1 --2- 172.21.15.2:0/2927578738 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580eb1df250 0x5580eb1e6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.849+0000 7f8baf682740 1 --2- 172.21.15.2:0/2927578738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580eb1de180 0x5580eb1e69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.849+0000 7f8baf682740 1 -- 172.21.15.2:0/2927578738 >> 172.21.15.2:0/2927578738 conn(0x5580eb1da210 msgr2=0x5580eb1e0080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.849+0000 7f8baf682740 1 -- 172.21.15.2:0/2927578738 shutdown_connections 2024-05-14T17:02:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.849+0000 7f8baf682740 1 -- 172.21.15.2:0/2927578738 wait complete. 2024-05-14T17:02:27.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.850+0000 7f8baf682740 20 client.25816 trim_cache size 0 max 16384 2024-05-14T17:02:27.566 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:02:26 smithi123 ceph-mon[35971]: pgmap v239: 97 pgs: 97 active+clean; 4.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.6 MiB/s wr, 735 op/s 2024-05-14T17:02:27.573 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:02:26 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/1846060470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:02:27.580 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:02:26 smithi151 ceph-mon[36066]: pgmap v239: 97 pgs: 97 active+clean; 4.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.6 MiB/s wr, 735 op/s 2024-05-14T17:02:27.587 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:02:26 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/1846060470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:02:27.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:25.858+0000 7fd000704700 1 -- 172.21.15.2:0/707783838 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcff806b220 msgr2=0x7fcff806b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:27.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:25.858+0000 7fd000704700 1 --2- 172.21.15.2:0/707783838 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcff806b220 0x7fcff806b6a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8009990 tx=0x7fcfe8009ca0 comp rx=0 tx=0).stop 2024-05-14T17:02:27.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:25.858+0000 7fd000704700 1 -- 172.21.15.2:0/707783838 shutdown_connections 2024-05-14T17:02:27.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:25.858+0000 7fd000704700 1 --2- 172.21.15.2:0/707783838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcff806bbe0 0x7fcff81103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:27.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:25.858+0000 7fd000704700 1 --2- 172.21.15.2:0/707783838 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcff81095b0 0x7fcff8109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:27.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:25.858+0000 7fd000704700 1 --2- 172.21.15.2:0/707783838 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcff806b220 0x7fcff806b6a0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:27.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:25.858+0000 7fd000704700 1 -- 172.21.15.2:0/707783838 >> 172.21.15.2:0/707783838 conn(0x7fcff80793f0 msgr2=0x7fcff8079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:27.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:25.858+0000 7fd000704700 1 -- 172.21.15.2:0/707783838 shutdown_connections 2024-05-14T17:02:27.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:25.858+0000 7fd000704700 1 -- 172.21.15.2:0/707783838 wait complete. 2024-05-14T17:02:27.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:25.859+0000 7fd000704700 1 Processor -- start 2024-05-14T17:02:27.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:25.859+0000 7fd000704700 1 -- start start 2024-05-14T17:02:27.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:25.859+0000 7fd000704700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcff806b220 0x7fcff819f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:27.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:25.859+0000 7fd000704700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcff806bbe0 0x7fcff819fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:27.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:25.859+0000 7fd000704700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcff81095b0 0x7fcff8199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:27.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:25.859+0000 7fd000704700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcff81136c0 con 0x7fcff806bbe0 2024-05-14T17:02:27.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:25.860+0000 7fd000704700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcff8113540 con 0x7fcff81095b0 2024-05-14T17:02:27.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:25.860+0000 7fd000704700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcff8113840 con 0x7fcff806b220 2024-05-14T17:02:27.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:25.860+0000 7fcffeca1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcff81095b0 0x7fcff8199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:27.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:25.860+0000 7fcffeca1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcff81095b0 0x7fcff8199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:47328/0 (socket says 172.21.15.2:47328) 2024-05-14T17:02:27.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:25.860+0000 7fcffeca1700 1 -- 172.21.15.2:0/1846060470 learned_addr learned my addr 172.21.15.2:0/1846060470 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:02:27.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:25.860+0000 7fcffdc9f700 1 --2- 172.21.15.2:0/1846060470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcff806bbe0 0x7fcff819fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:27.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:25.860+0000 7fcffeca1700 1 -- 172.21.15.2:0/1846060470 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcff806b220 msgr2=0x7fcff819f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:27.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:25.860+0000 7fcffe4a0700 1 --2- 172.21.15.2:0/1846060470 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcff806b220 0x7fcff819f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:27.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:25.860+0000 7fcffeca1700 1 --2- 172.21.15.2:0/1846060470 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcff806b220 0x7fcff819f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:27.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:25.860+0000 7fcffeca1700 1 -- 172.21.15.2:0/1846060470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcff806bbe0 msgr2=0x7fcff819fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:27.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:25.860+0000 7fcffeca1700 1 --2- 172.21.15.2:0/1846060470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcff806bbe0 0x7fcff819fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:27.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:25.860+0000 7fcffeca1700 1 -- 172.21.15.2:0/1846060470 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff819a1a0 con 0x7fcff81095b0 2024-05-14T17:02:27.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:25.861+0000 7fcffeca1700 1 --2- 172.21.15.2:0/1846060470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcff81095b0 0x7fcff8199a40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcff000c370 tx=0x7fcff000c730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:27.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:25.861+0000 7fcfef7fe700 1 -- 172.21.15.2:0/1846060470 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff00090d0 con 0x7fcff81095b0 2024-05-14T17:02:27.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:25.861+0000 7fd000704700 1 -- 172.21.15.2:0/1846060470 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcff819a430 con 0x7fcff81095b0 2024-05-14T17:02:27.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:25.861+0000 7fd000704700 1 -- 172.21.15.2:0/1846060470 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcff806cf60 con 0x7fcff81095b0 2024-05-14T17:02:27.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:25.861+0000 7fcfef7fe700 1 -- 172.21.15.2:0/1846060470 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcff0009270 con 0x7fcff81095b0 2024-05-14T17:02:27.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:25.861+0000 7fcfef7fe700 1 -- 172.21.15.2:0/1846060470 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff0004840 con 0x7fcff81095b0 2024-05-14T17:02:27.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:25.863+0000 7fd000704700 1 -- 172.21.15.2:0/1846060470 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfdc005330 con 0x7fcff81095b0 2024-05-14T17:02:27.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:25.865+0000 7fcfef7fe700 1 -- 172.21.15.2:0/1846060470 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fcff001a020 con 0x7fcff81095b0 2024-05-14T17:02:27.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:25.865+0000 7fcfef7fe700 1 --2- 172.21.15.2:0/1846060470 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcfe40b6f50 0x7fcfe40b9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:28.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:25.865+0000 7fcffe4a0700 1 --2- 172.21.15.2:0/1846060470 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcfe40b6f50 0x7fcfe40b9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:28.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:25.866+0000 7fcfef7fe700 1 -- 172.21.15.2:0/1846060470 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7fcff00e1e20 con 0x7fcff81095b0 2024-05-14T17:02:28.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:25.866+0000 7fcffe4a0700 1 --2- 172.21.15.2:0/1846060470 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcfe40b6f50 0x7fcfe40b9410 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcff819ab80 tx=0x7fcff4009260 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:28.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.871+0000 7f8baf682740 1 Processor -- start 2024-05-14T17:02:28.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:25.871+0000 7fcfef7fe700 1 -- 172.21.15.2:0/1846060470 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcff0010040 con 0x7fcff81095b0 2024-05-14T17:02:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.871+0000 7f8baf682740 1 -- start start 2024-05-14T17:02:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.871+0000 7f8baf682740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580eb1de180 0x5580eb1dfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.871+0000 7f8baf682740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580eb1df250 0x5580eb1e03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.871+0000 7f8baf682740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580eb1ed170 0x5580eb1e6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.871+0000 7f8baf682740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba81aed30 con 0x5580eb1ed170 2024-05-14T17:02:28.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.871+0000 7f8baf682740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba81ac5d0 con 0x5580eb1de180 2024-05-14T17:02:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.871+0000 7f8baf682740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba81aa440 con 0x5580eb1df250 2024-05-14T17:02:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.871+0000 7f8ba5d9b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580eb1df250 0x5580eb1e03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.871+0000 7f8ba5d9b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580eb1df250 0x5580eb1e03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:54384/0 (socket says 172.21.15.2:54384) 2024-05-14T17:02:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.871+0000 7f8ba5d9b640 1 -- 172.21.15.2:0/2719058372 learned_addr learned my addr 172.21.15.2:0/2719058372 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:02:28.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.871+0000 7f8ba6d9d640 1 --2- 172.21.15.2:0/2719058372 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580eb1ed170 0x5580eb1e6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:28.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.872+0000 7f8ba659c640 1 --2- 172.21.15.2:0/2719058372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580eb1de180 0x5580eb1dfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:28.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.872+0000 7f8ba5d9b640 1 -- 172.21.15.2:0/2719058372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580eb1de180 msgr2=0x5580eb1dfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:28.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.872+0000 7f8ba5d9b640 1 --2- 172.21.15.2:0/2719058372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580eb1de180 0x5580eb1dfe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.872+0000 7f8ba5d9b640 1 -- 172.21.15.2:0/2719058372 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580eb1ed170 msgr2=0x5580eb1e6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:28.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.872+0000 7f8ba5d9b640 1 --2- 172.21.15.2:0/2719058372 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580eb1ed170 0x5580eb1e6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:28.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.872+0000 7f8ba5d9b640 1 -- 172.21.15.2:0/2719058372 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580eb1e1030 con 0x5580eb1df250 2024-05-14T17:02:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.872+0000 7f8ba5d9b640 1 --2- 172.21.15.2:0/2719058372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580eb1df250 0x5580eb1e03a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c0a5d50 tx=0x7f8b9c010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd581f26f36d5965 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.872+0000 7f8b8ffff640 1 -- 172.21.15.2:0/2719058372 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c0b23d0 con 0x5580eb1df250 2024-05-14T17:02:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.872+0000 7f8b8ffff640 1 -- 172.21.15.2:0/2719058372 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b9c0adfd0 con 0x5580eb1df250 2024-05-14T17:02:28.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.872+0000 7f8b8ffff640 1 -- 172.21.15.2:0/2719058372 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c0a5640 con 0x5580eb1df250 2024-05-14T17:02:28.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.873+0000 7f8baf682740 1 -- 172.21.15.2:0/2719058372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580eb1df250 msgr2=0x5580eb1e03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:28.397 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.873+0000 7f8baf682740 1 --2- 172.21.15.2:0/2719058372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580eb1df250 0x5580eb1e03a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c0a5d50 tx=0x7f8b9c010860 comp rx=0 tx=0).stop 2024-05-14T17:02:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.873+0000 7f8baf682740 1 -- 172.21.15.2:0/2719058372 shutdown_connections 2024-05-14T17:02:28.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.873+0000 7f8baf682740 1 --2- 172.21.15.2:0/2719058372 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580eb1ed170 0x5580eb1e6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.873+0000 7f8baf682740 1 --2- 172.21.15.2:0/2719058372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580eb1df250 0x5580eb1e03a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:28.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.873+0000 7f8baf682740 1 --2- 172.21.15.2:0/2719058372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580eb1de180 0x5580eb1dfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.873+0000 7f8baf682740 1 -- 172.21.15.2:0/2719058372 >> 172.21.15.2:0/2719058372 conn(0x5580eb1da210 msgr2=0x5580eb1df750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:28.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.874+0000 7f8baf682740 1 -- 172.21.15.2:0/2719058372 shutdown_connections 2024-05-14T17:02:28.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.874+0000 7f8baf682740 1 -- 172.21.15.2:0/2719058372 wait complete. 2024-05-14T17:02:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.875+0000 7f8baf682740 1 -- start start 2024-05-14T17:02:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.875+0000 7f8baf682740 1 Processor -- start 2024-05-14T17:02:28.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.875+0000 7f8baf682740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580eb1df250 0x5580eb20d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.876+0000 7f8baf682740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580eb1ed170 0x5580eb20a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.876+0000 7f8ba659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580eb1df250 0x5580eb20d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.876+0000 7f8ba659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580eb1df250 0x5580eb20d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:47340/0 (socket says 172.21.15.2:47340) 2024-05-14T17:02:28.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.876+0000 7f8ba5d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580eb1ed170 0x5580eb20a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.876+0000 7f8ba5d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580eb1ed170 0x5580eb20a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36078/0 (socket says 172.21.15.2:36078) 2024-05-14T17:02:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.876+0000 7f8ba659c640 1 -- 172.21.15.2:0/2094540770 learned_addr learned my addr 172.21.15.2:0/2094540770 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:02:28.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.876+0000 7f8baf682740 1 --2- 172.21.15.2:0/2094540770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580eb2057a0 0x5580eb20ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.876+0000 7f8baf682740 1 -- 172.21.15.2:0/2094540770 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba81aaa90 con 0x5580eb1ed170 2024-05-14T17:02:28.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.876+0000 7f8baf682740 1 -- 172.21.15.2:0/2094540770 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba81abf00 con 0x5580eb1df250 2024-05-14T17:02:28.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.876+0000 7f8baf682740 1 -- 172.21.15.2:0/2094540770 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba81ab160 con 0x5580eb2057a0 2024-05-14T17:02:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.876+0000 7f8baf682740 10 client.0 _ll_register_callbacks cb 0x7f8ba0031ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:02:28.691 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.876+0000 7f8ba6d9d640 1 --2- 172.21.15.2:0/2094540770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580eb2057a0 0x5580eb20ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:28.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.876+0000 7f8ba659c640 1 -- 172.21.15.2:0/2094540770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580eb2057a0 msgr2=0x5580eb20ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:28.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.876+0000 7f8ba659c640 1 --2- 172.21.15.2:0/2094540770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580eb2057a0 0x5580eb20ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.876+0000 7f8ba659c640 1 -- 172.21.15.2:0/2094540770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580eb1ed170 msgr2=0x5580eb20a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:28.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.876+0000 7f8ba659c640 1 --2- 172.21.15.2:0/2094540770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580eb1ed170 0x5580eb20a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.876+0000 7f8ba659c640 1 -- 172.21.15.2:0/2094540770 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580eb20b3d0 con 0x5580eb1df250 2024-05-14T17:02:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.877+0000 7f8ba659c640 1 --2- 172.21.15.2:0/2094540770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580eb1df250 0x5580eb20d5c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8b9801b040 tx=0x7f8b98004870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22b0b4dff3a34dfc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.877+0000 7f8b8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T17:02:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.877+0000 7f8b8dffb640 1 -- 172.21.15.2:0/2094540770 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b98033060 con 0x5580eb1df250 2024-05-14T17:02:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.877+0000 7f8baf682740 1 -- 172.21.15.2:0/2094540770 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b9c0adfd0 con 0x5580eb1df250 2024-05-14T17:02:28.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.877+0000 7f8b8dffb640 1 -- 172.21.15.2:0/2094540770 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f8b9802e5e0 con 0x5580eb1df250 2024-05-14T17:02:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.877+0000 7f8b8dffb640 1 -- 172.21.15.2:0/2094540770 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f8b98011d10 con 0x5580eb1df250 2024-05-14T17:02:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.877+0000 7f8baf682740 10 client.25841 fetch_fsmap learned FSMap version 40 2024-05-14T17:02:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.877+0000 7f8baf682740 1 -- 172.21.15.2:0/2094540770 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5580eb26fc60 con 0x5580eb1df250 2024-05-14T17:02:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.878+0000 7f8b8dffb640 1 -- 172.21.15.2:0/2094540770 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b9c0adfd0 con 0x5580eb1df250 2024-05-14T17:02:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.878+0000 7f8b8dffb640 1 -- 172.21.15.2:0/2094540770 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7f8b98019430 con 0x5580eb1df250 2024-05-14T17:02:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.878+0000 7f8baf682740 10 client.25841 fetch_fsmap finished waiting for FSMap version 40 2024-05-14T17:02:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.878+0000 7f8baf682740 10 client.25841 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:02:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.878+0000 7f8baf682740 20 client.25841 populate_metadata read hostname 'smithi002' 2024-05-14T17:02:28.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.878+0000 7f8baf682740 1 --2- 172.21.15.2:0/2094540770 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5580eb270220 0x5580eb272640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.878+0000 7f8baf682740 4 client.25841 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:02:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.878+0000 7f8baf682740 1 -- 172.21.15.2:0/2094540770 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5580eb1da060 con 0x5580eb270220 2024-05-14T17:02:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.878+0000 7f8ba5d9b640 1 --2- 172.21.15.2:0/2094540770 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5580eb270220 0x5580eb272640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:29.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.879+0000 7f8b8dffb640 10 client.25841 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:02:29.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.879+0000 7f8ba5d9b640 1 --2- 172.21.15.2:0/2094540770 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5580eb270220 0x5580eb272640 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af621848e4cb2131 server_cookie=fad50fbc88a7d56a in_seq=0 out_seq=0 2024-05-14T17:02:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.891+0000 7f8b8dffb640 1 -- 172.21.15.2:0/2094540770 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5580eb1da060 con 0x5580eb270220 2024-05-14T17:02:29.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.891+0000 7f8b8dffb640 10 client.25841 handle_command_reply: tid=0 2024-05-14T17:02:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.891+0000 7f8baf682740 1 client.25841 shutdown 2024-05-14T17:02:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.891+0000 7f8baf682740 1 -- 172.21.15.2:0/2094540770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580eb1df250 msgr2=0x5580eb20d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.891+0000 7f8baf682740 1 --2- 172.21.15.2:0/2094540770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580eb1df250 0x5580eb20d5c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8b9801b040 tx=0x7f8b98004870 comp rx=0 tx=0).stop 2024-05-14T17:02:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.891+0000 7f8baf682740 1 -- 172.21.15.2:0/2094540770 shutdown_connections 2024-05-14T17:02:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.891+0000 7f8baf682740 1 --2- 172.21.15.2:0/2094540770 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5580eb270220 0x5580eb272640 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.891+0000 7f8baf682740 1 --2- 172.21.15.2:0/2094540770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580eb2057a0 0x5580eb20ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.891+0000 7f8b8dffb640 0 client.25841 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:02:29.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.891+0000 7f8baf682740 1 --2- 172.21.15.2:0/2094540770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580eb1ed170 0x5580eb20a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:29.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.891+0000 7f8baf682740 1 --2- 172.21.15.2:0/2094540770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5580eb1df250 0x5580eb20d5c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.892+0000 7f8baf682740 1 -- 172.21.15.2:0/2094540770 >> 172.21.15.2:0/2094540770 conn(0x5580eb1da210 msgr2=0x5580eb1dffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.892+0000 7f8baf682740 1 -- 172.21.15.2:0/2094540770 shutdown_connections 2024-05-14T17:02:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.892+0000 7f8baf682740 1 -- 172.21.15.2:0/2094540770 wait complete. 2024-05-14T17:02:29.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.892+0000 7f8baf682740 20 client.25841 trim_cache size 0 max 16384 2024-05-14T17:02:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:02:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:02:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:02:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.892+0000 7f8bac96a640 1 -- 172.21.15.2:0/65138333 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f8b900b2a10 msgr2=0x7f8b900b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.892+0000 7f8bac96a640 1 --2- 172.21.15.2:0/65138333 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f8b900b2a10 0x7f8b900b4ed0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0005e10 tx=0x7f8ba003d040 comp rx=0 tx=0).stop 2024-05-14T17:02:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.892+0000 7f8bac96a640 1 -- 172.21.15.2:0/65138333 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ba81033f0 msgr2=0x7f8ba8199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:29.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.892+0000 7f8bac96a640 1 --2- 172.21.15.2:0/65138333 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ba81033f0 0x7f8ba8199720 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c00e9f0 tx=0x7f8b9c00eeb0 comp rx=0 tx=0).stop 2024-05-14T17:02:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.892+0000 7f8bac96a640 1 -- 172.21.15.2:0/65138333 shutdown_connections 2024-05-14T17:02:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.892+0000 7f8bac96a640 1 --2- 172.21.15.2:0/65138333 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f8b900b2a10 0x7f8b900b4ed0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.892+0000 7f8bac96a640 1 --2- 172.21.15.2:0/65138333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba8108a40 0x7f8ba819da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.892+0000 7f8bac96a640 1 --2- 172.21.15.2:0/65138333 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ba81033f0 0x7f8ba8199720 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.892+0000 7f8bac96a640 1 --2- 172.21.15.2:0/65138333 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ba8102a30 0x7f8ba81991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.892+0000 7f8bac96a640 1 -- 172.21.15.2:0/65138333 >> 172.21.15.2:0/65138333 conn(0x7f8ba80fe500 msgr2=0x7f8ba80fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.893+0000 7f8bac96a640 1 -- 172.21.15.2:0/65138333 shutdown_connections 2024-05-14T17:02:29.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:25.893+0000 7f8bac96a640 1 -- 172.21.15.2:0/65138333 wait complete. 2024-05-14T17:02:29.415 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:02:28 smithi123 ceph-mon[35971]: pgmap v240: 97 pgs: 97 active+clean; 4.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.5 MiB/s wr, 719 op/s 2024-05-14T17:02:29.421 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:02:28 smithi151 ceph-mon[36066]: pgmap v240: 97 pgs: 97 active+clean; 4.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.5 MiB/s wr, 719 op/s 2024-05-14T17:02:29.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:26.259+0000 7fd000704700 1 -- 172.21.15.2:0/1846060470 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcfdc0058f0 con 0x7fcff81095b0 2024-05-14T17:02:29.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:26.260+0000 7fcfef7fe700 1 -- 172.21.15.2:0/1846060470 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11854 (secure 0 0 0) 0x7fcff00a9800 con 0x7fcff81095b0 2024-05-14T17:02:29.475 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:02:29.543 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2275696923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2275696923},{"type":"v1","addr":"172.21.15.2:6835","nonce":2275696923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/716718761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":716718761},{"type":"v1","addr":"172.21.15.2:6839","nonce":716718761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/3644815179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3644815179},{"type":"v1","addr":"172.21.15.2:6841","nonce":3644815179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1880336731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1880336731},{"type":"v1","addr":"172.21.15.151:6833","nonce":1880336731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2415605548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2415605548},{"type":"v1","addr":"172.21.15.2:6837","nonce":2415605548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2635167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2635167},{"type":"v1","addr":"172.21.15.123:6833","nonce":2635167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:51:34.106919+0000","modified":"2024-05-14T17:01:13.455170+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24455,"mds_2":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24415":{"gid":24415,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/762044339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":762044339},{"type":"v1","addr":"172.21.15.151:6835","nonce":762044339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/132043151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":132043151},{"type":"v1","addr":"172.21.15.151:6837","nonce":132043151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/4107834375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":4107834375},{"type":"v1","addr":"172.21.15.151:6839","nonce":4107834375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.123:6835/1198670013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1198670013},{"type":"v1","addr":"172.21.15.123:6835","nonce":1198670013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/333426645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":333426645},{"type":"v1","addr":"172.21.15.123:6837","nonce":333426645}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/580275877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":580275877},{"type":"v1","addr":"172.21.15.123:6839","nonce":580275877}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24431,24455,24479]},"id":1}]} 2024-05-14T17:02:29.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:26.263+0000 7fd000704700 1 -- 172.21.15.2:0/1846060470 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcfe40b6f50 msgr2=0x7fcfe40b9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:29.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:26.263+0000 7fd000704700 1 --2- 172.21.15.2:0/1846060470 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcfe40b6f50 0x7fcfe40b9410 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcff819ab80 tx=0x7fcff4009260 comp rx=0 tx=0).stop 2024-05-14T17:02:29.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:26.263+0000 7fd000704700 1 -- 172.21.15.2:0/1846060470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcff81095b0 msgr2=0x7fcff8199a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:29.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:26.263+0000 7fd000704700 1 --2- 172.21.15.2:0/1846060470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcff81095b0 0x7fcff8199a40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcff000c370 tx=0x7fcff000c730 comp rx=0 tx=0).stop 2024-05-14T17:02:29.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:26.263+0000 7fd000704700 1 -- 172.21.15.2:0/1846060470 shutdown_connections 2024-05-14T17:02:29.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:26.263+0000 7fd000704700 1 --2- 172.21.15.2:0/1846060470 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcfe40b6f50 0x7fcfe40b9410 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:29.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:26.263+0000 7fd000704700 1 --2- 172.21.15.2:0/1846060470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcff81095b0 0x7fcff8199a40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:29.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:26.263+0000 7fd000704700 1 --2- 172.21.15.2:0/1846060470 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcff806b220 0x7fcff819f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:29.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:26.263+0000 7fd000704700 1 --2- 172.21.15.2:0/1846060470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcff806bbe0 0x7fcff819fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:29.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:26.263+0000 7fd000704700 1 -- 172.21.15.2:0/1846060470 >> 172.21.15.2:0/1846060470 conn(0x7fcff80793f0 msgr2=0x7fcff81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:29.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:26.263+0000 7fd000704700 1 -- 172.21.15.2:0/1846060470 shutdown_connections 2024-05-14T17:02:29.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:26.263+0000 7fd000704700 1 -- 172.21.15.2:0/1846060470 wait complete. 2024-05-14T17:02:29.683 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 40 2024-05-14T17:02:29.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:02:26 smithi002 ceph-mon[31538]: pgmap v239: 97 pgs: 97 active+clean; 4.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.6 MiB/s wr, 735 op/s 2024-05-14T17:02:29.753 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:02:26 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/1846060470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:02:29.763 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 430.0 s, 2542.6 tps, lat 39.228 ms stddev 42.899 2024-05-14T17:02:29.801 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 151.3 --await-for 45.0' 2024-05-14T17:02:29.810 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 151.3 --await-for 45.0 2024-05-14T17:02:29.821 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:02:28 smithi002 ceph-mon[31538]: pgmap v240: 97 pgs: 97 active+clean; 4.0 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.5 MiB/s wr, 719 op/s 2024-05-14T17:02:30.194 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:02:30.662 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:02:30 smithi002 ceph-mon[31538]: pgmap v241: 97 pgs: 97 active+clean; 4.0 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 9.1 MiB/s wr, 894 op/s 2024-05-14T17:02:30.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:02:30 smithi151 ceph-mon[36066]: pgmap v241: 97 pgs: 97 active+clean; 4.0 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 9.1 MiB/s wr, 894 op/s 2024-05-14T17:02:30.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:02:30 smithi123 ceph-mon[35971]: pgmap v241: 97 pgs: 97 active+clean; 4.0 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 9.1 MiB/s wr, 894 op/s 2024-05-14T17:02:31.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 435.0 s, 2294.0 tps, lat 43.672 ms stddev 51.183 2024-05-14T17:02:32.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.538+0000 7f57ced1a700 1 -- 172.21.15.2:0/1697581603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57c811cd20 msgr2=0x7f57c811f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:32.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.538+0000 7f57ced1a700 1 --2- 172.21.15.2:0/1697581603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57c811cd20 0x7f57c811f110 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f57b8009f90 tx=0x7f57b800b330 comp rx=0 tx=0).stop 2024-05-14T17:02:32.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.538+0000 7f57ced1a700 1 -- 172.21.15.2:0/1697581603 shutdown_connections 2024-05-14T17:02:32.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.538+0000 7f57ced1a700 1 --2- 172.21.15.2:0/1697581603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57c811cd20 0x7f57c811f110 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:32.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.538+0000 7f57ced1a700 1 --2- 172.21.15.2:0/1697581603 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f57c8111d90 0x7f57c810a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:32.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.538+0000 7f57ced1a700 1 --2- 172.21.15.2:0/1697581603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57c810aa80 0x7f57c810af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:32.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.538+0000 7f57ced1a700 1 -- 172.21.15.2:0/1697581603 >> 172.21.15.2:0/1697581603 conn(0x7f57c806e6b0 msgr2=0x7f57c806eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:32.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.539+0000 7f57ced1a700 1 -- 172.21.15.2:0/1697581603 shutdown_connections 2024-05-14T17:02:32.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.539+0000 7f57ced1a700 1 -- 172.21.15.2:0/1697581603 wait complete. 2024-05-14T17:02:32.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.539+0000 7f57ced1a700 1 Processor -- start 2024-05-14T17:02:32.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.539+0000 7f57ced1a700 1 -- start start 2024-05-14T17:02:32.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.540+0000 7f57ced1a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f57c810aa80 0x7f57c81ae670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:32.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.540+0000 7f57ced1a700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57c8111d90 0x7f57c81aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:32.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.540+0000 7f57ced1a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57c811cd20 0x7f57c81a8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:32.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.540+0000 7f57ced1a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f57c8122540 con 0x7f57c810aa80 2024-05-14T17:02:32.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.540+0000 7f57ced1a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f57c81223c0 con 0x7f57c811cd20 2024-05-14T17:02:32.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.540+0000 7f57ced1a700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f57c81226c0 con 0x7f57c8111d90 2024-05-14T17:02:32.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.540+0000 7f57ccab6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f57c810aa80 0x7f57c81ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:32.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.540+0000 7f57cd2b7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57c811cd20 0x7f57c81a8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:32.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.540+0000 7f57cd2b7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57c811cd20 0x7f57c81a8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:47360/0 (socket says 172.21.15.2:47360) 2024-05-14T17:02:32.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.540+0000 7f57cd2b7700 1 -- 172.21.15.2:0/567594536 learned_addr learned my addr 172.21.15.2:0/567594536 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:02:32.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.540+0000 7f57c7fff700 1 --2- 172.21.15.2:0/567594536 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57c8111d90 0x7f57c81aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:33.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.540+0000 7f57cd2b7700 1 -- 172.21.15.2:0/567594536 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57c8111d90 msgr2=0x7f57c81aebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:33.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.540+0000 7f57cd2b7700 1 --2- 172.21.15.2:0/567594536 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57c8111d90 0x7f57c81aebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:33.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.540+0000 7f57cd2b7700 1 -- 172.21.15.2:0/567594536 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f57c810aa80 msgr2=0x7f57c81ae670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:33.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.540+0000 7f57cd2b7700 1 --2- 172.21.15.2:0/567594536 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f57c810aa80 0x7f57c81ae670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:33.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.540+0000 7f57cd2b7700 1 -- 172.21.15.2:0/567594536 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57c81a9020 con 0x7f57c811cd20 2024-05-14T17:02:33.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.540+0000 7f57ccab6700 1 --2- 172.21.15.2:0/567594536 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f57c810aa80 0x7f57c81ae670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:02:33.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.540+0000 7f57cd2b7700 1 --2- 172.21.15.2:0/567594536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57c811cd20 0x7f57c81a8740 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f57b800b6c0 tx=0x7f57b800cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:33.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.541+0000 7f57c5ffb700 1 -- 172.21.15.2:0/567594536 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b8021070 con 0x7f57c811cd20 2024-05-14T17:02:33.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.541+0000 7f57c5ffb700 1 -- 172.21.15.2:0/567594536 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57b801d450 con 0x7f57c811cd20 2024-05-14T17:02:33.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.541+0000 7f57ced1a700 1 -- 172.21.15.2:0/567594536 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57c81a92b0 con 0x7f57c811cd20 2024-05-14T17:02:33.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.541+0000 7f57c5ffb700 1 -- 172.21.15.2:0/567594536 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b80052a0 con 0x7f57c811cd20 2024-05-14T17:02:33.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.541+0000 7f57ced1a700 1 -- 172.21.15.2:0/567594536 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57c81b5420 con 0x7f57c811cd20 2024-05-14T17:02:33.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.546+0000 7f57c5ffb700 1 -- 172.21.15.2:0/567594536 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f57b8024070 con 0x7f57c811cd20 2024-05-14T17:02:33.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.547+0000 7f57d95b0b80 1 Processor -- start 2024-05-14T17:02:33.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.547+0000 7f57d95b0b80 1 -- start start 2024-05-14T17:02:33.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.547+0000 7f57c5ffb700 1 --2- 172.21.15.2:0/567594536 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f57b00b2aa0 0x7f57b00b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:33.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.547+0000 7f57c7fff700 1 --2- 172.21.15.2:0/567594536 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f57b00b2aa0 0x7f57b00b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:33.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.548+0000 7f57c5ffb700 1 -- 172.21.15.2:0/567594536 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f57b80e35f0 con 0x7f57c811cd20 2024-05-14T17:02:33.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.548+0000 7f57d95b0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f77aee30 0x55d7f77af210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:33.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.548+0000 7f57d95b0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f77af750 0x55d7f77a9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:33.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.548+0000 7f57d95b0b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f77a9b60 0x55d7f77a9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:33.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.548+0000 7f57c7fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f77af750 0x55d7f77a9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:33.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.548+0000 7f57d95b0b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d7f7707e20 con 0x55d7f77a9b60 2024-05-14T17:02:33.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.548+0000 7f57d95b0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d7f76f79c0 con 0x55d7f77af750 2024-05-14T17:02:33.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.548+0000 7f57d95b0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d7f76fba00 con 0x55d7f77aee30 2024-05-14T17:02:33.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.548+0000 7f57c7fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f77af750 0x55d7f77a9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:47370/0 (socket says 172.21.15.2:47370) 2024-05-14T17:02:33.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.548+0000 7f57c7fff700 1 -- 172.21.15.2:0/2206779907 learned_addr learned my addr 172.21.15.2:0/2206779907 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:02:33.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.548+0000 7f57c7fff700 1 --2- 172.21.15.2:0/567594536 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f57b00b2aa0 0x7f57b00b4f60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f57c81a9f00 tx=0x7f57c000b410 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:33.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.548+0000 7f57ccab6700 1 --2- 172.21.15.2:0/2206779907 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f77aee30 0x55d7f77af210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:33.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.549+0000 7f57ccab6700 1 -- 172.21.15.2:0/2206779907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f77af750 msgr2=0x55d7f77a9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:33.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.549+0000 7f57ccab6700 1 --2- 172.21.15.2:0/2206779907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f77af750 0x55d7f77a9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:33.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.549+0000 7f57ccab6700 1 -- 172.21.15.2:0/2206779907 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f77a9b60 msgr2=0x55d7f77a9fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T17:02:33.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.549+0000 7f57ccab6700 1 --2- 172.21.15.2:0/2206779907 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f77a9b60 0x55d7f77a9fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:33.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.549+0000 7f57ccab6700 1 -- 172.21.15.2:0/2206779907 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7f7647a60 con 0x55d7f77aee30 2024-05-14T17:02:33.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.549+0000 7f57ccab6700 1 --2- 172.21.15.2:0/2206779907 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f77aee30 0x55d7f77af210 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f57bc011440 tx=0x7f57bc011750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c42361f94d6016a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:33.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.549+0000 7f57aeffd700 1 -- 172.21.15.2:0/2206779907 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57bc004230 con 0x55d7f77aee30 2024-05-14T17:02:33.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.549+0000 7f57aeffd700 1 -- 172.21.15.2:0/2206779907 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57bc0043d0 con 0x55d7f77aee30 2024-05-14T17:02:33.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.550+0000 7f57aeffd700 1 -- 172.21.15.2:0/2206779907 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57bc00e8a0 con 0x55d7f77aee30 2024-05-14T17:02:33.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.550+0000 7f57d95b0b80 1 -- 172.21.15.2:0/2206779907 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f77aee30 msgr2=0x55d7f77af210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:33.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.550+0000 7f57d95b0b80 1 --2- 172.21.15.2:0/2206779907 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f77aee30 0x55d7f77af210 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f57bc011440 tx=0x7f57bc011750 comp rx=0 tx=0).stop 2024-05-14T17:02:33.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.552+0000 7f57d95b0b80 1 -- 172.21.15.2:0/2206779907 shutdown_connections 2024-05-14T17:02:33.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.552+0000 7f57d95b0b80 1 --2- 172.21.15.2:0/2206779907 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f77a9b60 0x55d7f77a9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:33.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.552+0000 7f57d95b0b80 1 --2- 172.21.15.2:0/2206779907 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f77aee30 0x55d7f77af210 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:33.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.552+0000 7f57d95b0b80 1 --2- 172.21.15.2:0/2206779907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f77af750 0x55d7f77a9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:33.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.552+0000 7f57d95b0b80 1 -- 172.21.15.2:0/2206779907 >> 172.21.15.2:0/2206779907 conn(0x55d7f76d43e0 msgr2=0x55d7f77a3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:33.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.552+0000 7f57d95b0b80 1 -- 172.21.15.2:0/2206779907 shutdown_connections 2024-05-14T17:02:33.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.552+0000 7f57d95b0b80 1 -- 172.21.15.2:0/2206779907 wait complete. 2024-05-14T17:02:33.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.552+0000 7f57d95b0b80 1 -- start start 2024-05-14T17:02:33.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.552+0000 7f57d95b0b80 1 Processor -- start 2024-05-14T17:02:33.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.553+0000 7f57d95b0b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f77a9b60 0x55d7f77b5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:33.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.553+0000 7f57d95b0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f77aee30 0x55d7f77b2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:33.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.554+0000 7f57d95b0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f77af750 0x55d7f77b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:33.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.554+0000 7f57ccab6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f77a9b60 0x55d7f77b5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:33.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.554+0000 7f57d95b0b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d7f75fe130 con 0x55d7f77a9b60 2024-05-14T17:02:33.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.554+0000 7f57ccab6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f77a9b60 0x55d7f77b5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36116/0 (socket says 172.21.15.2:36116) 2024-05-14T17:02:33.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.554+0000 7f57ccab6700 1 -- 172.21.15.2:0/281427340 learned_addr learned my addr 172.21.15.2:0/281427340 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:02:33.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.554+0000 7f57d95b0b80 1 -- 172.21.15.2:0/281427340 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d7f7699950 con 0x55d7f77af750 2024-05-14T17:02:33.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.554+0000 7f57d95b0b80 1 -- 172.21.15.2:0/281427340 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d7f77b61e0 con 0x55d7f77aee30 2024-05-14T17:02:33.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.554+0000 7f57d95b0b80 10 client.0 _ll_register_callbacks cb 0x55d7f768e710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:02:33.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.554+0000 7f57c7fff700 1 --2- 172.21.15.2:0/281427340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f77aee30 0x55d7f77b2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:34.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.554+0000 7f57cd2b7700 1 --2- 172.21.15.2:0/281427340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f77af750 0x55d7f77b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:34.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.554+0000 7f57ccab6700 1 -- 172.21.15.2:0/281427340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f77aee30 msgr2=0x55d7f77b2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:34.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.554+0000 7f57ccab6700 1 --2- 172.21.15.2:0/281427340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f77aee30 0x55d7f77b2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:34.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.554+0000 7f57ccab6700 1 -- 172.21.15.2:0/281427340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f77af750 msgr2=0x55d7f77b2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:34.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.554+0000 7f57ccab6700 1 --2- 172.21.15.2:0/281427340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f77af750 0x55d7f77b2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:34.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.554+0000 7f57ccab6700 1 -- 172.21.15.2:0/281427340 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7f7696820 con 0x55d7f77a9b60 2024-05-14T17:02:34.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.554+0000 7f57ccab6700 1 --2- 172.21.15.2:0/281427340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f77a9b60 0x55d7f77b5370 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f57bc01b040 tx=0x7f57bc005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bfd18c61ad71b3e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:34.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.554+0000 7f57a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:02:34.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.555+0000 7f57a7fff700 1 -- 172.21.15.2:0/281427340 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57bc027070 con 0x55d7f77a9b60 2024-05-14T17:02:34.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.555+0000 7f57d95b0b80 1 -- 172.21.15.2:0/281427340 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57bc0043d0 con 0x55d7f77a9b60 2024-05-14T17:02:34.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.555+0000 7f57a7fff700 1 -- 172.21.15.2:0/281427340 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f57bc004950 con 0x55d7f77a9b60 2024-05-14T17:02:34.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.555+0000 7f57a7fff700 1 -- 172.21.15.2:0/281427340 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f57bc031050 con 0x55d7f77a9b60 2024-05-14T17:02:34.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.555+0000 7f57d95b0b80 10 client.16173 fetch_fsmap learned FSMap version 40 2024-05-14T17:02:34.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.555+0000 7f57d95b0b80 1 -- 172.21.15.2:0/281427340 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d7f77c08b0 con 0x55d7f77a9b60 2024-05-14T17:02:34.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.555+0000 7f57a7fff700 1 -- 172.21.15.2:0/281427340 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57bc0043d0 con 0x55d7f77a9b60 2024-05-14T17:02:34.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.556+0000 7f57a7fff700 1 -- 172.21.15.2:0/281427340 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7f57bc00e540 con 0x55d7f77a9b60 2024-05-14T17:02:34.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.556+0000 7f57d95b0b80 10 client.16173 fetch_fsmap finished waiting for FSMap version 40 2024-05-14T17:02:34.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.556+0000 7f57d95b0b80 10 client.16173 resolve_mds: validated gid 24479 aka daemon mds.b 2024-05-14T17:02:34.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.556+0000 7f57d95b0b80 20 client.16173 populate_metadata read hostname 'smithi002' 2024-05-14T17:02:34.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.556+0000 7f57d95b0b80 1 --2- 172.21.15.2:0/281427340 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d7f77c2e40 0x55d7f77c5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:34.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.556+0000 7f57d95b0b80 4 client.16173 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:02:34.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.556+0000 7f57d95b0b80 1 -- 172.21.15.2:0/281427340 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7f77754f0 con 0x55d7f77c2e40 2024-05-14T17:02:34.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.556+0000 7f57c7fff700 1 --2- 172.21.15.2:0/281427340 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d7f77c2e40 0x55d7f77c5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:34.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.557+0000 7f57c7fff700 1 --2- 172.21.15.2:0/281427340 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d7f77c2e40 0x55d7f77c5230 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90c3f1b49d559bb6 server_cookie=46666e96c43449d0 in_seq=0 out_seq=0 2024-05-14T17:02:34.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.557+0000 7f57a7fff700 10 client.16173 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:02:34.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.558+0000 7f57a7fff700 1 -- 172.21.15.2:0/281427340 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d7f77754f0 con 0x55d7f77c2e40 2024-05-14T17:02:34.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.558+0000 7f57a7fff700 10 client.16173 handle_command_reply: tid=0 2024-05-14T17:02:34.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.558+0000 7f57d95b0b80 1 client.16173 shutdown 2024-05-14T17:02:34.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.558+0000 7f57d95b0b80 1 -- 172.21.15.2:0/281427340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f77a9b60 msgr2=0x55d7f77b5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:34.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.558+0000 7f57d95b0b80 1 --2- 172.21.15.2:0/281427340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f77a9b60 0x55d7f77b5370 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f57bc01b040 tx=0x7f57bc005270 comp rx=0 tx=0).stop 2024-05-14T17:02:34.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.558+0000 7f57d95b0b80 1 -- 172.21.15.2:0/281427340 shutdown_connections 2024-05-14T17:02:34.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.558+0000 7f57d95b0b80 1 --2- 172.21.15.2:0/281427340 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d7f77c2e40 0x55d7f77c5230 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:34.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.558+0000 7f57a7fff700 0 client.16173 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:02:34.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.558+0000 7f57d95b0b80 1 --2- 172.21.15.2:0/281427340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f77af750 0x55d7f77b2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:34.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.558+0000 7f57d95b0b80 1 --2- 172.21.15.2:0/281427340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f77a9b60 0x55d7f77b5370 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:34.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.558+0000 7f57d95b0b80 1 --2- 172.21.15.2:0/281427340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f77aee30 0x55d7f77b2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:34.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.558+0000 7f57d95b0b80 1 -- 172.21.15.2:0/281427340 >> 172.21.15.2:0/281427340 conn(0x55d7f76d43e0 msgr2=0x55d7f779e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:34.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.558+0000 7f57d95b0b80 1 -- 172.21.15.2:0/281427340 shutdown_connections 2024-05-14T17:02:34.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.558+0000 7f57d95b0b80 1 -- 172.21.15.2:0/281427340 wait complete. 2024-05-14T17:02:34.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.558+0000 7f57d95b0b80 20 client.16173 trim_cache size 0 max 16384 2024-05-14T17:02:34.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.598+0000 7f57d95b0b80 1 Processor -- start 2024-05-14T17:02:34.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.598+0000 7f57d95b0b80 1 -- start start 2024-05-14T17:02:34.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.599+0000 7f57d95b0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f77a9b60 0x55d7f77bf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:34.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.599+0000 7f57d95b0b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f77aee30 0x55d7f77b1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:34.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.599+0000 7f57d95b0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f77af750 0x55d7f77a57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:34.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.599+0000 7f57d95b0b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f57c81bf330 con 0x55d7f77aee30 2024-05-14T17:02:34.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.599+0000 7f57d95b0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f57c81bcb90 con 0x55d7f77af750 2024-05-14T17:02:34.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.599+0000 7f57d95b0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f57c81baa00 con 0x55d7f77a9b60 2024-05-14T17:02:34.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.599+0000 7f57ccab6700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f77a9b60 0x55d7f77bf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:34.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.599+0000 7f57ccab6700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f77a9b60 0x55d7f77bf140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:54422/0 (socket says 172.21.15.2:54422) 2024-05-14T17:02:34.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.599+0000 7f57ccab6700 1 -- 172.21.15.2:0/867541566 learned_addr learned my addr 172.21.15.2:0/867541566 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:02:34.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.599+0000 7f57c7fff700 1 --2- 172.21.15.2:0/867541566 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f77aee30 0x55d7f77b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:34.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.599+0000 7f57cd2b7700 1 --2- 172.21.15.2:0/867541566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f77af750 0x55d7f77a57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:34.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.599+0000 7f57ccab6700 1 -- 172.21.15.2:0/867541566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f77af750 msgr2=0x55d7f77a57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:34.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.599+0000 7f57ccab6700 1 --2- 172.21.15.2:0/867541566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f77af750 0x55d7f77a57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:34.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.599+0000 7f57ccab6700 1 -- 172.21.15.2:0/867541566 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f77aee30 msgr2=0x55d7f77b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:34.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.599+0000 7f57ccab6700 1 --2- 172.21.15.2:0/867541566 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f77aee30 0x55d7f77b1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:34.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.599+0000 7f57ccab6700 1 -- 172.21.15.2:0/867541566 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7f77b2fd0 con 0x55d7f77a9b60 2024-05-14T17:02:34.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.600+0000 7f57ccab6700 1 --2- 172.21.15.2:0/867541566 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f77a9b60 0x55d7f77bf140 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f57bc009350 tx=0x7f57bc005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39fae9742800204 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:34.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.600+0000 7f57ae7fc700 1 -- 172.21.15.2:0/867541566 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57bc03a070 con 0x55d7f77a9b60 2024-05-14T17:02:34.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.601+0000 7f57ae7fc700 1 -- 172.21.15.2:0/867541566 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57bc00e4c0 con 0x55d7f77a9b60 2024-05-14T17:02:34.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.601+0000 7f57ae7fc700 1 -- 172.21.15.2:0/867541566 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57bc027070 con 0x55d7f77a9b60 2024-05-14T17:02:34.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.602+0000 7f57d95b0b80 1 -- 172.21.15.2:0/867541566 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f77a9b60 msgr2=0x55d7f77bf140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:34.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.602+0000 7f57d95b0b80 1 --2- 172.21.15.2:0/867541566 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f77a9b60 0x55d7f77bf140 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f57bc009350 tx=0x7f57bc005e80 comp rx=0 tx=0).stop 2024-05-14T17:02:34.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.602+0000 7f57d95b0b80 1 -- 172.21.15.2:0/867541566 shutdown_connections 2024-05-14T17:02:34.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.602+0000 7f57d95b0b80 1 --2- 172.21.15.2:0/867541566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f77af750 0x55d7f77a57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:34.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.602+0000 7f57d95b0b80 1 --2- 172.21.15.2:0/867541566 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f77a9b60 0x55d7f77bf140 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:34.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.602+0000 7f57d95b0b80 1 --2- 172.21.15.2:0/867541566 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f77aee30 0x55d7f77b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:34.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.602+0000 7f57d95b0b80 1 -- 172.21.15.2:0/867541566 >> 172.21.15.2:0/867541566 conn(0x55d7f76d43e0 msgr2=0x55d7f77b4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:34.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.604+0000 7f57d95b0b80 1 -- 172.21.15.2:0/867541566 shutdown_connections 2024-05-14T17:02:34.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.604+0000 7f57d95b0b80 1 -- 172.21.15.2:0/867541566 wait complete. 2024-05-14T17:02:35.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.604+0000 7f57d95b0b80 1 -- start start 2024-05-14T17:02:35.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.604+0000 7f57d95b0b80 1 Processor -- start 2024-05-14T17:02:35.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.605+0000 7f57d95b0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f77a9b60 0x55d7f77bb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:35.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.605+0000 7f57d95b0b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f77aee30 0x55d7f77bbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:35.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.605+0000 7f57d95b0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f77af750 0x55d7f77b8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:35.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.605+0000 7f57d95b0b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f57c81bb050 con 0x55d7f77aee30 2024-05-14T17:02:35.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.605+0000 7f57d95b0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f57c81bc4c0 con 0x55d7f77af750 2024-05-14T17:02:35.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.605+0000 7f57d95b0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f57c81bb720 con 0x55d7f77a9b60 2024-05-14T17:02:35.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.605+0000 7f57d95b0b80 10 client.0 _ll_register_callbacks cb 0x55d7f768e710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:02:35.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.605+0000 7f57cd2b7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f77af750 0x55d7f77b8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:35.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.605+0000 7f57cd2b7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f77af750 0x55d7f77b8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:47386/0 (socket says 172.21.15.2:47386) 2024-05-14T17:02:35.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.605+0000 7f57cd2b7700 1 -- 172.21.15.2:0/4163885886 learned_addr learned my addr 172.21.15.2:0/4163885886 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:02:35.197 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:02:32 smithi002 ceph-mon[31538]: pgmap v242: 97 pgs: 97 active+clean; 4.0 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.2 MiB/s wr, 603 op/s 2024-05-14T17:02:35.212 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:02:32 smithi123 ceph-mon[35971]: pgmap v242: 97 pgs: 97 active+clean; 4.0 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.2 MiB/s wr, 603 op/s 2024-05-14T17:02:35.222 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:02:34 smithi123 ceph-mon[35971]: pgmap v243: 97 pgs: 97 active+clean; 4.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.9 MiB/s wr, 977 op/s 2024-05-14T17:02:35.232 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:02:32 smithi151 ceph-mon[36066]: pgmap v242: 97 pgs: 97 active+clean; 4.0 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.2 MiB/s wr, 603 op/s 2024-05-14T17:02:35.243 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:02:34 smithi151 ceph-mon[36066]: pgmap v243: 97 pgs: 97 active+clean; 4.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.9 MiB/s wr, 977 op/s 2024-05-14T17:02:35.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.605+0000 7f57c7fff700 1 --2- 172.21.15.2:0/4163885886 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f77aee30 0x55d7f77bbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:35.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.606+0000 7f57ccab6700 1 --2- 172.21.15.2:0/4163885886 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f77a9b60 0x55d7f77bb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:35.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.606+0000 7f57cd2b7700 1 -- 172.21.15.2:0/4163885886 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f77a9b60 msgr2=0x55d7f77bb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:35.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.606+0000 7f57cd2b7700 1 --2- 172.21.15.2:0/4163885886 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f77a9b60 0x55d7f77bb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:35.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.606+0000 7f57cd2b7700 1 -- 172.21.15.2:0/4163885886 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f77aee30 msgr2=0x55d7f77bbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:35.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.606+0000 7f57cd2b7700 1 --2- 172.21.15.2:0/4163885886 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f77aee30 0x55d7f77bbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:35.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.606+0000 7f57cd2b7700 1 -- 172.21.15.2:0/4163885886 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7f77b8ed0 con 0x55d7f77af750 2024-05-14T17:02:35.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.607+0000 7f57cd2b7700 1 --2- 172.21.15.2:0/4163885886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f77af750 0x55d7f77b8690 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f57b80e1340 tx=0x7f57b800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80ac68d0eb7b4d4c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:35.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.607+0000 7f57a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T17:02:35.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.608+0000 7f57a7fff700 1 -- 172.21.15.2:0/4163885886 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b802b050 con 0x55d7f77af750 2024-05-14T17:02:35.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.608+0000 7f57a7fff700 1 -- 172.21.15.2:0/4163885886 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f57b80a6bf0 con 0x55d7f77af750 2024-05-14T17:02:35.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.608+0000 7f57d95b0b80 1 -- 172.21.15.2:0/4163885886 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57bc00e4c0 con 0x55d7f77af750 2024-05-14T17:02:35.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.609+0000 7f57a7fff700 1 -- 172.21.15.2:0/4163885886 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f57b80ab060 con 0x55d7f77af750 2024-05-14T17:02:35.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.609+0000 7f57d95b0b80 10 client.25859 fetch_fsmap learned FSMap version 40 2024-05-14T17:02:35.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.609+0000 7f57d95b0b80 1 -- 172.21.15.2:0/4163885886 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d7f782e200 con 0x55d7f77af750 2024-05-14T17:02:35.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.609+0000 7f57a7fff700 1 -- 172.21.15.2:0/4163885886 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57b80aa960 con 0x55d7f77af750 2024-05-14T17:02:35.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.609+0000 7f57a7fff700 1 -- 172.21.15.2:0/4163885886 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7f57b8011070 con 0x55d7f77af750 2024-05-14T17:02:35.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.609+0000 7f57d95b0b80 10 client.25859 fetch_fsmap finished waiting for FSMap version 40 2024-05-14T17:02:35.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.609+0000 7f57d95b0b80 10 client.25859 resolve_mds: validated gid 24479 aka daemon mds.b 2024-05-14T17:02:35.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.609+0000 7f57d95b0b80 20 client.25859 populate_metadata read hostname 'smithi002' 2024-05-14T17:02:35.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.609+0000 7f57d95b0b80 1 --2- 172.21.15.2:0/4163885886 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d7f77c2e40 0x55d7f78309c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:35.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.609+0000 7f57d95b0b80 4 client.25859 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 151.3, "await_for": 45.0}] 2024-05-14T17:02:35.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.609+0000 7f57d95b0b80 1 -- 172.21.15.2:0/4163885886 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 151.3, "await_for": 45.0}) -- 0x7f57bc00e4c0 con 0x55d7f77c2e40 2024-05-14T17:02:35.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.610+0000 7f57ccab6700 1 --2- 172.21.15.2:0/4163885886 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d7f77c2e40 0x55d7f78309c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:35.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.610+0000 7f57a7fff700 10 client.25859 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:02:35.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.610+0000 7f57ccab6700 1 --2- 172.21.15.2:0/4163885886 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d7f77c2e40 0x55d7f78309c0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9c589334852d5ba server_cookie=33578db924d55a70 in_seq=0 out_seq=0 2024-05-14T17:02:35.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.844+0000 7f57a7fff700 1 -- 172.21.15.2:0/4163885886 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x7f57bc00e4c0 con 0x55d7f77c2e40 2024-05-14T17:02:35.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.844+0000 7f57a7fff700 10 client.25859 handle_command_reply: tid=0 2024-05-14T17:02:35.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.844+0000 7f57d95b0b80 1 client.25859 shutdown 2024-05-14T17:02:35.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.844+0000 7f57d95b0b80 1 -- 172.21.15.2:0/4163885886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f77af750 msgr2=0x55d7f77b8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:35.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.844+0000 7f57d95b0b80 1 --2- 172.21.15.2:0/4163885886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f77af750 0x55d7f77b8690 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f57b80e1340 tx=0x7f57b800b330 comp rx=0 tx=0).stop 2024-05-14T17:02:35.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.844+0000 7f57d95b0b80 1 -- 172.21.15.2:0/4163885886 shutdown_connections 2024-05-14T17:02:35.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.844+0000 7f57d95b0b80 1 --2- 172.21.15.2:0/4163885886 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d7f77c2e40 0x55d7f78309c0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:35.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.844+0000 7f57d95b0b80 1 --2- 172.21.15.2:0/4163885886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f77af750 0x55d7f77b8690 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:35.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.844+0000 7f57d95b0b80 1 --2- 172.21.15.2:0/4163885886 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f77a9b60 0x55d7f77bb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:35.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.844+0000 7f57d95b0b80 1 --2- 172.21.15.2:0/4163885886 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f77aee30 0x55d7f77bbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:35.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.844+0000 7f57a7fff700 0 client.25859 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:02:35.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.844+0000 7f57d95b0b80 1 -- 172.21.15.2:0/4163885886 >> 172.21.15.2:0/4163885886 conn(0x55d7f76d43e0 msgr2=0x55d7f77b3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:35.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.844+0000 7f57d95b0b80 1 -- 172.21.15.2:0/4163885886 shutdown_connections 2024-05-14T17:02:35.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.844+0000 7f57d95b0b80 1 -- 172.21.15.2:0/4163885886 wait complete. 2024-05-14T17:02:35.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.844+0000 7f57d95b0b80 20 client.25859 trim_cache size 0 max 16384 2024-05-14T17:02:36.002 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T17:02:36.012 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 40, 2024-05-14T17:02:36.038 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24479, 2024-05-14T17:02:36.046 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 26, 2024-05-14T17:02:36.056 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-05-14T17:02:36.066 INFO:teuthology.orchestra.run.smithi002.stdout: "8649c12": { 2024-05-14T17:02:36.074 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 26, 2024-05-14T17:02:36.084 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-05-14T17:02:36.096 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-05-14T17:02:36.106 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-05-14T17:02:36.121 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-05-14T17:02:36.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:02:36.144 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-05-14T17:02:36.162 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 151.3, 2024-05-14T17:02:36.172 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-05-14T17:02:36.181 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-05-14T17:02:36.216 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-05-14T17:02:36.224 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-05-14T17:02:36.235 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-05-14T17:02:36.307 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-05-14T17:02:36.317 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:02:36.326 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:02:36.336 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:02:36.344 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:02:36.357 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:02:36.366 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T17:02:36.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.845+0000 7f57ced1a700 1 -- 172.21.15.2:0/567594536 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f57b00b2aa0 msgr2=0x7f57b00b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:36.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.845+0000 7f57ced1a700 1 --2- 172.21.15.2:0/567594536 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f57b00b2aa0 0x7f57b00b4f60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f57c81a9f00 tx=0x7f57c000b410 comp rx=0 tx=0).stop 2024-05-14T17:02:36.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.845+0000 7f57ced1a700 1 -- 172.21.15.2:0/567594536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57c811cd20 msgr2=0x7f57c81a8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:36.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.845+0000 7f57ced1a700 1 --2- 172.21.15.2:0/567594536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57c811cd20 0x7f57c81a8740 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f57b800b6c0 tx=0x7f57b800cd70 comp rx=0 tx=0).stop 2024-05-14T17:02:36.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.845+0000 7f57ced1a700 1 -- 172.21.15.2:0/567594536 shutdown_connections 2024-05-14T17:02:36.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.845+0000 7f57ced1a700 1 --2- 172.21.15.2:0/567594536 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f57b00b2aa0 0x7f57b00b4f60 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:36.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.845+0000 7f57ced1a700 1 --2- 172.21.15.2:0/567594536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57c811cd20 0x7f57c81a8740 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:36.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.845+0000 7f57ced1a700 1 --2- 172.21.15.2:0/567594536 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f57c810aa80 0x7f57c81ae670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:36.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.845+0000 7f57ced1a700 1 --2- 172.21.15.2:0/567594536 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57c8111d90 0x7f57c81aebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:36.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.845+0000 7f57ced1a700 1 -- 172.21.15.2:0/567594536 >> 172.21.15.2:0/567594536 conn(0x7f57c806e6b0 msgr2=0x7f57c811ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:36.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.845+0000 7f57ced1a700 1 -- 172.21.15.2:0/567594536 shutdown_connections 2024-05-14T17:02:36.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:32.845+0000 7f57ced1a700 1 -- 172.21.15.2:0/567594536 wait complete. 2024-05-14T17:02:36.543 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:02:34 smithi002 ceph-mon[31538]: pgmap v243: 97 pgs: 97 active+clean; 4.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.9 MiB/s wr, 977 op/s 2024-05-14T17:02:36.553 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '8649c12', quiesce took 13.5 seconds. Will release after: 17.8 2024-05-14T17:02:36.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:02:36 smithi002 ceph-mon[31538]: pgmap v244: 97 pgs: 97 active+clean; 4.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.4 MiB/s wr, 654 op/s 2024-05-14T17:02:36.752 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 440.0 s, 436.5 tps, lat 41.154 ms stddev 47.688 2024-05-14T17:02:36.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:02:36 smithi151 ceph-mon[36066]: pgmap v244: 97 pgs: 97 active+clean; 4.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.4 MiB/s wr, 654 op/s 2024-05-14T17:02:36.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:02:36 smithi123 ceph-mon[35971]: pgmap v244: 97 pgs: 97 active+clean; 4.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.4 MiB/s wr, 654 op/s 2024-05-14T17:02:38.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:02:38 smithi002 ceph-mon[31538]: pgmap v245: 97 pgs: 97 active+clean; 4.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.4 MiB/s wr, 654 op/s 2024-05-14T17:02:39.379 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:02:38 smithi123 ceph-mon[35971]: pgmap v245: 97 pgs: 97 active+clean; 4.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.4 MiB/s wr, 654 op/s 2024-05-14T17:02:39.429 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:02:38 smithi151 ceph-mon[36066]: pgmap v245: 97 pgs: 97 active+clean; 4.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.4 MiB/s wr, 654 op/s 2024-05-14T17:02:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:38.999+0000 7f74adc54640 1 -- 172.21.15.2:0/3712214962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74a810d020 msgr2=0x7f74a810f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:39.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:38.999+0000 7f74adc54640 1 --2- 172.21.15.2:0/3712214962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74a810d020 0x7f74a810f4c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f74a0009ca0 tx=0x7f74a002a9e0 comp rx=0 tx=0).stop 2024-05-14T17:02:39.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:38.999+0000 7f74adc54640 1 -- 172.21.15.2:0/3712214962 shutdown_connections 2024-05-14T17:02:39.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:38.999+0000 7f74adc54640 1 --2- 172.21.15.2:0/3712214962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74a810d020 0x7f74a810f4c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:38.999+0000 7f74adc54640 1 --2- 172.21.15.2:0/3712214962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74a8104600 0x7f74a810cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:39.735 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:38.999+0000 7f74adc54640 1 --2- 172.21.15.2:0/3712214962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74a8103ce0 0x7f74a81040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:38.999+0000 7f74adc54640 1 -- 172.21.15.2:0/3712214962 >> 172.21.15.2:0/3712214962 conn(0x7f74a8078ad0 msgr2=0x7f74a8078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:38.999+0000 7f74adc54640 1 -- 172.21.15.2:0/3712214962 shutdown_connections 2024-05-14T17:02:39.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:38.999+0000 7f74adc54640 1 -- 172.21.15.2:0/3712214962 wait complete. 2024-05-14T17:02:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.000+0000 7f74adc54640 1 Processor -- start 2024-05-14T17:02:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.000+0000 7f74adc54640 1 -- start start 2024-05-14T17:02:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.000+0000 7f74adc54640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74a8103ce0 0x7f74a8199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.000+0000 7f74adc54640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74a8104600 0x7f74a81996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.000+0000 7f74adc54640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74a810d020 0x7f74a819da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.000+0000 7f74adc54640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f74a8112cd0 con 0x7f74a810d020 2024-05-14T17:02:39.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.000+0000 7f74adc54640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f74a8112b50 con 0x7f74a8103ce0 2024-05-14T17:02:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.000+0000 7f74adc54640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f74a8112e50 con 0x7f74a8104600 2024-05-14T17:02:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.000+0000 7f74a7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74a810d020 0x7f74a819da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.000+0000 7f74a7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74a810d020 0x7f74a819da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34720/0 (socket says 172.21.15.2:34720) 2024-05-14T17:02:40.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.000+0000 7f74a7fff640 1 -- 172.21.15.2:0/755848032 learned_addr learned my addr 172.21.15.2:0/755848032 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:02:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.000+0000 7f74a77fe640 1 --2- 172.21.15.2:0/755848032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74a8103ce0 0x7f74a8199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.000+0000 7f74a6ffd640 1 --2- 172.21.15.2:0/755848032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74a8104600 0x7f74a81996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.001+0000 7f74a7fff640 1 -- 172.21.15.2:0/755848032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74a8104600 msgr2=0x7f74a81996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:40.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.001+0000 7f74a7fff640 1 --2- 172.21.15.2:0/755848032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74a8104600 0x7f74a81996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.001+0000 7f74a7fff640 1 -- 172.21.15.2:0/755848032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74a8103ce0 msgr2=0x7f74a8199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.001+0000 7f74a7fff640 1 --2- 172.21.15.2:0/755848032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74a8103ce0 0x7f74a8199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.001+0000 7f74a7fff640 1 -- 172.21.15.2:0/755848032 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a819e160 con 0x7f74a810d020 2024-05-14T17:02:40.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.001+0000 7f74a7fff640 1 --2- 172.21.15.2:0/755848032 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74a810d020 0x7f74a819da30 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f74a0006010 tx=0x7f74a002fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:40.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.001+0000 7f74a4ff9640 1 -- 172.21.15.2:0/755848032 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a0033040 con 0x7f74a810d020 2024-05-14T17:02:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.001+0000 7f74a4ff9640 1 -- 172.21.15.2:0/755848032 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74a002e7a0 con 0x7f74a810d020 2024-05-14T17:02:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.001+0000 7f74adc54640 1 -- 172.21.15.2:0/755848032 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74a819e3f0 con 0x7f74a810d020 2024-05-14T17:02:40.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.001+0000 7f74a4ff9640 1 -- 172.21.15.2:0/755848032 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a0043720 con 0x7f74a810d020 2024-05-14T17:02:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.001+0000 7f74adc54640 1 -- 172.21.15.2:0/755848032 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74a81a5ce0 con 0x7f74a810d020 2024-05-14T17:02:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.004+0000 7f74b096b740 1 Processor -- start 2024-05-14T17:02:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.005+0000 7f74a4ff9640 1 -- 172.21.15.2:0/755848032 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f74a0036070 con 0x7f74a810d020 2024-05-14T17:02:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.005+0000 7f74b096b740 1 -- start start 2024-05-14T17:02:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.005+0000 7f74b096b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599501f0810 0x5599501f0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.005+0000 7f74b096b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5599501f1130 0x5599501ff9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:40.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.005+0000 7f74b096b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599501ffee0 0x5599502022d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.005+0000 7f74b096b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x559950163c30 con 0x5599501f1130 2024-05-14T17:02:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.005+0000 7f74b096b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5599501537d0 con 0x5599501f0810 2024-05-14T17:02:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.005+0000 7f74b096b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559950157810 con 0x5599501ffee0 2024-05-14T17:02:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.005+0000 7f74a77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5599501f1130 0x5599501ff9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:40.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.005+0000 7f74a77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5599501f1130 0x5599501ff9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34730/0 (socket says 172.21.15.2:34730) 2024-05-14T17:02:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.005+0000 7f74a77fe640 1 -- 172.21.15.2:0/2424184041 learned_addr learned my addr 172.21.15.2:0/2424184041 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:02:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.005+0000 7f74a6ffd640 1 --2- 172.21.15.2:0/2424184041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599501ffee0 0x5599502022d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:40.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.005+0000 7f74a6ffd640 1 --2- 172.21.15.2:0/2424184041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599501f0810 0x5599501f0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.005+0000 7f74a77fe640 1 -- 172.21.15.2:0/2424184041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599501ffee0 msgr2=0x5599502022d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.005+0000 7f74a77fe640 1 --2- 172.21.15.2:0/2424184041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599501ffee0 0x5599502022d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:40.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.005+0000 7f74a77fe640 1 -- 172.21.15.2:0/2424184041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599501f0810 msgr2=0x5599501f0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.005+0000 7f74a77fe640 1 --2- 172.21.15.2:0/2424184041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599501f0810 0x5599501f0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:40.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.005+0000 7f74a77fe640 1 -- 172.21.15.2:0/2424184041 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55994fe68210 con 0x5599501f1130 2024-05-14T17:02:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.006+0000 7f74a6ffd640 1 --2- 172.21.15.2:0/2424184041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599501ffee0 0x5599502022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:02:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.006+0000 7f74a4ff9640 1 --2- 172.21.15.2:0/755848032 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f74900b2a90 0x7f74900b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.006+0000 7f74a77fe640 1 --2- 172.21.15.2:0/2424184041 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5599501f1130 0x5599501ff9a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f7498009cf0 tx=0x7f7498011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b12ac05616dee1b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.006+0000 7f74a6ffd640 1 --2- 172.21.15.2:0/755848032 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f74900b2a90 0x7f74900b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:40.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.006+0000 7f74a4ff9640 1 -- 172.21.15.2:0/755848032 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f74a01032a0 con 0x7f74a810d020 2024-05-14T17:02:40.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.006+0000 7f748dffb640 1 -- 172.21.15.2:0/2424184041 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7498004230 con 0x5599501f1130 2024-05-14T17:02:40.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.006+0000 7f748dffb640 1 -- 172.21.15.2:0/2424184041 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74980043d0 con 0x5599501f1130 2024-05-14T17:02:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.006+0000 7f748dffb640 1 -- 172.21.15.2:0/2424184041 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7498019880 con 0x5599501f1130 2024-05-14T17:02:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.007+0000 7f74b096b740 1 -- 172.21.15.2:0/2424184041 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5599501f1130 msgr2=0x5599501ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.007+0000 7f74a6ffd640 1 --2- 172.21.15.2:0/755848032 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f74900b2a90 0x7f74900b4f50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x5599501edf90 tx=0x7f749c00b390 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.007+0000 7f74b096b740 1 --2- 172.21.15.2:0/2424184041 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5599501f1130 0x5599501ff9a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f7498009cf0 tx=0x7f7498011810 comp rx=0 tx=0).stop 2024-05-14T17:02:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.007+0000 7f74b096b740 1 -- 172.21.15.2:0/2424184041 shutdown_connections 2024-05-14T17:02:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.007+0000 7f74b096b740 1 --2- 172.21.15.2:0/2424184041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599501ffee0 0x5599502022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.007+0000 7f74b096b740 1 --2- 172.21.15.2:0/2424184041 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5599501f1130 0x5599501ff9a0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:40.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.007+0000 7f74b096b740 1 --2- 172.21.15.2:0/2424184041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599501f0810 0x5599501f0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:40.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.007+0000 7f74b096b740 1 -- 172.21.15.2:0/2424184041 >> 172.21.15.2:0/2424184041 conn(0x5599501ece00 msgr2=0x5599501ed1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:40.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.007+0000 7f74b096b740 1 -- 172.21.15.2:0/2424184041 shutdown_connections 2024-05-14T17:02:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.007+0000 7f74b096b740 1 -- 172.21.15.2:0/2424184041 wait complete. 2024-05-14T17:02:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.007+0000 7f74b096b740 1 -- start start 2024-05-14T17:02:40.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.007+0000 7f74b096b740 1 Processor -- start 2024-05-14T17:02:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.008+0000 7f74b096b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599501f0810 0x5599501ff840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.008+0000 7f74b096b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5599501f1130 0x5599501fcb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.008+0000 7f74b096b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599501ffee0 0x5599501fd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.008+0000 7f74b096b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55994fe39790 con 0x5599501f1130 2024-05-14T17:02:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.008+0000 7f74b096b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55995010f890 con 0x5599501ffee0 2024-05-14T17:02:40.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.008+0000 7f74b096b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559950202bb0 con 0x5599501f0810 2024-05-14T17:02:40.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.008+0000 7f74b096b740 10 client.0 _ll_register_callbacks cb 0x559950112d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:02:40.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.008+0000 7f74a6ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5599501f1130 0x5599501fcb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:40.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.008+0000 7f74a6ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5599501f1130 0x5599501fcb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34742/0 (socket says 172.21.15.2:34742) 2024-05-14T17:02:40.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.008+0000 7f74a6ffd640 1 -- 172.21.15.2:0/199816209 learned_addr learned my addr 172.21.15.2:0/199816209 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:02:40.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.008+0000 7f74a77fe640 1 --2- 172.21.15.2:0/199816209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599501f0810 0x5599501ff840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:40.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.008+0000 7f74a7fff640 1 --2- 172.21.15.2:0/199816209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599501ffee0 0x5599501fd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.008+0000 7f74a6ffd640 1 -- 172.21.15.2:0/199816209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599501f0810 msgr2=0x5599501ff840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:40.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.008+0000 7f74a6ffd640 1 --2- 172.21.15.2:0/199816209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599501f0810 0x5599501ff840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.008+0000 7f74a6ffd640 1 -- 172.21.15.2:0/199816209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599501ffee0 msgr2=0x5599501fd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:41.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.008+0000 7f74a6ffd640 1 --2- 172.21.15.2:0/199816209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599501ffee0 0x5599501fd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:41.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.008+0000 7f74a6ffd640 1 -- 172.21.15.2:0/199816209 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55994ff34620 con 0x5599501f1130 2024-05-14T17:02:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.008+0000 7f74a77fe640 1 --2- 172.21.15.2:0/199816209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599501f0810 0x5599501ff840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:02:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.009+0000 7f74a6ffd640 1 --2- 172.21.15.2:0/199816209 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5599501f1130 0x5599501fcb30 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f749c003eb0 tx=0x7f749c00b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f887290b0121562 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.009+0000 7f7486ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:02:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.009+0000 7f7486ffd640 1 -- 172.21.15.2:0/199816209 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749c012e00 con 0x5599501f1130 2024-05-14T17:02:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.009+0000 7f74b096b740 1 -- 172.21.15.2:0/199816209 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74980043d0 con 0x5599501f1130 2024-05-14T17:02:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.009+0000 7f7486ffd640 1 -- 172.21.15.2:0/199816209 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f749c01e600 con 0x5599501f1130 2024-05-14T17:02:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.009+0000 7f7486ffd640 1 -- 172.21.15.2:0/199816209 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f749c01e800 con 0x5599501f1130 2024-05-14T17:02:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.009+0000 7f74b096b740 10 client.16191 fetch_fsmap learned FSMap version 40 2024-05-14T17:02:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.009+0000 7f74b096b740 1 -- 172.21.15.2:0/199816209 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55995020e6f0 con 0x5599501f1130 2024-05-14T17:02:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.009+0000 7f7486ffd640 1 -- 172.21.15.2:0/199816209 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74980043d0 con 0x5599501f1130 2024-05-14T17:02:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.009+0000 7f7486ffd640 1 -- 172.21.15.2:0/199816209 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7f749c031b20 con 0x5599501f1130 2024-05-14T17:02:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.009+0000 7f74b096b740 10 client.16191 fetch_fsmap finished waiting for FSMap version 40 2024-05-14T17:02:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.009+0000 7f74b096b740 10 client.16191 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:02:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.009+0000 7f74b096b740 20 client.16191 populate_metadata read hostname 'smithi002' 2024-05-14T17:02:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.009+0000 7f74b096b740 1 --2- 172.21.15.2:0/199816209 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55995020fbb0 0x559950212020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.010+0000 7f74b096b740 4 client.16191 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:02:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.010+0000 7f74b096b740 1 -- 172.21.15.2:0/199816209 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559950165b80 con 0x55995020fbb0 2024-05-14T17:02:41.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.010+0000 7f74a77fe640 1 --2- 172.21.15.2:0/199816209 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55995020fbb0 0x559950212020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.010+0000 7f7486ffd640 10 client.16191 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:02:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.010+0000 7f74a77fe640 1 --2- 172.21.15.2:0/199816209 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55995020fbb0 0x559950212020 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a1bf508fa791e18 server_cookie=f317176b15ac0fd in_seq=0 out_seq=0 2024-05-14T17:02:41.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.011+0000 7f7486ffd640 1 -- 172.21.15.2:0/199816209 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559950165b80 con 0x55995020fbb0 2024-05-14T17:02:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.011+0000 7f7486ffd640 10 client.16191 handle_command_reply: tid=0 2024-05-14T17:02:41.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.011+0000 7f74b096b740 1 client.16191 shutdown 2024-05-14T17:02:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.011+0000 7f74b096b740 1 -- 172.21.15.2:0/199816209 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5599501f1130 msgr2=0x5599501fcb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.011+0000 7f74b096b740 1 --2- 172.21.15.2:0/199816209 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5599501f1130 0x5599501fcb30 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f749c003eb0 tx=0x7f749c00b6e0 comp rx=0 tx=0).stop 2024-05-14T17:02:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.011+0000 7f74b096b740 1 -- 172.21.15.2:0/199816209 shutdown_connections 2024-05-14T17:02:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.011+0000 7f74b096b740 1 --2- 172.21.15.2:0/199816209 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55995020fbb0 0x559950212020 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.011+0000 7f74b096b740 1 --2- 172.21.15.2:0/199816209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599501ffee0 0x5599501fd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:41.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.011+0000 7f74b096b740 1 --2- 172.21.15.2:0/199816209 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5599501f1130 0x5599501fcb30 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:41.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.011+0000 7f7486ffd640 0 client.16191 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:02:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.011+0000 7f74b096b740 1 --2- 172.21.15.2:0/199816209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599501f0810 0x5599501ff840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:41.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.011+0000 7f74b096b740 1 -- 172.21.15.2:0/199816209 >> 172.21.15.2:0/199816209 conn(0x5599501ece00 msgr2=0x5599501f2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:41.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.011+0000 7f74b096b740 1 -- 172.21.15.2:0/199816209 shutdown_connections 2024-05-14T17:02:41.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.011+0000 7f74b096b740 1 -- 172.21.15.2:0/199816209 wait complete. 2024-05-14T17:02:41.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.012+0000 7f74b096b740 20 client.16191 trim_cache size 0 max 16384 2024-05-14T17:02:41.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.027+0000 7f74b096b740 1 Processor -- start 2024-05-14T17:02:41.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.027+0000 7f74b096b740 1 -- start start 2024-05-14T17:02:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.027+0000 7f74b096b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599501f0810 0x5599501f2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.027+0000 7f74b096b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599501f1130 0x5599502178f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:41.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.027+0000 7f74b096b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5599501ffee0 0x559950217e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:41.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.027+0000 7f74b096b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f74a81aed90 con 0x5599501ffee0 2024-05-14T17:02:41.551 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.027+0000 7f74b096b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f74a81ac630 con 0x5599501f1130 2024-05-14T17:02:41.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.027+0000 7f74b096b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f74a81aa4a0 con 0x5599501f0810 2024-05-14T17:02:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.027+0000 7f74a7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5599501ffee0 0x559950217e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.027+0000 7f74a7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5599501ffee0 0x559950217e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34754/0 (socket says 172.21.15.2:34754) 2024-05-14T17:02:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.027+0000 7f74a7fff640 1 -- 172.21.15.2:0/3156853479 learned_addr learned my addr 172.21.15.2:0/3156853479 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:02:41.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.027+0000 7f74a6ffd640 1 --2- 172.21.15.2:0/3156853479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599501f1130 0x5599502178f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:41.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.028+0000 7f74a77fe640 1 --2- 172.21.15.2:0/3156853479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599501f0810 0x5599501f2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:41.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.028+0000 7f74a7fff640 1 -- 172.21.15.2:0/3156853479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599501f0810 msgr2=0x5599501f2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:41.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.028+0000 7f74a7fff640 1 --2- 172.21.15.2:0/3156853479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599501f0810 0x5599501f2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.028+0000 7f74a7fff640 1 -- 172.21.15.2:0/3156853479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599501f1130 msgr2=0x5599502178f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:41.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.028+0000 7f74a7fff640 1 --2- 172.21.15.2:0/3156853479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599501f1130 0x5599502178f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.028+0000 7f74a7fff640 1 -- 172.21.15.2:0/3156853479 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599501fd400 con 0x5599501ffee0 2024-05-14T17:02:41.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.028+0000 7f74a77fe640 1 --2- 172.21.15.2:0/3156853479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599501f0810 0x5599501f2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:02:41.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.028+0000 7f74a6ffd640 1 --2- 172.21.15.2:0/3156853479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599501f1130 0x5599502178f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:02:41.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.028+0000 7f74a7fff640 1 --2- 172.21.15.2:0/3156853479 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5599501ffee0 0x559950217e30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f74a0009dc0 tx=0x7f74a00c9da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be6ce10fc2a70f25 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.028+0000 7f748d7fa640 1 -- 172.21.15.2:0/3156853479 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a00ce930 con 0x5599501ffee0 2024-05-14T17:02:41.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.028+0000 7f748d7fa640 1 -- 172.21.15.2:0/3156853479 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74a00d33e0 con 0x5599501ffee0 2024-05-14T17:02:41.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.028+0000 7f748d7fa640 1 -- 172.21.15.2:0/3156853479 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a00c68c0 con 0x5599501ffee0 2024-05-14T17:02:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.028+0000 7f74b096b740 1 -- 172.21.15.2:0/3156853479 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5599501ffee0 msgr2=0x559950217e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:41.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.028+0000 7f74b096b740 1 --2- 172.21.15.2:0/3156853479 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5599501ffee0 0x559950217e30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f74a0009dc0 tx=0x7f74a00c9da0 comp rx=0 tx=0).stop 2024-05-14T17:02:41.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.029+0000 7f74b096b740 1 -- 172.21.15.2:0/3156853479 shutdown_connections 2024-05-14T17:02:41.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.029+0000 7f74b096b740 1 --2- 172.21.15.2:0/3156853479 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5599501ffee0 0x559950217e30 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:41.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.029+0000 7f74b096b740 1 --2- 172.21.15.2:0/3156853479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599501f1130 0x5599502178f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.029+0000 7f74b096b740 1 --2- 172.21.15.2:0/3156853479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599501f0810 0x5599501f2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:41.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.029+0000 7f74b096b740 1 -- 172.21.15.2:0/3156853479 >> 172.21.15.2:0/3156853479 conn(0x5599501ece00 msgr2=0x5599501f2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.029+0000 7f74b096b740 1 -- 172.21.15.2:0/3156853479 shutdown_connections 2024-05-14T17:02:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.029+0000 7f74b096b740 1 -- 172.21.15.2:0/3156853479 wait complete. 2024-05-14T17:02:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.029+0000 7f74b096b740 1 -- start start 2024-05-14T17:02:41.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.029+0000 7f74b096b740 1 Processor -- start 2024-05-14T17:02:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.029+0000 7f74b096b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599501f0810 0x559950203a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.029+0000 7f74b096b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5599501f1130 0x559950203f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.029+0000 7f74b096b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599501ffee0 0x559950204490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.029+0000 7f74b096b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f74a81aaaf0 con 0x5599501f1130 2024-05-14T17:02:41.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.029+0000 7f74b096b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f74a81abf60 con 0x5599501f0810 2024-05-14T17:02:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.029+0000 7f74b096b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f74a81ab1c0 con 0x5599501ffee0 2024-05-14T17:02:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.029+0000 7f74b096b740 10 client.0 _ll_register_callbacks cb 0x7f749c004420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:02:42.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.030+0000 7f74a6ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5599501f1130 0x559950203f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.030+0000 7f74a6ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5599501f1130 0x559950203f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34762/0 (socket says 172.21.15.2:34762) 2024-05-14T17:02:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.030+0000 7f74a6ffd640 1 -- 172.21.15.2:0/2361985648 learned_addr learned my addr 172.21.15.2:0/2361985648 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:02:42.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.030+0000 7f74a77fe640 1 --2- 172.21.15.2:0/2361985648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599501f0810 0x559950203a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.030+0000 7f74a7fff640 1 --2- 172.21.15.2:0/2361985648 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599501ffee0 0x559950204490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.030+0000 7f74a6ffd640 1 -- 172.21.15.2:0/2361985648 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599501ffee0 msgr2=0x559950204490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:42.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.030+0000 7f74a6ffd640 1 --2- 172.21.15.2:0/2361985648 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599501ffee0 0x559950204490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.030+0000 7f74a6ffd640 1 -- 172.21.15.2:0/2361985648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599501f0810 msgr2=0x559950203a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.030+0000 7f74a6ffd640 1 --2- 172.21.15.2:0/2361985648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599501f0810 0x559950203a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.030+0000 7f74a6ffd640 1 -- 172.21.15.2:0/2361985648 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599502794d0 con 0x5599501f1130 2024-05-14T17:02:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.030+0000 7f74a6ffd640 1 --2- 172.21.15.2:0/2361985648 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5599501f1130 0x559950203f50 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f749c028560 tx=0x7f749c012f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=257d2bae6389540c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.030+0000 7f7486ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:02:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.030+0000 7f7486ffd640 1 -- 172.21.15.2:0/2361985648 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749c03f060 con 0x5599501f1130 2024-05-14T17:02:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.030+0000 7f7486ffd640 1 -- 172.21.15.2:0/2361985648 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f749c037070 con 0x5599501f1130 2024-05-14T17:02:42.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.030+0000 7f74b096b740 1 -- 172.21.15.2:0/2361985648 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74a00d33e0 con 0x5599501f1130 2024-05-14T17:02:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.030+0000 7f7486ffd640 1 -- 172.21.15.2:0/2361985648 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f749c01e9a0 con 0x5599501f1130 2024-05-14T17:02:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.030+0000 7f74b096b740 10 client.16197 fetch_fsmap learned FSMap version 40 2024-05-14T17:02:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.030+0000 7f74b096b740 1 -- 172.21.15.2:0/2361985648 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55995027a730 con 0x5599501f1130 2024-05-14T17:02:42.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.031+0000 7f7486ffd640 1 -- 172.21.15.2:0/2361985648 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74a00d33e0 con 0x5599501f1130 2024-05-14T17:02:42.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.031+0000 7f7486ffd640 1 -- 172.21.15.2:0/2361985648 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 40) ==== 8350+0+0 (secure 0 0 0) 0x7f749c031480 con 0x5599501f1130 2024-05-14T17:02:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.031+0000 7f74b096b740 10 client.16197 fetch_fsmap finished waiting for FSMap version 40 2024-05-14T17:02:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.031+0000 7f74b096b740 10 client.16197 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:02:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.031+0000 7f74b096b740 20 client.16197 populate_metadata read hostname 'smithi002' 2024-05-14T17:02:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.031+0000 7f74b096b740 1 --2- 172.21.15.2:0/2361985648 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55995027af40 0x55995027d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.031+0000 7f74b096b740 4 client.16197 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:02:42.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.031+0000 7f74b096b740 1 -- 172.21.15.2:0/2361985648 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55995015f360 con 0x55995027af40 2024-05-14T17:02:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.031+0000 7f74a77fe640 1 --2- 172.21.15.2:0/2361985648 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55995027af40 0x55995027d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.031+0000 7f7486ffd640 10 client.16197 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:02:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.031+0000 7f74a77fe640 1 --2- 172.21.15.2:0/2361985648 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55995027af40 0x55995027d360 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6de29bd9b9431254 server_cookie=20b1042b884e1be8 in_seq=0 out_seq=0 2024-05-14T17:02:42.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.042+0000 7f7486ffd640 1 -- 172.21.15.2:0/2361985648 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55995015f360 con 0x55995027af40 2024-05-14T17:02:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.042+0000 7f7486ffd640 10 client.16197 handle_command_reply: tid=0 2024-05-14T17:02:42.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.042+0000 7f74b096b740 1 client.16197 shutdown 2024-05-14T17:02:42.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.042+0000 7f74b096b740 1 -- 172.21.15.2:0/2361985648 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5599501f1130 msgr2=0x559950203f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:42.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.042+0000 7f74b096b740 1 --2- 172.21.15.2:0/2361985648 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5599501f1130 0x559950203f50 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f749c028560 tx=0x7f749c012f00 comp rx=0 tx=0).stop 2024-05-14T17:02:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.043+0000 7f74b096b740 1 -- 172.21.15.2:0/2361985648 shutdown_connections 2024-05-14T17:02:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.043+0000 7f74b096b740 1 --2- 172.21.15.2:0/2361985648 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55995027af40 0x55995027d360 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:42.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.043+0000 7f74b096b740 1 --2- 172.21.15.2:0/2361985648 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599501ffee0 0x559950204490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.043+0000 7f74b096b740 1 --2- 172.21.15.2:0/2361985648 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5599501f1130 0x559950203f50 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.043+0000 7f7486ffd640 0 client.16197 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:02:42.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.043+0000 7f74b096b740 1 --2- 172.21.15.2:0/2361985648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599501f0810 0x559950203a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.043+0000 7f74b096b740 1 -- 172.21.15.2:0/2361985648 >> 172.21.15.2:0/2361985648 conn(0x5599501ece00 msgr2=0x5599501f2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.043+0000 7f74b096b740 1 -- 172.21.15.2:0/2361985648 shutdown_connections 2024-05-14T17:02:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.043+0000 7f74b096b740 1 -- 172.21.15.2:0/2361985648 wait complete. 2024-05-14T17:02:42.618 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.043+0000 7f74b096b740 20 client.16197 trim_cache size 0 max 16384 2024-05-14T17:02:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:02:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:02:42.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:02:42.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.043+0000 7f74adc54640 1 -- 172.21.15.2:0/755848032 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f74900b2a90 msgr2=0x7f74900b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:42.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.043+0000 7f74adc54640 1 --2- 172.21.15.2:0/755848032 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f74900b2a90 0x7f74900b4f50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x5599501edf90 tx=0x7f749c00b390 comp rx=0 tx=0).stop 2024-05-14T17:02:42.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.043+0000 7f74adc54640 1 -- 172.21.15.2:0/755848032 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74a810d020 msgr2=0x7f74a819da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.043+0000 7f74adc54640 1 --2- 172.21.15.2:0/755848032 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74a810d020 0x7f74a819da30 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f74a0006010 tx=0x7f74a002fbe0 comp rx=0 tx=0).stop 2024-05-14T17:02:42.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.043+0000 7f74adc54640 1 -- 172.21.15.2:0/755848032 shutdown_connections 2024-05-14T17:02:42.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.043+0000 7f74adc54640 1 --2- 172.21.15.2:0/755848032 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f74900b2a90 0x7f74900b4f50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.043+0000 7f74adc54640 1 --2- 172.21.15.2:0/755848032 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74a810d020 0x7f74a819da30 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.043+0000 7f74adc54640 1 --2- 172.21.15.2:0/755848032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74a8104600 0x7f74a81996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.043+0000 7f74adc54640 1 --2- 172.21.15.2:0/755848032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74a8103ce0 0x7f74a8199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:42.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.043+0000 7f74adc54640 1 -- 172.21.15.2:0/755848032 >> 172.21.15.2:0/755848032 conn(0x7f74a8078ad0 msgr2=0x7f74a80fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.044+0000 7f74adc54640 1 -- 172.21.15.2:0/755848032 shutdown_connections 2024-05-14T17:02:42.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:39.044+0000 7f74adc54640 1 -- 172.21.15.2:0/755848032 wait complete. 2024-05-14T17:02:42.810 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:02:40 smithi002 ceph-mon[31538]: pgmap v246: 97 pgs: 97 active+clean; 4.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.6 MiB/s wr, 665 op/s 2024-05-14T17:02:42.837 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 445.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:02:42.847 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:02:42 smithi002 ceph-mon[31538]: pgmap v247: 97 pgs: 97 active+clean; 4.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 3.9 MiB/s wr, 383 op/s 2024-05-14T17:02:42.855 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:02:40 smithi123 ceph-mon[35971]: pgmap v246: 97 pgs: 97 active+clean; 4.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.6 MiB/s wr, 665 op/s 2024-05-14T17:02:42.861 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:02:40 smithi151 ceph-mon[36066]: pgmap v246: 97 pgs: 97 active+clean; 4.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.6 MiB/s wr, 665 op/s 2024-05-14T17:02:42.868 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:02:42 smithi151 ceph-mon[36066]: pgmap v247: 97 pgs: 97 active+clean; 4.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 3.9 MiB/s wr, 383 op/s 2024-05-14T17:02:42.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:02:42 smithi123 ceph-mon[35971]: pgmap v247: 97 pgs: 97 active+clean; 4.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 3.9 MiB/s wr, 383 op/s 2024-05-14T17:02:44.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:02:44 smithi002 ceph-mon[31538]: pgmap v248: 97 pgs: 97 active+clean; 4.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 22 MiB/s wr, 1.29k op/s 2024-05-14T17:02:44.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:02:44 smithi151 ceph-mon[36066]: pgmap v248: 97 pgs: 97 active+clean; 4.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 22 MiB/s wr, 1.29k op/s 2024-05-14T17:02:44.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:02:44 smithi123 ceph-mon[35971]: pgmap v248: 97 pgs: 97 active+clean; 4.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 22 MiB/s wr, 1.29k op/s 2024-05-14T17:02:46.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:02:46 smithi002 ceph-mon[31538]: pgmap v249: 97 pgs: 97 active+clean; 4.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 18 MiB/s wr, 921 op/s 2024-05-14T17:02:46.779 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 450.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:02:46.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:02:46 smithi151 ceph-mon[36066]: pgmap v249: 97 pgs: 97 active+clean; 4.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 18 MiB/s wr, 921 op/s 2024-05-14T17:02:46.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:02:46 smithi123 ceph-mon[35971]: pgmap v249: 97 pgs: 97 active+clean; 4.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 18 MiB/s wr, 921 op/s 2024-05-14T17:02:48.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:02:48 smithi002 ceph-mon[31538]: pgmap v250: 97 pgs: 97 active+clean; 4.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 18 MiB/s wr, 921 op/s 2024-05-14T17:02:48.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:02:48 smithi151 ceph-mon[36066]: pgmap v250: 97 pgs: 97 active+clean; 4.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 18 MiB/s wr, 921 op/s 2024-05-14T17:02:48.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:02:48 smithi123 ceph-mon[35971]: pgmap v250: 97 pgs: 97 active+clean; 4.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 18 MiB/s wr, 921 op/s 2024-05-14T17:02:50.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:02:50 smithi002 ceph-mon[31538]: pgmap v251: 97 pgs: 97 active+clean; 4.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 21 MiB/s wr, 1.09k op/s 2024-05-14T17:02:50.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:02:50 smithi151 ceph-mon[36066]: pgmap v251: 97 pgs: 97 active+clean; 4.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 21 MiB/s wr, 1.09k op/s 2024-05-14T17:02:50.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:02:50 smithi123 ceph-mon[35971]: pgmap v251: 97 pgs: 97 active+clean; 4.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 21 MiB/s wr, 1.09k op/s 2024-05-14T17:02:51.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 455.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:02:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.145+0000 7fb485bda640 1 -- 172.21.15.2:0/3535907323 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb480107940 msgr2=0x7fb480107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:52.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.145+0000 7fb485bda640 1 --2- 172.21.15.2:0/3535907323 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb480107940 0x7fb480107d20 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb470009930 tx=0x7fb47002a420 comp rx=0 tx=0).stop 2024-05-14T17:02:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.145+0000 7fb485bda640 1 -- 172.21.15.2:0/3535907323 shutdown_connections 2024-05-14T17:02:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.145+0000 7fb485bda640 1 --2- 172.21.15.2:0/3535907323 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb480102340 0x7fb48010e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:52.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.145+0000 7fb485bda640 1 --2- 172.21.15.2:0/3535907323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb480101980 0x7fb480101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.145+0000 7fb485bda640 1 --2- 172.21.15.2:0/3535907323 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb480107940 0x7fb480107d20 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.145+0000 7fb485bda640 1 -- 172.21.15.2:0/3535907323 >> 172.21.15.2:0/3535907323 conn(0x7fb480078ad0 msgr2=0x7fb480078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.145+0000 7fb485bda640 1 -- 172.21.15.2:0/3535907323 shutdown_connections 2024-05-14T17:02:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.145+0000 7fb485bda640 1 -- 172.21.15.2:0/3535907323 wait complete. 2024-05-14T17:02:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.146+0000 7fb485bda640 1 Processor -- start 2024-05-14T17:02:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.146+0000 7fb485bda640 1 -- start start 2024-05-14T17:02:52.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.146+0000 7fb485bda640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb480101980 0x7fb4800fe910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:52.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.146+0000 7fb485bda640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb480102340 0x7fb4800fee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:52.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.146+0000 7fb485bda640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb480107940 0x7fb4800ff390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:52.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.146+0000 7fb485bda640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb480111c20 con 0x7fb480107940 2024-05-14T17:02:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.146+0000 7fb485bda640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb480111aa0 con 0x7fb480101980 2024-05-14T17:02:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.146+0000 7fb485bda640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb480111da0 con 0x7fb480102340 2024-05-14T17:02:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.147+0000 7fb47ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb480107940 0x7fb4800ff390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:52.416 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.147+0000 7fb47ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb480107940 0x7fb4800ff390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41732/0 (socket says 172.21.15.2:41732) 2024-05-14T17:02:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.147+0000 7fb47ffff640 1 -- 172.21.15.2:0/2057355524 learned_addr learned my addr 172.21.15.2:0/2057355524 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:02:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.147+0000 7fb47f7fe640 1 --2- 172.21.15.2:0/2057355524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb480101980 0x7fb4800fe910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.147+0000 7fb47effd640 1 --2- 172.21.15.2:0/2057355524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb480102340 0x7fb4800fee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:52.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.147+0000 7fb47ffff640 1 -- 172.21.15.2:0/2057355524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb480102340 msgr2=0x7fb4800fee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.147+0000 7fb47ffff640 1 --2- 172.21.15.2:0/2057355524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb480102340 0x7fb4800fee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:52.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.147+0000 7fb47ffff640 1 -- 172.21.15.2:0/2057355524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb480101980 msgr2=0x7fb4800fe910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:52.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.147+0000 7fb47ffff640 1 --2- 172.21.15.2:0/2057355524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb480101980 0x7fb4800fe910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.147+0000 7fb47ffff640 1 -- 172.21.15.2:0/2057355524 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4801a76f0 con 0x7fb480107940 2024-05-14T17:02:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.147+0000 7fb47ffff640 1 --2- 172.21.15.2:0/2057355524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb480107940 0x7fb4800ff390 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fb47800eaa0 tx=0x7fb47800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.147+0000 7fb47cff9640 1 -- 172.21.15.2:0/2057355524 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47800cdb0 con 0x7fb480107940 2024-05-14T17:02:52.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.147+0000 7fb47cff9640 1 -- 172.21.15.2:0/2057355524 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb478005020 con 0x7fb480107940 2024-05-14T17:02:52.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.147+0000 7fb485bda640 1 -- 172.21.15.2:0/2057355524 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4801a79e0 con 0x7fb480107940 2024-05-14T17:02:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.147+0000 7fb47cff9640 1 -- 172.21.15.2:0/2057355524 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4780116d0 con 0x7fb480107940 2024-05-14T17:02:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.147+0000 7fb485bda640 1 -- 172.21.15.2:0/2057355524 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4801a7ee0 con 0x7fb480107940 2024-05-14T17:02:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.151+0000 7fb4889c7740 1 Processor -- start 2024-05-14T17:02:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.151+0000 7fb47cff9640 1 -- 172.21.15.2:0/2057355524 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fb4780051d0 con 0x7fb480107940 2024-05-14T17:02:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.151+0000 7fb4889c7740 1 -- start start 2024-05-14T17:02:52.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.151+0000 7fb4889c7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f90cb8f0 0x55d7f90cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.151+0000 7fb4889c7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f90cc210 0x55d7f90d6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.151+0000 7fb4889c7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f90d6ad0 0x55d7f90d8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.151+0000 7fb4889c7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d7f903ec30 con 0x55d7f90cb8f0 2024-05-14T17:02:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.151+0000 7fb4889c7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d7f902e7d0 con 0x55d7f90d6ad0 2024-05-14T17:02:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.151+0000 7fb4889c7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d7f9032810 con 0x55d7f90cc210 2024-05-14T17:02:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.151+0000 7fb47effd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f90cb8f0 0x55d7f90cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.151+0000 7fb47effd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f90cb8f0 0x55d7f90cbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41746/0 (socket says 172.21.15.2:41746) 2024-05-14T17:02:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.151+0000 7fb47effd640 1 -- 172.21.15.2:0/3309726933 learned_addr learned my addr 172.21.15.2:0/3309726933 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:02:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.151+0000 7fb47effd640 1 --2- 172.21.15.2:0/3309726933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f90d6ad0 0x55d7f90d8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:52.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.151+0000 7fb47f7fe640 1 --2- 172.21.15.2:0/3309726933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f90cc210 0x55d7f90d6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.152+0000 7fb47effd640 1 -- 172.21.15.2:0/3309726933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f90cc210 msgr2=0x55d7f90d6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.152+0000 7fb47effd640 1 --2- 172.21.15.2:0/3309726933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f90cc210 0x55d7f90d6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:52.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.152+0000 7fb47effd640 1 -- 172.21.15.2:0/3309726933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f90d6ad0 msgr2=0x55d7f90d8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.152+0000 7fb47effd640 1 --2- 172.21.15.2:0/3309726933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f90d6ad0 0x55d7f90d8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:52.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.152+0000 7fb47effd640 1 -- 172.21.15.2:0/3309726933 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7f8d43210 con 0x55d7f90cb8f0 2024-05-14T17:02:52.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.152+0000 7fb47cff9640 1 --2- 172.21.15.2:0/2057355524 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb4680b2a40 0x7fb4680b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.152+0000 7fb47effd640 1 --2- 172.21.15.2:0/3309726933 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f90cb8f0 0x55d7f90cbcd0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb4740058a0 tx=0x7fb47400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2482c881fa692454 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:52.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.152+0000 7fb47f7fe640 1 --2- 172.21.15.2:0/2057355524 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb4680b2a40 0x7fb4680b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.152+0000 7fb47cff9640 1 -- 172.21.15.2:0/2057355524 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7fb4780e26d0 con 0x7fb480107940 2024-05-14T17:02:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.152+0000 7fb465ffb640 1 -- 172.21.15.2:0/3309726933 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb474004030 con 0x55d7f90cb8f0 2024-05-14T17:02:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.152+0000 7fb465ffb640 1 -- 172.21.15.2:0/3309726933 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4740035f0 con 0x55d7f90cb8f0 2024-05-14T17:02:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.152+0000 7fb465ffb640 1 -- 172.21.15.2:0/3309726933 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb474010660 con 0x55d7f90cb8f0 2024-05-14T17:02:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.153+0000 7fb4889c7740 1 -- 172.21.15.2:0/3309726933 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f90cb8f0 msgr2=0x55d7f90cbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:52.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.153+0000 7fb47f7fe640 1 --2- 172.21.15.2:0/2057355524 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb4680b2a40 0x7fb4680b4f00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb470031440 tx=0x7fb470005a20 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.153+0000 7fb4889c7740 1 --2- 172.21.15.2:0/3309726933 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f90cb8f0 0x55d7f90cbcd0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb4740058a0 tx=0x7fb47400b840 comp rx=0 tx=0).stop 2024-05-14T17:02:52.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.153+0000 7fb4889c7740 1 -- 172.21.15.2:0/3309726933 shutdown_connections 2024-05-14T17:02:52.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.153+0000 7fb4889c7740 1 --2- 172.21.15.2:0/3309726933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f90d6ad0 0x55d7f90d8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:52.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.153+0000 7fb4889c7740 1 --2- 172.21.15.2:0/3309726933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f90cc210 0x55d7f90d6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.153+0000 7fb4889c7740 1 --2- 172.21.15.2:0/3309726933 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f90cb8f0 0x55d7f90cbcd0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:53.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.153+0000 7fb4889c7740 1 -- 172.21.15.2:0/3309726933 >> 172.21.15.2:0/3309726933 conn(0x55d7f90c7e00 msgr2=0x55d7f90c81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:53.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.153+0000 7fb4889c7740 1 -- 172.21.15.2:0/3309726933 shutdown_connections 2024-05-14T17:02:53.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.153+0000 7fb4889c7740 1 -- 172.21.15.2:0/3309726933 wait complete. 2024-05-14T17:02:53.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.153+0000 7fb4889c7740 1 -- start start 2024-05-14T17:02:53.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.154+0000 7fb4889c7740 1 Processor -- start 2024-05-14T17:02:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.154+0000 7fb4889c7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f90cb8f0 0x55d7f90d6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.154+0000 7fb4889c7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f90cc210 0x55d7f90d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:53.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.154+0000 7fb4889c7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f90d6ad0 0x55d7f90d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.154+0000 7fb4889c7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d7f8d14790 con 0x55d7f90cc210 2024-05-14T17:02:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.154+0000 7fb4889c7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d7f8fea890 con 0x55d7f90cb8f0 2024-05-14T17:02:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.154+0000 7fb4889c7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d7f90d97a0 con 0x55d7f90d6ad0 2024-05-14T17:02:53.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.154+0000 7fb4889c7740 10 client.0 _ll_register_callbacks cb 0x55d7f8fedd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:02:53.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.154+0000 7fb47effd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f90cc210 0x55d7f90d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:53.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.154+0000 7fb47effd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f90cc210 0x55d7f90d36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41754/0 (socket says 172.21.15.2:41754) 2024-05-14T17:02:53.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.154+0000 7fb47effd640 1 -- 172.21.15.2:0/687684802 learned_addr learned my addr 172.21.15.2:0/687684802 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:02:53.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.154+0000 7fb47ffff640 1 --2- 172.21.15.2:0/687684802 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f90d6ad0 0x55d7f90d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:53.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.154+0000 7fb47f7fe640 1 --2- 172.21.15.2:0/687684802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f90cb8f0 0x55d7f90d6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.154+0000 7fb47effd640 1 -- 172.21.15.2:0/687684802 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f90d6ad0 msgr2=0x55d7f90d3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:53.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.154+0000 7fb47effd640 1 --2- 172.21.15.2:0/687684802 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f90d6ad0 0x55d7f90d3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.154+0000 7fb47effd640 1 -- 172.21.15.2:0/687684802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f90cb8f0 msgr2=0x55d7f90d6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.154+0000 7fb47effd640 1 --2- 172.21.15.2:0/687684802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f90cb8f0 0x55d7f90d6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.154+0000 7fb47effd640 1 -- 172.21.15.2:0/687684802 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7f8e0f620 con 0x55d7f90cc210 2024-05-14T17:02:53.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.154+0000 7fb47f7fe640 1 --2- 172.21.15.2:0/687684802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f90cb8f0 0x55d7f90d6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:02:53.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.155+0000 7fb47effd640 1 --2- 172.21.15.2:0/687684802 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f90cc210 0x55d7f90d36f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fb47400bfe0 tx=0x7fb474012300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33fcc4bc5fbe0c9d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:53.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.155+0000 7fb45effd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:02:53.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.155+0000 7fb45effd640 1 -- 172.21.15.2:0/687684802 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4740039b0 con 0x55d7f90cc210 2024-05-14T17:02:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.155+0000 7fb45effd640 1 -- 172.21.15.2:0/687684802 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7fb474003fe0 con 0x55d7f90cc210 2024-05-14T17:02:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.155+0000 7fb4889c7740 1 -- 172.21.15.2:0/687684802 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4740035f0 con 0x55d7f90cc210 2024-05-14T17:02:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.155+0000 7fb45effd640 1 -- 172.21.15.2:0/687684802 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb474003b50 con 0x55d7f90cc210 2024-05-14T17:02:53.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.155+0000 7fb4889c7740 10 client.16209 fetch_fsmap learned FSMap version 41 2024-05-14T17:02:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.155+0000 7fb4889c7740 1 -- 172.21.15.2:0/687684802 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55d7f90e51e0 con 0x55d7f90cc210 2024-05-14T17:02:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.155+0000 7fb45effd640 1 -- 172.21.15.2:0/687684802 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4740035f0 con 0x55d7f90cc210 2024-05-14T17:02:53.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.155+0000 7fb45effd640 1 -- 172.21.15.2:0/687684802 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fb47400e490 con 0x55d7f90cc210 2024-05-14T17:02:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.155+0000 7fb4889c7740 10 client.16209 fetch_fsmap finished waiting for FSMap version 41 2024-05-14T17:02:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.155+0000 7fb4889c7740 10 client.16209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:02:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.155+0000 7fb4889c7740 20 client.16209 populate_metadata read hostname 'smithi002' 2024-05-14T17:02:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.155+0000 7fb4889c7740 1 --2- 172.21.15.2:0/687684802 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d7f90e65e0 0x55d7f90e89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.155+0000 7fb4889c7740 4 client.16209 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:02:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.155+0000 7fb4889c7740 1 -- 172.21.15.2:0/687684802 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7f9040b80 con 0x55d7f90e65e0 2024-05-14T17:02:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.156+0000 7fb47f7fe640 1 --2- 172.21.15.2:0/687684802 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d7f90e65e0 0x55d7f90e89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.156+0000 7fb47f7fe640 1 --2- 172.21.15.2:0/687684802 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d7f90e65e0 0x55d7f90e89d0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=338c6794bf97e83f server_cookie=e95d9c0bdbed00fb in_seq=0 out_seq=0 2024-05-14T17:02:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.156+0000 7fb45effd640 10 client.16209 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:02:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.157+0000 7fb45effd640 1 -- 172.21.15.2:0/687684802 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d7f9040b80 con 0x55d7f90e65e0 2024-05-14T17:02:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.157+0000 7fb45effd640 10 client.16209 handle_command_reply: tid=0 2024-05-14T17:02:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.157+0000 7fb4889c7740 1 client.16209 shutdown 2024-05-14T17:02:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.157+0000 7fb4889c7740 1 -- 172.21.15.2:0/687684802 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f90cc210 msgr2=0x55d7f90d36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:53.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.157+0000 7fb4889c7740 1 --2- 172.21.15.2:0/687684802 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f90cc210 0x55d7f90d36f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fb47400bfe0 tx=0x7fb474012300 comp rx=0 tx=0).stop 2024-05-14T17:02:53.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.157+0000 7fb4889c7740 1 -- 172.21.15.2:0/687684802 shutdown_connections 2024-05-14T17:02:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.157+0000 7fb4889c7740 1 --2- 172.21.15.2:0/687684802 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d7f90e65e0 0x55d7f90e89d0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.157+0000 7fb4889c7740 1 --2- 172.21.15.2:0/687684802 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f90d6ad0 0x55d7f90d3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.157+0000 7fb4889c7740 1 --2- 172.21.15.2:0/687684802 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f90cc210 0x55d7f90d36f0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.157+0000 7fb45effd640 0 client.16209 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:02:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.157+0000 7fb4889c7740 1 --2- 172.21.15.2:0/687684802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f90cb8f0 0x55d7f90d6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.157+0000 7fb4889c7740 1 -- 172.21.15.2:0/687684802 >> 172.21.15.2:0/687684802 conn(0x55d7f90c7e00 msgr2=0x55d7f90c7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:53.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.157+0000 7fb4889c7740 1 -- 172.21.15.2:0/687684802 shutdown_connections 2024-05-14T17:02:53.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.157+0000 7fb4889c7740 1 -- 172.21.15.2:0/687684802 wait complete. 2024-05-14T17:02:53.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.157+0000 7fb4889c7740 20 client.16209 trim_cache size 0 max 16384 2024-05-14T17:02:53.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.172+0000 7fb4889c7740 1 Processor -- start 2024-05-14T17:02:53.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.173+0000 7fb4889c7740 1 -- start start 2024-05-14T17:02:53.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.173+0000 7fb4889c7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f90cb8f0 0x55d7f90e2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:53.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.173+0000 7fb4889c7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f90cc210 0x55d7f90d5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:53.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.173+0000 7fb4889c7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f90d6ad0 0x55d7f90ee160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.173+0000 7fb4889c7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb4801b3120 con 0x55d7f90cb8f0 2024-05-14T17:02:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.173+0000 7fb4889c7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb4801b09f0 con 0x55d7f90cc210 2024-05-14T17:02:53.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.173+0000 7fb4889c7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb4801ae860 con 0x55d7f90d6ad0 2024-05-14T17:02:53.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.173+0000 7fb47f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f90cb8f0 0x55d7f90e2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:53.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.173+0000 7fb47f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f90cb8f0 0x55d7f90e2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41760/0 (socket says 172.21.15.2:41760) 2024-05-14T17:02:53.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.173+0000 7fb47f7fe640 1 -- 172.21.15.2:0/1204670562 learned_addr learned my addr 172.21.15.2:0/1204670562 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:02:53.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.173+0000 7fb47effd640 1 --2- 172.21.15.2:0/1204670562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f90cc210 0x55d7f90d5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.173+0000 7fb47ffff640 1 --2- 172.21.15.2:0/1204670562 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f90d6ad0 0x55d7f90ee160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.173+0000 7fb47f7fe640 1 -- 172.21.15.2:0/1204670562 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f90d6ad0 msgr2=0x55d7f90ee160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.173+0000 7fb47f7fe640 1 --2- 172.21.15.2:0/1204670562 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f90d6ad0 0x55d7f90ee160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.173+0000 7fb47f7fe640 1 -- 172.21.15.2:0/1204670562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f90cc210 msgr2=0x55d7f90d5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.173+0000 7fb47f7fe640 1 --2- 172.21.15.2:0/1204670562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f90cc210 0x55d7f90d5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.173+0000 7fb47f7fe640 1 -- 172.21.15.2:0/1204670562 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7f90cc040 con 0x55d7f90cb8f0 2024-05-14T17:02:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.173+0000 7fb47ffff640 1 --2- 172.21.15.2:0/1204670562 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f90d6ad0 0x55d7f90ee160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:02:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.173+0000 7fb47f7fe640 1 --2- 172.21.15.2:0/1204670562 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f90cb8f0 0x55d7f90e2670 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb470005ab0 tx=0x7fb47003e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b42285fdbe4ebd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.174+0000 7fb4657fa640 1 -- 172.21.15.2:0/1204670562 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb470057070 con 0x55d7f90cb8f0 2024-05-14T17:02:54.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.174+0000 7fb4657fa640 1 -- 172.21.15.2:0/1204670562 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb470004e00 con 0x55d7f90cb8f0 2024-05-14T17:02:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.174+0000 7fb4657fa640 1 -- 172.21.15.2:0/1204670562 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb470035040 con 0x55d7f90cb8f0 2024-05-14T17:02:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.174+0000 7fb4889c7740 1 -- 172.21.15.2:0/1204670562 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f90cb8f0 msgr2=0x55d7f90e2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.174+0000 7fb4889c7740 1 --2- 172.21.15.2:0/1204670562 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f90cb8f0 0x55d7f90e2670 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb470005ab0 tx=0x7fb47003e000 comp rx=0 tx=0).stop 2024-05-14T17:02:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.174+0000 7fb4889c7740 1 -- 172.21.15.2:0/1204670562 shutdown_connections 2024-05-14T17:02:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.174+0000 7fb4889c7740 1 --2- 172.21.15.2:0/1204670562 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f90d6ad0 0x55d7f90ee160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:55.397 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.174+0000 7fb4889c7740 1 --2- 172.21.15.2:0/1204670562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f90cc210 0x55d7f90d5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:55.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.174+0000 7fb4889c7740 1 --2- 172.21.15.2:0/1204670562 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f90cb8f0 0x55d7f90e2670 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.174+0000 7fb4889c7740 1 -- 172.21.15.2:0/1204670562 >> 172.21.15.2:0/1204670562 conn(0x55d7f90c7e00 msgr2=0x55d7f90ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:55.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.174+0000 7fb4889c7740 1 -- 172.21.15.2:0/1204670562 shutdown_connections 2024-05-14T17:02:55.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.174+0000 7fb4889c7740 1 -- 172.21.15.2:0/1204670562 wait complete. 2024-05-14T17:02:55.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.175+0000 7fb4889c7740 1 -- start start 2024-05-14T17:02:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.175+0000 7fb4889c7740 1 Processor -- start 2024-05-14T17:02:55.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.175+0000 7fb4889c7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f90cb8f0 0x55d7f90de930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:55.503 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.175+0000 7fb4889c7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f90cc210 0x55d7f90dee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:55.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.175+0000 7fb4889c7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f90d6ad0 0x55d7f90d9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:55.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.175+0000 7fb4889c7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb4801aeeb0 con 0x55d7f90cc210 2024-05-14T17:02:55.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.175+0000 7fb4889c7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb4801b0320 con 0x55d7f90d6ad0 2024-05-14T17:02:55.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.175+0000 7fb4889c7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb4801af580 con 0x55d7f90cb8f0 2024-05-14T17:02:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.175+0000 7fb4889c7740 10 client.0 _ll_register_callbacks cb 0x7fb474003d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:02:55.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.175+0000 7fb47effd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f90cc210 0x55d7f90dee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.175+0000 7fb47effd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f90cc210 0x55d7f90dee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41776/0 (socket says 172.21.15.2:41776) 2024-05-14T17:02:55.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.175+0000 7fb47effd640 1 -- 172.21.15.2:0/1824597671 learned_addr learned my addr 172.21.15.2:0/1824597671 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:02:55.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.175+0000 7fb47ffff640 1 --2- 172.21.15.2:0/1824597671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f90d6ad0 0x55d7f90d9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:55.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.175+0000 7fb47f7fe640 1 --2- 172.21.15.2:0/1824597671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f90cb8f0 0x55d7f90de930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.176+0000 7fb47effd640 1 -- 172.21.15.2:0/1824597671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f90cb8f0 msgr2=0x55d7f90de930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.176+0000 7fb47effd640 1 --2- 172.21.15.2:0/1824597671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f90cb8f0 0x55d7f90de930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.176+0000 7fb47effd640 1 -- 172.21.15.2:0/1824597671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f90d6ad0 msgr2=0x55d7f90d9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:02:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.176+0000 7fb47effd640 1 --2- 172.21.15.2:0/1824597671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f90d6ad0 0x55d7f90d9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.176+0000 7fb47effd640 1 -- 172.21.15.2:0/1824597671 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7f90da580 con 0x55d7f90cc210 2024-05-14T17:02:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.176+0000 7fb47f7fe640 1 --2- 172.21.15.2:0/1824597671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f90cb8f0 0x55d7f90de930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:02:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.176+0000 7fb47effd640 1 --2- 172.21.15.2:0/1824597671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f90cc210 0x55d7f90dee70 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb47400c010 tx=0x7fb474005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7761570f51f9aa server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.176+0000 7fb45effd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:02:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.176+0000 7fb45effd640 1 -- 172.21.15.2:0/1824597671 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47400ece0 con 0x55d7f90cc210 2024-05-14T17:02:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.176+0000 7fb4889c7740 1 -- 172.21.15.2:0/1824597671 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb470004e00 con 0x55d7f90cc210 2024-05-14T17:02:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.176+0000 7fb45effd640 1 -- 172.21.15.2:0/1824597671 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7fb4740050c0 con 0x55d7f90cc210 2024-05-14T17:02:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.176+0000 7fb45effd640 1 -- 172.21.15.2:0/1824597671 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb47400ee80 con 0x55d7f90cc210 2024-05-14T17:02:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.176+0000 7fb4889c7740 10 client.16215 fetch_fsmap learned FSMap version 41 2024-05-14T17:02:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.176+0000 7fb4889c7740 1 -- 172.21.15.2:0/1824597671 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55d7f9150710 con 0x55d7f90cc210 2024-05-14T17:02:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.176+0000 7fb45effd640 1 -- 172.21.15.2:0/1824597671 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb470004e00 con 0x55d7f90cc210 2024-05-14T17:02:55.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.176+0000 7fb45effd640 1 -- 172.21.15.2:0/1824597671 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fb4740108b0 con 0x55d7f90cc210 2024-05-14T17:02:55.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.177+0000 7fb4889c7740 10 client.16215 fetch_fsmap finished waiting for FSMap version 41 2024-05-14T17:02:55.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.177+0000 7fb4889c7740 10 client.16215 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:02:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.177+0000 7fb4889c7740 20 client.16215 populate_metadata read hostname 'smithi002' 2024-05-14T17:02:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.177+0000 7fb4889c7740 1 --2- 172.21.15.2:0/1824597671 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d7f9150e30 0x55d7f9153250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.177+0000 7fb4889c7740 4 client.16215 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:02:55.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.177+0000 7fb4889c7740 1 -- 172.21.15.2:0/1824597671 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d7f903a360 con 0x55d7f9150e30 2024-05-14T17:02:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.177+0000 7fb47f7fe640 1 --2- 172.21.15.2:0/1824597671 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d7f9150e30 0x55d7f9153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:55.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.177+0000 7fb47f7fe640 1 --2- 172.21.15.2:0/1824597671 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d7f9150e30 0x55d7f9153250 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f29eb6f75987e59 server_cookie=460ca0d835f654e3 in_seq=0 out_seq=0 2024-05-14T17:02:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.177+0000 7fb45effd640 10 client.16215 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:02:55.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.188+0000 7fb45effd640 1 -- 172.21.15.2:0/1824597671 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d7f903a360 con 0x55d7f9150e30 2024-05-14T17:02:55.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.188+0000 7fb45effd640 10 client.16215 handle_command_reply: tid=0 2024-05-14T17:02:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.188+0000 7fb4889c7740 1 client.16215 shutdown 2024-05-14T17:02:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.188+0000 7fb4889c7740 1 -- 172.21.15.2:0/1824597671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f90cc210 msgr2=0x55d7f90dee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:55.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.188+0000 7fb4889c7740 1 --2- 172.21.15.2:0/1824597671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f90cc210 0x55d7f90dee70 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb47400c010 tx=0x7fb474005b90 comp rx=0 tx=0).stop 2024-05-14T17:02:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.188+0000 7fb4889c7740 1 -- 172.21.15.2:0/1824597671 shutdown_connections 2024-05-14T17:02:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.188+0000 7fb4889c7740 1 --2- 172.21.15.2:0/1824597671 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d7f9150e30 0x55d7f9153250 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.188+0000 7fb4889c7740 1 --2- 172.21.15.2:0/1824597671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d7f90d6ad0 0x55d7f90d9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:55.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.188+0000 7fb4889c7740 1 --2- 172.21.15.2:0/1824597671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7f90cc210 0x55d7f90dee70 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.188+0000 7fb4889c7740 1 --2- 172.21.15.2:0/1824597671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d7f90cb8f0 0x55d7f90de930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.188+0000 7fb45effd640 0 client.16215 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:02:56.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.188+0000 7fb4889c7740 1 -- 172.21.15.2:0/1824597671 >> 172.21.15.2:0/1824597671 conn(0x55d7f90c7e00 msgr2=0x55d7f90ce3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.188+0000 7fb4889c7740 1 -- 172.21.15.2:0/1824597671 shutdown_connections 2024-05-14T17:02:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.188+0000 7fb4889c7740 1 -- 172.21.15.2:0/1824597671 wait complete. 2024-05-14T17:02:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.188+0000 7fb4889c7740 20 client.16215 trim_cache size 0 max 16384 2024-05-14T17:02:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:02:56.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:02:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:02:56.073 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:02:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.189+0000 7fb485bda640 1 -- 172.21.15.2:0/2057355524 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb4680b2a40 msgr2=0x7fb4680b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.189+0000 7fb485bda640 1 --2- 172.21.15.2:0/2057355524 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb4680b2a40 0x7fb4680b4f00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb470031440 tx=0x7fb470005a20 comp rx=0 tx=0).stop 2024-05-14T17:02:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.189+0000 7fb485bda640 1 -- 172.21.15.2:0/2057355524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb480107940 msgr2=0x7fb4800ff390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.189+0000 7fb485bda640 1 --2- 172.21.15.2:0/2057355524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb480107940 0x7fb4800ff390 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fb47800eaa0 tx=0x7fb47800ef60 comp rx=0 tx=0).stop 2024-05-14T17:02:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.189+0000 7fb485bda640 1 -- 172.21.15.2:0/2057355524 shutdown_connections 2024-05-14T17:02:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.189+0000 7fb485bda640 1 --2- 172.21.15.2:0/2057355524 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb4680b2a40 0x7fb4680b4f00 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.189+0000 7fb485bda640 1 --2- 172.21.15.2:0/2057355524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb480107940 0x7fb4800ff390 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.189+0000 7fb485bda640 1 --2- 172.21.15.2:0/2057355524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb480102340 0x7fb4800fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.189+0000 7fb485bda640 1 --2- 172.21.15.2:0/2057355524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb480101980 0x7fb4800fe910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:56.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.189+0000 7fb485bda640 1 -- 172.21.15.2:0/2057355524 >> 172.21.15.2:0/2057355524 conn(0x7fb480078ad0 msgr2=0x7fb48006a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.189+0000 7fb485bda640 1 -- 172.21.15.2:0/2057355524 shutdown_connections 2024-05-14T17:02:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:02:52.189+0000 7fb485bda640 1 -- 172.21.15.2:0/2057355524 wait complete. 2024-05-14T17:02:56.233 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:02:52 smithi002 ceph-mon[31538]: pgmap v252: 97 pgs: 97 active+clean; 4.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 MiB/s wr, 1.08k op/s 2024-05-14T17:02:56.240 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:02:52 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:02:56.247 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:02:54 smithi002 ceph-mon[31538]: pgmap v253: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 21 MiB/s wr, 1.09k op/s 2024-05-14T17:02:56.253 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:02:52 smithi123 ceph-mon[35971]: pgmap v252: 97 pgs: 97 active+clean; 4.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 MiB/s wr, 1.08k op/s 2024-05-14T17:02:56.260 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:02:52 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:02:56.266 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:02:54 smithi123 ceph-mon[35971]: pgmap v253: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 21 MiB/s wr, 1.09k op/s 2024-05-14T17:02:56.273 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:02:52 smithi151 ceph-mon[36066]: pgmap v252: 97 pgs: 97 active+clean; 4.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 MiB/s wr, 1.08k op/s 2024-05-14T17:02:56.279 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:02:52 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:02:56.295 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:02:54 smithi151 ceph-mon[36066]: pgmap v253: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 21 MiB/s wr, 1.09k op/s 2024-05-14T17:02:56.522 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:02:56.644 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:02:56 smithi002 ceph-mon[31538]: pgmap v254: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.2 MiB/s wr, 178 op/s 2024-05-14T17:02:56.751 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 460.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:02:56.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:02:56 smithi151 ceph-mon[36066]: pgmap v254: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.2 MiB/s wr, 178 op/s 2024-05-14T17:02:56.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:02:56 smithi123 ceph-mon[35971]: pgmap v254: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.2 MiB/s wr, 178 op/s 2024-05-14T17:02:57.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:02:57 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:02:57.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:02:57 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:02:57.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:02:57 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:02:58.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.192+0000 7fc66bc31700 1 -- 172.21.15.2:0/624922225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6641088f0 msgr2=0x7fc664110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:58.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.192+0000 7fc66bc31700 1 --2- 172.21.15.2:0/624922225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6641088f0 0x7fc664110470 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fc65c00b0d0 tx=0x7fc65c00b3e0 comp rx=0 tx=0).stop 2024-05-14T17:02:58.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.193+0000 7fc66bc31700 1 -- 172.21.15.2:0/624922225 shutdown_connections 2024-05-14T17:02:58.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.193+0000 7fc66bc31700 1 --2- 172.21.15.2:0/624922225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6641088f0 0x7fc664110470 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:58.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.193+0000 7fc66bc31700 1 --2- 172.21.15.2:0/624922225 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6641037f0 0x7fc664103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:58.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.193+0000 7fc66bc31700 1 --2- 172.21.15.2:0/624922225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6641041a0 0x7fc6641081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:58.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.193+0000 7fc66bc31700 1 -- 172.21.15.2:0/624922225 >> 172.21.15.2:0/624922225 conn(0x7fc66407a6a0 msgr2=0x7fc66407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:58.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.193+0000 7fc66bc31700 1 -- 172.21.15.2:0/624922225 shutdown_connections 2024-05-14T17:02:58.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.193+0000 7fc66bc31700 1 -- 172.21.15.2:0/624922225 wait complete. 2024-05-14T17:02:58.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.193+0000 7fc66bc31700 1 Processor -- start 2024-05-14T17:02:58.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.194+0000 7fc66bc31700 1 -- start start 2024-05-14T17:02:58.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.194+0000 7fc66bc31700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6641037f0 0x7fc664075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:58.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.194+0000 7fc66bc31700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6641041a0 0x7fc664075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:58.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.194+0000 7fc66bc31700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6641088f0 0x7fc6640762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:58.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.194+0000 7fc66bc31700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc6641136c0 con 0x7fc6641037f0 2024-05-14T17:02:58.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.194+0000 7fc66bc31700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc664113540 con 0x7fc6641041a0 2024-05-14T17:02:58.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.194+0000 7fc66bc31700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc664113840 con 0x7fc6641088f0 2024-05-14T17:02:58.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.194+0000 7fc6699cd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6641037f0 0x7fc664075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:58.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.194+0000 7fc6699cd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6641037f0 0x7fc664075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43174/0 (socket says 172.21.15.2:43174) 2024-05-14T17:02:58.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.194+0000 7fc6699cd700 1 -- 172.21.15.2:0/3461547848 learned_addr learned my addr 172.21.15.2:0/3461547848 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:02:58.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.194+0000 7fc6691cc700 1 --2- 172.21.15.2:0/3461547848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6641041a0 0x7fc664075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:58.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.194+0000 7fc66a1ce700 1 --2- 172.21.15.2:0/3461547848 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6641088f0 0x7fc6640762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:58.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.194+0000 7fc6699cd700 1 -- 172.21.15.2:0/3461547848 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6641088f0 msgr2=0x7fc6640762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:58.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.194+0000 7fc6699cd700 1 --2- 172.21.15.2:0/3461547848 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6641088f0 0x7fc6640762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:58.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.194+0000 7fc6699cd700 1 -- 172.21.15.2:0/3461547848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6641041a0 msgr2=0x7fc664075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:58.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.194+0000 7fc6699cd700 1 --2- 172.21.15.2:0/3461547848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6641041a0 0x7fc664075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:58.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.194+0000 7fc6699cd700 1 -- 172.21.15.2:0/3461547848 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6641a8010 con 0x7fc6641037f0 2024-05-14T17:02:58.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.195+0000 7fc6699cd700 1 --2- 172.21.15.2:0/3461547848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6641037f0 0x7fc664075840 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc66000e9a0 tx=0x7fc66000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:58.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.195+0000 7fc65affd700 1 -- 172.21.15.2:0/3461547848 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66000ca10 con 0x7fc6641037f0 2024-05-14T17:02:58.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.195+0000 7fc65affd700 1 -- 172.21.15.2:0/3461547848 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc66000cbb0 con 0x7fc6641037f0 2024-05-14T17:02:58.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.195+0000 7fc66bc31700 1 -- 172.21.15.2:0/3461547848 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6641a8300 con 0x7fc6641037f0 2024-05-14T17:02:58.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.195+0000 7fc65affd700 1 -- 172.21.15.2:0/3461547848 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc660018680 con 0x7fc6641037f0 2024-05-14T17:02:58.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.195+0000 7fc66bc31700 1 -- 172.21.15.2:0/3461547848 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6641a8890 con 0x7fc6641037f0 2024-05-14T17:02:58.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.196+0000 7fc66bc31700 1 -- 172.21.15.2:0/3461547848 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc66406a230 con 0x7fc6641037f0 2024-05-14T17:02:58.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.198+0000 7fc65affd700 1 -- 172.21.15.2:0/3461547848 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fc660010430 con 0x7fc6641037f0 2024-05-14T17:02:58.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.199+0000 7fc65affd700 1 --2- 172.21.15.2:0/3461547848 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc6500b2a70 0x7fc6500b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:02:58.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.199+0000 7fc6691cc700 1 --2- 172.21.15.2:0/3461547848 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc6500b2a70 0x7fc6500b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:02:58.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.199+0000 7fc65affd700 1 -- 172.21.15.2:0/3461547848 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7fc6600e2ab0 con 0x7fc6641037f0 2024-05-14T17:02:58.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.199+0000 7fc6691cc700 1 --2- 172.21.15.2:0/3461547848 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc6500b2a70 0x7fc6500b4f30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc6641030c0 tx=0x7fc654009260 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:02:58.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.202+0000 7fc65affd700 1 -- 172.21.15.2:0/3461547848 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc6600aa430 con 0x7fc6641037f0 2024-05-14T17:02:58.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.464+0000 7fc66bc31700 1 -- 172.21.15.2:0/3461547848 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc66402e560 con 0x7fc6641037f0 2024-05-14T17:02:58.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.465+0000 7fc65affd700 1 -- 172.21.15.2:0/3461547848 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11853 (secure 0 0 0) 0x7fc6600a9d30 con 0x7fc6641037f0 2024-05-14T17:02:58.833 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:02:58.865 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2275696923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2275696923},{"type":"v1","addr":"172.21.15.2:6835","nonce":2275696923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/716718761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":716718761},{"type":"v1","addr":"172.21.15.2:6839","nonce":716718761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/3644815179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3644815179},{"type":"v1","addr":"172.21.15.2:6841","nonce":3644815179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1880336731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1880336731},{"type":"v1","addr":"172.21.15.151:6833","nonce":1880336731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2415605548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2415605548},{"type":"v1","addr":"172.21.15.2:6837","nonce":2415605548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2635167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2635167},{"type":"v1","addr":"172.21.15.123:6833","nonce":2635167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:51:34.106919+0000","modified":"2024-05-14T17:02:50.475577+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24455,"mds_2":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24415":{"gid":24415,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/762044339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":762044339},{"type":"v1","addr":"172.21.15.151:6835","nonce":762044339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/132043151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":132043151},{"type":"v1","addr":"172.21.15.151:6837","nonce":132043151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/4107834375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":4107834375},{"type":"v1","addr":"172.21.15.151:6839","nonce":4107834375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.123:6835/1198670013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1198670013},{"type":"v1","addr":"172.21.15.123:6835","nonce":1198670013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/333426645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":333426645},{"type":"v1","addr":"172.21.15.123:6837","nonce":333426645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/580275877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":580275877},{"type":"v1","addr":"172.21.15.123:6839","nonce":580275877}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24431,24455]},"id":1}]} 2024-05-14T17:02:58.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.467+0000 7fc66bc31700 1 -- 172.21.15.2:0/3461547848 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc6500b2a70 msgr2=0x7fc6500b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:58.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.467+0000 7fc66bc31700 1 --2- 172.21.15.2:0/3461547848 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc6500b2a70 0x7fc6500b4f30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc6641030c0 tx=0x7fc654009260 comp rx=0 tx=0).stop 2024-05-14T17:02:58.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.467+0000 7fc66bc31700 1 -- 172.21.15.2:0/3461547848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6641037f0 msgr2=0x7fc664075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:02:58.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.467+0000 7fc66bc31700 1 --2- 172.21.15.2:0/3461547848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6641037f0 0x7fc664075840 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc66000e9a0 tx=0x7fc66000ecb0 comp rx=0 tx=0).stop 2024-05-14T17:02:58.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.467+0000 7fc66bc31700 1 -- 172.21.15.2:0/3461547848 shutdown_connections 2024-05-14T17:02:58.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.467+0000 7fc66bc31700 1 --2- 172.21.15.2:0/3461547848 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc6500b2a70 0x7fc6500b4f30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:58.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.467+0000 7fc66bc31700 1 --2- 172.21.15.2:0/3461547848 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6641088f0 0x7fc6640762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:58.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.467+0000 7fc66bc31700 1 --2- 172.21.15.2:0/3461547848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6641037f0 0x7fc664075840 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:58.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.467+0000 7fc66bc31700 1 --2- 172.21.15.2:0/3461547848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6641041a0 0x7fc664075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:02:58.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.467+0000 7fc66bc31700 1 -- 172.21.15.2:0/3461547848 >> 172.21.15.2:0/3461547848 conn(0x7fc66407a6a0 msgr2=0x7fc664101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:02:58.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.467+0000 7fc66bc31700 1 -- 172.21.15.2:0/3461547848 shutdown_connections 2024-05-14T17:02:58.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:02:58.467+0000 7fc66bc31700 1 -- 172.21.15.2:0/3461547848 wait complete. 2024-05-14T17:02:58.987 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 41 2024-05-14T17:02:58.996 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:02:58 smithi002 ceph-mon[31538]: pgmap v255: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.2 MiB/s wr, 178 op/s 2024-05-14T17:02:59.004 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:02:58 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:02:59.041 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:02:58 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:02:59.068 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:02:58 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:02:59.076 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:02:58 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/3461547848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:02:59.084 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:02:58 smithi123 ceph-mon[35971]: pgmap v255: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.2 MiB/s wr, 178 op/s 2024-05-14T17:02:59.096 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:02:58 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:02:59.104 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:02:58 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:02:59.112 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:02:58 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:02:59.120 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:02:58 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/3461547848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:02:59.129 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:02:58 smithi151 ceph-mon[36066]: pgmap v255: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.2 MiB/s wr, 178 op/s 2024-05-14T17:02:59.136 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:02:58 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:02:59.163 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:02:58 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:02:59.171 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:02:58 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:02:59.181 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:02:58 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/3461547848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:02:59.191 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id 8649c12 --release --await' 2024-05-14T17:02:59.198 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id 8649c12 --release --await 2024-05-14T17:02:59.458 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:03:00.581 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:03:00 smithi002 ceph-mon[31538]: pgmap v256: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 3.2 MiB/s wr, 181 op/s 2024-05-14T17:03:00.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:03:00 smithi151 ceph-mon[36066]: pgmap v256: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 3.2 MiB/s wr, 181 op/s 2024-05-14T17:03:00.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:03:00 smithi123 ceph-mon[35971]: pgmap v256: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 3.2 MiB/s wr, 181 op/s 2024-05-14T17:03:01.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.122+0000 7f4b65c6b700 1 -- 172.21.15.2:0/1606974219 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b601084f0 msgr2=0x7f4b601088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:01.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.122+0000 7f4b65c6b700 1 --2- 172.21.15.2:0/1606974219 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b601084f0 0x7f4b601088d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f4b5000b0a0 tx=0x7f4b5000b3b0 comp rx=0 tx=0).stop 2024-05-14T17:03:01.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.122+0000 7f4b65c6b700 1 -- 172.21.15.2:0/1606974219 shutdown_connections 2024-05-14T17:03:01.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.123+0000 7f4b65c6b700 1 --2- 172.21.15.2:0/1606974219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b60102e20 0x7f4b6010f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:01.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.123+0000 7f4b65c6b700 1 --2- 172.21.15.2:0/1606974219 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b601084f0 0x7f4b601088d0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:01.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.123+0000 7f4b65c6b700 1 --2- 172.21.15.2:0/1606974219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b60102460 0x7f4b601028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:01.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.123+0000 7f4b65c6b700 1 -- 172.21.15.2:0/1606974219 >> 172.21.15.2:0/1606974219 conn(0x7f4b600794e0 msgr2=0x7f4b600798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:01.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.123+0000 7f4b65c6b700 1 -- 172.21.15.2:0/1606974219 shutdown_connections 2024-05-14T17:03:01.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.123+0000 7f4b65c6b700 1 -- 172.21.15.2:0/1606974219 wait complete. 2024-05-14T17:03:01.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.123+0000 7f4b65c6b700 1 Processor -- start 2024-05-14T17:03:01.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.123+0000 7f4b65c6b700 1 -- start start 2024-05-14T17:03:01.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.124+0000 7f4b65c6b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b60102460 0x7f4b6019f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:01.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.124+0000 7f4b65c6b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b60102e20 0x7f4b6019fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:01.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.124+0000 7f4b65c6b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b601084f0 0x7f4b60199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:01.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.124+0000 7f4b65c6b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4b60112600 con 0x7f4b60102e20 2024-05-14T17:03:01.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.124+0000 7f4b65c6b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4b60112480 con 0x7f4b601084f0 2024-05-14T17:03:01.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.124+0000 7f4b65c6b700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4b60112780 con 0x7f4b60102460 2024-05-14T17:03:01.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.124+0000 7f4b5effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b60102e20 0x7f4b6019fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:01.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.124+0000 7f4b5effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b60102e20 0x7f4b6019fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43196/0 (socket says 172.21.15.2:43196) 2024-05-14T17:03:01.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.124+0000 7f4b5effd700 1 -- 172.21.15.2:0/2614281377 learned_addr learned my addr 172.21.15.2:0/2614281377 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:03:01.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.124+0000 7f4b5f7fe700 1 --2- 172.21.15.2:0/2614281377 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b60102460 0x7f4b6019f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:01.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.124+0000 7f4b5ffff700 1 --2- 172.21.15.2:0/2614281377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b601084f0 0x7f4b60199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:01.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.124+0000 7f4b5effd700 1 -- 172.21.15.2:0/2614281377 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b60102460 msgr2=0x7f4b6019f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:01.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.124+0000 7f4b5effd700 1 --2- 172.21.15.2:0/2614281377 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b60102460 0x7f4b6019f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:01.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.124+0000 7f4b5effd700 1 -- 172.21.15.2:0/2614281377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b601084f0 msgr2=0x7f4b60199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:01.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.124+0000 7f4b5effd700 1 --2- 172.21.15.2:0/2614281377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b601084f0 0x7f4b60199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:01.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.124+0000 7f4b5effd700 1 -- 172.21.15.2:0/2614281377 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b6019a180 con 0x7f4b60102e20 2024-05-14T17:03:01.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.124+0000 7f4b5f7fe700 1 --2- 172.21.15.2:0/2614281377 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b60102460 0x7f4b6019f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:03:01.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.125+0000 7f4b5effd700 1 --2- 172.21.15.2:0/2614281377 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b60102e20 0x7f4b6019fda0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4b5400d950 tx=0x7f4b5400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:01.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.125+0000 7f4b5cff9700 1 -- 172.21.15.2:0/2614281377 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b54016070 con 0x7f4b60102e20 2024-05-14T17:03:01.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.125+0000 7f4b65c6b700 1 -- 172.21.15.2:0/2614281377 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b6019a470 con 0x7f4b60102e20 2024-05-14T17:03:01.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.125+0000 7f4b5cff9700 1 -- 172.21.15.2:0/2614281377 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b5400fd40 con 0x7f4b60102e20 2024-05-14T17:03:01.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.125+0000 7f4b5cff9700 1 -- 172.21.15.2:0/2614281377 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5400e3b0 con 0x7f4b60102e20 2024-05-14T17:03:01.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.125+0000 7f4b65c6b700 1 -- 172.21.15.2:0/2614281377 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b601a6730 con 0x7f4b60102e20 2024-05-14T17:03:01.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.128+0000 7f4b5cff9700 1 -- 172.21.15.2:0/2614281377 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f4b54019070 con 0x7f4b60102e20 2024-05-14T17:03:01.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.128+0000 7f4b5cff9700 1 --2- 172.21.15.2:0/2614281377 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f4b480b29f0 0x7f4b480b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:01.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.128+0000 7f4b5f7fe700 1 --2- 172.21.15.2:0/2614281377 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f4b480b29f0 0x7f4b480b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:01.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.128+0000 7f4b5cff9700 1 -- 172.21.15.2:0/2614281377 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f4b540e2110 con 0x7f4b60102e20 2024-05-14T17:03:01.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.129+0000 7f4b70501b80 1 Processor -- start 2024-05-14T17:03:01.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.129+0000 7f4b70501b80 1 -- start start 2024-05-14T17:03:01.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.129+0000 7f4b70501b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55590fd260b0 0x55590fd26490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:01.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.129+0000 7f4b70501b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590fd218c0 0x55590fd21d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:01.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.129+0000 7f4b70501b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55590fd22280 0x55590fd2dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:01.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.129+0000 7f4b70501b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55590fc7bd90 con 0x55590fd260b0 2024-05-14T17:03:01.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.129+0000 7f4b70501b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55590fc6b930 con 0x55590fd22280 2024-05-14T17:03:01.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.129+0000 7f4b70501b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55590fc6f970 con 0x55590fd218c0 2024-05-14T17:03:01.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.129+0000 7f4b5f7fe700 1 --2- 172.21.15.2:0/2614281377 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f4b480b29f0 0x7f4b480b4eb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4b50009fd0 tx=0x7f4b5000b7b0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:01.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.129+0000 7f4b5f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55590fd260b0 0x55590fd26490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:01.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.129+0000 7f4b5f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55590fd260b0 0x55590fd26490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43208/0 (socket says 172.21.15.2:43208) 2024-05-14T17:03:01.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.129+0000 7f4b5f7fe700 1 -- 172.21.15.2:0/3684511982 learned_addr learned my addr 172.21.15.2:0/3684511982 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:03:01.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.129+0000 7f4b5ffff700 1 --2- 172.21.15.2:0/3684511982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55590fd22280 0x55590fd2dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:01.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.129+0000 7f4b5effd700 1 --2- 172.21.15.2:0/3684511982 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590fd218c0 0x55590fd21d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:01.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.129+0000 7f4b5f7fe700 1 -- 172.21.15.2:0/3684511982 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590fd218c0 msgr2=0x55590fd21d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:01.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.129+0000 7f4b5f7fe700 1 --2- 172.21.15.2:0/3684511982 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590fd218c0 0x55590fd21d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:01.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.129+0000 7f4b5f7fe700 1 -- 172.21.15.2:0/3684511982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55590fd22280 msgr2=0x55590fd2dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:01.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.129+0000 7f4b5f7fe700 1 --2- 172.21.15.2:0/3684511982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55590fd22280 0x55590fd2dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:01.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.129+0000 7f4b5f7fe700 1 -- 172.21.15.2:0/3684511982 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55590fbbb9d0 con 0x55590fd260b0 2024-05-14T17:03:01.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.129+0000 7f4b5ffff700 1 --2- 172.21.15.2:0/3684511982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55590fd22280 0x55590fd2dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:03:01.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.129+0000 7f4b5effd700 1 --2- 172.21.15.2:0/3684511982 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590fd218c0 0x55590fd21d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:03:01.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.130+0000 7f4b5f7fe700 1 --2- 172.21.15.2:0/3684511982 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55590fd260b0 0x55590fd26490 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4b5000b0a0 tx=0x7f4b5001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc089008bc546f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:01.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.130+0000 7f4b45ffb700 1 -- 172.21.15.2:0/3684511982 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5002c070 con 0x55590fd260b0 2024-05-14T17:03:01.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.130+0000 7f4b45ffb700 1 -- 172.21.15.2:0/3684511982 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b50031910 con 0x55590fd260b0 2024-05-14T17:03:01.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.130+0000 7f4b45ffb700 1 -- 172.21.15.2:0/3684511982 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b500287f0 con 0x55590fd260b0 2024-05-14T17:03:02.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.131+0000 7f4b70501b80 1 -- 172.21.15.2:0/3684511982 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55590fd260b0 msgr2=0x55590fd26490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:02.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.131+0000 7f4b70501b80 1 --2- 172.21.15.2:0/3684511982 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55590fd260b0 0x55590fd26490 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4b5000b0a0 tx=0x7f4b5001d000 comp rx=0 tx=0).stop 2024-05-14T17:03:02.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.131+0000 7f4b70501b80 1 -- 172.21.15.2:0/3684511982 shutdown_connections 2024-05-14T17:03:02.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.131+0000 7f4b70501b80 1 --2- 172.21.15.2:0/3684511982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55590fd22280 0x55590fd2dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:02.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.131+0000 7f4b70501b80 1 --2- 172.21.15.2:0/3684511982 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55590fd260b0 0x55590fd26490 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:02.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.131+0000 7f4b70501b80 1 --2- 172.21.15.2:0/3684511982 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590fd218c0 0x55590fd21d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:02.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.131+0000 7f4b70501b80 1 -- 172.21.15.2:0/3684511982 >> 172.21.15.2:0/3684511982 conn(0x55590fc48350 msgr2=0x55590fd17ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:02.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.131+0000 7f4b70501b80 1 -- 172.21.15.2:0/3684511982 shutdown_connections 2024-05-14T17:03:02.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.131+0000 7f4b70501b80 1 -- 172.21.15.2:0/3684511982 wait complete. 2024-05-14T17:03:02.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.131+0000 7f4b70501b80 1 -- start start 2024-05-14T17:03:02.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.131+0000 7f4b70501b80 1 Processor -- start 2024-05-14T17:03:02.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.132+0000 7f4b70501b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590fd218c0 0x55590fd2dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:02.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.132+0000 7f4b70501b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55590fd22280 0x55590fd2af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:02.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.132+0000 7f4b70501b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55590fd260b0 0x55590fd2b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:02.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.132+0000 7f4b70501b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55590fb720a0 con 0x55590fd260b0 2024-05-14T17:03:02.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.132+0000 7f4b70501b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55590fc0d8c0 con 0x55590fd22280 2024-05-14T17:03:02.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.132+0000 7f4b70501b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55590fd2e510 con 0x55590fd218c0 2024-05-14T17:03:02.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.132+0000 7f4b70501b80 10 client.0 _ll_register_callbacks cb 0x55590fc02680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:03:02.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.132+0000 7f4b5ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55590fd260b0 0x55590fd2b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:02.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.132+0000 7f4b5ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55590fd260b0 0x55590fd2b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43214/0 (socket says 172.21.15.2:43214) 2024-05-14T17:03:02.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.132+0000 7f4b5ffff700 1 -- 172.21.15.2:0/2723329623 learned_addr learned my addr 172.21.15.2:0/2723329623 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:03:02.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.132+0000 7f4b5f7fe700 1 --2- 172.21.15.2:0/2723329623 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590fd218c0 0x55590fd2dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:02.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.132+0000 7f4b5effd700 1 --2- 172.21.15.2:0/2723329623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55590fd22280 0x55590fd2af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:02.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.132+0000 7f4b5ffff700 1 -- 172.21.15.2:0/2723329623 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590fd218c0 msgr2=0x55590fd2dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:02.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.132+0000 7f4b5ffff700 1 --2- 172.21.15.2:0/2723329623 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590fd218c0 0x55590fd2dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:02.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.132+0000 7f4b5ffff700 1 -- 172.21.15.2:0/2723329623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55590fd22280 msgr2=0x55590fd2af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:02.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.132+0000 7f4b5ffff700 1 --2- 172.21.15.2:0/2723329623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55590fd22280 0x55590fd2af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:02.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.132+0000 7f4b5ffff700 1 -- 172.21.15.2:0/2723329623 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55590fc0a790 con 0x55590fd260b0 2024-05-14T17:03:02.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.132+0000 7f4b5effd700 1 --2- 172.21.15.2:0/2723329623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55590fd22280 0x55590fd2af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:03:02.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.132+0000 7f4b5ffff700 1 --2- 172.21.15.2:0/2723329623 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55590fd260b0 0x55590fd2b4b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f4b58012850 tx=0x7f4b58012c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5910299503f5296 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:02.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.132+0000 7f4b3effd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:03:02.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.132+0000 7f4b3effd700 1 -- 172.21.15.2:0/2723329623 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5801f060 con 0x55590fd260b0 2024-05-14T17:03:02.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.133+0000 7f4b3effd700 1 -- 172.21.15.2:0/2723329623 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f4b58010cd0 con 0x55590fd260b0 2024-05-14T17:03:02.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.133+0000 7f4b70501b80 1 -- 172.21.15.2:0/2723329623 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b50031910 con 0x55590fd260b0 2024-05-14T17:03:02.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.133+0000 7f4b3effd700 1 -- 172.21.15.2:0/2723329623 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4b5801a430 con 0x55590fd260b0 2024-05-14T17:03:02.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.133+0000 7f4b70501b80 10 client.16233 fetch_fsmap learned FSMap version 41 2024-05-14T17:03:02.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.133+0000 7f4b70501b80 1 -- 172.21.15.2:0/2723329623 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55590fd38bf0 con 0x55590fd260b0 2024-05-14T17:03:02.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.133+0000 7f4b3effd700 1 -- 172.21.15.2:0/2723329623 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b50031910 con 0x55590fd260b0 2024-05-14T17:03:02.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.133+0000 7f4b3effd700 1 -- 172.21.15.2:0/2723329623 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f4b5800e3d0 con 0x55590fd260b0 2024-05-14T17:03:02.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.133+0000 7f4b70501b80 10 client.16233 fetch_fsmap finished waiting for FSMap version 41 2024-05-14T17:03:02.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.133+0000 7f4b70501b80 10 client.16233 resolve_mds: validated gid 24479 aka daemon mds.b 2024-05-14T17:03:02.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.133+0000 7f4b70501b80 20 client.16233 populate_metadata read hostname 'smithi002' 2024-05-14T17:03:02.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.133+0000 7f4b70501b80 1 --2- 172.21.15.2:0/2723329623 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55590fd3b280 0x55590fd3d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:02.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.133+0000 7f4b70501b80 4 client.16233 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:03:02.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.133+0000 7f4b70501b80 1 -- 172.21.15.2:0/2723329623 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55590fce9460 con 0x55590fd3b280 2024-05-14T17:03:02.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.133+0000 7f4b5f7fe700 1 --2- 172.21.15.2:0/2723329623 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55590fd3b280 0x55590fd3d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:02.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.134+0000 7f4b3effd700 10 client.16233 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:03:02.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.134+0000 7f4b5f7fe700 1 --2- 172.21.15.2:0/2723329623 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55590fd3b280 0x55590fd3d690 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89cebaba1f5180aa server_cookie=cd8ace0cf1939a15 in_seq=0 out_seq=0 2024-05-14T17:03:02.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.135+0000 7f4b3effd700 1 -- 172.21.15.2:0/2723329623 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55590fce9460 con 0x55590fd3b280 2024-05-14T17:03:02.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.135+0000 7f4b3effd700 10 client.16233 handle_command_reply: tid=0 2024-05-14T17:03:02.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.135+0000 7f4b70501b80 1 client.16233 shutdown 2024-05-14T17:03:02.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.135+0000 7f4b70501b80 1 -- 172.21.15.2:0/2723329623 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55590fd260b0 msgr2=0x55590fd2b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:02.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.135+0000 7f4b70501b80 1 --2- 172.21.15.2:0/2723329623 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55590fd260b0 0x55590fd2b4b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f4b58012850 tx=0x7f4b58012c10 comp rx=0 tx=0).stop 2024-05-14T17:03:02.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.135+0000 7f4b70501b80 1 -- 172.21.15.2:0/2723329623 shutdown_connections 2024-05-14T17:03:02.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.135+0000 7f4b70501b80 1 --2- 172.21.15.2:0/2723329623 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55590fd3b280 0x55590fd3d690 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:02.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.135+0000 7f4b70501b80 1 --2- 172.21.15.2:0/2723329623 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55590fd260b0 0x55590fd2b4b0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:02.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.135+0000 7f4b70501b80 1 --2- 172.21.15.2:0/2723329623 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590fd218c0 0x55590fd2dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:02.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.135+0000 7f4b70501b80 1 --2- 172.21.15.2:0/2723329623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55590fd22280 0x55590fd2af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:02.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.135+0000 7f4b70501b80 1 -- 172.21.15.2:0/2723329623 >> 172.21.15.2:0/2723329623 conn(0x55590fc48350 msgr2=0x55590fd27730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:02.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.135+0000 7f4b3effd700 0 client.16233 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:03:02.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.135+0000 7f4b70501b80 1 -- 172.21.15.2:0/2723329623 shutdown_connections 2024-05-14T17:03:02.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.135+0000 7f4b70501b80 1 -- 172.21.15.2:0/2723329623 wait complete. 2024-05-14T17:03:02.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.135+0000 7f4b70501b80 20 client.16233 trim_cache size 0 max 16384 2024-05-14T17:03:02.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.155+0000 7f4b70501b80 1 Processor -- start 2024-05-14T17:03:02.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.155+0000 7f4b70501b80 1 -- start start 2024-05-14T17:03:02.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.155+0000 7f4b70501b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590fd218c0 0x55590fd2b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:02.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.155+0000 7f4b70501b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55590fd22280 0x55590fd37420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:03.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.155+0000 7f4b70501b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55590fd260b0 0x55590fd12160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:03.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.155+0000 7f4b70501b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4b601ae410 con 0x55590fd22280 2024-05-14T17:03:03.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.155+0000 7f4b70501b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4b601abca0 con 0x55590fd260b0 2024-05-14T17:03:03.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.155+0000 7f4b70501b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4b601a9b30 con 0x55590fd218c0 2024-05-14T17:03:03.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.156+0000 7f4b5effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55590fd22280 0x55590fd37420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:03.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.156+0000 7f4b5effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55590fd22280 0x55590fd37420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43224/0 (socket says 172.21.15.2:43224) 2024-05-14T17:03:03.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.156+0000 7f4b5effd700 1 -- 172.21.15.2:0/1544003658 learned_addr learned my addr 172.21.15.2:0/1544003658 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:03:03.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.156+0000 7f4b5f7fe700 1 --2- 172.21.15.2:0/1544003658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590fd218c0 0x55590fd2b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:03.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.156+0000 7f4b5ffff700 1 --2- 172.21.15.2:0/1544003658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55590fd260b0 0x55590fd12160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:03.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.156+0000 7f4b5effd700 1 -- 172.21.15.2:0/1544003658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590fd218c0 msgr2=0x55590fd2b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:03.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.156+0000 7f4b5effd700 1 --2- 172.21.15.2:0/1544003658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590fd218c0 0x55590fd2b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:03.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.156+0000 7f4b5effd700 1 -- 172.21.15.2:0/1544003658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55590fd260b0 msgr2=0x55590fd12160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:03.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.156+0000 7f4b5effd700 1 --2- 172.21.15.2:0/1544003658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55590fd260b0 0x55590fd12160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:03.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.156+0000 7f4b5effd700 1 -- 172.21.15.2:0/1544003658 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55590fd294e0 con 0x55590fd22280 2024-05-14T17:03:03.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.156+0000 7f4b5ffff700 1 --2- 172.21.15.2:0/1544003658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55590fd260b0 0x55590fd12160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:03:03.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.156+0000 7f4b5effd700 1 --2- 172.21.15.2:0/1544003658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55590fd22280 0x55590fd37420 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f4b540a9380 tx=0x7f4b54010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=934cd78ce685d865 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:03.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.156+0000 7f4b457fa700 1 -- 172.21.15.2:0/1544003658 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b54020040 con 0x55590fd22280 2024-05-14T17:03:03.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.156+0000 7f4b457fa700 1 -- 172.21.15.2:0/1544003658 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b540a93f0 con 0x55590fd22280 2024-05-14T17:03:03.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.156+0000 7f4b457fa700 1 -- 172.21.15.2:0/1544003658 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b540b1dc0 con 0x55590fd22280 2024-05-14T17:03:03.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.157+0000 7f4b70501b80 1 -- 172.21.15.2:0/1544003658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55590fd22280 msgr2=0x55590fd37420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:03.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.157+0000 7f4b70501b80 1 --2- 172.21.15.2:0/1544003658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55590fd22280 0x55590fd37420 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f4b540a9380 tx=0x7f4b54010000 comp rx=0 tx=0).stop 2024-05-14T17:03:03.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.157+0000 7f4b70501b80 1 -- 172.21.15.2:0/1544003658 shutdown_connections 2024-05-14T17:03:03.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.157+0000 7f4b70501b80 1 --2- 172.21.15.2:0/1544003658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55590fd260b0 0x55590fd12160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:03.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.157+0000 7f4b70501b80 1 --2- 172.21.15.2:0/1544003658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590fd218c0 0x55590fd2b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:03.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.157+0000 7f4b70501b80 1 --2- 172.21.15.2:0/1544003658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55590fd22280 0x55590fd37420 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:03.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.157+0000 7f4b70501b80 1 -- 172.21.15.2:0/1544003658 >> 172.21.15.2:0/1544003658 conn(0x55590fc48350 msgr2=0x55590fd2c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:03.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.157+0000 7f4b70501b80 1 -- 172.21.15.2:0/1544003658 shutdown_connections 2024-05-14T17:03:03.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.157+0000 7f4b70501b80 1 -- 172.21.15.2:0/1544003658 wait complete. 2024-05-14T17:03:03.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.157+0000 7f4b70501b80 1 -- start start 2024-05-14T17:03:03.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.157+0000 7f4b70501b80 1 Processor -- start 2024-05-14T17:03:03.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.158+0000 7f4b70501b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55590fd218c0 0x55590fd33c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:03.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.158+0000 7f4b70501b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55590fd22280 0x55590fd23f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:03.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.158+0000 7f4b70501b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590fd260b0 0x55590fd24470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:03.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.158+0000 7f4b70501b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4b601aa180 con 0x55590fd218c0 2024-05-14T17:03:03.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.158+0000 7f4b70501b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4b601ab5d0 con 0x55590fd22280 2024-05-14T17:03:03.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.158+0000 7f4b70501b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4b601aa830 con 0x55590fd260b0 2024-05-14T17:03:03.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.158+0000 7f4b70501b80 10 client.0 _ll_register_callbacks cb 0x55590fc02680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:03:03.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.158+0000 7f4b5f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55590fd218c0 0x55590fd33c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:03.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.158+0000 7f4b5f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55590fd218c0 0x55590fd33c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43232/0 (socket says 172.21.15.2:43232) 2024-05-14T17:03:03.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.158+0000 7f4b5f7fe700 1 -- 172.21.15.2:0/1969986988 learned_addr learned my addr 172.21.15.2:0/1969986988 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:03:03.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.158+0000 7f4b5effd700 1 --2- 172.21.15.2:0/1969986988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55590fd22280 0x55590fd23f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:03.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.158+0000 7f4b5ffff700 1 --2- 172.21.15.2:0/1969986988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590fd260b0 0x55590fd24470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:03.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.158+0000 7f4b5f7fe700 1 -- 172.21.15.2:0/1969986988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590fd260b0 msgr2=0x55590fd24470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:03.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.158+0000 7f4b5f7fe700 1 --2- 172.21.15.2:0/1969986988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590fd260b0 0x55590fd24470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:03.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.158+0000 7f4b5f7fe700 1 -- 172.21.15.2:0/1969986988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55590fd22280 msgr2=0x55590fd23f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:03.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.158+0000 7f4b5f7fe700 1 --2- 172.21.15.2:0/1969986988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55590fd22280 0x55590fd23f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:03.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.158+0000 7f4b5f7fe700 1 -- 172.21.15.2:0/1969986988 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55590fd22cf0 con 0x55590fd218c0 2024-05-14T17:03:03.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.158+0000 7f4b5f7fe700 1 --2- 172.21.15.2:0/1969986988 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55590fd218c0 0x55590fd33c20 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f4b5000d010 tx=0x7f4b50007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad24dd1c69d03e88 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:03.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.158+0000 7f4b3effd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:03:03.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.159+0000 7f4b3effd700 1 -- 172.21.15.2:0/1969986988 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b50042060 con 0x55590fd218c0 2024-05-14T17:03:03.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.159+0000 7f4b70501b80 1 -- 172.21.15.2:0/1969986988 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b540a93f0 con 0x55590fd218c0 2024-05-14T17:03:03.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.159+0000 7f4b3effd700 1 -- 172.21.15.2:0/1969986988 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7f4b50003d50 con 0x55590fd218c0 2024-05-14T17:03:03.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.159+0000 7f4b3effd700 1 -- 172.21.15.2:0/1969986988 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4b50020040 con 0x55590fd218c0 2024-05-14T17:03:03.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.159+0000 7f4b70501b80 10 client.16239 fetch_fsmap learned FSMap version 41 2024-05-14T17:03:03.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.159+0000 7f4b70501b80 1 -- 172.21.15.2:0/1969986988 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55590fda6d60 con 0x55590fd218c0 2024-05-14T17:03:03.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.159+0000 7f4b3effd700 1 -- 172.21.15.2:0/1969986988 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b540a93f0 con 0x55590fd218c0 2024-05-14T17:03:03.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.159+0000 7f4b3effd700 1 -- 172.21.15.2:0/1969986988 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f4b50011070 con 0x55590fd218c0 2024-05-14T17:03:03.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.159+0000 7f4b70501b80 10 client.16239 fetch_fsmap finished waiting for FSMap version 41 2024-05-14T17:03:03.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.159+0000 7f4b70501b80 10 client.16239 resolve_mds: validated gid 24479 aka daemon mds.b 2024-05-14T17:03:03.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.159+0000 7f4b70501b80 20 client.16239 populate_metadata read hostname 'smithi002' 2024-05-14T17:03:03.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.159+0000 7f4b70501b80 1 --2- 172.21.15.2:0/1969986988 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55590fda6f90 0x55590fda9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:03.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.159+0000 7f4b70501b80 4 client.16239 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "8649c12", "release": true, "await": true}] 2024-05-14T17:03:03.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.159+0000 7f4b70501b80 1 -- 172.21.15.2:0/1969986988 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "quiesce db", "set_id": "8649c12", "release": true, "await": true}) -- 0x55590fc7dce0 con 0x55590fda6f90 2024-05-14T17:03:03.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.160+0000 7f4b5effd700 1 --2- 172.21.15.2:0/1969986988 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55590fda6f90 0x55590fda9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:03.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.160+0000 7f4b5effd700 1 --2- 172.21.15.2:0/1969986988 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55590fda6f90 0x55590fda9380 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff3d2863135c6f36 server_cookie=e749d1cb21b5450f in_seq=0 out_seq=0 2024-05-14T17:03:03.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.160+0000 7f4b3effd700 10 client.16239 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:03:03.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.170+0000 7f4b3effd700 1 -- 172.21.15.2:0/1969986988 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55590fc7dce0 con 0x55590fda6f90 2024-05-14T17:03:03.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.170+0000 7f4b3effd700 10 client.16239 handle_command_reply: tid=0 2024-05-14T17:03:03.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.170+0000 7f4b70501b80 1 client.16239 shutdown 2024-05-14T17:03:03.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.171+0000 7f4b70501b80 1 -- 172.21.15.2:0/1969986988 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55590fd218c0 msgr2=0x55590fd33c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:03.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.171+0000 7f4b70501b80 1 --2- 172.21.15.2:0/1969986988 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55590fd218c0 0x55590fd33c20 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f4b5000d010 tx=0x7f4b50007e60 comp rx=0 tx=0).stop 2024-05-14T17:03:03.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.171+0000 7f4b70501b80 1 -- 172.21.15.2:0/1969986988 shutdown_connections 2024-05-14T17:03:03.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.171+0000 7f4b70501b80 1 --2- 172.21.15.2:0/1969986988 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55590fda6f90 0x55590fda9380 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:04.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.171+0000 7f4b70501b80 1 --2- 172.21.15.2:0/1969986988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55590fd260b0 0x55590fd24470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:04.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.171+0000 7f4b70501b80 1 --2- 172.21.15.2:0/1969986988 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55590fd218c0 0x55590fd33c20 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:04.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.171+0000 7f4b70501b80 1 --2- 172.21.15.2:0/1969986988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55590fd22280 0x55590fd23f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:04.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.171+0000 7f4b3effd700 0 client.16239 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:03:04.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.171+0000 7f4b70501b80 1 -- 172.21.15.2:0/1969986988 >> 172.21.15.2:0/1969986988 conn(0x55590fc48350 msgr2=0x55590fd1cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:04.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.171+0000 7f4b70501b80 1 -- 172.21.15.2:0/1969986988 shutdown_connections 2024-05-14T17:03:04.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.171+0000 7f4b70501b80 1 -- 172.21.15.2:0/1969986988 wait complete. 2024-05-14T17:03:04.087 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.171+0000 7f4b70501b80 20 client.16239 trim_cache size 0 max 16384 2024-05-14T17:03:04.099 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T17:03:04.114 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 41, 2024-05-14T17:03:04.125 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24479, 2024-05-14T17:03:04.135 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 28, 2024-05-14T17:03:04.148 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-05-14T17:03:04.160 INFO:teuthology.orchestra.run.smithi002.stdout: "8649c12": { 2024-05-14T17:03:04.171 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 28, 2024-05-14T17:03:04.185 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-05-14T17:03:04.219 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-05-14T17:03:04.250 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-05-14T17:03:04.264 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-05-14T17:03:04.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:03:04.294 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-05-14T17:03:04.307 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 151.3, 2024-05-14T17:03:04.323 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-05-14T17:03:04.332 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-05-14T17:03:04.343 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-05-14T17:03:04.355 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-05-14T17:03:04.380 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-05-14T17:03:04.391 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-05-14T17:03:04.402 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:03:04.411 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:03:04.422 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:03:04.433 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:03:04.445 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:03:04.483 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T17:03:04.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.171+0000 7f4b65c6b700 1 -- 172.21.15.2:0/2614281377 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f4b480b29f0 msgr2=0x7f4b480b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:04.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.171+0000 7f4b65c6b700 1 --2- 172.21.15.2:0/2614281377 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f4b480b29f0 0x7f4b480b4eb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4b50009fd0 tx=0x7f4b5000b7b0 comp rx=0 tx=0).stop 2024-05-14T17:03:04.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.171+0000 7f4b65c6b700 1 -- 172.21.15.2:0/2614281377 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b60102e20 msgr2=0x7f4b6019fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:04.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.171+0000 7f4b65c6b700 1 --2- 172.21.15.2:0/2614281377 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b60102e20 0x7f4b6019fda0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4b5400d950 tx=0x7f4b5400dc60 comp rx=0 tx=0).stop 2024-05-14T17:03:04.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.171+0000 7f4b65c6b700 1 -- 172.21.15.2:0/2614281377 shutdown_connections 2024-05-14T17:03:04.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.171+0000 7f4b65c6b700 1 --2- 172.21.15.2:0/2614281377 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f4b480b29f0 0x7f4b480b4eb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:04.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.171+0000 7f4b65c6b700 1 --2- 172.21.15.2:0/2614281377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b601084f0 0x7f4b60199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:04.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.171+0000 7f4b65c6b700 1 --2- 172.21.15.2:0/2614281377 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b60102460 0x7f4b6019f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:04.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.171+0000 7f4b65c6b700 1 --2- 172.21.15.2:0/2614281377 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b60102e20 0x7f4b6019fda0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:04.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.172+0000 7f4b65c6b700 1 -- 172.21.15.2:0/2614281377 >> 172.21.15.2:0/2614281377 conn(0x7f4b600794e0 msgr2=0x7f4b6006bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:04.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.172+0000 7f4b65c6b700 1 -- 172.21.15.2:0/2614281377 shutdown_connections 2024-05-14T17:03:04.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:03:01.172+0000 7f4b65c6b700 1 -- 172.21.15.2:0/2614281377 wait complete. 2024-05-14T17:03:04.638 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 465.0 s, 108.3 tps, lat 5246.141 ms stddev 11075.241 2024-05-14T17:03:04.647 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:03:02 smithi002 ceph-mon[31538]: pgmap v257: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 38 KiB/s wr, 10 op/s 2024-05-14T17:03:04.655 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '8649c12', total seconds elapsed: 41.6 2024-05-14T17:03:04.666 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:03:02 smithi123 ceph-mon[35971]: pgmap v257: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 38 KiB/s wr, 10 op/s 2024-05-14T17:03:04.702 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:03:02 smithi151 ceph-mon[36066]: pgmap v257: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 38 KiB/s wr, 10 op/s 2024-05-14T17:03:04.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:03:04 smithi151 ceph-mon[36066]: pgmap v258: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 1022 KiB/s wr, 133 op/s 2024-05-14T17:03:04.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:03:04 smithi123 ceph-mon[35971]: pgmap v258: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 1022 KiB/s wr, 133 op/s 2024-05-14T17:03:04.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:03:04 smithi002 ceph-mon[31538]: pgmap v258: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 1022 KiB/s wr, 133 op/s 2024-05-14T17:03:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.314+0000 7ff729423640 1 -- 172.21.15.2:0/2541597950 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff72410bd10 msgr2=0x7ff724114460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:05.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.314+0000 7ff729423640 1 --2- 172.21.15.2:0/2541597950 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff72410bd10 0x7ff724114460 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff714009900 tx=0x7ff71402a480 comp rx=0 tx=0).stop 2024-05-14T17:03:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.314+0000 7ff729423640 1 -- 172.21.15.2:0/2541597950 shutdown_connections 2024-05-14T17:03:05.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.314+0000 7ff729423640 1 --2- 172.21.15.2:0/2541597950 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7241149a0 0x7ff724116d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:05.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.314+0000 7ff729423640 1 --2- 172.21.15.2:0/2541597950 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff72410bd10 0x7ff724114460 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.314+0000 7ff729423640 1 --2- 172.21.15.2:0/2541597950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff72410b3f0 0x7ff72410b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.314+0000 7ff729423640 1 -- 172.21.15.2:0/2541597950 >> 172.21.15.2:0/2541597950 conn(0x7ff72406dfb0 msgr2=0x7ff72406e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:05.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.314+0000 7ff729423640 1 -- 172.21.15.2:0/2541597950 shutdown_connections 2024-05-14T17:03:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.314+0000 7ff729423640 1 -- 172.21.15.2:0/2541597950 wait complete. 2024-05-14T17:03:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.315+0000 7ff729423640 1 Processor -- start 2024-05-14T17:03:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.315+0000 7ff729423640 1 -- start start 2024-05-14T17:03:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.315+0000 7ff729423640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff72410b3f0 0x7ff7241a7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:05.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.315+0000 7ff729423640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff72410bd10 0x7ff7241a7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.315+0000 7ff729423640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7241149a0 0x7ff7241a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:05.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.315+0000 7ff729423640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff72411a3b0 con 0x7ff72410bd10 2024-05-14T17:03:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.315+0000 7ff729423640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff72411a230 con 0x7ff7241149a0 2024-05-14T17:03:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.315+0000 7ff729423640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff72411a530 con 0x7ff72410b3f0 2024-05-14T17:03:05.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.315+0000 7ff7227fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff72410bd10 0x7ff7241a7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.316+0000 7ff722ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff72410b3f0 0x7ff7241a7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.316+0000 7ff722ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff72410b3f0 0x7ff7241a7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:41402/0 (socket says 172.21.15.2:41402) 2024-05-14T17:03:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.316+0000 7ff7227fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff72410bd10 0x7ff7241a7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49982/0 (socket says 172.21.15.2:49982) 2024-05-14T17:03:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.316+0000 7ff722ffd640 1 -- 172.21.15.2:0/1311572849 learned_addr learned my addr 172.21.15.2:0/1311572849 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:03:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.316+0000 7ff7237fe640 1 --2- 172.21.15.2:0/1311572849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7241149a0 0x7ff7241a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.316+0000 7ff722ffd640 1 -- 172.21.15.2:0/1311572849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7241149a0 msgr2=0x7ff7241a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.316+0000 7ff722ffd640 1 --2- 172.21.15.2:0/1311572849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7241149a0 0x7ff7241a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.316+0000 7ff722ffd640 1 -- 172.21.15.2:0/1311572849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff72410bd10 msgr2=0x7ff7241a7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:05.691 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.316+0000 7ff722ffd640 1 --2- 172.21.15.2:0/1311572849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff72410bd10 0x7ff7241a7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.316+0000 7ff722ffd640 1 -- 172.21.15.2:0/1311572849 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7241a1f50 con 0x7ff72410b3f0 2024-05-14T17:03:05.735 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.316+0000 7ff7227fc640 1 --2- 172.21.15.2:0/1311572849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff72410bd10 0x7ff7241a7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:03:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.316+0000 7ff722ffd640 1 --2- 172.21.15.2:0/1311572849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff72410b3f0 0x7ff7241a7610 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff70c00e9f0 tx=0x7ff70c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:05.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.316+0000 7ff71bfff640 1 -- 172.21.15.2:0/1311572849 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70c00cd80 con 0x7ff72410b3f0 2024-05-14T17:03:05.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.316+0000 7ff729423640 1 -- 172.21.15.2:0/1311572849 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7241a2240 con 0x7ff72410b3f0 2024-05-14T17:03:05.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.316+0000 7ff71bfff640 1 -- 172.21.15.2:0/1311572849 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff70c0108e0 con 0x7ff72410b3f0 2024-05-14T17:03:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.316+0000 7ff71bfff640 1 -- 172.21.15.2:0/1311572849 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70c018980 con 0x7ff72410b3f0 2024-05-14T17:03:05.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.316+0000 7ff729423640 1 -- 172.21.15.2:0/1311572849 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7241ae460 con 0x7ff72410b3f0 2024-05-14T17:03:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.320+0000 7ff72c21d740 1 Processor -- start 2024-05-14T17:03:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.320+0000 7ff72c21d740 1 -- start start 2024-05-14T17:03:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.320+0000 7ff72c21d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fdb4dce30 0x558fdb4db7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.321+0000 7ff72c21d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fdb4dbcf0 0x558fdb4dc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.321+0000 7ff72c21d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558fdb4ef170 0x558fdb4f1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.321+0000 7ff72c21d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x558fdb44ec30 con 0x558fdb4dce30 2024-05-14T17:03:05.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.321+0000 7ff72c21d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558fdb43e7d0 con 0x558fdb4dbcf0 2024-05-14T17:03:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.321+0000 7ff72c21d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558fdb442810 con 0x558fdb4ef170 2024-05-14T17:03:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.321+0000 7ff7237fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fdb4dce30 0x558fdb4db7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:05.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.321+0000 7ff7227fc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558fdb4ef170 0x558fdb4f1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:05.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.321+0000 7ff7237fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fdb4dce30 0x558fdb4db7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49986/0 (socket says 172.21.15.2:49986) 2024-05-14T17:03:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.321+0000 7ff7237fe640 1 -- 172.21.15.2:0/3626047546 learned_addr learned my addr 172.21.15.2:0/3626047546 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:03:06.014 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.321+0000 7ff7227fc640 1 -- 172.21.15.2:0/3626047546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fdb4dbcf0 msgr2=0x558fdb4dc170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T17:03:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.321+0000 7ff7227fc640 1 --2- 172.21.15.2:0/3626047546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fdb4dbcf0 0x558fdb4dc170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.321+0000 7ff7227fc640 1 -- 172.21.15.2:0/3626047546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fdb4dce30 msgr2=0x558fdb4db7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.321+0000 7ff7227fc640 1 --2- 172.21.15.2:0/3626047546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fdb4dce30 0x558fdb4db7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.321+0000 7ff7227fc640 1 -- 172.21.15.2:0/3626047546 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fdb153210 con 0x558fdb4ef170 2024-05-14T17:03:06.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.321+0000 7ff7237fe640 1 --2- 172.21.15.2:0/3626047546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fdb4dce30 0x558fdb4db7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:03:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.322+0000 7ff7227fc640 1 --2- 172.21.15.2:0/3626047546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558fdb4ef170 0x558fdb4f1560 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff714009230 tx=0x7ff714005d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb82d1c0a5293252 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.322+0000 7ff71bfff640 1 -- 172.21.15.2:0/1311572849 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7ff70c018b20 con 0x7ff72410b3f0 2024-05-14T17:03:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.322+0000 7ff7197fa640 1 -- 172.21.15.2:0/3626047546 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff714040070 con 0x558fdb4ef170 2024-05-14T17:03:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.322+0000 7ff7197fa640 1 -- 172.21.15.2:0/3626047546 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff714031860 con 0x558fdb4ef170 2024-05-14T17:03:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.322+0000 7ff7197fa640 1 -- 172.21.15.2:0/3626047546 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff714035040 con 0x558fdb4ef170 2024-05-14T17:03:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.322+0000 7ff71bfff640 1 --2- 172.21.15.2:0/1311572849 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ff7100b2a10 0x7ff7100b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.323+0000 7ff72c21d740 1 -- 172.21.15.2:0/3626047546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558fdb4ef170 msgr2=0x558fdb4f1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.323+0000 7ff72c21d740 1 --2- 172.21.15.2:0/3626047546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558fdb4ef170 0x558fdb4f1560 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff714009230 tx=0x7ff714005d90 comp rx=0 tx=0).stop 2024-05-14T17:03:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.323+0000 7ff71bfff640 1 -- 172.21.15.2:0/1311572849 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7ff70c0e29a0 con 0x7ff72410b3f0 2024-05-14T17:03:06.186 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.323+0000 7ff7237fe640 1 --2- 172.21.15.2:0/1311572849 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ff7100b2a10 0x7ff7100b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:06.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.323+0000 7ff72c21d740 1 -- 172.21.15.2:0/3626047546 shutdown_connections 2024-05-14T17:03:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.323+0000 7ff72c21d740 1 --2- 172.21.15.2:0/3626047546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558fdb4ef170 0x558fdb4f1560 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.323+0000 7ff72c21d740 1 --2- 172.21.15.2:0/3626047546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fdb4dbcf0 0x558fdb4dc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:06.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.323+0000 7ff72c21d740 1 --2- 172.21.15.2:0/3626047546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fdb4dce30 0x558fdb4db7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:06.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.323+0000 7ff72c21d740 1 -- 172.21.15.2:0/3626047546 >> 172.21.15.2:0/3626047546 conn(0x558fdb4d7e00 msgr2=0x558fdb4d81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:06.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.323+0000 7ff72c21d740 1 -- 172.21.15.2:0/3626047546 shutdown_connections 2024-05-14T17:03:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.323+0000 7ff72c21d740 1 -- 172.21.15.2:0/3626047546 wait complete. 2024-05-14T17:03:06.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.323+0000 7ff72c21d740 1 -- start start 2024-05-14T17:03:06.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.324+0000 7ff7237fe640 1 --2- 172.21.15.2:0/1311572849 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ff7100b2a10 0x7ff7100b4ed0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff71c009fd0 tx=0x7ff71c009500 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:06.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.324+0000 7ff72c21d740 1 Processor -- start 2024-05-14T17:03:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.325+0000 7ff72c21d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558fdb4dbcf0 0x558fdb4e1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.325+0000 7ff72c21d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fdb4dce30 0x558fdb4df360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:06.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.325+0000 7ff72c21d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fdb4ef170 0x558fdb4df9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.325+0000 7ff72c21d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x558fdb124790 con 0x558fdb4dce30 2024-05-14T17:03:06.401 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.325+0000 7ff72c21d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558fdb3fa890 con 0x558fdb4ef170 2024-05-14T17:03:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.325+0000 7ff722ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558fdb4dbcf0 0x558fdb4e1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.325+0000 7ff72c21d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558fdb4f1f10 con 0x558fdb4dbcf0 2024-05-14T17:03:06.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.325+0000 7ff722ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558fdb4dbcf0 0x558fdb4e1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:41426/0 (socket says 172.21.15.2:41426) 2024-05-14T17:03:06.503 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.325+0000 7ff722ffd640 1 -- 172.21.15.2:0/2196926463 learned_addr learned my addr 172.21.15.2:0/2196926463 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:03:06.514 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.325+0000 7ff7227fc640 1 --2- 172.21.15.2:0/2196926463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fdb4dce30 0x558fdb4df360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.325+0000 7ff72c21d740 10 client.0 _ll_register_callbacks cb 0x558fdb3fdd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:03:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.325+0000 7ff7237fe640 1 --2- 172.21.15.2:0/2196926463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fdb4ef170 0x558fdb4df9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.325+0000 7ff722ffd640 1 -- 172.21.15.2:0/2196926463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fdb4ef170 msgr2=0x558fdb4df9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:06.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.325+0000 7ff722ffd640 1 --2- 172.21.15.2:0/2196926463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fdb4ef170 0x558fdb4df9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.325+0000 7ff722ffd640 1 -- 172.21.15.2:0/2196926463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fdb4dce30 msgr2=0x558fdb4df360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:06.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.325+0000 7ff722ffd640 1 --2- 172.21.15.2:0/2196926463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fdb4dce30 0x558fdb4df360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.325+0000 7ff722ffd640 1 -- 172.21.15.2:0/2196926463 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fdb21f620 con 0x558fdb4dbcf0 2024-05-14T17:03:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.326+0000 7ff722ffd640 1 --2- 172.21.15.2:0/2196926463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558fdb4dbcf0 0x558fdb4e1fb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff70c0e01a0 tx=0x7ff70c0a9980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d386ff3f39a6b052 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.326+0000 7ff70a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-14T17:03:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.326+0000 7ff70a7fc640 1 -- 172.21.15.2:0/2196926463 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70c010430 con 0x558fdb4dbcf0 2024-05-14T17:03:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.326+0000 7ff70a7fc640 1 -- 172.21.15.2:0/2196926463 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7ff70c0105d0 con 0x558fdb4dbcf0 2024-05-14T17:03:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.326+0000 7ff72c21d740 1 -- 172.21.15.2:0/2196926463 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff714031860 con 0x558fdb4dbcf0 2024-05-14T17:03:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.326+0000 7ff70a7fc640 1 -- 172.21.15.2:0/2196926463 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff70c0a8f90 con 0x558fdb4dbcf0 2024-05-14T17:03:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.326+0000 7ff72c21d740 10 client.25906 fetch_fsmap learned FSMap version 41 2024-05-14T17:03:06.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.326+0000 7ff72c21d740 1 -- 172.21.15.2:0/2196926463 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558fdb4fdaa0 con 0x558fdb4dbcf0 2024-05-14T17:03:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.326+0000 7ff70a7fc640 1 -- 172.21.15.2:0/2196926463 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff714031860 con 0x558fdb4dbcf0 2024-05-14T17:03:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.326+0000 7ff70a7fc640 1 -- 172.21.15.2:0/2196926463 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7ff70c0a8a90 con 0x558fdb4dbcf0 2024-05-14T17:03:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.326+0000 7ff72c21d740 10 client.25906 fetch_fsmap finished waiting for FSMap version 41 2024-05-14T17:03:06.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.326+0000 7ff72c21d740 10 client.25906 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:03:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.326+0000 7ff72c21d740 20 client.25906 populate_metadata read hostname 'smithi002' 2024-05-14T17:03:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.326+0000 7ff72c21d740 1 --2- 172.21.15.2:0/2196926463 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x558fdb4fef60 0x558fdb501350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.326+0000 7ff72c21d740 4 client.25906 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:03:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.326+0000 7ff72c21d740 1 -- 172.21.15.2:0/2196926463 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fdb450b80 con 0x558fdb4fef60 2024-05-14T17:03:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.327+0000 7ff7227fc640 1 --2- 172.21.15.2:0/2196926463 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x558fdb4fef60 0x558fdb501350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.327+0000 7ff70a7fc640 10 client.25906 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:03:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.327+0000 7ff7227fc640 1 --2- 172.21.15.2:0/2196926463 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x558fdb4fef60 0x558fdb501350 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84824c687ac5e32c server_cookie=159b0f7497bcf6f in_seq=0 out_seq=0 2024-05-14T17:03:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.328+0000 7ff70a7fc640 1 -- 172.21.15.2:0/2196926463 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558fdb450b80 con 0x558fdb4fef60 2024-05-14T17:03:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.328+0000 7ff70a7fc640 10 client.25906 handle_command_reply: tid=0 2024-05-14T17:03:06.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.328+0000 7ff72c21d740 1 client.25906 shutdown 2024-05-14T17:03:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.328+0000 7ff72c21d740 1 -- 172.21.15.2:0/2196926463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558fdb4dbcf0 msgr2=0x558fdb4e1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:06.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.328+0000 7ff72c21d740 1 --2- 172.21.15.2:0/2196926463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558fdb4dbcf0 0x558fdb4e1fb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff70c0e01a0 tx=0x7ff70c0a9980 comp rx=0 tx=0).stop 2024-05-14T17:03:06.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.328+0000 7ff72c21d740 1 -- 172.21.15.2:0/2196926463 shutdown_connections 2024-05-14T17:03:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.328+0000 7ff72c21d740 1 --2- 172.21.15.2:0/2196926463 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x558fdb4fef60 0x558fdb501350 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.328+0000 7ff72c21d740 1 --2- 172.21.15.2:0/2196926463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fdb4ef170 0x558fdb4df9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.328+0000 7ff72c21d740 1 --2- 172.21.15.2:0/2196926463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fdb4dce30 0x558fdb4df360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.328+0000 7ff70a7fc640 0 client.25906 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:03:07.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.328+0000 7ff72c21d740 1 --2- 172.21.15.2:0/2196926463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558fdb4dbcf0 0x558fdb4e1fb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.328+0000 7ff72c21d740 1 -- 172.21.15.2:0/2196926463 >> 172.21.15.2:0/2196926463 conn(0x558fdb4d7e00 msgr2=0x558fdb4ddc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.328+0000 7ff72c21d740 1 -- 172.21.15.2:0/2196926463 shutdown_connections 2024-05-14T17:03:07.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.328+0000 7ff72c21d740 1 -- 172.21.15.2:0/2196926463 wait complete. 2024-05-14T17:03:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.328+0000 7ff72c21d740 20 client.25906 trim_cache size 0 max 16384 2024-05-14T17:03:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.350+0000 7ff72c21d740 1 Processor -- start 2024-05-14T17:03:07.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.350+0000 7ff72c21d740 1 -- start start 2024-05-14T17:03:07.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.350+0000 7ff72c21d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fdb4dbcf0 0x558fdb4d6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.350+0000 7ff72c21d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fdb4dce30 0x558fdb4e1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.350+0000 7ff72c21d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558fdb4ef170 0x558fdb506b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.350+0000 7ff72c21d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff7241b74a0 con 0x558fdb4dbcf0 2024-05-14T17:03:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.350+0000 7ff72c21d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff7241b4d70 con 0x558fdb4dce30 2024-05-14T17:03:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.350+0000 7ff72c21d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff7241b2be0 con 0x558fdb4ef170 2024-05-14T17:03:07.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.350+0000 7ff722ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fdb4dbcf0 0x558fdb4d6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.351+0000 7ff7237fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558fdb4ef170 0x558fdb506b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.351+0000 7ff722ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fdb4dbcf0 0x558fdb4d6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50004/0 (socket says 172.21.15.2:50004) 2024-05-14T17:03:07.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.351+0000 7ff722ffd640 1 -- 172.21.15.2:0/1293324612 learned_addr learned my addr 172.21.15.2:0/1293324612 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:03:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.351+0000 7ff7227fc640 1 --2- 172.21.15.2:0/1293324612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fdb4dce30 0x558fdb4e1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.351+0000 7ff7237fe640 1 -- 172.21.15.2:0/1293324612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fdb4dce30 msgr2=0x558fdb4e1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.351+0000 7ff7237fe640 1 --2- 172.21.15.2:0/1293324612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fdb4dce30 0x558fdb4e1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:07.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.351+0000 7ff7237fe640 1 -- 172.21.15.2:0/1293324612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fdb4dbcf0 msgr2=0x558fdb4d6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.351+0000 7ff7237fe640 1 --2- 172.21.15.2:0/1293324612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fdb4dbcf0 0x558fdb4d6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:07.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.351+0000 7ff7237fe640 1 -- 172.21.15.2:0/1293324612 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fdb5016c0 con 0x558fdb4ef170 2024-05-14T17:03:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.351+0000 7ff722ffd640 1 --2- 172.21.15.2:0/1293324612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fdb4dbcf0 0x558fdb4d6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:03:07.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.351+0000 7ff7237fe640 1 --2- 172.21.15.2:0/1293324612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558fdb4ef170 0x558fdb506b70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff71c005fd0 tx=0x7ff71c016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=905b9aeb1ac3688c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:07.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.351+0000 7ff718ff9640 1 -- 172.21.15.2:0/1293324612 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c01c410 con 0x558fdb4ef170 2024-05-14T17:03:07.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.351+0000 7ff718ff9640 1 -- 172.21.15.2:0/1293324612 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff71c02c070 con 0x558fdb4ef170 2024-05-14T17:03:07.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.351+0000 7ff718ff9640 1 -- 172.21.15.2:0/1293324612 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c0279c0 con 0x558fdb4ef170 2024-05-14T17:03:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.352+0000 7ff72c21d740 1 -- 172.21.15.2:0/1293324612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558fdb4ef170 msgr2=0x558fdb506b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.352+0000 7ff72c21d740 1 --2- 172.21.15.2:0/1293324612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558fdb4ef170 0x558fdb506b70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff71c005fd0 tx=0x7ff71c016000 comp rx=0 tx=0).stop 2024-05-14T17:03:07.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.352+0000 7ff72c21d740 1 -- 172.21.15.2:0/1293324612 shutdown_connections 2024-05-14T17:03:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.352+0000 7ff72c21d740 1 --2- 172.21.15.2:0/1293324612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558fdb4ef170 0x558fdb506b70 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.352+0000 7ff72c21d740 1 --2- 172.21.15.2:0/1293324612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fdb4dce30 0x558fdb4e1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.352+0000 7ff72c21d740 1 --2- 172.21.15.2:0/1293324612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fdb4dbcf0 0x558fdb4d6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.352+0000 7ff72c21d740 1 -- 172.21.15.2:0/1293324612 >> 172.21.15.2:0/1293324612 conn(0x558fdb4d7e00 msgr2=0x558fdb4d6870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:07.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.352+0000 7ff72c21d740 1 -- 172.21.15.2:0/1293324612 shutdown_connections 2024-05-14T17:03:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.352+0000 7ff72c21d740 1 -- 172.21.15.2:0/1293324612 wait complete. 2024-05-14T17:03:07.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.352+0000 7ff72c21d740 1 -- start start 2024-05-14T17:03:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.353+0000 7ff72c21d740 1 Processor -- start 2024-05-14T17:03:07.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.353+0000 7ff72c21d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558fdb4dbcf0 0x558fdb4f6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.353+0000 7ff72c21d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fdb4dce30 0x558fdb4f74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.353+0000 7ff72c21d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fdb4ef170 0x558fdb4f3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.353+0000 7ff72c21d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff7241b3230 con 0x558fdb4ef170 2024-05-14T17:03:07.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.353+0000 7ff72c21d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff7241b46a0 con 0x558fdb4dce30 2024-05-14T17:03:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.353+0000 7ff72c21d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff7241b3900 con 0x558fdb4dbcf0 2024-05-14T17:03:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.353+0000 7ff72c21d740 10 client.0 _ll_register_callbacks cb 0x7ff70c0a94a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:03:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.353+0000 7ff7227fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fdb4dce30 0x558fdb4f74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.353+0000 7ff7227fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fdb4dce30 0x558fdb4f74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:52680/0 (socket says 172.21.15.2:52680) 2024-05-14T17:03:07.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.353+0000 7ff7227fc640 1 -- 172.21.15.2:0/2904640086 learned_addr learned my addr 172.21.15.2:0/2904640086 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:03:07.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.353+0000 7ff722ffd640 1 --2- 172.21.15.2:0/2904640086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558fdb4dbcf0 0x558fdb4f6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.353+0000 7ff7237fe640 1 --2- 172.21.15.2:0/2904640086 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fdb4ef170 0x558fdb4f3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:07.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.354+0000 7ff7227fc640 1 -- 172.21.15.2:0/2904640086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558fdb4dbcf0 msgr2=0x558fdb4f6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.354+0000 7ff7227fc640 1 --2- 172.21.15.2:0/2904640086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558fdb4dbcf0 0x558fdb4f6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.354+0000 7ff7227fc640 1 -- 172.21.15.2:0/2904640086 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fdb4ef170 msgr2=0x558fdb4f3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.354+0000 7ff7227fc640 1 --2- 172.21.15.2:0/2904640086 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fdb4ef170 0x558fdb4f3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.354+0000 7ff7227fc640 1 -- 172.21.15.2:0/2904640086 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fdb4f3c10 con 0x558fdb4dce30 2024-05-14T17:03:07.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.354+0000 7ff722ffd640 1 --2- 172.21.15.2:0/2904640086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558fdb4dbcf0 0x558fdb4f6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:03:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.354+0000 7ff7227fc640 1 --2- 172.21.15.2:0/2904640086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fdb4dce30 0x558fdb4f74b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff714009900 tx=0x7ff714009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ad6c5bd6945e0ee server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.355+0000 7ff70a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T17:03:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.355+0000 7ff70a7fc640 1 -- 172.21.15.2:0/2904640086 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff714054030 con 0x558fdb4dce30 2024-05-14T17:03:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.355+0000 7ff70a7fc640 1 -- 172.21.15.2:0/2904640086 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8513+0+0 (secure 0 0 0) 0x7ff714004240 con 0x558fdb4dce30 2024-05-14T17:03:07.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.355+0000 7ff72c21d740 1 -- 172.21.15.2:0/2904640086 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff71c02c070 con 0x558fdb4dce30 2024-05-14T17:03:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.355+0000 7ff70a7fc640 1 -- 172.21.15.2:0/2904640086 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff71405f050 con 0x558fdb4dce30 2024-05-14T17:03:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.355+0000 7ff72c21d740 10 client.25931 fetch_fsmap learned FSMap version 41 2024-05-14T17:03:07.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.355+0000 7ff72c21d740 1 -- 172.21.15.2:0/2904640086 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558fdb56a190 con 0x558fdb4dce30 2024-05-14T17:03:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.355+0000 7ff70a7fc640 1 -- 172.21.15.2:0/2904640086 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff71c02c070 con 0x558fdb4dce30 2024-05-14T17:03:07.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.356+0000 7ff70a7fc640 1 -- 172.21.15.2:0/2904640086 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7ff714003710 con 0x558fdb4dce30 2024-05-14T17:03:07.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.356+0000 7ff72c21d740 10 client.25931 fetch_fsmap finished waiting for FSMap version 41 2024-05-14T17:03:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.356+0000 7ff72c21d740 10 client.25931 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:03:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.356+0000 7ff72c21d740 20 client.25931 populate_metadata read hostname 'smithi002' 2024-05-14T17:03:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.356+0000 7ff72c21d740 1 --2- 172.21.15.2:0/2904640086 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x558fdb56a590 0x558fdb56c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.356+0000 7ff72c21d740 4 client.25931 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:03:08.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.356+0000 7ff72c21d740 1 -- 172.21.15.2:0/2904640086 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558fdb44a360 con 0x558fdb56a590 2024-05-14T17:03:08.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.356+0000 7ff722ffd640 1 --2- 172.21.15.2:0/2904640086 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x558fdb56a590 0x558fdb56c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:08.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.356+0000 7ff722ffd640 1 --2- 172.21.15.2:0/2904640086 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x558fdb56a590 0x558fdb56c980 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b46e849235cc101e server_cookie=ab247672a1299156 in_seq=0 out_seq=0 2024-05-14T17:03:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.356+0000 7ff70a7fc640 10 client.25931 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:03:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.365+0000 7ff70a7fc640 1 -- 172.21.15.2:0/2904640086 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558fdb44a360 con 0x558fdb56a590 2024-05-14T17:03:08.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.365+0000 7ff70a7fc640 10 client.25931 handle_command_reply: tid=0 2024-05-14T17:03:08.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.365+0000 7ff72c21d740 1 client.25931 shutdown 2024-05-14T17:03:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.366+0000 7ff72c21d740 1 -- 172.21.15.2:0/2904640086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fdb4dce30 msgr2=0x558fdb4f74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.366+0000 7ff72c21d740 1 --2- 172.21.15.2:0/2904640086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fdb4dce30 0x558fdb4f74b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff714009900 tx=0x7ff714009260 comp rx=0 tx=0).stop 2024-05-14T17:03:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.366+0000 7ff72c21d740 1 -- 172.21.15.2:0/2904640086 shutdown_connections 2024-05-14T17:03:08.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.366+0000 7ff72c21d740 1 --2- 172.21.15.2:0/2904640086 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x558fdb56a590 0x558fdb56c980 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.366+0000 7ff70a7fc640 0 client.25931 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:03:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.366+0000 7ff72c21d740 1 --2- 172.21.15.2:0/2904640086 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558fdb4ef170 0x558fdb4f3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.366+0000 7ff72c21d740 1 --2- 172.21.15.2:0/2904640086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fdb4dce30 0x558fdb4f74b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.366+0000 7ff72c21d740 1 --2- 172.21.15.2:0/2904640086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558fdb4dbcf0 0x558fdb4f6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.366+0000 7ff72c21d740 1 -- 172.21.15.2:0/2904640086 >> 172.21.15.2:0/2904640086 conn(0x558fdb4d7e00 msgr2=0x558fdb4d7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:08.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.366+0000 7ff72c21d740 1 -- 172.21.15.2:0/2904640086 shutdown_connections 2024-05-14T17:03:08.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.366+0000 7ff72c21d740 1 -- 172.21.15.2:0/2904640086 wait complete. 2024-05-14T17:03:08.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.366+0000 7ff72c21d740 20 client.25931 trim_cache size 0 max 16384 2024-05-14T17:03:08.284 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:03:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:03:08.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:03:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.367+0000 7ff729423640 1 -- 172.21.15.2:0/1311572849 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ff7100b2a10 msgr2=0x7ff7100b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.367+0000 7ff729423640 1 --2- 172.21.15.2:0/1311572849 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ff7100b2a10 0x7ff7100b4ed0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff71c009fd0 tx=0x7ff71c009500 comp rx=0 tx=0).stop 2024-05-14T17:03:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.367+0000 7ff729423640 1 -- 172.21.15.2:0/1311572849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff72410b3f0 msgr2=0x7ff7241a7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.367+0000 7ff729423640 1 --2- 172.21.15.2:0/1311572849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff72410b3f0 0x7ff7241a7610 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff70c00e9f0 tx=0x7ff70c00eeb0 comp rx=0 tx=0).stop 2024-05-14T17:03:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.367+0000 7ff729423640 1 -- 172.21.15.2:0/1311572849 shutdown_connections 2024-05-14T17:03:08.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.367+0000 7ff729423640 1 --2- 172.21.15.2:0/1311572849 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ff7100b2a10 0x7ff7100b4ed0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:08.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.367+0000 7ff729423640 1 --2- 172.21.15.2:0/1311572849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7241149a0 0x7ff7241a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.367+0000 7ff729423640 1 --2- 172.21.15.2:0/1311572849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff72410bd10 0x7ff7241a7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.367+0000 7ff729423640 1 --2- 172.21.15.2:0/1311572849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff72410b3f0 0x7ff7241a7610 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:08.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.367+0000 7ff729423640 1 -- 172.21.15.2:0/1311572849 >> 172.21.15.2:0/1311572849 conn(0x7ff72406dfb0 msgr2=0x7ff72410a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:08.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.367+0000 7ff729423640 1 -- 172.21.15.2:0/1311572849 shutdown_connections 2024-05-14T17:03:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:05.367+0000 7ff729423640 1 -- 172.21.15.2:0/1311572849 wait complete. 2024-05-14T17:03:08.485 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 470.0 s, 2079.5 tps, lat 55.007 ms stddev 408.612 2024-05-14T17:03:08.498 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:03:06 smithi002 ceph-mon[31538]: pgmap v259: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 987 KiB/s wr, 126 op/s 2024-05-14T17:03:08.510 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:03:06 smithi123 ceph-mon[35971]: pgmap v259: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 987 KiB/s wr, 126 op/s 2024-05-14T17:03:08.524 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:03:06 smithi151 ceph-mon[36066]: pgmap v259: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 987 KiB/s wr, 126 op/s 2024-05-14T17:03:08.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:03:08 smithi151 ceph-mon[36066]: pgmap v260: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 987 KiB/s wr, 126 op/s 2024-05-14T17:03:08.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:03:08 smithi123 ceph-mon[35971]: pgmap v260: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 987 KiB/s wr, 126 op/s 2024-05-14T17:03:08.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:03:08 smithi002 ceph-mon[31538]: pgmap v260: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 987 KiB/s wr, 126 op/s 2024-05-14T17:03:09.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:03:09 smithi151 ceph-mon[36066]: osdmap e86: 12 total, 12 up, 12 in 2024-05-14T17:03:09.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:03:09 smithi123 ceph-mon[35971]: osdmap e86: 12 total, 12 up, 12 in 2024-05-14T17:03:09.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:03:09 smithi002 ceph-mon[31538]: osdmap e86: 12 total, 12 up, 12 in 2024-05-14T17:03:10.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:03:10 smithi151 ceph-mon[36066]: pgmap v262: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.1 MiB/s wr, 463 op/s 2024-05-14T17:03:10.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:03:10 smithi123 ceph-mon[35971]: pgmap v262: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.1 MiB/s wr, 463 op/s 2024-05-14T17:03:10.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:03:10 smithi002 ceph-mon[31538]: pgmap v262: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.1 MiB/s wr, 463 op/s 2024-05-14T17:03:11.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 475.0 s, 2558.4 tps, lat 39.161 ms stddev 47.929 2024-05-14T17:03:12.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:03:12 smithi151 ceph-mon[36066]: pgmap v263: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.1 MiB/s wr, 463 op/s 2024-05-14T17:03:12.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:03:12 smithi123 ceph-mon[35971]: pgmap v263: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.1 MiB/s wr, 463 op/s 2024-05-14T17:03:12.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:03:12 smithi002 ceph-mon[31538]: pgmap v263: 97 pgs: 97 active+clean; 4.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.1 MiB/s wr, 463 op/s 2024-05-14T17:03:14.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:03:14 smithi151 ceph-mon[36066]: pgmap v264: 97 pgs: 97 active+clean; 4.0 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 17 MiB/s wr, 1.68k op/s 2024-05-14T17:03:15.343 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:03:14 smithi002 ceph-mon[31538]: pgmap v264: 97 pgs: 97 active+clean; 4.0 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 17 MiB/s wr, 1.68k op/s 2024-05-14T17:03:15.415 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:03:14 smithi123 ceph-mon[35971]: pgmap v264: 97 pgs: 97 active+clean; 4.0 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 17 MiB/s wr, 1.68k op/s 2024-05-14T17:03:16.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 480.0 s, 2484.4 tps, lat 40.139 ms stddev 60.342 2024-05-14T17:03:16.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:03:16 smithi151 ceph-mon[36066]: pgmap v265: 97 pgs: 97 active+clean; 4.0 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 17 MiB/s wr, 1.68k op/s 2024-05-14T17:03:16.866 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:03:16 smithi002 ceph-mon[31538]: pgmap v265: 97 pgs: 97 active+clean; 4.0 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 17 MiB/s wr, 1.68k op/s 2024-05-14T17:03:16.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:03:16 smithi123 ceph-mon[35971]: pgmap v265: 97 pgs: 97 active+clean; 4.0 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 17 MiB/s wr, 1.68k op/s 2024-05-14T17:03:18.409 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:03:18 smithi123 ceph-mon[35971]: osdmap e87: 12 total, 12 up, 12 in 2024-05-14T17:03:18.421 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:03:18 smithi123 ceph-mon[35971]: pgmap v267: 97 pgs: 97 active+clean; 4.0 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 16 MiB/s wr, 1.54k op/s 2024-05-14T17:03:18.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:03:18 smithi002 ceph-mon[31538]: osdmap e87: 12 total, 12 up, 12 in 2024-05-14T17:03:18.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:03:18 smithi002 ceph-mon[31538]: pgmap v267: 97 pgs: 97 active+clean; 4.0 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 16 MiB/s wr, 1.54k op/s 2024-05-14T17:03:18.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.510+0000 7f89794a7640 1 -- 172.21.15.2:0/3549930661 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8974107940 msgr2=0x7f8974107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:18.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.510+0000 7f89794a7640 1 --2- 172.21.15.2:0/3549930661 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8974107940 0x7f8974107d20 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8964009930 tx=0x7f896402a420 comp rx=0 tx=0).stop 2024-05-14T17:03:18.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.510+0000 7f89794a7640 1 -- 172.21.15.2:0/3549930661 shutdown_connections 2024-05-14T17:03:18.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.510+0000 7f89794a7640 1 --2- 172.21.15.2:0/3549930661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8974102340 0x7f897410e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.511+0000 7f89794a7640 1 --2- 172.21.15.2:0/3549930661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8974101980 0x7f8974101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.511+0000 7f89794a7640 1 --2- 172.21.15.2:0/3549930661 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8974107940 0x7f8974107d20 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:18.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.511+0000 7f89794a7640 1 -- 172.21.15.2:0/3549930661 >> 172.21.15.2:0/3549930661 conn(0x7f8974078ad0 msgr2=0x7f8974078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:18.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.511+0000 7f89794a7640 1 -- 172.21.15.2:0/3549930661 shutdown_connections 2024-05-14T17:03:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.511+0000 7f89794a7640 1 -- 172.21.15.2:0/3549930661 wait complete. 2024-05-14T17:03:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.511+0000 7f89794a7640 1 Processor -- start 2024-05-14T17:03:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.512+0000 7f89794a7640 1 -- start start 2024-05-14T17:03:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.512+0000 7f89794a7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8974101980 0x7f897419b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.512+0000 7f89794a7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8974102340 0x7f897419b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.512+0000 7f8972ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8974101980 0x7f897419b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:18.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.512+0000 7f89794a7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8974107940 0x7f897419fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.512+0000 7f89794a7640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8974111c00 con 0x7f8974101980 2024-05-14T17:03:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.512+0000 7f89794a7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8974111a80 con 0x7f8974102340 2024-05-14T17:03:18.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.512+0000 7f89794a7640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8974111d80 con 0x7f8974107940 2024-05-14T17:03:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.512+0000 7f8972ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8974101980 0x7f897419b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44918/0 (socket says 172.21.15.2:44918) 2024-05-14T17:03:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.512+0000 7f8972ffd640 1 -- 172.21.15.2:0/3453403479 learned_addr learned my addr 172.21.15.2:0/3453403479 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:03:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.512+0000 7f89737fe640 1 --2- 172.21.15.2:0/3453403479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8974107940 0x7f897419fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.512+0000 7f896bfff640 1 --2- 172.21.15.2:0/3453403479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8974102340 0x7f897419b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.512+0000 7f8972ffd640 1 -- 172.21.15.2:0/3453403479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8974107940 msgr2=0x7f897419fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:18.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.512+0000 7f8972ffd640 1 --2- 172.21.15.2:0/3453403479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8974107940 0x7f897419fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.512+0000 7f8972ffd640 1 -- 172.21.15.2:0/3453403479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8974102340 msgr2=0x7f897419b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.512+0000 7f8972ffd640 1 --2- 172.21.15.2:0/3453403479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8974102340 0x7f897419b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:18.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.512+0000 7f8972ffd640 1 -- 172.21.15.2:0/3453403479 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89741a0370 con 0x7f8974101980 2024-05-14T17:03:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.512+0000 7f896bfff640 1 --2- 172.21.15.2:0/3453403479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8974102340 0x7f897419b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:03:18.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.513+0000 7f8972ffd640 1 --2- 172.21.15.2:0/3453403479 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8974101980 0x7f897419b380 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8964005940 tx=0x7f89640040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:18.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.513+0000 7f8970ff9640 1 -- 172.21.15.2:0/3453403479 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896403f030 con 0x7f8974101980 2024-05-14T17:03:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.513+0000 7f8970ff9640 1 -- 172.21.15.2:0/3453403479 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89640044c0 con 0x7f8974101980 2024-05-14T17:03:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.513+0000 7f89794a7640 1 -- 172.21.15.2:0/3453403479 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89741a0600 con 0x7f8974101980 2024-05-14T17:03:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.513+0000 7f8970ff9640 1 -- 172.21.15.2:0/3453403479 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8964035070 con 0x7f8974101980 2024-05-14T17:03:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.513+0000 7f89794a7640 1 -- 172.21.15.2:0/3453403479 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89741a7ef0 con 0x7f8974101980 2024-05-14T17:03:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.518+0000 7f8970ff9640 1 -- 172.21.15.2:0/3453403479 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f8964042030 con 0x7f8974101980 2024-05-14T17:03:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.518+0000 7f897c1c8740 1 Processor -- start 2024-05-14T17:03:19.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.518+0000 7f897c1c8740 1 -- start start 2024-05-14T17:03:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.518+0000 7f8970ff9640 1 --2- 172.21.15.2:0/3453403479 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f89600b2a90 0x7f89600b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.519+0000 7f89737fe640 1 --2- 172.21.15.2:0/3453403479 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f89600b2a90 0x7f89600b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:19.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.519+0000 7f897c1c8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55980adfb8f0 0x55980adfbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.519+0000 7f897c1c8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55980adfc210 0x55980ae06590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.519+0000 7f8970ff9640 1 -- 172.21.15.2:0/3453403479 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f89640c5720 con 0x7f8974101980 2024-05-14T17:03:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.519+0000 7f897c1c8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980ae06ad0 0x55980ae08ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.519+0000 7f897c1c8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55980ad6ec30 con 0x55980adfb8f0 2024-05-14T17:03:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.519+0000 7f8972ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55980adfb8f0 0x55980adfbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:19.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.519+0000 7f897c1c8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55980ad5e7d0 con 0x55980adfc210 2024-05-14T17:03:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.519+0000 7f897c1c8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55980ad62810 con 0x55980ae06ad0 2024-05-14T17:03:19.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.519+0000 7f89737fe640 1 --2- 172.21.15.2:0/3453403479 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f89600b2a90 0x7f89600b4f50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f896c005fd0 tx=0x7f896c009500 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.519+0000 7f89737fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980ae06ad0 0x55980ae08ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.519+0000 7f8972ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55980adfb8f0 0x55980adfbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44928/0 (socket says 172.21.15.2:44928) 2024-05-14T17:03:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.519+0000 7f89737fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980ae06ad0 0x55980ae08ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:59398/0 (socket says 172.21.15.2:59398) 2024-05-14T17:03:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.519+0000 7f8972ffd640 1 -- 172.21.15.2:0/113415208 learned_addr learned my addr 172.21.15.2:0/113415208 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:03:19.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.519+0000 7f896bfff640 1 --2- 172.21.15.2:0/113415208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55980adfc210 0x55980ae06590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.519+0000 7f89737fe640 1 -- 172.21.15.2:0/113415208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55980adfc210 msgr2=0x55980ae06590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:19.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.519+0000 7f89737fe640 1 --2- 172.21.15.2:0/113415208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55980adfc210 0x55980ae06590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.519+0000 7f89737fe640 1 -- 172.21.15.2:0/113415208 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55980adfb8f0 msgr2=0x55980adfbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.520+0000 7f89737fe640 1 --2- 172.21.15.2:0/113415208 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55980adfb8f0 0x55980adfbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.520+0000 7f89737fe640 1 -- 172.21.15.2:0/113415208 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55980aa73210 con 0x55980ae06ad0 2024-05-14T17:03:19.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.520+0000 7f896bfff640 1 --2- 172.21.15.2:0/113415208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55980adfc210 0x55980ae06590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:03:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.520+0000 7f8972ffd640 1 --2- 172.21.15.2:0/113415208 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55980adfb8f0 0x55980adfbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:03:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.520+0000 7f89737fe640 1 --2- 172.21.15.2:0/113415208 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980ae06ad0 0x55980ae08ec0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f896c003b00 tx=0x7f896c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=100ac916a804f61c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.520+0000 7f89697fa640 1 -- 172.21.15.2:0/113415208 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896c022070 con 0x55980ae06ad0 2024-05-14T17:03:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.520+0000 7f89697fa640 1 -- 172.21.15.2:0/113415208 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f896c004750 con 0x55980ae06ad0 2024-05-14T17:03:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.520+0000 7f89697fa640 1 -- 172.21.15.2:0/113415208 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896c01e3f0 con 0x55980ae06ad0 2024-05-14T17:03:19.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.521+0000 7f897c1c8740 1 -- 172.21.15.2:0/113415208 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980ae06ad0 msgr2=0x55980ae08ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.521+0000 7f897c1c8740 1 --2- 172.21.15.2:0/113415208 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980ae06ad0 0x55980ae08ec0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f896c003b00 tx=0x7f896c014000 comp rx=0 tx=0).stop 2024-05-14T17:03:19.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.521+0000 7f897c1c8740 1 -- 172.21.15.2:0/113415208 shutdown_connections 2024-05-14T17:03:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.521+0000 7f897c1c8740 1 --2- 172.21.15.2:0/113415208 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980ae06ad0 0x55980ae08ec0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:19.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.521+0000 7f897c1c8740 1 --2- 172.21.15.2:0/113415208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55980adfc210 0x55980ae06590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.521+0000 7f897c1c8740 1 --2- 172.21.15.2:0/113415208 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55980adfb8f0 0x55980adfbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.521+0000 7f897c1c8740 1 -- 172.21.15.2:0/113415208 >> 172.21.15.2:0/113415208 conn(0x55980adf7e00 msgr2=0x55980adf81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.521+0000 7f897c1c8740 1 -- 172.21.15.2:0/113415208 shutdown_connections 2024-05-14T17:03:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.522+0000 7f897c1c8740 1 -- 172.21.15.2:0/113415208 wait complete. 2024-05-14T17:03:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.522+0000 7f897c1c8740 1 -- start start 2024-05-14T17:03:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.522+0000 7f897c1c8740 1 Processor -- start 2024-05-14T17:03:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.522+0000 7f897c1c8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55980adfb8f0 0x55980ae06450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.522+0000 7f897c1c8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55980adfc210 0x55980ae03740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:19.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.522+0000 7f897c1c8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980ae06ad0 0x55980ae03cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:19.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.522+0000 7f897c1c8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55980aa44790 con 0x55980adfb8f0 2024-05-14T17:03:19.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.522+0000 7f897c1c8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55980ad1a890 con 0x55980adfc210 2024-05-14T17:03:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.522+0000 7f897c1c8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55980ae097a0 con 0x55980ae06ad0 2024-05-14T17:03:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.522+0000 7f897c1c8740 10 client.0 _ll_register_callbacks cb 0x55980ad1dd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:03:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.522+0000 7f8972ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55980adfb8f0 0x55980ae06450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.523+0000 7f896bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55980adfc210 0x55980ae03740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.523+0000 7f896bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55980adfc210 0x55980ae03740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:49282/0 (socket says 172.21.15.2:49282) 2024-05-14T17:03:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.523+0000 7f8972ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55980adfb8f0 0x55980ae06450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44930/0 (socket says 172.21.15.2:44930) 2024-05-14T17:03:19.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.523+0000 7f896bfff640 1 -- 172.21.15.2:0/438477947 learned_addr learned my addr 172.21.15.2:0/438477947 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:03:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.523+0000 7f89737fe640 1 --2- 172.21.15.2:0/438477947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980ae06ad0 0x55980ae03cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:19.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.523+0000 7f896bfff640 1 -- 172.21.15.2:0/438477947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980ae06ad0 msgr2=0x55980ae03cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.523+0000 7f896bfff640 1 --2- 172.21.15.2:0/438477947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980ae06ad0 0x55980ae03cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.523+0000 7f896bfff640 1 -- 172.21.15.2:0/438477947 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55980adfb8f0 msgr2=0x55980ae06450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.523+0000 7f896bfff640 1 --2- 172.21.15.2:0/438477947 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55980adfb8f0 0x55980ae06450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.523+0000 7f896bfff640 1 -- 172.21.15.2:0/438477947 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55980ab3f620 con 0x55980adfc210 2024-05-14T17:03:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.523+0000 7f8972ffd640 1 --2- 172.21.15.2:0/438477947 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55980adfb8f0 0x55980ae06450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:03:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.523+0000 7f896bfff640 1 --2- 172.21.15.2:0/438477947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55980adfc210 0x55980ae03740 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f895c013710 tx=0x7f895c013b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1ae9624934fcdae server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.523+0000 7f895a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T17:03:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.523+0000 7f895a7fc640 1 -- 172.21.15.2:0/438477947 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c011dd0 con 0x55980adfc210 2024-05-14T17:03:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.523+0000 7f897c1c8740 1 -- 172.21.15.2:0/438477947 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f896c004750 con 0x55980adfc210 2024-05-14T17:03:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.523+0000 7f895a7fc640 1 -- 172.21.15.2:0/438477947 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f895c004020 con 0x55980adfc210 2024-05-14T17:03:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.524+0000 7f895a7fc640 1 -- 172.21.15.2:0/438477947 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f895c004220 con 0x55980adfc210 2024-05-14T17:03:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.524+0000 7f897c1c8740 10 client.25940 fetch_fsmap learned FSMap version 41 2024-05-14T17:03:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.524+0000 7f897c1c8740 1 -- 172.21.15.2:0/438477947 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55980ae15280 con 0x55980adfc210 2024-05-14T17:03:20.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.524+0000 7f895a7fc640 1 -- 172.21.15.2:0/438477947 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f896c004750 con 0x55980adfc210 2024-05-14T17:03:20.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.524+0000 7f895a7fc640 1 -- 172.21.15.2:0/438477947 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f895c01a070 con 0x55980adfc210 2024-05-14T17:03:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.524+0000 7f897c1c8740 10 client.25940 fetch_fsmap finished waiting for FSMap version 41 2024-05-14T17:03:20.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.524+0000 7f897c1c8740 10 client.25940 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:03:20.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.525+0000 7f897c1c8740 20 client.25940 populate_metadata read hostname 'smithi002' 2024-05-14T17:03:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.525+0000 7f897c1c8740 1 --2- 172.21.15.2:0/438477947 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55980ae16850 0x55980ae18c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.525+0000 7f897c1c8740 4 client.25940 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:03:20.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.525+0000 7f897c1c8740 1 -- 172.21.15.2:0/438477947 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55980ad70b80 con 0x55980ae16850 2024-05-14T17:03:20.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.525+0000 7f8972ffd640 1 --2- 172.21.15.2:0/438477947 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55980ae16850 0x55980ae18c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.525+0000 7f895a7fc640 10 client.25940 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:03:20.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.525+0000 7f8972ffd640 1 --2- 172.21.15.2:0/438477947 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55980ae16850 0x55980ae18c70 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23eb068026440924 server_cookie=b18676839f864fcc in_seq=0 out_seq=0 2024-05-14T17:03:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.526+0000 7f895a7fc640 1 -- 172.21.15.2:0/438477947 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55980ad70b80 con 0x55980ae16850 2024-05-14T17:03:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.526+0000 7f895a7fc640 10 client.25940 handle_command_reply: tid=0 2024-05-14T17:03:20.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.526+0000 7f897c1c8740 1 client.25940 shutdown 2024-05-14T17:03:20.272 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:03:18 smithi151 ceph-mon[36066]: osdmap e87: 12 total, 12 up, 12 in 2024-05-14T17:03:20.298 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:03:18 smithi151 ceph-mon[36066]: pgmap v267: 97 pgs: 97 active+clean; 4.0 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 16 MiB/s wr, 1.54k op/s 2024-05-14T17:03:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.527+0000 7f897c1c8740 1 -- 172.21.15.2:0/438477947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55980adfc210 msgr2=0x55980ae03740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:20.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.528+0000 7f897c1c8740 1 --2- 172.21.15.2:0/438477947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55980adfc210 0x55980ae03740 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f895c013710 tx=0x7f895c013b10 comp rx=0 tx=0).stop 2024-05-14T17:03:20.378 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.529+0000 7f897c1c8740 1 -- 172.21.15.2:0/438477947 shutdown_connections 2024-05-14T17:03:20.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.529+0000 7f897c1c8740 1 --2- 172.21.15.2:0/438477947 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55980ae16850 0x55980ae18c70 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:20.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.529+0000 7f897c1c8740 1 --2- 172.21.15.2:0/438477947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980ae06ad0 0x55980ae03cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.529+0000 7f897c1c8740 1 --2- 172.21.15.2:0/438477947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55980adfc210 0x55980ae03740 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.529+0000 7f897c1c8740 1 --2- 172.21.15.2:0/438477947 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55980adfb8f0 0x55980ae06450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:20.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.529+0000 7f895a7fc640 0 client.25940 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:03:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.529+0000 7f897c1c8740 1 -- 172.21.15.2:0/438477947 >> 172.21.15.2:0/438477947 conn(0x55980adf7e00 msgr2=0x55980adf7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.529+0000 7f897c1c8740 1 -- 172.21.15.2:0/438477947 shutdown_connections 2024-05-14T17:03:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.529+0000 7f897c1c8740 1 -- 172.21.15.2:0/438477947 wait complete. 2024-05-14T17:03:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.529+0000 7f897c1c8740 20 client.25940 trim_cache size 0 max 16384 2024-05-14T17:03:20.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.554+0000 7f897c1c8740 1 Processor -- start 2024-05-14T17:03:20.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.554+0000 7f897c1c8740 1 -- start start 2024-05-14T17:03:20.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.554+0000 7f897c1c8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980adfb8f0 0x55980ae12670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:20.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.554+0000 7f897c1c8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55980adfc210 0x55980ae05f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:20.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.555+0000 7f897c1c8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55980ae06ad0 0x55980ae1e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.555+0000 7f897c1c8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f89741b3220 con 0x55980adfc210 2024-05-14T17:03:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.555+0000 7f8972ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980adfb8f0 0x55980ae12670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:20.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.555+0000 7f896bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55980adfc210 0x55980ae05f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.555+0000 7f8972ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980adfb8f0 0x55980ae12670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:59420/0 (socket says 172.21.15.2:59420) 2024-05-14T17:03:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.555+0000 7f8972ffd640 1 -- 172.21.15.2:0/1278788965 learned_addr learned my addr 172.21.15.2:0/1278788965 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:03:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.555+0000 7f896bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55980adfc210 0x55980ae05f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:44946/0 (socket says 172.21.15.2:44946) 2024-05-14T17:03:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.555+0000 7f897c1c8740 1 -- 172.21.15.2:0/1278788965 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f89741b0af0 con 0x55980ae06ad0 2024-05-14T17:03:20.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.555+0000 7f897c1c8740 1 -- 172.21.15.2:0/1278788965 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f89741ae960 con 0x55980adfb8f0 2024-05-14T17:03:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.555+0000 7f89737fe640 1 --2- 172.21.15.2:0/1278788965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55980ae06ad0 0x55980ae1e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.555+0000 7f89737fe640 1 -- 172.21.15.2:0/1278788965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980adfb8f0 msgr2=0x55980ae12670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.555+0000 7f89737fe640 1 --2- 172.21.15.2:0/1278788965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980adfb8f0 0x55980ae12670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.555+0000 7f89737fe640 1 -- 172.21.15.2:0/1278788965 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55980adfc210 msgr2=0x55980ae05f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:20.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.555+0000 7f89737fe640 1 --2- 172.21.15.2:0/1278788965 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55980adfc210 0x55980ae05f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.555+0000 7f89737fe640 1 -- 172.21.15.2:0/1278788965 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55980adf2510 con 0x55980ae06ad0 2024-05-14T17:03:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.556+0000 7f89737fe640 1 --2- 172.21.15.2:0/1278788965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55980ae06ad0 0x55980ae1e540 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f896c011ca0 tx=0x7f896c005760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=511d6d3781665258 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:20.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.556+0000 7f8968ff9640 1 -- 172.21.15.2:0/1278788965 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896c022070 con 0x55980ae06ad0 2024-05-14T17:03:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.556+0000 7f8968ff9640 1 -- 172.21.15.2:0/1278788965 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f896c005850 con 0x55980ae06ad0 2024-05-14T17:03:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.556+0000 7f8968ff9640 1 -- 172.21.15.2:0/1278788965 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896c026660 con 0x55980ae06ad0 2024-05-14T17:03:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.557+0000 7f897c1c8740 1 -- 172.21.15.2:0/1278788965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55980ae06ad0 msgr2=0x55980ae1e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.557+0000 7f897c1c8740 1 --2- 172.21.15.2:0/1278788965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55980ae06ad0 0x55980ae1e540 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f896c011ca0 tx=0x7f896c005760 comp rx=0 tx=0).stop 2024-05-14T17:03:20.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.557+0000 7f897c1c8740 1 -- 172.21.15.2:0/1278788965 shutdown_connections 2024-05-14T17:03:20.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.557+0000 7f897c1c8740 1 --2- 172.21.15.2:0/1278788965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55980ae06ad0 0x55980ae1e540 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.557+0000 7f897c1c8740 1 --2- 172.21.15.2:0/1278788965 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55980adfc210 0x55980ae05f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.557+0000 7f897c1c8740 1 --2- 172.21.15.2:0/1278788965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980adfb8f0 0x55980ae12670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:20.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.557+0000 7f897c1c8740 1 -- 172.21.15.2:0/1278788965 >> 172.21.15.2:0/1278788965 conn(0x55980adf7e00 msgr2=0x55980adfedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.558+0000 7f897c1c8740 1 -- 172.21.15.2:0/1278788965 shutdown_connections 2024-05-14T17:03:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.558+0000 7f897c1c8740 1 -- 172.21.15.2:0/1278788965 wait complete. 2024-05-14T17:03:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.558+0000 7f897c1c8740 1 -- start start 2024-05-14T17:03:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.558+0000 7f897c1c8740 1 Processor -- start 2024-05-14T17:03:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.559+0000 7f897c1c8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55980adfb8f0 0x55980ae0e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:21.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.559+0000 7f897c1c8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980adfc210 0x55980ae0ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.559+0000 7f897c1c8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55980ae06ad0 0x55980ae09d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.559+0000 7f897c1c8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f89741aefb0 con 0x55980ae06ad0 2024-05-14T17:03:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.559+0000 7f897c1c8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f89741b0420 con 0x55980adfb8f0 2024-05-14T17:03:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.559+0000 7f896bfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980adfc210 0x55980ae0ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.559+0000 7f897c1c8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f89741af680 con 0x55980adfc210 2024-05-14T17:03:21.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.559+0000 7f896bfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980adfc210 0x55980ae0ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:59424/0 (socket says 172.21.15.2:59424) 2024-05-14T17:03:21.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.559+0000 7f896bfff640 1 -- 172.21.15.2:0/1399004221 learned_addr learned my addr 172.21.15.2:0/1399004221 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:03:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.559+0000 7f897c1c8740 10 client.0 _ll_register_callbacks cb 0x7f895c009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:03:21.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.559+0000 7f8972ffd640 1 --2- 172.21.15.2:0/1399004221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55980adfb8f0 0x55980ae0e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:21.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.559+0000 7f89737fe640 1 --2- 172.21.15.2:0/1399004221 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55980ae06ad0 0x55980ae09d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:21.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.559+0000 7f896bfff640 1 -- 172.21.15.2:0/1399004221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55980adfb8f0 msgr2=0x55980ae0e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:21.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.559+0000 7f896bfff640 1 --2- 172.21.15.2:0/1399004221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55980adfb8f0 0x55980ae0e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.559+0000 7f896bfff640 1 -- 172.21.15.2:0/1399004221 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55980ae06ad0 msgr2=0x55980ae09d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:21.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.559+0000 7f896bfff640 1 --2- 172.21.15.2:0/1399004221 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55980ae06ad0 0x55980ae09d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.559+0000 7f896bfff640 1 -- 172.21.15.2:0/1399004221 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55980ae0a580 con 0x55980adfc210 2024-05-14T17:03:21.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.559+0000 7f896bfff640 1 --2- 172.21.15.2:0/1399004221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980adfc210 0x55980ae0ed60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f895c023750 tx=0x7f895c005990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d6f14a5d9ae506c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:21.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.559+0000 7f895a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-14T17:03:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.560+0000 7f895a7fc640 1 -- 172.21.15.2:0/1399004221 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c005c50 con 0x55980adfc210 2024-05-14T17:03:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.560+0000 7f897c1c8740 1 -- 172.21.15.2:0/1399004221 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f896c005850 con 0x55980adfc210 2024-05-14T17:03:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.560+0000 7f895a7fc640 1 -- 172.21.15.2:0/1399004221 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f895c0239e0 con 0x55980adfc210 2024-05-14T17:03:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.560+0000 7f895a7fc640 1 -- 172.21.15.2:0/1399004221 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f895c039050 con 0x55980adfc210 2024-05-14T17:03:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.560+0000 7f897c1c8740 10 client.25930 fetch_fsmap learned FSMap version 41 2024-05-14T17:03:21.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.560+0000 7f897c1c8740 1 -- 172.21.15.2:0/1399004221 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55980ae80aa0 con 0x55980adfc210 2024-05-14T17:03:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.560+0000 7f895a7fc640 1 -- 172.21.15.2:0/1399004221 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f896c005850 con 0x55980adfc210 2024-05-14T17:03:21.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.560+0000 7f895a7fc640 1 -- 172.21.15.2:0/1399004221 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f895c01a050 con 0x55980adfc210 2024-05-14T17:03:21.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.560+0000 7f897c1c8740 10 client.25930 fetch_fsmap finished waiting for FSMap version 41 2024-05-14T17:03:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.560+0000 7f897c1c8740 10 client.25930 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:03:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.560+0000 7f897c1c8740 20 client.25930 populate_metadata read hostname 'smithi002' 2024-05-14T17:03:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.561+0000 7f897c1c8740 1 --2- 172.21.15.2:0/1399004221 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55980ae812a0 0x55980ae836c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.561+0000 7f897c1c8740 4 client.25930 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:03:21.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.561+0000 7f897c1c8740 1 -- 172.21.15.2:0/1399004221 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55980ad6a360 con 0x55980ae812a0 2024-05-14T17:03:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.561+0000 7f8972ffd640 1 --2- 172.21.15.2:0/1399004221 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55980ae812a0 0x55980ae836c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.563+0000 7f8972ffd640 1 --2- 172.21.15.2:0/1399004221 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55980ae812a0 0x55980ae836c0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a78d888e5bccdc1a server_cookie=e532ae38afd10311 in_seq=0 out_seq=0 2024-05-14T17:03:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.563+0000 7f895a7fc640 10 client.25930 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:03:21.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.573+0000 7f895a7fc640 1 -- 172.21.15.2:0/1399004221 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55980ad6a360 con 0x55980ae812a0 2024-05-14T17:03:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.573+0000 7f895a7fc640 10 client.25930 handle_command_reply: tid=0 2024-05-14T17:03:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.573+0000 7f897c1c8740 1 client.25930 shutdown 2024-05-14T17:03:21.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.574+0000 7f897c1c8740 1 -- 172.21.15.2:0/1399004221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980adfc210 msgr2=0x55980ae0ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:21.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.574+0000 7f897c1c8740 1 --2- 172.21.15.2:0/1399004221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980adfc210 0x55980ae0ed60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f895c023750 tx=0x7f895c005990 comp rx=0 tx=0).stop 2024-05-14T17:03:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.574+0000 7f897c1c8740 1 -- 172.21.15.2:0/1399004221 shutdown_connections 2024-05-14T17:03:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.574+0000 7f897c1c8740 1 --2- 172.21.15.2:0/1399004221 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55980ae812a0 0x55980ae836c0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.574+0000 7f897c1c8740 1 --2- 172.21.15.2:0/1399004221 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55980ae06ad0 0x55980ae09d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.574+0000 7f897c1c8740 1 --2- 172.21.15.2:0/1399004221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55980adfc210 0x55980ae0ed60 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:21.788 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.574+0000 7f895a7fc640 0 client.25930 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:03:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.574+0000 7f897c1c8740 1 --2- 172.21.15.2:0/1399004221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55980adfb8f0 0x55980ae0e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.574+0000 7f897c1c8740 1 -- 172.21.15.2:0/1399004221 >> 172.21.15.2:0/1399004221 conn(0x55980adf7e00 msgr2=0x55980ae11660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.574+0000 7f897c1c8740 1 -- 172.21.15.2:0/1399004221 shutdown_connections 2024-05-14T17:03:21.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.574+0000 7f897c1c8740 1 -- 172.21.15.2:0/1399004221 wait complete. 2024-05-14T17:03:21.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.574+0000 7f897c1c8740 20 client.25930 trim_cache size 0 max 16384 2024-05-14T17:03:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:03:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:03:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:03:21.912 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:03:20 smithi123 ceph-mon[35971]: pgmap v268: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 17 MiB/s wr, 1.71k op/s 2024-05-14T17:03:21.921 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:03:20 smithi151 ceph-mon[36066]: pgmap v268: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 17 MiB/s wr, 1.71k op/s 2024-05-14T17:03:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.574+0000 7f89794a7640 1 -- 172.21.15.2:0/3453403479 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f89600b2a90 msgr2=0x7f89600b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.574+0000 7f89794a7640 1 --2- 172.21.15.2:0/3453403479 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f89600b2a90 0x7f89600b4f50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f896c005fd0 tx=0x7f896c009500 comp rx=0 tx=0).stop 2024-05-14T17:03:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.574+0000 7f89794a7640 1 -- 172.21.15.2:0/3453403479 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8974101980 msgr2=0x7f897419b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:22.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.574+0000 7f89794a7640 1 --2- 172.21.15.2:0/3453403479 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8974101980 0x7f897419b380 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8964005940 tx=0x7f89640040c0 comp rx=0 tx=0).stop 2024-05-14T17:03:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.575+0000 7f89794a7640 1 -- 172.21.15.2:0/3453403479 shutdown_connections 2024-05-14T17:03:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.575+0000 7f89794a7640 1 --2- 172.21.15.2:0/3453403479 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f89600b2a90 0x7f89600b4f50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.575+0000 7f89794a7640 1 --2- 172.21.15.2:0/3453403479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8974107940 0x7f897419fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:22.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.575+0000 7f89794a7640 1 --2- 172.21.15.2:0/3453403479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8974102340 0x7f897419b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:22.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.575+0000 7f89794a7640 1 --2- 172.21.15.2:0/3453403479 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8974101980 0x7f897419b380 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.575+0000 7f89794a7640 1 -- 172.21.15.2:0/3453403479 >> 172.21.15.2:0/3453403479 conn(0x7f8974078ad0 msgr2=0x7f897406a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.575+0000 7f89794a7640 1 -- 172.21.15.2:0/3453403479 shutdown_connections 2024-05-14T17:03:22.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:18.575+0000 7f89794a7640 1 -- 172.21.15.2:0/3453403479 wait complete. 2024-05-14T17:03:22.089 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:03:20 smithi002 ceph-mon[31538]: pgmap v268: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 17 MiB/s wr, 1.71k op/s 2024-05-14T17:03:22.101 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 485.0 s, 2521.8 tps, lat 39.831 ms stddev 45.260 2024-05-14T17:03:22.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:03:22 smithi002 ceph-mon[31538]: pgmap v269: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 17 MiB/s wr, 1.71k op/s 2024-05-14T17:03:22.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:03:22 smithi151 ceph-mon[36066]: pgmap v269: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 17 MiB/s wr, 1.71k op/s 2024-05-14T17:03:22.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:03:22 smithi123 ceph-mon[35971]: pgmap v269: 97 pgs: 97 active+clean; 4.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 17 MiB/s wr, 1.71k op/s 2024-05-14T17:03:24.711 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:03:24 smithi123 ceph-mon[35971]: pgmap v270: 97 pgs: 97 active+clean; 4.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 6.5 MiB/s wr, 685 op/s 2024-05-14T17:03:24.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:03:24 smithi002 ceph-mon[31538]: pgmap v270: 97 pgs: 97 active+clean; 4.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 6.5 MiB/s wr, 685 op/s 2024-05-14T17:03:24.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:03:24 smithi151 ceph-mon[36066]: pgmap v270: 97 pgs: 97 active+clean; 4.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 6.5 MiB/s wr, 685 op/s 2024-05-14T17:03:26.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:03:26 smithi002 ceph-mon[31538]: pgmap v271: 97 pgs: 97 active+clean; 4.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 6.5 MiB/s wr, 685 op/s 2024-05-14T17:03:26.761 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 490.0 s, 3272.2 tps, lat 30.424 ms stddev 31.839 2024-05-14T17:03:26.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:03:26 smithi151 ceph-mon[36066]: pgmap v271: 97 pgs: 97 active+clean; 4.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 6.5 MiB/s wr, 685 op/s 2024-05-14T17:03:26.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:03:26 smithi123 ceph-mon[35971]: pgmap v271: 97 pgs: 97 active+clean; 4.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 6.5 MiB/s wr, 685 op/s 2024-05-14T17:03:28.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:03:28 smithi002 ceph-mon[31538]: pgmap v272: 97 pgs: 97 active+clean; 4.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 6.4 MiB/s wr, 670 op/s 2024-05-14T17:03:28.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:03:28 smithi151 ceph-mon[36066]: pgmap v272: 97 pgs: 97 active+clean; 4.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 6.4 MiB/s wr, 670 op/s 2024-05-14T17:03:28.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:03:28 smithi123 ceph-mon[35971]: pgmap v272: 97 pgs: 97 active+clean; 4.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 6.4 MiB/s wr, 670 op/s 2024-05-14T17:03:30.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:03:30 smithi002 ceph-mon[31538]: pgmap v273: 97 pgs: 97 active+clean; 4.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 8.5 MiB/s wr, 877 op/s 2024-05-14T17:03:30.882 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:03:30 smithi151 ceph-mon[36066]: pgmap v273: 97 pgs: 97 active+clean; 4.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 8.5 MiB/s wr, 877 op/s 2024-05-14T17:03:30.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:03:30 smithi123 ceph-mon[35971]: pgmap v273: 97 pgs: 97 active+clean; 4.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 8.5 MiB/s wr, 877 op/s 2024-05-14T17:03:31.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.707+0000 7fbd3bd72640 1 -- 172.21.15.2:0/2949516421 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd34107940 msgr2=0x7fbd34107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.707+0000 7fbd3bd72640 1 --2- 172.21.15.2:0/2949516421 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd34107940 0x7fbd34107d20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbd2400dcc0 tx=0x7fbd2402e390 comp rx=0 tx=0).stop 2024-05-14T17:03:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.707+0000 7fbd3bd72640 1 -- 172.21.15.2:0/2949516421 shutdown_connections 2024-05-14T17:03:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.707+0000 7fbd3bd72640 1 --2- 172.21.15.2:0/2949516421 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd34102340 0x7fbd3410e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.707+0000 7fbd3bd72640 1 --2- 172.21.15.2:0/2949516421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd34101980 0x7fbd34101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:31.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.707+0000 7fbd3bd72640 1 --2- 172.21.15.2:0/2949516421 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd34107940 0x7fbd34107d20 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.707+0000 7fbd3bd72640 1 -- 172.21.15.2:0/2949516421 >> 172.21.15.2:0/2949516421 conn(0x7fbd34078ad0 msgr2=0x7fbd34078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.707+0000 7fbd3bd72640 1 -- 172.21.15.2:0/2949516421 shutdown_connections 2024-05-14T17:03:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.707+0000 7fbd3bd72640 1 -- 172.21.15.2:0/2949516421 wait complete. 2024-05-14T17:03:31.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.708+0000 7fbd3bd72640 1 Processor -- start 2024-05-14T17:03:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.708+0000 7fbd3bd72640 1 -- start start 2024-05-14T17:03:31.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.708+0000 7fbd3bd72640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd34101980 0x7fbd34197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:31.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.708+0000 7fbd3bd72640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd34102340 0x7fbd34197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:31.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.708+0000 7fbd3bd72640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd34107940 0x7fbd3419b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:31.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.708+0000 7fbd3bd72640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbd34110af0 con 0x7fbd34102340 2024-05-14T17:03:31.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.708+0000 7fbd3bd72640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbd34110970 con 0x7fbd34101980 2024-05-14T17:03:31.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.708+0000 7fbd3bd72640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbd34110c70 con 0x7fbd34107940 2024-05-14T17:03:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.708+0000 7fbd39b0e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd34101980 0x7fbd34197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.708+0000 7fbd3930d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd34102340 0x7fbd34197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.708+0000 7fbd39b0e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd34101980 0x7fbd34197010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:52522/0 (socket says 172.21.15.2:52522) 2024-05-14T17:03:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.708+0000 7fbd39b0e640 1 -- 172.21.15.2:0/2738399079 learned_addr learned my addr 172.21.15.2:0/2738399079 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:03:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.708+0000 7fbd3a30f640 1 --2- 172.21.15.2:0/2738399079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd34107940 0x7fbd3419b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.709+0000 7fbd39b0e640 1 -- 172.21.15.2:0/2738399079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd34107940 msgr2=0x7fbd3419b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.709+0000 7fbd39b0e640 1 --2- 172.21.15.2:0/2738399079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd34107940 0x7fbd3419b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:32.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.709+0000 7fbd39b0e640 1 -- 172.21.15.2:0/2738399079 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd34102340 msgr2=0x7fbd34197550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.709+0000 7fbd39b0e640 1 --2- 172.21.15.2:0/2738399079 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd34102340 0x7fbd34197550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.709+0000 7fbd39b0e640 1 -- 172.21.15.2:0/2738399079 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd3419c000 con 0x7fbd34101980 2024-05-14T17:03:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.709+0000 7fbd39b0e640 1 --2- 172.21.15.2:0/2738399079 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd34101980 0x7fbd34197010 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbd24000c00 tx=0x7fbd2400bae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:32.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.710+0000 7fbd32ffd640 1 -- 172.21.15.2:0/2738399079 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd24004050 con 0x7fbd34101980 2024-05-14T17:03:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.710+0000 7fbd32ffd640 1 -- 172.21.15.2:0/2738399079 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd240041f0 con 0x7fbd34101980 2024-05-14T17:03:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.710+0000 7fbd32ffd640 1 -- 172.21.15.2:0/2738399079 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2403e420 con 0x7fbd34101980 2024-05-14T17:03:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.710+0000 7fbd3bd72640 1 -- 172.21.15.2:0/2738399079 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd3419c290 con 0x7fbd34101980 2024-05-14T17:03:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.710+0000 7fbd3bd72640 1 -- 172.21.15.2:0/2738399079 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd341a3b80 con 0x7fbd34101980 2024-05-14T17:03:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.713+0000 7fbd3ea85740 1 Processor -- start 2024-05-14T17:03:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.714+0000 7fbd3ea85740 1 -- start start 2024-05-14T17:03:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.714+0000 7fbd3ea85740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fc980d8f0 0x557fc980dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.714+0000 7fbd3ea85740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fc980e210 0x557fc9818590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.714+0000 7fbd3ea85740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fc9818ad0 0x557fc981aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.714+0000 7fbd3ea85740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x557fc9780c30 con 0x557fc9818ad0 2024-05-14T17:03:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.714+0000 7fbd3ea85740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557fc97707d0 con 0x557fc980d8f0 2024-05-14T17:03:32.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.714+0000 7fbd3ea85740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557fc9774810 con 0x557fc980e210 2024-05-14T17:03:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.714+0000 7fbd3930d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fc9818ad0 0x557fc981aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.714+0000 7fbd3a30f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fc980d8f0 0x557fc980dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.714+0000 7fbd3930d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fc9818ad0 0x557fc981aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36166/0 (socket says 172.21.15.2:36166) 2024-05-14T17:03:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.714+0000 7fbd3930d640 1 -- 172.21.15.2:0/3029916468 learned_addr learned my addr 172.21.15.2:0/3029916468 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:03:32.397 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.714+0000 7fbd3930d640 1 -- 172.21.15.2:0/3029916468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fc980e210 msgr2=0x557fc9818590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T17:03:32.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.714+0000 7fbd3930d640 1 --2- 172.21.15.2:0/3029916468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fc980e210 0x557fc9818590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.714+0000 7fbd3930d640 1 -- 172.21.15.2:0/3029916468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fc980d8f0 msgr2=0x557fc980dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.714+0000 7fbd3930d640 1 --2- 172.21.15.2:0/3029916468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fc980d8f0 0x557fc980dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.714+0000 7fbd3930d640 1 -- 172.21.15.2:0/3029916468 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fc9485210 con 0x557fc9818ad0 2024-05-14T17:03:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.714+0000 7fbd32ffd640 1 -- 172.21.15.2:0/2738399079 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fbd24045020 con 0x7fbd34101980 2024-05-14T17:03:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.714+0000 7fbd3930d640 1 --2- 172.21.15.2:0/3029916468 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fc9818ad0 0x557fc981aec0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fbd28011470 tx=0x7fbd28011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b36030de12d27f5b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.715+0000 7fbd1bfff640 1 -- 172.21.15.2:0/3029916468 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd28004b20 con 0x557fc9818ad0 2024-05-14T17:03:32.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.715+0000 7fbd1bfff640 1 -- 172.21.15.2:0/3029916468 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd28004cc0 con 0x557fc9818ad0 2024-05-14T17:03:32.578 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.715+0000 7fbd1bfff640 1 -- 172.21.15.2:0/3029916468 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2800f540 con 0x557fc9818ad0 2024-05-14T17:03:32.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.715+0000 7fbd32ffd640 1 --2- 172.21.15.2:0/2738399079 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fbd1c0b2b50 0x7fbd1c0b5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.715+0000 7fbd3a30f640 1 --2- 172.21.15.2:0/2738399079 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fbd1c0b2b50 0x7fbd1c0b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:32.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.715+0000 7fbd32ffd640 1 -- 172.21.15.2:0/2738399079 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7fbd24105840 con 0x7fbd34101980 2024-05-14T17:03:32.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.715+0000 7fbd3ea85740 1 -- 172.21.15.2:0/3029916468 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fc9818ad0 msgr2=0x557fc981aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:32.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.715+0000 7fbd3ea85740 1 --2- 172.21.15.2:0/3029916468 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fc9818ad0 0x557fc981aec0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fbd28011470 tx=0x7fbd28011930 comp rx=0 tx=0).stop 2024-05-14T17:03:32.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.716+0000 7fbd3ea85740 1 -- 172.21.15.2:0/3029916468 shutdown_connections 2024-05-14T17:03:32.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.716+0000 7fbd3ea85740 1 --2- 172.21.15.2:0/3029916468 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fc9818ad0 0x557fc981aec0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:32.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.716+0000 7fbd3ea85740 1 --2- 172.21.15.2:0/3029916468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fc980e210 0x557fc9818590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.716+0000 7fbd3ea85740 1 --2- 172.21.15.2:0/3029916468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fc980d8f0 0x557fc980dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.716+0000 7fbd3ea85740 1 -- 172.21.15.2:0/3029916468 >> 172.21.15.2:0/3029916468 conn(0x557fc9809e00 msgr2=0x557fc980a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:32.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.716+0000 7fbd3a30f640 1 --2- 172.21.15.2:0/2738399079 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fbd1c0b2b50 0x7fbd1c0b5010 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c00b6c0 tx=0x7fbd2c00b2e0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.716+0000 7fbd3ea85740 1 -- 172.21.15.2:0/3029916468 shutdown_connections 2024-05-14T17:03:32.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.716+0000 7fbd3ea85740 1 -- 172.21.15.2:0/3029916468 wait complete. 2024-05-14T17:03:32.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.716+0000 7fbd3ea85740 1 -- start start 2024-05-14T17:03:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.716+0000 7fbd3ea85740 1 Processor -- start 2024-05-14T17:03:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.717+0000 7fbd3ea85740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fc980d8f0 0x557fc9818460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.717+0000 7fbd3ea85740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fc980e210 0x557fc9815750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.717+0000 7fbd3ea85740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fc9818ad0 0x557fc9815cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.717+0000 7fbd3ea85740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x557fc9456790 con 0x557fc980e210 2024-05-14T17:03:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.717+0000 7fbd3ea85740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557fc972c890 con 0x557fc9818ad0 2024-05-14T17:03:32.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.717+0000 7fbd3ea85740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557fc981b7a0 con 0x557fc980d8f0 2024-05-14T17:03:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.717+0000 7fbd3a30f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fc9818ad0 0x557fc9815cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.717+0000 7fbd3a30f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fc9818ad0 0x557fc9815cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:52536/0 (socket says 172.21.15.2:52536) 2024-05-14T17:03:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.717+0000 7fbd3a30f640 1 -- 172.21.15.2:0/3111065094 learned_addr learned my addr 172.21.15.2:0/3111065094 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:03:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.717+0000 7fbd3930d640 1 --2- 172.21.15.2:0/3111065094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fc980e210 0x557fc9815750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:32.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.717+0000 7fbd3ea85740 10 client.0 _ll_register_callbacks cb 0x557fc972fd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:03:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.717+0000 7fbd39b0e640 1 --2- 172.21.15.2:0/3111065094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fc980d8f0 0x557fc9818460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.718+0000 7fbd39b0e640 1 -- 172.21.15.2:0/3111065094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fc9818ad0 msgr2=0x557fc9815cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.718+0000 7fbd39b0e640 1 --2- 172.21.15.2:0/3111065094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fc9818ad0 0x557fc9815cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:32.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.718+0000 7fbd39b0e640 1 -- 172.21.15.2:0/3111065094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fc980e210 msgr2=0x557fc9815750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.718+0000 7fbd39b0e640 1 --2- 172.21.15.2:0/3111065094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fc980e210 0x557fc9815750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:32.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.718+0000 7fbd39b0e640 1 -- 172.21.15.2:0/3111065094 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fc9551620 con 0x557fc980d8f0 2024-05-14T17:03:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.718+0000 7fbd3a30f640 1 --2- 172.21.15.2:0/3111065094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fc9818ad0 0x557fc9815cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:03:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.718+0000 7fbd39b0e640 1 --2- 172.21.15.2:0/3111065094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fc980d8f0 0x557fc9818460 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbd241030a0 tx=0x7fbd240cc3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5052649f618bcdb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.718+0000 7fbd197fa640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-14T17:03:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.718+0000 7fbd3930d640 1 --2- 172.21.15.2:0/3111065094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fc980e210 0x557fc9815750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:03:33.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.718+0000 7fbd197fa640 1 -- 172.21.15.2:0/3111065094 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd240cd330 con 0x557fc980d8f0 2024-05-14T17:03:33.033 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.718+0000 7fbd3ea85740 1 -- 172.21.15.2:0/3111065094 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd28004cc0 con 0x557fc980d8f0 2024-05-14T17:03:33.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.718+0000 7fbd197fa640 1 -- 172.21.15.2:0/3111065094 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7fbd240c8ec0 con 0x557fc980d8f0 2024-05-14T17:03:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.719+0000 7fbd197fa640 1 -- 172.21.15.2:0/3111065094 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fbd240e7320 con 0x557fc980d8f0 2024-05-14T17:03:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.719+0000 7fbd3ea85740 10 client.25939 fetch_fsmap learned FSMap version 41 2024-05-14T17:03:33.062 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.719+0000 7fbd3ea85740 1 -- 172.21.15.2:0/3111065094 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x557fc9827220 con 0x557fc980d8f0 2024-05-14T17:03:33.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.719+0000 7fbd197fa640 1 -- 172.21.15.2:0/3111065094 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd28004cc0 con 0x557fc980d8f0 2024-05-14T17:03:33.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.719+0000 7fbd197fa640 1 -- 172.21.15.2:0/3111065094 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fbd240c85a0 con 0x557fc980d8f0 2024-05-14T17:03:33.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.719+0000 7fbd3ea85740 10 client.25939 fetch_fsmap finished waiting for FSMap version 41 2024-05-14T17:03:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.719+0000 7fbd3ea85740 10 client.25939 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:03:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.719+0000 7fbd3ea85740 20 client.25939 populate_metadata read hostname 'smithi002' 2024-05-14T17:03:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.719+0000 7fbd3ea85740 1 --2- 172.21.15.2:0/3111065094 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x557fc9828720 0x557fc982ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.719+0000 7fbd3ea85740 4 client.25939 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:03:33.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.719+0000 7fbd3ea85740 1 -- 172.21.15.2:0/3111065094 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557fc9782b80 con 0x557fc9828720 2024-05-14T17:03:33.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.719+0000 7fbd3930d640 1 --2- 172.21.15.2:0/3111065094 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x557fc9828720 0x557fc982ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.720+0000 7fbd197fa640 10 client.25939 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:03:33.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.720+0000 7fbd3930d640 1 --2- 172.21.15.2:0/3111065094 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x557fc9828720 0x557fc982ab50 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c9d26b8303df9d6 server_cookie=9088412f00c04462 in_seq=0 out_seq=0 2024-05-14T17:03:33.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.720+0000 7fbd197fa640 1 -- 172.21.15.2:0/3111065094 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557fc9782b80 con 0x557fc9828720 2024-05-14T17:03:33.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.720+0000 7fbd197fa640 10 client.25939 handle_command_reply: tid=0 2024-05-14T17:03:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.720+0000 7fbd3ea85740 1 client.25939 shutdown 2024-05-14T17:03:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.721+0000 7fbd3ea85740 1 -- 172.21.15.2:0/3111065094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fc980d8f0 msgr2=0x557fc9818460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:33.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.721+0000 7fbd3ea85740 1 --2- 172.21.15.2:0/3111065094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fc980d8f0 0x557fc9818460 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbd241030a0 tx=0x7fbd240cc3a0 comp rx=0 tx=0).stop 2024-05-14T17:03:33.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.721+0000 7fbd3ea85740 1 -- 172.21.15.2:0/3111065094 shutdown_connections 2024-05-14T17:03:33.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.721+0000 7fbd3ea85740 1 --2- 172.21.15.2:0/3111065094 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x557fc9828720 0x557fc982ab50 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:33.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.721+0000 7fbd3ea85740 1 --2- 172.21.15.2:0/3111065094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fc9818ad0 0x557fc9815cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:33.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.721+0000 7fbd3ea85740 1 --2- 172.21.15.2:0/3111065094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fc980e210 0x557fc9815750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:33.397 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.721+0000 7fbd3ea85740 1 --2- 172.21.15.2:0/3111065094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fc980d8f0 0x557fc9818460 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.721+0000 7fbd3ea85740 1 -- 172.21.15.2:0/3111065094 >> 172.21.15.2:0/3111065094 conn(0x557fc9809e00 msgr2=0x557fc9809100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.721+0000 7fbd197fa640 0 client.25939 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:03:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.721+0000 7fbd3ea85740 1 -- 172.21.15.2:0/3111065094 shutdown_connections 2024-05-14T17:03:33.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.721+0000 7fbd3ea85740 1 -- 172.21.15.2:0/3111065094 wait complete. 2024-05-14T17:03:33.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.721+0000 7fbd3ea85740 20 client.25939 trim_cache size 0 max 16384 2024-05-14T17:03:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.741+0000 7fbd3ea85740 1 Processor -- start 2024-05-14T17:03:33.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.741+0000 7fbd3ea85740 1 -- start start 2024-05-14T17:03:33.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.741+0000 7fbd3ea85740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fc980d8f0 0x557fc9824670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:33.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.741+0000 7fbd3ea85740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fc980e210 0x557fc9830460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.741+0000 7fbd3ea85740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fc9818ad0 0x557fc98309a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.741+0000 7fbd3ea85740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbd341aec40 con 0x557fc980e210 2024-05-14T17:03:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.741+0000 7fbd3ea85740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbd341ac510 con 0x557fc980d8f0 2024-05-14T17:03:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.741+0000 7fbd3ea85740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbd341aa380 con 0x557fc9818ad0 2024-05-14T17:03:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.741+0000 7fbd3930d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fc980e210 0x557fc9830460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.741+0000 7fbd3a30f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fc9818ad0 0x557fc98309a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.741+0000 7fbd3930d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fc980e210 0x557fc9830460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36194/0 (socket says 172.21.15.2:36194) 2024-05-14T17:03:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.741+0000 7fbd39b0e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fc980d8f0 0x557fc9824670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.741+0000 7fbd3930d640 1 -- 172.21.15.2:0/3027807615 learned_addr learned my addr 172.21.15.2:0/3027807615 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:03:33.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.741+0000 7fbd3930d640 1 -- 172.21.15.2:0/3027807615 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fc9818ad0 msgr2=0x557fc98309a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.741+0000 7fbd3930d640 1 --2- 172.21.15.2:0/3027807615 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fc9818ad0 0x557fc98309a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:33.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.741+0000 7fbd3930d640 1 -- 172.21.15.2:0/3027807615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fc980d8f0 msgr2=0x557fc9824670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:33.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.741+0000 7fbd3930d640 1 --2- 172.21.15.2:0/3027807615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fc980d8f0 0x557fc9824670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:33.696 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.742+0000 7fbd3930d640 1 -- 172.21.15.2:0/3027807615 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fc9816000 con 0x557fc980e210 2024-05-14T17:03:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.742+0000 7fbd39b0e640 1 --2- 172.21.15.2:0/3027807615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fc980d8f0 0x557fc9824670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:03:33.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.742+0000 7fbd3930d640 1 --2- 172.21.15.2:0/3027807615 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fc980e210 0x557fc9830460 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fbd28006010 tx=0x7fbd28004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c732b5ef07c540e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:33.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.742+0000 7fbd1b7fe640 1 -- 172.21.15.2:0/3027807615 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd28004590 con 0x557fc980e210 2024-05-14T17:03:33.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.742+0000 7fbd1b7fe640 1 -- 172.21.15.2:0/3027807615 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd28004730 con 0x557fc980e210 2024-05-14T17:03:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.742+0000 7fbd1b7fe640 1 -- 172.21.15.2:0/3027807615 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2800f9b0 con 0x557fc980e210 2024-05-14T17:03:33.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.743+0000 7fbd3ea85740 1 -- 172.21.15.2:0/3027807615 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fc980e210 msgr2=0x557fc9830460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:33.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.743+0000 7fbd3ea85740 1 --2- 172.21.15.2:0/3027807615 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fc980e210 0x557fc9830460 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fbd28006010 tx=0x7fbd28004410 comp rx=0 tx=0).stop 2024-05-14T17:03:33.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.743+0000 7fbd3ea85740 1 -- 172.21.15.2:0/3027807615 shutdown_connections 2024-05-14T17:03:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.743+0000 7fbd3ea85740 1 --2- 172.21.15.2:0/3027807615 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fc9818ad0 0x557fc98309a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.743+0000 7fbd3ea85740 1 --2- 172.21.15.2:0/3027807615 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fc980e210 0x557fc9830460 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.743+0000 7fbd3ea85740 1 --2- 172.21.15.2:0/3027807615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fc980d8f0 0x557fc9824670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.743+0000 7fbd3ea85740 1 -- 172.21.15.2:0/3027807615 >> 172.21.15.2:0/3027807615 conn(0x557fc9809e00 msgr2=0x557fc9809100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.743+0000 7fbd3ea85740 1 -- 172.21.15.2:0/3027807615 shutdown_connections 2024-05-14T17:03:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.743+0000 7fbd3ea85740 1 -- 172.21.15.2:0/3027807615 wait complete. 2024-05-14T17:03:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.743+0000 7fbd3ea85740 1 -- start start 2024-05-14T17:03:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.743+0000 7fbd3ea85740 1 Processor -- start 2024-05-14T17:03:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.743+0000 7fbd3ea85740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fc980d8f0 0x557fc981ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.743+0000 7fbd3ea85740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fc980e210 0x557fc981f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.743+0000 7fbd3ea85740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fc9818ad0 0x557fc9892020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:33.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.744+0000 7fbd3ea85740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbd341aa9d0 con 0x557fc980d8f0 2024-05-14T17:03:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.744+0000 7fbd3ea85740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbd341abe40 con 0x557fc9818ad0 2024-05-14T17:03:33.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.744+0000 7fbd3ea85740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbd341ab0a0 con 0x557fc980e210 2024-05-14T17:03:33.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.744+0000 7fbd3ea85740 10 client.0 _ll_register_callbacks cb 0x7fbd240cca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:03:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.744+0000 7fbd3930d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fc980e210 0x557fc981f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.744+0000 7fbd3930d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fc980e210 0x557fc981f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:53868/0 (socket says 172.21.15.2:53868) 2024-05-14T17:03:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.744+0000 7fbd3930d640 1 -- 172.21.15.2:0/2549949716 learned_addr learned my addr 172.21.15.2:0/2549949716 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:03:33.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.744+0000 7fbd3a30f640 1 --2- 172.21.15.2:0/2549949716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fc9818ad0 0x557fc9892020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:34.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.744+0000 7fbd39b0e640 1 --2- 172.21.15.2:0/2549949716 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fc980d8f0 0x557fc981ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.745+0000 7fbd39b0e640 1 -- 172.21.15.2:0/2549949716 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fc980e210 msgr2=0x557fc981f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.745+0000 7fbd39b0e640 1 --2- 172.21.15.2:0/2549949716 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fc980e210 0x557fc981f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.745+0000 7fbd39b0e640 1 -- 172.21.15.2:0/2549949716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fc9818ad0 msgr2=0x557fc9892020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.745+0000 7fbd39b0e640 1 --2- 172.21.15.2:0/2549949716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fc9818ad0 0x557fc9892020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:34.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.745+0000 7fbd39b0e640 1 -- 172.21.15.2:0/2549949716 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fc9892560 con 0x557fc980d8f0 2024-05-14T17:03:34.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.745+0000 7fbd39b0e640 1 --2- 172.21.15.2:0/2549949716 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fc980d8f0 0x557fc981ebf0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fbd240ccbc0 tx=0x7fbd240cc4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcfe776e58aee07e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:34.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.745+0000 7fbd197fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:03:34.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.745+0000 7fbd197fa640 1 -- 172.21.15.2:0/2549949716 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd240c8af0 con 0x557fc980d8f0 2024-05-14T17:03:34.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.745+0000 7fbd197fa640 1 -- 172.21.15.2:0/2549949716 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7fbd240d5810 con 0x557fc980d8f0 2024-05-14T17:03:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.746+0000 7fbd3ea85740 1 -- 172.21.15.2:0/2549949716 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd28004730 con 0x557fc980d8f0 2024-05-14T17:03:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.746+0000 7fbd197fa640 1 -- 172.21.15.2:0/2549949716 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fbd240cd330 con 0x557fc980d8f0 2024-05-14T17:03:34.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.746+0000 7fbd3ea85740 10 client.16290 fetch_fsmap learned FSMap version 41 2024-05-14T17:03:34.188 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 495.0 s, 2898.2 tps, lat 34.681 ms stddev 38.015 2024-05-14T17:03:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.747+0000 7fbd3ea85740 1 -- 172.21.15.2:0/2549949716 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x557fc9893770 con 0x557fc980d8f0 2024-05-14T17:03:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.747+0000 7fbd197fa640 1 -- 172.21.15.2:0/2549949716 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd28004730 con 0x557fc980d8f0 2024-05-14T17:03:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.747+0000 7fbd197fa640 1 -- 172.21.15.2:0/2549949716 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fbd240d5ce0 con 0x557fc980d8f0 2024-05-14T17:03:34.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.748+0000 7fbd3ea85740 10 client.16290 fetch_fsmap finished waiting for FSMap version 41 2024-05-14T17:03:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.748+0000 7fbd3ea85740 10 client.16290 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:03:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.748+0000 7fbd3ea85740 20 client.16290 populate_metadata read hostname 'smithi002' 2024-05-14T17:03:34.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.748+0000 7fbd3ea85740 1 --2- 172.21.15.2:0/2549949716 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x557fc9893d90 0x557fc98961b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:34.334 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.748+0000 7fbd3ea85740 4 client.16290 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:03:34.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.748+0000 7fbd3ea85740 1 -- 172.21.15.2:0/2549949716 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557fc977c360 con 0x557fc9893d90 2024-05-14T17:03:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.748+0000 7fbd3930d640 1 --2- 172.21.15.2:0/2549949716 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x557fc9893d90 0x557fc98961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.748+0000 7fbd197fa640 10 client.16290 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:03:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.748+0000 7fbd3930d640 1 --2- 172.21.15.2:0/2549949716 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x557fc9893d90 0x557fc98961b0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f349e9e12616764 server_cookie=c79c48a82c5e55cd in_seq=0 out_seq=0 2024-05-14T17:03:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.759+0000 7fbd197fa640 1 -- 172.21.15.2:0/2549949716 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557fc977c360 con 0x557fc9893d90 2024-05-14T17:03:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.759+0000 7fbd197fa640 10 client.16290 handle_command_reply: tid=0 2024-05-14T17:03:34.385 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.759+0000 7fbd3ea85740 1 client.16290 shutdown 2024-05-14T17:03:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.759+0000 7fbd3ea85740 1 -- 172.21.15.2:0/2549949716 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fc980d8f0 msgr2=0x557fc981ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:34.397 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.759+0000 7fbd3ea85740 1 --2- 172.21.15.2:0/2549949716 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fc980d8f0 0x557fc981ebf0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fbd240ccbc0 tx=0x7fbd240cc4d0 comp rx=0 tx=0).stop 2024-05-14T17:03:34.403 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.759+0000 7fbd3ea85740 1 -- 172.21.15.2:0/2549949716 shutdown_connections 2024-05-14T17:03:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.759+0000 7fbd3ea85740 1 --2- 172.21.15.2:0/2549949716 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x557fc9893d90 0x557fc98961b0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.759+0000 7fbd3ea85740 1 --2- 172.21.15.2:0/2549949716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557fc9818ad0 0x557fc9892020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.759+0000 7fbd3ea85740 1 --2- 172.21.15.2:0/2549949716 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557fc980e210 0x557fc981f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.759+0000 7fbd3ea85740 1 --2- 172.21.15.2:0/2549949716 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fc980d8f0 0x557fc981ebf0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:34.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.759+0000 7fbd197fa640 0 client.16290 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:03:34.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.759+0000 7fbd3ea85740 1 -- 172.21.15.2:0/2549949716 >> 172.21.15.2:0/2549949716 conn(0x557fc9809e00 msgr2=0x557fc9823660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:34.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.759+0000 7fbd3ea85740 1 -- 172.21.15.2:0/2549949716 shutdown_connections 2024-05-14T17:03:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.759+0000 7fbd3ea85740 1 -- 172.21.15.2:0/2549949716 wait complete. 2024-05-14T17:03:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.759+0000 7fbd3ea85740 20 client.16290 trim_cache size 0 max 16384 2024-05-14T17:03:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:03:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:03:34.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:03:34.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.760+0000 7fbd3bd72640 1 -- 172.21.15.2:0/2738399079 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fbd1c0b2b50 msgr2=0x7fbd1c0b5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.760+0000 7fbd3bd72640 1 --2- 172.21.15.2:0/2738399079 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fbd1c0b2b50 0x7fbd1c0b5010 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c00b6c0 tx=0x7fbd2c00b2e0 comp rx=0 tx=0).stop 2024-05-14T17:03:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.760+0000 7fbd3bd72640 1 -- 172.21.15.2:0/2738399079 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd34101980 msgr2=0x7fbd34197010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.760+0000 7fbd3bd72640 1 --2- 172.21.15.2:0/2738399079 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd34101980 0x7fbd34197010 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbd24000c00 tx=0x7fbd2400bae0 comp rx=0 tx=0).stop 2024-05-14T17:03:34.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.760+0000 7fbd3bd72640 1 -- 172.21.15.2:0/2738399079 shutdown_connections 2024-05-14T17:03:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.760+0000 7fbd3bd72640 1 --2- 172.21.15.2:0/2738399079 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fbd1c0b2b50 0x7fbd1c0b5010 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.760+0000 7fbd3bd72640 1 --2- 172.21.15.2:0/2738399079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd34107940 0x7fbd3419b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:34.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.760+0000 7fbd3bd72640 1 --2- 172.21.15.2:0/2738399079 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd34102340 0x7fbd34197550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:34.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.760+0000 7fbd3bd72640 1 --2- 172.21.15.2:0/2738399079 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd34101980 0x7fbd34197010 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.760+0000 7fbd3bd72640 1 -- 172.21.15.2:0/2738399079 >> 172.21.15.2:0/2738399079 conn(0x7fbd34078ad0 msgr2=0x7fbd341045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:34.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.761+0000 7fbd3bd72640 1 -- 172.21.15.2:0/2738399079 shutdown_connections 2024-05-14T17:03:34.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:31.761+0000 7fbd3bd72640 1 -- 172.21.15.2:0/2738399079 wait complete. 2024-05-14T17:03:34.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:03:32 smithi123 ceph-mon[35971]: pgmap v274: 97 pgs: 97 active+clean; 4.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.8 MiB/s wr, 595 op/s 2024-05-14T17:03:34.765 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:03:32 smithi151 ceph-mon[36066]: pgmap v274: 97 pgs: 97 active+clean; 4.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.8 MiB/s wr, 595 op/s 2024-05-14T17:03:34.774 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:03:32 smithi002 ceph-mon[31538]: pgmap v274: 97 pgs: 97 active+clean; 4.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.8 MiB/s wr, 595 op/s 2024-05-14T17:03:34.782 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:03:34 smithi002 ceph-mon[31538]: pgmap v275: 97 pgs: 97 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 9.8 MiB/s wr, 948 op/s 2024-05-14T17:03:34.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:03:34 smithi151 ceph-mon[36066]: pgmap v275: 97 pgs: 97 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 9.8 MiB/s wr, 948 op/s 2024-05-14T17:03:34.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:03:34 smithi123 ceph-mon[35971]: pgmap v275: 97 pgs: 97 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 9.8 MiB/s wr, 948 op/s 2024-05-14T17:03:36.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:03:36 smithi002 ceph-mon[31538]: pgmap v276: 97 pgs: 97 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.0 MiB/s wr, 659 op/s 2024-05-14T17:03:36.752 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 500.0 s, 3058.4 tps, lat 32.686 ms stddev 34.232 2024-05-14T17:03:36.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:03:36 smithi151 ceph-mon[36066]: pgmap v276: 97 pgs: 97 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.0 MiB/s wr, 659 op/s 2024-05-14T17:03:36.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:03:36 smithi123 ceph-mon[35971]: pgmap v276: 97 pgs: 97 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.0 MiB/s wr, 659 op/s 2024-05-14T17:03:38.409 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:03:38 smithi123 ceph-mon[35971]: pgmap v277: 97 pgs: 97 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.0 MiB/s wr, 659 op/s 2024-05-14T17:03:38.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:03:38 smithi002 ceph-mon[31538]: pgmap v277: 97 pgs: 97 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.0 MiB/s wr, 659 op/s 2024-05-14T17:03:38.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:03:38 smithi151 ceph-mon[36066]: pgmap v277: 97 pgs: 97 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.0 MiB/s wr, 659 op/s 2024-05-14T17:03:40.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:03:40 smithi002 ceph-mon[31538]: pgmap v278: 97 pgs: 97 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.9 MiB/s wr, 953 op/s 2024-05-14T17:03:41.485 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:03:40 smithi123 ceph-mon[35971]: pgmap v278: 97 pgs: 97 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.9 MiB/s wr, 953 op/s 2024-05-14T17:03:41.492 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:03:40 smithi151 ceph-mon[36066]: pgmap v278: 97 pgs: 97 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.9 MiB/s wr, 953 op/s 2024-05-14T17:03:41.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 505.0 s, 2812.6 tps, lat 35.481 ms stddev 37.977 2024-05-14T17:03:42.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:03:42 smithi002 ceph-mon[31538]: pgmap v279: 97 pgs: 97 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.8 MiB/s wr, 646 op/s 2024-05-14T17:03:42.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:03:42 smithi151 ceph-mon[36066]: pgmap v279: 97 pgs: 97 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.8 MiB/s wr, 646 op/s 2024-05-14T17:03:42.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:03:42 smithi123 ceph-mon[35971]: pgmap v279: 97 pgs: 97 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.8 MiB/s wr, 646 op/s 2024-05-14T17:03:44.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:03:44 smithi002 ceph-mon[31538]: pgmap v280: 97 pgs: 97 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 947 op/s 2024-05-14T17:03:44.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:03:44 smithi151 ceph-mon[36066]: pgmap v280: 97 pgs: 97 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 947 op/s 2024-05-14T17:03:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.892+0000 7fbd995da640 1 -- 172.21.15.2:0/4065835672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd94102360 msgr2=0x7fbd9410e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:44.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.892+0000 7fbd995da640 1 --2- 172.21.15.2:0/4065835672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd94102360 0x7fbd9410e820 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c00df70 tx=0x7fbd8c02e640 comp rx=0 tx=0).stop 2024-05-14T17:03:44.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.893+0000 7fbd995da640 1 -- 172.21.15.2:0/4065835672 shutdown_connections 2024-05-14T17:03:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.893+0000 7fbd995da640 1 --2- 172.21.15.2:0/4065835672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd94102360 0x7fbd9410e820 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.893+0000 7fbd995da640 1 --2- 172.21.15.2:0/4065835672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd941019a0 0x7fbd94101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.893+0000 7fbd995da640 1 --2- 172.21.15.2:0/4065835672 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd94107960 0x7fbd94107d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.893+0000 7fbd995da640 1 -- 172.21.15.2:0/4065835672 >> 172.21.15.2:0/4065835672 conn(0x7fbd94078aa0 msgr2=0x7fbd94078eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.893+0000 7fbd995da640 1 -- 172.21.15.2:0/4065835672 shutdown_connections 2024-05-14T17:03:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.893+0000 7fbd995da640 1 -- 172.21.15.2:0/4065835672 wait complete. 2024-05-14T17:03:44.972 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:03:44 smithi123 ceph-mon[35971]: pgmap v280: 97 pgs: 97 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 947 op/s 2024-05-14T17:03:44.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.894+0000 7fbd995da640 1 Processor -- start 2024-05-14T17:03:45.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.894+0000 7fbd995da640 1 -- start start 2024-05-14T17:03:45.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.894+0000 7fbd995da640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd941019a0 0x7fbd9419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.894+0000 7fbd995da640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd94102360 0x7fbd9419fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:45.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.894+0000 7fbd995da640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd94107960 0x7fbd941a3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.894+0000 7fbd995da640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbd94110b10 con 0x7fbd94102360 2024-05-14T17:03:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.894+0000 7fbd995da640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbd94110990 con 0x7fbd941019a0 2024-05-14T17:03:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.894+0000 7fbd995da640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbd94110c90 con 0x7fbd94107960 2024-05-14T17:03:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.894+0000 7fbd937fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd94107960 0x7fbd941a3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.894+0000 7fbd937fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd94107960 0x7fbd941a3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:56204/0 (socket says 172.21.15.2:56204) 2024-05-14T17:03:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.894+0000 7fbd937fe640 1 -- 172.21.15.2:0/1431949550 learned_addr learned my addr 172.21.15.2:0/1431949550 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:03:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.895+0000 7fbd92ffd640 1 --2- 172.21.15.2:0/1431949550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd941019a0 0x7fbd9419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.895+0000 7fbd927fc640 1 --2- 172.21.15.2:0/1431949550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd94102360 0x7fbd9419fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.895+0000 7fbd937fe640 1 -- 172.21.15.2:0/1431949550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd941019a0 msgr2=0x7fbd9419f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:45.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.895+0000 7fbd937fe640 1 --2- 172.21.15.2:0/1431949550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd941019a0 0x7fbd9419f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.895+0000 7fbd937fe640 1 -- 172.21.15.2:0/1431949550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd94102360 msgr2=0x7fbd9419fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:45.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.895+0000 7fbd937fe640 1 --2- 172.21.15.2:0/1431949550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd94102360 0x7fbd9419fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.895+0000 7fbd937fe640 1 -- 172.21.15.2:0/1431949550 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd941a4680 con 0x7fbd94107960 2024-05-14T17:03:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.895+0000 7fbd927fc640 1 --2- 172.21.15.2:0/1431949550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd94102360 0x7fbd9419fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:03:45.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.895+0000 7fbd937fe640 1 --2- 172.21.15.2:0/1431949550 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd94107960 0x7fbd941a3f50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c004af0 tx=0x7fbd8c034590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.895+0000 7fbd7bfff640 1 -- 172.21.15.2:0/1431949550 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c03a030 con 0x7fbd94107960 2024-05-14T17:03:45.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.895+0000 7fbd995da640 1 -- 172.21.15.2:0/1431949550 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd941a4910 con 0x7fbd94107960 2024-05-14T17:03:45.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.895+0000 7fbd7bfff640 1 -- 172.21.15.2:0/1431949550 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd8c03c070 con 0x7fbd94107960 2024-05-14T17:03:45.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.895+0000 7fbd7bfff640 1 -- 172.21.15.2:0/1431949550 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c004020 con 0x7fbd94107960 2024-05-14T17:03:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.896+0000 7fbd995da640 1 -- 172.21.15.2:0/1431949550 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd941ac200 con 0x7fbd94107960 2024-05-14T17:03:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.899+0000 7fbd9c3d0740 1 Processor -- start 2024-05-14T17:03:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.900+0000 7fbd7bfff640 1 -- 172.21.15.2:0/1431949550 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fbd8c03e030 con 0x7fbd94107960 2024-05-14T17:03:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.900+0000 7fbd9c3d0740 1 -- start start 2024-05-14T17:03:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.900+0000 7fbd9c3d0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564999c6a8f0 0x564999c6acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.900+0000 7fbd9c3d0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564999c6b210 0x564999c75590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.900+0000 7fbd9c3d0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564999c75ad0 0x564999c77ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.900+0000 7fbd9c3d0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x564999bddc30 con 0x564999c6a8f0 2024-05-14T17:03:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.900+0000 7fbd9c3d0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564999bcd7d0 con 0x564999c6b210 2024-05-14T17:03:45.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.900+0000 7fbd9c3d0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564999bd1810 con 0x564999c75ad0 2024-05-14T17:03:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.901+0000 7fbd92ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564999c6b210 0x564999c75590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.901+0000 7fbd927fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564999c6a8f0 0x564999c6acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.901+0000 7fbd927fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564999c6a8f0 0x564999c6acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38436/0 (socket says 172.21.15.2:38436) 2024-05-14T17:03:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.901+0000 7fbd927fc640 1 -- 172.21.15.2:0/1829916091 learned_addr learned my addr 172.21.15.2:0/1829916091 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:03:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.901+0000 7fbd927fc640 1 --2- 172.21.15.2:0/1829916091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564999c75ad0 0x564999c77ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.901+0000 7fbd92ffd640 1 -- 172.21.15.2:0/1829916091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564999c75ad0 msgr2=0x564999c77ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.901+0000 7fbd92ffd640 1 --2- 172.21.15.2:0/1829916091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564999c75ad0 0x564999c77ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.901+0000 7fbd92ffd640 1 -- 172.21.15.2:0/1829916091 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564999c6a8f0 msgr2=0x564999c6acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.901+0000 7fbd92ffd640 1 --2- 172.21.15.2:0/1829916091 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564999c6a8f0 0x564999c6acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.901+0000 7fbd92ffd640 1 -- 172.21.15.2:0/1829916091 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649998e2210 con 0x564999c6b210 2024-05-14T17:03:45.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.901+0000 7fbd7bfff640 1 --2- 172.21.15.2:0/1431949550 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fbd7c0b2a90 0x7fbd7c0b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:45.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.901+0000 7fbd92ffd640 1 --2- 172.21.15.2:0/1829916091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564999c6b210 0x564999c75590 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbd84009bc0 tx=0x7fbd840117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2670c2a2d05b5ea server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.901+0000 7fbd927fc640 1 --2- 172.21.15.2:0/1431949550 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fbd7c0b2a90 0x7fbd7c0b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.901+0000 7fbd7bfff640 1 -- 172.21.15.2:0/1431949550 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7fbd8c102650 con 0x7fbd94107960 2024-05-14T17:03:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.901+0000 7fbd797fa640 1 -- 172.21.15.2:0/1829916091 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd84004050 con 0x564999c6b210 2024-05-14T17:03:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.901+0000 7fbd797fa640 1 -- 172.21.15.2:0/1829916091 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd84016070 con 0x564999c6b210 2024-05-14T17:03:45.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.902+0000 7fbd797fa640 1 -- 172.21.15.2:0/1829916091 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8401b470 con 0x564999c6b210 2024-05-14T17:03:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.902+0000 7fbd9c3d0740 1 -- 172.21.15.2:0/1829916091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564999c6b210 msgr2=0x564999c75590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:45.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.902+0000 7fbd9c3d0740 1 --2- 172.21.15.2:0/1829916091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564999c6b210 0x564999c75590 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbd84009bc0 tx=0x7fbd840117b0 comp rx=0 tx=0).stop 2024-05-14T17:03:45.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.902+0000 7fbd927fc640 1 --2- 172.21.15.2:0/1431949550 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fbd7c0b2a90 0x7fbd7c0b4f50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x564999c68220 tx=0x7fbd88009260 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:45.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.903+0000 7fbd9c3d0740 1 -- 172.21.15.2:0/1829916091 shutdown_connections 2024-05-14T17:03:45.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.903+0000 7fbd9c3d0740 1 --2- 172.21.15.2:0/1829916091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564999c75ad0 0x564999c77ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.903+0000 7fbd9c3d0740 1 --2- 172.21.15.2:0/1829916091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564999c6b210 0x564999c75590 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.903+0000 7fbd9c3d0740 1 --2- 172.21.15.2:0/1829916091 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564999c6a8f0 0x564999c6acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.903+0000 7fbd9c3d0740 1 -- 172.21.15.2:0/1829916091 >> 172.21.15.2:0/1829916091 conn(0x564999c66e00 msgr2=0x564999c671e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.903+0000 7fbd9c3d0740 1 -- 172.21.15.2:0/1829916091 shutdown_connections 2024-05-14T17:03:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.903+0000 7fbd9c3d0740 1 -- 172.21.15.2:0/1829916091 wait complete. 2024-05-14T17:03:45.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.904+0000 7fbd9c3d0740 1 -- start start 2024-05-14T17:03:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.904+0000 7fbd9c3d0740 1 Processor -- start 2024-05-14T17:03:45.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.905+0000 7fbd9c3d0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564999c6a8f0 0x564999c75330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.905+0000 7fbd9c3d0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564999c6b210 0x564999c72620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.905+0000 7fbd9c3d0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564999c75ad0 0x564999c72b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:45.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.905+0000 7fbd9c3d0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5649998b3790 con 0x564999c75ad0 2024-05-14T17:03:45.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.905+0000 7fbd9c3d0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564999b89890 con 0x564999c6b210 2024-05-14T17:03:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.905+0000 7fbd9c3d0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564999c787a0 con 0x564999c6a8f0 2024-05-14T17:03:45.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.906+0000 7fbd9c3d0740 10 client.0 _ll_register_callbacks cb 0x564999b8cd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:03:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.906+0000 7fbd927fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564999c6b210 0x564999c72620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.906+0000 7fbd927fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564999c6b210 0x564999c72620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:40136/0 (socket says 172.21.15.2:40136) 2024-05-14T17:03:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.906+0000 7fbd927fc640 1 -- 172.21.15.2:0/2616219626 learned_addr learned my addr 172.21.15.2:0/2616219626 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:03:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.906+0000 7fbd937fe640 1 --2- 172.21.15.2:0/2616219626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564999c75ad0 0x564999c72b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:46.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.906+0000 7fbd92ffd640 1 --2- 172.21.15.2:0/2616219626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564999c6a8f0 0x564999c75330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.906+0000 7fbd927fc640 1 -- 172.21.15.2:0/2616219626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564999c6a8f0 msgr2=0x564999c75330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.906+0000 7fbd927fc640 1 --2- 172.21.15.2:0/2616219626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564999c6a8f0 0x564999c75330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.906+0000 7fbd927fc640 1 -- 172.21.15.2:0/2616219626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564999c75ad0 msgr2=0x564999c72b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.906+0000 7fbd927fc640 1 --2- 172.21.15.2:0/2616219626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564999c75ad0 0x564999c72b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.906+0000 7fbd927fc640 1 -- 172.21.15.2:0/2616219626 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649999ae620 con 0x564999c6b210 2024-05-14T17:03:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.906+0000 7fbd937fe640 1 --2- 172.21.15.2:0/2616219626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564999c75ad0 0x564999c72b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:03:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.906+0000 7fbd927fc640 1 --2- 172.21.15.2:0/2616219626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564999c6b210 0x564999c72620 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbd8801dd90 tx=0x7fbd88004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6673b262d1bba94e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.906+0000 7fbd76ff5640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T17:03:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.906+0000 7fbd76ff5640 1 -- 172.21.15.2:0/2616219626 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd88021070 con 0x564999c6b210 2024-05-14T17:03:46.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.907+0000 7fbd9c3d0740 1 -- 172.21.15.2:0/2616219626 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd84016070 con 0x564999c6b210 2024-05-14T17:03:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.907+0000 7fbd76ff5640 1 -- 172.21.15.2:0/2616219626 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7fbd88028670 con 0x564999c6b210 2024-05-14T17:03:46.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.907+0000 7fbd76ff5640 1 -- 172.21.15.2:0/2616219626 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fbd88036050 con 0x564999c6b210 2024-05-14T17:03:46.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.907+0000 7fbd9c3d0740 10 client.25976 fetch_fsmap learned FSMap version 41 2024-05-14T17:03:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.907+0000 7fbd9c3d0740 1 -- 172.21.15.2:0/2616219626 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x564999c84140 con 0x564999c6b210 2024-05-14T17:03:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.907+0000 7fbd76ff5640 1 -- 172.21.15.2:0/2616219626 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd84016070 con 0x564999c6b210 2024-05-14T17:03:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.907+0000 7fbd76ff5640 1 -- 172.21.15.2:0/2616219626 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fbd8801b800 con 0x564999c6b210 2024-05-14T17:03:46.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.907+0000 7fbd9c3d0740 10 client.25976 fetch_fsmap finished waiting for FSMap version 41 2024-05-14T17:03:46.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.907+0000 7fbd9c3d0740 10 client.25976 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:03:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.907+0000 7fbd9c3d0740 20 client.25976 populate_metadata read hostname 'smithi002' 2024-05-14T17:03:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.907+0000 7fbd9c3d0740 1 --2- 172.21.15.2:0/2616219626 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564999c85710 0x564999c87b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.907+0000 7fbd9c3d0740 4 client.25976 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:03:46.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.907+0000 7fbd9c3d0740 1 -- 172.21.15.2:0/2616219626 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564999bdfb80 con 0x564999c85710 2024-05-14T17:03:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.907+0000 7fbd92ffd640 1 --2- 172.21.15.2:0/2616219626 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564999c85710 0x564999c87b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:46.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.908+0000 7fbd92ffd640 1 --2- 172.21.15.2:0/2616219626 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564999c85710 0x564999c87b50 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1583ed033a6ba53c server_cookie=1cc62175b4656d47 in_seq=0 out_seq=0 2024-05-14T17:03:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.908+0000 7fbd76ff5640 10 client.25976 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:03:46.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.908+0000 7fbd76ff5640 1 -- 172.21.15.2:0/2616219626 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564999bdfb80 con 0x564999c85710 2024-05-14T17:03:46.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.908+0000 7fbd76ff5640 10 client.25976 handle_command_reply: tid=0 2024-05-14T17:03:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.909+0000 7fbd9c3d0740 1 client.25976 shutdown 2024-05-14T17:03:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.909+0000 7fbd9c3d0740 1 -- 172.21.15.2:0/2616219626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564999c6b210 msgr2=0x564999c72620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.909+0000 7fbd9c3d0740 1 --2- 172.21.15.2:0/2616219626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564999c6b210 0x564999c72620 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbd8801dd90 tx=0x7fbd88004730 comp rx=0 tx=0).stop 2024-05-14T17:03:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.909+0000 7fbd9c3d0740 1 -- 172.21.15.2:0/2616219626 shutdown_connections 2024-05-14T17:03:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.909+0000 7fbd9c3d0740 1 --2- 172.21.15.2:0/2616219626 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564999c85710 0x564999c87b50 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:46.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.909+0000 7fbd9c3d0740 1 --2- 172.21.15.2:0/2616219626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564999c75ad0 0x564999c72b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.909+0000 7fbd9c3d0740 1 --2- 172.21.15.2:0/2616219626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564999c6b210 0x564999c72620 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.909+0000 7fbd9c3d0740 1 --2- 172.21.15.2:0/2616219626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564999c6a8f0 0x564999c75330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.909+0000 7fbd76ff5640 0 client.25976 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:03:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.909+0000 7fbd9c3d0740 1 -- 172.21.15.2:0/2616219626 >> 172.21.15.2:0/2616219626 conn(0x564999c66e00 msgr2=0x564999c66100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.909+0000 7fbd9c3d0740 1 -- 172.21.15.2:0/2616219626 shutdown_connections 2024-05-14T17:03:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.909+0000 7fbd9c3d0740 1 -- 172.21.15.2:0/2616219626 wait complete. 2024-05-14T17:03:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.909+0000 7fbd9c3d0740 20 client.25976 trim_cache size 0 max 16384 2024-05-14T17:03:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.937+0000 7fbd9c3d0740 1 Processor -- start 2024-05-14T17:03:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.937+0000 7fbd9c3d0740 1 -- start start 2024-05-14T17:03:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.937+0000 7fbd9c3d0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564999c6a8f0 0x564999c6dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.937+0000 7fbd9c3d0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564999c6b210 0x564999c74e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:46.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.937+0000 7fbd9c3d0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564999c75ad0 0x564999c8d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:46.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.937+0000 7fbd9c3d0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbd941b7430 con 0x564999c6a8f0 2024-05-14T17:03:46.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.937+0000 7fbd9c3d0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbd941b4d00 con 0x564999c75ad0 2024-05-14T17:03:46.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.937+0000 7fbd9c3d0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbd941b2b70 con 0x564999c6b210 2024-05-14T17:03:46.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.937+0000 7fbd92ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564999c6a8f0 0x564999c6dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:46.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.938+0000 7fbd937fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564999c75ad0 0x564999c8d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:46.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.938+0000 7fbd92ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564999c6a8f0 0x564999c6dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38448/0 (socket says 172.21.15.2:38448) 2024-05-14T17:03:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.938+0000 7fbd92ffd640 1 -- 172.21.15.2:0/4134955466 learned_addr learned my addr 172.21.15.2:0/4134955466 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:03:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.938+0000 7fbd927fc640 1 --2- 172.21.15.2:0/4134955466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564999c6b210 0x564999c74e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.938+0000 7fbd937fe640 1 -- 172.21.15.2:0/4134955466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564999c6b210 msgr2=0x564999c74e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.938+0000 7fbd937fe640 1 --2- 172.21.15.2:0/4134955466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564999c6b210 0x564999c74e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.938+0000 7fbd937fe640 1 -- 172.21.15.2:0/4134955466 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564999c6a8f0 msgr2=0x564999c6dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:47.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.938+0000 7fbd937fe640 1 --2- 172.21.15.2:0/4134955466 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564999c6a8f0 0x564999c6dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.938+0000 7fbd937fe640 1 -- 172.21.15.2:0/4134955466 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564999c66c30 con 0x564999c75ad0 2024-05-14T17:03:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.938+0000 7fbd92ffd640 1 --2- 172.21.15.2:0/4134955466 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564999c6a8f0 0x564999c6dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:03:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.938+0000 7fbd937fe640 1 --2- 172.21.15.2:0/4134955466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564999c75ad0 0x564999c8d420 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c0ffda0 tx=0x7fbd8c00d910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78385e131870ae0d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.938+0000 7fbd78ff9640 1 -- 172.21.15.2:0/4134955466 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c0d20e0 con 0x564999c75ad0 2024-05-14T17:03:47.062 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.938+0000 7fbd78ff9640 1 -- 172.21.15.2:0/4134955466 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd8c0afe50 con 0x564999c75ad0 2024-05-14T17:03:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.938+0000 7fbd78ff9640 1 -- 172.21.15.2:0/4134955466 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c0eabe0 con 0x564999c75ad0 2024-05-14T17:03:47.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.939+0000 7fbd9c3d0740 1 -- 172.21.15.2:0/4134955466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564999c75ad0 msgr2=0x564999c8d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:47.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.939+0000 7fbd9c3d0740 1 --2- 172.21.15.2:0/4134955466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564999c75ad0 0x564999c8d420 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c0ffda0 tx=0x7fbd8c00d910 comp rx=0 tx=0).stop 2024-05-14T17:03:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.939+0000 7fbd9c3d0740 1 -- 172.21.15.2:0/4134955466 shutdown_connections 2024-05-14T17:03:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.939+0000 7fbd9c3d0740 1 --2- 172.21.15.2:0/4134955466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564999c75ad0 0x564999c8d420 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:47.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.939+0000 7fbd9c3d0740 1 --2- 172.21.15.2:0/4134955466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564999c6b210 0x564999c74e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.939+0000 7fbd9c3d0740 1 --2- 172.21.15.2:0/4134955466 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564999c6a8f0 0x564999c6dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:47.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.939+0000 7fbd9c3d0740 1 -- 172.21.15.2:0/4134955466 >> 172.21.15.2:0/4134955466 conn(0x564999c66e00 msgr2=0x564999c6cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:47.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.939+0000 7fbd9c3d0740 1 -- 172.21.15.2:0/4134955466 shutdown_connections 2024-05-14T17:03:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.939+0000 7fbd9c3d0740 1 -- 172.21.15.2:0/4134955466 wait complete. 2024-05-14T17:03:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.940+0000 7fbd9c3d0740 1 -- start start 2024-05-14T17:03:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.940+0000 7fbd9c3d0740 1 Processor -- start 2024-05-14T17:03:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.940+0000 7fbd9c3d0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564999c6a8f0 0x564999c7deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.940+0000 7fbd9c3d0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564999c6b210 0x564999c79220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.940+0000 7fbd9c3d0740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564999c75ad0 0x564999c79760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:47.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.940+0000 7fbd92ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564999c6a8f0 0x564999c7deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.940+0000 7fbd92ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564999c6a8f0 0x564999c7deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:56234/0 (socket says 172.21.15.2:56234) 2024-05-14T17:03:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.940+0000 7fbd9c3d0740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbd941b31c0 con 0x564999c75ad0 2024-05-14T17:03:47.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.940+0000 7fbd92ffd640 1 -- 172.21.15.2:0/670229356 learned_addr learned my addr 172.21.15.2:0/670229356 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:03:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.940+0000 7fbd9c3d0740 1 -- 172.21.15.2:0/670229356 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbd941b4630 con 0x564999c6b210 2024-05-14T17:03:47.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.940+0000 7fbd9c3d0740 1 -- 172.21.15.2:0/670229356 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbd941b3890 con 0x564999c6a8f0 2024-05-14T17:03:47.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.940+0000 7fbd927fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564999c6b210 0x564999c79220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:47.418 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.940+0000 7fbd937fe640 1 --2- 172.21.15.2:0/670229356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564999c75ad0 0x564999c79760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.941+0000 7fbd9c3d0740 10 client.0 _ll_register_callbacks cb 0x7fbd8801da40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:03:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.941+0000 7fbd927fc640 1 -- 172.21.15.2:0/670229356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564999c6a8f0 msgr2=0x564999c7deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:47.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.941+0000 7fbd927fc640 1 --2- 172.21.15.2:0/670229356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564999c6a8f0 0x564999c7deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:47.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.941+0000 7fbd927fc640 1 -- 172.21.15.2:0/670229356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564999c75ad0 msgr2=0x564999c79760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:47.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.941+0000 7fbd927fc640 1 --2- 172.21.15.2:0/670229356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564999c75ad0 0x564999c79760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.941+0000 7fbd927fc640 1 -- 172.21.15.2:0/670229356 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564999c79ea0 con 0x564999c6b210 2024-05-14T17:03:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.941+0000 7fbd9c3d0740 1 -- 172.21.15.2:0/670229356 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd8c0afe50 con 0x564999c6b210 2024-05-14T17:03:47.514 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.941+0000 7fbd92ffd640 1 --2- 172.21.15.2:0/670229356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564999c6a8f0 0x564999c7deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:03:47.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.941+0000 7fbd937fe640 1 --2- 172.21.15.2:0/670229356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564999c75ad0 0x564999c79760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:03:47.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.941+0000 7fbd927fc640 1 --2- 172.21.15.2:0/670229356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564999c6b210 0x564999c79220 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbd88005ab0 tx=0x7fbd88019cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fa3f7df0d3d6737 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.941+0000 7fbd76ff5640 10 client.25982 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T17:03:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.941+0000 7fbd76ff5640 1 -- 172.21.15.2:0/670229356 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd88021040 con 0x564999c6b210 2024-05-14T17:03:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.941+0000 7fbd76ff5640 1 -- 172.21.15.2:0/670229356 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7fbd8801b830 con 0x564999c6b210 2024-05-14T17:03:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.941+0000 7fbd76ff5640 1 -- 172.21.15.2:0/670229356 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fbd8801be60 con 0x564999c6b210 2024-05-14T17:03:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.941+0000 7fbd9c3d0740 10 client.25982 fetch_fsmap learned FSMap version 41 2024-05-14T17:03:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.941+0000 7fbd9c3d0740 1 -- 172.21.15.2:0/670229356 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=41}) -- 0x564999cefdd0 con 0x564999c6b210 2024-05-14T17:03:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.942+0000 7fbd76ff5640 1 -- 172.21.15.2:0/670229356 <== mon.2 v2:172.21.15.123:3300/0 4 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fbd88004020 con 0x564999c6b210 2024-05-14T17:03:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.942+0000 7fbd9c3d0740 10 client.25982 fetch_fsmap finished waiting for FSMap version 41 2024-05-14T17:03:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.942+0000 7fbd9c3d0740 10 client.25982 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:03:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.942+0000 7fbd9c3d0740 20 client.25982 populate_metadata read hostname 'smithi002' 2024-05-14T17:03:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.942+0000 7fbd9c3d0740 1 --2- 172.21.15.2:0/670229356 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564999cf0330 0x564999cf2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.942+0000 7fbd9c3d0740 4 client.25982 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:03:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.942+0000 7fbd9c3d0740 1 -- 172.21.15.2:0/670229356 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564999bd9360 con 0x564999cf0330 2024-05-14T17:03:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.942+0000 7fbd92ffd640 1 --2- 172.21.15.2:0/670229356 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564999cf0330 0x564999cf2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.942+0000 7fbd76ff5640 10 client.25982 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:03:47.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.942+0000 7fbd92ffd640 1 --2- 172.21.15.2:0/670229356 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564999cf0330 0x564999cf2750 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45a7676c4c517743 server_cookie=b3e15ea4d2969066 in_seq=0 out_seq=0 2024-05-14T17:03:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.972+0000 7fbd76ff5640 1 -- 172.21.15.2:0/670229356 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564999bd9360 con 0x564999cf0330 2024-05-14T17:03:47.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.972+0000 7fbd76ff5640 10 client.25982 handle_command_reply: tid=0 2024-05-14T17:03:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.972+0000 7fbd9c3d0740 1 client.25982 shutdown 2024-05-14T17:03:47.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.972+0000 7fbd9c3d0740 1 -- 172.21.15.2:0/670229356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564999c6b210 msgr2=0x564999c79220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:47.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.972+0000 7fbd9c3d0740 1 --2- 172.21.15.2:0/670229356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564999c6b210 0x564999c79220 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbd88005ab0 tx=0x7fbd88019cd0 comp rx=0 tx=0).stop 2024-05-14T17:03:47.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.972+0000 7fbd9c3d0740 1 -- 172.21.15.2:0/670229356 shutdown_connections 2024-05-14T17:03:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.972+0000 7fbd9c3d0740 1 --2- 172.21.15.2:0/670229356 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564999cf0330 0x564999cf2750 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:47.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.972+0000 7fbd9c3d0740 1 --2- 172.21.15.2:0/670229356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564999c75ad0 0x564999c79760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.972+0000 7fbd9c3d0740 1 --2- 172.21.15.2:0/670229356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564999c6b210 0x564999c79220 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.972+0000 7fbd76ff5640 0 client.25982 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:03:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.972+0000 7fbd9c3d0740 1 --2- 172.21.15.2:0/670229356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564999c6a8f0 0x564999c7deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.972+0000 7fbd9c3d0740 1 -- 172.21.15.2:0/670229356 >> 172.21.15.2:0/670229356 conn(0x564999c66e00 msgr2=0x564999c6cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.972+0000 7fbd9c3d0740 1 -- 172.21.15.2:0/670229356 shutdown_connections 2024-05-14T17:03:47.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.972+0000 7fbd9c3d0740 1 -- 172.21.15.2:0/670229356 wait complete. 2024-05-14T17:03:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.972+0000 7fbd9c3d0740 20 client.25982 trim_cache size 0 max 16384 2024-05-14T17:03:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:03:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:03:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:03:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.973+0000 7fbd995da640 1 -- 172.21.15.2:0/1431949550 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fbd7c0b2a90 msgr2=0x7fbd7c0b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.973+0000 7fbd995da640 1 --2- 172.21.15.2:0/1431949550 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fbd7c0b2a90 0x7fbd7c0b4f50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x564999c68220 tx=0x7fbd88009260 comp rx=0 tx=0).stop 2024-05-14T17:03:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.973+0000 7fbd995da640 1 -- 172.21.15.2:0/1431949550 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd94107960 msgr2=0x7fbd941a3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:48.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.973+0000 7fbd995da640 1 --2- 172.21.15.2:0/1431949550 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd94107960 0x7fbd941a3f50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c004af0 tx=0x7fbd8c034590 comp rx=0 tx=0).stop 2024-05-14T17:03:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.973+0000 7fbd995da640 1 -- 172.21.15.2:0/1431949550 shutdown_connections 2024-05-14T17:03:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.973+0000 7fbd995da640 1 --2- 172.21.15.2:0/1431949550 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fbd7c0b2a90 0x7fbd7c0b4f50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.973+0000 7fbd995da640 1 --2- 172.21.15.2:0/1431949550 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbd94107960 0x7fbd941a3f50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:48.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.973+0000 7fbd995da640 1 --2- 172.21.15.2:0/1431949550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd94102360 0x7fbd9419fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.973+0000 7fbd995da640 1 --2- 172.21.15.2:0/1431949550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd941019a0 0x7fbd9419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.973+0000 7fbd995da640 1 -- 172.21.15.2:0/1431949550 >> 172.21.15.2:0/1431949550 conn(0x7fbd94078aa0 msgr2=0x7fbd94104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.974+0000 7fbd995da640 1 -- 172.21.15.2:0/1431949550 shutdown_connections 2024-05-14T17:03:48.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:44.975+0000 7fbd995da640 1 -- 172.21.15.2:0/1431949550 wait complete. 2024-05-14T17:03:48.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:03:46 smithi002 ceph-mon[31538]: pgmap v281: 97 pgs: 97 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.1 MiB/s wr, 593 op/s 2024-05-14T17:03:48.264 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 510.0 s, 2446.0 tps, lat 40.953 ms stddev 45.598 2024-05-14T17:03:48.275 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:03:46 smithi123 ceph-mon[35971]: pgmap v281: 97 pgs: 97 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.1 MiB/s wr, 593 op/s 2024-05-14T17:03:48.284 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:03:46 smithi151 ceph-mon[36066]: pgmap v281: 97 pgs: 97 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.1 MiB/s wr, 593 op/s 2024-05-14T17:03:48.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:03:48 smithi002 ceph-mon[31538]: pgmap v282: 97 pgs: 97 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.1 MiB/s wr, 593 op/s 2024-05-14T17:03:48.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:03:48 smithi151 ceph-mon[36066]: pgmap v282: 97 pgs: 97 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.1 MiB/s wr, 593 op/s 2024-05-14T17:03:48.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:03:48 smithi123 ceph-mon[35971]: pgmap v282: 97 pgs: 97 active+clean; 4.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.1 MiB/s wr, 593 op/s 2024-05-14T17:03:50.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:03:50 smithi002 ceph-mon[31538]: pgmap v283: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 18 MiB/s wr, 1.08k op/s 2024-05-14T17:03:50.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:03:50 smithi151 ceph-mon[36066]: pgmap v283: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 18 MiB/s wr, 1.08k op/s 2024-05-14T17:03:50.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:03:50 smithi123 ceph-mon[35971]: pgmap v283: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 18 MiB/s wr, 1.08k op/s 2024-05-14T17:03:51.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 515.0 s, 2644.4 tps, lat 37.845 ms stddev 44.648 2024-05-14T17:03:52.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:03:52 smithi002 ceph-mon[31538]: pgmap v284: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 MiB/s wr, 789 op/s 2024-05-14T17:03:52.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:03:52 smithi151 ceph-mon[36066]: pgmap v284: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 MiB/s wr, 789 op/s 2024-05-14T17:03:52.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:03:52 smithi123 ceph-mon[35971]: pgmap v284: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 15 MiB/s wr, 789 op/s 2024-05-14T17:03:54.711 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:03:54 smithi123 ceph-mon[35971]: pgmap v285: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 24 MiB/s wr, 1.08k op/s 2024-05-14T17:03:54.752 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:03:54 smithi002 ceph-mon[31538]: pgmap v285: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 24 MiB/s wr, 1.08k op/s 2024-05-14T17:03:54.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:03:54 smithi151 ceph-mon[36066]: pgmap v285: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 24 MiB/s wr, 1.08k op/s 2024-05-14T17:03:56.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:03:56 smithi002 ceph-mon[31538]: pgmap v286: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 21 MiB/s wr, 783 op/s 2024-05-14T17:03:56.752 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 520.0 s, 2939.6 tps, lat 33.908 ms stddev 37.403 2024-05-14T17:03:56.853 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:03:56 smithi151 ceph-mon[36066]: pgmap v286: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 21 MiB/s wr, 783 op/s 2024-05-14T17:03:56.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:03:56 smithi123 ceph-mon[35971]: pgmap v286: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 21 MiB/s wr, 783 op/s 2024-05-14T17:03:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.113+0000 7f7bc559a640 1 -- 172.21.15.2:0/176730525 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc00318a0 con 0x7f7bc8102340 2024-05-14T17:03:58.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.114+0000 7f7bcc911640 1 -- 172.21.15.2:0/176730525 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bc8102340 msgr2=0x7f7bc810e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.114+0000 7f7bcc911640 1 --2- 172.21.15.2:0/176730525 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bc8102340 0x7f7bc810e800 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0009930 tx=0x7f7bc002a460 comp rx=0 tx=0).stop 2024-05-14T17:03:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.114+0000 7f7bcc911640 1 -- 172.21.15.2:0/176730525 shutdown_connections 2024-05-14T17:03:58.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.114+0000 7f7bcc911640 1 --2- 172.21.15.2:0/176730525 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bc8102340 0x7f7bc810e800 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:58.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.114+0000 7f7bcc911640 1 --2- 172.21.15.2:0/176730525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7bc8101980 0x7f7bc8101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.114+0000 7f7bcc911640 1 --2- 172.21.15.2:0/176730525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bc8107940 0x7f7bc8107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:58.180 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.114+0000 7f7bcc911640 1 -- 172.21.15.2:0/176730525 >> 172.21.15.2:0/176730525 conn(0x7f7bc8078ad0 msgr2=0x7f7bc8078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.114+0000 7f7bcc911640 1 -- 172.21.15.2:0/176730525 shutdown_connections 2024-05-14T17:03:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.114+0000 7f7bcc911640 1 -- 172.21.15.2:0/176730525 wait complete. 2024-05-14T17:03:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.115+0000 7f7bcc911640 1 Processor -- start 2024-05-14T17:03:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.116+0000 7f7bcc911640 1 -- start start 2024-05-14T17:03:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.116+0000 7f7bcc911640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7bc8101980 0x7f7bc81992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:58.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.116+0000 7f7bcc911640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bc8102340 0x7f7bc8199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.116+0000 7f7bcc911640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bc8107940 0x7f7bc819dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:58.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.116+0000 7f7bcc911640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc8111cf0 con 0x7f7bc8102340 2024-05-14T17:03:58.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.116+0000 7f7bcc911640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc8111b70 con 0x7f7bc8107940 2024-05-14T17:03:58.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.116+0000 7f7bcc911640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc8111e70 con 0x7f7bc8101980 2024-05-14T17:03:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.116+0000 7f7bc5d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bc8102340 0x7f7bc8199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.116+0000 7f7bc6d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bc8107940 0x7f7bc819dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:58.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.116+0000 7f7bc659c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7bc8101980 0x7f7bc81992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:58.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.116+0000 7f7bc6d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bc8107940 0x7f7bc819dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:39830/0 (socket says 172.21.15.2:39830) 2024-05-14T17:03:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.116+0000 7f7bc6d9d640 1 -- 172.21.15.2:0/3232215945 learned_addr learned my addr 172.21.15.2:0/3232215945 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:03:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.117+0000 7f7bc5d9b640 1 -- 172.21.15.2:0/3232215945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7bc8101980 msgr2=0x7f7bc81992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.117+0000 7f7bc5d9b640 1 --2- 172.21.15.2:0/3232215945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7bc8101980 0x7f7bc81992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:58.402 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.117+0000 7f7bc5d9b640 1 -- 172.21.15.2:0/3232215945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bc8107940 msgr2=0x7f7bc819dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:58.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.117+0000 7f7bc5d9b640 1 --2- 172.21.15.2:0/3232215945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bc8107940 0x7f7bc819dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:58.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.117+0000 7f7bc5d9b640 1 -- 172.21.15.2:0/3232215945 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc819e2d0 con 0x7f7bc8102340 2024-05-14T17:03:58.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.117+0000 7f7bc6d9d640 1 --2- 172.21.15.2:0/3232215945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bc8107940 0x7f7bc819dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:03:58.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.117+0000 7f7bc5d9b640 1 --2- 172.21.15.2:0/3232215945 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bc8102340 0x7f7bc8199820 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0009fd0 tx=0x7f7bb000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:58.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.117+0000 7f7bbf7fe640 1 -- 172.21.15.2:0/3232215945 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb0015020 con 0x7f7bc8102340 2024-05-14T17:03:58.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.117+0000 7f7bcc911640 1 -- 172.21.15.2:0/3232215945 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bc819e5c0 con 0x7f7bc8102340 2024-05-14T17:03:58.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.117+0000 7f7bbf7fe640 1 -- 172.21.15.2:0/3232215945 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bb0011ce0 con 0x7f7bc8102340 2024-05-14T17:03:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.117+0000 7f7bbf7fe640 1 -- 172.21.15.2:0/3232215945 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb0010e00 con 0x7f7bc8102340 2024-05-14T17:03:58.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.117+0000 7f7bcc911640 1 -- 172.21.15.2:0/3232215945 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bc80fe840 con 0x7f7bc8102340 2024-05-14T17:03:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.121+0000 7f7bcf621740 1 Processor -- start 2024-05-14T17:03:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.122+0000 7f7bcf621740 1 -- start start 2024-05-14T17:03:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.122+0000 7f7bcf621740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55823fe3d8f0 0x55823fe3dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.122+0000 7f7bbf7fe640 1 -- 172.21.15.2:0/3232215945 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f7bb0018020 con 0x7f7bc8102340 2024-05-14T17:03:58.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.122+0000 7f7bcf621740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55823fe3e210 0x55823fe48590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:58.578 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.122+0000 7f7bcf621740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823fe48ad0 0x55823fe4aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.122+0000 7f7bcf621740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55823fdb0c30 con 0x55823fe3e210 2024-05-14T17:03:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.122+0000 7f7bcf621740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55823fda07d0 con 0x55823fe3d8f0 2024-05-14T17:03:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.122+0000 7f7bcf621740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55823fda4810 con 0x55823fe48ad0 2024-05-14T17:03:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.122+0000 7f7bc659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55823fe3e210 0x55823fe48590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.122+0000 7f7bc6d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55823fe3d8f0 0x55823fe3dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.122+0000 7f7bc659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55823fe3e210 0x55823fe48590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55736/0 (socket says 172.21.15.2:55736) 2024-05-14T17:03:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.122+0000 7f7bc6d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55823fe3d8f0 0x55823fe3dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:39840/0 (socket says 172.21.15.2:39840) 2024-05-14T17:03:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.122+0000 7f7bc659c640 1 -- 172.21.15.2:0/3322188043 learned_addr learned my addr 172.21.15.2:0/3322188043 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:03:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.123+0000 7f7bc5d9b640 1 --2- 172.21.15.2:0/3322188043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823fe48ad0 0x55823fe4aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.124+0000 7f7bc5d9b640 1 -- 172.21.15.2:0/3322188043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55823fe3d8f0 msgr2=0x55823fe3dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.124+0000 7f7bc5d9b640 1 --2- 172.21.15.2:0/3322188043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55823fe3d8f0 0x55823fe3dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.124+0000 7f7bc5d9b640 1 -- 172.21.15.2:0/3322188043 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55823fe3e210 msgr2=0x55823fe48590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.124+0000 7f7bc5d9b640 1 --2- 172.21.15.2:0/3322188043 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55823fe3e210 0x55823fe48590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:58.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.124+0000 7f7bc5d9b640 1 -- 172.21.15.2:0/3322188043 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55823fab5210 con 0x55823fe48ad0 2024-05-14T17:03:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.124+0000 7f7bc659c640 1 --2- 172.21.15.2:0/3322188043 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55823fe3e210 0x55823fe48590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:03:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.124+0000 7f7bc6d9d640 1 --2- 172.21.15.2:0/3322188043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55823fe3d8f0 0x55823fe3dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:03:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.124+0000 7f7bc5d9b640 1 --2- 172.21.15.2:0/3322188043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823fe48ad0 0x55823fe4aec0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7bb00df500 tx=0x7f7bb00ea7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72d8442b5d67b51f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.124+0000 7f7bbcff9640 1 -- 172.21.15.2:0/3322188043 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb00f1070 con 0x55823fe48ad0 2024-05-14T17:03:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.124+0000 7f7bbcff9640 1 -- 172.21.15.2:0/3322188043 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bb00e7cb0 con 0x55823fe48ad0 2024-05-14T17:03:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.124+0000 7f7bbcff9640 1 -- 172.21.15.2:0/3322188043 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb00e6ca0 con 0x55823fe48ad0 2024-05-14T17:03:58.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.125+0000 7f7bbf7fe640 1 --2- 172.21.15.2:0/3232215945 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f7bb40b2a10 0x7f7bb40b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.125+0000 7f7bc6d9d640 1 --2- 172.21.15.2:0/3232215945 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f7bb40b2a10 0x7f7bb40b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:58.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.125+0000 7f7bbf7fe640 1 -- 172.21.15.2:0/3232215945 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f7bb00e1ca0 con 0x7f7bc8102340 2024-05-14T17:03:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.125+0000 7f7bc6d9d640 1 --2- 172.21.15.2:0/3232215945 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f7bb40b2a10 0x7f7bb40b4ed0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7bc00096c0 tx=0x7f7bc00058d0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:58.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.125+0000 7f7bcf621740 1 -- 172.21.15.2:0/3322188043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823fe48ad0 msgr2=0x55823fe4aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:58.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.125+0000 7f7bcf621740 1 --2- 172.21.15.2:0/3322188043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823fe48ad0 0x55823fe4aec0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7bb00df500 tx=0x7f7bb00ea7e0 comp rx=0 tx=0).stop 2024-05-14T17:03:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.125+0000 7f7bcf621740 1 -- 172.21.15.2:0/3322188043 shutdown_connections 2024-05-14T17:03:58.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.125+0000 7f7bcf621740 1 --2- 172.21.15.2:0/3322188043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823fe48ad0 0x55823fe4aec0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:58.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.125+0000 7f7bcf621740 1 --2- 172.21.15.2:0/3322188043 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55823fe3e210 0x55823fe48590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.125+0000 7f7bcf621740 1 --2- 172.21.15.2:0/3322188043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55823fe3d8f0 0x55823fe3dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:58.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.125+0000 7f7bcf621740 1 -- 172.21.15.2:0/3322188043 >> 172.21.15.2:0/3322188043 conn(0x55823fe39e00 msgr2=0x55823fe3a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:58.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.126+0000 7f7bcf621740 1 -- 172.21.15.2:0/3322188043 shutdown_connections 2024-05-14T17:03:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.126+0000 7f7bcf621740 1 -- 172.21.15.2:0/3322188043 wait complete. 2024-05-14T17:03:58.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.126+0000 7f7bcf621740 1 -- start start 2024-05-14T17:03:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.126+0000 7f7bcf621740 1 Processor -- start 2024-05-14T17:03:58.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.127+0000 7f7bcf621740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823fe3d8f0 0x55823fe484d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.127+0000 7f7bcf621740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55823fe3e210 0x55823fe457c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.127+0000 7f7bcf621740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55823fe48ad0 0x55823fe45dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.127+0000 7f7bcf621740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55823fa86790 con 0x55823fe3e210 2024-05-14T17:03:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.127+0000 7f7bc6d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55823fe48ad0 0x55823fe45dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:59.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.127+0000 7f7bc659c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823fe3d8f0 0x55823fe484d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:59.031 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.127+0000 7f7bc5d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55823fe3e210 0x55823fe457c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.127+0000 7f7bc6d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55823fe48ad0 0x55823fe45dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:39856/0 (socket says 172.21.15.2:39856) 2024-05-14T17:03:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.127+0000 7f7bc659c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823fe3d8f0 0x55823fe484d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:47002/0 (socket says 172.21.15.2:47002) 2024-05-14T17:03:59.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.127+0000 7f7bc6d9d640 1 -- 172.21.15.2:0/1242661446 learned_addr learned my addr 172.21.15.2:0/1242661446 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:03:59.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.127+0000 7f7bcf621740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55823fd5c890 con 0x55823fe48ad0 2024-05-14T17:03:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.127+0000 7f7bcf621740 1 -- 172.21.15.2:0/1242661446 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55823fe4b620 con 0x55823fe3d8f0 2024-05-14T17:03:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.128+0000 7f7bcf621740 10 client.0 _ll_register_callbacks cb 0x55823fd5fd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:03:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.128+0000 7f7bc659c640 1 -- 172.21.15.2:0/1242661446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55823fe48ad0 msgr2=0x55823fe45dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.128+0000 7f7bc659c640 1 --2- 172.21.15.2:0/1242661446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55823fe48ad0 0x55823fe45dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.128+0000 7f7bc659c640 1 -- 172.21.15.2:0/1242661446 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55823fe3e210 msgr2=0x55823fe457c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.128+0000 7f7bc659c640 1 --2- 172.21.15.2:0/1242661446 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55823fe3e210 0x55823fe457c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.128+0000 7f7bc659c640 1 -- 172.21.15.2:0/1242661446 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55823fb81620 con 0x55823fe3d8f0 2024-05-14T17:03:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.128+0000 7f7bc659c640 1 --2- 172.21.15.2:0/1242661446 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823fe3d8f0 0x55823fe484d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7bb80118e0 tx=0x7f7bb801a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3ca83cf3a006028 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.128+0000 7f7badffb640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-14T17:03:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.129+0000 7f7badffb640 1 -- 172.21.15.2:0/1242661446 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb800fe60 con 0x55823fe3d8f0 2024-05-14T17:03:59.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.129+0000 7f7bc5d9b640 1 --2- 172.21.15.2:0/1242661446 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55823fe3e210 0x55823fe457c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:03:59.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.129+0000 7f7bcf621740 1 -- 172.21.15.2:0/1242661446 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bb00e7cb0 con 0x55823fe3d8f0 2024-05-14T17:03:59.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.129+0000 7f7badffb640 1 -- 172.21.15.2:0/1242661446 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f7bb801d650 con 0x55823fe3d8f0 2024-05-14T17:03:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.129+0000 7f7bc6d9d640 1 --2- 172.21.15.2:0/1242661446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55823fe48ad0 0x55823fe45dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:03:59.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.129+0000 7f7badffb640 1 -- 172.21.15.2:0/1242661446 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7bb801d890 con 0x55823fe3d8f0 2024-05-14T17:03:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.129+0000 7f7bcf621740 10 client.25972 fetch_fsmap learned FSMap version 41 2024-05-14T17:03:59.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.129+0000 7f7bcf621740 1 -- 172.21.15.2:0/1242661446 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55823fe570e0 con 0x55823fe3d8f0 2024-05-14T17:03:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.129+0000 7f7badffb640 1 -- 172.21.15.2:0/1242661446 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bb80273e0 con 0x55823fe3d8f0 2024-05-14T17:03:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.129+0000 7f7badffb640 1 -- 172.21.15.2:0/1242661446 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f7bb801c3b0 con 0x55823fe3d8f0 2024-05-14T17:03:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.130+0000 7f7bcf621740 10 client.25972 fetch_fsmap finished waiting for FSMap version 41 2024-05-14T17:03:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.130+0000 7f7bcf621740 10 client.25972 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:03:59.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.130+0000 7f7bcf621740 20 client.25972 populate_metadata read hostname 'smithi002' 2024-05-14T17:03:59.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.130+0000 7f7bcf621740 1 --2- 172.21.15.2:0/1242661446 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55823fe585f0 0x55823fe5aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.130+0000 7f7bcf621740 4 client.25972 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:03:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.130+0000 7f7bcf621740 1 -- 172.21.15.2:0/1242661446 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7bb00e7cb0 con 0x55823fe585f0 2024-05-14T17:03:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.130+0000 7f7bc5d9b640 1 --2- 172.21.15.2:0/1242661446 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55823fe585f0 0x55823fe5aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.130+0000 7f7badffb640 10 client.25972 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:03:59.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.130+0000 7f7bc5d9b640 1 --2- 172.21.15.2:0/1242661446 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55823fe585f0 0x55823fe5aa80 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ac9b2663498c4bb server_cookie=f85cef56595a4366 in_seq=0 out_seq=0 2024-05-14T17:03:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.131+0000 7f7badffb640 1 -- 172.21.15.2:0/1242661446 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f7bb00e7cb0 con 0x55823fe585f0 2024-05-14T17:03:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.131+0000 7f7badffb640 10 client.25972 handle_command_reply: tid=0 2024-05-14T17:03:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.131+0000 7f7bcf621740 1 client.25972 shutdown 2024-05-14T17:03:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.132+0000 7f7bcf621740 1 -- 172.21.15.2:0/1242661446 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823fe3d8f0 msgr2=0x55823fe484d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:03:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.132+0000 7f7bcf621740 1 --2- 172.21.15.2:0/1242661446 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823fe3d8f0 0x55823fe484d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7bb80118e0 tx=0x7f7bb801a6a0 comp rx=0 tx=0).stop 2024-05-14T17:03:59.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.132+0000 7f7bcf621740 1 -- 172.21.15.2:0/1242661446 shutdown_connections 2024-05-14T17:03:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.132+0000 7f7bcf621740 1 --2- 172.21.15.2:0/1242661446 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55823fe585f0 0x55823fe5aa80 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.132+0000 7f7bcf621740 1 --2- 172.21.15.2:0/1242661446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55823fe48ad0 0x55823fe45dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:59.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.132+0000 7f7bcf621740 1 --2- 172.21.15.2:0/1242661446 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55823fe3e210 0x55823fe457c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.132+0000 7f7badffb640 0 client.25972 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:03:59.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.132+0000 7f7bcf621740 1 --2- 172.21.15.2:0/1242661446 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823fe3d8f0 0x55823fe484d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:59.633 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.132+0000 7f7bcf621740 1 -- 172.21.15.2:0/1242661446 >> 172.21.15.2:0/1242661446 conn(0x55823fe39e00 msgr2=0x55823fe39100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:03:59.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.132+0000 7f7bcf621740 1 -- 172.21.15.2:0/1242661446 shutdown_connections 2024-05-14T17:03:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.132+0000 7f7bcf621740 1 -- 172.21.15.2:0/1242661446 wait complete. 2024-05-14T17:03:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.133+0000 7f7bcf621740 20 client.25972 trim_cache size 0 max 16384 2024-05-14T17:03:59.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.159+0000 7f7bcf621740 1 Processor -- start 2024-05-14T17:03:59.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.159+0000 7f7bcf621740 1 -- start start 2024-05-14T17:03:59.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.160+0000 7f7bcf621740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55823fe3d8f0 0x55823fe3a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.160+0000 7f7bcf621740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55823fe3e210 0x55823fe544f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.160+0000 7f7bcf621740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823fe48ad0 0x55823fe48000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:03:59.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.160+0000 7f7bcf621740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc81aee10 con 0x55823fe3d8f0 2024-05-14T17:03:59.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.160+0000 7f7bcf621740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc81ac6e0 con 0x55823fe3e210 2024-05-14T17:03:59.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.160+0000 7f7bcf621740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc81aa590 con 0x55823fe48ad0 2024-05-14T17:03:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.160+0000 7f7bc659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55823fe3d8f0 0x55823fe3a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.160+0000 7f7bc6d9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823fe48ad0 0x55823fe48000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:59.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.160+0000 7f7bc659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55823fe3d8f0 0x55823fe3a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55750/0 (socket says 172.21.15.2:55750) 2024-05-14T17:03:59.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.160+0000 7f7bc5d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55823fe3e210 0x55823fe544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:03:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.160+0000 7f7bc659c640 1 -- 172.21.15.2:0/4235948747 learned_addr learned my addr 172.21.15.2:0/4235948747 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:03:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.160+0000 7f7bc6d9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823fe48ad0 0x55823fe48000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:47018/0 (socket says 172.21.15.2:47018) 2024-05-14T17:03:59.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.160+0000 7f7bc5d9b640 1 -- 172.21.15.2:0/4235948747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823fe48ad0 msgr2=0x55823fe48000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:59.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.160+0000 7f7bc5d9b640 1 --2- 172.21.15.2:0/4235948747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823fe48ad0 0x55823fe48000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.160+0000 7f7bc5d9b640 1 -- 172.21.15.2:0/4235948747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55823fe3d8f0 msgr2=0x55823fe3a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:03:59.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.160+0000 7f7bc5d9b640 1 --2- 172.21.15.2:0/4235948747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55823fe3d8f0 0x55823fe3a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:03:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.160+0000 7f7bc5d9b640 1 -- 172.21.15.2:0/4235948747 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55823fe3e040 con 0x55823fe3e210 2024-05-14T17:03:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.161+0000 7f7bc5d9b640 1 --2- 172.21.15.2:0/4235948747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55823fe3e210 0x55823fe544f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7bb00df560 tx=0x7f7bb0010380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ccbc8d0daabc113 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:03:59.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.161+0000 7f7baffff640 1 -- 172.21.15.2:0/4235948747 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb00f1040 con 0x55823fe3e210 2024-05-14T17:03:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.161+0000 7f7baffff640 1 -- 172.21.15.2:0/4235948747 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bb00a8ba0 con 0x55823fe3e210 2024-05-14T17:03:59.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.161+0000 7f7baffff640 1 -- 172.21.15.2:0/4235948747 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb00e7e60 con 0x55823fe3e210 2024-05-14T17:03:59.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.162+0000 7f7bcf621740 1 -- 172.21.15.2:0/4235948747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55823fe3e210 msgr2=0x55823fe544f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:00.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.162+0000 7f7bcf621740 1 --2- 172.21.15.2:0/4235948747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55823fe3e210 0x55823fe544f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7bb00df560 tx=0x7f7bb0010380 comp rx=0 tx=0).stop 2024-05-14T17:04:00.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.163+0000 7f7bcf621740 1 -- 172.21.15.2:0/4235948747 shutdown_connections 2024-05-14T17:04:00.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.163+0000 7f7bcf621740 1 --2- 172.21.15.2:0/4235948747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823fe48ad0 0x55823fe48000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.163+0000 7f7bcf621740 1 --2- 172.21.15.2:0/4235948747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55823fe3e210 0x55823fe544f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:00.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.163+0000 7f7bcf621740 1 --2- 172.21.15.2:0/4235948747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55823fe3d8f0 0x55823fe3a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.163+0000 7f7bcf621740 1 -- 172.21.15.2:0/4235948747 >> 172.21.15.2:0/4235948747 conn(0x55823fe39e00 msgr2=0x55823fe40dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.163+0000 7f7bcf621740 1 -- 172.21.15.2:0/4235948747 shutdown_connections 2024-05-14T17:04:00.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.163+0000 7f7bcf621740 1 -- 172.21.15.2:0/4235948747 wait complete. 2024-05-14T17:04:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.163+0000 7f7bcf621740 1 -- start start 2024-05-14T17:04:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.164+0000 7f7bcf621740 1 Processor -- start 2024-05-14T17:04:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.164+0000 7f7bcf621740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823fe3d8f0 0x55823fe50c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.164+0000 7f7bcf621740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55823fe3e210 0x55823fe4bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.164+0000 7f7bc5d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55823fe3e210 0x55823fe4bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.164+0000 7f7bc5d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55823fe3e210 0x55823fe4bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:39870/0 (socket says 172.21.15.2:39870) 2024-05-14T17:04:00.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.164+0000 7f7bc5d9b640 1 -- 172.21.15.2:0/1479395341 learned_addr learned my addr 172.21.15.2:0/1479395341 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:04:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.164+0000 7f7bc659c640 1 --2- 172.21.15.2:0/1479395341 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823fe3d8f0 0x55823fe50c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:00.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.165+0000 7f7bcf621740 1 --2- 172.21.15.2:0/1479395341 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55823fe48ad0 0x55823fe4c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:00.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.165+0000 7f7bcf621740 1 -- 172.21.15.2:0/1479395341 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc81aabe0 con 0x55823fe48ad0 2024-05-14T17:04:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.165+0000 7f7bcf621740 1 -- 172.21.15.2:0/1479395341 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc81ac010 con 0x55823fe3e210 2024-05-14T17:04:00.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.165+0000 7f7bcf621740 1 -- 172.21.15.2:0/1479395341 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc81ab270 con 0x55823fe3d8f0 2024-05-14T17:04:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.165+0000 7f7bcf621740 10 client.0 _ll_register_callbacks cb 0x7f7bb8011520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:04:00.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.165+0000 7f7bc6d9d640 1 --2- 172.21.15.2:0/1479395341 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55823fe48ad0 0x55823fe4c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:00.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.165+0000 7f7bc6d9d640 1 -- 172.21.15.2:0/1479395341 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823fe3d8f0 msgr2=0x55823fe50c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.165+0000 7f7bc6d9d640 1 --2- 172.21.15.2:0/1479395341 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823fe3d8f0 0x55823fe50c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.165+0000 7f7bc6d9d640 1 -- 172.21.15.2:0/1479395341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55823fe3e210 msgr2=0x55823fe4bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.165+0000 7f7bc6d9d640 1 --2- 172.21.15.2:0/1479395341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55823fe3e210 0x55823fe4bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.165+0000 7f7bc6d9d640 1 -- 172.21.15.2:0/1479395341 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55823fe4cba0 con 0x55823fe48ad0 2024-05-14T17:04:00.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.165+0000 7f7bc5d9b640 1 --2- 172.21.15.2:0/1479395341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55823fe3e210 0x55823fe4bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:04:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.166+0000 7f7bc6d9d640 1 --2- 172.21.15.2:0/1479395341 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55823fe48ad0 0x55823fe4c530 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0031440 tx=0x7f7bc003c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aa56e91db670930 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.166+0000 7f7badffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:04:00.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.166+0000 7f7badffb640 1 -- 172.21.15.2:0/1479395341 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0055060 con 0x55823fe48ad0 2024-05-14T17:04:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.166+0000 7f7badffb640 1 -- 172.21.15.2:0/1479395341 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8513+0+0 (secure 0 0 0) 0x7f7bc0003ac0 con 0x55823fe48ad0 2024-05-14T17:04:00.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.166+0000 7f7bcf621740 1 -- 172.21.15.2:0/1479395341 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bb00a8ba0 con 0x55823fe48ad0 2024-05-14T17:04:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.166+0000 7f7badffb640 1 -- 172.21.15.2:0/1479395341 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7bc0035040 con 0x55823fe48ad0 2024-05-14T17:04:00.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.166+0000 7f7bcf621740 10 client.16320 fetch_fsmap learned FSMap version 41 2024-05-14T17:04:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.166+0000 7f7bcf621740 1 -- 172.21.15.2:0/1479395341 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55823fec2de0 con 0x55823fe48ad0 2024-05-14T17:04:00.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.166+0000 7f7badffb640 1 -- 172.21.15.2:0/1479395341 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bb00a8ba0 con 0x55823fe48ad0 2024-05-14T17:04:00.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.167+0000 7f7badffb640 1 -- 172.21.15.2:0/1479395341 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f7bc003a040 con 0x55823fe48ad0 2024-05-14T17:04:00.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.167+0000 7f7bcf621740 10 client.16320 fetch_fsmap finished waiting for FSMap version 41 2024-05-14T17:04:00.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.167+0000 7f7bcf621740 10 client.16320 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:04:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.167+0000 7f7bcf621740 20 client.16320 populate_metadata read hostname 'smithi002' 2024-05-14T17:04:00.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.167+0000 7f7bcf621740 1 --2- 172.21.15.2:0/1479395341 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55823fe585f0 0x55823fec5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.167+0000 7f7bcf621740 4 client.16320 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:04:00.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.167+0000 7f7bcf621740 1 -- 172.21.15.2:0/1479395341 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55823fdb2b80 con 0x55823fe585f0 2024-05-14T17:04:00.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.167+0000 7f7bc659c640 1 --2- 172.21.15.2:0/1479395341 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55823fe585f0 0x55823fec5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:00.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.169+0000 7f7bc659c640 1 --2- 172.21.15.2:0/1479395341 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55823fe585f0 0x55823fec5600 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7077cfdbd25f83 server_cookie=4e06ccc008bec622 in_seq=0 out_seq=0 2024-05-14T17:04:00.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.169+0000 7f7badffb640 10 client.16320 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:04:00.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.178+0000 7f7badffb640 1 -- 172.21.15.2:0/1479395341 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55823fdb2b80 con 0x55823fe585f0 2024-05-14T17:04:00.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.178+0000 7f7badffb640 10 client.16320 handle_command_reply: tid=0 2024-05-14T17:04:00.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.178+0000 7f7bcf621740 1 client.16320 shutdown 2024-05-14T17:04:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.179+0000 7f7bcf621740 1 -- 172.21.15.2:0/1479395341 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55823fe48ad0 msgr2=0x55823fe4c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.179+0000 7f7bcf621740 1 --2- 172.21.15.2:0/1479395341 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55823fe48ad0 0x55823fe4c530 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0031440 tx=0x7f7bc003c040 comp rx=0 tx=0).stop 2024-05-14T17:04:00.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.179+0000 7f7bcf621740 1 -- 172.21.15.2:0/1479395341 shutdown_connections 2024-05-14T17:04:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.179+0000 7f7bcf621740 1 --2- 172.21.15.2:0/1479395341 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55823fe585f0 0x55823fec5600 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.179+0000 7f7bcf621740 1 --2- 172.21.15.2:0/1479395341 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55823fe48ad0 0x55823fe4c530 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.179+0000 7f7bcf621740 1 --2- 172.21.15.2:0/1479395341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55823fe3e210 0x55823fe4bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:00.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.179+0000 7f7bcf621740 1 --2- 172.21.15.2:0/1479395341 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55823fe3d8f0 0x55823fe50c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.179+0000 7f7bcf621740 1 -- 172.21.15.2:0/1479395341 >> 172.21.15.2:0/1479395341 conn(0x55823fe39e00 msgr2=0x55823fe40dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.179+0000 7f7badffb640 0 client.16320 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:04:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.179+0000 7f7bcf621740 1 -- 172.21.15.2:0/1479395341 shutdown_connections 2024-05-14T17:04:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.179+0000 7f7bcf621740 1 -- 172.21.15.2:0/1479395341 wait complete. 2024-05-14T17:04:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.179+0000 7f7bcf621740 20 client.16320 trim_cache size 0 max 16384 2024-05-14T17:04:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:04:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:04:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:04:00.967 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:03:58 smithi123 ceph-mon[35971]: pgmap v287: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 21 MiB/s wr, 783 op/s 2024-05-14T17:04:00.976 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:03:58 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:04:00.985 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:03:59 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:04:01.025 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:03:59 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:04:01.035 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:03:59 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:04:01.046 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:04:00 smithi123 ceph-mon[35971]: pgmap v288: 97 pgs: 97 active+clean; 4.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 28 MiB/s wr, 1.08k op/s 2024-05-14T17:04:01.056 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:03:58 smithi151 ceph-mon[36066]: pgmap v287: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 21 MiB/s wr, 783 op/s 2024-05-14T17:04:01.064 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:03:58 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:04:01.072 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:03:59 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:04:01.079 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:03:59 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:04:01.088 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:03:59 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:04:01.096 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:04:00 smithi151 ceph-mon[36066]: pgmap v288: 97 pgs: 97 active+clean; 4.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 28 MiB/s wr, 1.08k op/s 2024-05-14T17:04:01.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.179+0000 7f7bcc911640 1 -- 172.21.15.2:0/3232215945 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f7bb40b2a10 msgr2=0x7f7bb40b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:01.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.179+0000 7f7bcc911640 1 --2- 172.21.15.2:0/3232215945 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f7bb40b2a10 0x7f7bb40b4ed0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7bc00096c0 tx=0x7f7bc00058d0 comp rx=0 tx=0).stop 2024-05-14T17:04:01.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.180+0000 7f7bcc911640 1 -- 172.21.15.2:0/3232215945 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bc8102340 msgr2=0x7f7bc8199820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:01.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.180+0000 7f7bcc911640 1 --2- 172.21.15.2:0/3232215945 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bc8102340 0x7f7bc8199820 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0009fd0 tx=0x7f7bb000b870 comp rx=0 tx=0).stop 2024-05-14T17:04:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.180+0000 7f7bcc911640 1 -- 172.21.15.2:0/3232215945 shutdown_connections 2024-05-14T17:04:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.180+0000 7f7bcc911640 1 --2- 172.21.15.2:0/3232215945 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f7bb40b2a10 0x7f7bb40b4ed0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.180+0000 7f7bcc911640 1 --2- 172.21.15.2:0/3232215945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bc8107940 0x7f7bc819dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:01.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.180+0000 7f7bcc911640 1 --2- 172.21.15.2:0/3232215945 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bc8102340 0x7f7bc8199820 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:01.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.180+0000 7f7bcc911640 1 --2- 172.21.15.2:0/3232215945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7bc8101980 0x7f7bc81992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:01.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.180+0000 7f7bcc911640 1 -- 172.21.15.2:0/3232215945 >> 172.21.15.2:0/3232215945 conn(0x7f7bc8078ad0 msgr2=0x7f7bc8100b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:01.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.180+0000 7f7bcc911640 1 -- 172.21.15.2:0/3232215945 shutdown_connections 2024-05-14T17:04:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:03:58.180+0000 7f7bcc911640 1 -- 172.21.15.2:0/3232215945 wait complete. 2024-05-14T17:04:01.300 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:03:58 smithi002 ceph-mon[31538]: pgmap v287: 97 pgs: 97 active+clean; 4.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 21 MiB/s wr, 783 op/s 2024-05-14T17:04:01.310 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:03:58 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:04:01.323 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:03:59 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:04:01.333 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:03:59 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:04:01.340 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:03:59 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:04:01.350 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:04:00 smithi002 ceph-mon[31538]: pgmap v288: 97 pgs: 97 active+clean; 4.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 28 MiB/s wr, 1.08k op/s 2024-05-14T17:04:01.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 525.0 s, 1817.2 tps, lat 51.508 ms stddev 74.383 2024-05-14T17:04:02.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:04:02 smithi002 ceph-mon[31538]: pgmap v289: 97 pgs: 97 active+clean; 4.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 587 op/s 2024-05-14T17:04:02.796 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.4 2024-05-14T17:04:02.805 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:04:02.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:04:02 smithi151 ceph-mon[36066]: pgmap v289: 97 pgs: 97 active+clean; 4.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 587 op/s 2024-05-14T17:04:02.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:04:02 smithi123 ceph-mon[35971]: pgmap v289: 97 pgs: 97 active+clean; 4.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 587 op/s 2024-05-14T17:04:03.404 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:04:04.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:04:04 smithi002 ceph-mon[31538]: pgmap v290: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 20 MiB/s wr, 785 op/s 2024-05-14T17:04:04.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:04:04 smithi151 ceph-mon[36066]: pgmap v290: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 20 MiB/s wr, 785 op/s 2024-05-14T17:04:04.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:04:04 smithi123 ceph-mon[35971]: pgmap v290: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 20 MiB/s wr, 785 op/s 2024-05-14T17:04:05.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.536+0000 7fa875fbe700 1 -- 172.21.15.2:0/861351034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa870102b10 msgr2=0x7fa8701068d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:05.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.536+0000 7fa875fbe700 1 --2- 172.21.15.2:0/861351034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa870102b10 0x7fa8701068d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa86800b0a0 tx=0x7fa86800b3b0 comp rx=0 tx=0).stop 2024-05-14T17:04:05.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.537+0000 7fa875fbe700 1 -- 172.21.15.2:0/861351034 shutdown_connections 2024-05-14T17:04:05.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.537+0000 7fa875fbe700 1 --2- 172.21.15.2:0/861351034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa870107000 0x7fa87010eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:05.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.537+0000 7fa875fbe700 1 --2- 172.21.15.2:0/861351034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa870102160 0x7fa870102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:05.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.537+0000 7fa875fbe700 1 --2- 172.21.15.2:0/861351034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa870102b10 0x7fa8701068d0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:05.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.537+0000 7fa875fbe700 1 -- 172.21.15.2:0/861351034 >> 172.21.15.2:0/861351034 conn(0x7fa870077780 msgr2=0x7fa870077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:05.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.537+0000 7fa875fbe700 1 -- 172.21.15.2:0/861351034 shutdown_connections 2024-05-14T17:04:05.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.537+0000 7fa875fbe700 1 -- 172.21.15.2:0/861351034 wait complete. 2024-05-14T17:04:05.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.538+0000 7fa875fbe700 1 Processor -- start 2024-05-14T17:04:05.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.538+0000 7fa875fbe700 1 -- start start 2024-05-14T17:04:05.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.539+0000 7fa875fbe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa870102160 0x7fa87019d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:05.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.539+0000 7fa875fbe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa870102b10 0x7fa87019dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:05.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.539+0000 7fa875fbe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa870107000 0x7fa8701978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:05.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.539+0000 7fa875fbe700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa8701115d0 con 0x7fa870102160 2024-05-14T17:04:05.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.539+0000 7fa86effd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa870102b10 0x7fa87019dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:05.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.539+0000 7fa86ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa870107000 0x7fa8701978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:05.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.539+0000 7fa86ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa870107000 0x7fa8701978f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:46084/0 (socket says 172.21.15.2:46084) 2024-05-14T17:04:05.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.539+0000 7fa86ffff700 1 -- 172.21.15.2:0/1601434038 learned_addr learned my addr 172.21.15.2:0/1601434038 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:04:05.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.540+0000 7fa875fbe700 1 -- 172.21.15.2:0/1601434038 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa870111450 con 0x7fa870107000 2024-05-14T17:04:05.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.540+0000 7fa875fbe700 1 -- 172.21.15.2:0/1601434038 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa870111750 con 0x7fa870102b10 2024-05-14T17:04:05.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.540+0000 7fa86effd700 1 -- 172.21.15.2:0/1601434038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa870107000 msgr2=0x7fa8701978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:05.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.540+0000 7fa86effd700 1 --2- 172.21.15.2:0/1601434038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa870107000 0x7fa8701978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:05.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.540+0000 7fa86effd700 1 -- 172.21.15.2:0/1601434038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa870102160 msgr2=0x7fa87019d730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-14T17:04:05.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.540+0000 7fa86effd700 1 --2- 172.21.15.2:0/1601434038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa870102160 0x7fa87019d730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:05.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.540+0000 7fa86effd700 1 -- 172.21.15.2:0/1601434038 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa870198050 con 0x7fa870102b10 2024-05-14T17:04:06.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.540+0000 7fa86ffff700 1 --2- 172.21.15.2:0/1601434038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa870107000 0x7fa8701978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:04:06.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.541+0000 7fa86effd700 1 --2- 172.21.15.2:0/1601434038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa870102b10 0x7fa87019dc70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa868000f80 tx=0x7fa868004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:06.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.541+0000 7fa86cff9700 1 -- 172.21.15.2:0/1601434038 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa868021070 con 0x7fa870102b10 2024-05-14T17:04:06.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.541+0000 7fa86cff9700 1 -- 172.21.15.2:0/1601434038 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa868007e30 con 0x7fa870102b10 2024-05-14T17:04:06.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.541+0000 7fa86cff9700 1 -- 172.21.15.2:0/1601434038 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa868004620 con 0x7fa870102b10 2024-05-14T17:04:06.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.541+0000 7fa875fbe700 1 -- 172.21.15.2:0/1601434038 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8701982e0 con 0x7fa870102b10 2024-05-14T17:04:06.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.542+0000 7fa875fbe700 1 -- 172.21.15.2:0/1601434038 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8701a4600 con 0x7fa870102b10 2024-05-14T17:04:06.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.543+0000 7fa875fbe700 1 -- 172.21.15.2:0/1601434038 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa870103340 con 0x7fa870102b10 2024-05-14T17:04:06.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.545+0000 7fa86cff9700 1 -- 172.21.15.2:0/1601434038 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fa868052d10 con 0x7fa870102b10 2024-05-14T17:04:06.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.546+0000 7fa86cff9700 1 --2- 172.21.15.2:0/1601434038 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fa8580b4c80 0x7fa8580b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:06.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.546+0000 7fa86cff9700 1 -- 172.21.15.2:0/1601434038 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8557+0+0 (secure 0 0 0) 0x7fa86800f020 con 0x7fa870102b10 2024-05-14T17:04:06.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.550+0000 7fa86f7fe700 1 --2- 172.21.15.2:0/1601434038 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fa8580b4c80 0x7fa8580b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:06.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.553+0000 7fa86cff9700 1 -- 172.21.15.2:0/1601434038 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa86802a020 con 0x7fa870102b10 2024-05-14T17:04:06.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.567+0000 7fa86f7fe700 1 --2- 172.21.15.2:0/1601434038 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fa8580b4c80 0x7fa8580b7140 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa870198a30 tx=0x7fa86000a370 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:06.287 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:04:05 smithi123 ceph-mon[35971]: osdmap e88: 12 total, 12 up, 12 in 2024-05-14T17:04:06.296 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:04:05 smithi151 ceph-mon[36066]: osdmap e88: 12 total, 12 up, 12 in 2024-05-14T17:04:06.307 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:04:05 smithi002 ceph-mon[31538]: osdmap e88: 12 total, 12 up, 12 in 2024-05-14T17:04:06.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.953+0000 7fa875fbe700 1 -- 172.21.15.2:0/1601434038 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa870198e20 con 0x7fa870102b10 2024-05-14T17:04:06.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.954+0000 7fa86cff9700 1 -- 172.21.15.2:0/1601434038 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11853 (secure 0 0 0) 0x7fa8680ab400 con 0x7fa870102b10 2024-05-14T17:04:06.354 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:04:06.367 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2275696923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2275696923},{"type":"v1","addr":"172.21.15.2:6835","nonce":2275696923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/716718761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":716718761},{"type":"v1","addr":"172.21.15.2:6839","nonce":716718761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/3644815179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3644815179},{"type":"v1","addr":"172.21.15.2:6841","nonce":3644815179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1880336731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1880336731},{"type":"v1","addr":"172.21.15.151:6833","nonce":1880336731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2415605548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2415605548},{"type":"v1","addr":"172.21.15.2:6837","nonce":2415605548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2635167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2635167},{"type":"v1","addr":"172.21.15.123:6833","nonce":2635167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:51:34.106919+0000","modified":"2024-05-14T17:02:50.475577+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24455,"mds_2":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24415":{"gid":24415,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/762044339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":762044339},{"type":"v1","addr":"172.21.15.151:6835","nonce":762044339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/132043151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":132043151},{"type":"v1","addr":"172.21.15.151:6837","nonce":132043151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/4107834375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":4107834375},{"type":"v1","addr":"172.21.15.151:6839","nonce":4107834375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.123:6835/1198670013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1198670013},{"type":"v1","addr":"172.21.15.123:6835","nonce":1198670013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/333426645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":333426645},{"type":"v1","addr":"172.21.15.123:6837","nonce":333426645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/580275877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":580275877},{"type":"v1","addr":"172.21.15.123:6839","nonce":580275877}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24431,24455]},"id":1}]} 2024-05-14T17:04:06.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.956+0000 7fa875fbe700 1 -- 172.21.15.2:0/1601434038 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fa8580b4c80 msgr2=0x7fa8580b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:06.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.956+0000 7fa875fbe700 1 --2- 172.21.15.2:0/1601434038 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fa8580b4c80 0x7fa8580b7140 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa870198a30 tx=0x7fa86000a370 comp rx=0 tx=0).stop 2024-05-14T17:04:06.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.956+0000 7fa875fbe700 1 -- 172.21.15.2:0/1601434038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa870102b10 msgr2=0x7fa87019dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:06.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.956+0000 7fa875fbe700 1 --2- 172.21.15.2:0/1601434038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa870102b10 0x7fa87019dc70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa868000f80 tx=0x7fa868004d00 comp rx=0 tx=0).stop 2024-05-14T17:04:06.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.956+0000 7fa875fbe700 1 -- 172.21.15.2:0/1601434038 shutdown_connections 2024-05-14T17:04:06.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.956+0000 7fa875fbe700 1 --2- 172.21.15.2:0/1601434038 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fa8580b4c80 0x7fa8580b7140 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:06.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.956+0000 7fa875fbe700 1 --2- 172.21.15.2:0/1601434038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa870107000 0x7fa8701978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:06.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.956+0000 7fa875fbe700 1 --2- 172.21.15.2:0/1601434038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa870102160 0x7fa87019d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:06.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.956+0000 7fa875fbe700 1 --2- 172.21.15.2:0/1601434038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa870102b10 0x7fa87019dc70 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:06.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.956+0000 7fa875fbe700 1 -- 172.21.15.2:0/1601434038 >> 172.21.15.2:0/1601434038 conn(0x7fa870077780 msgr2=0x7fa870100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:06.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.957+0000 7fa875fbe700 1 -- 172.21.15.2:0/1601434038 shutdown_connections 2024-05-14T17:04:06.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:05.957+0000 7fa875fbe700 1 -- 172.21.15.2:0/1601434038 wait complete. 2024-05-14T17:04:06.554 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 41 2024-05-14T17:04:06.603 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:04:06 smithi002 ceph-mon[31538]: pgmap v292: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 21 MiB/s wr, 891 op/s 2024-05-14T17:04:06.612 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:04:06 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/1601434038' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:04:06.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 530.0 s, 1224.6 tps, lat 87.077 ms stddev 203.493 2024-05-14T17:04:06.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:04:06 smithi151 ceph-mon[36066]: pgmap v292: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 21 MiB/s wr, 891 op/s 2024-05-14T17:04:06.865 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:04:06 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/1601434038' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:04:06.943 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:04:06 smithi123 ceph-mon[35971]: pgmap v292: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 21 MiB/s wr, 891 op/s 2024-05-14T17:04:06.954 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:04:06 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/1601434038' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:04:07.461 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 140.4 --await-for 45.0' 2024-05-14T17:04:07.472 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 140.4 --await-for 45.0 2024-05-14T17:04:07.891 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:04:08.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:04:08 smithi002 ceph-mon[31538]: pgmap v293: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 21 MiB/s wr, 891 op/s 2024-05-14T17:04:08.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:04:08 smithi151 ceph-mon[36066]: pgmap v293: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 21 MiB/s wr, 891 op/s 2024-05-14T17:04:08.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:04:08 smithi123 ceph-mon[35971]: pgmap v293: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 21 MiB/s wr, 891 op/s 2024-05-14T17:04:10.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.038+0000 7f759cab1700 1 -- 172.21.15.2:0/1534796521 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75981096d0 msgr2=0x7f7598109ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:10.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.038+0000 7f759cab1700 1 --2- 172.21.15.2:0/1534796521 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75981096d0 0x7f7598109ab0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f7588009990 tx=0x7f7588009ca0 comp rx=0 tx=0).stop 2024-05-14T17:04:10.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.038+0000 7f759cab1700 1 -- 172.21.15.2:0/1534796521 shutdown_connections 2024-05-14T17:04:10.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.038+0000 7f759cab1700 1 --2- 172.21.15.2:0/1534796521 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7598103f80 0x7f7598110460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:10.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.038+0000 7f759cab1700 1 --2- 172.21.15.2:0/1534796521 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75981096d0 0x7f7598109ab0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:10.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.039+0000 7f759cab1700 1 --2- 172.21.15.2:0/1534796521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75981035c0 0x7f7598103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:10.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.039+0000 7f759cab1700 1 -- 172.21.15.2:0/1534796521 >> 172.21.15.2:0/1534796521 conn(0x7f759807a6f0 msgr2=0x7f759807ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:10.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.039+0000 7f759cab1700 1 -- 172.21.15.2:0/1534796521 shutdown_connections 2024-05-14T17:04:10.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.039+0000 7f759cab1700 1 -- 172.21.15.2:0/1534796521 wait complete. 2024-05-14T17:04:10.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.040+0000 7f759cab1700 1 Processor -- start 2024-05-14T17:04:10.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.040+0000 7f759cab1700 1 -- start start 2024-05-14T17:04:10.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.040+0000 7f759cab1700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75981035c0 0x7f759819f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:10.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.040+0000 7f759cab1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7598103f80 0x7f759819fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:10.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.040+0000 7f759cab1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75981096d0 0x7f75981999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:10.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.040+0000 7f759cab1700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7598113760 con 0x7f7598103f80 2024-05-14T17:04:10.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.040+0000 7f759cab1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f75981135e0 con 0x7f75981096d0 2024-05-14T17:04:10.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.040+0000 7f759cab1700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f75981138e0 con 0x7f75981035c0 2024-05-14T17:04:10.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.041+0000 7f759659c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75981035c0 0x7f759819f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:10.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.041+0000 7f759659c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75981035c0 0x7f759819f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:34304/0 (socket says 172.21.15.2:34304) 2024-05-14T17:04:10.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.041+0000 7f759659c700 1 -- 172.21.15.2:0/2481000733 learned_addr learned my addr 172.21.15.2:0/2481000733 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:04:10.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.041+0000 7f7596d9d700 1 --2- 172.21.15.2:0/2481000733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75981096d0 0x7f75981999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:10.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.042+0000 7f759659c700 1 -- 172.21.15.2:0/2481000733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75981096d0 msgr2=0x7f75981999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:10.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.042+0000 7f759659c700 1 --2- 172.21.15.2:0/2481000733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75981096d0 0x7f75981999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:10.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.042+0000 7f759659c700 1 -- 172.21.15.2:0/2481000733 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7598103f80 msgr2=0x7f759819fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:10.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.042+0000 7f759659c700 1 --2- 172.21.15.2:0/2481000733 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7598103f80 0x7f759819fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:10.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.042+0000 7f759659c700 1 -- 172.21.15.2:0/2481000733 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f759819a120 con 0x7f75981035c0 2024-05-14T17:04:10.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.042+0000 7f7596d9d700 1 --2- 172.21.15.2:0/2481000733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75981096d0 0x7f75981999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:04:10.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.043+0000 7f759659c700 1 --2- 172.21.15.2:0/2481000733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75981035c0 0x7f759819f7b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7588000c00 tx=0x7f758800bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:10.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.043+0000 7f758f7fe700 1 -- 172.21.15.2:0/2481000733 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7588027070 con 0x7f75981035c0 2024-05-14T17:04:10.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.043+0000 7f758f7fe700 1 -- 172.21.15.2:0/2481000733 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7588022470 con 0x7f75981035c0 2024-05-14T17:04:10.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.043+0000 7f758f7fe700 1 -- 172.21.15.2:0/2481000733 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7588017650 con 0x7f75981035c0 2024-05-14T17:04:10.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.043+0000 7f759cab1700 1 -- 172.21.15.2:0/2481000733 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f759819a3b0 con 0x7f75981035c0 2024-05-14T17:04:10.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.043+0000 7f759cab1700 1 -- 172.21.15.2:0/2481000733 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75981a6680 con 0x7f75981035c0 2024-05-14T17:04:10.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.046+0000 7f758f7fe700 1 -- 172.21.15.2:0/2481000733 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f758802a070 con 0x7f75981035c0 2024-05-14T17:04:10.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.046+0000 7f758f7fe700 1 --2- 172.21.15.2:0/2481000733 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f75780b2b20 0x7f75780b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:10.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.047+0000 7f758f7fe700 1 -- 172.21.15.2:0/2481000733 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8557+0+0 (secure 0 0 0) 0x7f75880e3320 con 0x7f75981035c0 2024-05-14T17:04:10.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.047+0000 7f7595d9b700 1 --2- 172.21.15.2:0/2481000733 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f75780b2b20 0x7f75780b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:10.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.047+0000 7f75a7347b80 1 Processor -- start 2024-05-14T17:04:10.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.047+0000 7f75a7347b80 1 -- start start 2024-05-14T17:04:10.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.047+0000 7f75a7347b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56139d43f5f0 0x56139d43f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:10.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.047+0000 7f75a7347b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56139d43bd30 0x56139d43c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:10.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.047+0000 7f75a7347b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56139d43c6d0 0x56139d4509a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:10.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.047+0000 7f75a7347b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56139d399f00 con 0x56139d43bd30 2024-05-14T17:04:10.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.047+0000 7f75a7347b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56139d389aa0 con 0x56139d43f5f0 2024-05-14T17:04:10.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.047+0000 7f75a7347b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56139d38dae0 con 0x56139d43c6d0 2024-05-14T17:04:10.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.048+0000 7f759659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56139d43f5f0 0x56139d43f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:10.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.048+0000 7f759659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56139d43f5f0 0x56139d43f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:46106/0 (socket says 172.21.15.2:46106) 2024-05-14T17:04:10.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.048+0000 7f7596d9d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56139d43c6d0 0x56139d4509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:10.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.048+0000 7f759659c700 1 -- 172.21.15.2:0/236060443 learned_addr learned my addr 172.21.15.2:0/236060443 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:04:10.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.048+0000 7f759659c700 1 -- 172.21.15.2:0/236060443 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56139d43c6d0 msgr2=0x56139d4509a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:10.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.048+0000 7f759659c700 1 --2- 172.21.15.2:0/236060443 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56139d43c6d0 0x56139d4509a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:10.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.048+0000 7f759659c700 1 -- 172.21.15.2:0/236060443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56139d43bd30 msgr2=0x56139d43c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:10.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.048+0000 7f759659c700 1 --2- 172.21.15.2:0/236060443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56139d43bd30 0x56139d43c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:10.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.048+0000 7f759659c700 1 -- 172.21.15.2:0/236060443 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56139d333140 con 0x56139d43f5f0 2024-05-14T17:04:10.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.048+0000 7f759659c700 1 --2- 172.21.15.2:0/236060443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56139d43f5f0 0x56139d43f9d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7588024000 tx=0x7f75880aa040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7355c611bb2835c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:10.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.049+0000 7f758cff9700 1 -- 172.21.15.2:0/236060443 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75880aa670 con 0x56139d43f5f0 2024-05-14T17:04:10.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.049+0000 7f7595d9b700 1 --2- 172.21.15.2:0/2481000733 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f75780b2b20 0x7f75780b4fe0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f759807aca0 tx=0x7f758000b040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:10.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.049+0000 7f758cff9700 1 -- 172.21.15.2:0/236060443 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75880aea20 con 0x56139d43f5f0 2024-05-14T17:04:10.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.050+0000 7f75a7347b80 1 -- 172.21.15.2:0/236060443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56139d43f5f0 msgr2=0x56139d43f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:10.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.050+0000 7f75a7347b80 1 --2- 172.21.15.2:0/236060443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56139d43f5f0 0x56139d43f9d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7588024000 tx=0x7f75880aa040 comp rx=0 tx=0).stop 2024-05-14T17:04:11.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.050+0000 7f75a7347b80 1 -- 172.21.15.2:0/236060443 shutdown_connections 2024-05-14T17:04:11.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.050+0000 7f75a7347b80 1 --2- 172.21.15.2:0/236060443 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56139d43c6d0 0x56139d4509a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:11.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.050+0000 7f75a7347b80 1 --2- 172.21.15.2:0/236060443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56139d43f5f0 0x56139d43f9d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:11.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.050+0000 7f75a7347b80 1 --2- 172.21.15.2:0/236060443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56139d43bd30 0x56139d43c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:11.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.050+0000 7f75a7347b80 1 -- 172.21.15.2:0/236060443 >> 172.21.15.2:0/236060443 conn(0x56139d368f40 msgr2=0x56139d436530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:11.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.050+0000 7f75a7347b80 1 -- 172.21.15.2:0/236060443 shutdown_connections 2024-05-14T17:04:11.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.050+0000 7f75a7347b80 1 -- 172.21.15.2:0/236060443 wait complete. 2024-05-14T17:04:11.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.051+0000 7f75a7347b80 1 -- start start 2024-05-14T17:04:11.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.051+0000 7f75a7347b80 1 Processor -- start 2024-05-14T17:04:11.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.051+0000 7f75a7347b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56139d43bd30 0x56139d450500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:11.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.051+0000 7f75a7347b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56139d43c6d0 0x56139d44d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:11.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.051+0000 7f75a7347b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56139d43f5f0 0x56139d44dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:11.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.051+0000 7f75a7347b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56139d290130 con 0x56139d43bd30 2024-05-14T17:04:11.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.051+0000 7f75a7347b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56139d2f1f30 con 0x56139d43c6d0 2024-05-14T17:04:11.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.051+0000 7f75a7347b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56139d450f40 con 0x56139d43f5f0 2024-05-14T17:04:11.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.051+0000 7f759659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56139d43bd30 0x56139d450500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:11.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.051+0000 7f759659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56139d43bd30 0x56139d450500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60068/0 (socket says 172.21.15.2:60068) 2024-05-14T17:04:11.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.051+0000 7f759659c700 1 -- 172.21.15.2:0/1244051528 learned_addr learned my addr 172.21.15.2:0/1244051528 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:04:11.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.051+0000 7f75a7347b80 10 client.0 _ll_register_callbacks cb 0x56139d324ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:04:11.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.051+0000 7f7595d9b700 1 --2- 172.21.15.2:0/1244051528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56139d43c6d0 0x56139d44d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:11.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.052+0000 7f7596d9d700 1 --2- 172.21.15.2:0/1244051528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56139d43f5f0 0x56139d44dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:11.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.052+0000 7f7595d9b700 1 -- 172.21.15.2:0/1244051528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56139d43f5f0 msgr2=0x56139d44dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:11.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.052+0000 7f7595d9b700 1 --2- 172.21.15.2:0/1244051528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56139d43f5f0 0x56139d44dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:11.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.052+0000 7f7595d9b700 1 -- 172.21.15.2:0/1244051528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56139d43bd30 msgr2=0x56139d450500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:11.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.052+0000 7f7595d9b700 1 --2- 172.21.15.2:0/1244051528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56139d43bd30 0x56139d450500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:11.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.052+0000 7f7595d9b700 1 -- 172.21.15.2:0/1244051528 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56139d2d9a60 con 0x56139d43c6d0 2024-05-14T17:04:11.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.052+0000 7f7595d9b700 1 --2- 172.21.15.2:0/1244051528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56139d43c6d0 0x56139d44d8a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f758000ea70 tx=0x7f7580015d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95548132688c94d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:11.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.052+0000 7f7585ffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T17:04:11.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.052+0000 7f7585ffb700 1 -- 172.21.15.2:0/1244051528 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7580004bb0 con 0x56139d43c6d0 2024-05-14T17:04:11.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.052+0000 7f75a7347b80 1 -- 172.21.15.2:0/1244051528 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75880aea20 con 0x56139d43c6d0 2024-05-14T17:04:11.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.052+0000 7f7585ffb700 1 -- 172.21.15.2:0/1244051528 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8557+0+0 (secure 0 0 0) 0x7f7580004d50 con 0x56139d43c6d0 2024-05-14T17:04:11.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.052+0000 7f7585ffb700 1 -- 172.21.15.2:0/1244051528 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f758002c050 con 0x56139d43c6d0 2024-05-14T17:04:11.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.052+0000 7f75a7347b80 10 client.26009 fetch_fsmap learned FSMap version 41 2024-05-14T17:04:11.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.052+0000 7f75a7347b80 1 -- 172.21.15.2:0/1244051528 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56139d3438f0 con 0x56139d43c6d0 2024-05-14T17:04:11.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.053+0000 7f7585ffb700 1 -- 172.21.15.2:0/1244051528 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75880aea20 con 0x56139d43c6d0 2024-05-14T17:04:11.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.053+0000 7f7585ffb700 1 -- 172.21.15.2:0/1244051528 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f758001fae0 con 0x56139d43c6d0 2024-05-14T17:04:11.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.053+0000 7f75a7347b80 10 client.26009 fetch_fsmap finished waiting for FSMap version 41 2024-05-14T17:04:11.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.053+0000 7f75a7347b80 10 client.26009 resolve_mds: validated gid 24479 aka daemon mds.b 2024-05-14T17:04:11.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.053+0000 7f75a7347b80 20 client.26009 populate_metadata read hostname 'smithi002' 2024-05-14T17:04:11.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.053+0000 7f75a7347b80 1 --2- 172.21.15.2:0/1244051528 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56139d45ddf0 0x56139d460210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:11.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.053+0000 7f75a7347b80 4 client.26009 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:04:11.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.053+0000 7f75a7347b80 1 -- 172.21.15.2:0/1244051528 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56139d39be50 con 0x56139d45ddf0 2024-05-14T17:04:11.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.053+0000 7f759659c700 1 --2- 172.21.15.2:0/1244051528 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56139d45ddf0 0x56139d460210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:11.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.054+0000 7f759659c700 1 --2- 172.21.15.2:0/1244051528 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56139d45ddf0 0x56139d460210 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c498024a1d1fe189 server_cookie=4ef8f917bb705ee5 in_seq=0 out_seq=0 2024-05-14T17:04:11.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.055+0000 7f7585ffb700 10 client.26009 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:04:11.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.055+0000 7f7585ffb700 1 -- 172.21.15.2:0/1244051528 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56139d39be50 con 0x56139d45ddf0 2024-05-14T17:04:11.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.055+0000 7f7585ffb700 10 client.26009 handle_command_reply: tid=0 2024-05-14T17:04:11.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.055+0000 7f75a7347b80 1 client.26009 shutdown 2024-05-14T17:04:11.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.055+0000 7f75a7347b80 1 -- 172.21.15.2:0/1244051528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56139d43c6d0 msgr2=0x56139d44d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:11.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.055+0000 7f75a7347b80 1 --2- 172.21.15.2:0/1244051528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56139d43c6d0 0x56139d44d8a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f758000ea70 tx=0x7f7580015d90 comp rx=0 tx=0).stop 2024-05-14T17:04:11.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.055+0000 7f75a7347b80 1 -- 172.21.15.2:0/1244051528 shutdown_connections 2024-05-14T17:04:11.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.055+0000 7f75a7347b80 1 --2- 172.21.15.2:0/1244051528 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56139d45ddf0 0x56139d460210 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:11.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.055+0000 7f75a7347b80 1 --2- 172.21.15.2:0/1244051528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56139d43f5f0 0x56139d44dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:11.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.055+0000 7f75a7347b80 1 --2- 172.21.15.2:0/1244051528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56139d43bd30 0x56139d450500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:11.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.055+0000 7f75a7347b80 1 --2- 172.21.15.2:0/1244051528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56139d43c6d0 0x56139d44d8a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:11.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.055+0000 7f75a7347b80 1 -- 172.21.15.2:0/1244051528 >> 172.21.15.2:0/1244051528 conn(0x56139d368f40 msgr2=0x56139d431050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:11.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.055+0000 7f7585ffb700 0 client.26009 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:04:11.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.055+0000 7f75a7347b80 1 -- 172.21.15.2:0/1244051528 shutdown_connections 2024-05-14T17:04:11.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.056+0000 7f75a7347b80 1 -- 172.21.15.2:0/1244051528 wait complete. 2024-05-14T17:04:11.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.056+0000 7f75a7347b80 20 client.26009 trim_cache size 0 max 16384 2024-05-14T17:04:11.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.082+0000 7f75a7347b80 1 Processor -- start 2024-05-14T17:04:11.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.082+0000 7f75a7347b80 1 -- start start 2024-05-14T17:04:11.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.082+0000 7f75a7347b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56139d43bd30 0x56139d44c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:11.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.082+0000 7f75a7347b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56139d43c6d0 0x56139d448fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:11.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.082+0000 7f75a7347b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56139d43f5f0 0x56139d431160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:11.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.082+0000 7f75a7347b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f75981ae3a0 con 0x56139d43c6d0 2024-05-14T17:04:11.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.082+0000 7f75a7347b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f75981abc30 con 0x56139d43bd30 2024-05-14T17:04:11.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.082+0000 7f75a7347b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f75981a9aa0 con 0x56139d43f5f0 2024-05-14T17:04:12.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.082+0000 7f759659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56139d43bd30 0x56139d44c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:12.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.082+0000 7f759659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56139d43bd30 0x56139d44c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:46122/0 (socket says 172.21.15.2:46122) 2024-05-14T17:04:12.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.082+0000 7f759659c700 1 -- 172.21.15.2:0/2331330119 learned_addr learned my addr 172.21.15.2:0/2331330119 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:04:12.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.083+0000 7f7596d9d700 1 --2- 172.21.15.2:0/2331330119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56139d43f5f0 0x56139d431160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:12.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.083+0000 7f7595d9b700 1 --2- 172.21.15.2:0/2331330119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56139d43c6d0 0x56139d448fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:12.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.083+0000 7f759659c700 1 -- 172.21.15.2:0/2331330119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56139d43f5f0 msgr2=0x56139d431160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:12.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.084+0000 7f759659c700 1 --2- 172.21.15.2:0/2331330119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56139d43f5f0 0x56139d431160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:12.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.084+0000 7f759659c700 1 -- 172.21.15.2:0/2331330119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56139d43c6d0 msgr2=0x56139d448fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:12.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.084+0000 7f759659c700 1 --2- 172.21.15.2:0/2331330119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56139d43c6d0 0x56139d448fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:12.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.084+0000 7f759659c700 1 -- 172.21.15.2:0/2331330119 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56139d43f240 con 0x56139d43bd30 2024-05-14T17:04:12.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.084+0000 7f759659c700 1 --2- 172.21.15.2:0/2331330119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56139d43bd30 0x56139d44c860 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f75880a78e0 tx=0x7f7588092040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8707fa389d5b93e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:12.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.084+0000 7f7587fff700 1 -- 172.21.15.2:0/2331330119 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75880aea20 con 0x56139d43bd30 2024-05-14T17:04:12.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.084+0000 7f7587fff700 1 -- 172.21.15.2:0/2331330119 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75880aaa70 con 0x56139d43bd30 2024-05-14T17:04:12.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.084+0000 7f7587fff700 1 -- 172.21.15.2:0/2331330119 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75880b3210 con 0x56139d43bd30 2024-05-14T17:04:12.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.085+0000 7f75a7347b80 1 -- 172.21.15.2:0/2331330119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56139d43bd30 msgr2=0x56139d44c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:12.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.085+0000 7f75a7347b80 1 --2- 172.21.15.2:0/2331330119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56139d43bd30 0x56139d44c860 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f75880a78e0 tx=0x7f7588092040 comp rx=0 tx=0).stop 2024-05-14T17:04:12.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.085+0000 7f75a7347b80 1 -- 172.21.15.2:0/2331330119 shutdown_connections 2024-05-14T17:04:12.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.085+0000 7f75a7347b80 1 --2- 172.21.15.2:0/2331330119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56139d43f5f0 0x56139d431160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:12.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.085+0000 7f75a7347b80 1 --2- 172.21.15.2:0/2331330119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56139d43bd30 0x56139d44c860 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:12.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.085+0000 7f75a7347b80 1 --2- 172.21.15.2:0/2331330119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56139d43c6d0 0x56139d448fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:12.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.085+0000 7f75a7347b80 1 -- 172.21.15.2:0/2331330119 >> 172.21.15.2:0/2331330119 conn(0x56139d368f40 msgr2=0x56139d43b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:12.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.085+0000 7f75a7347b80 1 -- 172.21.15.2:0/2331330119 shutdown_connections 2024-05-14T17:04:12.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.085+0000 7f75a7347b80 1 -- 172.21.15.2:0/2331330119 wait complete. 2024-05-14T17:04:12.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.086+0000 7f75a7347b80 1 -- start start 2024-05-14T17:04:12.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.086+0000 7f75a7347b80 1 Processor -- start 2024-05-14T17:04:12.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.086+0000 7f75a7347b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56139d43c6d0 0x56139d456570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:12.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.086+0000 7f75a7347b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56139d43f5f0 0x56139d452860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:12.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.087+0000 7f75a7347b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56139d452da0 0x56139d453220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:12.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.087+0000 7f75a7347b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f75981aa0f0 con 0x56139d452da0 2024-05-14T17:04:12.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.087+0000 7f759659c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56139d43c6d0 0x56139d456570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:12.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.087+0000 7f759659c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56139d43c6d0 0x56139d456570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:34324/0 (socket says 172.21.15.2:34324) 2024-05-14T17:04:12.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.087+0000 7f759659c700 1 -- 172.21.15.2:0/282406222 learned_addr learned my addr 172.21.15.2:0/282406222 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:04:12.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.087+0000 7f75a7347b80 1 -- 172.21.15.2:0/282406222 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f75981ab560 con 0x56139d43f5f0 2024-05-14T17:04:12.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.087+0000 7f7596d9d700 1 --2- 172.21.15.2:0/282406222 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56139d452da0 0x56139d453220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:12.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.087+0000 7f7595d9b700 1 --2- 172.21.15.2:0/282406222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56139d43f5f0 0x56139d452860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:12.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.087+0000 7f75a7347b80 1 -- 172.21.15.2:0/282406222 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f75981aa7c0 con 0x56139d43c6d0 2024-05-14T17:04:12.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.087+0000 7f75a7347b80 10 client.0 _ll_register_callbacks cb 0x56139d324ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:04:12.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.087+0000 7f759659c700 1 -- 172.21.15.2:0/282406222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56139d43f5f0 msgr2=0x56139d452860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:12.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.087+0000 7f759659c700 1 --2- 172.21.15.2:0/282406222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56139d43f5f0 0x56139d452860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:12.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.087+0000 7f759659c700 1 -- 172.21.15.2:0/282406222 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56139d452da0 msgr2=0x56139d453220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:12.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.087+0000 7f759659c700 1 --2- 172.21.15.2:0/282406222 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56139d452da0 0x56139d453220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:12.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.087+0000 7f759659c700 1 -- 172.21.15.2:0/282406222 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56139d4c8660 con 0x56139d43c6d0 2024-05-14T17:04:12.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.088+0000 7f7595d9b700 1 --2- 172.21.15.2:0/282406222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56139d43f5f0 0x56139d452860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:04:12.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.088+0000 7f759659c700 1 --2- 172.21.15.2:0/282406222 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56139d43c6d0 0x56139d456570 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f75880e0fd0 tx=0x7f7588009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=944b90c508ef0b9a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:12.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.088+0000 7f7585ffb700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-14T17:04:12.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.088+0000 7f7585ffb700 1 -- 172.21.15.2:0/282406222 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75880a9fc0 con 0x56139d43c6d0 2024-05-14T17:04:12.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.088+0000 7f7585ffb700 1 -- 172.21.15.2:0/282406222 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8557+0+0 (secure 0 0 0) 0x7f75880afdc0 con 0x56139d43c6d0 2024-05-14T17:04:12.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.088+0000 7f75a7347b80 1 -- 172.21.15.2:0/282406222 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75880aaa70 con 0x56139d43c6d0 2024-05-14T17:04:12.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.089+0000 7f7585ffb700 1 -- 172.21.15.2:0/282406222 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f75880a6900 con 0x56139d43c6d0 2024-05-14T17:04:12.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.089+0000 7f75a7347b80 10 client.26002 fetch_fsmap learned FSMap version 41 2024-05-14T17:04:12.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.089+0000 7f75a7347b80 1 -- 172.21.15.2:0/282406222 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56139d4c8660 con 0x56139d43c6d0 2024-05-14T17:04:12.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.089+0000 7f7585ffb700 1 -- 172.21.15.2:0/282406222 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75880aaa70 con 0x56139d43c6d0 2024-05-14T17:04:12.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.090+0000 7f7585ffb700 1 -- 172.21.15.2:0/282406222 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f7588090b90 con 0x56139d43c6d0 2024-05-14T17:04:12.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.090+0000 7f75a7347b80 10 client.26002 fetch_fsmap finished waiting for FSMap version 41 2024-05-14T17:04:12.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.090+0000 7f75a7347b80 10 client.26002 resolve_mds: validated gid 24479 aka daemon mds.b 2024-05-14T17:04:12.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.090+0000 7f75a7347b80 20 client.26002 populate_metadata read hostname 'smithi002' 2024-05-14T17:04:12.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.090+0000 7f75a7347b80 1 --2- 172.21.15.2:0/282406222 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56139d4c98d0 0x56139d4cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:12.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.090+0000 7f75a7347b80 4 client.26002 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.4, "await_for": 45.0}] 2024-05-14T17:04:12.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.090+0000 7f75a7347b80 1 -- 172.21.15.2:0/282406222 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.4, "await_for": 45.0}) -- 0x56139d395630 con 0x56139d4c98d0 2024-05-14T17:04:12.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.090+0000 7f7595d9b700 1 --2- 172.21.15.2:0/282406222 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56139d4c98d0 0x56139d4cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:12.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.092+0000 7f7595d9b700 1 --2- 172.21.15.2:0/282406222 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56139d4c98d0 0x56139d4cbcc0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88937832f125f46 server_cookie=954ed6050050d124 in_seq=0 out_seq=0 2024-05-14T17:04:12.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.092+0000 7f7585ffb700 10 client.26002 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:04:12.858 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:04:10 smithi123 ceph-mon[35971]: pgmap v294: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 40 MiB/s wr, 1.70k op/s 2024-05-14T17:04:12.866 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:04:10 smithi151 ceph-mon[36066]: pgmap v294: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 40 MiB/s wr, 1.70k op/s 2024-05-14T17:04:12.874 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:04:12 smithi151 ceph-mon[36066]: pgmap v295: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 40 MiB/s wr, 1.70k op/s 2024-05-14T17:04:12.883 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:04:12 smithi151 ceph-mon[36066]: osdmap e89: 12 total, 12 up, 12 in 2024-05-14T17:04:12.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.354+0000 7f7585ffb700 1 -- 172.21.15.2:0/282406222 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56139d395630 con 0x56139d4c98d0 2024-05-14T17:04:12.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.354+0000 7f7585ffb700 10 client.26002 handle_command_reply: tid=0 2024-05-14T17:04:12.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.354+0000 7f75a7347b80 1 client.26002 shutdown 2024-05-14T17:04:12.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.354+0000 7f75a7347b80 1 -- 172.21.15.2:0/282406222 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56139d43c6d0 msgr2=0x56139d456570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:12.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.354+0000 7f75a7347b80 1 --2- 172.21.15.2:0/282406222 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56139d43c6d0 0x56139d456570 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f75880e0fd0 tx=0x7f7588009ca0 comp rx=0 tx=0).stop 2024-05-14T17:04:12.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.354+0000 7f75a7347b80 1 -- 172.21.15.2:0/282406222 shutdown_connections 2024-05-14T17:04:13.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.354+0000 7f75a7347b80 1 --2- 172.21.15.2:0/282406222 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56139d4c98d0 0x56139d4cbcc0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:13.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.354+0000 7f75a7347b80 1 --2- 172.21.15.2:0/282406222 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56139d452da0 0x56139d453220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:13.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.354+0000 7f75a7347b80 1 --2- 172.21.15.2:0/282406222 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56139d43c6d0 0x56139d456570 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:13.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.354+0000 7f75a7347b80 1 --2- 172.21.15.2:0/282406222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56139d43f5f0 0x56139d452860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:13.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.354+0000 7f7585ffb700 0 client.26002 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:04:13.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.354+0000 7f75a7347b80 1 -- 172.21.15.2:0/282406222 >> 172.21.15.2:0/282406222 conn(0x56139d368f40 msgr2=0x56139d43b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:13.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.354+0000 7f75a7347b80 1 -- 172.21.15.2:0/282406222 shutdown_connections 2024-05-14T17:04:13.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.354+0000 7f75a7347b80 1 -- 172.21.15.2:0/282406222 wait complete. 2024-05-14T17:04:13.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.354+0000 7f75a7347b80 20 client.26002 trim_cache size 0 max 16384 2024-05-14T17:04:13.162 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T17:04:13.170 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 41, 2024-05-14T17:04:13.179 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24479, 2024-05-14T17:04:13.218 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 30, 2024-05-14T17:04:13.257 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-05-14T17:04:13.266 INFO:teuthology.orchestra.run.smithi002.stdout: "6c170f95": { 2024-05-14T17:04:13.275 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 30, 2024-05-14T17:04:13.284 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-05-14T17:04:13.292 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-05-14T17:04:13.300 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-05-14T17:04:13.308 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-05-14T17:04:13.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:04:13.326 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-05-14T17:04:13.361 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 140.4, 2024-05-14T17:04:13.372 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-05-14T17:04:13.382 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-05-14T17:04:13.391 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-05-14T17:04:13.402 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-05-14T17:04:13.410 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-05-14T17:04:13.419 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-05-14T17:04:13.445 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:04:13.470 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:04:13.507 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:04:13.514 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:04:13.521 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:04:13.530 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T17:04:13.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.355+0000 7f759cab1700 1 -- 172.21.15.2:0/2481000733 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f75780b2b20 msgr2=0x7f75780b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:13.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.355+0000 7f759cab1700 1 --2- 172.21.15.2:0/2481000733 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f75780b2b20 0x7f75780b4fe0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f759807aca0 tx=0x7f758000b040 comp rx=0 tx=0).stop 2024-05-14T17:04:13.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.355+0000 7f759cab1700 1 -- 172.21.15.2:0/2481000733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75981035c0 msgr2=0x7f759819f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:13.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.355+0000 7f759cab1700 1 --2- 172.21.15.2:0/2481000733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75981035c0 0x7f759819f7b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7588000c00 tx=0x7f758800bfd0 comp rx=0 tx=0).stop 2024-05-14T17:04:13.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.355+0000 7f759cab1700 1 -- 172.21.15.2:0/2481000733 shutdown_connections 2024-05-14T17:04:13.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.355+0000 7f759cab1700 1 --2- 172.21.15.2:0/2481000733 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f75780b2b20 0x7f75780b4fe0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:13.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.355+0000 7f759cab1700 1 --2- 172.21.15.2:0/2481000733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75981096d0 0x7f75981999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:13.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.355+0000 7f759cab1700 1 --2- 172.21.15.2:0/2481000733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75981035c0 0x7f759819f7b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:13.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.355+0000 7f759cab1700 1 --2- 172.21.15.2:0/2481000733 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7598103f80 0x7f759819fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:13.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.355+0000 7f759cab1700 1 -- 172.21.15.2:0/2481000733 >> 172.21.15.2:0/2481000733 conn(0x7f759807a6f0 msgr2=0x7f7598102040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:13.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.355+0000 7f759cab1700 1 -- 172.21.15.2:0/2481000733 shutdown_connections 2024-05-14T17:04:13.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:10.355+0000 7f759cab1700 1 -- 172.21.15.2:0/2481000733 wait complete. 2024-05-14T17:04:13.724 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:04:10 smithi002 ceph-mon[31538]: pgmap v294: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 40 MiB/s wr, 1.70k op/s 2024-05-14T17:04:13.735 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:04:12 smithi123 ceph-mon[35971]: pgmap v295: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 40 MiB/s wr, 1.70k op/s 2024-05-14T17:04:13.742 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:04:12 smithi123 ceph-mon[35971]: osdmap e89: 12 total, 12 up, 12 in 2024-05-14T17:04:13.751 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6c170f95', quiesce took 11.0 seconds. Will release after: 9.399999999999999 2024-05-14T17:04:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.287+0000 7f1d66932640 1 -- 172.21.15.2:0/4228489822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d60102330 msgr2=0x7f1d6010e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.287+0000 7f1d66932640 1 --2- 172.21.15.2:0/4228489822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d60102330 0x7f1d6010e840 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f1d58009cd0 tx=0x7f1d5802a8e0 comp rx=0 tx=0).stop 2024-05-14T17:04:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.287+0000 7f1d66932640 1 -- 172.21.15.2:0/4228489822 shutdown_connections 2024-05-14T17:04:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.287+0000 7f1d66932640 1 --2- 172.21.15.2:0/4228489822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d60102330 0x7f1d6010e840 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.287+0000 7f1d66932640 1 --2- 172.21.15.2:0/4228489822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d60101970 0x7f1d60101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:13.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.287+0000 7f1d66932640 1 --2- 172.21.15.2:0/4228489822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d60107980 0x7f1d60107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.287+0000 7f1d66932640 1 -- 172.21.15.2:0/4228489822 >> 172.21.15.2:0/4228489822 conn(0x7f1d60077cd0 msgr2=0x7f1d600780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.287+0000 7f1d66932640 1 -- 172.21.15.2:0/4228489822 shutdown_connections 2024-05-14T17:04:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.287+0000 7f1d66932640 1 -- 172.21.15.2:0/4228489822 wait complete. 2024-05-14T17:04:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.287+0000 7f1d66932640 1 Processor -- start 2024-05-14T17:04:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.287+0000 7f1d66932640 1 -- start start 2024-05-14T17:04:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.288+0000 7f1d66932640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d60101970 0x7f1d60103540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.288+0000 7f1d66932640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d60102330 0x7f1d60103a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.288+0000 7f1d66932640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d60107980 0x7f1d60103fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.288+0000 7f1d66932640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1d60111c90 con 0x7f1d60102330 2024-05-14T17:04:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.288+0000 7f1d66932640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1d60111b10 con 0x7f1d60101970 2024-05-14T17:04:14.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.288+0000 7f1d66932640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1d60111e10 con 0x7f1d60107980 2024-05-14T17:04:14.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.288+0000 7f1d5f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d60102330 0x7f1d60103a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:14.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.288+0000 7f1d5f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d60102330 0x7f1d60103a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60096/0 (socket says 172.21.15.2:60096) 2024-05-14T17:04:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.288+0000 7f1d5f7fe640 1 -- 172.21.15.2:0/4082165865 learned_addr learned my addr 172.21.15.2:0/4082165865 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:04:14.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.288+0000 7f1d5ffff640 1 --2- 172.21.15.2:0/4082165865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d60101970 0x7f1d60103540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.288+0000 7f1d64ecf640 1 --2- 172.21.15.2:0/4082165865 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d60107980 0x7f1d60103fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:14.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.288+0000 7f1d5f7fe640 1 -- 172.21.15.2:0/4082165865 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d60107980 msgr2=0x7f1d60103fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.288+0000 7f1d5f7fe640 1 --2- 172.21.15.2:0/4082165865 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d60107980 0x7f1d60103fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:14.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.288+0000 7f1d5f7fe640 1 -- 172.21.15.2:0/4082165865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d60101970 msgr2=0x7f1d60103540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.288+0000 7f1d5f7fe640 1 --2- 172.21.15.2:0/4082165865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d60101970 0x7f1d60103540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.288+0000 7f1d5f7fe640 1 -- 172.21.15.2:0/4082165865 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d601a7660 con 0x7f1d60102330 2024-05-14T17:04:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.289+0000 7f1d5f7fe640 1 --2- 172.21.15.2:0/4082165865 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d60102330 0x7f1d60103a80 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f1d5400e9f0 tx=0x7f1d5400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:14.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.289+0000 7f1d5d7fa640 1 -- 172.21.15.2:0/4082165865 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5400cd80 con 0x7f1d60102330 2024-05-14T17:04:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.289+0000 7f1d5d7fa640 1 -- 172.21.15.2:0/4082165865 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d540108e0 con 0x7f1d60102330 2024-05-14T17:04:14.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.289+0000 7f1d66932640 1 -- 172.21.15.2:0/4082165865 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d601a7950 con 0x7f1d60102330 2024-05-14T17:04:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.289+0000 7f1d5d7fa640 1 -- 172.21.15.2:0/4082165865 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d54018980 con 0x7f1d60102330 2024-05-14T17:04:14.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.289+0000 7f1d66932640 1 -- 172.21.15.2:0/4082165865 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d601a7ef0 con 0x7f1d60102330 2024-05-14T17:04:14.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.292+0000 7f1d6964e740 1 Processor -- start 2024-05-14T17:04:14.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.292+0000 7f1d6964e740 1 -- start start 2024-05-14T17:04:14.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.292+0000 7f1d5d7fa640 1 -- 172.21.15.2:0/4082165865 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f1d54010430 con 0x7f1d60102330 2024-05-14T17:04:14.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.292+0000 7f1d6964e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2354dc8f0 0x55d2354dccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:14.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.292+0000 7f1d6964e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2354dd210 0x55d2354e7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:14.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.293+0000 7f1d6964e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2354e7ad0 0x55d2354e9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:14.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.293+0000 7f1d6964e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d23544fc30 con 0x55d2354dd210 2024-05-14T17:04:14.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.293+0000 7f1d6964e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d23543f7d0 con 0x55d2354e7ad0 2024-05-14T17:04:14.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.293+0000 7f1d6964e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d235443810 con 0x55d2354dc8f0 2024-05-14T17:04:14.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.293+0000 7f1d5ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2354dd210 0x55d2354e7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:14.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.293+0000 7f1d5ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2354dd210 0x55d2354e7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60104/0 (socket says 172.21.15.2:60104) 2024-05-14T17:04:14.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.293+0000 7f1d5ffff640 1 -- 172.21.15.2:0/2307033073 learned_addr learned my addr 172.21.15.2:0/2307033073 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:04:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.293+0000 7f1d5f7fe640 1 --2- 172.21.15.2:0/2307033073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2354e7ad0 0x55d2354e9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:14.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.293+0000 7f1d64ecf640 1 --2- 172.21.15.2:0/2307033073 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2354dc8f0 0x55d2354dccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:14.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.293+0000 7f1d5ffff640 1 -- 172.21.15.2:0/2307033073 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2354dc8f0 msgr2=0x55d2354dccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:14.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.293+0000 7f1d5ffff640 1 --2- 172.21.15.2:0/2307033073 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2354dc8f0 0x55d2354dccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:14.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.293+0000 7f1d5ffff640 1 -- 172.21.15.2:0/2307033073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2354e7ad0 msgr2=0x55d2354e9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:14.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.293+0000 7f1d5ffff640 1 --2- 172.21.15.2:0/2307033073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2354e7ad0 0x55d2354e9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:14.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.293+0000 7f1d5ffff640 1 -- 172.21.15.2:0/2307033073 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d235154210 con 0x55d2354dd210 2024-05-14T17:04:14.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.293+0000 7f1d5f7fe640 1 --2- 172.21.15.2:0/2307033073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2354e7ad0 0x55d2354e9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:04:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.293+0000 7f1d5ffff640 1 --2- 172.21.15.2:0/2307033073 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2354dd210 0x55d2354e7590 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f1d5000bc90 tx=0x7f1d500107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d446129337c6435e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.293+0000 7f1d5d7fa640 1 --2- 172.21.15.2:0/4082165865 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f1d480b2ac0 0x7f1d480b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.293+0000 7f1d467fc640 1 -- 172.21.15.2:0/2307033073 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5001c480 con 0x55d2354dd210 2024-05-14T17:04:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.294+0000 7f1d64ecf640 1 --2- 172.21.15.2:0/4082165865 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f1d480b2ac0 0x7f1d480b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:14.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.294+0000 7f1d467fc640 1 -- 172.21.15.2:0/2307033073 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d5001cad0 con 0x55d2354dd210 2024-05-14T17:04:14.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.294+0000 7f1d467fc640 1 -- 172.21.15.2:0/2307033073 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d50011b50 con 0x55d2354dd210 2024-05-14T17:04:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.294+0000 7f1d5d7fa640 1 -- 172.21.15.2:0/4082165865 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8557+0+0 (secure 0 0 0) 0x7f1d540e2870 con 0x7f1d60102330 2024-05-14T17:04:14.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.294+0000 7f1d64ecf640 1 --2- 172.21.15.2:0/4082165865 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f1d480b2ac0 0x7f1d480b4f80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1d58004af0 tx=0x7f1d58005d80 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:14.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.294+0000 7f1d6964e740 1 -- 172.21.15.2:0/2307033073 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2354dd210 msgr2=0x55d2354e7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:14.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.294+0000 7f1d6964e740 1 --2- 172.21.15.2:0/2307033073 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2354dd210 0x55d2354e7590 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f1d5000bc90 tx=0x7f1d500107a0 comp rx=0 tx=0).stop 2024-05-14T17:04:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.294+0000 7f1d6964e740 1 -- 172.21.15.2:0/2307033073 shutdown_connections 2024-05-14T17:04:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.294+0000 7f1d6964e740 1 --2- 172.21.15.2:0/2307033073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2354e7ad0 0x55d2354e9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.294+0000 7f1d6964e740 1 --2- 172.21.15.2:0/2307033073 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2354dd210 0x55d2354e7590 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:14.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.294+0000 7f1d6964e740 1 --2- 172.21.15.2:0/2307033073 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2354dc8f0 0x55d2354dccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.294+0000 7f1d6964e740 1 -- 172.21.15.2:0/2307033073 >> 172.21.15.2:0/2307033073 conn(0x55d2354d8e00 msgr2=0x55d2354d91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:14.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.294+0000 7f1d6964e740 1 -- 172.21.15.2:0/2307033073 shutdown_connections 2024-05-14T17:04:14.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.294+0000 7f1d6964e740 1 -- 172.21.15.2:0/2307033073 wait complete. 2024-05-14T17:04:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.295+0000 7f1d6964e740 1 -- start start 2024-05-14T17:04:14.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.295+0000 7f1d6964e740 1 Processor -- start 2024-05-14T17:04:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.295+0000 7f1d6964e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2354dc8f0 0x55d2354e74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.295+0000 7f1d6964e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2354dd210 0x55d2354e47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:15.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.295+0000 7f1d6964e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2354e7ad0 0x55d2354e4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:15.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.295+0000 7f1d6964e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d235125790 con 0x55d2354dc8f0 2024-05-14T17:04:15.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.295+0000 7f1d6964e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d2353fb890 con 0x55d2354dd210 2024-05-14T17:04:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.295+0000 7f1d6964e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d2354ea7a0 con 0x55d2354e7ad0 2024-05-14T17:04:15.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.295+0000 7f1d6964e740 10 client.0 _ll_register_callbacks cb 0x55d2353fed60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:04:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.295+0000 7f1d5ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2354dc8f0 0x55d2354e74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.295+0000 7f1d5ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2354dc8f0 0x55d2354e74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60118/0 (socket says 172.21.15.2:60118) 2024-05-14T17:04:15.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.295+0000 7f1d5ffff640 1 -- 172.21.15.2:0/1847454639 learned_addr learned my addr 172.21.15.2:0/1847454639 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:04:15.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.296+0000 7f1d5f7fe640 1 --2- 172.21.15.2:0/1847454639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2354dd210 0x55d2354e47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:15.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.296+0000 7f1d64ecf640 1 --2- 172.21.15.2:0/1847454639 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2354e7ad0 0x55d2354e4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:15.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.296+0000 7f1d5ffff640 1 -- 172.21.15.2:0/1847454639 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2354e7ad0 msgr2=0x55d2354e4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:15.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.296+0000 7f1d5ffff640 1 --2- 172.21.15.2:0/1847454639 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2354e7ad0 0x55d2354e4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:15.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.296+0000 7f1d5ffff640 1 -- 172.21.15.2:0/1847454639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2354dd210 msgr2=0x55d2354e47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:15.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.296+0000 7f1d5ffff640 1 --2- 172.21.15.2:0/1847454639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2354dd210 0x55d2354e47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:15.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.296+0000 7f1d5ffff640 1 -- 172.21.15.2:0/1847454639 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d235220620 con 0x55d2354dc8f0 2024-05-14T17:04:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.296+0000 7f1d64ecf640 1 --2- 172.21.15.2:0/1847454639 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2354e7ad0 0x55d2354e4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:04:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.296+0000 7f1d5ffff640 1 --2- 172.21.15.2:0/1847454639 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2354dc8f0 0x55d2354e74d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f1d50010ca0 tx=0x7f1d5001ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ea2164d2c46371b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.296+0000 7f1d3f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:04:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.296+0000 7f1d3f7fe640 1 -- 172.21.15.2:0/1847454639 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5001c890 con 0x55d2354dc8f0 2024-05-14T17:04:15.284 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.296+0000 7f1d3f7fe640 1 -- 172.21.15.2:0/1847454639 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8557+0+0 (secure 0 0 0) 0x7f1d50004020 con 0x55d2354dc8f0 2024-05-14T17:04:15.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.296+0000 7f1d6964e740 1 -- 172.21.15.2:0/1847454639 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d5001cad0 con 0x55d2354dc8f0 2024-05-14T17:04:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.296+0000 7f1d3f7fe640 1 -- 172.21.15.2:0/1847454639 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f1d5002e050 con 0x55d2354dc8f0 2024-05-14T17:04:15.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.296+0000 7f1d6964e740 10 client.16347 fetch_fsmap learned FSMap version 41 2024-05-14T17:04:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.296+0000 7f1d6964e740 1 -- 172.21.15.2:0/1847454639 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55d2354f6240 con 0x55d2354dc8f0 2024-05-14T17:04:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.297+0000 7f1d3f7fe640 1 -- 172.21.15.2:0/1847454639 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d5001cad0 con 0x55d2354dc8f0 2024-05-14T17:04:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.297+0000 7f1d3f7fe640 1 -- 172.21.15.2:0/1847454639 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f1d50025070 con 0x55d2354dc8f0 2024-05-14T17:04:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.297+0000 7f1d6964e740 10 client.16347 fetch_fsmap finished waiting for FSMap version 41 2024-05-14T17:04:15.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.297+0000 7f1d6964e740 10 client.16347 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:04:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.297+0000 7f1d6964e740 20 client.16347 populate_metadata read hostname 'smithi002' 2024-05-14T17:04:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.297+0000 7f1d6964e740 1 --2- 172.21.15.2:0/1847454639 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d2354f75f0 0x55d2354f99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.297+0000 7f1d6964e740 4 client.16347 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:04:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.297+0000 7f1d6964e740 1 -- 172.21.15.2:0/1847454639 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d235451b80 con 0x55d2354f75f0 2024-05-14T17:04:15.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.297+0000 7f1d5f7fe640 1 --2- 172.21.15.2:0/1847454639 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d2354f75f0 0x55d2354f99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.297+0000 7f1d3f7fe640 10 client.16347 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:04:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.297+0000 7f1d5f7fe640 1 --2- 172.21.15.2:0/1847454639 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d2354f75f0 0x55d2354f99e0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a5e57dd063c2ba4 server_cookie=e8860409cad1c7e4 in_seq=0 out_seq=0 2024-05-14T17:04:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.298+0000 7f1d3f7fe640 1 -- 172.21.15.2:0/1847454639 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d235451b80 con 0x55d2354f75f0 2024-05-14T17:04:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.298+0000 7f1d3f7fe640 10 client.16347 handle_command_reply: tid=0 2024-05-14T17:04:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.298+0000 7f1d6964e740 1 client.16347 shutdown 2024-05-14T17:04:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.299+0000 7f1d6964e740 1 -- 172.21.15.2:0/1847454639 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2354dc8f0 msgr2=0x55d2354e74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:15.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.299+0000 7f1d6964e740 1 --2- 172.21.15.2:0/1847454639 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2354dc8f0 0x55d2354e74d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f1d50010ca0 tx=0x7f1d5001ceb0 comp rx=0 tx=0).stop 2024-05-14T17:04:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.299+0000 7f1d6964e740 1 -- 172.21.15.2:0/1847454639 shutdown_connections 2024-05-14T17:04:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.299+0000 7f1d6964e740 1 --2- 172.21.15.2:0/1847454639 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d2354f75f0 0x55d2354f99e0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.299+0000 7f1d6964e740 1 --2- 172.21.15.2:0/1847454639 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2354e7ad0 0x55d2354e4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.299+0000 7f1d6964e740 1 --2- 172.21.15.2:0/1847454639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2354dd210 0x55d2354e47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.299+0000 7f1d3f7fe640 0 client.16347 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:04:15.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.299+0000 7f1d6964e740 1 --2- 172.21.15.2:0/1847454639 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2354dc8f0 0x55d2354e74d0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.299+0000 7f1d6964e740 1 -- 172.21.15.2:0/1847454639 >> 172.21.15.2:0/1847454639 conn(0x55d2354d8e00 msgr2=0x55d2354d8100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.299+0000 7f1d6964e740 1 -- 172.21.15.2:0/1847454639 shutdown_connections 2024-05-14T17:04:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.299+0000 7f1d6964e740 1 -- 172.21.15.2:0/1847454639 wait complete. 2024-05-14T17:04:15.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.299+0000 7f1d6964e740 20 client.16347 trim_cache size 0 max 16384 2024-05-14T17:04:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.314+0000 7f1d6964e740 1 Processor -- start 2024-05-14T17:04:15.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.314+0000 7f1d6964e740 1 -- start start 2024-05-14T17:04:15.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.314+0000 7f1d6964e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2354dc8f0 0x55d2354d8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:15.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.314+0000 7f1d6964e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2354dd210 0x55d2354f3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:15.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.314+0000 7f1d6964e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2354e7ad0 0x55d2354ff040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:15.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.314+0000 7f1d6964e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1d601b31f0 con 0x55d2354dc8f0 2024-05-14T17:04:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.314+0000 7f1d6964e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1d601b0ac0 con 0x55d2354dd210 2024-05-14T17:04:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.314+0000 7f1d6964e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1d601ae930 con 0x55d2354e7ad0 2024-05-14T17:04:15.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.314+0000 7f1d5ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2354dc8f0 0x55d2354d8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:15.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.314+0000 7f1d5ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2354dc8f0 0x55d2354d8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60128/0 (socket says 172.21.15.2:60128) 2024-05-14T17:04:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.314+0000 7f1d5ffff640 1 -- 172.21.15.2:0/924803651 learned_addr learned my addr 172.21.15.2:0/924803651 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:04:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.315+0000 7f1d5f7fe640 1 --2- 172.21.15.2:0/924803651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2354dd210 0x55d2354f3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:15.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.315+0000 7f1d64ecf640 1 --2- 172.21.15.2:0/924803651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2354e7ad0 0x55d2354ff040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:15.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.315+0000 7f1d5ffff640 1 -- 172.21.15.2:0/924803651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2354e7ad0 msgr2=0x55d2354ff040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:15.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.315+0000 7f1d5ffff640 1 --2- 172.21.15.2:0/924803651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2354e7ad0 0x55d2354ff040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.315+0000 7f1d5ffff640 1 -- 172.21.15.2:0/924803651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2354dd210 msgr2=0x55d2354f3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:15.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.315+0000 7f1d5ffff640 1 --2- 172.21.15.2:0/924803651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2354dd210 0x55d2354f3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.315+0000 7f1d5ffff640 1 -- 172.21.15.2:0/924803651 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2354e5140 con 0x55d2354dc8f0 2024-05-14T17:04:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.315+0000 7f1d5ffff640 1 --2- 172.21.15.2:0/924803651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2354dc8f0 0x55d2354d8100 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f1d5000b3d0 tx=0x7f1d50005890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997198777bdda1fb server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.315+0000 7f1d45ffb640 1 -- 172.21.15.2:0/924803651 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5001c690 con 0x55d2354dc8f0 2024-05-14T17:04:15.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.315+0000 7f1d45ffb640 1 -- 172.21.15.2:0/924803651 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d500056a0 con 0x55d2354dc8f0 2024-05-14T17:04:15.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.315+0000 7f1d45ffb640 1 -- 172.21.15.2:0/924803651 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d500035f0 con 0x55d2354dc8f0 2024-05-14T17:04:15.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.316+0000 7f1d6964e740 1 -- 172.21.15.2:0/924803651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2354dc8f0 msgr2=0x55d2354d8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:15.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.316+0000 7f1d6964e740 1 --2- 172.21.15.2:0/924803651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2354dc8f0 0x55d2354d8100 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f1d5000b3d0 tx=0x7f1d50005890 comp rx=0 tx=0).stop 2024-05-14T17:04:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.316+0000 7f1d6964e740 1 -- 172.21.15.2:0/924803651 shutdown_connections 2024-05-14T17:04:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.316+0000 7f1d6964e740 1 --2- 172.21.15.2:0/924803651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2354e7ad0 0x55d2354ff040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:16.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.316+0000 7f1d6964e740 1 --2- 172.21.15.2:0/924803651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2354dd210 0x55d2354f3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.316+0000 7f1d6964e740 1 --2- 172.21.15.2:0/924803651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2354dc8f0 0x55d2354d8100 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.316+0000 7f1d6964e740 1 -- 172.21.15.2:0/924803651 >> 172.21.15.2:0/924803651 conn(0x55d2354d8e00 msgr2=0x55d2354d91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.316+0000 7f1d6964e740 1 -- 172.21.15.2:0/924803651 shutdown_connections 2024-05-14T17:04:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.316+0000 7f1d6964e740 1 -- 172.21.15.2:0/924803651 wait complete. 2024-05-14T17:04:16.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.316+0000 7f1d6964e740 1 -- start start 2024-05-14T17:04:16.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.316+0000 7f1d6964e740 1 Processor -- start 2024-05-14T17:04:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.317+0000 7f1d6964e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2354dc8f0 0x55d2354efc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.317+0000 7f1d6964e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2354dd210 0x55d2354ecfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:16.187 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.317+0000 7f1d6964e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2354e7ad0 0x55d2354ed510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:16.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.317+0000 7f1d6964e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1d601aef80 con 0x55d2354dc8f0 2024-05-14T17:04:16.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.317+0000 7f1d6964e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1d601b03f0 con 0x55d2354e7ad0 2024-05-14T17:04:16.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.317+0000 7f1d6964e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1d601af650 con 0x55d2354dd210 2024-05-14T17:04:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.317+0000 7f1d6964e740 10 client.0 _ll_register_callbacks cb 0x7f1d5000b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:04:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.317+0000 7f1d5ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2354dc8f0 0x55d2354efc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:16.252 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.317+0000 7f1d5ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2354dc8f0 0x55d2354efc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60134/0 (socket says 172.21.15.2:60134) 2024-05-14T17:04:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.317+0000 7f1d5ffff640 1 -- 172.21.15.2:0/2930465658 learned_addr learned my addr 172.21.15.2:0/2930465658 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:04:16.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.317+0000 7f1d5f7fe640 1 --2- 172.21.15.2:0/2930465658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2354dd210 0x55d2354ecfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:16.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.317+0000 7f1d64ecf640 1 --2- 172.21.15.2:0/2930465658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2354e7ad0 0x55d2354ed510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:16.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.317+0000 7f1d5ffff640 1 -- 172.21.15.2:0/2930465658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2354dd210 msgr2=0x55d2354ecfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.317+0000 7f1d5ffff640 1 --2- 172.21.15.2:0/2930465658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2354dd210 0x55d2354ecfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.317+0000 7f1d5ffff640 1 -- 172.21.15.2:0/2930465658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2354e7ad0 msgr2=0x55d2354ed510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.317+0000 7f1d5ffff640 1 --2- 172.21.15.2:0/2930465658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2354e7ad0 0x55d2354ed510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:16.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.317+0000 7f1d5ffff640 1 -- 172.21.15.2:0/2930465658 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2354eda50 con 0x55d2354dc8f0 2024-05-14T17:04:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.317+0000 7f1d64ecf640 1 --2- 172.21.15.2:0/2930465658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2354e7ad0 0x55d2354ed510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:04:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.317+0000 7f1d5ffff640 1 --2- 172.21.15.2:0/2930465658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2354dc8f0 0x55d2354efc40 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f1d5001ce90 tx=0x7f1d5000b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4512ef0927cc010d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.317+0000 7f1d3f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:04:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.317+0000 7f1d3f7fe640 1 -- 172.21.15.2:0/2930465658 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d50038040 con 0x55d2354dc8f0 2024-05-14T17:04:16.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.317+0000 7f1d3f7fe640 1 -- 172.21.15.2:0/2930465658 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8557+0+0 (secure 0 0 0) 0x7f1d50011440 con 0x55d2354dc8f0 2024-05-14T17:04:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.317+0000 7f1d6964e740 1 -- 172.21.15.2:0/2930465658 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d500056a0 con 0x55d2354dc8f0 2024-05-14T17:04:16.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.318+0000 7f1d3f7fe640 1 -- 172.21.15.2:0/2930465658 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f1d5004a050 con 0x55d2354dc8f0 2024-05-14T17:04:16.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.318+0000 7f1d6964e740 10 client.16353 fetch_fsmap learned FSMap version 41 2024-05-14T17:04:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.318+0000 7f1d6964e740 1 -- 172.21.15.2:0/2930465658 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55d235561920 con 0x55d2354dc8f0 2024-05-14T17:04:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.318+0000 7f1d3f7fe640 1 -- 172.21.15.2:0/2930465658 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d500056a0 con 0x55d2354dc8f0 2024-05-14T17:04:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.318+0000 7f1d3f7fe640 1 -- 172.21.15.2:0/2930465658 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f1d5003b070 con 0x55d2354dc8f0 2024-05-14T17:04:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.318+0000 7f1d6964e740 10 client.16353 fetch_fsmap finished waiting for FSMap version 41 2024-05-14T17:04:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.318+0000 7f1d6964e740 10 client.16353 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:04:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.318+0000 7f1d6964e740 20 client.16353 populate_metadata read hostname 'smithi002' 2024-05-14T17:04:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.318+0000 7f1d6964e740 1 --2- 172.21.15.2:0/2930465658 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d235561f60 0x55d235564380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.318+0000 7f1d6964e740 4 client.16353 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:04:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.318+0000 7f1d6964e740 1 -- 172.21.15.2:0/2930465658 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d23544b360 con 0x55d235561f60 2024-05-14T17:04:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.318+0000 7f1d5f7fe640 1 --2- 172.21.15.2:0/2930465658 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d235561f60 0x55d235564380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.318+0000 7f1d3f7fe640 10 client.16353 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:04:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.318+0000 7f1d5f7fe640 1 --2- 172.21.15.2:0/2930465658 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d235561f60 0x55d235564380 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5102997cbf4caed4 server_cookie=9fb54b6b70023ef0 in_seq=0 out_seq=0 2024-05-14T17:04:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.331+0000 7f1d3f7fe640 1 -- 172.21.15.2:0/2930465658 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d23544b360 con 0x55d235561f60 2024-05-14T17:04:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.331+0000 7f1d3f7fe640 10 client.16353 handle_command_reply: tid=0 2024-05-14T17:04:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.331+0000 7f1d6964e740 1 client.16353 shutdown 2024-05-14T17:04:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.331+0000 7f1d6964e740 1 -- 172.21.15.2:0/2930465658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2354dc8f0 msgr2=0x55d2354efc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:16.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.331+0000 7f1d6964e740 1 --2- 172.21.15.2:0/2930465658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2354dc8f0 0x55d2354efc40 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f1d5001ce90 tx=0x7f1d5000b820 comp rx=0 tx=0).stop 2024-05-14T17:04:16.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.331+0000 7f1d6964e740 1 -- 172.21.15.2:0/2930465658 shutdown_connections 2024-05-14T17:04:16.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.331+0000 7f1d6964e740 1 --2- 172.21.15.2:0/2930465658 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d235561f60 0x55d235564380 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.331+0000 7f1d6964e740 1 --2- 172.21.15.2:0/2930465658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2354e7ad0 0x55d2354ed510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.331+0000 7f1d6964e740 1 --2- 172.21.15.2:0/2930465658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2354dd210 0x55d2354ecfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:16.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.331+0000 7f1d3f7fe640 0 client.16353 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:04:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.331+0000 7f1d6964e740 1 --2- 172.21.15.2:0/2930465658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d2354dc8f0 0x55d2354efc40 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.331+0000 7f1d6964e740 1 -- 172.21.15.2:0/2930465658 >> 172.21.15.2:0/2930465658 conn(0x55d2354d8e00 msgr2=0x55d2354d8100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.331+0000 7f1d6964e740 1 -- 172.21.15.2:0/2930465658 shutdown_connections 2024-05-14T17:04:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.331+0000 7f1d6964e740 1 -- 172.21.15.2:0/2930465658 wait complete. 2024-05-14T17:04:16.939 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.331+0000 7f1d6964e740 20 client.16353 trim_cache size 0 max 16384 2024-05-14T17:04:16.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:04:16.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:04:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:04:16.973 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:04:14 smithi123 ceph-mon[35971]: pgmap v297: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 35 MiB/s wr, 1.45k op/s 2024-05-14T17:04:17.003 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:04:16 smithi123 ceph-mon[35971]: pgmap v298: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 32 MiB/s wr, 1.34k op/s 2024-05-14T17:04:17.010 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:04:14 smithi151 ceph-mon[36066]: pgmap v297: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 35 MiB/s wr, 1.45k op/s 2024-05-14T17:04:17.018 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:04:16 smithi151 ceph-mon[36066]: pgmap v298: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 32 MiB/s wr, 1.34k op/s 2024-05-14T17:04:17.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.332+0000 7f1d66932640 1 -- 172.21.15.2:0/4082165865 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f1d480b2ac0 msgr2=0x7f1d480b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:17.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.332+0000 7f1d66932640 1 --2- 172.21.15.2:0/4082165865 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f1d480b2ac0 0x7f1d480b4f80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1d58004af0 tx=0x7f1d58005d80 comp rx=0 tx=0).stop 2024-05-14T17:04:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.332+0000 7f1d66932640 1 -- 172.21.15.2:0/4082165865 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d60102330 msgr2=0x7f1d60103a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.332+0000 7f1d66932640 1 --2- 172.21.15.2:0/4082165865 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d60102330 0x7f1d60103a80 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f1d5400e9f0 tx=0x7f1d5400eeb0 comp rx=0 tx=0).stop 2024-05-14T17:04:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.332+0000 7f1d66932640 1 -- 172.21.15.2:0/4082165865 shutdown_connections 2024-05-14T17:04:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.332+0000 7f1d66932640 1 --2- 172.21.15.2:0/4082165865 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f1d480b2ac0 0x7f1d480b4f80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.332+0000 7f1d66932640 1 --2- 172.21.15.2:0/4082165865 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d60107980 0x7f1d60103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.332+0000 7f1d66932640 1 --2- 172.21.15.2:0/4082165865 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d60102330 0x7f1d60103a80 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:17.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.332+0000 7f1d66932640 1 --2- 172.21.15.2:0/4082165865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d60101970 0x7f1d60103540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.332+0000 7f1d66932640 1 -- 172.21.15.2:0/4082165865 >> 172.21.15.2:0/4082165865 conn(0x7f1d60077cd0 msgr2=0x7f1d6006b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.332+0000 7f1d66932640 1 -- 172.21.15.2:0/4082165865 shutdown_connections 2024-05-14T17:04:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:11.332+0000 7f1d66932640 1 -- 172.21.15.2:0/4082165865 wait complete. 2024-05-14T17:04:17.280 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 535.0 s, 1632.5 tps, lat 41.075 ms stddev 51.271 2024-05-14T17:04:17.288 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:04:12 smithi002 ceph-mon[31538]: pgmap v295: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 40 MiB/s wr, 1.70k op/s 2024-05-14T17:04:17.312 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:04:12 smithi002 ceph-mon[31538]: osdmap e89: 12 total, 12 up, 12 in 2024-05-14T17:04:17.321 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:04:14 smithi002 ceph-mon[31538]: pgmap v297: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 35 MiB/s wr, 1.45k op/s 2024-05-14T17:04:17.328 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:04:16 smithi002 ceph-mon[31538]: pgmap v298: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 32 MiB/s wr, 1.34k op/s 2024-05-14T17:04:17.345 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 540.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:04:18.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:04:18 smithi002 ceph-mon[31538]: pgmap v299: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 32 MiB/s wr, 1.34k op/s 2024-05-14T17:04:18.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:04:18 smithi151 ceph-mon[36066]: pgmap v299: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 32 MiB/s wr, 1.34k op/s 2024-05-14T17:04:18.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:04:18 smithi123 ceph-mon[35971]: pgmap v299: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 32 MiB/s wr, 1.34k op/s 2024-05-14T17:04:20.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:04:20 smithi002 ceph-mon[31538]: pgmap v300: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 3.2 MiB/s wr, 177 op/s 2024-05-14T17:04:20.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:04:20 smithi151 ceph-mon[36066]: pgmap v300: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 3.2 MiB/s wr, 177 op/s 2024-05-14T17:04:20.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:04:20 smithi123 ceph-mon[35971]: pgmap v300: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 3.2 MiB/s wr, 177 op/s 2024-05-14T17:04:21.751 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 545.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:04:22.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:04:22 smithi002 ceph-mon[31538]: pgmap v301: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 3.2 MiB/s wr, 177 op/s 2024-05-14T17:04:22.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:04:22 smithi151 ceph-mon[36066]: pgmap v301: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 3.2 MiB/s wr, 177 op/s 2024-05-14T17:04:22.915 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:04:22 smithi123 ceph-mon[35971]: pgmap v301: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 3.2 MiB/s wr, 177 op/s 2024-05-14T17:04:23.158 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:04:23.433 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:04:24.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.435+0000 7fb658262640 1 -- 172.21.15.2:0/4165731727 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb65011c0f0 msgr2=0x7fb65011e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:24.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.435+0000 7fb658262640 1 --2- 172.21.15.2:0/4165731727 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb65011c0f0 0x7fb65011e4e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fb648009900 tx=0x7fb64802a410 comp rx=0 tx=0).stop 2024-05-14T17:04:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.436+0000 7fb658262640 1 -- 172.21.15.2:0/4165731727 shutdown_connections 2024-05-14T17:04:24.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.436+0000 7fb658262640 1 --2- 172.21.15.2:0/4165731727 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb65011c0f0 0x7fb65011e4e0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.436+0000 7fb658262640 1 --2- 172.21.15.2:0/4165731727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb65006ef90 0x7fb65011bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.436+0000 7fb658262640 1 --2- 172.21.15.2:0/4165731727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb65006e670 0x7fb65006ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:24.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.436+0000 7fb658262640 1 -- 172.21.15.2:0/4165731727 >> 172.21.15.2:0/4165731727 conn(0x7fb65006dcd0 msgr2=0x7fb65010ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.436+0000 7fb658262640 1 -- 172.21.15.2:0/4165731727 shutdown_connections 2024-05-14T17:04:24.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.436+0000 7fb658262640 1 -- 172.21.15.2:0/4165731727 wait complete. 2024-05-14T17:04:24.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.436+0000 7fb658262640 1 Processor -- start 2024-05-14T17:04:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.437+0000 7fb658262640 1 -- start start 2024-05-14T17:04:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.437+0000 7fb658262640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb65006e670 0x7fb6501a7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.437+0000 7fb658262640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb65006ef90 0x7fb6501a84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.437+0000 7fb658262640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb65011c0f0 0x7fb6501ac7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.437+0000 7fb658262640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb650121b00 con 0x7fb65006ef90 2024-05-14T17:04:24.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.437+0000 7fb658262640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb650121980 con 0x7fb65011c0f0 2024-05-14T17:04:24.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.437+0000 7fb658262640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb650121c80 con 0x7fb65006e670 2024-05-14T17:04:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.437+0000 7fb6557fd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb65006ef90 0x7fb6501a84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.437+0000 7fb6557fd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb65006ef90 0x7fb6501a84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33704/0 (socket says 172.21.15.2:33704) 2024-05-14T17:04:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.437+0000 7fb6557fd640 1 -- 172.21.15.2:0/286850315 learned_addr learned my addr 172.21.15.2:0/286850315 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:04:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.437+0000 7fb655ffe640 1 --2- 172.21.15.2:0/286850315 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb65006e670 0x7fb6501a7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.437+0000 7fb6567ff640 1 --2- 172.21.15.2:0/286850315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb65011c0f0 0x7fb6501ac7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.437+0000 7fb6557fd640 1 -- 172.21.15.2:0/286850315 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb65006e670 msgr2=0x7fb6501a7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.437+0000 7fb6557fd640 1 --2- 172.21.15.2:0/286850315 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb65006e670 0x7fb6501a7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:24.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.437+0000 7fb6557fd640 1 -- 172.21.15.2:0/286850315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb65011c0f0 msgr2=0x7fb6501ac7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:24.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.437+0000 7fb6557fd640 1 --2- 172.21.15.2:0/286850315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb65011c0f0 0x7fb6501ac7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.437+0000 7fb6557fd640 1 -- 172.21.15.2:0/286850315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6501acf20 con 0x7fb65006ef90 2024-05-14T17:04:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.437+0000 7fb6557fd640 1 --2- 172.21.15.2:0/286850315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb65006ef90 0x7fb6501a84c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fb64000d950 tx=0x7fb64000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.438+0000 7fb646ffd640 1 -- 172.21.15.2:0/286850315 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640017070 con 0x7fb65006ef90 2024-05-14T17:04:24.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.438+0000 7fb646ffd640 1 -- 172.21.15.2:0/286850315 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb640009c30 con 0x7fb65006ef90 2024-05-14T17:04:24.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.438+0000 7fb658262640 1 -- 172.21.15.2:0/286850315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6501ad210 con 0x7fb65006ef90 2024-05-14T17:04:24.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.438+0000 7fb646ffd640 1 -- 172.21.15.2:0/286850315 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64000f620 con 0x7fb65006ef90 2024-05-14T17:04:24.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.438+0000 7fb658262640 1 -- 172.21.15.2:0/286850315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6501b4aa0 con 0x7fb65006ef90 2024-05-14T17:04:24.857 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:04:24 smithi151 ceph-mon[36066]: pgmap v302: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 2.9 MiB/s wr, 164 op/s 2024-05-14T17:04:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.441+0000 7fb65af77740 1 Processor -- start 2024-05-14T17:04:24.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.441+0000 7fb65af77740 1 -- start start 2024-05-14T17:04:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.441+0000 7fb646ffd640 1 -- 172.21.15.2:0/286850315 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fb640010460 con 0x7fb65006ef90 2024-05-14T17:04:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.442+0000 7fb65af77740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b681da78f0 0x55b681da7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:24.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.442+0000 7fb65af77740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b681da8210 0x55b681db2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:24.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.442+0000 7fb65af77740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b681db2ad0 0x55b681db4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.442+0000 7fb65af77740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b681d1ac30 con 0x55b681da8210 2024-05-14T17:04:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.442+0000 7fb65af77740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b681d0a7d0 con 0x55b681da78f0 2024-05-14T17:04:24.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.442+0000 7fb65af77740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b681d0e810 con 0x55b681db2ad0 2024-05-14T17:04:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.442+0000 7fb655ffe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b681da8210 0x55b681db2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:24.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.442+0000 7fb655ffe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b681da8210 0x55b681db2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33706/0 (socket says 172.21.15.2:33706) 2024-05-14T17:04:25.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.442+0000 7fb655ffe640 1 -- 172.21.15.2:0/2494929805 learned_addr learned my addr 172.21.15.2:0/2494929805 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:04:25.011 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.442+0000 7fb6557fd640 1 --2- 172.21.15.2:0/2494929805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b681db2ad0 0x55b681db4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.442+0000 7fb6567ff640 1 --2- 172.21.15.2:0/2494929805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b681da78f0 0x55b681da7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:25.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.442+0000 7fb655ffe640 1 -- 172.21.15.2:0/2494929805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b681db2ad0 msgr2=0x55b681db4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:25.031 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.442+0000 7fb655ffe640 1 --2- 172.21.15.2:0/2494929805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b681db2ad0 0x55b681db4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.442+0000 7fb655ffe640 1 -- 172.21.15.2:0/2494929805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b681da78f0 msgr2=0x55b681da7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.442+0000 7fb655ffe640 1 --2- 172.21.15.2:0/2494929805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b681da78f0 0x55b681da7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:25.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.442+0000 7fb655ffe640 1 -- 172.21.15.2:0/2494929805 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b681a1f210 con 0x55b681da8210 2024-05-14T17:04:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.442+0000 7fb6557fd640 1 --2- 172.21.15.2:0/2494929805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b681db2ad0 0x55b681db4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:04:25.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.442+0000 7fb655ffe640 1 --2- 172.21.15.2:0/2494929805 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b681da8210 0x55b681db2590 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb64c003a20 tx=0x7fb64c003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37f2e455f70ebf8c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:25.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.442+0000 7fb63bfff640 1 -- 172.21.15.2:0/2494929805 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c004860 con 0x55b681da8210 2024-05-14T17:04:25.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.443+0000 7fb646ffd640 1 --2- 172.21.15.2:0/286850315 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb63c0b2a10 0x7fb63c0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:25.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.443+0000 7fb63bfff640 1 -- 172.21.15.2:0/2494929805 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb64c015030 con 0x55b681da8210 2024-05-14T17:04:25.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.443+0000 7fb63bfff640 1 -- 172.21.15.2:0/2494929805 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c010920 con 0x55b681da8210 2024-05-14T17:04:25.177 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.443+0000 7fb6567ff640 1 --2- 172.21.15.2:0/286850315 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb63c0b2a10 0x7fb63c0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.443+0000 7fb646ffd640 1 -- 172.21.15.2:0/286850315 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7fb6400e21c0 con 0x7fb65006ef90 2024-05-14T17:04:25.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.443+0000 7fb65af77740 1 -- 172.21.15.2:0/2494929805 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b681da8210 msgr2=0x55b681db2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.443+0000 7fb65af77740 1 --2- 172.21.15.2:0/2494929805 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b681da8210 0x55b681db2590 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb64c003a20 tx=0x7fb64c003ee0 comp rx=0 tx=0).stop 2024-05-14T17:04:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.443+0000 7fb6567ff640 1 --2- 172.21.15.2:0/286850315 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb63c0b2a10 0x7fb63c0b4ed0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb648005940 tx=0x7fb6480058d0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.443+0000 7fb65af77740 1 -- 172.21.15.2:0/2494929805 shutdown_connections 2024-05-14T17:04:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.443+0000 7fb65af77740 1 --2- 172.21.15.2:0/2494929805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b681db2ad0 0x55b681db4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:25.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.443+0000 7fb65af77740 1 --2- 172.21.15.2:0/2494929805 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b681da8210 0x55b681db2590 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.443+0000 7fb65af77740 1 --2- 172.21.15.2:0/2494929805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b681da78f0 0x55b681da7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:25.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.443+0000 7fb65af77740 1 -- 172.21.15.2:0/2494929805 >> 172.21.15.2:0/2494929805 conn(0x55b681da3e00 msgr2=0x55b681da41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.443+0000 7fb65af77740 1 -- 172.21.15.2:0/2494929805 shutdown_connections 2024-05-14T17:04:25.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.443+0000 7fb65af77740 1 -- 172.21.15.2:0/2494929805 wait complete. 2024-05-14T17:04:25.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.444+0000 7fb65af77740 1 -- start start 2024-05-14T17:04:25.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.444+0000 7fb65af77740 1 Processor -- start 2024-05-14T17:04:25.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.444+0000 7fb65af77740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b681da78f0 0x55b681db23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.444+0000 7fb65af77740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b681da8210 0x55b681daf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:25.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.444+0000 7fb65af77740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b681db2ad0 0x55b681dafbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.444+0000 7fb65af77740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b6819f0790 con 0x55b681da8210 2024-05-14T17:04:25.385 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.444+0000 7fb65af77740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b681cc6890 con 0x55b681db2ad0 2024-05-14T17:04:25.395 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.444+0000 7fb65af77740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b681db57a0 con 0x55b681da78f0 2024-05-14T17:04:25.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.444+0000 7fb65af77740 10 client.0 _ll_register_callbacks cb 0x55b681cc9d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:04:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.445+0000 7fb6557fd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b681da8210 0x55b681daf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.445+0000 7fb6557fd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b681da8210 0x55b681daf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33712/0 (socket says 172.21.15.2:33712) 2024-05-14T17:04:25.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.445+0000 7fb6557fd640 1 -- 172.21.15.2:0/3923316459 learned_addr learned my addr 172.21.15.2:0/3923316459 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:04:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.445+0000 7fb6567ff640 1 --2- 172.21.15.2:0/3923316459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b681db2ad0 0x55b681dafbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:25.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.445+0000 7fb655ffe640 1 --2- 172.21.15.2:0/3923316459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b681da78f0 0x55b681db23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.445+0000 7fb6557fd640 1 -- 172.21.15.2:0/3923316459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b681da78f0 msgr2=0x55b681db23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:25.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.445+0000 7fb6557fd640 1 --2- 172.21.15.2:0/3923316459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b681da78f0 0x55b681db23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.445+0000 7fb6557fd640 1 -- 172.21.15.2:0/3923316459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b681db2ad0 msgr2=0x55b681dafbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:25.549 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.445+0000 7fb6557fd640 1 --2- 172.21.15.2:0/3923316459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b681db2ad0 0x55b681dafbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:25.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.445+0000 7fb6557fd640 1 -- 172.21.15.2:0/3923316459 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b681aeb620 con 0x55b681da8210 2024-05-14T17:04:25.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.445+0000 7fb6567ff640 1 --2- 172.21.15.2:0/3923316459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b681db2ad0 0x55b681dafbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:04:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.445+0000 7fb6557fd640 1 --2- 172.21.15.2:0/3923316459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b681da8210 0x55b681daf6a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb6400a91a0 tx=0x7fb6400a93a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3de7d6b3881bdcd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.445+0000 7fb6397fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:04:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.445+0000 7fb6397fa640 1 -- 172.21.15.2:0/3923316459 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640020050 con 0x55b681da8210 2024-05-14T17:04:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.445+0000 7fb6397fa640 1 -- 172.21.15.2:0/3923316459 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7fb6400b1e70 con 0x55b681da8210 2024-05-14T17:04:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.445+0000 7fb65af77740 1 -- 172.21.15.2:0/3923316459 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb64c015030 con 0x55b681da8210 2024-05-14T17:04:25.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.445+0000 7fb6397fa640 1 -- 172.21.15.2:0/3923316459 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb6400a8fb0 con 0x55b681da8210 2024-05-14T17:04:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.445+0000 7fb65af77740 10 client.16365 fetch_fsmap learned FSMap version 41 2024-05-14T17:04:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.445+0000 7fb65af77740 1 -- 172.21.15.2:0/3923316459 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55b681dc1180 con 0x55b681da8210 2024-05-14T17:04:25.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.446+0000 7fb6397fa640 1 -- 172.21.15.2:0/3923316459 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb64c015030 con 0x55b681da8210 2024-05-14T17:04:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.446+0000 7fb6397fa640 1 -- 172.21.15.2:0/3923316459 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fb6400a3bc0 con 0x55b681da8210 2024-05-14T17:04:25.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.446+0000 7fb65af77740 10 client.16365 fetch_fsmap finished waiting for FSMap version 41 2024-05-14T17:04:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.446+0000 7fb65af77740 10 client.16365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:04:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.446+0000 7fb65af77740 20 client.16365 populate_metadata read hostname 'smithi002' 2024-05-14T17:04:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.446+0000 7fb65af77740 1 --2- 172.21.15.2:0/3923316459 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55b681dc2640 0x55b681dc4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.446+0000 7fb65af77740 4 client.16365 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:04:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.446+0000 7fb65af77740 1 -- 172.21.15.2:0/3923316459 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b681d1cb80 con 0x55b681dc2640 2024-05-14T17:04:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.446+0000 7fb655ffe640 1 --2- 172.21.15.2:0/3923316459 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55b681dc2640 0x55b681dc4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.447+0000 7fb655ffe640 1 --2- 172.21.15.2:0/3923316459 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55b681dc2640 0x55b681dc4a50 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22c575cf185da1cd server_cookie=3461aa861e7e95c3 in_seq=0 out_seq=0 2024-05-14T17:04:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.447+0000 7fb6397fa640 10 client.16365 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:04:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.447+0000 7fb6397fa640 1 -- 172.21.15.2:0/3923316459 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b681d1cb80 con 0x55b681dc2640 2024-05-14T17:04:25.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.447+0000 7fb6397fa640 10 client.16365 handle_command_reply: tid=0 2024-05-14T17:04:25.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.447+0000 7fb65af77740 1 client.16365 shutdown 2024-05-14T17:04:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.447+0000 7fb65af77740 1 -- 172.21.15.2:0/3923316459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b681da8210 msgr2=0x55b681daf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:25.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.447+0000 7fb65af77740 1 --2- 172.21.15.2:0/3923316459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b681da8210 0x55b681daf6a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb6400a91a0 tx=0x7fb6400a93a0 comp rx=0 tx=0).stop 2024-05-14T17:04:25.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.448+0000 7fb65af77740 1 -- 172.21.15.2:0/3923316459 shutdown_connections 2024-05-14T17:04:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.448+0000 7fb65af77740 1 --2- 172.21.15.2:0/3923316459 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55b681dc2640 0x55b681dc4a50 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.448+0000 7fb65af77740 1 --2- 172.21.15.2:0/3923316459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b681db2ad0 0x55b681dafbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.448+0000 7fb65af77740 1 --2- 172.21.15.2:0/3923316459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b681da8210 0x55b681daf6a0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.448+0000 7fb65af77740 1 --2- 172.21.15.2:0/3923316459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b681da78f0 0x55b681db23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.448+0000 7fb6397fa640 0 client.16365 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:04:25.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.448+0000 7fb65af77740 1 -- 172.21.15.2:0/3923316459 >> 172.21.15.2:0/3923316459 conn(0x55b681da3e00 msgr2=0x55b681da3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.448+0000 7fb65af77740 1 -- 172.21.15.2:0/3923316459 shutdown_connections 2024-05-14T17:04:25.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.448+0000 7fb65af77740 1 -- 172.21.15.2:0/3923316459 wait complete. 2024-05-14T17:04:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.448+0000 7fb65af77740 20 client.16365 trim_cache size 0 max 16384 2024-05-14T17:04:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.463+0000 7fb65af77740 1 Processor -- start 2024-05-14T17:04:25.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.463+0000 7fb65af77740 1 -- start start 2024-05-14T17:04:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.463+0000 7fb65af77740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b681da78f0 0x55b681daaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.463+0000 7fb65af77740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b681da8210 0x55b681db1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.463+0000 7fb65af77740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b681db2ad0 0x55b681dca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:25.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.463+0000 7fb65af77740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb6501bfcf0 con 0x55b681da8210 2024-05-14T17:04:25.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.463+0000 7fb65af77740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb6501bd5c0 con 0x55b681da78f0 2024-05-14T17:04:26.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.463+0000 7fb65af77740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb6501bb430 con 0x55b681db2ad0 2024-05-14T17:04:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.463+0000 7fb6557fd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b681da8210 0x55b681db1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:26.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.463+0000 7fb6557fd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b681da8210 0x55b681db1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33722/0 (socket says 172.21.15.2:33722) 2024-05-14T17:04:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.463+0000 7fb6557fd640 1 -- 172.21.15.2:0/1640920871 learned_addr learned my addr 172.21.15.2:0/1640920871 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:04:26.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.463+0000 7fb655ffe640 1 --2- 172.21.15.2:0/1640920871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b681da78f0 0x55b681daaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:26.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.464+0000 7fb6557fd640 1 -- 172.21.15.2:0/1640920871 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b681db2ad0 msgr2=0x55b681dca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:26.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.464+0000 7fb6557fd640 1 --2- 172.21.15.2:0/1640920871 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b681db2ad0 0x55b681dca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.464+0000 7fb6557fd640 1 -- 172.21.15.2:0/1640920871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b681da78f0 msgr2=0x55b681daaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.464+0000 7fb6557fd640 1 --2- 172.21.15.2:0/1640920871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b681da78f0 0x55b681daaa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.464+0000 7fb6557fd640 1 -- 172.21.15.2:0/1640920871 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b681dadbf0 con 0x55b681da8210 2024-05-14T17:04:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.464+0000 7fb655ffe640 1 --2- 172.21.15.2:0/1640920871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b681da78f0 0x55b681daaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:04:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.464+0000 7fb6557fd640 1 --2- 172.21.15.2:0/1640920871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b681da8210 0x55b681db1ee0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb6400a9150 tx=0x7fb6400e58e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad2bac3867c370a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.464+0000 7fb63b7fe640 1 -- 172.21.15.2:0/1640920871 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640020070 con 0x55b681da8210 2024-05-14T17:04:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.464+0000 7fb63b7fe640 1 -- 172.21.15.2:0/1640920871 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6400a9ba0 con 0x55b681da8210 2024-05-14T17:04:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.464+0000 7fb63b7fe640 1 -- 172.21.15.2:0/1640920871 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6400a3bc0 con 0x55b681da8210 2024-05-14T17:04:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.465+0000 7fb65af77740 1 -- 172.21.15.2:0/1640920871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b681da8210 msgr2=0x55b681db1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.465+0000 7fb65af77740 1 --2- 172.21.15.2:0/1640920871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b681da8210 0x55b681db1ee0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb6400a9150 tx=0x7fb6400e58e0 comp rx=0 tx=0).stop 2024-05-14T17:04:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.465+0000 7fb65af77740 1 -- 172.21.15.2:0/1640920871 shutdown_connections 2024-05-14T17:04:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.465+0000 7fb65af77740 1 --2- 172.21.15.2:0/1640920871 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b681db2ad0 0x55b681dca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.465+0000 7fb65af77740 1 --2- 172.21.15.2:0/1640920871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b681da8210 0x55b681db1ee0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.465+0000 7fb65af77740 1 --2- 172.21.15.2:0/1640920871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b681da78f0 0x55b681daaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.465+0000 7fb65af77740 1 -- 172.21.15.2:0/1640920871 >> 172.21.15.2:0/1640920871 conn(0x55b681da3e00 msgr2=0x55b681daa370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:26.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.465+0000 7fb65af77740 1 -- 172.21.15.2:0/1640920871 shutdown_connections 2024-05-14T17:04:26.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.465+0000 7fb65af77740 1 -- 172.21.15.2:0/1640920871 wait complete. 2024-05-14T17:04:26.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.465+0000 7fb65af77740 1 -- start start 2024-05-14T17:04:26.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.465+0000 7fb65af77740 1 Processor -- start 2024-05-14T17:04:26.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.465+0000 7fb65af77740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b681da78f0 0x55b681db6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:26.396 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.465+0000 7fb65af77740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b681da8210 0x55b681db7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:26.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.465+0000 7fb65af77740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b681db2ad0 0x55b681e2bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.466+0000 7fb65af77740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb6501bba80 con 0x55b681da8210 2024-05-14T17:04:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.466+0000 7fb65af77740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb6501bcef0 con 0x55b681da78f0 2024-05-14T17:04:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.466+0000 7fb65af77740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb6501bc150 con 0x55b681db2ad0 2024-05-14T17:04:26.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.466+0000 7fb65af77740 10 client.0 _ll_register_callbacks cb 0x7fb6400a5360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:04:26.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.466+0000 7fb6557fd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b681da8210 0x55b681db7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:26.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.466+0000 7fb6557fd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b681da8210 0x55b681db7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33732/0 (socket says 172.21.15.2:33732) 2024-05-14T17:04:26.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.466+0000 7fb6557fd640 1 -- 172.21.15.2:0/2494717990 learned_addr learned my addr 172.21.15.2:0/2494717990 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:04:26.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.466+0000 7fb6567ff640 1 --2- 172.21.15.2:0/2494717990 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b681db2ad0 0x55b681e2bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:26.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.466+0000 7fb655ffe640 1 --2- 172.21.15.2:0/2494717990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b681da78f0 0x55b681db6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.466+0000 7fb6557fd640 1 -- 172.21.15.2:0/2494717990 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b681db2ad0 msgr2=0x55b681e2bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.466+0000 7fb6557fd640 1 --2- 172.21.15.2:0/2494717990 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b681db2ad0 0x55b681e2bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.466+0000 7fb6557fd640 1 -- 172.21.15.2:0/2494717990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b681da78f0 msgr2=0x55b681db6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:26.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.466+0000 7fb6557fd640 1 --2- 172.21.15.2:0/2494717990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b681da78f0 0x55b681db6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:26.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.466+0000 7fb6557fd640 1 -- 172.21.15.2:0/2494717990 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b681e2c450 con 0x55b681da8210 2024-05-14T17:04:26.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.466+0000 7fb6557fd640 1 --2- 172.21.15.2:0/2494717990 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b681da8210 0x55b681db7160 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb6400dff10 tx=0x7fb6400b2120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=481fe7f82cee4644 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:26.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.466+0000 7fb6397fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:04:26.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.466+0000 7fb6397fa640 1 -- 172.21.15.2:0/2494717990 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640020070 con 0x55b681da8210 2024-05-14T17:04:26.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.466+0000 7fb6397fa640 1 -- 172.21.15.2:0/2494717990 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7fb6400a8910 con 0x55b681da8210 2024-05-14T17:04:26.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.466+0000 7fb65af77740 1 -- 172.21.15.2:0/2494717990 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6400a9ba0 con 0x55b681da8210 2024-05-14T17:04:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.467+0000 7fb6397fa640 1 -- 172.21.15.2:0/2494717990 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb640009de0 con 0x55b681da8210 2024-05-14T17:04:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.467+0000 7fb65af77740 10 client.16371 fetch_fsmap learned FSMap version 41 2024-05-14T17:04:26.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.467+0000 7fb65af77740 1 -- 172.21.15.2:0/2494717990 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55b681e2c450 con 0x55b681da8210 2024-05-14T17:04:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.467+0000 7fb6397fa640 1 -- 172.21.15.2:0/2494717990 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6400a9ba0 con 0x55b681da8210 2024-05-14T17:04:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.467+0000 7fb6397fa640 1 -- 172.21.15.2:0/2494717990 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fb6400ca200 con 0x55b681da8210 2024-05-14T17:04:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.467+0000 7fb65af77740 10 client.16371 fetch_fsmap finished waiting for FSMap version 41 2024-05-14T17:04:26.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.467+0000 7fb65af77740 10 client.16371 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:04:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.467+0000 7fb65af77740 20 client.16371 populate_metadata read hostname 'smithi002' 2024-05-14T17:04:26.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.467+0000 7fb65af77740 1 --2- 172.21.15.2:0/2494717990 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55b681e2d930 0x55b681e2fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.467+0000 7fb65af77740 4 client.16371 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:04:26.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.467+0000 7fb65af77740 1 -- 172.21.15.2:0/2494717990 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b681d16360 con 0x55b681e2d930 2024-05-14T17:04:26.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.467+0000 7fb655ffe640 1 --2- 172.21.15.2:0/2494717990 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55b681e2d930 0x55b681e2fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:26.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.468+0000 7fb655ffe640 1 --2- 172.21.15.2:0/2494717990 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55b681e2d930 0x55b681e2fd50 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eed3eb9ffad8c35d server_cookie=c63e549bb3536349 in_seq=0 out_seq=0 2024-05-14T17:04:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.468+0000 7fb6397fa640 10 client.16371 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:04:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.479+0000 7fb6397fa640 1 -- 172.21.15.2:0/2494717990 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b681d16360 con 0x55b681e2d930 2024-05-14T17:04:26.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.480+0000 7fb6397fa640 10 client.16371 handle_command_reply: tid=0 2024-05-14T17:04:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.480+0000 7fb65af77740 1 client.16371 shutdown 2024-05-14T17:04:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.480+0000 7fb65af77740 1 -- 172.21.15.2:0/2494717990 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b681da8210 msgr2=0x55b681db7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.480+0000 7fb65af77740 1 --2- 172.21.15.2:0/2494717990 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b681da8210 0x55b681db7160 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb6400dff10 tx=0x7fb6400b2120 comp rx=0 tx=0).stop 2024-05-14T17:04:26.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.480+0000 7fb65af77740 1 -- 172.21.15.2:0/2494717990 shutdown_connections 2024-05-14T17:04:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.480+0000 7fb65af77740 1 --2- 172.21.15.2:0/2494717990 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55b681e2d930 0x55b681e2fd50 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:26.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.480+0000 7fb65af77740 1 --2- 172.21.15.2:0/2494717990 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b681db2ad0 0x55b681e2bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.480+0000 7fb6397fa640 0 client.16371 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:04:26.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.480+0000 7fb65af77740 1 --2- 172.21.15.2:0/2494717990 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b681da8210 0x55b681db7160 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.480+0000 7fb65af77740 1 --2- 172.21.15.2:0/2494717990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b681da78f0 0x55b681db6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:26.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.480+0000 7fb65af77740 1 -- 172.21.15.2:0/2494717990 >> 172.21.15.2:0/2494717990 conn(0x55b681da3e00 msgr2=0x55b681da3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.480+0000 7fb65af77740 1 -- 172.21.15.2:0/2494717990 shutdown_connections 2024-05-14T17:04:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.480+0000 7fb65af77740 1 -- 172.21.15.2:0/2494717990 wait complete. 2024-05-14T17:04:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.480+0000 7fb65af77740 20 client.16371 trim_cache size 0 max 16384 2024-05-14T17:04:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:04:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:04:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:04:27.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.481+0000 7fb658262640 1 -- 172.21.15.2:0/286850315 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb63c0b2a10 msgr2=0x7fb63c0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.481+0000 7fb658262640 1 --2- 172.21.15.2:0/286850315 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb63c0b2a10 0x7fb63c0b4ed0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb648005940 tx=0x7fb6480058d0 comp rx=0 tx=0).stop 2024-05-14T17:04:27.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.481+0000 7fb658262640 1 -- 172.21.15.2:0/286850315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb65006ef90 msgr2=0x7fb6501a84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.481+0000 7fb658262640 1 --2- 172.21.15.2:0/286850315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb65006ef90 0x7fb6501a84c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fb64000d950 tx=0x7fb64000de10 comp rx=0 tx=0).stop 2024-05-14T17:04:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.481+0000 7fb658262640 1 -- 172.21.15.2:0/286850315 shutdown_connections 2024-05-14T17:04:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.481+0000 7fb658262640 1 --2- 172.21.15.2:0/286850315 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb63c0b2a10 0x7fb63c0b4ed0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.481+0000 7fb658262640 1 --2- 172.21.15.2:0/286850315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb65011c0f0 0x7fb6501ac7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.481+0000 7fb658262640 1 --2- 172.21.15.2:0/286850315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb65006ef90 0x7fb6501a84c0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.481+0000 7fb658262640 1 --2- 172.21.15.2:0/286850315 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb65006e670 0x7fb6501a7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.481+0000 7fb658262640 1 -- 172.21.15.2:0/286850315 >> 172.21.15.2:0/286850315 conn(0x7fb65006dcd0 msgr2=0x7fb650071af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:27.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.481+0000 7fb658262640 1 -- 172.21.15.2:0/286850315 shutdown_connections 2024-05-14T17:04:27.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:24.481+0000 7fb658262640 1 -- 172.21.15.2:0/286850315 wait complete. 2024-05-14T17:04:27.198 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:04:26 smithi151 ceph-mon[36066]: pgmap v303: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 712 KiB/s wr, 42 op/s 2024-05-14T17:04:27.208 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:04:26 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/3844935437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:04:27.216 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:04:24 smithi002 ceph-mon[31538]: pgmap v302: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 2.9 MiB/s wr, 164 op/s 2024-05-14T17:04:27.224 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:04:24 smithi123 ceph-mon[35971]: pgmap v302: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 2.9 MiB/s wr, 164 op/s 2024-05-14T17:04:27.231 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:04:26 smithi123 ceph-mon[35971]: pgmap v303: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 712 KiB/s wr, 42 op/s 2024-05-14T17:04:27.240 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:04:26 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/3844935437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:04:27.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.254+0000 7f45b1afb700 1 -- 172.21.15.2:0/3934086354 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45ac102e00 msgr2=0x7f45ac10f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:27.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.254+0000 7f45b1afb700 1 --2- 172.21.15.2:0/3934086354 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45ac102e00 0x7f45ac10f2f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f45a400b0a0 tx=0x7f45a400b3b0 comp rx=0 tx=0).stop 2024-05-14T17:04:27.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.254+0000 7f45b1afb700 1 -- 172.21.15.2:0/3934086354 shutdown_connections 2024-05-14T17:04:27.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.254+0000 7f45b1afb700 1 --2- 172.21.15.2:0/3934086354 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45ac102e00 0x7f45ac10f2f0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:27.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.254+0000 7f45b1afb700 1 --2- 172.21.15.2:0/3934086354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45ac108520 0x7f45ac108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:27.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.254+0000 7f45b1afb700 1 --2- 172.21.15.2:0/3934086354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45ac102440 0x7f45ac1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:27.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.254+0000 7f45b1afb700 1 -- 172.21.15.2:0/3934086354 >> 172.21.15.2:0/3934086354 conn(0x7f45ac0793f0 msgr2=0x7f45ac079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:27.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.254+0000 7f45b1afb700 1 -- 172.21.15.2:0/3934086354 shutdown_connections 2024-05-14T17:04:27.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.254+0000 7f45b1afb700 1 -- 172.21.15.2:0/3934086354 wait complete. 2024-05-14T17:04:27.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.255+0000 7f45b1afb700 1 Processor -- start 2024-05-14T17:04:27.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.255+0000 7f45b1afb700 1 -- start start 2024-05-14T17:04:27.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.255+0000 7f45b1afb700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45ac102440 0x7f45ac104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:27.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.255+0000 7f45b1afb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45ac102e00 0x7f45ac1045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:27.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.255+0000 7f45b1afb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45ac108520 0x7f45ac104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:27.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.256+0000 7f45b1afb700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac1125d0 con 0x7f45ac108520 2024-05-14T17:04:27.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.256+0000 7f45b1afb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac112450 con 0x7f45ac102e00 2024-05-14T17:04:27.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.256+0000 7f45b1afb700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac112750 con 0x7f45ac102440 2024-05-14T17:04:27.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.256+0000 7f45abfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45ac108520 0x7f45ac104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:27.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.256+0000 7f45abfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45ac108520 0x7f45ac104b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33752/0 (socket says 172.21.15.2:33752) 2024-05-14T17:04:27.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.256+0000 7f45abfff700 1 -- 172.21.15.2:0/3844935437 learned_addr learned my addr 172.21.15.2:0/3844935437 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:04:27.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.256+0000 7f45ab7fe700 1 --2- 172.21.15.2:0/3844935437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45ac102440 0x7f45ac104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:27.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.256+0000 7f45aaffd700 1 --2- 172.21.15.2:0/3844935437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45ac102e00 0x7f45ac1045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:27.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.256+0000 7f45abfff700 1 -- 172.21.15.2:0/3844935437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45ac102440 msgr2=0x7f45ac104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:27.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.256+0000 7f45abfff700 1 --2- 172.21.15.2:0/3844935437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45ac102440 0x7f45ac104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:27.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.256+0000 7f45abfff700 1 -- 172.21.15.2:0/3844935437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45ac102e00 msgr2=0x7f45ac1045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:27.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.256+0000 7f45abfff700 1 --2- 172.21.15.2:0/3844935437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45ac102e00 0x7f45ac1045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:27.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.256+0000 7f45abfff700 1 -- 172.21.15.2:0/3844935437 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45ac1a8020 con 0x7f45ac108520 2024-05-14T17:04:27.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.256+0000 7f45abfff700 1 --2- 172.21.15.2:0/3844935437 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45ac108520 0x7f45ac104b00 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f45a400b830 tx=0x7f45a4004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:27.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.256+0000 7f45a8ff9700 1 -- 172.21.15.2:0/3844935437 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a4021070 con 0x7f45ac108520 2024-05-14T17:04:27.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.256+0000 7f45a8ff9700 1 -- 172.21.15.2:0/3844935437 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45a4007e30 con 0x7f45ac108520 2024-05-14T17:04:27.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.256+0000 7f45a8ff9700 1 -- 172.21.15.2:0/3844935437 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a40047d0 con 0x7f45ac108520 2024-05-14T17:04:27.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.256+0000 7f45b1afb700 1 -- 172.21.15.2:0/3844935437 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45ac1a82b0 con 0x7f45ac108520 2024-05-14T17:04:27.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.257+0000 7f45b1afb700 1 -- 172.21.15.2:0/3844935437 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45ac1a8760 con 0x7f45ac108520 2024-05-14T17:04:27.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.257+0000 7f45b1afb700 1 -- 172.21.15.2:0/3844935437 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45ac06a230 con 0x7f45ac108520 2024-05-14T17:04:27.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.259+0000 7f45a8ff9700 1 -- 172.21.15.2:0/3844935437 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f45a4007980 con 0x7f45ac108520 2024-05-14T17:04:27.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.260+0000 7f45a8ff9700 1 --2- 172.21.15.2:0/3844935437 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f45940b29f0 0x7f45940b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:27.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.260+0000 7f45ab7fe700 1 --2- 172.21.15.2:0/3844935437 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f45940b29f0 0x7f45940b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:27.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.260+0000 7f45a8ff9700 1 -- 172.21.15.2:0/3844935437 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f45a400f020 con 0x7f45ac108520 2024-05-14T17:04:27.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.260+0000 7f45ab7fe700 1 --2- 172.21.15.2:0/3844935437 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f45940b29f0 0x7f45940b4eb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f459c00abd0 tx=0x7f459c00a380 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:27.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.263+0000 7f45a8ff9700 1 -- 172.21.15.2:0/3844935437 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f45a402a020 con 0x7f45ac108520 2024-05-14T17:04:27.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.533+0000 7f45b1afb700 1 -- 172.21.15.2:0/3844935437 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45ac1a8a30 con 0x7f45ac108520 2024-05-14T17:04:27.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.533+0000 7f45a8ff9700 1 -- 172.21.15.2:0/3844935437 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11853 (secure 0 0 0) 0x7f45a40aba40 con 0x7f45ac108520 2024-05-14T17:04:27.774 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:04:27.851 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2275696923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2275696923},{"type":"v1","addr":"172.21.15.2:6835","nonce":2275696923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/716718761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":716718761},{"type":"v1","addr":"172.21.15.2:6839","nonce":716718761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/3644815179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3644815179},{"type":"v1","addr":"172.21.15.2:6841","nonce":3644815179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1880336731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1880336731},{"type":"v1","addr":"172.21.15.151:6833","nonce":1880336731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2415605548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2415605548},{"type":"v1","addr":"172.21.15.2:6837","nonce":2415605548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2635167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2635167},{"type":"v1","addr":"172.21.15.123:6833","nonce":2635167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:51:34.106919+0000","modified":"2024-05-14T17:02:50.475577+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24455,"mds_2":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24415":{"gid":24415,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/762044339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":762044339},{"type":"v1","addr":"172.21.15.151:6835","nonce":762044339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/132043151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":132043151},{"type":"v1","addr":"172.21.15.151:6837","nonce":132043151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/4107834375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":4107834375},{"type":"v1","addr":"172.21.15.151:6839","nonce":4107834375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.123:6835/1198670013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1198670013},{"type":"v1","addr":"172.21.15.123:6835","nonce":1198670013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/333426645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":333426645},{"type":"v1","addr":"172.21.15.123:6837","nonce":333426645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/580275877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":580275877},{"type":"v1","addr":"172.21.15.123:6839","nonce":580275877}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24431,24455]},"id":1}]} 2024-05-14T17:04:27.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.536+0000 7f45b1afb700 1 -- 172.21.15.2:0/3844935437 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f45940b29f0 msgr2=0x7f45940b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:27.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.536+0000 7f45b1afb700 1 --2- 172.21.15.2:0/3844935437 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f45940b29f0 0x7f45940b4eb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f459c00abd0 tx=0x7f459c00a380 comp rx=0 tx=0).stop 2024-05-14T17:04:27.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.536+0000 7f45b1afb700 1 -- 172.21.15.2:0/3844935437 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45ac108520 msgr2=0x7f45ac104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:27.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.536+0000 7f45b1afb700 1 --2- 172.21.15.2:0/3844935437 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45ac108520 0x7f45ac104b00 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f45a400b830 tx=0x7f45a4004ce0 comp rx=0 tx=0).stop 2024-05-14T17:04:27.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.536+0000 7f45b1afb700 1 -- 172.21.15.2:0/3844935437 shutdown_connections 2024-05-14T17:04:27.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.536+0000 7f45b1afb700 1 --2- 172.21.15.2:0/3844935437 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f45940b29f0 0x7f45940b4eb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:27.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.536+0000 7f45b1afb700 1 --2- 172.21.15.2:0/3844935437 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45ac108520 0x7f45ac104b00 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:27.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.536+0000 7f45b1afb700 1 --2- 172.21.15.2:0/3844935437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45ac102440 0x7f45ac104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:27.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.536+0000 7f45b1afb700 1 --2- 172.21.15.2:0/3844935437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45ac102e00 0x7f45ac1045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:27.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.536+0000 7f45b1afb700 1 -- 172.21.15.2:0/3844935437 >> 172.21.15.2:0/3844935437 conn(0x7f45ac0793f0 msgr2=0x7f45ac106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:27.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.536+0000 7f45b1afb700 1 -- 172.21.15.2:0/3844935437 shutdown_connections 2024-05-14T17:04:27.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:25.536+0000 7f45b1afb700 1 -- 172.21.15.2:0/3844935437 wait complete. 2024-05-14T17:04:27.967 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 41 2024-05-14T17:04:27.976 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id 6c170f95 --release --await' 2024-05-14T17:04:27.984 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id 6c170f95 --release --await 2024-05-14T17:04:27.995 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 550.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:04:28.056 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:04:26 smithi002 ceph-mon[31538]: pgmap v303: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 712 KiB/s wr, 42 op/s 2024-05-14T17:04:28.065 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:04:26 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/3844935437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:04:28.340 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:04:28.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:04:28 smithi002 ceph-mon[31538]: pgmap v304: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 40 KiB/s wr, 9 op/s 2024-05-14T17:04:28.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:04:28 smithi151 ceph-mon[36066]: pgmap v304: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 40 KiB/s wr, 9 op/s 2024-05-14T17:04:28.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:04:28 smithi123 ceph-mon[35971]: pgmap v304: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 40 KiB/s wr, 9 op/s 2024-05-14T17:04:29.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:04:29 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:04:29.874 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:04:29 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:04:29.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:04:29 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:04:30.138 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.138+0000 7f45a5115700 1 -- 172.21.15.2:0/2342750066 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45a0102e00 msgr2=0x7f45a010f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:30.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.138+0000 7f45a5115700 1 --2- 172.21.15.2:0/2342750066 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45a0102e00 0x7f45a010f2f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f459800b0a0 tx=0x7f459800b3b0 comp rx=0 tx=0).stop 2024-05-14T17:04:30.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.138+0000 7f45a5115700 1 -- 172.21.15.2:0/2342750066 shutdown_connections 2024-05-14T17:04:30.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.138+0000 7f45a5115700 1 --2- 172.21.15.2:0/2342750066 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45a0102e00 0x7f45a010f2f0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:30.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.138+0000 7f45a5115700 1 --2- 172.21.15.2:0/2342750066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45a0108520 0x7f45a0108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:30.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.138+0000 7f45a5115700 1 --2- 172.21.15.2:0/2342750066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45a0102440 0x7f45a01028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:30.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.138+0000 7f45a5115700 1 -- 172.21.15.2:0/2342750066 >> 172.21.15.2:0/2342750066 conn(0x7f45a00793f0 msgr2=0x7f45a0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:30.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.138+0000 7f45a5115700 1 -- 172.21.15.2:0/2342750066 shutdown_connections 2024-05-14T17:04:30.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.138+0000 7f45a5115700 1 -- 172.21.15.2:0/2342750066 wait complete. 2024-05-14T17:04:30.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.139+0000 7f45a5115700 1 Processor -- start 2024-05-14T17:04:30.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.139+0000 7f45a5115700 1 -- start start 2024-05-14T17:04:30.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.140+0000 7f45a5115700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45a0102440 0x7f45a019be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:30.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.140+0000 7f45a5115700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45a0102e00 0x7f45a019c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:30.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.140+0000 7f45a5115700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45a0108520 0x7f45a01a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:30.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.140+0000 7f45a5115700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f45a01125f0 con 0x7f45a0102440 2024-05-14T17:04:30.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.140+0000 7f45a5115700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f45a0112470 con 0x7f45a0108520 2024-05-14T17:04:30.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.140+0000 7f45a5115700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f45a0112770 con 0x7f45a0102e00 2024-05-14T17:04:30.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.140+0000 7f459e59c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45a0102e00 0x7f45a019c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:30.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.140+0000 7f459ed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45a0102440 0x7f45a019be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:30.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.140+0000 7f459e59c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45a0102e00 0x7f45a019c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:35122/0 (socket says 172.21.15.2:35122) 2024-05-14T17:04:30.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.140+0000 7f459e59c700 1 -- 172.21.15.2:0/1169224616 learned_addr learned my addr 172.21.15.2:0/1169224616 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:04:30.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.140+0000 7f459f59e700 1 --2- 172.21.15.2:0/1169224616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45a0108520 0x7f45a01a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:30.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.140+0000 7f459e59c700 1 -- 172.21.15.2:0/1169224616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45a0108520 msgr2=0x7f45a01a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:30.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.140+0000 7f459e59c700 1 --2- 172.21.15.2:0/1169224616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45a0108520 0x7f45a01a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:30.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.140+0000 7f459e59c700 1 -- 172.21.15.2:0/1169224616 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45a0102440 msgr2=0x7f45a019be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:30.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.140+0000 7f459e59c700 1 --2- 172.21.15.2:0/1169224616 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45a0102440 0x7f45a019be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:30.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.140+0000 7f459e59c700 1 -- 172.21.15.2:0/1169224616 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a01a0e40 con 0x7f45a0102e00 2024-05-14T17:04:30.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.141+0000 7f459e59c700 1 --2- 172.21.15.2:0/1169224616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45a0102e00 0x7f45a019c370 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f459400d950 tx=0x7f459400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:30.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.141+0000 7f4587fff700 1 -- 172.21.15.2:0/1169224616 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4594016070 con 0x7f45a0102e00 2024-05-14T17:04:30.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.141+0000 7f45a5115700 1 -- 172.21.15.2:0/1169224616 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45a01a1130 con 0x7f45a0102e00 2024-05-14T17:04:30.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.141+0000 7f4587fff700 1 -- 172.21.15.2:0/1169224616 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f459400fd40 con 0x7f45a0102e00 2024-05-14T17:04:30.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.141+0000 7f4587fff700 1 -- 172.21.15.2:0/1169224616 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459400e3b0 con 0x7f45a0102e00 2024-05-14T17:04:30.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.141+0000 7f45a5115700 1 -- 172.21.15.2:0/1169224616 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45a01a89b0 con 0x7f45a0102e00 2024-05-14T17:04:30.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.144+0000 7f4587fff700 1 -- 172.21.15.2:0/1169224616 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f4594019070 con 0x7f45a0102e00 2024-05-14T17:04:30.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.144+0000 7f45af9abb80 1 Processor -- start 2024-05-14T17:04:30.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.144+0000 7f4587fff700 1 --2- 172.21.15.2:0/1169224616 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f45880b29f0 0x7f45880b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:30.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.144+0000 7f45af9abb80 1 -- start start 2024-05-14T17:04:30.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.144+0000 7f459f59e700 1 --2- 172.21.15.2:0/1169224616 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f45880b29f0 0x7f45880b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:30.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.145+0000 7f4587fff700 1 -- 172.21.15.2:0/1169224616 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f45940e20b0 con 0x7f45a0102e00 2024-05-14T17:04:30.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.145+0000 7f45af9abb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624f0d2ada0 0x5624f0d2b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:30.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.145+0000 7f45af9abb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624f0d2b6c0 0x5624f0d254b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:30.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.145+0000 7f45af9abb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624f0d25ad0 0x5624f0d25f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:30.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.145+0000 7f45af9abb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5624f0c83d90 con 0x5624f0d2ada0 2024-05-14T17:04:30.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.145+0000 7f45af9abb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5624f0c73930 con 0x5624f0d2b6c0 2024-05-14T17:04:30.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.145+0000 7f45af9abb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5624f0c77970 con 0x5624f0d25ad0 2024-05-14T17:04:30.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.145+0000 7f459ed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624f0d2ada0 0x5624f0d2b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:30.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.145+0000 7f459ed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624f0d2ada0 0x5624f0d2b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33800/0 (socket says 172.21.15.2:33800) 2024-05-14T17:04:30.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.145+0000 7f459ed9d700 1 -- 172.21.15.2:0/3025239216 learned_addr learned my addr 172.21.15.2:0/3025239216 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:04:30.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.145+0000 7f459f59e700 1 --2- 172.21.15.2:0/3025239216 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624f0d25ad0 0x5624f0d25f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:30.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.145+0000 7f459e59c700 1 --2- 172.21.15.2:0/3025239216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624f0d2b6c0 0x5624f0d254b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:30.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.145+0000 7f459f59e700 1 --2- 172.21.15.2:0/1169224616 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f45880b29f0 0x7f45880b4eb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4598009fd0 tx=0x7f4598009130 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:30.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.145+0000 7f459ed9d700 1 -- 172.21.15.2:0/3025239216 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624f0d25ad0 msgr2=0x5624f0d25f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:30.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.145+0000 7f459ed9d700 1 --2- 172.21.15.2:0/3025239216 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624f0d25ad0 0x5624f0d25f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:30.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.145+0000 7f459ed9d700 1 -- 172.21.15.2:0/3025239216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624f0d2b6c0 msgr2=0x5624f0d254b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:31.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.145+0000 7f459ed9d700 1 --2- 172.21.15.2:0/3025239216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624f0d2b6c0 0x5624f0d254b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:31.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.145+0000 7f459ed9d700 1 -- 172.21.15.2:0/3025239216 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624f0bc39d0 con 0x5624f0d2ada0 2024-05-14T17:04:31.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.145+0000 7f459e59c700 1 --2- 172.21.15.2:0/3025239216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624f0d2b6c0 0x5624f0d254b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:04:31.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.145+0000 7f459ed9d700 1 --2- 172.21.15.2:0/3025239216 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624f0d2ada0 0x5624f0d2b180 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f45900046d0 tx=0x7f459000b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4545da8ec5096658 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:31.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.145+0000 7f45857fa700 1 -- 172.21.15.2:0/3025239216 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4590004a30 con 0x5624f0d2ada0 2024-05-14T17:04:31.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.146+0000 7f45857fa700 1 -- 172.21.15.2:0/3025239216 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4590015070 con 0x5624f0d2ada0 2024-05-14T17:04:31.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.146+0000 7f45857fa700 1 -- 172.21.15.2:0/3025239216 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4590003c70 con 0x5624f0d2ada0 2024-05-14T17:04:31.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.146+0000 7f45af9abb80 1 -- 172.21.15.2:0/3025239216 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624f0d2ada0 msgr2=0x5624f0d2b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:31.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.146+0000 7f45af9abb80 1 --2- 172.21.15.2:0/3025239216 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624f0d2ada0 0x5624f0d2b180 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f45900046d0 tx=0x7f459000b9e0 comp rx=0 tx=0).stop 2024-05-14T17:04:31.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.146+0000 7f45af9abb80 1 -- 172.21.15.2:0/3025239216 shutdown_connections 2024-05-14T17:04:31.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.146+0000 7f45af9abb80 1 --2- 172.21.15.2:0/3025239216 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624f0d25ad0 0x5624f0d25f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:31.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.146+0000 7f45af9abb80 1 --2- 172.21.15.2:0/3025239216 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624f0d2ada0 0x5624f0d2b180 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:31.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.146+0000 7f45af9abb80 1 --2- 172.21.15.2:0/3025239216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624f0d2b6c0 0x5624f0d254b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:31.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.146+0000 7f45af9abb80 1 -- 172.21.15.2:0/3025239216 >> 172.21.15.2:0/3025239216 conn(0x5624f0c50350 msgr2=0x5624f0d1fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:31.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.146+0000 7f45af9abb80 1 -- 172.21.15.2:0/3025239216 shutdown_connections 2024-05-14T17:04:31.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.146+0000 7f45af9abb80 1 -- 172.21.15.2:0/3025239216 wait complete. 2024-05-14T17:04:31.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.147+0000 7f45af9abb80 1 -- start start 2024-05-14T17:04:31.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.147+0000 7f45af9abb80 1 Processor -- start 2024-05-14T17:04:31.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.147+0000 7f45af9abb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624f0d25ad0 0x5624f0d313d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:31.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.147+0000 7f45af9abb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624f0d2ada0 0x5624f0d2e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:31.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.147+0000 7f45af9abb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624f0d2b6c0 0x5624f0d2ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:31.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.147+0000 7f45af9abb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5624f0b7a0a0 con 0x5624f0d2ada0 2024-05-14T17:04:31.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.147+0000 7f45af9abb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5624f0c158c0 con 0x5624f0d2b6c0 2024-05-14T17:04:31.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.147+0000 7f45af9abb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5624f0d32100 con 0x5624f0d25ad0 2024-05-14T17:04:31.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.147+0000 7f45af9abb80 10 client.0 _ll_register_callbacks cb 0x5624f0c0a680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:04:31.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.147+0000 7f459e59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624f0d2ada0 0x5624f0d2e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:31.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.147+0000 7f459e59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624f0d2ada0 0x5624f0d2e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33806/0 (socket says 172.21.15.2:33806) 2024-05-14T17:04:31.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.147+0000 7f459e59c700 1 -- 172.21.15.2:0/344747057 learned_addr learned my addr 172.21.15.2:0/344747057 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:04:31.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.148+0000 7f459f59e700 1 --2- 172.21.15.2:0/344747057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624f0d2b6c0 0x5624f0d2ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:31.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.148+0000 7f459ed9d700 1 --2- 172.21.15.2:0/344747057 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624f0d25ad0 0x5624f0d313d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:31.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.148+0000 7f459e59c700 1 -- 172.21.15.2:0/344747057 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624f0d25ad0 msgr2=0x5624f0d313d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:31.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.148+0000 7f459e59c700 1 --2- 172.21.15.2:0/344747057 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624f0d25ad0 0x5624f0d313d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:31.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.148+0000 7f459e59c700 1 -- 172.21.15.2:0/344747057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624f0d2b6c0 msgr2=0x5624f0d2ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:31.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.148+0000 7f459e59c700 1 --2- 172.21.15.2:0/344747057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624f0d2b6c0 0x5624f0d2ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:31.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.148+0000 7f459e59c700 1 -- 172.21.15.2:0/344747057 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624f0c12790 con 0x5624f0d2ada0 2024-05-14T17:04:31.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.148+0000 7f459f59e700 1 --2- 172.21.15.2:0/344747057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624f0d2b6c0 0x5624f0d2ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:04:31.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.148+0000 7f459ed9d700 1 --2- 172.21.15.2:0/344747057 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624f0d25ad0 0x5624f0d313d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:04:31.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.148+0000 7f459e59c700 1 --2- 172.21.15.2:0/344747057 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624f0d2ada0 0x5624f0d2e770 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f45940df860 tx=0x7f45940a9390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed1a9326a2ff01f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:31.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.148+0000 7f4582ff5700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:04:31.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.148+0000 7f4582ff5700 1 -- 172.21.15.2:0/344747057 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4594020040 con 0x5624f0d2ada0 2024-05-14T17:04:31.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.148+0000 7f4582ff5700 1 -- 172.21.15.2:0/344747057 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f45940a95f0 con 0x5624f0d2ada0 2024-05-14T17:04:31.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.148+0000 7f45af9abb80 1 -- 172.21.15.2:0/344747057 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4590015070 con 0x5624f0d2ada0 2024-05-14T17:04:31.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.148+0000 7f4582ff5700 1 -- 172.21.15.2:0/344747057 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f45940a97f0 con 0x5624f0d2ada0 2024-05-14T17:04:31.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.148+0000 7f45af9abb80 10 client.16389 fetch_fsmap learned FSMap version 42 2024-05-14T17:04:31.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.148+0000 7f45af9abb80 1 -- 172.21.15.2:0/344747057 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5624f0d3c780 con 0x5624f0d2ada0 2024-05-14T17:04:31.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.148+0000 7f4582ff5700 1 -- 172.21.15.2:0/344747057 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4590015070 con 0x5624f0d2ada0 2024-05-14T17:04:31.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.148+0000 7f4582ff5700 1 -- 172.21.15.2:0/344747057 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 42) ==== 8342+0+0 (secure 0 0 0) 0x7f45940a8780 con 0x5624f0d2ada0 2024-05-14T17:04:31.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.149+0000 7f45af9abb80 10 client.16389 fetch_fsmap finished waiting for FSMap version 42 2024-05-14T17:04:31.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.149+0000 7f45af9abb80 10 client.16389 resolve_mds: validated gid 24479 aka daemon mds.b 2024-05-14T17:04:31.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.149+0000 7f45af9abb80 20 client.16389 populate_metadata read hostname 'smithi002' 2024-05-14T17:04:31.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.149+0000 7f45af9abb80 1 --2- 172.21.15.2:0/344747057 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5624f0d3ed10 0x5624f0d41100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:31.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.149+0000 7f45af9abb80 4 client.16389 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:04:31.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.149+0000 7f45af9abb80 1 -- 172.21.15.2:0/344747057 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624f0cf1460 con 0x5624f0d3ed10 2024-05-14T17:04:31.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.149+0000 7f459ed9d700 1 --2- 172.21.15.2:0/344747057 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5624f0d3ed10 0x5624f0d41100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:31.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.149+0000 7f459ed9d700 1 --2- 172.21.15.2:0/344747057 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5624f0d3ed10 0x5624f0d41100 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48b2a5d06e7ef0f0 server_cookie=b0384a51be4ff201 in_seq=0 out_seq=0 2024-05-14T17:04:31.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.149+0000 7f4582ff5700 10 client.16389 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:04:31.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.150+0000 7f4582ff5700 1 -- 172.21.15.2:0/344747057 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5624f0cf1460 con 0x5624f0d3ed10 2024-05-14T17:04:32.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.150+0000 7f4582ff5700 10 client.16389 handle_command_reply: tid=0 2024-05-14T17:04:32.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.150+0000 7f45af9abb80 1 client.16389 shutdown 2024-05-14T17:04:32.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.150+0000 7f45af9abb80 1 -- 172.21.15.2:0/344747057 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624f0d2ada0 msgr2=0x5624f0d2e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:32.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.150+0000 7f45af9abb80 1 --2- 172.21.15.2:0/344747057 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624f0d2ada0 0x5624f0d2e770 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f45940df860 tx=0x7f45940a9390 comp rx=0 tx=0).stop 2024-05-14T17:04:32.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.150+0000 7f45af9abb80 1 -- 172.21.15.2:0/344747057 shutdown_connections 2024-05-14T17:04:32.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.150+0000 7f45af9abb80 1 --2- 172.21.15.2:0/344747057 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5624f0d3ed10 0x5624f0d41100 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:32.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.150+0000 7f45af9abb80 1 --2- 172.21.15.2:0/344747057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624f0d2b6c0 0x5624f0d2ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:32.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.150+0000 7f45af9abb80 1 --2- 172.21.15.2:0/344747057 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624f0d25ad0 0x5624f0d313d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:32.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.150+0000 7f4582ff5700 0 client.16389 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:04:32.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.150+0000 7f45af9abb80 1 --2- 172.21.15.2:0/344747057 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624f0d2ada0 0x5624f0d2e770 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:32.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.150+0000 7f45af9abb80 1 -- 172.21.15.2:0/344747057 >> 172.21.15.2:0/344747057 conn(0x5624f0c50350 msgr2=0x5624f0d1a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:32.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.150+0000 7f45af9abb80 1 -- 172.21.15.2:0/344747057 shutdown_connections 2024-05-14T17:04:32.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.151+0000 7f45af9abb80 1 -- 172.21.15.2:0/344747057 wait complete. 2024-05-14T17:04:32.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.151+0000 7f45af9abb80 20 client.16389 trim_cache size 0 max 16384 2024-05-14T17:04:32.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.171+0000 7f45af9abb80 1 Processor -- start 2024-05-14T17:04:32.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.171+0000 7f45af9abb80 1 -- start start 2024-05-14T17:04:32.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.171+0000 7f45af9abb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624f0d25ad0 0x5624f0d2d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:32.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.171+0000 7f45af9abb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624f0d2ada0 0x5624f0d3b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:32.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.171+0000 7f45af9abb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624f0d2b6c0 0x5624f0d2c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:32.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.171+0000 7f45af9abb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f45a01b28c0 con 0x5624f0d2ada0 2024-05-14T17:04:32.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.171+0000 7f45af9abb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f45a01b0150 con 0x5624f0d2b6c0 2024-05-14T17:04:32.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.171+0000 7f45af9abb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f45a01adfc0 con 0x5624f0d25ad0 2024-05-14T17:04:32.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.171+0000 7f459e59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624f0d2ada0 0x5624f0d3b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:32.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.171+0000 7f459e59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624f0d2ada0 0x5624f0d3b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33816/0 (socket says 172.21.15.2:33816) 2024-05-14T17:04:32.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.171+0000 7f459e59c700 1 -- 172.21.15.2:0/3502757142 learned_addr learned my addr 172.21.15.2:0/3502757142 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:04:32.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.171+0000 7f459f59e700 1 --2- 172.21.15.2:0/3502757142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624f0d2b6c0 0x5624f0d2c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:32.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.171+0000 7f459ed9d700 1 --2- 172.21.15.2:0/3502757142 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624f0d25ad0 0x5624f0d2d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:32.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.171+0000 7f459e59c700 1 -- 172.21.15.2:0/3502757142 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624f0d25ad0 msgr2=0x5624f0d2d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:32.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.171+0000 7f459e59c700 1 --2- 172.21.15.2:0/3502757142 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624f0d25ad0 0x5624f0d2d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:32.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.171+0000 7f459e59c700 1 -- 172.21.15.2:0/3502757142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624f0d2b6c0 msgr2=0x5624f0d2c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:32.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.171+0000 7f459e59c700 1 --2- 172.21.15.2:0/3502757142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624f0d2b6c0 0x5624f0d2c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:32.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.171+0000 7f459e59c700 1 -- 172.21.15.2:0/3502757142 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624f0d31740 con 0x5624f0d2ada0 2024-05-14T17:04:32.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.171+0000 7f459ed9d700 1 --2- 172.21.15.2:0/3502757142 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624f0d25ad0 0x5624f0d2d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:04:32.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.171+0000 7f459f59e700 1 --2- 172.21.15.2:0/3502757142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624f0d2b6c0 0x5624f0d2c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:04:32.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.172+0000 7f459e59c700 1 --2- 172.21.15.2:0/3502757142 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624f0d2ada0 0x5624f0d3b010 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f459400dfb0 tx=0x7f45940a8e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1508f5142a006b02 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:32.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.172+0000 7f4584ff9700 1 -- 172.21.15.2:0/3502757142 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4594020070 con 0x5624f0d2ada0 2024-05-14T17:04:32.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.172+0000 7f4584ff9700 1 -- 172.21.15.2:0/3502757142 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45940b1d60 con 0x5624f0d2ada0 2024-05-14T17:04:32.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.172+0000 7f4584ff9700 1 -- 172.21.15.2:0/3502757142 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45940a8830 con 0x5624f0d2ada0 2024-05-14T17:04:32.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.172+0000 7f45af9abb80 1 -- 172.21.15.2:0/3502757142 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624f0d2ada0 msgr2=0x5624f0d3b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:32.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.172+0000 7f45af9abb80 1 --2- 172.21.15.2:0/3502757142 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624f0d2ada0 0x5624f0d3b010 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f459400dfb0 tx=0x7f45940a8e10 comp rx=0 tx=0).stop 2024-05-14T17:04:32.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.172+0000 7f45af9abb80 1 -- 172.21.15.2:0/3502757142 shutdown_connections 2024-05-14T17:04:32.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.172+0000 7f45af9abb80 1 --2- 172.21.15.2:0/3502757142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624f0d2b6c0 0x5624f0d2c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:32.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.172+0000 7f45af9abb80 1 --2- 172.21.15.2:0/3502757142 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624f0d25ad0 0x5624f0d2d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:32.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.172+0000 7f45af9abb80 1 --2- 172.21.15.2:0/3502757142 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624f0d2ada0 0x5624f0d3b010 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:32.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.172+0000 7f45af9abb80 1 -- 172.21.15.2:0/3502757142 >> 172.21.15.2:0/3502757142 conn(0x5624f0c50350 msgr2=0x5624f0d29960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:32.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.172+0000 7f45af9abb80 1 -- 172.21.15.2:0/3502757142 shutdown_connections 2024-05-14T17:04:32.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.172+0000 7f45af9abb80 1 -- 172.21.15.2:0/3502757142 wait complete. 2024-05-14T17:04:32.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.173+0000 7f45af9abb80 1 -- start start 2024-05-14T17:04:32.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.173+0000 7f45af9abb80 1 Processor -- start 2024-05-14T17:04:32.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.173+0000 7f45af9abb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624f0d25ad0 0x5624f0d37570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:32.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.173+0000 7f45af9abb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624f0d2ada0 0x5624f0d37ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:32.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.173+0000 7f45af9abb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624f0d2b6c0 0x5624f0d343c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:32.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.173+0000 7f45af9abb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f45a01ae610 con 0x5624f0d2b6c0 2024-05-14T17:04:32.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.173+0000 7f45af9abb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f45a01afa80 con 0x5624f0d25ad0 2024-05-14T17:04:32.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.173+0000 7f45af9abb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f45a01aece0 con 0x5624f0d2ada0 2024-05-14T17:04:32.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.173+0000 7f45af9abb80 10 client.0 _ll_register_callbacks cb 0x5624f0d21eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:04:32.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.173+0000 7f459f59e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624f0d2b6c0 0x5624f0d343c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:32.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.173+0000 7f459f59e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624f0d2b6c0 0x5624f0d343c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33826/0 (socket says 172.21.15.2:33826) 2024-05-14T17:04:32.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.173+0000 7f459f59e700 1 -- 172.21.15.2:0/3355714858 learned_addr learned my addr 172.21.15.2:0/3355714858 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:04:32.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.173+0000 7f459ed9d700 1 --2- 172.21.15.2:0/3355714858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624f0d25ad0 0x5624f0d37570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:32.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.173+0000 7f459e59c700 1 --2- 172.21.15.2:0/3355714858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624f0d2ada0 0x5624f0d37ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:32.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.173+0000 7f459f59e700 1 -- 172.21.15.2:0/3355714858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624f0d2ada0 msgr2=0x5624f0d37ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:32.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.174+0000 7f459f59e700 1 --2- 172.21.15.2:0/3355714858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624f0d2ada0 0x5624f0d37ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:32.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.174+0000 7f459f59e700 1 -- 172.21.15.2:0/3355714858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624f0d25ad0 msgr2=0x5624f0d37570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:32.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.174+0000 7f459f59e700 1 --2- 172.21.15.2:0/3355714858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624f0d25ad0 0x5624f0d37570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:33.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.174+0000 7f459f59e700 1 -- 172.21.15.2:0/3355714858 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624f0d34c00 con 0x5624f0d2b6c0 2024-05-14T17:04:33.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.174+0000 7f459f59e700 1 --2- 172.21.15.2:0/3355714858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624f0d2b6c0 0x5624f0d343c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4598015000 tx=0x7f459800b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5e169d44c57eba server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:33.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.174+0000 7f4582ff5700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:04:33.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.174+0000 7f4582ff5700 1 -- 172.21.15.2:0/3355714858 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4598008e60 con 0x5624f0d2b6c0 2024-05-14T17:04:33.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.174+0000 7f45af9abb80 1 -- 172.21.15.2:0/3355714858 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45940b1d60 con 0x5624f0d2b6c0 2024-05-14T17:04:33.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.174+0000 7f4582ff5700 1 -- 172.21.15.2:0/3355714858 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f4598033470 con 0x5624f0d2b6c0 2024-05-14T17:04:33.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.174+0000 7f4582ff5700 1 -- 172.21.15.2:0/3355714858 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f4598033670 con 0x5624f0d2b6c0 2024-05-14T17:04:33.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.174+0000 7f45af9abb80 10 client.16395 fetch_fsmap learned FSMap version 42 2024-05-14T17:04:33.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.174+0000 7f45af9abb80 1 -- 172.21.15.2:0/3355714858 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5624f0daa090 con 0x5624f0d2b6c0 2024-05-14T17:04:33.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.174+0000 7f4582ff5700 1 -- 172.21.15.2:0/3355714858 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45940b1d60 con 0x5624f0d2b6c0 2024-05-14T17:04:33.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.174+0000 7f4582ff5700 1 -- 172.21.15.2:0/3355714858 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 42) ==== 8342+0+0 (secure 0 0 0) 0x7f4598011070 con 0x5624f0d2b6c0 2024-05-14T17:04:33.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.174+0000 7f45af9abb80 10 client.16395 fetch_fsmap finished waiting for FSMap version 42 2024-05-14T17:04:33.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.174+0000 7f45af9abb80 10 client.16395 resolve_mds: validated gid 24479 aka daemon mds.b 2024-05-14T17:04:33.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.174+0000 7f45af9abb80 20 client.16395 populate_metadata read hostname 'smithi002' 2024-05-14T17:04:33.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.174+0000 7f45af9abb80 1 --2- 172.21.15.2:0/3355714858 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5624f0daa7b0 0x5624f0dacbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:33.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.175+0000 7f45af9abb80 4 client.16395 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6c170f95", "release": true, "await": true}] 2024-05-14T17:04:33.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.175+0000 7f45af9abb80 1 -- 172.21.15.2:0/3355714858 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6c170f95", "release": true, "await": true}) -- 0x5624f0c85ce0 con 0x5624f0daa7b0 2024-05-14T17:04:33.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.175+0000 7f459ed9d700 1 --2- 172.21.15.2:0/3355714858 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5624f0daa7b0 0x5624f0dacbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:33.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.175+0000 7f459ed9d700 1 --2- 172.21.15.2:0/3355714858 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5624f0daa7b0 0x5624f0dacbd0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6bb8a0929db39d1 server_cookie=7f8b79893d86569 in_seq=0 out_seq=0 2024-05-14T17:04:33.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.175+0000 7f4582ff5700 10 client.16395 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:04:33.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.185+0000 7f4582ff5700 1 -- 172.21.15.2:0/3355714858 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5624f0c85ce0 con 0x5624f0daa7b0 2024-05-14T17:04:33.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.185+0000 7f4582ff5700 10 client.16395 handle_command_reply: tid=0 2024-05-14T17:04:33.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.185+0000 7f45af9abb80 1 client.16395 shutdown 2024-05-14T17:04:33.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.185+0000 7f45af9abb80 1 -- 172.21.15.2:0/3355714858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624f0d2b6c0 msgr2=0x5624f0d343c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:33.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.185+0000 7f45af9abb80 1 --2- 172.21.15.2:0/3355714858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624f0d2b6c0 0x5624f0d343c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4598015000 tx=0x7f459800b3b0 comp rx=0 tx=0).stop 2024-05-14T17:04:33.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.185+0000 7f45af9abb80 1 -- 172.21.15.2:0/3355714858 shutdown_connections 2024-05-14T17:04:33.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.185+0000 7f45af9abb80 1 --2- 172.21.15.2:0/3355714858 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5624f0daa7b0 0x5624f0dacbd0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:33.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.185+0000 7f45af9abb80 1 --2- 172.21.15.2:0/3355714858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5624f0d2b6c0 0x5624f0d343c0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:33.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.185+0000 7f4582ff5700 0 client.16395 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:04:33.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.185+0000 7f45af9abb80 1 --2- 172.21.15.2:0/3355714858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624f0d25ad0 0x5624f0d37570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:33.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.185+0000 7f45af9abb80 1 --2- 172.21.15.2:0/3355714858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5624f0d2ada0 0x5624f0d37ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:33.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.185+0000 7f45af9abb80 1 -- 172.21.15.2:0/3355714858 >> 172.21.15.2:0/3355714858 conn(0x5624f0c50350 msgr2=0x5624f0d2c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:33.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.185+0000 7f45af9abb80 1 -- 172.21.15.2:0/3355714858 shutdown_connections 2024-05-14T17:04:33.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.185+0000 7f45af9abb80 1 -- 172.21.15.2:0/3355714858 wait complete. 2024-05-14T17:04:33.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.186+0000 7f45af9abb80 20 client.16395 trim_cache size 0 max 16384 2024-05-14T17:04:33.450 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T17:04:33.493 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 42, 2024-05-14T17:04:33.502 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24479, 2024-05-14T17:04:33.510 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 32, 2024-05-14T17:04:33.518 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-05-14T17:04:33.526 INFO:teuthology.orchestra.run.smithi002.stdout: "6c170f95": { 2024-05-14T17:04:33.540 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 32, 2024-05-14T17:04:33.548 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-05-14T17:04:33.557 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-05-14T17:04:33.565 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-05-14T17:04:33.573 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-05-14T17:04:33.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:04:33.590 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-05-14T17:04:33.611 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 140.4, 2024-05-14T17:04:33.618 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-05-14T17:04:33.634 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-05-14T17:04:33.677 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-05-14T17:04:33.685 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-05-14T17:04:33.701 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RELEASED", 2024-05-14T17:04:33.728 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-05-14T17:04:33.736 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:04:33.744 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:04:33.752 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:04:33.759 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:04:33.777 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:04:33.786 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T17:04:33.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.186+0000 7f45a5115700 1 -- 172.21.15.2:0/1169224616 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f45880b29f0 msgr2=0x7f45880b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:33.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.186+0000 7f45a5115700 1 --2- 172.21.15.2:0/1169224616 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f45880b29f0 0x7f45880b4eb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4598009fd0 tx=0x7f4598009130 comp rx=0 tx=0).stop 2024-05-14T17:04:33.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.186+0000 7f45a5115700 1 -- 172.21.15.2:0/1169224616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45a0102e00 msgr2=0x7f45a019c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:33.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.186+0000 7f45a5115700 1 --2- 172.21.15.2:0/1169224616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45a0102e00 0x7f45a019c370 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f459400d950 tx=0x7f459400dc60 comp rx=0 tx=0).stop 2024-05-14T17:04:33.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.186+0000 7f45a5115700 1 -- 172.21.15.2:0/1169224616 shutdown_connections 2024-05-14T17:04:33.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.186+0000 7f45a5115700 1 --2- 172.21.15.2:0/1169224616 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f45880b29f0 0x7f45880b4eb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:33.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.186+0000 7f45a5115700 1 --2- 172.21.15.2:0/1169224616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45a0108520 0x7f45a01a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:33.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.186+0000 7f45a5115700 1 --2- 172.21.15.2:0/1169224616 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45a0102440 0x7f45a019be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:33.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.186+0000 7f45a5115700 1 --2- 172.21.15.2:0/1169224616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f45a0102e00 0x7f45a019c370 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:33.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.186+0000 7f45a5115700 1 -- 172.21.15.2:0/1169224616 >> 172.21.15.2:0/1169224616 conn(0x7f45a00793f0 msgr2=0x7f45a006b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:33.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.186+0000 7f45a5115700 1 -- 172.21.15.2:0/1169224616 shutdown_connections 2024-05-14T17:04:33.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:04:30.186+0000 7f45a5115700 1 -- 172.21.15.2:0/1169224616 wait complete. 2024-05-14T17:04:33.955 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:04:30 smithi002 ceph-mon[31538]: pgmap v305: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 60 KiB/s wr, 14 op/s 2024-05-14T17:04:33.965 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 555.0 s, 938.1 tps, lat 461.539 ms stddev 2907.182 2024-05-14T17:04:33.975 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:04:32 smithi002 ceph-mon[31538]: pgmap v306: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 52 KiB/s wr, 11 op/s 2024-05-14T17:04:33.995 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6c170f95', total seconds elapsed: 31.2 2024-05-14T17:04:34.005 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:04:30 smithi123 ceph-mon[35971]: pgmap v305: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 60 KiB/s wr, 14 op/s 2024-05-14T17:04:34.012 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:04:32 smithi123 ceph-mon[35971]: pgmap v306: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 52 KiB/s wr, 11 op/s 2024-05-14T17:04:34.021 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:04:30 smithi151 ceph-mon[36066]: pgmap v305: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 60 KiB/s wr, 14 op/s 2024-05-14T17:04:34.031 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:04:32 smithi151 ceph-mon[36066]: pgmap v306: 97 pgs: 97 active+clean; 4.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 52 KiB/s wr, 11 op/s 2024-05-14T17:04:34.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:04:34 smithi151 ceph-mon[36066]: pgmap v307: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 2.0 MiB/s wr, 120 op/s 2024-05-14T17:04:34.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:04:34 smithi123 ceph-mon[35971]: pgmap v307: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 2.0 MiB/s wr, 120 op/s 2024-05-14T17:04:34.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:04:34 smithi002 ceph-mon[31538]: pgmap v307: 97 pgs: 97 active+clean; 4.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 2.0 MiB/s wr, 120 op/s 2024-05-14T17:04:36.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 560.0 s, 2346.0 tps, lat 42.787 ms stddev 64.119 2024-05-14T17:04:36.794 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:04:36 smithi002 ceph-mon[31538]: pgmap v308: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 5.5 MiB/s wr, 404 op/s 2024-05-14T17:04:36.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:04:36 smithi151 ceph-mon[36066]: pgmap v308: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 5.5 MiB/s wr, 404 op/s 2024-05-14T17:04:36.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:04:36 smithi123 ceph-mon[35971]: pgmap v308: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 5.5 MiB/s wr, 404 op/s 2024-05-14T17:04:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.614+0000 7f0a50f51640 1 -- 172.21.15.2:0/1806394980 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a400308b0 con 0x7f0a4c109f90 2024-05-14T17:04:37.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.615+0000 7f0a541b7640 1 -- 172.21.15.2:0/1806394980 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a4c109f90 msgr2=0x7f0a4c10a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.615+0000 7f0a541b7640 1 --2- 172.21.15.2:0/1806394980 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a4c109f90 0x7f0a4c10a3f0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0a40009860 tx=0x7f0a4002a380 comp rx=0 tx=0).stop 2024-05-14T17:04:37.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.615+0000 7f0a541b7640 1 -- 172.21.15.2:0/1806394980 shutdown_connections 2024-05-14T17:04:37.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.615+0000 7f0a541b7640 1 --2- 172.21.15.2:0/1806394980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a4c117ae0 0x7f0a4c119ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:37.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.615+0000 7f0a541b7640 1 --2- 172.21.15.2:0/1806394980 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a4c109f90 0x7f0a4c10a3f0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.615+0000 7f0a541b7640 1 --2- 172.21.15.2:0/1806394980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a4c10d150 0x7f0a4c109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.615+0000 7f0a541b7640 1 -- 172.21.15.2:0/1806394980 >> 172.21.15.2:0/1806394980 conn(0x7f0a4c06db20 msgr2=0x7f0a4c06df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.620+0000 7f0a541b7640 1 -- 172.21.15.2:0/1806394980 shutdown_connections 2024-05-14T17:04:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.622+0000 7f0a541b7640 1 -- 172.21.15.2:0/1806394980 wait complete. 2024-05-14T17:04:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.623+0000 7f0a541b7640 1 Processor -- start 2024-05-14T17:04:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.625+0000 7f0a541b7640 1 -- start start 2024-05-14T17:04:37.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.625+0000 7f0a541b7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a4c109f90 0x7f0a4c117410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:37.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.625+0000 7f0a541b7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a4c10d150 0x7f0a4c123540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.625+0000 7f0a541b7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a4c117ae0 0x7f0a4c123b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:37.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.625+0000 7f0a541b7640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4c10e6b0 con 0x7f0a4c109f90 2024-05-14T17:04:37.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.625+0000 7f0a541b7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4c10e530 con 0x7f0a4c10d150 2024-05-14T17:04:37.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.625+0000 7f0a541b7640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4c10e830 con 0x7f0a4c117ae0 2024-05-14T17:04:37.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.627+0000 7f0a51752640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a4c10d150 0x7f0a4c123540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.627+0000 7f0a51752640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a4c10d150 0x7f0a4c123540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:33822/0 (socket says 172.21.15.2:33822) 2024-05-14T17:04:37.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.627+0000 7f0a51752640 1 -- 172.21.15.2:0/779923027 learned_addr learned my addr 172.21.15.2:0/779923027 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:04:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.627+0000 7f0a52754640 1 --2- 172.21.15.2:0/779923027 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a4c117ae0 0x7f0a4c123b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.628+0000 7f0a51752640 1 -- 172.21.15.2:0/779923027 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a4c117ae0 msgr2=0x7f0a4c123b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.628+0000 7f0a51752640 1 --2- 172.21.15.2:0/779923027 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a4c117ae0 0x7f0a4c123b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.628+0000 7f0a51752640 1 -- 172.21.15.2:0/779923027 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a4c109f90 msgr2=0x7f0a4c117410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.628+0000 7f0a51752640 1 --2- 172.21.15.2:0/779923027 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a4c109f90 0x7f0a4c117410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.628+0000 7f0a51752640 1 -- 172.21.15.2:0/779923027 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a4c1bcc40 con 0x7f0a4c10d150 2024-05-14T17:04:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.632+0000 7f0a51752640 1 --2- 172.21.15.2:0/779923027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a4c10d150 0x7f0a4c123540 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0a4002a8f0 tx=0x7f0a40032cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.634+0000 7f0a3effd640 1 -- 172.21.15.2:0/779923027 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a40037030 con 0x7f0a4c10d150 2024-05-14T17:04:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.634+0000 7f0a3effd640 1 -- 172.21.15.2:0/779923027 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a40041070 con 0x7f0a4c10d150 2024-05-14T17:04:38.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.634+0000 7f0a3effd640 1 -- 172.21.15.2:0/779923027 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a40005580 con 0x7f0a4c10d150 2024-05-14T17:04:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.635+0000 7f0a541b7640 1 -- 172.21.15.2:0/779923027 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a4c1bce10 con 0x7f0a4c10d150 2024-05-14T17:04:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.635+0000 7f0a541b7640 1 -- 172.21.15.2:0/779923027 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a4c1bd1c0 con 0x7f0a4c10d150 2024-05-14T17:04:38.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.642+0000 7f0a56fa8740 1 Processor -- start 2024-05-14T17:04:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.642+0000 7f0a56fa8740 1 -- start start 2024-05-14T17:04:38.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.643+0000 7f0a56fa8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f74e90e8f0 0x55f74e90ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.643+0000 7f0a56fa8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f74e90f210 0x55f74e919590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.643+0000 7f0a56fa8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f74e919ad0 0x55f74e91bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:38.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.643+0000 7f0a56fa8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f74e881c30 con 0x55f74e90f210 2024-05-14T17:04:38.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.643+0000 7f0a56fa8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f74e8717d0 con 0x55f74e90e8f0 2024-05-14T17:04:38.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.643+0000 7f0a56fa8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f74e875810 con 0x55f74e919ad0 2024-05-14T17:04:38.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.643+0000 7f0a52754640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f74e90e8f0 0x55f74e90ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.643+0000 7f0a52754640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f74e90e8f0 0x55f74e90ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:33828/0 (socket says 172.21.15.2:33828) 2024-05-14T17:04:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.643+0000 7f0a52754640 1 -- 172.21.15.2:0/1943651814 learned_addr learned my addr 172.21.15.2:0/1943651814 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:04:38.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.643+0000 7f0a52754640 1 -- 172.21.15.2:0/1943651814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f74e919ad0 msgr2=0x55f74e91bec0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T17:04:38.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.643+0000 7f0a52754640 1 --2- 172.21.15.2:0/1943651814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f74e919ad0 0x55f74e91bec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.643+0000 7f0a52754640 1 -- 172.21.15.2:0/1943651814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f74e90f210 msgr2=0x55f74e919590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T17:04:38.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.643+0000 7f0a52754640 1 --2- 172.21.15.2:0/1943651814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f74e90f210 0x55f74e919590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:38.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.643+0000 7f0a52754640 1 -- 172.21.15.2:0/1943651814 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f74e586210 con 0x55f74e90e8f0 2024-05-14T17:04:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.644+0000 7f0a52754640 1 --2- 172.21.15.2:0/1943651814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f74e90e8f0 0x55f74e90ecd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0a48009bc0 tx=0x7f0a48033820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74bdcd8d5f0e9365 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:38.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.644+0000 7f0a37fff640 1 -- 172.21.15.2:0/1943651814 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a48009e30 con 0x55f74e90e8f0 2024-05-14T17:04:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.644+0000 7f0a37fff640 1 -- 172.21.15.2:0/1943651814 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a480054d0 con 0x55f74e90e8f0 2024-05-14T17:04:38.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.645+0000 7f0a56fa8740 1 -- 172.21.15.2:0/1943651814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f74e90e8f0 msgr2=0x55f74e90ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:38.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.645+0000 7f0a56fa8740 1 --2- 172.21.15.2:0/1943651814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f74e90e8f0 0x55f74e90ecd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0a48009bc0 tx=0x7f0a48033820 comp rx=0 tx=0).stop 2024-05-14T17:04:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.645+0000 7f0a56fa8740 1 -- 172.21.15.2:0/1943651814 shutdown_connections 2024-05-14T17:04:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.645+0000 7f0a56fa8740 1 --2- 172.21.15.2:0/1943651814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f74e919ad0 0x55f74e91bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:38.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.645+0000 7f0a56fa8740 1 --2- 172.21.15.2:0/1943651814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f74e90f210 0x55f74e919590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.645+0000 7f0a56fa8740 1 --2- 172.21.15.2:0/1943651814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f74e90e8f0 0x55f74e90ecd0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.645+0000 7f0a56fa8740 1 -- 172.21.15.2:0/1943651814 >> 172.21.15.2:0/1943651814 conn(0x55f74e90ae00 msgr2=0x55f74e90b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:38.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.650+0000 7f0a3effd640 1 -- 172.21.15.2:0/779923027 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f0a40004050 con 0x7f0a4c10d150 2024-05-14T17:04:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.651+0000 7f0a3effd640 1 --2- 172.21.15.2:0/779923027 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f0a380b2ae0 0x7f0a380b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.651+0000 7f0a56fa8740 1 -- 172.21.15.2:0/1943651814 shutdown_connections 2024-05-14T17:04:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.652+0000 7f0a3effd640 1 -- 172.21.15.2:0/779923027 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f0a40101cc0 con 0x7f0a4c10d150 2024-05-14T17:04:38.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.652+0000 7f0a56fa8740 1 -- 172.21.15.2:0/1943651814 wait complete. 2024-05-14T17:04:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.652+0000 7f0a56fa8740 1 -- start start 2024-05-14T17:04:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.652+0000 7f0a56fa8740 1 Processor -- start 2024-05-14T17:04:38.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.653+0000 7f0a51f53640 1 --2- 172.21.15.2:0/779923027 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f0a380b2ae0 0x7f0a380b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:38.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.653+0000 7f0a56fa8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f74e90e8f0 0x55f74e919540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.653+0000 7f0a56fa8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f74e916830 0x55f74e916c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:38.633 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.653+0000 7f0a56fa8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f74e917290 0x55f74e925ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.653+0000 7f0a56fa8740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f74e557790 con 0x55f74e917290 2024-05-14T17:04:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.653+0000 7f0a56fa8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f74e82d890 con 0x55f74e90e8f0 2024-05-14T17:04:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.653+0000 7f0a56fa8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f74e91c7f0 con 0x55f74e916830 2024-05-14T17:04:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.653+0000 7f0a56fa8740 10 client.0 _ll_register_callbacks cb 0x55f74e830d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:04:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.654+0000 7f0a51f53640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f74e90e8f0 0x55f74e919540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.654+0000 7f0a51f53640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f74e90e8f0 0x55f74e919540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:33834/0 (socket says 172.21.15.2:33834) 2024-05-14T17:04:38.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.654+0000 7f0a51f53640 1 -- 172.21.15.2:0/4001177662 learned_addr learned my addr 172.21.15.2:0/4001177662 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:04:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.654+0000 7f0a51752640 1 --2- 172.21.15.2:0/4001177662 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f74e916830 0x55f74e916c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.655+0000 7f0a51f53640 1 -- 172.21.15.2:0/4001177662 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f74e916830 msgr2=0x55f74e916c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:38.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.655+0000 7f0a51f53640 1 --2- 172.21.15.2:0/4001177662 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f74e916830 0x55f74e916c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:38.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.655+0000 7f0a51f53640 1 -- 172.21.15.2:0/4001177662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f74e917290 msgr2=0x55f74e925ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:38.735 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.655+0000 7f0a51f53640 1 --2- 172.21.15.2:0/4001177662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f74e917290 0x55f74e925ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.656+0000 7f0a51f53640 1 -- 172.21.15.2:0/4001177662 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f74e652620 con 0x55f74e90e8f0 2024-05-14T17:04:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.656+0000 7f0a51f53640 1 --2- 172.21.15.2:0/779923027 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f0a380b2ae0 0x7f0a380b4fa0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0a440125e0 tx=0x7f0a440129e0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.656+0000 7f0a51f53640 1 --2- 172.21.15.2:0/4001177662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f74e90e8f0 0x55f74e919540 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0a44012ee0 tx=0x7f0a44018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5409ff18686ea362 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.656+0000 7f0a357fa640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T17:04:38.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.656+0000 7f0a357fa640 1 -- 172.21.15.2:0/4001177662 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a440120d0 con 0x55f74e90e8f0 2024-05-14T17:04:38.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.656+0000 7f0a357fa640 1 -- 172.21.15.2:0/4001177662 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f0a44009040 con 0x55f74e90e8f0 2024-05-14T17:04:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.656+0000 7f0a56fa8740 1 -- 172.21.15.2:0/4001177662 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a480054d0 con 0x55f74e90e8f0 2024-05-14T17:04:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.657+0000 7f0a357fa640 1 -- 172.21.15.2:0/4001177662 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f0a4402b050 con 0x55f74e90e8f0 2024-05-14T17:04:38.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.657+0000 7f0a56fa8740 10 client.26078 fetch_fsmap learned FSMap version 42 2024-05-14T17:04:38.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.657+0000 7f0a56fa8740 1 -- 172.21.15.2:0/4001177662 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f74e9289d0 con 0x55f74e90e8f0 2024-05-14T17:04:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.657+0000 7f0a357fa640 1 -- 172.21.15.2:0/4001177662 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a480054d0 con 0x55f74e90e8f0 2024-05-14T17:04:38.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.658+0000 7f0a357fa640 1 -- 172.21.15.2:0/4001177662 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 42) ==== 8342+0+0 (secure 0 0 0) 0x7f0a44026030 con 0x55f74e90e8f0 2024-05-14T17:04:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.658+0000 7f0a56fa8740 10 client.26078 fetch_fsmap finished waiting for FSMap version 42 2024-05-14T17:04:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.658+0000 7f0a56fa8740 10 client.26078 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:04:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.658+0000 7f0a56fa8740 20 client.26078 populate_metadata read hostname 'smithi002' 2024-05-14T17:04:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.658+0000 7f0a56fa8740 1 --2- 172.21.15.2:0/4001177662 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55f74e929f50 0x55f74e92c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.658+0000 7f0a56fa8740 4 client.26078 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:04:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.658+0000 7f0a56fa8740 1 -- 172.21.15.2:0/4001177662 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f74e883b80 con 0x55f74e929f50 2024-05-14T17:04:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.661+0000 7f0a51752640 1 --2- 172.21.15.2:0/4001177662 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55f74e929f50 0x55f74e92c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.661+0000 7f0a357fa640 10 client.26078 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:04:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.661+0000 7f0a51752640 1 --2- 172.21.15.2:0/4001177662 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55f74e929f50 0x55f74e92c360 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f06007b51fb7a205 server_cookie=819366b9e22eee93 in_seq=0 out_seq=0 2024-05-14T17:04:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.663+0000 7f0a357fa640 1 -- 172.21.15.2:0/4001177662 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f74e883b80 con 0x55f74e929f50 2024-05-14T17:04:39.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.663+0000 7f0a357fa640 10 client.26078 handle_command_reply: tid=0 2024-05-14T17:04:39.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.663+0000 7f0a56fa8740 1 client.26078 shutdown 2024-05-14T17:04:39.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.666+0000 7f0a56fa8740 1 -- 172.21.15.2:0/4001177662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f74e90e8f0 msgr2=0x55f74e919540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.666+0000 7f0a56fa8740 1 --2- 172.21.15.2:0/4001177662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f74e90e8f0 0x55f74e919540 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0a44012ee0 tx=0x7f0a44018000 comp rx=0 tx=0).stop 2024-05-14T17:04:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.666+0000 7f0a56fa8740 1 -- 172.21.15.2:0/4001177662 shutdown_connections 2024-05-14T17:04:39.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.666+0000 7f0a56fa8740 1 --2- 172.21.15.2:0/4001177662 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55f74e929f50 0x55f74e92c360 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.666+0000 7f0a56fa8740 1 --2- 172.21.15.2:0/4001177662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f74e917290 0x55f74e925ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:39.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.666+0000 7f0a56fa8740 1 --2- 172.21.15.2:0/4001177662 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f74e916830 0x55f74e916c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.666+0000 7f0a56fa8740 1 --2- 172.21.15.2:0/4001177662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f74e90e8f0 0x55f74e919540 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.666+0000 7f0a56fa8740 1 -- 172.21.15.2:0/4001177662 >> 172.21.15.2:0/4001177662 conn(0x55f74e90ae00 msgr2=0x55f74e90a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.667+0000 7f0a357fa640 0 client.26078 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:04:39.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.667+0000 7f0a56fa8740 1 -- 172.21.15.2:0/4001177662 shutdown_connections 2024-05-14T17:04:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.667+0000 7f0a56fa8740 1 -- 172.21.15.2:0/4001177662 wait complete. 2024-05-14T17:04:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.667+0000 7f0a56fa8740 20 client.26078 trim_cache size 0 max 16384 2024-05-14T17:04:39.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.700+0000 7f0a56fa8740 1 Processor -- start 2024-05-14T17:04:39.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.700+0000 7f0a56fa8740 1 -- start start 2024-05-14T17:04:39.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.701+0000 7f0a56fa8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f74e90e8f0 0x55f74e915d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:39.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.701+0000 7f0a56fa8740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f74e916830 0x55f74e919070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:39.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.701+0000 7f0a51f53640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f74e90e8f0 0x55f74e915d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.701+0000 7f0a51f53640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f74e90e8f0 0x55f74e915d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:33848/0 (socket says 172.21.15.2:33848) 2024-05-14T17:04:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.701+0000 7f0a51f53640 1 -- 172.21.15.2:0/2503627721 learned_addr learned my addr 172.21.15.2:0/2503627721 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:04:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.702+0000 7f0a56fa8740 1 --2- 172.21.15.2:0/2503627721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f74e917290 0x55f74e925780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.702+0000 7f0a56fa8740 1 -- 172.21.15.2:0/2503627721 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4c07ff00 con 0x55f74e916830 2024-05-14T17:04:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.702+0000 7f0a52754640 1 --2- 172.21.15.2:0/2503627721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f74e917290 0x55f74e925780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.702+0000 7f0a56fa8740 1 -- 172.21.15.2:0/2503627721 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4c07d810 con 0x55f74e90e8f0 2024-05-14T17:04:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.702+0000 7f0a56fa8740 1 -- 172.21.15.2:0/2503627721 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4c124a50 con 0x55f74e917290 2024-05-14T17:04:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.702+0000 7f0a51752640 1 --2- 172.21.15.2:0/2503627721 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f74e916830 0x55f74e919070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.703+0000 7f0a51f53640 1 -- 172.21.15.2:0/2503627721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f74e917290 msgr2=0x55f74e925780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.703+0000 7f0a51f53640 1 --2- 172.21.15.2:0/2503627721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f74e917290 0x55f74e925780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.703+0000 7f0a51f53640 1 -- 172.21.15.2:0/2503627721 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f74e916830 msgr2=0x55f74e919070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.703+0000 7f0a51f53640 1 --2- 172.21.15.2:0/2503627721 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f74e916830 0x55f74e919070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:39.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.703+0000 7f0a51f53640 1 -- 172.21.15.2:0/2503627721 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f74e90ac30 con 0x55f74e90e8f0 2024-05-14T17:04:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.703+0000 7f0a51f53640 1 --2- 172.21.15.2:0/2503627721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f74e90e8f0 0x55f74e915d70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0a44000f80 tx=0x7f0a440049e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e118d992c8ae88e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.703+0000 7f0a377fe640 1 -- 172.21.15.2:0/2503627721 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4402b050 con 0x55f74e90e8f0 2024-05-14T17:04:39.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.704+0000 7f0a377fe640 1 -- 172.21.15.2:0/2503627721 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a44009c50 con 0x55f74e90e8f0 2024-05-14T17:04:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.704+0000 7f0a377fe640 1 -- 172.21.15.2:0/2503627721 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a440120d0 con 0x55f74e90e8f0 2024-05-14T17:04:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.705+0000 7f0a56fa8740 1 -- 172.21.15.2:0/2503627721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f74e90e8f0 msgr2=0x55f74e915d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.705+0000 7f0a56fa8740 1 --2- 172.21.15.2:0/2503627721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f74e90e8f0 0x55f74e915d70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0a44000f80 tx=0x7f0a440049e0 comp rx=0 tx=0).stop 2024-05-14T17:04:39.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.705+0000 7f0a56fa8740 1 -- 172.21.15.2:0/2503627721 shutdown_connections 2024-05-14T17:04:39.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.705+0000 7f0a56fa8740 1 --2- 172.21.15.2:0/2503627721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f74e917290 0x55f74e925780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.705+0000 7f0a56fa8740 1 --2- 172.21.15.2:0/2503627721 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f74e916830 0x55f74e919070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:39.618 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.705+0000 7f0a56fa8740 1 --2- 172.21.15.2:0/2503627721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f74e90e8f0 0x55f74e915d70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:39.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.705+0000 7f0a56fa8740 1 -- 172.21.15.2:0/2503627721 >> 172.21.15.2:0/2503627721 conn(0x55f74e90ae00 msgr2=0x55f74e910c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.706+0000 7f0a56fa8740 1 -- 172.21.15.2:0/2503627721 shutdown_connections 2024-05-14T17:04:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.706+0000 7f0a56fa8740 1 -- 172.21.15.2:0/2503627721 wait complete. 2024-05-14T17:04:39.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.706+0000 7f0a56fa8740 1 -- start start 2024-05-14T17:04:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.707+0000 7f0a56fa8740 1 Processor -- start 2024-05-14T17:04:39.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.707+0000 7f0a56fa8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f74e90e8f0 0x55f74e921b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.708+0000 7f0a56fa8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f74e916830 0x55f74e922040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.708+0000 7f0a51f53640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f74e90e8f0 0x55f74e921b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.708+0000 7f0a51f53640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f74e90e8f0 0x55f74e921b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:58600/0 (socket says 172.21.15.2:58600) 2024-05-14T17:04:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.708+0000 7f0a51f53640 1 -- 172.21.15.2:0/1488070025 learned_addr learned my addr 172.21.15.2:0/1488070025 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:04:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.708+0000 7f0a56fa8740 1 --2- 172.21.15.2:0/1488070025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f74e917290 0x55f74e912d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:39.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.708+0000 7f0a51752640 1 --2- 172.21.15.2:0/1488070025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f74e916830 0x55f74e922040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.708+0000 7f0a56fa8740 1 -- 172.21.15.2:0/1488070025 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4c07bcd0 con 0x55f74e917290 2024-05-14T17:04:39.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.708+0000 7f0a56fa8740 1 -- 172.21.15.2:0/1488070025 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4c07d140 con 0x55f74e916830 2024-05-14T17:04:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.709+0000 7f0a56fa8740 1 -- 172.21.15.2:0/1488070025 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4c07c3a0 con 0x55f74e90e8f0 2024-05-14T17:04:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.709+0000 7f0a52754640 1 --2- 172.21.15.2:0/1488070025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f74e917290 0x55f74e912d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.709+0000 7f0a51752640 1 -- 172.21.15.2:0/1488070025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f74e90e8f0 msgr2=0x55f74e921b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:39.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.709+0000 7f0a51752640 1 --2- 172.21.15.2:0/1488070025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f74e90e8f0 0x55f74e921b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.709+0000 7f0a51752640 1 -- 172.21.15.2:0/1488070025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f74e917290 msgr2=0x55f74e912d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.709+0000 7f0a51752640 1 --2- 172.21.15.2:0/1488070025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f74e917290 0x55f74e912d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:39.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.709+0000 7f0a51752640 1 -- 172.21.15.2:0/1488070025 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f74e90c010 con 0x55f74e916830 2024-05-14T17:04:39.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.710+0000 7f0a51752640 1 --2- 172.21.15.2:0/1488070025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f74e916830 0x55f74e922040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0a400ff4c0 tx=0x7f0a40005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=765184bf6a801f77 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:39.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.710+0000 7f0a56fa8740 10 client.0 _ll_register_callbacks cb 0x7f0a4400e9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:04:39.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.710+0000 7f0a56fa8740 1 -- 172.21.15.2:0/1488070025 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a44009c50 con 0x55f74e916830 2024-05-14T17:04:40.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.710+0000 7f0a357fa640 10 client.26084 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T17:04:40.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.710+0000 7f0a357fa640 1 -- 172.21.15.2:0/1488070025 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a400e2070 con 0x55f74e916830 2024-05-14T17:04:40.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.710+0000 7f0a357fa640 1 -- 172.21.15.2:0/1488070025 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f0a400c5360 con 0x55f74e916830 2024-05-14T17:04:40.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.711+0000 7f0a357fa640 1 -- 172.21.15.2:0/1488070025 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f0a400c8be0 con 0x55f74e916830 2024-05-14T17:04:40.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.711+0000 7f0a56fa8740 10 client.26084 fetch_fsmap learned FSMap version 42 2024-05-14T17:04:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.711+0000 7f0a56fa8740 1 -- 172.21.15.2:0/1488070025 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=42}) -- 0x55f74e993ea0 con 0x55f74e916830 2024-05-14T17:04:40.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.711+0000 7f0a357fa640 1 -- 172.21.15.2:0/1488070025 <== mon.2 v2:172.21.15.123:3300/0 4 ==== fsmap(e 42) ==== 8342+0+0 (secure 0 0 0) 0x7f0a400ff630 con 0x55f74e916830 2024-05-14T17:04:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.711+0000 7f0a56fa8740 10 client.26084 fetch_fsmap finished waiting for FSMap version 42 2024-05-14T17:04:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.711+0000 7f0a56fa8740 10 client.26084 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:04:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.712+0000 7f0a56fa8740 20 client.26084 populate_metadata read hostname 'smithi002' 2024-05-14T17:04:40.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.712+0000 7f0a56fa8740 1 --2- 172.21.15.2:0/1488070025 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55f74e994500 0x55f74e9969a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.712+0000 7f0a56fa8740 4 client.26084 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:04:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.712+0000 7f0a56fa8740 1 -- 172.21.15.2:0/1488070025 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f0a44009c50 con 0x55f74e994500 2024-05-14T17:04:40.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.712+0000 7f0a51f53640 1 --2- 172.21.15.2:0/1488070025 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55f74e994500 0x55f74e9969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.713+0000 7f0a357fa640 10 client.26084 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:04:40.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.713+0000 7f0a51f53640 1 --2- 172.21.15.2:0/1488070025 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55f74e994500 0x55f74e9969a0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a24a9a8316b5536 server_cookie=91fc8ef5702b54c1 in_seq=0 out_seq=0 2024-05-14T17:04:40.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.727+0000 7f0a357fa640 1 -- 172.21.15.2:0/1488070025 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f0a44009c50 con 0x55f74e994500 2024-05-14T17:04:40.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.727+0000 7f0a357fa640 10 client.26084 handle_command_reply: tid=0 2024-05-14T17:04:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.727+0000 7f0a56fa8740 1 client.26084 shutdown 2024-05-14T17:04:40.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.728+0000 7f0a56fa8740 1 -- 172.21.15.2:0/1488070025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f74e916830 msgr2=0x55f74e922040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.728+0000 7f0a56fa8740 1 --2- 172.21.15.2:0/1488070025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f74e916830 0x55f74e922040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0a400ff4c0 tx=0x7f0a40005720 comp rx=0 tx=0).stop 2024-05-14T17:04:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.729+0000 7f0a56fa8740 1 -- 172.21.15.2:0/1488070025 shutdown_connections 2024-05-14T17:04:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.729+0000 7f0a56fa8740 1 --2- 172.21.15.2:0/1488070025 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55f74e994500 0x55f74e9969a0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.729+0000 7f0a56fa8740 1 --2- 172.21.15.2:0/1488070025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f74e917290 0x55f74e912d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.729+0000 7f0a357fa640 0 client.26084 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:04:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.729+0000 7f0a56fa8740 1 --2- 172.21.15.2:0/1488070025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f74e916830 0x55f74e922040 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.729+0000 7f0a56fa8740 1 --2- 172.21.15.2:0/1488070025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f74e90e8f0 0x55f74e921b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.729+0000 7f0a56fa8740 1 -- 172.21.15.2:0/1488070025 >> 172.21.15.2:0/1488070025 conn(0x55f74e90ae00 msgr2=0x55f74e911530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:40.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.729+0000 7f0a56fa8740 1 -- 172.21.15.2:0/1488070025 shutdown_connections 2024-05-14T17:04:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.729+0000 7f0a56fa8740 1 -- 172.21.15.2:0/1488070025 wait complete. 2024-05-14T17:04:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.729+0000 7f0a56fa8740 20 client.26084 trim_cache size 0 max 16384 2024-05-14T17:04:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:04:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:04:40.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:04:40.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.730+0000 7f0a541b7640 1 -- 172.21.15.2:0/779923027 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f0a380b2ae0 msgr2=0x7f0a380b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.730+0000 7f0a541b7640 1 --2- 172.21.15.2:0/779923027 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f0a380b2ae0 0x7f0a380b4fa0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0a440125e0 tx=0x7f0a440129e0 comp rx=0 tx=0).stop 2024-05-14T17:04:40.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.730+0000 7f0a541b7640 1 -- 172.21.15.2:0/779923027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a4c10d150 msgr2=0x7f0a4c123540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.730+0000 7f0a541b7640 1 --2- 172.21.15.2:0/779923027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a4c10d150 0x7f0a4c123540 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0a4002a8f0 tx=0x7f0a40032cd0 comp rx=0 tx=0).stop 2024-05-14T17:04:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.732+0000 7f0a541b7640 1 -- 172.21.15.2:0/779923027 shutdown_connections 2024-05-14T17:04:40.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.732+0000 7f0a541b7640 1 --2- 172.21.15.2:0/779923027 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f0a380b2ae0 0x7f0a380b4fa0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.732+0000 7f0a541b7640 1 --2- 172.21.15.2:0/779923027 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a4c117ae0 0x7f0a4c123b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:40.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.732+0000 7f0a541b7640 1 --2- 172.21.15.2:0/779923027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a4c10d150 0x7f0a4c123540 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.732+0000 7f0a541b7640 1 --2- 172.21.15.2:0/779923027 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a4c109f90 0x7f0a4c117410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:40.551 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.732+0000 7f0a541b7640 1 -- 172.21.15.2:0/779923027 >> 172.21.15.2:0/779923027 conn(0x7f0a4c06db20 msgr2=0x7f0a4c117ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.732+0000 7f0a541b7640 1 -- 172.21.15.2:0/779923027 shutdown_connections 2024-05-14T17:04:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:37.732+0000 7f0a541b7640 1 -- 172.21.15.2:0/779923027 wait complete. 2024-05-14T17:04:40.578 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:04:38 smithi002 ceph-mon[31538]: pgmap v309: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 5.5 MiB/s wr, 404 op/s 2024-05-14T17:04:40.599 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:04:38 smithi123 ceph-mon[35971]: pgmap v309: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 5.5 MiB/s wr, 404 op/s 2024-05-14T17:04:40.647 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:04:38 smithi151 ceph-mon[36066]: pgmap v309: 97 pgs: 97 active+clean; 4.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 5.5 MiB/s wr, 404 op/s 2024-05-14T17:04:40.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:04:40 smithi151 ceph-mon[36066]: pgmap v310: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 18 MiB/s wr, 1.42k op/s 2024-05-14T17:04:40.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:04:40 smithi123 ceph-mon[35971]: pgmap v310: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 18 MiB/s wr, 1.42k op/s 2024-05-14T17:04:40.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:04:40 smithi002 ceph-mon[31538]: pgmap v310: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 18 MiB/s wr, 1.42k op/s 2024-05-14T17:04:41.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 565.0 s, 2952.0 tps, lat 33.456 ms stddev 38.634 2024-05-14T17:04:42.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:04:42 smithi151 ceph-mon[36066]: pgmap v311: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 18 MiB/s wr, 1.42k op/s 2024-05-14T17:04:42.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:04:42 smithi123 ceph-mon[35971]: pgmap v311: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 18 MiB/s wr, 1.42k op/s 2024-05-14T17:04:42.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:04:42 smithi002 ceph-mon[31538]: pgmap v311: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 18 MiB/s wr, 1.42k op/s 2024-05-14T17:04:44.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:04:44 smithi151 ceph-mon[36066]: pgmap v312: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 22 MiB/s wr, 1.65k op/s 2024-05-14T17:04:44.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:04:44 smithi123 ceph-mon[35971]: pgmap v312: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 22 MiB/s wr, 1.65k op/s 2024-05-14T17:04:44.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:04:44 smithi002 ceph-mon[31538]: pgmap v312: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 22 MiB/s wr, 1.65k op/s 2024-05-14T17:04:46.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 570.0 s, 2723.2 tps, lat 36.803 ms stddev 40.215 2024-05-14T17:04:46.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:04:46 smithi151 ceph-mon[36066]: pgmap v313: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 21 MiB/s wr, 1.62k op/s 2024-05-14T17:04:46.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:04:46 smithi123 ceph-mon[35971]: pgmap v313: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 21 MiB/s wr, 1.62k op/s 2024-05-14T17:04:46.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:04:46 smithi002 ceph-mon[31538]: pgmap v313: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 21 MiB/s wr, 1.62k op/s 2024-05-14T17:04:48.808 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:04:48 smithi002 ceph-mon[31538]: pgmap v314: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 18 MiB/s wr, 1.33k op/s 2024-05-14T17:04:48.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:04:48 smithi151 ceph-mon[36066]: pgmap v314: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 18 MiB/s wr, 1.33k op/s 2024-05-14T17:04:48.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:04:48 smithi123 ceph-mon[35971]: pgmap v314: 97 pgs: 97 active+clean; 4.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 18 MiB/s wr, 1.33k op/s 2024-05-14T17:04:50.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:04:50 smithi151 ceph-mon[36066]: pgmap v315: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 21 MiB/s wr, 1.60k op/s 2024-05-14T17:04:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.884+0000 7fc6afa37640 1 -- 172.21.15.2:0/3063540475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6a810bd10 msgr2=0x7fc6a8114460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.884+0000 7fc6afa37640 1 --2- 172.21.15.2:0/3063540475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6a810bd10 0x7fc6a8114460 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc698009900 tx=0x7fc69802a480 comp rx=0 tx=0).stop 2024-05-14T17:04:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.885+0000 7fc6afa37640 1 -- 172.21.15.2:0/3063540475 shutdown_connections 2024-05-14T17:04:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.885+0000 7fc6afa37640 1 --2- 172.21.15.2:0/3063540475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6a81149a0 0x7fc6a8116d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.885+0000 7fc6afa37640 1 --2- 172.21.15.2:0/3063540475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6a810bd10 0x7fc6a8114460 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.885+0000 7fc6afa37640 1 --2- 172.21.15.2:0/3063540475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6a810b3f0 0x7fc6a810b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.885+0000 7fc6afa37640 1 -- 172.21.15.2:0/3063540475 >> 172.21.15.2:0/3063540475 conn(0x7fc6a806dfb0 msgr2=0x7fc6a806e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.885+0000 7fc6afa37640 1 -- 172.21.15.2:0/3063540475 shutdown_connections 2024-05-14T17:04:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.885+0000 7fc6afa37640 1 -- 172.21.15.2:0/3063540475 wait complete. 2024-05-14T17:04:50.949 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:04:50 smithi123 ceph-mon[35971]: pgmap v315: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 21 MiB/s wr, 1.60k op/s 2024-05-14T17:04:50.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.886+0000 7fc6afa37640 1 Processor -- start 2024-05-14T17:04:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.886+0000 7fc6afa37640 1 -- start start 2024-05-14T17:04:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.886+0000 7fc6afa37640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6a810b3f0 0x7fc6a81a6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.886+0000 7fc6afa37640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6a810bd10 0x7fc6a81a6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.886+0000 7fc6afa37640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6a81149a0 0x7fc6a81a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:51.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.886+0000 7fc6afa37640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a811a3f0 con 0x7fc6a81149a0 2024-05-14T17:04:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.886+0000 7fc6afa37640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a811a270 con 0x7fc6a810bd10 2024-05-14T17:04:51.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.886+0000 7fc6afa37640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a811a570 con 0x7fc6a810b3f0 2024-05-14T17:04:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.886+0000 7fc6adfd4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6a81149a0 0x7fc6a81a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.886+0000 7fc6adfd4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6a81149a0 0x7fc6a81a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41474/0 (socket says 172.21.15.2:41474) 2024-05-14T17:04:51.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.886+0000 7fc6adfd4640 1 -- 172.21.15.2:0/3489784546 learned_addr learned my addr 172.21.15.2:0/3489784546 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:04:51.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.886+0000 7fc6ad7d3640 1 --2- 172.21.15.2:0/3489784546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6a810b3f0 0x7fc6a81a6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.886+0000 7fc6acfd2640 1 --2- 172.21.15.2:0/3489784546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6a810bd10 0x7fc6a81a6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.887+0000 7fc6adfd4640 1 -- 172.21.15.2:0/3489784546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6a810b3f0 msgr2=0x7fc6a81a6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.887+0000 7fc6adfd4640 1 --2- 172.21.15.2:0/3489784546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6a810b3f0 0x7fc6a81a6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.887+0000 7fc6adfd4640 1 -- 172.21.15.2:0/3489784546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6a810bd10 msgr2=0x7fc6a81a6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:51.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.887+0000 7fc6adfd4640 1 --2- 172.21.15.2:0/3489784546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6a810bd10 0x7fc6a81a6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.887+0000 7fc6adfd4640 1 -- 172.21.15.2:0/3489784546 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6a81a0fb0 con 0x7fc6a81149a0 2024-05-14T17:04:51.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.887+0000 7fc6adfd4640 1 --2- 172.21.15.2:0/3489784546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6a81149a0 0x7fc6a81a06f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc6a000eb10 tx=0x7fc6a000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.887+0000 7fc69e7fc640 1 -- 172.21.15.2:0/3489784546 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0004b20 con 0x7fc6a81149a0 2024-05-14T17:04:51.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.887+0000 7fc69e7fc640 1 -- 172.21.15.2:0/3489784546 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6a0004cc0 con 0x7fc6a81149a0 2024-05-14T17:04:51.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.887+0000 7fc69e7fc640 1 -- 172.21.15.2:0/3489784546 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0004b20 con 0x7fc6a81149a0 2024-05-14T17:04:51.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.889+0000 7fc6afa37640 1 -- 172.21.15.2:0/3489784546 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6a81a1240 con 0x7fc6a81149a0 2024-05-14T17:04:51.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.889+0000 7fc6afa37640 1 -- 172.21.15.2:0/3489784546 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6a810a2e0 con 0x7fc6a81149a0 2024-05-14T17:04:51.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.893+0000 7fc6b282d740 1 Processor -- start 2024-05-14T17:04:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.895+0000 7fc69e7fc640 1 -- 172.21.15.2:0/3489784546 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fc6a0005000 con 0x7fc6a81149a0 2024-05-14T17:04:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.896+0000 7fc69e7fc640 1 --2- 172.21.15.2:0/3489784546 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc6940b4e10 0x7fc6940b72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.897+0000 7fc69e7fc640 1 -- 172.21.15.2:0/3489784546 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7fc6a00a6170 con 0x7fc6a81149a0 2024-05-14T17:04:51.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.897+0000 7fc6b282d740 1 -- start start 2024-05-14T17:04:51.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.898+0000 7fc6b282d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567762d0e30 0x5567762cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:51.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.898+0000 7fc6b282d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567762cfcf0 0x5567762d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.898+0000 7fc6b282d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567762e3170 0x5567762e5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:51.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.898+0000 7fc6b282d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x556776242c30 con 0x5567762e3170 2024-05-14T17:04:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.898+0000 7fc6b282d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5567762327d0 con 0x5567762cfcf0 2024-05-14T17:04:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.898+0000 7fc6b282d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556776236810 con 0x5567762d0e30 2024-05-14T17:04:51.416 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.899+0000 7fc6adfd4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567762e3170 0x5567762e5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:51.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.899+0000 7fc6adfd4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567762e3170 0x5567762e5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:41480/0 (socket says 172.21.15.2:41480) 2024-05-14T17:04:51.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.899+0000 7fc6adfd4640 1 -- 172.21.15.2:0/1154211418 learned_addr learned my addr 172.21.15.2:0/1154211418 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:04:51.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.900+0000 7fc6adfd4640 1 -- 172.21.15.2:0/1154211418 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567762d0e30 msgr2=0x5567762cf7b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T17:04:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.900+0000 7fc6adfd4640 1 --2- 172.21.15.2:0/1154211418 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567762d0e30 0x5567762cf7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.900+0000 7fc6adfd4640 1 -- 172.21.15.2:0/1154211418 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567762cfcf0 msgr2=0x5567762d0170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T17:04:51.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.900+0000 7fc6adfd4640 1 --2- 172.21.15.2:0/1154211418 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567762cfcf0 0x5567762d0170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.900+0000 7fc6adfd4640 1 -- 172.21.15.2:0/1154211418 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556775f47210 con 0x5567762e3170 2024-05-14T17:04:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.900+0000 7fc6adfd4640 1 --2- 172.21.15.2:0/1154211418 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567762e3170 0x5567762e5560 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc6a000eff0 tx=0x7fc6a00af050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe3bae3c07a280d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:51.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.900+0000 7fc6acfd2640 1 --2- 172.21.15.2:0/3489784546 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc6940b4e10 0x7fc6940b72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.901+0000 7fc6937fe640 1 -- 172.21.15.2:0/1154211418 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a00a6370 con 0x5567762e3170 2024-05-14T17:04:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.901+0000 7fc6937fe640 1 -- 172.21.15.2:0/1154211418 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6a001ce50 con 0x5567762e3170 2024-05-14T17:04:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.901+0000 7fc6937fe640 1 -- 172.21.15.2:0/1154211418 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a00a6510 con 0x5567762e3170 2024-05-14T17:04:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.901+0000 7fc6acfd2640 1 --2- 172.21.15.2:0/3489784546 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc6940b4e10 0x7fc6940b72d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc698005ab0 tx=0x7fc698005a20 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.903+0000 7fc6b282d740 1 -- 172.21.15.2:0/1154211418 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567762e3170 msgr2=0x5567762e5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:51.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.903+0000 7fc6b282d740 1 --2- 172.21.15.2:0/1154211418 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567762e3170 0x5567762e5560 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc6a000eff0 tx=0x7fc6a00af050 comp rx=0 tx=0).stop 2024-05-14T17:04:51.634 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.909+0000 7fc6b282d740 1 -- 172.21.15.2:0/1154211418 shutdown_connections 2024-05-14T17:04:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.909+0000 7fc6b282d740 1 --2- 172.21.15.2:0/1154211418 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567762e3170 0x5567762e5560 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:51.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.909+0000 7fc6b282d740 1 --2- 172.21.15.2:0/1154211418 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567762cfcf0 0x5567762d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:51.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.909+0000 7fc6b282d740 1 --2- 172.21.15.2:0/1154211418 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567762d0e30 0x5567762cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:51.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.909+0000 7fc6b282d740 1 -- 172.21.15.2:0/1154211418 >> 172.21.15.2:0/1154211418 conn(0x5567762cbe00 msgr2=0x5567762cc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.918+0000 7fc6b282d740 1 -- 172.21.15.2:0/1154211418 shutdown_connections 2024-05-14T17:04:51.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.922+0000 7fc6b282d740 1 -- 172.21.15.2:0/1154211418 wait complete. 2024-05-14T17:04:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.923+0000 7fc6b282d740 1 -- start start 2024-05-14T17:04:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.923+0000 7fc6b282d740 1 Processor -- start 2024-05-14T17:04:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.925+0000 7fc6b282d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567762cfcf0 0x5567762dcac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.925+0000 7fc6b282d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567762d0e30 0x5567762dd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:51.788 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.925+0000 7fc6b282d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567762e3170 0x5567762e27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:51.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.925+0000 7fc6b282d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x556775f18790 con 0x5567762d0e30 2024-05-14T17:04:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.925+0000 7fc6b282d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5567761ee890 con 0x5567762e3170 2024-05-14T17:04:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.925+0000 7fc6b282d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5567762d6d80 con 0x5567762cfcf0 2024-05-14T17:04:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.925+0000 7fc6b282d740 10 client.0 _ll_register_callbacks cb 0x5567761f1d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:04:51.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.926+0000 7fc6ad7d3640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567762cfcf0 0x5567762dcac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.926+0000 7fc6ad7d3640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567762cfcf0 0x5567762dcac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:39250/0 (socket says 172.21.15.2:39250) 2024-05-14T17:04:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.926+0000 7fc6ad7d3640 1 -- 172.21.15.2:0/211959135 learned_addr learned my addr 172.21.15.2:0/211959135 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:04:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.926+0000 7fc6ad7d3640 1 -- 172.21.15.2:0/211959135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567762e3170 msgr2=0x5567762e27e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T17:04:51.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.926+0000 7fc6ad7d3640 1 --2- 172.21.15.2:0/211959135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567762e3170 0x5567762e27e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.926+0000 7fc6ad7d3640 1 -- 172.21.15.2:0/211959135 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567762d0e30 msgr2=0x5567762dd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:51.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.926+0000 7fc6ad7d3640 1 --2- 172.21.15.2:0/211959135 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567762d0e30 0x5567762dd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.926+0000 7fc6ad7d3640 1 -- 172.21.15.2:0/211959135 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556776013620 con 0x5567762cfcf0 2024-05-14T17:04:51.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.927+0000 7fc6ad7d3640 1 --2- 172.21.15.2:0/211959135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567762cfcf0 0x5567762dcac0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc6a400b940 tx=0x7fc6a400be00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b8e903c8b065d8f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.927+0000 7fc690ff9640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-14T17:04:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.927+0000 7fc690ff9640 1 -- 172.21.15.2:0/211959135 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4005020 con 0x5567762cfcf0 2024-05-14T17:04:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.927+0000 7fc6b282d740 1 -- 172.21.15.2:0/211959135 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6a001ce50 con 0x5567762cfcf0 2024-05-14T17:04:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.927+0000 7fc690ff9640 1 -- 172.21.15.2:0/211959135 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7fc6a40051c0 con 0x5567762cfcf0 2024-05-14T17:04:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.927+0000 7fc690ff9640 1 -- 172.21.15.2:0/211959135 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc6a40053c0 con 0x5567762cfcf0 2024-05-14T17:04:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.930+0000 7fc6b282d740 10 client.26071 fetch_fsmap learned FSMap version 43 2024-05-14T17:04:52.011 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.930+0000 7fc6b282d740 1 -- 172.21.15.2:0/211959135 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5567762fa350 con 0x5567762cfcf0 2024-05-14T17:04:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.931+0000 7fc690ff9640 1 -- 172.21.15.2:0/211959135 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6a001ce50 con 0x5567762cfcf0 2024-05-14T17:04:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.931+0000 7fc690ff9640 1 -- 172.21.15.2:0/211959135 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7fc6a40116e0 con 0x5567762cfcf0 2024-05-14T17:04:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.931+0000 7fc6b282d740 10 client.26071 fetch_fsmap finished waiting for FSMap version 43 2024-05-14T17:04:52.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.931+0000 7fc6b282d740 10 client.26071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:04:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.931+0000 7fc6b282d740 20 client.26071 populate_metadata read hostname 'smithi002' 2024-05-14T17:04:52.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.931+0000 7fc6b282d740 1 --2- 172.21.15.2:0/211959135 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5567762fbc00 0x5567762fe0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:52.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.931+0000 7fc6b282d740 4 client.26071 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:04:52.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.931+0000 7fc6b282d740 1 -- 172.21.15.2:0/211959135 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556776244b80 con 0x5567762fbc00 2024-05-14T17:04:52.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.931+0000 7fc6acfd2640 1 --2- 172.21.15.2:0/211959135 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5567762fbc00 0x5567762fe0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:52.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.932+0000 7fc690ff9640 10 client.26071 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:04:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.933+0000 7fc6acfd2640 1 --2- 172.21.15.2:0/211959135 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5567762fbc00 0x5567762fe0c0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6149d002e507233 server_cookie=7172648054808b42 in_seq=0 out_seq=0 2024-05-14T17:04:52.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.934+0000 7fc690ff9640 1 -- 172.21.15.2:0/211959135 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556776244b80 con 0x5567762fbc00 2024-05-14T17:04:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.934+0000 7fc690ff9640 10 client.26071 handle_command_reply: tid=0 2024-05-14T17:04:52.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.934+0000 7fc6b282d740 1 client.26071 shutdown 2024-05-14T17:04:52.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.934+0000 7fc6b282d740 1 -- 172.21.15.2:0/211959135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567762cfcf0 msgr2=0x5567762dcac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.934+0000 7fc6b282d740 1 --2- 172.21.15.2:0/211959135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567762cfcf0 0x5567762dcac0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc6a400b940 tx=0x7fc6a400be00 comp rx=0 tx=0).stop 2024-05-14T17:04:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.934+0000 7fc6b282d740 1 -- 172.21.15.2:0/211959135 shutdown_connections 2024-05-14T17:04:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.934+0000 7fc6b282d740 1 --2- 172.21.15.2:0/211959135 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5567762fbc00 0x5567762fe0c0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.934+0000 7fc6b282d740 1 --2- 172.21.15.2:0/211959135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567762e3170 0x5567762e27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.934+0000 7fc6b282d740 1 --2- 172.21.15.2:0/211959135 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567762d0e30 0x5567762dd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.935+0000 7fc690ff9640 0 client.26071 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:04:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.935+0000 7fc6b282d740 1 --2- 172.21.15.2:0/211959135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567762cfcf0 0x5567762dcac0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.935+0000 7fc6b282d740 1 -- 172.21.15.2:0/211959135 >> 172.21.15.2:0/211959135 conn(0x5567762cbe00 msgr2=0x5567762d1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.935+0000 7fc6b282d740 1 -- 172.21.15.2:0/211959135 shutdown_connections 2024-05-14T17:04:52.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.935+0000 7fc6b282d740 1 -- 172.21.15.2:0/211959135 wait complete. 2024-05-14T17:04:52.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.935+0000 7fc6b282d740 20 client.26071 trim_cache size 0 max 16384 2024-05-14T17:04:52.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.964+0000 7fc6b282d740 1 Processor -- start 2024-05-14T17:04:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.964+0000 7fc6b282d740 1 -- start start 2024-05-14T17:04:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.965+0000 7fc6b282d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567762cfcf0 0x5567762e2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:52.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.965+0000 7fc6b282d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567762d0e30 0x5567763039d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:52.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.965+0000 7fc6ad7d3640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567762cfcf0 0x5567762e2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:52.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.965+0000 7fc6ad7d3640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567762cfcf0 0x5567762e2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:48386/0 (socket says 172.21.15.2:48386) 2024-05-14T17:04:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.966+0000 7fc6ad7d3640 1 -- 172.21.15.2:0/4257697507 learned_addr learned my addr 172.21.15.2:0/4257697507 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:04:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.966+0000 7fc6b282d740 1 --2- 172.21.15.2:0/4257697507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567762e3170 0x556776303f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.966+0000 7fc6b282d740 1 -- 172.21.15.2:0/4257697507 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a81b7600 con 0x5567762d0e30 2024-05-14T17:04:52.402 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.966+0000 7fc6b282d740 1 -- 172.21.15.2:0/4257697507 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a81b4ea0 con 0x5567762cfcf0 2024-05-14T17:04:52.418 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.966+0000 7fc6adfd4640 1 --2- 172.21.15.2:0/4257697507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567762e3170 0x556776303f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.966+0000 7fc6acfd2640 1 --2- 172.21.15.2:0/4257697507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567762d0e30 0x5567763039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.966+0000 7fc6b282d740 1 -- 172.21.15.2:0/4257697507 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a81b2d10 con 0x5567762e3170 2024-05-14T17:04:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.967+0000 7fc6ad7d3640 1 -- 172.21.15.2:0/4257697507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567762e3170 msgr2=0x556776303f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.967+0000 7fc6ad7d3640 1 --2- 172.21.15.2:0/4257697507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567762e3170 0x556776303f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.967+0000 7fc6ad7d3640 1 -- 172.21.15.2:0/4257697507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567762d0e30 msgr2=0x5567763039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.967+0000 7fc6ad7d3640 1 --2- 172.21.15.2:0/4257697507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567762d0e30 0x5567763039d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.967+0000 7fc6ad7d3640 1 -- 172.21.15.2:0/4257697507 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567762d8e10 con 0x5567762cfcf0 2024-05-14T17:04:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.967+0000 7fc6adfd4640 1 --2- 172.21.15.2:0/4257697507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567762e3170 0x556776303f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:04:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.967+0000 7fc6ad7d3640 1 --2- 172.21.15.2:0/4257697507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567762cfcf0 0x5567762e2750 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4001eb0 tx=0x7fc6a4005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b3d49491c38dde0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:52.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.967+0000 7fc692ffd640 1 -- 172.21.15.2:0/4257697507 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4031070 con 0x5567762cfcf0 2024-05-14T17:04:52.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.967+0000 7fc692ffd640 1 -- 172.21.15.2:0/4257697507 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6a4015070 con 0x5567762cfcf0 2024-05-14T17:04:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.967+0000 7fc692ffd640 1 -- 172.21.15.2:0/4257697507 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4026040 con 0x5567762cfcf0 2024-05-14T17:04:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.968+0000 7fc6b282d740 1 -- 172.21.15.2:0/4257697507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567762cfcf0 msgr2=0x5567762e2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:52.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.968+0000 7fc6b282d740 1 --2- 172.21.15.2:0/4257697507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567762cfcf0 0x5567762e2750 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4001eb0 tx=0x7fc6a4005770 comp rx=0 tx=0).stop 2024-05-14T17:04:52.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.969+0000 7fc6b282d740 1 -- 172.21.15.2:0/4257697507 shutdown_connections 2024-05-14T17:04:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.969+0000 7fc6b282d740 1 --2- 172.21.15.2:0/4257697507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567762e3170 0x556776303f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:52.618 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.969+0000 7fc6b282d740 1 --2- 172.21.15.2:0/4257697507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567762d0e30 0x5567763039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:52.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.969+0000 7fc6b282d740 1 --2- 172.21.15.2:0/4257697507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567762cfcf0 0x5567762e2750 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.969+0000 7fc6b282d740 1 -- 172.21.15.2:0/4257697507 >> 172.21.15.2:0/4257697507 conn(0x5567762cbe00 msgr2=0x5567762cb100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:52.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.970+0000 7fc6b282d740 1 -- 172.21.15.2:0/4257697507 shutdown_connections 2024-05-14T17:04:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.970+0000 7fc6b282d740 1 -- 172.21.15.2:0/4257697507 wait complete. 2024-05-14T17:04:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.970+0000 7fc6b282d740 1 -- start start 2024-05-14T17:04:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.971+0000 7fc6b282d740 1 Processor -- start 2024-05-14T17:04:52.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.971+0000 7fc6b282d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567762cfcf0 0x5567762d4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.972+0000 7fc6b282d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567762d0e30 0x55677630f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:52.754 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:04:51 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:04:52.763 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:04:51 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:04:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.972+0000 7fc6b282d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567762e3170 0x55677630f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.972+0000 7fc6b282d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a81b3360 con 0x5567762e3170 2024-05-14T17:04:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.972+0000 7fc6ad7d3640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567762cfcf0 0x5567762d4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:52.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.972+0000 7fc6b282d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a81b47d0 con 0x5567762d0e30 2024-05-14T17:04:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.972+0000 7fc6b282d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a81b3a30 con 0x5567762cfcf0 2024-05-14T17:04:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.972+0000 7fc6ad7d3640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567762cfcf0 0x5567762d4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:39272/0 (socket says 172.21.15.2:39272) 2024-05-14T17:04:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.972+0000 7fc6ad7d3640 1 -- 172.21.15.2:0/2784185682 learned_addr learned my addr 172.21.15.2:0/2784185682 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:04:52.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.972+0000 7fc6acfd2640 1 --2- 172.21.15.2:0/2784185682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567762d0e30 0x55677630f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.972+0000 7fc6b282d740 10 client.0 _ll_register_callbacks cb 0x7fc6a40044e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:04:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.972+0000 7fc6adfd4640 1 --2- 172.21.15.2:0/2784185682 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567762e3170 0x55677630f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.972+0000 7fc6ad7d3640 1 -- 172.21.15.2:0/2784185682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567762d0e30 msgr2=0x55677630f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.972+0000 7fc6ad7d3640 1 --2- 172.21.15.2:0/2784185682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567762d0e30 0x55677630f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:52.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.972+0000 7fc6ad7d3640 1 -- 172.21.15.2:0/2784185682 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567762e3170 msgr2=0x55677630f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:04:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.972+0000 7fc6ad7d3640 1 --2- 172.21.15.2:0/2784185682 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567762e3170 0x55677630f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.972+0000 7fc6ad7d3640 1 -- 172.21.15.2:0/2784185682 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567762dc2a0 con 0x5567762cfcf0 2024-05-14T17:04:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.972+0000 7fc6acfd2640 1 --2- 172.21.15.2:0/2784185682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567762d0e30 0x55677630f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:04:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.974+0000 7fc6ad7d3640 1 --2- 172.21.15.2:0/2784185682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567762cfcf0 0x5567762d4f30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc6a40041e0 tx=0x7fc6a4003fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c38d2a2743637d59 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:04:53.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.974+0000 7fc690ff9640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-14T17:04:53.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.974+0000 7fc690ff9640 1 -- 172.21.15.2:0/2784185682 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4026070 con 0x5567762cfcf0 2024-05-14T17:04:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.974+0000 7fc690ff9640 1 -- 172.21.15.2:0/2784185682 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7fc6a4011640 con 0x5567762cfcf0 2024-05-14T17:04:53.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.974+0000 7fc6b282d740 1 -- 172.21.15.2:0/2784185682 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6a4015070 con 0x5567762cfcf0 2024-05-14T17:04:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.975+0000 7fc690ff9640 1 -- 172.21.15.2:0/2784185682 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc6a4031070 con 0x5567762cfcf0 2024-05-14T17:04:53.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.975+0000 7fc6b282d740 10 client.26077 fetch_fsmap learned FSMap version 43 2024-05-14T17:04:53.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.975+0000 7fc6b282d740 1 -- 172.21.15.2:0/2784185682 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x556776366560 con 0x5567762cfcf0 2024-05-14T17:04:53.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.976+0000 7fc690ff9640 1 -- 172.21.15.2:0/2784185682 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6a4015070 con 0x5567762cfcf0 2024-05-14T17:04:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.977+0000 7fc690ff9640 1 -- 172.21.15.2:0/2784185682 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7fc6a4036780 con 0x5567762cfcf0 2024-05-14T17:04:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.977+0000 7fc6b282d740 10 client.26077 fetch_fsmap finished waiting for FSMap version 43 2024-05-14T17:04:53.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.977+0000 7fc6b282d740 10 client.26077 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:04:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.977+0000 7fc6b282d740 20 client.26077 populate_metadata read hostname 'smithi002' 2024-05-14T17:04:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.977+0000 7fc6b282d740 1 --2- 172.21.15.2:0/2784185682 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x556776366b70 0x556776368f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:04:53.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.978+0000 7fc6b282d740 4 client.26077 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:04:53.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.978+0000 7fc6b282d740 1 -- 172.21.15.2:0/2784185682 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55677623e360 con 0x556776366b70 2024-05-14T17:04:53.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.978+0000 7fc6acfd2640 1 --2- 172.21.15.2:0/2784185682 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x556776366b70 0x556776368f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:04:53.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.979+0000 7fc6acfd2640 1 --2- 172.21.15.2:0/2784185682 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x556776366b70 0x556776368f60 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=281492109cb156ae server_cookie=e774855407ec45d5 in_seq=0 out_seq=0 2024-05-14T17:04:53.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:50.979+0000 7fc690ff9640 10 client.26077 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:04:53.266 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:04:50 smithi002 ceph-mon[31538]: pgmap v315: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 21 MiB/s wr, 1.60k op/s 2024-05-14T17:04:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:51.003+0000 7fc690ff9640 1 -- 172.21.15.2:0/2784185682 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55677623e360 con 0x556776366b70 2024-05-14T17:04:53.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:51.003+0000 7fc690ff9640 10 client.26077 handle_command_reply: tid=0 2024-05-14T17:04:53.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:51.003+0000 7fc6b282d740 1 client.26077 shutdown 2024-05-14T17:04:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:51.004+0000 7fc6b282d740 1 -- 172.21.15.2:0/2784185682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567762cfcf0 msgr2=0x5567762d4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:51.004+0000 7fc6b282d740 1 --2- 172.21.15.2:0/2784185682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567762cfcf0 0x5567762d4f30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc6a40041e0 tx=0x7fc6a4003fa0 comp rx=0 tx=0).stop 2024-05-14T17:04:53.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:51.004+0000 7fc6b282d740 1 -- 172.21.15.2:0/2784185682 shutdown_connections 2024-05-14T17:04:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:51.004+0000 7fc6b282d740 1 --2- 172.21.15.2:0/2784185682 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x556776366b70 0x556776368f60 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:53.334 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:51.004+0000 7fc6b282d740 1 --2- 172.21.15.2:0/2784185682 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5567762e3170 0x55677630f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:53.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:51.004+0000 7fc6b282d740 1 --2- 172.21.15.2:0/2784185682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567762d0e30 0x55677630f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:53.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:51.004+0000 7fc6b282d740 1 --2- 172.21.15.2:0/2784185682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567762cfcf0 0x5567762d4f30 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:53.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:51.004+0000 7fc6b282d740 1 -- 172.21.15.2:0/2784185682 >> 172.21.15.2:0/2784185682 conn(0x5567762cbe00 msgr2=0x5567762d1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:51.004+0000 7fc690ff9640 0 client.26077 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:04:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:51.005+0000 7fc6b282d740 1 -- 172.21.15.2:0/2784185682 shutdown_connections 2024-05-14T17:04:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:51.005+0000 7fc6b282d740 1 -- 172.21.15.2:0/2784185682 wait complete. 2024-05-14T17:04:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:51.005+0000 7fc6b282d740 20 client.26077 trim_cache size 0 max 16384 2024-05-14T17:04:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:04:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:04:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:04:53.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:51.007+0000 7fc6afa37640 1 -- 172.21.15.2:0/3489784546 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc6940b4e10 msgr2=0x7fc6940b72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:53.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:51.007+0000 7fc6afa37640 1 --2- 172.21.15.2:0/3489784546 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc6940b4e10 0x7fc6940b72d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc698005ab0 tx=0x7fc698005a20 comp rx=0 tx=0).stop 2024-05-14T17:04:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:51.007+0000 7fc6afa37640 1 -- 172.21.15.2:0/3489784546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6a81149a0 msgr2=0x7fc6a81a06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:04:53.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:51.007+0000 7fc6afa37640 1 --2- 172.21.15.2:0/3489784546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6a81149a0 0x7fc6a81a06f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc6a000eb10 tx=0x7fc6a000c2d0 comp rx=0 tx=0).stop 2024-05-14T17:04:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:51.007+0000 7fc6afa37640 1 -- 172.21.15.2:0/3489784546 shutdown_connections 2024-05-14T17:04:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:51.007+0000 7fc6afa37640 1 --2- 172.21.15.2:0/3489784546 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc6940b4e10 0x7fc6940b72d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:51.007+0000 7fc6afa37640 1 --2- 172.21.15.2:0/3489784546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6a81149a0 0x7fc6a81a06f0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:51.007+0000 7fc6afa37640 1 --2- 172.21.15.2:0/3489784546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6a810bd10 0x7fc6a81a6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:51.007+0000 7fc6afa37640 1 --2- 172.21.15.2:0/3489784546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6a810b3f0 0x7fc6a81a6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:04:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:51.007+0000 7fc6afa37640 1 -- 172.21.15.2:0/3489784546 >> 172.21.15.2:0/3489784546 conn(0x7fc6a806dfb0 msgr2=0x7fc6a8071770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:04:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:51.008+0000 7fc6afa37640 1 -- 172.21.15.2:0/3489784546 shutdown_connections 2024-05-14T17:04:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:04:51.009+0000 7fc6afa37640 1 -- 172.21.15.2:0/3489784546 wait complete. 2024-05-14T17:04:53.607 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 575.0 s, 2811.0 tps, lat 35.867 ms stddev 41.154 2024-05-14T17:04:53.614 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:04:52 smithi123 ceph-mon[35971]: pgmap v316: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 8.8 MiB/s wr, 584 op/s 2024-05-14T17:04:53.620 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:04:52 smithi151 ceph-mon[36066]: pgmap v316: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 8.8 MiB/s wr, 584 op/s 2024-05-14T17:04:53.626 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:04:51 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:04:53.632 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:04:52 smithi002 ceph-mon[31538]: pgmap v316: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 8.8 MiB/s wr, 584 op/s 2024-05-14T17:04:54.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:04:54 smithi151 ceph-mon[36066]: pgmap v317: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 12 MiB/s wr, 825 op/s 2024-05-14T17:04:54.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:04:54 smithi123 ceph-mon[35971]: pgmap v317: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 12 MiB/s wr, 825 op/s 2024-05-14T17:04:54.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:04:54 smithi002 ceph-mon[31538]: pgmap v317: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 12 MiB/s wr, 825 op/s 2024-05-14T17:04:56.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 580.0 s, 2704.4 tps, lat 36.928 ms stddev 41.537 2024-05-14T17:04:56.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:04:56 smithi151 ceph-mon[36066]: pgmap v318: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 8.8 MiB/s wr, 654 op/s 2024-05-14T17:04:56.870 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:04:56 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:04:56.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:04:56 smithi123 ceph-mon[35971]: pgmap v318: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 8.8 MiB/s wr, 654 op/s 2024-05-14T17:04:56.918 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:04:56 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:04:56.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:04:56 smithi002 ceph-mon[31538]: pgmap v318: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 8.8 MiB/s wr, 654 op/s 2024-05-14T17:04:57.038 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:04:56 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:04:58.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:04:58 smithi151 ceph-mon[36066]: pgmap v319: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 7.7 MiB/s wr, 575 op/s 2024-05-14T17:04:58.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:04:58 smithi123 ceph-mon[35971]: pgmap v319: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 7.7 MiB/s wr, 575 op/s 2024-05-14T17:04:58.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:04:58 smithi002 ceph-mon[31538]: pgmap v319: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 7.7 MiB/s wr, 575 op/s 2024-05-14T17:04:59.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:04:59 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:04:59.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:04:59 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:05:00.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:04:59 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:05:00.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:00 smithi123 ceph-mon[35971]: pgmap v320: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 11 MiB/s wr, 858 op/s 2024-05-14T17:05:00.923 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:00 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:05:00.932 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:00 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:05:00.945 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:00 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:05:00.955 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:00 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:05:00.965 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:00 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:05:00.979 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:00 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:05:00.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:00 smithi151 ceph-mon[36066]: pgmap v320: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 11 MiB/s wr, 858 op/s 2024-05-14T17:05:00.999 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:00 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:05:01.011 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:00 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:05:01.023 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:00 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:05:01.036 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:00 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:05:01.050 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:00 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:05:01.058 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:00 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:05:01.069 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:00 smithi002 ceph-mon[31538]: pgmap v320: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 11 MiB/s wr, 858 op/s 2024-05-14T17:05:01.116 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:00 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:05:01.135 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:00 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:05:01.147 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:00 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:05:01.157 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:00 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:05:01.167 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:00 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:05:01.180 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:00 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:05:01.747 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 585.0 s, 2209.8 tps, lat 36.696 ms stddev 42.113 2024-05-14T17:05:02.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:02 smithi123 ceph-mon[35971]: pgmap v321: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 7.5 MiB/s wr, 587 op/s 2024-05-14T17:05:02.937 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:02 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:05:02.951 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:02 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:05:02.965 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:02 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:05:02.983 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:02 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:05:02.996 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:02 smithi002 ceph-mon[31538]: pgmap v321: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 7.5 MiB/s wr, 587 op/s 2024-05-14T17:05:03.009 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:02 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:05:03.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:02 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:05:03.033 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:02 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:05:03.048 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:02 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:05:03.138 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:02 smithi151 ceph-mon[36066]: pgmap v321: 97 pgs: 97 active+clean; 4.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 7.5 MiB/s wr, 587 op/s 2024-05-14T17:05:03.183 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:02 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:05:03.197 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:02 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:05:03.210 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:02 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:05:03.223 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:02 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:05:03.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:03 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:05:03.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:03 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:05:03.940 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:03 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:05:03.948 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:03 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:05:03.958 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:03 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:05:03.981 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:03 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:05:03.991 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:03 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:05:03.999 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:03 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:05:04.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:03 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:05:04.015 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:03 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:05:04.024 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:03 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:05:04.032 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:03 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:05:04.100 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:03 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:05:04.114 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:03 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:05:04.124 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:03 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:05:04.136 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:03 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:05:04.146 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:03 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:05:04.156 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:03 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:05:04.187 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:03 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:05:04.199 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:03 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:05:04.213 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:03 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:05:04.230 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:03 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:05:04.242 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:03 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:05:04.253 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:03 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:05:04.266 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:03 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:05:04.280 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:03 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:05:04.293 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:03 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:05:04.303 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:03 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:05:04.316 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:03 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:05:04.366 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:03 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:05:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.156+0000 7fb1d3746640 1 -- 172.21.15.2:0/958987523 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1cc1092b0 msgr2=0x7fb1cc109690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.156+0000 7fb1d3746640 1 --2- 172.21.15.2:0/958987523 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1cc1092b0 0x7fb1cc109690 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc009900 tx=0x7fb1bc02a410 comp rx=0 tx=0).stop 2024-05-14T17:05:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.157+0000 7fb1d3746640 1 -- 172.21.15.2:0/958987523 shutdown_connections 2024-05-14T17:05:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.157+0000 7fb1d3746640 1 --2- 172.21.15.2:0/958987523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1cc10e380 0x7fb1cc115f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:04.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.157+0000 7fb1d3746640 1 --2- 172.21.15.2:0/958987523 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1cc109c60 0x7fb1cc10dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:04.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.157+0000 7fb1d3746640 1 --2- 172.21.15.2:0/958987523 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1cc1092b0 0x7fb1cc109690 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.157+0000 7fb1d3746640 1 -- 172.21.15.2:0/958987523 >> 172.21.15.2:0/958987523 conn(0x7fb1cc076fe0 msgr2=0x7fb1cc079400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.157+0000 7fb1d3746640 1 -- 172.21.15.2:0/958987523 shutdown_connections 2024-05-14T17:05:04.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.157+0000 7fb1d3746640 1 -- 172.21.15.2:0/958987523 wait complete. 2024-05-14T17:05:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.158+0000 7fb1d3746640 1 Processor -- start 2024-05-14T17:05:04.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.158+0000 7fb1d3746640 1 -- start start 2024-05-14T17:05:04.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.158+0000 7fb1d3746640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1cc1092b0 0x7fb1cc083040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.158+0000 7fb1d3746640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1cc10e380 0x7fb1cc07c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:04.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.158+0000 7fb1d3746640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1cc07c6d0 0x7fb1cc07cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:04.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.158+0000 7fb1d3746640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb1cc1192c0 con 0x7fb1cc10e380 2024-05-14T17:05:04.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.158+0000 7fb1d3746640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb1cc119140 con 0x7fb1cc07c6d0 2024-05-14T17:05:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.158+0000 7fb1d3746640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb1cc119440 con 0x7fb1cc1092b0 2024-05-14T17:05:04.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.158+0000 7fb1d1ce3640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1cc07c6d0 0x7fb1cc07cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.158+0000 7fb1d0ce1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1cc10e380 0x7fb1cc07c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.158+0000 7fb1d1ce3640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1cc07c6d0 0x7fb1cc07cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:52954/0 (socket says 172.21.15.2:52954) 2024-05-14T17:05:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.158+0000 7fb1d0ce1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1cc10e380 0x7fb1cc07c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50974/0 (socket says 172.21.15.2:50974) 2024-05-14T17:05:04.691 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.158+0000 7fb1d1ce3640 1 -- 172.21.15.2:0/1736631059 learned_addr learned my addr 172.21.15.2:0/1736631059 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:05:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.159+0000 7fb1d14e2640 1 --2- 172.21.15.2:0/1736631059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1cc1092b0 0x7fb1cc083040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.159+0000 7fb1d1ce3640 1 -- 172.21.15.2:0/1736631059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1cc1092b0 msgr2=0x7fb1cc083040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.159+0000 7fb1d1ce3640 1 --2- 172.21.15.2:0/1736631059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1cc1092b0 0x7fb1cc083040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:04.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.159+0000 7fb1d1ce3640 1 -- 172.21.15.2:0/1736631059 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1cc10e380 msgr2=0x7fb1cc07c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.159+0000 7fb1d1ce3640 1 --2- 172.21.15.2:0/1736631059 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1cc10e380 0x7fb1cc07c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.159+0000 7fb1d1ce3640 1 -- 172.21.15.2:0/1736631059 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1cc080b20 con 0x7fb1cc07c6d0 2024-05-14T17:05:04.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.159+0000 7fb1d14e2640 1 --2- 172.21.15.2:0/1736631059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1cc1092b0 0x7fb1cc083040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:05:04.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.159+0000 7fb1d0ce1640 1 --2- 172.21.15.2:0/1736631059 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1cc10e380 0x7fb1cc07c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:05:04.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.159+0000 7fb1d1ce3640 1 --2- 172.21.15.2:0/1736631059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1cc07c6d0 0x7fb1cc07cb80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb1c400eb40 tx=0x7fb1c400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:04.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.159+0000 7fb1c27fc640 1 -- 172.21.15.2:0/1736631059 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c4004b20 con 0x7fb1cc07c6d0 2024-05-14T17:05:04.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.159+0000 7fb1c27fc640 1 -- 172.21.15.2:0/1736631059 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1c4004cc0 con 0x7fb1cc07c6d0 2024-05-14T17:05:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.159+0000 7fb1d3746640 1 -- 172.21.15.2:0/1736631059 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1cc080e10 con 0x7fb1cc07c6d0 2024-05-14T17:05:04.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.159+0000 7fb1c27fc640 1 -- 172.21.15.2:0/1736631059 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c4010690 con 0x7fb1cc07c6d0 2024-05-14T17:05:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.159+0000 7fb1d3746640 1 -- 172.21.15.2:0/1736631059 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1cc081310 con 0x7fb1cc07c6d0 2024-05-14T17:05:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.163+0000 7fb1d6463740 1 Processor -- start 2024-05-14T17:05:04.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.164+0000 7fb1c27fc640 1 -- 172.21.15.2:0/1736631059 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fb1c4005020 con 0x7fb1cc07c6d0 2024-05-14T17:05:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.164+0000 7fb1d6463740 1 -- start start 2024-05-14T17:05:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.165+0000 7fb1d6463740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604b50978f0 0x5604b5097cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.165+0000 7fb1d6463740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604b5098210 0x5604b50a2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.165+0000 7fb1d6463740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604b50a2ad0 0x5604b50a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:04.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.165+0000 7fb1d6463740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5604b500ac30 con 0x5604b50a2ad0 2024-05-14T17:05:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.165+0000 7fb1d6463740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5604b4ffa7d0 con 0x5604b5098210 2024-05-14T17:05:05.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.165+0000 7fb1d6463740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5604b4ffe810 con 0x5604b50978f0 2024-05-14T17:05:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.165+0000 7fb1d0ce1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604b50978f0 0x5604b5097cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.165+0000 7fb1d0ce1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604b50978f0 0x5604b5097cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:33410/0 (socket says 172.21.15.2:33410) 2024-05-14T17:05:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.165+0000 7fb1d0ce1640 1 -- 172.21.15.2:0/1955968533 learned_addr learned my addr 172.21.15.2:0/1955968533 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:05:05.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.166+0000 7fb1d0ce1640 1 --2- 172.21.15.2:0/1955968533 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604b50a2ad0 0x5604b50a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:05.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.166+0000 7fb1d14e2640 1 --2- 172.21.15.2:0/1955968533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604b5098210 0x5604b50a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.166+0000 7fb1c27fc640 1 --2- 172.21.15.2:0/1736631059 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb1b80b2a90 0x7fb1b80b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.166+0000 7fb1d0ce1640 1 -- 172.21.15.2:0/1955968533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604b5098210 msgr2=0x5604b50a2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.166+0000 7fb1d0ce1640 1 --2- 172.21.15.2:0/1955968533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604b5098210 0x5604b50a2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.166+0000 7fb1d0ce1640 1 -- 172.21.15.2:0/1955968533 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604b50a2ad0 msgr2=0x5604b50a4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:05.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.166+0000 7fb1d0ce1640 1 --2- 172.21.15.2:0/1955968533 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604b50a2ad0 0x5604b50a4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:05.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.166+0000 7fb1d0ce1640 1 -- 172.21.15.2:0/1955968533 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604b4d0f210 con 0x5604b50978f0 2024-05-14T17:05:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.166+0000 7fb1d14e2640 1 --2- 172.21.15.2:0/1955968533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604b5098210 0x5604b50a2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:05:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.167+0000 7fb1d0ce1640 1 --2- 172.21.15.2:0/1955968533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604b50978f0 0x5604b5097cd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb1c80058c0 tx=0x7fb1c8005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=808968a7784804a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:05.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.167+0000 7fb1d1ce3640 1 --2- 172.21.15.2:0/1736631059 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb1b80b2a90 0x7fb1b80b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.167+0000 7fb1c27fc640 1 -- 172.21.15.2:0/1736631059 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7fb1c40e2c20 con 0x7fb1cc07c6d0 2024-05-14T17:05:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.167+0000 7fb1b77fe640 1 -- 172.21.15.2:0/1955968533 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c8033050 con 0x5604b50978f0 2024-05-14T17:05:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.167+0000 7fb1b77fe640 1 -- 172.21.15.2:0/1955968533 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1c80331f0 con 0x5604b50978f0 2024-05-14T17:05:05.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.168+0000 7fb1b77fe640 1 -- 172.21.15.2:0/1955968533 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c80035f0 con 0x5604b50978f0 2024-05-14T17:05:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.168+0000 7fb1d6463740 1 -- 172.21.15.2:0/1955968533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604b50978f0 msgr2=0x5604b5097cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:05.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.168+0000 7fb1d6463740 1 --2- 172.21.15.2:0/1955968533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604b50978f0 0x5604b5097cd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb1c80058c0 tx=0x7fb1c8005850 comp rx=0 tx=0).stop 2024-05-14T17:05:05.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.168+0000 7fb1d1ce3640 1 --2- 172.21.15.2:0/1736631059 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb1b80b2a90 0x7fb1b80b4f50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb1c40a7110 tx=0x7fb1c4005800 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.168+0000 7fb1d6463740 1 -- 172.21.15.2:0/1955968533 shutdown_connections 2024-05-14T17:05:05.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.168+0000 7fb1d6463740 1 --2- 172.21.15.2:0/1955968533 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604b50a2ad0 0x5604b50a4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.168+0000 7fb1d6463740 1 --2- 172.21.15.2:0/1955968533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604b5098210 0x5604b50a2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.168+0000 7fb1d6463740 1 --2- 172.21.15.2:0/1955968533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604b50978f0 0x5604b5097cd0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.168+0000 7fb1d6463740 1 -- 172.21.15.2:0/1955968533 >> 172.21.15.2:0/1955968533 conn(0x5604b5093e00 msgr2=0x5604b50941e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.168+0000 7fb1d6463740 1 -- 172.21.15.2:0/1955968533 shutdown_connections 2024-05-14T17:05:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.168+0000 7fb1d6463740 1 -- 172.21.15.2:0/1955968533 wait complete. 2024-05-14T17:05:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.169+0000 7fb1d6463740 1 -- start start 2024-05-14T17:05:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.169+0000 7fb1d6463740 1 Processor -- start 2024-05-14T17:05:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.169+0000 7fb1d6463740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604b50978f0 0x5604b50a2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.169+0000 7fb1d6463740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604b5098210 0x5604b509f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.169+0000 7fb1d6463740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604b50a2ad0 0x5604b509fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.169+0000 7fb1d6463740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5604b4ce0790 con 0x5604b50a2ad0 2024-05-14T17:05:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.169+0000 7fb1d6463740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5604b4fb6890 con 0x5604b5098210 2024-05-14T17:05:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.169+0000 7fb1d6463740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5604b50a57f0 con 0x5604b50978f0 2024-05-14T17:05:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.169+0000 7fb1d6463740 10 client.0 _ll_register_callbacks cb 0x5604b4fb9d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:05:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.170+0000 7fb1d14e2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604b5098210 0x5604b509f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.170+0000 7fb1d14e2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604b5098210 0x5604b509f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:52978/0 (socket says 172.21.15.2:52978) 2024-05-14T17:05:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.170+0000 7fb1d14e2640 1 -- 172.21.15.2:0/626150892 learned_addr learned my addr 172.21.15.2:0/626150892 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:05:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.170+0000 7fb1d0ce1640 1 --2- 172.21.15.2:0/626150892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604b50a2ad0 0x5604b509fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:05.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.170+0000 7fb1d0ce1640 1 --2- 172.21.15.2:0/626150892 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604b50978f0 0x5604b50a2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:05.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.170+0000 7fb1d14e2640 1 -- 172.21.15.2:0/626150892 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604b50978f0 msgr2=0x5604b50a2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:05.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.170+0000 7fb1d14e2640 1 --2- 172.21.15.2:0/626150892 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604b50978f0 0x5604b50a2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:05.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.170+0000 7fb1d14e2640 1 -- 172.21.15.2:0/626150892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604b50a2ad0 msgr2=0x5604b509fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.170+0000 7fb1d14e2640 1 --2- 172.21.15.2:0/626150892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604b50a2ad0 0x5604b509fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:05.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.170+0000 7fb1d14e2640 1 -- 172.21.15.2:0/626150892 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604b4ddb620 con 0x5604b5098210 2024-05-14T17:05:05.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.170+0000 7fb1d0ce1640 1 --2- 172.21.15.2:0/626150892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604b50a2ad0 0x5604b509fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:05:05.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.172+0000 7fb1d14e2640 1 --2- 172.21.15.2:0/626150892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604b5098210 0x5604b509f710 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc07d8d0 tx=0x7fb1bc005230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2702c1ffb0127dab server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:05.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.172+0000 7fb1b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T17:05:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.172+0000 7fb1b4ff9640 1 -- 172.21.15.2:0/626150892 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc0035f0 con 0x5604b5098210 2024-05-14T17:05:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.172+0000 7fb1d6463740 1 -- 172.21.15.2:0/626150892 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1c80331f0 con 0x5604b5098210 2024-05-14T17:05:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.172+0000 7fb1b4ff9640 1 -- 172.21.15.2:0/626150892 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7fb1bc004d50 con 0x5604b5098210 2024-05-14T17:05:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.172+0000 7fb1b4ff9640 1 -- 172.21.15.2:0/626150892 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fb1bc0035f0 con 0x5604b5098210 2024-05-14T17:05:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.173+0000 7fb1d6463740 10 client.26108 fetch_fsmap learned FSMap version 44 2024-05-14T17:05:05.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.173+0000 7fb1d6463740 1 -- 172.21.15.2:0/626150892 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5604b50b10e0 con 0x5604b5098210 2024-05-14T17:05:05.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.173+0000 7fb1b4ff9640 1 -- 172.21.15.2:0/626150892 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1c80331f0 con 0x5604b5098210 2024-05-14T17:05:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.173+0000 7fb1b4ff9640 1 -- 172.21.15.2:0/626150892 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 44) ==== 8350+0+0 (secure 0 0 0) 0x7fb1bc044070 con 0x5604b5098210 2024-05-14T17:05:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.173+0000 7fb1d6463740 10 client.26108 fetch_fsmap finished waiting for FSMap version 44 2024-05-14T17:05:06.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.173+0000 7fb1d6463740 10 client.26108 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:05:06.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.173+0000 7fb1d6463740 20 client.26108 populate_metadata read hostname 'smithi002' 2024-05-14T17:05:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.173+0000 7fb1d6463740 1 --2- 172.21.15.2:0/626150892 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5604b50b25a0 0x5604b50b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:06.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.173+0000 7fb1d6463740 4 client.26108 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:05:06.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.173+0000 7fb1d6463740 1 -- 172.21.15.2:0/626150892 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604b500cb80 con 0x5604b50b25a0 2024-05-14T17:05:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.174+0000 7fb1d0ce1640 1 --2- 172.21.15.2:0/626150892 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5604b50b25a0 0x5604b50b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.174+0000 7fb1d0ce1640 1 --2- 172.21.15.2:0/626150892 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5604b50b25a0 0x5604b50b49c0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9655b8c052544ee5 server_cookie=dbefff3362fa917a in_seq=0 out_seq=0 2024-05-14T17:05:06.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.174+0000 7fb1b4ff9640 10 client.26108 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:05:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.175+0000 7fb1b4ff9640 1 -- 172.21.15.2:0/626150892 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5604b500cb80 con 0x5604b50b25a0 2024-05-14T17:05:06.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.175+0000 7fb1b4ff9640 10 client.26108 handle_command_reply: tid=0 2024-05-14T17:05:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.175+0000 7fb1d6463740 1 client.26108 shutdown 2024-05-14T17:05:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.175+0000 7fb1d6463740 1 -- 172.21.15.2:0/626150892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604b5098210 msgr2=0x5604b509f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.175+0000 7fb1d6463740 1 --2- 172.21.15.2:0/626150892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604b5098210 0x5604b509f710 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc07d8d0 tx=0x7fb1bc005230 comp rx=0 tx=0).stop 2024-05-14T17:05:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.175+0000 7fb1d6463740 1 -- 172.21.15.2:0/626150892 shutdown_connections 2024-05-14T17:05:06.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.175+0000 7fb1d6463740 1 --2- 172.21.15.2:0/626150892 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5604b50b25a0 0x5604b50b49c0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:06.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.175+0000 7fb1b4ff9640 0 client.26108 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:05:06.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.175+0000 7fb1d6463740 1 --2- 172.21.15.2:0/626150892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604b50a2ad0 0x5604b509fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.175+0000 7fb1d6463740 1 --2- 172.21.15.2:0/626150892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604b5098210 0x5604b509f710 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:06.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.175+0000 7fb1d6463740 1 --2- 172.21.15.2:0/626150892 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604b50978f0 0x5604b50a2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.175+0000 7fb1d6463740 1 -- 172.21.15.2:0/626150892 >> 172.21.15.2:0/626150892 conn(0x5604b5093e00 msgr2=0x5604b5093100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:06.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.175+0000 7fb1d6463740 1 -- 172.21.15.2:0/626150892 shutdown_connections 2024-05-14T17:05:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.175+0000 7fb1d6463740 1 -- 172.21.15.2:0/626150892 wait complete. 2024-05-14T17:05:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.175+0000 7fb1d6463740 20 client.26108 trim_cache size 0 max 16384 2024-05-14T17:05:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.209+0000 7fb1d6463740 1 Processor -- start 2024-05-14T17:05:06.378 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.209+0000 7fb1d6463740 1 -- start start 2024-05-14T17:05:06.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.210+0000 7fb1d6463740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604b50978f0 0x5604b50ba270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.210+0000 7fb1d6463740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604b5098210 0x5604b50ba7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.210+0000 7fb1d6463740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604b50a2ad0 0x5604b50aaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.210+0000 7fb1d6463740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb1cc1b77f0 con 0x5604b5098210 2024-05-14T17:05:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.210+0000 7fb1d6463740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb1cc074510 con 0x5604b50a2ad0 2024-05-14T17:05:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.210+0000 7fb1d6463740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb1cc072300 con 0x5604b50978f0 2024-05-14T17:05:06.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.210+0000 7fb1d0ce1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604b50978f0 0x5604b50ba270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:06.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.210+0000 7fb1d0ce1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604b50978f0 0x5604b50ba270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:33424/0 (socket says 172.21.15.2:33424) 2024-05-14T17:05:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.210+0000 7fb1d0ce1640 1 -- 172.21.15.2:0/4158203158 learned_addr learned my addr 172.21.15.2:0/4158203158 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:05:06.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.210+0000 7fb1d0ce1640 1 --2- 172.21.15.2:0/4158203158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604b50a2ad0 0x5604b50aaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.211+0000 7fb1d0ce1640 1 -- 172.21.15.2:0/4158203158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604b50978f0 msgr2=0x5604b50ba270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.211+0000 7fb1d0ce1640 1 --2- 172.21.15.2:0/4158203158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604b50978f0 0x5604b50ba270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.211+0000 7fb1d0ce1640 1 -- 172.21.15.2:0/4158203158 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604b5098210 msgr2=0x5604b50ba7b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T17:05:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.211+0000 7fb1d14e2640 1 --2- 172.21.15.2:0/4158203158 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604b5098210 0x5604b50ba7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.211+0000 7fb1d0ce1640 1 --2- 172.21.15.2:0/4158203158 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604b5098210 0x5604b50ba7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.211+0000 7fb1d0ce1640 1 -- 172.21.15.2:0/4158203158 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604b509ffe0 con 0x5604b50a2ad0 2024-05-14T17:05:06.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.212+0000 7fb1d0ce1640 1 --2- 172.21.15.2:0/4158203158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604b50a2ad0 0x5604b50aaa80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8009730 tx=0x7fb1c8005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6798037ebbbc7092 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:06.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.212+0000 7fb1b6ffd640 1 -- 172.21.15.2:0/4158203158 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c800fd90 con 0x5604b50a2ad0 2024-05-14T17:05:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.212+0000 7fb1b6ffd640 1 -- 172.21.15.2:0/4158203158 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1c80054f0 con 0x5604b50a2ad0 2024-05-14T17:05:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.212+0000 7fb1b6ffd640 1 -- 172.21.15.2:0/4158203158 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c80035f0 con 0x5604b50a2ad0 2024-05-14T17:05:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.213+0000 7fb1d6463740 1 -- 172.21.15.2:0/4158203158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604b50a2ad0 msgr2=0x5604b50aaa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:06.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.213+0000 7fb1d6463740 1 --2- 172.21.15.2:0/4158203158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604b50a2ad0 0x5604b50aaa80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8009730 tx=0x7fb1c8005850 comp rx=0 tx=0).stop 2024-05-14T17:05:06.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.213+0000 7fb1d6463740 1 -- 172.21.15.2:0/4158203158 shutdown_connections 2024-05-14T17:05:06.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.213+0000 7fb1d6463740 1 --2- 172.21.15.2:0/4158203158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604b50a2ad0 0x5604b50aaa80 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.213+0000 7fb1d6463740 1 --2- 172.21.15.2:0/4158203158 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604b5098210 0x5604b50ba7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.213+0000 7fb1d6463740 1 --2- 172.21.15.2:0/4158203158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604b50978f0 0x5604b50ba270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:06.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.213+0000 7fb1d6463740 1 -- 172.21.15.2:0/4158203158 >> 172.21.15.2:0/4158203158 conn(0x5604b5093e00 msgr2=0x5604b5099b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.213+0000 7fb1d6463740 1 -- 172.21.15.2:0/4158203158 shutdown_connections 2024-05-14T17:05:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.213+0000 7fb1d6463740 1 -- 172.21.15.2:0/4158203158 wait complete. 2024-05-14T17:05:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.214+0000 7fb1d6463740 1 -- start start 2024-05-14T17:05:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.214+0000 7fb1d6463740 1 Processor -- start 2024-05-14T17:05:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.214+0000 7fb1d6463740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604b50978f0 0x5604b50aadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.214+0000 7fb1d6463740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604b5098210 0x5604b509bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:06.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.214+0000 7fb1d6463740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604b50a2ad0 0x5604b509c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:06.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.215+0000 7fb1d6463740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb1cc0729d0 con 0x5604b50a2ad0 2024-05-14T17:05:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.215+0000 7fb1d6463740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb1cc073e40 con 0x5604b5098210 2024-05-14T17:05:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.215+0000 7fb1d6463740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb1cc0730a0 con 0x5604b50978f0 2024-05-14T17:05:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.215+0000 7fb1d6463740 10 client.0 _ll_register_callbacks cb 0x7fb1bc009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:05:06.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.215+0000 7fb1d14e2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604b5098210 0x5604b509bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.215+0000 7fb1d0ce1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604b50a2ad0 0x5604b509c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.215+0000 7fb1d14e2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604b5098210 0x5604b509bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:52988/0 (socket says 172.21.15.2:52988) 2024-05-14T17:05:07.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.215+0000 7fb1d0ce1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604b50a2ad0 0x5604b509c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51002/0 (socket says 172.21.15.2:51002) 2024-05-14T17:05:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.215+0000 7fb1d14e2640 1 -- 172.21.15.2:0/1377258821 learned_addr learned my addr 172.21.15.2:0/1377258821 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:05:07.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.215+0000 7fb1d0ce1640 1 --2- 172.21.15.2:0/1377258821 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604b50978f0 0x5604b50aadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.215+0000 7fb1d14e2640 1 -- 172.21.15.2:0/1377258821 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604b50978f0 msgr2=0x5604b50aadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:07.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.215+0000 7fb1d14e2640 1 --2- 172.21.15.2:0/1377258821 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604b50978f0 0x5604b50aadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.215+0000 7fb1d14e2640 1 -- 172.21.15.2:0/1377258821 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604b50a2ad0 msgr2=0x5604b509c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.215+0000 7fb1d14e2640 1 --2- 172.21.15.2:0/1377258821 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604b50a2ad0 0x5604b509c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.215+0000 7fb1d14e2640 1 -- 172.21.15.2:0/1377258821 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604b509cb10 con 0x5604b5098210 2024-05-14T17:05:07.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.216+0000 7fb1d14e2640 1 --2- 172.21.15.2:0/1377258821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604b5098210 0x5604b509bfe0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc009900 tx=0x7fb1bc02a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5e9ae9d29e9d9d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.216+0000 7fb1b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T17:05:07.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.216+0000 7fb1b4ff9640 1 -- 172.21.15.2:0/1377258821 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc0035f0 con 0x5604b5098210 2024-05-14T17:05:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.216+0000 7fb1b4ff9640 1 -- 172.21.15.2:0/1377258821 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7fb1bc03bdb0 con 0x5604b5098210 2024-05-14T17:05:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.216+0000 7fb1d6463740 1 -- 172.21.15.2:0/1377258821 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1c80054f0 con 0x5604b5098210 2024-05-14T17:05:07.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.217+0000 7fb1b4ff9640 1 -- 172.21.15.2:0/1377258821 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fb1bc005d50 con 0x5604b5098210 2024-05-14T17:05:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.217+0000 7fb1d6463740 10 client.26114 fetch_fsmap learned FSMap version 44 2024-05-14T17:05:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.217+0000 7fb1d6463740 1 -- 172.21.15.2:0/1377258821 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5604b511cc10 con 0x5604b5098210 2024-05-14T17:05:07.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.217+0000 7fb1b4ff9640 1 -- 172.21.15.2:0/1377258821 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1bc005630 con 0x5604b5098210 2024-05-14T17:05:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.217+0000 7fb1b4ff9640 1 -- 172.21.15.2:0/1377258821 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 44) ==== 8350+0+0 (secure 0 0 0) 0x7fb1bc044070 con 0x5604b5098210 2024-05-14T17:05:07.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.218+0000 7fb1d6463740 10 client.26114 fetch_fsmap finished waiting for FSMap version 44 2024-05-14T17:05:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.218+0000 7fb1d6463740 10 client.26114 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:05:07.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.218+0000 7fb1d6463740 20 client.26114 populate_metadata read hostname 'smithi002' 2024-05-14T17:05:07.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.218+0000 7fb1d6463740 1 --2- 172.21.15.2:0/1377258821 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5604b511d360 0x5604b511f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.218+0000 7fb1d6463740 4 client.26114 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:05:07.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.218+0000 7fb1d6463740 1 -- 172.21.15.2:0/1377258821 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fb1c80054f0 con 0x5604b511d360 2024-05-14T17:05:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.218+0000 7fb1d0ce1640 1 --2- 172.21.15.2:0/1377258821 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5604b511d360 0x5604b511f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:07.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.218+0000 7fb1b4ff9640 10 client.26114 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:05:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.218+0000 7fb1d0ce1640 1 --2- 172.21.15.2:0/1377258821 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5604b511d360 0x5604b511f780 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b53f8cef698012d6 server_cookie=f57c66671e601cb7 in_seq=0 out_seq=0 2024-05-14T17:05:07.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.281+0000 7fb1b4ff9640 1 -- 172.21.15.2:0/1377258821 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7fb1c80054f0 con 0x5604b511d360 2024-05-14T17:05:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.281+0000 7fb1b4ff9640 10 client.26114 handle_command_reply: tid=0 2024-05-14T17:05:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.282+0000 7fb1d6463740 1 client.26114 shutdown 2024-05-14T17:05:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.284+0000 7fb1d6463740 1 -- 172.21.15.2:0/1377258821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604b5098210 msgr2=0x5604b509bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:07.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.284+0000 7fb1d6463740 1 --2- 172.21.15.2:0/1377258821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604b5098210 0x5604b509bfe0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc009900 tx=0x7fb1bc02a410 comp rx=0 tx=0).stop 2024-05-14T17:05:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.287+0000 7fb1d6463740 1 -- 172.21.15.2:0/1377258821 shutdown_connections 2024-05-14T17:05:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.287+0000 7fb1d6463740 1 --2- 172.21.15.2:0/1377258821 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5604b511d360 0x5604b511f780 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.287+0000 7fb1b4ff9640 0 client.26114 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:05:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.287+0000 7fb1d6463740 1 --2- 172.21.15.2:0/1377258821 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5604b50a2ad0 0x5604b509c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:07.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.287+0000 7fb1d6463740 1 --2- 172.21.15.2:0/1377258821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604b5098210 0x5604b509bfe0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.287+0000 7fb1d6463740 1 --2- 172.21.15.2:0/1377258821 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604b50978f0 0x5604b50aadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.287+0000 7fb1d6463740 1 -- 172.21.15.2:0/1377258821 >> 172.21.15.2:0/1377258821 conn(0x5604b5093e00 msgr2=0x5604b50992a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.288+0000 7fb1d6463740 1 -- 172.21.15.2:0/1377258821 shutdown_connections 2024-05-14T17:05:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.288+0000 7fb1d6463740 1 -- 172.21.15.2:0/1377258821 wait complete. 2024-05-14T17:05:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.288+0000 7fb1d6463740 20 client.26114 trim_cache size 0 max 16384 2024-05-14T17:05:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:05:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:05:07.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:05:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.294+0000 7fb1d3746640 1 -- 172.21.15.2:0/1736631059 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb1b80b2a90 msgr2=0x7fb1b80b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.294+0000 7fb1d3746640 1 --2- 172.21.15.2:0/1736631059 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb1b80b2a90 0x7fb1b80b4f50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb1c40a7110 tx=0x7fb1c4005800 comp rx=0 tx=0).stop 2024-05-14T17:05:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.294+0000 7fb1d3746640 1 -- 172.21.15.2:0/1736631059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1cc07c6d0 msgr2=0x7fb1cc07cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.294+0000 7fb1d3746640 1 --2- 172.21.15.2:0/1736631059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1cc07c6d0 0x7fb1cc07cb80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb1c400eb40 tx=0x7fb1c400c2d0 comp rx=0 tx=0).stop 2024-05-14T17:05:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.296+0000 7fb1d3746640 1 -- 172.21.15.2:0/1736631059 shutdown_connections 2024-05-14T17:05:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.296+0000 7fb1d3746640 1 --2- 172.21.15.2:0/1736631059 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb1b80b2a90 0x7fb1b80b4f50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.296+0000 7fb1d3746640 1 --2- 172.21.15.2:0/1736631059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1cc07c6d0 0x7fb1cc07cb80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:08.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.296+0000 7fb1d3746640 1 --2- 172.21.15.2:0/1736631059 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1cc10e380 0x7fb1cc07c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.296+0000 7fb1d3746640 1 --2- 172.21.15.2:0/1736631059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb1cc1092b0 0x7fb1cc083040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:08.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.296+0000 7fb1d3746640 1 -- 172.21.15.2:0/1736631059 >> 172.21.15.2:0/1736631059 conn(0x7fb1cc076fe0 msgr2=0x7fb1cc079350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.296+0000 7fb1d3746640 1 -- 172.21.15.2:0/1736631059 shutdown_connections 2024-05-14T17:05:08.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:04.296+0000 7fb1d3746640 1 -- 172.21.15.2:0/1736631059 wait complete. 2024-05-14T17:05:08.120 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:04 smithi002 ceph-mon[31538]: Adjusting osd_memory_target on smithi151 to 176.2M 2024-05-14T17:05:08.128 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:04 smithi002 ceph-mon[31538]: Unable to set osd_memory_target on smithi151 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-05-14T17:05:08.137 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:04 smithi002 ceph-mon[31538]: Adjusting osd_memory_target on smithi123 to 176.2M 2024-05-14T17:05:08.145 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:04 smithi002 ceph-mon[31538]: Unable to set osd_memory_target on smithi123 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-05-14T17:05:08.154 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:04 smithi002 ceph-mon[31538]: pgmap v322: 97 pgs: 97 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 25 MiB/s wr, 1.57k op/s 2024-05-14T17:05:08.163 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:04 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:05:08.171 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:04 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:05:08.179 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:04 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:05:08.189 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:04 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:05:08.199 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:04 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:05:08.256 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:04 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:05:08.276 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:04 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:05:08.286 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:05 smithi002 ceph-mon[31538]: Adjusting osd_memory_target on smithi002 to 177.2M 2024-05-14T17:05:08.294 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:05 smithi002 ceph-mon[31538]: Unable to set osd_memory_target on smithi002 to 185879142: error parsing value: Value '185879142' is below minimum 939524096 2024-05-14T17:05:08.304 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 590.0 s, 1746.4 tps, lat 68.169 ms stddev 163.913 2024-05-14T17:05:08.315 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:06 smithi002 ceph-mon[31538]: pgmap v323: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 29 MiB/s wr, 1.75k op/s 2024-05-14T17:05:08.338 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:04 smithi151 ceph-mon[36066]: Adjusting osd_memory_target on smithi151 to 176.2M 2024-05-14T17:05:08.352 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:04 smithi151 ceph-mon[36066]: Unable to set osd_memory_target on smithi151 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-05-14T17:05:08.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:04 smithi151 ceph-mon[36066]: Adjusting osd_memory_target on smithi123 to 176.2M 2024-05-14T17:05:08.374 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:04 smithi151 ceph-mon[36066]: Unable to set osd_memory_target on smithi123 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-05-14T17:05:08.388 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:04 smithi151 ceph-mon[36066]: pgmap v322: 97 pgs: 97 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 25 MiB/s wr, 1.57k op/s 2024-05-14T17:05:08.400 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:04 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:05:08.410 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:04 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:05:08.419 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:04 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:05:08.432 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:04 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:05:08.443 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:04 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:05:08.455 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:04 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:05:08.523 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:04 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:05:08.544 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:05 smithi151 ceph-mon[36066]: Adjusting osd_memory_target on smithi002 to 177.2M 2024-05-14T17:05:08.556 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:05 smithi151 ceph-mon[36066]: Unable to set osd_memory_target on smithi002 to 185879142: error parsing value: Value '185879142' is below minimum 939524096 2024-05-14T17:05:08.573 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:06 smithi151 ceph-mon[36066]: pgmap v323: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 29 MiB/s wr, 1.75k op/s 2024-05-14T17:05:08.587 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:04 smithi123 ceph-mon[35971]: Adjusting osd_memory_target on smithi151 to 176.2M 2024-05-14T17:05:08.601 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:04 smithi123 ceph-mon[35971]: Unable to set osd_memory_target on smithi151 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-05-14T17:05:08.629 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:04 smithi123 ceph-mon[35971]: Adjusting osd_memory_target on smithi123 to 176.2M 2024-05-14T17:05:08.646 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:04 smithi123 ceph-mon[35971]: Unable to set osd_memory_target on smithi123 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-05-14T17:05:08.660 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:04 smithi123 ceph-mon[35971]: pgmap v322: 97 pgs: 97 active+clean; 4.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 25 MiB/s wr, 1.57k op/s 2024-05-14T17:05:08.673 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:04 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:05:08.686 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:04 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:05:08.702 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:04 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:05:08.713 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:04 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:05:08.728 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:04 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:05:08.742 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:04 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:05:08.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:04 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:05:08.772 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:05 smithi123 ceph-mon[35971]: Adjusting osd_memory_target on smithi002 to 177.2M 2024-05-14T17:05:08.844 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:05 smithi123 ceph-mon[35971]: Unable to set osd_memory_target on smithi002 to 185879142: error parsing value: Value '185879142' is below minimum 939524096 2024-05-14T17:05:08.861 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:06 smithi123 ceph-mon[35971]: pgmap v323: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 29 MiB/s wr, 1.75k op/s 2024-05-14T17:05:08.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:08 smithi123 ceph-mon[35971]: pgmap v324: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 MiB/s wr, 1.68k op/s 2024-05-14T17:05:08.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:08 smithi002 ceph-mon[31538]: pgmap v324: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 MiB/s wr, 1.68k op/s 2024-05-14T17:05:09.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:08 smithi151 ceph-mon[36066]: pgmap v324: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 MiB/s wr, 1.68k op/s 2024-05-14T17:05:10.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:10 smithi123 ceph-mon[35971]: pgmap v325: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 34 MiB/s wr, 2.10k op/s 2024-05-14T17:05:10.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:10 smithi002 ceph-mon[31538]: pgmap v325: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 34 MiB/s wr, 2.10k op/s 2024-05-14T17:05:11.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:10 smithi151 ceph-mon[36066]: pgmap v325: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 34 MiB/s wr, 2.10k op/s 2024-05-14T17:05:11.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 595.0 s, 2654.6 tps, lat 37.761 ms stddev 46.249 2024-05-14T17:05:11.891 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.9 2024-05-14T17:05:11.939 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:05:12.380 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:05:12.835 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:12 smithi002 ceph-mon[31538]: pgmap v326: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 30 MiB/s wr, 1.82k op/s 2024-05-14T17:05:12.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:12 smithi123 ceph-mon[35971]: pgmap v326: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 30 MiB/s wr, 1.82k op/s 2024-05-14T17:05:13.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:12 smithi151 ceph-mon[36066]: pgmap v326: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 30 MiB/s wr, 1.82k op/s 2024-05-14T17:05:14.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.597+0000 7f364241e700 1 -- 172.21.15.2:0/13040737 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f363c103850 msgr2=0x7f363c103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:14.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.597+0000 7f364241e700 1 --2- 172.21.15.2:0/13040737 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f363c103850 0x7f363c103c30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f362c009990 tx=0x7f362c009ca0 comp rx=0 tx=0).stop 2024-05-14T17:05:14.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.598+0000 7f364241e700 1 -- 172.21.15.2:0/13040737 shutdown_connections 2024-05-14T17:05:14.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.598+0000 7f364241e700 1 --2- 172.21.15.2:0/13040737 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f363c108920 0x7f363c1104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:14.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.598+0000 7f364241e700 1 --2- 172.21.15.2:0/13040737 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f363c103850 0x7f363c103c30 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:14.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.598+0000 7f364241e700 1 --2- 172.21.15.2:0/13040737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f363c104200 0x7f363c1081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:14.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.598+0000 7f364241e700 1 -- 172.21.15.2:0/13040737 >> 172.21.15.2:0/13040737 conn(0x7f363c07a620 msgr2=0x7f363c07aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:14.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.599+0000 7f364241e700 1 -- 172.21.15.2:0/13040737 shutdown_connections 2024-05-14T17:05:14.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.599+0000 7f364241e700 1 -- 172.21.15.2:0/13040737 wait complete. 2024-05-14T17:05:14.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.600+0000 7f364241e700 1 Processor -- start 2024-05-14T17:05:14.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.600+0000 7f364241e700 1 -- start start 2024-05-14T17:05:14.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.600+0000 7f364241e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f363c103850 0x7f363c10a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:14.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.600+0000 7f364241e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f363c104200 0x7f363c10a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:14.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.601+0000 7f363bfff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f363c103850 0x7f363c10a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:14.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.601+0000 7f364241e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f363c108920 0x7f363c10af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:14.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.601+0000 7f364241e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f363c113750 con 0x7f363c104200 2024-05-14T17:05:14.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.601+0000 7f364241e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f363c1135d0 con 0x7f363c108920 2024-05-14T17:05:14.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.601+0000 7f364241e700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f363c1138d0 con 0x7f363c103850 2024-05-14T17:05:14.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.601+0000 7f363bfff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f363c103850 0x7f363c10a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:60012/0 (socket says 172.21.15.2:60012) 2024-05-14T17:05:14.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.601+0000 7f363bfff700 1 -- 172.21.15.2:0/3803913057 learned_addr learned my addr 172.21.15.2:0/3803913057 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:05:14.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.601+0000 7f363b7fe700 1 --2- 172.21.15.2:0/3803913057 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f363c104200 0x7f363c10a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:14.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.601+0000 7f36409bb700 1 --2- 172.21.15.2:0/3803913057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f363c108920 0x7f363c10af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:14.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.602+0000 7f363b7fe700 1 -- 172.21.15.2:0/3803913057 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f363c103850 msgr2=0x7f363c10a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:14.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.602+0000 7f363b7fe700 1 --2- 172.21.15.2:0/3803913057 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f363c103850 0x7f363c10a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:14.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.602+0000 7f363b7fe700 1 -- 172.21.15.2:0/3803913057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f363c108920 msgr2=0x7f363c10af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:14.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.602+0000 7f363b7fe700 1 --2- 172.21.15.2:0/3803913057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f363c108920 0x7f363c10af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:14.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.602+0000 7f363b7fe700 1 -- 172.21.15.2:0/3803913057 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f363c1ac4b0 con 0x7f363c104200 2024-05-14T17:05:14.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.602+0000 7f363bfff700 1 --2- 172.21.15.2:0/3803913057 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f363c103850 0x7f363c10a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:05:15.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.602+0000 7f363b7fe700 1 --2- 172.21.15.2:0/3803913057 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f363c104200 0x7f363c10a9f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f363000e9f0 tx=0x7f363000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:15.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.603+0000 7f36397fa700 1 -- 172.21.15.2:0/3803913057 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363000cb70 con 0x7f363c104200 2024-05-14T17:05:15.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.603+0000 7f36397fa700 1 -- 172.21.15.2:0/3803913057 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f363000ee90 con 0x7f363c104200 2024-05-14T17:05:15.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.603+0000 7f36397fa700 1 -- 172.21.15.2:0/3803913057 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3630003d40 con 0x7f363c104200 2024-05-14T17:05:15.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.603+0000 7f364241e700 1 -- 172.21.15.2:0/3803913057 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f363c1ac7a0 con 0x7f363c104200 2024-05-14T17:05:15.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.603+0000 7f364241e700 1 -- 172.21.15.2:0/3803913057 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f363c1acc90 con 0x7f363c104200 2024-05-14T17:05:15.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.605+0000 7f364241e700 1 -- 172.21.15.2:0/3803913057 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f363c104cb0 con 0x7f363c104200 2024-05-14T17:05:15.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.607+0000 7f36397fa700 1 -- 172.21.15.2:0/3803913057 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f3630003ee0 con 0x7f363c104200 2024-05-14T17:05:15.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.609+0000 7f36397fa700 1 --2- 172.21.15.2:0/3803913057 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f36240b2bc0 0x7f36240b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:15.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.609+0000 7f363bfff700 1 --2- 172.21.15.2:0/3803913057 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f36240b2bc0 0x7f36240b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:15.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.609+0000 7f36397fa700 1 -- 172.21.15.2:0/3803913057 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f36300e2650 con 0x7f363c104200 2024-05-14T17:05:15.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.610+0000 7f363bfff700 1 --2- 172.21.15.2:0/3803913057 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f36240b2bc0 0x7f36240b5080 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f362c00b790 tx=0x7f362c01a040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:15.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:14.617+0000 7f36397fa700 1 -- 172.21.15.2:0/3803913057 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f36300a9fd0 con 0x7f363c104200 2024-05-14T17:05:15.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:14 smithi151 ceph-mon[36066]: pgmap v327: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 32 MiB/s wr, 1.99k op/s 2024-05-14T17:05:15.201 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:14 smithi123 ceph-mon[35971]: pgmap v327: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 32 MiB/s wr, 1.99k op/s 2024-05-14T17:05:15.209 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:14 smithi002 ceph-mon[31538]: pgmap v327: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 32 MiB/s wr, 1.99k op/s 2024-05-14T17:05:15.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:15.055+0000 7f364241e700 1 -- 172.21.15.2:0/3803913057 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f363c06a230 con 0x7f363c104200 2024-05-14T17:05:15.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:15.058+0000 7f36397fa700 1 -- 172.21.15.2:0/3803913057 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11854 (secure 0 0 0) 0x7f363c06a230 con 0x7f363c104200 2024-05-14T17:05:15.302 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:05:15.341 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2275696923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2275696923},{"type":"v1","addr":"172.21.15.2:6835","nonce":2275696923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/716718761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":716718761},{"type":"v1","addr":"172.21.15.2:6839","nonce":716718761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/3644815179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3644815179},{"type":"v1","addr":"172.21.15.2:6841","nonce":3644815179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1880336731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1880336731},{"type":"v1","addr":"172.21.15.151:6833","nonce":1880336731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2415605548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2415605548},{"type":"v1","addr":"172.21.15.2:6837","nonce":2415605548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2635167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2635167},{"type":"v1","addr":"172.21.15.123:6833","nonce":2635167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:51:34.106919+0000","modified":"2024-05-14T17:04:55.478062+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24455,"mds_2":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24415":{"gid":24415,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/762044339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":762044339},{"type":"v1","addr":"172.21.15.151:6835","nonce":762044339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/132043151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":132043151},{"type":"v1","addr":"172.21.15.151:6837","nonce":132043151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/4107834375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":4107834375},{"type":"v1","addr":"172.21.15.151:6839","nonce":4107834375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.123:6835/1198670013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1198670013},{"type":"v1","addr":"172.21.15.123:6835","nonce":1198670013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/333426645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":333426645},{"type":"v1","addr":"172.21.15.123:6837","nonce":333426645}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/580275877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":580275877},{"type":"v1","addr":"172.21.15.123:6839","nonce":580275877}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24431,24455]},"id":1}]} 2024-05-14T17:05:15.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:15.062+0000 7f364241e700 1 -- 172.21.15.2:0/3803913057 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f36240b2bc0 msgr2=0x7f36240b5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:15.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:15.062+0000 7f364241e700 1 --2- 172.21.15.2:0/3803913057 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f36240b2bc0 0x7f36240b5080 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f362c00b790 tx=0x7f362c01a040 comp rx=0 tx=0).stop 2024-05-14T17:05:15.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:15.062+0000 7f364241e700 1 -- 172.21.15.2:0/3803913057 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f363c104200 msgr2=0x7f363c10a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:15.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:15.062+0000 7f364241e700 1 --2- 172.21.15.2:0/3803913057 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f363c104200 0x7f363c10a9f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f363000e9f0 tx=0x7f363000ed00 comp rx=0 tx=0).stop 2024-05-14T17:05:15.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:15.062+0000 7f364241e700 1 -- 172.21.15.2:0/3803913057 shutdown_connections 2024-05-14T17:05:15.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:15.062+0000 7f364241e700 1 --2- 172.21.15.2:0/3803913057 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f36240b2bc0 0x7f36240b5080 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:15.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:15.062+0000 7f364241e700 1 --2- 172.21.15.2:0/3803913057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f363c108920 0x7f363c10af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:15.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:15.062+0000 7f364241e700 1 --2- 172.21.15.2:0/3803913057 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f363c103850 0x7f363c10a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:15.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:15.062+0000 7f364241e700 1 --2- 172.21.15.2:0/3803913057 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f363c104200 0x7f363c10a9f0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:15.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:15.062+0000 7f364241e700 1 -- 172.21.15.2:0/3803913057 >> 172.21.15.2:0/3803913057 conn(0x7f363c07a620 msgr2=0x7f363c101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:15.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:15.062+0000 7f364241e700 1 -- 172.21.15.2:0/3803913057 shutdown_connections 2024-05-14T17:05:15.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:15.062+0000 7f364241e700 1 -- 172.21.15.2:0/3803913057 wait complete. 2024-05-14T17:05:15.521 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 44 2024-05-14T17:05:15.715 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:15 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/3803913057' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:05:15.753 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db / --timeout 90 --expiration 137.9 --await-for 45.0' 2024-05-14T17:05:15.801 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db / --timeout 90 --expiration 137.9 --await-for 45.0 2024-05-14T17:05:15.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:15 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/3803913057' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:05:16.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:15 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/3803913057' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:05:16.225 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:05:16.746 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 600.0 s, 2912.4 tps, lat 33.922 ms stddev 38.686 2024-05-14T17:05:16.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:16 smithi123 ceph-mon[35971]: pgmap v328: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 16 MiB/s wr, 1.11k op/s 2024-05-14T17:05:16.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:16 smithi002 ceph-mon[31538]: pgmap v328: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 16 MiB/s wr, 1.11k op/s 2024-05-14T17:05:17.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:16 smithi151 ceph-mon[36066]: pgmap v328: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 16 MiB/s wr, 1.11k op/s 2024-05-14T17:05:17.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.438+0000 7fd51adda640 1 -- 172.21.15.2:0/4291860374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd514108a40 msgr2=0x7fd514108e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.438+0000 7fd51adda640 1 --2- 172.21.15.2:0/4291860374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd514108a40 0x7fd514108e20 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd50800fb40 tx=0x7fd5080329c0 comp rx=0 tx=0).stop 2024-05-14T17:05:17.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.438+0000 7fd51adda640 1 -- 172.21.15.2:0/4291860374 shutdown_connections 2024-05-14T17:05:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.438+0000 7fd51adda640 1 --2- 172.21.15.2:0/4291860374 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5141033f0 0x7fd51410f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.438+0000 7fd51adda640 1 --2- 172.21.15.2:0/4291860374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd514102a30 0x7fd514102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:17.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.438+0000 7fd51adda640 1 --2- 172.21.15.2:0/4291860374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd514108a40 0x7fd514108e20 secure :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd50800fb40 tx=0x7fd5080329c0 comp rx=0 tx=0).stop 2024-05-14T17:05:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.438+0000 7fd51adda640 1 -- 172.21.15.2:0/4291860374 >> 172.21.15.2:0/4291860374 conn(0x7fd5140fe520 msgr2=0x7fd514100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.438+0000 7fd51adda640 1 -- 172.21.15.2:0/4291860374 shutdown_connections 2024-05-14T17:05:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.439+0000 7fd51adda640 1 -- 172.21.15.2:0/4291860374 wait complete. 2024-05-14T17:05:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.440+0000 7fd51adda640 1 Processor -- start 2024-05-14T17:05:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.440+0000 7fd51adda640 1 -- start start 2024-05-14T17:05:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.440+0000 7fd51adda640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd514102a30 0x7fd514197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.440+0000 7fd51adda640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5141033f0 0x7fd5141977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.440+0000 7fd51adda640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd51419bb20 0x7fd51419bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.440+0000 7fd51adda640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd514110b40 con 0x7fd51419bb20 2024-05-14T17:05:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.440+0000 7fd51adda640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd5141109c0 con 0x7fd514102a30 2024-05-14T17:05:17.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.440+0000 7fd51adda640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd514110cc0 con 0x7fd5141033f0 2024-05-14T17:05:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.440+0000 7fd519377640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd51419bb20 0x7fd51419bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.440+0000 7fd519377640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd51419bb20 0x7fd51419bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36468/0 (socket says 172.21.15.2:36468) 2024-05-14T17:05:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.440+0000 7fd519377640 1 -- 172.21.15.2:0/3915358387 learned_addr learned my addr 172.21.15.2:0/3915358387 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:05:17.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.440+0000 7fd518b76640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd514102a30 0x7fd514197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.440+0000 7fd513fff640 1 --2- 172.21.15.2:0/3915358387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5141033f0 0x7fd5141977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.441+0000 7fd519377640 1 -- 172.21.15.2:0/3915358387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5141033f0 msgr2=0x7fd5141977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:17.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.441+0000 7fd519377640 1 --2- 172.21.15.2:0/3915358387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5141033f0 0x7fd5141977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:17.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.441+0000 7fd519377640 1 -- 172.21.15.2:0/3915358387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd514102a30 msgr2=0x7fd514197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.441+0000 7fd519377640 1 --2- 172.21.15.2:0/3915358387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd514102a30 0x7fd514197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:18.033 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.441+0000 7fd519377640 1 -- 172.21.15.2:0/3915358387 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd51419c610 con 0x7fd51419bb20 2024-05-14T17:05:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.441+0000 7fd519377640 1 --2- 172.21.15.2:0/3915358387 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd51419bb20 0x7fd51419bfd0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fd50c00eb10 tx=0x7fd50c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.442+0000 7fd511ffb640 1 -- 172.21.15.2:0/3915358387 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50c00ce20 con 0x7fd51419bb20 2024-05-14T17:05:18.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.442+0000 7fd511ffb640 1 -- 172.21.15.2:0/3915358387 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd50c005020 con 0x7fd51419bb20 2024-05-14T17:05:18.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.442+0000 7fd511ffb640 1 -- 172.21.15.2:0/3915358387 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50c010690 con 0x7fd51419bb20 2024-05-14T17:05:18.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.442+0000 7fd51adda640 1 -- 172.21.15.2:0/3915358387 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd51419c900 con 0x7fd51419bb20 2024-05-14T17:05:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.442+0000 7fd51adda640 1 -- 172.21.15.2:0/3915358387 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5141a40a0 con 0x7fd51419bb20 2024-05-14T17:05:18.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.445+0000 7fd511ffb640 1 -- 172.21.15.2:0/3915358387 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fd50c005510 con 0x7fd51419bb20 2024-05-14T17:05:18.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.445+0000 7fd51dbcb740 1 Processor -- start 2024-05-14T17:05:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.445+0000 7fd51dbcb740 1 -- start start 2024-05-14T17:05:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.445+0000 7fd51dbcb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644f31c08f0 0x5644f31c0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.445+0000 7fd51dbcb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644f31c1210 0x5644f31cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.446+0000 7fd51dbcb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644f31cbad0 0x5644f31cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:18.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.446+0000 7fd51dbcb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5644f3133c30 con 0x5644f31c1210 2024-05-14T17:05:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.446+0000 7fd51dbcb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5644f31237d0 con 0x5644f31c08f0 2024-05-14T17:05:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.446+0000 7fd51dbcb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5644f3127810 con 0x5644f31cbad0 2024-05-14T17:05:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.446+0000 7fd518b76640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644f31c1210 0x5644f31cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.446+0000 7fd518b76640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644f31c1210 0x5644f31cb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36472/0 (socket says 172.21.15.2:36472) 2024-05-14T17:05:18.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.446+0000 7fd518b76640 1 -- 172.21.15.2:0/4174056875 learned_addr learned my addr 172.21.15.2:0/4174056875 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:05:18.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.446+0000 7fd513fff640 1 --2- 172.21.15.2:0/4174056875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644f31c08f0 0x5644f31c0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.446+0000 7fd513fff640 1 --2- 172.21.15.2:0/4174056875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644f31cbad0 0x5644f31cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.446+0000 7fd511ffb640 1 --2- 172.21.15.2:0/3915358387 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fd5040b2a10 0x7fd5040b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.446+0000 7fd518b76640 1 -- 172.21.15.2:0/4174056875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644f31cbad0 msgr2=0x5644f31cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.446+0000 7fd518b76640 1 --2- 172.21.15.2:0/4174056875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644f31cbad0 0x5644f31cdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:18.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.446+0000 7fd518b76640 1 -- 172.21.15.2:0/4174056875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644f31c08f0 msgr2=0x5644f31c0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:18.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.446+0000 7fd518b76640 1 --2- 172.21.15.2:0/4174056875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644f31c08f0 0x5644f31c0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:18.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.446+0000 7fd518b76640 1 -- 172.21.15.2:0/4174056875 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644f2e38210 con 0x5644f31c1210 2024-05-14T17:05:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.446+0000 7fd513fff640 1 --2- 172.21.15.2:0/4174056875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644f31c08f0 0x5644f31c0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:05:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.446+0000 7fd519377640 1 --2- 172.21.15.2:0/3915358387 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fd5040b2a10 0x7fd5040b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:18.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.446+0000 7fd511ffb640 1 -- 172.21.15.2:0/3915358387 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7fd50c0e2bb0 con 0x7fd51419bb20 2024-05-14T17:05:18.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.446+0000 7fd518b76640 1 --2- 172.21.15.2:0/4174056875 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644f31c1210 0x5644f31cb590 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd508005ab0 tx=0x7fd508005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9966d6bdf79fb5d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.447+0000 7fd4feffd640 1 -- 172.21.15.2:0/4174056875 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd508004030 con 0x5644f31c1210 2024-05-14T17:05:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.447+0000 7fd4feffd640 1 -- 172.21.15.2:0/4174056875 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd50800e660 con 0x5644f31c1210 2024-05-14T17:05:18.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.447+0000 7fd4feffd640 1 -- 172.21.15.2:0/4174056875 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd508003850 con 0x5644f31c1210 2024-05-14T17:05:18.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.447+0000 7fd519377640 1 --2- 172.21.15.2:0/3915358387 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fd5040b2a10 0x7fd5040b4ed0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd50c0e03b0 tx=0x7fd50c0767c0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:18.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.448+0000 7fd51dbcb740 1 -- 172.21.15.2:0/4174056875 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644f31c1210 msgr2=0x5644f31cb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:18.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.448+0000 7fd51dbcb740 1 --2- 172.21.15.2:0/4174056875 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644f31c1210 0x5644f31cb590 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd508005ab0 tx=0x7fd508005a20 comp rx=0 tx=0).stop 2024-05-14T17:05:18.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.448+0000 7fd51dbcb740 1 -- 172.21.15.2:0/4174056875 shutdown_connections 2024-05-14T17:05:18.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.448+0000 7fd51dbcb740 1 --2- 172.21.15.2:0/4174056875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644f31cbad0 0x5644f31cdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:18.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.448+0000 7fd51dbcb740 1 --2- 172.21.15.2:0/4174056875 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644f31c1210 0x5644f31cb590 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.448+0000 7fd51dbcb740 1 --2- 172.21.15.2:0/4174056875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644f31c08f0 0x5644f31c0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.448+0000 7fd51dbcb740 1 -- 172.21.15.2:0/4174056875 >> 172.21.15.2:0/4174056875 conn(0x5644f31bce00 msgr2=0x5644f31bd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.448+0000 7fd51dbcb740 1 -- 172.21.15.2:0/4174056875 shutdown_connections 2024-05-14T17:05:18.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.448+0000 7fd51dbcb740 1 -- 172.21.15.2:0/4174056875 wait complete. 2024-05-14T17:05:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.449+0000 7fd51dbcb740 1 -- start start 2024-05-14T17:05:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.449+0000 7fd51dbcb740 1 Processor -- start 2024-05-14T17:05:18.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.449+0000 7fd51dbcb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644f31c08f0 0x5644f31cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.449+0000 7fd51dbcb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644f31c1210 0x5644f31c8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:18.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.449+0000 7fd51dbcb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644f31cbad0 0x5644f31c8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:18.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.449+0000 7fd51dbcb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5644f2e09790 con 0x5644f31c08f0 2024-05-14T17:05:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.449+0000 7fd51dbcb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5644f30df890 con 0x5644f31cbad0 2024-05-14T17:05:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.449+0000 7fd51dbcb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5644f31ce740 con 0x5644f31c1210 2024-05-14T17:05:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.449+0000 7fd51dbcb740 10 client.0 _ll_register_callbacks cb 0x5644f30e2d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:05:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.449+0000 7fd513fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644f31c08f0 0x5644f31cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:18.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.449+0000 7fd513fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644f31c08f0 0x5644f31cb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36480/0 (socket says 172.21.15.2:36480) 2024-05-14T17:05:18.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.449+0000 7fd513fff640 1 -- 172.21.15.2:0/679236253 learned_addr learned my addr 172.21.15.2:0/679236253 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:05:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.449+0000 7fd518b76640 1 --2- 172.21.15.2:0/679236253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644f31c1210 0x5644f31c8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.450+0000 7fd513fff640 1 --2- 172.21.15.2:0/679236253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644f31cbad0 0x5644f31c8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.450+0000 7fd513fff640 1 -- 172.21.15.2:0/679236253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644f31c1210 msgr2=0x5644f31c8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.450+0000 7fd513fff640 1 --2- 172.21.15.2:0/679236253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644f31c1210 0x5644f31c8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.450+0000 7fd513fff640 1 -- 172.21.15.2:0/679236253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644f31cbad0 msgr2=0x5644f31c8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.450+0000 7fd513fff640 1 --2- 172.21.15.2:0/679236253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644f31cbad0 0x5644f31c8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:18.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.450+0000 7fd513fff640 1 -- 172.21.15.2:0/679236253 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644f2f04620 con 0x5644f31c08f0 2024-05-14T17:05:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.450+0000 7fd513fff640 1 --2- 172.21.15.2:0/679236253 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644f31c08f0 0x5644f31cb4a0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd50001ac50 tx=0x7fd500009470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc45f246f735cff5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.450+0000 7fd4f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:05:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.451+0000 7fd4f3fff640 1 -- 172.21.15.2:0/679236253 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50001ae00 con 0x5644f31c08f0 2024-05-14T17:05:18.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.451+0000 7fd4f3fff640 1 -- 172.21.15.2:0/679236253 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7fd500015e00 con 0x5644f31c08f0 2024-05-14T17:05:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.451+0000 7fd51dbcb740 1 -- 172.21.15.2:0/679236253 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd50800e660 con 0x5644f31c08f0 2024-05-14T17:05:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.451+0000 7fd4f3fff640 1 -- 172.21.15.2:0/679236253 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fd50002a050 con 0x5644f31c08f0 2024-05-14T17:05:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.451+0000 7fd51dbcb740 10 client.16452 fetch_fsmap learned FSMap version 44 2024-05-14T17:05:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.451+0000 7fd51dbcb740 1 -- 172.21.15.2:0/679236253 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5644f31d9fb0 con 0x5644f31c08f0 2024-05-14T17:05:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.451+0000 7fd4f3fff640 1 -- 172.21.15.2:0/679236253 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd50800e660 con 0x5644f31c08f0 2024-05-14T17:05:19.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.452+0000 7fd4f3fff640 1 -- 172.21.15.2:0/679236253 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 44) ==== 8350+0+0 (secure 0 0 0) 0x7fd500018b30 con 0x5644f31c08f0 2024-05-14T17:05:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.452+0000 7fd51dbcb740 10 client.16452 fetch_fsmap finished waiting for FSMap version 44 2024-05-14T17:05:19.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.452+0000 7fd51dbcb740 10 client.16452 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:05:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.452+0000 7fd51dbcb740 20 client.16452 populate_metadata read hostname 'smithi002' 2024-05-14T17:05:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.452+0000 7fd51dbcb740 1 --2- 172.21.15.2:0/679236253 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5644f31db470 0x5644f31dd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.452+0000 7fd51dbcb740 4 client.16452 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:05:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.452+0000 7fd51dbcb740 1 -- 172.21.15.2:0/679236253 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644f3135b80 con 0x5644f31db470 2024-05-14T17:05:19.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.452+0000 7fd518b76640 1 --2- 172.21.15.2:0/679236253 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5644f31db470 0x5644f31dd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.452+0000 7fd4f3fff640 10 client.16452 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:05:19.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.452+0000 7fd518b76640 1 --2- 172.21.15.2:0/679236253 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5644f31db470 0x5644f31dd860 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8b1cd3dbf6f8d19 server_cookie=4c50c82265c4d65a in_seq=0 out_seq=0 2024-05-14T17:05:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.454+0000 7fd4f3fff640 1 -- 172.21.15.2:0/679236253 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5644f3135b80 con 0x5644f31db470 2024-05-14T17:05:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.454+0000 7fd4f3fff640 10 client.16452 handle_command_reply: tid=0 2024-05-14T17:05:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.454+0000 7fd51dbcb740 1 client.16452 shutdown 2024-05-14T17:05:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.454+0000 7fd51dbcb740 1 -- 172.21.15.2:0/679236253 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644f31c08f0 msgr2=0x5644f31cb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:19.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.454+0000 7fd51dbcb740 1 --2- 172.21.15.2:0/679236253 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644f31c08f0 0x5644f31cb4a0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd50001ac50 tx=0x7fd500009470 comp rx=0 tx=0).stop 2024-05-14T17:05:19.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.454+0000 7fd51dbcb740 1 -- 172.21.15.2:0/679236253 shutdown_connections 2024-05-14T17:05:19.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.454+0000 7fd51dbcb740 1 --2- 172.21.15.2:0/679236253 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5644f31db470 0x5644f31dd860 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.454+0000 7fd51dbcb740 1 --2- 172.21.15.2:0/679236253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644f31cbad0 0x5644f31c8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.454+0000 7fd51dbcb740 1 --2- 172.21.15.2:0/679236253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644f31c1210 0x5644f31c8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.454+0000 7fd51dbcb740 1 --2- 172.21.15.2:0/679236253 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644f31c08f0 0x5644f31cb4a0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:19.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.454+0000 7fd51dbcb740 1 -- 172.21.15.2:0/679236253 >> 172.21.15.2:0/679236253 conn(0x5644f31bce00 msgr2=0x5644f31bc100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:19.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.454+0000 7fd4f3fff640 0 client.16452 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:05:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.455+0000 7fd51dbcb740 1 -- 172.21.15.2:0/679236253 shutdown_connections 2024-05-14T17:05:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.455+0000 7fd51dbcb740 1 -- 172.21.15.2:0/679236253 wait complete. 2024-05-14T17:05:19.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.455+0000 7fd51dbcb740 20 client.16452 trim_cache size 0 max 16384 2024-05-14T17:05:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.474+0000 7fd51dbcb740 1 Processor -- start 2024-05-14T17:05:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.474+0000 7fd51dbcb740 1 -- start start 2024-05-14T17:05:19.403 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.475+0000 7fd51dbcb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644f31c08f0 0x5644f31bc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:19.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.475+0000 7fd51dbcb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644f31c1210 0x5644f31cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:19.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.475+0000 7fd51dbcb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644f31cbad0 0x5644f31c3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:19.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.475+0000 7fd51dbcb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd5141af3e0 con 0x5644f31c08f0 2024-05-14T17:05:19.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.475+0000 7fd51dbcb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd5141acc50 con 0x5644f31cbad0 2024-05-14T17:05:19.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.475+0000 7fd51dbcb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd5141aaac0 con 0x5644f31c1210 2024-05-14T17:05:19.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.475+0000 7fd513fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644f31c08f0 0x5644f31bc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.475+0000 7fd513fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644f31c08f0 0x5644f31bc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36486/0 (socket says 172.21.15.2:36486) 2024-05-14T17:05:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.475+0000 7fd513fff640 1 -- 172.21.15.2:0/2640912242 learned_addr learned my addr 172.21.15.2:0/2640912242 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:05:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.475+0000 7fd518b76640 1 --2- 172.21.15.2:0/2640912242 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644f31c1210 0x5644f31cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.475+0000 7fd513fff640 1 --2- 172.21.15.2:0/2640912242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644f31cbad0 0x5644f31c3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.475+0000 7fd518b76640 1 -- 172.21.15.2:0/2640912242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644f31cbad0 msgr2=0x5644f31c3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.475+0000 7fd518b76640 1 --2- 172.21.15.2:0/2640912242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644f31cbad0 0x5644f31c3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.475+0000 7fd518b76640 1 -- 172.21.15.2:0/2640912242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644f31c08f0 msgr2=0x5644f31bc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.475+0000 7fd518b76640 1 --2- 172.21.15.2:0/2640912242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644f31c08f0 0x5644f31bc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.475+0000 7fd518b76640 1 -- 172.21.15.2:0/2640912242 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644f31bc930 con 0x5644f31c1210 2024-05-14T17:05:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.475+0000 7fd513fff640 1 --2- 172.21.15.2:0/2640912242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644f31c08f0 0x5644f31bc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:05:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.475+0000 7fd518b76640 1 --2- 172.21.15.2:0/2640912242 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644f31c1210 0x5644f31cafd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd508006010 tx=0x7fd508000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21b419d0e749232 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.476+0000 7fd4fe7fc640 1 -- 172.21.15.2:0/2640912242 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50805e070 con 0x5644f31c1210 2024-05-14T17:05:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.476+0000 7fd4fe7fc640 1 -- 172.21.15.2:0/2640912242 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd50800f5e0 con 0x5644f31c1210 2024-05-14T17:05:19.696 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.476+0000 7fd4fe7fc640 1 -- 172.21.15.2:0/2640912242 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd508059070 con 0x5644f31c1210 2024-05-14T17:05:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.477+0000 7fd51dbcb740 1 -- 172.21.15.2:0/2640912242 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644f31c1210 msgr2=0x5644f31cafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.477+0000 7fd51dbcb740 1 --2- 172.21.15.2:0/2640912242 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644f31c1210 0x5644f31cafd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd508006010 tx=0x7fd508000bc0 comp rx=0 tx=0).stop 2024-05-14T17:05:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.477+0000 7fd51dbcb740 1 -- 172.21.15.2:0/2640912242 shutdown_connections 2024-05-14T17:05:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.477+0000 7fd51dbcb740 1 --2- 172.21.15.2:0/2640912242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644f31cbad0 0x5644f31c3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:19.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.477+0000 7fd51dbcb740 1 --2- 172.21.15.2:0/2640912242 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644f31c1210 0x5644f31cafd0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:19.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.477+0000 7fd51dbcb740 1 --2- 172.21.15.2:0/2640912242 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644f31c08f0 0x5644f31bc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.477+0000 7fd51dbcb740 1 -- 172.21.15.2:0/2640912242 >> 172.21.15.2:0/2640912242 conn(0x5644f31bce00 msgr2=0x5644f31c3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:19.788 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.477+0000 7fd51dbcb740 1 -- 172.21.15.2:0/2640912242 shutdown_connections 2024-05-14T17:05:19.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.477+0000 7fd51dbcb740 1 -- 172.21.15.2:0/2640912242 wait complete. 2024-05-14T17:05:19.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.477+0000 7fd51dbcb740 1 -- start start 2024-05-14T17:05:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.477+0000 7fd51dbcb740 1 Processor -- start 2024-05-14T17:05:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.478+0000 7fd51dbcb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644f31c08f0 0x5644f31cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.478+0000 7fd51dbcb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644f31c1210 0x5644f31cfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:19.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.478+0000 7fd51dbcb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644f31cbad0 0x5644f31d0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.478+0000 7fd51dbcb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd5141ab110 con 0x5644f31cbad0 2024-05-14T17:05:19.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.478+0000 7fd51dbcb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd5141ac580 con 0x5644f31c1210 2024-05-14T17:05:19.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.478+0000 7fd51dbcb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd5141ab7e0 con 0x5644f31c08f0 2024-05-14T17:05:19.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.478+0000 7fd51dbcb740 10 client.0 _ll_register_callbacks cb 0x7fd50001a890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:05:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.478+0000 7fd513fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644f31cbad0 0x5644f31d0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.478+0000 7fd513fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644f31c08f0 0x5644f31cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.478+0000 7fd518b76640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644f31c1210 0x5644f31cfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.478+0000 7fd513fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644f31c08f0 0x5644f31cf5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:60084/0 (socket says 172.21.15.2:60084) 2024-05-14T17:05:20.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.478+0000 7fd518b76640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644f31c1210 0x5644f31cfaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:55780/0 (socket says 172.21.15.2:55780) 2024-05-14T17:05:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.478+0000 7fd513fff640 1 -- 172.21.15.2:0/2563148486 learned_addr learned my addr 172.21.15.2:0/2563148486 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:05:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.479+0000 7fd513fff640 1 -- 172.21.15.2:0/2563148486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644f31c1210 msgr2=0x5644f31cfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.479+0000 7fd513fff640 1 --2- 172.21.15.2:0/2563148486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644f31c1210 0x5644f31cfaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:20.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.479+0000 7fd513fff640 1 -- 172.21.15.2:0/2563148486 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644f31cbad0 msgr2=0x5644f31d0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.479+0000 7fd513fff640 1 --2- 172.21.15.2:0/2563148486 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644f31cbad0 0x5644f31d0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.479+0000 7fd513fff640 1 -- 172.21.15.2:0/2563148486 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644f3244d30 con 0x5644f31c08f0 2024-05-14T17:05:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.479+0000 7fd513fff640 1 --2- 172.21.15.2:0/2563148486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644f31c08f0 0x5644f31cf5b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd500006b90 tx=0x7fd500025770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed308ad46716a45c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:20.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.479+0000 7fd4f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-14T17:05:20.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.479+0000 7fd4f3fff640 1 -- 172.21.15.2:0/2563148486 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50001a2e0 con 0x5644f31c08f0 2024-05-14T17:05:20.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.479+0000 7fd4f3fff640 1 -- 172.21.15.2:0/2563148486 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7fd500025930 con 0x5644f31c08f0 2024-05-14T17:05:20.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.479+0000 7fd51dbcb740 1 -- 172.21.15.2:0/2563148486 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5644f312f360 con 0x5644f31c08f0 2024-05-14T17:05:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.480+0000 7fd4f3fff640 1 -- 172.21.15.2:0/2563148486 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fd50003e050 con 0x5644f31c08f0 2024-05-14T17:05:20.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.480+0000 7fd51dbcb740 10 client.26116 fetch_fsmap learned FSMap version 44 2024-05-14T17:05:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.481+0000 7fd51dbcb740 1 -- 172.21.15.2:0/2563148486 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5644f3246020 con 0x5644f31c08f0 2024-05-14T17:05:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.481+0000 7fd4f3fff640 1 -- 172.21.15.2:0/2563148486 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5644f312f360 con 0x5644f31c08f0 2024-05-14T17:05:20.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.481+0000 7fd4f3fff640 1 -- 172.21.15.2:0/2563148486 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 44) ==== 8350+0+0 (secure 0 0 0) 0x7fd500018a90 con 0x5644f31c08f0 2024-05-14T17:05:20.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.481+0000 7fd51dbcb740 10 client.26116 fetch_fsmap finished waiting for FSMap version 44 2024-05-14T17:05:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.481+0000 7fd51dbcb740 10 client.26116 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:05:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.481+0000 7fd51dbcb740 20 client.26116 populate_metadata read hostname 'smithi002' 2024-05-14T17:05:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.481+0000 7fd51dbcb740 1 --2- 172.21.15.2:0/2563148486 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5644f3246620 0x5644f3248a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.481+0000 7fd51dbcb740 4 client.26116 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:05:20.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.481+0000 7fd51dbcb740 1 -- 172.21.15.2:0/2563148486 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5644f30a95f0 con 0x5644f3246620 2024-05-14T17:05:20.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.481+0000 7fd518b76640 1 --2- 172.21.15.2:0/2563148486 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5644f3246620 0x5644f3248a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.482+0000 7fd518b76640 1 --2- 172.21.15.2:0/2563148486 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5644f3246620 0x5644f3248a40 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8ab38c2df8ceb92 server_cookie=32b3c597261419f4 in_seq=0 out_seq=0 2024-05-14T17:05:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.482+0000 7fd4f3fff640 10 client.26116 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:05:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.496+0000 7fd4f3fff640 1 -- 172.21.15.2:0/2563148486 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5644f30a95f0 con 0x5644f3246620 2024-05-14T17:05:20.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.496+0000 7fd4f3fff640 10 client.26116 handle_command_reply: tid=0 2024-05-14T17:05:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.496+0000 7fd51dbcb740 1 client.26116 shutdown 2024-05-14T17:05:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.497+0000 7fd51dbcb740 1 -- 172.21.15.2:0/2563148486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644f31c08f0 msgr2=0x5644f31cf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.497+0000 7fd51dbcb740 1 --2- 172.21.15.2:0/2563148486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644f31c08f0 0x5644f31cf5b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd500006b90 tx=0x7fd500025770 comp rx=0 tx=0).stop 2024-05-14T17:05:20.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.497+0000 7fd51dbcb740 1 -- 172.21.15.2:0/2563148486 shutdown_connections 2024-05-14T17:05:20.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.497+0000 7fd51dbcb740 1 --2- 172.21.15.2:0/2563148486 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5644f3246620 0x5644f3248a40 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:20.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.497+0000 7fd51dbcb740 1 --2- 172.21.15.2:0/2563148486 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644f31cbad0 0x5644f31d0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:20.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.497+0000 7fd51dbcb740 1 --2- 172.21.15.2:0/2563148486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644f31c1210 0x5644f31cfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.497+0000 7fd4f3fff640 0 client.26116 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:05:20.384 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.497+0000 7fd51dbcb740 1 --2- 172.21.15.2:0/2563148486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644f31c08f0 0x5644f31cf5b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:20.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.497+0000 7fd51dbcb740 1 -- 172.21.15.2:0/2563148486 >> 172.21.15.2:0/2563148486 conn(0x5644f31bce00 msgr2=0x5644f31c2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.498+0000 7fd51dbcb740 1 -- 172.21.15.2:0/2563148486 shutdown_connections 2024-05-14T17:05:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.498+0000 7fd51dbcb740 1 -- 172.21.15.2:0/2563148486 wait complete. 2024-05-14T17:05:20.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.498+0000 7fd51dbcb740 20 client.26116 trim_cache size 0 max 16384 2024-05-14T17:05:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:05:20.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:05:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:05:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.498+0000 7fd51adda640 1 -- 172.21.15.2:0/3915358387 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fd5040b2a10 msgr2=0x7fd5040b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:20.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.498+0000 7fd51adda640 1 --2- 172.21.15.2:0/3915358387 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fd5040b2a10 0x7fd5040b4ed0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd50c0e03b0 tx=0x7fd50c0767c0 comp rx=0 tx=0).stop 2024-05-14T17:05:20.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.498+0000 7fd51adda640 1 -- 172.21.15.2:0/3915358387 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd51419bb20 msgr2=0x7fd51419bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:20.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.498+0000 7fd51adda640 1 --2- 172.21.15.2:0/3915358387 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd51419bb20 0x7fd51419bfd0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fd50c00eb10 tx=0x7fd50c00ef10 comp rx=0 tx=0).stop 2024-05-14T17:05:20.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.498+0000 7fd51adda640 1 -- 172.21.15.2:0/3915358387 shutdown_connections 2024-05-14T17:05:20.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.498+0000 7fd51adda640 1 --2- 172.21.15.2:0/3915358387 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fd5040b2a10 0x7fd5040b4ed0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.498+0000 7fd51adda640 1 --2- 172.21.15.2:0/3915358387 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd51419bb20 0x7fd51419bfd0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.498+0000 7fd51adda640 1 --2- 172.21.15.2:0/3915358387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5141033f0 0x7fd5141977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.498+0000 7fd51adda640 1 --2- 172.21.15.2:0/3915358387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd514102a30 0x7fd514197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.498+0000 7fd51adda640 1 -- 172.21.15.2:0/3915358387 >> 172.21.15.2:0/3915358387 conn(0x7fd5140fe520 msgr2=0x7fd5140fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:20.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.498+0000 7fd51adda640 1 -- 172.21.15.2:0/3915358387 shutdown_connections 2024-05-14T17:05:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:17.499+0000 7fd51adda640 1 -- 172.21.15.2:0/3915358387 wait complete. 2024-05-14T17:05:20.629 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:18 smithi151 ceph-mon[36066]: pgmap v329: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.1 MiB/s wr, 696 op/s 2024-05-14T17:05:20.639 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:18 smithi123 ceph-mon[35971]: pgmap v329: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.1 MiB/s wr, 696 op/s 2024-05-14T17:05:20.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.259+0000 7f833ec2f700 1 -- 172.21.15.2:0/1893080886 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8338105fd0 msgr2=0x7f8338106480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:20.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.259+0000 7f833ec2f700 1 --2- 172.21.15.2:0/1893080886 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8338105fd0 0x7f8338106480 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f833001c0d0 tx=0x7f833001c3e0 comp rx=0 tx=0).stop 2024-05-14T17:05:20.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.260+0000 7f833ec2f700 1 -- 172.21.15.2:0/1893080886 shutdown_connections 2024-05-14T17:05:20.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.260+0000 7f833ec2f700 1 --2- 172.21.15.2:0/1893080886 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8338105fd0 0x7f8338106480 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:20.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.260+0000 7f833ec2f700 1 --2- 172.21.15.2:0/1893080886 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8338108560 0x7f833806bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:20.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.260+0000 7f833ec2f700 1 --2- 172.21.15.2:0/1893080886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83381089d0 0x7f833806c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:20.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.260+0000 7f833ec2f700 1 -- 172.21.15.2:0/1893080886 >> 172.21.15.2:0/1893080886 conn(0x7f83380793f0 msgr2=0x7f8338079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:20.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.261+0000 7f833ec2f700 1 -- 172.21.15.2:0/1893080886 shutdown_connections 2024-05-14T17:05:20.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.261+0000 7f833ec2f700 1 -- 172.21.15.2:0/1893080886 wait complete. 2024-05-14T17:05:20.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.262+0000 7f833ec2f700 1 Processor -- start 2024-05-14T17:05:20.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.262+0000 7f833ec2f700 1 -- start start 2024-05-14T17:05:20.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.262+0000 7f833ec2f700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8338105fd0 0x7f833810efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:20.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.262+0000 7f833ec2f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8338108560 0x7f833810f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:20.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.262+0000 7f833ec2f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83381089d0 0x7f8338116d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:20.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.263+0000 7f833ec2f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f833810b4d0 con 0x7f8338108560 2024-05-14T17:05:20.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.263+0000 7f833ec2f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f833810b350 con 0x7f83381089d0 2024-05-14T17:05:20.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.263+0000 7f833ec2f700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f833810b650 con 0x7f8338105fd0 2024-05-14T17:05:20.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.263+0000 7f8337fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8338108560 0x7f833810f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:20.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.263+0000 7f8337fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8338108560 0x7f833810f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36508/0 (socket says 172.21.15.2:36508) 2024-05-14T17:05:20.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.263+0000 7f8337fff700 1 -- 172.21.15.2:0/1064358236 learned_addr learned my addr 172.21.15.2:0/1064358236 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:05:20.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.263+0000 7f833d1cc700 1 --2- 172.21.15.2:0/1064358236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83381089d0 0x7f8338116d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:20.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.263+0000 7f833c9cb700 1 --2- 172.21.15.2:0/1064358236 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8338105fd0 0x7f833810efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:20.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.263+0000 7f8337fff700 1 -- 172.21.15.2:0/1064358236 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8338105fd0 msgr2=0x7f833810efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:21.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.263+0000 7f8337fff700 1 --2- 172.21.15.2:0/1064358236 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8338105fd0 0x7f833810efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:21.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.263+0000 7f8337fff700 1 -- 172.21.15.2:0/1064358236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83381089d0 msgr2=0x7f8338116d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:21.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.263+0000 7f8337fff700 1 --2- 172.21.15.2:0/1064358236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83381089d0 0x7f8338116d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:21.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.263+0000 7f8337fff700 1 -- 172.21.15.2:0/1064358236 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8338073c70 con 0x7f8338108560 2024-05-14T17:05:21.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.264+0000 7f833d1cc700 1 --2- 172.21.15.2:0/1064358236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83381089d0 0x7f8338116d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:05:21.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.264+0000 7f8337fff700 1 --2- 172.21.15.2:0/1064358236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8338108560 0x7f833810f520 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f832800e940 tx=0x7f832800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:21.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.264+0000 7f8335ffb700 1 -- 172.21.15.2:0/1064358236 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832800cbd0 con 0x7f8338108560 2024-05-14T17:05:21.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.264+0000 7f8335ffb700 1 -- 172.21.15.2:0/1064358236 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f832800cd70 con 0x7f8338108560 2024-05-14T17:05:21.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.264+0000 7f8335ffb700 1 -- 172.21.15.2:0/1064358236 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8328018680 con 0x7f8338108560 2024-05-14T17:05:21.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.264+0000 7f833ec2f700 1 -- 172.21.15.2:0/1064358236 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8338073f60 con 0x7f8338108560 2024-05-14T17:05:21.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.265+0000 7f833ec2f700 1 -- 172.21.15.2:0/1064358236 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8338074450 con 0x7f8338108560 2024-05-14T17:05:21.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.270+0000 7f83494c5b80 1 Processor -- start 2024-05-14T17:05:21.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.270+0000 7f83494c5b80 1 -- start start 2024-05-14T17:05:21.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.271+0000 7f83494c5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba046ee4b0 0x55ba046ee890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:21.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.271+0000 7f83494c5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba046eedd0 0x55ba046e9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:21.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.271+0000 7f83494c5b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba046ea350 0x55ba046ea7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:21.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.271+0000 7f83494c5b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ba04647f00 con 0x55ba046ea350 2024-05-14T17:05:21.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.271+0000 7f833d1cc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba046ee4b0 0x55ba046ee890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:21.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.271+0000 7f833d1cc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba046ee4b0 0x55ba046ee890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:60112/0 (socket says 172.21.15.2:60112) 2024-05-14T17:05:21.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.271+0000 7f833d1cc700 1 -- 172.21.15.2:0/1532152786 learned_addr learned my addr 172.21.15.2:0/1532152786 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:05:21.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.271+0000 7f83494c5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ba04637aa0 con 0x55ba046eedd0 2024-05-14T17:05:21.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.271+0000 7f83494c5b80 1 -- 172.21.15.2:0/1532152786 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ba0463bae0 con 0x55ba046ee4b0 2024-05-14T17:05:21.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.271+0000 7f8337fff700 1 --2- 172.21.15.2:0/1532152786 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba046ea350 0x55ba046ea7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:21.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.271+0000 7f833c9cb700 1 --2- 172.21.15.2:0/1532152786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba046eedd0 0x55ba046e9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:21.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.272+0000 7f833c9cb700 1 -- 172.21.15.2:0/1532152786 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba046ee4b0 msgr2=0x55ba046ee890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:21.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.272+0000 7f833c9cb700 1 --2- 172.21.15.2:0/1532152786 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba046ee4b0 0x55ba046ee890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:21.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.272+0000 7f833c9cb700 1 -- 172.21.15.2:0/1532152786 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba046ea350 msgr2=0x55ba046ea7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:21.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.272+0000 7f833c9cb700 1 --2- 172.21.15.2:0/1532152786 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba046ea350 0x55ba046ea7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:21.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.272+0000 7f833c9cb700 1 -- 172.21.15.2:0/1532152786 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba045e1140 con 0x55ba046eedd0 2024-05-14T17:05:21.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.272+0000 7f833d1cc700 1 --2- 172.21.15.2:0/1532152786 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba046ee4b0 0x55ba046ee890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:05:21.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.272+0000 7f833c9cb700 1 --2- 172.21.15.2:0/1532152786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba046eedd0 0x55ba046e9d30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f832c009b40 tx=0x7f832c009e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c213dbf9c3519a2f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:21.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.273+0000 7f831effd700 1 -- 172.21.15.2:0/1532152786 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c016070 con 0x55ba046eedd0 2024-05-14T17:05:21.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.273+0000 7f831effd700 1 -- 172.21.15.2:0/1532152786 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f832c005020 con 0x55ba046eedd0 2024-05-14T17:05:21.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.273+0000 7f831effd700 1 -- 172.21.15.2:0/1532152786 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c00ec60 con 0x55ba046eedd0 2024-05-14T17:05:21.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.274+0000 7f83494c5b80 1 -- 172.21.15.2:0/1532152786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba046eedd0 msgr2=0x55ba046e9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:21.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.274+0000 7f83494c5b80 1 --2- 172.21.15.2:0/1532152786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba046eedd0 0x55ba046e9d30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f832c009b40 tx=0x7f832c009e50 comp rx=0 tx=0).stop 2024-05-14T17:05:21.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.274+0000 7f83494c5b80 1 -- 172.21.15.2:0/1532152786 shutdown_connections 2024-05-14T17:05:21.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.274+0000 7f83494c5b80 1 --2- 172.21.15.2:0/1532152786 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba046ea350 0x55ba046ea7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:21.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.274+0000 7f83494c5b80 1 --2- 172.21.15.2:0/1532152786 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba046ee4b0 0x55ba046ee890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:21.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.274+0000 7f83494c5b80 1 --2- 172.21.15.2:0/1532152786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba046eedd0 0x55ba046e9d30 secure :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f832c009b40 tx=0x7f832c009e50 comp rx=0 tx=0).stop 2024-05-14T17:05:21.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.274+0000 7f83494c5b80 1 -- 172.21.15.2:0/1532152786 >> 172.21.15.2:0/1532152786 conn(0x55ba04616f40 msgr2=0x55ba046e4530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:21.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.275+0000 7f83494c5b80 1 -- 172.21.15.2:0/1532152786 shutdown_connections 2024-05-14T17:05:21.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.275+0000 7f83494c5b80 1 -- 172.21.15.2:0/1532152786 wait complete. 2024-05-14T17:05:21.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.275+0000 7f8335ffb700 1 -- 172.21.15.2:0/1064358236 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f8328010430 con 0x7f8338108560 2024-05-14T17:05:21.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.275+0000 7f83494c5b80 1 -- start start 2024-05-14T17:05:21.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.275+0000 7f83494c5b80 1 Processor -- start 2024-05-14T17:05:21.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.276+0000 7f83494c5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba046ea350 0x55ba046f5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:21.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.276+0000 7f83494c5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba046ee4b0 0x55ba046f3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:21.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.276+0000 7f83494c5b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba046f3760 0x55ba046f3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:21.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.277+0000 7f83494c5b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ba0453e130 con 0x55ba046f3760 2024-05-14T17:05:21.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.277+0000 7f833d1cc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba046ea350 0x55ba046f5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:21.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.277+0000 7f833d1cc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba046ea350 0x55ba046f5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:55814/0 (socket says 172.21.15.2:55814) 2024-05-14T17:05:21.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.277+0000 7f833d1cc700 1 -- 172.21.15.2:0/4260426884 learned_addr learned my addr 172.21.15.2:0/4260426884 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:05:21.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.277+0000 7f83494c5b80 1 -- 172.21.15.2:0/4260426884 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ba0459ff30 con 0x55ba046ea350 2024-05-14T17:05:22.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.277+0000 7f833c9cb700 1 --2- 172.21.15.2:0/4260426884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba046ee4b0 0x55ba046f3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:22.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.277+0000 7f8337fff700 1 --2- 172.21.15.2:0/4260426884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba046f3760 0x55ba046f3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:22.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.277+0000 7f83494c5b80 1 -- 172.21.15.2:0/4260426884 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ba046f6910 con 0x55ba046ee4b0 2024-05-14T17:05:22.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.277+0000 7f83494c5b80 10 client.0 _ll_register_callbacks cb 0x55ba045d2ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:05:22.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.277+0000 7f833d1cc700 1 -- 172.21.15.2:0/4260426884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba046ee4b0 msgr2=0x55ba046f3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:22.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.277+0000 7f833d1cc700 1 --2- 172.21.15.2:0/4260426884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba046ee4b0 0x55ba046f3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:22.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.277+0000 7f833d1cc700 1 -- 172.21.15.2:0/4260426884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba046f3760 msgr2=0x55ba046f3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:22.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.277+0000 7f833d1cc700 1 --2- 172.21.15.2:0/4260426884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba046f3760 0x55ba046f3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:22.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.277+0000 7f833d1cc700 1 -- 172.21.15.2:0/4260426884 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba04587a60 con 0x55ba046ea350 2024-05-14T17:05:22.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.277+0000 7f8337fff700 1 --2- 172.21.15.2:0/4260426884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba046f3760 0x55ba046f3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:05:22.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.278+0000 7f833d1cc700 1 --2- 172.21.15.2:0/4260426884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba046ea350 0x55ba046f5e10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f833001cec0 tx=0x7f833001cef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bf2e05ebd0791e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:22.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.278+0000 7f8317fff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T17:05:22.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.278+0000 7f8335ffb700 1 --2- 172.21.15.2:0/1064358236 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f83200b2b00 0x7f83200b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:22.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.278+0000 7f8317fff700 1 -- 172.21.15.2:0/4260426884 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83300184f0 con 0x55ba046ea350 2024-05-14T17:05:22.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.278+0000 7f8317fff700 1 -- 172.21.15.2:0/4260426884 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f8330018690 con 0x55ba046ea350 2024-05-14T17:05:23.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.278+0000 7f83494c5b80 1 -- 172.21.15.2:0/4260426884 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f832c005020 con 0x55ba046ea350 2024-05-14T17:05:23.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.278+0000 7f833c9cb700 1 --2- 172.21.15.2:0/1064358236 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f83200b2b00 0x7f83200b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:23.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.278+0000 7f8317fff700 1 -- 172.21.15.2:0/4260426884 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f83300188d0 con 0x55ba046ea350 2024-05-14T17:05:23.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.278+0000 7f83494c5b80 10 client.26144 fetch_fsmap learned FSMap version 44 2024-05-14T17:05:23.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.278+0000 7f83494c5b80 1 -- 172.21.15.2:0/4260426884 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ba045f18f0 con 0x55ba046ea350 2024-05-14T17:05:23.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.278+0000 7f8335ffb700 1 -- 172.21.15.2:0/1064358236 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f83280e8320 con 0x7f8338108560 2024-05-14T17:05:23.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.278+0000 7f8317fff700 1 -- 172.21.15.2:0/4260426884 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f832c005020 con 0x55ba046ea350 2024-05-14T17:05:23.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.279+0000 7f8317fff700 1 -- 172.21.15.2:0/4260426884 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 44) ==== 8350+0+0 (secure 0 0 0) 0x7f8330022070 con 0x55ba046ea350 2024-05-14T17:05:23.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.279+0000 7f83494c5b80 10 client.26144 fetch_fsmap finished waiting for FSMap version 44 2024-05-14T17:05:23.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.279+0000 7f83494c5b80 10 client.26144 resolve_mds: validated gid 24479 aka daemon mds.b 2024-05-14T17:05:23.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.279+0000 7f83494c5b80 20 client.26144 populate_metadata read hostname 'smithi002' 2024-05-14T17:05:24.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.279+0000 7f83494c5b80 1 --2- 172.21.15.2:0/4260426884 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ba04703ae0 0x55ba04705f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:24.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.279+0000 7f83494c5b80 4 client.26144 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:05:24.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.279+0000 7f83494c5b80 1 -- 172.21.15.2:0/4260426884 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba04649e50 con 0x55ba04703ae0 2024-05-14T17:05:24.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.279+0000 7f8337fff700 1 --2- 172.21.15.2:0/4260426884 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ba04703ae0 0x55ba04705f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:24.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.279+0000 7f833c9cb700 1 --2- 172.21.15.2:0/1064358236 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f83200b2b00 0x7f83200b4fc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f832c000c00 tx=0x7f832c01c040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:24.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.280+0000 7f8337fff700 1 --2- 172.21.15.2:0/4260426884 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ba04703ae0 0x55ba04705f10 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96709ae71bd633d server_cookie=e91c337f5084a0f5 in_seq=0 out_seq=0 2024-05-14T17:05:24.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.280+0000 7f8317fff700 10 client.26144 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:05:24.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.281+0000 7f8317fff700 1 -- 172.21.15.2:0/4260426884 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ba04649e50 con 0x55ba04703ae0 2024-05-14T17:05:24.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.281+0000 7f8317fff700 10 client.26144 handle_command_reply: tid=0 2024-05-14T17:05:24.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.281+0000 7f83494c5b80 1 client.26144 shutdown 2024-05-14T17:05:24.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.281+0000 7f83494c5b80 1 -- 172.21.15.2:0/4260426884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba046ea350 msgr2=0x55ba046f5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:24.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.281+0000 7f83494c5b80 1 --2- 172.21.15.2:0/4260426884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba046ea350 0x55ba046f5e10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f833001cec0 tx=0x7f833001cef0 comp rx=0 tx=0).stop 2024-05-14T17:05:24.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.281+0000 7f83494c5b80 1 -- 172.21.15.2:0/4260426884 shutdown_connections 2024-05-14T17:05:25.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.281+0000 7f83494c5b80 1 --2- 172.21.15.2:0/4260426884 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ba04703ae0 0x55ba04705f10 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:25.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.281+0000 7f83494c5b80 1 --2- 172.21.15.2:0/4260426884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba046f3760 0x55ba046f3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:25.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.281+0000 7f83494c5b80 1 --2- 172.21.15.2:0/4260426884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba046ea350 0x55ba046f5e10 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:25.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.281+0000 7f83494c5b80 1 --2- 172.21.15.2:0/4260426884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba046ee4b0 0x55ba046f3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:25.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.281+0000 7f83494c5b80 1 -- 172.21.15.2:0/4260426884 >> 172.21.15.2:0/4260426884 conn(0x55ba04616f40 msgr2=0x55ba046df050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:25.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.281+0000 7f8317fff700 0 client.26144 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:05:25.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.282+0000 7f83494c5b80 1 -- 172.21.15.2:0/4260426884 shutdown_connections 2024-05-14T17:05:25.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.282+0000 7f83494c5b80 1 -- 172.21.15.2:0/4260426884 wait complete. 2024-05-14T17:05:25.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.282+0000 7f83494c5b80 20 client.26144 trim_cache size 0 max 16384 2024-05-14T17:05:25.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.310+0000 7f83494c5b80 1 Processor -- start 2024-05-14T17:05:25.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.310+0000 7f83494c5b80 1 -- start start 2024-05-14T17:05:25.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.311+0000 7f83494c5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba046ea350 0x55ba046e9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:25.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.311+0000 7f83494c5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba046ee4b0 0x55ba046df160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:25.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.311+0000 7f83494c5b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba046f3760 0x55ba0470b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:26.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.311+0000 7f83494c5b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f83381b6df0 con 0x55ba046f3760 2024-05-14T17:05:26.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.311+0000 7f83494c5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f83381b4620 con 0x55ba046ea350 2024-05-14T17:05:26.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.311+0000 7f83494c5b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f83381b24e0 con 0x55ba046ee4b0 2024-05-14T17:05:26.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.311+0000 7f8337fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba046ee4b0 0x55ba046df160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:26.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.311+0000 7f8337fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba046ee4b0 0x55ba046df160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:60136/0 (socket says 172.21.15.2:60136) 2024-05-14T17:05:26.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.311+0000 7f8337fff700 1 -- 172.21.15.2:0/415984289 learned_addr learned my addr 172.21.15.2:0/415984289 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:05:26.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.311+0000 7f833c9cb700 1 --2- 172.21.15.2:0/415984289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba046ea350 0x55ba046e9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:26.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.311+0000 7f833d1cc700 1 --2- 172.21.15.2:0/415984289 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba046f3760 0x55ba0470b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:26.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.311+0000 7f8337fff700 1 -- 172.21.15.2:0/415984289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba046ea350 msgr2=0x55ba046e9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:26.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.311+0000 7f8337fff700 1 --2- 172.21.15.2:0/415984289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba046ea350 0x55ba046e9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:26.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.311+0000 7f8337fff700 1 -- 172.21.15.2:0/415984289 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba046f3760 msgr2=0x55ba0470b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:26.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.311+0000 7f8337fff700 1 --2- 172.21.15.2:0/415984289 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba046f3760 0x55ba0470b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:26.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.311+0000 7f8337fff700 1 -- 172.21.15.2:0/415984289 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba046f16f0 con 0x55ba046ee4b0 2024-05-14T17:05:26.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.311+0000 7f833d1cc700 1 --2- 172.21.15.2:0/415984289 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba046f3760 0x55ba0470b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:05:26.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.312+0000 7f8337fff700 1 --2- 172.21.15.2:0/415984289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba046ee4b0 0x55ba046df160 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f83280af530 tx=0x7f83280ea040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad7fd0318e7e41f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:27.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.312+0000 7f831e7fc700 1 -- 172.21.15.2:0/415984289 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83280c9070 con 0x55ba046ee4b0 2024-05-14T17:05:27.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.312+0000 7f831e7fc700 1 -- 172.21.15.2:0/415984289 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83280af0b0 con 0x55ba046ee4b0 2024-05-14T17:05:27.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.312+0000 7f831e7fc700 1 -- 172.21.15.2:0/415984289 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83280b3950 con 0x55ba046ee4b0 2024-05-14T17:05:27.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.313+0000 7f83494c5b80 1 -- 172.21.15.2:0/415984289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba046ee4b0 msgr2=0x55ba046df160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:27.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.313+0000 7f83494c5b80 1 --2- 172.21.15.2:0/415984289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba046ee4b0 0x55ba046df160 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f83280af530 tx=0x7f83280ea040 comp rx=0 tx=0).stop 2024-05-14T17:05:27.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.314+0000 7f83494c5b80 1 -- 172.21.15.2:0/415984289 shutdown_connections 2024-05-14T17:05:27.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.314+0000 7f83494c5b80 1 --2- 172.21.15.2:0/415984289 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba046f3760 0x55ba0470b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:27.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.314+0000 7f83494c5b80 1 --2- 172.21.15.2:0/415984289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba046ea350 0x55ba046e9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:27.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.314+0000 7f83494c5b80 1 --2- 172.21.15.2:0/415984289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba046ee4b0 0x55ba046df160 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:27.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.314+0000 7f83494c5b80 1 -- 172.21.15.2:0/415984289 >> 172.21.15.2:0/415984289 conn(0x55ba04616f40 msgr2=0x55ba046ed7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:27.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.314+0000 7f83494c5b80 1 -- 172.21.15.2:0/415984289 shutdown_connections 2024-05-14T17:05:27.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.314+0000 7f83494c5b80 1 -- 172.21.15.2:0/415984289 wait complete. 2024-05-14T17:05:27.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.315+0000 7f83494c5b80 1 -- start start 2024-05-14T17:05:27.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.315+0000 7f83494c5b80 1 Processor -- start 2024-05-14T17:05:28.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.315+0000 7f83494c5b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba046ea350 0x55ba046f6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:28.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.315+0000 7f83494c5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba046ee4b0 0x55ba046f7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:28.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.315+0000 7f83494c5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba046f3760 0x55ba0476e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:28.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.316+0000 7f83494c5b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f83381b2b30 con 0x55ba046ea350 2024-05-14T17:05:28.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.316+0000 7f83494c5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f83381b3f50 con 0x55ba046f3760 2024-05-14T17:05:28.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.316+0000 7f83494c5b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f83381b31b0 con 0x55ba046ee4b0 2024-05-14T17:05:28.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.316+0000 7f833c9cb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba046ea350 0x55ba046f6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:28.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.316+0000 7f83494c5b80 10 client.0 _ll_register_callbacks cb 0x55ba045d2ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:05:28.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.316+0000 7f833c9cb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba046ea350 0x55ba046f6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36544/0 (socket says 172.21.15.2:36544) 2024-05-14T17:05:28.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.316+0000 7f8337fff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba046ee4b0 0x55ba046f7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:28.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.316+0000 7f833c9cb700 1 -- 172.21.15.2:0/2495083076 learned_addr learned my addr 172.21.15.2:0/2495083076 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:05:28.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.316+0000 7f833d1cc700 1 --2- 172.21.15.2:0/2495083076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba046f3760 0x55ba0476e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:28.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.316+0000 7f833c9cb700 1 -- 172.21.15.2:0/2495083076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba046ee4b0 msgr2=0x55ba046f7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:29.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.316+0000 7f833c9cb700 1 --2- 172.21.15.2:0/2495083076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba046ee4b0 0x55ba046f7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:29.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.316+0000 7f833c9cb700 1 -- 172.21.15.2:0/2495083076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba046f3760 msgr2=0x55ba0476e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:29.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.316+0000 7f833c9cb700 1 --2- 172.21.15.2:0/2495083076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba046f3760 0x55ba0476e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:29.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.316+0000 7f833c9cb700 1 -- 172.21.15.2:0/2495083076 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba0476e950 con 0x55ba046ea350 2024-05-14T17:05:29.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.316+0000 7f833d1cc700 1 --2- 172.21.15.2:0/2495083076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba046f3760 0x55ba0476e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:05:29.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.316+0000 7f833c9cb700 1 --2- 172.21.15.2:0/2495083076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba046ea350 0x55ba046f6e40 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f832c009b40 tx=0x7f832c00e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93c4da0c72b5345 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:29.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.316+0000 7f8317fff700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:05:29.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.317+0000 7f8317fff700 1 -- 172.21.15.2:0/2495083076 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c016040 con 0x55ba046ea350 2024-05-14T17:05:29.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.317+0000 7f8317fff700 1 -- 172.21.15.2:0/2495083076 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f832c004ac0 con 0x55ba046ea350 2024-05-14T17:05:29.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.317+0000 7f83494c5b80 1 -- 172.21.15.2:0/2495083076 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83280af0b0 con 0x55ba046ea350 2024-05-14T17:05:29.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.317+0000 7f8317fff700 1 -- 172.21.15.2:0/2495083076 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f832c040050 con 0x55ba046ea350 2024-05-14T17:05:30.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.317+0000 7f83494c5b80 10 client.16476 fetch_fsmap learned FSMap version 44 2024-05-14T17:05:30.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.317+0000 7f83494c5b80 1 -- 172.21.15.2:0/2495083076 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ba0476fc10 con 0x55ba046ea350 2024-05-14T17:05:30.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.318+0000 7f8317fff700 1 -- 172.21.15.2:0/2495083076 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83280af0b0 con 0x55ba046ea350 2024-05-14T17:05:30.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.318+0000 7f8317fff700 1 -- 172.21.15.2:0/2495083076 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 44) ==== 8350+0+0 (secure 0 0 0) 0x7f832c00f040 con 0x55ba046ea350 2024-05-14T17:05:30.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.319+0000 7f83494c5b80 10 client.16476 fetch_fsmap finished waiting for FSMap version 44 2024-05-14T17:05:30.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.319+0000 7f83494c5b80 10 client.16476 resolve_mds: validated gid 24479 aka daemon mds.b 2024-05-14T17:05:30.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.319+0000 7f83494c5b80 20 client.16476 populate_metadata read hostname 'smithi002' 2024-05-14T17:05:30.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.319+0000 7f83494c5b80 1 --2- 172.21.15.2:0/2495083076 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ba047700a0 0x55ba047724c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:30.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.319+0000 7f83494c5b80 4 client.16476 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.9, "await_for": 45.0}] 2024-05-14T17:05:30.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.319+0000 7f83494c5b80 1 -- 172.21.15.2:0/2495083076 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.9, "await_for": 45.0}) -- 0x55ba04643630 con 0x55ba047700a0 2024-05-14T17:05:30.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.319+0000 7f8337fff700 1 --2- 172.21.15.2:0/2495083076 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ba047700a0 0x55ba047724c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:30.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.320+0000 7f8317fff700 10 client.16476 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:05:30.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.320+0000 7f8337fff700 1 --2- 172.21.15.2:0/2495083076 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ba047700a0 0x55ba047724c0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=846678de357add86 server_cookie=7f5fd90c5dc7ee03 in_seq=0 out_seq=0 2024-05-14T17:05:30.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.634+0000 7f8317fff700 1 -- 172.21.15.2:0/2495083076 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55ba04643630 con 0x55ba047700a0 2024-05-14T17:05:31.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.634+0000 7f8317fff700 10 client.16476 handle_command_reply: tid=0 2024-05-14T17:05:31.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.634+0000 7f83494c5b80 1 client.16476 shutdown 2024-05-14T17:05:31.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.635+0000 7f83494c5b80 1 -- 172.21.15.2:0/2495083076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba046ea350 msgr2=0x55ba046f6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:31.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.635+0000 7f83494c5b80 1 --2- 172.21.15.2:0/2495083076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba046ea350 0x55ba046f6e40 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f832c009b40 tx=0x7f832c00e7e0 comp rx=0 tx=0).stop 2024-05-14T17:05:31.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.635+0000 7f83494c5b80 1 -- 172.21.15.2:0/2495083076 shutdown_connections 2024-05-14T17:05:31.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.635+0000 7f83494c5b80 1 --2- 172.21.15.2:0/2495083076 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ba047700a0 0x55ba047724c0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:31.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.635+0000 7f83494c5b80 1 --2- 172.21.15.2:0/2495083076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba046f3760 0x55ba0476e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:31.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.635+0000 7f83494c5b80 1 --2- 172.21.15.2:0/2495083076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ba046ea350 0x55ba046f6e40 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:31.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.635+0000 7f8317fff700 0 client.16476 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:05:31.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.635+0000 7f83494c5b80 1 --2- 172.21.15.2:0/2495083076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ba046ee4b0 0x55ba046f7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:31.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.635+0000 7f83494c5b80 1 -- 172.21.15.2:0/2495083076 >> 172.21.15.2:0/2495083076 conn(0x55ba04616f40 msgr2=0x55ba046fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:31.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.635+0000 7f83494c5b80 1 -- 172.21.15.2:0/2495083076 shutdown_connections 2024-05-14T17:05:32.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.635+0000 7f83494c5b80 1 -- 172.21.15.2:0/2495083076 wait complete. 2024-05-14T17:05:32.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.635+0000 7f83494c5b80 20 client.16476 trim_cache size 0 max 16384 2024-05-14T17:05:32.145 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T17:05:32.234 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 44, 2024-05-14T17:05:32.323 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24479, 2024-05-14T17:05:32.369 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 34, 2024-05-14T17:05:32.462 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-05-14T17:05:32.531 INFO:teuthology.orchestra.run.smithi002.stdout: "e0c7677c": { 2024-05-14T17:05:32.599 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 34, 2024-05-14T17:05:32.743 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 0.0, 2024-05-14T17:05:32.797 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-05-14T17:05:32.843 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-05-14T17:05:32.938 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-05-14T17:05:33.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:05:33.085 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-05-14T17:05:33.135 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 137.9, 2024-05-14T17:05:33.238 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-05-14T17:05:33.325 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-05-14T17:05:33.412 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-05-14T17:05:33.466 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-05-14T17:05:33.510 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-05-14T17:05:33.573 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-05-14T17:05:33.628 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:05:33.703 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:05:33.772 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:05:33.839 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:05:33.980 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:05:34.061 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T17:05:34.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.636+0000 7f833ec2f700 1 -- 172.21.15.2:0/1064358236 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f83200b2b00 msgr2=0x7f83200b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:34.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.636+0000 7f833ec2f700 1 --2- 172.21.15.2:0/1064358236 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f83200b2b00 0x7f83200b4fc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f832c000c00 tx=0x7f832c01c040 comp rx=0 tx=0).stop 2024-05-14T17:05:34.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.636+0000 7f833ec2f700 1 -- 172.21.15.2:0/1064358236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8338108560 msgr2=0x7f833810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:34.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.636+0000 7f833ec2f700 1 --2- 172.21.15.2:0/1064358236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8338108560 0x7f833810f520 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f832800e940 tx=0x7f832800ed00 comp rx=0 tx=0).stop 2024-05-14T17:05:34.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.636+0000 7f833ec2f700 1 -- 172.21.15.2:0/1064358236 shutdown_connections 2024-05-14T17:05:34.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.636+0000 7f833ec2f700 1 --2- 172.21.15.2:0/1064358236 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f83200b2b00 0x7f83200b4fc0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:34.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.636+0000 7f833ec2f700 1 --2- 172.21.15.2:0/1064358236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83381089d0 0x7f8338116d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:34.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.636+0000 7f833ec2f700 1 --2- 172.21.15.2:0/1064358236 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8338105fd0 0x7f833810efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:34.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.636+0000 7f833ec2f700 1 --2- 172.21.15.2:0/1064358236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8338108560 0x7f833810f520 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:34.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.636+0000 7f833ec2f700 1 -- 172.21.15.2:0/1064358236 >> 172.21.15.2:0/1064358236 conn(0x7f83380793f0 msgr2=0x7f83380ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:34.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.636+0000 7f833ec2f700 1 -- 172.21.15.2:0/1064358236 shutdown_connections 2024-05-14T17:05:34.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:18.636+0000 7f833ec2f700 1 -- 172.21.15.2:0/1064358236 wait complete. 2024-05-14T17:05:35.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:18 smithi002 ceph-mon[31538]: pgmap v329: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.1 MiB/s wr, 696 op/s 2024-05-14T17:05:35.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:20 smithi002 ceph-mon[31538]: pgmap v330: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 13 MiB/s wr, 986 op/s 2024-05-14T17:05:35.086 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 605.0 s, 769.1 tps, lat 41.682 ms stddev 47.756 2024-05-14T17:05:35.120 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'e0c7677c', quiesce took 23.2 seconds. Will release after: -5.300000000000001 2024-05-14T17:05:35.154 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:22 smithi002 ceph-mon[31538]: pgmap v331: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.9 MiB/s wr, 566 op/s 2024-05-14T17:05:35.211 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:24 smithi002 ceph-mon[31538]: pgmap v332: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 7.4 MiB/s wr, 603 op/s 2024-05-14T17:05:35.308 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 610.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:05:35.362 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:26 smithi002 ceph-mon[31538]: pgmap v333: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.7 MiB/s wr, 433 op/s 2024-05-14T17:05:35.430 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:28 smithi002 ceph-mon[31538]: pgmap v334: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 4.2 MiB/s wr, 327 op/s 2024-05-14T17:05:35.549 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:20 smithi151 ceph-mon[36066]: pgmap v330: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 13 MiB/s wr, 986 op/s 2024-05-14T17:05:35.611 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:22 smithi151 ceph-mon[36066]: pgmap v331: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.9 MiB/s wr, 566 op/s 2024-05-14T17:05:35.665 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:24 smithi151 ceph-mon[36066]: pgmap v332: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 7.4 MiB/s wr, 603 op/s 2024-05-14T17:05:35.735 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:26 smithi151 ceph-mon[36066]: pgmap v333: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.7 MiB/s wr, 433 op/s 2024-05-14T17:05:35.909 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:28 smithi151 ceph-mon[36066]: pgmap v334: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 4.2 MiB/s wr, 327 op/s 2024-05-14T17:05:35.986 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:30 smithi151 ceph-mon[36066]: pgmap v335: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.3 MiB/s wr, 331 op/s 2024-05-14T17:05:36.019 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:32 smithi151 ceph-mon[36066]: pgmap v336: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 460 KiB/s wr, 40 op/s 2024-05-14T17:05:36.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:34 smithi151 ceph-mon[36066]: pgmap v337: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 483 KiB/s wr, 48 op/s 2024-05-14T17:05:36.192 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:20 smithi123 ceph-mon[35971]: pgmap v330: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 13 MiB/s wr, 986 op/s 2024-05-14T17:05:36.299 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:22 smithi123 ceph-mon[35971]: pgmap v331: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.9 MiB/s wr, 566 op/s 2024-05-14T17:05:36.362 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:24 smithi123 ceph-mon[35971]: pgmap v332: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 7.4 MiB/s wr, 603 op/s 2024-05-14T17:05:36.398 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:26 smithi123 ceph-mon[35971]: pgmap v333: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.7 MiB/s wr, 433 op/s 2024-05-14T17:05:36.473 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:28 smithi123 ceph-mon[35971]: pgmap v334: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 4.2 MiB/s wr, 327 op/s 2024-05-14T17:05:36.530 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:30 smithi123 ceph-mon[35971]: pgmap v335: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.3 MiB/s wr, 331 op/s 2024-05-14T17:05:36.647 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:32 smithi123 ceph-mon[35971]: pgmap v336: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 460 KiB/s wr, 40 op/s 2024-05-14T17:05:36.705 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:34 smithi123 ceph-mon[35971]: pgmap v337: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 483 KiB/s wr, 48 op/s 2024-05-14T17:05:36.750 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:05:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.602+0000 7f33523f0640 1 -- 172.21.15.2:0/1571917307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f334c107940 msgr2=0x7f334c107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:36.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.602+0000 7f33523f0640 1 --2- 172.21.15.2:0/1571917307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f334c107940 0x7f334c107d20 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f333c009930 tx=0x7f333c02a420 comp rx=0 tx=0).stop 2024-05-14T17:05:36.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.602+0000 7f33523f0640 1 -- 172.21.15.2:0/1571917307 shutdown_connections 2024-05-14T17:05:36.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.602+0000 7f33523f0640 1 --2- 172.21.15.2:0/1571917307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f334c102340 0x7f334c10e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.602+0000 7f33523f0640 1 --2- 172.21.15.2:0/1571917307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f334c101980 0x7f334c101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.602+0000 7f33523f0640 1 --2- 172.21.15.2:0/1571917307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f334c107940 0x7f334c107d20 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:37.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.602+0000 7f33523f0640 1 -- 172.21.15.2:0/1571917307 >> 172.21.15.2:0/1571917307 conn(0x7f334c078ad0 msgr2=0x7f334c078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.602+0000 7f33523f0640 1 -- 172.21.15.2:0/1571917307 shutdown_connections 2024-05-14T17:05:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.602+0000 7f33523f0640 1 -- 172.21.15.2:0/1571917307 wait complete. 2024-05-14T17:05:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.603+0000 7f33523f0640 1 Processor -- start 2024-05-14T17:05:37.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.603+0000 7f33523f0640 1 -- start start 2024-05-14T17:05:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.603+0000 7f33523f0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f334c101980 0x7f334c199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:37.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.603+0000 7f33523f0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f334c102340 0x7f334c199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:37.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.603+0000 7f33523f0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f334c107940 0x7f334c19db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.604+0000 7f33523f0640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f334c111c20 con 0x7f334c101980 2024-05-14T17:05:37.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.604+0000 7f33523f0640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f334c111aa0 con 0x7f334c107940 2024-05-14T17:05:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.604+0000 7f33523f0640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f334c111da0 con 0x7f334c102340 2024-05-14T17:05:37.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.604+0000 7f334bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f334c101980 0x7f334c199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:37.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.604+0000 7f334bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f334c101980 0x7f334c199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49310/0 (socket says 172.21.15.2:49310) 2024-05-14T17:05:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.604+0000 7f334bfff640 1 -- 172.21.15.2:0/4049304038 learned_addr learned my addr 172.21.15.2:0/4049304038 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:05:38.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.604+0000 7f334b7fe640 1 --2- 172.21.15.2:0/4049304038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f334c102340 0x7f334c199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.604+0000 7f335098d640 1 --2- 172.21.15.2:0/4049304038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f334c107940 0x7f334c19db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:38.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.604+0000 7f334bfff640 1 -- 172.21.15.2:0/4049304038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f334c102340 msgr2=0x7f334c199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.604+0000 7f334bfff640 1 --2- 172.21.15.2:0/4049304038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f334c102340 0x7f334c199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.604+0000 7f334bfff640 1 -- 172.21.15.2:0/4049304038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f334c107940 msgr2=0x7f334c19db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.604+0000 7f334bfff640 1 --2- 172.21.15.2:0/4049304038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f334c107940 0x7f334c19db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.604+0000 7f334bfff640 1 -- 172.21.15.2:0/4049304038 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f334c19e230 con 0x7f334c101980 2024-05-14T17:05:38.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.604+0000 7f335098d640 1 --2- 172.21.15.2:0/4049304038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f334c107940 0x7f334c19db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:05:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.604+0000 7f334bfff640 1 --2- 172.21.15.2:0/4049304038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f334c101980 0x7f334c199240 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f333c004770 tx=0x7f333c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.604+0000 7f33497fa640 1 -- 172.21.15.2:0/4049304038 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c03f070 con 0x7f334c101980 2024-05-14T17:05:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.605+0000 7f33523f0640 1 -- 172.21.15.2:0/4049304038 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f334c19e4c0 con 0x7f334c101980 2024-05-14T17:05:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.605+0000 7f33497fa640 1 -- 172.21.15.2:0/4049304038 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f333c03b460 con 0x7f334c101980 2024-05-14T17:05:38.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.605+0000 7f33497fa640 1 -- 172.21.15.2:0/4049304038 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c035070 con 0x7f334c101980 2024-05-14T17:05:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.605+0000 7f33523f0640 1 -- 172.21.15.2:0/4049304038 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f334c1a5db0 con 0x7f334c101980 2024-05-14T17:05:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.607+0000 7f33497fa640 1 -- 172.21.15.2:0/4049304038 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f333c03b8d0 con 0x7f334c101980 2024-05-14T17:05:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.607+0000 7f33551e7740 1 Processor -- start 2024-05-14T17:05:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.607+0000 7f33551e7740 1 -- start start 2024-05-14T17:05:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.607+0000 7f33551e7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da921448f0 0x55da92144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:39.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.607+0000 7f33551e7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da92145210 0x55da9214f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:39.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.607+0000 7f33551e7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da9214fad0 0x55da92151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.607+0000 7f33551e7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55da920b7c30 con 0x55da92145210 2024-05-14T17:05:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.607+0000 7f33551e7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55da920a77d0 con 0x55da921448f0 2024-05-14T17:05:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.607+0000 7f33551e7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55da920ab810 con 0x55da9214fad0 2024-05-14T17:05:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.608+0000 7f334bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da92145210 0x55da9214f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.608+0000 7f334bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da92145210 0x55da9214f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49318/0 (socket says 172.21.15.2:49318) 2024-05-14T17:05:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.608+0000 7f334bfff640 1 -- 172.21.15.2:0/3049149128 learned_addr learned my addr 172.21.15.2:0/3049149128 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:05:39.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.608+0000 7f335098d640 1 --2- 172.21.15.2:0/3049149128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da921448f0 0x55da92144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.608+0000 7f334b7fe640 1 --2- 172.21.15.2:0/3049149128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da9214fad0 0x55da92151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:40.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.608+0000 7f334bfff640 1 -- 172.21.15.2:0/3049149128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da9214fad0 msgr2=0x55da92151ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:40.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.608+0000 7f334bfff640 1 --2- 172.21.15.2:0/3049149128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da9214fad0 0x55da92151ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:40.186 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.608+0000 7f334bfff640 1 -- 172.21.15.2:0/3049149128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da921448f0 msgr2=0x55da92144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.608+0000 7f334bfff640 1 --2- 172.21.15.2:0/3049149128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da921448f0 0x55da92144cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.608+0000 7f334bfff640 1 -- 172.21.15.2:0/3049149128 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da91dbc210 con 0x55da92145210 2024-05-14T17:05:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.608+0000 7f335098d640 1 --2- 172.21.15.2:0/3049149128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da921448f0 0x55da92144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:05:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.608+0000 7f33497fa640 1 --2- 172.21.15.2:0/4049304038 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f33340b2a10 0x7f33340b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:40.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.608+0000 7f335098d640 1 --2- 172.21.15.2:0/4049304038 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f33340b2a10 0x7f33340b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:40.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.608+0000 7f334bfff640 1 --2- 172.21.15.2:0/3049149128 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da92145210 0x55da9214f590 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f333c100050 tx=0x7f333c106bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f49b3341201ae0d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.608+0000 7f33497fa640 1 -- 172.21.15.2:0/4049304038 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f333c102990 con 0x7f334c101980 2024-05-14T17:05:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.609+0000 7f33327fc640 1 -- 172.21.15.2:0/3049149128 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c10d040 con 0x55da92145210 2024-05-14T17:05:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.609+0000 7f33327fc640 1 -- 172.21.15.2:0/3049149128 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f333c0c53f0 con 0x55da92145210 2024-05-14T17:05:40.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.609+0000 7f33327fc640 1 -- 172.21.15.2:0/3049149128 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c0c5b30 con 0x55da92145210 2024-05-14T17:05:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.609+0000 7f33551e7740 1 -- 172.21.15.2:0/3049149128 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da92145210 msgr2=0x55da9214f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.609+0000 7f335098d640 1 --2- 172.21.15.2:0/4049304038 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f33340b2a10 0x7f33340b4ed0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f334400a4c0 tx=0x7f33440096f0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:41.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.609+0000 7f33551e7740 1 --2- 172.21.15.2:0/3049149128 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da92145210 0x55da9214f590 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f333c100050 tx=0x7f333c106bb0 comp rx=0 tx=0).stop 2024-05-14T17:05:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.609+0000 7f33551e7740 1 -- 172.21.15.2:0/3049149128 shutdown_connections 2024-05-14T17:05:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.609+0000 7f33551e7740 1 --2- 172.21.15.2:0/3049149128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da9214fad0 0x55da92151ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.609+0000 7f33551e7740 1 --2- 172.21.15.2:0/3049149128 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da92145210 0x55da9214f590 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:41.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.609+0000 7f33551e7740 1 --2- 172.21.15.2:0/3049149128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da921448f0 0x55da92144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.609+0000 7f33551e7740 1 -- 172.21.15.2:0/3049149128 >> 172.21.15.2:0/3049149128 conn(0x55da92140e00 msgr2=0x55da921411e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.609+0000 7f33551e7740 1 -- 172.21.15.2:0/3049149128 shutdown_connections 2024-05-14T17:05:41.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.609+0000 7f33551e7740 1 -- 172.21.15.2:0/3049149128 wait complete. 2024-05-14T17:05:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.610+0000 7f33551e7740 1 -- start start 2024-05-14T17:05:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.610+0000 7f33551e7740 1 Processor -- start 2024-05-14T17:05:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.610+0000 7f33551e7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da921448f0 0x55da9214f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.610+0000 7f33551e7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da92145210 0x55da9214c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.610+0000 7f33551e7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da9214fad0 0x55da9214cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:42.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.610+0000 7f33551e7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55da91d8d790 con 0x55da921448f0 2024-05-14T17:05:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.610+0000 7f33551e7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55da92063890 con 0x55da9214fad0 2024-05-14T17:05:42.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.610+0000 7f33551e7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55da921524c0 con 0x55da92145210 2024-05-14T17:05:42.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.610+0000 7f33551e7740 10 client.0 _ll_register_callbacks cb 0x55da92066d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:05:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.611+0000 7f334bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da921448f0 0x55da9214f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.611+0000 7f334bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da921448f0 0x55da9214f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49320/0 (socket says 172.21.15.2:49320) 2024-05-14T17:05:42.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.611+0000 7f334bfff640 1 -- 172.21.15.2:0/4185584740 learned_addr learned my addr 172.21.15.2:0/4185584740 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:05:42.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.611+0000 7f334b7fe640 1 --2- 172.21.15.2:0/4185584740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da92145210 0x55da9214c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:42.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.611+0000 7f334bfff640 1 -- 172.21.15.2:0/4185584740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da92145210 msgr2=0x55da9214c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:42.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.611+0000 7f334bfff640 1 --2- 172.21.15.2:0/4185584740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da92145210 0x55da9214c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.611+0000 7f334bfff640 1 -- 172.21.15.2:0/4185584740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da9214fad0 msgr2=0x55da9214cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.611+0000 7f334bfff640 1 --2- 172.21.15.2:0/4185584740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da9214fad0 0x55da9214cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.611+0000 7f334bfff640 1 -- 172.21.15.2:0/4185584740 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da91e88620 con 0x55da921448f0 2024-05-14T17:05:43.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.611+0000 7f334bfff640 1 --2- 172.21.15.2:0/4185584740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da921448f0 0x55da9214f490 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f333c100190 tx=0x7f333c03b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e8d1a2a0967221 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:43.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.611+0000 7f332b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:05:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.611+0000 7f332b7fe640 1 -- 172.21.15.2:0/4185584740 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c048040 con 0x55da921448f0 2024-05-14T17:05:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.611+0000 7f332b7fe640 1 -- 172.21.15.2:0/4185584740 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f333c106e00 con 0x55da921448f0 2024-05-14T17:05:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.611+0000 7f33551e7740 1 -- 172.21.15.2:0/4185584740 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f333c0c53f0 con 0x55da921448f0 2024-05-14T17:05:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.611+0000 7f332b7fe640 1 -- 172.21.15.2:0/4185584740 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f333c10d040 con 0x55da921448f0 2024-05-14T17:05:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.611+0000 7f33551e7740 10 client.16488 fetch_fsmap learned FSMap version 44 2024-05-14T17:05:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.611+0000 7f33551e7740 1 -- 172.21.15.2:0/4185584740 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55da9215dee0 con 0x55da921448f0 2024-05-14T17:05:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.611+0000 7f332b7fe640 1 -- 172.21.15.2:0/4185584740 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f333c0c53f0 con 0x55da921448f0 2024-05-14T17:05:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.612+0000 7f332b7fe640 1 -- 172.21.15.2:0/4185584740 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 44) ==== 8350+0+0 (secure 0 0 0) 0x7f333c04b060 con 0x55da921448f0 2024-05-14T17:05:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.612+0000 7f33551e7740 10 client.16488 fetch_fsmap finished waiting for FSMap version 44 2024-05-14T17:05:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.612+0000 7f33551e7740 10 client.16488 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:05:44.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.612+0000 7f33551e7740 20 client.16488 populate_metadata read hostname 'smithi002' 2024-05-14T17:05:44.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.612+0000 7f33551e7740 1 --2- 172.21.15.2:0/4185584740 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55da9215f230 0x55da92161620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:44.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.612+0000 7f33551e7740 4 client.16488 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:05:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.612+0000 7f33551e7740 1 -- 172.21.15.2:0/4185584740 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da920b9b80 con 0x55da9215f230 2024-05-14T17:05:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.612+0000 7f334b7fe640 1 --2- 172.21.15.2:0/4185584740 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55da9215f230 0x55da92161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.612+0000 7f332b7fe640 10 client.16488 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:05:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.612+0000 7f334b7fe640 1 --2- 172.21.15.2:0/4185584740 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55da9215f230 0x55da92161620 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1df96ebc4dc4d863 server_cookie=480cbb89d7a6e2bf in_seq=0 out_seq=0 2024-05-14T17:05:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.613+0000 7f332b7fe640 1 -- 172.21.15.2:0/4185584740 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55da920b9b80 con 0x55da9215f230 2024-05-14T17:05:44.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.613+0000 7f332b7fe640 10 client.16488 handle_command_reply: tid=0 2024-05-14T17:05:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.613+0000 7f33551e7740 1 client.16488 shutdown 2024-05-14T17:05:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.613+0000 7f33551e7740 1 -- 172.21.15.2:0/4185584740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da921448f0 msgr2=0x55da9214f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.613+0000 7f33551e7740 1 --2- 172.21.15.2:0/4185584740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da921448f0 0x55da9214f490 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f333c100190 tx=0x7f333c03b840 comp rx=0 tx=0).stop 2024-05-14T17:05:44.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.613+0000 7f33551e7740 1 -- 172.21.15.2:0/4185584740 shutdown_connections 2024-05-14T17:05:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.613+0000 7f33551e7740 1 --2- 172.21.15.2:0/4185584740 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55da9215f230 0x55da92161620 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:45.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.613+0000 7f33551e7740 1 --2- 172.21.15.2:0/4185584740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da9214fad0 0x55da9214cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.613+0000 7f33551e7740 1 --2- 172.21.15.2:0/4185584740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da92145210 0x55da9214c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.613+0000 7f33551e7740 1 --2- 172.21.15.2:0/4185584740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da921448f0 0x55da9214f490 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.613+0000 7f332b7fe640 0 client.16488 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:05:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.613+0000 7f33551e7740 1 -- 172.21.15.2:0/4185584740 >> 172.21.15.2:0/4185584740 conn(0x55da92140e00 msgr2=0x55da92140100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.614+0000 7f33551e7740 1 -- 172.21.15.2:0/4185584740 shutdown_connections 2024-05-14T17:05:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.614+0000 7f33551e7740 1 -- 172.21.15.2:0/4185584740 wait complete. 2024-05-14T17:05:45.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.614+0000 7f33551e7740 20 client.16488 trim_cache size 0 max 16384 2024-05-14T17:05:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.629+0000 7f33551e7740 1 Processor -- start 2024-05-14T17:05:45.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.629+0000 7f33551e7740 1 -- start start 2024-05-14T17:05:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.629+0000 7f33551e7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da921448f0 0x55da9215b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:46.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.629+0000 7f33551e7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da92145210 0x55da92166ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.629+0000 7f33551e7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da9214fad0 0x55da92167420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.629+0000 7f33551e7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f334c1b1060 con 0x55da9214fad0 2024-05-14T17:05:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.629+0000 7f33551e7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f334c1ae8d0 con 0x55da92145210 2024-05-14T17:05:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.629+0000 7f33551e7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f334c1ac720 con 0x55da921448f0 2024-05-14T17:05:46.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.629+0000 7f335098d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da9214fad0 0x55da92167420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.629+0000 7f335098d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da9214fad0 0x55da92167420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49334/0 (socket says 172.21.15.2:49334) 2024-05-14T17:05:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.629+0000 7f335098d640 1 -- 172.21.15.2:0/1665370170 learned_addr learned my addr 172.21.15.2:0/1665370170 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:05:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.630+0000 7f334b7fe640 1 --2- 172.21.15.2:0/1665370170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da92145210 0x55da92166ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:46.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.630+0000 7f334bfff640 1 --2- 172.21.15.2:0/1665370170 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da921448f0 0x55da9215b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.630+0000 7f335098d640 1 -- 172.21.15.2:0/1665370170 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da921448f0 msgr2=0x55da9215b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:46.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.630+0000 7f335098d640 1 --2- 172.21.15.2:0/1665370170 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da921448f0 0x55da9215b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.630+0000 7f335098d640 1 -- 172.21.15.2:0/1665370170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da92145210 msgr2=0x55da92166ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.630+0000 7f335098d640 1 --2- 172.21.15.2:0/1665370170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da92145210 0x55da92166ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:47.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.630+0000 7f335098d640 1 -- 172.21.15.2:0/1665370170 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da92145040 con 0x55da9214fad0 2024-05-14T17:05:47.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.630+0000 7f334bfff640 1 --2- 172.21.15.2:0/1665370170 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da921448f0 0x55da9215b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:05:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.630+0000 7f335098d640 1 --2- 172.21.15.2:0/1665370170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da9214fad0 0x55da92167420 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f334401a880 tx=0x7f334400e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3baed71d389f078c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:47.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.630+0000 7f3331ffb640 1 -- 172.21.15.2:0/1665370170 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334401a2e0 con 0x55da9214fad0 2024-05-14T17:05:47.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.630+0000 7f3331ffb640 1 -- 172.21.15.2:0/1665370170 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33440188c0 con 0x55da9214fad0 2024-05-14T17:05:47.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.630+0000 7f3331ffb640 1 -- 172.21.15.2:0/1665370170 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33440238b0 con 0x55da9214fad0 2024-05-14T17:05:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.631+0000 7f33551e7740 1 -- 172.21.15.2:0/1665370170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da9214fad0 msgr2=0x55da92167420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.631+0000 7f33551e7740 1 --2- 172.21.15.2:0/1665370170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da9214fad0 0x55da92167420 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f334401a880 tx=0x7f334400e050 comp rx=0 tx=0).stop 2024-05-14T17:05:47.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.631+0000 7f33551e7740 1 -- 172.21.15.2:0/1665370170 shutdown_connections 2024-05-14T17:05:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.631+0000 7f33551e7740 1 --2- 172.21.15.2:0/1665370170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da9214fad0 0x55da92167420 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:48.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.631+0000 7f33551e7740 1 --2- 172.21.15.2:0/1665370170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da92145210 0x55da92166ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:48.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.631+0000 7f33551e7740 1 --2- 172.21.15.2:0/1665370170 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da921448f0 0x55da9215b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.631+0000 7f33551e7740 1 -- 172.21.15.2:0/1665370170 >> 172.21.15.2:0/1665370170 conn(0x55da92140e00 msgr2=0x55da9214b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.631+0000 7f33551e7740 1 -- 172.21.15.2:0/1665370170 shutdown_connections 2024-05-14T17:05:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.631+0000 7f33551e7740 1 -- 172.21.15.2:0/1665370170 wait complete. 2024-05-14T17:05:48.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.631+0000 7f33551e7740 1 -- start start 2024-05-14T17:05:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.631+0000 7f33551e7740 1 Processor -- start 2024-05-14T17:05:48.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.632+0000 7f33551e7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da921448f0 0x55da92157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.632+0000 7f33551e7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da92145210 0x55da92155050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.632+0000 7f33551e7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da9214fad0 0x55da921555b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:48.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.632+0000 7f33551e7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f334c1acd90 con 0x55da92145210 2024-05-14T17:05:49.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.632+0000 7f33551e7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f334c1ae200 con 0x55da9214fad0 2024-05-14T17:05:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.632+0000 7f33551e7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f334c1ad460 con 0x55da921448f0 2024-05-14T17:05:49.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.632+0000 7f33551e7740 10 client.0 _ll_register_callbacks cb 0x7f333c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:05:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.632+0000 7f334b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da92145210 0x55da92155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.632+0000 7f334b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da92145210 0x55da92155050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49342/0 (socket says 172.21.15.2:49342) 2024-05-14T17:05:49.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.632+0000 7f334b7fe640 1 -- 172.21.15.2:0/2836383598 learned_addr learned my addr 172.21.15.2:0/2836383598 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:05:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.632+0000 7f334bfff640 1 --2- 172.21.15.2:0/2836383598 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da921448f0 0x55da92157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:49.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.632+0000 7f335098d640 1 --2- 172.21.15.2:0/2836383598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da9214fad0 0x55da921555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.632+0000 7f334b7fe640 1 -- 172.21.15.2:0/2836383598 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da921448f0 msgr2=0x55da92157cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.632+0000 7f334b7fe640 1 --2- 172.21.15.2:0/2836383598 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da921448f0 0x55da92157cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.632+0000 7f334b7fe640 1 -- 172.21.15.2:0/2836383598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da9214fad0 msgr2=0x55da921555b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:49.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.632+0000 7f334b7fe640 1 --2- 172.21.15.2:0/2836383598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da9214fad0 0x55da921555b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.632+0000 7f334b7fe640 1 -- 172.21.15.2:0/2836383598 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da92155e30 con 0x55da92145210 2024-05-14T17:05:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.632+0000 7f334b7fe640 1 --2- 172.21.15.2:0/2836383598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da92145210 0x55da92155050 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x55da92141f90 tx=0x7f334000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d99061fdc73ce2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:50.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.632+0000 7f332b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:05:50.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.632+0000 7f332b7fe640 1 -- 172.21.15.2:0/2836383598 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334001d5b0 con 0x55da92145210 2024-05-14T17:05:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.632+0000 7f332b7fe640 1 -- 172.21.15.2:0/2836383598 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f334001d750 con 0x55da92145210 2024-05-14T17:05:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.633+0000 7f33551e7740 1 -- 172.21.15.2:0/2836383598 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33440188c0 con 0x55da92145210 2024-05-14T17:05:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.633+0000 7f332b7fe640 1 -- 172.21.15.2:0/2836383598 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f3340003e90 con 0x55da92145210 2024-05-14T17:05:50.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.633+0000 7f33551e7740 10 client.16494 fetch_fsmap learned FSMap version 44 2024-05-14T17:05:50.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.633+0000 7f33551e7740 1 -- 172.21.15.2:0/2836383598 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55da921c9af0 con 0x55da92145210 2024-05-14T17:05:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.633+0000 7f332b7fe640 1 -- 172.21.15.2:0/2836383598 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33440188c0 con 0x55da92145210 2024-05-14T17:05:51.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.633+0000 7f332b7fe640 1 -- 172.21.15.2:0/2836383598 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 44) ==== 8350+0+0 (secure 0 0 0) 0x7f3340011440 con 0x55da92145210 2024-05-14T17:05:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.633+0000 7f33551e7740 10 client.16494 fetch_fsmap finished waiting for FSMap version 44 2024-05-14T17:05:51.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.633+0000 7f33551e7740 10 client.16494 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:05:51.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.633+0000 7f33551e7740 20 client.16494 populate_metadata read hostname 'smithi002' 2024-05-14T17:05:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.633+0000 7f33551e7740 1 --2- 172.21.15.2:0/2836383598 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55da921ca250 0x55da921cc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.633+0000 7f33551e7740 4 client.16494 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:05:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.633+0000 7f33551e7740 1 -- 172.21.15.2:0/2836383598 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55da920b3360 con 0x55da921ca250 2024-05-14T17:05:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.633+0000 7f334bfff640 1 --2- 172.21.15.2:0/2836383598 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55da921ca250 0x55da921cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:51.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.634+0000 7f332b7fe640 10 client.16494 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:05:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.634+0000 7f334bfff640 1 --2- 172.21.15.2:0/2836383598 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55da921ca250 0x55da921cc670 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a13747a3e3b19c8 server_cookie=a9e352406ed14f3 in_seq=0 out_seq=0 2024-05-14T17:05:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.649+0000 7f332b7fe640 1 -- 172.21.15.2:0/2836383598 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55da920b3360 con 0x55da921ca250 2024-05-14T17:05:51.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.650+0000 7f332b7fe640 10 client.16494 handle_command_reply: tid=0 2024-05-14T17:05:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.650+0000 7f33551e7740 1 client.16494 shutdown 2024-05-14T17:05:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.650+0000 7f33551e7740 1 -- 172.21.15.2:0/2836383598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da92145210 msgr2=0x55da92155050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.650+0000 7f33551e7740 1 --2- 172.21.15.2:0/2836383598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da92145210 0x55da92155050 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x55da92141f90 tx=0x7f334000bf10 comp rx=0 tx=0).stop 2024-05-14T17:05:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.650+0000 7f33551e7740 1 -- 172.21.15.2:0/2836383598 shutdown_connections 2024-05-14T17:05:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.650+0000 7f33551e7740 1 --2- 172.21.15.2:0/2836383598 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55da921ca250 0x55da921cc670 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.650+0000 7f33551e7740 1 --2- 172.21.15.2:0/2836383598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da9214fad0 0x55da921555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.650+0000 7f33551e7740 1 --2- 172.21.15.2:0/2836383598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da92145210 0x55da92155050 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.650+0000 7f332b7fe640 0 client.16494 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:05:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.650+0000 7f33551e7740 1 --2- 172.21.15.2:0/2836383598 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da921448f0 0x55da92157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.650+0000 7f33551e7740 1 -- 172.21.15.2:0/2836383598 >> 172.21.15.2:0/2836383598 conn(0x55da92140e00 msgr2=0x55da9215a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:52.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.650+0000 7f33551e7740 1 -- 172.21.15.2:0/2836383598 shutdown_connections 2024-05-14T17:05:52.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.650+0000 7f33551e7740 1 -- 172.21.15.2:0/2836383598 wait complete. 2024-05-14T17:05:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.650+0000 7f33551e7740 20 client.16494 trim_cache size 0 max 16384 2024-05-14T17:05:53.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:05:53.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:05:53.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:05:53.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.651+0000 7f33523f0640 1 -- 172.21.15.2:0/4049304038 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f33340b2a10 msgr2=0x7f33340b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.651+0000 7f33523f0640 1 --2- 172.21.15.2:0/4049304038 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f33340b2a10 0x7f33340b4ed0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f334400a4c0 tx=0x7f33440096f0 comp rx=0 tx=0).stop 2024-05-14T17:05:53.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.651+0000 7f33523f0640 1 -- 172.21.15.2:0/4049304038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f334c101980 msgr2=0x7f334c199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:53.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.651+0000 7f33523f0640 1 --2- 172.21.15.2:0/4049304038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f334c101980 0x7f334c199240 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f333c004770 tx=0x7f333c0042f0 comp rx=0 tx=0).stop 2024-05-14T17:05:53.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.651+0000 7f33523f0640 1 -- 172.21.15.2:0/4049304038 shutdown_connections 2024-05-14T17:05:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.651+0000 7f33523f0640 1 --2- 172.21.15.2:0/4049304038 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f33340b2a10 0x7f33340b4ed0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.651+0000 7f33523f0640 1 --2- 172.21.15.2:0/4049304038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f334c107940 0x7f334c19db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.651+0000 7f33523f0640 1 --2- 172.21.15.2:0/4049304038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f334c102340 0x7f334c199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.651+0000 7f33523f0640 1 --2- 172.21.15.2:0/4049304038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f334c101980 0x7f334c199240 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.651+0000 7f33523f0640 1 -- 172.21.15.2:0/4049304038 >> 172.21.15.2:0/4049304038 conn(0x7f334c078ad0 msgr2=0x7f334c06a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.651+0000 7f33523f0640 1 -- 172.21.15.2:0/4049304038 shutdown_connections 2024-05-14T17:05:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:30.651+0000 7f33523f0640 1 -- 172.21.15.2:0/4049304038 wait complete. 2024-05-14T17:05:53.840 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:30 smithi002 ceph-mon[31538]: pgmap v335: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.3 MiB/s wr, 331 op/s 2024-05-14T17:05:53.940 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 615.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:05:53.991 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:36 smithi123 ceph-mon[35971]: pgmap v338: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 34 KiB/s wr, 10 op/s 2024-05-14T17:05:54.027 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:38 smithi123 ceph-mon[35971]: pgmap v339: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 34 KiB/s wr, 10 op/s 2024-05-14T17:05:54.060 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:40 smithi123 ceph-mon[35971]: pgmap v340: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-05-14T17:05:54.094 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:42 smithi123 ceph-mon[35971]: pgmap v341: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.2 MiB/s wr, 121 op/s 2024-05-14T17:05:54.127 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:44 smithi123 ceph-mon[35971]: pgmap v342: 97 pgs: 97 active+clean; 4.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.8 MiB/s wr, 540 op/s 2024-05-14T17:05:54.166 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:46 smithi123 ceph-mon[35971]: pgmap v343: 97 pgs: 97 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 14 MiB/s wr, 710 op/s 2024-05-14T17:05:54.223 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:47 smithi123 ceph-mon[35971]: pgmap v344: 97 pgs: 97 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 14 MiB/s wr, 710 op/s 2024-05-14T17:05:54.338 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:50 smithi123 ceph-mon[35971]: pgmap v345: 97 pgs: 97 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 14 MiB/s wr, 716 op/s 2024-05-14T17:05:54.410 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:52 smithi123 ceph-mon[35971]: pgmap v346: 97 pgs: 97 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 13 MiB/s wr, 602 op/s 2024-05-14T17:05:54.485 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:36 smithi151 ceph-mon[36066]: pgmap v338: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 34 KiB/s wr, 10 op/s 2024-05-14T17:05:54.555 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:38 smithi151 ceph-mon[36066]: pgmap v339: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 34 KiB/s wr, 10 op/s 2024-05-14T17:05:54.696 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:40 smithi151 ceph-mon[36066]: pgmap v340: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-05-14T17:05:54.777 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:42 smithi151 ceph-mon[36066]: pgmap v341: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.2 MiB/s wr, 121 op/s 2024-05-14T17:05:54.845 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:44 smithi151 ceph-mon[36066]: pgmap v342: 97 pgs: 97 active+clean; 4.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.8 MiB/s wr, 540 op/s 2024-05-14T17:05:54.902 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:46 smithi151 ceph-mon[36066]: pgmap v343: 97 pgs: 97 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 14 MiB/s wr, 710 op/s 2024-05-14T17:05:54.936 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:47 smithi151 ceph-mon[36066]: pgmap v344: 97 pgs: 97 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 14 MiB/s wr, 710 op/s 2024-05-14T17:05:54.969 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:50 smithi151 ceph-mon[36066]: pgmap v345: 97 pgs: 97 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 14 MiB/s wr, 716 op/s 2024-05-14T17:05:55.011 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:52 smithi151 ceph-mon[36066]: pgmap v346: 97 pgs: 97 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 13 MiB/s wr, 602 op/s 2024-05-14T17:05:55.045 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:32 smithi002 ceph-mon[31538]: pgmap v336: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 460 KiB/s wr, 40 op/s 2024-05-14T17:05:55.170 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:34 smithi002 ceph-mon[31538]: pgmap v337: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 483 KiB/s wr, 48 op/s 2024-05-14T17:05:55.294 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 620.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:05:55.328 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:36 smithi002 ceph-mon[31538]: pgmap v338: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 34 KiB/s wr, 10 op/s 2024-05-14T17:05:55.466 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:38 smithi002 ceph-mon[31538]: pgmap v339: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 34 KiB/s wr, 10 op/s 2024-05-14T17:05:55.536 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:40 smithi002 ceph-mon[31538]: pgmap v340: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-05-14T17:05:55.635 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 625.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:05:55.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:42 smithi002 ceph-mon[31538]: pgmap v341: 97 pgs: 97 active+clean; 4.8 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.2 MiB/s wr, 121 op/s 2024-05-14T17:05:55.821 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:54 smithi123 ceph-mon[35971]: pgmap v347: 97 pgs: 97 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 13 MiB/s wr, 605 op/s 2024-05-14T17:05:55.901 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:54 smithi151 ceph-mon[36066]: pgmap v347: 97 pgs: 97 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 13 MiB/s wr, 605 op/s 2024-05-14T17:05:56.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.755+0000 7f9d6b704640 1 -- 172.21.15.2:0/1571055230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d64102a50 msgr2=0x7f9d64102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.755+0000 7f9d6b704640 1 --2- 172.21.15.2:0/1571055230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d64102a50 0x7f9d64102ed0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f9d54009930 tx=0x7f9d5402a420 comp rx=0 tx=0).stop 2024-05-14T17:05:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.755+0000 7f9d6b704640 1 -- 172.21.15.2:0/1571055230 shutdown_connections 2024-05-14T17:05:56.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.755+0000 7f9d6b704640 1 --2- 172.21.15.2:0/1571055230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d64103410 0x7f9d6410f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.755+0000 7f9d6b704640 1 --2- 172.21.15.2:0/1571055230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d64102a50 0x7f9d64102ed0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.755+0000 7f9d6b704640 1 --2- 172.21.15.2:0/1571055230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d64108a60 0x7f9d64108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.755+0000 7f9d6b704640 1 -- 172.21.15.2:0/1571055230 >> 172.21.15.2:0/1571055230 conn(0x7f9d640fe520 msgr2=0x7f9d64100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:05:56.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.755+0000 7f9d6b704640 1 -- 172.21.15.2:0/1571055230 shutdown_connections 2024-05-14T17:05:56.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.755+0000 7f9d6b704640 1 -- 172.21.15.2:0/1571055230 wait complete. 2024-05-14T17:05:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.756+0000 7f9d6b704640 1 Processor -- start 2024-05-14T17:05:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.756+0000 7f9d6b704640 1 -- start start 2024-05-14T17:05:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.756+0000 7f9d6b704640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d64102a50 0x7f9d64199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.756+0000 7f9d6b704640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d64103410 0x7f9d64199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:56.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.756+0000 7f9d6b704640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d64108a60 0x7f9d6419dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.756+0000 7f9d6b704640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9d64112d20 con 0x7f9d64103410 2024-05-14T17:05:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.756+0000 7f9d6b704640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9d64112ba0 con 0x7f9d64102a50 2024-05-14T17:05:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.756+0000 7f9d6b704640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9d64112ea0 con 0x7f9d64108a60 2024-05-14T17:05:57.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.756+0000 7f9d68c9f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d64103410 0x7f9d64199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.756+0000 7f9d68c9f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d64103410 0x7f9d64199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48120/0 (socket says 172.21.15.2:48120) 2024-05-14T17:05:57.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.756+0000 7f9d68c9f640 1 -- 172.21.15.2:0/3638457519 learned_addr learned my addr 172.21.15.2:0/3638457519 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:05:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.757+0000 7f9d69ca1640 1 --2- 172.21.15.2:0/3638457519 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d64108a60 0x7f9d6419dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:57.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.757+0000 7f9d694a0640 1 --2- 172.21.15.2:0/3638457519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d64102a50 0x7f9d64199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:57.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.757+0000 7f9d68c9f640 1 -- 172.21.15.2:0/3638457519 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d64108a60 msgr2=0x7f9d6419dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.757+0000 7f9d68c9f640 1 --2- 172.21.15.2:0/3638457519 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d64108a60 0x7f9d6419dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:57.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.757+0000 7f9d68c9f640 1 -- 172.21.15.2:0/3638457519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d64102a50 msgr2=0x7f9d64199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.757+0000 7f9d68c9f640 1 --2- 172.21.15.2:0/3638457519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d64102a50 0x7f9d64199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.757+0000 7f9d68c9f640 1 -- 172.21.15.2:0/3638457519 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d6419e1f0 con 0x7f9d64103410 2024-05-14T17:05:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.757+0000 7f9d694a0640 1 --2- 172.21.15.2:0/3638457519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d64102a50 0x7f9d64199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:05:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.757+0000 7f9d68c9f640 1 --2- 172.21.15.2:0/3638457519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d64103410 0x7f9d64199740 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9d54004770 tx=0x7f9d540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.757+0000 7f9d5a7fc640 1 -- 172.21.15.2:0/3638457519 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5403f070 con 0x7f9d64103410 2024-05-14T17:05:58.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.757+0000 7f9d5a7fc640 1 -- 172.21.15.2:0/3638457519 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d5403b460 con 0x7f9d64103410 2024-05-14T17:05:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.757+0000 7f9d6b704640 1 -- 172.21.15.2:0/3638457519 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d6419e480 con 0x7f9d64103410 2024-05-14T17:05:58.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.757+0000 7f9d5a7fc640 1 -- 172.21.15.2:0/3638457519 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d54035070 con 0x7f9d64103410 2024-05-14T17:05:58.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.757+0000 7f9d6b704640 1 -- 172.21.15.2:0/3638457519 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d641a5d70 con 0x7f9d64103410 2024-05-14T17:05:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.760+0000 7f9d6e419740 1 Processor -- start 2024-05-14T17:05:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.760+0000 7f9d5a7fc640 1 -- 172.21.15.2:0/3638457519 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f9d5403b8d0 con 0x7f9d64103410 2024-05-14T17:05:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.760+0000 7f9d6e419740 1 -- start start 2024-05-14T17:05:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.761+0000 7f9d6e419740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef003468f0 0x55ef00346cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:58.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.761+0000 7f9d6e419740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef00347210 0x55ef00351590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.761+0000 7f9d6e419740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef00351ad0 0x55ef00353ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.761+0000 7f9d6e419740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ef002b9c30 con 0x55ef003468f0 2024-05-14T17:05:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.761+0000 7f9d6e419740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ef002a97d0 con 0x55ef00347210 2024-05-14T17:05:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.761+0000 7f9d6e419740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ef002ad810 con 0x55ef00351ad0 2024-05-14T17:05:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.761+0000 7f9d69ca1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef003468f0 0x55ef00346cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:58.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.761+0000 7f9d69ca1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef003468f0 0x55ef00346cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48124/0 (socket says 172.21.15.2:48124) 2024-05-14T17:05:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.761+0000 7f9d69ca1640 1 -- 172.21.15.2:0/203993569 learned_addr learned my addr 172.21.15.2:0/203993569 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:05:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.761+0000 7f9d68c9f640 1 --2- 172.21.15.2:0/203993569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef00351ad0 0x55ef00353ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:59.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.761+0000 7f9d694a0640 1 --2- 172.21.15.2:0/203993569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef00347210 0x55ef00351590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:59.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.761+0000 7f9d69ca1640 1 -- 172.21.15.2:0/203993569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef00351ad0 msgr2=0x55ef00353ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.761+0000 7f9d69ca1640 1 --2- 172.21.15.2:0/203993569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef00351ad0 0x55ef00353ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:59.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.761+0000 7f9d69ca1640 1 -- 172.21.15.2:0/203993569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef00347210 msgr2=0x55ef00351590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:05:59.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.761+0000 7f9d69ca1640 1 --2- 172.21.15.2:0/203993569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef00347210 0x55ef00351590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:05:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.761+0000 7f9d69ca1640 1 -- 172.21.15.2:0/203993569 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eefffbe210 con 0x55ef003468f0 2024-05-14T17:05:59.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.761+0000 7f9d68c9f640 1 --2- 172.21.15.2:0/203993569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef00351ad0 0x55ef00353ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:05:59.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.761+0000 7f9d5a7fc640 1 --2- 172.21.15.2:0/3638457519 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f9d500b2a10 0x7f9d500b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:05:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.761+0000 7f9d69ca1640 1 --2- 172.21.15.2:0/203993569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef003468f0 0x55ef00346cd0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c010370 tx=0x7f9d5c010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bef6ed498ed66e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:05:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.761+0000 7f9d694a0640 1 --2- 172.21.15.2:0/3638457519 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f9d500b2a10 0x7f9d500b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:05:59.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.761+0000 7f9d5a7fc640 1 -- 172.21.15.2:0/3638457519 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f9d54102990 con 0x7f9d64103410 2024-05-14T17:05:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.762+0000 7f9d4f7fe640 1 -- 172.21.15.2:0/203993569 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c016070 con 0x55ef003468f0 2024-05-14T17:05:59.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.762+0000 7f9d4f7fe640 1 -- 172.21.15.2:0/203993569 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d5c004d20 con 0x55ef003468f0 2024-05-14T17:05:59.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.762+0000 7f9d4f7fe640 1 -- 172.21.15.2:0/203993569 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c005420 con 0x55ef003468f0 2024-05-14T17:05:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.762+0000 7f9d6e419740 1 -- 172.21.15.2:0/203993569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef003468f0 msgr2=0x55ef00346cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:05:59.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.762+0000 7f9d694a0640 1 --2- 172.21.15.2:0/3638457519 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f9d500b2a10 0x7f9d500b4ed0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x55ef00347e90 tx=0x7f9d6000b3a0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.762+0000 7f9d6e419740 1 --2- 172.21.15.2:0/203993569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef003468f0 0x55ef00346cd0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c010370 tx=0x7f9d5c010830 comp rx=0 tx=0).stop 2024-05-14T17:06:00.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.762+0000 7f9d6e419740 1 -- 172.21.15.2:0/203993569 shutdown_connections 2024-05-14T17:06:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.762+0000 7f9d6e419740 1 --2- 172.21.15.2:0/203993569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef00351ad0 0x55ef00353ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.762+0000 7f9d6e419740 1 --2- 172.21.15.2:0/203993569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef00347210 0x55ef00351590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.762+0000 7f9d6e419740 1 --2- 172.21.15.2:0/203993569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef003468f0 0x55ef00346cd0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.762+0000 7f9d6e419740 1 -- 172.21.15.2:0/203993569 >> 172.21.15.2:0/203993569 conn(0x55ef00342e00 msgr2=0x55ef003431e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.763+0000 7f9d6e419740 1 -- 172.21.15.2:0/203993569 shutdown_connections 2024-05-14T17:06:00.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.763+0000 7f9d6e419740 1 -- 172.21.15.2:0/203993569 wait complete. 2024-05-14T17:06:00.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.763+0000 7f9d6e419740 1 -- start start 2024-05-14T17:06:00.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.763+0000 7f9d6e419740 1 Processor -- start 2024-05-14T17:06:00.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.763+0000 7f9d6e419740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef003468f0 0x55ef00351500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.763+0000 7f9d6e419740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef00347210 0x55ef0034e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.763+0000 7f9d6e419740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef00351ad0 0x55ef0034ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.763+0000 7f9d6e419740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55eefff8f790 con 0x55ef00347210 2024-05-14T17:06:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.763+0000 7f9d6e419740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ef00265890 con 0x55ef00351ad0 2024-05-14T17:06:01.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.763+0000 7f9d6e419740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ef003547a0 con 0x55ef003468f0 2024-05-14T17:06:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.764+0000 7f9d6e419740 10 client.0 _ll_register_callbacks cb 0x55ef00268d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:06:01.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.764+0000 7f9d68c9f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef00347210 0x55ef0034e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.764+0000 7f9d68c9f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef00347210 0x55ef0034e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48138/0 (socket says 172.21.15.2:48138) 2024-05-14T17:06:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.764+0000 7f9d68c9f640 1 -- 172.21.15.2:0/4072332962 learned_addr learned my addr 172.21.15.2:0/4072332962 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:06:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.764+0000 7f9d69ca1640 1 --2- 172.21.15.2:0/4072332962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef00351ad0 0x55ef0034ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.764+0000 7f9d694a0640 1 --2- 172.21.15.2:0/4072332962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef003468f0 0x55ef00351500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.764+0000 7f9d68c9f640 1 -- 172.21.15.2:0/4072332962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef003468f0 msgr2=0x55ef00351500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.764+0000 7f9d68c9f640 1 --2- 172.21.15.2:0/4072332962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef003468f0 0x55ef00351500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.764+0000 7f9d68c9f640 1 -- 172.21.15.2:0/4072332962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef00351ad0 msgr2=0x55ef0034ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:01.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.764+0000 7f9d68c9f640 1 --2- 172.21.15.2:0/4072332962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef00351ad0 0x55ef0034ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.764+0000 7f9d68c9f640 1 -- 172.21.15.2:0/4072332962 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef0008a620 con 0x55ef00347210 2024-05-14T17:06:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.764+0000 7f9d68c9f640 1 --2- 172.21.15.2:0/4072332962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef00347210 0x55ef0034e7f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9d540c6ee0 tx=0x7f9d54005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aadc60c22939fa87 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:01.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.764+0000 7f9d4cff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:06:02.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.764+0000 7f9d4cff9640 1 -- 172.21.15.2:0/4072332962 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d54048050 con 0x55ef00347210 2024-05-14T17:06:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.764+0000 7f9d4cff9640 1 -- 172.21.15.2:0/4072332962 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f9d540c5f00 con 0x55ef00347210 2024-05-14T17:06:02.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.764+0000 7f9d6e419740 1 -- 172.21.15.2:0/4072332962 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d5c004d20 con 0x55ef00347210 2024-05-14T17:06:02.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.764+0000 7f9d4cff9640 1 -- 172.21.15.2:0/4072332962 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f9d540c95c0 con 0x55ef00347210 2024-05-14T17:06:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.764+0000 7f9d6e419740 10 client.16506 fetch_fsmap learned FSMap version 44 2024-05-14T17:06:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.764+0000 7f9d6e419740 1 -- 172.21.15.2:0/4072332962 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ef00360270 con 0x55ef00347210 2024-05-14T17:06:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.765+0000 7f9d4cff9640 1 -- 172.21.15.2:0/4072332962 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d5c004d20 con 0x55ef00347210 2024-05-14T17:06:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.765+0000 7f9d4cff9640 1 -- 172.21.15.2:0/4072332962 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 44) ==== 8350+0+0 (secure 0 0 0) 0x7f9d5404b060 con 0x55ef00347210 2024-05-14T17:06:02.618 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.765+0000 7f9d6e419740 10 client.16506 fetch_fsmap finished waiting for FSMap version 44 2024-05-14T17:06:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.765+0000 7f9d6e419740 10 client.16506 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:06:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.765+0000 7f9d6e419740 20 client.16506 populate_metadata read hostname 'smithi002' 2024-05-14T17:06:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.765+0000 7f9d6e419740 1 --2- 172.21.15.2:0/4072332962 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ef00361750 0x55ef00363bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.765+0000 7f9d6e419740 4 client.16506 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:06:03.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.765+0000 7f9d6e419740 1 -- 172.21.15.2:0/4072332962 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef002bbb80 con 0x55ef00361750 2024-05-14T17:06:03.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.765+0000 7f9d694a0640 1 --2- 172.21.15.2:0/4072332962 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ef00361750 0x55ef00363bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.766+0000 7f9d694a0640 1 --2- 172.21.15.2:0/4072332962 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ef00361750 0x55ef00363bb0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd20461cef388eb1 server_cookie=40ecf650068b6c52 in_seq=0 out_seq=0 2024-05-14T17:06:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.766+0000 7f9d4cff9640 10 client.16506 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:06:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.766+0000 7f9d4cff9640 1 -- 172.21.15.2:0/4072332962 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ef002bbb80 con 0x55ef00361750 2024-05-14T17:06:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.766+0000 7f9d4cff9640 10 client.16506 handle_command_reply: tid=0 2024-05-14T17:06:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.766+0000 7f9d6e419740 1 client.16506 shutdown 2024-05-14T17:06:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.767+0000 7f9d6e419740 1 -- 172.21.15.2:0/4072332962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef00347210 msgr2=0x55ef0034e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.767+0000 7f9d6e419740 1 --2- 172.21.15.2:0/4072332962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef00347210 0x55ef0034e7f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9d540c6ee0 tx=0x7f9d54005be0 comp rx=0 tx=0).stop 2024-05-14T17:06:03.661 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.767+0000 7f9d6e419740 1 -- 172.21.15.2:0/4072332962 shutdown_connections 2024-05-14T17:06:03.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.767+0000 7f9d6e419740 1 --2- 172.21.15.2:0/4072332962 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ef00361750 0x55ef00363bb0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:03.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.767+0000 7f9d6e419740 1 --2- 172.21.15.2:0/4072332962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef00351ad0 0x55ef0034ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:03.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.767+0000 7f9d4cff9640 0 client.16506 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:06:03.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.767+0000 7f9d6e419740 1 --2- 172.21.15.2:0/4072332962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef00347210 0x55ef0034e7f0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:03.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.767+0000 7f9d6e419740 1 --2- 172.21.15.2:0/4072332962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef003468f0 0x55ef00351500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.767+0000 7f9d6e419740 1 -- 172.21.15.2:0/4072332962 >> 172.21.15.2:0/4072332962 conn(0x55ef00342e00 msgr2=0x55ef00342100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.767+0000 7f9d6e419740 1 -- 172.21.15.2:0/4072332962 shutdown_connections 2024-05-14T17:06:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.767+0000 7f9d6e419740 1 -- 172.21.15.2:0/4072332962 wait complete. 2024-05-14T17:06:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.767+0000 7f9d6e419740 20 client.16506 trim_cache size 0 max 16384 2024-05-14T17:06:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.782+0000 7f9d6e419740 1 Processor -- start 2024-05-14T17:06:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.782+0000 7f9d6e419740 1 -- start start 2024-05-14T17:06:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.783+0000 7f9d6e419740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef003468f0 0x55ef00351030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:04.549 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.783+0000 7f9d6e419740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef00347210 0x55ef00351570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.783+0000 7f9d6e419740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef00351ad0 0x55ef0035d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.783+0000 7f9d6e419740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9d641b0fa0 con 0x55ef003468f0 2024-05-14T17:06:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.783+0000 7f9d6e419740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9d641ae810 con 0x55ef00351ad0 2024-05-14T17:06:04.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.783+0000 7f9d6e419740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9d641ac680 con 0x55ef00347210 2024-05-14T17:06:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.783+0000 7f9d694a0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef003468f0 0x55ef00351030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.783+0000 7f9d694a0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef003468f0 0x55ef00351030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48144/0 (socket says 172.21.15.2:48144) 2024-05-14T17:06:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.783+0000 7f9d694a0640 1 -- 172.21.15.2:0/1225457329 learned_addr learned my addr 172.21.15.2:0/1225457329 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:06:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.783+0000 7f9d69ca1640 1 --2- 172.21.15.2:0/1225457329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef00351ad0 0x55ef0035d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:05.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.783+0000 7f9d68c9f640 1 --2- 172.21.15.2:0/1225457329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef00347210 0x55ef00351570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.783+0000 7f9d694a0640 1 -- 172.21.15.2:0/1225457329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef00347210 msgr2=0x55ef00351570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.783+0000 7f9d694a0640 1 --2- 172.21.15.2:0/1225457329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef00347210 0x55ef00351570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:05.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.783+0000 7f9d694a0640 1 -- 172.21.15.2:0/1225457329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef00351ad0 msgr2=0x55ef0035d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.783+0000 7f9d694a0640 1 --2- 172.21.15.2:0/1225457329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef00351ad0 0x55ef0035d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.783+0000 7f9d694a0640 1 -- 172.21.15.2:0/1225457329 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef0034f2a0 con 0x55ef003468f0 2024-05-14T17:06:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.783+0000 7f9d694a0640 1 --2- 172.21.15.2:0/1225457329 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef003468f0 0x55ef00351030 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x55ef00364420 tx=0x7f9d60014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a26a55f35cbb686 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.783+0000 7f9d4effd640 1 -- 172.21.15.2:0/1225457329 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6002e070 con 0x55ef003468f0 2024-05-14T17:06:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.783+0000 7f9d4effd640 1 -- 172.21.15.2:0/1225457329 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d600211d0 con 0x55ef003468f0 2024-05-14T17:06:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.783+0000 7f9d4effd640 1 -- 172.21.15.2:0/1225457329 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d60021b40 con 0x55ef003468f0 2024-05-14T17:06:05.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.784+0000 7f9d6e419740 1 -- 172.21.15.2:0/1225457329 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef003468f0 msgr2=0x55ef00351030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.784+0000 7f9d6e419740 1 --2- 172.21.15.2:0/1225457329 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef003468f0 0x55ef00351030 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x55ef00364420 tx=0x7f9d60014000 comp rx=0 tx=0).stop 2024-05-14T17:06:06.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.784+0000 7f9d6e419740 1 -- 172.21.15.2:0/1225457329 shutdown_connections 2024-05-14T17:06:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.784+0000 7f9d6e419740 1 --2- 172.21.15.2:0/1225457329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef00351ad0 0x55ef0035d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.784+0000 7f9d6e419740 1 --2- 172.21.15.2:0/1225457329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef00347210 0x55ef00351570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:06.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.784+0000 7f9d6e419740 1 --2- 172.21.15.2:0/1225457329 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef003468f0 0x55ef00351030 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.784+0000 7f9d6e419740 1 -- 172.21.15.2:0/1225457329 >> 172.21.15.2:0/1225457329 conn(0x55ef00342e00 msgr2=0x55ef0034d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.784+0000 7f9d6e419740 1 -- 172.21.15.2:0/1225457329 shutdown_connections 2024-05-14T17:06:06.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.784+0000 7f9d6e419740 1 -- 172.21.15.2:0/1225457329 wait complete. 2024-05-14T17:06:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.784+0000 7f9d6e419740 1 -- start start 2024-05-14T17:06:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.784+0000 7f9d6e419740 1 Processor -- start 2024-05-14T17:06:07.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.785+0000 7f9d6e419740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef003468f0 0x55ef0035a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.785+0000 7f9d6e419740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef00347210 0x55ef00357430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.785+0000 7f9d6e419740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef00351ad0 0x55ef00357b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.785+0000 7f9d6e419740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9d641accd0 con 0x55ef00347210 2024-05-14T17:06:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.785+0000 7f9d6e419740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9d641ae140 con 0x55ef00351ad0 2024-05-14T17:06:07.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.785+0000 7f9d6e419740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9d641ad3a0 con 0x55ef003468f0 2024-05-14T17:06:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.785+0000 7f9d6e419740 10 client.0 _ll_register_callbacks cb 0x7f9d54009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:06:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.785+0000 7f9d68c9f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef00347210 0x55ef00357430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.785+0000 7f9d68c9f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef00347210 0x55ef00357430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48152/0 (socket says 172.21.15.2:48152) 2024-05-14T17:06:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.785+0000 7f9d68c9f640 1 -- 172.21.15.2:0/342962836 learned_addr learned my addr 172.21.15.2:0/342962836 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:06:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.785+0000 7f9d69ca1640 1 --2- 172.21.15.2:0/342962836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef00351ad0 0x55ef00357b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.785+0000 7f9d694a0640 1 --2- 172.21.15.2:0/342962836 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef003468f0 0x55ef0035a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:07.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.785+0000 7f9d68c9f640 1 -- 172.21.15.2:0/342962836 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef003468f0 msgr2=0x55ef0035a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.785+0000 7f9d68c9f640 1 --2- 172.21.15.2:0/342962836 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef003468f0 0x55ef0035a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.785+0000 7f9d68c9f640 1 -- 172.21.15.2:0/342962836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef00351ad0 msgr2=0x55ef00357b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:08.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.785+0000 7f9d68c9f640 1 --2- 172.21.15.2:0/342962836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef00351ad0 0x55ef00357b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.785+0000 7f9d68c9f640 1 -- 172.21.15.2:0/342962836 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef003cb040 con 0x55ef00347210 2024-05-14T17:06:08.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.785+0000 7f9d68c9f640 1 --2- 172.21.15.2:0/342962836 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef00347210 0x55ef00357430 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f9d540ce1e0 tx=0x7f9d5403bc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a192390c882cdd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:08.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.785+0000 7f9d4cff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:06:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.786+0000 7f9d4cff9640 1 -- 172.21.15.2:0/342962836 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d540e8060 con 0x55ef00347210 2024-05-14T17:06:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.786+0000 7f9d4cff9640 1 -- 172.21.15.2:0/342962836 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f9d54105ff0 con 0x55ef00347210 2024-05-14T17:06:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.786+0000 7f9d6e419740 1 -- 172.21.15.2:0/342962836 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d600211d0 con 0x55ef00347210 2024-05-14T17:06:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.786+0000 7f9d4cff9640 1 -- 172.21.15.2:0/342962836 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f9d54048070 con 0x55ef00347210 2024-05-14T17:06:08.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.786+0000 7f9d6e419740 10 client.16512 fetch_fsmap learned FSMap version 44 2024-05-14T17:06:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.786+0000 7f9d6e419740 1 -- 172.21.15.2:0/342962836 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ef003cc1b0 con 0x55ef00347210 2024-05-14T17:06:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.786+0000 7f9d4cff9640 1 -- 172.21.15.2:0/342962836 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d600211d0 con 0x55ef00347210 2024-05-14T17:06:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.786+0000 7f9d4cff9640 1 -- 172.21.15.2:0/342962836 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 44) ==== 8350+0+0 (secure 0 0 0) 0x7f9d540c8940 con 0x55ef00347210 2024-05-14T17:06:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.786+0000 7f9d6e419740 10 client.16512 fetch_fsmap finished waiting for FSMap version 44 2024-05-14T17:06:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.786+0000 7f9d6e419740 10 client.16512 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:06:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.786+0000 7f9d6e419740 20 client.16512 populate_metadata read hostname 'smithi002' 2024-05-14T17:06:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.786+0000 7f9d6e419740 1 --2- 172.21.15.2:0/342962836 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ef003cc7f0 0x55ef003cec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.786+0000 7f9d6e419740 4 client.16512 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:06:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.786+0000 7f9d6e419740 1 -- 172.21.15.2:0/342962836 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ef002b5360 con 0x55ef003cc7f0 2024-05-14T17:06:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.786+0000 7f9d694a0640 1 --2- 172.21.15.2:0/342962836 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ef003cc7f0 0x55ef003cec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.787+0000 7f9d4cff9640 10 client.16512 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:06:09.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.787+0000 7f9d694a0640 1 --2- 172.21.15.2:0/342962836 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ef003cc7f0 0x55ef003cec10 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b7a84ef43727093 server_cookie=8408c6b02837d1da in_seq=0 out_seq=0 2024-05-14T17:06:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.797+0000 7f9d4cff9640 1 -- 172.21.15.2:0/342962836 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ef002b5360 con 0x55ef003cc7f0 2024-05-14T17:06:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.797+0000 7f9d4cff9640 10 client.16512 handle_command_reply: tid=0 2024-05-14T17:06:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.798+0000 7f9d6e419740 1 client.16512 shutdown 2024-05-14T17:06:09.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.798+0000 7f9d6e419740 1 -- 172.21.15.2:0/342962836 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef00347210 msgr2=0x55ef00357430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:09.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.798+0000 7f9d6e419740 1 --2- 172.21.15.2:0/342962836 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef00347210 0x55ef00357430 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f9d540ce1e0 tx=0x7f9d5403bc70 comp rx=0 tx=0).stop 2024-05-14T17:06:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.798+0000 7f9d6e419740 1 -- 172.21.15.2:0/342962836 shutdown_connections 2024-05-14T17:06:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.798+0000 7f9d6e419740 1 --2- 172.21.15.2:0/342962836 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ef003cc7f0 0x55ef003cec10 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.798+0000 7f9d6e419740 1 --2- 172.21.15.2:0/342962836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef00351ad0 0x55ef00357b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.798+0000 7f9d6e419740 1 --2- 172.21.15.2:0/342962836 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef00347210 0x55ef00357430 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.798+0000 7f9d4cff9640 0 client.16512 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:06:10.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.798+0000 7f9d6e419740 1 --2- 172.21.15.2:0/342962836 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef003468f0 0x55ef0035a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.798+0000 7f9d6e419740 1 -- 172.21.15.2:0/342962836 >> 172.21.15.2:0/342962836 conn(0x55ef00342e00 msgr2=0x55ef00350020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.798+0000 7f9d6e419740 1 -- 172.21.15.2:0/342962836 shutdown_connections 2024-05-14T17:06:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.798+0000 7f9d6e419740 1 -- 172.21.15.2:0/342962836 wait complete. 2024-05-14T17:06:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.798+0000 7f9d6e419740 20 client.16512 trim_cache size 0 max 16384 2024-05-14T17:06:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:06:10.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:06:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:06:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.799+0000 7f9d6b704640 1 -- 172.21.15.2:0/3638457519 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f9d500b2a10 msgr2=0x7f9d500b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:11.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.799+0000 7f9d6b704640 1 --2- 172.21.15.2:0/3638457519 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f9d500b2a10 0x7f9d500b4ed0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x55ef00347e90 tx=0x7f9d6000b3a0 comp rx=0 tx=0).stop 2024-05-14T17:06:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.799+0000 7f9d6b704640 1 -- 172.21.15.2:0/3638457519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d64103410 msgr2=0x7f9d64199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.799+0000 7f9d6b704640 1 --2- 172.21.15.2:0/3638457519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d64103410 0x7f9d64199740 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9d54004770 tx=0x7f9d540042f0 comp rx=0 tx=0).stop 2024-05-14T17:06:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.799+0000 7f9d6b704640 1 -- 172.21.15.2:0/3638457519 shutdown_connections 2024-05-14T17:06:11.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.799+0000 7f9d6b704640 1 --2- 172.21.15.2:0/3638457519 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f9d500b2a10 0x7f9d500b4ed0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.799+0000 7f9d6b704640 1 --2- 172.21.15.2:0/3638457519 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d64108a60 0x7f9d6419dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:11.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.799+0000 7f9d6b704640 1 --2- 172.21.15.2:0/3638457519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d64103410 0x7f9d64199740 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:11.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.799+0000 7f9d6b704640 1 --2- 172.21.15.2:0/3638457519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d64102a50 0x7f9d64199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.799+0000 7f9d6b704640 1 -- 172.21.15.2:0/3638457519 >> 172.21.15.2:0/3638457519 conn(0x7f9d640fe520 msgr2=0x7f9d640fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.799+0000 7f9d6b704640 1 -- 172.21.15.2:0/3638457519 shutdown_connections 2024-05-14T17:06:11.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:43.799+0000 7f9d6b704640 1 -- 172.21.15.2:0/3638457519 wait complete. 2024-05-14T17:06:11.843 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:44 smithi002 ceph-mon[31538]: pgmap v342: 97 pgs: 97 active+clean; 4.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.8 MiB/s wr, 540 op/s 2024-05-14T17:06:11.883 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 630.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:06:11.964 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:46 smithi002 ceph-mon[31538]: pgmap v343: 97 pgs: 97 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 14 MiB/s wr, 710 op/s 2024-05-14T17:06:12.039 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:47 smithi002 ceph-mon[31538]: pgmap v344: 97 pgs: 97 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 14 MiB/s wr, 710 op/s 2024-05-14T17:06:12.110 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:50 smithi002 ceph-mon[31538]: pgmap v345: 97 pgs: 97 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 14 MiB/s wr, 716 op/s 2024-05-14T17:06:12.169 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 635.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:06:12.290 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:52 smithi002 ceph-mon[31538]: pgmap v346: 97 pgs: 97 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 13 MiB/s wr, 602 op/s 2024-05-14T17:06:12.381 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:54 smithi002 ceph-mon[31538]: pgmap v347: 97 pgs: 97 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 13 MiB/s wr, 605 op/s 2024-05-14T17:06:12.477 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:06:12.556 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:56 smithi123 ceph-mon[35971]: pgmap v348: 97 pgs: 97 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.1 MiB/s wr, 187 op/s 2024-05-14T17:06:12.593 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:58 smithi123 ceph-mon[35971]: pgmap v349: 97 pgs: 97 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-05-14T17:06:12.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:05:58 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/2682417187' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:06:12.693 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:06:00 smithi123 ceph-mon[35971]: pgmap v350: 97 pgs: 97 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-05-14T17:06:12.740 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:06:02 smithi123 ceph-mon[35971]: pgmap v351: 97 pgs: 97 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 20 KiB/s wr, 9 op/s 2024-05-14T17:06:12.813 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:06:04 smithi123 ceph-mon[35971]: pgmap v352: 97 pgs: 97 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 69 KiB/s wr, 29 op/s 2024-05-14T17:06:12.925 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:06:04 smithi123 ceph-mon[35971]: osdmap e90: 12 total, 12 up, 12 in 2024-05-14T17:06:12.960 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:06:04 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:06:12.993 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:06:05 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:06:13.051 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:06:05 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:06:13.158 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:06:05 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:06:13.235 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:06:06 smithi123 ceph-mon[35971]: pgmap v354: 97 pgs: 4 active+clean+snaptrim, 81 active+clean, 12 active+clean+snaptrim_wait; 4.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 78 KiB/s wr, 30 op/s 2024-05-14T17:06:13.283 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:06:08 smithi123 ceph-mon[35971]: pgmap v355: 97 pgs: 4 active+clean+snaptrim, 81 active+clean, 12 active+clean+snaptrim_wait; 4.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 78 KiB/s wr, 30 op/s 2024-05-14T17:06:13.318 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:06:09 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:06:13.352 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:06:10 smithi123 ceph-mon[35971]: pgmap v356: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 10 MiB/s wr, 467 op/s 2024-05-14T17:06:13.385 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:56 smithi151 ceph-mon[36066]: pgmap v348: 97 pgs: 97 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.1 MiB/s wr, 187 op/s 2024-05-14T17:06:13.418 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:58 smithi151 ceph-mon[36066]: pgmap v349: 97 pgs: 97 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-05-14T17:06:13.452 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:05:58 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/2682417187' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:06:13.485 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:06:00 smithi151 ceph-mon[36066]: pgmap v350: 97 pgs: 97 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-05-14T17:06:13.622 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:06:02 smithi151 ceph-mon[36066]: pgmap v351: 97 pgs: 97 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 20 KiB/s wr, 9 op/s 2024-05-14T17:06:13.693 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:06:04 smithi151 ceph-mon[36066]: pgmap v352: 97 pgs: 97 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 69 KiB/s wr, 29 op/s 2024-05-14T17:06:13.775 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:06:04 smithi151 ceph-mon[36066]: osdmap e90: 12 total, 12 up, 12 in 2024-05-14T17:06:13.835 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:06:04 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:06:13.984 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:06:05 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:06:14.100 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:06:05 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:06:14.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:06:05 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:06:14.185 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:06:06 smithi151 ceph-mon[36066]: pgmap v354: 97 pgs: 4 active+clean+snaptrim, 81 active+clean, 12 active+clean+snaptrim_wait; 4.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 78 KiB/s wr, 30 op/s 2024-05-14T17:06:14.219 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:06:08 smithi151 ceph-mon[36066]: pgmap v355: 97 pgs: 4 active+clean+snaptrim, 81 active+clean, 12 active+clean+snaptrim_wait; 4.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 78 KiB/s wr, 30 op/s 2024-05-14T17:06:14.252 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:06:09 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:06:14.323 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:06:10 smithi151 ceph-mon[36066]: pgmap v356: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 10 MiB/s wr, 467 op/s 2024-05-14T17:06:14.385 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:56 smithi002 ceph-mon[31538]: pgmap v348: 97 pgs: 97 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.1 MiB/s wr, 187 op/s 2024-05-14T17:06:14.448 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 640.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:06:14.614 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:06:12 smithi123 ceph-mon[35971]: pgmap v357: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 10 MiB/s wr, 467 op/s 2024-05-14T17:06:14.685 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:06:12 smithi123 ceph-mon[35971]: osdmap e91: 12 total, 12 up, 12 in 2024-05-14T17:06:14.824 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:06:12 smithi151 ceph-mon[36066]: pgmap v357: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 10 MiB/s wr, 467 op/s 2024-05-14T17:06:14.891 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:06:12 smithi151 ceph-mon[36066]: osdmap e91: 12 total, 12 up, 12 in 2024-05-14T17:06:14.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:06:14 smithi123 ceph-mon[35971]: pgmap v359: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 454 op/s 2024-05-14T17:06:15.003 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:06:14 smithi151 ceph-mon[36066]: pgmap v359: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 454 op/s 2024-05-14T17:06:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.903+0000 7f4a5a571640 1 -- 172.21.15.2:0/219051872 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a5410d0b0 msgr2=0x7f4a5410f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:15.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.903+0000 7f4a5a571640 1 --2- 172.21.15.2:0/219051872 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a5410d0b0 0x7f4a5410f4e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c009ca0 tx=0x7f4a4c02a950 comp rx=0 tx=0).stop 2024-05-14T17:06:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.903+0000 7f4a5a571640 1 -- 172.21.15.2:0/219051872 shutdown_connections 2024-05-14T17:06:15.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.903+0000 7f4a5a571640 1 --2- 172.21.15.2:0/219051872 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a5410d0b0 0x7f4a5410f4e0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.903+0000 7f4a5a571640 1 --2- 172.21.15.2:0/219051872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a54104600 0x7f4a5410cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.903+0000 7f4a5a571640 1 --2- 172.21.15.2:0/219051872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a54103ce0 0x7f4a541040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.903+0000 7f4a5a571640 1 -- 172.21.15.2:0/219051872 >> 172.21.15.2:0/219051872 conn(0x7f4a54078b60 msgr2=0x7f4a54078f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.903+0000 7f4a5a571640 1 -- 172.21.15.2:0/219051872 shutdown_connections 2024-05-14T17:06:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.903+0000 7f4a5a571640 1 -- 172.21.15.2:0/219051872 wait complete. 2024-05-14T17:06:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.904+0000 7f4a5a571640 1 Processor -- start 2024-05-14T17:06:15.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.904+0000 7f4a5a571640 1 -- start start 2024-05-14T17:06:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.904+0000 7f4a5a571640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a54103ce0 0x7f4a54199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.904+0000 7f4a5a571640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a54104600 0x7f4a541996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:16.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.904+0000 7f4a5a571640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a5410d0b0 0x7f4a5419da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.904+0000 7f4a5a571640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4a54112cf0 con 0x7f4a54103ce0 2024-05-14T17:06:16.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.904+0000 7f4a5a571640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4a54112b70 con 0x7f4a54104600 2024-05-14T17:06:16.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.904+0000 7f4a5a571640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4a54112e70 con 0x7f4a5410d0b0 2024-05-14T17:06:16.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.904+0000 7f4a53fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a54103ce0 0x7f4a54199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.904+0000 7f4a53fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a54103ce0 0x7f4a54199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35516/0 (socket says 172.21.15.2:35516) 2024-05-14T17:06:16.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.904+0000 7f4a53fff640 1 -- 172.21.15.2:0/1163847173 learned_addr learned my addr 172.21.15.2:0/1163847173 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:06:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.905+0000 7f4a537fe640 1 --2- 172.21.15.2:0/1163847173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a54104600 0x7f4a541996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.905+0000 7f4a58b0e640 1 --2- 172.21.15.2:0/1163847173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a5410d0b0 0x7f4a5419da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.905+0000 7f4a53fff640 1 -- 172.21.15.2:0/1163847173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a5410d0b0 msgr2=0x7f4a5419da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.905+0000 7f4a53fff640 1 --2- 172.21.15.2:0/1163847173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a5410d0b0 0x7f4a5419da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.905+0000 7f4a53fff640 1 -- 172.21.15.2:0/1163847173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a54104600 msgr2=0x7f4a541996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.905+0000 7f4a53fff640 1 --2- 172.21.15.2:0/1163847173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a54104600 0x7f4a541996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.905+0000 7f4a53fff640 1 -- 172.21.15.2:0/1163847173 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a5419e130 con 0x7f4a54103ce0 2024-05-14T17:06:17.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.905+0000 7f4a58b0e640 1 --2- 172.21.15.2:0/1163847173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a5410d0b0 0x7f4a5419da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:06:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.905+0000 7f4a53fff640 1 --2- 172.21.15.2:0/1163847173 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a54103ce0 0x7f4a54199190 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f4a4400e9f0 tx=0x7f4a4400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:17.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.905+0000 7f4a517fa640 1 -- 172.21.15.2:0/1163847173 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4400cd80 con 0x7f4a54103ce0 2024-05-14T17:06:17.178 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.905+0000 7f4a5a571640 1 -- 172.21.15.2:0/1163847173 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a5419e420 con 0x7f4a54103ce0 2024-05-14T17:06:17.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.905+0000 7f4a517fa640 1 -- 172.21.15.2:0/1163847173 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a440054d0 con 0x7f4a54103ce0 2024-05-14T17:06:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.905+0000 7f4a517fa640 1 -- 172.21.15.2:0/1163847173 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a44010690 con 0x7f4a54103ce0 2024-05-14T17:06:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.905+0000 7f4a5a571640 1 -- 172.21.15.2:0/1163847173 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a541a5cb0 con 0x7f4a54103ce0 2024-05-14T17:06:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.908+0000 7f4a5d365740 1 Processor -- start 2024-05-14T17:06:17.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.908+0000 7f4a5d365740 1 -- start start 2024-05-14T17:06:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.908+0000 7f4a517fa640 1 -- 172.21.15.2:0/1163847173 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f4a44004e90 con 0x7f4a54103ce0 2024-05-14T17:06:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.908+0000 7f4a5d365740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d7948f0 0x56070d794cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.908+0000 7f4a5d365740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56070d795210 0x56070d79f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:17.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.908+0000 7f4a5d365740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56070d79fad0 0x56070d7a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:17.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.908+0000 7f4a5d365740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56070d707c30 con 0x56070d79fad0 2024-05-14T17:06:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.908+0000 7f4a5d365740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56070d6f77d0 con 0x56070d795210 2024-05-14T17:06:18.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.908+0000 7f4a5d365740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56070d6fb810 con 0x56070d7948f0 2024-05-14T17:06:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.909+0000 7f4a537fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56070d79fad0 0x56070d7a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:18.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.909+0000 7f4a537fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56070d79fad0 0x56070d7a1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35524/0 (socket says 172.21.15.2:35524) 2024-05-14T17:06:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.909+0000 7f4a537fe640 1 -- 172.21.15.2:0/1213221168 learned_addr learned my addr 172.21.15.2:0/1213221168 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:06:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.909+0000 7f4a58b0e640 1 --2- 172.21.15.2:0/1213221168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d7948f0 0x56070d794cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.909+0000 7f4a53fff640 1 --2- 172.21.15.2:0/1213221168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56070d795210 0x56070d79f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.909+0000 7f4a537fe640 1 -- 172.21.15.2:0/1213221168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d7948f0 msgr2=0x56070d794cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:18.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.909+0000 7f4a537fe640 1 --2- 172.21.15.2:0/1213221168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d7948f0 0x56070d794cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:18.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.909+0000 7f4a537fe640 1 -- 172.21.15.2:0/1213221168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56070d795210 msgr2=0x56070d79f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.909+0000 7f4a537fe640 1 --2- 172.21.15.2:0/1213221168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56070d795210 0x56070d79f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.909+0000 7f4a537fe640 1 -- 172.21.15.2:0/1213221168 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56070d40c210 con 0x56070d79fad0 2024-05-14T17:06:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.909+0000 7f4a53fff640 1 --2- 172.21.15.2:0/1213221168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56070d795210 0x56070d79f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:06:18.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.909+0000 7f4a537fe640 1 --2- 172.21.15.2:0/1213221168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56070d79fad0 0x56070d7a1ec0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f4a4800bc30 tx=0x7f4a480107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2695f8d165adcbf0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.909+0000 7f4a3a7fc640 1 -- 172.21.15.2:0/1213221168 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4801c480 con 0x56070d79fad0 2024-05-14T17:06:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.909+0000 7f4a3a7fc640 1 -- 172.21.15.2:0/1213221168 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a4801cad0 con 0x56070d79fad0 2024-05-14T17:06:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.909+0000 7f4a3a7fc640 1 -- 172.21.15.2:0/1213221168 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a48011a90 con 0x56070d79fad0 2024-05-14T17:06:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.910+0000 7f4a517fa640 1 --2- 172.21.15.2:0/1163847173 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f4a3c0b2a10 0x7f4a3c0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:19.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.910+0000 7f4a58b0e640 1 --2- 172.21.15.2:0/1163847173 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f4a3c0b2a10 0x7f4a3c0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.910+0000 7f4a517fa640 1 -- 172.21.15.2:0/1163847173 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f4a440e2ac0 con 0x7f4a54103ce0 2024-05-14T17:06:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.910+0000 7f4a5d365740 1 -- 172.21.15.2:0/1213221168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56070d79fad0 msgr2=0x56070d7a1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:19.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.910+0000 7f4a5d365740 1 --2- 172.21.15.2:0/1213221168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56070d79fad0 0x56070d7a1ec0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f4a4800bc30 tx=0x7f4a480107a0 comp rx=0 tx=0).stop 2024-05-14T17:06:19.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.910+0000 7f4a58b0e640 1 --2- 172.21.15.2:0/1163847173 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f4a3c0b2a10 0x7f4a3c0b4ed0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c004af0 tx=0x7f4a4c005d80 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.910+0000 7f4a5d365740 1 -- 172.21.15.2:0/1213221168 shutdown_connections 2024-05-14T17:06:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.910+0000 7f4a5d365740 1 --2- 172.21.15.2:0/1213221168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56070d79fad0 0x56070d7a1ec0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.910+0000 7f4a5d365740 1 --2- 172.21.15.2:0/1213221168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56070d795210 0x56070d79f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.910+0000 7f4a5d365740 1 --2- 172.21.15.2:0/1213221168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d7948f0 0x56070d794cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.910+0000 7f4a5d365740 1 -- 172.21.15.2:0/1213221168 >> 172.21.15.2:0/1213221168 conn(0x56070d790e00 msgr2=0x56070d7911e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.910+0000 7f4a5d365740 1 -- 172.21.15.2:0/1213221168 shutdown_connections 2024-05-14T17:06:20.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.910+0000 7f4a5d365740 1 -- 172.21.15.2:0/1213221168 wait complete. 2024-05-14T17:06:20.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.911+0000 7f4a5d365740 1 -- start start 2024-05-14T17:06:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.911+0000 7f4a5d365740 1 Processor -- start 2024-05-14T17:06:20.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.911+0000 7f4a5d365740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56070d7948f0 0x56070d79f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.911+0000 7f4a5d365740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d795210 0x56070d79c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:20.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.911+0000 7f4a5d365740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56070d79fad0 0x56070d79cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.911+0000 7f4a5d365740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56070d3dd790 con 0x56070d7948f0 2024-05-14T17:06:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.911+0000 7f4a5d365740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56070d6b3890 con 0x56070d79fad0 2024-05-14T17:06:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.911+0000 7f4a5d365740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56070d7a27a0 con 0x56070d795210 2024-05-14T17:06:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.911+0000 7f4a5d365740 10 client.0 _ll_register_callbacks cb 0x56070d6b6d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:06:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.911+0000 7f4a53fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56070d7948f0 0x56070d79f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.911+0000 7f4a53fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56070d7948f0 0x56070d79f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35530/0 (socket says 172.21.15.2:35530) 2024-05-14T17:06:21.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.911+0000 7f4a53fff640 1 -- 172.21.15.2:0/2162178026 learned_addr learned my addr 172.21.15.2:0/2162178026 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:06:21.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.912+0000 7f4a537fe640 1 --2- 172.21.15.2:0/2162178026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d795210 0x56070d79c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:21.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.912+0000 7f4a58b0e640 1 --2- 172.21.15.2:0/2162178026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56070d79fad0 0x56070d79cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.912+0000 7f4a53fff640 1 -- 172.21.15.2:0/2162178026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d795210 msgr2=0x56070d79c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.912+0000 7f4a53fff640 1 --2- 172.21.15.2:0/2162178026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d795210 0x56070d79c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.912+0000 7f4a53fff640 1 -- 172.21.15.2:0/2162178026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56070d79fad0 msgr2=0x56070d79cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:21.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.912+0000 7f4a53fff640 1 --2- 172.21.15.2:0/2162178026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56070d79fad0 0x56070d79cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:21.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.912+0000 7f4a53fff640 1 -- 172.21.15.2:0/2162178026 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56070d4d8620 con 0x56070d7948f0 2024-05-14T17:06:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.912+0000 7f4a53fff640 1 --2- 172.21.15.2:0/2162178026 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56070d7948f0 0x56070d79f400 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4a440e0270 tx=0x7f4a440a9c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab4143f6e6a7c741 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.912+0000 7f4a337fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:06:21.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.912+0000 7f4a337fe640 1 -- 172.21.15.2:0/2162178026 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a44005180 con 0x56070d7948f0 2024-05-14T17:06:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.912+0000 7f4a337fe640 1 -- 172.21.15.2:0/2162178026 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f4a440b2770 con 0x56070d7948f0 2024-05-14T17:06:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.912+0000 7f4a5d365740 1 -- 172.21.15.2:0/2162178026 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a4801cad0 con 0x56070d7948f0 2024-05-14T17:06:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.912+0000 7f4a337fe640 1 -- 172.21.15.2:0/2162178026 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f4a440a98b0 con 0x56070d7948f0 2024-05-14T17:06:22.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.912+0000 7f4a5d365740 10 client.16524 fetch_fsmap learned FSMap version 44 2024-05-14T17:06:22.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.912+0000 7f4a5d365740 1 -- 172.21.15.2:0/2162178026 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x56070d7ae200 con 0x56070d7948f0 2024-05-14T17:06:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.913+0000 7f4a337fe640 1 -- 172.21.15.2:0/2162178026 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a4801cad0 con 0x56070d7948f0 2024-05-14T17:06:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.913+0000 7f4a337fe640 1 -- 172.21.15.2:0/2162178026 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 44) ==== 8350+0+0 (secure 0 0 0) 0x7f4a440a5710 con 0x56070d7948f0 2024-05-14T17:06:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.913+0000 7f4a5d365740 10 client.16524 fetch_fsmap finished waiting for FSMap version 44 2024-05-14T17:06:22.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.913+0000 7f4a5d365740 10 client.16524 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:06:22.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.913+0000 7f4a5d365740 20 client.16524 populate_metadata read hostname 'smithi002' 2024-05-14T17:06:22.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.913+0000 7f4a5d365740 1 --2- 172.21.15.2:0/2162178026 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56070d7af730 0x56070d7b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:22.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.913+0000 7f4a5d365740 4 client.16524 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:06:22.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.913+0000 7f4a5d365740 1 -- 172.21.15.2:0/2162178026 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56070d709b80 con 0x56070d7af730 2024-05-14T17:06:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.913+0000 7f4a537fe640 1 --2- 172.21.15.2:0/2162178026 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56070d7af730 0x56070d7b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.914+0000 7f4a337fe640 10 client.16524 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:06:22.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.914+0000 7f4a537fe640 1 --2- 172.21.15.2:0/2162178026 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56070d7af730 0x56070d7b1b20 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=340a95d975be560d server_cookie=cf059400325c2cd7 in_seq=0 out_seq=0 2024-05-14T17:06:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.914+0000 7f4a337fe640 1 -- 172.21.15.2:0/2162178026 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56070d709b80 con 0x56070d7af730 2024-05-14T17:06:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.914+0000 7f4a337fe640 10 client.16524 handle_command_reply: tid=0 2024-05-14T17:06:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.914+0000 7f4a5d365740 1 client.16524 shutdown 2024-05-14T17:06:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.914+0000 7f4a5d365740 1 -- 172.21.15.2:0/2162178026 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56070d7948f0 msgr2=0x56070d79f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.914+0000 7f4a5d365740 1 --2- 172.21.15.2:0/2162178026 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56070d7948f0 0x56070d79f400 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4a440e0270 tx=0x7f4a440a9c30 comp rx=0 tx=0).stop 2024-05-14T17:06:23.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.915+0000 7f4a5d365740 1 -- 172.21.15.2:0/2162178026 shutdown_connections 2024-05-14T17:06:23.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.915+0000 7f4a5d365740 1 --2- 172.21.15.2:0/2162178026 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56070d7af730 0x56070d7b1b20 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.915+0000 7f4a5d365740 1 --2- 172.21.15.2:0/2162178026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56070d79fad0 0x56070d79cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.915+0000 7f4a5d365740 1 --2- 172.21.15.2:0/2162178026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d795210 0x56070d79c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.915+0000 7f4a5d365740 1 --2- 172.21.15.2:0/2162178026 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56070d7948f0 0x56070d79f400 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:23.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.915+0000 7f4a337fe640 0 client.16524 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:06:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.915+0000 7f4a5d365740 1 -- 172.21.15.2:0/2162178026 >> 172.21.15.2:0/2162178026 conn(0x56070d790e00 msgr2=0x56070d790100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:23.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.915+0000 7f4a5d365740 1 -- 172.21.15.2:0/2162178026 shutdown_connections 2024-05-14T17:06:23.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.915+0000 7f4a5d365740 1 -- 172.21.15.2:0/2162178026 wait complete. 2024-05-14T17:06:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.915+0000 7f4a5d365740 20 client.16524 trim_cache size 0 max 16384 2024-05-14T17:06:24.178 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.930+0000 7f4a5d365740 1 Processor -- start 2024-05-14T17:06:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.930+0000 7f4a5d365740 1 -- start start 2024-05-14T17:06:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.930+0000 7f4a5d365740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56070d7948f0 0x56070d790880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.930+0000 7f4a5d365740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d795210 0x56070d797dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.930+0000 7f4a5d365740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56070d79fad0 0x56070d79ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:24.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.930+0000 7f4a5d365740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4a541b0ee0 con 0x56070d7948f0 2024-05-14T17:06:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.930+0000 7f4a5d365740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4a541ae750 con 0x56070d79fad0 2024-05-14T17:06:24.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.930+0000 7f4a5d365740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4a541ac5c0 con 0x56070d795210 2024-05-14T17:06:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.931+0000 7f4a53fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56070d7948f0 0x56070d790880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:24.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.931+0000 7f4a53fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56070d7948f0 0x56070d790880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35542/0 (socket says 172.21.15.2:35542) 2024-05-14T17:06:24.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.931+0000 7f4a53fff640 1 -- 172.21.15.2:0/2848585465 learned_addr learned my addr 172.21.15.2:0/2848585465 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:06:24.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.931+0000 7f4a537fe640 1 --2- 172.21.15.2:0/2848585465 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d795210 0x56070d797dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:25.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.931+0000 7f4a58b0e640 1 --2- 172.21.15.2:0/2848585465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56070d79fad0 0x56070d79ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:25.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.931+0000 7f4a53fff640 1 -- 172.21.15.2:0/2848585465 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d795210 msgr2=0x56070d797dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.931+0000 7f4a53fff640 1 --2- 172.21.15.2:0/2848585465 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d795210 0x56070d797dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.931+0000 7f4a53fff640 1 -- 172.21.15.2:0/2848585465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56070d79fad0 msgr2=0x56070d79ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.931+0000 7f4a53fff640 1 --2- 172.21.15.2:0/2848585465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56070d79fad0 0x56070d79ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.931+0000 7f4a53fff640 1 -- 172.21.15.2:0/2848585465 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56070d79ac40 con 0x56070d7948f0 2024-05-14T17:06:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.931+0000 7f4a58b0e640 1 --2- 172.21.15.2:0/2848585465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56070d79fad0 0x56070d79ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:06:25.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.931+0000 7f4a53fff640 1 --2- 172.21.15.2:0/2848585465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56070d7948f0 0x56070d790880 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f4a440a7010 tx=0x7f4a440a5d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0424cb6a06ef65d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.931+0000 7f4a39ffb640 1 -- 172.21.15.2:0/2848585465 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a44005180 con 0x56070d7948f0 2024-05-14T17:06:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.931+0000 7f4a39ffb640 1 -- 172.21.15.2:0/2848585465 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a440aa4a0 con 0x56070d7948f0 2024-05-14T17:06:25.548 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.931+0000 7f4a39ffb640 1 -- 172.21.15.2:0/2848585465 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a440b2770 con 0x56070d7948f0 2024-05-14T17:06:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.932+0000 7f4a5d365740 1 -- 172.21.15.2:0/2848585465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56070d7948f0 msgr2=0x56070d790880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:25.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.932+0000 7f4a5d365740 1 --2- 172.21.15.2:0/2848585465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56070d7948f0 0x56070d790880 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f4a440a7010 tx=0x7f4a440a5d60 comp rx=0 tx=0).stop 2024-05-14T17:06:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.932+0000 7f4a5d365740 1 -- 172.21.15.2:0/2848585465 shutdown_connections 2024-05-14T17:06:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.932+0000 7f4a5d365740 1 --2- 172.21.15.2:0/2848585465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56070d79fad0 0x56070d79ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.932+0000 7f4a5d365740 1 --2- 172.21.15.2:0/2848585465 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d795210 0x56070d797dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.932+0000 7f4a5d365740 1 --2- 172.21.15.2:0/2848585465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56070d7948f0 0x56070d790880 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:26.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.932+0000 7f4a5d365740 1 -- 172.21.15.2:0/2848585465 >> 172.21.15.2:0/2848585465 conn(0x56070d790e00 msgr2=0x56070d7962a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.932+0000 7f4a5d365740 1 -- 172.21.15.2:0/2848585465 shutdown_connections 2024-05-14T17:06:26.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.932+0000 7f4a5d365740 1 -- 172.21.15.2:0/2848585465 wait complete. 2024-05-14T17:06:26.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.933+0000 7f4a5d365740 1 -- start start 2024-05-14T17:06:26.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.933+0000 7f4a5d365740 1 Processor -- start 2024-05-14T17:06:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.933+0000 7f4a5d365740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56070d7948f0 0x56070d7a3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.933+0000 7f4a5d365740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56070d795210 0x56070d7a3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:26.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.933+0000 7f4a5d365740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d79fad0 0x56070d7a42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:26.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.933+0000 7f4a5d365740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4a541acc10 con 0x56070d795210 2024-05-14T17:06:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.933+0000 7f4a5d365740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4a541ae080 con 0x56070d7948f0 2024-05-14T17:06:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.933+0000 7f4a5d365740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4a541ad2e0 con 0x56070d79fad0 2024-05-14T17:06:26.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.933+0000 7f4a5d365740 10 client.0 _ll_register_callbacks cb 0x7f4a44005680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:06:27.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.934+0000 7f4a537fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56070d795210 0x56070d7a3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:27.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.934+0000 7f4a537fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56070d795210 0x56070d7a3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35548/0 (socket says 172.21.15.2:35548) 2024-05-14T17:06:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.934+0000 7f4a537fe640 1 -- 172.21.15.2:0/1701185645 learned_addr learned my addr 172.21.15.2:0/1701185645 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:06:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.934+0000 7f4a53fff640 1 --2- 172.21.15.2:0/1701185645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56070d7948f0 0x56070d7a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:27.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.934+0000 7f4a58b0e640 1 --2- 172.21.15.2:0/1701185645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d79fad0 0x56070d7a42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.934+0000 7f4a537fe640 1 -- 172.21.15.2:0/1701185645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d79fad0 msgr2=0x56070d7a42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.934+0000 7f4a537fe640 1 --2- 172.21.15.2:0/1701185645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d79fad0 0x56070d7a42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:27.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.934+0000 7f4a537fe640 1 -- 172.21.15.2:0/1701185645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56070d7948f0 msgr2=0x56070d7a3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:27.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.934+0000 7f4a537fe640 1 --2- 172.21.15.2:0/1701185645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56070d7948f0 0x56070d7a3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.934+0000 7f4a537fe640 1 -- 172.21.15.2:0/1701185645 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56070d819230 con 0x56070d795210 2024-05-14T17:06:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.934+0000 7f4a58b0e640 1 --2- 172.21.15.2:0/1701185645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d79fad0 0x56070d7a42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:06:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.934+0000 7f4a537fe640 1 --2- 172.21.15.2:0/1701185645 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56070d795210 0x56070d7a3d70 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f4a48011010 tx=0x7f4a4800bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5909d1ab4322c001 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:27.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.934+0000 7f4a337fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:06:27.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.934+0000 7f4a337fe640 1 -- 172.21.15.2:0/1701185645 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a48005610 con 0x56070d795210 2024-05-14T17:06:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.934+0000 7f4a337fe640 1 -- 172.21.15.2:0/1701185645 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7f4a48004680 con 0x56070d795210 2024-05-14T17:06:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.934+0000 7f4a5d365740 1 -- 172.21.15.2:0/1701185645 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a440aa4a0 con 0x56070d795210 2024-05-14T17:06:27.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.935+0000 7f4a337fe640 1 -- 172.21.15.2:0/1701185645 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f4a48004020 con 0x56070d795210 2024-05-14T17:06:27.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.935+0000 7f4a5d365740 10 client.16530 fetch_fsmap learned FSMap version 44 2024-05-14T17:06:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.935+0000 7f4a5d365740 1 -- 172.21.15.2:0/1701185645 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x56070d81a4b0 con 0x56070d795210 2024-05-14T17:06:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.935+0000 7f4a337fe640 1 -- 172.21.15.2:0/1701185645 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a440aa4a0 con 0x56070d795210 2024-05-14T17:06:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.935+0000 7f4a337fe640 1 -- 172.21.15.2:0/1701185645 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 44) ==== 8350+0+0 (secure 0 0 0) 0x7f4a48003780 con 0x56070d795210 2024-05-14T17:06:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.935+0000 7f4a5d365740 10 client.16530 fetch_fsmap finished waiting for FSMap version 44 2024-05-14T17:06:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.935+0000 7f4a5d365740 10 client.16530 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:06:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.935+0000 7f4a5d365740 20 client.16530 populate_metadata read hostname 'smithi002' 2024-05-14T17:06:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.935+0000 7f4a5d365740 1 --2- 172.21.15.2:0/1701185645 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56070d81a9a0 0x56070d81cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:28.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.935+0000 7f4a5d365740 4 client.16530 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:06:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.935+0000 7f4a5d365740 1 -- 172.21.15.2:0/1701185645 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56070d703360 con 0x56070d81a9a0 2024-05-14T17:06:28.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.935+0000 7f4a53fff640 1 --2- 172.21.15.2:0/1701185645 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56070d81a9a0 0x56070d81cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.936+0000 7f4a337fe640 10 client.16530 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:06:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.936+0000 7f4a53fff640 1 --2- 172.21.15.2:0/1701185645 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56070d81a9a0 0x56070d81cdc0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=652245544563b643 server_cookie=472e14c7a7f567c7 in_seq=0 out_seq=0 2024-05-14T17:06:28.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.948+0000 7f4a337fe640 1 -- 172.21.15.2:0/1701185645 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56070d703360 con 0x56070d81a9a0 2024-05-14T17:06:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.948+0000 7f4a337fe640 10 client.16530 handle_command_reply: tid=0 2024-05-14T17:06:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.948+0000 7f4a5d365740 1 client.16530 shutdown 2024-05-14T17:06:29.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.948+0000 7f4a5d365740 1 -- 172.21.15.2:0/1701185645 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56070d795210 msgr2=0x56070d7a3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:29.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.948+0000 7f4a5d365740 1 --2- 172.21.15.2:0/1701185645 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56070d795210 0x56070d7a3d70 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f4a48011010 tx=0x7f4a4800bfc0 comp rx=0 tx=0).stop 2024-05-14T17:06:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.949+0000 7f4a5d365740 1 -- 172.21.15.2:0/1701185645 shutdown_connections 2024-05-14T17:06:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.949+0000 7f4a5d365740 1 --2- 172.21.15.2:0/1701185645 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56070d81a9a0 0x56070d81cdc0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:29.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.949+0000 7f4a5d365740 1 --2- 172.21.15.2:0/1701185645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56070d79fad0 0x56070d7a42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.949+0000 7f4a5d365740 1 --2- 172.21.15.2:0/1701185645 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56070d795210 0x56070d7a3d70 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:29.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.949+0000 7f4a337fe640 0 client.16530 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:06:29.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.949+0000 7f4a5d365740 1 --2- 172.21.15.2:0/1701185645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56070d7948f0 0x56070d7a3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.949+0000 7f4a5d365740 1 -- 172.21.15.2:0/1701185645 >> 172.21.15.2:0/1701185645 conn(0x56070d790e00 msgr2=0x56070d797c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.949+0000 7f4a5d365740 1 -- 172.21.15.2:0/1701185645 shutdown_connections 2024-05-14T17:06:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.949+0000 7f4a5d365740 1 -- 172.21.15.2:0/1701185645 wait complete. 2024-05-14T17:06:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.949+0000 7f4a5d365740 20 client.16530 trim_cache size 0 max 16384 2024-05-14T17:06:30.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:06:30.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:06:30.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:06:30.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.949+0000 7f4a5a571640 1 -- 172.21.15.2:0/1163847173 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f4a3c0b2a10 msgr2=0x7f4a3c0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.949+0000 7f4a5a571640 1 --2- 172.21.15.2:0/1163847173 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f4a3c0b2a10 0x7f4a3c0b4ed0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c004af0 tx=0x7f4a4c005d80 comp rx=0 tx=0).stop 2024-05-14T17:06:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.949+0000 7f4a5a571640 1 -- 172.21.15.2:0/1163847173 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a54103ce0 msgr2=0x7f4a54199190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.949+0000 7f4a5a571640 1 --2- 172.21.15.2:0/1163847173 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a54103ce0 0x7f4a54199190 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f4a4400e9f0 tx=0x7f4a4400edf0 comp rx=0 tx=0).stop 2024-05-14T17:06:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.949+0000 7f4a5a571640 1 -- 172.21.15.2:0/1163847173 shutdown_connections 2024-05-14T17:06:30.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.949+0000 7f4a5a571640 1 --2- 172.21.15.2:0/1163847173 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f4a3c0b2a10 0x7f4a3c0b4ed0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.950+0000 7f4a5a571640 1 --2- 172.21.15.2:0/1163847173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a5410d0b0 0x7f4a5419da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:30.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.950+0000 7f4a5a571640 1 --2- 172.21.15.2:0/1163847173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a54104600 0x7f4a541996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:31.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.950+0000 7f4a5a571640 1 --2- 172.21.15.2:0/1163847173 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a54103ce0 0x7f4a54199190 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.950+0000 7f4a5a571640 1 -- 172.21.15.2:0/1163847173 >> 172.21.15.2:0/1163847173 conn(0x7f4a54078b60 msgr2=0x7f4a540fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.950+0000 7f4a5a571640 1 -- 172.21.15.2:0/1163847173 shutdown_connections 2024-05-14T17:06:31.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:05:56.950+0000 7f4a5a571640 1 -- 172.21.15.2:0/1163847173 wait complete. 2024-05-14T17:06:31.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.467+0000 7feada625700 1 -- 172.21.15.2:0/2512598923 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fead406bbe0 msgr2=0x7fead41103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:31.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.467+0000 7feada625700 1 --2- 172.21.15.2:0/2512598923 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fead406bbe0 0x7fead41103d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7feacc00b0a0 tx=0x7feacc00b3b0 comp rx=0 tx=0).stop 2024-05-14T17:06:31.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.467+0000 7feada625700 1 -- 172.21.15.2:0/2512598923 shutdown_connections 2024-05-14T17:06:31.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.467+0000 7feada625700 1 --2- 172.21.15.2:0/2512598923 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fead406bbe0 0x7fead41103d0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:31.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.467+0000 7feada625700 1 --2- 172.21.15.2:0/2512598923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fead41095b0 0x7fead4109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:31.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.467+0000 7feada625700 1 --2- 172.21.15.2:0/2512598923 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fead406b220 0x7fead406b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:31.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.467+0000 7feada625700 1 -- 172.21.15.2:0/2512598923 >> 172.21.15.2:0/2512598923 conn(0x7fead40793f0 msgr2=0x7fead4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:32.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.467+0000 7feada625700 1 -- 172.21.15.2:0/2512598923 shutdown_connections 2024-05-14T17:06:32.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.467+0000 7feada625700 1 -- 172.21.15.2:0/2512598923 wait complete. 2024-05-14T17:06:32.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.468+0000 7feada625700 1 Processor -- start 2024-05-14T17:06:32.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.468+0000 7feada625700 1 -- start start 2024-05-14T17:06:32.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.468+0000 7feada625700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fead406b220 0x7fead419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:32.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.468+0000 7feada625700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fead406bbe0 0x7fead419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:32.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.468+0000 7feada625700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fead41095b0 0x7fead41999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:32.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.468+0000 7feada625700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fead41136f0 con 0x7fead41095b0 2024-05-14T17:06:32.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.468+0000 7feada625700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fead4113570 con 0x7fead406b220 2024-05-14T17:06:32.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.468+0000 7feada625700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fead4113870 con 0x7fead406bbe0 2024-05-14T17:06:32.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.468+0000 7fead8bc2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fead41095b0 0x7fead41999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:32.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.468+0000 7fead8bc2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fead41095b0 0x7fead41999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35578/0 (socket says 172.21.15.2:35578) 2024-05-14T17:06:32.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.469+0000 7fead8bc2700 1 -- 172.21.15.2:0/2682417187 learned_addr learned my addr 172.21.15.2:0/2682417187 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:06:32.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.469+0000 7fead37fe700 1 --2- 172.21.15.2:0/2682417187 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fead406bbe0 0x7fead419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:33.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.469+0000 7fead3fff700 1 --2- 172.21.15.2:0/2682417187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fead406b220 0x7fead419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:33.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.469+0000 7fead8bc2700 1 -- 172.21.15.2:0/2682417187 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fead406bbe0 msgr2=0x7fead419fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:33.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.469+0000 7fead8bc2700 1 --2- 172.21.15.2:0/2682417187 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fead406bbe0 0x7fead419fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:33.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.469+0000 7fead8bc2700 1 -- 172.21.15.2:0/2682417187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fead406b220 msgr2=0x7fead419f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:33.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.469+0000 7fead8bc2700 1 --2- 172.21.15.2:0/2682417187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fead406b220 0x7fead419f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:33.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.469+0000 7fead8bc2700 1 -- 172.21.15.2:0/2682417187 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fead419a150 con 0x7fead41095b0 2024-05-14T17:06:33.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.469+0000 7fead8bc2700 1 --2- 172.21.15.2:0/2682417187 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fead41095b0 0x7fead41999f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7feacc00b830 tx=0x7feacc004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:33.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.469+0000 7fead17fa700 1 -- 172.21.15.2:0/2682417187 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc021070 con 0x7fead41095b0 2024-05-14T17:06:33.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.469+0000 7feada625700 1 -- 172.21.15.2:0/2682417187 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fead419a3e0 con 0x7fead41095b0 2024-05-14T17:06:33.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.469+0000 7fead17fa700 1 -- 172.21.15.2:0/2682417187 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feacc007e30 con 0x7fead41095b0 2024-05-14T17:06:33.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.469+0000 7fead17fa700 1 -- 172.21.15.2:0/2682417187 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc004700 con 0x7fead41095b0 2024-05-14T17:06:33.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.469+0000 7feada625700 1 -- 172.21.15.2:0/2682417187 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fead41a66b0 con 0x7fead41095b0 2024-05-14T17:06:34.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.470+0000 7feada625700 1 -- 172.21.15.2:0/2682417187 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fead406a230 con 0x7fead41095b0 2024-05-14T17:06:34.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.472+0000 7fead17fa700 1 -- 172.21.15.2:0/2682417187 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7feacc007980 con 0x7fead41095b0 2024-05-14T17:06:34.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.472+0000 7fead17fa700 1 --2- 172.21.15.2:0/2682417187 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7feabc0b29f0 0x7feabc0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:34.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.472+0000 7fead3fff700 1 --2- 172.21.15.2:0/2682417187 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7feabc0b29f0 0x7feabc0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:34.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.472+0000 7fead17fa700 1 -- 172.21.15.2:0/2682417187 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8513+0+0 (secure 0 0 0) 0x7feacc00f020 con 0x7fead41095b0 2024-05-14T17:06:34.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.473+0000 7fead3fff700 1 --2- 172.21.15.2:0/2682417187 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7feabc0b29f0 0x7feabc0b4eb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fead4100b10 tx=0x7feac400b410 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:34.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.477+0000 7fead17fa700 1 -- 172.21.15.2:0/2682417187 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7feacc02a020 con 0x7fead41095b0 2024-05-14T17:06:34.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.743+0000 7feada625700 1 -- 172.21.15.2:0/2682417187 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fead419aed0 con 0x7fead41095b0 2024-05-14T17:06:34.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.744+0000 7fead17fa700 1 -- 172.21.15.2:0/2682417187 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11854 (secure 0 0 0) 0x7feacc0ab550 con 0x7fead41095b0 2024-05-14T17:06:34.637 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:06:34.669 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2275696923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2275696923},{"type":"v1","addr":"172.21.15.2:6835","nonce":2275696923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/716718761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":716718761},{"type":"v1","addr":"172.21.15.2:6839","nonce":716718761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/3644815179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3644815179},{"type":"v1","addr":"172.21.15.2:6841","nonce":3644815179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1880336731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1880336731},{"type":"v1","addr":"172.21.15.151:6833","nonce":1880336731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2415605548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2415605548},{"type":"v1","addr":"172.21.15.2:6837","nonce":2415605548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2635167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2635167},{"type":"v1","addr":"172.21.15.123:6833","nonce":2635167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:51:34.106919+0000","modified":"2024-05-14T17:04:55.478062+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24455,"mds_2":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24415":{"gid":24415,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/762044339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":762044339},{"type":"v1","addr":"172.21.15.151:6835","nonce":762044339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/132043151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":132043151},{"type":"v1","addr":"172.21.15.151:6837","nonce":132043151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/4107834375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":4107834375},{"type":"v1","addr":"172.21.15.151:6839","nonce":4107834375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.123:6835/1198670013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1198670013},{"type":"v1","addr":"172.21.15.123:6835","nonce":1198670013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/333426645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":333426645},{"type":"v1","addr":"172.21.15.123:6837","nonce":333426645}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/580275877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":580275877},{"type":"v1","addr":"172.21.15.123:6839","nonce":580275877}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24431,24455]},"id":1}]} 2024-05-14T17:06:34.705 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:06:16 smithi123 ceph-mon[35971]: pgmap v360: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 450 op/s 2024-05-14T17:06:34.768 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:06:18 smithi123 ceph-mon[35971]: pgmap v361: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 450 op/s 2024-05-14T17:06:34.856 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:06:20 smithi123 ceph-mon[35971]: pgmap v362: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-05-14T17:06:34.939 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:06:22 smithi123 ceph-mon[35971]: pgmap v363: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-05-14T17:06:35.033 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:06:24 smithi123 ceph-mon[35971]: pgmap v364: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-05-14T17:06:35.101 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:06:26 smithi123 ceph-mon[35971]: pgmap v365: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 767 B/s wr, 5 op/s 2024-05-14T17:06:35.136 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:06:28 smithi123 ceph-mon[35971]: pgmap v366: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 767 B/s wr, 5 op/s 2024-05-14T17:06:35.245 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:06:30 smithi123 ceph-mon[35971]: pgmap v367: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-05-14T17:06:35.306 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:06:32 smithi123 ceph-mon[35971]: pgmap v368: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-05-14T17:06:35.472 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:06:16 smithi151 ceph-mon[36066]: pgmap v360: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 450 op/s 2024-05-14T17:06:35.553 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:06:18 smithi151 ceph-mon[36066]: pgmap v361: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 450 op/s 2024-05-14T17:06:35.611 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:06:20 smithi151 ceph-mon[36066]: pgmap v362: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-05-14T17:06:35.721 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:06:22 smithi151 ceph-mon[36066]: pgmap v363: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-05-14T17:06:35.796 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:06:24 smithi151 ceph-mon[36066]: pgmap v364: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-05-14T17:06:35.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:06:26 smithi151 ceph-mon[36066]: pgmap v365: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 767 B/s wr, 5 op/s 2024-05-14T17:06:35.993 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:06:28 smithi151 ceph-mon[36066]: pgmap v366: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 767 B/s wr, 5 op/s 2024-05-14T17:06:36.079 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:06:30 smithi151 ceph-mon[36066]: pgmap v367: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-05-14T17:06:36.155 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:06:32 smithi151 ceph-mon[36066]: pgmap v368: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-05-14T17:06:36.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.746+0000 7feada625700 1 -- 172.21.15.2:0/2682417187 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7feabc0b29f0 msgr2=0x7feabc0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:36.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.746+0000 7feada625700 1 --2- 172.21.15.2:0/2682417187 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7feabc0b29f0 0x7feabc0b4eb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fead4100b10 tx=0x7feac400b410 comp rx=0 tx=0).stop 2024-05-14T17:06:36.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.746+0000 7feada625700 1 -- 172.21.15.2:0/2682417187 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fead41095b0 msgr2=0x7fead41999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:36.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.746+0000 7feada625700 1 --2- 172.21.15.2:0/2682417187 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fead41095b0 0x7fead41999f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7feacc00b830 tx=0x7feacc004ce0 comp rx=0 tx=0).stop 2024-05-14T17:06:36.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.746+0000 7feada625700 1 -- 172.21.15.2:0/2682417187 shutdown_connections 2024-05-14T17:06:36.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.746+0000 7feada625700 1 --2- 172.21.15.2:0/2682417187 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7feabc0b29f0 0x7feabc0b4eb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:36.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.746+0000 7feada625700 1 --2- 172.21.15.2:0/2682417187 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fead41095b0 0x7fead41999f0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:36.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.746+0000 7feada625700 1 --2- 172.21.15.2:0/2682417187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fead406b220 0x7fead419f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:36.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.746+0000 7feada625700 1 --2- 172.21.15.2:0/2682417187 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fead406bbe0 0x7fead419fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:36.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.746+0000 7feada625700 1 -- 172.21.15.2:0/2682417187 >> 172.21.15.2:0/2682417187 conn(0x7fead40793f0 msgr2=0x7fead4101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:37.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.746+0000 7feada625700 1 -- 172.21.15.2:0/2682417187 shutdown_connections 2024-05-14T17:06:37.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:05:57.746+0000 7feada625700 1 -- 172.21.15.2:0/2682417187 wait complete. 2024-05-14T17:06:37.153 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 44 2024-05-14T17:06:37.215 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --set-id e0c7677c --release --await' 2024-05-14T17:06:37.278 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --set-id e0c7677c --release --await 2024-05-14T17:06:37.375 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:58 smithi002 ceph-mon[31538]: pgmap v349: 97 pgs: 97 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-05-14T17:06:37.511 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:05:58 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/2682417187' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:06:37.620 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:06:00 smithi002 ceph-mon[31538]: pgmap v350: 97 pgs: 97 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-05-14T17:06:37.712 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 645.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:06:37.761 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:06:02 smithi002 ceph-mon[31538]: pgmap v351: 97 pgs: 97 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 20 KiB/s wr, 9 op/s 2024-05-14T17:06:37.880 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:06:04 smithi002 ceph-mon[31538]: pgmap v352: 97 pgs: 97 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 69 KiB/s wr, 29 op/s 2024-05-14T17:06:37.947 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:06:04 smithi002 ceph-mon[31538]: osdmap e90: 12 total, 12 up, 12 in 2024-05-14T17:06:38.092 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:06:04 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:06:38.234 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:06:05 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:06:38.339 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:06:05 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:06:38.407 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:06:05 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:06:38.468 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:06:06 smithi002 ceph-mon[31538]: pgmap v354: 97 pgs: 4 active+clean+snaptrim, 81 active+clean, 12 active+clean+snaptrim_wait; 4.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 78 KiB/s wr, 30 op/s 2024-05-14T17:06:38.525 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 650.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:06:38.581 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:06:08 smithi002 ceph-mon[31538]: pgmap v355: 97 pgs: 4 active+clean+snaptrim, 81 active+clean, 12 active+clean+snaptrim_wait; 4.9 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 78 KiB/s wr, 30 op/s 2024-05-14T17:06:38.658 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:06:09 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:06:38.863 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:06:34 smithi123 ceph-mon[35971]: pgmap v369: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-05-14T17:06:38.914 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:06:36 smithi123 ceph-mon[35971]: pgmap v370: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 KiB/s wr, 8 op/s 2024-05-14T17:06:38.974 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:06:34 smithi151 ceph-mon[36066]: pgmap v369: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-05-14T17:06:39.078 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:06:36 smithi151 ceph-mon[36066]: pgmap v370: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 KiB/s wr, 8 op/s 2024-05-14T17:06:39.171 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:06:38 smithi151 ceph-mon[36066]: pgmap v371: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 KiB/s wr, 8 op/s 2024-05-14T17:06:39.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:06:38 smithi123 ceph-mon[35971]: pgmap v371: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 KiB/s wr, 8 op/s 2024-05-14T17:06:39.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.053+0000 7f834199e640 1 -- 172.21.15.2:0/4059265272 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f833c102ab0 msgr2=0x7f833c106380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:39.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.053+0000 7f834199e640 1 --2- 172.21.15.2:0/4059265272 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f833c102ab0 0x7f833c106380 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f832c009cd0 tx=0x7f832c02a8d0 comp rx=0 tx=0).stop 2024-05-14T17:06:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.053+0000 7f834199e640 1 -- 172.21.15.2:0/4059265272 shutdown_connections 2024-05-14T17:06:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.053+0000 7f834199e640 1 --2- 172.21.15.2:0/4059265272 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f833c106ab0 0x7f833c10e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.053+0000 7f834199e640 1 --2- 172.21.15.2:0/4059265272 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f833c102ab0 0x7f833c106380 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.053+0000 7f834199e640 1 --2- 172.21.15.2:0/4059265272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f833c102100 0x7f833c1024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:39.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.053+0000 7f834199e640 1 -- 172.21.15.2:0/4059265272 >> 172.21.15.2:0/4059265272 conn(0x7f833c077c90 msgr2=0x7f833c0ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:39.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.053+0000 7f834199e640 1 -- 172.21.15.2:0/4059265272 shutdown_connections 2024-05-14T17:06:39.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.053+0000 7f834199e640 1 -- 172.21.15.2:0/4059265272 wait complete. 2024-05-14T17:06:39.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.054+0000 7f834199e640 1 Processor -- start 2024-05-14T17:06:39.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.054+0000 7f834199e640 1 -- start start 2024-05-14T17:06:40.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.054+0000 7f834199e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f833c102100 0x7f833c199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:40.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.054+0000 7f834199e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f833c106ab0 0x7f833c199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.054+0000 7f834199e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f833c19d9c0 0x7f833c19de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:40.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.054+0000 7f834199e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f833c112b40 con 0x7f833c19d9c0 2024-05-14T17:06:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.054+0000 7f834199e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f833c1129c0 con 0x7f833c102100 2024-05-14T17:06:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.054+0000 7f834199e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f833c112cc0 con 0x7f833c106ab0 2024-05-14T17:06:40.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.055+0000 7f834119d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f833c19d9c0 0x7f833c19de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.055+0000 7f834119d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f833c19d9c0 0x7f833c19de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40066/0 (socket says 172.21.15.2:40066) 2024-05-14T17:06:40.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.055+0000 7f834119d640 1 -- 172.21.15.2:0/3488500794 learned_addr learned my addr 172.21.15.2:0/3488500794 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:06:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.055+0000 7f834099c640 1 --2- 172.21.15.2:0/3488500794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f833c102100 0x7f833c199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.055+0000 7f833bfff640 1 --2- 172.21.15.2:0/3488500794 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f833c106ab0 0x7f833c199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:40.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.055+0000 7f834119d640 1 -- 172.21.15.2:0/3488500794 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f833c106ab0 msgr2=0x7f833c199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.055+0000 7f834119d640 1 --2- 172.21.15.2:0/3488500794 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f833c106ab0 0x7f833c199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:40.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.055+0000 7f834119d640 1 -- 172.21.15.2:0/3488500794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f833c102100 msgr2=0x7f833c199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.055+0000 7f834119d640 1 --2- 172.21.15.2:0/3488500794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f833c102100 0x7f833c199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.055+0000 7f834119d640 1 -- 172.21.15.2:0/3488500794 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f833c19e4b0 con 0x7f833c19d9c0 2024-05-14T17:06:41.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.055+0000 7f834099c640 1 --2- 172.21.15.2:0/3488500794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f833c102100 0x7f833c199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:06:41.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.055+0000 7f834119d640 1 --2- 172.21.15.2:0/3488500794 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f833c19d9c0 0x7f833c19de70 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f833400d950 tx=0x7f833400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:41.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.055+0000 7f8339ffb640 1 -- 172.21.15.2:0/3488500794 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8334017030 con 0x7f833c19d9c0 2024-05-14T17:06:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.055+0000 7f8339ffb640 1 -- 172.21.15.2:0/3488500794 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8334004020 con 0x7f833c19d9c0 2024-05-14T17:06:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.055+0000 7f834199e640 1 -- 172.21.15.2:0/3488500794 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f833c19e7a0 con 0x7f833c19d9c0 2024-05-14T17:06:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.055+0000 7f8339ffb640 1 -- 172.21.15.2:0/3488500794 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8334003b50 con 0x7f833c19d9c0 2024-05-14T17:06:41.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.056+0000 7f834199e640 1 -- 172.21.15.2:0/3488500794 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f833c1a5f10 con 0x7f833c19d9c0 2024-05-14T17:06:41.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.058+0000 7f8339ffb640 1 -- 172.21.15.2:0/3488500794 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f833401a070 con 0x7f833c19d9c0 2024-05-14T17:06:41.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.058+0000 7f83459f1740 1 Processor -- start 2024-05-14T17:06:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.058+0000 7f83459f1740 1 -- start start 2024-05-14T17:06:41.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.059+0000 7f83459f1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654547549c0 0x565454754da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.059+0000 7f83459f1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5654547552e0 0x565454763b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.059+0000 7f83459f1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654547640d0 0x5654547664c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.059+0000 7f83459f1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5654546c7c30 con 0x5654547549c0 2024-05-14T17:06:41.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.059+0000 7f83459f1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5654546b77d0 con 0x5654547640d0 2024-05-14T17:06:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.059+0000 7f83459f1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5654546bb810 con 0x5654547552e0 2024-05-14T17:06:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.059+0000 7f833bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654547549c0 0x565454754da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.059+0000 7f833bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654547549c0 0x565454754da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40070/0 (socket says 172.21.15.2:40070) 2024-05-14T17:06:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.059+0000 7f833bfff640 1 -- 172.21.15.2:0/879412153 learned_addr learned my addr 172.21.15.2:0/879412153 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:06:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.059+0000 7f834099c640 1 --2- 172.21.15.2:0/879412153 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5654547552e0 0x565454763b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:42.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.059+0000 7f8339ffb640 1 --2- 172.21.15.2:0/3488500794 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f83280b2950 0x7f83280b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.059+0000 7f833bfff640 1 --2- 172.21.15.2:0/879412153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654547640d0 0x5654547664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.059+0000 7f833bfff640 1 -- 172.21.15.2:0/879412153 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5654547552e0 msgr2=0x565454763b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.059+0000 7f833bfff640 1 --2- 172.21.15.2:0/879412153 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5654547552e0 0x565454763b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.059+0000 7f833bfff640 1 -- 172.21.15.2:0/879412153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654547640d0 msgr2=0x5654547664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.059+0000 7f833bfff640 1 --2- 172.21.15.2:0/879412153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654547640d0 0x5654547664c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.059+0000 7f833bfff640 1 -- 172.21.15.2:0/879412153 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654543cc210 con 0x5654547549c0 2024-05-14T17:06:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.059+0000 7f834119d640 1 --2- 172.21.15.2:0/3488500794 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f83280b2950 0x7f83280b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:43.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.060+0000 7f8339ffb640 1 -- 172.21.15.2:0/3488500794 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8557+0+0 (secure 0 0 0) 0x7f83340e1cf0 con 0x7f833c19d9c0 2024-05-14T17:06:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.060+0000 7f833bfff640 1 --2- 172.21.15.2:0/879412153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654547549c0 0x565454754da0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f832c009cd0 tx=0x7f832c009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c5d20fe5b00255 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:43.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.060+0000 7f8332ffd640 1 -- 172.21.15.2:0/879412153 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c031420 con 0x5654547549c0 2024-05-14T17:06:43.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.060+0000 7f834119d640 1 --2- 172.21.15.2:0/3488500794 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f83280b2950 0x7f83280b4e10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f83340a6220 tx=0x7f83340d85c0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.060+0000 7f8332ffd640 1 -- 172.21.15.2:0/879412153 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f832c036040 con 0x5654547549c0 2024-05-14T17:06:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.060+0000 7f8332ffd640 1 -- 172.21.15.2:0/879412153 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c004390 con 0x5654547549c0 2024-05-14T17:06:43.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.061+0000 7f83459f1740 1 -- 172.21.15.2:0/879412153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654547549c0 msgr2=0x565454754da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.061+0000 7f83459f1740 1 --2- 172.21.15.2:0/879412153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654547549c0 0x565454754da0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f832c009cd0 tx=0x7f832c009360 comp rx=0 tx=0).stop 2024-05-14T17:06:43.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.061+0000 7f83459f1740 1 -- 172.21.15.2:0/879412153 shutdown_connections 2024-05-14T17:06:43.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.061+0000 7f83459f1740 1 --2- 172.21.15.2:0/879412153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654547640d0 0x5654547664c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:43.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.061+0000 7f83459f1740 1 --2- 172.21.15.2:0/879412153 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5654547552e0 0x565454763b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.061+0000 7f83459f1740 1 --2- 172.21.15.2:0/879412153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654547549c0 0x565454754da0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.061+0000 7f83459f1740 1 -- 172.21.15.2:0/879412153 >> 172.21.15.2:0/879412153 conn(0x565454750ff0 msgr2=0x5654547513d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:43.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.061+0000 7f83459f1740 1 -- 172.21.15.2:0/879412153 shutdown_connections 2024-05-14T17:06:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.061+0000 7f83459f1740 1 -- 172.21.15.2:0/879412153 wait complete. 2024-05-14T17:06:44.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.061+0000 7f83459f1740 1 -- start start 2024-05-14T17:06:44.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.061+0000 7f83459f1740 1 Processor -- start 2024-05-14T17:06:44.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.062+0000 7f83459f1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5654547549c0 0x565454763a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.062+0000 7f83459f1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654547552e0 0x565454760d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.062+0000 7f83459f1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654547640d0 0x565454761290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.062+0000 7f83459f1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56545439d790 con 0x5654547640d0 2024-05-14T17:06:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.062+0000 7f83459f1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x565454673890 con 0x5654547552e0 2024-05-14T17:06:44.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.062+0000 7f83459f1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x565454766d40 con 0x5654547549c0 2024-05-14T17:06:44.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.062+0000 7f83459f1740 10 client.0 _ll_register_callbacks cb 0x565454676d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:06:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.062+0000 7f833bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654547640d0 0x565454761290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.062+0000 7f833bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654547640d0 0x565454761290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40082/0 (socket says 172.21.15.2:40082) 2024-05-14T17:06:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.062+0000 7f833bfff640 1 -- 172.21.15.2:0/3599528187 learned_addr learned my addr 172.21.15.2:0/3599528187 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:06:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.062+0000 7f833bfff640 1 --2- 172.21.15.2:0/3599528187 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5654547549c0 0x565454763a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.062+0000 7f834099c640 1 --2- 172.21.15.2:0/3599528187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654547552e0 0x565454760d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:45.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.062+0000 7f833bfff640 1 -- 172.21.15.2:0/3599528187 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5654547549c0 msgr2=0x565454763a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.062+0000 7f833bfff640 1 --2- 172.21.15.2:0/3599528187 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5654547549c0 0x565454763a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.062+0000 7f833bfff640 1 -- 172.21.15.2:0/3599528187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654547552e0 msgr2=0x565454760d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.062+0000 7f833bfff640 1 --2- 172.21.15.2:0/3599528187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654547552e0 0x565454760d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:45.382 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.062+0000 7f833bfff640 1 -- 172.21.15.2:0/3599528187 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565454498620 con 0x5654547640d0 2024-05-14T17:06:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.062+0000 7f834099c640 1 --2- 172.21.15.2:0/3599528187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654547552e0 0x565454760d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:06:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.062+0000 7f833bfff640 1 --2- 172.21.15.2:0/3599528187 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654547640d0 0x565454761290 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f832c009430 tx=0x7f832c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e945d78e2b81ba76 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.062+0000 7f831bfff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:06:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.063+0000 7f831bfff640 1 -- 172.21.15.2:0/3599528187 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c054060 con 0x5654547640d0 2024-05-14T17:06:45.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.063+0000 7f831bfff640 1 -- 172.21.15.2:0/3599528187 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8557+0+0 (secure 0 0 0) 0x7f832c004780 con 0x5654547640d0 2024-05-14T17:06:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.063+0000 7f83459f1740 1 -- 172.21.15.2:0/3599528187 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f832c036040 con 0x5654547640d0 2024-05-14T17:06:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.063+0000 7f831bfff640 1 -- 172.21.15.2:0/3599528187 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f832c040070 con 0x5654547640d0 2024-05-14T17:06:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.063+0000 7f83459f1740 10 client.16548 fetch_fsmap learned FSMap version 45 2024-05-14T17:06:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.063+0000 7f83459f1740 1 -- 172.21.15.2:0/3599528187 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x565454772810 con 0x5654547640d0 2024-05-14T17:06:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.063+0000 7f831bfff640 1 -- 172.21.15.2:0/3599528187 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f832c036040 con 0x5654547640d0 2024-05-14T17:06:46.186 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.063+0000 7f831bfff640 1 -- 172.21.15.2:0/3599528187 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f832c030590 con 0x5654547640d0 2024-05-14T17:06:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.063+0000 7f83459f1740 10 client.16548 fetch_fsmap finished waiting for FSMap version 45 2024-05-14T17:06:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.063+0000 7f83459f1740 10 client.16548 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:06:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.063+0000 7f83459f1740 20 client.16548 populate_metadata read hostname 'smithi002' 2024-05-14T17:06:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.063+0000 7f83459f1740 1 --2- 172.21.15.2:0/3599528187 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x565454773b60 0x565454775f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:46.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.063+0000 7f83459f1740 4 client.16548 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:06:46.550 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.063+0000 7f83459f1740 1 -- 172.21.15.2:0/3599528187 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654546c9b80 con 0x565454773b60 2024-05-14T17:06:46.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.064+0000 7f834099c640 1 --2- 172.21.15.2:0/3599528187 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x565454773b60 0x565454775f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:46.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.064+0000 7f834099c640 1 --2- 172.21.15.2:0/3599528187 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x565454773b60 0x565454775f50 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=206801fb1714a818 server_cookie=7cc0398aed048b63 in_seq=0 out_seq=0 2024-05-14T17:06:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.064+0000 7f831bfff640 10 client.16548 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:06:46.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.065+0000 7f831bfff640 1 -- 172.21.15.2:0/3599528187 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5654546c9b80 con 0x565454773b60 2024-05-14T17:06:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.065+0000 7f831bfff640 10 client.16548 handle_command_reply: tid=0 2024-05-14T17:06:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.065+0000 7f83459f1740 1 client.16548 shutdown 2024-05-14T17:06:47.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.065+0000 7f83459f1740 1 -- 172.21.15.2:0/3599528187 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654547640d0 msgr2=0x565454761290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:47.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.065+0000 7f83459f1740 1 --2- 172.21.15.2:0/3599528187 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654547640d0 0x565454761290 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f832c009430 tx=0x7f832c000f40 comp rx=0 tx=0).stop 2024-05-14T17:06:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.065+0000 7f83459f1740 1 -- 172.21.15.2:0/3599528187 shutdown_connections 2024-05-14T17:06:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.065+0000 7f83459f1740 1 --2- 172.21.15.2:0/3599528187 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x565454773b60 0x565454775f50 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:47.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.065+0000 7f83459f1740 1 --2- 172.21.15.2:0/3599528187 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654547640d0 0x565454761290 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.065+0000 7f831bfff640 0 client.16548 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:06:47.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.065+0000 7f83459f1740 1 --2- 172.21.15.2:0/3599528187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654547552e0 0x565454760d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.065+0000 7f83459f1740 1 --2- 172.21.15.2:0/3599528187 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5654547549c0 0x565454763a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.065+0000 7f83459f1740 1 -- 172.21.15.2:0/3599528187 >> 172.21.15.2:0/3599528187 conn(0x565454750ff0 msgr2=0x565454750350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:47.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.065+0000 7f83459f1740 1 -- 172.21.15.2:0/3599528187 shutdown_connections 2024-05-14T17:06:47.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.065+0000 7f83459f1740 1 -- 172.21.15.2:0/3599528187 wait complete. 2024-05-14T17:06:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.065+0000 7f83459f1740 20 client.16548 trim_cache size 0 max 16384 2024-05-14T17:06:48.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.081+0000 7f83459f1740 1 Processor -- start 2024-05-14T17:06:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.081+0000 7f83459f1740 1 -- start start 2024-05-14T17:06:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.081+0000 7f83459f1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654547549c0 0x56545476fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.081+0000 7f83459f1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5654547552e0 0x565454763570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:48.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.081+0000 7f83459f1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654547640d0 0x56545477b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.081+0000 7f83459f1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f833c1aebf0 con 0x5654547549c0 2024-05-14T17:06:48.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.081+0000 7f83459f1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f833c1ac460 con 0x5654547640d0 2024-05-14T17:06:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.081+0000 7f83459f1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f833c1aa2d0 con 0x5654547552e0 2024-05-14T17:06:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.081+0000 7f833bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654547549c0 0x56545476fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.081+0000 7f833bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654547549c0 0x56545476fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40088/0 (socket says 172.21.15.2:40088) 2024-05-14T17:06:48.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.081+0000 7f833bfff640 1 -- 172.21.15.2:0/331053894 learned_addr learned my addr 172.21.15.2:0/331053894 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:06:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.081+0000 7f833bfff640 1 -- 172.21.15.2:0/331053894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5654547552e0 msgr2=0x565454763570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.081+0000 7f833bfff640 1 --2- 172.21.15.2:0/331053894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5654547552e0 0x565454763570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:49.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.081+0000 7f833bfff640 1 -- 172.21.15.2:0/331053894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654547640d0 msgr2=0x56545477b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.081+0000 7f833bfff640 1 --2- 172.21.15.2:0/331053894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654547640d0 0x56545477b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.081+0000 7f833bfff640 1 -- 172.21.15.2:0/331053894 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565454750e20 con 0x5654547549c0 2024-05-14T17:06:49.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.082+0000 7f833bfff640 1 --2- 172.21.15.2:0/331053894 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654547549c0 0x56545476fc80 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f832c009980 tx=0x7f832c005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a2de9d9aff093ef server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.082+0000 7f83327fc640 1 -- 172.21.15.2:0/331053894 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c054040 con 0x5654547549c0 2024-05-14T17:06:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.082+0000 7f83327fc640 1 -- 172.21.15.2:0/331053894 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f832c036040 con 0x5654547549c0 2024-05-14T17:06:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.082+0000 7f83327fc640 1 -- 172.21.15.2:0/331053894 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c030590 con 0x5654547549c0 2024-05-14T17:06:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.082+0000 7f83459f1740 1 -- 172.21.15.2:0/331053894 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654547549c0 msgr2=0x56545476fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.082+0000 7f83459f1740 1 --2- 172.21.15.2:0/331053894 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654547549c0 0x56545476fc80 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f832c009980 tx=0x7f832c005bf0 comp rx=0 tx=0).stop 2024-05-14T17:06:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.082+0000 7f83459f1740 1 -- 172.21.15.2:0/331053894 shutdown_connections 2024-05-14T17:06:49.943 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.082+0000 7f83459f1740 1 --2- 172.21.15.2:0/331053894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654547640d0 0x56545477b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:50.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.082+0000 7f83459f1740 1 --2- 172.21.15.2:0/331053894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5654547552e0 0x565454763570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.082+0000 7f83459f1740 1 --2- 172.21.15.2:0/331053894 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654547549c0 0x56545476fc80 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.082+0000 7f83459f1740 1 -- 172.21.15.2:0/331053894 >> 172.21.15.2:0/331053894 conn(0x565454750ff0 msgr2=0x56545475c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.083+0000 7f83459f1740 1 -- 172.21.15.2:0/331053894 shutdown_connections 2024-05-14T17:06:50.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.083+0000 7f83459f1740 1 -- 172.21.15.2:0/331053894 wait complete. 2024-05-14T17:06:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.083+0000 7f83459f1740 1 -- start start 2024-05-14T17:06:50.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.083+0000 7f83459f1740 1 Processor -- start 2024-05-14T17:06:50.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.083+0000 7f83459f1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5654547549c0 0x565454767ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.083+0000 7f83459f1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654547552e0 0x5654547680e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.083+0000 7f83459f1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654547640d0 0x565454768620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.083+0000 7f83459f1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f833c1aa920 con 0x5654547552e0 2024-05-14T17:06:50.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.083+0000 7f83459f1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f833c1abd90 con 0x5654547640d0 2024-05-14T17:06:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.083+0000 7f83459f1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f833c1aaff0 con 0x5654547549c0 2024-05-14T17:06:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.084+0000 7f83459f1740 10 client.0 _ll_register_callbacks cb 0x7f832c031cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:06:51.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.084+0000 7f834099c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654547552e0 0x5654547680e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:51.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.084+0000 7f834099c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654547552e0 0x5654547680e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40094/0 (socket says 172.21.15.2:40094) 2024-05-14T17:06:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.084+0000 7f834099c640 1 -- 172.21.15.2:0/3663947369 learned_addr learned my addr 172.21.15.2:0/3663947369 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:06:51.418 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.084+0000 7f833bfff640 1 --2- 172.21.15.2:0/3663947369 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5654547549c0 0x565454767ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.084+0000 7f833bfff640 1 --2- 172.21.15.2:0/3663947369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654547640d0 0x565454768620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:51.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.084+0000 7f834099c640 1 -- 172.21.15.2:0/3663947369 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5654547549c0 msgr2=0x565454767ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:51.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.084+0000 7f834099c640 1 --2- 172.21.15.2:0/3663947369 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5654547549c0 0x565454767ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.084+0000 7f834099c640 1 -- 172.21.15.2:0/3663947369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654547640d0 msgr2=0x565454768620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:06:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.084+0000 7f834099c640 1 --2- 172.21.15.2:0/3663947369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654547640d0 0x565454768620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.084+0000 7f834099c640 1 -- 172.21.15.2:0/3663947369 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654547dd130 con 0x5654547552e0 2024-05-14T17:06:52.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.084+0000 7f833bfff640 1 --2- 172.21.15.2:0/3663947369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654547640d0 0x565454768620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:06:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.084+0000 7f834099c640 1 --2- 172.21.15.2:0/3663947369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654547552e0 0x5654547680e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x565454776520 tx=0x7f8324010670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e4cdb36169c896 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:06:52.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.084+0000 7f831bfff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:06:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.084+0000 7f831bfff640 1 -- 172.21.15.2:0/3663947369 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8324003ac0 con 0x5654547552e0 2024-05-14T17:06:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.084+0000 7f831bfff640 1 -- 172.21.15.2:0/3663947369 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8557+0+0 (secure 0 0 0) 0x7f8324003c60 con 0x5654547552e0 2024-05-14T17:06:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.084+0000 7f83459f1740 1 -- 172.21.15.2:0/3663947369 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f832c036040 con 0x5654547552e0 2024-05-14T17:06:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.085+0000 7f831bfff640 1 -- 172.21.15.2:0/3663947369 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f832402d050 con 0x5654547552e0 2024-05-14T17:06:52.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.085+0000 7f83459f1740 10 client.16554 fetch_fsmap learned FSMap version 45 2024-05-14T17:06:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.085+0000 7f83459f1740 1 -- 172.21.15.2:0/3663947369 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5654547de4b0 con 0x5654547552e0 2024-05-14T17:06:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.085+0000 7f831bfff640 1 -- 172.21.15.2:0/3663947369 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f832c036040 con 0x5654547552e0 2024-05-14T17:06:52.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.085+0000 7f831bfff640 1 -- 172.21.15.2:0/3663947369 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f832401f070 con 0x5654547552e0 2024-05-14T17:06:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.085+0000 7f83459f1740 10 client.16554 fetch_fsmap finished waiting for FSMap version 45 2024-05-14T17:06:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.085+0000 7f83459f1740 10 client.16554 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:06:52.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.085+0000 7f83459f1740 20 client.16554 populate_metadata read hostname 'smithi002' 2024-05-14T17:06:53.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.085+0000 7f83459f1740 1 --2- 172.21.15.2:0/3663947369 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5654547dece0 0x5654547e1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:53.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.085+0000 7f83459f1740 4 client.16554 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:06:53.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.085+0000 7f83459f1740 1 -- 172.21.15.2:0/3663947369 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5654546c3360 con 0x5654547dece0 2024-05-14T17:06:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.085+0000 7f833bfff640 1 --2- 172.21.15.2:0/3663947369 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5654547dece0 0x5654547e1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.086+0000 7f833bfff640 1 --2- 172.21.15.2:0/3663947369 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5654547dece0 0x5654547e1100 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b75e111523f1b9ff server_cookie=3805896306e81c61 in_seq=0 out_seq=0 2024-05-14T17:06:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.086+0000 7f831bfff640 10 client.16554 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:06:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.097+0000 7f831bfff640 1 -- 172.21.15.2:0/3663947369 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5654546c3360 con 0x5654547dece0 2024-05-14T17:06:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.097+0000 7f831bfff640 10 client.16554 handle_command_reply: tid=0 2024-05-14T17:06:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.097+0000 7f83459f1740 1 client.16554 shutdown 2024-05-14T17:06:53.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.097+0000 7f83459f1740 1 -- 172.21.15.2:0/3663947369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654547552e0 msgr2=0x5654547680e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.097+0000 7f83459f1740 1 --2- 172.21.15.2:0/3663947369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654547552e0 0x5654547680e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x565454776520 tx=0x7f8324010670 comp rx=0 tx=0).stop 2024-05-14T17:06:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.097+0000 7f83459f1740 1 -- 172.21.15.2:0/3663947369 shutdown_connections 2024-05-14T17:06:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.097+0000 7f83459f1740 1 --2- 172.21.15.2:0/3663947369 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5654547dece0 0x5654547e1100 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:54.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.097+0000 7f83459f1740 1 --2- 172.21.15.2:0/3663947369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5654547640d0 0x565454768620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.097+0000 7f83459f1740 1 --2- 172.21.15.2:0/3663947369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5654547552e0 0x5654547680e0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:54.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.097+0000 7f831bfff640 0 client.16554 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:06:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.097+0000 7f83459f1740 1 --2- 172.21.15.2:0/3663947369 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5654547549c0 0x565454767ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.097+0000 7f83459f1740 1 -- 172.21.15.2:0/3663947369 >> 172.21.15.2:0/3663947369 conn(0x565454750ff0 msgr2=0x56545474fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.097+0000 7f83459f1740 1 -- 172.21.15.2:0/3663947369 shutdown_connections 2024-05-14T17:06:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.097+0000 7f83459f1740 1 -- 172.21.15.2:0/3663947369 wait complete. 2024-05-14T17:06:54.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.097+0000 7f83459f1740 20 client.16554 trim_cache size 0 max 16384 2024-05-14T17:06:54.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:06:54.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:06:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:06:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.098+0000 7f834199e640 1 -- 172.21.15.2:0/3488500794 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f83280b2950 msgr2=0x7f83280b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.098+0000 7f834199e640 1 --2- 172.21.15.2:0/3488500794 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f83280b2950 0x7f83280b4e10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f83340a6220 tx=0x7f83340d85c0 comp rx=0 tx=0).stop 2024-05-14T17:06:55.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.098+0000 7f834199e640 1 -- 172.21.15.2:0/3488500794 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f833c19d9c0 msgr2=0x7f833c19de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.098+0000 7f834199e640 1 --2- 172.21.15.2:0/3488500794 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f833c19d9c0 0x7f833c19de70 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f833400d950 tx=0x7f833400de10 comp rx=0 tx=0).stop 2024-05-14T17:06:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.098+0000 7f834199e640 1 -- 172.21.15.2:0/3488500794 shutdown_connections 2024-05-14T17:06:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.098+0000 7f834199e640 1 --2- 172.21.15.2:0/3488500794 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f83280b2950 0x7f83280b4e10 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.098+0000 7f834199e640 1 --2- 172.21.15.2:0/3488500794 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f833c19d9c0 0x7f833c19de70 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:55.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.098+0000 7f834199e640 1 --2- 172.21.15.2:0/3488500794 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f833c106ab0 0x7f833c199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.098+0000 7f834199e640 1 --2- 172.21.15.2:0/3488500794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f833c102100 0x7f833c199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:55.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.098+0000 7f834199e640 1 -- 172.21.15.2:0/3488500794 >> 172.21.15.2:0/3488500794 conn(0x7f833c077c90 msgr2=0x7f833c104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:55.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.098+0000 7f834199e640 1 -- 172.21.15.2:0/3488500794 shutdown_connections 2024-05-14T17:06:55.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:10.098+0000 7f834199e640 1 -- 172.21.15.2:0/3488500794 wait complete. 2024-05-14T17:06:55.954 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:06:10 smithi002 ceph-mon[31538]: pgmap v356: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 10 MiB/s wr, 467 op/s 2024-05-14T17:06:55.995 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 655.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:06:56.037 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:06:12 smithi002 ceph-mon[31538]: pgmap v357: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 10 MiB/s wr, 467 op/s 2024-05-14T17:06:56.156 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:06:12 smithi002 ceph-mon[31538]: osdmap e91: 12 total, 12 up, 12 in 2024-05-14T17:06:56.227 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:06:14 smithi002 ceph-mon[31538]: pgmap v359: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 454 op/s 2024-05-14T17:06:56.314 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 660.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:06:56.440 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:06:40 smithi123 ceph-mon[35971]: pgmap v372: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 36 KiB/s wr, 14 op/s 2024-05-14T17:06:56.516 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:06:41 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:06:56.586 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:06:42 smithi123 ceph-mon[35971]: pgmap v373: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 8 op/s 2024-05-14T17:06:56.629 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:06:44 smithi123 ceph-mon[35971]: pgmap v374: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-05-14T17:06:56.671 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:06:46 smithi123 ceph-mon[35971]: pgmap v375: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-05-14T17:06:56.711 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:06:48 smithi123 ceph-mon[35971]: pgmap v376: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-05-14T17:06:56.753 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:06:50 smithi123 ceph-mon[35971]: pgmap v377: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-05-14T17:06:56.901 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:06:52 smithi123 ceph-mon[35971]: pgmap v378: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T17:06:56.978 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:06:54 smithi123 ceph-mon[35971]: pgmap v379: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-05-14T17:06:57.086 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:06:40 smithi151 ceph-mon[36066]: pgmap v372: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 36 KiB/s wr, 14 op/s 2024-05-14T17:06:57.172 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:06:41 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:06:57.258 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:06:42 smithi151 ceph-mon[36066]: pgmap v373: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 8 op/s 2024-05-14T17:06:57.349 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:06:44 smithi151 ceph-mon[36066]: pgmap v374: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-05-14T17:06:57.418 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:06:46 smithi151 ceph-mon[36066]: pgmap v375: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-05-14T17:06:57.453 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:06:48 smithi151 ceph-mon[36066]: pgmap v376: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-05-14T17:06:57.518 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:06:50 smithi151 ceph-mon[36066]: pgmap v377: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-05-14T17:06:57.564 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:06:52 smithi151 ceph-mon[36066]: pgmap v378: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T17:06:57.626 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:06:54 smithi151 ceph-mon[36066]: pgmap v379: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-05-14T17:06:57.705 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:06:16 smithi002 ceph-mon[31538]: pgmap v360: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 450 op/s 2024-05-14T17:06:57.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:06:18 smithi002 ceph-mon[31538]: pgmap v361: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 450 op/s 2024-05-14T17:06:57.870 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:06:20 smithi002 ceph-mon[31538]: pgmap v362: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-05-14T17:06:57.947 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 665.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:06:58.032 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:06:22 smithi002 ceph-mon[31538]: pgmap v363: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-05-14T17:06:58.239 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:06:56 smithi123 ceph-mon[35971]: pgmap v380: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 5 op/s 2024-05-14T17:06:58.287 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:06:56 smithi151 ceph-mon[36066]: pgmap v380: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 5 op/s 2024-05-14T17:06:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.199+0000 7ffb0359d640 1 -- 172.21.15.2:0/3167655824 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffafc107980 msgr2=0x7ffafc107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:06:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.199+0000 7ffb0359d640 1 --2- 172.21.15.2:0/3167655824 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffafc107980 0x7ffafc107d60 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ffaf80099b0 tx=0x7ffaf802c650 comp rx=0 tx=0).stop 2024-05-14T17:06:58.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.199+0000 7ffb0359d640 1 -- 172.21.15.2:0/3167655824 shutdown_connections 2024-05-14T17:06:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.199+0000 7ffb0359d640 1 --2- 172.21.15.2:0/3167655824 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffafc102330 0x7ffafc10e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.199+0000 7ffb0359d640 1 --2- 172.21.15.2:0/3167655824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffafc101970 0x7ffafc101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.199+0000 7ffb0359d640 1 --2- 172.21.15.2:0/3167655824 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffafc107980 0x7ffafc107d60 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:06:58.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.199+0000 7ffb0359d640 1 -- 172.21.15.2:0/3167655824 >> 172.21.15.2:0/3167655824 conn(0x7ffafc077cd0 msgr2=0x7ffafc0780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:06:58.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.200+0000 7ffb0359d640 1 -- 172.21.15.2:0/3167655824 shutdown_connections 2024-05-14T17:06:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.200+0000 7ffb0359d640 1 -- 172.21.15.2:0/3167655824 wait complete. 2024-05-14T17:06:59.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.200+0000 7ffb0359d640 1 Processor -- start 2024-05-14T17:06:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.200+0000 7ffb0359d640 1 -- start start 2024-05-14T17:06:59.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.201+0000 7ffb0359d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffafc101970 0x7ffafc19b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.201+0000 7ffb0359d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffafc102330 0x7ffafc19b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.201+0000 7ffb0359d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffafc107980 0x7ffafc19fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:06:59.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.201+0000 7ffb0359d640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ffafc110b30 con 0x7ffafc102330 2024-05-14T17:06:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.201+0000 7ffb0359d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffafc1109b0 con 0x7ffafc107980 2024-05-14T17:06:59.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.201+0000 7ffb0359d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffafc110cb0 con 0x7ffafc101970 2024-05-14T17:06:59.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.201+0000 7ffb00b38640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffafc102330 0x7ffafc19b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:06:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.201+0000 7ffb00b38640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffafc102330 0x7ffafc19b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57428/0 (socket says 172.21.15.2:57428) 2024-05-14T17:06:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.201+0000 7ffb00b38640 1 -- 172.21.15.2:0/2544413621 learned_addr learned my addr 172.21.15.2:0/2544413621 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:07:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.201+0000 7ffb01339640 1 --2- 172.21.15.2:0/2544413621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffafc101970 0x7ffafc19b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:00.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.201+0000 7ffb01b3a640 1 --2- 172.21.15.2:0/2544413621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffafc107980 0x7ffafc19fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.201+0000 7ffb00b38640 1 -- 172.21.15.2:0/2544413621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffafc101970 msgr2=0x7ffafc19b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:00.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.201+0000 7ffb00b38640 1 --2- 172.21.15.2:0/2544413621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffafc101970 0x7ffafc19b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.201+0000 7ffb00b38640 1 -- 172.21.15.2:0/2544413621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffafc107980 msgr2=0x7ffafc19fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.201+0000 7ffb00b38640 1 --2- 172.21.15.2:0/2544413621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffafc107980 0x7ffafc19fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.201+0000 7ffb00b38640 1 -- 172.21.15.2:0/2544413621 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffafc1a0360 con 0x7ffafc102330 2024-05-14T17:07:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.201+0000 7ffb01b3a640 1 --2- 172.21.15.2:0/2544413621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffafc107980 0x7ffafc19fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:07:00.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.201+0000 7ffb00b38640 1 --2- 172.21.15.2:0/2544413621 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffafc102330 0x7ffafc19b900 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ffaec00d950 tx=0x7ffaec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:00.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.201+0000 7ffaf27fc640 1 -- 172.21.15.2:0/2544413621 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaec017070 con 0x7ffafc102330 2024-05-14T17:07:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.201+0000 7ffaf27fc640 1 -- 172.21.15.2:0/2544413621 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffaec009c30 con 0x7ffafc102330 2024-05-14T17:07:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.201+0000 7ffb0359d640 1 -- 172.21.15.2:0/2544413621 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffafc1a0650 con 0x7ffafc102330 2024-05-14T17:07:00.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.201+0000 7ffaf27fc640 1 -- 172.21.15.2:0/2544413621 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaec00f620 con 0x7ffafc102330 2024-05-14T17:07:01.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.202+0000 7ffb0359d640 1 -- 172.21.15.2:0/2544413621 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffafc1a7ee0 con 0x7ffafc102330 2024-05-14T17:07:01.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.204+0000 7ffaf27fc640 1 -- 172.21.15.2:0/2544413621 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7ffaec010460 con 0x7ffafc102330 2024-05-14T17:07:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.204+0000 7ffb06394740 1 Processor -- start 2024-05-14T17:07:01.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.205+0000 7ffb06394740 1 -- start start 2024-05-14T17:07:01.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.205+0000 7ffb06394740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de5fc8f8f0 0x55de5fc8fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.205+0000 7ffb06394740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5fc90210 0x55de5fc9a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.205+0000 7ffb06394740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de5fc9aad0 0x55de5fc9cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.205+0000 7ffb06394740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55de5fc02c30 con 0x55de5fc90210 2024-05-14T17:07:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.205+0000 7ffb06394740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55de5fbf27d0 con 0x55de5fc8f8f0 2024-05-14T17:07:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.205+0000 7ffb06394740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55de5fbf6810 con 0x55de5fc9aad0 2024-05-14T17:07:01.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.205+0000 7ffb01339640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5fc90210 0x55de5fc9a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.205+0000 7ffb01339640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5fc90210 0x55de5fc9a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57442/0 (socket says 172.21.15.2:57442) 2024-05-14T17:07:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.205+0000 7ffb01339640 1 -- 172.21.15.2:0/785796177 learned_addr learned my addr 172.21.15.2:0/785796177 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:07:02.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.205+0000 7ffb01b3a640 1 --2- 172.21.15.2:0/785796177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de5fc8f8f0 0x55de5fc8fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.205+0000 7ffb00b38640 1 --2- 172.21.15.2:0/785796177 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de5fc9aad0 0x55de5fc9cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:02.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.205+0000 7ffaf27fc640 1 --2- 172.21.15.2:0/2544413621 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ffae80b2a90 0x7ffae80b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.205+0000 7ffb01b3a640 1 --2- 172.21.15.2:0/2544413621 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ffae80b2a90 0x7ffae80b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.205+0000 7ffb01339640 1 -- 172.21.15.2:0/785796177 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de5fc9aad0 msgr2=0x55de5fc9cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:02.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.205+0000 7ffb01339640 1 --2- 172.21.15.2:0/785796177 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de5fc9aad0 0x55de5fc9cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.205+0000 7ffb01339640 1 -- 172.21.15.2:0/785796177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de5fc8f8f0 msgr2=0x55de5fc8fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.205+0000 7ffb01339640 1 --2- 172.21.15.2:0/785796177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de5fc8f8f0 0x55de5fc8fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.205+0000 7ffb01339640 1 -- 172.21.15.2:0/785796177 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de5f907210 con 0x55de5fc90210 2024-05-14T17:07:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.206+0000 7ffaf27fc640 1 -- 172.21.15.2:0/2544413621 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7ffaec0e2200 con 0x7ffafc102330 2024-05-14T17:07:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.206+0000 7ffb01339640 1 --2- 172.21.15.2:0/785796177 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5fc90210 0x55de5fc9a590 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ffaf802cb70 tx=0x7ffaf8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1d69a52d93ac9d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:03.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.206+0000 7ffb01b3a640 1 --2- 172.21.15.2:0/2544413621 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ffae80b2a90 0x7ffae80b4f50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4009370 tx=0x7ffaf4004ab0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.206+0000 7ffae77fe640 1 -- 172.21.15.2:0/785796177 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8043070 con 0x55de5fc90210 2024-05-14T17:07:03.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.206+0000 7ffae77fe640 1 -- 172.21.15.2:0/785796177 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffaf8004030 con 0x55de5fc90210 2024-05-14T17:07:03.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.206+0000 7ffae77fe640 1 -- 172.21.15.2:0/785796177 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8037040 con 0x55de5fc90210 2024-05-14T17:07:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.207+0000 7ffb06394740 1 -- 172.21.15.2:0/785796177 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5fc90210 msgr2=0x55de5fc9a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.207+0000 7ffb06394740 1 --2- 172.21.15.2:0/785796177 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5fc90210 0x55de5fc9a590 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ffaf802cb70 tx=0x7ffaf8005fd0 comp rx=0 tx=0).stop 2024-05-14T17:07:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.207+0000 7ffb06394740 1 -- 172.21.15.2:0/785796177 shutdown_connections 2024-05-14T17:07:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.207+0000 7ffb06394740 1 --2- 172.21.15.2:0/785796177 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de5fc9aad0 0x55de5fc9cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:03.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.207+0000 7ffb06394740 1 --2- 172.21.15.2:0/785796177 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5fc90210 0x55de5fc9a590 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:03.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.207+0000 7ffb06394740 1 --2- 172.21.15.2:0/785796177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de5fc8f8f0 0x55de5fc8fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.207+0000 7ffb06394740 1 -- 172.21.15.2:0/785796177 >> 172.21.15.2:0/785796177 conn(0x55de5fc8be00 msgr2=0x55de5fc8c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:03.939 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.207+0000 7ffb06394740 1 -- 172.21.15.2:0/785796177 shutdown_connections 2024-05-14T17:07:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.207+0000 7ffb06394740 1 -- 172.21.15.2:0/785796177 wait complete. 2024-05-14T17:07:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.207+0000 7ffb06394740 1 -- start start 2024-05-14T17:07:04.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.207+0000 7ffb06394740 1 Processor -- start 2024-05-14T17:07:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.208+0000 7ffb06394740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5fc8f8f0 0x55de5fc9a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.208+0000 7ffb06394740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de5fc90210 0x55de5fc97650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.208+0000 7ffb06394740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de5fc9aad0 0x55de5fc97b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:04.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.208+0000 7ffb06394740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55de5f8d8790 con 0x55de5fc8f8f0 2024-05-14T17:07:04.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.208+0000 7ffb06394740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55de5fbae890 con 0x55de5fc9aad0 2024-05-14T17:07:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.208+0000 7ffb06394740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55de5fc9d740 con 0x55de5fc90210 2024-05-14T17:07:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.208+0000 7ffb06394740 10 client.0 _ll_register_callbacks cb 0x55de5fbb1d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:07:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.208+0000 7ffb01339640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5fc8f8f0 0x55de5fc9a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.208+0000 7ffb01339640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5fc8f8f0 0x55de5fc9a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57446/0 (socket says 172.21.15.2:57446) 2024-05-14T17:07:05.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.208+0000 7ffb01339640 1 -- 172.21.15.2:0/413885655 learned_addr learned my addr 172.21.15.2:0/413885655 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:07:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.208+0000 7ffb00b38640 1 --2- 172.21.15.2:0/413885655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de5fc90210 0x55de5fc97650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:05.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.208+0000 7ffb01b3a640 1 --2- 172.21.15.2:0/413885655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de5fc9aad0 0x55de5fc97b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.208+0000 7ffb01339640 1 -- 172.21.15.2:0/413885655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de5fc90210 msgr2=0x55de5fc97650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.208+0000 7ffb01339640 1 --2- 172.21.15.2:0/413885655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de5fc90210 0x55de5fc97650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:05.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.208+0000 7ffb01339640 1 -- 172.21.15.2:0/413885655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de5fc9aad0 msgr2=0x55de5fc97b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.208+0000 7ffb01339640 1 --2- 172.21.15.2:0/413885655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de5fc9aad0 0x55de5fc97b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.208+0000 7ffb01339640 1 -- 172.21.15.2:0/413885655 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de5f9d3620 con 0x55de5fc8f8f0 2024-05-14T17:07:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.208+0000 7ffb01339640 1 --2- 172.21.15.2:0/413885655 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5fc8f8f0 0x55de5fc9a360 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8005c20 tx=0x7ffaf8004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaf6d7943baecfee server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.208+0000 7ffae4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:07:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.209+0000 7ffae4ff9640 1 -- 172.21.15.2:0/413885655 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf804b020 con 0x55de5fc8f8f0 2024-05-14T17:07:06.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.209+0000 7ffae4ff9640 1 -- 172.21.15.2:0/413885655 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7ffaf80334a0 con 0x55de5fc8f8f0 2024-05-14T17:07:06.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.209+0000 7ffb06394740 1 -- 172.21.15.2:0/413885655 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffaf8004030 con 0x55de5fc8f8f0 2024-05-14T17:07:06.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.209+0000 7ffae4ff9640 1 -- 172.21.15.2:0/413885655 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7ffaf8056050 con 0x55de5fc8f8f0 2024-05-14T17:07:06.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.209+0000 7ffb06394740 10 client.16566 fetch_fsmap learned FSMap version 45 2024-05-14T17:07:06.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.209+0000 7ffb06394740 1 -- 172.21.15.2:0/413885655 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55de5fca9080 con 0x55de5fc8f8f0 2024-05-14T17:07:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.209+0000 7ffae4ff9640 1 -- 172.21.15.2:0/413885655 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffaf8004030 con 0x55de5fc8f8f0 2024-05-14T17:07:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.209+0000 7ffae4ff9640 1 -- 172.21.15.2:0/413885655 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7ffaf80035f0 con 0x55de5fc8f8f0 2024-05-14T17:07:07.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.209+0000 7ffb06394740 10 client.16566 fetch_fsmap finished waiting for FSMap version 45 2024-05-14T17:07:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.209+0000 7ffb06394740 10 client.16566 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:07:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.209+0000 7ffb06394740 20 client.16566 populate_metadata read hostname 'smithi002' 2024-05-14T17:07:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.209+0000 7ffb06394740 1 --2- 172.21.15.2:0/413885655 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55de5fcaa3d0 0x55de5fcac7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:07.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.209+0000 7ffb06394740 4 client.16566 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:07:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.209+0000 7ffb06394740 1 -- 172.21.15.2:0/413885655 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de5fc04b80 con 0x55de5fcaa3d0 2024-05-14T17:07:07.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.210+0000 7ffb00b38640 1 --2- 172.21.15.2:0/413885655 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55de5fcaa3d0 0x55de5fcac7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.210+0000 7ffb00b38640 1 --2- 172.21.15.2:0/413885655 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55de5fcaa3d0 0x55de5fcac7c0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8d0cdc745ab5cef server_cookie=95d1305b5a39322b in_seq=0 out_seq=0 2024-05-14T17:07:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.210+0000 7ffae4ff9640 10 client.16566 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:07:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.211+0000 7ffae4ff9640 1 -- 172.21.15.2:0/413885655 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55de5fc04b80 con 0x55de5fcaa3d0 2024-05-14T17:07:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.211+0000 7ffae4ff9640 10 client.16566 handle_command_reply: tid=0 2024-05-14T17:07:07.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.211+0000 7ffb06394740 1 client.16566 shutdown 2024-05-14T17:07:08.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.211+0000 7ffb06394740 1 -- 172.21.15.2:0/413885655 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5fc8f8f0 msgr2=0x55de5fc9a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:08.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.211+0000 7ffb06394740 1 --2- 172.21.15.2:0/413885655 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5fc8f8f0 0x55de5fc9a360 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8005c20 tx=0x7ffaf8004410 comp rx=0 tx=0).stop 2024-05-14T17:07:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.211+0000 7ffb06394740 1 -- 172.21.15.2:0/413885655 shutdown_connections 2024-05-14T17:07:08.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.211+0000 7ffb06394740 1 --2- 172.21.15.2:0/413885655 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55de5fcaa3d0 0x55de5fcac7c0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.211+0000 7ffb06394740 1 --2- 172.21.15.2:0/413885655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de5fc9aad0 0x55de5fc97b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:08.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.211+0000 7ffb06394740 1 --2- 172.21.15.2:0/413885655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de5fc90210 0x55de5fc97650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.211+0000 7ffae4ff9640 0 client.16566 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:07:08.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.211+0000 7ffb06394740 1 --2- 172.21.15.2:0/413885655 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5fc8f8f0 0x55de5fc9a360 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.211+0000 7ffb06394740 1 -- 172.21.15.2:0/413885655 >> 172.21.15.2:0/413885655 conn(0x55de5fc8be00 msgr2=0x55de5fc8b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.211+0000 7ffb06394740 1 -- 172.21.15.2:0/413885655 shutdown_connections 2024-05-14T17:07:08.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.211+0000 7ffb06394740 1 -- 172.21.15.2:0/413885655 wait complete. 2024-05-14T17:07:08.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.211+0000 7ffb06394740 20 client.16566 trim_cache size 0 max 16384 2024-05-14T17:07:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.227+0000 7ffb06394740 1 Processor -- start 2024-05-14T17:07:08.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.227+0000 7ffb06394740 1 -- start start 2024-05-14T17:07:08.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.227+0000 7ffb06394740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de5fc8f8f0 0x55de5fc86f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.227+0000 7ffb06394740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5fc90210 0x55de5fca6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.227+0000 7ffb06394740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de5fc9aad0 0x55de5fcb1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.227+0000 7ffb06394740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ffafc1b3060 con 0x55de5fc90210 2024-05-14T17:07:09.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.227+0000 7ffb06394740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffafc1b08d0 con 0x55de5fc8f8f0 2024-05-14T17:07:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.227+0000 7ffb06394740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffafc1ae740 con 0x55de5fc9aad0 2024-05-14T17:07:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.227+0000 7ffb00b38640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5fc90210 0x55de5fca6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.227+0000 7ffb00b38640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5fc90210 0x55de5fca6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57456/0 (socket says 172.21.15.2:57456) 2024-05-14T17:07:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.227+0000 7ffb00b38640 1 -- 172.21.15.2:0/2452692930 learned_addr learned my addr 172.21.15.2:0/2452692930 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:07:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.227+0000 7ffb01b3a640 1 --2- 172.21.15.2:0/2452692930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de5fc9aad0 0x55de5fcb1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.227+0000 7ffb01339640 1 --2- 172.21.15.2:0/2452692930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de5fc8f8f0 0x55de5fc86f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.227+0000 7ffb00b38640 1 -- 172.21.15.2:0/2452692930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de5fc9aad0 msgr2=0x55de5fcb1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:09.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.227+0000 7ffb00b38640 1 --2- 172.21.15.2:0/2452692930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de5fc9aad0 0x55de5fcb1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:10.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.227+0000 7ffb00b38640 1 -- 172.21.15.2:0/2452692930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de5fc8f8f0 msgr2=0x55de5fc86f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.227+0000 7ffb00b38640 1 --2- 172.21.15.2:0/2452692930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de5fc8f8f0 0x55de5fc86f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.227+0000 7ffb00b38640 1 -- 172.21.15.2:0/2452692930 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de5fc96600 con 0x55de5fc90210 2024-05-14T17:07:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.227+0000 7ffb01339640 1 --2- 172.21.15.2:0/2452692930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de5fc8f8f0 0x55de5fc86f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:07:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.228+0000 7ffb00b38640 1 --2- 172.21.15.2:0/2452692930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5fc90210 0x55de5fca6610 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ffaec0a9410 tx=0x7ffaec091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef9ca4a76b8d16d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.228+0000 7ffae6ffd640 1 -- 172.21.15.2:0/2452692930 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaec020040 con 0x55de5fc90210 2024-05-14T17:07:10.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.228+0000 7ffae6ffd640 1 -- 172.21.15.2:0/2452692930 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffaec009de0 con 0x55de5fc90210 2024-05-14T17:07:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.228+0000 7ffae6ffd640 1 -- 172.21.15.2:0/2452692930 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaec0dfb70 con 0x55de5fc90210 2024-05-14T17:07:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.228+0000 7ffb06394740 1 -- 172.21.15.2:0/2452692930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5fc90210 msgr2=0x55de5fca6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.228+0000 7ffb06394740 1 --2- 172.21.15.2:0/2452692930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5fc90210 0x55de5fca6610 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ffaec0a9410 tx=0x7ffaec091040 comp rx=0 tx=0).stop 2024-05-14T17:07:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.228+0000 7ffb06394740 1 -- 172.21.15.2:0/2452692930 shutdown_connections 2024-05-14T17:07:11.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.228+0000 7ffb06394740 1 --2- 172.21.15.2:0/2452692930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de5fc9aad0 0x55de5fcb1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.228+0000 7ffb06394740 1 --2- 172.21.15.2:0/2452692930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5fc90210 0x55de5fca6610 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:11.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.228+0000 7ffb06394740 1 --2- 172.21.15.2:0/2452692930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de5fc8f8f0 0x55de5fc86f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:11.250 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.228+0000 7ffb06394740 1 -- 172.21.15.2:0/2452692930 >> 172.21.15.2:0/2452692930 conn(0x55de5fc8be00 msgr2=0x55de5fc8b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.228+0000 7ffb06394740 1 -- 172.21.15.2:0/2452692930 shutdown_connections 2024-05-14T17:07:11.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.228+0000 7ffb06394740 1 -- 172.21.15.2:0/2452692930 wait complete. 2024-05-14T17:07:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.229+0000 7ffb06394740 1 -- start start 2024-05-14T17:07:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.229+0000 7ffb06394740 1 Processor -- start 2024-05-14T17:07:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.229+0000 7ffb06394740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de5fc8f8f0 0x55de5fca07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.229+0000 7ffb06394740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de5fc90210 0x55de5fca0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.229+0000 7ffb06394740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5fc9aad0 0x55de5fca1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.229+0000 7ffb06394740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ffafc1aed90 con 0x55de5fc9aad0 2024-05-14T17:07:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.229+0000 7ffb06394740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffafc1b0200 con 0x55de5fc8f8f0 2024-05-14T17:07:12.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.229+0000 7ffb06394740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffafc1af460 con 0x55de5fc90210 2024-05-14T17:07:12.237 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.229+0000 7ffb06394740 10 client.0 _ll_register_callbacks cb 0x7ffaf8009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:07:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.229+0000 7ffb01b3a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5fc9aad0 0x55de5fca1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.229+0000 7ffb01b3a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5fc9aad0 0x55de5fca1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57472/0 (socket says 172.21.15.2:57472) 2024-05-14T17:07:12.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.229+0000 7ffb01b3a640 1 -- 172.21.15.2:0/2595849396 learned_addr learned my addr 172.21.15.2:0/2595849396 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:07:12.551 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.230+0000 7ffb00b38640 1 --2- 172.21.15.2:0/2595849396 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de5fc90210 0x55de5fca0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.230+0000 7ffb01339640 1 --2- 172.21.15.2:0/2595849396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de5fc8f8f0 0x55de5fca07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.230+0000 7ffb01b3a640 1 -- 172.21.15.2:0/2595849396 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de5fc90210 msgr2=0x55de5fca0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.230+0000 7ffb01b3a640 1 --2- 172.21.15.2:0/2595849396 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de5fc90210 0x55de5fca0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.230+0000 7ffb01b3a640 1 -- 172.21.15.2:0/2595849396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de5fc8f8f0 msgr2=0x55de5fca07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.230+0000 7ffb01b3a640 1 --2- 172.21.15.2:0/2595849396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de5fc8f8f0 0x55de5fca07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.230+0000 7ffb01b3a640 1 -- 172.21.15.2:0/2595849396 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de5fc90040 con 0x55de5fc9aad0 2024-05-14T17:07:13.178 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.230+0000 7ffb01b3a640 1 --2- 172.21.15.2:0/2595849396 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5fc9aad0 0x55de5fca1220 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4003dc0 tx=0x7ffaf4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=591127fd866da9bc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.230+0000 7ffae4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:07:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.230+0000 7ffae4ff9640 1 -- 172.21.15.2:0/2595849396 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf4028070 con 0x55de5fc9aad0 2024-05-14T17:07:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.230+0000 7ffae4ff9640 1 -- 172.21.15.2:0/2595849396 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7ffaf4004500 con 0x55de5fc9aad0 2024-05-14T17:07:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.230+0000 7ffb06394740 1 -- 172.21.15.2:0/2595849396 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffaec009de0 con 0x55de5fc9aad0 2024-05-14T17:07:13.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.230+0000 7ffae4ff9640 1 -- 172.21.15.2:0/2595849396 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7ffaf4034050 con 0x55de5fc9aad0 2024-05-14T17:07:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.230+0000 7ffb06394740 10 client.16572 fetch_fsmap learned FSMap version 45 2024-05-14T17:07:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.230+0000 7ffb06394740 1 -- 172.21.15.2:0/2595849396 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55de5fd14d20 con 0x55de5fc9aad0 2024-05-14T17:07:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.231+0000 7ffae4ff9640 1 -- 172.21.15.2:0/2595849396 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffaec009de0 con 0x55de5fc9aad0 2024-05-14T17:07:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.231+0000 7ffae4ff9640 1 -- 172.21.15.2:0/2595849396 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7ffaf4022040 con 0x55de5fc9aad0 2024-05-14T17:07:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.231+0000 7ffb06394740 10 client.16572 fetch_fsmap finished waiting for FSMap version 45 2024-05-14T17:07:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.231+0000 7ffb06394740 10 client.16572 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:07:14.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.231+0000 7ffb06394740 20 client.16572 populate_metadata read hostname 'smithi002' 2024-05-14T17:07:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.231+0000 7ffb06394740 1 --2- 172.21.15.2:0/2595849396 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55de5fd154a0 0x55de5fd178c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.231+0000 7ffb06394740 4 client.16572 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:07:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.231+0000 7ffb06394740 1 -- 172.21.15.2:0/2595849396 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55de5fbfe360 con 0x55de5fd154a0 2024-05-14T17:07:14.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.231+0000 7ffb01339640 1 --2- 172.21.15.2:0/2595849396 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55de5fd154a0 0x55de5fd178c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:14.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.231+0000 7ffb01339640 1 --2- 172.21.15.2:0/2595849396 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55de5fd154a0 0x55de5fd178c0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f7cbb40cde1bbec server_cookie=afeeb42c01355a5f in_seq=0 out_seq=0 2024-05-14T17:07:14.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.231+0000 7ffae4ff9640 10 client.16572 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:07:14.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.242+0000 7ffae4ff9640 1 -- 172.21.15.2:0/2595849396 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55de5fbfe360 con 0x55de5fd154a0 2024-05-14T17:07:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.242+0000 7ffae4ff9640 10 client.16572 handle_command_reply: tid=0 2024-05-14T17:07:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.242+0000 7ffb06394740 1 client.16572 shutdown 2024-05-14T17:07:14.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.243+0000 7ffb06394740 1 -- 172.21.15.2:0/2595849396 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5fc9aad0 msgr2=0x55de5fca1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.243+0000 7ffb06394740 1 --2- 172.21.15.2:0/2595849396 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5fc9aad0 0x55de5fca1220 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4003dc0 tx=0x7ffaf4004a40 comp rx=0 tx=0).stop 2024-05-14T17:07:14.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.243+0000 7ffb06394740 1 -- 172.21.15.2:0/2595849396 shutdown_connections 2024-05-14T17:07:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.243+0000 7ffb06394740 1 --2- 172.21.15.2:0/2595849396 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55de5fd154a0 0x55de5fd178c0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:15.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.243+0000 7ffb06394740 1 --2- 172.21.15.2:0/2595849396 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de5fc9aad0 0x55de5fca1220 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:15.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.243+0000 7ffb06394740 1 --2- 172.21.15.2:0/2595849396 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55de5fc90210 0x55de5fca0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:15.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.243+0000 7ffb06394740 1 --2- 172.21.15.2:0/2595849396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de5fc8f8f0 0x55de5fca07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.243+0000 7ffae4ff9640 0 client.16572 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:07:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.243+0000 7ffb06394740 1 -- 172.21.15.2:0/2595849396 >> 172.21.15.2:0/2595849396 conn(0x55de5fc8be00 msgr2=0x55de5fca5600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:15.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.243+0000 7ffb06394740 1 -- 172.21.15.2:0/2595849396 shutdown_connections 2024-05-14T17:07:15.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.243+0000 7ffb06394740 1 -- 172.21.15.2:0/2595849396 wait complete. 2024-05-14T17:07:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.243+0000 7ffb06394740 20 client.16572 trim_cache size 0 max 16384 2024-05-14T17:07:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:07:15.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:07:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:07:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.243+0000 7ffb0359d640 1 -- 172.21.15.2:0/2544413621 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ffae80b2a90 msgr2=0x7ffae80b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.243+0000 7ffb0359d640 1 --2- 172.21.15.2:0/2544413621 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ffae80b2a90 0x7ffae80b4f50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4009370 tx=0x7ffaf4004ab0 comp rx=0 tx=0).stop 2024-05-14T17:07:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.243+0000 7ffb0359d640 1 -- 172.21.15.2:0/2544413621 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffafc102330 msgr2=0x7ffafc19b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:16.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.244+0000 7ffb0359d640 1 --2- 172.21.15.2:0/2544413621 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffafc102330 0x7ffafc19b900 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ffaec00d950 tx=0x7ffaec00de10 comp rx=0 tx=0).stop 2024-05-14T17:07:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.244+0000 7ffb0359d640 1 -- 172.21.15.2:0/2544413621 shutdown_connections 2024-05-14T17:07:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.244+0000 7ffb0359d640 1 --2- 172.21.15.2:0/2544413621 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ffae80b2a90 0x7ffae80b4f50 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.244+0000 7ffb0359d640 1 --2- 172.21.15.2:0/2544413621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffafc107980 0x7ffafc19fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.244+0000 7ffb0359d640 1 --2- 172.21.15.2:0/2544413621 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffafc102330 0x7ffafc19b900 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.244+0000 7ffb0359d640 1 --2- 172.21.15.2:0/2544413621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffafc101970 0x7ffafc19b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:16.634 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.244+0000 7ffb0359d640 1 -- 172.21.15.2:0/2544413621 >> 172.21.15.2:0/2544413621 conn(0x7ffafc077cd0 msgr2=0x7ffafc0ff6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.244+0000 7ffb0359d640 1 -- 172.21.15.2:0/2544413621 shutdown_connections 2024-05-14T17:07:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:23.244+0000 7ffb0359d640 1 -- 172.21.15.2:0/2544413621 wait complete. 2024-05-14T17:07:16.952 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:06:24 smithi002 ceph-mon[31538]: pgmap v364: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-05-14T17:07:17.054 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 670.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:07:17.178 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:06:26 smithi002 ceph-mon[31538]: pgmap v365: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 767 B/s wr, 5 op/s 2024-05-14T17:07:17.296 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:06:28 smithi002 ceph-mon[31538]: pgmap v366: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 767 B/s wr, 5 op/s 2024-05-14T17:07:17.338 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:06:30 smithi002 ceph-mon[31538]: pgmap v367: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-05-14T17:07:17.407 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 675.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:07:17.475 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:06:32 smithi002 ceph-mon[31538]: pgmap v368: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-05-14T17:07:17.543 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:06:34 smithi002 ceph-mon[31538]: pgmap v369: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-05-14T17:07:17.588 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:06:58 smithi123 ceph-mon[35971]: pgmap v381: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 5 op/s 2024-05-14T17:07:17.627 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:07:00 smithi123 ceph-mon[35971]: pgmap v382: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-05-14T17:07:17.669 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:07:02 smithi123 ceph-mon[35971]: pgmap v383: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 5 op/s 2024-05-14T17:07:17.739 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:07:04 smithi123 ceph-mon[35971]: pgmap v384: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 81 KiB/s wr, 27 op/s 2024-05-14T17:07:17.847 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:07:04 smithi123 ceph-mon[35971]: osdmap e92: 12 total, 12 up, 12 in 2024-05-14T17:07:17.918 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:07:05 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:07:17.977 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:07:06 smithi123 ceph-mon[35971]: pgmap v386: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 82 KiB/s wr, 29 op/s 2024-05-14T17:07:18.074 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:07:06 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:07:18.170 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:07:06 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:07:18.241 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:07:06 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:07:18.358 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:07:08 smithi123 ceph-mon[35971]: pgmap v387: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 82 KiB/s wr, 29 op/s 2024-05-14T17:07:18.403 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:07:10 smithi123 ceph-mon[35971]: pgmap v388: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 89 KiB/s wr, 32 op/s 2024-05-14T17:07:18.453 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:07:12 smithi123 ceph-mon[35971]: pgmap v389: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 89 KiB/s wr, 32 op/s 2024-05-14T17:07:18.550 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:07:12 smithi123 ceph-mon[35971]: osdmap e93: 12 total, 12 up, 12 in 2024-05-14T17:07:18.622 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:07:14 smithi123 ceph-mon[35971]: pgmap v391: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-14T17:07:18.705 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:07:16 smithi123 ceph-mon[35971]: pgmap v392: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-14T17:07:18.745 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:06:58 smithi151 ceph-mon[36066]: pgmap v381: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 5 op/s 2024-05-14T17:07:18.795 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:07:00 smithi151 ceph-mon[36066]: pgmap v382: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-05-14T17:07:18.900 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:07:02 smithi151 ceph-mon[36066]: pgmap v383: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 5 op/s 2024-05-14T17:07:18.965 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:07:04 smithi151 ceph-mon[36066]: pgmap v384: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 81 KiB/s wr, 27 op/s 2024-05-14T17:07:19.038 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:07:04 smithi151 ceph-mon[36066]: osdmap e92: 12 total, 12 up, 12 in 2024-05-14T17:07:19.107 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:07:05 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:07:19.153 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:07:06 smithi151 ceph-mon[36066]: pgmap v386: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 82 KiB/s wr, 29 op/s 2024-05-14T17:07:19.203 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:07:06 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:07:19.278 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:07:06 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:07:19.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:07:06 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:07:19.476 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:07:08 smithi151 ceph-mon[36066]: pgmap v387: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 82 KiB/s wr, 29 op/s 2024-05-14T17:07:19.530 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:07:10 smithi151 ceph-mon[36066]: pgmap v388: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 89 KiB/s wr, 32 op/s 2024-05-14T17:07:19.608 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:07:12 smithi151 ceph-mon[36066]: pgmap v389: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 89 KiB/s wr, 32 op/s 2024-05-14T17:07:19.659 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:07:12 smithi151 ceph-mon[36066]: osdmap e93: 12 total, 12 up, 12 in 2024-05-14T17:07:19.782 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:07:14 smithi151 ceph-mon[36066]: pgmap v391: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-14T17:07:19.847 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:07:16 smithi151 ceph-mon[36066]: pgmap v392: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-14T17:07:19.897 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:07:18 smithi151 ceph-mon[36066]: pgmap v393: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-14T17:07:19.945 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:07:18 smithi123 ceph-mon[35971]: pgmap v393: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-14T17:07:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.347+0000 7f14b2e75640 1 -- 172.21.15.2:0/3786114695 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14ac103ce0 msgr2=0x7f14ac1040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.347+0000 7f14b2e75640 1 --2- 172.21.15.2:0/3786114695 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14ac103ce0 0x7f14ac1040c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f14a0009900 tx=0x7f14a002a510 comp rx=0 tx=0).stop 2024-05-14T17:07:20.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.347+0000 7f14b2e75640 1 -- 172.21.15.2:0/3786114695 shutdown_connections 2024-05-14T17:07:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.347+0000 7f14b2e75640 1 --2- 172.21.15.2:0/3786114695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14ac10d020 0x7f14ac10f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.347+0000 7f14b2e75640 1 --2- 172.21.15.2:0/3786114695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14ac104600 0x7f14ac10cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:20.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.347+0000 7f14b2e75640 1 --2- 172.21.15.2:0/3786114695 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14ac103ce0 0x7f14ac1040c0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:20.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.347+0000 7f14b2e75640 1 -- 172.21.15.2:0/3786114695 >> 172.21.15.2:0/3786114695 conn(0x7f14ac078ad0 msgr2=0x7f14ac078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.347+0000 7f14b2e75640 1 -- 172.21.15.2:0/3786114695 shutdown_connections 2024-05-14T17:07:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.347+0000 7f14b2e75640 1 -- 172.21.15.2:0/3786114695 wait complete. 2024-05-14T17:07:20.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.348+0000 7f14b2e75640 1 Processor -- start 2024-05-14T17:07:20.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.348+0000 7f14b2e75640 1 -- start start 2024-05-14T17:07:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.348+0000 7f14b2e75640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14ac103ce0 0x7f14ac199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:20.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.348+0000 7f14b2e75640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14ac104600 0x7f14ac199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:20.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.348+0000 7f14b2e75640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14ac10d020 0x7f14ac19daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.348+0000 7f14b2e75640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac112cd0 con 0x7f14ac10d020 2024-05-14T17:07:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.348+0000 7f14b2e75640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac112b50 con 0x7f14ac104600 2024-05-14T17:07:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.348+0000 7f14b2e75640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac112e50 con 0x7f14ac103ce0 2024-05-14T17:07:21.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.348+0000 7f14b1412640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14ac10d020 0x7f14ac19daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:21.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.348+0000 7f14b1412640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14ac10d020 0x7f14ac19daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38686/0 (socket says 172.21.15.2:38686) 2024-05-14T17:07:21.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.348+0000 7f14b1412640 1 -- 172.21.15.2:0/1489468254 learned_addr learned my addr 172.21.15.2:0/1489468254 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:07:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.348+0000 7f14b0c11640 1 --2- 172.21.15.2:0/1489468254 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14ac103ce0 0x7f14ac199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:21.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.348+0000 7f14abfff640 1 --2- 172.21.15.2:0/1489468254 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14ac104600 0x7f14ac199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:21.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.348+0000 7f14b1412640 1 -- 172.21.15.2:0/1489468254 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14ac103ce0 msgr2=0x7f14ac199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:21.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.348+0000 7f14b1412640 1 --2- 172.21.15.2:0/1489468254 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14ac103ce0 0x7f14ac199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.348+0000 7f14b1412640 1 -- 172.21.15.2:0/1489468254 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14ac104600 msgr2=0x7f14ac199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.348+0000 7f14b1412640 1 --2- 172.21.15.2:0/1489468254 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14ac104600 0x7f14ac199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.348+0000 7f14b1412640 1 -- 172.21.15.2:0/1489468254 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14ac19e1d0 con 0x7f14ac10d020 2024-05-14T17:07:21.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.349+0000 7f14b1412640 1 --2- 172.21.15.2:0/1489468254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14ac10d020 0x7f14ac19daa0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f14a400cad0 tx=0x7f14a400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:21.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.349+0000 7f14a9ffb640 1 -- 172.21.15.2:0/1489468254 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a400ae20 con 0x7f14ac10d020 2024-05-14T17:07:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.349+0000 7f14a9ffb640 1 -- 172.21.15.2:0/1489468254 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14a4005020 con 0x7f14ac10d020 2024-05-14T17:07:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.349+0000 7f14b2e75640 1 -- 172.21.15.2:0/1489468254 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14ac19e4c0 con 0x7f14ac10d020 2024-05-14T17:07:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.349+0000 7f14a9ffb640 1 -- 172.21.15.2:0/1489468254 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a400f6b0 con 0x7f14ac10d020 2024-05-14T17:07:22.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.349+0000 7f14b2e75640 1 -- 172.21.15.2:0/1489468254 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14ac1a5d50 con 0x7f14ac10d020 2024-05-14T17:07:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.351+0000 7f14a9ffb640 1 -- 172.21.15.2:0/1489468254 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f14a40051d0 con 0x7f14ac10d020 2024-05-14T17:07:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.352+0000 7f14b5b98740 1 Processor -- start 2024-05-14T17:07:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.352+0000 7f14b5b98740 1 -- start start 2024-05-14T17:07:22.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.352+0000 7f14b5b98740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56213a6f77d0 0x56213a6f7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:22.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.352+0000 7f14b5b98740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56213a6f80f0 0x56213a7069a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:22.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.352+0000 7f14b5b98740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56213a706ee0 0x56213a7092d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.352+0000 7f14b5b98740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56213a66ac30 con 0x56213a6f80f0 2024-05-14T17:07:22.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.352+0000 7f14b5b98740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56213a65a7d0 con 0x56213a6f77d0 2024-05-14T17:07:22.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.352+0000 7f14b5b98740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56213a65e810 con 0x56213a706ee0 2024-05-14T17:07:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.353+0000 7f14b0c11640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56213a6f80f0 0x56213a7069a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.353+0000 7f14b0c11640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56213a6f80f0 0x56213a7069a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38700/0 (socket says 172.21.15.2:38700) 2024-05-14T17:07:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.353+0000 7f14b0c11640 1 -- 172.21.15.2:0/56856804 learned_addr learned my addr 172.21.15.2:0/56856804 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:07:23.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.353+0000 7f14a9ffb640 1 --2- 172.21.15.2:0/1489468254 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f149c0b2a90 0x7f149c0b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.353+0000 7f14abfff640 1 --2- 172.21.15.2:0/56856804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56213a6f77d0 0x56213a6f7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.353+0000 7f14abfff640 1 --2- 172.21.15.2:0/56856804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56213a706ee0 0x56213a7092d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.353+0000 7f14b1412640 1 --2- 172.21.15.2:0/1489468254 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f149c0b2a90 0x7f149c0b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.353+0000 7f14b0c11640 1 -- 172.21.15.2:0/56856804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56213a706ee0 msgr2=0x56213a7092d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.353+0000 7f14b0c11640 1 --2- 172.21.15.2:0/56856804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56213a706ee0 0x56213a7092d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:23.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.353+0000 7f14b0c11640 1 -- 172.21.15.2:0/56856804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56213a6f77d0 msgr2=0x56213a6f7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.353+0000 7f14b0c11640 1 --2- 172.21.15.2:0/56856804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56213a6f77d0 0x56213a6f7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.353+0000 7f14b0c11640 1 -- 172.21.15.2:0/56856804 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56213a36f210 con 0x56213a6f80f0 2024-05-14T17:07:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.353+0000 7f14abfff640 1 --2- 172.21.15.2:0/56856804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56213a706ee0 0x56213a7092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:07:23.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.353+0000 7f14a9ffb640 1 -- 172.21.15.2:0/1489468254 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f14a40e2570 con 0x7f14ac10d020 2024-05-14T17:07:23.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.353+0000 7f14b0c11640 1 --2- 172.21.15.2:0/56856804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56213a6f80f0 0x56213a7069a0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f14a0033000 tx=0x7f14a0032cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eb7712c1ff1920f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.353+0000 7f14b1412640 1 --2- 172.21.15.2:0/1489468254 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f149c0b2a90 0x7f149c0b4f50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f14a40a97e0 tx=0x7f14a40cc040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:23.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.353+0000 7f1496ffd640 1 -- 172.21.15.2:0/56856804 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a0035040 con 0x56213a6f80f0 2024-05-14T17:07:23.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.353+0000 7f1496ffd640 1 -- 172.21.15.2:0/56856804 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14a00316a0 con 0x56213a6f80f0 2024-05-14T17:07:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.353+0000 7f1496ffd640 1 -- 172.21.15.2:0/56856804 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a0005650 con 0x56213a6f80f0 2024-05-14T17:07:24.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.354+0000 7f14b5b98740 1 -- 172.21.15.2:0/56856804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56213a6f80f0 msgr2=0x56213a7069a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.354+0000 7f14b5b98740 1 --2- 172.21.15.2:0/56856804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56213a6f80f0 0x56213a7069a0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f14a0033000 tx=0x7f14a0032cd0 comp rx=0 tx=0).stop 2024-05-14T17:07:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.354+0000 7f14b5b98740 1 -- 172.21.15.2:0/56856804 shutdown_connections 2024-05-14T17:07:24.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.354+0000 7f14b5b98740 1 --2- 172.21.15.2:0/56856804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56213a706ee0 0x56213a7092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:24.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.354+0000 7f14b5b98740 1 --2- 172.21.15.2:0/56856804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56213a6f80f0 0x56213a7069a0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.354+0000 7f14b5b98740 1 --2- 172.21.15.2:0/56856804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56213a6f77d0 0x56213a6f7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:24.514 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.354+0000 7f14b5b98740 1 -- 172.21.15.2:0/56856804 >> 172.21.15.2:0/56856804 conn(0x56213a6f3e00 msgr2=0x56213a6f41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.354+0000 7f14b5b98740 1 -- 172.21.15.2:0/56856804 shutdown_connections 2024-05-14T17:07:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.354+0000 7f14b5b98740 1 -- 172.21.15.2:0/56856804 wait complete. 2024-05-14T17:07:24.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.355+0000 7f14b5b98740 1 -- start start 2024-05-14T17:07:24.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.355+0000 7f14b5b98740 1 Processor -- start 2024-05-14T17:07:24.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.355+0000 7f14b5b98740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56213a6f77d0 0x56213a706830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:24.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.355+0000 7f14b5b98740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56213a6f80f0 0x56213a703b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:25.014 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.355+0000 7f14b5b98740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56213a706ee0 0x56213a704080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.355+0000 7f14b5b98740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56213a340790 con 0x56213a6f77d0 2024-05-14T17:07:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.355+0000 7f14b5b98740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56213a616890 con 0x56213a706ee0 2024-05-14T17:07:25.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.355+0000 7f14b5b98740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56213a709b50 con 0x56213a6f80f0 2024-05-14T17:07:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.356+0000 7f14b5b98740 10 client.0 _ll_register_callbacks cb 0x56213a619d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:07:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.356+0000 7f14abfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56213a6f77d0 0x56213a706830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:25.378 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.356+0000 7f14abfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56213a6f77d0 0x56213a706830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38702/0 (socket says 172.21.15.2:38702) 2024-05-14T17:07:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.356+0000 7f14abfff640 1 -- 172.21.15.2:0/4139002528 learned_addr learned my addr 172.21.15.2:0/4139002528 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:07:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.356+0000 7f14b0c11640 1 --2- 172.21.15.2:0/4139002528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56213a6f80f0 0x56213a703b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:25.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.356+0000 7f14abfff640 1 --2- 172.21.15.2:0/4139002528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56213a706ee0 0x56213a704080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:25.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.356+0000 7f14abfff640 1 -- 172.21.15.2:0/4139002528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56213a6f80f0 msgr2=0x56213a703b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.356+0000 7f14abfff640 1 --2- 172.21.15.2:0/4139002528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56213a6f80f0 0x56213a703b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.356+0000 7f14abfff640 1 -- 172.21.15.2:0/4139002528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56213a706ee0 msgr2=0x56213a704080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.356+0000 7f14abfff640 1 --2- 172.21.15.2:0/4139002528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56213a706ee0 0x56213a704080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.356+0000 7f14abfff640 1 -- 172.21.15.2:0/4139002528 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56213a43b620 con 0x56213a6f77d0 2024-05-14T17:07:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.356+0000 7f14abfff640 1 --2- 172.21.15.2:0/4139002528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56213a6f77d0 0x56213a706830 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1498009e30 tx=0x7f1498004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=728c204cd10a7b00 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.356+0000 7f148bfff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:07:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.356+0000 7f148bfff640 1 -- 172.21.15.2:0/4139002528 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149800ed40 con 0x56213a6f77d0 2024-05-14T17:07:26.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.356+0000 7f14b5b98740 1 -- 172.21.15.2:0/4139002528 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14a00316a0 con 0x56213a6f77d0 2024-05-14T17:07:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.356+0000 7f148bfff640 1 -- 172.21.15.2:0/4139002528 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f149800f420 con 0x56213a6f77d0 2024-05-14T17:07:26.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.356+0000 7f148bfff640 1 -- 172.21.15.2:0/4139002528 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f149800f620 con 0x56213a6f77d0 2024-05-14T17:07:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.357+0000 7f14b5b98740 10 client.16584 fetch_fsmap learned FSMap version 45 2024-05-14T17:07:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.357+0000 7f14b5b98740 1 -- 172.21.15.2:0/4139002528 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x56213a715680 con 0x56213a6f77d0 2024-05-14T17:07:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.357+0000 7f148bfff640 1 -- 172.21.15.2:0/4139002528 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14a00316a0 con 0x56213a6f77d0 2024-05-14T17:07:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.357+0000 7f148bfff640 1 -- 172.21.15.2:0/4139002528 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f149802a850 con 0x56213a6f77d0 2024-05-14T17:07:26.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.357+0000 7f14b5b98740 10 client.16584 fetch_fsmap finished waiting for FSMap version 45 2024-05-14T17:07:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.357+0000 7f14b5b98740 10 client.16584 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:07:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.357+0000 7f14b5b98740 20 client.16584 populate_metadata read hostname 'smithi002' 2024-05-14T17:07:27.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.357+0000 7f14b5b98740 1 --2- 172.21.15.2:0/4139002528 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56213a716af0 0x56213a718f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.357+0000 7f14b5b98740 4 client.16584 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:07:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.357+0000 7f14b5b98740 1 -- 172.21.15.2:0/4139002528 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56213a66cb80 con 0x56213a716af0 2024-05-14T17:07:27.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.357+0000 7f14b0c11640 1 --2- 172.21.15.2:0/4139002528 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56213a716af0 0x56213a718f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:27.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.358+0000 7f148bfff640 10 client.16584 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:07:27.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.358+0000 7f14b0c11640 1 --2- 172.21.15.2:0/4139002528 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56213a716af0 0x56213a718f20 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f601efcf9a4a4ef5 server_cookie=d5a678a6f1dfc0ea in_seq=0 out_seq=0 2024-05-14T17:07:27.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.358+0000 7f148bfff640 1 -- 172.21.15.2:0/4139002528 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56213a66cb80 con 0x56213a716af0 2024-05-14T17:07:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.358+0000 7f148bfff640 10 client.16584 handle_command_reply: tid=0 2024-05-14T17:07:27.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.358+0000 7f14b5b98740 1 client.16584 shutdown 2024-05-14T17:07:27.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.359+0000 7f14b5b98740 1 -- 172.21.15.2:0/4139002528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56213a6f77d0 msgr2=0x56213a706830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:27.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.359+0000 7f14b5b98740 1 --2- 172.21.15.2:0/4139002528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56213a6f77d0 0x56213a706830 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1498009e30 tx=0x7f1498004730 comp rx=0 tx=0).stop 2024-05-14T17:07:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.359+0000 7f14b5b98740 1 -- 172.21.15.2:0/4139002528 shutdown_connections 2024-05-14T17:07:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.359+0000 7f14b5b98740 1 --2- 172.21.15.2:0/4139002528 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56213a716af0 0x56213a718f20 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:27.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.359+0000 7f14b5b98740 1 --2- 172.21.15.2:0/4139002528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56213a706ee0 0x56213a704080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.359+0000 7f148bfff640 0 client.16584 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:07:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.359+0000 7f14b5b98740 1 --2- 172.21.15.2:0/4139002528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56213a6f80f0 0x56213a703b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.359+0000 7f14b5b98740 1 --2- 172.21.15.2:0/4139002528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56213a6f77d0 0x56213a706830 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:28.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.359+0000 7f14b5b98740 1 -- 172.21.15.2:0/4139002528 >> 172.21.15.2:0/4139002528 conn(0x56213a6f3e00 msgr2=0x56213a6f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:28.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.359+0000 7f14b5b98740 1 -- 172.21.15.2:0/4139002528 shutdown_connections 2024-05-14T17:07:28.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.359+0000 7f14b5b98740 1 -- 172.21.15.2:0/4139002528 wait complete. 2024-05-14T17:07:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.359+0000 7f14b5b98740 20 client.16584 trim_cache size 0 max 16384 2024-05-14T17:07:28.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.375+0000 7f14b5b98740 1 Processor -- start 2024-05-14T17:07:28.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.375+0000 7f14b5b98740 1 -- start start 2024-05-14T17:07:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.375+0000 7f14b5b98740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56213a6f77d0 0x56213a6ff1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:28.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.375+0000 7f14b5b98740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56213a6f80f0 0x56213a712a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:28.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.376+0000 7f14b5b98740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56213a706ee0 0x56213a706360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.376+0000 7f14b5b98740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac1aedd0 con 0x56213a6f77d0 2024-05-14T17:07:29.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.376+0000 7f14b5b98740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac1ac640 con 0x56213a706ee0 2024-05-14T17:07:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.376+0000 7f14b5b98740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac1aa4b0 con 0x56213a6f80f0 2024-05-14T17:07:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.376+0000 7f14abfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56213a6f77d0 0x56213a6ff1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.376+0000 7f14abfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56213a6f77d0 0x56213a6ff1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38708/0 (socket says 172.21.15.2:38708) 2024-05-14T17:07:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.376+0000 7f14abfff640 1 -- 172.21.15.2:0/958335588 learned_addr learned my addr 172.21.15.2:0/958335588 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:07:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.376+0000 7f14b0c11640 1 --2- 172.21.15.2:0/958335588 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56213a6f80f0 0x56213a712a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.376+0000 7f14abfff640 1 --2- 172.21.15.2:0/958335588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56213a706ee0 0x56213a706360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.376+0000 7f14abfff640 1 -- 172.21.15.2:0/958335588 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56213a6f80f0 msgr2=0x56213a712a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.376+0000 7f14abfff640 1 --2- 172.21.15.2:0/958335588 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56213a6f80f0 0x56213a712a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:29.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.376+0000 7f14abfff640 1 -- 172.21.15.2:0/958335588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56213a706ee0 msgr2=0x56213a706360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.376+0000 7f14abfff640 1 --2- 172.21.15.2:0/958335588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56213a706ee0 0x56213a706360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.376+0000 7f14abfff640 1 -- 172.21.15.2:0/958335588 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56213a6ee510 con 0x56213a6f77d0 2024-05-14T17:07:30.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.376+0000 7f14abfff640 1 --2- 172.21.15.2:0/958335588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56213a6f77d0 0x56213a6ff1d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f1498006010 tx=0x7f1498005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9deeba83d075e3b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:30.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.376+0000 7f14967fc640 1 -- 172.21.15.2:0/958335588 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149800ec10 con 0x56213a6f77d0 2024-05-14T17:07:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.376+0000 7f14967fc640 1 -- 172.21.15.2:0/958335588 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1498031020 con 0x56213a6f77d0 2024-05-14T17:07:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.376+0000 7f14967fc640 1 -- 172.21.15.2:0/958335588 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149802a840 con 0x56213a6f77d0 2024-05-14T17:07:30.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.377+0000 7f14b5b98740 1 -- 172.21.15.2:0/958335588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56213a6f77d0 msgr2=0x56213a6ff1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.377+0000 7f14b5b98740 1 --2- 172.21.15.2:0/958335588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56213a6f77d0 0x56213a6ff1d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f1498006010 tx=0x7f1498005bb0 comp rx=0 tx=0).stop 2024-05-14T17:07:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.377+0000 7f14b5b98740 1 -- 172.21.15.2:0/958335588 shutdown_connections 2024-05-14T17:07:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.377+0000 7f14b5b98740 1 --2- 172.21.15.2:0/958335588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56213a706ee0 0x56213a706360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.377+0000 7f14b5b98740 1 --2- 172.21.15.2:0/958335588 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56213a6f80f0 0x56213a712a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:30.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.377+0000 7f14b5b98740 1 --2- 172.21.15.2:0/958335588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56213a6f77d0 0x56213a6ff1d0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:30.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.377+0000 7f14b5b98740 1 -- 172.21.15.2:0/958335588 >> 172.21.15.2:0/958335588 conn(0x56213a6f3e00 msgr2=0x56213a6eef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.377+0000 7f14b5b98740 1 -- 172.21.15.2:0/958335588 shutdown_connections 2024-05-14T17:07:31.011 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.377+0000 7f14b5b98740 1 -- 172.21.15.2:0/958335588 wait complete. 2024-05-14T17:07:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.378+0000 7f14b5b98740 1 -- start start 2024-05-14T17:07:31.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.378+0000 7f14b5b98740 1 Processor -- start 2024-05-14T17:07:31.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.378+0000 7f14b5b98740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56213a6f77d0 0x56213a70f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.378+0000 7f14b5b98740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56213a6f80f0 0x56213a70a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:31.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.378+0000 7f14b5b98740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56213a706ee0 0x56213a70a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.378+0000 7f14b5b98740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac1aab00 con 0x56213a6f80f0 2024-05-14T17:07:31.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.378+0000 7f14b5b98740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac1abf70 con 0x56213a6f77d0 2024-05-14T17:07:32.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.378+0000 7f14b5b98740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac1ab1d0 con 0x56213a706ee0 2024-05-14T17:07:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.378+0000 7f14b5b98740 10 client.0 _ll_register_callbacks cb 0x7f1498011980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:07:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.378+0000 7f14b0c11640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56213a6f80f0 0x56213a70a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:32.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.378+0000 7f14abfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56213a6f77d0 0x56213a70f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.378+0000 7f14b0c11640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56213a6f80f0 0x56213a70a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38724/0 (socket says 172.21.15.2:38724) 2024-05-14T17:07:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.378+0000 7f14b0c11640 1 -- 172.21.15.2:0/446836989 learned_addr learned my addr 172.21.15.2:0/446836989 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:07:32.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.378+0000 7f14abfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56213a6f77d0 0x56213a70f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:59556/0 (socket says 172.21.15.2:59556) 2024-05-14T17:07:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.378+0000 7f14abfff640 1 --2- 172.21.15.2:0/446836989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56213a706ee0 0x56213a70a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:32.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.379+0000 7f14b0c11640 1 -- 172.21.15.2:0/446836989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56213a706ee0 msgr2=0x56213a70a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:32.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.379+0000 7f14b0c11640 1 --2- 172.21.15.2:0/446836989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56213a706ee0 0x56213a70a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.379+0000 7f14b0c11640 1 -- 172.21.15.2:0/446836989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56213a6f77d0 msgr2=0x56213a70f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.379+0000 7f14b0c11640 1 --2- 172.21.15.2:0/446836989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56213a6f77d0 0x56213a70f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.379+0000 7f14b0c11640 1 -- 172.21.15.2:0/446836989 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56213a70aed0 con 0x56213a6f80f0 2024-05-14T17:07:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.379+0000 7f14abfff640 1 --2- 172.21.15.2:0/446836989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56213a6f77d0 0x56213a70f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:07:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.379+0000 7f14b0c11640 1 --2- 172.21.15.2:0/446836989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56213a6f80f0 0x56213a70a450 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f14a00052e0 tx=0x7f14a0031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faf75f75215e8c08 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.379+0000 7f148bfff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:07:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.379+0000 7f148bfff640 1 -- 172.21.15.2:0/446836989 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a0056060 con 0x56213a6f80f0 2024-05-14T17:07:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.379+0000 7f148bfff640 1 -- 172.21.15.2:0/446836989 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f14a0031520 con 0x56213a6f80f0 2024-05-14T17:07:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.379+0000 7f14b5b98740 1 -- 172.21.15.2:0/446836989 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1498031020 con 0x56213a6f80f0 2024-05-14T17:07:33.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.379+0000 7f148bfff640 1 -- 172.21.15.2:0/446836989 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f14a0035040 con 0x56213a6f80f0 2024-05-14T17:07:33.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.379+0000 7f14b5b98740 10 client.16590 fetch_fsmap learned FSMap version 45 2024-05-14T17:07:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.379+0000 7f14b5b98740 1 -- 172.21.15.2:0/446836989 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x56213a781130 con 0x56213a6f80f0 2024-05-14T17:07:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.379+0000 7f148bfff640 1 -- 172.21.15.2:0/446836989 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1498031020 con 0x56213a6f80f0 2024-05-14T17:07:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.379+0000 7f148bfff640 1 -- 172.21.15.2:0/446836989 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f14a0032440 con 0x56213a6f80f0 2024-05-14T17:07:34.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.380+0000 7f14b5b98740 10 client.16590 fetch_fsmap finished waiting for FSMap version 45 2024-05-14T17:07:34.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.380+0000 7f14b5b98740 10 client.16590 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:07:34.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.380+0000 7f14b5b98740 20 client.16590 populate_metadata read hostname 'smithi002' 2024-05-14T17:07:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.380+0000 7f14b5b98740 1 --2- 172.21.15.2:0/446836989 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56213a781430 0x56213a783850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:34.402 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.380+0000 7f14b5b98740 4 client.16590 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:07:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.380+0000 7f14b5b98740 1 -- 172.21.15.2:0/446836989 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56213a666360 con 0x56213a781430 2024-05-14T17:07:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.380+0000 7f14abfff640 1 --2- 172.21.15.2:0/446836989 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56213a781430 0x56213a783850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.380+0000 7f14abfff640 1 --2- 172.21.15.2:0/446836989 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56213a781430 0x56213a783850 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f0fdaf777a45055 server_cookie=579348cdaed5d5fa in_seq=0 out_seq=0 2024-05-14T17:07:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.380+0000 7f148bfff640 10 client.16590 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:07:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.390+0000 7f148bfff640 1 -- 172.21.15.2:0/446836989 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56213a666360 con 0x56213a781430 2024-05-14T17:07:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.390+0000 7f148bfff640 10 client.16590 handle_command_reply: tid=0 2024-05-14T17:07:34.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.390+0000 7f14b5b98740 1 client.16590 shutdown 2024-05-14T17:07:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.391+0000 7f14b5b98740 1 -- 172.21.15.2:0/446836989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56213a6f80f0 msgr2=0x56213a70a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:35.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.391+0000 7f14b5b98740 1 --2- 172.21.15.2:0/446836989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56213a6f80f0 0x56213a70a450 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f14a00052e0 tx=0x7f14a0031400 comp rx=0 tx=0).stop 2024-05-14T17:07:35.219 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.391+0000 7f14b5b98740 1 -- 172.21.15.2:0/446836989 shutdown_connections 2024-05-14T17:07:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.391+0000 7f14b5b98740 1 --2- 172.21.15.2:0/446836989 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56213a781430 0x56213a783850 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.391+0000 7f14b5b98740 1 --2- 172.21.15.2:0/446836989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56213a706ee0 0x56213a70a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.391+0000 7f14b5b98740 1 --2- 172.21.15.2:0/446836989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56213a6f80f0 0x56213a70a450 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:35.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.391+0000 7f148bfff640 0 client.16590 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:07:35.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.391+0000 7f14b5b98740 1 --2- 172.21.15.2:0/446836989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56213a6f77d0 0x56213a70f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:35.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.391+0000 7f14b5b98740 1 -- 172.21.15.2:0/446836989 >> 172.21.15.2:0/446836989 conn(0x56213a6f3e00 msgr2=0x56213a6f2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.391+0000 7f14b5b98740 1 -- 172.21.15.2:0/446836989 shutdown_connections 2024-05-14T17:07:35.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.391+0000 7f14b5b98740 1 -- 172.21.15.2:0/446836989 wait complete. 2024-05-14T17:07:36.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.391+0000 7f14b5b98740 20 client.16590 trim_cache size 0 max 16384 2024-05-14T17:07:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:07:36.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:07:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:07:36.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.392+0000 7f14b2e75640 1 -- 172.21.15.2:0/1489468254 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f149c0b2a90 msgr2=0x7f149c0b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.392+0000 7f14b2e75640 1 --2- 172.21.15.2:0/1489468254 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f149c0b2a90 0x7f149c0b4f50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f14a40a97e0 tx=0x7f14a40cc040 comp rx=0 tx=0).stop 2024-05-14T17:07:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.392+0000 7f14b2e75640 1 -- 172.21.15.2:0/1489468254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14ac10d020 msgr2=0x7f14ac19daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.392+0000 7f14b2e75640 1 --2- 172.21.15.2:0/1489468254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14ac10d020 0x7f14ac19daa0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f14a400cad0 tx=0x7f14a400cf90 comp rx=0 tx=0).stop 2024-05-14T17:07:36.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.392+0000 7f14b2e75640 1 -- 172.21.15.2:0/1489468254 shutdown_connections 2024-05-14T17:07:36.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.392+0000 7f14b2e75640 1 --2- 172.21.15.2:0/1489468254 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f149c0b2a90 0x7f149c0b4f50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:36.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.392+0000 7f14b2e75640 1 --2- 172.21.15.2:0/1489468254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14ac10d020 0x7f14ac19daa0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.392+0000 7f14b2e75640 1 --2- 172.21.15.2:0/1489468254 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14ac104600 0x7f14ac199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:37.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.392+0000 7f14b2e75640 1 --2- 172.21.15.2:0/1489468254 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14ac103ce0 0x7f14ac199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:37.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.392+0000 7f14b2e75640 1 -- 172.21.15.2:0/1489468254 >> 172.21.15.2:0/1489468254 conn(0x7f14ac078ad0 msgr2=0x7f14ac0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.392+0000 7f14b2e75640 1 -- 172.21.15.2:0/1489468254 shutdown_connections 2024-05-14T17:07:37.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:36.392+0000 7f14b2e75640 1 -- 172.21.15.2:0/1489468254 wait complete. 2024-05-14T17:07:37.412 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 680.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:07:37.516 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:06:36 smithi002 ceph-mon[31538]: pgmap v370: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 KiB/s wr, 8 op/s 2024-05-14T17:07:37.607 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:06:38 smithi002 ceph-mon[31538]: pgmap v371: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 KiB/s wr, 8 op/s 2024-05-14T17:07:37.653 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:06:40 smithi002 ceph-mon[31538]: pgmap v372: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 36 KiB/s wr, 14 op/s 2024-05-14T17:07:37.748 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 685.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:07:37.829 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:06:41 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:07:37.884 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:06:42 smithi002 ceph-mon[31538]: pgmap v373: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 8 op/s 2024-05-14T17:07:38.012 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:06:44 smithi002 ceph-mon[31538]: pgmap v374: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-05-14T17:07:38.093 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 690.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:07:38.177 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:06:46 smithi002 ceph-mon[31538]: pgmap v375: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-05-14T17:07:38.267 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:06:48 smithi002 ceph-mon[31538]: pgmap v376: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-05-14T17:07:38.338 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:07:20 smithi151 ceph-mon[36066]: pgmap v394: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-05-14T17:07:38.382 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:07:22 smithi151 ceph-mon[36066]: pgmap v395: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-05-14T17:07:38.417 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:07:24 smithi151 ceph-mon[36066]: pgmap v396: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 11 op/s 2024-05-14T17:07:38.459 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:07:26 smithi151 ceph-mon[36066]: pgmap v397: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-05-14T17:07:38.501 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:07:28 smithi151 ceph-mon[36066]: pgmap v398: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-05-14T17:07:38.542 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:07:30 smithi151 ceph-mon[36066]: pgmap v399: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-05-14T17:07:38.584 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:07:31 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:07:38.626 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:07:32 smithi151 ceph-mon[36066]: pgmap v400: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:07:38.668 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:07:34 smithi151 ceph-mon[36066]: pgmap v401: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.1 KiB/s wr, 7 op/s 2024-05-14T17:07:38.709 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:07:36 smithi151 ceph-mon[36066]: pgmap v402: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 KiB/s wr, 5 op/s 2024-05-14T17:07:38.795 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:07:20 smithi123 ceph-mon[35971]: pgmap v394: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-05-14T17:07:38.876 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:07:22 smithi123 ceph-mon[35971]: pgmap v395: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-05-14T17:07:38.947 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:07:24 smithi123 ceph-mon[35971]: pgmap v396: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 11 op/s 2024-05-14T17:07:39.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:07:26 smithi123 ceph-mon[35971]: pgmap v397: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-05-14T17:07:39.051 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:07:28 smithi123 ceph-mon[35971]: pgmap v398: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-05-14T17:07:39.120 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:07:30 smithi123 ceph-mon[35971]: pgmap v399: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-05-14T17:07:39.159 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:07:31 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:07:39.233 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:07:32 smithi123 ceph-mon[35971]: pgmap v400: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:07:39.302 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:07:34 smithi123 ceph-mon[35971]: pgmap v401: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.1 KiB/s wr, 7 op/s 2024-05-14T17:07:39.363 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:07:36 smithi123 ceph-mon[35971]: pgmap v402: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 KiB/s wr, 5 op/s 2024-05-14T17:07:39.482 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:07:38 smithi123 ceph-mon[35971]: pgmap v403: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 KiB/s wr, 5 op/s 2024-05-14T17:07:39.518 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:07:38 smithi151 ceph-mon[36066]: pgmap v403: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 KiB/s wr, 5 op/s 2024-05-14T17:07:39.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.493+0000 7f6931254640 1 -- 172.21.15.2:0/4044839444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f692c104600 msgr2=0x7f692c10cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.493+0000 7f6931254640 1 --2- 172.21.15.2:0/4044839444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f692c104600 0x7f692c10cae0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6920009900 tx=0x7f692002a510 comp rx=0 tx=0).stop 2024-05-14T17:07:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.493+0000 7f6931254640 1 -- 172.21.15.2:0/4044839444 shutdown_connections 2024-05-14T17:07:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.493+0000 7f6931254640 1 --2- 172.21.15.2:0/4044839444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f692c10d020 0x7f692c10f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:39.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.493+0000 7f6931254640 1 --2- 172.21.15.2:0/4044839444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f692c104600 0x7f692c10cae0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.493+0000 7f6931254640 1 --2- 172.21.15.2:0/4044839444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f692c103ce0 0x7f692c1040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.494+0000 7f6931254640 1 -- 172.21.15.2:0/4044839444 >> 172.21.15.2:0/4044839444 conn(0x7f692c078ad0 msgr2=0x7f692c078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:40.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.494+0000 7f6931254640 1 -- 172.21.15.2:0/4044839444 shutdown_connections 2024-05-14T17:07:40.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.494+0000 7f6931254640 1 -- 172.21.15.2:0/4044839444 wait complete. 2024-05-14T17:07:40.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.494+0000 7f6931254640 1 Processor -- start 2024-05-14T17:07:40.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.495+0000 7f6931254640 1 -- start start 2024-05-14T17:07:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.495+0000 7f6931254640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f692c103ce0 0x7f692c199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.495+0000 7f6931254640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f692c104600 0x7f692c1996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:40.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.495+0000 7f6931254640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f692c10d020 0x7f692c19da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.495+0000 7f6931254640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f692c112cd0 con 0x7f692c104600 2024-05-14T17:07:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.495+0000 7f6931254640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f692c112b50 con 0x7f692c10d020 2024-05-14T17:07:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.495+0000 7f6931254640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f692c112e50 con 0x7f692c103ce0 2024-05-14T17:07:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.495+0000 7f692a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f692c104600 0x7f692c1996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:41.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.495+0000 7f692a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f692c104600 0x7f692c1996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58430/0 (socket says 172.21.15.2:58430) 2024-05-14T17:07:41.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.495+0000 7f692a59c640 1 -- 172.21.15.2:0/46303632 learned_addr learned my addr 172.21.15.2:0/46303632 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:07:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.495+0000 7f692ad9d640 1 --2- 172.21.15.2:0/46303632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f692c103ce0 0x7f692c199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:41.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.495+0000 7f692b59e640 1 --2- 172.21.15.2:0/46303632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f692c10d020 0x7f692c19da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:41.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.495+0000 7f692a59c640 1 -- 172.21.15.2:0/46303632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f692c103ce0 msgr2=0x7f692c199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.495+0000 7f692a59c640 1 --2- 172.21.15.2:0/46303632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f692c103ce0 0x7f692c199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:41.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.495+0000 7f692a59c640 1 -- 172.21.15.2:0/46303632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f692c10d020 msgr2=0x7f692c19da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.495+0000 7f692a59c640 1 --2- 172.21.15.2:0/46303632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f692c10d020 0x7f692c19da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:41.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.495+0000 7f692a59c640 1 -- 172.21.15.2:0/46303632 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f692c19e160 con 0x7f692c104600 2024-05-14T17:07:41.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.496+0000 7f692a59c640 1 --2- 172.21.15.2:0/46303632 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f692c104600 0x7f692c1996b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f6920006010 tx=0x7f69200042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:41.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.496+0000 7f6913fff640 1 -- 172.21.15.2:0/46303632 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692003f030 con 0x7f692c104600 2024-05-14T17:07:42.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.496+0000 7f6931254640 1 -- 172.21.15.2:0/46303632 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f692c19e3f0 con 0x7f692c104600 2024-05-14T17:07:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.496+0000 7f6913fff640 1 -- 172.21.15.2:0/46303632 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6920003800 con 0x7f692c104600 2024-05-14T17:07:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.496+0000 7f6913fff640 1 -- 172.21.15.2:0/46303632 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6920035070 con 0x7f692c104600 2024-05-14T17:07:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.496+0000 7f6931254640 1 -- 172.21.15.2:0/46303632 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f692c1a5ce0 con 0x7f692c104600 2024-05-14T17:07:42.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.499+0000 7f6913fff640 1 -- 172.21.15.2:0/46303632 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f6920003cb0 con 0x7f692c104600 2024-05-14T17:07:42.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.499+0000 7f6933f6b740 1 Processor -- start 2024-05-14T17:07:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.499+0000 7f6933f6b740 1 -- start start 2024-05-14T17:07:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.499+0000 7f6933f6b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622df0738f0 0x5622df073cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:42.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.499+0000 7f6933f6b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622df074210 0x5622df07e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.499+0000 7f6933f6b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622df07ead0 0x5622df080ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:42.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.499+0000 7f6933f6b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5622defe6c30 con 0x5622df074210 2024-05-14T17:07:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.499+0000 7f6933f6b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5622defd67d0 con 0x5622df0738f0 2024-05-14T17:07:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.499+0000 7f6933f6b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5622defda810 con 0x5622df07ead0 2024-05-14T17:07:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.499+0000 7f692ad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622df074210 0x5622df07e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:43.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.499+0000 7f692ad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622df074210 0x5622df07e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58442/0 (socket says 172.21.15.2:58442) 2024-05-14T17:07:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.499+0000 7f692ad9d640 1 -- 172.21.15.2:0/1239278288 learned_addr learned my addr 172.21.15.2:0/1239278288 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:07:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.499+0000 7f692b59e640 1 --2- 172.21.15.2:0/1239278288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622df0738f0 0x5622df073cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:43.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.499+0000 7f692a59c640 1 --2- 172.21.15.2:0/1239278288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622df07ead0 0x5622df080ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.500+0000 7f6913fff640 1 --2- 172.21.15.2:0/46303632 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f69140b2a10 0x7f69140b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.500+0000 7f692ad9d640 1 -- 172.21.15.2:0/1239278288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622df07ead0 msgr2=0x5622df080ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.500+0000 7f692ad9d640 1 --2- 172.21.15.2:0/1239278288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622df07ead0 0x5622df080ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.500+0000 7f692ad9d640 1 -- 172.21.15.2:0/1239278288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622df0738f0 msgr2=0x5622df073cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.500+0000 7f692ad9d640 1 --2- 172.21.15.2:0/1239278288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622df0738f0 0x5622df073cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:43.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.500+0000 7f692ad9d640 1 -- 172.21.15.2:0/1239278288 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622deceb210 con 0x5622df074210 2024-05-14T17:07:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.500+0000 7f692a59c640 1 --2- 172.21.15.2:0/1239278288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622df07ead0 0x5622df080ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:07:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.500+0000 7f692b59e640 1 --2- 172.21.15.2:0/46303632 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f69140b2a10 0x7f69140b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.500+0000 7f6913fff640 1 -- 172.21.15.2:0/46303632 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f6920102eb0 con 0x7f692c104600 2024-05-14T17:07:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.500+0000 7f692ad9d640 1 --2- 172.21.15.2:0/1239278288 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622df074210 0x5622df07e590 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f691c00ad00 tx=0x7f691c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38de63f6bd710808 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.500+0000 7f69117fa640 1 -- 172.21.15.2:0/1239278288 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c01d070 con 0x5622df074210 2024-05-14T17:07:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.500+0000 7f69117fa640 1 -- 172.21.15.2:0/1239278288 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f691c018af0 con 0x5622df074210 2024-05-14T17:07:44.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.500+0000 7f69117fa640 1 -- 172.21.15.2:0/1239278288 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c010b40 con 0x5622df074210 2024-05-14T17:07:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.501+0000 7f6933f6b740 1 -- 172.21.15.2:0/1239278288 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622df074210 msgr2=0x5622df07e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.501+0000 7f692b59e640 1 --2- 172.21.15.2:0/46303632 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f69140b2a10 0x7f69140b4ed0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6924007fc0 tx=0x7f6924004a40 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.501+0000 7f6933f6b740 1 --2- 172.21.15.2:0/1239278288 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622df074210 0x5622df07e590 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f691c00ad00 tx=0x7f691c00f790 comp rx=0 tx=0).stop 2024-05-14T17:07:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.501+0000 7f6933f6b740 1 -- 172.21.15.2:0/1239278288 shutdown_connections 2024-05-14T17:07:44.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.501+0000 7f6933f6b740 1 --2- 172.21.15.2:0/1239278288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622df07ead0 0x5622df080ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.501+0000 7f6933f6b740 1 --2- 172.21.15.2:0/1239278288 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622df074210 0x5622df07e590 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.501+0000 7f6933f6b740 1 --2- 172.21.15.2:0/1239278288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622df0738f0 0x5622df073cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.501+0000 7f6933f6b740 1 -- 172.21.15.2:0/1239278288 >> 172.21.15.2:0/1239278288 conn(0x5622df06fe00 msgr2=0x5622df0701e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.501+0000 7f6933f6b740 1 -- 172.21.15.2:0/1239278288 shutdown_connections 2024-05-14T17:07:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.501+0000 7f6933f6b740 1 -- 172.21.15.2:0/1239278288 wait complete. 2024-05-14T17:07:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.501+0000 7f6933f6b740 1 -- start start 2024-05-14T17:07:45.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.501+0000 7f6933f6b740 1 Processor -- start 2024-05-14T17:07:45.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.502+0000 7f6933f6b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622df0738f0 0x5622df088750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.502+0000 7f6933f6b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622df074210 0x5622df088c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:45.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.502+0000 7f6933f6b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622df07ead0 0x5622df082ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.502+0000 7f6933f6b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5622decbc790 con 0x5622df07ead0 2024-05-14T17:07:45.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.502+0000 7f6933f6b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5622def92890 con 0x5622df0738f0 2024-05-14T17:07:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.502+0000 7f6933f6b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5622df0817a0 con 0x5622df074210 2024-05-14T17:07:45.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.502+0000 7f6933f6b740 10 client.0 _ll_register_callbacks cb 0x5622def95d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:07:45.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.502+0000 7f692b59e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622df07ead0 0x5622df082ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.502+0000 7f692b59e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622df07ead0 0x5622df082ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58448/0 (socket says 172.21.15.2:58448) 2024-05-14T17:07:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.502+0000 7f692b59e640 1 -- 172.21.15.2:0/629430211 learned_addr learned my addr 172.21.15.2:0/629430211 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:07:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.502+0000 7f692a59c640 1 --2- 172.21.15.2:0/629430211 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622df074210 0x5622df088c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.502+0000 7f692ad9d640 1 --2- 172.21.15.2:0/629430211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622df0738f0 0x5622df088750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.502+0000 7f692b59e640 1 -- 172.21.15.2:0/629430211 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622df074210 msgr2=0x5622df088c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.502+0000 7f692b59e640 1 --2- 172.21.15.2:0/629430211 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622df074210 0x5622df088c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.502+0000 7f692b59e640 1 -- 172.21.15.2:0/629430211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622df0738f0 msgr2=0x5622df088750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.502+0000 7f692b59e640 1 --2- 172.21.15.2:0/629430211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622df0738f0 0x5622df088750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.502+0000 7f692b59e640 1 -- 172.21.15.2:0/629430211 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622dedb7620 con 0x5622df07ead0 2024-05-14T17:07:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.502+0000 7f692ad9d640 1 --2- 172.21.15.2:0/629430211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622df0738f0 0x5622df088750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:07:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.502+0000 7f692b59e640 1 --2- 172.21.15.2:0/629430211 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622df07ead0 0x5622df082ac0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f69240049d0 tx=0x7f6924004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=711f11b7190d802d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:46.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.502+0000 7f690eff5640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:07:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.503+0000 7f690eff5640 1 -- 172.21.15.2:0/629430211 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6924010bc0 con 0x5622df07ead0 2024-05-14T17:07:46.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.503+0000 7f690eff5640 1 -- 172.21.15.2:0/629430211 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f6924010d60 con 0x5622df07ead0 2024-05-14T17:07:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.503+0000 7f6933f6b740 1 -- 172.21.15.2:0/629430211 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f691c018af0 con 0x5622df07ead0 2024-05-14T17:07:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.503+0000 7f690eff5640 1 -- 172.21.15.2:0/629430211 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f692401f400 con 0x5622df07ead0 2024-05-14T17:07:46.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.503+0000 7f6933f6b740 10 client.16602 fetch_fsmap learned FSMap version 46 2024-05-14T17:07:46.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.503+0000 7f6933f6b740 1 -- 172.21.15.2:0/629430211 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5622df0916b0 con 0x5622df07ead0 2024-05-14T17:07:46.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.503+0000 7f690eff5640 1 -- 172.21.15.2:0/629430211 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f691c018af0 con 0x5622df07ead0 2024-05-14T17:07:46.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.503+0000 7f690eff5640 1 -- 172.21.15.2:0/629430211 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 46) ==== 8342+0+0 (secure 0 0 0) 0x7f692401b690 con 0x5622df07ead0 2024-05-14T17:07:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.503+0000 7f6933f6b740 10 client.16602 fetch_fsmap finished waiting for FSMap version 46 2024-05-14T17:07:47.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.503+0000 7f6933f6b740 10 client.16602 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:07:47.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.503+0000 7f6933f6b740 20 client.16602 populate_metadata read hostname 'smithi002' 2024-05-14T17:07:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.503+0000 7f6933f6b740 1 --2- 172.21.15.2:0/629430211 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5622df092be0 0x5622df095010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.503+0000 7f6933f6b740 4 client.16602 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:07:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.503+0000 7f6933f6b740 1 -- 172.21.15.2:0/629430211 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622defe8b80 con 0x5622df092be0 2024-05-14T17:07:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.503+0000 7f692ad9d640 1 --2- 172.21.15.2:0/629430211 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5622df092be0 0x5622df095010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:47.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.504+0000 7f692ad9d640 1 --2- 172.21.15.2:0/629430211 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5622df092be0 0x5622df095010 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edc2f06f7b6bfa server_cookie=204fec3d3cfe2513 in_seq=0 out_seq=0 2024-05-14T17:07:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.504+0000 7f690eff5640 10 client.16602 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:07:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.505+0000 7f690eff5640 1 -- 172.21.15.2:0/629430211 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5622defe8b80 con 0x5622df092be0 2024-05-14T17:07:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.505+0000 7f690eff5640 10 client.16602 handle_command_reply: tid=0 2024-05-14T17:07:47.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.505+0000 7f6933f6b740 1 client.16602 shutdown 2024-05-14T17:07:47.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.505+0000 7f6933f6b740 1 -- 172.21.15.2:0/629430211 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622df07ead0 msgr2=0x5622df082ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:47.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.505+0000 7f6933f6b740 1 --2- 172.21.15.2:0/629430211 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622df07ead0 0x5622df082ac0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f69240049d0 tx=0x7f6924004ab0 comp rx=0 tx=0).stop 2024-05-14T17:07:47.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.505+0000 7f6933f6b740 1 -- 172.21.15.2:0/629430211 shutdown_connections 2024-05-14T17:07:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.505+0000 7f6933f6b740 1 --2- 172.21.15.2:0/629430211 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5622df092be0 0x5622df095010 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.505+0000 7f6933f6b740 1 --2- 172.21.15.2:0/629430211 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622df07ead0 0x5622df082ac0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:48.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.505+0000 7f6933f6b740 1 --2- 172.21.15.2:0/629430211 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622df074210 0x5622df088c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.505+0000 7f690eff5640 0 client.16602 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:07:48.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.505+0000 7f6933f6b740 1 --2- 172.21.15.2:0/629430211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622df0738f0 0x5622df088750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.505+0000 7f6933f6b740 1 -- 172.21.15.2:0/629430211 >> 172.21.15.2:0/629430211 conn(0x5622df06fe00 msgr2=0x5622df06f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.505+0000 7f6933f6b740 1 -- 172.21.15.2:0/629430211 shutdown_connections 2024-05-14T17:07:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.505+0000 7f6933f6b740 1 -- 172.21.15.2:0/629430211 wait complete. 2024-05-14T17:07:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.505+0000 7f6933f6b740 20 client.16602 trim_cache size 0 max 16384 2024-05-14T17:07:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.520+0000 7f6933f6b740 1 Processor -- start 2024-05-14T17:07:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.520+0000 7f6933f6b740 1 -- start start 2024-05-14T17:07:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.520+0000 7f6933f6b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622df0738f0 0x5622df088980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:48.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.520+0000 7f6933f6b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622df074210 0x5622df09a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.520+0000 7f6933f6b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622df07ead0 0x5622df09aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:48.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.520+0000 7f6933f6b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f692c1aec80 con 0x5622df0738f0 2024-05-14T17:07:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.520+0000 7f6933f6b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f692c1ac520 con 0x5622df07ead0 2024-05-14T17:07:49.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.520+0000 7f6933f6b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f692c1aa390 con 0x5622df074210 2024-05-14T17:07:49.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.521+0000 7f692ad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622df0738f0 0x5622df088980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.521+0000 7f692ad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622df0738f0 0x5622df088980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58462/0 (socket says 172.21.15.2:58462) 2024-05-14T17:07:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.521+0000 7f692ad9d640 1 -- 172.21.15.2:0/3444383677 learned_addr learned my addr 172.21.15.2:0/3444383677 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:07:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.521+0000 7f692a59c640 1 --2- 172.21.15.2:0/3444383677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622df074210 0x5622df09a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.521+0000 7f692ad9d640 1 -- 172.21.15.2:0/3444383677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622df074210 msgr2=0x5622df09a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.521+0000 7f692ad9d640 1 --2- 172.21.15.2:0/3444383677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622df074210 0x5622df09a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.521+0000 7f692ad9d640 1 -- 172.21.15.2:0/3444383677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622df07ead0 msgr2=0x5622df09aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.521+0000 7f692ad9d640 1 --2- 172.21.15.2:0/3444383677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622df07ead0 0x5622df09aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.521+0000 7f692ad9d640 1 -- 172.21.15.2:0/3444383677 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622df0709d0 con 0x5622df0738f0 2024-05-14T17:07:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.521+0000 7f692a59c640 1 --2- 172.21.15.2:0/3444383677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622df074210 0x5622df09a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:07:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.521+0000 7f692ad9d640 1 --2- 172.21.15.2:0/3444383677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622df0738f0 0x5622df088980 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f691c00ad00 tx=0x7f691c010780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b485c784a3c74029 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.521+0000 7f6910ff9640 1 -- 172.21.15.2:0/3444383677 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c01d040 con 0x5622df0738f0 2024-05-14T17:07:50.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.521+0000 7f6910ff9640 1 -- 172.21.15.2:0/3444383677 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f691c02e020 con 0x5622df0738f0 2024-05-14T17:07:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.521+0000 7f6910ff9640 1 -- 172.21.15.2:0/3444383677 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c010c10 con 0x5622df0738f0 2024-05-14T17:07:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.522+0000 7f6933f6b740 1 -- 172.21.15.2:0/3444383677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622df0738f0 msgr2=0x5622df088980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:50.180 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.522+0000 7f6933f6b740 1 --2- 172.21.15.2:0/3444383677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622df0738f0 0x5622df088980 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f691c00ad00 tx=0x7f691c010780 comp rx=0 tx=0).stop 2024-05-14T17:07:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.522+0000 7f6933f6b740 1 -- 172.21.15.2:0/3444383677 shutdown_connections 2024-05-14T17:07:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.522+0000 7f6933f6b740 1 --2- 172.21.15.2:0/3444383677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622df07ead0 0x5622df09aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:50.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.522+0000 7f6933f6b740 1 --2- 172.21.15.2:0/3444383677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622df074210 0x5622df09a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:50.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.522+0000 7f6933f6b740 1 --2- 172.21.15.2:0/3444383677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622df0738f0 0x5622df088980 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:50.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.522+0000 7f6933f6b740 1 -- 172.21.15.2:0/3444383677 >> 172.21.15.2:0/3444383677 conn(0x5622df06fe00 msgr2=0x5622df07b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.522+0000 7f6933f6b740 1 -- 172.21.15.2:0/3444383677 shutdown_connections 2024-05-14T17:07:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.522+0000 7f6933f6b740 1 -- 172.21.15.2:0/3444383677 wait complete. 2024-05-14T17:07:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.522+0000 7f6933f6b740 1 -- start start 2024-05-14T17:07:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.522+0000 7f6933f6b740 1 Processor -- start 2024-05-14T17:07:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.522+0000 7f6933f6b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622df0738f0 0x5622df08f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:50.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.523+0000 7f6933f6b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622df074210 0x5622df08c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.523+0000 7f6933f6b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622df07ead0 0x5622df08c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.523+0000 7f6933f6b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f692c1aa9e0 con 0x5622df07ead0 2024-05-14T17:07:51.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.523+0000 7f6933f6b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f692c1abe50 con 0x5622df0738f0 2024-05-14T17:07:51.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.523+0000 7f6933f6b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f692c1ab0b0 con 0x5622df074210 2024-05-14T17:07:51.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.523+0000 7f6933f6b740 10 client.0 _ll_register_callbacks cb 0x7f692400f750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:07:51.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.523+0000 7f692b59e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622df07ead0 0x5622df08c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.523+0000 7f692b59e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622df07ead0 0x5622df08c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58468/0 (socket says 172.21.15.2:58468) 2024-05-14T17:07:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.523+0000 7f692b59e640 1 -- 172.21.15.2:0/3771158006 learned_addr learned my addr 172.21.15.2:0/3771158006 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:07:51.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.523+0000 7f692a59c640 1 --2- 172.21.15.2:0/3771158006 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622df074210 0x5622df08c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.523+0000 7f692ad9d640 1 --2- 172.21.15.2:0/3771158006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622df0738f0 0x5622df08f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:51.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.523+0000 7f692b59e640 1 -- 172.21.15.2:0/3771158006 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622df074210 msgr2=0x5622df08c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.523+0000 7f692b59e640 1 --2- 172.21.15.2:0/3771158006 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622df074210 0x5622df08c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:51.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.523+0000 7f692b59e640 1 -- 172.21.15.2:0/3771158006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622df0738f0 msgr2=0x5622df08f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:07:51.943 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.523+0000 7f692b59e640 1 --2- 172.21.15.2:0/3771158006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622df0738f0 0x5622df08f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.523+0000 7f692b59e640 1 -- 172.21.15.2:0/3771158006 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622df08cfb0 con 0x5622df07ead0 2024-05-14T17:07:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.523+0000 7f692b59e640 1 --2- 172.21.15.2:0/3771158006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622df07ead0 0x5622df08c970 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f692400e3d0 tx=0x7f69240071c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=627baa98400cc533 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:07:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.523+0000 7f690eff5640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:07:52.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.523+0000 7f690eff5640 1 -- 172.21.15.2:0/3771158006 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6924023020 con 0x5622df07ead0 2024-05-14T17:07:52.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.523+0000 7f690eff5640 1 -- 172.21.15.2:0/3771158006 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f692401b5d0 con 0x5622df07ead0 2024-05-14T17:07:52.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.523+0000 7f6933f6b740 1 -- 172.21.15.2:0/3771158006 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f691c02e020 con 0x5622df07ead0 2024-05-14T17:07:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.523+0000 7f690eff5640 1 -- 172.21.15.2:0/3771158006 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f6924040050 con 0x5622df07ead0 2024-05-14T17:07:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.523+0000 7f6933f6b740 10 client.16608 fetch_fsmap learned FSMap version 46 2024-05-14T17:07:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.524+0000 7f6933f6b740 1 -- 172.21.15.2:0/3771158006 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5622df0fd300 con 0x5622df07ead0 2024-05-14T17:07:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.524+0000 7f690eff5640 1 -- 172.21.15.2:0/3771158006 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f691c02e020 con 0x5622df07ead0 2024-05-14T17:07:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.524+0000 7f690eff5640 1 -- 172.21.15.2:0/3771158006 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 46) ==== 8342+0+0 (secure 0 0 0) 0x7f692403b030 con 0x5622df07ead0 2024-05-14T17:07:52.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.524+0000 7f6933f6b740 10 client.16608 fetch_fsmap finished waiting for FSMap version 46 2024-05-14T17:07:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.524+0000 7f6933f6b740 10 client.16608 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:07:53.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.524+0000 7f6933f6b740 20 client.16608 populate_metadata read hostname 'smithi002' 2024-05-14T17:07:53.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.524+0000 7f6933f6b740 1 --2- 172.21.15.2:0/3771158006 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5622df0fda10 0x5622df0ffe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.524+0000 7f6933f6b740 4 client.16608 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:07:53.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.524+0000 7f6933f6b740 1 -- 172.21.15.2:0/3771158006 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5622defe2360 con 0x5622df0fda10 2024-05-14T17:07:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.524+0000 7f692ad9d640 1 --2- 172.21.15.2:0/3771158006 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5622df0fda10 0x5622df0ffe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.524+0000 7f692ad9d640 1 --2- 172.21.15.2:0/3771158006 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5622df0fda10 0x5622df0ffe30 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c270b95ee37448c0 server_cookie=cf35379c20da4c20 in_seq=0 out_seq=0 2024-05-14T17:07:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.524+0000 7f690eff5640 10 client.16608 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:07:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.530+0000 7f690eff5640 1 -- 172.21.15.2:0/3771158006 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5622defe2360 con 0x5622df0fda10 2024-05-14T17:07:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.530+0000 7f690eff5640 10 client.16608 handle_command_reply: tid=0 2024-05-14T17:07:53.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.530+0000 7f6933f6b740 1 client.16608 shutdown 2024-05-14T17:07:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.530+0000 7f6933f6b740 1 -- 172.21.15.2:0/3771158006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622df07ead0 msgr2=0x5622df08c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:54.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.530+0000 7f6933f6b740 1 --2- 172.21.15.2:0/3771158006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622df07ead0 0x5622df08c970 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f692400e3d0 tx=0x7f69240071c0 comp rx=0 tx=0).stop 2024-05-14T17:07:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.530+0000 7f6933f6b740 1 -- 172.21.15.2:0/3771158006 shutdown_connections 2024-05-14T17:07:54.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.530+0000 7f6933f6b740 1 --2- 172.21.15.2:0/3771158006 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5622df0fda10 0x5622df0ffe30 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:54.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.530+0000 7f6933f6b740 1 --2- 172.21.15.2:0/3771158006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5622df07ead0 0x5622df08c970 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.530+0000 7f690eff5640 0 client.16608 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:07:54.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.530+0000 7f6933f6b740 1 --2- 172.21.15.2:0/3771158006 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5622df074210 0x5622df08c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.530+0000 7f6933f6b740 1 --2- 172.21.15.2:0/3771158006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622df0738f0 0x5622df08f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.530+0000 7f6933f6b740 1 -- 172.21.15.2:0/3771158006 >> 172.21.15.2:0/3771158006 conn(0x5622df06fe00 msgr2=0x5622df07b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:54.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.530+0000 7f6933f6b740 1 -- 172.21.15.2:0/3771158006 shutdown_connections 2024-05-14T17:07:54.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.530+0000 7f6933f6b740 1 -- 172.21.15.2:0/3771158006 wait complete. 2024-05-14T17:07:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.530+0000 7f6933f6b740 20 client.16608 trim_cache size 0 max 16384 2024-05-14T17:07:54.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:07:54.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:07:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:07:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.530+0000 7f6931254640 1 -- 172.21.15.2:0/46303632 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f69140b2a10 msgr2=0x7f69140b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:55.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.530+0000 7f6931254640 1 --2- 172.21.15.2:0/46303632 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f69140b2a10 0x7f69140b4ed0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6924007fc0 tx=0x7f6924004a40 comp rx=0 tx=0).stop 2024-05-14T17:07:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.530+0000 7f6931254640 1 -- 172.21.15.2:0/46303632 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f692c104600 msgr2=0x7f692c1996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.530+0000 7f6931254640 1 --2- 172.21.15.2:0/46303632 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f692c104600 0x7f692c1996b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f6920006010 tx=0x7f69200042f0 comp rx=0 tx=0).stop 2024-05-14T17:07:55.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.531+0000 7f6931254640 1 -- 172.21.15.2:0/46303632 shutdown_connections 2024-05-14T17:07:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.531+0000 7f6931254640 1 --2- 172.21.15.2:0/46303632 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f69140b2a10 0x7f69140b4ed0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.531+0000 7f6931254640 1 --2- 172.21.15.2:0/46303632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f692c10d020 0x7f692c19da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:55.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.531+0000 7f6931254640 1 --2- 172.21.15.2:0/46303632 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f692c104600 0x7f692c1996b0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:55.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.531+0000 7f6931254640 1 --2- 172.21.15.2:0/46303632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f692c103ce0 0x7f692c199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.531+0000 7f6931254640 1 -- 172.21.15.2:0/46303632 >> 172.21.15.2:0/46303632 conn(0x7f692c078ad0 msgr2=0x7f692c0ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.531+0000 7f6931254640 1 -- 172.21.15.2:0/46303632 shutdown_connections 2024-05-14T17:07:56.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:06:49.531+0000 7f6931254640 1 -- 172.21.15.2:0/46303632 wait complete. 2024-05-14T17:07:56.257 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:06:50 smithi002 ceph-mon[31538]: pgmap v377: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-05-14T17:07:56.360 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 695.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:07:56.429 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:06:52 smithi002 ceph-mon[31538]: pgmap v378: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T17:07:56.501 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:06:54 smithi002 ceph-mon[31538]: pgmap v379: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-05-14T17:07:56.533 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 700.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:07:56.566 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:07:40 smithi123 ceph-mon[35971]: pgmap v404: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 1.2 MiB/s wr, 94 op/s 2024-05-14T17:07:56.638 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:07:42 smithi123 ceph-mon[35971]: pgmap v405: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 1.2 MiB/s wr, 92 op/s 2024-05-14T17:07:56.696 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:07:44 smithi123 ceph-mon[35971]: pgmap v406: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 2.9 MiB/s wr, 267 op/s 2024-05-14T17:07:56.764 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:07:46 smithi123 ceph-mon[35971]: pgmap v407: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 3.8 MiB/s wr, 353 op/s 2024-05-14T17:07:56.853 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:07:48 smithi123 ceph-mon[35971]: pgmap v408: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 3.8 MiB/s wr, 353 op/s 2024-05-14T17:07:56.928 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:07:50 smithi123 ceph-mon[35971]: pgmap v409: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 6.5 MiB/s wr, 612 op/s 2024-05-14T17:07:56.985 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:07:52 smithi123 ceph-mon[35971]: pgmap v410: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 5.4 MiB/s wr, 523 op/s 2024-05-14T17:07:57.092 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:07:53 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:07:57.144 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:07:54 smithi123 ceph-mon[35971]: pgmap v411: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 7.0 MiB/s wr, 694 op/s 2024-05-14T17:07:57.192 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:07:40 smithi151 ceph-mon[36066]: pgmap v404: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 1.2 MiB/s wr, 94 op/s 2024-05-14T17:07:57.253 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:07:42 smithi151 ceph-mon[36066]: pgmap v405: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 1.2 MiB/s wr, 92 op/s 2024-05-14T17:07:57.316 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:07:44 smithi151 ceph-mon[36066]: pgmap v406: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 2.9 MiB/s wr, 267 op/s 2024-05-14T17:07:57.376 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:07:46 smithi151 ceph-mon[36066]: pgmap v407: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 3.8 MiB/s wr, 353 op/s 2024-05-14T17:07:57.453 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:07:48 smithi151 ceph-mon[36066]: pgmap v408: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 3.8 MiB/s wr, 353 op/s 2024-05-14T17:07:57.526 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:07:50 smithi151 ceph-mon[36066]: pgmap v409: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 6.5 MiB/s wr, 612 op/s 2024-05-14T17:07:57.565 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:07:52 smithi151 ceph-mon[36066]: pgmap v410: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 5.4 MiB/s wr, 523 op/s 2024-05-14T17:07:57.603 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:07:53 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:07:57.702 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:07:54 smithi151 ceph-mon[36066]: pgmap v411: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 7.0 MiB/s wr, 694 op/s 2024-05-14T17:07:57.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:06:56 smithi002 ceph-mon[31538]: pgmap v380: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 5 op/s 2024-05-14T17:07:57.843 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:06:58 smithi002 ceph-mon[31538]: pgmap v381: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 5 op/s 2024-05-14T17:07:57.897 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:07:00 smithi002 ceph-mon[31538]: pgmap v382: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-05-14T17:07:57.933 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 705.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:07:57.971 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:07:56 smithi123 ceph-mon[35971]: pgmap v412: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 6.8 MiB/s wr, 644 op/s 2024-05-14T17:07:57.999 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:07:56 smithi151 ceph-mon[36066]: pgmap v412: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 6.8 MiB/s wr, 644 op/s 2024-05-14T17:07:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.634+0000 7f52b7311640 1 -- 172.21.15.2:0/2916297382 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52b0102a50 msgr2=0x7f52b0102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:58.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.634+0000 7f52b7311640 1 --2- 172.21.15.2:0/2916297382 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52b0102a50 0x7f52b0102ed0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f52a0009930 tx=0x7f52a002a420 comp rx=0 tx=0).stop 2024-05-14T17:07:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.634+0000 7f52b7311640 1 -- 172.21.15.2:0/2916297382 shutdown_connections 2024-05-14T17:07:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.634+0000 7f52b7311640 1 --2- 172.21.15.2:0/2916297382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52b0103410 0x7f52b010f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.634+0000 7f52b7311640 1 --2- 172.21.15.2:0/2916297382 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52b0102a50 0x7f52b0102ed0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.634+0000 7f52b7311640 1 --2- 172.21.15.2:0/2916297382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b0108a60 0x7f52b0108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:58.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.634+0000 7f52b7311640 1 -- 172.21.15.2:0/2916297382 >> 172.21.15.2:0/2916297382 conn(0x7f52b00fe520 msgr2=0x7f52b0100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:07:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.634+0000 7f52b7311640 1 -- 172.21.15.2:0/2916297382 shutdown_connections 2024-05-14T17:07:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.634+0000 7f52b7311640 1 -- 172.21.15.2:0/2916297382 wait complete. 2024-05-14T17:07:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.635+0000 7f52b7311640 1 Processor -- start 2024-05-14T17:07:58.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.635+0000 7f52b7311640 1 -- start start 2024-05-14T17:07:58.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.635+0000 7f52b7311640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52b0102a50 0x7f52b0199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.635+0000 7f52b7311640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b0103410 0x7f52b01996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.635+0000 7f52b7311640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52b0108a60 0x7f52b019da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:07:58.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.635+0000 7f52b7311640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f52b0112d20 con 0x7f52b0102a50 2024-05-14T17:07:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.635+0000 7f52b7311640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f52b0112ba0 con 0x7f52b0103410 2024-05-14T17:07:59.033 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.635+0000 7f52b7311640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f52b0112ea0 con 0x7f52b0108a60 2024-05-14T17:07:59.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.635+0000 7f52b50ad640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52b0102a50 0x7f52b0199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:59.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.635+0000 7f52b50ad640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52b0102a50 0x7f52b0199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43116/0 (socket says 172.21.15.2:43116) 2024-05-14T17:07:59.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.635+0000 7f52b50ad640 1 -- 172.21.15.2:0/3598808786 learned_addr learned my addr 172.21.15.2:0/3598808786 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:07:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.635+0000 7f52b48ac640 1 --2- 172.21.15.2:0/3598808786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b0103410 0x7f52b01996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:59.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.635+0000 7f52b58ae640 1 --2- 172.21.15.2:0/3598808786 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52b0108a60 0x7f52b019da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:07:59.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.635+0000 7f52b50ad640 1 -- 172.21.15.2:0/3598808786 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52b0108a60 msgr2=0x7f52b019da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:59.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.635+0000 7f52b50ad640 1 --2- 172.21.15.2:0/3598808786 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52b0108a60 0x7f52b019da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.635+0000 7f52b50ad640 1 -- 172.21.15.2:0/3598808786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b0103410 msgr2=0x7f52b01996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:07:59.727 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.635+0000 7f52b50ad640 1 --2- 172.21.15.2:0/3598808786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b0103410 0x7f52b01996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:07:59.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.635+0000 7f52b50ad640 1 -- 172.21.15.2:0/3598808786 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b019e150 con 0x7f52b0102a50 2024-05-14T17:07:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.635+0000 7f52b48ac640 1 --2- 172.21.15.2:0/3598808786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b0103410 0x7f52b01996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:07:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.636+0000 7f52b50ad640 1 --2- 172.21.15.2:0/3598808786 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52b0102a50 0x7f52b0199160 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00d950 tx=0x7f52ac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:00.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.636+0000 7f52a67fc640 1 -- 172.21.15.2:0/3598808786 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac017030 con 0x7f52b0102a50 2024-05-14T17:08:00.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.636+0000 7f52a67fc640 1 -- 172.21.15.2:0/3598808786 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52ac004020 con 0x7f52b0102a50 2024-05-14T17:08:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.636+0000 7f52b7311640 1 -- 172.21.15.2:0/3598808786 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b019e440 con 0x7f52b0102a50 2024-05-14T17:08:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.636+0000 7f52a67fc640 1 -- 172.21.15.2:0/3598808786 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac0047a0 con 0x7f52b0102a50 2024-05-14T17:08:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.636+0000 7f52b7311640 1 -- 172.21.15.2:0/3598808786 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b01a5cd0 con 0x7f52b0102a50 2024-05-14T17:08:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.639+0000 7f52a67fc640 1 -- 172.21.15.2:0/3598808786 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f52ac009a30 con 0x7f52b0102a50 2024-05-14T17:08:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.639+0000 7f52ba022740 1 Processor -- start 2024-05-14T17:08:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.639+0000 7f52ba022740 1 -- start start 2024-05-14T17:08:00.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.639+0000 7f52ba022740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562fc4408f0 0x5562fc440cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:00.693 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.639+0000 7f52ba022740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562fc441210 0x5562fc44b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.639+0000 7f52ba022740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562fc44bad0 0x5562fc44dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.639+0000 7f52ba022740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5562fc3b3c30 con 0x5562fc441210 2024-05-14T17:08:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.639+0000 7f52ba022740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5562fc3a37d0 con 0x5562fc4408f0 2024-05-14T17:08:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.639+0000 7f52ba022740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5562fc3a7810 con 0x5562fc44bad0 2024-05-14T17:08:01.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.639+0000 7f52b50ad640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562fc441210 0x5562fc44b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.639+0000 7f52b50ad640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562fc441210 0x5562fc44b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43124/0 (socket says 172.21.15.2:43124) 2024-05-14T17:08:01.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.639+0000 7f52b50ad640 1 -- 172.21.15.2:0/75626473 learned_addr learned my addr 172.21.15.2:0/75626473 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:08:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.640+0000 7f52b58ae640 1 --2- 172.21.15.2:0/75626473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562fc4408f0 0x5562fc440cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:01.186 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.640+0000 7f52b48ac640 1 --2- 172.21.15.2:0/75626473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562fc44bad0 0x5562fc44dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.640+0000 7f52b50ad640 1 -- 172.21.15.2:0/75626473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562fc44bad0 msgr2=0x5562fc44dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:01.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.640+0000 7f52b50ad640 1 --2- 172.21.15.2:0/75626473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562fc44bad0 0x5562fc44dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.640+0000 7f52b50ad640 1 -- 172.21.15.2:0/75626473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562fc4408f0 msgr2=0x5562fc440cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.640+0000 7f52b50ad640 1 --2- 172.21.15.2:0/75626473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562fc4408f0 0x5562fc440cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:01.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.640+0000 7f52a67fc640 1 --2- 172.21.15.2:0/3598808786 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f529c0b2a10 0x7f529c0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.640+0000 7f52b50ad640 1 -- 172.21.15.2:0/75626473 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562fc0b8210 con 0x5562fc441210 2024-05-14T17:08:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.640+0000 7f52b48ac640 1 --2- 172.21.15.2:0/3598808786 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f529c0b2a10 0x7f529c0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.640+0000 7f52a67fc640 1 -- 172.21.15.2:0/3598808786 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f52ac0e2090 con 0x7f52b0102a50 2024-05-14T17:08:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.640+0000 7f52b50ad640 1 --2- 172.21.15.2:0/75626473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562fc441210 0x5562fc44b590 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f52ac0df750 tx=0x7f52ac0a5a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf6313e64631b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.640+0000 7f529b7fe640 1 -- 172.21.15.2:0/75626473 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac0a9140 con 0x5562fc441210 2024-05-14T17:08:01.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.640+0000 7f529b7fe640 1 -- 172.21.15.2:0/75626473 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52ac0056a0 con 0x5562fc441210 2024-05-14T17:08:02.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.640+0000 7f529b7fe640 1 -- 172.21.15.2:0/75626473 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac0a8ae0 con 0x5562fc441210 2024-05-14T17:08:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.641+0000 7f52ba022740 1 -- 172.21.15.2:0/75626473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562fc441210 msgr2=0x5562fc44b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.641+0000 7f52ba022740 1 --2- 172.21.15.2:0/75626473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562fc441210 0x5562fc44b590 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f52ac0df750 tx=0x7f52ac0a5a80 comp rx=0 tx=0).stop 2024-05-14T17:08:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.641+0000 7f52b48ac640 1 --2- 172.21.15.2:0/3598808786 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f529c0b2a10 0x7f529c0b4ed0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f52a0000c00 tx=0x7f52a00058b0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.641+0000 7f52ba022740 1 -- 172.21.15.2:0/75626473 shutdown_connections 2024-05-14T17:08:02.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.641+0000 7f52ba022740 1 --2- 172.21.15.2:0/75626473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562fc44bad0 0x5562fc44dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:02.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.641+0000 7f52ba022740 1 --2- 172.21.15.2:0/75626473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562fc441210 0x5562fc44b590 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.641+0000 7f52ba022740 1 --2- 172.21.15.2:0/75626473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562fc4408f0 0x5562fc440cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:02.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.641+0000 7f52ba022740 1 -- 172.21.15.2:0/75626473 >> 172.21.15.2:0/75626473 conn(0x5562fc43ce00 msgr2=0x5562fc43d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.641+0000 7f52ba022740 1 -- 172.21.15.2:0/75626473 shutdown_connections 2024-05-14T17:08:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.641+0000 7f52ba022740 1 -- 172.21.15.2:0/75626473 wait complete. 2024-05-14T17:08:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.642+0000 7f52ba022740 1 -- start start 2024-05-14T17:08:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.642+0000 7f52ba022740 1 Processor -- start 2024-05-14T17:08:03.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.642+0000 7f52ba022740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562fc4408f0 0x5562fc44b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.642+0000 7f52ba022740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562fc441210 0x5562fc4487c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:03.236 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.642+0000 7f52ba022740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562fc44bad0 0x5562fc448dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:03.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.642+0000 7f52ba022740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5562fc089790 con 0x5562fc44bad0 2024-05-14T17:08:03.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.642+0000 7f52ba022740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5562fc35f890 con 0x5562fc4408f0 2024-05-14T17:08:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.642+0000 7f52ba022740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5562fc44e7a0 con 0x5562fc441210 2024-05-14T17:08:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.642+0000 7f52ba022740 10 client.0 _ll_register_callbacks cb 0x5562fc362d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:08:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.642+0000 7f52b58ae640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562fc44bad0 0x5562fc448dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:03.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.642+0000 7f52b58ae640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562fc44bad0 0x5562fc448dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43132/0 (socket says 172.21.15.2:43132) 2024-05-14T17:08:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.642+0000 7f52b58ae640 1 -- 172.21.15.2:0/4121374324 learned_addr learned my addr 172.21.15.2:0/4121374324 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:08:03.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.643+0000 7f52b50ad640 1 --2- 172.21.15.2:0/4121374324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562fc4408f0 0x5562fc44b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.643+0000 7f52b48ac640 1 --2- 172.21.15.2:0/4121374324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562fc441210 0x5562fc4487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:03.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.643+0000 7f52b58ae640 1 -- 172.21.15.2:0/4121374324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562fc441210 msgr2=0x5562fc4487c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.643+0000 7f52b58ae640 1 --2- 172.21.15.2:0/4121374324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562fc441210 0x5562fc4487c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:03.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.643+0000 7f52b58ae640 1 -- 172.21.15.2:0/4121374324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562fc4408f0 msgr2=0x5562fc44b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:03.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.643+0000 7f52b58ae640 1 --2- 172.21.15.2:0/4121374324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562fc4408f0 0x5562fc44b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.643+0000 7f52b58ae640 1 -- 172.21.15.2:0/4121374324 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562fc184620 con 0x5562fc44bad0 2024-05-14T17:08:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.643+0000 7f52b58ae640 1 --2- 172.21.15.2:0/4121374324 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562fc44bad0 0x5562fc448dd0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f52a8010ad0 tx=0x7f52a8010f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a72d5d908b616 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.643+0000 7f5298ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:08:04.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.643+0000 7f5298ff9640 1 -- 172.21.15.2:0/4121374324 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a801d060 con 0x5562fc44bad0 2024-05-14T17:08:04.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.643+0000 7f5298ff9640 1 -- 172.21.15.2:0/4121374324 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f52a800ed80 con 0x5562fc44bad0 2024-05-14T17:08:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.643+0000 7f52ba022740 1 -- 172.21.15.2:0/4121374324 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52ac0056a0 con 0x5562fc44bad0 2024-05-14T17:08:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.643+0000 7f5298ff9640 1 -- 172.21.15.2:0/4121374324 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f52a8021410 con 0x5562fc44bad0 2024-05-14T17:08:04.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.643+0000 7f52ba022740 10 client.16620 fetch_fsmap learned FSMap version 46 2024-05-14T17:08:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.643+0000 7f52ba022740 1 -- 172.21.15.2:0/4121374324 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5562fc45a2a0 con 0x5562fc44bad0 2024-05-14T17:08:04.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.643+0000 7f5298ff9640 1 -- 172.21.15.2:0/4121374324 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52ac0056a0 con 0x5562fc44bad0 2024-05-14T17:08:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.644+0000 7f5298ff9640 1 -- 172.21.15.2:0/4121374324 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 46) ==== 8342+0+0 (secure 0 0 0) 0x7f52a8020440 con 0x5562fc44bad0 2024-05-14T17:08:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.644+0000 7f52ba022740 10 client.16620 fetch_fsmap finished waiting for FSMap version 46 2024-05-14T17:08:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.644+0000 7f52ba022740 10 client.16620 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:08:04.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.644+0000 7f52ba022740 20 client.16620 populate_metadata read hostname 'smithi002' 2024-05-14T17:08:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.644+0000 7f52ba022740 1 --2- 172.21.15.2:0/4121374324 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5562fc45b7b0 0x5562fc45dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:04.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.644+0000 7f52ba022740 4 client.16620 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:08:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.644+0000 7f52ba022740 1 -- 172.21.15.2:0/4121374324 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562fc3b5b80 con 0x5562fc45b7b0 2024-05-14T17:08:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.644+0000 7f52b50ad640 1 --2- 172.21.15.2:0/4121374324 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5562fc45b7b0 0x5562fc45dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.645+0000 7f52b50ad640 1 --2- 172.21.15.2:0/4121374324 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5562fc45b7b0 0x5562fc45dba0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da929377afd87961 server_cookie=262493163dd770c8 in_seq=0 out_seq=0 2024-05-14T17:08:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.645+0000 7f5298ff9640 10 client.16620 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:08:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.645+0000 7f5298ff9640 1 -- 172.21.15.2:0/4121374324 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5562fc3b5b80 con 0x5562fc45b7b0 2024-05-14T17:08:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.645+0000 7f5298ff9640 10 client.16620 handle_command_reply: tid=0 2024-05-14T17:08:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.645+0000 7f52ba022740 1 client.16620 shutdown 2024-05-14T17:08:05.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.646+0000 7f52ba022740 1 -- 172.21.15.2:0/4121374324 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562fc44bad0 msgr2=0x5562fc448dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.646+0000 7f52ba022740 1 --2- 172.21.15.2:0/4121374324 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562fc44bad0 0x5562fc448dd0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f52a8010ad0 tx=0x7f52a8010f90 comp rx=0 tx=0).stop 2024-05-14T17:08:05.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.646+0000 7f52ba022740 1 -- 172.21.15.2:0/4121374324 shutdown_connections 2024-05-14T17:08:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.646+0000 7f52ba022740 1 --2- 172.21.15.2:0/4121374324 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5562fc45b7b0 0x5562fc45dba0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:05.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.646+0000 7f52ba022740 1 --2- 172.21.15.2:0/4121374324 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562fc44bad0 0x5562fc448dd0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.646+0000 7f52ba022740 1 --2- 172.21.15.2:0/4121374324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562fc441210 0x5562fc4487c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:05.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.646+0000 7f5298ff9640 0 client.16620 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:08:05.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.646+0000 7f52ba022740 1 --2- 172.21.15.2:0/4121374324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562fc4408f0 0x5562fc44b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.646+0000 7f52ba022740 1 -- 172.21.15.2:0/4121374324 >> 172.21.15.2:0/4121374324 conn(0x5562fc43ce00 msgr2=0x5562fc43c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.646+0000 7f52ba022740 1 -- 172.21.15.2:0/4121374324 shutdown_connections 2024-05-14T17:08:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.646+0000 7f52ba022740 1 -- 172.21.15.2:0/4121374324 wait complete. 2024-05-14T17:08:06.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.646+0000 7f52ba022740 20 client.16620 trim_cache size 0 max 16384 2024-05-14T17:08:06.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.661+0000 7f52ba022740 1 Processor -- start 2024-05-14T17:08:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.661+0000 7f52ba022740 1 -- start start 2024-05-14T17:08:06.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.661+0000 7f52ba022740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562fc4408f0 0x5562fc457670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.661+0000 7f52ba022740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562fc441210 0x5562fc442a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.661+0000 7f52ba022740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562fc44bad0 0x5562fc44b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.661+0000 7f52ba022740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01b0ee0 con 0x5562fc44bad0 2024-05-14T17:08:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.661+0000 7f52ba022740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01ae780 con 0x5562fc441210 2024-05-14T17:08:06.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.661+0000 7f52ba022740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01ac5f0 con 0x5562fc4408f0 2024-05-14T17:08:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.661+0000 7f52b58ae640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562fc44bad0 0x5562fc44b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:06.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.661+0000 7f52b58ae640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562fc44bad0 0x5562fc44b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43140/0 (socket says 172.21.15.2:43140) 2024-05-14T17:08:06.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.661+0000 7f52b58ae640 1 -- 172.21.15.2:0/3600875963 learned_addr learned my addr 172.21.15.2:0/3600875963 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:08:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.661+0000 7f52b50ad640 1 --2- 172.21.15.2:0/3600875963 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562fc4408f0 0x5562fc457670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:07.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.661+0000 7f52b58ae640 1 -- 172.21.15.2:0/3600875963 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562fc4408f0 msgr2=0x5562fc457670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.661+0000 7f52b58ae640 1 --2- 172.21.15.2:0/3600875963 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562fc4408f0 0x5562fc457670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.661+0000 7f52b58ae640 1 -- 172.21.15.2:0/3600875963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562fc441210 msgr2=0x5562fc442a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.661+0000 7f52b48ac640 1 --2- 172.21.15.2:0/3600875963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562fc441210 0x5562fc442a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.661+0000 7f52b58ae640 1 --2- 172.21.15.2:0/3600875963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562fc441210 0x5562fc442a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.661+0000 7f52b58ae640 1 -- 172.21.15.2:0/3600875963 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562fc45df10 con 0x5562fc44bad0 2024-05-14T17:08:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.661+0000 7f52b48ac640 1 --2- 172.21.15.2:0/3600875963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562fc441210 0x5562fc442a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:08:07.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.661+0000 7f52b58ae640 1 --2- 172.21.15.2:0/3600875963 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562fc44bad0 0x5562fc44b000 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f52a8002230 tx=0x7f52a8005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=307159fe81d1f40e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.661+0000 7f529affd640 1 -- 172.21.15.2:0/3600875963 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a800ed80 con 0x5562fc44bad0 2024-05-14T17:08:07.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.661+0000 7f529affd640 1 -- 172.21.15.2:0/3600875963 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52a8020ad0 con 0x5562fc44bad0 2024-05-14T17:08:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.661+0000 7f529affd640 1 -- 172.21.15.2:0/3600875963 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8021810 con 0x5562fc44bad0 2024-05-14T17:08:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.662+0000 7f52ba022740 1 -- 172.21.15.2:0/3600875963 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562fc44bad0 msgr2=0x5562fc44b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:07.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.662+0000 7f52ba022740 1 --2- 172.21.15.2:0/3600875963 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562fc44bad0 0x5562fc44b000 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f52a8002230 tx=0x7f52a8005f10 comp rx=0 tx=0).stop 2024-05-14T17:08:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.662+0000 7f52ba022740 1 -- 172.21.15.2:0/3600875963 shutdown_connections 2024-05-14T17:08:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.662+0000 7f52ba022740 1 --2- 172.21.15.2:0/3600875963 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562fc44bad0 0x5562fc44b000 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.662+0000 7f52ba022740 1 --2- 172.21.15.2:0/3600875963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562fc441210 0x5562fc442a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:08.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.662+0000 7f52ba022740 1 --2- 172.21.15.2:0/3600875963 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562fc4408f0 0x5562fc457670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:08.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.662+0000 7f52ba022740 1 -- 172.21.15.2:0/3600875963 >> 172.21.15.2:0/3600875963 conn(0x5562fc43ce00 msgr2=0x5562fc4422f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.662+0000 7f52ba022740 1 -- 172.21.15.2:0/3600875963 shutdown_connections 2024-05-14T17:08:08.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.662+0000 7f52ba022740 1 -- 172.21.15.2:0/3600875963 wait complete. 2024-05-14T17:08:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.662+0000 7f52ba022740 1 -- start start 2024-05-14T17:08:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.663+0000 7f52ba022740 1 Processor -- start 2024-05-14T17:08:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.663+0000 7f52ba022740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562fc4408f0 0x5562fc453e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:08.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.663+0000 7f52ba022740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562fc441210 0x5562fc44f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:08.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.663+0000 7f52ba022740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562fc44bad0 0x5562fc44f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.663+0000 7f52ba022740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01acc40 con 0x5562fc4408f0 2024-05-14T17:08:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.663+0000 7f52ba022740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01ae0b0 con 0x5562fc44bad0 2024-05-14T17:08:08.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.663+0000 7f52ba022740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01ad310 con 0x5562fc441210 2024-05-14T17:08:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.663+0000 7f52ba022740 10 client.0 _ll_register_callbacks cb 0x7f52a8010660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:08:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.663+0000 7f52b50ad640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562fc4408f0 0x5562fc453e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:09.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.663+0000 7f52b50ad640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562fc4408f0 0x5562fc453e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:43150/0 (socket says 172.21.15.2:43150) 2024-05-14T17:08:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.663+0000 7f52b50ad640 1 -- 172.21.15.2:0/3603880488 learned_addr learned my addr 172.21.15.2:0/3603880488 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:08:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.663+0000 7f52b58ae640 1 --2- 172.21.15.2:0/3603880488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562fc44bad0 0x5562fc44f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.663+0000 7f52b48ac640 1 --2- 172.21.15.2:0/3603880488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562fc441210 0x5562fc44f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:09.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.663+0000 7f52b50ad640 1 -- 172.21.15.2:0/3603880488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562fc441210 msgr2=0x5562fc44f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.663+0000 7f52b50ad640 1 --2- 172.21.15.2:0/3603880488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562fc441210 0x5562fc44f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:09.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.663+0000 7f52b50ad640 1 -- 172.21.15.2:0/3603880488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562fc44bad0 msgr2=0x5562fc44f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.663+0000 7f52b50ad640 1 --2- 172.21.15.2:0/3603880488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562fc44bad0 0x5562fc44f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.663+0000 7f52b50ad640 1 -- 172.21.15.2:0/3603880488 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562fc44fe60 con 0x5562fc4408f0 2024-05-14T17:08:09.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.663+0000 7f52b58ae640 1 --2- 172.21.15.2:0/3603880488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562fc44bad0 0x5562fc44f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:08:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.664+0000 7f52b50ad640 1 --2- 172.21.15.2:0/3603880488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562fc4408f0 0x5562fc453e70 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f52ac0a9070 tx=0x7f52ac0044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e5705a92d48696 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.664+0000 7f5298ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:08:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.664+0000 7f5298ff9640 1 -- 172.21.15.2:0/3603880488 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac0a8cc0 con 0x5562fc4408f0 2024-05-14T17:08:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.664+0000 7f5298ff9640 1 -- 172.21.15.2:0/3603880488 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8513+0+0 (secure 0 0 0) 0x7f52ac0041d0 con 0x5562fc4408f0 2024-05-14T17:08:10.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.664+0000 7f52ba022740 1 -- 172.21.15.2:0/3603880488 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52a8020ad0 con 0x5562fc4408f0 2024-05-14T17:08:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.664+0000 7f5298ff9640 1 -- 172.21.15.2:0/3603880488 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f52ac0a8ae0 con 0x5562fc4408f0 2024-05-14T17:08:10.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.664+0000 7f52ba022740 10 client.16626 fetch_fsmap learned FSMap version 46 2024-05-14T17:08:10.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.664+0000 7f52ba022740 1 -- 172.21.15.2:0/3603880488 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5562fc4c5ec0 con 0x5562fc4408f0 2024-05-14T17:08:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.664+0000 7f5298ff9640 1 -- 172.21.15.2:0/3603880488 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52a8020ad0 con 0x5562fc4408f0 2024-05-14T17:08:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.664+0000 7f5298ff9640 1 -- 172.21.15.2:0/3603880488 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 46) ==== 8342+0+0 (secure 0 0 0) 0x7f52ac0a5600 con 0x5562fc4408f0 2024-05-14T17:08:10.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.664+0000 7f52ba022740 10 client.16626 fetch_fsmap finished waiting for FSMap version 46 2024-05-14T17:08:10.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.664+0000 7f52ba022740 10 client.16626 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:08:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.664+0000 7f52ba022740 20 client.16626 populate_metadata read hostname 'smithi002' 2024-05-14T17:08:10.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.664+0000 7f52ba022740 1 --2- 172.21.15.2:0/3603880488 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5562fc4c61b0 0x5562fc4c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:11.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.664+0000 7f52ba022740 4 client.16626 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:08:11.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.664+0000 7f52ba022740 1 -- 172.21.15.2:0/3603880488 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5562fc3af360 con 0x5562fc4c61b0 2024-05-14T17:08:11.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.665+0000 7f52b48ac640 1 --2- 172.21.15.2:0/3603880488 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5562fc4c61b0 0x5562fc4c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:11.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.665+0000 7f52b48ac640 1 --2- 172.21.15.2:0/3603880488 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5562fc4c61b0 0x5562fc4c85a0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d559316c501a964 server_cookie=8a174c9856f04520 in_seq=0 out_seq=0 2024-05-14T17:08:11.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.665+0000 7f5298ff9640 10 client.16626 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:08:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.678+0000 7f5298ff9640 1 -- 172.21.15.2:0/3603880488 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5562fc3af360 con 0x5562fc4c61b0 2024-05-14T17:08:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.678+0000 7f5298ff9640 10 client.16626 handle_command_reply: tid=0 2024-05-14T17:08:11.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.678+0000 7f52ba022740 1 client.16626 shutdown 2024-05-14T17:08:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.679+0000 7f52ba022740 1 -- 172.21.15.2:0/3603880488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562fc4408f0 msgr2=0x5562fc453e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.679+0000 7f52ba022740 1 --2- 172.21.15.2:0/3603880488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562fc4408f0 0x5562fc453e70 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f52ac0a9070 tx=0x7f52ac0044d0 comp rx=0 tx=0).stop 2024-05-14T17:08:11.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.679+0000 7f52ba022740 1 -- 172.21.15.2:0/3603880488 shutdown_connections 2024-05-14T17:08:11.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.679+0000 7f52ba022740 1 --2- 172.21.15.2:0/3603880488 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5562fc4c61b0 0x5562fc4c85a0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.679+0000 7f52ba022740 1 --2- 172.21.15.2:0/3603880488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5562fc44bad0 0x5562fc44f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:11.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.679+0000 7f52ba022740 1 --2- 172.21.15.2:0/3603880488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562fc441210 0x5562fc44f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.679+0000 7f52ba022740 1 --2- 172.21.15.2:0/3603880488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5562fc4408f0 0x5562fc453e70 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.679+0000 7f5298ff9640 0 client.16626 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:08:12.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.679+0000 7f52ba022740 1 -- 172.21.15.2:0/3603880488 >> 172.21.15.2:0/3603880488 conn(0x5562fc43ce00 msgr2=0x5562fc43c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:12.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.679+0000 7f52ba022740 1 -- 172.21.15.2:0/3603880488 shutdown_connections 2024-05-14T17:08:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.679+0000 7f52ba022740 1 -- 172.21.15.2:0/3603880488 wait complete. 2024-05-14T17:08:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.679+0000 7f52ba022740 20 client.16626 trim_cache size 0 max 16384 2024-05-14T17:08:12.250 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:08:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:08:12.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:08:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.680+0000 7f52b7311640 1 -- 172.21.15.2:0/3598808786 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f529c0b2a10 msgr2=0x7f529c0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.680+0000 7f52b7311640 1 --2- 172.21.15.2:0/3598808786 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f529c0b2a10 0x7f529c0b4ed0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f52a0000c00 tx=0x7f52a00058b0 comp rx=0 tx=0).stop 2024-05-14T17:08:12.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.680+0000 7f52b7311640 1 -- 172.21.15.2:0/3598808786 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52b0102a50 msgr2=0x7f52b0199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.680+0000 7f52b7311640 1 --2- 172.21.15.2:0/3598808786 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52b0102a50 0x7f52b0199160 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00d950 tx=0x7f52ac00de10 comp rx=0 tx=0).stop 2024-05-14T17:08:12.696 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.680+0000 7f52b7311640 1 -- 172.21.15.2:0/3598808786 shutdown_connections 2024-05-14T17:08:12.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.680+0000 7f52b7311640 1 --2- 172.21.15.2:0/3598808786 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f529c0b2a10 0x7f529c0b4ed0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.680+0000 7f52b7311640 1 --2- 172.21.15.2:0/3598808786 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52b0108a60 0x7f52b019da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.680+0000 7f52b7311640 1 --2- 172.21.15.2:0/3598808786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b0103410 0x7f52b01996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.680+0000 7f52b7311640 1 --2- 172.21.15.2:0/3598808786 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52b0102a50 0x7f52b0199160 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:13.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.680+0000 7f52b7311640 1 -- 172.21.15.2:0/3598808786 >> 172.21.15.2:0/3598808786 conn(0x7f52b00fe520 msgr2=0x7f52b00fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.680+0000 7f52b7311640 1 -- 172.21.15.2:0/3598808786 shutdown_connections 2024-05-14T17:08:13.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:02.680+0000 7f52b7311640 1 -- 172.21.15.2:0/3598808786 wait complete. 2024-05-14T17:08:13.340 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:07:02 smithi002 ceph-mon[31538]: pgmap v383: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 5 op/s 2024-05-14T17:08:13.421 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:07:04 smithi002 ceph-mon[31538]: pgmap v384: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 81 KiB/s wr, 27 op/s 2024-05-14T17:08:13.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:07:04 smithi002 ceph-mon[31538]: osdmap e92: 12 total, 12 up, 12 in 2024-05-14T17:08:13.549 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:07:05 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:08:13.662 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 710.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:08:13.769 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:07:06 smithi002 ceph-mon[31538]: pgmap v386: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 82 KiB/s wr, 29 op/s 2024-05-14T17:08:13.888 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:07:06 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:08:13.997 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:07:06 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:08:14.065 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:07:06 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:08:14.133 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:07:08 smithi002 ceph-mon[31538]: pgmap v387: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 82 KiB/s wr, 29 op/s 2024-05-14T17:08:14.178 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:07:10 smithi002 ceph-mon[31538]: pgmap v388: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 89 KiB/s wr, 32 op/s 2024-05-14T17:08:14.247 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 715.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:08:14.332 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:07:12 smithi002 ceph-mon[31538]: pgmap v389: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 89 KiB/s wr, 32 op/s 2024-05-14T17:08:14.428 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:07:12 smithi002 ceph-mon[31538]: osdmap e93: 12 total, 12 up, 12 in 2024-05-14T17:08:14.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:07:14 smithi002 ceph-mon[31538]: pgmap v391: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-14T17:08:14.557 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:07:58 smithi123 ceph-mon[35971]: pgmap v413: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.8 MiB/s wr, 555 op/s 2024-05-14T17:08:14.788 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:07:58 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:08:14.834 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:08:00 smithi123 ceph-mon[35971]: pgmap v414: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 8.5 MiB/s wr, 830 op/s 2024-05-14T17:08:14.867 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:08:01 smithi123 ceph-mon[35971]: pgmap v415: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 5.8 MiB/s wr, 571 op/s 2024-05-14T17:08:14.900 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:08:04 smithi123 ceph-mon[35971]: pgmap v416: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 7.8 MiB/s wr, 792 op/s 2024-05-14T17:08:14.963 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:08:06 smithi123 ceph-mon[35971]: pgmap v417: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 11 MiB/s wr, 942 op/s 2024-05-14T17:08:15.078 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:08:06 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:08:15.167 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:08:07 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:08:15.255 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:08:07 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:08:15.384 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:08:07 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:08:15.451 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:08:08 smithi123 ceph-mon[35971]: pgmap v418: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 9.9 MiB/s wr, 817 op/s 2024-05-14T17:08:15.521 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:08:10 smithi123 ceph-mon[35971]: osdmap e94: 12 total, 12 up, 12 in 2024-05-14T17:08:15.584 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:08:10 smithi123 ceph-mon[35971]: pgmap v420: 97 pgs: 97 active+clean; 4.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 31 MiB/s wr, 2.17k op/s 2024-05-14T17:08:15.640 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:08:12 smithi123 ceph-mon[35971]: pgmap v421: 97 pgs: 97 active+clean; 4.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 31 MiB/s wr, 2.17k op/s 2024-05-14T17:08:15.694 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:07:58 smithi151 ceph-mon[36066]: pgmap v413: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.8 MiB/s wr, 555 op/s 2024-05-14T17:08:15.726 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:07:58 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:08:15.759 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:08:00 smithi151 ceph-mon[36066]: pgmap v414: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 8.5 MiB/s wr, 830 op/s 2024-05-14T17:08:15.903 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:08:01 smithi151 ceph-mon[36066]: pgmap v415: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 5.8 MiB/s wr, 571 op/s 2024-05-14T17:08:15.982 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:08:04 smithi151 ceph-mon[36066]: pgmap v416: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 7.8 MiB/s wr, 792 op/s 2024-05-14T17:08:16.085 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:08:06 smithi151 ceph-mon[36066]: pgmap v417: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 11 MiB/s wr, 942 op/s 2024-05-14T17:08:16.182 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:08:06 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:08:16.243 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:08:07 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:08:16.308 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:08:07 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:08:16.371 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:08:07 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:08:16.509 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:08:08 smithi151 ceph-mon[36066]: pgmap v418: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 9.9 MiB/s wr, 817 op/s 2024-05-14T17:08:16.616 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:08:10 smithi151 ceph-mon[36066]: osdmap e94: 12 total, 12 up, 12 in 2024-05-14T17:08:16.668 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:08:10 smithi151 ceph-mon[36066]: pgmap v420: 97 pgs: 97 active+clean; 4.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 31 MiB/s wr, 2.17k op/s 2024-05-14T17:08:16.793 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:08:12 smithi151 ceph-mon[36066]: pgmap v421: 97 pgs: 97 active+clean; 4.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 31 MiB/s wr, 2.17k op/s 2024-05-14T17:08:16.898 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:08:14 smithi123 ceph-mon[35971]: pgmap v422: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 31 MiB/s wr, 2.18k op/s 2024-05-14T17:08:17.000 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:08:14 smithi151 ceph-mon[36066]: pgmap v422: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 31 MiB/s wr, 2.18k op/s 2024-05-14T17:08:17.054 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:08:16 smithi151 ceph-mon[36066]: pgmap v423: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 26 MiB/s wr, 1.86k op/s 2024-05-14T17:08:17.128 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:08:16 smithi123 ceph-mon[35971]: pgmap v423: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 26 MiB/s wr, 1.86k op/s 2024-05-14T17:08:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.782+0000 7f9d762c8640 1 -- 172.21.15.2:0/2889608016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d70107980 msgr2=0x7f9d70107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.782+0000 7f9d762c8640 1 --2- 172.21.15.2:0/2889608016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d70107980 0x7f9d70107d60 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f9d60009930 tx=0x7f9d6002a420 comp rx=0 tx=0).stop 2024-05-14T17:08:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.783+0000 7f9d762c8640 1 -- 172.21.15.2:0/2889608016 shutdown_connections 2024-05-14T17:08:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.783+0000 7f9d762c8640 1 --2- 172.21.15.2:0/2889608016 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d70102330 0x7f9d7010e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.783+0000 7f9d762c8640 1 --2- 172.21.15.2:0/2889608016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d70101970 0x7f9d70101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.783+0000 7f9d762c8640 1 --2- 172.21.15.2:0/2889608016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d70107980 0x7f9d70107d60 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.783+0000 7f9d762c8640 1 -- 172.21.15.2:0/2889608016 >> 172.21.15.2:0/2889608016 conn(0x7f9d70077cd0 msgr2=0x7f9d700780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.783+0000 7f9d762c8640 1 -- 172.21.15.2:0/2889608016 shutdown_connections 2024-05-14T17:08:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.783+0000 7f9d762c8640 1 -- 172.21.15.2:0/2889608016 wait complete. 2024-05-14T17:08:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.783+0000 7f9d762c8640 1 Processor -- start 2024-05-14T17:08:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.783+0000 7f9d762c8640 1 -- start start 2024-05-14T17:08:17.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.784+0000 7f9d762c8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d70101970 0x7f9d7019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.784+0000 7f9d762c8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d70102330 0x7f9d7019b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:18.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.784+0000 7f9d762c8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d70107980 0x7f9d7019fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.784+0000 7f9d762c8640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9d70111c90 con 0x7f9d70107980 2024-05-14T17:08:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.784+0000 7f9d762c8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9d70111b10 con 0x7f9d70102330 2024-05-14T17:08:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.784+0000 7f9d762c8640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9d70111e10 con 0x7f9d70101970 2024-05-14T17:08:18.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.784+0000 7f9d74865640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d70107980 0x7f9d7019fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:18.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.784+0000 7f9d74865640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d70107980 0x7f9d7019fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36988/0 (socket says 172.21.15.2:36988) 2024-05-14T17:08:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.784+0000 7f9d74865640 1 -- 172.21.15.2:0/3511935037 learned_addr learned my addr 172.21.15.2:0/3511935037 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:08:18.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.784+0000 7f9d6ffff640 1 --2- 172.21.15.2:0/3511935037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d70101970 0x7f9d7019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:18.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.784+0000 7f9d6f7fe640 1 --2- 172.21.15.2:0/3511935037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d70102330 0x7f9d7019b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.784+0000 7f9d74865640 1 -- 172.21.15.2:0/3511935037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d70101970 msgr2=0x7f9d7019b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:18.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.784+0000 7f9d74865640 1 --2- 172.21.15.2:0/3511935037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d70101970 0x7f9d7019b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.784+0000 7f9d74865640 1 -- 172.21.15.2:0/3511935037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d70102330 msgr2=0x7f9d7019b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.784+0000 7f9d74865640 1 --2- 172.21.15.2:0/3511935037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d70102330 0x7f9d7019b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:19.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.784+0000 7f9d74865640 1 -- 172.21.15.2:0/3511935037 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d701a0350 con 0x7f9d70107980 2024-05-14T17:08:19.382 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.784+0000 7f9d6f7fe640 1 --2- 172.21.15.2:0/3511935037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d70102330 0x7f9d7019b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:08:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.784+0000 7f9d6ffff640 1 --2- 172.21.15.2:0/3511935037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d70101970 0x7f9d7019b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:08:19.503 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.784+0000 7f9d74865640 1 --2- 172.21.15.2:0/3511935037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d70107980 0x7f9d7019fc20 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f9d6800eb10 tx=0x7f9d6800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.785+0000 7f9d6d7fa640 1 -- 172.21.15.2:0/3511935037 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d68004b20 con 0x7f9d70107980 2024-05-14T17:08:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.785+0000 7f9d6d7fa640 1 -- 172.21.15.2:0/3511935037 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d68004cc0 con 0x7f9d70107980 2024-05-14T17:08:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.785+0000 7f9d762c8640 1 -- 172.21.15.2:0/3511935037 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d701a0640 con 0x7f9d70107980 2024-05-14T17:08:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.785+0000 7f9d6d7fa640 1 -- 172.21.15.2:0/3511935037 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d68010670 con 0x7f9d70107980 2024-05-14T17:08:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.785+0000 7f9d762c8640 1 -- 172.21.15.2:0/3511935037 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d701a7ed0 con 0x7f9d70107980 2024-05-14T17:08:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.788+0000 7f9d6d7fa640 1 -- 172.21.15.2:0/3511935037 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f9d68005020 con 0x7f9d70107980 2024-05-14T17:08:20.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.788+0000 7f9d78fe3740 1 Processor -- start 2024-05-14T17:08:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.788+0000 7f9d78fe3740 1 -- start start 2024-05-14T17:08:20.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.788+0000 7f9d78fe3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa294828f0 0x55aa29482cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:20.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.788+0000 7f9d78fe3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa29483210 0x55aa2948d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.788+0000 7f9d78fe3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa2948dad0 0x55aa2948fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:20.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.788+0000 7f9d78fe3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55aa293f5c30 con 0x55aa294828f0 2024-05-14T17:08:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.788+0000 7f9d78fe3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55aa293e57d0 con 0x55aa29483210 2024-05-14T17:08:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.788+0000 7f9d78fe3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55aa293e9810 con 0x55aa2948dad0 2024-05-14T17:08:20.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.788+0000 7f9d6f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa294828f0 0x55aa29482cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.788+0000 7f9d6f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa294828f0 0x55aa29482cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36990/0 (socket says 172.21.15.2:36990) 2024-05-14T17:08:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.788+0000 7f9d6f7fe640 1 -- 172.21.15.2:0/2501997452 learned_addr learned my addr 172.21.15.2:0/2501997452 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:08:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.788+0000 7f9d6ffff640 1 --2- 172.21.15.2:0/2501997452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa29483210 0x55aa2948d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:21.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.788+0000 7f9d6f7fe640 1 -- 172.21.15.2:0/2501997452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa2948dad0 msgr2=0x55aa2948fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:21.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.788+0000 7f9d6f7fe640 1 --2- 172.21.15.2:0/2501997452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa2948dad0 0x55aa2948fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.788+0000 7f9d6f7fe640 1 -- 172.21.15.2:0/2501997452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa29483210 msgr2=0x55aa2948d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.788+0000 7f9d6f7fe640 1 --2- 172.21.15.2:0/2501997452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa29483210 0x55aa2948d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.789+0000 7f9d6d7fa640 1 --2- 172.21.15.2:0/3511935037 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f9d580b2a90 0x7f9d580b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.789+0000 7f9d6f7fe640 1 -- 172.21.15.2:0/2501997452 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa290fa210 con 0x55aa294828f0 2024-05-14T17:08:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.789+0000 7f9d6ffff640 1 --2- 172.21.15.2:0/3511935037 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f9d580b2a90 0x7f9d580b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:21.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.789+0000 7f9d6f7fe640 1 --2- 172.21.15.2:0/2501997452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa294828f0 0x55aa29482cd0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f9d640058c0 tx=0x7f9d64011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5126ea7a5f3dd0c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.789+0000 7f9d6d7fa640 1 -- 172.21.15.2:0/3511935037 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f9d680e2bb0 con 0x7f9d70107980 2024-05-14T17:08:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.789+0000 7f9d567fc640 1 -- 172.21.15.2:0/2501997452 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d64011e60 con 0x55aa294828f0 2024-05-14T17:08:22.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.789+0000 7f9d567fc640 1 -- 172.21.15.2:0/2501997452 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d640128a0 con 0x55aa294828f0 2024-05-14T17:08:22.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.789+0000 7f9d567fc640 1 -- 172.21.15.2:0/2501997452 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6400f880 con 0x55aa294828f0 2024-05-14T17:08:22.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.790+0000 7f9d78fe3740 1 -- 172.21.15.2:0/2501997452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa294828f0 msgr2=0x55aa29482cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.790+0000 7f9d6ffff640 1 --2- 172.21.15.2:0/3511935037 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f9d580b2a90 0x7f9d580b4f50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9d60006010 tx=0x7f9d60005b50 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.790+0000 7f9d78fe3740 1 --2- 172.21.15.2:0/2501997452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa294828f0 0x55aa29482cd0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f9d640058c0 tx=0x7f9d64011750 comp rx=0 tx=0).stop 2024-05-14T17:08:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.790+0000 7f9d78fe3740 1 -- 172.21.15.2:0/2501997452 shutdown_connections 2024-05-14T17:08:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.790+0000 7f9d78fe3740 1 --2- 172.21.15.2:0/2501997452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa2948dad0 0x55aa2948fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.790+0000 7f9d78fe3740 1 --2- 172.21.15.2:0/2501997452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa29483210 0x55aa2948d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:22.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.790+0000 7f9d78fe3740 1 --2- 172.21.15.2:0/2501997452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa294828f0 0x55aa29482cd0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.790+0000 7f9d78fe3740 1 -- 172.21.15.2:0/2501997452 >> 172.21.15.2:0/2501997452 conn(0x55aa2947ee00 msgr2=0x55aa2947f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:22.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.790+0000 7f9d78fe3740 1 -- 172.21.15.2:0/2501997452 shutdown_connections 2024-05-14T17:08:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.790+0000 7f9d78fe3740 1 -- 172.21.15.2:0/2501997452 wait complete. 2024-05-14T17:08:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.790+0000 7f9d78fe3740 1 -- start start 2024-05-14T17:08:22.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.790+0000 7f9d78fe3740 1 Processor -- start 2024-05-14T17:08:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.791+0000 7f9d78fe3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa294828f0 0x55aa2948d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.791+0000 7f9d78fe3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa29483210 0x55aa2948a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.791+0000 7f9d78fe3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa2948dad0 0x55aa2948ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.791+0000 7f9d78fe3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55aa290cb790 con 0x55aa2948dad0 2024-05-14T17:08:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.791+0000 7f9d78fe3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55aa293a1890 con 0x55aa294828f0 2024-05-14T17:08:23.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.791+0000 7f9d78fe3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55aa294907a0 con 0x55aa29483210 2024-05-14T17:08:23.383 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.791+0000 7f9d78fe3740 10 client.0 _ll_register_callbacks cb 0x55aa293a4d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:08:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.791+0000 7f9d74865640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa2948dad0 0x55aa2948ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:23.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.791+0000 7f9d74865640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa2948dad0 0x55aa2948ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37004/0 (socket says 172.21.15.2:37004) 2024-05-14T17:08:23.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.791+0000 7f9d74865640 1 -- 172.21.15.2:0/2777896715 learned_addr learned my addr 172.21.15.2:0/2777896715 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:08:23.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.791+0000 7f9d6f7fe640 1 --2- 172.21.15.2:0/2777896715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa29483210 0x55aa2948a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.791+0000 7f9d6ffff640 1 --2- 172.21.15.2:0/2777896715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa294828f0 0x55aa2948d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.791+0000 7f9d74865640 1 -- 172.21.15.2:0/2777896715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa29483210 msgr2=0x55aa2948a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:23.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.791+0000 7f9d74865640 1 --2- 172.21.15.2:0/2777896715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa29483210 0x55aa2948a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:23.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.791+0000 7f9d74865640 1 -- 172.21.15.2:0/2777896715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa294828f0 msgr2=0x55aa2948d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:23.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.791+0000 7f9d74865640 1 --2- 172.21.15.2:0/2777896715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa294828f0 0x55aa2948d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.791+0000 7f9d74865640 1 -- 172.21.15.2:0/2777896715 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa291c6620 con 0x55aa2948dad0 2024-05-14T17:08:24.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.791+0000 7f9d6ffff640 1 --2- 172.21.15.2:0/2777896715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa294828f0 0x55aa2948d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:08:24.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.791+0000 7f9d74865640 1 --2- 172.21.15.2:0/2777896715 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa2948dad0 0x55aa2948ac90 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f9d680a5f10 tx=0x7f9d680a9d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805873d35cd8c32f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:24.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.791+0000 7f9d4f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:08:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.792+0000 7f9d4f7fe640 1 -- 172.21.15.2:0/2777896715 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d680aa590 con 0x55aa2948dad0 2024-05-14T17:08:24.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.792+0000 7f9d4f7fe640 1 -- 172.21.15.2:0/2777896715 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f9d680b2860 con 0x55aa2948dad0 2024-05-14T17:08:24.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.792+0000 7f9d78fe3740 1 -- 172.21.15.2:0/2777896715 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d640128a0 con 0x55aa2948dad0 2024-05-14T17:08:24.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.792+0000 7f9d4f7fe640 1 -- 172.21.15.2:0/2777896715 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f9d680a9e90 con 0x55aa2948dad0 2024-05-14T17:08:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.792+0000 7f9d78fe3740 10 client.16638 fetch_fsmap learned FSMap version 46 2024-05-14T17:08:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.792+0000 7f9d78fe3740 1 -- 172.21.15.2:0/2777896715 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55aa2949c260 con 0x55aa2948dad0 2024-05-14T17:08:24.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.792+0000 7f9d4f7fe640 1 -- 172.21.15.2:0/2777896715 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d640128a0 con 0x55aa2948dad0 2024-05-14T17:08:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.792+0000 7f9d4f7fe640 1 -- 172.21.15.2:0/2777896715 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 46) ==== 8342+0+0 (secure 0 0 0) 0x7f9d680a5920 con 0x55aa2948dad0 2024-05-14T17:08:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.792+0000 7f9d78fe3740 10 client.16638 fetch_fsmap finished waiting for FSMap version 46 2024-05-14T17:08:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.792+0000 7f9d78fe3740 10 client.16638 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:08:25.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.792+0000 7f9d78fe3740 20 client.16638 populate_metadata read hostname 'smithi002' 2024-05-14T17:08:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.792+0000 7f9d78fe3740 1 --2- 172.21.15.2:0/2777896715 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55aa2949d7c0 0x55aa2949fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:25.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.792+0000 7f9d78fe3740 4 client.16638 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:08:25.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.792+0000 7f9d78fe3740 1 -- 172.21.15.2:0/2777896715 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa293f7b80 con 0x55aa2949d7c0 2024-05-14T17:08:25.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.793+0000 7f9d6ffff640 1 --2- 172.21.15.2:0/2777896715 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55aa2949d7c0 0x55aa2949fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.793+0000 7f9d6ffff640 1 --2- 172.21.15.2:0/2777896715 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55aa2949d7c0 0x55aa2949fc10 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b23d15cd47d339ec server_cookie=b0b88b0884804c6b in_seq=0 out_seq=0 2024-05-14T17:08:25.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.793+0000 7f9d4f7fe640 10 client.16638 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:08:25.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.794+0000 7f9d4f7fe640 1 -- 172.21.15.2:0/2777896715 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55aa293f7b80 con 0x55aa2949d7c0 2024-05-14T17:08:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.794+0000 7f9d4f7fe640 10 client.16638 handle_command_reply: tid=0 2024-05-14T17:08:25.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.794+0000 7f9d78fe3740 1 client.16638 shutdown 2024-05-14T17:08:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.794+0000 7f9d78fe3740 1 -- 172.21.15.2:0/2777896715 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa2948dad0 msgr2=0x55aa2948ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.794+0000 7f9d78fe3740 1 --2- 172.21.15.2:0/2777896715 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa2948dad0 0x55aa2948ac90 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f9d680a5f10 tx=0x7f9d680a9d90 comp rx=0 tx=0).stop 2024-05-14T17:08:25.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.794+0000 7f9d78fe3740 1 -- 172.21.15.2:0/2777896715 shutdown_connections 2024-05-14T17:08:25.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.794+0000 7f9d78fe3740 1 --2- 172.21.15.2:0/2777896715 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55aa2949d7c0 0x55aa2949fc10 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.794+0000 7f9d78fe3740 1 --2- 172.21.15.2:0/2777896715 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa2948dad0 0x55aa2948ac90 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.794+0000 7f9d4f7fe640 0 client.16638 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:08:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.794+0000 7f9d78fe3740 1 --2- 172.21.15.2:0/2777896715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa29483210 0x55aa2948a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.794+0000 7f9d78fe3740 1 --2- 172.21.15.2:0/2777896715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa294828f0 0x55aa2948d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.794+0000 7f9d78fe3740 1 -- 172.21.15.2:0/2777896715 >> 172.21.15.2:0/2777896715 conn(0x55aa2947ee00 msgr2=0x55aa2947e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.794+0000 7f9d78fe3740 1 -- 172.21.15.2:0/2777896715 shutdown_connections 2024-05-14T17:08:26.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.794+0000 7f9d78fe3740 1 -- 172.21.15.2:0/2777896715 wait complete. 2024-05-14T17:08:26.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.794+0000 7f9d78fe3740 20 client.16638 trim_cache size 0 max 16384 2024-05-14T17:08:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.810+0000 7f9d78fe3740 1 Processor -- start 2024-05-14T17:08:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.810+0000 7f9d78fe3740 1 -- start start 2024-05-14T17:08:26.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.810+0000 7f9d78fe3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa294828f0 0x55aa29499670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:26.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.810+0000 7f9d78fe3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa29483210 0x55aa2948cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.810+0000 7f9d78fe3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa2948dad0 0x55aa294a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.810+0000 7f9d78fe3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9d701b30f0 con 0x55aa294828f0 2024-05-14T17:08:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.810+0000 7f9d78fe3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9d701b0990 con 0x55aa29483210 2024-05-14T17:08:27.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.810+0000 7f9d78fe3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9d701ae800 con 0x55aa2948dad0 2024-05-14T17:08:27.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.810+0000 7f9d6ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa294828f0 0x55aa29499670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:27.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.810+0000 7f9d6ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa294828f0 0x55aa29499670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37018/0 (socket says 172.21.15.2:37018) 2024-05-14T17:08:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.810+0000 7f9d6ffff640 1 -- 172.21.15.2:0/147292195 learned_addr learned my addr 172.21.15.2:0/147292195 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:08:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.810+0000 7f9d74865640 1 --2- 172.21.15.2:0/147292195 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa2948dad0 0x55aa294a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.810+0000 7f9d6f7fe640 1 --2- 172.21.15.2:0/147292195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa29483210 0x55aa2948cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:27.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.810+0000 7f9d6ffff640 1 -- 172.21.15.2:0/147292195 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa2948dad0 msgr2=0x55aa294a54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.810+0000 7f9d6ffff640 1 --2- 172.21.15.2:0/147292195 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa2948dad0 0x55aa294a54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:27.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.810+0000 7f9d6ffff640 1 -- 172.21.15.2:0/147292195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa29483210 msgr2=0x55aa2948cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:27.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.810+0000 7f9d6ffff640 1 --2- 172.21.15.2:0/147292195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa29483210 0x55aa2948cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.810+0000 7f9d6ffff640 1 -- 172.21.15.2:0/147292195 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa2947a830 con 0x55aa294828f0 2024-05-14T17:08:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.810+0000 7f9d74865640 1 --2- 172.21.15.2:0/147292195 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa2948dad0 0x55aa294a54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:08:28.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.810+0000 7f9d6f7fe640 1 --2- 172.21.15.2:0/147292195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa29483210 0x55aa2948cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:08:28.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.810+0000 7f9d6ffff640 1 --2- 172.21.15.2:0/147292195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa294828f0 0x55aa29499670 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9d60031440 tx=0x7f9d6003e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=983d5af0e748045e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.811+0000 7f9d55ffb640 1 -- 172.21.15.2:0/147292195 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d60035040 con 0x55aa294828f0 2024-05-14T17:08:28.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.811+0000 7f9d55ffb640 1 -- 172.21.15.2:0/147292195 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d60005630 con 0x55aa294828f0 2024-05-14T17:08:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.811+0000 7f9d55ffb640 1 -- 172.21.15.2:0/147292195 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d60003850 con 0x55aa294828f0 2024-05-14T17:08:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.811+0000 7f9d78fe3740 1 -- 172.21.15.2:0/147292195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa294828f0 msgr2=0x55aa29499670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.811+0000 7f9d78fe3740 1 --2- 172.21.15.2:0/147292195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa294828f0 0x55aa29499670 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9d60031440 tx=0x7f9d6003e000 comp rx=0 tx=0).stop 2024-05-14T17:08:28.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.811+0000 7f9d78fe3740 1 -- 172.21.15.2:0/147292195 shutdown_connections 2024-05-14T17:08:28.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.811+0000 7f9d78fe3740 1 --2- 172.21.15.2:0/147292195 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa2948dad0 0x55aa294a54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.811+0000 7f9d78fe3740 1 --2- 172.21.15.2:0/147292195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa29483210 0x55aa2948cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.811+0000 7f9d78fe3740 1 --2- 172.21.15.2:0/147292195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa294828f0 0x55aa29499670 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.811+0000 7f9d78fe3740 1 -- 172.21.15.2:0/147292195 >> 172.21.15.2:0/147292195 conn(0x55aa2947ee00 msgr2=0x55aa29484b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.811+0000 7f9d78fe3740 1 -- 172.21.15.2:0/147292195 shutdown_connections 2024-05-14T17:08:29.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.811+0000 7f9d78fe3740 1 -- 172.21.15.2:0/147292195 wait complete. 2024-05-14T17:08:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.812+0000 7f9d78fe3740 1 -- start start 2024-05-14T17:08:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.812+0000 7f9d78fe3740 1 Processor -- start 2024-05-14T17:08:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.812+0000 7f9d78fe3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa294828f0 0x55aa29495d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.812+0000 7f9d78fe3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa29483210 0x55aa294910f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.812+0000 7f9d78fe3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa2948dad0 0x55aa29491630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.812+0000 7f9d78fe3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9d701aee50 con 0x55aa294828f0 2024-05-14T17:08:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.812+0000 7f9d78fe3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9d701b02c0 con 0x55aa29483210 2024-05-14T17:08:29.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.812+0000 7f9d78fe3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9d701af520 con 0x55aa2948dad0 2024-05-14T17:08:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.812+0000 7f9d78fe3740 10 client.0 _ll_register_callbacks cb 0x7f9d680a5d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:08:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.812+0000 7f9d6ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa294828f0 0x55aa29495d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.812+0000 7f9d6ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa294828f0 0x55aa29495d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37020/0 (socket says 172.21.15.2:37020) 2024-05-14T17:08:29.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.812+0000 7f9d6ffff640 1 -- 172.21.15.2:0/4040091382 learned_addr learned my addr 172.21.15.2:0/4040091382 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:08:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.812+0000 7f9d6f7fe640 1 --2- 172.21.15.2:0/4040091382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa29483210 0x55aa294910f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:29.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.812+0000 7f9d74865640 1 --2- 172.21.15.2:0/4040091382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa2948dad0 0x55aa29491630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.813+0000 7f9d6ffff640 1 -- 172.21.15.2:0/4040091382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa2948dad0 msgr2=0x55aa29491630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:30.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.813+0000 7f9d6ffff640 1 --2- 172.21.15.2:0/4040091382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa2948dad0 0x55aa29491630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:30.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.813+0000 7f9d6ffff640 1 -- 172.21.15.2:0/4040091382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa29483210 msgr2=0x55aa294910f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.813+0000 7f9d6ffff640 1 --2- 172.21.15.2:0/4040091382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa29483210 0x55aa294910f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:30.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.813+0000 7f9d6ffff640 1 -- 172.21.15.2:0/4040091382 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa29491c20 con 0x55aa294828f0 2024-05-14T17:08:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.813+0000 7f9d74865640 1 --2- 172.21.15.2:0/4040091382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa2948dad0 0x55aa29491630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:08:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.813+0000 7f9d6ffff640 1 --2- 172.21.15.2:0/4040091382 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa294828f0 0x55aa29495d30 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f9d60000c00 tx=0x7f9d60004d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7073d6fcc88157ba server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:30.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.813+0000 7f9d4f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:08:30.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.813+0000 7f9d4f7fe640 1 -- 172.21.15.2:0/4040091382 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d60035070 con 0x55aa294828f0 2024-05-14T17:08:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.813+0000 7f9d4f7fe640 1 -- 172.21.15.2:0/4040091382 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f9d60051450 con 0x55aa294828f0 2024-05-14T17:08:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.813+0000 7f9d78fe3740 1 -- 172.21.15.2:0/4040091382 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55aa293f1360 con 0x55aa294828f0 2024-05-14T17:08:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.813+0000 7f9d4f7fe640 1 -- 172.21.15.2:0/4040091382 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f9d6006a050 con 0x55aa294828f0 2024-05-14T17:08:30.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.813+0000 7f9d78fe3740 10 client.16644 fetch_fsmap learned FSMap version 46 2024-05-14T17:08:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.813+0000 7f9d78fe3740 1 -- 172.21.15.2:0/4040091382 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55aa29507df0 con 0x55aa294828f0 2024-05-14T17:08:30.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.813+0000 7f9d4f7fe640 1 -- 172.21.15.2:0/4040091382 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55aa293f1360 con 0x55aa294828f0 2024-05-14T17:08:31.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.813+0000 7f9d4f7fe640 1 -- 172.21.15.2:0/4040091382 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 46) ==== 8342+0+0 (secure 0 0 0) 0x7f9d60051ce0 con 0x55aa294828f0 2024-05-14T17:08:31.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.813+0000 7f9d78fe3740 10 client.16644 fetch_fsmap finished waiting for FSMap version 46 2024-05-14T17:08:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.813+0000 7f9d78fe3740 10 client.16644 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:08:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.813+0000 7f9d78fe3740 20 client.16644 populate_metadata read hostname 'smithi002' 2024-05-14T17:08:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.813+0000 7f9d78fe3740 1 --2- 172.21.15.2:0/4040091382 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55aa295084d0 0x55aa2950a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:31.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.814+0000 7f9d78fe3740 4 client.16644 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:08:31.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.814+0000 7f9d78fe3740 1 -- 172.21.15.2:0/4040091382 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aa2936b5f0 con 0x55aa295084d0 2024-05-14T17:08:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.814+0000 7f9d6f7fe640 1 --2- 172.21.15.2:0/4040091382 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55aa295084d0 0x55aa2950a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.814+0000 7f9d4f7fe640 10 client.16644 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:08:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.814+0000 7f9d6f7fe640 1 --2- 172.21.15.2:0/4040091382 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55aa295084d0 0x55aa2950a8f0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c08a0d734ccd123a server_cookie=2551c25c2017ec2c in_seq=0 out_seq=0 2024-05-14T17:08:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.825+0000 7f9d4f7fe640 1 -- 172.21.15.2:0/4040091382 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55aa2936b5f0 con 0x55aa295084d0 2024-05-14T17:08:31.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.825+0000 7f9d4f7fe640 10 client.16644 handle_command_reply: tid=0 2024-05-14T17:08:31.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.825+0000 7f9d78fe3740 1 client.16644 shutdown 2024-05-14T17:08:31.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.825+0000 7f9d78fe3740 1 -- 172.21.15.2:0/4040091382 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa294828f0 msgr2=0x55aa29495d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.825+0000 7f9d78fe3740 1 --2- 172.21.15.2:0/4040091382 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa294828f0 0x55aa29495d30 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f9d60000c00 tx=0x7f9d60004d40 comp rx=0 tx=0).stop 2024-05-14T17:08:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.825+0000 7f9d78fe3740 1 -- 172.21.15.2:0/4040091382 shutdown_connections 2024-05-14T17:08:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.825+0000 7f9d78fe3740 1 --2- 172.21.15.2:0/4040091382 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55aa295084d0 0x55aa2950a8f0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.825+0000 7f9d78fe3740 1 --2- 172.21.15.2:0/4040091382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa2948dad0 0x55aa29491630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.825+0000 7f9d78fe3740 1 --2- 172.21.15.2:0/4040091382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa29483210 0x55aa294910f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.825+0000 7f9d78fe3740 1 --2- 172.21.15.2:0/4040091382 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55aa294828f0 0x55aa29495d30 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:32.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.825+0000 7f9d78fe3740 1 -- 172.21.15.2:0/4040091382 >> 172.21.15.2:0/4040091382 conn(0x55aa2947ee00 msgr2=0x55aa294842f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.825+0000 7f9d4f7fe640 0 client.16644 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:08:32.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.825+0000 7f9d78fe3740 1 -- 172.21.15.2:0/4040091382 shutdown_connections 2024-05-14T17:08:32.696 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.825+0000 7f9d78fe3740 1 -- 172.21.15.2:0/4040091382 wait complete. 2024-05-14T17:08:32.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.825+0000 7f9d78fe3740 20 client.16644 trim_cache size 0 max 16384 2024-05-14T17:08:32.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:08:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:08:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:08:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.826+0000 7f9d762c8640 1 -- 172.21.15.2:0/3511935037 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f9d580b2a90 msgr2=0x7f9d580b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.826+0000 7f9d762c8640 1 --2- 172.21.15.2:0/3511935037 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f9d580b2a90 0x7f9d580b4f50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9d60006010 tx=0x7f9d60005b50 comp rx=0 tx=0).stop 2024-05-14T17:08:33.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.826+0000 7f9d762c8640 1 -- 172.21.15.2:0/3511935037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d70107980 msgr2=0x7f9d7019fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.826+0000 7f9d762c8640 1 --2- 172.21.15.2:0/3511935037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d70107980 0x7f9d7019fc20 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f9d6800eb10 tx=0x7f9d6800c2d0 comp rx=0 tx=0).stop 2024-05-14T17:08:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.826+0000 7f9d762c8640 1 -- 172.21.15.2:0/3511935037 shutdown_connections 2024-05-14T17:08:33.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.826+0000 7f9d762c8640 1 --2- 172.21.15.2:0/3511935037 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f9d580b2a90 0x7f9d580b4f50 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:33.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.826+0000 7f9d762c8640 1 --2- 172.21.15.2:0/3511935037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d70107980 0x7f9d7019fc20 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.826+0000 7f9d762c8640 1 --2- 172.21.15.2:0/3511935037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d70102330 0x7f9d7019b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:33.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.826+0000 7f9d762c8640 1 --2- 172.21.15.2:0/3511935037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d70101970 0x7f9d7019b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.826+0000 7f9d762c8640 1 -- 172.21.15.2:0/3511935037 >> 172.21.15.2:0/3511935037 conn(0x7f9d70077cd0 msgr2=0x7f9d700ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.826+0000 7f9d762c8640 1 -- 172.21.15.2:0/3511935037 shutdown_connections 2024-05-14T17:08:33.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:15.826+0000 7f9d762c8640 1 -- 172.21.15.2:0/3511935037 wait complete. 2024-05-14T17:08:33.856 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 720.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:08:34.016 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:07:16 smithi002 ceph-mon[31538]: pgmap v392: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-14T17:08:34.143 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:07:18 smithi002 ceph-mon[31538]: pgmap v393: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-14T17:08:34.222 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:07:20 smithi002 ceph-mon[31538]: pgmap v394: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-05-14T17:08:34.296 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 725.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:08:34.354 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:07:22 smithi002 ceph-mon[31538]: pgmap v395: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-05-14T17:08:34.457 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:07:24 smithi002 ceph-mon[31538]: pgmap v396: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 11 op/s 2024-05-14T17:08:34.557 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 730.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:08:34.661 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:07:26 smithi002 ceph-mon[31538]: pgmap v397: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-05-14T17:08:34.695 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:08:18 smithi123 ceph-mon[35971]: osdmap e95: 12 total, 12 up, 12 in 2024-05-14T17:08:34.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:08:18 smithi123 ceph-mon[35971]: pgmap v425: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 3.9 MiB/s wr, 409 op/s 2024-05-14T17:08:34.888 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:08:20 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:08:35.042 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:08:20 smithi123 ceph-mon[35971]: pgmap v426: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 10 MiB/s wr, 982 op/s 2024-05-14T17:08:35.106 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:08:22 smithi123 ceph-mon[35971]: pgmap v427: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 10 MiB/s wr, 982 op/s 2024-05-14T17:08:35.278 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:08:24 smithi123 ceph-mon[35971]: pgmap v428: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 19 MiB/s wr, 1.69k op/s 2024-05-14T17:08:35.356 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:08:26 smithi123 ceph-mon[35971]: pgmap v429: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 20 MiB/s wr, 1.76k op/s 2024-05-14T17:08:35.408 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:08:28 smithi123 ceph-mon[35971]: pgmap v430: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 20 MiB/s wr, 1.72k op/s 2024-05-14T17:08:35.459 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:08:30 smithi123 ceph-mon[35971]: pgmap v431: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 19 MiB/s wr, 1.73k op/s 2024-05-14T17:08:35.493 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:08:32 smithi123 ceph-mon[35971]: pgmap v432: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 1.20k op/s 2024-05-14T17:08:35.526 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:08:18 smithi151 ceph-mon[36066]: osdmap e95: 12 total, 12 up, 12 in 2024-05-14T17:08:35.621 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:08:18 smithi151 ceph-mon[36066]: pgmap v425: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 3.9 MiB/s wr, 409 op/s 2024-05-14T17:08:35.672 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:08:20 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:08:35.787 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:08:20 smithi151 ceph-mon[36066]: pgmap v426: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 10 MiB/s wr, 982 op/s 2024-05-14T17:08:35.893 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:08:22 smithi151 ceph-mon[36066]: pgmap v427: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 10 MiB/s wr, 982 op/s 2024-05-14T17:08:35.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:08:24 smithi151 ceph-mon[36066]: pgmap v428: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 19 MiB/s wr, 1.69k op/s 2024-05-14T17:08:36.118 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:08:26 smithi151 ceph-mon[36066]: pgmap v429: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 20 MiB/s wr, 1.76k op/s 2024-05-14T17:08:36.151 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:08:28 smithi151 ceph-mon[36066]: pgmap v430: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 20 MiB/s wr, 1.72k op/s 2024-05-14T17:08:36.225 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:08:30 smithi151 ceph-mon[36066]: pgmap v431: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 19 MiB/s wr, 1.73k op/s 2024-05-14T17:08:36.376 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:08:32 smithi151 ceph-mon[36066]: pgmap v432: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 1.20k op/s 2024-05-14T17:08:36.461 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:08:34 smithi123 ceph-mon[35971]: pgmap v433: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 16 MiB/s wr, 1.41k op/s 2024-05-14T17:08:36.574 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:08:34 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:08:36.627 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:08:34 smithi151 ceph-mon[36066]: pgmap v433: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 16 MiB/s wr, 1.41k op/s 2024-05-14T17:08:36.691 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:08:34 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:08:36.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.927+0000 7f278dcc4640 1 -- 172.21.15.2:0/2357772952 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2788103410 msgr2=0x7f278810f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:36.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.927+0000 7f278dcc4640 1 --2- 172.21.15.2:0/2357772952 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2788103410 0x7f278810f920 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f2780009cd0 tx=0x7f278002a8e0 comp rx=0 tx=0).stop 2024-05-14T17:08:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.927+0000 7f278dcc4640 1 -- 172.21.15.2:0/2357772952 shutdown_connections 2024-05-14T17:08:37.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.927+0000 7f278dcc4640 1 --2- 172.21.15.2:0/2357772952 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2788103410 0x7f278810f920 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.927+0000 7f278dcc4640 1 --2- 172.21.15.2:0/2357772952 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2788102a50 0x7f2788102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.927+0000 7f278dcc4640 1 --2- 172.21.15.2:0/2357772952 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2788108a60 0x7f2788108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.927+0000 7f278dcc4640 1 -- 172.21.15.2:0/2357772952 >> 172.21.15.2:0/2357772952 conn(0x7f27880fe520 msgr2=0x7f2788100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:37.385 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.927+0000 7f278dcc4640 1 -- 172.21.15.2:0/2357772952 shutdown_connections 2024-05-14T17:08:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.927+0000 7f278dcc4640 1 -- 172.21.15.2:0/2357772952 wait complete. 2024-05-14T17:08:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.928+0000 7f278dcc4640 1 Processor -- start 2024-05-14T17:08:37.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.928+0000 7f278dcc4640 1 -- start start 2024-05-14T17:08:37.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.928+0000 7f278dcc4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2788102a50 0x7f27881991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.928+0000 7f278dcc4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2788103410 0x7f2788199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.928+0000 7f278dcc4640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2788108a60 0x7f278819da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:37.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.928+0000 7f278dcc4640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2788112d20 con 0x7f2788102a50 2024-05-14T17:08:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.928+0000 7f278dcc4640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2788112ba0 con 0x7f2788103410 2024-05-14T17:08:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.928+0000 7f278dcc4640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2788112ea0 con 0x7f2788108a60 2024-05-14T17:08:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.929+0000 7f27877fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2788102a50 0x7f27881991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:38.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.929+0000 7f27877fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2788102a50 0x7f27881991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47050/0 (socket says 172.21.15.2:47050) 2024-05-14T17:08:38.263 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.929+0000 7f27877fe640 1 -- 172.21.15.2:0/3275047171 learned_addr learned my addr 172.21.15.2:0/3275047171 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:08:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.929+0000 7f2787fff640 1 --2- 172.21.15.2:0/3275047171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2788108a60 0x7f278819da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.929+0000 7f2786ffd640 1 --2- 172.21.15.2:0/3275047171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2788103410 0x7f2788199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.929+0000 7f27877fe640 1 -- 172.21.15.2:0/3275047171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2788108a60 msgr2=0x7f278819da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:38.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.929+0000 7f27877fe640 1 --2- 172.21.15.2:0/3275047171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2788108a60 0x7f278819da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.929+0000 7f27877fe640 1 -- 172.21.15.2:0/3275047171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2788103410 msgr2=0x7f2788199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:38.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.929+0000 7f27877fe640 1 --2- 172.21.15.2:0/3275047171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2788103410 0x7f2788199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:38.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.929+0000 7f27877fe640 1 -- 172.21.15.2:0/3275047171 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f278819e160 con 0x7f2788102a50 2024-05-14T17:08:38.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.929+0000 7f2786ffd640 1 --2- 172.21.15.2:0/3275047171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2788103410 0x7f2788199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:08:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.929+0000 7f27877fe640 1 --2- 172.21.15.2:0/3275047171 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2788102a50 0x7f27881991c0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f277800d770 tx=0x7f277800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:39.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.929+0000 7f2784ff9640 1 -- 172.21.15.2:0/3275047171 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2778017070 con 0x7f2788102a50 2024-05-14T17:08:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.929+0000 7f278dcc4640 1 -- 172.21.15.2:0/3275047171 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f278819e450 con 0x7f2788102a50 2024-05-14T17:08:39.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.929+0000 7f278dcc4640 1 -- 172.21.15.2:0/3275047171 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27881a5ce0 con 0x7f2788102a50 2024-05-14T17:08:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.929+0000 7f2784ff9640 1 -- 172.21.15.2:0/3275047171 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27780054d0 con 0x7f2788102a50 2024-05-14T17:08:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.929+0000 7f2784ff9640 1 -- 172.21.15.2:0/3275047171 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2778004c30 con 0x7f2788102a50 2024-05-14T17:08:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.932+0000 7f2784ff9640 1 -- 172.21.15.2:0/3275047171 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f2778004eb0 con 0x7f2788102a50 2024-05-14T17:08:39.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.932+0000 7f27909da740 1 Processor -- start 2024-05-14T17:08:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.932+0000 7f27909da740 1 -- start start 2024-05-14T17:08:39.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.932+0000 7f27909da740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558320cb7810 0x558320cb7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:39.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.932+0000 7f27909da740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558320cb8130 0x558320cc69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:39.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.932+0000 7f27909da740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558320cc6ee0 0x558320cc92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.932+0000 7f27909da740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x558320c2ac30 con 0x558320cb7810 2024-05-14T17:08:40.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.932+0000 7f27909da740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558320c1a7d0 con 0x558320cb8130 2024-05-14T17:08:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.932+0000 7f27909da740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558320c1e810 con 0x558320cc6ee0 2024-05-14T17:08:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.932+0000 7f2787fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558320cb7810 0x558320cb7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.932+0000 7f2787fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558320cb7810 0x558320cb7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47064/0 (socket says 172.21.15.2:47064) 2024-05-14T17:08:40.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.932+0000 7f2787fff640 1 -- 172.21.15.2:0/2884856909 learned_addr learned my addr 172.21.15.2:0/2884856909 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:08:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.933+0000 7f27877fe640 1 --2- 172.21.15.2:0/2884856909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558320cb8130 0x558320cc69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:40.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.933+0000 7f2786ffd640 1 --2- 172.21.15.2:0/2884856909 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558320cc6ee0 0x558320cc92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.933+0000 7f2787fff640 1 -- 172.21.15.2:0/2884856909 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558320cc6ee0 msgr2=0x558320cc92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.933+0000 7f2787fff640 1 --2- 172.21.15.2:0/2884856909 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558320cc6ee0 0x558320cc92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:40.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.933+0000 7f2787fff640 1 -- 172.21.15.2:0/2884856909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558320cb8130 msgr2=0x558320cc69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:40.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.933+0000 7f2787fff640 1 --2- 172.21.15.2:0/2884856909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558320cb8130 0x558320cc69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.933+0000 7f2787fff640 1 -- 172.21.15.2:0/2884856909 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55832092f210 con 0x558320cb7810 2024-05-14T17:08:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.933+0000 7f2784ff9640 1 --2- 172.21.15.2:0/3275047171 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f27700b2a90 0x7f27700b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.933+0000 7f2787fff640 1 --2- 172.21.15.2:0/2884856909 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558320cb7810 0x558320cb7bf0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f2780035000 tx=0x7f2780009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d856690ac0df3fa server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.933+0000 7f27877fe640 1 --2- 172.21.15.2:0/3275047171 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f27700b2a90 0x7f27700b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:41.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.933+0000 7f2784ff9640 1 -- 172.21.15.2:0/3275047171 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f27780e20d0 con 0x7f2788102a50 2024-05-14T17:08:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.933+0000 7f276dffb640 1 -- 172.21.15.2:0/2884856909 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2780031850 con 0x558320cb7810 2024-05-14T17:08:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.933+0000 7f276dffb640 1 -- 172.21.15.2:0/2884856909 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2780036040 con 0x558320cb7810 2024-05-14T17:08:41.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.933+0000 7f276dffb640 1 -- 172.21.15.2:0/2884856909 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2780004440 con 0x558320cb7810 2024-05-14T17:08:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.934+0000 7f27909da740 1 -- 172.21.15.2:0/2884856909 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558320cb7810 msgr2=0x558320cb7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:41.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.934+0000 7f27909da740 1 --2- 172.21.15.2:0/2884856909 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558320cb7810 0x558320cb7bf0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f2780035000 tx=0x7f2780009a20 comp rx=0 tx=0).stop 2024-05-14T17:08:41.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.934+0000 7f27877fe640 1 --2- 172.21.15.2:0/3275047171 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f27700b2a90 0x7f27700b4f50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f27780df8d0 tx=0x7f27780e4040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:41.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.934+0000 7f27909da740 1 -- 172.21.15.2:0/2884856909 shutdown_connections 2024-05-14T17:08:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.934+0000 7f27909da740 1 --2- 172.21.15.2:0/2884856909 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558320cc6ee0 0x558320cc92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.934+0000 7f27909da740 1 --2- 172.21.15.2:0/2884856909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558320cb8130 0x558320cc69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:41.744 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.934+0000 7f27909da740 1 --2- 172.21.15.2:0/2884856909 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558320cb7810 0x558320cb7bf0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.934+0000 7f27909da740 1 -- 172.21.15.2:0/2884856909 >> 172.21.15.2:0/2884856909 conn(0x558320cb3e00 msgr2=0x558320cb41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:41.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.934+0000 7f27909da740 1 -- 172.21.15.2:0/2884856909 shutdown_connections 2024-05-14T17:08:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.934+0000 7f27909da740 1 -- 172.21.15.2:0/2884856909 wait complete. 2024-05-14T17:08:42.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.934+0000 7f27909da740 1 -- start start 2024-05-14T17:08:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.935+0000 7f27909da740 1 Processor -- start 2024-05-14T17:08:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.935+0000 7f27909da740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558320cb7810 0x558320cc6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.935+0000 7f27909da740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558320cb8130 0x558320cc3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.935+0000 7f27909da740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558320cc6ee0 0x558320cc40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:42.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.935+0000 7f27909da740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x558320900790 con 0x558320cb7810 2024-05-14T17:08:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.935+0000 7f27909da740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558320bd6890 con 0x558320cc6ee0 2024-05-14T17:08:42.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.935+0000 7f27909da740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558320cc9b50 con 0x558320cb8130 2024-05-14T17:08:42.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.935+0000 7f27909da740 10 client.0 _ll_register_callbacks cb 0x558320bd9d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:08:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.935+0000 7f2787fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558320cb7810 0x558320cc6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.935+0000 7f2787fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558320cb7810 0x558320cc6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47078/0 (socket says 172.21.15.2:47078) 2024-05-14T17:08:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.935+0000 7f2787fff640 1 -- 172.21.15.2:0/3027237443 learned_addr learned my addr 172.21.15.2:0/3027237443 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:08:43.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.935+0000 7f2786ffd640 1 --2- 172.21.15.2:0/3027237443 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558320cb8130 0x558320cc3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.935+0000 7f2787fff640 1 --2- 172.21.15.2:0/3027237443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558320cc6ee0 0x558320cc40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.935+0000 7f2787fff640 1 -- 172.21.15.2:0/3027237443 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558320cb8130 msgr2=0x558320cc3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.935+0000 7f2787fff640 1 --2- 172.21.15.2:0/3027237443 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558320cb8130 0x558320cc3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.935+0000 7f2787fff640 1 -- 172.21.15.2:0/3027237443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558320cc6ee0 msgr2=0x558320cc40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:43.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.935+0000 7f2787fff640 1 --2- 172.21.15.2:0/3027237443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558320cc6ee0 0x558320cc40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.935+0000 7f2787fff640 1 -- 172.21.15.2:0/3027237443 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583209fb620 con 0x558320cb7810 2024-05-14T17:08:43.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.936+0000 7f2787fff640 1 --2- 172.21.15.2:0/3027237443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558320cb7810 0x558320cc6800 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f2780009620 tx=0x7f27800040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f56dd772a703ac7b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:43.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.936+0000 7f2766ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:08:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.936+0000 7f2766ffd640 1 -- 172.21.15.2:0/3027237443 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2780004b40 con 0x558320cb7810 2024-05-14T17:08:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.936+0000 7f2766ffd640 1 -- 172.21.15.2:0/3027237443 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f2780004ce0 con 0x558320cb7810 2024-05-14T17:08:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.936+0000 7f27909da740 1 -- 172.21.15.2:0/3027237443 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2780036040 con 0x558320cb7810 2024-05-14T17:08:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.936+0000 7f2766ffd640 1 -- 172.21.15.2:0/3027237443 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f278005d050 con 0x558320cb7810 2024-05-14T17:08:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.936+0000 7f27909da740 10 client.16656 fetch_fsmap learned FSMap version 46 2024-05-14T17:08:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.936+0000 7f27909da740 1 -- 172.21.15.2:0/3027237443 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x558320cd5590 con 0x558320cb7810 2024-05-14T17:08:44.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.936+0000 7f2766ffd640 1 -- 172.21.15.2:0/3027237443 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2780036040 con 0x558320cb7810 2024-05-14T17:08:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.936+0000 7f2766ffd640 1 -- 172.21.15.2:0/3027237443 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 46) ==== 8342+0+0 (secure 0 0 0) 0x7f2780030070 con 0x558320cb7810 2024-05-14T17:08:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.936+0000 7f27909da740 10 client.16656 fetch_fsmap finished waiting for FSMap version 46 2024-05-14T17:08:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.936+0000 7f27909da740 10 client.16656 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:08:44.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.936+0000 7f27909da740 20 client.16656 populate_metadata read hostname 'smithi002' 2024-05-14T17:08:44.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.937+0000 7f27909da740 1 --2- 172.21.15.2:0/3027237443 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x558320cd68e0 0x558320cd8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.937+0000 7f27909da740 4 client.16656 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:08:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.937+0000 7f27909da740 1 -- 172.21.15.2:0/3027237443 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558320c2cb80 con 0x558320cd68e0 2024-05-14T17:08:44.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.937+0000 7f2786ffd640 1 --2- 172.21.15.2:0/3027237443 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x558320cd68e0 0x558320cd8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:45.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.937+0000 7f2786ffd640 1 --2- 172.21.15.2:0/3027237443 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x558320cd68e0 0x558320cd8cd0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=698fc0611300f390 server_cookie=21241c478ee96876 in_seq=0 out_seq=0 2024-05-14T17:08:45.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.937+0000 7f2766ffd640 10 client.16656 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:08:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.938+0000 7f2766ffd640 1 -- 172.21.15.2:0/3027237443 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558320c2cb80 con 0x558320cd68e0 2024-05-14T17:08:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.938+0000 7f2766ffd640 10 client.16656 handle_command_reply: tid=0 2024-05-14T17:08:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.938+0000 7f27909da740 1 client.16656 shutdown 2024-05-14T17:08:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.938+0000 7f27909da740 1 -- 172.21.15.2:0/3027237443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558320cb7810 msgr2=0x558320cc6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:45.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.938+0000 7f27909da740 1 --2- 172.21.15.2:0/3027237443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558320cb7810 0x558320cc6800 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f2780009620 tx=0x7f27800040e0 comp rx=0 tx=0).stop 2024-05-14T17:08:45.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.938+0000 7f27909da740 1 -- 172.21.15.2:0/3027237443 shutdown_connections 2024-05-14T17:08:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.938+0000 7f27909da740 1 --2- 172.21.15.2:0/3027237443 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x558320cd68e0 0x558320cd8cd0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.938+0000 7f27909da740 1 --2- 172.21.15.2:0/3027237443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558320cc6ee0 0x558320cc40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.938+0000 7f27909da740 1 --2- 172.21.15.2:0/3027237443 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558320cb8130 0x558320cc3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:45.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.938+0000 7f2766ffd640 0 client.16656 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:08:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.938+0000 7f27909da740 1 --2- 172.21.15.2:0/3027237443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558320cb7810 0x558320cc6800 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:46.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.938+0000 7f27909da740 1 -- 172.21.15.2:0/3027237443 >> 172.21.15.2:0/3027237443 conn(0x558320cb3e00 msgr2=0x558320cb9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:46.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.938+0000 7f27909da740 1 -- 172.21.15.2:0/3027237443 shutdown_connections 2024-05-14T17:08:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.938+0000 7f27909da740 1 -- 172.21.15.2:0/3027237443 wait complete. 2024-05-14T17:08:46.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.938+0000 7f27909da740 20 client.16656 trim_cache size 0 max 16384 2024-05-14T17:08:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.954+0000 7f27909da740 1 Processor -- start 2024-05-14T17:08:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.954+0000 7f27909da740 1 -- start start 2024-05-14T17:08:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.954+0000 7f27909da740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558320cb7810 0x558320cd2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.954+0000 7f27909da740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558320cb8130 0x558320cc63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.954+0000 7f27909da740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558320cc6ee0 0x558320cde330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:46.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.954+0000 7f27909da740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f27881aed50 con 0x558320cb7810 2024-05-14T17:08:46.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.954+0000 7f27909da740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f27881ac5f0 con 0x558320cb8130 2024-05-14T17:08:46.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.954+0000 7f27909da740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f27881aa460 con 0x558320cc6ee0 2024-05-14T17:08:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.954+0000 7f2787fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558320cb7810 0x558320cd2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:47.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.954+0000 7f2787fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558320cb7810 0x558320cd2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47082/0 (socket says 172.21.15.2:47082) 2024-05-14T17:08:47.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.954+0000 7f2787fff640 1 -- 172.21.15.2:0/2029912358 learned_addr learned my addr 172.21.15.2:0/2029912358 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:08:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.954+0000 7f2786ffd640 1 --2- 172.21.15.2:0/2029912358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558320cb8130 0x558320cc63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:47.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.954+0000 7f2787fff640 1 -- 172.21.15.2:0/2029912358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558320cc6ee0 msgr2=0x558320cde330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:47.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.954+0000 7f2787fff640 1 --2- 172.21.15.2:0/2029912358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558320cc6ee0 0x558320cde330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:47.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.954+0000 7f2787fff640 1 -- 172.21.15.2:0/2029912358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558320cb8130 msgr2=0x558320cc63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.954+0000 7f2787fff640 1 --2- 172.21.15.2:0/2029912358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558320cb8130 0x558320cc63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:47.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.954+0000 7f2787fff640 1 -- 172.21.15.2:0/2029912358 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558320cb7f60 con 0x558320cb7810 2024-05-14T17:08:47.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.955+0000 7f2787fff640 1 --2- 172.21.15.2:0/2029912358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558320cb7810 0x558320cd2a20 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f2780009cd0 tx=0x7f2780031d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d9b185fec590f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:47.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.955+0000 7f276d7fa640 1 -- 172.21.15.2:0/2029912358 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278004a070 con 0x558320cb7810 2024-05-14T17:08:48.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.955+0000 7f276d7fa640 1 -- 172.21.15.2:0/2029912358 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2780036070 con 0x558320cb7810 2024-05-14T17:08:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.955+0000 7f276d7fa640 1 -- 172.21.15.2:0/2029912358 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27800305a0 con 0x558320cb7810 2024-05-14T17:08:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.955+0000 7f27909da740 1 -- 172.21.15.2:0/2029912358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558320cb7810 msgr2=0x558320cd2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.955+0000 7f27909da740 1 --2- 172.21.15.2:0/2029912358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558320cb7810 0x558320cd2a20 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f2780009cd0 tx=0x7f2780031d70 comp rx=0 tx=0).stop 2024-05-14T17:08:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.955+0000 7f27909da740 1 -- 172.21.15.2:0/2029912358 shutdown_connections 2024-05-14T17:08:48.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.955+0000 7f27909da740 1 --2- 172.21.15.2:0/2029912358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558320cc6ee0 0x558320cde330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:48.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.955+0000 7f27909da740 1 --2- 172.21.15.2:0/2029912358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558320cb8130 0x558320cc63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.955+0000 7f27909da740 1 --2- 172.21.15.2:0/2029912358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558320cb7810 0x558320cd2a20 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.955+0000 7f27909da740 1 -- 172.21.15.2:0/2029912358 >> 172.21.15.2:0/2029912358 conn(0x558320cb3e00 msgr2=0x558320cbf1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.955+0000 7f27909da740 1 -- 172.21.15.2:0/2029912358 shutdown_connections 2024-05-14T17:08:48.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.956+0000 7f27909da740 1 -- 172.21.15.2:0/2029912358 wait complete. 2024-05-14T17:08:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.956+0000 7f27909da740 1 -- start start 2024-05-14T17:08:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.956+0000 7f27909da740 1 Processor -- start 2024-05-14T17:08:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.956+0000 7f27909da740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558320cb7810 0x558320ccf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.956+0000 7f27909da740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558320cb8130 0x558320cca880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.956+0000 7f27909da740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558320cc6ee0 0x558320ccafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:49.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.956+0000 7f27909da740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f27881aaab0 con 0x558320cc6ee0 2024-05-14T17:08:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.956+0000 7f27909da740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f27881abf20 con 0x558320cb8130 2024-05-14T17:08:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.956+0000 7f27909da740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f27881ab180 con 0x558320cb7810 2024-05-14T17:08:49.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.956+0000 7f27909da740 10 client.0 _ll_register_callbacks cb 0x7f27800315b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:08:49.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.956+0000 7f2787fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558320cc6ee0 0x558320ccafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.956+0000 7f2787fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558320cb7810 0x558320ccf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.957+0000 7f2787fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558320cb7810 0x558320ccf4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:56900/0 (socket says 172.21.15.2:56900) 2024-05-14T17:08:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.957+0000 7f2787fff640 1 -- 172.21.15.2:0/58956308 learned_addr learned my addr 172.21.15.2:0/58956308 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:08:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.957+0000 7f2786ffd640 1 --2- 172.21.15.2:0/58956308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558320cb8130 0x558320cca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.957+0000 7f2787fff640 1 -- 172.21.15.2:0/58956308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558320cb8130 msgr2=0x558320cca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.957+0000 7f2787fff640 1 --2- 172.21.15.2:0/58956308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558320cb8130 0x558320cca880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.957+0000 7f2787fff640 1 -- 172.21.15.2:0/58956308 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558320cc6ee0 msgr2=0x558320ccafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.957+0000 7f2787fff640 1 --2- 172.21.15.2:0/58956308 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558320cc6ee0 0x558320ccafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:50.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.957+0000 7f2787fff640 1 -- 172.21.15.2:0/58956308 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558320d3fee0 con 0x558320cb7810 2024-05-14T17:08:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.957+0000 7f2786ffd640 1 --2- 172.21.15.2:0/58956308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558320cb8130 0x558320cca880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:08:50.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.957+0000 7f2787fff640 1 --2- 172.21.15.2:0/58956308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558320cb7810 0x558320ccf4c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2780030930 tx=0x7f278002a8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e524054fab24f4a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.957+0000 7f2766ffd640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-14T17:08:50.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.957+0000 7f2766ffd640 1 -- 172.21.15.2:0/58956308 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278004a030 con 0x558320cb7810 2024-05-14T17:08:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.957+0000 7f27909da740 1 -- 172.21.15.2:0/58956308 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2780036070 con 0x558320cb7810 2024-05-14T17:08:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.957+0000 7f2766ffd640 1 -- 172.21.15.2:0/58956308 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f2780046890 con 0x558320cb7810 2024-05-14T17:08:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.957+0000 7f2766ffd640 1 -- 172.21.15.2:0/58956308 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f2780066040 con 0x558320cb7810 2024-05-14T17:08:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.957+0000 7f27909da740 10 client.26278 fetch_fsmap learned FSMap version 46 2024-05-14T17:08:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.958+0000 7f27909da740 1 -- 172.21.15.2:0/58956308 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x558320d410b0 con 0x558320cb7810 2024-05-14T17:08:51.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.958+0000 7f2766ffd640 1 -- 172.21.15.2:0/58956308 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2780036070 con 0x558320cb7810 2024-05-14T17:08:51.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.958+0000 7f2766ffd640 1 -- 172.21.15.2:0/58956308 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 46) ==== 8342+0+0 (secure 0 0 0) 0x7f2780030040 con 0x558320cb7810 2024-05-14T17:08:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.958+0000 7f27909da740 10 client.26278 fetch_fsmap finished waiting for FSMap version 46 2024-05-14T17:08:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.958+0000 7f27909da740 10 client.26278 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:08:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.958+0000 7f27909da740 20 client.26278 populate_metadata read hostname 'smithi002' 2024-05-14T17:08:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.958+0000 7f27909da740 1 --2- 172.21.15.2:0/58956308 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x558320d41670 0x558320d43a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.958+0000 7f27909da740 4 client.26278 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:08:51.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.958+0000 7f27909da740 1 -- 172.21.15.2:0/58956308 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558320c26360 con 0x558320d41670 2024-05-14T17:08:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.958+0000 7f2786ffd640 1 --2- 172.21.15.2:0/58956308 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x558320d41670 0x558320d43a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.959+0000 7f2766ffd640 10 client.26278 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:08:51.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.959+0000 7f2786ffd640 1 --2- 172.21.15.2:0/58956308 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x558320d41670 0x558320d43a60 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3700b8b244b976d1 server_cookie=5b7ad3094b8041ae in_seq=0 out_seq=0 2024-05-14T17:08:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.964+0000 7f2766ffd640 1 -- 172.21.15.2:0/58956308 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558320c26360 con 0x558320d41670 2024-05-14T17:08:52.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.964+0000 7f2766ffd640 10 client.26278 handle_command_reply: tid=0 2024-05-14T17:08:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.964+0000 7f27909da740 1 client.26278 shutdown 2024-05-14T17:08:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.964+0000 7f27909da740 1 -- 172.21.15.2:0/58956308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558320cb7810 msgr2=0x558320ccf4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:52.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.964+0000 7f27909da740 1 --2- 172.21.15.2:0/58956308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558320cb7810 0x558320ccf4c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2780030930 tx=0x7f278002a8e0 comp rx=0 tx=0).stop 2024-05-14T17:08:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.964+0000 7f27909da740 1 -- 172.21.15.2:0/58956308 shutdown_connections 2024-05-14T17:08:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.964+0000 7f27909da740 1 --2- 172.21.15.2:0/58956308 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x558320d41670 0x558320d43a60 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.964+0000 7f27909da740 1 --2- 172.21.15.2:0/58956308 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558320cc6ee0 0x558320ccafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.964+0000 7f2766ffd640 0 client.26278 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:08:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.964+0000 7f27909da740 1 --2- 172.21.15.2:0/58956308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558320cb8130 0x558320cca880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:52.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.964+0000 7f27909da740 1 --2- 172.21.15.2:0/58956308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558320cb7810 0x558320ccf4c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:52.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.964+0000 7f27909da740 1 -- 172.21.15.2:0/58956308 >> 172.21.15.2:0/58956308 conn(0x558320cb3e00 msgr2=0x558320cb2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.964+0000 7f27909da740 1 -- 172.21.15.2:0/58956308 shutdown_connections 2024-05-14T17:08:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.964+0000 7f27909da740 1 -- 172.21.15.2:0/58956308 wait complete. 2024-05-14T17:08:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.964+0000 7f27909da740 20 client.26278 trim_cache size 0 max 16384 2024-05-14T17:08:53.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:08:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:08:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:08:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.965+0000 7f278dcc4640 1 -- 172.21.15.2:0/3275047171 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f27700b2a90 msgr2=0x7f27700b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:53.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.965+0000 7f278dcc4640 1 --2- 172.21.15.2:0/3275047171 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f27700b2a90 0x7f27700b4f50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f27780df8d0 tx=0x7f27780e4040 comp rx=0 tx=0).stop 2024-05-14T17:08:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.965+0000 7f278dcc4640 1 -- 172.21.15.2:0/3275047171 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2788102a50 msgr2=0x7f27881991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.965+0000 7f278dcc4640 1 --2- 172.21.15.2:0/3275047171 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2788102a50 0x7f27881991c0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f277800d770 tx=0x7f277800dc30 comp rx=0 tx=0).stop 2024-05-14T17:08:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.965+0000 7f278dcc4640 1 -- 172.21.15.2:0/3275047171 shutdown_connections 2024-05-14T17:08:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.965+0000 7f278dcc4640 1 --2- 172.21.15.2:0/3275047171 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f27700b2a90 0x7f27700b4f50 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:53.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.965+0000 7f278dcc4640 1 --2- 172.21.15.2:0/3275047171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2788108a60 0x7f278819da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.965+0000 7f278dcc4640 1 --2- 172.21.15.2:0/3275047171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2788103410 0x7f2788199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:53.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.965+0000 7f278dcc4640 1 --2- 172.21.15.2:0/3275047171 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2788102a50 0x7f27881991c0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:53.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.965+0000 7f278dcc4640 1 -- 172.21.15.2:0/3275047171 >> 172.21.15.2:0/3275047171 conn(0x7f27880fe520 msgr2=0x7f27880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.965+0000 7f278dcc4640 1 -- 172.21.15.2:0/3275047171 shutdown_connections 2024-05-14T17:08:54.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:28.965+0000 7f278dcc4640 1 -- 172.21.15.2:0/3275047171 wait complete. 2024-05-14T17:08:54.091 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:07:28 smithi002 ceph-mon[31538]: pgmap v398: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-05-14T17:08:54.152 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:07:30 smithi002 ceph-mon[31538]: pgmap v399: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-05-14T17:08:54.217 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 735.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-14T17:08:54.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:07:31 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:08:54.408 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:07:32 smithi002 ceph-mon[31538]: pgmap v400: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:08:54.440 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:07:34 smithi002 ceph-mon[31538]: pgmap v401: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.1 KiB/s wr, 7 op/s 2024-05-14T17:08:54.474 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 740.0 s, 44.6 tps, lat 58346.889 ms stddev 68259.436 2024-05-14T17:08:54.548 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:07:36 smithi002 ceph-mon[31538]: pgmap v402: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 KiB/s wr, 5 op/s 2024-05-14T17:08:54.656 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:08:54.744 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:07:38 smithi002 ceph-mon[31538]: pgmap v403: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.1 KiB/s wr, 5 op/s 2024-05-14T17:08:54.782 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:08:36 smithi123 ceph-mon[35971]: pgmap v434: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.2 MiB/s wr, 644 op/s 2024-05-14T17:08:54.842 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:08:38 smithi123 ceph-mon[35971]: pgmap v435: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.0 MiB/s wr, 530 op/s 2024-05-14T17:08:54.935 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:08:40 smithi123 ceph-mon[35971]: pgmap v436: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 7.3 MiB/s wr, 776 op/s 2024-05-14T17:08:55.103 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:08:42 smithi123 ceph-mon[35971]: pgmap v437: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 4.7 MiB/s wr, 510 op/s 2024-05-14T17:08:55.174 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:08:44 smithi123 ceph-mon[35971]: pgmap v438: 97 pgs: 97 active+clean; 4.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 6.8 MiB/s wr, 725 op/s 2024-05-14T17:08:55.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:08:46 smithi123 ceph-mon[35971]: pgmap v439: 97 pgs: 97 active+clean; 4.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 6.6 MiB/s wr, 639 op/s 2024-05-14T17:08:55.401 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:08:48 smithi123 ceph-mon[35971]: pgmap v440: 97 pgs: 97 active+clean; 4.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 6.2 MiB/s wr, 582 op/s 2024-05-14T17:08:55.466 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:08:50 smithi123 ceph-mon[35971]: pgmap v441: 97 pgs: 97 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 15 MiB/s wr, 939 op/s 2024-05-14T17:08:55.564 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:08:52 smithi123 ceph-mon[35971]: pgmap v442: 97 pgs: 97 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 12 MiB/s wr, 692 op/s 2024-05-14T17:08:55.699 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:08:36 smithi151 ceph-mon[36066]: pgmap v434: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.2 MiB/s wr, 644 op/s 2024-05-14T17:08:55.819 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:08:38 smithi151 ceph-mon[36066]: pgmap v435: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.0 MiB/s wr, 530 op/s 2024-05-14T17:08:55.879 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:08:40 smithi151 ceph-mon[36066]: pgmap v436: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 7.3 MiB/s wr, 776 op/s 2024-05-14T17:08:55.960 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:08:42 smithi151 ceph-mon[36066]: pgmap v437: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 4.7 MiB/s wr, 510 op/s 2024-05-14T17:08:55.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:08:44 smithi151 ceph-mon[36066]: pgmap v438: 97 pgs: 97 active+clean; 4.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 6.8 MiB/s wr, 725 op/s 2024-05-14T17:08:56.244 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:08:46 smithi151 ceph-mon[36066]: pgmap v439: 97 pgs: 97 active+clean; 4.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 6.6 MiB/s wr, 639 op/s 2024-05-14T17:08:56.315 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:08:48 smithi151 ceph-mon[36066]: pgmap v440: 97 pgs: 97 active+clean; 4.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 6.2 MiB/s wr, 582 op/s 2024-05-14T17:08:56.419 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:08:50 smithi151 ceph-mon[36066]: pgmap v441: 97 pgs: 97 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 15 MiB/s wr, 939 op/s 2024-05-14T17:08:56.465 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:08:52 smithi151 ceph-mon[36066]: pgmap v442: 97 pgs: 97 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 12 MiB/s wr, 692 op/s 2024-05-14T17:08:56.592 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:08:54 smithi123 ceph-mon[35971]: pgmap v443: 97 pgs: 97 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 18 MiB/s wr, 898 op/s 2024-05-14T17:08:56.704 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:08:54 smithi151 ceph-mon[36066]: pgmap v443: 97 pgs: 97 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 18 MiB/s wr, 898 op/s 2024-05-14T17:08:56.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.224+0000 7f3ba6b1b700 1 -- 172.21.15.2:0/445780849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ba010a540 msgr2=0x7f3ba010a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:56.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.224+0000 7f3ba6b1b700 1 --2- 172.21.15.2:0/445780849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ba010a540 0x7f3ba010a920 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3b94009990 tx=0x7f3b94009ca0 comp rx=0 tx=0).stop 2024-05-14T17:08:56.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.224+0000 7f3ba6b1b700 1 -- 172.21.15.2:0/445780849 shutdown_connections 2024-05-14T17:08:56.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.224+0000 7f3ba6b1b700 1 --2- 172.21.15.2:0/445780849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ba0116690 0x7f3ba0118a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:57.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.224+0000 7f3ba6b1b700 1 --2- 172.21.15.2:0/445780849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ba010a540 0x7f3ba010a920 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:57.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.224+0000 7f3ba6b1b700 1 --2- 172.21.15.2:0/445780849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ba010ae60 0x7f3ba0116150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:57.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.224+0000 7f3ba6b1b700 1 -- 172.21.15.2:0/445780849 >> 172.21.15.2:0/445780849 conn(0x7f3ba006e6b0 msgr2=0x7f3ba006eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:08:57.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.225+0000 7f3ba6b1b700 1 -- 172.21.15.2:0/445780849 shutdown_connections 2024-05-14T17:08:57.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.225+0000 7f3ba6b1b700 1 -- 172.21.15.2:0/445780849 wait complete. 2024-05-14T17:08:57.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.227+0000 7f3ba6b1b700 1 Processor -- start 2024-05-14T17:08:57.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.230+0000 7f3ba6b1b700 1 -- start start 2024-05-14T17:08:57.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.230+0000 7f3ba6b1b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ba010a540 0x7f3ba01ad6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:57.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.230+0000 7f3ba6b1b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ba010ae60 0x7f3ba01adbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:57.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.230+0000 7f3ba6b1b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ba0116690 0x7f3ba0115350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:57.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.230+0000 7f3ba6b1b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba011bf70 con 0x7f3ba010ae60 2024-05-14T17:08:57.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.230+0000 7f3ba6b1b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba011bdf0 con 0x7f3ba0116690 2024-05-14T17:08:57.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.230+0000 7f3ba6b1b700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba011c0f0 con 0x7f3ba010a540 2024-05-14T17:08:57.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.231+0000 7f3ba50b8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ba0116690 0x7f3ba0115350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:58.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.234+0000 7f3b9ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ba010ae60 0x7f3ba01adbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:58.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.234+0000 7f3b9ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ba010ae60 0x7f3ba01adbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39936/0 (socket says 172.21.15.2:39936) 2024-05-14T17:08:58.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.234+0000 7f3b9ffff700 1 -- 172.21.15.2:0/604334412 learned_addr learned my addr 172.21.15.2:0/604334412 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:08:58.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.236+0000 7f3ba48b7700 1 --2- 172.21.15.2:0/604334412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ba010a540 0x7f3ba01ad6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:58.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.239+0000 7f3b9ffff700 1 -- 172.21.15.2:0/604334412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ba010a540 msgr2=0x7f3ba01ad6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:58.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.239+0000 7f3b9ffff700 1 --2- 172.21.15.2:0/604334412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ba010a540 0x7f3ba01ad6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:58.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.239+0000 7f3b9ffff700 1 -- 172.21.15.2:0/604334412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ba0116690 msgr2=0x7f3ba0115350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:08:58.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.239+0000 7f3b9ffff700 1 --2- 172.21.15.2:0/604334412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ba0116690 0x7f3ba0115350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:58.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.239+0000 7f3b9ffff700 1 -- 172.21.15.2:0/604334412 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ba0115ae0 con 0x7f3ba010ae60 2024-05-14T17:08:58.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.246+0000 7f3b9ffff700 1 --2- 172.21.15.2:0/604334412 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ba010ae60 0x7f3ba01adbf0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c00e9c0 tx=0x7f3b8c00ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:58.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.250+0000 7f3b9dffb700 1 -- 172.21.15.2:0/604334412 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c00cbd0 con 0x7f3ba010ae60 2024-05-14T17:08:58.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.250+0000 7f3b9dffb700 1 -- 172.21.15.2:0/604334412 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b8c00cd70 con 0x7f3ba010ae60 2024-05-14T17:08:58.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.250+0000 7f3b9dffb700 1 -- 172.21.15.2:0/604334412 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c010430 con 0x7f3ba010ae60 2024-05-14T17:08:58.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.252+0000 7f3ba6b1b700 1 -- 172.21.15.2:0/604334412 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ba0115dd0 con 0x7f3ba010ae60 2024-05-14T17:08:58.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.252+0000 7f3ba6b1b700 1 -- 172.21.15.2:0/604334412 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ba01b55e0 con 0x7f3ba010ae60 2024-05-14T17:08:58.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.257+0000 7f3bb13b1b80 1 Processor -- start 2024-05-14T17:08:58.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.259+0000 7f3bb13b1b80 1 -- start start 2024-05-14T17:08:58.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.259+0000 7f3bb13b1b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10692d420 0x55d10692d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:59.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.259+0000 7f3bb13b1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d10692dd40 0x55d106928ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:59.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.259+0000 7f3bb13b1b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1069292c0 0x55d106929720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:08:59.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.259+0000 7f3bb13b1b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d106886e70 con 0x55d1069292c0 2024-05-14T17:08:59.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.259+0000 7f3bb13b1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d106876a10 con 0x55d10692dd40 2024-05-14T17:08:59.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.259+0000 7f3bb13b1b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d10687aa50 con 0x55d10692d420 2024-05-14T17:08:59.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.260+0000 7f3ba48b7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d10692dd40 0x55d106928ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:59.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.260+0000 7f3ba50b8700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10692d420 0x55d10692d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:08:59.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.260+0000 7f3ba50b8700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10692d420 0x55d10692d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:48050/0 (socket says 172.21.15.2:48050) 2024-05-14T17:08:59.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.260+0000 7f3ba50b8700 1 -- 172.21.15.2:0/2060443829 learned_addr learned my addr 172.21.15.2:0/2060443829 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:08:59.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.261+0000 7f3ba50b8700 1 -- 172.21.15.2:0/2060443829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d10692dd40 msgr2=0x55d106928ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:59.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.261+0000 7f3ba50b8700 1 --2- 172.21.15.2:0/2060443829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d10692dd40 0x55d106928ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:59.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.261+0000 7f3ba50b8700 1 -- 172.21.15.2:0/2060443829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1069292c0 msgr2=0x55d106929720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:08:59.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.261+0000 7f3ba50b8700 1 --2- 172.21.15.2:0/2060443829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1069292c0 0x55d106929720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:08:59.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.261+0000 7f3ba50b8700 1 -- 172.21.15.2:0/2060443829 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1068200b0 con 0x55d10692d420 2024-05-14T17:08:59.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.262+0000 7f3ba50b8700 1 --2- 172.21.15.2:0/2060443829 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10692d420 0x55d10692d800 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3b98009d20 tx=0x7f3b980077e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38ca2bdf5ee4caa8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:08:59.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.262+0000 7f3b9dffb700 1 -- 172.21.15.2:0/604334412 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f3b8c004d70 con 0x7f3ba010ae60 2024-05-14T17:09:00.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.264+0000 7f3b9dffb700 1 --2- 172.21.15.2:0/604334412 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f3b900b2bc0 0x7f3b900b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:00.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.264+0000 7f3b9dffb700 1 -- 172.21.15.2:0/604334412 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f3b8c0e4b60 con 0x7f3ba010ae60 2024-05-14T17:09:00.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.265+0000 7f3ba48b7700 1 --2- 172.21.15.2:0/604334412 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f3b900b2bc0 0x7f3b900b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:00.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.265+0000 7f3ba48b7700 1 --2- 172.21.15.2:0/604334412 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f3b900b2bc0 0x7f3b900b5080 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x55d106924060 tx=0x7f3b9401a040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:00.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.267+0000 7f3b8affd700 1 -- 172.21.15.2:0/2060443829 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b98004bb0 con 0x55d10692d420 2024-05-14T17:09:00.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.267+0000 7f3b8affd700 1 -- 172.21.15.2:0/2060443829 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b98004d50 con 0x55d10692d420 2024-05-14T17:09:00.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.267+0000 7f3b8affd700 1 -- 172.21.15.2:0/2060443829 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b98014670 con 0x55d10692d420 2024-05-14T17:09:00.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.268+0000 7f3bb13b1b80 1 -- 172.21.15.2:0/2060443829 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10692d420 msgr2=0x55d10692d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:00.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.268+0000 7f3bb13b1b80 1 --2- 172.21.15.2:0/2060443829 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10692d420 0x55d10692d800 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3b98009d20 tx=0x7f3b980077e0 comp rx=0 tx=0).stop 2024-05-14T17:09:00.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.269+0000 7f3bb13b1b80 1 -- 172.21.15.2:0/2060443829 shutdown_connections 2024-05-14T17:09:01.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.269+0000 7f3bb13b1b80 1 --2- 172.21.15.2:0/2060443829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1069292c0 0x55d106929720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:01.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.269+0000 7f3bb13b1b80 1 --2- 172.21.15.2:0/2060443829 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10692d420 0x55d10692d800 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:01.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.269+0000 7f3bb13b1b80 1 --2- 172.21.15.2:0/2060443829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d10692dd40 0x55d106928ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:01.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.269+0000 7f3bb13b1b80 1 -- 172.21.15.2:0/2060443829 >> 172.21.15.2:0/2060443829 conn(0x55d106855eb0 msgr2=0x55d1069234a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:01.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.270+0000 7f3bb13b1b80 1 -- 172.21.15.2:0/2060443829 shutdown_connections 2024-05-14T17:09:01.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.274+0000 7f3bb13b1b80 1 -- 172.21.15.2:0/2060443829 wait complete. 2024-05-14T17:09:01.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.274+0000 7f3bb13b1b80 1 -- start start 2024-05-14T17:09:01.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.274+0000 7f3bb13b1b80 1 Processor -- start 2024-05-14T17:09:01.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.276+0000 7f3bb13b1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d1069292c0 0x55d106934be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:01.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.276+0000 7f3bb13b1b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d10692d420 0x55d106931f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:01.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.276+0000 7f3bb13b1b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10692dd40 0x55d1069324c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:01.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.276+0000 7f3bb13b1b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d10677d0a0 con 0x55d10692d420 2024-05-14T17:09:01.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.276+0000 7f3bb13b1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d1067deea0 con 0x55d1069292c0 2024-05-14T17:09:02.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.276+0000 7f3bb13b1b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d106946890 con 0x55d10692dd40 2024-05-14T17:09:02.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.276+0000 7f3bb13b1b80 10 client.0 _ll_register_callbacks cb 0x55d106811a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:09:02.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.276+0000 7f3ba50b8700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10692dd40 0x55d1069324c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:02.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.276+0000 7f3ba50b8700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10692dd40 0x55d1069324c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:48062/0 (socket says 172.21.15.2:48062) 2024-05-14T17:09:02.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.277+0000 7f3ba50b8700 1 -- 172.21.15.2:0/2119728917 learned_addr learned my addr 172.21.15.2:0/2119728917 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:09:02.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.277+0000 7f3b9ffff700 1 --2- 172.21.15.2:0/2119728917 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d10692d420 0x55d106931f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:02.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.277+0000 7f3ba50b8700 1 -- 172.21.15.2:0/2119728917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d1069292c0 msgr2=0x55d106934be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:02.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.277+0000 7f3ba50b8700 1 --2- 172.21.15.2:0/2119728917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d1069292c0 0x55d106934be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:02.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.277+0000 7f3ba50b8700 1 -- 172.21.15.2:0/2119728917 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d10692d420 msgr2=0x55d106931f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:02.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.277+0000 7f3ba50b8700 1 --2- 172.21.15.2:0/2119728917 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d10692d420 0x55d106931f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:02.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.277+0000 7f3ba50b8700 1 -- 172.21.15.2:0/2119728917 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1067c69d0 con 0x55d10692dd40 2024-05-14T17:09:02.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.277+0000 7f3b9ffff700 1 --2- 172.21.15.2:0/2119728917 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d10692d420 0x55d106931f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:09:03.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.277+0000 7f3ba50b8700 1 --2- 172.21.15.2:0/2119728917 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10692dd40 0x55d1069324c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3b98005980 tx=0x7f3b980056c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2901b8873bed1962 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:03.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.277+0000 7f3b7ffff700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-14T17:09:03.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.279+0000 7f3b7ffff700 1 -- 172.21.15.2:0/2119728917 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b98022020 con 0x55d10692dd40 2024-05-14T17:09:03.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.279+0000 7f3b7ffff700 1 -- 172.21.15.2:0/2119728917 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f3b98004ad0 con 0x55d10692dd40 2024-05-14T17:09:03.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.280+0000 7f3bb13b1b80 1 -- 172.21.15.2:0/2119728917 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b98004d50 con 0x55d10692dd40 2024-05-14T17:09:03.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.280+0000 7f3b7ffff700 1 -- 172.21.15.2:0/2119728917 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f3b9801e420 con 0x55d10692dd40 2024-05-14T17:09:03.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.280+0000 7f3bb13b1b80 10 client.26290 fetch_fsmap learned FSMap version 47 2024-05-14T17:09:03.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.280+0000 7f3bb13b1b80 1 -- 172.21.15.2:0/2119728917 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55d106830860 con 0x55d10692dd40 2024-05-14T17:09:03.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.283+0000 7f3b7ffff700 1 -- 172.21.15.2:0/2119728917 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b98004d50 con 0x55d10692dd40 2024-05-14T17:09:03.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.283+0000 7f3b7ffff700 1 -- 172.21.15.2:0/2119728917 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 47) ==== 8338+0+0 (secure 0 0 0) 0x7f3b9802f020 con 0x55d10692dd40 2024-05-14T17:09:03.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.284+0000 7f3bb13b1b80 10 client.26290 fetch_fsmap finished waiting for FSMap version 47 2024-05-14T17:09:03.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.284+0000 7f3bb13b1b80 10 client.26290 resolve_mds: validated gid 24479 aka daemon mds.b 2024-05-14T17:09:03.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.284+0000 7f3bb13b1b80 20 client.26290 populate_metadata read hostname 'smithi002' 2024-05-14T17:09:03.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.284+0000 7f3bb13b1b80 1 --2- 172.21.15.2:0/2119728917 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d106953720 0x55d106955b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:04.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.284+0000 7f3bb13b1b80 4 client.26290 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:09:04.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.284+0000 7f3bb13b1b80 1 -- 172.21.15.2:0/2119728917 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d106888dc0 con 0x55d106953720 2024-05-14T17:09:04.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.285+0000 7f3ba48b7700 1 --2- 172.21.15.2:0/2119728917 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d106953720 0x55d106955b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:04.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.286+0000 7f3ba48b7700 1 --2- 172.21.15.2:0/2119728917 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d106953720 0x55d106955b60 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eadc7db4b817912 server_cookie=83408225f3416329 in_seq=0 out_seq=0 2024-05-14T17:09:04.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.286+0000 7f3b7ffff700 10 client.26290 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:09:04.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.287+0000 7f3b7ffff700 1 -- 172.21.15.2:0/2119728917 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d106888dc0 con 0x55d106953720 2024-05-14T17:09:04.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.287+0000 7f3b7ffff700 10 client.26290 handle_command_reply: tid=0 2024-05-14T17:09:04.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.287+0000 7f3bb13b1b80 1 client.26290 shutdown 2024-05-14T17:09:04.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.290+0000 7f3bb13b1b80 1 -- 172.21.15.2:0/2119728917 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10692dd40 msgr2=0x55d1069324c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:04.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.290+0000 7f3bb13b1b80 1 --2- 172.21.15.2:0/2119728917 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10692dd40 0x55d1069324c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3b98005980 tx=0x7f3b980056c0 comp rx=0 tx=0).stop 2024-05-14T17:09:04.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.290+0000 7f3bb13b1b80 1 -- 172.21.15.2:0/2119728917 shutdown_connections 2024-05-14T17:09:04.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.290+0000 7f3bb13b1b80 1 --2- 172.21.15.2:0/2119728917 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d106953720 0x55d106955b60 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:04.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.290+0000 7f3bb13b1b80 1 --2- 172.21.15.2:0/2119728917 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10692dd40 0x55d1069324c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:04.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.290+0000 7f3bb13b1b80 1 --2- 172.21.15.2:0/2119728917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d1069292c0 0x55d106934be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:04.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.290+0000 7f3bb13b1b80 1 --2- 172.21.15.2:0/2119728917 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d10692d420 0x55d106931f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:04.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.290+0000 7f3bb13b1b80 1 -- 172.21.15.2:0/2119728917 >> 172.21.15.2:0/2119728917 conn(0x55d106855eb0 msgr2=0x55d10691dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:05.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.290+0000 7f3b7ffff700 0 client.26290 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:09:05.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.290+0000 7f3bb13b1b80 1 -- 172.21.15.2:0/2119728917 shutdown_connections 2024-05-14T17:09:05.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.291+0000 7f3bb13b1b80 1 -- 172.21.15.2:0/2119728917 wait complete. 2024-05-14T17:09:05.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.291+0000 7f3bb13b1b80 20 client.26290 trim_cache size 0 max 16384 2024-05-14T17:09:05.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.329+0000 7f3bb13b1b80 1 Processor -- start 2024-05-14T17:09:05.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.330+0000 7f3bb13b1b80 1 -- start start 2024-05-14T17:09:05.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.330+0000 7f3bb13b1b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1069292c0 0x55d10691e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:05.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.330+0000 7f3bb13b1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d10692d420 0x55d10692c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:05.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.330+0000 7f3bb13b1b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10692dd40 0x55d10692f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:05.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.330+0000 7f3bb13b1b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba01b6e50 con 0x55d1069292c0 2024-05-14T17:09:05.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.330+0000 7f3bb13b1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba01a8ab0 con 0x55d10692d420 2024-05-14T17:09:05.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.330+0000 7f3bb13b1b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba01a6920 con 0x55d10692dd40 2024-05-14T17:09:05.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.331+0000 7f3ba50b8700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10692dd40 0x55d10692f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:06.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.331+0000 7f3ba50b8700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10692dd40 0x55d10692f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:48074/0 (socket says 172.21.15.2:48074) 2024-05-14T17:09:06.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.331+0000 7f3ba50b8700 1 -- 172.21.15.2:0/3736322241 learned_addr learned my addr 172.21.15.2:0/3736322241 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:09:06.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.331+0000 7f3ba48b7700 1 --2- 172.21.15.2:0/3736322241 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1069292c0 0x55d10691e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:06.255 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.331+0000 7f3b9ffff700 1 --2- 172.21.15.2:0/3736322241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d10692d420 0x55d10692c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:06.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.331+0000 7f3b9ffff700 1 -- 172.21.15.2:0/3736322241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10692dd40 msgr2=0x55d10692f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:06.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.331+0000 7f3b9ffff700 1 --2- 172.21.15.2:0/3736322241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10692dd40 0x55d10692f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:06.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.331+0000 7f3b9ffff700 1 -- 172.21.15.2:0/3736322241 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1069292c0 msgr2=0x55d10691e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:06.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.331+0000 7f3b9ffff700 1 --2- 172.21.15.2:0/3736322241 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1069292c0 0x55d10691e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:06.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.331+0000 7f3b9ffff700 1 -- 172.21.15.2:0/3736322241 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1069304f0 con 0x55d10692d420 2024-05-14T17:09:06.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.332+0000 7f3b9ffff700 1 --2- 172.21.15.2:0/3736322241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d10692d420 0x55d10692c7f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c0e2250 tx=0x7f3b8c0ab550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99b69047da9cf607 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:06.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.332+0000 7f3b8a7fc700 1 -- 172.21.15.2:0/3736322241 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c0af070 con 0x55d10692d420 2024-05-14T17:09:06.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.333+0000 7f3b8a7fc700 1 -- 172.21.15.2:0/3736322241 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b8c0ab8f0 con 0x55d10692d420 2024-05-14T17:09:06.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.333+0000 7f3b8a7fc700 1 -- 172.21.15.2:0/3736322241 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c0b47b0 con 0x55d10692d420 2024-05-14T17:09:07.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.335+0000 7f3bb13b1b80 1 -- 172.21.15.2:0/3736322241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d10692d420 msgr2=0x55d10692c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:07.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.335+0000 7f3bb13b1b80 1 --2- 172.21.15.2:0/3736322241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d10692d420 0x55d10692c7f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c0e2250 tx=0x7f3b8c0ab550 comp rx=0 tx=0).stop 2024-05-14T17:09:07.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.335+0000 7f3bb13b1b80 1 -- 172.21.15.2:0/3736322241 shutdown_connections 2024-05-14T17:09:07.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.335+0000 7f3bb13b1b80 1 --2- 172.21.15.2:0/3736322241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10692dd40 0x55d10692f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:07.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.335+0000 7f3bb13b1b80 1 --2- 172.21.15.2:0/3736322241 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1069292c0 0x55d10691e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:07.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.335+0000 7f3bb13b1b80 1 --2- 172.21.15.2:0/3736322241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d10692d420 0x55d10692c7f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:07.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.336+0000 7f3bb13b1b80 1 -- 172.21.15.2:0/3736322241 >> 172.21.15.2:0/3736322241 conn(0x55d106855eb0 msgr2=0x55d10692c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:07.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.337+0000 7f3bb13b1b80 1 -- 172.21.15.2:0/3736322241 shutdown_connections 2024-05-14T17:09:07.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.337+0000 7f3bb13b1b80 1 -- 172.21.15.2:0/3736322241 wait complete. 2024-05-14T17:09:07.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.337+0000 7f3bb13b1b80 1 -- start start 2024-05-14T17:09:07.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.337+0000 7f3bb13b1b80 1 Processor -- start 2024-05-14T17:09:07.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.338+0000 7f3bb13b1b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1069292c0 0x55d106959780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:07.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.338+0000 7f3bb13b1b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10692d420 0x55d106959cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:07.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.338+0000 7f3bb13b1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d10692dd40 0x55d1069b6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:08.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.338+0000 7f3bb13b1b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba01a6f70 con 0x55d1069292c0 2024-05-14T17:09:08.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.338+0000 7f3bb13b1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba01a83e0 con 0x55d10692dd40 2024-05-14T17:09:08.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.338+0000 7f3bb13b1b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba01a7640 con 0x55d10692d420 2024-05-14T17:09:08.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.338+0000 7f3b9ffff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10692d420 0x55d106959cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:08.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.338+0000 7f3b9ffff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10692d420 0x55d106959cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:48076/0 (socket says 172.21.15.2:48076) 2024-05-14T17:09:08.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.338+0000 7f3b9ffff700 1 -- 172.21.15.2:0/2766272892 learned_addr learned my addr 172.21.15.2:0/2766272892 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:09:08.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.338+0000 7f3ba50b8700 1 --2- 172.21.15.2:0/2766272892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d10692dd40 0x55d1069b6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:08.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.338+0000 7f3ba48b7700 1 --2- 172.21.15.2:0/2766272892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1069292c0 0x55d106959780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:08.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.338+0000 7f3bb13b1b80 10 client.0 _ll_register_callbacks cb 0x55d106811a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:09:08.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.338+0000 7f3b9ffff700 1 -- 172.21.15.2:0/2766272892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d10692dd40 msgr2=0x55d1069b6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:08.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.338+0000 7f3b9ffff700 1 --2- 172.21.15.2:0/2766272892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d10692dd40 0x55d1069b6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:08.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.338+0000 7f3b9ffff700 1 -- 172.21.15.2:0/2766272892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1069292c0 msgr2=0x55d106959780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:08.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.338+0000 7f3b9ffff700 1 --2- 172.21.15.2:0/2766272892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1069292c0 0x55d106959780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:08.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.338+0000 7f3b9ffff700 1 -- 172.21.15.2:0/2766272892 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1069b7570 con 0x55d10692d420 2024-05-14T17:09:09.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.339+0000 7f3b9ffff700 1 --2- 172.21.15.2:0/2766272892 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10692d420 0x55d106959cc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c00c5b0 tx=0x7f3b8c005870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=471907f85c518536 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:09.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.339+0000 7f3b7ffff700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-14T17:09:09.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.339+0000 7f3b7ffff700 1 -- 172.21.15.2:0/2766272892 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c0b9060 con 0x55d10692d420 2024-05-14T17:09:09.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.339+0000 7f3b7ffff700 1 -- 172.21.15.2:0/2766272892 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f3b8c0b49c0 con 0x55d10692d420 2024-05-14T17:09:09.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.340+0000 7f3bb13b1b80 1 -- 172.21.15.2:0/2766272892 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b8c0ab8f0 con 0x55d10692d420 2024-05-14T17:09:09.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.340+0000 7f3b7ffff700 1 -- 172.21.15.2:0/2766272892 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f3b8c0af040 con 0x55d10692d420 2024-05-14T17:09:09.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.342+0000 7f3bb13b1b80 10 client.26296 fetch_fsmap learned FSMap version 47 2024-05-14T17:09:09.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.342+0000 7f3bb13b1b80 1 -- 172.21.15.2:0/2766272892 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55d1069be8e0 con 0x55d10692d420 2024-05-14T17:09:09.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.342+0000 7f3b7ffff700 1 -- 172.21.15.2:0/2766272892 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b8c0b0190 con 0x55d10692d420 2024-05-14T17:09:09.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.342+0000 7f3b7ffff700 1 -- 172.21.15.2:0/2766272892 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 47) ==== 8338+0+0 (secure 0 0 0) 0x7f3b8c0aabc0 con 0x55d10692d420 2024-05-14T17:09:09.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.343+0000 7f3bb13b1b80 10 client.26296 fetch_fsmap finished waiting for FSMap version 47 2024-05-14T17:09:09.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.343+0000 7f3bb13b1b80 10 client.26296 resolve_mds: validated gid 24479 aka daemon mds.b 2024-05-14T17:09:09.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.343+0000 7f3bb13b1b80 20 client.26296 populate_metadata read hostname 'smithi002' 2024-05-14T17:09:09.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.343+0000 7f3bb13b1b80 1 --2- 172.21.15.2:0/2766272892 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d1069bee30 0x55d1069c1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:09.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.343+0000 7f3bb13b1b80 4 client.26296 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "e0c7677c", "release": true, "await": true}] 2024-05-14T17:09:10.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.343+0000 7f3bb13b1b80 1 -- 172.21.15.2:0/2766272892 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "quiesce db", "set_id": "e0c7677c", "release": true, "await": true}) -- 0x7f3b8c0ab8f0 con 0x55d1069bee30 2024-05-14T17:09:10.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.343+0000 7f3ba48b7700 1 --2- 172.21.15.2:0/2766272892 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d1069bee30 0x55d1069c1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:10.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.344+0000 7f3ba48b7700 1 --2- 172.21.15.2:0/2766272892 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d1069bee30 0x55d1069c1250 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79d52429c91fb75b server_cookie=97f97040128d99ee in_seq=0 out_seq=0 2024-05-14T17:09:10.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.344+0000 7f3b7ffff700 10 client.26296 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:09:10.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.344+0000 7f3b7ffff700 1 -- 172.21.15.2:0/2766272892 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: -1 ) ==== 8+0+592 (crc 0 0 0) 0x7f3b8c0ab8f0 con 0x55d1069bee30 2024-05-14T17:09:10.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.344+0000 7f3b7ffff700 10 client.26296 handle_command_reply: tid=0 2024-05-14T17:09:10.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.344+0000 7f3bb13b1b80 1 client.26296 shutdown 2024-05-14T17:09:10.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.345+0000 7f3bb13b1b80 1 -- 172.21.15.2:0/2766272892 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10692d420 msgr2=0x55d106959cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:10.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.345+0000 7f3bb13b1b80 1 --2- 172.21.15.2:0/2766272892 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10692d420 0x55d106959cc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c00c5b0 tx=0x7f3b8c005870 comp rx=0 tx=0).stop 2024-05-14T17:09:10.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.345+0000 7f3bb13b1b80 1 -- 172.21.15.2:0/2766272892 shutdown_connections 2024-05-14T17:09:10.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.345+0000 7f3bb13b1b80 1 --2- 172.21.15.2:0/2766272892 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d1069bee30 0x55d1069c1250 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:11.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.345+0000 7f3bb13b1b80 1 --2- 172.21.15.2:0/2766272892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d10692dd40 0x55d1069b6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:11.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.345+0000 7f3b7ffff700 0 client.26296 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:09:11.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.345+0000 7f3bb13b1b80 1 --2- 172.21.15.2:0/2766272892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d1069292c0 0x55d106959780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:11.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.345+0000 7f3bb13b1b80 1 --2- 172.21.15.2:0/2766272892 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d10692d420 0x55d106959cc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:11.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.346+0000 7f3bb13b1b80 1 -- 172.21.15.2:0/2766272892 >> 172.21.15.2:0/2766272892 conn(0x55d106855eb0 msgr2=0x55d10692c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:11.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.346+0000 7f3bb13b1b80 1 -- 172.21.15.2:0/2766272892 shutdown_connections 2024-05-14T17:09:11.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.346+0000 7f3bb13b1b80 1 -- 172.21.15.2:0/2766272892 wait complete. 2024-05-14T17:09:11.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:07:40.346+0000 7f3bb13b1b80 20 client.26296 trim_cache size 0 max 16384 2024-05-14T17:09:11.494 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T17:09:11.650 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 47, 2024-05-14T17:09:11.751 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24479, 2024-05-14T17:09:11.779 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 35, 2024-05-14T17:09:11.812 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": { 2024-05-14T17:09:11.869 INFO:teuthology.orchestra.run.smithi002.stdout: "e0c7677c": { 2024-05-14T17:09:11.951 INFO:teuthology.orchestra.run.smithi002.stdout: "version": 35, 2024-05-14T17:09:12.016 INFO:teuthology.orchestra.run.smithi002.stdout: "age_ref": 3.8, 2024-05-14T17:09:12.148 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-05-14T17:09:12.238 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "EXPIRED", 2024-05-14T17:09:12.300 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0 2024-05-14T17:09:12.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:09:12.456 INFO:teuthology.orchestra.run.smithi002.stdout: "timeout": 90.0, 2024-05-14T17:09:12.488 INFO:teuthology.orchestra.run.smithi002.stdout: "expiration": 137.9, 2024-05-14T17:09:12.563 INFO:teuthology.orchestra.run.smithi002.stdout: "members": { 2024-05-14T17:09:12.646 INFO:teuthology.orchestra.run.smithi002.stdout: "file:/": { 2024-05-14T17:09:12.782 INFO:teuthology.orchestra.run.smithi002.stdout: "excluded": false, 2024-05-14T17:09:12.854 INFO:teuthology.orchestra.run.smithi002.stdout: "state": { 2024-05-14T17:09:12.936 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "QUIESCED", 2024-05-14T17:09:13.004 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 137.9 2024-05-14T17:09:13.082 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:09:13.143 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:09:13.174 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:09:13.207 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:09:13.241 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:09:13.274 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T17:09:13.333 INFO:teuthology.orchestra.run.smithi002.stderr:Error EPERM: 2024-05-14T17:09:13.378 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:07:40 smithi002 ceph-mon[31538]: pgmap v404: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 1.2 MiB/s wr, 94 op/s 2024-05-14T17:09:13.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T17:09:13.554 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 745.0 s, 1951.0 tps, lat 136.084 ms stddev 3435.535 2024-05-14T17:09:13.747 ERROR:tasks.quiescer.fs.[cephfs]:Couldn't release set 'e0c7677c' with rc: 1 (EPERM), stdout: { "epoch": 47, "leader": 24479, "set_version": 35, "sets": { "e0c7677c": { "version": 35, "age_ref": 3.8, "state": { "name": "EXPIRED", "age": 0.0 }, "timeout": 90.0, "expiration": 137.9, "members": { "file:/": { "excluded": false, "state": { "name": "QUIESCED", "age": 137.9 } } } } } } 2024-05-14T17:09:13.823 ERROR:tasks.quiescer.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/qa/tasks/quiescer.py", line 354, in _run self.do_quiesce(d) File "/home/teuthworker/src/git.ceph.com_ceph-c_9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/qa/tasks/quiescer.py", line 330, in do_quiesce raise RuntimeError(f"Error releasing set '{set_id}': {rcinfo(rc)}") RuntimeError: Error releasing set 'e0c7677c': 1 (EPERM) 2024-05-14T17:09:13.905 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:08:56 smithi123 ceph-mon[35971]: pgmap v444: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 17 MiB/s wr, 754 op/s 2024-05-14T17:09:13.971 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:08:58 smithi123 ceph-mon[35971]: pgmap v445: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 16 MiB/s wr, 633 op/s 2024-05-14T17:09:14.060 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:09:00 smithi123 ceph-mon[35971]: pgmap v446: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 22 MiB/s wr, 904 op/s 2024-05-14T17:09:14.128 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:09:02 smithi123 ceph-mon[35971]: pgmap v447: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 547 op/s 2024-05-14T17:09:14.211 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:09:04 smithi123 ceph-mon[35971]: pgmap v448: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 17 MiB/s wr, 708 op/s 2024-05-14T17:09:14.271 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:09:06 smithi123 ceph-mon[35971]: pgmap v449: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 544 op/s 2024-05-14T17:09:14.338 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:09:07 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:09:14.517 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:09:08 smithi123 ceph-mon[35971]: pgmap v450: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 10 MiB/s wr, 472 op/s 2024-05-14T17:09:14.563 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:09:08 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:09:14.626 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:09:08 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:09:14.658 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:09:08 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:09:14.691 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:09:10 smithi123 ceph-mon[35971]: osdmap e96: 12 total, 12 up, 12 in 2024-05-14T17:09:14.724 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:09:10 smithi123 ceph-mon[35971]: pgmap v452: 97 pgs: 97 active+clean; 5.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 30 MiB/s wr, 1.91k op/s 2024-05-14T17:09:14.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:09:12 smithi123 ceph-mon[35971]: pgmap v453: 97 pgs: 97 active+clean; 5.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 30 MiB/s wr, 1.91k op/s 2024-05-14T17:09:14.800 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:08:56 smithi151 ceph-mon[36066]: pgmap v444: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 17 MiB/s wr, 754 op/s 2024-05-14T17:09:14.879 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:08:58 smithi151 ceph-mon[36066]: pgmap v445: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 16 MiB/s wr, 633 op/s 2024-05-14T17:09:14.935 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:09:00 smithi151 ceph-mon[36066]: pgmap v446: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 22 MiB/s wr, 904 op/s 2024-05-14T17:09:14.991 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:09:02 smithi151 ceph-mon[36066]: pgmap v447: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 547 op/s 2024-05-14T17:09:15.080 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:09:04 smithi151 ceph-mon[36066]: pgmap v448: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 17 MiB/s wr, 708 op/s 2024-05-14T17:09:15.124 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:09:06 smithi151 ceph-mon[36066]: pgmap v449: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 544 op/s 2024-05-14T17:09:15.199 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:09:07 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:09:15.294 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:09:08 smithi151 ceph-mon[36066]: pgmap v450: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 10 MiB/s wr, 472 op/s 2024-05-14T17:09:15.382 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:09:08 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:09:15.489 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:09:08 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:09:15.525 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:09:08 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:09:15.559 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:09:10 smithi151 ceph-mon[36066]: osdmap e96: 12 total, 12 up, 12 in 2024-05-14T17:09:15.639 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:09:10 smithi151 ceph-mon[36066]: pgmap v452: 97 pgs: 97 active+clean; 5.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 30 MiB/s wr, 1.91k op/s 2024-05-14T17:09:15.705 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:09:12 smithi151 ceph-mon[36066]: pgmap v453: 97 pgs: 97 active+clean; 5.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 30 MiB/s wr, 1.91k op/s 2024-05-14T17:09:15.810 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:09:14 smithi151 ceph-mon[36066]: pgmap v454: 97 pgs: 97 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 33 MiB/s wr, 2.07k op/s 2024-05-14T17:09:15.888 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:09:14 smithi123 ceph-mon[35971]: pgmap v454: 97 pgs: 97 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 33 MiB/s wr, 2.07k op/s 2024-05-14T17:09:15.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.082+0000 7fea4e4c8640 1 -- 172.21.15.2:0/4230007346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea48108a70 msgr2=0x7fea48108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.082+0000 7fea4e4c8640 1 --2- 172.21.15.2:0/4230007346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea48108a70 0x7fea48108e50 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fea38009930 tx=0x7fea3802a420 comp rx=0 tx=0).stop 2024-05-14T17:09:16.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.082+0000 7fea4e4c8640 1 -- 172.21.15.2:0/4230007346 shutdown_connections 2024-05-14T17:09:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.082+0000 7fea4e4c8640 1 --2- 172.21.15.2:0/4230007346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea48103420 0x7fea4810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:16.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.082+0000 7fea4e4c8640 1 --2- 172.21.15.2:0/4230007346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea48102a60 0x7fea48102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.082+0000 7fea4e4c8640 1 --2- 172.21.15.2:0/4230007346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea48108a70 0x7fea48108e50 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:16.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.082+0000 7fea4e4c8640 1 -- 172.21.15.2:0/4230007346 >> 172.21.15.2:0/4230007346 conn(0x7fea48079ee0 msgr2=0x7fea4807a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.082+0000 7fea4e4c8640 1 -- 172.21.15.2:0/4230007346 shutdown_connections 2024-05-14T17:09:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.083+0000 7fea4e4c8640 1 -- 172.21.15.2:0/4230007346 wait complete. 2024-05-14T17:09:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.083+0000 7fea4e4c8640 1 Processor -- start 2024-05-14T17:09:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.084+0000 7fea4e4c8640 1 -- start start 2024-05-14T17:09:16.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.084+0000 7fea4e4c8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea48102a60 0x7fea481991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.084+0000 7fea4e4c8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea48103420 0x7fea481996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:17.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.084+0000 7fea4e4c8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea48108a70 0x7fea4819da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:17.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.084+0000 7fea4e4c8640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fea48112d80 con 0x7fea48103420 2024-05-14T17:09:17.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.084+0000 7fea4e4c8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fea48112c00 con 0x7fea48108a70 2024-05-14T17:09:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.084+0000 7fea4e4c8640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fea48112f00 con 0x7fea48102a60 2024-05-14T17:09:17.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.084+0000 7fea477fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea48103420 0x7fea481996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.084+0000 7fea4ca65640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea48108a70 0x7fea4819da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.084+0000 7fea477fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea48103420 0x7fea481996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39996/0 (socket says 172.21.15.2:39996) 2024-05-14T17:09:17.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.084+0000 7fea477fe640 1 -- 172.21.15.2:0/488783520 learned_addr learned my addr 172.21.15.2:0/488783520 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:09:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.084+0000 7fea47fff640 1 --2- 172.21.15.2:0/488783520 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea48102a60 0x7fea481991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.084+0000 7fea477fe640 1 -- 172.21.15.2:0/488783520 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea48102a60 msgr2=0x7fea481991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.084+0000 7fea477fe640 1 --2- 172.21.15.2:0/488783520 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea48102a60 0x7fea481991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.084+0000 7fea477fe640 1 -- 172.21.15.2:0/488783520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea48108a70 msgr2=0x7fea4819da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.084+0000 7fea477fe640 1 --2- 172.21.15.2:0/488783520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea48108a70 0x7fea4819da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:17.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.084+0000 7fea477fe640 1 -- 172.21.15.2:0/488783520 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea4819e190 con 0x7fea48103420 2024-05-14T17:09:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.085+0000 7fea477fe640 1 --2- 172.21.15.2:0/488783520 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea48103420 0x7fea481996e0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00e9f0 tx=0x7fea3c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:18.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.085+0000 7fea457fa640 1 -- 172.21.15.2:0/488783520 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c00cd80 con 0x7fea48103420 2024-05-14T17:09:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.085+0000 7fea457fa640 1 -- 172.21.15.2:0/488783520 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea3c0108e0 con 0x7fea48103420 2024-05-14T17:09:18.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.085+0000 7fea4e4c8640 1 -- 172.21.15.2:0/488783520 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea4819e480 con 0x7fea48103420 2024-05-14T17:09:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.085+0000 7fea457fa640 1 -- 172.21.15.2:0/488783520 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c018980 con 0x7fea48103420 2024-05-14T17:09:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.085+0000 7fea4e4c8640 1 -- 172.21.15.2:0/488783520 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea48074c80 con 0x7fea48103420 2024-05-14T17:09:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.088+0000 7fea511de740 1 Processor -- start 2024-05-14T17:09:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.089+0000 7fea511de740 1 -- start start 2024-05-14T17:09:18.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.089+0000 7fea511de740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d06423f100 0x55d06423f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.089+0000 7fea511de740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d06423fa20 0x55d064249da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:18.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.089+0000 7fea511de740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d06424a2e0 0x55d06424c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.089+0000 7fea511de740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d0641a9c30 con 0x55d06423fa20 2024-05-14T17:09:18.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.089+0000 7fea511de740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d0641997d0 con 0x55d06424a2e0 2024-05-14T17:09:18.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.089+0000 7fea511de740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d06419d810 con 0x55d06423f100 2024-05-14T17:09:19.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.089+0000 7fea4ca65640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d06423f100 0x55d06423f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.089+0000 7fea47fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d06423fa20 0x55d064249da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:19.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.089+0000 7fea47fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d06423fa20 0x55d064249da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40012/0 (socket says 172.21.15.2:40012) 2024-05-14T17:09:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.089+0000 7fea47fff640 1 -- 172.21.15.2:0/1334136239 learned_addr learned my addr 172.21.15.2:0/1334136239 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:09:19.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.090+0000 7fea47fff640 1 -- 172.21.15.2:0/1334136239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d06423f100 msgr2=0x55d06423f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:19.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.090+0000 7fea47fff640 1 --2- 172.21.15.2:0/1334136239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d06423f100 0x55d06423f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:19.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.090+0000 7fea47fff640 1 -- 172.21.15.2:0/1334136239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d06424a2e0 msgr2=0x55d06424c6d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T17:09:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.090+0000 7fea47fff640 1 --2- 172.21.15.2:0/1334136239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d06424a2e0 0x55d06424c6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:19.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.090+0000 7fea47fff640 1 -- 172.21.15.2:0/1334136239 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d063eae210 con 0x55d06423fa20 2024-05-14T17:09:19.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.090+0000 7fea47fff640 1 --2- 172.21.15.2:0/1334136239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d06423fa20 0x55d064249da0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fea3802aa10 tx=0x7fea38005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b261053335b75de7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.090+0000 7fea457fa640 1 -- 172.21.15.2:0/488783520 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fea3c018b20 con 0x7fea48103420 2024-05-14T17:09:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.090+0000 7fea2e7fc640 1 -- 172.21.15.2:0/1334136239 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38035040 con 0x55d06423fa20 2024-05-14T17:09:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.090+0000 7fea2e7fc640 1 -- 172.21.15.2:0/1334136239 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea380043d0 con 0x55d06423fa20 2024-05-14T17:09:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.090+0000 7fea2e7fc640 1 -- 172.21.15.2:0/1334136239 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea380035f0 con 0x55d06423fa20 2024-05-14T17:09:19.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.091+0000 7fea457fa640 1 --2- 172.21.15.2:0/488783520 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fea300b2ae0 0x7fea300b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:19.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.091+0000 7fea511de740 1 -- 172.21.15.2:0/1334136239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d06423fa20 msgr2=0x55d064249da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:19.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.091+0000 7fea511de740 1 --2- 172.21.15.2:0/1334136239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d06423fa20 0x55d064249da0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fea3802aa10 tx=0x7fea38005fd0 comp rx=0 tx=0).stop 2024-05-14T17:09:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.091+0000 7fea511de740 1 -- 172.21.15.2:0/1334136239 shutdown_connections 2024-05-14T17:09:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.091+0000 7fea511de740 1 --2- 172.21.15.2:0/1334136239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d06424a2e0 0x55d06424c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.091+0000 7fea511de740 1 --2- 172.21.15.2:0/1334136239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d06423fa20 0x55d064249da0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:20.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.091+0000 7fea511de740 1 --2- 172.21.15.2:0/1334136239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d06423f100 0x55d06423f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.091+0000 7fea511de740 1 -- 172.21.15.2:0/1334136239 >> 172.21.15.2:0/1334136239 conn(0x55d06423b610 msgr2=0x55d06423b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.091+0000 7fea457fa640 1 -- 172.21.15.2:0/488783520 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7fea3c0e28d0 con 0x7fea48103420 2024-05-14T17:09:20.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.091+0000 7fea4ca65640 1 --2- 172.21.15.2:0/488783520 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fea300b2ae0 0x7fea300b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.092+0000 7fea511de740 1 -- 172.21.15.2:0/1334136239 shutdown_connections 2024-05-14T17:09:20.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.092+0000 7fea511de740 1 -- 172.21.15.2:0/1334136239 wait complete. 2024-05-14T17:09:20.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.092+0000 7fea511de740 1 -- start start 2024-05-14T17:09:20.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.092+0000 7fea4ca65640 1 --2- 172.21.15.2:0/488783520 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fea300b2ae0 0x7fea300b4fa0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fea40005fd0 tx=0x7fea40012040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.092+0000 7fea511de740 1 Processor -- start 2024-05-14T17:09:20.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.093+0000 7fea511de740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d06423f100 0x55d064249d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.093+0000 7fea511de740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d06423fa20 0x55d064247050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:20.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.093+0000 7fea511de740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d06424a2e0 0x55d064247790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:20.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.093+0000 7fea511de740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d063e7f790 con 0x55d06423fa20 2024-05-14T17:09:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.093+0000 7fea511de740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d064155890 con 0x55d06423f100 2024-05-14T17:09:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.093+0000 7fea511de740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d06424cf50 con 0x55d06424a2e0 2024-05-14T17:09:20.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.093+0000 7fea511de740 10 client.0 _ll_register_callbacks cb 0x55d064158d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:09:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.093+0000 7fea4ca65640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d06424a2e0 0x55d064247790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:21.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.093+0000 7fea477fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d06423fa20 0x55d064247050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:21.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.093+0000 7fea47fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d06423f100 0x55d064249d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.093+0000 7fea4ca65640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d06424a2e0 0x55d064247790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:48110/0 (socket says 172.21.15.2:48110) 2024-05-14T17:09:21.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.093+0000 7fea47fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d06423f100 0x55d064249d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:54462/0 (socket says 172.21.15.2:54462) 2024-05-14T17:09:21.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.093+0000 7fea4ca65640 1 -- 172.21.15.2:0/4222080709 learned_addr learned my addr 172.21.15.2:0/4222080709 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:09:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.093+0000 7fea47fff640 1 -- 172.21.15.2:0/4222080709 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d06424a2e0 msgr2=0x55d064247790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.093+0000 7fea47fff640 1 --2- 172.21.15.2:0/4222080709 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d06424a2e0 0x55d064247790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.093+0000 7fea47fff640 1 -- 172.21.15.2:0/4222080709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d06423fa20 msgr2=0x55d064247050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.093+0000 7fea47fff640 1 --2- 172.21.15.2:0/4222080709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d06423fa20 0x55d064247050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:21.578 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.093+0000 7fea47fff640 1 -- 172.21.15.2:0/4222080709 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d063f7a620 con 0x55d06423f100 2024-05-14T17:09:21.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.093+0000 7fea47fff640 1 --2- 172.21.15.2:0/4222080709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d06423f100 0x55d064249d60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fea38009850 tx=0x7fea3802a420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c495af866c19a8de server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:21.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.093+0000 7fea277fe640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T17:09:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.093+0000 7fea277fe640 1 -- 172.21.15.2:0/4222080709 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38052060 con 0x55d06423f100 2024-05-14T17:09:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.093+0000 7fea277fe640 1 -- 172.21.15.2:0/4222080709 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7fea38003800 con 0x55d06423f100 2024-05-14T17:09:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.094+0000 7fea511de740 1 -- 172.21.15.2:0/4222080709 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea380043d0 con 0x55d06423f100 2024-05-14T17:09:21.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.094+0000 7fea277fe640 1 -- 172.21.15.2:0/4222080709 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fea38035070 con 0x55d06423f100 2024-05-14T17:09:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.094+0000 7fea511de740 10 client.26315 fetch_fsmap learned FSMap version 47 2024-05-14T17:09:22.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.094+0000 7fea511de740 1 -- 172.21.15.2:0/4222080709 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55d064258a90 con 0x55d06423f100 2024-05-14T17:09:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.094+0000 7fea277fe640 1 -- 172.21.15.2:0/4222080709 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea380043d0 con 0x55d06423f100 2024-05-14T17:09:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.094+0000 7fea277fe640 1 -- 172.21.15.2:0/4222080709 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 47) ==== 8338+0+0 (secure 0 0 0) 0x7fea380052b0 con 0x55d06423f100 2024-05-14T17:09:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.095+0000 7fea511de740 10 client.26315 fetch_fsmap finished waiting for FSMap version 47 2024-05-14T17:09:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.095+0000 7fea511de740 10 client.26315 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:09:22.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.095+0000 7fea511de740 20 client.26315 populate_metadata read hostname 'smithi002' 2024-05-14T17:09:22.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.095+0000 7fea511de740 1 --2- 172.21.15.2:0/4222080709 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d064259de0 0x55d06425c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:22.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.095+0000 7fea511de740 4 client.26315 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:09:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.095+0000 7fea511de740 1 -- 172.21.15.2:0/4222080709 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0641abb80 con 0x55d064259de0 2024-05-14T17:09:22.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.095+0000 7fea477fe640 1 --2- 172.21.15.2:0/4222080709 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d064259de0 0x55d06425c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:22.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.095+0000 7fea477fe640 1 --2- 172.21.15.2:0/4222080709 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d064259de0 0x55d06425c1d0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50b570e32d382400 server_cookie=77ce4dc5b370905f in_seq=0 out_seq=0 2024-05-14T17:09:22.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.095+0000 7fea277fe640 10 client.26315 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:09:22.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.096+0000 7fea277fe640 1 -- 172.21.15.2:0/4222080709 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d0641abb80 con 0x55d064259de0 2024-05-14T17:09:23.033 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.096+0000 7fea277fe640 10 client.26315 handle_command_reply: tid=0 2024-05-14T17:09:23.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.096+0000 7fea511de740 1 client.26315 shutdown 2024-05-14T17:09:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.096+0000 7fea511de740 1 -- 172.21.15.2:0/4222080709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d06423f100 msgr2=0x55d064249d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.096+0000 7fea511de740 1 --2- 172.21.15.2:0/4222080709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d06423f100 0x55d064249d60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fea38009850 tx=0x7fea3802a420 comp rx=0 tx=0).stop 2024-05-14T17:09:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.096+0000 7fea511de740 1 -- 172.21.15.2:0/4222080709 shutdown_connections 2024-05-14T17:09:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.096+0000 7fea511de740 1 --2- 172.21.15.2:0/4222080709 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d064259de0 0x55d06425c1d0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.096+0000 7fea511de740 1 --2- 172.21.15.2:0/4222080709 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d06424a2e0 0x55d064247790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:23.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.096+0000 7fea277fe640 0 client.26315 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:09:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.096+0000 7fea511de740 1 --2- 172.21.15.2:0/4222080709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d06423fa20 0x55d064247050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.096+0000 7fea511de740 1 --2- 172.21.15.2:0/4222080709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d06423f100 0x55d064249d60 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:23.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.096+0000 7fea511de740 1 -- 172.21.15.2:0/4222080709 >> 172.21.15.2:0/4222080709 conn(0x55d06423b610 msgr2=0x55d064240ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.096+0000 7fea511de740 1 -- 172.21.15.2:0/4222080709 shutdown_connections 2024-05-14T17:09:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.096+0000 7fea511de740 1 -- 172.21.15.2:0/4222080709 wait complete. 2024-05-14T17:09:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.096+0000 7fea511de740 20 client.26315 trim_cache size 0 max 16384 2024-05-14T17:09:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.115+0000 7fea511de740 1 Processor -- start 2024-05-14T17:09:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.116+0000 7fea511de740 1 -- start start 2024-05-14T17:09:24.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.116+0000 7fea511de740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d06423f100 0x55d0642561b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:24.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.116+0000 7fea511de740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d06423fa20 0x55d064232100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:24.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.116+0000 7fea511de740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d06424a2e0 0x55d064249890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:24.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.116+0000 7fea511de740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fea481aec60 con 0x55d06424a2e0 2024-05-14T17:09:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.116+0000 7fea511de740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fea481ac530 con 0x55d06423f100 2024-05-14T17:09:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.116+0000 7fea511de740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fea481aa3a0 con 0x55d06423fa20 2024-05-14T17:09:24.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.116+0000 7fea47fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d06423f100 0x55d0642561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.116+0000 7fea477fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d06423fa20 0x55d064232100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.116+0000 7fea477fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d06423fa20 0x55d064232100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:48122/0 (socket says 172.21.15.2:48122) 2024-05-14T17:09:24.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.116+0000 7fea477fe640 1 -- 172.21.15.2:0/2412267290 learned_addr learned my addr 172.21.15.2:0/2412267290 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:09:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.116+0000 7fea47fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d06423f100 0x55d0642561b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:54478/0 (socket says 172.21.15.2:54478) 2024-05-14T17:09:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.116+0000 7fea4ca65640 1 --2- 172.21.15.2:0/2412267290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d06424a2e0 0x55d064249890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.116+0000 7fea47fff640 1 -- 172.21.15.2:0/2412267290 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d06423fa20 msgr2=0x55d064232100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.116+0000 7fea47fff640 1 --2- 172.21.15.2:0/2412267290 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d06423fa20 0x55d064232100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.116+0000 7fea47fff640 1 -- 172.21.15.2:0/2412267290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d06424a2e0 msgr2=0x55d064249890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:25.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.116+0000 7fea47fff640 1 --2- 172.21.15.2:0/2412267290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d06424a2e0 0x55d064249890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:25.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.116+0000 7fea47fff640 1 -- 172.21.15.2:0/2412267290 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d064247b00 con 0x55d06423f100 2024-05-14T17:09:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.116+0000 7fea4ca65640 1 --2- 172.21.15.2:0/2412267290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d06424a2e0 0x55d064249890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:09:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.116+0000 7fea477fe640 1 --2- 172.21.15.2:0/2412267290 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d06423fa20 0x55d064232100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:09:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.117+0000 7fea47fff640 1 --2- 172.21.15.2:0/2412267290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d06423f100 0x55d0642561b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fea38005700 tx=0x7fea38005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bac8049a586968 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.117+0000 7fea2dffb640 1 -- 172.21.15.2:0/2412267290 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38052040 con 0x55d06423f100 2024-05-14T17:09:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.117+0000 7fea2dffb640 1 -- 172.21.15.2:0/2412267290 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea38031dc0 con 0x55d06423f100 2024-05-14T17:09:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.117+0000 7fea2dffb640 1 -- 172.21.15.2:0/2412267290 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38049070 con 0x55d06423f100 2024-05-14T17:09:25.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.118+0000 7fea511de740 1 -- 172.21.15.2:0/2412267290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d06423f100 msgr2=0x55d0642561b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:25.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.118+0000 7fea511de740 1 --2- 172.21.15.2:0/2412267290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d06423f100 0x55d0642561b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fea38005700 tx=0x7fea38005fd0 comp rx=0 tx=0).stop 2024-05-14T17:09:26.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.118+0000 7fea511de740 1 -- 172.21.15.2:0/2412267290 shutdown_connections 2024-05-14T17:09:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.118+0000 7fea511de740 1 --2- 172.21.15.2:0/2412267290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d06424a2e0 0x55d064249890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.118+0000 7fea511de740 1 --2- 172.21.15.2:0/2412267290 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d06423fa20 0x55d064232100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.118+0000 7fea511de740 1 --2- 172.21.15.2:0/2412267290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d06423f100 0x55d0642561b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:26.385 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.118+0000 7fea511de740 1 -- 172.21.15.2:0/2412267290 >> 172.21.15.2:0/2412267290 conn(0x55d06423b610 msgr2=0x55d064241a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.118+0000 7fea511de740 1 -- 172.21.15.2:0/2412267290 shutdown_connections 2024-05-14T17:09:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.118+0000 7fea511de740 1 -- 172.21.15.2:0/2412267290 wait complete. 2024-05-14T17:09:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.118+0000 7fea511de740 1 -- start start 2024-05-14T17:09:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.119+0000 7fea511de740 1 Processor -- start 2024-05-14T17:09:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.119+0000 7fea511de740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d06423f100 0x55d06425c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:26.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.119+0000 7fea511de740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d06423fa20 0x55d06425ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.119+0000 7fea511de740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d06424a2e0 0x55d0642c22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.119+0000 7fea511de740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fea481aa9f0 con 0x55d06423f100 2024-05-14T17:09:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.119+0000 7fea511de740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fea481abe60 con 0x55d06423fa20 2024-05-14T17:09:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.119+0000 7fea511de740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fea481ab0c0 con 0x55d06424a2e0 2024-05-14T17:09:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.119+0000 7fea47fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d06423f100 0x55d06425c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:27.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.119+0000 7fea511de740 10 client.0 _ll_register_callbacks cb 0x7fea38004580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:09:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.119+0000 7fea47fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d06423f100 0x55d06425c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40032/0 (socket says 172.21.15.2:40032) 2024-05-14T17:09:27.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.119+0000 7fea47fff640 1 -- 172.21.15.2:0/4139286736 learned_addr learned my addr 172.21.15.2:0/4139286736 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:09:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.119+0000 7fea477fe640 1 --2- 172.21.15.2:0/4139286736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d06423fa20 0x55d06425ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.119+0000 7fea4ca65640 1 --2- 172.21.15.2:0/4139286736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d06424a2e0 0x55d0642c22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.120+0000 7fea47fff640 1 -- 172.21.15.2:0/4139286736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d06424a2e0 msgr2=0x55d0642c22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:27.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.120+0000 7fea47fff640 1 --2- 172.21.15.2:0/4139286736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d06424a2e0 0x55d0642c22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:27.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.120+0000 7fea47fff640 1 -- 172.21.15.2:0/4139286736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d06423fa20 msgr2=0x55d06425ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.120+0000 7fea47fff640 1 --2- 172.21.15.2:0/4139286736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d06423fa20 0x55d06425ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.120+0000 7fea47fff640 1 -- 172.21.15.2:0/4139286736 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0642c2880 con 0x55d06423f100 2024-05-14T17:09:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.120+0000 7fea47fff640 1 --2- 172.21.15.2:0/4139286736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d06423f100 0x55d06425c540 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fea38004650 tx=0x7fea3802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e47d95384fbf3f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.120+0000 7fea277fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:09:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.120+0000 7fea277fe640 1 -- 172.21.15.2:0/4139286736 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38049070 con 0x55d06423f100 2024-05-14T17:09:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.120+0000 7fea277fe640 1 -- 172.21.15.2:0/4139286736 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7fea38003a10 con 0x55d06423f100 2024-05-14T17:09:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.120+0000 7fea511de740 1 -- 172.21.15.2:0/4139286736 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea38031dc0 con 0x55d06423f100 2024-05-14T17:09:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.120+0000 7fea277fe640 1 -- 172.21.15.2:0/4139286736 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fea3806f050 con 0x55d06423f100 2024-05-14T17:09:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.120+0000 7fea511de740 10 client.16692 fetch_fsmap learned FSMap version 47 2024-05-14T17:09:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.120+0000 7fea511de740 1 -- 172.21.15.2:0/4139286736 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55d0642c3b90 con 0x55d06423f100 2024-05-14T17:09:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.120+0000 7fea277fe640 1 -- 172.21.15.2:0/4139286736 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea38064020 con 0x55d06423f100 2024-05-14T17:09:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.121+0000 7fea277fe640 1 -- 172.21.15.2:0/4139286736 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 47) ==== 8338+0+0 (secure 0 0 0) 0x7fea3805e020 con 0x55d06423f100 2024-05-14T17:09:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.121+0000 7fea511de740 10 client.16692 fetch_fsmap finished waiting for FSMap version 47 2024-05-14T17:09:28.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.121+0000 7fea511de740 10 client.16692 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:09:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.121+0000 7fea511de740 20 client.16692 populate_metadata read hostname 'smithi002' 2024-05-14T17:09:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.121+0000 7fea511de740 1 --2- 172.21.15.2:0/4139286736 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d0642c4230 0x55d0642c6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.121+0000 7fea511de740 4 client.16692 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:09:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.121+0000 7fea511de740 1 -- 172.21.15.2:0/4139286736 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fea38031dc0 con 0x55d0642c4230 2024-05-14T17:09:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.121+0000 7fea477fe640 1 --2- 172.21.15.2:0/4139286736 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d0642c4230 0x55d0642c6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.121+0000 7fea277fe640 10 client.16692 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:09:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.121+0000 7fea477fe640 1 --2- 172.21.15.2:0/4139286736 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d0642c4230 0x55d0642c6650 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d68d506e551f1e8 server_cookie=8efcca02b0740a6f in_seq=0 out_seq=0 2024-05-14T17:09:29.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.198+0000 7fea277fe640 1 -- 172.21.15.2:0/4139286736 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7fea38031dc0 con 0x55d0642c4230 2024-05-14T17:09:29.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.198+0000 7fea277fe640 10 client.16692 handle_command_reply: tid=0 2024-05-14T17:09:29.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.198+0000 7fea511de740 1 client.16692 shutdown 2024-05-14T17:09:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.198+0000 7fea511de740 1 -- 172.21.15.2:0/4139286736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d06423f100 msgr2=0x55d06425c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.198+0000 7fea511de740 1 --2- 172.21.15.2:0/4139286736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d06423f100 0x55d06425c540 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fea38004650 tx=0x7fea3802a420 comp rx=0 tx=0).stop 2024-05-14T17:09:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.198+0000 7fea511de740 1 -- 172.21.15.2:0/4139286736 shutdown_connections 2024-05-14T17:09:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.198+0000 7fea511de740 1 --2- 172.21.15.2:0/4139286736 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d0642c4230 0x55d0642c6650 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:29.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.198+0000 7fea511de740 1 --2- 172.21.15.2:0/4139286736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d06424a2e0 0x55d0642c22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:29.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.198+0000 7fea511de740 1 --2- 172.21.15.2:0/4139286736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d06423fa20 0x55d06425ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:30.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.198+0000 7fea511de740 1 --2- 172.21.15.2:0/4139286736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d06423f100 0x55d06425c540 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.198+0000 7fea511de740 1 -- 172.21.15.2:0/4139286736 >> 172.21.15.2:0/4139286736 conn(0x55d06423b610 msgr2=0x55d064231f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:30.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.198+0000 7fea277fe640 0 client.16692 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:09:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.198+0000 7fea511de740 1 -- 172.21.15.2:0/4139286736 shutdown_connections 2024-05-14T17:09:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.198+0000 7fea511de740 1 -- 172.21.15.2:0/4139286736 wait complete. 2024-05-14T17:09:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.199+0000 7fea511de740 20 client.16692 trim_cache size 0 max 16384 2024-05-14T17:09:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:09:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:09:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:09:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.199+0000 7fea4e4c8640 1 -- 172.21.15.2:0/488783520 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fea300b2ae0 msgr2=0x7fea300b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:30.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.199+0000 7fea4e4c8640 1 --2- 172.21.15.2:0/488783520 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fea300b2ae0 0x7fea300b4fa0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fea40005fd0 tx=0x7fea40012040 comp rx=0 tx=0).stop 2024-05-14T17:09:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.199+0000 7fea4e4c8640 1 -- 172.21.15.2:0/488783520 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea48103420 msgr2=0x7fea481996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.199+0000 7fea4e4c8640 1 --2- 172.21.15.2:0/488783520 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea48103420 0x7fea481996e0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00e9f0 tx=0x7fea3c00eeb0 comp rx=0 tx=0).stop 2024-05-14T17:09:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.200+0000 7fea4e4c8640 1 -- 172.21.15.2:0/488783520 shutdown_connections 2024-05-14T17:09:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.200+0000 7fea4e4c8640 1 --2- 172.21.15.2:0/488783520 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fea300b2ae0 0x7fea300b4fa0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:31.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.200+0000 7fea4e4c8640 1 --2- 172.21.15.2:0/488783520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea48108a70 0x7fea4819da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:31.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.200+0000 7fea4e4c8640 1 --2- 172.21.15.2:0/488783520 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea48103420 0x7fea481996e0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:31.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.200+0000 7fea4e4c8640 1 --2- 172.21.15.2:0/488783520 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea48102a60 0x7fea481991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.200+0000 7fea4e4c8640 1 -- 172.21.15.2:0/488783520 >> 172.21.15.2:0/488783520 conn(0x7fea48079ee0 msgr2=0x7fea481002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:31.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.200+0000 7fea4e4c8640 1 -- 172.21.15.2:0/488783520 shutdown_connections 2024-05-14T17:09:31.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:42.200+0000 7fea4e4c8640 1 -- 172.21.15.2:0/488783520 wait complete. 2024-05-14T17:09:31.493 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:07:42 smithi002 ceph-mon[31538]: pgmap v405: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 1.2 MiB/s wr, 92 op/s 2024-05-14T17:09:31.653 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:07:44 smithi002 ceph-mon[31538]: pgmap v406: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 2.9 MiB/s wr, 267 op/s 2024-05-14T17:09:31.710 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 750.0 s, 2117.4 tps, lat 47.765 ms stddev 52.191 2024-05-14T17:09:31.751 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:07:46 smithi002 ceph-mon[31538]: pgmap v407: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 3.8 MiB/s wr, 353 op/s 2024-05-14T17:09:31.800 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:07:48 smithi002 ceph-mon[31538]: pgmap v408: 97 pgs: 97 active+clean; 4.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 3.8 MiB/s wr, 353 op/s 2024-05-14T17:09:31.851 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:07:50 smithi002 ceph-mon[31538]: pgmap v409: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 6.5 MiB/s wr, 612 op/s 2024-05-14T17:09:31.901 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 755.0 s, 2438.8 tps, lat 40.854 ms stddev 43.975 2024-05-14T17:09:31.962 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:07:52 smithi002 ceph-mon[31538]: pgmap v410: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 5.4 MiB/s wr, 523 op/s 2024-05-14T17:09:32.104 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:07:53 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:09:32.195 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:07:54 smithi002 ceph-mon[31538]: pgmap v411: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 7.0 MiB/s wr, 694 op/s 2024-05-14T17:09:32.261 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:09:16 smithi123 ceph-mon[35971]: pgmap v455: 97 pgs: 97 active+clean; 4.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 34 MiB/s wr, 2.10k op/s 2024-05-14T17:09:32.475 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:09:18 smithi123 ceph-mon[35971]: osdmap e97: 12 total, 12 up, 12 in 2024-05-14T17:09:32.553 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:09:18 smithi123 ceph-mon[35971]: pgmap v457: 97 pgs: 97 active+clean; 4.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 9.2 MiB/s wr, 535 op/s 2024-05-14T17:09:32.642 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:09:20 smithi123 ceph-mon[35971]: pgmap v458: 97 pgs: 97 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 13 MiB/s wr, 758 op/s 2024-05-14T17:09:32.699 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:09:22 smithi123 ceph-mon[35971]: pgmap v459: 97 pgs: 97 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 13 MiB/s wr, 758 op/s 2024-05-14T17:09:32.747 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:09:24 smithi123 ceph-mon[35971]: pgmap v460: 97 pgs: 97 active+clean; 4.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 20 MiB/s wr, 1.43k op/s 2024-05-14T17:09:32.826 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:09:26 smithi123 ceph-mon[35971]: pgmap v461: 97 pgs: 97 active+clean; 4.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 24 MiB/s wr, 1.81k op/s 2024-05-14T17:09:32.886 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:09:28 smithi123 ceph-mon[35971]: pgmap v462: 97 pgs: 97 active+clean; 4.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 23 MiB/s wr, 1.76k op/s 2024-05-14T17:09:32.975 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:09:30 smithi123 ceph-mon[35971]: pgmap v463: 97 pgs: 97 active+clean; 4.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 26 MiB/s wr, 2.01k op/s 2024-05-14T17:09:33.036 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:09:16 smithi151 ceph-mon[36066]: pgmap v455: 97 pgs: 97 active+clean; 4.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 34 MiB/s wr, 2.10k op/s 2024-05-14T17:09:33.088 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:09:18 smithi151 ceph-mon[36066]: osdmap e97: 12 total, 12 up, 12 in 2024-05-14T17:09:33.202 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:09:18 smithi151 ceph-mon[36066]: pgmap v457: 97 pgs: 97 active+clean; 4.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 9.2 MiB/s wr, 535 op/s 2024-05-14T17:09:33.258 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:09:20 smithi151 ceph-mon[36066]: pgmap v458: 97 pgs: 97 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 13 MiB/s wr, 758 op/s 2024-05-14T17:09:33.329 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:09:22 smithi151 ceph-mon[36066]: pgmap v459: 97 pgs: 97 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 13 MiB/s wr, 758 op/s 2024-05-14T17:09:33.404 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:09:24 smithi151 ceph-mon[36066]: pgmap v460: 97 pgs: 97 active+clean; 4.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 20 MiB/s wr, 1.43k op/s 2024-05-14T17:09:33.452 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:09:26 smithi151 ceph-mon[36066]: pgmap v461: 97 pgs: 97 active+clean; 4.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 24 MiB/s wr, 1.81k op/s 2024-05-14T17:09:33.510 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:09:28 smithi151 ceph-mon[36066]: pgmap v462: 97 pgs: 97 active+clean; 4.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 23 MiB/s wr, 1.76k op/s 2024-05-14T17:09:33.560 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:09:30 smithi151 ceph-mon[36066]: pgmap v463: 97 pgs: 97 active+clean; 4.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 26 MiB/s wr, 2.01k op/s 2024-05-14T17:09:33.610 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:09:32 smithi151 ceph-mon[36066]: pgmap v464: 97 pgs: 97 active+clean; 4.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 21 MiB/s wr, 1.79k op/s 2024-05-14T17:09:33.701 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:09:32 smithi123 ceph-mon[35971]: pgmap v464: 97 pgs: 97 active+clean; 4.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 21 MiB/s wr, 1.79k op/s 2024-05-14T17:09:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.333+0000 7f0466cc8640 1 -- 172.21.15.2:0/1977070757 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0460107a90 msgr2=0x7f0460107e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.333+0000 7f0466cc8640 1 --2- 172.21.15.2:0/1977070757 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0460107a90 0x7f0460107e70 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f0454009900 tx=0x7f045402a410 comp rx=0 tx=0).stop 2024-05-14T17:09:33.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.333+0000 7f0466cc8640 1 -- 172.21.15.2:0/1977070757 shutdown_connections 2024-05-14T17:09:34.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.333+0000 7f0466cc8640 1 --2- 172.21.15.2:0/1977070757 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04601025d0 0x7f046010e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.333+0000 7f0466cc8640 1 --2- 172.21.15.2:0/1977070757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0460101c30 0x7f0460102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.333+0000 7f0466cc8640 1 --2- 172.21.15.2:0/1977070757 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0460107a90 0x7f0460107e70 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:34.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.333+0000 7f0466cc8640 1 -- 172.21.15.2:0/1977070757 >> 172.21.15.2:0/1977070757 conn(0x7f0460078930 msgr2=0x7f0460078d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:34.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.333+0000 7f0466cc8640 1 -- 172.21.15.2:0/1977070757 shutdown_connections 2024-05-14T17:09:34.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.333+0000 7f0466cc8640 1 -- 172.21.15.2:0/1977070757 wait complete. 2024-05-14T17:09:34.514 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.334+0000 7f0466cc8640 1 Processor -- start 2024-05-14T17:09:34.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.334+0000 7f0466cc8640 1 -- start start 2024-05-14T17:09:34.734 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.334+0000 7f0466cc8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0460101c30 0x7f046019b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.334+0000 7f0466cc8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04601025d0 0x7f046019b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.334+0000 7f0466cc8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0460107a90 0x7f046019fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:34.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.334+0000 7f0466cc8640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0460110b40 con 0x7f0460101c30 2024-05-14T17:09:35.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.334+0000 7f0466cc8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f04601109c0 con 0x7f04601025d0 2024-05-14T17:09:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.334+0000 7f0466cc8640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0460110cc0 con 0x7f0460107a90 2024-05-14T17:09:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.334+0000 7f0464a64640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0460101c30 0x7f046019b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.334+0000 7f045ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04601025d0 0x7f046019b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.334+0000 7f0464a64640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0460101c30 0x7f046019b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60186/0 (socket says 172.21.15.2:60186) 2024-05-14T17:09:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.334+0000 7f045ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04601025d0 0x7f046019b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:44346/0 (socket says 172.21.15.2:44346) 2024-05-14T17:09:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.334+0000 7f0464a64640 1 -- 172.21.15.2:0/2219863949 learned_addr learned my addr 172.21.15.2:0/2219863949 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:09:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.334+0000 7f0465265640 1 --2- 172.21.15.2:0/2219863949 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0460107a90 0x7f046019fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:35.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.335+0000 7f045ffff640 1 -- 172.21.15.2:0/2219863949 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0460107a90 msgr2=0x7f046019fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.335+0000 7f045ffff640 1 --2- 172.21.15.2:0/2219863949 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0460107a90 0x7f046019fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:35.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.335+0000 7f045ffff640 1 -- 172.21.15.2:0/2219863949 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0460101c30 msgr2=0x7f046019b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.335+0000 7f045ffff640 1 --2- 172.21.15.2:0/2219863949 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0460101c30 0x7f046019b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:36.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.335+0000 7f045ffff640 1 -- 172.21.15.2:0/2219863949 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04601a02a0 con 0x7f04601025d0 2024-05-14T17:09:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.335+0000 7f045ffff640 1 --2- 172.21.15.2:0/2219863949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04601025d0 0x7f046019b7f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f044c009800 tx=0x7f044c009cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:36.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.335+0000 7f045dffb640 1 -- 172.21.15.2:0/2219863949 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c004020 con 0x7f04601025d0 2024-05-14T17:09:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.335+0000 7f0466cc8640 1 -- 172.21.15.2:0/2219863949 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04601a0590 con 0x7f04601025d0 2024-05-14T17:09:36.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.335+0000 7f045dffb640 1 -- 172.21.15.2:0/2219863949 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f044c00ed60 con 0x7f04601025d0 2024-05-14T17:09:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.335+0000 7f045dffb640 1 -- 172.21.15.2:0/2219863949 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c003a30 con 0x7f04601025d0 2024-05-14T17:09:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.335+0000 7f0466cc8640 1 -- 172.21.15.2:0/2219863949 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04601a7e20 con 0x7f04601025d0 2024-05-14T17:09:36.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.339+0000 7f04699e7740 1 Processor -- start 2024-05-14T17:09:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.340+0000 7f04699e7740 1 -- start start 2024-05-14T17:09:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.340+0000 7f04699e7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562927547100 0x5629275474e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:36.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.340+0000 7f04699e7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562927547a20 0x562927551da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:36.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.340+0000 7f04699e7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629275522e0 0x5629275546d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:36.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.340+0000 7f04699e7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5629274b1c30 con 0x562927547a20 2024-05-14T17:09:36.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.340+0000 7f04699e7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5629274a17d0 con 0x5629275522e0 2024-05-14T17:09:36.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.340+0000 7f04699e7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5629274a5810 con 0x562927547100 2024-05-14T17:09:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.341+0000 7f0464a64640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562927547a20 0x562927551da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.341+0000 7f0465265640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562927547100 0x5629275474e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.341+0000 7f0464a64640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562927547a20 0x562927551da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60188/0 (socket says 172.21.15.2:60188) 2024-05-14T17:09:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.341+0000 7f0464a64640 1 -- 172.21.15.2:0/2680590279 learned_addr learned my addr 172.21.15.2:0/2680590279 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:09:37.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.341+0000 7f045dffb640 1 -- 172.21.15.2:0/2219863949 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f044c01b020 con 0x7f04601025d0 2024-05-14T17:09:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.341+0000 7f0465265640 1 -- 172.21.15.2:0/2680590279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629275522e0 msgr2=0x5629275546d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:37.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.341+0000 7f045ffff640 1 --2- 172.21.15.2:0/2680590279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629275522e0 0x5629275546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.341+0000 7f0465265640 1 --2- 172.21.15.2:0/2680590279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629275522e0 0x5629275546d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:37.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.341+0000 7f0465265640 1 -- 172.21.15.2:0/2680590279 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562927547a20 msgr2=0x562927551da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.341+0000 7f0465265640 1 --2- 172.21.15.2:0/2680590279 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562927547a20 0x562927551da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:37.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.341+0000 7f0465265640 1 -- 172.21.15.2:0/2680590279 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629271b6210 con 0x562927547100 2024-05-14T17:09:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.341+0000 7f0464a64640 1 --2- 172.21.15.2:0/2680590279 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562927547a20 0x562927551da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:09:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.342+0000 7f045dffb640 1 --2- 172.21.15.2:0/2219863949 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f04500b2a10 0x7f04500b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:37.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.342+0000 7f0464a64640 1 --2- 172.21.15.2:0/2219863949 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f04500b2a10 0x7f04500b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:38.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.342+0000 7f045dffb640 1 -- 172.21.15.2:0/2219863949 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f044c0e1ce0 con 0x7f04601025d0 2024-05-14T17:09:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.342+0000 7f0465265640 1 --2- 172.21.15.2:0/2680590279 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562927547100 0x5629275474e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0458009ef0 tx=0x7f0458011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5feafa5e780c23cd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:38.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.343+0000 7f0464a64640 1 --2- 172.21.15.2:0/2219863949 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f04500b2a10 0x7f04500b4ed0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0454005ec0 tx=0x7f04540046c0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:38.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.343+0000 7f044affd640 1 -- 172.21.15.2:0/2680590279 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0458004d20 con 0x562927547100 2024-05-14T17:09:38.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.343+0000 7f044affd640 1 -- 172.21.15.2:0/2680590279 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0458004030 con 0x562927547100 2024-05-14T17:09:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.343+0000 7f044affd640 1 -- 172.21.15.2:0/2680590279 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0458005280 con 0x562927547100 2024-05-14T17:09:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.344+0000 7f04699e7740 1 -- 172.21.15.2:0/2680590279 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562927547100 msgr2=0x5629275474e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.344+0000 7f04699e7740 1 --2- 172.21.15.2:0/2680590279 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562927547100 0x5629275474e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0458009ef0 tx=0x7f0458011810 comp rx=0 tx=0).stop 2024-05-14T17:09:38.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.344+0000 7f04699e7740 1 -- 172.21.15.2:0/2680590279 shutdown_connections 2024-05-14T17:09:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.344+0000 7f04699e7740 1 --2- 172.21.15.2:0/2680590279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629275522e0 0x5629275546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.344+0000 7f04699e7740 1 --2- 172.21.15.2:0/2680590279 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562927547a20 0x562927551da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.344+0000 7f04699e7740 1 --2- 172.21.15.2:0/2680590279 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562927547100 0x5629275474e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.344+0000 7f04699e7740 1 -- 172.21.15.2:0/2680590279 >> 172.21.15.2:0/2680590279 conn(0x562927543610 msgr2=0x5629275439f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.345+0000 7f04699e7740 1 -- 172.21.15.2:0/2680590279 shutdown_connections 2024-05-14T17:09:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.345+0000 7f04699e7740 1 -- 172.21.15.2:0/2680590279 wait complete. 2024-05-14T17:09:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.346+0000 7f04699e7740 1 -- start start 2024-05-14T17:09:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.346+0000 7f04699e7740 1 Processor -- start 2024-05-14T17:09:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.346+0000 7f04699e7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562927547100 0x562927551cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.347+0000 7f04699e7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562927547a20 0x56292754efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:39.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.347+0000 7f04699e7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5629275522e0 0x56292754f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.347+0000 7f04699e7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x562927187790 con 0x562927547a20 2024-05-14T17:09:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.347+0000 7f04699e7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56292745d890 con 0x562927547100 2024-05-14T17:09:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.347+0000 7f04699e7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562927554fb0 con 0x5629275522e0 2024-05-14T17:09:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.347+0000 7f04699e7740 10 client.0 _ll_register_callbacks cb 0x562927460d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:09:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.347+0000 7f0464a64640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562927547100 0x562927551cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.347+0000 7f0464a64640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562927547100 0x562927551cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:44362/0 (socket says 172.21.15.2:44362) 2024-05-14T17:09:39.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.347+0000 7f0464a64640 1 -- 172.21.15.2:0/220308591 learned_addr learned my addr 172.21.15.2:0/220308591 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:09:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.347+0000 7f045ffff640 1 --2- 172.21.15.2:0/220308591 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562927547a20 0x56292754efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:39.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.347+0000 7f0464a64640 1 -- 172.21.15.2:0/220308591 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5629275522e0 msgr2=0x56292754f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:40.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.347+0000 7f0465265640 1 --2- 172.21.15.2:0/220308591 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5629275522e0 0x56292754f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:40.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.347+0000 7f0464a64640 1 --2- 172.21.15.2:0/220308591 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5629275522e0 0x56292754f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.347+0000 7f0464a64640 1 -- 172.21.15.2:0/220308591 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562927547a20 msgr2=0x56292754efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.347+0000 7f0464a64640 1 --2- 172.21.15.2:0/220308591 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562927547a20 0x56292754efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.347+0000 7f0464a64640 1 -- 172.21.15.2:0/220308591 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562927282620 con 0x562927547100 2024-05-14T17:09:40.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.347+0000 7f045ffff640 1 --2- 172.21.15.2:0/220308591 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562927547a20 0x56292754efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:09:40.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.348+0000 7f0464a64640 1 --2- 172.21.15.2:0/220308591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562927547100 0x562927551cb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f04540096c0 tx=0x7f0454009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9dc75c069fe4692 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.348+0000 7f043ffff640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T17:09:40.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.348+0000 7f043ffff640 1 -- 172.21.15.2:0/220308591 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0454004750 con 0x562927547100 2024-05-14T17:09:40.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.348+0000 7f043ffff640 1 -- 172.21.15.2:0/220308591 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f04540048f0 con 0x562927547100 2024-05-14T17:09:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.348+0000 7f04699e7740 1 -- 172.21.15.2:0/220308591 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0458004030 con 0x562927547100 2024-05-14T17:09:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.349+0000 7f043ffff640 1 -- 172.21.15.2:0/220308591 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f0454004af0 con 0x562927547100 2024-05-14T17:09:41.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.349+0000 7f04699e7740 10 client.26330 fetch_fsmap learned FSMap version 48 2024-05-14T17:09:41.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.349+0000 7f04699e7740 1 -- 172.21.15.2:0/220308591 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x562927560a30 con 0x562927547100 2024-05-14T17:09:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.349+0000 7f043ffff640 1 -- 172.21.15.2:0/220308591 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0458004030 con 0x562927547100 2024-05-14T17:09:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.349+0000 7f043ffff640 1 -- 172.21.15.2:0/220308591 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 48) ==== 8342+0+0 (secure 0 0 0) 0x7f0454044030 con 0x562927547100 2024-05-14T17:09:41.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.349+0000 7f04699e7740 10 client.26330 fetch_fsmap finished waiting for FSMap version 48 2024-05-14T17:09:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.349+0000 7f04699e7740 10 client.26330 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:09:41.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.349+0000 7f04699e7740 20 client.26330 populate_metadata read hostname 'smithi002' 2024-05-14T17:09:41.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.349+0000 7f04699e7740 1 --2- 172.21.15.2:0/220308591 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x562927561ef0 0x5629275642e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.349+0000 7f04699e7740 4 client.26330 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:09:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.349+0000 7f04699e7740 1 -- 172.21.15.2:0/220308591 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629274b3b80 con 0x562927561ef0 2024-05-14T17:09:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.350+0000 7f045ffff640 1 --2- 172.21.15.2:0/220308591 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x562927561ef0 0x5629275642e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:41.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.350+0000 7f043ffff640 10 client.26330 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:09:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.350+0000 7f045ffff640 1 --2- 172.21.15.2:0/220308591 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x562927561ef0 0x5629275642e0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=297d174601123a17 server_cookie=201cb622f980f1b5 in_seq=0 out_seq=0 2024-05-14T17:09:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.351+0000 7f043ffff640 1 -- 172.21.15.2:0/220308591 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5629274b3b80 con 0x562927561ef0 2024-05-14T17:09:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.351+0000 7f043ffff640 10 client.26330 handle_command_reply: tid=0 2024-05-14T17:09:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.351+0000 7f04699e7740 1 client.26330 shutdown 2024-05-14T17:09:42.334 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.351+0000 7f04699e7740 1 -- 172.21.15.2:0/220308591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562927547100 msgr2=0x562927551cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:42.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.351+0000 7f04699e7740 1 --2- 172.21.15.2:0/220308591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562927547100 0x562927551cb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f04540096c0 tx=0x7f0454009260 comp rx=0 tx=0).stop 2024-05-14T17:09:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.351+0000 7f04699e7740 1 -- 172.21.15.2:0/220308591 shutdown_connections 2024-05-14T17:09:42.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.351+0000 7f04699e7740 1 --2- 172.21.15.2:0/220308591 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x562927561ef0 0x5629275642e0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:42.661 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.351+0000 7f04699e7740 1 --2- 172.21.15.2:0/220308591 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5629275522e0 0x56292754f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:42.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.351+0000 7f04699e7740 1 --2- 172.21.15.2:0/220308591 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562927547a20 0x56292754efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.351+0000 7f04699e7740 1 --2- 172.21.15.2:0/220308591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562927547100 0x562927551cb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.351+0000 7f043ffff640 0 client.26330 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:09:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.351+0000 7f04699e7740 1 -- 172.21.15.2:0/220308591 >> 172.21.15.2:0/220308591 conn(0x562927543610 msgr2=0x562927548b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.351+0000 7f04699e7740 1 -- 172.21.15.2:0/220308591 shutdown_connections 2024-05-14T17:09:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.351+0000 7f04699e7740 1 -- 172.21.15.2:0/220308591 wait complete. 2024-05-14T17:09:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.352+0000 7f04699e7740 20 client.26330 trim_cache size 0 max 16384 2024-05-14T17:09:43.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.377+0000 7f04699e7740 1 Processor -- start 2024-05-14T17:09:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.381+0000 7f04699e7740 1 -- start start 2024-05-14T17:09:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.382+0000 7f04699e7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562927547100 0x56292753a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.382+0000 7f04699e7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562927547a20 0x562927549210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.382+0000 7f04699e7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5629275522e0 0x5629275517e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:43.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.382+0000 7f04699e7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f04601aecd0 con 0x562927547100 2024-05-14T17:09:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.382+0000 7f04699e7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f04601ac570 con 0x562927547a20 2024-05-14T17:09:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.383+0000 7f04699e7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f04601aa420 con 0x5629275522e0 2024-05-14T17:09:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.384+0000 7f045ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562927547a20 0x562927549210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.384+0000 7f045ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562927547a20 0x562927549210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:44364/0 (socket says 172.21.15.2:44364) 2024-05-14T17:09:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.384+0000 7f045ffff640 1 -- 172.21.15.2:0/1310678744 learned_addr learned my addr 172.21.15.2:0/1310678744 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:09:43.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.384+0000 7f0464a64640 1 --2- 172.21.15.2:0/1310678744 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562927547100 0x56292753a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:43.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.385+0000 7f0464a64640 1 -- 172.21.15.2:0/1310678744 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5629275522e0 msgr2=0x5629275517e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T17:09:43.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.385+0000 7f0464a64640 1 --2- 172.21.15.2:0/1310678744 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5629275522e0 0x5629275517e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.385+0000 7f0464a64640 1 -- 172.21.15.2:0/1310678744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562927547a20 msgr2=0x562927549210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.385+0000 7f0464a64640 1 --2- 172.21.15.2:0/1310678744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562927547a20 0x562927549210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.385+0000 7f0464a64640 1 -- 172.21.15.2:0/1310678744 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56292754f920 con 0x562927547100 2024-05-14T17:09:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.385+0000 7f0464a64640 1 --2- 172.21.15.2:0/1310678744 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562927547100 0x56292753a100 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f04540037b0 tx=0x7f0454004fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d651a8eabd0e8142 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.391+0000 7f044a7fc640 1 -- 172.21.15.2:0/1310678744 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0454004750 con 0x562927547100 2024-05-14T17:09:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.391+0000 7f044a7fc640 1 -- 172.21.15.2:0/1310678744 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f045405f070 con 0x562927547100 2024-05-14T17:09:44.618 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.391+0000 7f044a7fc640 1 -- 172.21.15.2:0/1310678744 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0454031910 con 0x562927547100 2024-05-14T17:09:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.392+0000 7f04699e7740 1 -- 172.21.15.2:0/1310678744 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562927547100 msgr2=0x56292753a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.392+0000 7f04699e7740 1 --2- 172.21.15.2:0/1310678744 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562927547100 0x56292753a100 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f04540037b0 tx=0x7f0454004fe0 comp rx=0 tx=0).stop 2024-05-14T17:09:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.396+0000 7f04699e7740 1 -- 172.21.15.2:0/1310678744 shutdown_connections 2024-05-14T17:09:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.396+0000 7f04699e7740 1 --2- 172.21.15.2:0/1310678744 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5629275522e0 0x5629275517e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.396+0000 7f04699e7740 1 --2- 172.21.15.2:0/1310678744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562927547a20 0x562927549210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.396+0000 7f04699e7740 1 --2- 172.21.15.2:0/1310678744 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562927547100 0x56292753a100 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.396+0000 7f04699e7740 1 -- 172.21.15.2:0/1310678744 >> 172.21.15.2:0/1310678744 conn(0x562927543610 msgr2=0x562927548b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:45.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.403+0000 7f04699e7740 1 -- 172.21.15.2:0/1310678744 shutdown_connections 2024-05-14T17:09:45.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.408+0000 7f04699e7740 1 -- 172.21.15.2:0/1310678744 wait complete. 2024-05-14T17:09:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.408+0000 7f04699e7740 1 -- start start 2024-05-14T17:09:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.408+0000 7f04699e7740 1 Processor -- start 2024-05-14T17:09:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.411+0000 7f04699e7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562927547100 0x56292755a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.411+0000 7f04699e7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562927547a20 0x562927555c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.411+0000 7f04699e7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5629275522e0 0x562927556380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.411+0000 7f04699e7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f04601aaa70 con 0x562927547100 2024-05-14T17:09:45.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.411+0000 7f04699e7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f04601abea0 con 0x562927547a20 2024-05-14T17:09:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.411+0000 7f04699e7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f04601ab100 con 0x5629275522e0 2024-05-14T17:09:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.411+0000 7f04699e7740 10 client.0 _ll_register_callbacks cb 0x7f0454031520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:09:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.412+0000 7f045ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562927547a20 0x562927555c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.412+0000 7f045ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562927547a20 0x562927555c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:44380/0 (socket says 172.21.15.2:44380) 2024-05-14T17:09:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.412+0000 7f045ffff640 1 -- 172.21.15.2:0/2454768866 learned_addr learned my addr 172.21.15.2:0/2454768866 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:09:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.412+0000 7f0464a64640 1 --2- 172.21.15.2:0/2454768866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562927547100 0x56292755a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:46.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.412+0000 7f045ffff640 1 -- 172.21.15.2:0/2454768866 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5629275522e0 msgr2=0x562927556380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T17:09:46.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.412+0000 7f045ffff640 1 --2- 172.21.15.2:0/2454768866 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5629275522e0 0x562927556380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:46.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.412+0000 7f045ffff640 1 -- 172.21.15.2:0/2454768866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562927547100 msgr2=0x56292755a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.412+0000 7f045ffff640 1 --2- 172.21.15.2:0/2454768866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562927547100 0x56292755a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.412+0000 7f045ffff640 1 -- 172.21.15.2:0/2454768866 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629275dc740 con 0x562927547a20 2024-05-14T17:09:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.413+0000 7f045ffff640 1 --2- 172.21.15.2:0/2454768866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562927547a20 0x562927555c40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f044c0a61d0 tx=0x7f044c00b520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6196ff017a252748 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.413+0000 7f043ffff640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T17:09:47.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.413+0000 7f043ffff640 1 -- 172.21.15.2:0/2454768866 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c0ad310 con 0x562927547a20 2024-05-14T17:09:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.413+0000 7f04699e7740 1 -- 172.21.15.2:0/2454768866 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f045405f070 con 0x562927547a20 2024-05-14T17:09:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.413+0000 7f043ffff640 1 -- 172.21.15.2:0/2454768866 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f044c0df650 con 0x562927547a20 2024-05-14T17:09:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.415+0000 7f043ffff640 1 -- 172.21.15.2:0/2454768866 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f044c0df850 con 0x562927547a20 2024-05-14T17:09:47.419 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.415+0000 7f04699e7740 10 client.26336 fetch_fsmap learned FSMap version 48 2024-05-14T17:09:47.514 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.415+0000 7f04699e7740 1 -- 172.21.15.2:0/2454768866 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5629275dd940 con 0x562927547a20 2024-05-14T17:09:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.416+0000 7f043ffff640 1 -- 172.21.15.2:0/2454768866 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f045405f070 con 0x562927547a20 2024-05-14T17:09:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.416+0000 7f043ffff640 1 -- 172.21.15.2:0/2454768866 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 48) ==== 8342+0+0 (secure 0 0 0) 0x7f044c0a81c0 con 0x562927547a20 2024-05-14T17:09:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.416+0000 7f04699e7740 10 client.26336 fetch_fsmap finished waiting for FSMap version 48 2024-05-14T17:09:47.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.416+0000 7f04699e7740 10 client.26336 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:09:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.416+0000 7f04699e7740 20 client.26336 populate_metadata read hostname 'smithi002' 2024-05-14T17:09:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.416+0000 7f04699e7740 1 --2- 172.21.15.2:0/2454768866 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5629275de0f0 0x5629275e0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.416+0000 7f04699e7740 4 client.26336 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:09:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.416+0000 7f04699e7740 1 -- 172.21.15.2:0/2454768866 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5629274ad360 con 0x5629275de0f0 2024-05-14T17:09:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.417+0000 7f0464a64640 1 --2- 172.21.15.2:0/2454768866 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5629275de0f0 0x5629275e0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.424+0000 7f0464a64640 1 --2- 172.21.15.2:0/2454768866 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5629275de0f0 0x5629275e0510 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54246a72f500fb26 server_cookie=5376fc85039e69a8 in_seq=0 out_seq=0 2024-05-14T17:09:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.425+0000 7f043ffff640 10 client.26336 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:09:48.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.463+0000 7f043ffff640 1 -- 172.21.15.2:0/2454768866 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5629274ad360 con 0x5629275de0f0 2024-05-14T17:09:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.463+0000 7f043ffff640 10 client.26336 handle_command_reply: tid=0 2024-05-14T17:09:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.463+0000 7f04699e7740 1 client.26336 shutdown 2024-05-14T17:09:48.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.463+0000 7f04699e7740 1 -- 172.21.15.2:0/2454768866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562927547a20 msgr2=0x562927555c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:48.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.463+0000 7f04699e7740 1 --2- 172.21.15.2:0/2454768866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562927547a20 0x562927555c40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f044c0a61d0 tx=0x7f044c00b520 comp rx=0 tx=0).stop 2024-05-14T17:09:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.463+0000 7f04699e7740 1 -- 172.21.15.2:0/2454768866 shutdown_connections 2024-05-14T17:09:48.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.463+0000 7f04699e7740 1 --2- 172.21.15.2:0/2454768866 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5629275de0f0 0x5629275e0510 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:49.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.463+0000 7f04699e7740 1 --2- 172.21.15.2:0/2454768866 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5629275522e0 0x562927556380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.463+0000 7f04699e7740 1 --2- 172.21.15.2:0/2454768866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562927547a20 0x562927555c40 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:49.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.463+0000 7f043ffff640 0 client.26336 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:09:49.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.463+0000 7f04699e7740 1 --2- 172.21.15.2:0/2454768866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562927547100 0x56292755a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.463+0000 7f04699e7740 1 -- 172.21.15.2:0/2454768866 >> 172.21.15.2:0/2454768866 conn(0x562927543610 msgr2=0x56292754a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.463+0000 7f04699e7740 1 -- 172.21.15.2:0/2454768866 shutdown_connections 2024-05-14T17:09:49.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.464+0000 7f04699e7740 1 -- 172.21.15.2:0/2454768866 wait complete. 2024-05-14T17:09:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.464+0000 7f04699e7740 20 client.26336 trim_cache size 0 max 16384 2024-05-14T17:09:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:09:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:09:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:09:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.464+0000 7f0466cc8640 1 -- 172.21.15.2:0/2219863949 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f04500b2a10 msgr2=0x7f04500b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.464+0000 7f0466cc8640 1 --2- 172.21.15.2:0/2219863949 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f04500b2a10 0x7f04500b4ed0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0454005ec0 tx=0x7f04540046c0 comp rx=0 tx=0).stop 2024-05-14T17:09:50.283 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.464+0000 7f0466cc8640 1 -- 172.21.15.2:0/2219863949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04601025d0 msgr2=0x7f046019b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:50.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.464+0000 7f0466cc8640 1 --2- 172.21.15.2:0/2219863949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04601025d0 0x7f046019b7f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f044c009800 tx=0x7f044c009cc0 comp rx=0 tx=0).stop 2024-05-14T17:09:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.464+0000 7f0466cc8640 1 -- 172.21.15.2:0/2219863949 shutdown_connections 2024-05-14T17:09:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.464+0000 7f0466cc8640 1 --2- 172.21.15.2:0/2219863949 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f04500b2a10 0x7f04500b4ed0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:50.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.464+0000 7f0466cc8640 1 --2- 172.21.15.2:0/2219863949 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0460107a90 0x7f046019fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:50.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.464+0000 7f0466cc8640 1 --2- 172.21.15.2:0/2219863949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04601025d0 0x7f046019b7f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:50.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.464+0000 7f0466cc8640 1 --2- 172.21.15.2:0/2219863949 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0460101c30 0x7f046019b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:50.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.464+0000 7f0466cc8640 1 -- 172.21.15.2:0/2219863949 >> 172.21.15.2:0/2219863949 conn(0x7f0460078930 msgr2=0x7f0460100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.465+0000 7f0466cc8640 1 -- 172.21.15.2:0/2219863949 shutdown_connections 2024-05-14T17:09:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:07:55.465+0000 7f0466cc8640 1 -- 172.21.15.2:0/2219863949 wait complete. 2024-05-14T17:09:51.036 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 760.0 s, 2503.6 tps, lat 40.001 ms stddev 44.107 2024-05-14T17:09:51.111 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:07:56 smithi002 ceph-mon[31538]: pgmap v412: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 6.8 MiB/s wr, 644 op/s 2024-05-14T17:09:51.193 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:07:58 smithi002 ceph-mon[31538]: pgmap v413: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.8 MiB/s wr, 555 op/s 2024-05-14T17:09:51.322 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:07:58 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:09:51.394 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:08:00 smithi002 ceph-mon[31538]: pgmap v414: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 8.5 MiB/s wr, 830 op/s 2024-05-14T17:09:51.466 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 765.0 s, 2068.2 tps, lat 46.001 ms stddev 56.317 2024-05-14T17:09:51.505 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:08:01 smithi002 ceph-mon[31538]: pgmap v415: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 5.8 MiB/s wr, 571 op/s 2024-05-14T17:09:51.594 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:08:04 smithi002 ceph-mon[31538]: pgmap v416: 97 pgs: 97 active+clean; 4.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 7.8 MiB/s wr, 792 op/s 2024-05-14T17:09:51.665 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 770.0 s, 1146.2 tps, lat 91.375 ms stddev 169.840 2024-05-14T17:09:51.761 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:08:06 smithi002 ceph-mon[31538]: pgmap v417: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 11 MiB/s wr, 942 op/s 2024-05-14T17:09:51.850 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:08:06 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:09:51.944 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:08:07 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:09:52.008 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:08:07 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:09:52.098 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:08:07 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:09:52.170 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:09:34 smithi123 ceph-mon[35971]: pgmap v465: 97 pgs: 97 active+clean; 4.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 25 MiB/s wr, 2.00k op/s 2024-05-14T17:09:52.205 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:09:36 smithi123 ceph-mon[35971]: pgmap v466: 97 pgs: 97 active+clean; 4.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 15 MiB/s wr, 1.22k op/s 2024-05-14T17:09:52.238 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:09:38 smithi123 ceph-mon[35971]: pgmap v467: 97 pgs: 97 active+clean; 4.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 10 MiB/s wr, 826 op/s 2024-05-14T17:09:52.272 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:09:40 smithi123 ceph-mon[35971]: pgmap v468: 97 pgs: 97 active+clean; 5.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-05-14T17:09:52.305 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:09:42 smithi123 ceph-mon[35971]: pgmap v469: 97 pgs: 97 active+clean; 5.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.8 MiB/s wr, 584 op/s 2024-05-14T17:09:52.338 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:09:44 smithi123 ceph-mon[35971]: pgmap v470: 97 pgs: 97 active+clean; 5.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 12 MiB/s wr, 834 op/s 2024-05-14T17:09:52.372 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:09:46 smithi123 ceph-mon[35971]: pgmap v471: 97 pgs: 97 active+clean; 5.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.8 MiB/s wr, 662 op/s 2024-05-14T17:09:52.438 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:09:48 smithi123 ceph-mon[35971]: pgmap v472: 97 pgs: 97 active+clean; 5.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.6 MiB/s wr, 635 op/s 2024-05-14T17:09:52.505 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:09:50 smithi123 ceph-mon[35971]: pgmap v473: 97 pgs: 97 active+clean; 5.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 12 MiB/s wr, 877 op/s 2024-05-14T17:09:52.583 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:09:34 smithi151 ceph-mon[36066]: pgmap v465: 97 pgs: 97 active+clean; 4.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 25 MiB/s wr, 2.00k op/s 2024-05-14T17:09:52.679 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:09:36 smithi151 ceph-mon[36066]: pgmap v466: 97 pgs: 97 active+clean; 4.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 15 MiB/s wr, 1.22k op/s 2024-05-14T17:09:52.730 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:09:38 smithi151 ceph-mon[36066]: pgmap v467: 97 pgs: 97 active+clean; 4.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 10 MiB/s wr, 826 op/s 2024-05-14T17:09:52.859 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:09:40 smithi151 ceph-mon[36066]: pgmap v468: 97 pgs: 97 active+clean; 5.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-05-14T17:09:52.910 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:09:42 smithi151 ceph-mon[36066]: pgmap v469: 97 pgs: 97 active+clean; 5.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.8 MiB/s wr, 584 op/s 2024-05-14T17:09:52.962 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:09:44 smithi151 ceph-mon[36066]: pgmap v470: 97 pgs: 97 active+clean; 5.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 12 MiB/s wr, 834 op/s 2024-05-14T17:09:53.018 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:09:46 smithi151 ceph-mon[36066]: pgmap v471: 97 pgs: 97 active+clean; 5.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.8 MiB/s wr, 662 op/s 2024-05-14T17:09:53.096 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:09:48 smithi151 ceph-mon[36066]: pgmap v472: 97 pgs: 97 active+clean; 5.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.6 MiB/s wr, 635 op/s 2024-05-14T17:09:53.130 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:09:50 smithi151 ceph-mon[36066]: pgmap v473: 97 pgs: 97 active+clean; 5.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 12 MiB/s wr, 877 op/s 2024-05-14T17:09:53.198 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:09:52 smithi123 ceph-mon[35971]: pgmap v474: 97 pgs: 97 active+clean; 5.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.7 MiB/s wr, 648 op/s 2024-05-14T17:09:53.255 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:09:52 smithi151 ceph-mon[36066]: pgmap v474: 97 pgs: 97 active+clean; 5.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.7 MiB/s wr, 648 op/s 2024-05-14T17:09:53.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.596+0000 7f2279589640 1 -- 172.21.15.2:0/1671079598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2274101980 msgr2=0x7f2274101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.596+0000 7f2279589640 1 --2- 172.21.15.2:0/1671079598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2274101980 0x7f2274101e00 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f2268009930 tx=0x7f226802a420 comp rx=0 tx=0).stop 2024-05-14T17:09:53.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.596+0000 7f2279589640 1 -- 172.21.15.2:0/1671079598 shutdown_connections 2024-05-14T17:09:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.596+0000 7f2279589640 1 --2- 172.21.15.2:0/1671079598 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2274102340 0x7f227410e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.596+0000 7f2279589640 1 --2- 172.21.15.2:0/1671079598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2274101980 0x7f2274101e00 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:53.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.596+0000 7f2279589640 1 --2- 172.21.15.2:0/1671079598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2274107940 0x7f2274107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.596+0000 7f2279589640 1 -- 172.21.15.2:0/1671079598 >> 172.21.15.2:0/1671079598 conn(0x7f2274078ad0 msgr2=0x7f2274078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:53.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.596+0000 7f2279589640 1 -- 172.21.15.2:0/1671079598 shutdown_connections 2024-05-14T17:09:53.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.596+0000 7f2279589640 1 -- 172.21.15.2:0/1671079598 wait complete. 2024-05-14T17:09:54.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.596+0000 7f2279589640 1 Processor -- start 2024-05-14T17:09:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.597+0000 7f2279589640 1 -- start start 2024-05-14T17:09:54.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.597+0000 7f2279589640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2274101980 0x7f2274197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:54.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.597+0000 7f2279589640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2274102340 0x7f22741975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.597+0000 7f2279589640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2274107940 0x7f227419b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.597+0000 7f2279589640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2274110b40 con 0x7f2274102340 2024-05-14T17:09:54.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.597+0000 7f2279589640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f22741109c0 con 0x7f2274107940 2024-05-14T17:09:54.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.597+0000 7f2279589640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2274110cc0 con 0x7f2274101980 2024-05-14T17:09:54.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.597+0000 7f22727fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2274102340 0x7f22741975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.597+0000 7f22727fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2274102340 0x7f22741975a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57114/0 (socket says 172.21.15.2:57114) 2024-05-14T17:09:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.597+0000 7f22727fc640 1 -- 172.21.15.2:0/2843261035 learned_addr learned my addr 172.21.15.2:0/2843261035 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:09:55.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.597+0000 7f2272ffd640 1 --2- 172.21.15.2:0/2843261035 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2274101980 0x7f2274197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.597+0000 7f22737fe640 1 --2- 172.21.15.2:0/2843261035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2274107940 0x7f227419b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:55.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.597+0000 7f22727fc640 1 -- 172.21.15.2:0/2843261035 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2274101980 msgr2=0x7f2274197060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:55.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.597+0000 7f22727fc640 1 --2- 172.21.15.2:0/2843261035 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2274101980 0x7f2274197060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.597+0000 7f22727fc640 1 -- 172.21.15.2:0/2843261035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2274107940 msgr2=0x7f227419b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:55.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.597+0000 7f22727fc640 1 --2- 172.21.15.2:0/2843261035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2274107940 0x7f227419b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:55.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.597+0000 7f22727fc640 1 -- 172.21.15.2:0/2843261035 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f227419c050 con 0x7f2274102340 2024-05-14T17:09:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.597+0000 7f2272ffd640 1 --2- 172.21.15.2:0/2843261035 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2274101980 0x7f2274197060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:09:55.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.597+0000 7f22727fc640 1 --2- 172.21.15.2:0/2843261035 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2274102340 0x7f22741975a0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f2268005ab0 tx=0x7f2268003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:55.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.598+0000 7f225bfff640 1 -- 172.21.15.2:0/2843261035 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226803f070 con 0x7f2274102340 2024-05-14T17:09:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.598+0000 7f2279589640 1 -- 172.21.15.2:0/2843261035 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f227419c2e0 con 0x7f2274102340 2024-05-14T17:09:55.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.598+0000 7f225bfff640 1 -- 172.21.15.2:0/2843261035 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2268031b50 con 0x7f2274102340 2024-05-14T17:09:55.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.598+0000 7f225bfff640 1 -- 172.21.15.2:0/2843261035 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2268035030 con 0x7f2274102340 2024-05-14T17:09:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.598+0000 7f2279589640 1 -- 172.21.15.2:0/2843261035 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22741a3bd0 con 0x7f2274102340 2024-05-14T17:09:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.602+0000 7f225bfff640 1 -- 172.21.15.2:0/2843261035 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f2268042070 con 0x7f2274102340 2024-05-14T17:09:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.602+0000 7f227c37c740 1 Processor -- start 2024-05-14T17:09:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.602+0000 7f227c37c740 1 -- start start 2024-05-14T17:09:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.602+0000 7f227c37c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfeec458f0 0x55dfeec45cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:56.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.602+0000 7f227c37c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfeec46210 0x55dfeec50590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.603+0000 7f227c37c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfeec50ad0 0x55dfeec52ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.603+0000 7f227c37c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55dfeebb8c30 con 0x55dfeec46210 2024-05-14T17:09:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.603+0000 7f227c37c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55dfeeba87d0 con 0x55dfeec50ad0 2024-05-14T17:09:56.578 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.603+0000 7f227c37c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55dfeebac810 con 0x55dfeec458f0 2024-05-14T17:09:56.618 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.603+0000 7f22737fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfeec458f0 0x55dfeec45cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.603+0000 7f225bfff640 1 --2- 172.21.15.2:0/2843261035 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f225c0b4c20 0x7f225c0b70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.603+0000 7f22737fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfeec458f0 0x55dfeec45cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:43244/0 (socket says 172.21.15.2:43244) 2024-05-14T17:09:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.603+0000 7f22737fe640 1 -- 172.21.15.2:0/2254199300 learned_addr learned my addr 172.21.15.2:0/2254199300 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:09:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.603+0000 7f2272ffd640 1 --2- 172.21.15.2:0/2254199300 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfeec46210 0x55dfeec50590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.603+0000 7f22727fc640 1 --2- 172.21.15.2:0/2254199300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfeec50ad0 0x55dfeec52ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.603+0000 7f22737fe640 1 --2- 172.21.15.2:0/2843261035 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f225c0b4c20 0x7f225c0b70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:57.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.603+0000 7f22737fe640 1 -- 172.21.15.2:0/2254199300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfeec50ad0 msgr2=0x55dfeec52ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:57.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.603+0000 7f22737fe640 1 --2- 172.21.15.2:0/2254199300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfeec50ad0 0x55dfeec52ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.603+0000 7f22737fe640 1 -- 172.21.15.2:0/2254199300 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfeec46210 msgr2=0x55dfeec50590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:09:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.603+0000 7f22737fe640 1 --2- 172.21.15.2:0/2254199300 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfeec46210 0x55dfeec50590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:57.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.603+0000 7f22737fe640 1 -- 172.21.15.2:0/2254199300 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfee8bd210 con 0x55dfeec458f0 2024-05-14T17:09:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.603+0000 7f22727fc640 1 --2- 172.21.15.2:0/2254199300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfeec50ad0 0x55dfeec52ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:09:57.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.603+0000 7f225bfff640 1 -- 172.21.15.2:0/2843261035 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f2268102cd0 con 0x7f2274102340 2024-05-14T17:09:57.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.603+0000 7f22737fe640 1 --2- 172.21.15.2:0/2254199300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfeec458f0 0x55dfeec45cd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f226c005c40 tx=0x7f226c00f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd2d3d15d558b999 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.603+0000 7f22597fa640 1 -- 172.21.15.2:0/2254199300 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226c022020 con 0x55dfeec458f0 2024-05-14T17:09:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.603+0000 7f22737fe640 1 --2- 172.21.15.2:0/2843261035 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f225c0b4c20 0x7f225c0b70e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f226c023000 tx=0x7f226c021000 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:09:58.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.604+0000 7f22597fa640 1 -- 172.21.15.2:0/2254199300 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f226c013170 con 0x55dfeec458f0 2024-05-14T17:09:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.604+0000 7f22597fa640 1 -- 172.21.15.2:0/2254199300 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226c0107d0 con 0x55dfeec458f0 2024-05-14T17:09:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.604+0000 7f227c37c740 1 -- 172.21.15.2:0/2254199300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfeec458f0 msgr2=0x55dfeec45cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:09:58.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.604+0000 7f227c37c740 1 --2- 172.21.15.2:0/2254199300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfeec458f0 0x55dfeec45cd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f226c005c40 tx=0x7f226c00f3c0 comp rx=0 tx=0).stop 2024-05-14T17:09:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.604+0000 7f227c37c740 1 -- 172.21.15.2:0/2254199300 shutdown_connections 2024-05-14T17:09:58.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.604+0000 7f227c37c740 1 --2- 172.21.15.2:0/2254199300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfeec50ad0 0x55dfeec52ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.604+0000 7f227c37c740 1 --2- 172.21.15.2:0/2254199300 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfeec46210 0x55dfeec50590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.604+0000 7f227c37c740 1 --2- 172.21.15.2:0/2254199300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfeec458f0 0x55dfeec45cd0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:09:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.604+0000 7f227c37c740 1 -- 172.21.15.2:0/2254199300 >> 172.21.15.2:0/2254199300 conn(0x55dfeec41e00 msgr2=0x55dfeec421e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:09:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.604+0000 7f227c37c740 1 -- 172.21.15.2:0/2254199300 shutdown_connections 2024-05-14T17:09:58.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.604+0000 7f227c37c740 1 -- 172.21.15.2:0/2254199300 wait complete. 2024-05-14T17:09:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.605+0000 7f227c37c740 1 -- start start 2024-05-14T17:09:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.605+0000 7f227c37c740 1 Processor -- start 2024-05-14T17:09:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.605+0000 7f227c37c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfeec458f0 0x55dfeec504e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:59.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.605+0000 7f227c37c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfeec46210 0x55dfeec4d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.605+0000 7f227c37c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfeec50ad0 0x55dfeec4dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:09:59.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.605+0000 7f227c37c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55dfee88e790 con 0x55dfeec458f0 2024-05-14T17:09:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.605+0000 7f227c37c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55dfeeb64890 con 0x55dfeec46210 2024-05-14T17:09:59.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.605+0000 7f227c37c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55dfeec537a0 con 0x55dfeec50ad0 2024-05-14T17:09:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.606+0000 7f227c37c740 10 client.0 _ll_register_callbacks cb 0x55dfeeb67d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:09:59.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.606+0000 7f2272ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfeec458f0 0x55dfeec504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:59.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.606+0000 7f22727fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfeec46210 0x55dfeec4d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:09:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.606+0000 7f2272ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfeec458f0 0x55dfeec504e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57134/0 (socket says 172.21.15.2:57134) 2024-05-14T17:09:59.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.606+0000 7f2272ffd640 1 -- 172.21.15.2:0/3988108471 learned_addr learned my addr 172.21.15.2:0/3988108471 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:10:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.606+0000 7f22737fe640 1 --2- 172.21.15.2:0/3988108471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfeec50ad0 0x55dfeec4dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:00.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.606+0000 7f2272ffd640 1 -- 172.21.15.2:0/3988108471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfeec50ad0 msgr2=0x55dfeec4dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.606+0000 7f2272ffd640 1 --2- 172.21.15.2:0/3988108471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfeec50ad0 0x55dfeec4dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.606+0000 7f2272ffd640 1 -- 172.21.15.2:0/3988108471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfeec46210 msgr2=0x55dfeec4d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:00.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.606+0000 7f2272ffd640 1 --2- 172.21.15.2:0/3988108471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfeec46210 0x55dfeec4d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.606+0000 7f2272ffd640 1 -- 172.21.15.2:0/3988108471 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfee989620 con 0x55dfeec458f0 2024-05-14T17:10:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.606+0000 7f22727fc640 1 --2- 172.21.15.2:0/3988108471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfeec46210 0x55dfeec4d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:10:00.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.606+0000 7f2272ffd640 1 --2- 172.21.15.2:0/3988108471 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfeec458f0 0x55dfeec504e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f2264012700 tx=0x7f2264012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=762ab7d5046ce406 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:00.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.606+0000 7f2256ff5640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:10:00.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.606+0000 7f2256ff5640 1 -- 172.21.15.2:0/3988108471 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2264010e80 con 0x55dfeec458f0 2024-05-14T17:10:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.606+0000 7f2256ff5640 1 -- 172.21.15.2:0/3988108471 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f2264004020 con 0x55dfeec458f0 2024-05-14T17:10:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.606+0000 7f227c37c740 1 -- 172.21.15.2:0/3988108471 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f226c013170 con 0x55dfeec458f0 2024-05-14T17:10:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.607+0000 7f2256ff5640 1 -- 172.21.15.2:0/3988108471 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f2264004220 con 0x55dfeec458f0 2024-05-14T17:10:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.607+0000 7f227c37c740 10 client.16722 fetch_fsmap learned FSMap version 49 2024-05-14T17:10:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.607+0000 7f227c37c740 1 -- 172.21.15.2:0/3988108471 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55dfeec5f2e0 con 0x55dfeec458f0 2024-05-14T17:10:01.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.607+0000 7f2256ff5640 1 -- 172.21.15.2:0/3988108471 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f226c013170 con 0x55dfeec458f0 2024-05-14T17:10:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.607+0000 7f2256ff5640 1 -- 172.21.15.2:0/3988108471 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 49) ==== 8346+0+0 (secure 0 0 0) 0x7f2264024520 con 0x55dfeec458f0 2024-05-14T17:10:01.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.607+0000 7f227c37c740 10 client.16722 fetch_fsmap finished waiting for FSMap version 49 2024-05-14T17:10:01.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.607+0000 7f227c37c740 10 client.16722 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:10:01.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.607+0000 7f227c37c740 20 client.16722 populate_metadata read hostname 'smithi002' 2024-05-14T17:10:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.607+0000 7f227c37c740 1 --2- 172.21.15.2:0/3988108471 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55dfeec607f0 0x55dfeec62be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.607+0000 7f227c37c740 4 client.16722 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:10:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.607+0000 7f227c37c740 1 -- 172.21.15.2:0/3988108471 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfeebbab80 con 0x55dfeec607f0 2024-05-14T17:10:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.607+0000 7f22727fc640 1 --2- 172.21.15.2:0/3988108471 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55dfeec607f0 0x55dfeec62be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.608+0000 7f22727fc640 1 --2- 172.21.15.2:0/3988108471 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55dfeec607f0 0x55dfeec62be0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10f74b73f3187e96 server_cookie=9518c4a245ca62f4 in_seq=0 out_seq=0 2024-05-14T17:10:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.608+0000 7f2256ff5640 10 client.16722 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:10:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.608+0000 7f2256ff5640 1 -- 172.21.15.2:0/3988108471 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dfeebbab80 con 0x55dfeec607f0 2024-05-14T17:10:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.608+0000 7f2256ff5640 10 client.16722 handle_command_reply: tid=0 2024-05-14T17:10:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.608+0000 7f227c37c740 1 client.16722 shutdown 2024-05-14T17:10:02.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.608+0000 7f227c37c740 1 -- 172.21.15.2:0/3988108471 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfeec458f0 msgr2=0x55dfeec504e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:02.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.608+0000 7f227c37c740 1 --2- 172.21.15.2:0/3988108471 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfeec458f0 0x55dfeec504e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f2264012700 tx=0x7f2264012bc0 comp rx=0 tx=0).stop 2024-05-14T17:10:02.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.608+0000 7f227c37c740 1 -- 172.21.15.2:0/3988108471 shutdown_connections 2024-05-14T17:10:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.608+0000 7f227c37c740 1 --2- 172.21.15.2:0/3988108471 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55dfeec607f0 0x55dfeec62be0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:02.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.608+0000 7f227c37c740 1 --2- 172.21.15.2:0/3988108471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfeec50ad0 0x55dfeec4dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.609+0000 7f2256ff5640 0 client.16722 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:10:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.609+0000 7f227c37c740 1 --2- 172.21.15.2:0/3988108471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfeec46210 0x55dfeec4d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.609+0000 7f227c37c740 1 --2- 172.21.15.2:0/3988108471 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfeec458f0 0x55dfeec504e0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.609+0000 7f227c37c740 1 -- 172.21.15.2:0/3988108471 >> 172.21.15.2:0/3988108471 conn(0x55dfeec41e00 msgr2=0x55dfeec41100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:02.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.609+0000 7f227c37c740 1 -- 172.21.15.2:0/3988108471 shutdown_connections 2024-05-14T17:10:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.609+0000 7f227c37c740 1 -- 172.21.15.2:0/3988108471 wait complete. 2024-05-14T17:10:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.609+0000 7f227c37c740 20 client.16722 trim_cache size 0 max 16384 2024-05-14T17:10:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.630+0000 7f227c37c740 1 Processor -- start 2024-05-14T17:10:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.630+0000 7f227c37c740 1 -- start start 2024-05-14T17:10:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.630+0000 7f227c37c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfeec458f0 0x55dfeec50010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.630+0000 7f227c37c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfeec46210 0x55dfeec48b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.630+0000 7f227c37c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfeec50ad0 0x55dfeec684b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:03.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.630+0000 7f227c37c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f22741aed10 con 0x55dfeec458f0 2024-05-14T17:10:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.630+0000 7f227c37c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f22741ac580 con 0x55dfeec46210 2024-05-14T17:10:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.630+0000 7f227c37c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f22741aa3f0 con 0x55dfeec50ad0 2024-05-14T17:10:03.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.630+0000 7f2272ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfeec458f0 0x55dfeec50010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:03.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.630+0000 7f2272ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfeec458f0 0x55dfeec50010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57150/0 (socket says 172.21.15.2:57150) 2024-05-14T17:10:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.630+0000 7f2272ffd640 1 -- 172.21.15.2:0/2386783046 learned_addr learned my addr 172.21.15.2:0/2386783046 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:10:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.630+0000 7f22727fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfeec46210 0x55dfeec48b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.630+0000 7f22737fe640 1 --2- 172.21.15.2:0/2386783046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfeec50ad0 0x55dfeec684b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.630+0000 7f2272ffd640 1 -- 172.21.15.2:0/2386783046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfeec50ad0 msgr2=0x55dfeec684b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.630+0000 7f2272ffd640 1 --2- 172.21.15.2:0/2386783046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfeec50ad0 0x55dfeec684b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:03.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.630+0000 7f2272ffd640 1 -- 172.21.15.2:0/2386783046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfeec46210 msgr2=0x55dfeec48b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.630+0000 7f2272ffd640 1 --2- 172.21.15.2:0/2386783046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfeec46210 0x55dfeec48b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.630+0000 7f2272ffd640 1 -- 172.21.15.2:0/2386783046 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfeec4bd20 con 0x55dfeec458f0 2024-05-14T17:10:04.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.630+0000 7f22727fc640 1 --2- 172.21.15.2:0/2386783046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfeec46210 0x55dfeec48b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:10:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.631+0000 7f2272ffd640 1 --2- 172.21.15.2:0/2386783046 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfeec458f0 0x55dfeec50010 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2264024960 tx=0x7f226400bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67d6a079d82a4260 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:04.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.631+0000 7f2258ff9640 1 -- 172.21.15.2:0/2386783046 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2264033070 con 0x55dfeec458f0 2024-05-14T17:10:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.631+0000 7f2258ff9640 1 -- 172.21.15.2:0/2386783046 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2264005850 con 0x55dfeec458f0 2024-05-14T17:10:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.631+0000 7f2258ff9640 1 -- 172.21.15.2:0/2386783046 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226402e6a0 con 0x55dfeec458f0 2024-05-14T17:10:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.632+0000 7f227c37c740 1 -- 172.21.15.2:0/2386783046 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfeec458f0 msgr2=0x55dfeec50010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.632+0000 7f227c37c740 1 --2- 172.21.15.2:0/2386783046 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfeec458f0 0x55dfeec50010 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2264024960 tx=0x7f226400bab0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.632+0000 7f227c37c740 1 -- 172.21.15.2:0/2386783046 shutdown_connections 2024-05-14T17:10:04.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.632+0000 7f227c37c740 1 --2- 172.21.15.2:0/2386783046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfeec50ad0 0x55dfeec684b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.632+0000 7f227c37c740 1 --2- 172.21.15.2:0/2386783046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfeec46210 0x55dfeec48b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.632+0000 7f227c37c740 1 --2- 172.21.15.2:0/2386783046 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfeec458f0 0x55dfeec50010 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:04.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.632+0000 7f227c37c740 1 -- 172.21.15.2:0/2386783046 >> 172.21.15.2:0/2386783046 conn(0x55dfeec41e00 msgr2=0x55dfeec48410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.632+0000 7f227c37c740 1 -- 172.21.15.2:0/2386783046 shutdown_connections 2024-05-14T17:10:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.632+0000 7f227c37c740 1 -- 172.21.15.2:0/2386783046 wait complete. 2024-05-14T17:10:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.632+0000 7f227c37c740 1 -- start start 2024-05-14T17:10:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.632+0000 7f227c37c740 1 Processor -- start 2024-05-14T17:10:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.633+0000 7f227c37c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfeec458f0 0x55dfeec54340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:05.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.633+0000 7f227c37c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfeec46210 0x55dfeec54880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.633+0000 7f227c37c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfeec50ad0 0x55dfeec54dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:05.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.633+0000 7f227c37c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f22741aaa40 con 0x55dfeec458f0 2024-05-14T17:10:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.633+0000 7f227c37c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f22741abeb0 con 0x55dfeec46210 2024-05-14T17:10:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.633+0000 7f227c37c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f22741ab110 con 0x55dfeec50ad0 2024-05-14T17:10:05.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.633+0000 7f22737fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfeec50ad0 0x55dfeec54dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.633+0000 7f227c37c740 10 client.0 _ll_register_callbacks cb 0x7f226400be20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:10:05.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.633+0000 7f22737fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfeec50ad0 0x55dfeec54dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:43266/0 (socket says 172.21.15.2:43266) 2024-05-14T17:10:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.633+0000 7f22737fe640 1 -- 172.21.15.2:0/366820856 learned_addr learned my addr 172.21.15.2:0/366820856 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:10:05.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.633+0000 7f22727fc640 1 --2- 172.21.15.2:0/366820856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfeec46210 0x55dfeec54880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.634+0000 7f22727fc640 1 -- 172.21.15.2:0/366820856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfeec50ad0 msgr2=0x55dfeec54dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.634+0000 7f22727fc640 1 --2- 172.21.15.2:0/366820856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfeec50ad0 0x55dfeec54dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:05.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.634+0000 7f22727fc640 1 -- 172.21.15.2:0/366820856 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfeec458f0 msgr2=0x55dfeec54340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T17:10:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.634+0000 7f22727fc640 1 --2- 172.21.15.2:0/366820856 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfeec458f0 0x55dfeec54340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.634+0000 7f22727fc640 1 -- 172.21.15.2:0/366820856 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfeecc3020 con 0x55dfeec46210 2024-05-14T17:10:05.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.634+0000 7f22737fe640 1 --2- 172.21.15.2:0/366820856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfeec50ad0 0x55dfeec54dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:10:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.634+0000 7f22727fc640 1 --2- 172.21.15.2:0/366820856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfeec46210 0x55dfeec54880 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f22681003c0 tx=0x7f22680938a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36a6a2c81178212b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.634+0000 7f2256ff5640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T17:10:06.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.635+0000 7f2256ff5640 1 -- 172.21.15.2:0/366820856 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2268049040 con 0x55dfeec46210 2024-05-14T17:10:06.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.635+0000 7f2256ff5640 1 -- 172.21.15.2:0/366820856 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f22680c6290 con 0x55dfeec46210 2024-05-14T17:10:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.635+0000 7f227c37c740 1 -- 172.21.15.2:0/366820856 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2264005850 con 0x55dfeec46210 2024-05-14T17:10:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.635+0000 7f2256ff5640 1 -- 172.21.15.2:0/366820856 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f2268100640 con 0x55dfeec46210 2024-05-14T17:10:06.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.635+0000 7f227c37c740 10 client.26354 fetch_fsmap learned FSMap version 49 2024-05-14T17:10:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.635+0000 7f227c37c740 1 -- 172.21.15.2:0/366820856 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55dfeecc4330 con 0x55dfeec46210 2024-05-14T17:10:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.636+0000 7f2256ff5640 1 -- 172.21.15.2:0/366820856 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2264005850 con 0x55dfeec46210 2024-05-14T17:10:06.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.636+0000 7f2256ff5640 1 -- 172.21.15.2:0/366820856 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 49) ==== 8346+0+0 (secure 0 0 0) 0x7f22680c9260 con 0x55dfeec46210 2024-05-14T17:10:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.636+0000 7f227c37c740 10 client.26354 fetch_fsmap finished waiting for FSMap version 49 2024-05-14T17:10:06.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.636+0000 7f227c37c740 10 client.26354 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:10:06.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.636+0000 7f227c37c740 20 client.26354 populate_metadata read hostname 'smithi002' 2024-05-14T17:10:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.636+0000 7f227c37c740 1 --2- 172.21.15.2:0/366820856 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55dfeec607f0 0x55dfeecc6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:07.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.636+0000 7f227c37c740 4 client.26354 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:10:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.636+0000 7f227c37c740 1 -- 172.21.15.2:0/366820856 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dfeebb4360 con 0x55dfeec607f0 2024-05-14T17:10:07.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.636+0000 7f2272ffd640 1 --2- 172.21.15.2:0/366820856 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55dfeec607f0 0x55dfeecc6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:07.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.636+0000 7f2256ff5640 10 client.26354 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:10:07.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.636+0000 7f2272ffd640 1 --2- 172.21.15.2:0/366820856 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55dfeec607f0 0x55dfeecc6a50 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fdb5287b0a33eb5 server_cookie=4970beefec4d534e in_seq=0 out_seq=0 2024-05-14T17:10:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.652+0000 7f2256ff5640 1 -- 172.21.15.2:0/366820856 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55dfeebb4360 con 0x55dfeec607f0 2024-05-14T17:10:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.652+0000 7f2256ff5640 10 client.26354 handle_command_reply: tid=0 2024-05-14T17:10:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.652+0000 7f227c37c740 1 client.26354 shutdown 2024-05-14T17:10:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.652+0000 7f227c37c740 1 -- 172.21.15.2:0/366820856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfeec46210 msgr2=0x55dfeec54880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.652+0000 7f227c37c740 1 --2- 172.21.15.2:0/366820856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfeec46210 0x55dfeec54880 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f22681003c0 tx=0x7f22680938a0 comp rx=0 tx=0).stop 2024-05-14T17:10:07.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.652+0000 7f227c37c740 1 -- 172.21.15.2:0/366820856 shutdown_connections 2024-05-14T17:10:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.652+0000 7f227c37c740 1 --2- 172.21.15.2:0/366820856 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55dfeec607f0 0x55dfeecc6a50 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:07.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.652+0000 7f227c37c740 1 --2- 172.21.15.2:0/366820856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfeec50ad0 0x55dfeec54dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.652+0000 7f227c37c740 1 --2- 172.21.15.2:0/366820856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dfeec46210 0x55dfeec54880 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.652+0000 7f227c37c740 1 --2- 172.21.15.2:0/366820856 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dfeec458f0 0x55dfeec54340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.652+0000 7f2256ff5640 0 client.26354 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:10:08.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.652+0000 7f227c37c740 1 -- 172.21.15.2:0/366820856 >> 172.21.15.2:0/366820856 conn(0x55dfeec41e00 msgr2=0x55dfeec48ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:08.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.652+0000 7f227c37c740 1 -- 172.21.15.2:0/366820856 shutdown_connections 2024-05-14T17:10:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.652+0000 7f227c37c740 1 -- 172.21.15.2:0/366820856 wait complete. 2024-05-14T17:10:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.652+0000 7f227c37c740 20 client.26354 trim_cache size 0 max 16384 2024-05-14T17:10:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:10:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:10:08.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:10:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.653+0000 7f2279589640 1 -- 172.21.15.2:0/2843261035 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f225c0b4c20 msgr2=0x7f225c0b70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:08.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.653+0000 7f2279589640 1 --2- 172.21.15.2:0/2843261035 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f225c0b4c20 0x7f225c0b70e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f226c023000 tx=0x7f226c021000 comp rx=0 tx=0).stop 2024-05-14T17:10:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.653+0000 7f2279589640 1 -- 172.21.15.2:0/2843261035 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2274102340 msgr2=0x7f22741975a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.653+0000 7f2279589640 1 --2- 172.21.15.2:0/2843261035 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2274102340 0x7f22741975a0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f2268005ab0 tx=0x7f2268003b90 comp rx=0 tx=0).stop 2024-05-14T17:10:08.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.653+0000 7f2279589640 1 -- 172.21.15.2:0/2843261035 shutdown_connections 2024-05-14T17:10:08.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.653+0000 7f2279589640 1 --2- 172.21.15.2:0/2843261035 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f225c0b4c20 0x7f225c0b70e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.653+0000 7f2279589640 1 --2- 172.21.15.2:0/2843261035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2274107940 0x7f227419b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.653+0000 7f2279589640 1 --2- 172.21.15.2:0/2843261035 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2274102340 0x7f22741975a0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.653+0000 7f2279589640 1 --2- 172.21.15.2:0/2843261035 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2274101980 0x7f2274197060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:09.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.653+0000 7f2279589640 1 -- 172.21.15.2:0/2843261035 >> 172.21.15.2:0/2843261035 conn(0x7f2274078ad0 msgr2=0x7f2274104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.653+0000 7f2279589640 1 -- 172.21.15.2:0/2843261035 shutdown_connections 2024-05-14T17:10:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:08.653+0000 7f2279589640 1 -- 172.21.15.2:0/2843261035 wait complete. 2024-05-14T17:10:09.250 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:08:08 smithi002 ceph-mon[31538]: pgmap v418: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 9.9 MiB/s wr, 817 op/s 2024-05-14T17:10:09.322 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:08:10 smithi002 ceph-mon[31538]: osdmap e94: 12 total, 12 up, 12 in 2024-05-14T17:10:09.440 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:08:10 smithi002 ceph-mon[31538]: pgmap v420: 97 pgs: 97 active+clean; 4.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 31 MiB/s wr, 2.17k op/s 2024-05-14T17:10:09.471 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 775.0 s, 2195.6 tps, lat 45.699 ms stddev 63.887 2024-05-14T17:10:09.504 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:08:12 smithi002 ceph-mon[31538]: pgmap v421: 97 pgs: 97 active+clean; 4.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 31 MiB/s wr, 2.17k op/s 2024-05-14T17:10:09.557 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:08:14 smithi002 ceph-mon[31538]: pgmap v422: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 31 MiB/s wr, 2.18k op/s 2024-05-14T17:10:09.587 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 780.0 s, 2876.2 tps, lat 34.725 ms stddev 39.575 2024-05-14T17:10:09.620 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:09:54 smithi123 ceph-mon[35971]: pgmap v475: 97 pgs: 97 active+clean; 5.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 11 MiB/s wr, 841 op/s 2024-05-14T17:10:09.673 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:09:56 smithi123 ceph-mon[35971]: pgmap v476: 97 pgs: 97 active+clean; 5.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.5 MiB/s wr, 584 op/s 2024-05-14T17:10:09.740 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:09:58 smithi123 ceph-mon[35971]: pgmap v477: 97 pgs: 97 active+clean; 5.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.1 MiB/s wr, 579 op/s 2024-05-14T17:10:09.842 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:10:00 smithi123 ceph-mon[35971]: pgmap v478: 97 pgs: 97 active+clean; 5.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 9.5 MiB/s wr, 770 op/s 2024-05-14T17:10:09.932 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:10:00 smithi123 ceph-mon[35971]: overall HEALTH_OK 2024-05-14T17:10:10.014 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:10:02 smithi123 ceph-mon[35971]: pgmap v479: 97 pgs: 97 active+clean; 5.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.1 MiB/s wr, 528 op/s 2024-05-14T17:10:10.099 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:10:04 smithi123 ceph-mon[35971]: pgmap v480: 97 pgs: 97 active+clean; 5.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 7.8 MiB/s wr, 727 op/s 2024-05-14T17:10:10.158 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:10:04 smithi123 ceph-mon[35971]: osdmap e98: 12 total, 12 up, 12 in 2024-05-14T17:10:10.219 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:10:06 smithi123 ceph-mon[35971]: pgmap v482: 97 pgs: 97 active+clean; 5.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.6 MiB/s wr, 561 op/s 2024-05-14T17:10:10.336 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:10:08 smithi123 ceph-mon[35971]: pgmap v483: 97 pgs: 97 active+clean; 5.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 10 MiB/s wr, 797 op/s 2024-05-14T17:10:10.399 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:10:08 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:10:10.462 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:09:54 smithi151 ceph-mon[36066]: pgmap v475: 97 pgs: 97 active+clean; 5.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 11 MiB/s wr, 841 op/s 2024-05-14T17:10:10.541 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:09:56 smithi151 ceph-mon[36066]: pgmap v476: 97 pgs: 97 active+clean; 5.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.5 MiB/s wr, 584 op/s 2024-05-14T17:10:10.652 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:09:58 smithi151 ceph-mon[36066]: pgmap v477: 97 pgs: 97 active+clean; 5.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.1 MiB/s wr, 579 op/s 2024-05-14T17:10:10.714 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:10:00 smithi151 ceph-mon[36066]: pgmap v478: 97 pgs: 97 active+clean; 5.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 9.5 MiB/s wr, 770 op/s 2024-05-14T17:10:10.826 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:10:00 smithi151 ceph-mon[36066]: overall HEALTH_OK 2024-05-14T17:10:10.867 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:10:02 smithi151 ceph-mon[36066]: pgmap v479: 97 pgs: 97 active+clean; 5.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.1 MiB/s wr, 528 op/s 2024-05-14T17:10:10.934 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:10:04 smithi151 ceph-mon[36066]: pgmap v480: 97 pgs: 97 active+clean; 5.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 7.8 MiB/s wr, 727 op/s 2024-05-14T17:10:10.996 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:10:04 smithi151 ceph-mon[36066]: osdmap e98: 12 total, 12 up, 12 in 2024-05-14T17:10:11.085 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:10:06 smithi151 ceph-mon[36066]: pgmap v482: 97 pgs: 97 active+clean; 5.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.6 MiB/s wr, 561 op/s 2024-05-14T17:10:11.120 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:10:08 smithi151 ceph-mon[36066]: pgmap v483: 97 pgs: 97 active+clean; 5.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 10 MiB/s wr, 797 op/s 2024-05-14T17:10:11.154 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:10:08 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:10:11.189 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:08:16 smithi002 ceph-mon[31538]: pgmap v423: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 26 MiB/s wr, 1.86k op/s 2024-05-14T17:10:11.221 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:08:18 smithi002 ceph-mon[31538]: osdmap e95: 12 total, 12 up, 12 in 2024-05-14T17:10:11.327 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:08:18 smithi002 ceph-mon[31538]: pgmap v425: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 3.9 MiB/s wr, 409 op/s 2024-05-14T17:10:11.430 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:08:20 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:10:11.614 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:08:20 smithi002 ceph-mon[31538]: pgmap v426: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 10 MiB/s wr, 982 op/s 2024-05-14T17:10:11.715 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:10:09 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:10:11.767 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:10:09 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:10:11.865 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:10:09 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:10:11.899 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:10:10 smithi123 ceph-mon[35971]: pgmap v484: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 29 MiB/s wr, 1.84k op/s 2024-05-14T17:10:11.931 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:10:09 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:10:12.046 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:10:09 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:10:12.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:10:09 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:10:12.184 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:10:10 smithi151 ceph-mon[36066]: pgmap v484: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 29 MiB/s wr, 1.84k op/s 2024-05-14T17:10:12.215 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 785.0 s, 785.3 tps, lat 59.070 ms stddev 114.148 2024-05-14T17:10:12.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.757+0000 7f27a026d640 1 -- 172.21.15.2:0/1069681102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f279810f340 msgr2=0x7f2798111730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.757+0000 7f27a026d640 1 --2- 172.21.15.2:0/1069681102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f279810f340 0x7f2798111730 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f2788009ca0 tx=0x7f278802a950 comp rx=0 tx=0).stop 2024-05-14T17:10:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.757+0000 7f27a026d640 1 -- 172.21.15.2:0/1069681102 shutdown_connections 2024-05-14T17:10:12.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.757+0000 7f27a026d640 1 --2- 172.21.15.2:0/1069681102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f279810f340 0x7f2798111730 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.757+0000 7f27a026d640 1 --2- 172.21.15.2:0/1069681102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2798075640 0x7f279810ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.757+0000 7f27a026d640 1 --2- 172.21.15.2:0/1069681102 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2798074d20 0x7f2798075100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.757+0000 7f27a026d640 1 -- 172.21.15.2:0/1069681102 >> 172.21.15.2:0/1069681102 conn(0x7f2798079ee0 msgr2=0x7f279807a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.757+0000 7f27a026d640 1 -- 172.21.15.2:0/1069681102 shutdown_connections 2024-05-14T17:10:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.757+0000 7f27a026d640 1 -- 172.21.15.2:0/1069681102 wait complete. 2024-05-14T17:10:12.788 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.758+0000 7f27a026d640 1 Processor -- start 2024-05-14T17:10:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.758+0000 7f27a026d640 1 -- start start 2024-05-14T17:10:12.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.758+0000 7f27a026d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2798074d20 0x7f2798102a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.758+0000 7f27a026d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2798075640 0x7f2798102fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.758+0000 7f27a026d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f279810f340 0x7f27981034e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.758+0000 7f27a026d640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2798114ef0 con 0x7f2798074d20 2024-05-14T17:10:13.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.758+0000 7f27a026d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2798114d70 con 0x7f279810f340 2024-05-14T17:10:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.758+0000 7f27a026d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2798115070 con 0x7f2798075640 2024-05-14T17:10:13.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.758+0000 7f279e009640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2798074d20 0x7f2798102a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.758+0000 7f279e009640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2798074d20 0x7f2798102a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57450/0 (socket says 172.21.15.2:57450) 2024-05-14T17:10:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.758+0000 7f279e009640 1 -- 172.21.15.2:0/2457094292 learned_addr learned my addr 172.21.15.2:0/2457094292 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:10:13.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.758+0000 7f279e80a640 1 --2- 172.21.15.2:0/2457094292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f279810f340 0x7f27981034e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:13.548 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.759+0000 7f279d808640 1 --2- 172.21.15.2:0/2457094292 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2798075640 0x7f2798102fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.759+0000 7f279e009640 1 -- 172.21.15.2:0/2457094292 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2798075640 msgr2=0x7f2798102fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:13.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.759+0000 7f279e009640 1 --2- 172.21.15.2:0/2457094292 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2798075640 0x7f2798102fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.759+0000 7f279e009640 1 -- 172.21.15.2:0/2457094292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f279810f340 msgr2=0x7f27981034e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.759+0000 7f279e009640 1 --2- 172.21.15.2:0/2457094292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f279810f340 0x7f27981034e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.759+0000 7f279e009640 1 -- 172.21.15.2:0/2457094292 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27981aba20 con 0x7f2798074d20 2024-05-14T17:10:13.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.759+0000 7f279e009640 1 --2- 172.21.15.2:0/2457094292 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2798074d20 0x7f2798102a60 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f279000ea20 tx=0x7f279000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.759+0000 7f27977fe640 1 -- 172.21.15.2:0/2457094292 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279000cd80 con 0x7f2798074d20 2024-05-14T17:10:14.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.759+0000 7f27977fe640 1 -- 172.21.15.2:0/2457094292 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27900108e0 con 0x7f2798074d20 2024-05-14T17:10:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.759+0000 7f27a026d640 1 -- 172.21.15.2:0/2457094292 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27981abd10 con 0x7f2798074d20 2024-05-14T17:10:14.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.759+0000 7f27977fe640 1 -- 172.21.15.2:0/2457094292 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790018840 con 0x7f2798074d20 2024-05-14T17:10:14.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.759+0000 7f27a026d640 1 -- 172.21.15.2:0/2457094292 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27981ac250 con 0x7f2798074d20 2024-05-14T17:10:14.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.762+0000 7f27977fe640 1 -- 172.21.15.2:0/2457094292 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f2790010430 con 0x7f2798074d20 2024-05-14T17:10:14.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.762+0000 7f27a2f82740 1 Processor -- start 2024-05-14T17:10:14.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.762+0000 7f27a2f82740 1 -- start start 2024-05-14T17:10:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.763+0000 7f27a2f82740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557237d7c8f0 0x557237d7ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.763+0000 7f27a2f82740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557237d7d210 0x557237d87590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.763+0000 7f27a2f82740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557237d87ad0 0x557237d89ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.763+0000 7f27a2f82740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x557237cefc30 con 0x557237d7d210 2024-05-14T17:10:15.011 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.763+0000 7f27a2f82740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557237cdf7d0 con 0x557237d87ad0 2024-05-14T17:10:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.763+0000 7f27a2f82740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557237ce3810 con 0x557237d7c8f0 2024-05-14T17:10:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.763+0000 7f279e009640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557237d7d210 0x557237d87590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:15.178 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.763+0000 7f279e009640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557237d7d210 0x557237d87590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57464/0 (socket says 172.21.15.2:57464) 2024-05-14T17:10:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.763+0000 7f279e009640 1 -- 172.21.15.2:0/235330760 learned_addr learned my addr 172.21.15.2:0/235330760 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:10:15.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.763+0000 7f279e80a640 1 --2- 172.21.15.2:0/235330760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557237d7c8f0 0x557237d7ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.763+0000 7f279d808640 1 --2- 172.21.15.2:0/235330760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557237d87ad0 0x557237d89ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.763+0000 7f279e009640 1 -- 172.21.15.2:0/235330760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557237d7c8f0 msgr2=0x557237d7ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:15.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.763+0000 7f279e009640 1 --2- 172.21.15.2:0/235330760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557237d7c8f0 0x557237d7ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.763+0000 7f279e009640 1 -- 172.21.15.2:0/235330760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557237d87ad0 msgr2=0x557237d89ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.763+0000 7f279e009640 1 --2- 172.21.15.2:0/235330760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557237d87ad0 0x557237d89ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.763+0000 7f279e009640 1 -- 172.21.15.2:0/235330760 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572379f4210 con 0x557237d7d210 2024-05-14T17:10:15.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.763+0000 7f27977fe640 1 --2- 172.21.15.2:0/2457094292 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f27800b2a90 0x7f27800b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.763+0000 7f279d808640 1 --2- 172.21.15.2:0/235330760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557237d87ad0 0x557237d89ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:10:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.764+0000 7f279e80a640 1 --2- 172.21.15.2:0/2457094292 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f27800b2a90 0x7f27800b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.764+0000 7f279e009640 1 --2- 172.21.15.2:0/235330760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557237d7d210 0x557237d87590 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f27900e0020 tx=0x7f27900a6290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce646fdf0e74e60 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.764+0000 7f27977fe640 1 -- 172.21.15.2:0/2457094292 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f27900e2960 con 0x7f2798074d20 2024-05-14T17:10:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.764+0000 7f2794bfe640 1 -- 172.21.15.2:0/235330760 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790026030 con 0x557237d7d210 2024-05-14T17:10:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.764+0000 7f2794bfe640 1 -- 172.21.15.2:0/235330760 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27900b23e0 con 0x557237d7d210 2024-05-14T17:10:16.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.764+0000 7f2794bfe640 1 -- 172.21.15.2:0/235330760 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27900a93b0 con 0x557237d7d210 2024-05-14T17:10:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.765+0000 7f27a2f82740 1 -- 172.21.15.2:0/235330760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557237d7d210 msgr2=0x557237d87590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:16.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.765+0000 7f27a2f82740 1 --2- 172.21.15.2:0/235330760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557237d7d210 0x557237d87590 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f27900e0020 tx=0x7f27900a6290 comp rx=0 tx=0).stop 2024-05-14T17:10:16.634 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.765+0000 7f279e80a640 1 --2- 172.21.15.2:0/2457094292 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f27800b2a90 0x7f27800b4f50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2788005e30 tx=0x7f2788005c50 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.765+0000 7f27a2f82740 1 -- 172.21.15.2:0/235330760 shutdown_connections 2024-05-14T17:10:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.765+0000 7f27a2f82740 1 --2- 172.21.15.2:0/235330760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557237d87ad0 0x557237d89ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:16.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.765+0000 7f27a2f82740 1 --2- 172.21.15.2:0/235330760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557237d7d210 0x557237d87590 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:16.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.765+0000 7f27a2f82740 1 --2- 172.21.15.2:0/235330760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557237d7c8f0 0x557237d7ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.765+0000 7f27a2f82740 1 -- 172.21.15.2:0/235330760 >> 172.21.15.2:0/235330760 conn(0x557237d78e00 msgr2=0x557237d791e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:17.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.765+0000 7f27a2f82740 1 -- 172.21.15.2:0/235330760 shutdown_connections 2024-05-14T17:10:17.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.765+0000 7f27a2f82740 1 -- 172.21.15.2:0/235330760 wait complete. 2024-05-14T17:10:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.765+0000 7f27a2f82740 1 -- start start 2024-05-14T17:10:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.765+0000 7f27a2f82740 1 Processor -- start 2024-05-14T17:10:17.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.766+0000 7f27a2f82740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557237d7c8f0 0x557237d87460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.766+0000 7f27a2f82740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557237d7d210 0x557237d84750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.766+0000 7f27a2f82740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557237d87ad0 0x557237d84cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.766+0000 7f27a2f82740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5572379c5790 con 0x557237d7d210 2024-05-14T17:10:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.766+0000 7f27a2f82740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557237c9b890 con 0x557237d7c8f0 2024-05-14T17:10:17.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.766+0000 7f27a2f82740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557237d8a7a0 con 0x557237d87ad0 2024-05-14T17:10:18.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.766+0000 7f27a2f82740 10 client.0 _ll_register_callbacks cb 0x557237c9ed60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:10:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.766+0000 7f279d808640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557237d7d210 0x557237d84750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.766+0000 7f279d808640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557237d7d210 0x557237d84750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57466/0 (socket says 172.21.15.2:57466) 2024-05-14T17:10:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.766+0000 7f279d808640 1 -- 172.21.15.2:0/4196559784 learned_addr learned my addr 172.21.15.2:0/4196559784 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:10:18.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.766+0000 7f279e009640 1 --2- 172.21.15.2:0/4196559784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557237d7c8f0 0x557237d87460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.766+0000 7f279e80a640 1 --2- 172.21.15.2:0/4196559784 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557237d87ad0 0x557237d84cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.766+0000 7f279d808640 1 -- 172.21.15.2:0/4196559784 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557237d87ad0 msgr2=0x557237d84cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.766+0000 7f279d808640 1 --2- 172.21.15.2:0/4196559784 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557237d87ad0 0x557237d84cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.766+0000 7f279d808640 1 -- 172.21.15.2:0/4196559784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557237d7c8f0 msgr2=0x557237d87460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:18.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.766+0000 7f279d808640 1 --2- 172.21.15.2:0/4196559784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557237d7c8f0 0x557237d87460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:18.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.766+0000 7f279d808640 1 -- 172.21.15.2:0/4196559784 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557237ac0620 con 0x557237d7d210 2024-05-14T17:10:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.766+0000 7f279d808640 1 --2- 172.21.15.2:0/4196559784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557237d7d210 0x557237d84750 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f278c013560 tx=0x7f278c013a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8c369b8e7990530 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.766+0000 7f277dffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:10:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.766+0000 7f277dffb640 1 -- 172.21.15.2:0/4196559784 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c020060 con 0x557237d7d210 2024-05-14T17:10:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.766+0000 7f277dffb640 1 -- 172.21.15.2:0/4196559784 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f278c004460 con 0x557237d7d210 2024-05-14T17:10:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.766+0000 7f27a2f82740 1 -- 172.21.15.2:0/4196559784 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557237cf1b80 con 0x557237d7d210 2024-05-14T17:10:19.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.767+0000 7f277dffb640 1 -- 172.21.15.2:0/4196559784 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f278c011660 con 0x557237d7d210 2024-05-14T17:10:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.767+0000 7f27a2f82740 10 client.16737 fetch_fsmap learned FSMap version 50 2024-05-14T17:10:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.767+0000 7f27a2f82740 1 -- 172.21.15.2:0/4196559784 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x557237d96290 con 0x557237d7d210 2024-05-14T17:10:19.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.767+0000 7f277dffb640 1 -- 172.21.15.2:0/4196559784 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557237cf1b80 con 0x557237d7d210 2024-05-14T17:10:19.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.767+0000 7f277dffb640 1 -- 172.21.15.2:0/4196559784 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) ==== 8350+0+0 (secure 0 0 0) 0x7f278c01a030 con 0x557237d7d210 2024-05-14T17:10:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.767+0000 7f27a2f82740 10 client.16737 fetch_fsmap finished waiting for FSMap version 50 2024-05-14T17:10:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.767+0000 7f27a2f82740 10 client.16737 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:10:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.767+0000 7f27a2f82740 20 client.16737 populate_metadata read hostname 'smithi002' 2024-05-14T17:10:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.767+0000 7f27a2f82740 1 --2- 172.21.15.2:0/4196559784 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x557237d977c0 0x557237d99bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.767+0000 7f27a2f82740 4 client.16737 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:10:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.767+0000 7f27a2f82740 1 -- 172.21.15.2:0/4196559784 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557237ceb360 con 0x557237d977c0 2024-05-14T17:10:20.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.767+0000 7f279e009640 1 --2- 172.21.15.2:0/4196559784 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x557237d977c0 0x557237d99bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.768+0000 7f279e009640 1 --2- 172.21.15.2:0/4196559784 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x557237d977c0 0x557237d99bb0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68114e40700d5323 server_cookie=1f3260c6cc16302d in_seq=0 out_seq=0 2024-05-14T17:10:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.768+0000 7f277dffb640 10 client.16737 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:10:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.769+0000 7f277dffb640 1 -- 172.21.15.2:0/4196559784 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557237ceb360 con 0x557237d977c0 2024-05-14T17:10:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.769+0000 7f277dffb640 10 client.16737 handle_command_reply: tid=0 2024-05-14T17:10:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.769+0000 7f27a2f82740 1 client.16737 shutdown 2024-05-14T17:10:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.769+0000 7f27a2f82740 1 -- 172.21.15.2:0/4196559784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557237d7d210 msgr2=0x557237d84750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:20.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.769+0000 7f27a2f82740 1 --2- 172.21.15.2:0/4196559784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557237d7d210 0x557237d84750 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f278c013560 tx=0x7f278c013a20 comp rx=0 tx=0).stop 2024-05-14T17:10:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.769+0000 7f27a2f82740 1 -- 172.21.15.2:0/4196559784 shutdown_connections 2024-05-14T17:10:20.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.769+0000 7f27a2f82740 1 --2- 172.21.15.2:0/4196559784 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x557237d977c0 0x557237d99bb0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.769+0000 7f27a2f82740 1 --2- 172.21.15.2:0/4196559784 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557237d87ad0 0x557237d84cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:20.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.769+0000 7f277dffb640 0 client.16737 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:10:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.769+0000 7f27a2f82740 1 --2- 172.21.15.2:0/4196559784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557237d7d210 0x557237d84750 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.769+0000 7f27a2f82740 1 --2- 172.21.15.2:0/4196559784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557237d7c8f0 0x557237d87460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.769+0000 7f27a2f82740 1 -- 172.21.15.2:0/4196559784 >> 172.21.15.2:0/4196559784 conn(0x557237d78e00 msgr2=0x557237d78100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:21.031 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.769+0000 7f27a2f82740 1 -- 172.21.15.2:0/4196559784 shutdown_connections 2024-05-14T17:10:21.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.769+0000 7f27a2f82740 1 -- 172.21.15.2:0/4196559784 wait complete. 2024-05-14T17:10:21.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.769+0000 7f27a2f82740 20 client.16737 trim_cache size 0 max 16384 2024-05-14T17:10:21.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.784+0000 7f27a2f82740 1 Processor -- start 2024-05-14T17:10:21.283 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.784+0000 7f27a2f82740 1 -- start start 2024-05-14T17:10:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.785+0000 7f27a2f82740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557237d7c8f0 0x557237d93670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.785+0000 7f27a2f82740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557237d7d210 0x557237d86f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.785+0000 7f27a2f82740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557237d87ad0 0x557237d9f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.785+0000 7f27a2f82740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f27981b75d0 con 0x557237d7d210 2024-05-14T17:10:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.785+0000 7f27a2f82740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f27981b4e40 con 0x557237d87ad0 2024-05-14T17:10:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.785+0000 7f27a2f82740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f27981b2cb0 con 0x557237d7c8f0 2024-05-14T17:10:21.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.785+0000 7f279d808640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557237d7d210 0x557237d86f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:22.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.785+0000 7f279d808640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557237d7d210 0x557237d86f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57474/0 (socket says 172.21.15.2:57474) 2024-05-14T17:10:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.785+0000 7f279d808640 1 -- 172.21.15.2:0/3506358008 learned_addr learned my addr 172.21.15.2:0/3506358008 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:10:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.785+0000 7f279e009640 1 --2- 172.21.15.2:0/3506358008 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557237d7c8f0 0x557237d93670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.785+0000 7f279d808640 1 -- 172.21.15.2:0/3506358008 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557237d7c8f0 msgr2=0x557237d93670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.785+0000 7f279d808640 1 --2- 172.21.15.2:0/3506358008 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557237d7c8f0 0x557237d93670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.785+0000 7f279e80a640 1 --2- 172.21.15.2:0/3506358008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557237d87ad0 0x557237d9f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.785+0000 7f279d808640 1 -- 172.21.15.2:0/3506358008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557237d87ad0 msgr2=0x557237d9f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:22.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.785+0000 7f279d808640 1 --2- 172.21.15.2:0/3506358008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557237d87ad0 0x557237d9f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.785+0000 7f279d808640 1 -- 172.21.15.2:0/3506358008 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557237d85040 con 0x557237d7d210 2024-05-14T17:10:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.785+0000 7f279e009640 1 --2- 172.21.15.2:0/3506358008 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557237d7c8f0 0x557237d93670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:10:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.785+0000 7f279e80a640 1 --2- 172.21.15.2:0/3506358008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557237d87ad0 0x557237d9f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:10:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.786+0000 7f279d808640 1 --2- 172.21.15.2:0/3506358008 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557237d7d210 0x557237d86f90 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f278c003d00 tx=0x7f278c00f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f9831bd6373b185 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.786+0000 7f277ffff640 1 -- 172.21.15.2:0/3506358008 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c032070 con 0x557237d7d210 2024-05-14T17:10:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.786+0000 7f277ffff640 1 -- 172.21.15.2:0/3506358008 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f278c003aa0 con 0x557237d7d210 2024-05-14T17:10:23.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.786+0000 7f277ffff640 1 -- 172.21.15.2:0/3506358008 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c00f7a0 con 0x557237d7d210 2024-05-14T17:10:23.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.786+0000 7f27a2f82740 1 -- 172.21.15.2:0/3506358008 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557237d7d210 msgr2=0x557237d86f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.786+0000 7f27a2f82740 1 --2- 172.21.15.2:0/3506358008 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557237d7d210 0x557237d86f90 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f278c003d00 tx=0x7f278c00f450 comp rx=0 tx=0).stop 2024-05-14T17:10:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.786+0000 7f27a2f82740 1 -- 172.21.15.2:0/3506358008 shutdown_connections 2024-05-14T17:10:23.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.786+0000 7f27a2f82740 1 --2- 172.21.15.2:0/3506358008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557237d87ad0 0x557237d9f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.786+0000 7f27a2f82740 1 --2- 172.21.15.2:0/3506358008 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557237d7d210 0x557237d86f90 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.786+0000 7f27a2f82740 1 --2- 172.21.15.2:0/3506358008 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557237d7c8f0 0x557237d93670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:23.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.786+0000 7f27a2f82740 1 -- 172.21.15.2:0/3506358008 >> 172.21.15.2:0/3506358008 conn(0x557237d78e00 msgr2=0x557237d7eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.786+0000 7f27a2f82740 1 -- 172.21.15.2:0/3506358008 shutdown_connections 2024-05-14T17:10:23.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.787+0000 7f27a2f82740 1 -- 172.21.15.2:0/3506358008 wait complete. 2024-05-14T17:10:23.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.787+0000 7f27a2f82740 1 -- start start 2024-05-14T17:10:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.787+0000 7f27a2f82740 1 Processor -- start 2024-05-14T17:10:23.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.787+0000 7f27a2f82740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557237d7c8f0 0x557237d8fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.787+0000 7f27a2f82740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557237d7d210 0x557237d8b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:24.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.787+0000 7f27a2f82740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557237d87ad0 0x557237d8b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:24.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.787+0000 7f27a2f82740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f27981b3300 con 0x557237d7d210 2024-05-14T17:10:24.237 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.787+0000 7f27a2f82740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f27981b4770 con 0x557237d87ad0 2024-05-14T17:10:24.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.787+0000 7f27a2f82740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f27981b39d0 con 0x557237d7c8f0 2024-05-14T17:10:24.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.787+0000 7f27a2f82740 10 client.0 _ll_register_callbacks cb 0x7f278c009c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:10:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.787+0000 7f279d808640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557237d7d210 0x557237d8b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:24.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.787+0000 7f279d808640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557237d7d210 0x557237d8b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57490/0 (socket says 172.21.15.2:57490) 2024-05-14T17:10:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.787+0000 7f279d808640 1 -- 172.21.15.2:0/116643478 learned_addr learned my addr 172.21.15.2:0/116643478 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:10:24.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.787+0000 7f279e80a640 1 --2- 172.21.15.2:0/116643478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557237d87ad0 0x557237d8b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.787+0000 7f279e009640 1 --2- 172.21.15.2:0/116643478 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557237d7c8f0 0x557237d8fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.788+0000 7f279d808640 1 -- 172.21.15.2:0/116643478 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557237d7c8f0 msgr2=0x557237d8fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:24.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.788+0000 7f279d808640 1 --2- 172.21.15.2:0/116643478 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557237d7c8f0 0x557237d8fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.788+0000 7f279d808640 1 -- 172.21.15.2:0/116643478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557237d87ad0 msgr2=0x557237d8b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.788+0000 7f279d808640 1 --2- 172.21.15.2:0/116643478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557237d87ad0 0x557237d8b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:25.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.788+0000 7f279d808640 1 -- 172.21.15.2:0/116643478 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557237d8bd20 con 0x557237d7d210 2024-05-14T17:10:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.788+0000 7f279e80a640 1 --2- 172.21.15.2:0/116643478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557237d87ad0 0x557237d8b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:10:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.788+0000 7f279d808640 1 --2- 172.21.15.2:0/116643478 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557237d7d210 0x557237d8b170 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f278c0114b0 tx=0x7f278c011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=372e7eaa3e4298b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:25.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.788+0000 7f277dffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:10:25.378 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.788+0000 7f277dffb640 1 -- 172.21.15.2:0/116643478 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c032040 con 0x557237d7d210 2024-05-14T17:10:25.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.788+0000 7f277dffb640 1 -- 172.21.15.2:0/116643478 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f278c0037f0 con 0x557237d7d210 2024-05-14T17:10:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.788+0000 7f27a2f82740 1 -- 172.21.15.2:0/116643478 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f278c003aa0 con 0x557237d7d210 2024-05-14T17:10:25.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.788+0000 7f277dffb640 1 -- 172.21.15.2:0/116643478 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f278c043050 con 0x557237d7d210 2024-05-14T17:10:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.788+0000 7f27a2f82740 10 client.16743 fetch_fsmap learned FSMap version 50 2024-05-14T17:10:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.788+0000 7f27a2f82740 1 -- 172.21.15.2:0/116643478 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x557237e01ed0 con 0x557237d7d210 2024-05-14T17:10:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.788+0000 7f277dffb640 1 -- 172.21.15.2:0/116643478 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f278c003aa0 con 0x557237d7d210 2024-05-14T17:10:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.788+0000 7f277dffb640 1 -- 172.21.15.2:0/116643478 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 50) ==== 8350+0+0 (secure 0 0 0) 0x7f278c03e030 con 0x557237d7d210 2024-05-14T17:10:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.789+0000 7f27a2f82740 10 client.16743 fetch_fsmap finished waiting for FSMap version 50 2024-05-14T17:10:25.939 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.789+0000 7f27a2f82740 10 client.16743 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:10:26.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.789+0000 7f27a2f82740 20 client.16743 populate_metadata read hostname 'smithi002' 2024-05-14T17:10:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.789+0000 7f27a2f82740 1 --2- 172.21.15.2:0/116643478 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x557237e02560 0x557237e04980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.789+0000 7f27a2f82740 4 client.16743 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:10:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.789+0000 7f27a2f82740 1 -- 172.21.15.2:0/116643478 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557237c655f0 con 0x557237e02560 2024-05-14T17:10:26.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.789+0000 7f279e009640 1 --2- 172.21.15.2:0/116643478 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x557237e02560 0x557237e04980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:26.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.789+0000 7f279e009640 1 --2- 172.21.15.2:0/116643478 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x557237e02560 0x557237e04980 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77c7aabfcc195e9c server_cookie=e0e91fd14eaede40 in_seq=0 out_seq=0 2024-05-14T17:10:26.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.789+0000 7f277dffb640 10 client.16743 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:10:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.852+0000 7f277dffb640 1 -- 172.21.15.2:0/116643478 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557237c655f0 con 0x557237e02560 2024-05-14T17:10:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.852+0000 7f277dffb640 10 client.16743 handle_command_reply: tid=0 2024-05-14T17:10:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.853+0000 7f27a2f82740 1 client.16743 shutdown 2024-05-14T17:10:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.854+0000 7f27a2f82740 1 -- 172.21.15.2:0/116643478 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557237d7d210 msgr2=0x557237d8b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.855+0000 7f27a2f82740 1 --2- 172.21.15.2:0/116643478 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557237d7d210 0x557237d8b170 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f278c0114b0 tx=0x7f278c011e70 comp rx=0 tx=0).stop 2024-05-14T17:10:26.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.856+0000 7f27a2f82740 1 -- 172.21.15.2:0/116643478 shutdown_connections 2024-05-14T17:10:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.856+0000 7f27a2f82740 1 --2- 172.21.15.2:0/116643478 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x557237e02560 0x557237e04980 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:27.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.856+0000 7f27a2f82740 1 --2- 172.21.15.2:0/116643478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557237d87ad0 0x557237d8b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.856+0000 7f27a2f82740 1 --2- 172.21.15.2:0/116643478 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557237d7d210 0x557237d8b170 secure :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f278c0114b0 tx=0x7f278c011e70 comp rx=0 tx=0).stop 2024-05-14T17:10:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.856+0000 7f27a2f82740 1 --2- 172.21.15.2:0/116643478 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557237d7c8f0 0x557237d8fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.856+0000 7f27a2f82740 1 -- 172.21.15.2:0/116643478 >> 172.21.15.2:0/116643478 conn(0x557237d78e00 msgr2=0x557237d77870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:27.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.856+0000 7f277dffb640 0 client.16743 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:10:27.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.857+0000 7f27a2f82740 1 -- 172.21.15.2:0/116643478 shutdown_connections 2024-05-14T17:10:27.634 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.857+0000 7f27a2f82740 1 -- 172.21.15.2:0/116643478 wait complete. 2024-05-14T17:10:27.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.857+0000 7f27a2f82740 20 client.16743 trim_cache size 0 max 16384 2024-05-14T17:10:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:10:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:10:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:10:28.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.858+0000 7f27a026d640 1 -- 172.21.15.2:0/2457094292 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f27800b2a90 msgr2=0x7f27800b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.858+0000 7f27a026d640 1 --2- 172.21.15.2:0/2457094292 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f27800b2a90 0x7f27800b4f50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2788005e30 tx=0x7f2788005c50 comp rx=0 tx=0).stop 2024-05-14T17:10:28.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.858+0000 7f27a026d640 1 -- 172.21.15.2:0/2457094292 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2798074d20 msgr2=0x7f2798102a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.858+0000 7f27a026d640 1 --2- 172.21.15.2:0/2457094292 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2798074d20 0x7f2798102a60 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f279000ea20 tx=0x7f279000eee0 comp rx=0 tx=0).stop 2024-05-14T17:10:28.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.858+0000 7f27a026d640 1 -- 172.21.15.2:0/2457094292 shutdown_connections 2024-05-14T17:10:28.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.858+0000 7f27a026d640 1 --2- 172.21.15.2:0/2457094292 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f27800b2a90 0x7f27800b4f50 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.858+0000 7f27a026d640 1 --2- 172.21.15.2:0/2457094292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f279810f340 0x7f27981034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:28.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.858+0000 7f27a026d640 1 --2- 172.21.15.2:0/2457094292 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2798075640 0x7f2798102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:28.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.858+0000 7f27a026d640 1 --2- 172.21.15.2:0/2457094292 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2798074d20 0x7f2798102a60 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.858+0000 7f27a026d640 1 -- 172.21.15.2:0/2457094292 >> 172.21.15.2:0/2457094292 conn(0x7f2798079ee0 msgr2=0x7f27980ff930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.858+0000 7f27a026d640 1 -- 172.21.15.2:0/2457094292 shutdown_connections 2024-05-14T17:10:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:21.858+0000 7f27a026d640 1 -- 172.21.15.2:0/2457094292 wait complete. 2024-05-14T17:10:29.045 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:08:22 smithi002 ceph-mon[31538]: pgmap v427: 97 pgs: 97 active+clean; 4.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 10 MiB/s wr, 982 op/s 2024-05-14T17:10:29.106 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:08:24 smithi002 ceph-mon[31538]: pgmap v428: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 19 MiB/s wr, 1.69k op/s 2024-05-14T17:10:29.150 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 790.0 s, 2276.3 tps, lat 67.222 ms stddev 266.748 2024-05-14T17:10:29.254 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:08:26 smithi002 ceph-mon[31538]: pgmap v429: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 20 MiB/s wr, 1.76k op/s 2024-05-14T17:10:29.356 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:08:28 smithi002 ceph-mon[31538]: pgmap v430: 97 pgs: 97 active+clean; 4.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 20 MiB/s wr, 1.72k op/s 2024-05-14T17:10:29.432 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:08:30 smithi002 ceph-mon[31538]: pgmap v431: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 19 MiB/s wr, 1.73k op/s 2024-05-14T17:10:29.539 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 795.0 s, 1673.4 tps, lat 53.642 ms stddev 61.171 2024-05-14T17:10:29.601 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:08:32 smithi002 ceph-mon[31538]: pgmap v432: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 1.20k op/s 2024-05-14T17:10:29.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:10:12 smithi123 ceph-mon[35971]: pgmap v485: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 29 MiB/s wr, 1.84k op/s 2024-05-14T17:10:29.750 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:10:12 smithi123 ceph-mon[35971]: osdmap e99: 12 total, 12 up, 12 in 2024-05-14T17:10:29.844 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:10:14 smithi123 ceph-mon[35971]: pgmap v487: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 30 MiB/s wr, 1.96k op/s 2024-05-14T17:10:29.880 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:10:16 smithi123 ceph-mon[35971]: pgmap v488: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 31 MiB/s wr, 1.99k op/s 2024-05-14T17:10:29.913 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:10:18 smithi123 ceph-mon[35971]: pgmap v489: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 27 MiB/s wr, 1.77k op/s 2024-05-14T17:10:29.947 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:10:20 smithi123 ceph-mon[35971]: pgmap v490: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.3 MiB/s wr, 741 op/s 2024-05-14T17:10:29.980 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:10:22 smithi123 ceph-mon[35971]: pgmap v491: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.3 MiB/s wr, 741 op/s 2024-05-14T17:10:30.023 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:10:24 smithi123 ceph-mon[35971]: pgmap v492: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.9 MiB/s wr, 715 op/s 2024-05-14T17:10:30.307 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:10:26 smithi123 ceph-mon[35971]: pgmap v493: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 3.8 MiB/s wr, 387 op/s 2024-05-14T17:10:30.355 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:10:28 smithi123 ceph-mon[35971]: pgmap v494: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 3.3 MiB/s wr, 341 op/s 2024-05-14T17:10:30.447 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:10:12 smithi151 ceph-mon[36066]: pgmap v485: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 29 MiB/s wr, 1.84k op/s 2024-05-14T17:10:30.503 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:10:12 smithi151 ceph-mon[36066]: osdmap e99: 12 total, 12 up, 12 in 2024-05-14T17:10:30.622 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:10:14 smithi151 ceph-mon[36066]: pgmap v487: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 30 MiB/s wr, 1.96k op/s 2024-05-14T17:10:30.711 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:10:16 smithi151 ceph-mon[36066]: pgmap v488: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 31 MiB/s wr, 1.99k op/s 2024-05-14T17:10:30.794 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:10:18 smithi151 ceph-mon[36066]: pgmap v489: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 27 MiB/s wr, 1.77k op/s 2024-05-14T17:10:30.888 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:10:20 smithi151 ceph-mon[36066]: pgmap v490: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.3 MiB/s wr, 741 op/s 2024-05-14T17:10:30.922 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:10:22 smithi151 ceph-mon[36066]: pgmap v491: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.3 MiB/s wr, 741 op/s 2024-05-14T17:10:31.090 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:10:24 smithi151 ceph-mon[36066]: pgmap v492: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.9 MiB/s wr, 715 op/s 2024-05-14T17:10:31.164 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:10:26 smithi151 ceph-mon[36066]: pgmap v493: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 3.8 MiB/s wr, 387 op/s 2024-05-14T17:10:31.226 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:10:28 smithi151 ceph-mon[36066]: pgmap v494: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 3.3 MiB/s wr, 341 op/s 2024-05-14T17:10:31.311 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:10:30 smithi123 ceph-mon[35971]: pgmap v495: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 3.2 MiB/s wr, 340 op/s 2024-05-14T17:10:31.411 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:10:30 smithi151 ceph-mon[36066]: pgmap v495: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 3.2 MiB/s wr, 340 op/s 2024-05-14T17:10:31.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.962+0000 7fcd94883640 1 -- 172.21.15.2:0/130839894 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd90108a70 msgr2=0x7fcd90108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:31.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.962+0000 7fcd94883640 1 --2- 172.21.15.2:0/130839894 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd90108a70 0x7fcd90108e50 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fcd80009930 tx=0x7fcd8002a420 comp rx=0 tx=0).stop 2024-05-14T17:10:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.962+0000 7fcd94883640 1 -- 172.21.15.2:0/130839894 shutdown_connections 2024-05-14T17:10:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.962+0000 7fcd94883640 1 --2- 172.21.15.2:0/130839894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd90103420 0x7fcd9010f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.962+0000 7fcd94883640 1 --2- 172.21.15.2:0/130839894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd90102a60 0x7fcd90102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.962+0000 7fcd94883640 1 --2- 172.21.15.2:0/130839894 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd90108a70 0x7fcd90108e50 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:31.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.962+0000 7fcd94883640 1 -- 172.21.15.2:0/130839894 >> 172.21.15.2:0/130839894 conn(0x7fcd90079ee0 msgr2=0x7fcd9007a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:31.943 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.962+0000 7fcd94883640 1 -- 172.21.15.2:0/130839894 shutdown_connections 2024-05-14T17:10:32.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.962+0000 7fcd94883640 1 -- 172.21.15.2:0/130839894 wait complete. 2024-05-14T17:10:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.963+0000 7fcd94883640 1 Processor -- start 2024-05-14T17:10:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.963+0000 7fcd94883640 1 -- start start 2024-05-14T17:10:32.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.963+0000 7fcd94883640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd90102a60 0x7fcd901991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.963+0000 7fcd94883640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd90103420 0x7fcd90199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.963+0000 7fcd94883640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd90108a70 0x7fcd9019da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:32.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.963+0000 7fcd94883640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcd90112d80 con 0x7fcd90103420 2024-05-14T17:10:32.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.963+0000 7fcd94883640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcd90112c00 con 0x7fcd90108a70 2024-05-14T17:10:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.963+0000 7fcd94883640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcd90112f00 con 0x7fcd90102a60 2024-05-14T17:10:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.963+0000 7fcd8dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd90103420 0x7fcd90199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.963+0000 7fcd8dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd90103420 0x7fcd90199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53914/0 (socket says 172.21.15.2:53914) 2024-05-14T17:10:33.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.964+0000 7fcd8dd9b640 1 -- 172.21.15.2:0/990819103 learned_addr learned my addr 172.21.15.2:0/990819103 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:10:33.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.964+0000 7fcd8e59c640 1 --2- 172.21.15.2:0/990819103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd90102a60 0x7fcd901991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.964+0000 7fcd8ed9d640 1 --2- 172.21.15.2:0/990819103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd90108a70 0x7fcd9019da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.964+0000 7fcd8dd9b640 1 -- 172.21.15.2:0/990819103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd90102a60 msgr2=0x7fcd901991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:33.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.964+0000 7fcd8dd9b640 1 --2- 172.21.15.2:0/990819103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd90102a60 0x7fcd901991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:33.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.964+0000 7fcd8dd9b640 1 -- 172.21.15.2:0/990819103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd90108a70 msgr2=0x7fcd9019da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:33.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.964+0000 7fcd8dd9b640 1 --2- 172.21.15.2:0/990819103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd90108a70 0x7fcd9019da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:33.402 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.964+0000 7fcd8dd9b640 1 -- 172.21.15.2:0/990819103 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd9019e1b0 con 0x7fcd90103420 2024-05-14T17:10:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.964+0000 7fcd8dd9b640 1 --2- 172.21.15.2:0/990819103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd90103420 0x7fcd90199700 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fcd7800e9f0 tx=0x7fcd7800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.964+0000 7fcd877fe640 1 -- 172.21.15.2:0/990819103 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7800cd80 con 0x7fcd90103420 2024-05-14T17:10:33.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.964+0000 7fcd877fe640 1 -- 172.21.15.2:0/990819103 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd780054d0 con 0x7fcd90103420 2024-05-14T17:10:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.964+0000 7fcd94883640 1 -- 172.21.15.2:0/990819103 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd9019e4a0 con 0x7fcd90103420 2024-05-14T17:10:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.964+0000 7fcd877fe640 1 -- 172.21.15.2:0/990819103 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd78010690 con 0x7fcd90103420 2024-05-14T17:10:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.964+0000 7fcd94883640 1 -- 172.21.15.2:0/990819103 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd901a5d30 con 0x7fcd90103420 2024-05-14T17:10:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.967+0000 7fcd877fe640 1 -- 172.21.15.2:0/990819103 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fcd78004e90 con 0x7fcd90103420 2024-05-14T17:10:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.967+0000 7fcd975a3740 1 Processor -- start 2024-05-14T17:10:34.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.967+0000 7fcd975a3740 1 -- start start 2024-05-14T17:10:34.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.968+0000 7fcd975a3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6e3e327f0 0x55d6e3e32bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.968+0000 7fcd975a3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d6e3e33110 0x55d6e3e419a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.968+0000 7fcd975a3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6e3e41ee0 0x55d6e3e442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.968+0000 7fcd975a3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d6e3da5c30 con 0x55d6e3e327f0 2024-05-14T17:10:34.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.968+0000 7fcd975a3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d6e3d957d0 con 0x55d6e3e41ee0 2024-05-14T17:10:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.968+0000 7fcd975a3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d6e3d99810 con 0x55d6e3e33110 2024-05-14T17:10:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.968+0000 7fcd8ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6e3e327f0 0x55d6e3e32bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.968+0000 7fcd8ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6e3e327f0 0x55d6e3e32bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53920/0 (socket says 172.21.15.2:53920) 2024-05-14T17:10:35.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.968+0000 7fcd8ed9d640 1 -- 172.21.15.2:0/2457372999 learned_addr learned my addr 172.21.15.2:0/2457372999 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:10:35.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.968+0000 7fcd8e59c640 1 --2- 172.21.15.2:0/2457372999 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d6e3e33110 0x55d6e3e419a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.968+0000 7fcd8dd9b640 1 --2- 172.21.15.2:0/2457372999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6e3e41ee0 0x55d6e3e442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.968+0000 7fcd877fe640 1 --2- 172.21.15.2:0/990819103 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcd7c0b2a10 0x7fcd7c0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:35.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.968+0000 7fcd8ed9d640 1 -- 172.21.15.2:0/2457372999 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d6e3e33110 msgr2=0x55d6e3e419a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:35.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.968+0000 7fcd8ed9d640 1 --2- 172.21.15.2:0/2457372999 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d6e3e33110 0x55d6e3e419a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:35.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.968+0000 7fcd8ed9d640 1 -- 172.21.15.2:0/2457372999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6e3e41ee0 msgr2=0x55d6e3e442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.968+0000 7fcd8ed9d640 1 --2- 172.21.15.2:0/2457372999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6e3e41ee0 0x55d6e3e442d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.968+0000 7fcd8ed9d640 1 -- 172.21.15.2:0/2457372999 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6e3aaa210 con 0x55d6e3e327f0 2024-05-14T17:10:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.968+0000 7fcd8dd9b640 1 --2- 172.21.15.2:0/2457372999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6e3e41ee0 0x55d6e3e442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:10:35.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.968+0000 7fcd877fe640 1 -- 172.21.15.2:0/990819103 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fcd780e2ac0 con 0x7fcd90103420 2024-05-14T17:10:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.968+0000 7fcd8ed9d640 1 --2- 172.21.15.2:0/990819103 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcd7c0b2a10 0x7fcd7c0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:35.939 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.969+0000 7fcd8ed9d640 1 --2- 172.21.15.2:0/2457372999 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6e3e327f0 0x55d6e3e32bd0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fcd88009e50 tx=0x7fcd88010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27184d713ca7e110 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.969+0000 7fcd84ff9640 1 -- 172.21.15.2:0/2457372999 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd88021070 con 0x55d6e3e327f0 2024-05-14T17:10:36.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.969+0000 7fcd84ff9640 1 -- 172.21.15.2:0/2457372999 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd8801bd80 con 0x55d6e3e327f0 2024-05-14T17:10:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.969+0000 7fcd84ff9640 1 -- 172.21.15.2:0/2457372999 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd88016070 con 0x55d6e3e327f0 2024-05-14T17:10:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.970+0000 7fcd975a3740 1 -- 172.21.15.2:0/2457372999 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6e3e327f0 msgr2=0x55d6e3e32bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:36.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.970+0000 7fcd975a3740 1 --2- 172.21.15.2:0/2457372999 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6e3e327f0 0x55d6e3e32bd0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fcd88009e50 tx=0x7fcd88010740 comp rx=0 tx=0).stop 2024-05-14T17:10:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.970+0000 7fcd8ed9d640 1 --2- 172.21.15.2:0/990819103 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcd7c0b2a10 0x7fcd7c0b4ed0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcd8801e000 tx=0x7fcd88021000 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.970+0000 7fcd975a3740 1 -- 172.21.15.2:0/2457372999 shutdown_connections 2024-05-14T17:10:36.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.970+0000 7fcd975a3740 1 --2- 172.21.15.2:0/2457372999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6e3e41ee0 0x55d6e3e442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.970+0000 7fcd975a3740 1 --2- 172.21.15.2:0/2457372999 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d6e3e33110 0x55d6e3e419a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:36.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.970+0000 7fcd975a3740 1 --2- 172.21.15.2:0/2457372999 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6e3e327f0 0x55d6e3e32bd0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:36.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.970+0000 7fcd975a3740 1 -- 172.21.15.2:0/2457372999 >> 172.21.15.2:0/2457372999 conn(0x55d6e3e2ee00 msgr2=0x55d6e3e2f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:36.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.970+0000 7fcd975a3740 1 -- 172.21.15.2:0/2457372999 shutdown_connections 2024-05-14T17:10:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.970+0000 7fcd975a3740 1 -- 172.21.15.2:0/2457372999 wait complete. 2024-05-14T17:10:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.970+0000 7fcd975a3740 1 -- start start 2024-05-14T17:10:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.970+0000 7fcd975a3740 1 Processor -- start 2024-05-14T17:10:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.970+0000 7fcd975a3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d6e3e327f0 0x55d6e3e41820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:37.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.971+0000 7fcd975a3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6e3e33110 0x55d6e3e3eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:37.400 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.971+0000 7fcd975a3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6e3e41ee0 0x55d6e3e3f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:37.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.971+0000 7fcd975a3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55d6e3a7b790 con 0x55d6e3e33110 2024-05-14T17:10:37.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.971+0000 7fcd975a3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d6e3d51890 con 0x55d6e3e41ee0 2024-05-14T17:10:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.971+0000 7fcd975a3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d6e3e44bb0 con 0x55d6e3e327f0 2024-05-14T17:10:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.971+0000 7fcd975a3740 10 client.0 _ll_register_callbacks cb 0x55d6e3d54d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:10:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.971+0000 7fcd8dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6e3e33110 0x55d6e3e3eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.971+0000 7fcd8dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6e3e33110 0x55d6e3e3eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53936/0 (socket says 172.21.15.2:53936) 2024-05-14T17:10:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.971+0000 7fcd8dd9b640 1 -- 172.21.15.2:0/2901489901 learned_addr learned my addr 172.21.15.2:0/2901489901 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:10:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.971+0000 7fcd8e59c640 1 --2- 172.21.15.2:0/2901489901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d6e3e327f0 0x55d6e3e41820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.971+0000 7fcd8ed9d640 1 --2- 172.21.15.2:0/2901489901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6e3e41ee0 0x55d6e3e3f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:38.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.971+0000 7fcd8dd9b640 1 -- 172.21.15.2:0/2901489901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d6e3e327f0 msgr2=0x55d6e3e41820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:38.378 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.971+0000 7fcd8dd9b640 1 --2- 172.21.15.2:0/2901489901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d6e3e327f0 0x55d6e3e41820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.971+0000 7fcd8dd9b640 1 -- 172.21.15.2:0/2901489901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6e3e41ee0 msgr2=0x55d6e3e3f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:38.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.971+0000 7fcd8dd9b640 1 --2- 172.21.15.2:0/2901489901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6e3e41ee0 0x55d6e3e3f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.971+0000 7fcd8dd9b640 1 -- 172.21.15.2:0/2901489901 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6e3b76620 con 0x55d6e3e33110 2024-05-14T17:10:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.971+0000 7fcd8ed9d640 1 --2- 172.21.15.2:0/2901489901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6e3e41ee0 0x55d6e3e3f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:10:38.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.971+0000 7fcd8dd9b640 1 --2- 172.21.15.2:0/2901489901 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6e3e33110 0x55d6e3e3eb10 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fcd780e0270 tx=0x7fcd780a5d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b170912464d26540 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.971+0000 7fcd75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:10:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.971+0000 7fcd75ffb640 1 -- 172.21.15.2:0/2901489901 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd780ad070 con 0x55d6e3e33110 2024-05-14T17:10:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.971+0000 7fcd75ffb640 1 -- 172.21.15.2:0/2901489901 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fcd780aa000 con 0x55d6e3e33110 2024-05-14T17:10:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.971+0000 7fcd975a3740 1 -- 172.21.15.2:0/2901489901 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd8801bd80 con 0x55d6e3e33110 2024-05-14T17:10:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.971+0000 7fcd75ffb640 1 -- 172.21.15.2:0/2901489901 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fcd780a98b0 con 0x55d6e3e33110 2024-05-14T17:10:39.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.971+0000 7fcd975a3740 10 client.16755 fetch_fsmap learned FSMap version 51 2024-05-14T17:10:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.972+0000 7fcd975a3740 1 -- 172.21.15.2:0/2901489901 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55d6e3e503f0 con 0x55d6e3e33110 2024-05-14T17:10:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.972+0000 7fcd75ffb640 1 -- 172.21.15.2:0/2901489901 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd8801bd80 con 0x55d6e3e33110 2024-05-14T17:10:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.972+0000 7fcd75ffb640 1 -- 172.21.15.2:0/2901489901 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 51) ==== 8354+0+0 (secure 0 0 0) 0x7fcd780b2770 con 0x55d6e3e33110 2024-05-14T17:10:39.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.972+0000 7fcd975a3740 10 client.16755 fetch_fsmap finished waiting for FSMap version 51 2024-05-14T17:10:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.972+0000 7fcd975a3740 10 client.16755 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:10:39.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.972+0000 7fcd975a3740 20 client.16755 populate_metadata read hostname 'smithi002' 2024-05-14T17:10:39.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.972+0000 7fcd975a3740 1 --2- 172.21.15.2:0/2901489901 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d6e3e518d0 0x55d6e3e53cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.972+0000 7fcd975a3740 4 client.16755 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:10:39.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.972+0000 7fcd975a3740 1 -- 172.21.15.2:0/2901489901 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6e3da7b80 con 0x55d6e3e518d0 2024-05-14T17:10:39.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.972+0000 7fcd8e59c640 1 --2- 172.21.15.2:0/2901489901 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d6e3e518d0 0x55d6e3e53cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.973+0000 7fcd75ffb640 10 client.16755 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:10:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.973+0000 7fcd8e59c640 1 --2- 172.21.15.2:0/2901489901 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d6e3e518d0 0x55d6e3e53cc0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5618d81d19c6ae4b server_cookie=e3472faf9ffe527f in_seq=0 out_seq=0 2024-05-14T17:10:40.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.973+0000 7fcd75ffb640 1 -- 172.21.15.2:0/2901489901 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d6e3da7b80 con 0x55d6e3e518d0 2024-05-14T17:10:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.973+0000 7fcd75ffb640 10 client.16755 handle_command_reply: tid=0 2024-05-14T17:10:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.974+0000 7fcd975a3740 1 client.16755 shutdown 2024-05-14T17:10:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.974+0000 7fcd975a3740 1 -- 172.21.15.2:0/2901489901 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6e3e33110 msgr2=0x55d6e3e3eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.974+0000 7fcd975a3740 1 --2- 172.21.15.2:0/2901489901 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6e3e33110 0x55d6e3e3eb10 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fcd780e0270 tx=0x7fcd780a5d70 comp rx=0 tx=0).stop 2024-05-14T17:10:40.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.974+0000 7fcd975a3740 1 -- 172.21.15.2:0/2901489901 shutdown_connections 2024-05-14T17:10:40.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.974+0000 7fcd975a3740 1 --2- 172.21.15.2:0/2901489901 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d6e3e518d0 0x55d6e3e53cc0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:40.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.974+0000 7fcd975a3740 1 --2- 172.21.15.2:0/2901489901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6e3e41ee0 0x55d6e3e3f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.974+0000 7fcd975a3740 1 --2- 172.21.15.2:0/2901489901 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6e3e33110 0x55d6e3e3eb10 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.974+0000 7fcd75ffb640 0 client.16755 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:10:40.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.974+0000 7fcd975a3740 1 --2- 172.21.15.2:0/2901489901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d6e3e327f0 0x55d6e3e41820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:40.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.974+0000 7fcd975a3740 1 -- 172.21.15.2:0/2901489901 >> 172.21.15.2:0/2901489901 conn(0x55d6e3e2ee00 msgr2=0x55d6e3e34c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:40.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.974+0000 7fcd975a3740 1 -- 172.21.15.2:0/2901489901 shutdown_connections 2024-05-14T17:10:40.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.974+0000 7fcd975a3740 1 -- 172.21.15.2:0/2901489901 wait complete. 2024-05-14T17:10:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.974+0000 7fcd975a3740 20 client.16755 trim_cache size 0 max 16384 2024-05-14T17:10:41.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.989+0000 7fcd975a3740 1 Processor -- start 2024-05-14T17:10:41.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.989+0000 7fcd975a3740 1 -- start start 2024-05-14T17:10:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.989+0000 7fcd975a3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6e3e327f0 0x55d6e3e3a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.989+0000 7fcd975a3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d6e3e33110 0x55d6e3e4da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:41.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.989+0000 7fcd975a3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6e3e41ee0 0x55d6e3e2df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.989+0000 7fcd975a3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcd901aecd0 con 0x55d6e3e41ee0 2024-05-14T17:10:41.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.989+0000 7fcd975a3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcd901ac540 con 0x55d6e3e327f0 2024-05-14T17:10:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.989+0000 7fcd975a3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcd901aa3b0 con 0x55d6e3e33110 2024-05-14T17:10:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.989+0000 7fcd8ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6e3e41ee0 0x55d6e3e2df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.990+0000 7fcd8ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6e3e41ee0 0x55d6e3e2df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53938/0 (socket says 172.21.15.2:53938) 2024-05-14T17:10:41.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.990+0000 7fcd8ed9d640 1 -- 172.21.15.2:0/1585862875 learned_addr learned my addr 172.21.15.2:0/1585862875 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:10:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.990+0000 7fcd8dd9b640 1 --2- 172.21.15.2:0/1585862875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d6e3e33110 0x55d6e3e4da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:42.031 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.990+0000 7fcd8e59c640 1 --2- 172.21.15.2:0/1585862875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6e3e327f0 0x55d6e3e3a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.990+0000 7fcd8ed9d640 1 -- 172.21.15.2:0/1585862875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d6e3e33110 msgr2=0x55d6e3e4da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:42.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.990+0000 7fcd8ed9d640 1 --2- 172.21.15.2:0/1585862875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d6e3e33110 0x55d6e3e4da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:42.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.990+0000 7fcd8ed9d640 1 -- 172.21.15.2:0/1585862875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6e3e327f0 msgr2=0x55d6e3e3a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.990+0000 7fcd8ed9d640 1 --2- 172.21.15.2:0/1585862875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6e3e327f0 0x55d6e3e3a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:42.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.990+0000 7fcd8ed9d640 1 -- 172.21.15.2:0/1585862875 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6e3e2ec30 con 0x55d6e3e41ee0 2024-05-14T17:10:42.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.990+0000 7fcd8dd9b640 1 --2- 172.21.15.2:0/1585862875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d6e3e33110 0x55d6e3e4da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:10:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.990+0000 7fcd8ed9d640 1 --2- 172.21.15.2:0/1585862875 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6e3e41ee0 0x55d6e3e2df80 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fcd88010b80 tx=0x7fcd88024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce2afdf09862dcfc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.990+0000 7fcd77fff640 1 -- 172.21.15.2:0/1585862875 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd88035070 con 0x55d6e3e41ee0 2024-05-14T17:10:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.990+0000 7fcd77fff640 1 -- 172.21.15.2:0/1585862875 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd880128e0 con 0x55d6e3e41ee0 2024-05-14T17:10:42.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.990+0000 7fcd77fff640 1 -- 172.21.15.2:0/1585862875 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd88016040 con 0x55d6e3e41ee0 2024-05-14T17:10:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.991+0000 7fcd975a3740 1 -- 172.21.15.2:0/1585862875 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6e3e41ee0 msgr2=0x55d6e3e2df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.991+0000 7fcd975a3740 1 --2- 172.21.15.2:0/1585862875 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6e3e41ee0 0x55d6e3e2df80 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fcd88010b80 tx=0x7fcd88024000 comp rx=0 tx=0).stop 2024-05-14T17:10:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.991+0000 7fcd975a3740 1 -- 172.21.15.2:0/1585862875 shutdown_connections 2024-05-14T17:10:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.991+0000 7fcd975a3740 1 --2- 172.21.15.2:0/1585862875 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6e3e41ee0 0x55d6e3e2df80 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:43.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.991+0000 7fcd975a3740 1 --2- 172.21.15.2:0/1585862875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d6e3e33110 0x55d6e3e4da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.991+0000 7fcd975a3740 1 --2- 172.21.15.2:0/1585862875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6e3e327f0 0x55d6e3e3a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.991+0000 7fcd975a3740 1 -- 172.21.15.2:0/1585862875 >> 172.21.15.2:0/1585862875 conn(0x55d6e3e2ee00 msgr2=0x55d6e3e2d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:43.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.991+0000 7fcd975a3740 1 -- 172.21.15.2:0/1585862875 shutdown_connections 2024-05-14T17:10:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.991+0000 7fcd975a3740 1 -- 172.21.15.2:0/1585862875 wait complete. 2024-05-14T17:10:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.991+0000 7fcd975a3740 1 -- start start 2024-05-14T17:10:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.991+0000 7fcd975a3740 1 Processor -- start 2024-05-14T17:10:43.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.991+0000 7fcd975a3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6e3e327f0 0x55d6e3e47aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.991+0000 7fcd975a3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d6e3e33110 0x55d6e3e47fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.991+0000 7fcd975a3740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6e3e41ee0 0x55d6e3e48520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:43.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.991+0000 7fcd975a3740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcd901aaa00 con 0x55d6e3e41ee0 2024-05-14T17:10:44.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.991+0000 7fcd975a3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcd901abe70 con 0x55d6e3e327f0 2024-05-14T17:10:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.991+0000 7fcd975a3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcd901ab0d0 con 0x55d6e3e33110 2024-05-14T17:10:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.992+0000 7fcd975a3740 10 client.0 _ll_register_callbacks cb 0x7fcd780a96f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:10:44.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.992+0000 7fcd8ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6e3e41ee0 0x55d6e3e48520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.992+0000 7fcd8ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6e3e41ee0 0x55d6e3e48520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53952/0 (socket says 172.21.15.2:53952) 2024-05-14T17:10:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.992+0000 7fcd8ed9d640 1 -- 172.21.15.2:0/140581763 learned_addr learned my addr 172.21.15.2:0/140581763 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:10:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.992+0000 7fcd8dd9b640 1 --2- 172.21.15.2:0/140581763 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d6e3e33110 0x55d6e3e47fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.992+0000 7fcd8e59c640 1 --2- 172.21.15.2:0/140581763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6e3e327f0 0x55d6e3e47aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:44.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.992+0000 7fcd8ed9d640 1 -- 172.21.15.2:0/140581763 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d6e3e33110 msgr2=0x55d6e3e47fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:44.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.992+0000 7fcd8ed9d640 1 --2- 172.21.15.2:0/140581763 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d6e3e33110 0x55d6e3e47fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.992+0000 7fcd8ed9d640 1 -- 172.21.15.2:0/140581763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6e3e327f0 msgr2=0x55d6e3e47aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.992+0000 7fcd8ed9d640 1 --2- 172.21.15.2:0/140581763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6e3e327f0 0x55d6e3e47aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.992+0000 7fcd8ed9d640 1 -- 172.21.15.2:0/140581763 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6e3e39480 con 0x55d6e3e41ee0 2024-05-14T17:10:45.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.992+0000 7fcd8ed9d640 1 --2- 172.21.15.2:0/140581763 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6e3e41ee0 0x55d6e3e48520 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fcd880092c0 tx=0x7fcd88004c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8bee8b9f84a34a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:45.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.992+0000 7fcd75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:10:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.992+0000 7fcd75ffb640 1 -- 172.21.15.2:0/140581763 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd88035070 con 0x55d6e3e41ee0 2024-05-14T17:10:45.284 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.992+0000 7fcd75ffb640 1 -- 172.21.15.2:0/140581763 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fcd8801bae0 con 0x55d6e3e41ee0 2024-05-14T17:10:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.992+0000 7fcd975a3740 1 -- 172.21.15.2:0/140581763 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd880128e0 con 0x55d6e3e41ee0 2024-05-14T17:10:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.992+0000 7fcd75ffb640 1 -- 172.21.15.2:0/140581763 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fcd88016070 con 0x55d6e3e41ee0 2024-05-14T17:10:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.992+0000 7fcd975a3740 10 client.16761 fetch_fsmap learned FSMap version 51 2024-05-14T17:10:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.992+0000 7fcd975a3740 1 -- 172.21.15.2:0/140581763 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55d6e3ec09c0 con 0x55d6e3e41ee0 2024-05-14T17:10:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.993+0000 7fcd75ffb640 1 -- 172.21.15.2:0/140581763 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd880128e0 con 0x55d6e3e41ee0 2024-05-14T17:10:45.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.993+0000 7fcd75ffb640 1 -- 172.21.15.2:0/140581763 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 51) ==== 8354+0+0 (secure 0 0 0) 0x7fcd8800f040 con 0x55d6e3e41ee0 2024-05-14T17:10:45.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.993+0000 7fcd975a3740 10 client.16761 fetch_fsmap finished waiting for FSMap version 51 2024-05-14T17:10:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.993+0000 7fcd975a3740 10 client.16761 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:10:45.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.993+0000 7fcd975a3740 20 client.16761 populate_metadata read hostname 'smithi002' 2024-05-14T17:10:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.993+0000 7fcd975a3740 1 --2- 172.21.15.2:0/140581763 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d6e3ec0dc0 0x55d6e3ec31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.993+0000 7fcd975a3740 4 client.16761 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:10:46.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.993+0000 7fcd975a3740 1 -- 172.21.15.2:0/140581763 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d6e3da1360 con 0x55d6e3ec0dc0 2024-05-14T17:10:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.993+0000 7fcd8e59c640 1 --2- 172.21.15.2:0/140581763 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d6e3ec0dc0 0x55d6e3ec31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:46.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.993+0000 7fcd75ffb640 10 client.16761 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:10:46.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:34.993+0000 7fcd8e59c640 1 --2- 172.21.15.2:0/140581763 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d6e3ec0dc0 0x55d6e3ec31e0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=330749d07ae1bf69 server_cookie=a18392e00433c79d in_seq=0 out_seq=0 2024-05-14T17:10:46.238 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:08:34 smithi002 ceph-mon[31538]: pgmap v433: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 16 MiB/s wr, 1.41k op/s 2024-05-14T17:10:46.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:08:34 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:10:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:35.025+0000 7fcd75ffb640 1 -- 172.21.15.2:0/140581763 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d6e3da1360 con 0x55d6e3ec0dc0 2024-05-14T17:10:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:35.025+0000 7fcd75ffb640 10 client.16761 handle_command_reply: tid=0 2024-05-14T17:10:46.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:35.025+0000 7fcd975a3740 1 client.16761 shutdown 2024-05-14T17:10:46.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:35.025+0000 7fcd975a3740 1 -- 172.21.15.2:0/140581763 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6e3e41ee0 msgr2=0x55d6e3e48520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:46.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:35.025+0000 7fcd975a3740 1 --2- 172.21.15.2:0/140581763 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6e3e41ee0 0x55d6e3e48520 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fcd880092c0 tx=0x7fcd88004c90 comp rx=0 tx=0).stop 2024-05-14T17:10:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:35.025+0000 7fcd975a3740 1 -- 172.21.15.2:0/140581763 shutdown_connections 2024-05-14T17:10:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:35.025+0000 7fcd975a3740 1 --2- 172.21.15.2:0/140581763 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55d6e3ec0dc0 0x55d6e3ec31e0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:35.025+0000 7fcd975a3740 1 --2- 172.21.15.2:0/140581763 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6e3e41ee0 0x55d6e3e48520 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:46.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:35.025+0000 7fcd975a3740 1 --2- 172.21.15.2:0/140581763 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d6e3e33110 0x55d6e3e47fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:35.025+0000 7fcd975a3740 1 --2- 172.21.15.2:0/140581763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d6e3e327f0 0x55d6e3e47aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:35.025+0000 7fcd975a3740 1 -- 172.21.15.2:0/140581763 >> 172.21.15.2:0/140581763 conn(0x55d6e3e2ee00 msgr2=0x55d6e3e4ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:47.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:35.025+0000 7fcd75ffb640 0 client.16761 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:10:47.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:35.026+0000 7fcd975a3740 1 -- 172.21.15.2:0/140581763 shutdown_connections 2024-05-14T17:10:47.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:35.026+0000 7fcd975a3740 1 -- 172.21.15.2:0/140581763 wait complete. 2024-05-14T17:10:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:35.026+0000 7fcd975a3740 20 client.16761 trim_cache size 0 max 16384 2024-05-14T17:10:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:10:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:10:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:10:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:35.026+0000 7fcd94883640 1 -- 172.21.15.2:0/990819103 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcd7c0b2a10 msgr2=0x7fcd7c0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:35.026+0000 7fcd94883640 1 --2- 172.21.15.2:0/990819103 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcd7c0b2a10 0x7fcd7c0b4ed0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcd8801e000 tx=0x7fcd88021000 comp rx=0 tx=0).stop 2024-05-14T17:10:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:35.026+0000 7fcd94883640 1 -- 172.21.15.2:0/990819103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd90103420 msgr2=0x7fcd90199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:35.026+0000 7fcd94883640 1 --2- 172.21.15.2:0/990819103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd90103420 0x7fcd90199700 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fcd7800e9f0 tx=0x7fcd7800edf0 comp rx=0 tx=0).stop 2024-05-14T17:10:48.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:35.027+0000 7fcd94883640 1 -- 172.21.15.2:0/990819103 shutdown_connections 2024-05-14T17:10:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:35.027+0000 7fcd94883640 1 --2- 172.21.15.2:0/990819103 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcd7c0b2a10 0x7fcd7c0b4ed0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:35.027+0000 7fcd94883640 1 --2- 172.21.15.2:0/990819103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd90108a70 0x7fcd9019da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:35.027+0000 7fcd94883640 1 --2- 172.21.15.2:0/990819103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd90103420 0x7fcd90199700 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:35.027+0000 7fcd94883640 1 --2- 172.21.15.2:0/990819103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd90102a60 0x7fcd901991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:35.027+0000 7fcd94883640 1 -- 172.21.15.2:0/990819103 >> 172.21.15.2:0/990819103 conn(0x7fcd90079ee0 msgr2=0x7fcd901002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:48.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:35.027+0000 7fcd94883640 1 -- 172.21.15.2:0/990819103 shutdown_connections 2024-05-14T17:10:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:35.027+0000 7fcd94883640 1 -- 172.21.15.2:0/990819103 wait complete. 2024-05-14T17:10:48.338 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 800.0 s, 983.6 tps, lat 112.741 ms stddev 542.753 2024-05-14T17:10:48.372 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:08:36 smithi002 ceph-mon[31538]: pgmap v434: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.2 MiB/s wr, 644 op/s 2024-05-14T17:10:48.404 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:08:38 smithi002 ceph-mon[31538]: pgmap v435: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.0 MiB/s wr, 530 op/s 2024-05-14T17:10:48.437 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:08:40 smithi002 ceph-mon[31538]: pgmap v436: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 7.3 MiB/s wr, 776 op/s 2024-05-14T17:10:48.471 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:10:32 smithi151 ceph-mon[36066]: pgmap v496: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 818 KiB/s wr, 109 op/s 2024-05-14T17:10:48.504 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:10:34 smithi151 ceph-mon[36066]: pgmap v497: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-05-14T17:10:48.554 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:10:36 smithi151 ceph-mon[36066]: pgmap v498: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 750 KiB/s wr, 102 op/s 2024-05-14T17:10:48.587 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:10:38 smithi151 ceph-mon[36066]: pgmap v499: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 767 KiB/s wr, 104 op/s 2024-05-14T17:10:48.621 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:10:40 smithi151 ceph-mon[36066]: pgmap v500: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 1021 KiB/s wr, 140 op/s 2024-05-14T17:10:48.654 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:10:42 smithi151 ceph-mon[36066]: pgmap v501: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 721 KiB/s wr, 96 op/s 2024-05-14T17:10:48.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:10:43 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:10:48.721 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:10:44 smithi151 ceph-mon[36066]: pgmap v502: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 1.8 MiB/s wr, 173 op/s 2024-05-14T17:10:48.754 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:10:46 smithi151 ceph-mon[36066]: pgmap v503: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 1.5 MiB/s wr, 134 op/s 2024-05-14T17:10:48.788 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:10:32 smithi123 ceph-mon[35971]: pgmap v496: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 818 KiB/s wr, 109 op/s 2024-05-14T17:10:48.821 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:10:34 smithi123 ceph-mon[35971]: pgmap v497: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-05-14T17:10:48.854 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:10:36 smithi123 ceph-mon[35971]: pgmap v498: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 750 KiB/s wr, 102 op/s 2024-05-14T17:10:48.888 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:10:38 smithi123 ceph-mon[35971]: pgmap v499: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 767 KiB/s wr, 104 op/s 2024-05-14T17:10:48.921 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:10:40 smithi123 ceph-mon[35971]: pgmap v500: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 1021 KiB/s wr, 140 op/s 2024-05-14T17:10:48.954 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:10:42 smithi123 ceph-mon[35971]: pgmap v501: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 721 KiB/s wr, 96 op/s 2024-05-14T17:10:48.988 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:10:43 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:10:49.021 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:10:44 smithi123 ceph-mon[35971]: pgmap v502: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 1.8 MiB/s wr, 173 op/s 2024-05-14T17:10:49.057 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:10:46 smithi123 ceph-mon[35971]: pgmap v503: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 1.5 MiB/s wr, 134 op/s 2024-05-14T17:10:49.113 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 805.0 s, 2374.0 tps, lat 41.959 ms stddev 50.165 2024-05-14T17:10:49.147 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:08:42 smithi002 ceph-mon[31538]: pgmap v437: 97 pgs: 97 active+clean; 4.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 4.7 MiB/s wr, 510 op/s 2024-05-14T17:10:49.180 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:08:44 smithi002 ceph-mon[31538]: pgmap v438: 97 pgs: 97 active+clean; 4.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 6.8 MiB/s wr, 725 op/s 2024-05-14T17:10:49.213 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 810.0 s, 1752.0 tps, lat 57.220 ms stddev 67.230 2024-05-14T17:10:49.247 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:08:46 smithi002 ceph-mon[31538]: pgmap v439: 97 pgs: 97 active+clean; 4.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 6.6 MiB/s wr, 639 op/s 2024-05-14T17:10:49.283 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:10:48 smithi151 ceph-mon[36066]: pgmap v504: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 1.5 MiB/s wr, 129 op/s 2024-05-14T17:10:49.313 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:10:48 smithi123 ceph-mon[35971]: pgmap v504: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 1.5 MiB/s wr, 129 op/s 2024-05-14T17:10:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.165+0000 7f231cc72640 1 -- 172.21.15.2:0/23530313 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2318103cc0 msgr2=0x7f23181040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.165+0000 7f231cc72640 1 --2- 172.21.15.2:0/23530313 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2318103cc0 0x7f23181040a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f2308009900 tx=0x7f230802a480 comp rx=0 tx=0).stop 2024-05-14T17:10:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.165+0000 7f231cc72640 1 -- 172.21.15.2:0/23530313 shutdown_connections 2024-05-14T17:10:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.165+0000 7f231cc72640 1 --2- 172.21.15.2:0/23530313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f231810d090 0x7f231810f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.165+0000 7f231cc72640 1 --2- 172.21.15.2:0/23530313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23181045e0 0x7f231810cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.165+0000 7f231cc72640 1 --2- 172.21.15.2:0/23530313 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2318103cc0 0x7f23181040a0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.165+0000 7f231cc72640 1 -- 172.21.15.2:0/23530313 >> 172.21.15.2:0/23530313 conn(0x7f2318078b60 msgr2=0x7f2318078f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.165+0000 7f231cc72640 1 -- 172.21.15.2:0/23530313 shutdown_connections 2024-05-14T17:10:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.165+0000 7f231cc72640 1 -- 172.21.15.2:0/23530313 wait complete. 2024-05-14T17:10:49.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.166+0000 7f231cc72640 1 Processor -- start 2024-05-14T17:10:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.166+0000 7f231cc72640 1 -- start start 2024-05-14T17:10:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.166+0000 7f231cc72640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2318103cc0 0x7f2318101910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.166+0000 7f231cc72640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23181045e0 0x7f2318101e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.166+0000 7f231cc72640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f231810d090 0x7f2318102390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.166+0000 7f231659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2318103cc0 0x7f2318101910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.166+0000 7f231659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2318103cc0 0x7f2318101910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:52824/0 (socket says 172.21.15.2:52824) 2024-05-14T17:10:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.167+0000 7f2315d9b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23181045e0 0x7f2318101e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.167+0000 7f2315d9b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23181045e0 0x7f2318101e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:34228/0 (socket says 172.21.15.2:34228) 2024-05-14T17:10:50.014 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.167+0000 7f231cc72640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2318112d40 con 0x7f231810d090 2024-05-14T17:10:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.167+0000 7f231cc72640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2318112bc0 con 0x7f2318103cc0 2024-05-14T17:10:50.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.167+0000 7f231cc72640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2318112ec0 con 0x7f23181045e0 2024-05-14T17:10:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.167+0000 7f231659c640 1 -- 172.21.15.2:0/2485154996 learned_addr learned my addr 172.21.15.2:0/2485154996 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:10:50.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.167+0000 7f2316d9d640 1 --2- 172.21.15.2:0/2485154996 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f231810d090 0x7f2318102390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.168+0000 7f2315d9b640 1 -- 172.21.15.2:0/2485154996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2318103cc0 msgr2=0x7f2318101910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:50.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.168+0000 7f2315d9b640 1 --2- 172.21.15.2:0/2485154996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2318103cc0 0x7f2318101910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:50.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.168+0000 7f2315d9b640 1 -- 172.21.15.2:0/2485154996 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f231810d090 msgr2=0x7f2318102390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.168+0000 7f2315d9b640 1 --2- 172.21.15.2:0/2485154996 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f231810d090 0x7f2318102390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:50.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.168+0000 7f2315d9b640 1 -- 172.21.15.2:0/2485154996 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23181a76c0 con 0x7f23181045e0 2024-05-14T17:10:50.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.168+0000 7f2315d9b640 1 --2- 172.21.15.2:0/2485154996 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23181045e0 0x7f2318101e50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f230c00e9f0 tx=0x7f230c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.168+0000 7f23077fe640 1 -- 172.21.15.2:0/2485154996 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230c00ccc0 con 0x7f23181045e0 2024-05-14T17:10:50.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.168+0000 7f231cc72640 1 -- 172.21.15.2:0/2485154996 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23181a79b0 con 0x7f23181045e0 2024-05-14T17:10:50.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.168+0000 7f231cc72640 1 -- 172.21.15.2:0/2485154996 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23181a7f00 con 0x7f23181045e0 2024-05-14T17:10:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.168+0000 7f23077fe640 1 -- 172.21.15.2:0/2485154996 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f230c0044d0 con 0x7f23181045e0 2024-05-14T17:10:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.168+0000 7f23077fe640 1 -- 172.21.15.2:0/2485154996 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230c01c070 con 0x7f23181045e0 2024-05-14T17:10:50.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.171+0000 7f231f993740 1 Processor -- start 2024-05-14T17:10:50.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.172+0000 7f231f993740 1 -- start start 2024-05-14T17:10:50.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.172+0000 7f231f993740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559e559ab100 0x559e559ab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.172+0000 7f231f993740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e559aba20 0x559e559b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.172+0000 7f231f993740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e559b62e0 0x559e559b86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:50.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.172+0000 7f231f993740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x559e55915c30 con 0x559e559ab100 2024-05-14T17:10:50.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.172+0000 7f231f993740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559e559057d0 con 0x559e559b62e0 2024-05-14T17:10:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.172+0000 7f231f993740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559e55909810 con 0x559e559aba20 2024-05-14T17:10:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.172+0000 7f2316d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559e559ab100 0x559e559ab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.172+0000 7f2316d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559e559ab100 0x559e559ab4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40856/0 (socket says 172.21.15.2:40856) 2024-05-14T17:10:50.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.172+0000 7f2316d9d640 1 -- 172.21.15.2:0/4213537678 learned_addr learned my addr 172.21.15.2:0/4213537678 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:10:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.172+0000 7f231659c640 1 --2- 172.21.15.2:0/4213537678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e559aba20 0x559e559b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.173+0000 7f2316d9d640 1 -- 172.21.15.2:0/4213537678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e559aba20 msgr2=0x559e559b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.173+0000 7f2316d9d640 1 --2- 172.21.15.2:0/4213537678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e559aba20 0x559e559b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.173+0000 7f2316d9d640 1 -- 172.21.15.2:0/4213537678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e559b62e0 msgr2=0x559e559b86d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T17:10:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.173+0000 7f2316d9d640 1 --2- 172.21.15.2:0/4213537678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e559b62e0 0x559e559b86d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.173+0000 7f2316d9d640 1 -- 172.21.15.2:0/4213537678 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e5561a210 con 0x559e559ab100 2024-05-14T17:10:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.173+0000 7f2316d9d640 1 --2- 172.21.15.2:0/4213537678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559e559ab100 0x559e559ab4e0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f2310009f40 tx=0x7f2310010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0d001eb0d1f9814 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:51.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.173+0000 7f23077fe640 1 -- 172.21.15.2:0/2485154996 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f230c005020 con 0x7f23181045e0 2024-05-14T17:10:51.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.173+0000 7f2304ff9640 1 -- 172.21.15.2:0/4213537678 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2310003b70 con 0x559e559ab100 2024-05-14T17:10:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.173+0000 7f2304ff9640 1 -- 172.21.15.2:0/4213537678 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2310003d10 con 0x559e559ab100 2024-05-14T17:10:51.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.173+0000 7f2304ff9640 1 -- 172.21.15.2:0/4213537678 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2310011a10 con 0x559e559ab100 2024-05-14T17:10:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.174+0000 7f23077fe640 1 --2- 172.21.15.2:0/2485154996 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f23000b2a90 0x7f23000b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:51.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.174+0000 7f23077fe640 1 -- 172.21.15.2:0/2485154996 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f230c0e2750 con 0x7f23181045e0 2024-05-14T17:10:51.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.175+0000 7f231f993740 1 -- 172.21.15.2:0/4213537678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559e559ab100 msgr2=0x559e559ab4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:51.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.175+0000 7f231f993740 1 --2- 172.21.15.2:0/4213537678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559e559ab100 0x559e559ab4e0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f2310009f40 tx=0x7f2310010860 comp rx=0 tx=0).stop 2024-05-14T17:10:51.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.175+0000 7f231659c640 1 --2- 172.21.15.2:0/2485154996 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f23000b2a90 0x7f23000b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.175+0000 7f231f993740 1 -- 172.21.15.2:0/4213537678 shutdown_connections 2024-05-14T17:10:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.175+0000 7f231f993740 1 --2- 172.21.15.2:0/4213537678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e559b62e0 0x559e559b86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:51.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.175+0000 7f231f993740 1 --2- 172.21.15.2:0/4213537678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e559aba20 0x559e559b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.175+0000 7f231f993740 1 --2- 172.21.15.2:0/4213537678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559e559ab100 0x559e559ab4e0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:51.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.175+0000 7f231f993740 1 -- 172.21.15.2:0/4213537678 >> 172.21.15.2:0/4213537678 conn(0x559e559a7610 msgr2=0x559e559a79f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.175+0000 7f231f993740 1 -- 172.21.15.2:0/4213537678 shutdown_connections 2024-05-14T17:10:51.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.175+0000 7f231f993740 1 -- 172.21.15.2:0/4213537678 wait complete. 2024-05-14T17:10:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.175+0000 7f231659c640 1 --2- 172.21.15.2:0/2485154996 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f23000b2a90 0x7f23000b4f50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x559e559ac6a0 tx=0x7f2308005a20 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.176+0000 7f231f993740 1 -- start start 2024-05-14T17:10:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.176+0000 7f231f993740 1 Processor -- start 2024-05-14T17:10:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.176+0000 7f231f993740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e559ab100 0x559e559b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:51.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.176+0000 7f231f993740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559e559aba20 0x559e559b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.176+0000 7f231f993740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e559b62e0 0x559e559b3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.176+0000 7f231f993740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x559e555eb790 con 0x559e559aba20 2024-05-14T17:10:52.033 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.176+0000 7f231f993740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559e558c1890 con 0x559e559ab100 2024-05-14T17:10:52.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.176+0000 7f231f993740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559e559b8fb0 con 0x559e559b62e0 2024-05-14T17:10:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.176+0000 7f2315d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559e559aba20 0x559e559b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.176+0000 7f2315d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559e559aba20 0x559e559b3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40872/0 (socket says 172.21.15.2:40872) 2024-05-14T17:10:52.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.176+0000 7f2315d9b640 1 -- 172.21.15.2:0/3795612217 learned_addr learned my addr 172.21.15.2:0/3795612217 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:10:52.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.176+0000 7f231f993740 10 client.0 _ll_register_callbacks cb 0x559e558c4d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:10:52.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.176+0000 7f231659c640 1 --2- 172.21.15.2:0/3795612217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e559ab100 0x559e559b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.176+0000 7f2316d9d640 1 --2- 172.21.15.2:0/3795612217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e559b62e0 0x559e559b3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:52.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.177+0000 7f2315d9b640 1 -- 172.21.15.2:0/3795612217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e559b62e0 msgr2=0x559e559b3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:52.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.177+0000 7f2315d9b640 1 --2- 172.21.15.2:0/3795612217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e559b62e0 0x559e559b3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:52.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.177+0000 7f2315d9b640 1 -- 172.21.15.2:0/3795612217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e559ab100 msgr2=0x559e559b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.177+0000 7f2315d9b640 1 --2- 172.21.15.2:0/3795612217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e559ab100 0x559e559b30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.177+0000 7f2315d9b640 1 -- 172.21.15.2:0/3795612217 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e556e6620 con 0x559e559aba20 2024-05-14T17:10:52.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.177+0000 7f2315d9b640 1 --2- 172.21.15.2:0/3795612217 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559e559aba20 0x559e559b3630 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f230c0a96d0 tx=0x7f230c0a9f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a876e22d883122a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.177+0000 7f22fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:10:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.178+0000 7f22fdffb640 1 -- 172.21.15.2:0/3795612217 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230c0cc320 con 0x559e559aba20 2024-05-14T17:10:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.178+0000 7f22fdffb640 1 -- 172.21.15.2:0/3795612217 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f230c0a8c30 con 0x559e559aba20 2024-05-14T17:10:52.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.178+0000 7f231f993740 1 -- 172.21.15.2:0/3795612217 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2310003d10 con 0x559e559aba20 2024-05-14T17:10:52.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.178+0000 7f22fdffb640 1 -- 172.21.15.2:0/3795612217 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f230c0b2170 con 0x559e559aba20 2024-05-14T17:10:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.178+0000 7f231f993740 10 client.16773 fetch_fsmap learned FSMap version 51 2024-05-14T17:10:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.178+0000 7f231f993740 1 -- 172.21.15.2:0/3795612217 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x559e559c4980 con 0x559e559aba20 2024-05-14T17:10:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.178+0000 7f22fdffb640 1 -- 172.21.15.2:0/3795612217 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2310003d10 con 0x559e559aba20 2024-05-14T17:10:52.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.179+0000 7f22fdffb640 1 -- 172.21.15.2:0/3795612217 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 51) ==== 8354+0+0 (secure 0 0 0) 0x7f230c0a5340 con 0x559e559aba20 2024-05-14T17:10:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.179+0000 7f231f993740 10 client.16773 fetch_fsmap finished waiting for FSMap version 51 2024-05-14T17:10:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.179+0000 7f231f993740 10 client.16773 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:10:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.179+0000 7f231f993740 20 client.16773 populate_metadata read hostname 'smithi002' 2024-05-14T17:10:52.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.179+0000 7f231f993740 1 --2- 172.21.15.2:0/3795612217 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x559e559c5df0 0x559e559c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.179+0000 7f231f993740 4 client.16773 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:10:53.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.179+0000 7f231f993740 1 -- 172.21.15.2:0/3795612217 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e55917b80 con 0x559e559c5df0 2024-05-14T17:10:53.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.179+0000 7f231659c640 1 --2- 172.21.15.2:0/3795612217 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x559e559c5df0 0x559e559c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:53.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.180+0000 7f231659c640 1 --2- 172.21.15.2:0/3795612217 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x559e559c5df0 0x559e559c81e0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2caa33eb81c535b0 server_cookie=218a8fc899a1cf29 in_seq=0 out_seq=0 2024-05-14T17:10:53.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.180+0000 7f22fdffb640 10 client.16773 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:10:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.181+0000 7f22fdffb640 1 -- 172.21.15.2:0/3795612217 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559e55917b80 con 0x559e559c5df0 2024-05-14T17:10:53.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.181+0000 7f22fdffb640 10 client.16773 handle_command_reply: tid=0 2024-05-14T17:10:53.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.181+0000 7f231f993740 1 client.16773 shutdown 2024-05-14T17:10:53.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.182+0000 7f231f993740 1 -- 172.21.15.2:0/3795612217 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559e559aba20 msgr2=0x559e559b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:53.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.182+0000 7f231f993740 1 --2- 172.21.15.2:0/3795612217 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559e559aba20 0x559e559b3630 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f230c0a96d0 tx=0x7f230c0a9f70 comp rx=0 tx=0).stop 2024-05-14T17:10:53.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.182+0000 7f231f993740 1 -- 172.21.15.2:0/3795612217 shutdown_connections 2024-05-14T17:10:53.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.182+0000 7f231f993740 1 --2- 172.21.15.2:0/3795612217 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x559e559c5df0 0x559e559c81e0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:53.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.182+0000 7f231f993740 1 --2- 172.21.15.2:0/3795612217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e559b62e0 0x559e559b3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.182+0000 7f231f993740 1 --2- 172.21.15.2:0/3795612217 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559e559aba20 0x559e559b3630 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.182+0000 7f22fdffb640 0 client.16773 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:10:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.182+0000 7f231f993740 1 --2- 172.21.15.2:0/3795612217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e559ab100 0x559e559b30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.182+0000 7f231f993740 1 -- 172.21.15.2:0/3795612217 >> 172.21.15.2:0/3795612217 conn(0x559e559a7610 msgr2=0x559e559acb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.182+0000 7f231f993740 1 -- 172.21.15.2:0/3795612217 shutdown_connections 2024-05-14T17:10:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.182+0000 7f231f993740 1 -- 172.21.15.2:0/3795612217 wait complete. 2024-05-14T17:10:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.182+0000 7f231f993740 20 client.16773 trim_cache size 0 max 16384 2024-05-14T17:10:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.204+0000 7f231f993740 1 Processor -- start 2024-05-14T17:10:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.205+0000 7f231f993740 1 -- start start 2024-05-14T17:10:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.205+0000 7f231f993740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e559ab100 0x559e559b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.205+0000 7f231f993740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559e559aba20 0x559e559cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:53.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.205+0000 7f2315d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559e559aba20 0x559e559cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.205+0000 7f2315d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559e559aba20 0x559e559cdab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40886/0 (socket says 172.21.15.2:40886) 2024-05-14T17:10:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.205+0000 7f231659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e559ab100 0x559e559b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.205+0000 7f231659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e559ab100 0x559e559b37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:52840/0 (socket says 172.21.15.2:52840) 2024-05-14T17:10:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.205+0000 7f2315d9b640 1 -- 172.21.15.2:0/3527196227 learned_addr learned my addr 172.21.15.2:0/3527196227 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:10:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.206+0000 7f231f993740 1 --2- 172.21.15.2:0/3527196227 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e559b62e0 0x559e559cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.206+0000 7f231f993740 1 -- 172.21.15.2:0/3527196227 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f23181aee40 con 0x559e559aba20 2024-05-14T17:10:54.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.206+0000 7f231f993740 1 -- 172.21.15.2:0/3527196227 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f23181ac6b0 con 0x559e559ab100 2024-05-14T17:10:54.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.206+0000 7f231f993740 1 -- 172.21.15.2:0/3527196227 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f23181aa540 con 0x559e559b62e0 2024-05-14T17:10:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.206+0000 7f2316d9d640 1 --2- 172.21.15.2:0/3527196227 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e559b62e0 0x559e559cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.206+0000 7f231659c640 1 -- 172.21.15.2:0/3527196227 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e559b62e0 msgr2=0x559e559cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.206+0000 7f231659c640 1 --2- 172.21.15.2:0/3527196227 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e559b62e0 0x559e559cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:54.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.206+0000 7f231659c640 1 -- 172.21.15.2:0/3527196227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559e559aba20 msgr2=0x559e559cdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.206+0000 7f231659c640 1 --2- 172.21.15.2:0/3527196227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559e559aba20 0x559e559cdab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.206+0000 7f231659c640 1 -- 172.21.15.2:0/3527196227 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e559a7440 con 0x559e559ab100 2024-05-14T17:10:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.206+0000 7f2315d9b640 1 --2- 172.21.15.2:0/3527196227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559e559aba20 0x559e559cdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:10:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.206+0000 7f231659c640 1 --2- 172.21.15.2:0/3527196227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e559ab100 0x559e559b37c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2308000c00 tx=0x7f2308040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0939c00d38301ba server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.206+0000 7f22fffff640 1 -- 172.21.15.2:0/3527196227 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2308035040 con 0x559e559ab100 2024-05-14T17:10:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.206+0000 7f22fffff640 1 -- 172.21.15.2:0/3527196227 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23080548d0 con 0x559e559ab100 2024-05-14T17:10:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.206+0000 7f22fffff640 1 -- 172.21.15.2:0/3527196227 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2308031cb0 con 0x559e559ab100 2024-05-14T17:10:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.207+0000 7f231f993740 1 -- 172.21.15.2:0/3527196227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e559ab100 msgr2=0x559e559b37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.207+0000 7f231f993740 1 --2- 172.21.15.2:0/3527196227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e559ab100 0x559e559b37c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2308000c00 tx=0x7f2308040000 comp rx=0 tx=0).stop 2024-05-14T17:10:54.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.208+0000 7f231f993740 1 -- 172.21.15.2:0/3527196227 shutdown_connections 2024-05-14T17:10:54.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.208+0000 7f231f993740 1 --2- 172.21.15.2:0/3527196227 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e559b62e0 0x559e559cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:54.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.208+0000 7f231f993740 1 --2- 172.21.15.2:0/3527196227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559e559aba20 0x559e559cdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.208+0000 7f231f993740 1 --2- 172.21.15.2:0/3527196227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e559ab100 0x559e559b37c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:54.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.208+0000 7f231f993740 1 -- 172.21.15.2:0/3527196227 >> 172.21.15.2:0/3527196227 conn(0x559e559a7610 msgr2=0x559e559b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.209+0000 7f231f993740 1 -- 172.21.15.2:0/3527196227 shutdown_connections 2024-05-14T17:10:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.209+0000 7f231f993740 1 -- 172.21.15.2:0/3527196227 wait complete. 2024-05-14T17:10:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.209+0000 7f231f993740 1 -- start start 2024-05-14T17:10:54.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.209+0000 7f231f993740 1 Processor -- start 2024-05-14T17:10:54.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.209+0000 7f231f993740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e559ab100 0x559e559bc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:54.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.209+0000 7f231f993740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e559aba20 0x559e559bc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:54.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.209+0000 7f231f993740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559e559b62e0 0x559e55a2f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:54.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.210+0000 7f2315d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e559aba20 0x559e559bc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:55.011 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.210+0000 7f2315d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e559aba20 0x559e559bc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:52854/0 (socket says 172.21.15.2:52854) 2024-05-14T17:10:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.210+0000 7f231659c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e559ab100 0x559e559bc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:55.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.210+0000 7f231659c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e559ab100 0x559e559bc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:34274/0 (socket says 172.21.15.2:34274) 2024-05-14T17:10:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.210+0000 7f2316d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559e559b62e0 0x559e55a2f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:55.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.210+0000 7f2316d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559e559b62e0 0x559e55a2f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40892/0 (socket says 172.21.15.2:40892) 2024-05-14T17:10:55.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.210+0000 7f231f993740 1 -- 172.21.15.2:0/2605672315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f23181aab90 con 0x559e559b62e0 2024-05-14T17:10:55.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.210+0000 7f2315d9b640 1 -- 172.21.15.2:0/2605672315 learned_addr learned my addr 172.21.15.2:0/2605672315 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:10:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.210+0000 7f231f993740 1 -- 172.21.15.2:0/2605672315 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f23181abfe0 con 0x559e559aba20 2024-05-14T17:10:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.210+0000 7f231f993740 1 -- 172.21.15.2:0/2605672315 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f23181ab240 con 0x559e559ab100 2024-05-14T17:10:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.210+0000 7f231f993740 10 client.0 _ll_register_callbacks cb 0x7f230c0e02d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:10:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.210+0000 7f231659c640 1 -- 172.21.15.2:0/2605672315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e559aba20 msgr2=0x559e559bc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.210+0000 7f231659c640 1 --2- 172.21.15.2:0/2605672315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e559aba20 0x559e559bc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.210+0000 7f231659c640 1 -- 172.21.15.2:0/2605672315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559e559b62e0 msgr2=0x559e55a2f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:55.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.210+0000 7f231659c640 1 --2- 172.21.15.2:0/2605672315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559e559b62e0 0x559e55a2f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:55.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.210+0000 7f231659c640 1 -- 172.21.15.2:0/2605672315 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e55a2fbb0 con 0x559e559ab100 2024-05-14T17:10:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.210+0000 7f2315d9b640 1 --2- 172.21.15.2:0/2605672315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e559aba20 0x559e559bc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:10:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.211+0000 7f231659c640 1 --2- 172.21.15.2:0/2605672315 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e559ab100 0x559e559bc4a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2308003610 tx=0x7f230802a480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45e769241fb31eb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:55.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.211+0000 7f22fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-14T17:10:55.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.211+0000 7f22fdffb640 1 -- 172.21.15.2:0/2605672315 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2308062060 con 0x559e559ab100 2024-05-14T17:10:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.211+0000 7f22fdffb640 1 -- 172.21.15.2:0/2605672315 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f2308054420 con 0x559e559ab100 2024-05-14T17:10:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.211+0000 7f231f993740 1 -- 172.21.15.2:0/2605672315 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23080548d0 con 0x559e559ab100 2024-05-14T17:10:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.211+0000 7f22fdffb640 1 -- 172.21.15.2:0/2605672315 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f2308054620 con 0x559e559ab100 2024-05-14T17:10:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.211+0000 7f231f993740 10 client.26395 fetch_fsmap learned FSMap version 51 2024-05-14T17:10:55.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.211+0000 7f231f993740 1 -- 172.21.15.2:0/2605672315 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x559e55a30e60 con 0x559e559ab100 2024-05-14T17:10:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.212+0000 7f22fdffb640 1 -- 172.21.15.2:0/2605672315 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2308031de0 con 0x559e559ab100 2024-05-14T17:10:55.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.212+0000 7f22fdffb640 1 -- 172.21.15.2:0/2605672315 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 51) ==== 8354+0+0 (secure 0 0 0) 0x7f2308044030 con 0x559e559ab100 2024-05-14T17:10:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.212+0000 7f231f993740 10 client.26395 fetch_fsmap finished waiting for FSMap version 51 2024-05-14T17:10:55.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.212+0000 7f231f993740 10 client.26395 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:10:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.212+0000 7f231f993740 20 client.26395 populate_metadata read hostname 'smithi002' 2024-05-14T17:10:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.212+0000 7f231f993740 1 --2- 172.21.15.2:0/2605672315 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x559e55a31590 0x559e55a339b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.212+0000 7f231f993740 4 client.26395 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:10:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.212+0000 7f231f993740 1 -- 172.21.15.2:0/2605672315 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f23080548d0 con 0x559e55a31590 2024-05-14T17:10:55.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.212+0000 7f2315d9b640 1 --2- 172.21.15.2:0/2605672315 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x559e55a31590 0x559e55a339b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.213+0000 7f2315d9b640 1 --2- 172.21.15.2:0/2605672315 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x559e55a31590 0x559e55a339b0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52925e3fc48b59cc server_cookie=3971d107d8de4289 in_seq=0 out_seq=0 2024-05-14T17:10:56.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:48.213+0000 7f22fdffb640 10 client.26395 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:10:56.062 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:08:48 smithi002 ceph-mon[31538]: pgmap v440: 97 pgs: 97 active+clean; 4.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 6.2 MiB/s wr, 582 op/s 2024-05-14T17:10:56.085 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:08:50 smithi002 ceph-mon[31538]: pgmap v441: 97 pgs: 97 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 15 MiB/s wr, 939 op/s 2024-05-14T17:10:56.104 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 815.0 s, 1926.0 tps, lat 51.159 ms stddev 54.898 2024-05-14T17:10:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:51.980+0000 7f22fdffb640 1 -- 172.21.15.2:0/2605672315 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f23080548d0 con 0x559e55a31590 2024-05-14T17:10:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:51.980+0000 7f22fdffb640 10 client.26395 handle_command_reply: tid=0 2024-05-14T17:10:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:51.980+0000 7f231f993740 1 client.26395 shutdown 2024-05-14T17:10:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:51.980+0000 7f231f993740 1 -- 172.21.15.2:0/2605672315 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e559ab100 msgr2=0x559e559bc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:56.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:51.980+0000 7f231f993740 1 --2- 172.21.15.2:0/2605672315 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e559ab100 0x559e559bc4a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2308003610 tx=0x7f230802a480 comp rx=0 tx=0).stop 2024-05-14T17:10:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:51.980+0000 7f231f993740 1 -- 172.21.15.2:0/2605672315 shutdown_connections 2024-05-14T17:10:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:51.980+0000 7f231f993740 1 --2- 172.21.15.2:0/2605672315 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x559e55a31590 0x559e55a339b0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:51.980+0000 7f231f993740 1 --2- 172.21.15.2:0/2605672315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559e559b62e0 0x559e55a2f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:51.980+0000 7f231f993740 1 --2- 172.21.15.2:0/2605672315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e559aba20 0x559e559bc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:51.980+0000 7f231f993740 1 --2- 172.21.15.2:0/2605672315 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e559ab100 0x559e559bc4a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:51.980+0000 7f231f993740 1 -- 172.21.15.2:0/2605672315 >> 172.21.15.2:0/2605672315 conn(0x559e559a7610 msgr2=0x559e559b3920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:51.981+0000 7f22fdffb640 0 client.26395 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:10:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:51.981+0000 7f231f993740 1 -- 172.21.15.2:0/2605672315 shutdown_connections 2024-05-14T17:10:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:51.981+0000 7f231f993740 1 -- 172.21.15.2:0/2605672315 wait complete. 2024-05-14T17:10:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:51.981+0000 7f231f993740 20 client.26395 trim_cache size 0 max 16384 2024-05-14T17:10:56.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:10:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:10:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:10:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:51.982+0000 7f231cc72640 1 -- 172.21.15.2:0/2485154996 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f23000b2a90 msgr2=0x7f23000b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:56.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:51.982+0000 7f231cc72640 1 --2- 172.21.15.2:0/2485154996 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f23000b2a90 0x7f23000b4f50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x559e559ac6a0 tx=0x7f2308005a20 comp rx=0 tx=0).stop 2024-05-14T17:10:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:51.982+0000 7f231cc72640 1 -- 172.21.15.2:0/2485154996 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23181045e0 msgr2=0x7f2318101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:56.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:51.982+0000 7f231cc72640 1 --2- 172.21.15.2:0/2485154996 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23181045e0 0x7f2318101e50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f230c00e9f0 tx=0x7f230c00eeb0 comp rx=0 tx=0).stop 2024-05-14T17:10:56.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:51.982+0000 7f231cc72640 1 -- 172.21.15.2:0/2485154996 shutdown_connections 2024-05-14T17:10:56.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:51.982+0000 7f231cc72640 1 --2- 172.21.15.2:0/2485154996 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f23000b2a90 0x7f23000b4f50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:51.982+0000 7f231cc72640 1 --2- 172.21.15.2:0/2485154996 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f231810d090 0x7f2318102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:51.982+0000 7f231cc72640 1 --2- 172.21.15.2:0/2485154996 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23181045e0 0x7f2318101e50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:51.982+0000 7f231cc72640 1 --2- 172.21.15.2:0/2485154996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2318103cc0 0x7f2318101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:56.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:51.982+0000 7f231cc72640 1 -- 172.21.15.2:0/2485154996 >> 172.21.15.2:0/2485154996 conn(0x7f2318078b60 msgr2=0x7f23180fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:51.982+0000 7f231cc72640 1 -- 172.21.15.2:0/2485154996 shutdown_connections 2024-05-14T17:10:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:08:51.982+0000 7f231cc72640 1 -- 172.21.15.2:0/2485154996 wait complete. 2024-05-14T17:10:56.780 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:10:50 smithi151 ceph-mon[36066]: pgmap v505: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 1.4 MiB/s wr, 126 op/s 2024-05-14T17:10:56.796 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:10:52 smithi151 ceph-mon[36066]: pgmap v506: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.1 MiB/s wr, 82 op/s 2024-05-14T17:10:56.813 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:10:54 smithi151 ceph-mon[36066]: pgmap v507: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 1.2 MiB/s wr, 89 op/s 2024-05-14T17:10:56.830 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:10:50 smithi123 ceph-mon[35971]: pgmap v505: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 1.4 MiB/s wr, 126 op/s 2024-05-14T17:10:56.872 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:10:52 smithi123 ceph-mon[35971]: pgmap v506: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.1 MiB/s wr, 82 op/s 2024-05-14T17:10:56.888 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:10:54 smithi123 ceph-mon[35971]: pgmap v507: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 1.2 MiB/s wr, 89 op/s 2024-05-14T17:10:56.906 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:08:52 smithi002 ceph-mon[31538]: pgmap v442: 97 pgs: 97 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 12 MiB/s wr, 692 op/s 2024-05-14T17:10:56.947 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:08:54 smithi002 ceph-mon[31538]: pgmap v443: 97 pgs: 97 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 18 MiB/s wr, 898 op/s 2024-05-14T17:10:56.964 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 820.0 s, 2180.8 tps, lat 46.412 ms stddev 57.225 2024-05-14T17:10:56.982 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:08:56 smithi002 ceph-mon[31538]: pgmap v444: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 17 MiB/s wr, 754 op/s 2024-05-14T17:10:56.997 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:08:58 smithi002 ceph-mon[31538]: pgmap v445: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 16 MiB/s wr, 633 op/s 2024-05-14T17:10:57.014 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:09:00 smithi002 ceph-mon[31538]: pgmap v446: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 22 MiB/s wr, 904 op/s 2024-05-14T17:10:57.063 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 825.0 s, 1496.0 tps, lat 64.703 ms stddev 97.319 2024-05-14T17:10:57.081 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:09:02 smithi002 ceph-mon[31538]: pgmap v447: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 547 op/s 2024-05-14T17:10:57.120 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:09:04 smithi002 ceph-mon[31538]: pgmap v448: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 17 MiB/s wr, 708 op/s 2024-05-14T17:10:57.140 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:10:56 smithi151 ceph-mon[36066]: pgmap v508: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 52 KiB/s wr, 13 op/s 2024-05-14T17:10:57.156 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:10:56 smithi123 ceph-mon[35971]: pgmap v508: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 52 KiB/s wr, 13 op/s 2024-05-14T17:10:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.172+0000 7f9a80483640 1 -- 172.21.15.2:0/1434444370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a78111240 msgr2=0x7f9a78109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.172+0000 7f9a80483640 1 --2- 172.21.15.2:0/1434444370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a78111240 0x7f9a78109a50 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f9a74009930 tx=0x7f9a7402a4d0 comp rx=0 tx=0).stop 2024-05-14T17:10:57.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.175+0000 7f9a80483640 1 -- 172.21.15.2:0/1434444370 shutdown_connections 2024-05-14T17:10:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.175+0000 7f9a80483640 1 --2- 172.21.15.2:0/1434444370 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a7811c230 0x7f9a7811e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:57.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.175+0000 7f9a80483640 1 --2- 172.21.15.2:0/1434444370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a78111620 0x7f9a78109f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.175+0000 7f9a80483640 1 --2- 172.21.15.2:0/1434444370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a78111240 0x7f9a78109a50 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.175+0000 7f9a80483640 1 -- 172.21.15.2:0/1434444370 >> 172.21.15.2:0/1434444370 conn(0x7f9a7806db20 msgr2=0x7f9a7806df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:57.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.176+0000 7f9a80483640 1 -- 172.21.15.2:0/1434444370 shutdown_connections 2024-05-14T17:10:57.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.179+0000 7f9a80483640 1 -- 172.21.15.2:0/1434444370 wait complete. 2024-05-14T17:10:57.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.179+0000 7f9a80483640 1 Processor -- start 2024-05-14T17:10:57.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.180+0000 7f9a80483640 1 -- start start 2024-05-14T17:10:57.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.180+0000 7f9a80483640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a78111240 0x7f9a781a8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:57.414 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.180+0000 7f9a80483640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a7811c230 0x7f9a781a8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:57.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.180+0000 7f9a80483640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a781ac8e0 0x7f9a781acd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.180+0000 7f9a80483640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9a78121b00 con 0x7f9a7811c230 2024-05-14T17:10:57.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.180+0000 7f9a80483640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9a78121980 con 0x7f9a781ac8e0 2024-05-14T17:10:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.180+0000 7f9a80483640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9a78121c80 con 0x7f9a78111240 2024-05-14T17:10:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.182+0000 7f9a7da1e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a7811c230 0x7f9a781a8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:57.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.182+0000 7f9a7da1e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a7811c230 0x7f9a781a8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38796/0 (socket says 172.21.15.2:38796) 2024-05-14T17:10:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.182+0000 7f9a7da1e640 1 -- 172.21.15.2:0/162058144 learned_addr learned my addr 172.21.15.2:0/162058144 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:10:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.182+0000 7f9a7ea20640 1 --2- 172.21.15.2:0/162058144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a781ac8e0 0x7f9a781acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.182+0000 7f9a7e21f640 1 --2- 172.21.15.2:0/162058144 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a78111240 0x7f9a781a8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.182+0000 7f9a7da1e640 1 -- 172.21.15.2:0/162058144 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a78111240 msgr2=0x7f9a781a8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.182+0000 7f9a7da1e640 1 --2- 172.21.15.2:0/162058144 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a78111240 0x7f9a781a8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.182+0000 7f9a7da1e640 1 -- 172.21.15.2:0/162058144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a781ac8e0 msgr2=0x7f9a781acd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:57.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.183+0000 7f9a7da1e640 1 --2- 172.21.15.2:0/162058144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a781ac8e0 0x7f9a781acd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.183+0000 7f9a7da1e640 1 -- 172.21.15.2:0/162058144 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a781ad3d0 con 0x7f9a7811c230 2024-05-14T17:10:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.184+0000 7f9a7da1e640 1 --2- 172.21.15.2:0/162058144 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a7811c230 0x7f9a781a8560 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f9a6800d770 tx=0x7f9a6800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.184+0000 7f9a7ea20640 1 --2- 172.21.15.2:0/162058144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a781ac8e0 0x7f9a781acd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:10:57.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.187+0000 7f9a6f7fe640 1 -- 172.21.15.2:0/162058144 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a68017070 con 0x7f9a7811c230 2024-05-14T17:10:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.187+0000 7f9a80483640 1 -- 172.21.15.2:0/162058144 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a781ad6c0 con 0x7f9a7811c230 2024-05-14T17:10:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.188+0000 7f9a80483640 1 -- 172.21.15.2:0/162058144 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a781b4eb0 con 0x7f9a7811c230 2024-05-14T17:10:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.188+0000 7f9a6f7fe640 1 -- 172.21.15.2:0/162058144 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a68009c30 con 0x7f9a7811c230 2024-05-14T17:10:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.188+0000 7f9a6f7fe640 1 -- 172.21.15.2:0/162058144 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6800f620 con 0x7f9a7811c230 2024-05-14T17:10:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.192+0000 7f9a831a4740 1 Processor -- start 2024-05-14T17:10:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.192+0000 7f9a831a4740 1 -- start start 2024-05-14T17:10:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.194+0000 7f9a6f7fe640 1 -- 172.21.15.2:0/162058144 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f9a6801a070 con 0x7f9a7811c230 2024-05-14T17:10:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.196+0000 7f9a6f7fe640 1 --2- 172.21.15.2:0/162058144 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f9a640b2ba0 0x7f9a640b5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.196+0000 7f9a6f7fe640 1 -- 172.21.15.2:0/162058144 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f9a680a5d10 con 0x7f9a7811c230 2024-05-14T17:10:57.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.197+0000 7f9a831a4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56262112a8f0 0x56262112acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.197+0000 7f9a831a4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56262112b210 0x562621135590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.197+0000 7f9a831a4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562621135ad0 0x562621137ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.197+0000 7f9a831a4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56262109dc30 con 0x56262112b210 2024-05-14T17:10:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.197+0000 7f9a831a4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56262108d7d0 con 0x56262112a8f0 2024-05-14T17:10:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.197+0000 7f9a831a4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562621091810 con 0x562621135ad0 2024-05-14T17:10:58.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.197+0000 7f9a7ea20640 1 --2- 172.21.15.2:0/162058144 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f9a640b2ba0 0x7f9a640b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.197+0000 7f9a7e21f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56262112a8f0 0x56262112acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.197+0000 7f9a7e21f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56262112a8f0 0x56262112acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:37414/0 (socket says 172.21.15.2:37414) 2024-05-14T17:10:58.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.197+0000 7f9a7e21f640 1 -- 172.21.15.2:0/3698287056 learned_addr learned my addr 172.21.15.2:0/3698287056 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:10:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.198+0000 7f9a7da1e640 1 --2- 172.21.15.2:0/3698287056 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56262112b210 0x562621135590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.200+0000 7f9a7e21f640 1 -- 172.21.15.2:0/3698287056 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562621135ad0 msgr2=0x562621137ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:58.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.200+0000 7f9a7e21f640 1 --2- 172.21.15.2:0/3698287056 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562621135ad0 0x562621137ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.200+0000 7f9a7e21f640 1 -- 172.21.15.2:0/3698287056 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56262112b210 msgr2=0x562621135590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.200+0000 7f9a7e21f640 1 --2- 172.21.15.2:0/3698287056 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56262112b210 0x562621135590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.200+0000 7f9a7e21f640 1 -- 172.21.15.2:0/3698287056 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562620da2210 con 0x56262112a8f0 2024-05-14T17:10:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.201+0000 7f9a7ea20640 1 --2- 172.21.15.2:0/162058144 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f9a640b2ba0 0x7f9a640b5060 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9a70005cc0 tx=0x7f9a700087c0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.205+0000 7f9a7e21f640 1 --2- 172.21.15.2:0/3698287056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56262112a8f0 0x56262112acd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9a74000c00 tx=0x7f9a74004170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7bfa5ca0f3f03a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.206+0000 7f9a6cbfe640 1 -- 172.21.15.2:0/3698287056 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7403f070 con 0x56262112a8f0 2024-05-14T17:10:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.206+0000 7f9a6cbfe640 1 -- 172.21.15.2:0/3698287056 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a740054f0 con 0x56262112a8f0 2024-05-14T17:10:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.206+0000 7f9a6cbfe640 1 -- 172.21.15.2:0/3698287056 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a74035030 con 0x56262112a8f0 2024-05-14T17:10:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.207+0000 7f9a831a4740 1 -- 172.21.15.2:0/3698287056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56262112a8f0 msgr2=0x56262112acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:58.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.207+0000 7f9a831a4740 1 --2- 172.21.15.2:0/3698287056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56262112a8f0 0x56262112acd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9a74000c00 tx=0x7f9a74004170 comp rx=0 tx=0).stop 2024-05-14T17:10:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.208+0000 7f9a831a4740 1 -- 172.21.15.2:0/3698287056 shutdown_connections 2024-05-14T17:10:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.208+0000 7f9a831a4740 1 --2- 172.21.15.2:0/3698287056 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562621135ad0 0x562621137ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:58.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.208+0000 7f9a831a4740 1 --2- 172.21.15.2:0/3698287056 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56262112b210 0x562621135590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:58.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.208+0000 7f9a831a4740 1 --2- 172.21.15.2:0/3698287056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56262112a8f0 0x56262112acd0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:58.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.208+0000 7f9a831a4740 1 -- 172.21.15.2:0/3698287056 >> 172.21.15.2:0/3698287056 conn(0x562621126e00 msgr2=0x5626211271e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.208+0000 7f9a831a4740 1 -- 172.21.15.2:0/3698287056 shutdown_connections 2024-05-14T17:10:58.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.208+0000 7f9a831a4740 1 -- 172.21.15.2:0/3698287056 wait complete. 2024-05-14T17:10:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.209+0000 7f9a831a4740 1 -- start start 2024-05-14T17:10:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.209+0000 7f9a831a4740 1 Processor -- start 2024-05-14T17:10:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.209+0000 7f9a831a4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56262112a8f0 0x5626211353f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.210+0000 7f9a831a4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56262112b210 0x5626211326e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.210+0000 7f9a831a4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562621135ad0 0x562621132c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.210+0000 7f9a831a4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x562620d73790 con 0x56262112a8f0 2024-05-14T17:10:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.210+0000 7f9a831a4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562621049890 con 0x56262112b210 2024-05-14T17:10:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.210+0000 7f9a831a4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562621138740 con 0x562621135ad0 2024-05-14T17:10:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.210+0000 7f9a831a4740 10 client.0 _ll_register_callbacks cb 0x56262104cd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:10:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.210+0000 7f9a7ea20640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562621135ad0 0x562621132c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.210+0000 7f9a7ea20640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562621135ad0 0x562621132c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:37616/0 (socket says 172.21.15.2:37616) 2024-05-14T17:10:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.210+0000 7f9a7ea20640 1 -- 172.21.15.2:0/969238891 learned_addr learned my addr 172.21.15.2:0/969238891 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:10:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.212+0000 7f9a7e21f640 1 --2- 172.21.15.2:0/969238891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56262112a8f0 0x5626211353f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:58.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.213+0000 7f9a7ea20640 1 -- 172.21.15.2:0/969238891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56262112b210 msgr2=0x5626211326e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T17:10:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.213+0000 7f9a7ea20640 1 --2- 172.21.15.2:0/969238891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56262112b210 0x5626211326e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.213+0000 7f9a7ea20640 1 -- 172.21.15.2:0/969238891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56262112a8f0 msgr2=0x5626211353f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:10:58.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.213+0000 7f9a7ea20640 1 --2- 172.21.15.2:0/969238891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56262112a8f0 0x5626211353f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.213+0000 7f9a7ea20640 1 -- 172.21.15.2:0/969238891 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562620e6e620 con 0x562621135ad0 2024-05-14T17:10:58.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.214+0000 7f9a7ea20640 1 --2- 172.21.15.2:0/969238891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562621135ad0 0x562621132c20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9a70011df0 tx=0x7f9a70004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed3602bbb5068227 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.215+0000 7f9a61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-14T17:10:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.215+0000 7f9a61ffb640 1 -- 172.21.15.2:0/969238891 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a70022020 con 0x562621135ad0 2024-05-14T17:10:59.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.215+0000 7f9a831a4740 1 -- 172.21.15.2:0/969238891 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a740054f0 con 0x562621135ad0 2024-05-14T17:10:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.215+0000 7f9a61ffb640 1 -- 172.21.15.2:0/969238891 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f9a7001e450 con 0x562621135ad0 2024-05-14T17:10:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.218+0000 7f9a61ffb640 1 -- 172.21.15.2:0/969238891 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f9a7002d050 con 0x562621135ad0 2024-05-14T17:10:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.218+0000 7f9a831a4740 10 client.26398 fetch_fsmap learned FSMap version 51 2024-05-14T17:10:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.218+0000 7f9a831a4740 1 -- 172.21.15.2:0/969238891 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x562621143fd0 con 0x562621135ad0 2024-05-14T17:10:59.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.220+0000 7f9a61ffb640 1 -- 172.21.15.2:0/969238891 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a740054f0 con 0x562621135ad0 2024-05-14T17:10:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.221+0000 7f9a61ffb640 1 -- 172.21.15.2:0/969238891 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 51) ==== 8354+0+0 (secure 0 0 0) 0x7f9a7001d880 con 0x562621135ad0 2024-05-14T17:10:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.221+0000 7f9a831a4740 10 client.26398 fetch_fsmap finished waiting for FSMap version 51 2024-05-14T17:10:59.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.221+0000 7f9a831a4740 10 client.26398 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:10:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.221+0000 7f9a831a4740 20 client.26398 populate_metadata read hostname 'smithi002' 2024-05-14T17:10:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.221+0000 7f9a831a4740 1 --2- 172.21.15.2:0/969238891 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x562621145960 0x562621147de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:59.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.221+0000 7f9a831a4740 4 client.26398 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:10:59.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.221+0000 7f9a831a4740 1 -- 172.21.15.2:0/969238891 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56262109fb80 con 0x562621145960 2024-05-14T17:10:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.228+0000 7f9a7e21f640 1 --2- 172.21.15.2:0/969238891 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x562621145960 0x562621147de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:59.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.234+0000 7f9a7e21f640 1 --2- 172.21.15.2:0/969238891 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x562621145960 0x562621147de0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e421adbb2da6a532 server_cookie=e2c307ac5950f674 in_seq=0 out_seq=0 2024-05-14T17:10:59.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.235+0000 7f9a61ffb640 10 client.26398 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:10:59.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.236+0000 7f9a61ffb640 1 -- 172.21.15.2:0/969238891 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56262109fb80 con 0x562621145960 2024-05-14T17:10:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.236+0000 7f9a61ffb640 10 client.26398 handle_command_reply: tid=0 2024-05-14T17:10:59.385 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.236+0000 7f9a831a4740 1 client.26398 shutdown 2024-05-14T17:10:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.242+0000 7f9a831a4740 1 -- 172.21.15.2:0/969238891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562621135ad0 msgr2=0x562621132c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:10:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.242+0000 7f9a831a4740 1 --2- 172.21.15.2:0/969238891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562621135ad0 0x562621132c20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9a70011df0 tx=0x7f9a70004a40 comp rx=0 tx=0).stop 2024-05-14T17:10:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.244+0000 7f9a831a4740 1 -- 172.21.15.2:0/969238891 shutdown_connections 2024-05-14T17:10:59.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.244+0000 7f9a831a4740 1 --2- 172.21.15.2:0/969238891 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x562621145960 0x562621147de0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.244+0000 7f9a831a4740 1 --2- 172.21.15.2:0/969238891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562621135ad0 0x562621132c20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.244+0000 7f9a831a4740 1 --2- 172.21.15.2:0/969238891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56262112b210 0x5626211326e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.244+0000 7f9a831a4740 1 --2- 172.21.15.2:0/969238891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56262112a8f0 0x5626211353f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.244+0000 7f9a831a4740 1 -- 172.21.15.2:0/969238891 >> 172.21.15.2:0/969238891 conn(0x562621126e00 msgr2=0x562621126100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:10:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.244+0000 7f9a61ffb640 0 client.26398 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:10:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.244+0000 7f9a831a4740 1 -- 172.21.15.2:0/969238891 shutdown_connections 2024-05-14T17:10:59.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.244+0000 7f9a831a4740 1 -- 172.21.15.2:0/969238891 wait complete. 2024-05-14T17:10:59.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.244+0000 7f9a831a4740 20 client.26398 trim_cache size 0 max 16384 2024-05-14T17:10:59.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.281+0000 7f9a831a4740 1 Processor -- start 2024-05-14T17:10:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.281+0000 7f9a831a4740 1 -- start start 2024-05-14T17:10:59.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.282+0000 7f9a831a4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56262112a8f0 0x56262112cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.283+0000 7f9a7e21f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56262112a8f0 0x56262112cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:10:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.283+0000 7f9a831a4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56262112b210 0x562621141640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.283+0000 7f9a831a4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562621135ad0 0x562621134f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:10:59.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.283+0000 7f9a831a4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5c010fd0 con 0x56262112a8f0 2024-05-14T17:10:59.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.283+0000 7f9a831a4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5c00e840 con 0x562621135ad0 2024-05-14T17:10:59.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.283+0000 7f9a831a4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5c00c740 con 0x56262112b210 2024-05-14T17:10:59.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.283+0000 7f9a7e21f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56262112a8f0 0x56262112cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38812/0 (socket says 172.21.15.2:38812) 2024-05-14T17:10:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.283+0000 7f9a7e21f640 1 -- 172.21.15.2:0/4213273740 learned_addr learned my addr 172.21.15.2:0/4213273740 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:10:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.284+0000 7f9a7e21f640 1 -- 172.21.15.2:0/4213273740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56262112b210 msgr2=0x562621141640 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T17:10:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.284+0000 7f9a7e21f640 1 --2- 172.21.15.2:0/4213273740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56262112b210 0x562621141640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.284+0000 7f9a7e21f640 1 -- 172.21.15.2:0/4213273740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562621135ad0 msgr2=0x562621134f20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-14T17:10:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.284+0000 7f9a7e21f640 1 --2- 172.21.15.2:0/4213273740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562621135ad0 0x562621134f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:10:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.284+0000 7f9a7e21f640 1 -- 172.21.15.2:0/4213273740 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626211279d0 con 0x56262112a8f0 2024-05-14T17:10:59.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.284+0000 7f9a7e21f640 1 --2- 172.21.15.2:0/4213273740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56262112a8f0 0x56262112cd70 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f9a7402aac0 tx=0x7f9a74009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3abc40e6cfa1b752 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:10:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.285+0000 7f9a63fff640 1 -- 172.21.15.2:0/4213273740 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7403f050 con 0x56262112a8f0 2024-05-14T17:11:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.285+0000 7f9a63fff640 1 -- 172.21.15.2:0/4213273740 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a74003d60 con 0x56262112a8f0 2024-05-14T17:11:00.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.286+0000 7f9a63fff640 1 -- 172.21.15.2:0/4213273740 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a74035040 con 0x56262112a8f0 2024-05-14T17:11:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.287+0000 7f9a831a4740 1 -- 172.21.15.2:0/4213273740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56262112a8f0 msgr2=0x56262112cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:00.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.287+0000 7f9a831a4740 1 --2- 172.21.15.2:0/4213273740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56262112a8f0 0x56262112cd70 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f9a7402aac0 tx=0x7f9a74009540 comp rx=0 tx=0).stop 2024-05-14T17:11:00.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.288+0000 7f9a831a4740 1 -- 172.21.15.2:0/4213273740 shutdown_connections 2024-05-14T17:11:00.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.288+0000 7f9a831a4740 1 --2- 172.21.15.2:0/4213273740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562621135ad0 0x562621134f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.288+0000 7f9a831a4740 1 --2- 172.21.15.2:0/4213273740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56262112b210 0x562621141640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.288+0000 7f9a831a4740 1 --2- 172.21.15.2:0/4213273740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56262112a8f0 0x56262112cd70 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.288+0000 7f9a831a4740 1 -- 172.21.15.2:0/4213273740 >> 172.21.15.2:0/4213273740 conn(0x562621126e00 msgr2=0x56262112dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:00.178 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.290+0000 7f9a831a4740 1 -- 172.21.15.2:0/4213273740 shutdown_connections 2024-05-14T17:11:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.290+0000 7f9a831a4740 1 -- 172.21.15.2:0/4213273740 wait complete. 2024-05-14T17:11:00.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.291+0000 7f9a831a4740 1 -- start start 2024-05-14T17:11:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.291+0000 7f9a831a4740 1 Processor -- start 2024-05-14T17:11:00.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.292+0000 7f9a831a4740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56262112a8f0 0x562621138980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.293+0000 7f9a831a4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56262112b210 0x562621138ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.293+0000 7f9a831a4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562621135ad0 0x562621139400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:00.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.293+0000 7f9a831a4740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5c00cd00 con 0x56262112a8f0 2024-05-14T17:11:00.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.293+0000 7f9a831a4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5c00e170 con 0x562621135ad0 2024-05-14T17:11:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.293+0000 7f9a831a4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5c00d3d0 con 0x56262112b210 2024-05-14T17:11:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.293+0000 7f9a831a4740 10 client.0 _ll_register_callbacks cb 0x7f9a70011ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.295+0000 7f9a7ea20640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562621135ad0 0x562621139400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.295+0000 7f9a7da1e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56262112b210 0x562621138ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.295+0000 7f9a7ea20640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562621135ad0 0x562621139400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:37442/0 (socket says 172.21.15.2:37442) 2024-05-14T17:11:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.295+0000 7f9a7ea20640 1 -- 172.21.15.2:0/1591948514 learned_addr learned my addr 172.21.15.2:0/1591948514 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:11:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.296+0000 7f9a7e21f640 1 --2- 172.21.15.2:0/1591948514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56262112a8f0 0x562621138980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:00.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.296+0000 7f9a7da1e640 1 -- 172.21.15.2:0/1591948514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562621135ad0 msgr2=0x562621139400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.296+0000 7f9a7da1e640 1 --2- 172.21.15.2:0/1591948514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562621135ad0 0x562621139400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:00.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.296+0000 7f9a7da1e640 1 -- 172.21.15.2:0/1591948514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56262112a8f0 msgr2=0x562621138980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.296+0000 7f9a7da1e640 1 --2- 172.21.15.2:0/1591948514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56262112a8f0 0x562621138980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.296+0000 7f9a7da1e640 1 -- 172.21.15.2:0/1591948514 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626211bf3d0 con 0x56262112b210 2024-05-14T17:11:00.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.296+0000 7f9a7e21f640 1 --2- 172.21.15.2:0/1591948514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56262112a8f0 0x562621138980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:11:00.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.297+0000 7f9a7da1e640 1 --2- 172.21.15.2:0/1591948514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56262112b210 0x562621138ec0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9a680dfb90 tx=0x7f9a680b0de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ab80943c5f18b40 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.297+0000 7f9a61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-14T17:11:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.297+0000 7f9a61ffb640 1 -- 172.21.15.2:0/1591948514 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a680a5f10 con 0x56262112b210 2024-05-14T17:11:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.297+0000 7f9a61ffb640 1 -- 172.21.15.2:0/1591948514 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f9a680a5410 con 0x56262112b210 2024-05-14T17:11:00.696 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.298+0000 7f9a831a4740 1 -- 172.21.15.2:0/1591948514 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a74003d60 con 0x56262112b210 2024-05-14T17:11:00.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.298+0000 7f9a61ffb640 1 -- 172.21.15.2:0/1591948514 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f9a680a60b0 con 0x56262112b210 2024-05-14T17:11:00.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.299+0000 7f9a831a4740 10 client.26401 fetch_fsmap learned FSMap version 51 2024-05-14T17:11:00.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.299+0000 7f9a831a4740 1 -- 172.21.15.2:0/1591948514 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5626211c06e0 con 0x56262112b210 2024-05-14T17:11:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.299+0000 7f9a61ffb640 1 -- 172.21.15.2:0/1591948514 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a74003d60 con 0x56262112b210 2024-05-14T17:11:00.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.299+0000 7f9a61ffb640 1 -- 172.21.15.2:0/1591948514 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 51) ==== 8354+0+0 (secure 0 0 0) 0x7f9a680b20f0 con 0x56262112b210 2024-05-14T17:11:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.300+0000 7f9a831a4740 10 client.26401 fetch_fsmap finished waiting for FSMap version 51 2024-05-14T17:11:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.300+0000 7f9a831a4740 10 client.26401 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:11:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.300+0000 7f9a831a4740 20 client.26401 populate_metadata read hostname 'smithi002' 2024-05-14T17:11:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.300+0000 7f9a831a4740 1 --2- 172.21.15.2:0/1591948514 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5626211c0ee0 0x5626211c3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.300+0000 7f9a831a4740 4 client.26401 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:11:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.300+0000 7f9a831a4740 1 -- 172.21.15.2:0/1591948514 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562621122630 con 0x5626211c0ee0 2024-05-14T17:11:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.302+0000 7f9a7e21f640 1 --2- 172.21.15.2:0/1591948514 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5626211c0ee0 0x5626211c3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.305+0000 7f9a7e21f640 1 --2- 172.21.15.2:0/1591948514 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5626211c0ee0 0x5626211c3300 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11b1a0908282af12 server_cookie=7bb6f243e0dc9acd in_seq=0 out_seq=0 2024-05-14T17:11:00.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.305+0000 7f9a61ffb640 10 client.26401 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:11:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.841+0000 7f9a61ffb640 1 -- 172.21.15.2:0/1591948514 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562621122630 con 0x5626211c0ee0 2024-05-14T17:11:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.841+0000 7f9a61ffb640 10 client.26401 handle_command_reply: tid=0 2024-05-14T17:11:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.841+0000 7f9a831a4740 1 client.26401 shutdown 2024-05-14T17:11:01.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.842+0000 7f9a831a4740 1 -- 172.21.15.2:0/1591948514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56262112b210 msgr2=0x562621138ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:01.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.842+0000 7f9a831a4740 1 --2- 172.21.15.2:0/1591948514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56262112b210 0x562621138ec0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9a680dfb90 tx=0x7f9a680b0de0 comp rx=0 tx=0).stop 2024-05-14T17:11:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.842+0000 7f9a831a4740 1 -- 172.21.15.2:0/1591948514 shutdown_connections 2024-05-14T17:11:01.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.842+0000 7f9a831a4740 1 --2- 172.21.15.2:0/1591948514 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5626211c0ee0 0x5626211c3300 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:01.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.842+0000 7f9a831a4740 1 --2- 172.21.15.2:0/1591948514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562621135ad0 0x562621139400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:01.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.842+0000 7f9a831a4740 1 --2- 172.21.15.2:0/1591948514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56262112b210 0x562621138ec0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.842+0000 7f9a831a4740 1 --2- 172.21.15.2:0/1591948514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56262112a8f0 0x562621138980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.842+0000 7f9a831a4740 1 -- 172.21.15.2:0/1591948514 >> 172.21.15.2:0/1591948514 conn(0x562621126e00 msgr2=0x562621125870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.842+0000 7f9a61ffb640 0 client.26401 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:11:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.842+0000 7f9a831a4740 1 -- 172.21.15.2:0/1591948514 shutdown_connections 2024-05-14T17:11:01.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.842+0000 7f9a831a4740 1 -- 172.21.15.2:0/1591948514 wait complete. 2024-05-14T17:11:01.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.842+0000 7f9a831a4740 20 client.26401 trim_cache size 0 max 16384 2024-05-14T17:11:01.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:11:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:11:01.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:11:01.324 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:10:58 smithi151 ceph-mon[36066]: pgmap v509: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 199 KiB/s wr, 27 op/s 2024-05-14T17:11:01.339 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:11:00 smithi151 ceph-mon[36066]: pgmap v510: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-05-14T17:11:01.355 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:10:58 smithi123 ceph-mon[35971]: pgmap v509: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 199 KiB/s wr, 27 op/s 2024-05-14T17:11:01.372 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:11:00 smithi123 ceph-mon[35971]: pgmap v510: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-05-14T17:11:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.845+0000 7f9a80483640 1 -- 172.21.15.2:0/162058144 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f9a640b2ba0 msgr2=0x7f9a640b5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:01.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.845+0000 7f9a80483640 1 --2- 172.21.15.2:0/162058144 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f9a640b2ba0 0x7f9a640b5060 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9a70005cc0 tx=0x7f9a700087c0 comp rx=0 tx=0).stop 2024-05-14T17:11:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.845+0000 7f9a80483640 1 -- 172.21.15.2:0/162058144 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a7811c230 msgr2=0x7f9a781a8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.845+0000 7f9a80483640 1 --2- 172.21.15.2:0/162058144 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a7811c230 0x7f9a781a8560 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f9a6800d770 tx=0x7f9a6800dc30 comp rx=0 tx=0).stop 2024-05-14T17:11:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.845+0000 7f9a80483640 1 -- 172.21.15.2:0/162058144 shutdown_connections 2024-05-14T17:11:01.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.845+0000 7f9a80483640 1 --2- 172.21.15.2:0/162058144 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f9a640b2ba0 0x7f9a640b5060 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.845+0000 7f9a80483640 1 --2- 172.21.15.2:0/162058144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a781ac8e0 0x7f9a781acd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.845+0000 7f9a80483640 1 --2- 172.21.15.2:0/162058144 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a7811c230 0x7f9a781a8560 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.845+0000 7f9a80483640 1 --2- 172.21.15.2:0/162058144 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9a78111240 0x7f9a781a8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.846+0000 7f9a80483640 1 -- 172.21.15.2:0/162058144 >> 172.21.15.2:0/162058144 conn(0x7f9a7806db20 msgr2=0x7f9a7810ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.847+0000 7f9a80483640 1 -- 172.21.15.2:0/162058144 shutdown_connections 2024-05-14T17:11:01.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:05.847+0000 7f9a80483640 1 -- 172.21.15.2:0/162058144 wait complete. 2024-05-14T17:11:01.621 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 830.0 s, 278.4 tps, lat 358.192 ms stddev 322.227 2024-05-14T17:11:01.640 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:09:06 smithi002 ceph-mon[31538]: pgmap v449: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 544 op/s 2024-05-14T17:11:01.656 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:09:07 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:11:01.673 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:09:08 smithi002 ceph-mon[31538]: pgmap v450: 97 pgs: 97 active+clean; 4.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 10 MiB/s wr, 472 op/s 2024-05-14T17:11:01.707 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:09:08 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:11:01.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:09:08 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:11:01.782 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:09:08 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:11:01.807 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:09:10 smithi002 ceph-mon[31538]: osdmap e96: 12 total, 12 up, 12 in 2024-05-14T17:11:01.832 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:09:10 smithi002 ceph-mon[31538]: pgmap v452: 97 pgs: 97 active+clean; 5.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 30 MiB/s wr, 1.91k op/s 2024-05-14T17:11:01.857 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 835.0 s, 2068.4 tps, lat 50.067 ms stddev 71.845 2024-05-14T17:11:01.882 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:09:12 smithi002 ceph-mon[31538]: pgmap v453: 97 pgs: 97 active+clean; 5.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 30 MiB/s wr, 1.91k op/s 2024-05-14T17:11:01.907 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:09:14 smithi002 ceph-mon[31538]: pgmap v454: 97 pgs: 97 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 33 MiB/s wr, 2.07k op/s 2024-05-14T17:11:01.966 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 840.0 s, 1942.4 tps, lat 51.650 ms stddev 56.923 2024-05-14T17:11:01.991 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:09:16 smithi002 ceph-mon[31538]: pgmap v455: 97 pgs: 97 active+clean; 4.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 34 MiB/s wr, 2.10k op/s 2024-05-14T17:11:02.015 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:09:18 smithi002 ceph-mon[31538]: osdmap e97: 12 total, 12 up, 12 in 2024-05-14T17:11:02.040 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:09:18 smithi002 ceph-mon[31538]: pgmap v457: 97 pgs: 97 active+clean; 4.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 9.2 MiB/s wr, 535 op/s 2024-05-14T17:11:02.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.983+0000 7f2c34dd4640 1 -- 172.21.15.2:0/434205914 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c30107970 msgr2=0x7f2c30107d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:02.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.983+0000 7f2c34dd4640 1 --2- 172.21.15.2:0/434205914 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c30107970 0x7f2c30107d50 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f2c20009930 tx=0x7f2c2002a420 comp rx=0 tx=0).stop 2024-05-14T17:11:02.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.983+0000 7f2c34dd4640 1 -- 172.21.15.2:0/434205914 shutdown_connections 2024-05-14T17:11:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.983+0000 7f2c34dd4640 1 --2- 172.21.15.2:0/434205914 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c30102370 0x7f2c3010e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.983+0000 7f2c34dd4640 1 --2- 172.21.15.2:0/434205914 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c301019b0 0x7f2c30101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.983+0000 7f2c34dd4640 1 --2- 172.21.15.2:0/434205914 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c30107970 0x7f2c30107d50 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.983+0000 7f2c34dd4640 1 -- 172.21.15.2:0/434205914 >> 172.21.15.2:0/434205914 conn(0x7f2c30078aa0 msgr2=0x7f2c30078eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:02.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.983+0000 7f2c34dd4640 1 -- 172.21.15.2:0/434205914 shutdown_connections 2024-05-14T17:11:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.983+0000 7f2c34dd4640 1 -- 172.21.15.2:0/434205914 wait complete. 2024-05-14T17:11:02.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.984+0000 7f2c34dd4640 1 Processor -- start 2024-05-14T17:11:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.984+0000 7f2c34dd4640 1 -- start start 2024-05-14T17:11:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.984+0000 7f2c34dd4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c301019b0 0x7f2c30199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.984+0000 7f2c34dd4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c30102370 0x7f2c3019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.984+0000 7f2c34dd4640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c30107970 0x7f2c301a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.984+0000 7f2c34dd4640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2c30111c50 con 0x7f2c30102370 2024-05-14T17:11:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.984+0000 7f2c34dd4640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2c30111ad0 con 0x7f2c301019b0 2024-05-14T17:11:02.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.984+0000 7f2c34dd4640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2c30111dd0 con 0x7f2c30107970 2024-05-14T17:11:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.985+0000 7f2c2e59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c301019b0 0x7f2c30199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:02.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.985+0000 7f2c2e59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c301019b0 0x7f2c30199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:46068/0 (socket says 172.21.15.2:46068) 2024-05-14T17:11:02.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.985+0000 7f2c27fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c30102370 0x7f2c3019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.985+0000 7f2c2e59c640 1 -- 172.21.15.2:0/2176103077 learned_addr learned my addr 172.21.15.2:0/2176103077 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:11:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.985+0000 7f2c2ed9d640 1 --2- 172.21.15.2:0/2176103077 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c30107970 0x7f2c301a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.985+0000 7f2c2e59c640 1 -- 172.21.15.2:0/2176103077 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c30107970 msgr2=0x7f2c301a5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:02.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.985+0000 7f2c2e59c640 1 --2- 172.21.15.2:0/2176103077 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c30107970 0x7f2c301a5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.985+0000 7f2c2e59c640 1 -- 172.21.15.2:0/2176103077 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c30102370 msgr2=0x7f2c3019a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.985+0000 7f2c2e59c640 1 --2- 172.21.15.2:0/2176103077 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c30102370 0x7f2c3019a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.985+0000 7f2c2e59c640 1 -- 172.21.15.2:0/2176103077 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c301a5570 con 0x7f2c301019b0 2024-05-14T17:11:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.985+0000 7f2c27fff640 1 --2- 172.21.15.2:0/2176103077 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c30102370 0x7f2c3019a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:11:02.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.985+0000 7f2c2e59c640 1 --2- 172.21.15.2:0/2176103077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c301019b0 0x7f2c30199fb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2c20005ab0 tx=0x7f2c20003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.985+0000 7f2c277fe640 1 -- 172.21.15.2:0/2176103077 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2003f070 con 0x7f2c301019b0 2024-05-14T17:11:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.985+0000 7f2c34dd4640 1 -- 172.21.15.2:0/2176103077 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c301a5740 con 0x7f2c301019b0 2024-05-14T17:11:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.985+0000 7f2c277fe640 1 -- 172.21.15.2:0/2176103077 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c20031b50 con 0x7f2c301019b0 2024-05-14T17:11:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.986+0000 7f2c277fe640 1 -- 172.21.15.2:0/2176103077 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c20035030 con 0x7f2c301019b0 2024-05-14T17:11:03.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.986+0000 7f2c34dd4640 1 -- 172.21.15.2:0/2176103077 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c301a5c00 con 0x7f2c301019b0 2024-05-14T17:11:03.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.989+0000 7f2c277fe640 1 -- 172.21.15.2:0/2176103077 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f2c20042070 con 0x7f2c301019b0 2024-05-14T17:11:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.989+0000 7f2c37bcd740 1 Processor -- start 2024-05-14T17:11:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.989+0000 7f2c37bcd740 1 -- start start 2024-05-14T17:11:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.989+0000 7f2c37bcd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2df048e30 0x55b2df0477b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.989+0000 7f2c37bcd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df047cf0 0x55b2df048170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:03.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.989+0000 7f2c37bcd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2df05b170 0x55b2df05d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.989+0000 7f2c37bcd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b2defbac30 con 0x55b2df048e30 2024-05-14T17:11:03.384 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.989+0000 7f2c37bcd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b2defaa7d0 con 0x55b2df047cf0 2024-05-14T17:11:03.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.989+0000 7f2c37bcd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b2defae810 con 0x55b2df05b170 2024-05-14T17:11:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.990+0000 7f2c2ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2df048e30 0x55b2df0477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.990+0000 7f2c2ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2df048e30 0x55b2df0477b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56212/0 (socket says 172.21.15.2:56212) 2024-05-14T17:11:03.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.990+0000 7f2c2ed9d640 1 -- 172.21.15.2:0/425442439 learned_addr learned my addr 172.21.15.2:0/425442439 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:11:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.990+0000 7f2c27fff640 1 --2- 172.21.15.2:0/425442439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2df05b170 0x55b2df05d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.990+0000 7f2c2e59c640 1 --2- 172.21.15.2:0/425442439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df047cf0 0x55b2df048170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:03.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.990+0000 7f2c2ed9d640 1 -- 172.21.15.2:0/425442439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2df05b170 msgr2=0x55b2df05d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:03.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.990+0000 7f2c2ed9d640 1 --2- 172.21.15.2:0/425442439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2df05b170 0x55b2df05d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.990+0000 7f2c2ed9d640 1 -- 172.21.15.2:0/425442439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df047cf0 msgr2=0x55b2df048170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:03.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.990+0000 7f2c2ed9d640 1 --2- 172.21.15.2:0/425442439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df047cf0 0x55b2df048170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:03.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.990+0000 7f2c2ed9d640 1 -- 172.21.15.2:0/425442439 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2decbf210 con 0x55b2df048e30 2024-05-14T17:11:03.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.991+0000 7f2c2ed9d640 1 --2- 172.21.15.2:0/425442439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2df048e30 0x55b2df0477b0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f2c28009f70 tx=0x7f2c28010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c229c3fd80b91e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.991+0000 7f2c24ff9640 1 -- 172.21.15.2:0/425442439 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c28016070 con 0x55b2df048e30 2024-05-14T17:11:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.992+0000 7f2c277fe640 1 --2- 172.21.15.2:0/2176103077 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f2c1c0b2a10 0x7f2c1c0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:03.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.992+0000 7f2c2e59c640 1 --2- 172.21.15.2:0/2176103077 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f2c1c0b2a10 0x7f2c1c0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.992+0000 7f2c24ff9640 1 -- 172.21.15.2:0/425442439 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c28004d20 con 0x55b2df048e30 2024-05-14T17:11:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.992+0000 7f2c24ff9640 1 -- 172.21.15.2:0/425442439 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c280127d0 con 0x55b2df048e30 2024-05-14T17:11:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.993+0000 7f2c277fe640 1 -- 172.21.15.2:0/2176103077 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f2c20102c80 con 0x7f2c301019b0 2024-05-14T17:11:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.993+0000 7f2c37bcd740 1 -- 172.21.15.2:0/425442439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2df048e30 msgr2=0x55b2df0477b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.993+0000 7f2c2e59c640 1 --2- 172.21.15.2:0/2176103077 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f2c1c0b2a10 0x7f2c1c0b4ed0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2c20100480 tx=0x7f2c20009260 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:04.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.993+0000 7f2c37bcd740 1 --2- 172.21.15.2:0/425442439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2df048e30 0x55b2df0477b0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f2c28009f70 tx=0x7f2c28010800 comp rx=0 tx=0).stop 2024-05-14T17:11:04.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.994+0000 7f2c37bcd740 1 -- 172.21.15.2:0/425442439 shutdown_connections 2024-05-14T17:11:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.994+0000 7f2c37bcd740 1 --2- 172.21.15.2:0/425442439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2df05b170 0x55b2df05d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:04.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.994+0000 7f2c37bcd740 1 --2- 172.21.15.2:0/425442439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df047cf0 0x55b2df048170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.994+0000 7f2c37bcd740 1 --2- 172.21.15.2:0/425442439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2df048e30 0x55b2df0477b0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.994+0000 7f2c37bcd740 1 -- 172.21.15.2:0/425442439 >> 172.21.15.2:0/425442439 conn(0x55b2df043e00 msgr2=0x55b2df0441e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.994+0000 7f2c37bcd740 1 -- 172.21.15.2:0/425442439 shutdown_connections 2024-05-14T17:11:04.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.994+0000 7f2c37bcd740 1 -- 172.21.15.2:0/425442439 wait complete. 2024-05-14T17:11:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.994+0000 7f2c37bcd740 1 -- start start 2024-05-14T17:11:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.994+0000 7f2c37bcd740 1 Processor -- start 2024-05-14T17:11:04.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.995+0000 7f2c37bcd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2df047cf0 0x55b2df054a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:04.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.995+0000 7f2c37bcd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df048e30 0x55b2df054f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:04.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.995+0000 7f2c37bcd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2df05b170 0x55b2df05a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.995+0000 7f2c37bcd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55b2dec90790 con 0x55b2df05b170 2024-05-14T17:11:04.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.995+0000 7f2c37bcd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b2def66890 con 0x55b2df048e30 2024-05-14T17:11:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.995+0000 7f2c37bcd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b2df04ede0 con 0x55b2df047cf0 2024-05-14T17:11:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.995+0000 7f2c37bcd740 10 client.0 _ll_register_callbacks cb 0x55b2def69d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.995+0000 7f2c2ed9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2df047cf0 0x55b2df054a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:04.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.995+0000 7f2c2ed9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2df047cf0 0x55b2df054a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:35644/0 (socket says 172.21.15.2:35644) 2024-05-14T17:11:04.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.995+0000 7f2c2ed9d640 1 -- 172.21.15.2:0/4255973777 learned_addr learned my addr 172.21.15.2:0/4255973777 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:11:04.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.995+0000 7f2c27fff640 1 --2- 172.21.15.2:0/4255973777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df048e30 0x55b2df054f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:04.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.995+0000 7f2c2ed9d640 1 --2- 172.21.15.2:0/4255973777 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2df05b170 0x55b2df05a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:04.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.995+0000 7f2c2ed9d640 1 -- 172.21.15.2:0/4255973777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df048e30 msgr2=0x55b2df054f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.995+0000 7f2c2ed9d640 1 --2- 172.21.15.2:0/4255973777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df048e30 0x55b2df054f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.995+0000 7f2c2ed9d640 1 -- 172.21.15.2:0/4255973777 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2df05b170 msgr2=0x55b2df05a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.995+0000 7f2c2ed9d640 1 --2- 172.21.15.2:0/4255973777 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2df05b170 0x55b2df05a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.995+0000 7f2c2ed9d640 1 -- 172.21.15.2:0/4255973777 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2ded8b620 con 0x55b2df047cf0 2024-05-14T17:11:05.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.996+0000 7f2c2ed9d640 1 --2- 172.21.15.2:0/4255973777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2df047cf0 0x55b2df054a00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2c280059c0 tx=0x7f2c28012450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bce18a8a0331108 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:05.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.996+0000 7f2c15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-14T17:11:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.996+0000 7f2c15ffb640 1 -- 172.21.15.2:0/4255973777 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c28016040 con 0x55b2df047cf0 2024-05-14T17:11:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.996+0000 7f2c15ffb640 1 -- 172.21.15.2:0/4255973777 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f2c28003970 con 0x55b2df047cf0 2024-05-14T17:11:05.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.996+0000 7f2c37bcd740 1 -- 172.21.15.2:0/4255973777 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c28004d20 con 0x55b2df047cf0 2024-05-14T17:11:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.996+0000 7f2c15ffb640 1 -- 172.21.15.2:0/4255973777 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f2c28012780 con 0x55b2df047cf0 2024-05-14T17:11:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.997+0000 7f2c37bcd740 10 client.26413 fetch_fsmap learned FSMap version 51 2024-05-14T17:11:05.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.997+0000 7f2c37bcd740 1 -- 172.21.15.2:0/4255973777 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55b2df072230 con 0x55b2df047cf0 2024-05-14T17:11:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.997+0000 7f2c15ffb640 1 -- 172.21.15.2:0/4255973777 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c28004d20 con 0x55b2df047cf0 2024-05-14T17:11:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.997+0000 7f2c15ffb640 1 -- 172.21.15.2:0/4255973777 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 51) ==== 8354+0+0 (secure 0 0 0) 0x7f2c28025070 con 0x55b2df047cf0 2024-05-14T17:11:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.997+0000 7f2c37bcd740 10 client.26413 fetch_fsmap finished waiting for FSMap version 51 2024-05-14T17:11:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.997+0000 7f2c37bcd740 10 client.26413 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:11:05.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.997+0000 7f2c37bcd740 20 client.26413 populate_metadata read hostname 'smithi002' 2024-05-14T17:11:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.997+0000 7f2c37bcd740 1 --2- 172.21.15.2:0/4255973777 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55b2df073690 0x55b2df075a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:05.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.997+0000 7f2c37bcd740 4 client.26413 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:11:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.997+0000 7f2c37bcd740 1 -- 172.21.15.2:0/4255973777 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2defbcb80 con 0x55b2df073690 2024-05-14T17:11:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.997+0000 7f2c27fff640 1 --2- 172.21.15.2:0/4255973777 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55b2df073690 0x55b2df075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:05.744 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.998+0000 7f2c27fff640 1 --2- 172.21.15.2:0/4255973777 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55b2df073690 0x55b2df075a80 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=510fb615f8b93e1b server_cookie=c6baf80f9efeff51 in_seq=0 out_seq=0 2024-05-14T17:11:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.998+0000 7f2c15ffb640 10 client.26413 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:11:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.999+0000 7f2c15ffb640 1 -- 172.21.15.2:0/4255973777 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b2defbcb80 con 0x55b2df073690 2024-05-14T17:11:05.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.999+0000 7f2c15ffb640 10 client.26413 handle_command_reply: tid=0 2024-05-14T17:11:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.999+0000 7f2c37bcd740 1 client.26413 shutdown 2024-05-14T17:11:05.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.999+0000 7f2c37bcd740 1 -- 172.21.15.2:0/4255973777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2df047cf0 msgr2=0x55b2df054a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.999+0000 7f2c37bcd740 1 --2- 172.21.15.2:0/4255973777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2df047cf0 0x55b2df054a00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2c280059c0 tx=0x7f2c28012450 comp rx=0 tx=0).stop 2024-05-14T17:11:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.999+0000 7f2c37bcd740 1 -- 172.21.15.2:0/4255973777 shutdown_connections 2024-05-14T17:11:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.999+0000 7f2c37bcd740 1 --2- 172.21.15.2:0/4255973777 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55b2df073690 0x55b2df075a80 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.999+0000 7f2c37bcd740 1 --2- 172.21.15.2:0/4255973777 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2df05b170 0x55b2df05a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:06.237 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.999+0000 7f2c37bcd740 1 --2- 172.21.15.2:0/4255973777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df048e30 0x55b2df054f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:06.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.999+0000 7f2c37bcd740 1 --2- 172.21.15.2:0/4255973777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2df047cf0 0x55b2df054a00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:06.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.999+0000 7f2c37bcd740 1 -- 172.21.15.2:0/4255973777 >> 172.21.15.2:0/4255973777 conn(0x55b2df043e00 msgr2=0x55b2df049c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.999+0000 7f2c15ffb640 0 client.26413 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:11:06.397 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.999+0000 7f2c37bcd740 1 -- 172.21.15.2:0/4255973777 shutdown_connections 2024-05-14T17:11:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.999+0000 7f2c37bcd740 1 -- 172.21.15.2:0/4255973777 wait complete. 2024-05-14T17:11:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:18.999+0000 7f2c37bcd740 20 client.26413 trim_cache size 0 max 16384 2024-05-14T17:11:06.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.021+0000 7f2c37bcd740 1 Processor -- start 2024-05-14T17:11:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.021+0000 7f2c37bcd740 1 -- start start 2024-05-14T17:11:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.022+0000 7f2c37bcd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2df047cf0 0x55b2df043880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:06.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.022+0000 7f2c37bcd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2df048e30 0x55b2df049aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.022+0000 7f2c37bcd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df05b170 0x55b2df07b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.022+0000 7f2c37bcd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2c301aee20 con 0x55b2df047cf0 2024-05-14T17:11:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.022+0000 7f2c37bcd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2c301ac690 con 0x55b2df05b170 2024-05-14T17:11:06.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.022+0000 7f2c37bcd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2c301aa500 con 0x55b2df048e30 2024-05-14T17:11:06.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.022+0000 7f2c27fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2df048e30 0x55b2df049aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:06.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.022+0000 7f2c27fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2df048e30 0x55b2df049aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:35650/0 (socket says 172.21.15.2:35650) 2024-05-14T17:11:06.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.022+0000 7f2c27fff640 1 -- 172.21.15.2:0/3772874355 learned_addr learned my addr 172.21.15.2:0/3772874355 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:11:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.022+0000 7f2c2ed9d640 1 --2- 172.21.15.2:0/3772874355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df05b170 0x55b2df07b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.022+0000 7f2c2ed9d640 1 --2- 172.21.15.2:0/3772874355 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2df047cf0 0x55b2df043880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:06.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.023+0000 7f2c27fff640 1 -- 172.21.15.2:0/3772874355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df05b170 msgr2=0x55b2df07b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:06.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.023+0000 7f2c27fff640 1 --2- 172.21.15.2:0/3772874355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df05b170 0x55b2df07b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.023+0000 7f2c27fff640 1 -- 172.21.15.2:0/3772874355 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2df047cf0 msgr2=0x55b2df043880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.023+0000 7f2c27fff640 1 --2- 172.21.15.2:0/3772874355 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2df047cf0 0x55b2df043880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:07.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.023+0000 7f2c27fff640 1 -- 172.21.15.2:0/3772874355 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2df04d800 con 0x55b2df048e30 2024-05-14T17:11:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.023+0000 7f2c27fff640 1 --2- 172.21.15.2:0/3772874355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2df048e30 0x55b2df049aa0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x55b2df045220 tx=0x7f2c18005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ba679de4e7c843e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:07.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.023+0000 7f2c17fff640 1 -- 172.21.15.2:0/3772874355 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c18012be0 con 0x55b2df048e30 2024-05-14T17:11:07.237 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.023+0000 7f2c17fff640 1 -- 172.21.15.2:0/3772874355 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c18012d80 con 0x55b2df048e30 2024-05-14T17:11:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.023+0000 7f2c17fff640 1 -- 172.21.15.2:0/3772874355 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c180047a0 con 0x55b2df048e30 2024-05-14T17:11:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.024+0000 7f2c37bcd740 1 -- 172.21.15.2:0/3772874355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2df048e30 msgr2=0x55b2df049aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.024+0000 7f2c37bcd740 1 --2- 172.21.15.2:0/3772874355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2df048e30 0x55b2df049aa0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x55b2df045220 tx=0x7f2c18005f90 comp rx=0 tx=0).stop 2024-05-14T17:11:07.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.024+0000 7f2c37bcd740 1 -- 172.21.15.2:0/3772874355 shutdown_connections 2024-05-14T17:11:07.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.024+0000 7f2c37bcd740 1 --2- 172.21.15.2:0/3772874355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df05b170 0x55b2df07b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.024+0000 7f2c37bcd740 1 --2- 172.21.15.2:0/3772874355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2df048e30 0x55b2df049aa0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.024+0000 7f2c37bcd740 1 --2- 172.21.15.2:0/3772874355 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2df047cf0 0x55b2df043880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.024+0000 7f2c37bcd740 1 -- 172.21.15.2:0/3772874355 >> 172.21.15.2:0/3772874355 conn(0x55b2df043e00 msgr2=0x55b2df049390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:07.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.024+0000 7f2c37bcd740 1 -- 172.21.15.2:0/3772874355 shutdown_connections 2024-05-14T17:11:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.024+0000 7f2c37bcd740 1 -- 172.21.15.2:0/3772874355 wait complete. 2024-05-14T17:11:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.025+0000 7f2c37bcd740 1 -- start start 2024-05-14T17:11:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.025+0000 7f2c37bcd740 1 Processor -- start 2024-05-14T17:11:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.025+0000 7f2c37bcd740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2df047cf0 0x55b2df087080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.025+0000 7f2c37bcd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df048e30 0x55b2df04be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.025+0000 7f2c37bcd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2df05b170 0x55b2df04c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.025+0000 7f2c37bcd740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2c301aab50 con 0x55b2df047cf0 2024-05-14T17:11:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.025+0000 7f2c37bcd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2c301abfc0 con 0x55b2df048e30 2024-05-14T17:11:07.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.025+0000 7f2c37bcd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2c301ab220 con 0x55b2df05b170 2024-05-14T17:11:07.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.025+0000 7f2c37bcd740 10 client.0 _ll_register_callbacks cb 0x7f2c28004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:07.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.025+0000 7f2c2ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2df047cf0 0x55b2df087080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:08.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.025+0000 7f2c2ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2df047cf0 0x55b2df087080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56232/0 (socket says 172.21.15.2:56232) 2024-05-14T17:11:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.025+0000 7f2c2ed9d640 1 -- 172.21.15.2:0/1264292092 learned_addr learned my addr 172.21.15.2:0/1264292092 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:11:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.025+0000 7f2c27fff640 1 --2- 172.21.15.2:0/1264292092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df048e30 0x55b2df04be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.026+0000 7f2c2ed9d640 1 --2- 172.21.15.2:0/1264292092 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2df05b170 0x55b2df04c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.026+0000 7f2c27fff640 1 -- 172.21.15.2:0/1264292092 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2df05b170 msgr2=0x55b2df04c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.026+0000 7f2c27fff640 1 --2- 172.21.15.2:0/1264292092 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2df05b170 0x55b2df04c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:08.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.026+0000 7f2c27fff640 1 -- 172.21.15.2:0/1264292092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2df047cf0 msgr2=0x55b2df087080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:08.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.026+0000 7f2c27fff640 1 --2- 172.21.15.2:0/1264292092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2df047cf0 0x55b2df087080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:08.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.026+0000 7f2c27fff640 1 -- 172.21.15.2:0/1264292092 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2df04cbc0 con 0x55b2df048e30 2024-05-14T17:11:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.026+0000 7f2c2ed9d640 1 --2- 172.21.15.2:0/1264292092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2df047cf0 0x55b2df087080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:11:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.026+0000 7f2c27fff640 1 --2- 172.21.15.2:0/1264292092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df048e30 0x55b2df04be50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2c18000c00 tx=0x7f2c18009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=669fdd46f872fd09 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:08.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.026+0000 7f2c15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-05-14T17:11:08.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.026+0000 7f2c15ffb640 1 -- 172.21.15.2:0/1264292092 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1801f890 con 0x55b2df048e30 2024-05-14T17:11:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.026+0000 7f2c15ffb640 1 -- 172.21.15.2:0/1264292092 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f2c1801fa30 con 0x55b2df048e30 2024-05-14T17:11:08.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.026+0000 7f2c37bcd740 1 -- 172.21.15.2:0/1264292092 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c18012d80 con 0x55b2df048e30 2024-05-14T17:11:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.027+0000 7f2c15ffb640 1 -- 172.21.15.2:0/1264292092 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f2c18005020 con 0x55b2df048e30 2024-05-14T17:11:08.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.027+0000 7f2c37bcd740 10 client.26423 fetch_fsmap learned FSMap version 51 2024-05-14T17:11:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.027+0000 7f2c37bcd740 1 -- 172.21.15.2:0/1264292092 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55b2df0dda50 con 0x55b2df048e30 2024-05-14T17:11:08.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.027+0000 7f2c15ffb640 1 -- 172.21.15.2:0/1264292092 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c18012d80 con 0x55b2df048e30 2024-05-14T17:11:08.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.027+0000 7f2c15ffb640 1 -- 172.21.15.2:0/1264292092 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 51) ==== 8354+0+0 (secure 0 0 0) 0x7f2c180035f0 con 0x55b2df048e30 2024-05-14T17:11:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.027+0000 7f2c37bcd740 10 client.26423 fetch_fsmap finished waiting for FSMap version 51 2024-05-14T17:11:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.027+0000 7f2c37bcd740 10 client.26423 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:11:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.028+0000 7f2c37bcd740 20 client.26423 populate_metadata read hostname 'smithi002' 2024-05-14T17:11:08.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.028+0000 7f2c37bcd740 1 --2- 172.21.15.2:0/1264292092 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55b2df0de070 0x55b2df0e0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:09.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.028+0000 7f2c37bcd740 4 client.26423 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:11:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.028+0000 7f2c37bcd740 1 -- 172.21.15.2:0/1264292092 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b2defb6360 con 0x55b2df0de070 2024-05-14T17:11:09.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.028+0000 7f2c2ed9d640 1 --2- 172.21.15.2:0/1264292092 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55b2df0de070 0x55b2df0e0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.028+0000 7f2c15ffb640 10 client.26423 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:11:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.028+0000 7f2c2ed9d640 1 --2- 172.21.15.2:0/1264292092 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55b2df0de070 0x55b2df0e0490 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a092bf2b82676ca server_cookie=2a15a62fbe4d06b2 in_seq=0 out_seq=0 2024-05-14T17:11:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.078+0000 7f2c15ffb640 1 -- 172.21.15.2:0/1264292092 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b2defb6360 con 0x55b2df0de070 2024-05-14T17:11:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.078+0000 7f2c15ffb640 10 client.26423 handle_command_reply: tid=0 2024-05-14T17:11:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.079+0000 7f2c37bcd740 1 client.26423 shutdown 2024-05-14T17:11:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.082+0000 7f2c37bcd740 1 -- 172.21.15.2:0/1264292092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df048e30 msgr2=0x55b2df04be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.083+0000 7f2c37bcd740 1 --2- 172.21.15.2:0/1264292092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df048e30 0x55b2df04be50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2c18000c00 tx=0x7f2c18009f60 comp rx=0 tx=0).stop 2024-05-14T17:11:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.083+0000 7f2c37bcd740 1 -- 172.21.15.2:0/1264292092 shutdown_connections 2024-05-14T17:11:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.083+0000 7f2c37bcd740 1 --2- 172.21.15.2:0/1264292092 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55b2df0de070 0x55b2df0e0490 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:09.548 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.083+0000 7f2c37bcd740 1 --2- 172.21.15.2:0/1264292092 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b2df05b170 0x55b2df04c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.083+0000 7f2c37bcd740 1 --2- 172.21.15.2:0/1264292092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b2df048e30 0x55b2df04be50 secure :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2c18000c00 tx=0x7f2c18009f60 comp rx=0 tx=0).stop 2024-05-14T17:11:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.083+0000 7f2c37bcd740 1 --2- 172.21.15.2:0/1264292092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2df047cf0 0x55b2df087080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.083+0000 7f2c37bcd740 1 -- 172.21.15.2:0/1264292092 >> 172.21.15.2:0/1264292092 conn(0x55b2df043e00 msgr2=0x55b2df059680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.083+0000 7f2c15ffb640 0 client.26423 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:11:09.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.084+0000 7f2c37bcd740 1 -- 172.21.15.2:0/1264292092 shutdown_connections 2024-05-14T17:11:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.084+0000 7f2c37bcd740 1 -- 172.21.15.2:0/1264292092 wait complete. 2024-05-14T17:11:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.084+0000 7f2c37bcd740 20 client.26423 trim_cache size 0 max 16384 2024-05-14T17:11:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:11:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:11:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:11:09.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.086+0000 7f2c34dd4640 1 -- 172.21.15.2:0/2176103077 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f2c1c0b2a10 msgr2=0x7f2c1c0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:10.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.086+0000 7f2c34dd4640 1 --2- 172.21.15.2:0/2176103077 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f2c1c0b2a10 0x7f2c1c0b4ed0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2c20100480 tx=0x7f2c20009260 comp rx=0 tx=0).stop 2024-05-14T17:11:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.086+0000 7f2c34dd4640 1 -- 172.21.15.2:0/2176103077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c301019b0 msgr2=0x7f2c30199fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.086+0000 7f2c34dd4640 1 --2- 172.21.15.2:0/2176103077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c301019b0 0x7f2c30199fb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2c20005ab0 tx=0x7f2c20003b90 comp rx=0 tx=0).stop 2024-05-14T17:11:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.086+0000 7f2c34dd4640 1 -- 172.21.15.2:0/2176103077 shutdown_connections 2024-05-14T17:11:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.086+0000 7f2c34dd4640 1 --2- 172.21.15.2:0/2176103077 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f2c1c0b2a10 0x7f2c1c0b4ed0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.086+0000 7f2c34dd4640 1 --2- 172.21.15.2:0/2176103077 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c30107970 0x7f2c301a5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:10.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.086+0000 7f2c34dd4640 1 --2- 172.21.15.2:0/2176103077 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c30102370 0x7f2c3019a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.086+0000 7f2c34dd4640 1 --2- 172.21.15.2:0/2176103077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c301019b0 0x7f2c30199fb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.086+0000 7f2c34dd4640 1 -- 172.21.15.2:0/2176103077 >> 172.21.15.2:0/2176103077 conn(0x7f2c30078aa0 msgr2=0x7f2c30104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.087+0000 7f2c34dd4640 1 -- 172.21.15.2:0/2176103077 shutdown_connections 2024-05-14T17:11:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:19.087+0000 7f2c34dd4640 1 -- 172.21.15.2:0/2176103077 wait complete. 2024-05-14T17:11:10.424 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:09:20 smithi002 ceph-mon[31538]: pgmap v458: 97 pgs: 97 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 13 MiB/s wr, 758 op/s 2024-05-14T17:11:10.458 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 845.0 s, 1803.2 tps, lat 55.392 ms stddev 91.040 2024-05-14T17:11:10.491 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:09:22 smithi002 ceph-mon[31538]: pgmap v459: 97 pgs: 97 active+clean; 4.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 13 MiB/s wr, 758 op/s 2024-05-14T17:11:10.523 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:09:24 smithi002 ceph-mon[31538]: pgmap v460: 97 pgs: 97 active+clean; 4.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 20 MiB/s wr, 1.43k op/s 2024-05-14T17:11:10.557 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:11:02 smithi151 ceph-mon[36066]: pgmap v511: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-05-14T17:11:10.615 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:11:04 smithi151 ceph-mon[36066]: pgmap v512: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-05-14T17:11:10.649 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:11:04 smithi151 ceph-mon[36066]: osdmap e100: 12 total, 12 up, 12 in 2024-05-14T17:11:10.682 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:11:06 smithi151 ceph-mon[36066]: pgmap v514: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-05-14T17:11:10.715 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:11:08 smithi151 ceph-mon[36066]: pgmap v515: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-05-14T17:11:10.749 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:11:02 smithi123 ceph-mon[35971]: pgmap v511: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-05-14T17:11:10.783 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:11:04 smithi123 ceph-mon[35971]: pgmap v512: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-05-14T17:11:10.816 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:11:04 smithi123 ceph-mon[35971]: osdmap e100: 12 total, 12 up, 12 in 2024-05-14T17:11:10.849 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:11:06 smithi123 ceph-mon[35971]: pgmap v514: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-05-14T17:11:10.883 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:11:08 smithi123 ceph-mon[35971]: pgmap v515: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-05-14T17:11:10.916 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 850.0 s, 1762.4 tps, lat 56.494 ms stddev 74.050 2024-05-14T17:11:10.950 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:09:26 smithi002 ceph-mon[31538]: pgmap v461: 97 pgs: 97 active+clean; 4.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 24 MiB/s wr, 1.81k op/s 2024-05-14T17:11:11.009 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:09:28 smithi002 ceph-mon[31538]: pgmap v462: 97 pgs: 97 active+clean; 4.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 23 MiB/s wr, 1.76k op/s 2024-05-14T17:11:11.041 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:09:30 smithi002 ceph-mon[31538]: pgmap v463: 97 pgs: 97 active+clean; 4.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 26 MiB/s wr, 2.01k op/s 2024-05-14T17:11:11.075 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 855.0 s, 2010.6 tps, lat 49.974 ms stddev 55.439 2024-05-14T17:11:11.109 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:11:10 smithi151 ceph-mon[36066]: pgmap v516: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 89 KiB/s wr, 32 op/s 2024-05-14T17:11:11.141 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:11:10 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:11:11.183 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:11:10 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:11:11.216 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:11:10 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:11:11.250 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:11:10 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:11:11.286 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:11:10 smithi123 ceph-mon[35971]: pgmap v516: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 89 KiB/s wr, 32 op/s 2024-05-14T17:11:11.316 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:11:10 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:11:11.350 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:11:10 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:11:11.383 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:11:10 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:11:11.419 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:11:10 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:11:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.201+0000 7f8ac9693640 1 -- 172.21.15.2:0/4195369621 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ac4108a70 msgr2=0x7f8ac4108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.201+0000 7f8ac9693640 1 --2- 172.21.15.2:0/4195369621 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ac4108a70 0x7f8ac4108e50 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009930 tx=0x7f8ab402a420 comp rx=0 tx=0).stop 2024-05-14T17:11:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.201+0000 7f8ac9693640 1 -- 172.21.15.2:0/4195369621 shutdown_connections 2024-05-14T17:11:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.201+0000 7f8ac9693640 1 --2- 172.21.15.2:0/4195369621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ac4103420 0x7f8ac410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:11.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.201+0000 7f8ac9693640 1 --2- 172.21.15.2:0/4195369621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ac4102a60 0x7f8ac4102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.201+0000 7f8ac9693640 1 --2- 172.21.15.2:0/4195369621 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ac4108a70 0x7f8ac4108e50 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.201+0000 7f8ac9693640 1 -- 172.21.15.2:0/4195369621 >> 172.21.15.2:0/4195369621 conn(0x7f8ac4079ee0 msgr2=0x7f8ac407a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.201+0000 7f8ac9693640 1 -- 172.21.15.2:0/4195369621 shutdown_connections 2024-05-14T17:11:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.201+0000 7f8ac9693640 1 -- 172.21.15.2:0/4195369621 wait complete. 2024-05-14T17:11:11.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.201+0000 7f8ac9693640 1 Processor -- start 2024-05-14T17:11:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.202+0000 7f8ac9693640 1 -- start start 2024-05-14T17:11:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.202+0000 7f8ac9693640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ac4102a60 0x7f8ac41991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.203+0000 7f8ac9693640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ac4103420 0x7f8ac4199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.203+0000 7f8ac9693640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ac4108a70 0x7f8ac419da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.203+0000 7f8ac9693640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac4112d80 con 0x7f8ac4108a70 2024-05-14T17:11:12.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.203+0000 7f8ac9693640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac4112c00 con 0x7f8ac4103420 2024-05-14T17:11:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.203+0000 7f8ac9693640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac4112f00 con 0x7f8ac4102a60 2024-05-14T17:11:12.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.203+0000 7f8ac27fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ac4103420 0x7f8ac4199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.203+0000 7f8ac2ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ac4102a60 0x7f8ac41991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:12.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.203+0000 7f8ac37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ac4108a70 0x7f8ac419da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:12.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.203+0000 7f8ac27fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ac4103420 0x7f8ac4199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:43142/0 (socket says 172.21.15.2:43142) 2024-05-14T17:11:12.239 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.203+0000 7f8ac2ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ac4102a60 0x7f8ac41991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:41508/0 (socket says 172.21.15.2:41508) 2024-05-14T17:11:12.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.203+0000 7f8ac27fc640 1 -- 172.21.15.2:0/4157836446 learned_addr learned my addr 172.21.15.2:0/4157836446 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:11:12.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.203+0000 7f8ac37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ac4108a70 0x7f8ac419da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54178/0 (socket says 172.21.15.2:54178) 2024-05-14T17:11:12.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.203+0000 7f8ac27fc640 1 -- 172.21.15.2:0/4157836446 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ac4102a60 msgr2=0x7f8ac41991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.203+0000 7f8ac27fc640 1 --2- 172.21.15.2:0/4157836446 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ac4102a60 0x7f8ac41991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.203+0000 7f8ac27fc640 1 -- 172.21.15.2:0/4157836446 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ac4108a70 msgr2=0x7f8ac419da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.203+0000 7f8ac27fc640 1 --2- 172.21.15.2:0/4157836446 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ac4108a70 0x7f8ac419da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:12.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.203+0000 7f8ac27fc640 1 -- 172.21.15.2:0/4157836446 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ac419e180 con 0x7f8ac4103420 2024-05-14T17:11:12.551 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.203+0000 7f8ac37fe640 1 --2- 172.21.15.2:0/4157836446 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ac4108a70 0x7f8ac419da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:11:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.203+0000 7f8ac27fc640 1 --2- 172.21.15.2:0/4157836446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ac4103420 0x7f8ac4199720 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8ab800e9f0 tx=0x7f8ab800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:12.618 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.203+0000 7f8aabfff640 1 -- 172.21.15.2:0/4157836446 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab800cd80 con 0x7f8ac4103420 2024-05-14T17:11:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.203+0000 7f8aabfff640 1 -- 172.21.15.2:0/4157836446 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ab80108e0 con 0x7f8ac4103420 2024-05-14T17:11:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.204+0000 7f8ac9693640 1 -- 172.21.15.2:0/4157836446 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ac419e470 con 0x7f8ac4103420 2024-05-14T17:11:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.204+0000 7f8aabfff640 1 -- 172.21.15.2:0/4157836446 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab8018980 con 0x7f8ac4103420 2024-05-14T17:11:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.204+0000 7f8ac9693640 1 -- 172.21.15.2:0/4157836446 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ac41a5d00 con 0x7f8ac4103420 2024-05-14T17:11:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.207+0000 7f8acc3b2740 1 Processor -- start 2024-05-14T17:11:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.208+0000 7f8acc3b2740 1 -- start start 2024-05-14T17:11:12.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.208+0000 7f8aabfff640 1 -- 172.21.15.2:0/4157836446 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f8ab8010430 con 0x7f8ac4103420 2024-05-14T17:11:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.208+0000 7f8acc3b2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df9aa598f0 0x55df9aa59cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.208+0000 7f8acc3b2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df9aa5a210 0x55df9aa64590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.208+0000 7f8acc3b2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df9aa64ad0 0x55df9aa66ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:13.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.208+0000 7f8acc3b2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55df9a9ccc30 con 0x55df9aa5a210 2024-05-14T17:11:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.208+0000 7f8acc3b2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55df9a9bc7d0 con 0x55df9aa64ad0 2024-05-14T17:11:13.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.208+0000 7f8acc3b2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55df9a9c0810 con 0x55df9aa598f0 2024-05-14T17:11:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.208+0000 7f8ac2ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df9aa5a210 0x55df9aa64590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:13.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.208+0000 7f8ac2ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df9aa5a210 0x55df9aa64590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54182/0 (socket says 172.21.15.2:54182) 2024-05-14T17:11:13.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.208+0000 7f8ac2ffd640 1 -- 172.21.15.2:0/2982427837 learned_addr learned my addr 172.21.15.2:0/2982427837 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:11:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.208+0000 7f8ac37fe640 1 --2- 172.21.15.2:0/2982427837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df9aa598f0 0x55df9aa59cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.208+0000 7f8ac27fc640 1 --2- 172.21.15.2:0/2982427837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df9aa64ad0 0x55df9aa66ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:13.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.208+0000 7f8ac2ffd640 1 -- 172.21.15.2:0/2982427837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df9aa598f0 msgr2=0x55df9aa59cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.208+0000 7f8ac2ffd640 1 --2- 172.21.15.2:0/2982427837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df9aa598f0 0x55df9aa59cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.208+0000 7f8ac2ffd640 1 -- 172.21.15.2:0/2982427837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df9aa64ad0 msgr2=0x55df9aa66ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:13.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.208+0000 7f8ac2ffd640 1 --2- 172.21.15.2:0/2982427837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df9aa64ad0 0x55df9aa66ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:13.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.208+0000 7f8ac2ffd640 1 -- 172.21.15.2:0/2982427837 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df9a6d1210 con 0x55df9aa5a210 2024-05-14T17:11:13.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.209+0000 7f8ac2ffd640 1 --2- 172.21.15.2:0/2982427837 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df9aa5a210 0x55df9aa64590 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4006010 tx=0x7f8ab4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265627b42b3940a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:13.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.209+0000 7f8aabfff640 1 --2- 172.21.15.2:0/4157836446 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f8aac0b2a10 0x7f8aac0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.209+0000 7f8aa97fa640 1 -- 172.21.15.2:0/2982427837 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4035040 con 0x55df9aa5a210 2024-05-14T17:11:13.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.209+0000 7f8aa97fa640 1 -- 172.21.15.2:0/2982427837 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ab40054f0 con 0x55df9aa5a210 2024-05-14T17:11:13.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.209+0000 7f8ac37fe640 1 --2- 172.21.15.2:0/4157836446 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f8aac0b2a10 0x7f8aac0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.209+0000 7f8aabfff640 1 -- 172.21.15.2:0/4157836446 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f8ab80e2940 con 0x7f8ac4103420 2024-05-14T17:11:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.209+0000 7f8aa97fa640 1 -- 172.21.15.2:0/2982427837 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab40035f0 con 0x55df9aa5a210 2024-05-14T17:11:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.210+0000 7f8acc3b2740 1 -- 172.21.15.2:0/2982427837 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df9aa5a210 msgr2=0x55df9aa64590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:13.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.210+0000 7f8ac37fe640 1 --2- 172.21.15.2:0/4157836446 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f8aac0b2a10 0x7f8aac0b4ed0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8abc005fd0 tx=0x7f8abc009500 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.210+0000 7f8acc3b2740 1 --2- 172.21.15.2:0/2982427837 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df9aa5a210 0x55df9aa64590 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4006010 tx=0x7f8ab4004040 comp rx=0 tx=0).stop 2024-05-14T17:11:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.210+0000 7f8acc3b2740 1 -- 172.21.15.2:0/2982427837 shutdown_connections 2024-05-14T17:11:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.210+0000 7f8acc3b2740 1 --2- 172.21.15.2:0/2982427837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df9aa64ad0 0x55df9aa66ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.210+0000 7f8acc3b2740 1 --2- 172.21.15.2:0/2982427837 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df9aa5a210 0x55df9aa64590 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:14.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.210+0000 7f8acc3b2740 1 --2- 172.21.15.2:0/2982427837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df9aa598f0 0x55df9aa59cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:14.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.210+0000 7f8acc3b2740 1 -- 172.21.15.2:0/2982427837 >> 172.21.15.2:0/2982427837 conn(0x55df9aa55e00 msgr2=0x55df9aa561e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.210+0000 7f8acc3b2740 1 -- 172.21.15.2:0/2982427837 shutdown_connections 2024-05-14T17:11:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.210+0000 7f8acc3b2740 1 -- 172.21.15.2:0/2982427837 wait complete. 2024-05-14T17:11:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.210+0000 7f8acc3b2740 1 -- start start 2024-05-14T17:11:14.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.210+0000 7f8acc3b2740 1 Processor -- start 2024-05-14T17:11:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.210+0000 7f8acc3b2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df9aa598f0 0x55df9aa64440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.210+0000 7f8acc3b2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df9aa5a210 0x55df9aa61730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:14.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.210+0000 7f8acc3b2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df9aa64ad0 0x55df9aa61c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:14.403 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.210+0000 7f8acc3b2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55df9a6a2790 con 0x55df9aa5a210 2024-05-14T17:11:14.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.210+0000 7f8acc3b2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55df9a978890 con 0x55df9aa64ad0 2024-05-14T17:11:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.210+0000 7f8acc3b2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55df9aa67740 con 0x55df9aa598f0 2024-05-14T17:11:14.503 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.210+0000 7f8acc3b2740 10 client.0 _ll_register_callbacks cb 0x55df9a97bd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:14.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.211+0000 7f8ac27fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df9aa5a210 0x55df9aa61730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:14.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.211+0000 7f8ac2ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df9aa598f0 0x55df9aa64440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.211+0000 7f8ac27fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df9aa5a210 0x55df9aa61730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54192/0 (socket says 172.21.15.2:54192) 2024-05-14T17:11:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.211+0000 7f8ac27fc640 1 -- 172.21.15.2:0/3029205612 learned_addr learned my addr 172.21.15.2:0/3029205612 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:11:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.211+0000 7f8ac2ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df9aa598f0 0x55df9aa64440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:41528/0 (socket says 172.21.15.2:41528) 2024-05-14T17:11:14.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.211+0000 7f8ac37fe640 1 --2- 172.21.15.2:0/3029205612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df9aa64ad0 0x55df9aa61c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.211+0000 7f8ac27fc640 1 -- 172.21.15.2:0/3029205612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df9aa598f0 msgr2=0x55df9aa64440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:14.788 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.211+0000 7f8ac27fc640 1 --2- 172.21.15.2:0/3029205612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df9aa598f0 0x55df9aa64440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.211+0000 7f8ac27fc640 1 -- 172.21.15.2:0/3029205612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df9aa64ad0 msgr2=0x55df9aa61c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.211+0000 7f8ac27fc640 1 --2- 172.21.15.2:0/3029205612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df9aa64ad0 0x55df9aa61c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:14.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.211+0000 7f8ac27fc640 1 -- 172.21.15.2:0/3029205612 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df9a79d620 con 0x55df9aa5a210 2024-05-14T17:11:14.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.211+0000 7f8ac27fc640 1 --2- 172.21.15.2:0/3029205612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df9aa5a210 0x55df9aa61730 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f8ab800c6a0 tx=0x7f8ab80e2c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a40765d7335624a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.211+0000 7f8aa6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:11:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.211+0000 7f8aa6ff5640 1 -- 172.21.15.2:0/3029205612 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab8010720 con 0x55df9aa5a210 2024-05-14T17:11:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.211+0000 7f8acc3b2740 1 -- 172.21.15.2:0/3029205612 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ab40054f0 con 0x55df9aa5a210 2024-05-14T17:11:15.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.211+0000 7f8aa6ff5640 1 -- 172.21.15.2:0/3029205612 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f8ab8010a90 con 0x55df9aa5a210 2024-05-14T17:11:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.211+0000 7f8aa6ff5640 1 -- 172.21.15.2:0/3029205612 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f8ab80a97e0 con 0x55df9aa5a210 2024-05-14T17:11:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.211+0000 7f8acc3b2740 10 client.16824 fetch_fsmap learned FSMap version 51 2024-05-14T17:11:15.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.211+0000 7f8acc3b2740 1 -- 172.21.15.2:0/3029205612 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55df9aa731f0 con 0x55df9aa5a210 2024-05-14T17:11:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.211+0000 7f8aa6ff5640 1 -- 172.21.15.2:0/3029205612 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ab40054f0 con 0x55df9aa5a210 2024-05-14T17:11:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.212+0000 7f8aa6ff5640 1 -- 172.21.15.2:0/3029205612 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 51) ==== 8354+0+0 (secure 0 0 0) 0x7f8ab80a5f60 con 0x55df9aa5a210 2024-05-14T17:11:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.212+0000 7f8acc3b2740 10 client.16824 fetch_fsmap finished waiting for FSMap version 51 2024-05-14T17:11:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.212+0000 7f8acc3b2740 10 client.16824 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:11:15.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.212+0000 7f8acc3b2740 20 client.16824 populate_metadata read hostname 'smithi002' 2024-05-14T17:11:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.212+0000 7f8acc3b2740 1 --2- 172.21.15.2:0/3029205612 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55df9aa746b0 0x55df9aa76b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.212+0000 7f8acc3b2740 4 client.16824 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:11:15.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.212+0000 7f8acc3b2740 1 -- 172.21.15.2:0/3029205612 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df9a9ceb80 con 0x55df9aa746b0 2024-05-14T17:11:15.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.212+0000 7f8ac2ffd640 1 --2- 172.21.15.2:0/3029205612 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55df9aa746b0 0x55df9aa76b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.212+0000 7f8aa6ff5640 10 client.16824 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:11:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.212+0000 7f8ac2ffd640 1 --2- 172.21.15.2:0/3029205612 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55df9aa746b0 0x55df9aa76b10 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53c20da58314feda server_cookie=147a9588a5f4699d in_seq=0 out_seq=0 2024-05-14T17:11:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.214+0000 7f8aa6ff5640 1 -- 172.21.15.2:0/3029205612 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55df9a9ceb80 con 0x55df9aa746b0 2024-05-14T17:11:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.214+0000 7f8aa6ff5640 10 client.16824 handle_command_reply: tid=0 2024-05-14T17:11:15.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.214+0000 7f8acc3b2740 1 client.16824 shutdown 2024-05-14T17:11:15.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.214+0000 7f8acc3b2740 1 -- 172.21.15.2:0/3029205612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df9aa5a210 msgr2=0x55df9aa61730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:15.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.214+0000 7f8acc3b2740 1 --2- 172.21.15.2:0/3029205612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df9aa5a210 0x55df9aa61730 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f8ab800c6a0 tx=0x7f8ab80e2c40 comp rx=0 tx=0).stop 2024-05-14T17:11:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.214+0000 7f8acc3b2740 1 -- 172.21.15.2:0/3029205612 shutdown_connections 2024-05-14T17:11:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.214+0000 7f8acc3b2740 1 --2- 172.21.15.2:0/3029205612 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55df9aa746b0 0x55df9aa76b10 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.214+0000 7f8acc3b2740 1 --2- 172.21.15.2:0/3029205612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df9aa64ad0 0x55df9aa61c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.214+0000 7f8aa6ff5640 0 client.16824 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:11:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.214+0000 7f8acc3b2740 1 --2- 172.21.15.2:0/3029205612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df9aa5a210 0x55df9aa61730 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.214+0000 7f8acc3b2740 1 --2- 172.21.15.2:0/3029205612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df9aa598f0 0x55df9aa64440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.214+0000 7f8acc3b2740 1 -- 172.21.15.2:0/3029205612 >> 172.21.15.2:0/3029205612 conn(0x55df9aa55e00 msgr2=0x55df9aa55100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.215+0000 7f8acc3b2740 1 -- 172.21.15.2:0/3029205612 shutdown_connections 2024-05-14T17:11:16.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.215+0000 7f8acc3b2740 1 -- 172.21.15.2:0/3029205612 wait complete. 2024-05-14T17:11:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.215+0000 7f8acc3b2740 20 client.16824 trim_cache size 0 max 16384 2024-05-14T17:11:16.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.234+0000 7f8acc3b2740 1 Processor -- start 2024-05-14T17:11:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.234+0000 7f8acc3b2740 1 -- start start 2024-05-14T17:11:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.234+0000 7f8acc3b2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df9aa598f0 0x55df9aa5cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:16.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.234+0000 7f8acc3b2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df9aa5a210 0x55df9aa63f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:16.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.234+0000 7f8acc3b2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df9aa64ad0 0x55df9aa7c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.234+0000 7f8acc3b2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac41b0f50 con 0x55df9aa5a210 2024-05-14T17:11:16.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.234+0000 7f8acc3b2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac41ae7c0 con 0x55df9aa64ad0 2024-05-14T17:11:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.234+0000 7f8acc3b2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac41ac630 con 0x55df9aa598f0 2024-05-14T17:11:16.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.234+0000 7f8ac27fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df9aa5a210 0x55df9aa63f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.234+0000 7f8ac27fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df9aa5a210 0x55df9aa63f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54208/0 (socket says 172.21.15.2:54208) 2024-05-14T17:11:16.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.234+0000 7f8ac27fc640 1 -- 172.21.15.2:0/2016599176 learned_addr learned my addr 172.21.15.2:0/2016599176 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:11:16.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.234+0000 7f8ac37fe640 1 --2- 172.21.15.2:0/2016599176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df9aa64ad0 0x55df9aa7c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.234+0000 7f8ac2ffd640 1 --2- 172.21.15.2:0/2016599176 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df9aa598f0 0x55df9aa5cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.234+0000 7f8ac27fc640 1 -- 172.21.15.2:0/2016599176 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df9aa598f0 msgr2=0x55df9aa5cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.235+0000 7f8ac27fc640 1 --2- 172.21.15.2:0/2016599176 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df9aa598f0 0x55df9aa5cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:16.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.235+0000 7f8ac27fc640 1 -- 172.21.15.2:0/2016599176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df9aa64ad0 msgr2=0x55df9aa7c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:16.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.235+0000 7f8ac27fc640 1 --2- 172.21.15.2:0/2016599176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df9aa64ad0 0x55df9aa7c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.235+0000 7f8ac27fc640 1 -- 172.21.15.2:0/2016599176 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df9aa62000 con 0x55df9aa5a210 2024-05-14T17:11:17.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.235+0000 7f8ac27fc640 1 --2- 172.21.15.2:0/2016599176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df9aa5a210 0x55df9aa63f70 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f8ab80e0690 tx=0x7f8ab8005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6522bd68032dbd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:17.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.235+0000 7f8aa8ff9640 1 -- 172.21.15.2:0/2016599176 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab80a5b90 con 0x55df9aa5a210 2024-05-14T17:11:17.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.235+0000 7f8aa8ff9640 1 -- 172.21.15.2:0/2016599176 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ab8010720 con 0x55df9aa5a210 2024-05-14T17:11:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.235+0000 7f8aa8ff9640 1 -- 172.21.15.2:0/2016599176 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab80a5f60 con 0x55df9aa5a210 2024-05-14T17:11:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.236+0000 7f8acc3b2740 1 -- 172.21.15.2:0/2016599176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df9aa5a210 msgr2=0x55df9aa63f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.236+0000 7f8acc3b2740 1 --2- 172.21.15.2:0/2016599176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df9aa5a210 0x55df9aa63f70 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f8ab80e0690 tx=0x7f8ab8005ab0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.236+0000 7f8acc3b2740 1 -- 172.21.15.2:0/2016599176 shutdown_connections 2024-05-14T17:11:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.236+0000 7f8acc3b2740 1 --2- 172.21.15.2:0/2016599176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df9aa64ad0 0x55df9aa7c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.236+0000 7f8acc3b2740 1 --2- 172.21.15.2:0/2016599176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df9aa5a210 0x55df9aa63f70 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.236+0000 7f8acc3b2740 1 --2- 172.21.15.2:0/2016599176 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df9aa598f0 0x55df9aa5cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.236+0000 7f8acc3b2740 1 -- 172.21.15.2:0/2016599176 >> 172.21.15.2:0/2016599176 conn(0x55df9aa55e00 msgr2=0x55df9aa5bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.236+0000 7f8acc3b2740 1 -- 172.21.15.2:0/2016599176 shutdown_connections 2024-05-14T17:11:17.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.236+0000 7f8acc3b2740 1 -- 172.21.15.2:0/2016599176 wait complete. 2024-05-14T17:11:17.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.236+0000 7f8acc3b2740 1 -- start start 2024-05-14T17:11:17.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.236+0000 7f8acc3b2740 1 Processor -- start 2024-05-14T17:11:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.236+0000 7f8acc3b2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df9aa598f0 0x55df9aa6ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:17.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.236+0000 7f8acc3b2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df9aa5a210 0x55df9aa69180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.236+0000 7f8acc3b2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df9aa64ad0 0x55df9aa696c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.236+0000 7f8acc3b2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac41acc80 con 0x55df9aa598f0 2024-05-14T17:11:17.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.236+0000 7f8acc3b2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac41ae0f0 con 0x55df9aa5a210 2024-05-14T17:11:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.236+0000 7f8acc3b2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac41ad350 con 0x55df9aa64ad0 2024-05-14T17:11:17.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.236+0000 7f8acc3b2740 10 client.0 _ll_register_callbacks cb 0x7f8ab80a9aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:17.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.236+0000 7f8ac2ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df9aa598f0 0x55df9aa6ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.236+0000 7f8ac2ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df9aa598f0 0x55df9aa6ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54218/0 (socket says 172.21.15.2:54218) 2024-05-14T17:11:17.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.236+0000 7f8ac2ffd640 1 -- 172.21.15.2:0/1258611357 learned_addr learned my addr 172.21.15.2:0/1258611357 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:11:17.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.236+0000 7f8ac37fe640 1 --2- 172.21.15.2:0/1258611357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df9aa64ad0 0x55df9aa696c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:17.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.237+0000 7f8ac27fc640 1 --2- 172.21.15.2:0/1258611357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df9aa5a210 0x55df9aa69180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:18.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.237+0000 7f8ac2ffd640 1 -- 172.21.15.2:0/1258611357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df9aa64ad0 msgr2=0x55df9aa696c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.237+0000 7f8ac2ffd640 1 --2- 172.21.15.2:0/1258611357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df9aa64ad0 0x55df9aa696c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.237+0000 7f8ac2ffd640 1 -- 172.21.15.2:0/1258611357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df9aa5a210 msgr2=0x55df9aa69180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.237+0000 7f8ac2ffd640 1 --2- 172.21.15.2:0/1258611357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df9aa5a210 0x55df9aa69180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.237+0000 7f8ac2ffd640 1 -- 172.21.15.2:0/1258611357 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df9aa69e00 con 0x55df9aa598f0 2024-05-14T17:11:18.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.237+0000 7f8ac2ffd640 1 --2- 172.21.15.2:0/1258611357 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df9aa598f0 0x55df9aa6ce00 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f8ab40057c0 tx=0x7f8ab402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f57a1ddc72bc2c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.237+0000 7f8aa6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:11:18.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.237+0000 7f8aa6ff5640 1 -- 172.21.15.2:0/1258611357 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4035040 con 0x55df9aa598f0 2024-05-14T17:11:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.237+0000 7f8aa6ff5640 1 -- 172.21.15.2:0/1258611357 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f8ab40048c0 con 0x55df9aa598f0 2024-05-14T17:11:18.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.237+0000 7f8acc3b2740 1 -- 172.21.15.2:0/1258611357 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55df9a9c8360 con 0x55df9aa598f0 2024-05-14T17:11:18.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.237+0000 7f8aa6ff5640 1 -- 172.21.15.2:0/1258611357 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f8ab4061050 con 0x55df9aa598f0 2024-05-14T17:11:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.237+0000 7f8acc3b2740 10 client.16830 fetch_fsmap learned FSMap version 51 2024-05-14T17:11:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.237+0000 7f8acc3b2740 1 -- 172.21.15.2:0/1258611357 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55df9aadff40 con 0x55df9aa598f0 2024-05-14T17:11:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.237+0000 7f8aa6ff5640 1 -- 172.21.15.2:0/1258611357 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55df9a9c8360 con 0x55df9aa598f0 2024-05-14T17:11:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.238+0000 7f8aa6ff5640 1 -- 172.21.15.2:0/1258611357 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 51) ==== 8354+0+0 (secure 0 0 0) 0x7f8ab40050a0 con 0x55df9aa598f0 2024-05-14T17:11:18.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.238+0000 7f8acc3b2740 10 client.16830 fetch_fsmap finished waiting for FSMap version 51 2024-05-14T17:11:18.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.238+0000 7f8acc3b2740 10 client.16830 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:11:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.238+0000 7f8acc3b2740 20 client.16830 populate_metadata read hostname 'smithi002' 2024-05-14T17:11:18.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.238+0000 7f8acc3b2740 1 --2- 172.21.15.2:0/1258611357 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55df9aae0280 0x55df9aae26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:18.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.238+0000 7f8acc3b2740 4 client.16830 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:11:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.238+0000 7f8acc3b2740 1 -- 172.21.15.2:0/1258611357 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55df9a9425f0 con 0x55df9aae0280 2024-05-14T17:11:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.238+0000 7f8ac27fc640 1 --2- 172.21.15.2:0/1258611357 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55df9aae0280 0x55df9aae26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:18.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.240+0000 7f8ac27fc640 1 --2- 172.21.15.2:0/1258611357 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55df9aae0280 0x55df9aae26a0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8d7a443a766c064 server_cookie=8cc9202c9d9c6e51 in_seq=0 out_seq=0 2024-05-14T17:11:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.240+0000 7f8aa6ff5640 10 client.16830 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:11:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.291+0000 7f8aa6ff5640 1 -- 172.21.15.2:0/1258611357 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55df9a9425f0 con 0x55df9aae0280 2024-05-14T17:11:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.291+0000 7f8aa6ff5640 10 client.16830 handle_command_reply: tid=0 2024-05-14T17:11:19.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.291+0000 7f8acc3b2740 1 client.16830 shutdown 2024-05-14T17:11:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.297+0000 7f8acc3b2740 1 -- 172.21.15.2:0/1258611357 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df9aa598f0 msgr2=0x55df9aa6ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:19.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.297+0000 7f8acc3b2740 1 --2- 172.21.15.2:0/1258611357 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df9aa598f0 0x55df9aa6ce00 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f8ab40057c0 tx=0x7f8ab402a420 comp rx=0 tx=0).stop 2024-05-14T17:11:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.298+0000 7f8acc3b2740 1 -- 172.21.15.2:0/1258611357 shutdown_connections 2024-05-14T17:11:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.298+0000 7f8acc3b2740 1 --2- 172.21.15.2:0/1258611357 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55df9aae0280 0x55df9aae26a0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:19.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.298+0000 7f8acc3b2740 1 --2- 172.21.15.2:0/1258611357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df9aa64ad0 0x55df9aa696c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.298+0000 7f8aa6ff5640 0 client.16830 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:11:19.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.298+0000 7f8acc3b2740 1 --2- 172.21.15.2:0/1258611357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df9aa5a210 0x55df9aa69180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:19.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.298+0000 7f8acc3b2740 1 --2- 172.21.15.2:0/1258611357 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55df9aa598f0 0x55df9aa6ce00 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:19.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.298+0000 7f8acc3b2740 1 -- 172.21.15.2:0/1258611357 >> 172.21.15.2:0/1258611357 conn(0x55df9aa55e00 msgr2=0x55df9aa54870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.303+0000 7f8acc3b2740 1 -- 172.21.15.2:0/1258611357 shutdown_connections 2024-05-14T17:11:19.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.304+0000 7f8acc3b2740 1 -- 172.21.15.2:0/1258611357 wait complete. 2024-05-14T17:11:19.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:11:19.549 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:11:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:11:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.307+0000 7f8acc3b2740 20 client.16830 trim_cache size 0 max 16384 2024-05-14T17:11:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.311+0000 7f8ac9693640 1 -- 172.21.15.2:0/4157836446 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f8aac0b2a10 msgr2=0x7f8aac0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.311+0000 7f8ac9693640 1 --2- 172.21.15.2:0/4157836446 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f8aac0b2a10 0x7f8aac0b4ed0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8abc005fd0 tx=0x7f8abc009500 comp rx=0 tx=0).stop 2024-05-14T17:11:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.311+0000 7f8ac9693640 1 -- 172.21.15.2:0/4157836446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ac4103420 msgr2=0x7f8ac4199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:19.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.311+0000 7f8ac9693640 1 --2- 172.21.15.2:0/4157836446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ac4103420 0x7f8ac4199720 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8ab800e9f0 tx=0x7f8ab800eeb0 comp rx=0 tx=0).stop 2024-05-14T17:11:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.315+0000 7f8ac9693640 1 -- 172.21.15.2:0/4157836446 shutdown_connections 2024-05-14T17:11:19.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.315+0000 7f8ac9693640 1 --2- 172.21.15.2:0/4157836446 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f8aac0b2a10 0x7f8aac0b4ed0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.315+0000 7f8ac9693640 1 --2- 172.21.15.2:0/4157836446 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ac4108a70 0x7f8ac419da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.315+0000 7f8ac9693640 1 --2- 172.21.15.2:0/4157836446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ac4103420 0x7f8ac4199720 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.315+0000 7f8ac9693640 1 --2- 172.21.15.2:0/4157836446 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ac4102a60 0x7f8ac41991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.315+0000 7f8ac9693640 1 -- 172.21.15.2:0/4157836446 >> 172.21.15.2:0/4157836446 conn(0x7f8ac4079ee0 msgr2=0x7f8ac41002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.315+0000 7f8ac9693640 1 -- 172.21.15.2:0/4157836446 shutdown_connections 2024-05-14T17:11:20.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:32.315+0000 7f8ac9693640 1 -- 172.21.15.2:0/4157836446 wait complete. 2024-05-14T17:11:20.117 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:09:32 smithi002 ceph-mon[31538]: pgmap v464: 97 pgs: 97 active+clean; 4.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 21 MiB/s wr, 1.79k op/s 2024-05-14T17:11:20.150 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:09:34 smithi002 ceph-mon[31538]: pgmap v465: 97 pgs: 97 active+clean; 4.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 25 MiB/s wr, 2.00k op/s 2024-05-14T17:11:20.184 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 860.0 s, 1872.6 tps, lat 53.154 ms stddev 57.303 2024-05-14T17:11:20.217 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:09:36 smithi002 ceph-mon[31538]: pgmap v466: 97 pgs: 97 active+clean; 4.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 15 MiB/s wr, 1.22k op/s 2024-05-14T17:11:20.250 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:09:38 smithi002 ceph-mon[31538]: pgmap v467: 97 pgs: 97 active+clean; 4.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 10 MiB/s wr, 826 op/s 2024-05-14T17:11:20.283 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:09:40 smithi002 ceph-mon[31538]: pgmap v468: 97 pgs: 97 active+clean; 5.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-05-14T17:11:20.317 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 865.0 s, 2263.4 tps, lat 44.160 ms stddev 55.009 2024-05-14T17:11:20.359 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:09:42 smithi002 ceph-mon[31538]: pgmap v469: 97 pgs: 97 active+clean; 5.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.8 MiB/s wr, 584 op/s 2024-05-14T17:11:20.392 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:09:44 smithi002 ceph-mon[31538]: pgmap v470: 97 pgs: 97 active+clean; 5.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 12 MiB/s wr, 834 op/s 2024-05-14T17:11:20.426 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:11:12 smithi123 ceph-mon[35971]: pgmap v517: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 89 KiB/s wr, 32 op/s 2024-05-14T17:11:20.458 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:11:12 smithi123 ceph-mon[35971]: osdmap e101: 12 total, 12 up, 12 in 2024-05-14T17:11:20.492 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:11:14 smithi123 ceph-mon[35971]: pgmap v519: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.0 MiB/s wr, 691 op/s 2024-05-14T17:11:20.525 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:11:16 smithi123 ceph-mon[35971]: pgmap v520: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.8 MiB/s wr, 774 op/s 2024-05-14T17:11:20.558 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:11:18 smithi123 ceph-mon[35971]: pgmap v521: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.8 MiB/s wr, 774 op/s 2024-05-14T17:11:20.592 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:11:19 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:11:20.625 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:11:12 smithi151 ceph-mon[36066]: pgmap v517: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 89 KiB/s wr, 32 op/s 2024-05-14T17:11:20.667 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:11:12 smithi151 ceph-mon[36066]: osdmap e101: 12 total, 12 up, 12 in 2024-05-14T17:11:20.707 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:11:14 smithi151 ceph-mon[36066]: pgmap v519: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.0 MiB/s wr, 691 op/s 2024-05-14T17:11:20.742 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:11:16 smithi151 ceph-mon[36066]: pgmap v520: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.8 MiB/s wr, 774 op/s 2024-05-14T17:11:20.775 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:11:18 smithi151 ceph-mon[36066]: pgmap v521: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.8 MiB/s wr, 774 op/s 2024-05-14T17:11:20.809 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:11:19 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:11:20.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.448+0000 7ff5af1da640 1 -- 172.21.15.2:0/898679251 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5a81045f0 msgr2=0x7ff5a810cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.448+0000 7ff5af1da640 1 --2- 172.21.15.2:0/898679251 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5a81045f0 0x7ff5a810cad0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ff594009900 tx=0x7ff59402a480 comp rx=0 tx=0).stop 2024-05-14T17:11:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.448+0000 7ff5af1da640 1 -- 172.21.15.2:0/898679251 shutdown_connections 2024-05-14T17:11:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.448+0000 7ff5af1da640 1 --2- 172.21.15.2:0/898679251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5a810d0a0 0x7ff5a810f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.448+0000 7ff5af1da640 1 --2- 172.21.15.2:0/898679251 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5a81045f0 0x7ff5a810cad0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:21.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.448+0000 7ff5af1da640 1 --2- 172.21.15.2:0/898679251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5a8103cd0 0x7ff5a81040b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.448+0000 7ff5af1da640 1 -- 172.21.15.2:0/898679251 >> 172.21.15.2:0/898679251 conn(0x7ff5a8078aa0 msgr2=0x7ff5a8078eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.448+0000 7ff5af1da640 1 -- 172.21.15.2:0/898679251 shutdown_connections 2024-05-14T17:11:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.448+0000 7ff5af1da640 1 -- 172.21.15.2:0/898679251 wait complete. 2024-05-14T17:11:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.449+0000 7ff5af1da640 1 Processor -- start 2024-05-14T17:11:21.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.449+0000 7ff5af1da640 1 -- start start 2024-05-14T17:11:21.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.449+0000 7ff5af1da640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5a8103cd0 0x7ff5a8196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.449+0000 7ff5af1da640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5a81045f0 0x7ff5a8197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.449+0000 7ff5af1da640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5a810d0a0 0x7ff5a819b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.449+0000 7ff5af1da640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a8103420 con 0x7ff5a810d0a0 2024-05-14T17:11:21.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.449+0000 7ff5af1da640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a81032a0 con 0x7ff5a8103cd0 2024-05-14T17:11:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.449+0000 7ff5af1da640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a81035a0 con 0x7ff5a81045f0 2024-05-14T17:11:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.450+0000 7ff5a7fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5a81045f0 0x7ff5a8197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.450+0000 7ff5ad777640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5a810d0a0 0x7ff5a819b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:21.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.450+0000 7ff5a7fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5a81045f0 0x7ff5a8197510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:53822/0 (socket says 172.21.15.2:53822) 2024-05-14T17:11:21.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.450+0000 7ff5a7fff640 1 -- 172.21.15.2:0/2579983069 learned_addr learned my addr 172.21.15.2:0/2579983069 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:11:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.450+0000 7ff5acf76640 1 --2- 172.21.15.2:0/2579983069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5a8103cd0 0x7ff5a8196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.450+0000 7ff5a7fff640 1 -- 172.21.15.2:0/2579983069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5a8103cd0 msgr2=0x7ff5a8196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:21.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.450+0000 7ff5a7fff640 1 --2- 172.21.15.2:0/2579983069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5a8103cd0 0x7ff5a8196fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.450+0000 7ff5a7fff640 1 -- 172.21.15.2:0/2579983069 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5a810d0a0 msgr2=0x7ff5a819b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.450+0000 7ff5a7fff640 1 --2- 172.21.15.2:0/2579983069 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5a810d0a0 0x7ff5a819b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.450+0000 7ff5a7fff640 1 -- 172.21.15.2:0/2579983069 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a819bf70 con 0x7ff5a81045f0 2024-05-14T17:11:21.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.450+0000 7ff5ad777640 1 --2- 172.21.15.2:0/2579983069 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5a810d0a0 0x7ff5a819b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:11:21.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.450+0000 7ff5a7fff640 1 --2- 172.21.15.2:0/2579983069 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5a81045f0 0x7ff5a8197510 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff594005940 tx=0x7ff594004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.450+0000 7ff5a5ffb640 1 -- 172.21.15.2:0/2579983069 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59403f030 con 0x7ff5a81045f0 2024-05-14T17:11:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.450+0000 7ff5a5ffb640 1 -- 172.21.15.2:0/2579983069 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff594003aa0 con 0x7ff5a81045f0 2024-05-14T17:11:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.450+0000 7ff5af1da640 1 -- 172.21.15.2:0/2579983069 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5a819c200 con 0x7ff5a81045f0 2024-05-14T17:11:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.450+0000 7ff5a5ffb640 1 -- 172.21.15.2:0/2579983069 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff594035070 con 0x7ff5a81045f0 2024-05-14T17:11:22.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.451+0000 7ff5af1da640 1 -- 172.21.15.2:0/2579983069 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5a81a3af0 con 0x7ff5a81045f0 2024-05-14T17:11:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.455+0000 7ff5a5ffb640 1 -- 172.21.15.2:0/2579983069 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7ff594042030 con 0x7ff5a81045f0 2024-05-14T17:11:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.456+0000 7ff5b1fcb740 1 Processor -- start 2024-05-14T17:11:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.456+0000 7ff5b1fcb740 1 -- start start 2024-05-14T17:11:22.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.456+0000 7ff5b1fcb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4af8c8f0 0x558b4af8ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.456+0000 7ff5b1fcb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b4af8d210 0x558b4af97590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.456+0000 7ff5b1fcb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b4af97ad0 0x558b4af99ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.456+0000 7ff5b1fcb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x558b4aeffc30 con 0x558b4af8d210 2024-05-14T17:11:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.456+0000 7ff5b1fcb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558b4aeef7d0 con 0x558b4af97ad0 2024-05-14T17:11:22.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.456+0000 7ff5b1fcb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558b4aef3810 con 0x558b4af8c8f0 2024-05-14T17:11:22.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.456+0000 7ff5acf76640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b4af8d210 0x558b4af97590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:22.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.456+0000 7ff5a7fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b4af97ad0 0x558b4af99ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.456+0000 7ff5acf76640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b4af8d210 0x558b4af97590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53504/0 (socket says 172.21.15.2:53504) 2024-05-14T17:11:22.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.456+0000 7ff5acf76640 1 -- 172.21.15.2:0/2065117844 learned_addr learned my addr 172.21.15.2:0/2065117844 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:11:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.456+0000 7ff5ad777640 1 --2- 172.21.15.2:0/2065117844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4af8c8f0 0x558b4af8ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.456+0000 7ff5acf76640 1 -- 172.21.15.2:0/2065117844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4af8c8f0 msgr2=0x558b4af8ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.456+0000 7ff5acf76640 1 --2- 172.21.15.2:0/2065117844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4af8c8f0 0x558b4af8ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.457+0000 7ff5acf76640 1 -- 172.21.15.2:0/2065117844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b4af97ad0 msgr2=0x558b4af99ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.457+0000 7ff5acf76640 1 --2- 172.21.15.2:0/2065117844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b4af97ad0 0x558b4af99ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.457+0000 7ff5acf76640 1 -- 172.21.15.2:0/2065117844 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b4ac04210 con 0x558b4af8d210 2024-05-14T17:11:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.457+0000 7ff5ad777640 1 --2- 172.21.15.2:0/2065117844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4af8c8f0 0x558b4af8ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:11:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.457+0000 7ff5a7fff640 1 --2- 172.21.15.2:0/2065117844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b4af97ad0 0x558b4af99ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:11:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.457+0000 7ff5a5ffb640 1 --2- 172.21.15.2:0/2579983069 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ff5980b6e20 0x7ff5980b92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.457+0000 7ff5ad777640 1 --2- 172.21.15.2:0/2579983069 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ff5980b6e20 0x7ff5980b92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.457+0000 7ff5a5ffb640 1 -- 172.21.15.2:0/2579983069 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7ff594102c70 con 0x7ff5a81045f0 2024-05-14T17:11:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.457+0000 7ff5acf76640 1 --2- 172.21.15.2:0/2065117844 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b4af8d210 0x558b4af97590 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ff59c009cf0 tx=0x7ff59c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7423b39fd7729a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.457+0000 7ff592ffd640 1 -- 172.21.15.2:0/2065117844 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c004230 con 0x558b4af8d210 2024-05-14T17:11:23.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.457+0000 7ff592ffd640 1 -- 172.21.15.2:0/2065117844 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff59c0043d0 con 0x558b4af8d210 2024-05-14T17:11:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.457+0000 7ff592ffd640 1 -- 172.21.15.2:0/2065117844 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c019880 con 0x558b4af8d210 2024-05-14T17:11:23.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.458+0000 7ff5b1fcb740 1 -- 172.21.15.2:0/2065117844 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b4af8d210 msgr2=0x558b4af97590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.458+0000 7ff5ad777640 1 --2- 172.21.15.2:0/2579983069 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ff5980b6e20 0x7ff5980b92e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0005fd0 tx=0x7ff5a0009600 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.458+0000 7ff5b1fcb740 1 --2- 172.21.15.2:0/2065117844 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b4af8d210 0x558b4af97590 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ff59c009cf0 tx=0x7ff59c011810 comp rx=0 tx=0).stop 2024-05-14T17:11:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.458+0000 7ff5b1fcb740 1 -- 172.21.15.2:0/2065117844 shutdown_connections 2024-05-14T17:11:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.458+0000 7ff5b1fcb740 1 --2- 172.21.15.2:0/2065117844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b4af97ad0 0x558b4af99ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.458+0000 7ff5b1fcb740 1 --2- 172.21.15.2:0/2065117844 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b4af8d210 0x558b4af97590 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:23.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.458+0000 7ff5b1fcb740 1 --2- 172.21.15.2:0/2065117844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4af8c8f0 0x558b4af8ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:23.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.458+0000 7ff5b1fcb740 1 -- 172.21.15.2:0/2065117844 >> 172.21.15.2:0/2065117844 conn(0x558b4af88e00 msgr2=0x558b4af891e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.458+0000 7ff5b1fcb740 1 -- 172.21.15.2:0/2065117844 shutdown_connections 2024-05-14T17:11:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.458+0000 7ff5b1fcb740 1 -- 172.21.15.2:0/2065117844 wait complete. 2024-05-14T17:11:23.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.459+0000 7ff5b1fcb740 1 -- start start 2024-05-14T17:11:23.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.459+0000 7ff5b1fcb740 1 Processor -- start 2024-05-14T17:11:23.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.460+0000 7ff5b1fcb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4af8c8f0 0x558b4af97450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:23.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.460+0000 7ff5b1fcb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b4af8d210 0x558b4af94740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:23.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.460+0000 7ff5b1fcb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b4af97ad0 0x558b4af94cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:23.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.460+0000 7ff5b1fcb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x558b4abd5790 con 0x558b4af8d210 2024-05-14T17:11:23.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.460+0000 7ff5a7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b4af8d210 0x558b4af94740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:23.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.460+0000 7ff5b1fcb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558b4aeab890 con 0x558b4af97ad0 2024-05-14T17:11:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.460+0000 7ff5acf76640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4af8c8f0 0x558b4af97450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.460+0000 7ff5a7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b4af8d210 0x558b4af94740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53508/0 (socket says 172.21.15.2:53508) 2024-05-14T17:11:24.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.460+0000 7ff5a7fff640 1 -- 172.21.15.2:0/2900783696 learned_addr learned my addr 172.21.15.2:0/2900783696 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:11:24.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.460+0000 7ff5b1fcb740 1 -- 172.21.15.2:0/2900783696 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558b4af9a7a0 con 0x558b4af8c8f0 2024-05-14T17:11:24.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.460+0000 7ff5ad777640 1 --2- 172.21.15.2:0/2900783696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b4af97ad0 0x558b4af94cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:24.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.460+0000 7ff5b1fcb740 10 client.0 _ll_register_callbacks cb 0x558b4aeaed60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:24.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.460+0000 7ff5a7fff640 1 -- 172.21.15.2:0/2900783696 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4af8c8f0 msgr2=0x558b4af97450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.460+0000 7ff5a7fff640 1 --2- 172.21.15.2:0/2900783696 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4af8c8f0 0x558b4af97450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:24.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.460+0000 7ff5a7fff640 1 -- 172.21.15.2:0/2900783696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b4af97ad0 msgr2=0x558b4af94cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:24.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.460+0000 7ff5a7fff640 1 --2- 172.21.15.2:0/2900783696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b4af97ad0 0x558b4af94cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:24.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.460+0000 7ff5a7fff640 1 -- 172.21.15.2:0/2900783696 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b4acd0620 con 0x558b4af8d210 2024-05-14T17:11:24.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.461+0000 7ff5a7fff640 1 --2- 172.21.15.2:0/2900783696 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b4af8d210 0x558b4af94740 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff5940c7270 tx=0x7ff594005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42dfb287cd308ce server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.461+0000 7ff587fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:11:24.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.461+0000 7ff587fff640 1 -- 172.21.15.2:0/2900783696 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff594049070 con 0x558b4af8d210 2024-05-14T17:11:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.461+0000 7ff587fff640 1 -- 172.21.15.2:0/2900783696 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7ff59403bcc0 con 0x558b4af8d210 2024-05-14T17:11:24.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.461+0000 7ff5b1fcb740 1 -- 172.21.15.2:0/2900783696 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff59c0043d0 con 0x558b4af8d210 2024-05-14T17:11:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.461+0000 7ff587fff640 1 -- 172.21.15.2:0/2900783696 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7ff5940c5ec0 con 0x558b4af8d210 2024-05-14T17:11:24.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.461+0000 7ff5b1fcb740 10 client.16842 fetch_fsmap learned FSMap version 51 2024-05-14T17:11:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.461+0000 7ff5b1fcb740 1 -- 172.21.15.2:0/2900783696 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x558b4afae9d0 con 0x558b4af8d210 2024-05-14T17:11:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.461+0000 7ff587fff640 1 -- 172.21.15.2:0/2900783696 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5940c9b10 con 0x558b4af8d210 2024-05-14T17:11:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.461+0000 7ff587fff640 1 -- 172.21.15.2:0/2900783696 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 51) ==== 8354+0+0 (secure 0 0 0) 0x7ff5940d2920 con 0x558b4af8d210 2024-05-14T17:11:24.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.461+0000 7ff5b1fcb740 10 client.16842 fetch_fsmap finished waiting for FSMap version 51 2024-05-14T17:11:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.461+0000 7ff5b1fcb740 10 client.16842 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:11:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.461+0000 7ff5b1fcb740 20 client.16842 populate_metadata read hostname 'smithi002' 2024-05-14T17:11:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.462+0000 7ff5b1fcb740 1 --2- 172.21.15.2:0/2900783696 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x558b4afaff00 0x558b4afb2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.462+0000 7ff5b1fcb740 4 client.16842 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:11:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.462+0000 7ff5b1fcb740 1 -- 172.21.15.2:0/2900783696 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff59c0043d0 con 0x558b4afaff00 2024-05-14T17:11:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.462+0000 7ff5acf76640 1 --2- 172.21.15.2:0/2900783696 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x558b4afaff00 0x558b4afb2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:24.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.462+0000 7ff587fff640 10 client.16842 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:11:25.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.462+0000 7ff5acf76640 1 --2- 172.21.15.2:0/2900783696 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x558b4afaff00 0x558b4afb2380 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54d17dc7b7f0254 server_cookie=64cd667abed1fe8d in_seq=0 out_seq=0 2024-05-14T17:11:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.463+0000 7ff587fff640 1 -- 172.21.15.2:0/2900783696 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7ff59c0043d0 con 0x558b4afaff00 2024-05-14T17:11:25.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.463+0000 7ff587fff640 10 client.16842 handle_command_reply: tid=0 2024-05-14T17:11:25.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.463+0000 7ff5b1fcb740 1 client.16842 shutdown 2024-05-14T17:11:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.463+0000 7ff5b1fcb740 1 -- 172.21.15.2:0/2900783696 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b4af8d210 msgr2=0x558b4af94740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.463+0000 7ff5b1fcb740 1 --2- 172.21.15.2:0/2900783696 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b4af8d210 0x558b4af94740 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff5940c7270 tx=0x7ff594005c90 comp rx=0 tx=0).stop 2024-05-14T17:11:25.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.463+0000 7ff5b1fcb740 1 -- 172.21.15.2:0/2900783696 shutdown_connections 2024-05-14T17:11:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.463+0000 7ff5b1fcb740 1 --2- 172.21.15.2:0/2900783696 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x558b4afaff00 0x558b4afb2380 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.463+0000 7ff5b1fcb740 1 --2- 172.21.15.2:0/2900783696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b4af97ad0 0x558b4af94cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:25.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.463+0000 7ff5b1fcb740 1 --2- 172.21.15.2:0/2900783696 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b4af8d210 0x558b4af94740 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:25.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.463+0000 7ff5b1fcb740 1 --2- 172.21.15.2:0/2900783696 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4af8c8f0 0x558b4af97450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.463+0000 7ff5b1fcb740 1 -- 172.21.15.2:0/2900783696 >> 172.21.15.2:0/2900783696 conn(0x558b4af88e00 msgr2=0x558b4af88100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.463+0000 7ff587fff640 0 client.16842 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:11:25.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.463+0000 7ff5b1fcb740 1 -- 172.21.15.2:0/2900783696 shutdown_connections 2024-05-14T17:11:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.463+0000 7ff5b1fcb740 1 -- 172.21.15.2:0/2900783696 wait complete. 2024-05-14T17:11:25.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.464+0000 7ff5b1fcb740 20 client.16842 trim_cache size 0 max 16384 2024-05-14T17:11:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.486+0000 7ff5b1fcb740 1 Processor -- start 2024-05-14T17:11:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.486+0000 7ff5b1fcb740 1 -- start start 2024-05-14T17:11:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.486+0000 7ff5b1fcb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b4af8c8f0 0x558b4afb7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.487+0000 7ff5b1fcb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4af8d210 0x558b4afb8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.487+0000 7ff5b1fcb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b4af97ad0 0x558b4af919c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:25.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.487+0000 7ff5b1fcb740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a81aed40 con 0x558b4af97ad0 2024-05-14T17:11:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.487+0000 7ff5b1fcb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a81ac5b0 con 0x558b4af8c8f0 2024-05-14T17:11:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.487+0000 7ff5b1fcb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a81aa420 con 0x558b4af8d210 2024-05-14T17:11:25.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.487+0000 7ff5ad777640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b4af97ad0 0x558b4af919c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.487+0000 7ff5ad777640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b4af97ad0 0x558b4af919c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53510/0 (socket says 172.21.15.2:53510) 2024-05-14T17:11:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.487+0000 7ff5ad777640 1 -- 172.21.15.2:0/1456264704 learned_addr learned my addr 172.21.15.2:0/1456264704 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:11:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.487+0000 7ff5a7fff640 1 --2- 172.21.15.2:0/1456264704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4af8d210 0x558b4afb8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.487+0000 7ff5ad777640 1 -- 172.21.15.2:0/1456264704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4af8d210 msgr2=0x558b4afb8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.487+0000 7ff5ad777640 1 --2- 172.21.15.2:0/1456264704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4af8d210 0x558b4afb8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.487+0000 7ff5ad777640 1 -- 172.21.15.2:0/1456264704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b4af8c8f0 msgr2=0x558b4afb7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.487+0000 7ff5ad777640 1 --2- 172.21.15.2:0/1456264704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b4af8c8f0 0x558b4afb7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.487+0000 7ff5ad777640 1 -- 172.21.15.2:0/1456264704 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b4af88c30 con 0x558b4af97ad0 2024-05-14T17:11:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.487+0000 7ff5ad777640 1 --2- 172.21.15.2:0/1456264704 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b4af97ad0 0x558b4af919c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0013e30 tx=0x7ff5a000b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a925d2df434e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:26.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.488+0000 7ff5927fc640 1 -- 172.21.15.2:0/1456264704 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a002c070 con 0x558b4af97ad0 2024-05-14T17:11:26.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.488+0000 7ff5927fc640 1 -- 172.21.15.2:0/1456264704 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5a001c910 con 0x558b4af97ad0 2024-05-14T17:11:26.396 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.488+0000 7ff5927fc640 1 -- 172.21.15.2:0/1456264704 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a00277e0 con 0x558b4af97ad0 2024-05-14T17:11:26.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.489+0000 7ff5b1fcb740 1 -- 172.21.15.2:0/1456264704 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b4af97ad0 msgr2=0x558b4af919c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:26.488 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.489+0000 7ff5b1fcb740 1 --2- 172.21.15.2:0/1456264704 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b4af97ad0 0x558b4af919c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0013e30 tx=0x7ff5a000b000 comp rx=0 tx=0).stop 2024-05-14T17:11:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.489+0000 7ff5b1fcb740 1 -- 172.21.15.2:0/1456264704 shutdown_connections 2024-05-14T17:11:26.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.489+0000 7ff5b1fcb740 1 --2- 172.21.15.2:0/1456264704 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b4af97ad0 0x558b4af919c0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.489+0000 7ff5b1fcb740 1 --2- 172.21.15.2:0/1456264704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4af8d210 0x558b4afb8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:26.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.489+0000 7ff5b1fcb740 1 --2- 172.21.15.2:0/1456264704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b4af8c8f0 0x558b4afb7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.489+0000 7ff5b1fcb740 1 -- 172.21.15.2:0/1456264704 >> 172.21.15.2:0/1456264704 conn(0x558b4af88e00 msgr2=0x558b4afaaeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:26.788 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.489+0000 7ff5b1fcb740 1 -- 172.21.15.2:0/1456264704 shutdown_connections 2024-05-14T17:11:26.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.489+0000 7ff5b1fcb740 1 -- 172.21.15.2:0/1456264704 wait complete. 2024-05-14T17:11:26.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.489+0000 7ff5b1fcb740 1 -- start start 2024-05-14T17:11:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.489+0000 7ff5b1fcb740 1 Processor -- start 2024-05-14T17:11:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.491+0000 7ff5b1fcb740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b4af8c8f0 0x558b4afa8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.491+0000 7ff5b1fcb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4af8d210 0x558b4afa8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.491+0000 7ff5b1fcb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b4af97ad0 0x558b4afa5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.491+0000 7ff5a7fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4af8d210 0x558b4afa8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.491+0000 7ff5a7fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4af8d210 0x558b4afa8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:53848/0 (socket says 172.21.15.2:53848) 2024-05-14T17:11:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.491+0000 7ff5a7fff640 1 -- 172.21.15.2:0/1289260562 learned_addr learned my addr 172.21.15.2:0/1289260562 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:11:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.492+0000 7ff5b1fcb740 1 -- 172.21.15.2:0/1289260562 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a81aaa70 con 0x558b4af8c8f0 2024-05-14T17:11:27.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.492+0000 7ff5b1fcb740 1 -- 172.21.15.2:0/1289260562 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a81abee0 con 0x558b4af97ad0 2024-05-14T17:11:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.492+0000 7ff5b1fcb740 1 -- 172.21.15.2:0/1289260562 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a81ab140 con 0x558b4af8d210 2024-05-14T17:11:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.492+0000 7ff5b1fcb740 10 client.0 _ll_register_callbacks cb 0x7ff5941064b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.492+0000 7ff5a7fff640 1 -- 172.21.15.2:0/1289260562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b4af97ad0 msgr2=0x558b4afa5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.492+0000 7ff5a7fff640 1 --2- 172.21.15.2:0/1289260562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b4af97ad0 0x558b4afa5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.492+0000 7ff5a7fff640 1 -- 172.21.15.2:0/1289260562 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b4af8c8f0 msgr2=0x558b4afa8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:27.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.492+0000 7ff5a7fff640 1 --2- 172.21.15.2:0/1289260562 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b4af8c8f0 0x558b4afa8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.492+0000 7ff5a7fff640 1 -- 172.21.15.2:0/1289260562 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b4afa6100 con 0x558b4af8d210 2024-05-14T17:11:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.492+0000 7ff5b1fcb740 1 -- 172.21.15.2:0/1289260562 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5a001c910 con 0x558b4af8d210 2024-05-14T17:11:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.492+0000 7ff5a7fff640 1 --2- 172.21.15.2:0/1289260562 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4af8d210 0x558b4afa8870 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff594032af0 tx=0x7ff594003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21b92432fd0b35a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:27.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.492+0000 7ff587fff640 10 client.26455 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-14T17:11:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.493+0000 7ff587fff640 1 -- 172.21.15.2:0/1289260562 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5940e8020 con 0x558b4af8d210 2024-05-14T17:11:27.664 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.493+0000 7ff587fff640 1 -- 172.21.15.2:0/1289260562 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7ff594100d20 con 0x558b4af8d210 2024-05-14T17:11:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.493+0000 7ff587fff640 1 -- 172.21.15.2:0/1289260562 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7ff594049070 con 0x558b4af8d210 2024-05-14T17:11:27.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.493+0000 7ff5b1fcb740 10 client.26455 fetch_fsmap learned FSMap version 51 2024-05-14T17:11:27.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.493+0000 7ff5b1fcb740 1 -- 172.21.15.2:0/1289260562 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({fsmap=51}) -- 0x558b4b01a2b0 con 0x558b4af8d210 2024-05-14T17:11:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.494+0000 7ff587fff640 1 -- 172.21.15.2:0/1289260562 <== mon.1 v2:172.21.15.151:3300/0 4 ==== fsmap(e 51) ==== 8354+0+0 (secure 0 0 0) 0x7ff5940c5970 con 0x558b4af8d210 2024-05-14T17:11:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.494+0000 7ff5b1fcb740 10 client.26455 fetch_fsmap finished waiting for FSMap version 51 2024-05-14T17:11:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.494+0000 7ff5b1fcb740 10 client.26455 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:11:27.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.494+0000 7ff5b1fcb740 20 client.26455 populate_metadata read hostname 'smithi002' 2024-05-14T17:11:28.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.494+0000 7ff5b1fcb740 1 --2- 172.21.15.2:0/1289260562 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x558b4b01a730 0x558b4b01cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:28.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.494+0000 7ff5b1fcb740 4 client.26455 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:11:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.494+0000 7ff5b1fcb740 1 -- 172.21.15.2:0/1289260562 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558b4af01b80 con 0x558b4b01a730 2024-05-14T17:11:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.494+0000 7ff5acf76640 1 --2- 172.21.15.2:0/1289260562 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x558b4b01a730 0x558b4b01cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.494+0000 7ff587fff640 10 client.26455 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:11:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.494+0000 7ff5acf76640 1 --2- 172.21.15.2:0/1289260562 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x558b4b01a730 0x558b4b01cb80 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5551cfd38738518 server_cookie=8c32ac0d74b2ffc1 in_seq=0 out_seq=0 2024-05-14T17:11:28.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.510+0000 7ff587fff640 1 -- 172.21.15.2:0/1289260562 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558b4af01b80 con 0x558b4b01a730 2024-05-14T17:11:28.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.510+0000 7ff587fff640 10 client.26455 handle_command_reply: tid=0 2024-05-14T17:11:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.510+0000 7ff5b1fcb740 1 client.26455 shutdown 2024-05-14T17:11:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.512+0000 7ff5b1fcb740 1 -- 172.21.15.2:0/1289260562 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4af8d210 msgr2=0x558b4afa8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:28.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.512+0000 7ff5b1fcb740 1 --2- 172.21.15.2:0/1289260562 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4af8d210 0x558b4afa8870 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff594032af0 tx=0x7ff594003820 comp rx=0 tx=0).stop 2024-05-14T17:11:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.512+0000 7ff5b1fcb740 1 -- 172.21.15.2:0/1289260562 shutdown_connections 2024-05-14T17:11:28.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.512+0000 7ff5b1fcb740 1 --2- 172.21.15.2:0/1289260562 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x558b4b01a730 0x558b4b01cb80 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.512+0000 7ff5b1fcb740 1 --2- 172.21.15.2:0/1289260562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558b4af97ad0 0x558b4afa5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:28.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.512+0000 7ff5b1fcb740 1 --2- 172.21.15.2:0/1289260562 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558b4af8d210 0x558b4afa8870 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:28.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.512+0000 7ff5b1fcb740 1 --2- 172.21.15.2:0/1289260562 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b4af8c8f0 0x558b4afa8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.512+0000 7ff5b1fcb740 1 -- 172.21.15.2:0/1289260562 >> 172.21.15.2:0/1289260562 conn(0x558b4af88e00 msgr2=0x558b4afab740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:28.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.512+0000 7ff587fff640 0 client.26455 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:11:28.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.512+0000 7ff5b1fcb740 1 -- 172.21.15.2:0/1289260562 shutdown_connections 2024-05-14T17:11:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.512+0000 7ff5b1fcb740 1 -- 172.21.15.2:0/1289260562 wait complete. 2024-05-14T17:11:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.512+0000 7ff5b1fcb740 20 client.26455 trim_cache size 0 max 16384 2024-05-14T17:11:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:11:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:11:28.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:11:28.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.514+0000 7ff5af1da640 1 -- 172.21.15.2:0/2579983069 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ff5980b6e20 msgr2=0x7ff5980b92e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:28.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.514+0000 7ff5af1da640 1 --2- 172.21.15.2:0/2579983069 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ff5980b6e20 0x7ff5980b92e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0005fd0 tx=0x7ff5a0009600 comp rx=0 tx=0).stop 2024-05-14T17:11:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.514+0000 7ff5af1da640 1 -- 172.21.15.2:0/2579983069 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5a81045f0 msgr2=0x7ff5a8197510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.514+0000 7ff5af1da640 1 --2- 172.21.15.2:0/2579983069 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5a81045f0 0x7ff5a8197510 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff594005940 tx=0x7ff594004270 comp rx=0 tx=0).stop 2024-05-14T17:11:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.514+0000 7ff5af1da640 1 -- 172.21.15.2:0/2579983069 shutdown_connections 2024-05-14T17:11:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.514+0000 7ff5af1da640 1 --2- 172.21.15.2:0/2579983069 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ff5980b6e20 0x7ff5980b92e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.514+0000 7ff5af1da640 1 --2- 172.21.15.2:0/2579983069 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff5a810d0a0 0x7ff5a819b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.514+0000 7ff5af1da640 1 --2- 172.21.15.2:0/2579983069 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5a81045f0 0x7ff5a8197510 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.514+0000 7ff5af1da640 1 --2- 172.21.15.2:0/2579983069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5a8103cd0 0x7ff5a8196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:29.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.514+0000 7ff5af1da640 1 -- 172.21.15.2:0/2579983069 >> 172.21.15.2:0/2579983069 conn(0x7ff5a8078aa0 msgr2=0x7ff5a80fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:29.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.514+0000 7ff5af1da640 1 -- 172.21.15.2:0/2579983069 shutdown_connections 2024-05-14T17:11:29.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:45.514+0000 7ff5af1da640 1 -- 172.21.15.2:0/2579983069 wait complete. 2024-05-14T17:11:29.382 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 870.0 s, 2654.0 tps, lat 37.408 ms stddev 47.315 2024-05-14T17:11:29.415 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:09:46 smithi002 ceph-mon[31538]: pgmap v471: 97 pgs: 97 active+clean; 5.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.8 MiB/s wr, 662 op/s 2024-05-14T17:11:29.448 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:09:48 smithi002 ceph-mon[31538]: pgmap v472: 97 pgs: 97 active+clean; 5.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.6 MiB/s wr, 635 op/s 2024-05-14T17:11:29.482 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:09:50 smithi002 ceph-mon[31538]: pgmap v473: 97 pgs: 97 active+clean; 5.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 12 MiB/s wr, 877 op/s 2024-05-14T17:11:29.515 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 875.0 s, 1706.4 tps, lat 58.899 ms stddev 64.724 2024-05-14T17:11:29.549 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:09:52 smithi002 ceph-mon[31538]: pgmap v474: 97 pgs: 97 active+clean; 5.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.7 MiB/s wr, 648 op/s 2024-05-14T17:11:29.582 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:09:54 smithi002 ceph-mon[31538]: pgmap v475: 97 pgs: 97 active+clean; 5.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 11 MiB/s wr, 841 op/s 2024-05-14T17:11:29.624 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 880.0 s, 2244.6 tps, lat 44.776 ms stddev 54.612 2024-05-14T17:11:29.658 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:09:56 smithi002 ceph-mon[31538]: pgmap v476: 97 pgs: 97 active+clean; 5.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.5 MiB/s wr, 584 op/s 2024-05-14T17:11:29.690 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:09:58 smithi002 ceph-mon[31538]: pgmap v477: 97 pgs: 97 active+clean; 5.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.1 MiB/s wr, 579 op/s 2024-05-14T17:11:29.725 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:11:20 smithi123 ceph-mon[35971]: pgmap v522: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 7.8 MiB/s wr, 774 op/s 2024-05-14T17:11:29.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:11:22 smithi123 ceph-mon[35971]: pgmap v523: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 7.8 MiB/s wr, 774 op/s 2024-05-14T17:11:29.790 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:11:24 smithi123 ceph-mon[35971]: pgmap v524: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.9 MiB/s wr, 691 op/s 2024-05-14T17:11:29.848 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:11:26 smithi123 ceph-mon[35971]: pgmap v525: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 846 KiB/s wr, 88 op/s 2024-05-14T17:11:29.891 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:11:28 smithi123 ceph-mon[35971]: pgmap v526: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 44 KiB/s wr, 9 op/s 2024-05-14T17:11:29.924 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:11:20 smithi151 ceph-mon[36066]: pgmap v522: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 7.8 MiB/s wr, 774 op/s 2024-05-14T17:11:29.957 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:11:22 smithi151 ceph-mon[36066]: pgmap v523: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 7.8 MiB/s wr, 774 op/s 2024-05-14T17:11:29.990 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:11:24 smithi151 ceph-mon[36066]: pgmap v524: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.9 MiB/s wr, 691 op/s 2024-05-14T17:11:30.024 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:11:26 smithi151 ceph-mon[36066]: pgmap v525: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 846 KiB/s wr, 88 op/s 2024-05-14T17:11:30.057 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:11:28 smithi151 ceph-mon[36066]: pgmap v526: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 44 KiB/s wr, 9 op/s 2024-05-14T17:11:30.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.650+0000 7f587ea72640 1 -- 172.21.15.2:0/968122520 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5878101980 msgr2=0x7f5878101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.650+0000 7f587ea72640 1 --2- 172.21.15.2:0/968122520 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5878101980 0x7f5878101e00 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f5870009cd0 tx=0x7f587002a8e0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.650+0000 7f587ea72640 1 -- 172.21.15.2:0/968122520 shutdown_connections 2024-05-14T17:11:30.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.650+0000 7f587ea72640 1 --2- 172.21.15.2:0/968122520 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5878102340 0x7f587810e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.650+0000 7f587ea72640 1 --2- 172.21.15.2:0/968122520 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5878101980 0x7f5878101e00 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.650+0000 7f587ea72640 1 --2- 172.21.15.2:0/968122520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5878107940 0x7f5878107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.650+0000 7f587ea72640 1 -- 172.21.15.2:0/968122520 >> 172.21.15.2:0/968122520 conn(0x7f5878078ad0 msgr2=0x7f5878078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.650+0000 7f587ea72640 1 -- 172.21.15.2:0/968122520 shutdown_connections 2024-05-14T17:11:30.382 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.650+0000 7f587ea72640 1 -- 172.21.15.2:0/968122520 wait complete. 2024-05-14T17:11:30.416 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.651+0000 7f587ea72640 1 Processor -- start 2024-05-14T17:11:30.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.651+0000 7f587ea72640 1 -- start start 2024-05-14T17:11:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.651+0000 7f587ea72640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5878101980 0x7f5878197d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.652+0000 7f587ea72640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5878102340 0x7f58781982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.652+0000 7f587ea72640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5878107940 0x7f5878198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.652+0000 7f587ea72640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5878111c50 con 0x7f5878107940 2024-05-14T17:11:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.652+0000 7f587ea72640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5878111ad0 con 0x7f5878102340 2024-05-14T17:11:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.652+0000 7f587ea72640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5878111dd0 con 0x7f5878101980 2024-05-14T17:11:30.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.652+0000 7f587d00f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5878107940 0x7f5878198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.652+0000 7f5877fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5878102340 0x7f58781982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:30.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.652+0000 7f587d00f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5878107940 0x7f5878198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32792/0 (socket says 172.21.15.2:32792) 2024-05-14T17:11:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.652+0000 7f587d00f640 1 -- 172.21.15.2:0/724150732 learned_addr learned my addr 172.21.15.2:0/724150732 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:11:30.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.652+0000 7f587c80e640 1 --2- 172.21.15.2:0/724150732 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5878101980 0x7f5878197d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.652+0000 7f5877fff640 1 -- 172.21.15.2:0/724150732 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5878101980 msgr2=0x7f5878197d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:30.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.652+0000 7f5877fff640 1 --2- 172.21.15.2:0/724150732 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5878101980 0x7f5878197d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.652+0000 7f5877fff640 1 -- 172.21.15.2:0/724150732 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5878107940 msgr2=0x7f5878198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.652+0000 7f5877fff640 1 --2- 172.21.15.2:0/724150732 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5878107940 0x7f5878198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:31.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.652+0000 7f5877fff640 1 -- 172.21.15.2:0/724150732 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58781a75f0 con 0x7f5878102340 2024-05-14T17:11:31.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.653+0000 7f5877fff640 1 --2- 172.21.15.2:0/724150732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5878102340 0x7f58781982c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5870000f80 tx=0x7f5870003970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.653+0000 7f5875ffb640 1 -- 172.21.15.2:0/724150732 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5870004040 con 0x7f5878102340 2024-05-14T17:11:31.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.653+0000 7f5875ffb640 1 -- 172.21.15.2:0/724150732 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5870004690 con 0x7f5878102340 2024-05-14T17:11:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.653+0000 7f587ea72640 1 -- 172.21.15.2:0/724150732 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58781a7880 con 0x7f5878102340 2024-05-14T17:11:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.653+0000 7f5875ffb640 1 -- 172.21.15.2:0/724150732 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5870030e70 con 0x7f5878102340 2024-05-14T17:11:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.653+0000 7f587ea72640 1 -- 172.21.15.2:0/724150732 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58781a7dd0 con 0x7f5878102340 2024-05-14T17:11:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.657+0000 7f588178e740 1 Processor -- start 2024-05-14T17:11:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.657+0000 7f588178e740 1 -- start start 2024-05-14T17:11:31.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.657+0000 7f588178e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644740d58f0 0x5644740d5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.657+0000 7f588178e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644740d6210 0x5644740f15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.657+0000 7f588178e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644740f1ae0 0x5644740f3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.657+0000 7f588178e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x564474048c30 con 0x5644740d6210 2024-05-14T17:11:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.657+0000 7f588178e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5644740387d0 con 0x5644740f1ae0 2024-05-14T17:11:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.657+0000 7f588178e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56447403c810 con 0x5644740d58f0 2024-05-14T17:11:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.658+0000 7f587d00f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644740d58f0 0x5644740d5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.658+0000 7f587d00f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644740d58f0 0x5644740d5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:45194/0 (socket says 172.21.15.2:45194) 2024-05-14T17:11:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.658+0000 7f587d00f640 1 -- 172.21.15.2:0/1988483593 learned_addr learned my addr 172.21.15.2:0/1988483593 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:11:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.658+0000 7f587d00f640 1 -- 172.21.15.2:0/1988483593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644740f1ae0 msgr2=0x5644740f3ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T17:11:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.658+0000 7f587d00f640 1 --2- 172.21.15.2:0/1988483593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644740f1ae0 0x5644740f3ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.658+0000 7f587d00f640 1 -- 172.21.15.2:0/1988483593 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644740d6210 msgr2=0x5644740f15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.658+0000 7f587d00f640 1 --2- 172.21.15.2:0/1988483593 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644740d6210 0x5644740f15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.658+0000 7f587d00f640 1 -- 172.21.15.2:0/1988483593 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564473d4d210 con 0x5644740d58f0 2024-05-14T17:11:31.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.658+0000 7f587d00f640 1 --2- 172.21.15.2:0/1988483593 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644740d58f0 0x5644740d5cd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f586c009d20 tx=0x7f586c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ababfbae21838794 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.658+0000 7f585effd640 1 -- 172.21.15.2:0/1988483593 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586c027070 con 0x5644740d58f0 2024-05-14T17:11:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.659+0000 7f585effd640 1 -- 172.21.15.2:0/1988483593 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f586c022440 con 0x5644740d58f0 2024-05-14T17:11:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.659+0000 7f585effd640 1 -- 172.21.15.2:0/1988483593 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586c00f600 con 0x5644740d58f0 2024-05-14T17:11:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.659+0000 7f5875ffb640 1 -- 172.21.15.2:0/724150732 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f58700305a0 con 0x7f5878102340 2024-05-14T17:11:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.660+0000 7f588178e740 1 -- 172.21.15.2:0/1988483593 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644740d58f0 msgr2=0x5644740d5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.660+0000 7f588178e740 1 --2- 172.21.15.2:0/1988483593 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644740d58f0 0x5644740d5cd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f586c009d20 tx=0x7f586c011810 comp rx=0 tx=0).stop 2024-05-14T17:11:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.660+0000 7f588178e740 1 -- 172.21.15.2:0/1988483593 shutdown_connections 2024-05-14T17:11:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.660+0000 7f588178e740 1 --2- 172.21.15.2:0/1988483593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644740f1ae0 0x5644740f3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.660+0000 7f588178e740 1 --2- 172.21.15.2:0/1988483593 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644740d6210 0x5644740f15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.660+0000 7f588178e740 1 --2- 172.21.15.2:0/1988483593 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644740d58f0 0x5644740d5cd0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:32.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.660+0000 7f588178e740 1 -- 172.21.15.2:0/1988483593 >> 172.21.15.2:0/1988483593 conn(0x5644740d1e00 msgr2=0x5644740d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:32.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.660+0000 7f588178e740 1 -- 172.21.15.2:0/1988483593 shutdown_connections 2024-05-14T17:11:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.660+0000 7f588178e740 1 -- 172.21.15.2:0/1988483593 wait complete. 2024-05-14T17:11:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.660+0000 7f5875ffb640 1 --2- 172.21.15.2:0/724150732 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f58600b2920 0x7f58600b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.661+0000 7f588178e740 1 -- start start 2024-05-14T17:11:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.661+0000 7f587c80e640 1 --2- 172.21.15.2:0/724150732 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f58600b2920 0x7f58600b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:32.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.661+0000 7f5875ffb640 1 -- 172.21.15.2:0/724150732 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f587002e030 con 0x7f5878102340 2024-05-14T17:11:32.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.661+0000 7f588178e740 1 Processor -- start 2024-05-14T17:11:32.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.661+0000 7f587c80e640 1 --2- 172.21.15.2:0/724150732 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f58600b2920 0x7f58600b4de0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5868004700 tx=0x7f586800a530 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:32.634 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.662+0000 7f588178e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644740d58f0 0x5644740f1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.662+0000 7f588178e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644740d6210 0x5644740ee760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:32.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.662+0000 7f588178e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644740f1ae0 0x5644740eece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:32.735 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.662+0000 7f588178e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x564473d1e790 con 0x5644740d58f0 2024-05-14T17:11:32.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.662+0000 7f588178e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564473ff4890 con 0x5644740d6210 2024-05-14T17:11:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.662+0000 7f588178e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5644740f4800 con 0x5644740f1ae0 2024-05-14T17:11:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.662+0000 7f588178e740 10 client.0 _ll_register_callbacks cb 0x564473ff7d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:32.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.662+0000 7f587c80e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644740d58f0 0x5644740f1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:32.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.662+0000 7f587c80e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644740d58f0 0x5644740f1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32810/0 (socket says 172.21.15.2:32810) 2024-05-14T17:11:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.662+0000 7f587c80e640 1 -- 172.21.15.2:0/3557615422 learned_addr learned my addr 172.21.15.2:0/3557615422 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:11:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.662+0000 7f587d00f640 1 --2- 172.21.15.2:0/3557615422 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644740f1ae0 0x5644740eece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.662+0000 7f5877fff640 1 --2- 172.21.15.2:0/3557615422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644740d6210 0x5644740ee760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.662+0000 7f587c80e640 1 -- 172.21.15.2:0/3557615422 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644740f1ae0 msgr2=0x5644740eece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.662+0000 7f587c80e640 1 --2- 172.21.15.2:0/3557615422 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644740f1ae0 0x5644740eece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:33.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.662+0000 7f587c80e640 1 -- 172.21.15.2:0/3557615422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644740d6210 msgr2=0x5644740ee760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.662+0000 7f587c80e640 1 --2- 172.21.15.2:0/3557615422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644740d6210 0x5644740ee760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:33.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.662+0000 7f587c80e640 1 -- 172.21.15.2:0/3557615422 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564473e19620 con 0x5644740d58f0 2024-05-14T17:11:33.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.662+0000 7f587d00f640 1 --2- 172.21.15.2:0/3557615422 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644740f1ae0 0x5644740eece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:11:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.662+0000 7f5877fff640 1 --2- 172.21.15.2:0/3557615422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644740d6210 0x5644740ee760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:11:33.402 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.662+0000 7f587c80e640 1 --2- 172.21.15.2:0/3557615422 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644740d58f0 0x5644740f1470 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f58680109b0 tx=0x7f586800a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f0535a761f6fcdc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.662+0000 7f5857fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:11:33.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.662+0000 7f5857fff640 1 -- 172.21.15.2:0/3557615422 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5868016070 con 0x5644740d58f0 2024-05-14T17:11:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.663+0000 7f5857fff640 1 -- 172.21.15.2:0/3557615422 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f586801bd40 con 0x5644740d58f0 2024-05-14T17:11:33.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.663+0000 7f588178e740 1 -- 172.21.15.2:0/3557615422 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f586c022440 con 0x5644740d58f0 2024-05-14T17:11:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.663+0000 7f5857fff640 1 -- 172.21.15.2:0/3557615422 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f5868027480 con 0x5644740d58f0 2024-05-14T17:11:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.663+0000 7f588178e740 10 client.16854 fetch_fsmap learned FSMap version 51 2024-05-14T17:11:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.663+0000 7f588178e740 1 -- 172.21.15.2:0/3557615422 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5644741002e0 con 0x5644740d58f0 2024-05-14T17:11:33.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.663+0000 7f5857fff640 1 -- 172.21.15.2:0/3557615422 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f586c022440 con 0x5644740d58f0 2024-05-14T17:11:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.663+0000 7f5857fff640 1 -- 172.21.15.2:0/3557615422 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 51) ==== 8354+0+0 (secure 0 0 0) 0x7f586801c5c0 con 0x5644740d58f0 2024-05-14T17:11:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.663+0000 7f588178e740 10 client.16854 fetch_fsmap finished waiting for FSMap version 51 2024-05-14T17:11:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.663+0000 7f588178e740 10 client.16854 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:11:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.663+0000 7f588178e740 20 client.16854 populate_metadata read hostname 'smithi002' 2024-05-14T17:11:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.663+0000 7f588178e740 1 --2- 172.21.15.2:0/3557615422 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5644741017f0 0x564474103be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:33.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.663+0000 7f588178e740 4 client.16854 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:11:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.663+0000 7f588178e740 1 -- 172.21.15.2:0/3557615422 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56447404ab80 con 0x5644741017f0 2024-05-14T17:11:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.664+0000 7f5877fff640 1 --2- 172.21.15.2:0/3557615422 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5644741017f0 0x564474103be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.664+0000 7f5877fff640 1 --2- 172.21.15.2:0/3557615422 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5644741017f0 0x564474103be0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faea9b32153f879d server_cookie=eca5753312d0be9e in_seq=0 out_seq=0 2024-05-14T17:11:34.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.665+0000 7f5857fff640 10 client.16854 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:11:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.665+0000 7f5857fff640 1 -- 172.21.15.2:0/3557615422 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f58700a4e50 con 0x5644741017f0 2024-05-14T17:11:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.665+0000 7f5857fff640 10 client.16854 handle_command_reply: tid=0 2024-05-14T17:11:34.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.665+0000 7f588178e740 1 client.16854 shutdown 2024-05-14T17:11:34.186 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.665+0000 7f588178e740 1 -- 172.21.15.2:0/3557615422 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644740d58f0 msgr2=0x5644740f1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.665+0000 7f588178e740 1 --2- 172.21.15.2:0/3557615422 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644740d58f0 0x5644740f1470 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f58680109b0 tx=0x7f586800a2d0 comp rx=0 tx=0).stop 2024-05-14T17:11:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.665+0000 7f588178e740 1 -- 172.21.15.2:0/3557615422 shutdown_connections 2024-05-14T17:11:34.317 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.665+0000 7f588178e740 1 --2- 172.21.15.2:0/3557615422 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5644741017f0 0x564474103be0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:34.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.665+0000 7f588178e740 1 --2- 172.21.15.2:0/3557615422 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644740f1ae0 0x5644740eece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:34.402 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.665+0000 7f5857fff640 0 client.16854 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:11:34.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.665+0000 7f588178e740 1 --2- 172.21.15.2:0/3557615422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644740d6210 0x5644740ee760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:34.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.665+0000 7f588178e740 1 --2- 172.21.15.2:0/3557615422 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644740d58f0 0x5644740f1470 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:34.503 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.665+0000 7f588178e740 1 -- 172.21.15.2:0/3557615422 >> 172.21.15.2:0/3557615422 conn(0x5644740d1e00 msgr2=0x5644740d1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.666+0000 7f588178e740 1 -- 172.21.15.2:0/3557615422 shutdown_connections 2024-05-14T17:11:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.666+0000 7f588178e740 1 -- 172.21.15.2:0/3557615422 wait complete. 2024-05-14T17:11:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.666+0000 7f588178e740 20 client.16854 trim_cache size 0 max 16384 2024-05-14T17:11:34.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.687+0000 7f588178e740 1 Processor -- start 2024-05-14T17:11:34.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.687+0000 7f588178e740 1 -- start start 2024-05-14T17:11:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.688+0000 7f588178e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644740d58f0 0x5644740ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:34.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.688+0000 7f588178e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644740d6210 0x5644740d8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.688+0000 7f588178e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644740f1ae0 0x5644740f0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.688+0000 7f588178e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5878073060 con 0x5644740d6210 2024-05-14T17:11:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.688+0000 7f5877fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644740d6210 0x5644740d8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.688+0000 7f5877fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644740d6210 0x5644740d8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32814/0 (socket says 172.21.15.2:32814) 2024-05-14T17:11:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.688+0000 7f5877fff640 1 -- 172.21.15.2:0/1914977082 learned_addr learned my addr 172.21.15.2:0/1914977082 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:11:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.688+0000 7f588178e740 1 -- 172.21.15.2:0/1914977082 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f58780708d0 con 0x5644740d58f0 2024-05-14T17:11:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.688+0000 7f588178e740 1 -- 172.21.15.2:0/1914977082 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f587806e760 con 0x5644740f1ae0 2024-05-14T17:11:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.688+0000 7f587c80e640 1 --2- 172.21.15.2:0/1914977082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644740d58f0 0x5644740ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.688+0000 7f587d00f640 1 --2- 172.21.15.2:0/1914977082 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644740f1ae0 0x5644740f0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.688+0000 7f587c80e640 1 -- 172.21.15.2:0/1914977082 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644740f1ae0 msgr2=0x5644740f0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.688+0000 7f587c80e640 1 --2- 172.21.15.2:0/1914977082 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644740f1ae0 0x5644740f0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.688+0000 7f587c80e640 1 -- 172.21.15.2:0/1914977082 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644740d6210 msgr2=0x5644740d8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:35.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.688+0000 7f587c80e640 1 --2- 172.21.15.2:0/1914977082 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644740d6210 0x5644740d8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:35.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.689+0000 7f587c80e640 1 -- 172.21.15.2:0/1914977082 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564474103f50 con 0x5644740d58f0 2024-05-14T17:11:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.689+0000 7f587c80e640 1 --2- 172.21.15.2:0/1914977082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644740d58f0 0x5644740ccf90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f586801cba0 tx=0x7f586800a860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a982ec517ec8bf20 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.689+0000 7f585e7fc640 1 -- 172.21.15.2:0/1914977082 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586803e020 con 0x5644740d58f0 2024-05-14T17:11:35.395 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.689+0000 7f585e7fc640 1 -- 172.21.15.2:0/1914977082 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f586801be00 con 0x5644740d58f0 2024-05-14T17:11:35.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.689+0000 7f585e7fc640 1 -- 172.21.15.2:0/1914977082 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5868034040 con 0x5644740d58f0 2024-05-14T17:11:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.690+0000 7f588178e740 1 -- 172.21.15.2:0/1914977082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644740d58f0 msgr2=0x5644740ccf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.690+0000 7f588178e740 1 --2- 172.21.15.2:0/1914977082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644740d58f0 0x5644740ccf90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f586801cba0 tx=0x7f586800a860 comp rx=0 tx=0).stop 2024-05-14T17:11:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.690+0000 7f588178e740 1 -- 172.21.15.2:0/1914977082 shutdown_connections 2024-05-14T17:11:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.690+0000 7f588178e740 1 --2- 172.21.15.2:0/1914977082 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644740f1ae0 0x5644740f0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:35.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.690+0000 7f588178e740 1 --2- 172.21.15.2:0/1914977082 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644740d6210 0x5644740d8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.690+0000 7f588178e740 1 --2- 172.21.15.2:0/1914977082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644740d58f0 0x5644740ccf90 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:35.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.690+0000 7f588178e740 1 -- 172.21.15.2:0/1914977082 >> 172.21.15.2:0/1914977082 conn(0x5644740d1e00 msgr2=0x5644740d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:35.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.690+0000 7f588178e740 1 -- 172.21.15.2:0/1914977082 shutdown_connections 2024-05-14T17:11:35.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.690+0000 7f588178e740 1 -- 172.21.15.2:0/1914977082 wait complete. 2024-05-14T17:11:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.690+0000 7f588178e740 1 -- start start 2024-05-14T17:11:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.691+0000 7f588178e740 1 Processor -- start 2024-05-14T17:11:35.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.691+0000 7f588178e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644740d58f0 0x5644740f9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.691+0000 7f588178e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644740d6210 0x5644740d9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:35.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.691+0000 7f588178e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644740f1ae0 0x5644740da520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.691+0000 7f588178e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f587806edb0 con 0x5644740d58f0 2024-05-14T17:11:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.691+0000 7f588178e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5878070200 con 0x5644740d6210 2024-05-14T17:11:36.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.691+0000 7f588178e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f587806f460 con 0x5644740f1ae0 2024-05-14T17:11:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.691+0000 7f588178e740 10 client.0 _ll_register_callbacks cb 0x7f5868010660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.692+0000 7f587c80e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644740d58f0 0x5644740f9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.692+0000 7f587c80e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644740d58f0 0x5644740f9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32824/0 (socket says 172.21.15.2:32824) 2024-05-14T17:11:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.692+0000 7f5877fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644740d6210 0x5644740d9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.692+0000 7f587c80e640 1 -- 172.21.15.2:0/3761268891 learned_addr learned my addr 172.21.15.2:0/3761268891 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:11:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.692+0000 7f587c80e640 1 -- 172.21.15.2:0/3761268891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644740f1ae0 msgr2=0x5644740da520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-14T17:11:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.692+0000 7f587c80e640 1 --2- 172.21.15.2:0/3761268891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644740f1ae0 0x5644740da520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.692+0000 7f587c80e640 1 -- 172.21.15.2:0/3761268891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644740d6210 msgr2=0x5644740d9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.692+0000 7f587c80e640 1 --2- 172.21.15.2:0/3761268891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644740d6210 0x5644740d9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.692+0000 7f587c80e640 1 -- 172.21.15.2:0/3761268891 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644740daa60 con 0x5644740d58f0 2024-05-14T17:11:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.693+0000 7f587c80e640 1 --2- 172.21.15.2:0/3761268891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644740d58f0 0x5644740f9c40 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f586800a5c0 tx=0x7f5868005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54528af89cfab26c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:36.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.693+0000 7f5857fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:11:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.693+0000 7f5857fff640 1 -- 172.21.15.2:0/3761268891 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586803e070 con 0x5644740d58f0 2024-05-14T17:11:36.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.693+0000 7f5857fff640 1 -- 172.21.15.2:0/3761268891 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f5868005040 con 0x5644740d58f0 2024-05-14T17:11:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.693+0000 7f588178e740 1 -- 172.21.15.2:0/3761268891 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f586801be00 con 0x5644740d58f0 2024-05-14T17:11:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.693+0000 7f5857fff640 1 -- 172.21.15.2:0/3761268891 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f5868051050 con 0x5644740d58f0 2024-05-14T17:11:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.693+0000 7f588178e740 10 client.16860 fetch_fsmap learned FSMap version 51 2024-05-14T17:11:36.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.694+0000 7f588178e740 1 -- 172.21.15.2:0/3761268891 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x56447416bdc0 con 0x5644740d58f0 2024-05-14T17:11:36.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.694+0000 7f5857fff640 1 -- 172.21.15.2:0/3761268891 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f586801be00 con 0x5644740d58f0 2024-05-14T17:11:36.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.694+0000 7f5857fff640 1 -- 172.21.15.2:0/3761268891 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 51) ==== 8354+0+0 (secure 0 0 0) 0x7f586804c030 con 0x5644740d58f0 2024-05-14T17:11:36.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.695+0000 7f588178e740 10 client.16860 fetch_fsmap finished waiting for FSMap version 51 2024-05-14T17:11:36.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.695+0000 7f588178e740 10 client.16860 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:11:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.695+0000 7f588178e740 20 client.16860 populate_metadata read hostname 'smithi002' 2024-05-14T17:11:36.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.695+0000 7f588178e740 1 --2- 172.21.15.2:0/3761268891 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5644741017f0 0x56447416e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.695+0000 7f588178e740 4 client.16860 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:11:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.695+0000 7f588178e740 1 -- 172.21.15.2:0/3761268891 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564474044360 con 0x5644741017f0 2024-05-14T17:11:36.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.695+0000 7f5877fff640 1 --2- 172.21.15.2:0/3761268891 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5644741017f0 0x56447416e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:37.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.695+0000 7f5857fff640 10 client.16860 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:11:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.695+0000 7f5877fff640 1 --2- 172.21.15.2:0/3761268891 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5644741017f0 0x56447416e540 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db0f8df548271aa8 server_cookie=f1357ee6196a2a0a in_seq=0 out_seq=0 2024-05-14T17:11:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.711+0000 7f5857fff640 1 -- 172.21.15.2:0/3761268891 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564474044360 con 0x5644741017f0 2024-05-14T17:11:37.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.711+0000 7f5857fff640 10 client.16860 handle_command_reply: tid=0 2024-05-14T17:11:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.711+0000 7f588178e740 1 client.16860 shutdown 2024-05-14T17:11:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.711+0000 7f588178e740 1 -- 172.21.15.2:0/3761268891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644740d58f0 msgr2=0x5644740f9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.711+0000 7f588178e740 1 --2- 172.21.15.2:0/3761268891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644740d58f0 0x5644740f9c40 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f586800a5c0 tx=0x7f5868005b10 comp rx=0 tx=0).stop 2024-05-14T17:11:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.711+0000 7f588178e740 1 -- 172.21.15.2:0/3761268891 shutdown_connections 2024-05-14T17:11:37.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.711+0000 7f588178e740 1 --2- 172.21.15.2:0/3761268891 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5644741017f0 0x56447416e540 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:37.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.711+0000 7f588178e740 1 --2- 172.21.15.2:0/3761268891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644740f1ae0 0x5644740da520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:37.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.711+0000 7f588178e740 1 --2- 172.21.15.2:0/3761268891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5644740d6210 0x5644740d9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:37.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.711+0000 7f588178e740 1 --2- 172.21.15.2:0/3761268891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5644740d58f0 0x5644740f9c40 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:37.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.711+0000 7f5857fff640 0 client.16860 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:11:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.711+0000 7f588178e740 1 -- 172.21.15.2:0/3761268891 >> 172.21.15.2:0/3761268891 conn(0x5644740d1e00 msgr2=0x5644740ccf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:37.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.711+0000 7f588178e740 1 -- 172.21.15.2:0/3761268891 shutdown_connections 2024-05-14T17:11:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.711+0000 7f588178e740 1 -- 172.21.15.2:0/3761268891 wait complete. 2024-05-14T17:11:37.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.711+0000 7f588178e740 20 client.16860 trim_cache size 0 max 16384 2024-05-14T17:11:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:11:37.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:11:37.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:11:37.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.712+0000 7f587ea72640 1 -- 172.21.15.2:0/724150732 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f58600b2920 msgr2=0x7f58600b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.712+0000 7f587ea72640 1 --2- 172.21.15.2:0/724150732 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f58600b2920 0x7f58600b4de0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5868004700 tx=0x7f586800a530 comp rx=0 tx=0).stop 2024-05-14T17:11:37.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.712+0000 7f587ea72640 1 -- 172.21.15.2:0/724150732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5878102340 msgr2=0x7f58781982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.712+0000 7f587ea72640 1 --2- 172.21.15.2:0/724150732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5878102340 0x7f58781982c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5870000f80 tx=0x7f5870003970 comp rx=0 tx=0).stop 2024-05-14T17:11:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.712+0000 7f587ea72640 1 -- 172.21.15.2:0/724150732 shutdown_connections 2024-05-14T17:11:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.712+0000 7f587ea72640 1 --2- 172.21.15.2:0/724150732 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f58600b2920 0x7f58600b4de0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.712+0000 7f587ea72640 1 --2- 172.21.15.2:0/724150732 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5878107940 0x7f5878198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.712+0000 7f587ea72640 1 --2- 172.21.15.2:0/724150732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5878102340 0x7f58781982c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.712+0000 7f587ea72640 1 --2- 172.21.15.2:0/724150732 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5878101980 0x7f5878197d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.712+0000 7f587ea72640 1 -- 172.21.15.2:0/724150732 >> 172.21.15.2:0/724150732 conn(0x7f5878078ad0 msgr2=0x7f5878104360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:38.180 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.712+0000 7f587ea72640 1 -- 172.21.15.2:0/724150732 shutdown_connections 2024-05-14T17:11:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:09:58.712+0000 7f587ea72640 1 -- 172.21.15.2:0/724150732 wait complete. 2024-05-14T17:11:38.247 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:10:00 smithi002 ceph-mon[31538]: pgmap v478: 97 pgs: 97 active+clean; 5.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 9.5 MiB/s wr, 770 op/s 2024-05-14T17:11:38.280 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:10:00 smithi002 ceph-mon[31538]: overall HEALTH_OK 2024-05-14T17:11:38.314 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 885.0 s, 1320.8 tps, lat 72.440 ms stddev 101.207 2024-05-14T17:11:38.347 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:10:02 smithi002 ceph-mon[31538]: pgmap v479: 97 pgs: 97 active+clean; 5.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.1 MiB/s wr, 528 op/s 2024-05-14T17:11:38.380 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:10:04 smithi002 ceph-mon[31538]: pgmap v480: 97 pgs: 97 active+clean; 5.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 7.8 MiB/s wr, 727 op/s 2024-05-14T17:11:38.414 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:10:04 smithi002 ceph-mon[31538]: osdmap e98: 12 total, 12 up, 12 in 2024-05-14T17:11:38.447 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 890.0 s, 293.2 tps, lat 356.821 ms stddev 541.684 2024-05-14T17:11:38.517 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:10:06 smithi002 ceph-mon[31538]: pgmap v482: 97 pgs: 97 active+clean; 5.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.6 MiB/s wr, 561 op/s 2024-05-14T17:11:38.547 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:10:08 smithi002 ceph-mon[31538]: pgmap v483: 97 pgs: 97 active+clean; 5.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 10 MiB/s wr, 797 op/s 2024-05-14T17:11:38.581 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:10:08 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:11:38.614 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:10:09 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:11:38.647 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:10:09 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:11:38.681 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:10:09 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:11:38.714 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:10:10 smithi002 ceph-mon[31538]: pgmap v484: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 29 MiB/s wr, 1.84k op/s 2024-05-14T17:11:38.748 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 895.0 s, 1970.0 tps, lat 50.281 ms stddev 61.565 2024-05-14T17:11:38.781 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:11:30 smithi123 ceph-mon[35971]: pgmap v527: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 58 KiB/s wr, 14 op/s 2024-05-14T17:11:38.814 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:11:32 smithi123 ceph-mon[35971]: pgmap v528: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 9 op/s 2024-05-14T17:11:38.847 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:11:34 smithi123 ceph-mon[35971]: pgmap v529: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 60 KiB/s wr, 15 op/s 2024-05-14T17:11:38.881 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:11:36 smithi123 ceph-mon[35971]: pgmap v530: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 KiB/s wr, 11 op/s 2024-05-14T17:11:38.914 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:11:30 smithi151 ceph-mon[36066]: pgmap v527: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 58 KiB/s wr, 14 op/s 2024-05-14T17:11:38.947 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:11:32 smithi151 ceph-mon[36066]: pgmap v528: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 9 op/s 2024-05-14T17:11:38.981 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:11:34 smithi151 ceph-mon[36066]: pgmap v529: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 60 KiB/s wr, 15 op/s 2024-05-14T17:11:39.014 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:11:36 smithi151 ceph-mon[36066]: pgmap v530: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 KiB/s wr, 11 op/s 2024-05-14T17:11:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.836+0000 7f784ada4640 1 -- 172.21.15.2:0/2150257810 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7844108c50 msgr2=0x7f7844109030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:39.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.836+0000 7f784ada4640 1 --2- 172.21.15.2:0/2150257810 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7844108c50 0x7f7844109030 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f7840009900 tx=0x7f784002a480 comp rx=0 tx=0).stop 2024-05-14T17:11:39.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.836+0000 7f784ada4640 1 -- 172.21.15.2:0/2150257810 shutdown_connections 2024-05-14T17:11:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.836+0000 7f784ada4640 1 --2- 172.21.15.2:0/2150257810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f784406bc40 0x7f784410f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.836+0000 7f784ada4640 1 --2- 172.21.15.2:0/2150257810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f784406b280 0x7f784406b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.836+0000 7f784ada4640 1 --2- 172.21.15.2:0/2150257810 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7844108c50 0x7f7844109030 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:39.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.836+0000 7f784ada4640 1 -- 172.21.15.2:0/2150257810 >> 172.21.15.2:0/2150257810 conn(0x7f7844077750 msgr2=0x7f7844077b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:39.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.836+0000 7f784ada4640 1 -- 172.21.15.2:0/2150257810 shutdown_connections 2024-05-14T17:11:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.836+0000 7f784ada4640 1 -- 172.21.15.2:0/2150257810 wait complete. 2024-05-14T17:11:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.837+0000 7f784ada4640 1 Processor -- start 2024-05-14T17:11:39.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.837+0000 7f784ada4640 1 -- start start 2024-05-14T17:11:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.837+0000 7f784ada4640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f784406b280 0x7f7844199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.837+0000 7f784ada4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f784406bc40 0x7f7844199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.837+0000 7f784ada4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7844108c50 0x7f784419da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.837+0000 7f784ada4640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7844112d20 con 0x7f784406bc40 2024-05-14T17:11:39.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.837+0000 7f784ada4640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7844112ba0 con 0x7f7844108c50 2024-05-14T17:11:39.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.837+0000 7f784ada4640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7844112ea0 con 0x7f784406b280 2024-05-14T17:11:39.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.837+0000 7f783bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f784406bc40 0x7f7844199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.837+0000 7f783bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f784406bc40 0x7f7844199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38414/0 (socket says 172.21.15.2:38414) 2024-05-14T17:11:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.837+0000 7f783bfff640 1 -- 172.21.15.2:0/82032345 learned_addr learned my addr 172.21.15.2:0/82032345 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:11:39.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.837+0000 7f7849341640 1 --2- 172.21.15.2:0/82032345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7844108c50 0x7f784419da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.837+0000 7f7848b40640 1 --2- 172.21.15.2:0/82032345 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f784406b280 0x7f7844199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.838+0000 7f7849341640 1 -- 172.21.15.2:0/82032345 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f784406b280 msgr2=0x7f7844199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:39.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.838+0000 7f7849341640 1 --2- 172.21.15.2:0/82032345 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f784406b280 0x7f7844199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:39.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.838+0000 7f7849341640 1 -- 172.21.15.2:0/82032345 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f784406bc40 msgr2=0x7f7844199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:39.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.838+0000 7f7849341640 1 --2- 172.21.15.2:0/82032345 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f784406bc40 0x7f7844199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:40.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.838+0000 7f7849341640 1 -- 172.21.15.2:0/82032345 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f784419e130 con 0x7f7844108c50 2024-05-14T17:11:40.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.838+0000 7f7849341640 1 --2- 172.21.15.2:0/82032345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7844108c50 0x7f784419da00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f783c009fd0 tx=0x7f783c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.838+0000 7f7839ffb640 1 -- 172.21.15.2:0/82032345 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783c00ce20 con 0x7f7844108c50 2024-05-14T17:11:40.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.838+0000 7f7839ffb640 1 -- 172.21.15.2:0/82032345 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f783c005020 con 0x7f7844108c50 2024-05-14T17:11:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.838+0000 7f7839ffb640 1 -- 172.21.15.2:0/82032345 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783c010690 con 0x7f7844108c50 2024-05-14T17:11:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.839+0000 7f784ada4640 1 -- 172.21.15.2:0/82032345 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f784419e420 con 0x7f7844108c50 2024-05-14T17:11:40.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.839+0000 7f784ada4640 1 -- 172.21.15.2:0/82032345 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78441a5cb0 con 0x7f7844108c50 2024-05-14T17:11:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.842+0000 7f784db98740 1 Processor -- start 2024-05-14T17:11:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.843+0000 7f7839ffb640 1 -- 172.21.15.2:0/82032345 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f783c0054d0 con 0x7f7844108c50 2024-05-14T17:11:40.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.843+0000 7f784db98740 1 -- start start 2024-05-14T17:11:40.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.843+0000 7f7839ffb640 1 --2- 172.21.15.2:0/82032345 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f782c0b2a90 0x7f782c0b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.843+0000 7f784db98740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560bb328b8f0 0x560bb328bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.843+0000 7f784db98740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560bb328c210 0x560bb3296590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.844+0000 7f784db98740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bb3296ad0 0x560bb3298ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:40.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.844+0000 7f784db98740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x560bb31fec30 con 0x560bb328c210 2024-05-14T17:11:40.551 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.844+0000 7f784db98740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560bb31ee7d0 con 0x560bb3296ad0 2024-05-14T17:11:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.844+0000 7f784db98740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560bb31f2810 con 0x560bb328b8f0 2024-05-14T17:11:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.844+0000 7f7839ffb640 1 -- 172.21.15.2:0/82032345 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8557+0+0 (secure 0 0 0) 0x7f783c0e2b40 con 0x7f7844108c50 2024-05-14T17:11:40.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.844+0000 7f783bfff640 1 --2- 172.21.15.2:0/82032345 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f782c0b2a90 0x7f782c0b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:40.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.844+0000 7f783bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560bb328c210 0x560bb3296590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.844+0000 7f7848b40640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560bb328b8f0 0x560bb328bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.844+0000 7f7848b40640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560bb328b8f0 0x560bb328bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:39856/0 (socket says 172.21.15.2:39856) 2024-05-14T17:11:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.844+0000 7f7848b40640 1 -- 172.21.15.2:0/2546621545 learned_addr learned my addr 172.21.15.2:0/2546621545 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:11:40.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.844+0000 7f7849341640 1 --2- 172.21.15.2:0/2546621545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bb3296ad0 0x560bb3298ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.844+0000 7f7848b40640 1 -- 172.21.15.2:0/2546621545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bb3296ad0 msgr2=0x560bb3298ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:40.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.844+0000 7f7848b40640 1 --2- 172.21.15.2:0/2546621545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bb3296ad0 0x560bb3298ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:40.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.844+0000 7f7848b40640 1 -- 172.21.15.2:0/2546621545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560bb328c210 msgr2=0x560bb3296590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:40.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.844+0000 7f7848b40640 1 --2- 172.21.15.2:0/2546621545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560bb328c210 0x560bb3296590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.844+0000 7f7848b40640 1 -- 172.21.15.2:0/2546621545 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bb2f03210 con 0x560bb328b8f0 2024-05-14T17:11:41.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.844+0000 7f783bfff640 1 --2- 172.21.15.2:0/2546621545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560bb328c210 0x560bb3296590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:11:41.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.844+0000 7f7848b40640 1 --2- 172.21.15.2:0/2546621545 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560bb328b8f0 0x560bb328bcd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7840009900 tx=0x7f7840032b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=998162fc150ff9d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.845+0000 7f783bfff640 1 --2- 172.21.15.2:0/82032345 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f782c0b2a90 0x7f782c0b4f50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7834009810 tx=0x7f7834009c30 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.845+0000 7f782affd640 1 -- 172.21.15.2:0/2546621545 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7840035070 con 0x560bb328b8f0 2024-05-14T17:11:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.845+0000 7f782affd640 1 -- 172.21.15.2:0/2546621545 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7840031430 con 0x560bb328b8f0 2024-05-14T17:11:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.845+0000 7f782affd640 1 -- 172.21.15.2:0/2546621545 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7840005040 con 0x560bb328b8f0 2024-05-14T17:11:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.846+0000 7f784db98740 1 -- 172.21.15.2:0/2546621545 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560bb328b8f0 msgr2=0x560bb328bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.846+0000 7f784db98740 1 --2- 172.21.15.2:0/2546621545 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560bb328b8f0 0x560bb328bcd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7840009900 tx=0x7f7840032b30 comp rx=0 tx=0).stop 2024-05-14T17:11:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.846+0000 7f784db98740 1 -- 172.21.15.2:0/2546621545 shutdown_connections 2024-05-14T17:11:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.846+0000 7f784db98740 1 --2- 172.21.15.2:0/2546621545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bb3296ad0 0x560bb3298ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.846+0000 7f784db98740 1 --2- 172.21.15.2:0/2546621545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560bb328c210 0x560bb3296590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.846+0000 7f784db98740 1 --2- 172.21.15.2:0/2546621545 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560bb328b8f0 0x560bb328bcd0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:41.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.846+0000 7f784db98740 1 -- 172.21.15.2:0/2546621545 >> 172.21.15.2:0/2546621545 conn(0x560bb3287e00 msgr2=0x560bb32881e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.846+0000 7f784db98740 1 -- 172.21.15.2:0/2546621545 shutdown_connections 2024-05-14T17:11:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.846+0000 7f784db98740 1 -- 172.21.15.2:0/2546621545 wait complete. 2024-05-14T17:11:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.846+0000 7f784db98740 1 -- start start 2024-05-14T17:11:41.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.846+0000 7f784db98740 1 Processor -- start 2024-05-14T17:11:41.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.847+0000 7f784db98740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bb328b8f0 0x560bb3296420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:41.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.847+0000 7f784db98740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560bb328c210 0x560bb3293710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:41.744 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.847+0000 7f784db98740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560bb3296ad0 0x560bb3293c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.847+0000 7f784db98740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x560bb2ed4790 con 0x560bb3296ad0 2024-05-14T17:11:41.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.847+0000 7f784db98740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560bb31aa890 con 0x560bb328b8f0 2024-05-14T17:11:41.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.847+0000 7f784db98740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560bb3299740 con 0x560bb328c210 2024-05-14T17:11:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.847+0000 7f784db98740 10 client.0 _ll_register_callbacks cb 0x560bb31add60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.847+0000 7f7849341640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560bb3296ad0 0x560bb3293c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.847+0000 7f7849341640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560bb3296ad0 0x560bb3293c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38428/0 (socket says 172.21.15.2:38428) 2024-05-14T17:11:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.847+0000 7f7849341640 1 -- 172.21.15.2:0/3236961511 learned_addr learned my addr 172.21.15.2:0/3236961511 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:11:42.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.847+0000 7f783bfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560bb328c210 0x560bb3293710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.847+0000 7f7848b40640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bb328b8f0 0x560bb3296420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.847+0000 7f7849341640 1 -- 172.21.15.2:0/3236961511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560bb328c210 msgr2=0x560bb3293710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:42.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.847+0000 7f7849341640 1 --2- 172.21.15.2:0/3236961511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560bb328c210 0x560bb3293710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.847+0000 7f7849341640 1 -- 172.21.15.2:0/3236961511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bb328b8f0 msgr2=0x560bb3296420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.847+0000 7f7849341640 1 --2- 172.21.15.2:0/3236961511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bb328b8f0 0x560bb3296420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.847+0000 7f7849341640 1 -- 172.21.15.2:0/3236961511 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bb2fcf620 con 0x560bb3296ad0 2024-05-14T17:11:42.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.847+0000 7f783bfff640 1 --2- 172.21.15.2:0/3236961511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560bb328c210 0x560bb3293710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:11:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.847+0000 7f7848b40640 1 --2- 172.21.15.2:0/3236961511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bb328b8f0 0x560bb3296420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:11:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.847+0000 7f7849341640 1 --2- 172.21.15.2:0/3236961511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560bb3296ad0 0x560bb3293c70 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f783c00c760 tx=0x7f783c0a9a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=339fadeaa227fd89 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:42.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.847+0000 7f7823fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:11:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.848+0000 7f7823fff640 1 -- 172.21.15.2:0/3236961511 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783c0b2c60 con 0x560bb3296ad0 2024-05-14T17:11:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.848+0000 7f784db98740 1 -- 172.21.15.2:0/3236961511 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7840031430 con 0x560bb3296ad0 2024-05-14T17:11:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.848+0000 7f7823fff640 1 -- 172.21.15.2:0/3236961511 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8557+0+0 (secure 0 0 0) 0x7f783c0a9270 con 0x560bb3296ad0 2024-05-14T17:11:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.848+0000 7f7823fff640 1 -- 172.21.15.2:0/3236961511 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f783c0a5770 con 0x560bb3296ad0 2024-05-14T17:11:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.848+0000 7f784db98740 10 client.16872 fetch_fsmap learned FSMap version 51 2024-05-14T17:11:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.848+0000 7f784db98740 1 -- 172.21.15.2:0/3236961511 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x560bb32a51b0 con 0x560bb3296ad0 2024-05-14T17:11:42.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.848+0000 7f7823fff640 1 -- 172.21.15.2:0/3236961511 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7840031430 con 0x560bb3296ad0 2024-05-14T17:11:42.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.848+0000 7f7823fff640 1 -- 172.21.15.2:0/3236961511 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 51) ==== 8354+0+0 (secure 0 0 0) 0x7f783c0c1070 con 0x560bb3296ad0 2024-05-14T17:11:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.848+0000 7f784db98740 10 client.16872 fetch_fsmap finished waiting for FSMap version 51 2024-05-14T17:11:42.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.848+0000 7f784db98740 10 client.16872 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:11:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.848+0000 7f784db98740 20 client.16872 populate_metadata read hostname 'smithi002' 2024-05-14T17:11:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.848+0000 7f784db98740 1 --2- 172.21.15.2:0/3236961511 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x560bb32a6670 0x560bb32a8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:42.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.848+0000 7f784db98740 4 client.16872 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:11:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.848+0000 7f784db98740 1 -- 172.21.15.2:0/3236961511 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bb3200b80 con 0x560bb32a6670 2024-05-14T17:11:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.848+0000 7f7848b40640 1 --2- 172.21.15.2:0/3236961511 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x560bb32a6670 0x560bb32a8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.849+0000 7f7848b40640 1 --2- 172.21.15.2:0/3236961511 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x560bb32a6670 0x560bb32a8a80 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40fc70b2f6a5345a server_cookie=a8398f0094d35164 in_seq=0 out_seq=0 2024-05-14T17:11:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.849+0000 7f7823fff640 10 client.16872 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:11:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.849+0000 7f7823fff640 1 -- 172.21.15.2:0/3236961511 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560bb3200b80 con 0x560bb32a6670 2024-05-14T17:11:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.849+0000 7f7823fff640 10 client.16872 handle_command_reply: tid=0 2024-05-14T17:11:43.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.850+0000 7f784db98740 1 client.16872 shutdown 2024-05-14T17:11:43.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.850+0000 7f784db98740 1 -- 172.21.15.2:0/3236961511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560bb3296ad0 msgr2=0x560bb3293c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.850+0000 7f784db98740 1 --2- 172.21.15.2:0/3236961511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560bb3296ad0 0x560bb3293c70 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f783c00c760 tx=0x7f783c0a9a30 comp rx=0 tx=0).stop 2024-05-14T17:11:43.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.850+0000 7f784db98740 1 -- 172.21.15.2:0/3236961511 shutdown_connections 2024-05-14T17:11:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.850+0000 7f784db98740 1 --2- 172.21.15.2:0/3236961511 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x560bb32a6670 0x560bb32a8a80 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.850+0000 7f784db98740 1 --2- 172.21.15.2:0/3236961511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560bb3296ad0 0x560bb3293c70 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.850+0000 7f784db98740 1 --2- 172.21.15.2:0/3236961511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560bb328c210 0x560bb3293710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:43.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.850+0000 7f784db98740 1 --2- 172.21.15.2:0/3236961511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bb328b8f0 0x560bb3296420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.850+0000 7f784db98740 1 -- 172.21.15.2:0/3236961511 >> 172.21.15.2:0/3236961511 conn(0x560bb3287e00 msgr2=0x560bb3287100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:43.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.851+0000 7f7823fff640 0 client.16872 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:11:43.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.851+0000 7f784db98740 1 -- 172.21.15.2:0/3236961511 shutdown_connections 2024-05-14T17:11:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.851+0000 7f784db98740 1 -- 172.21.15.2:0/3236961511 wait complete. 2024-05-14T17:11:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.851+0000 7f784db98740 20 client.16872 trim_cache size 0 max 16384 2024-05-14T17:11:43.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.870+0000 7f784db98740 1 Processor -- start 2024-05-14T17:11:43.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.870+0000 7f784db98740 1 -- start start 2024-05-14T17:11:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.870+0000 7f784db98740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560bb328b8f0 0x560bb32a2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:43.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.870+0000 7f784db98740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bb328c210 0x560bb3286f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.870+0000 7f784db98740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560bb3296ad0 0x560bb32ae390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.870+0000 7f784db98740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f78441aeda0 con 0x560bb3296ad0 2024-05-14T17:11:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.870+0000 7f784db98740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f78441ac610 con 0x560bb328c210 2024-05-14T17:11:43.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.870+0000 7f784db98740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f78441aa480 con 0x560bb328b8f0 2024-05-14T17:11:43.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.870+0000 7f7848b40640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560bb328b8f0 0x560bb32a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:43.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.870+0000 7f7849341640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560bb3296ad0 0x560bb32ae390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.870+0000 7f783bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bb328c210 0x560bb3286f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.870+0000 7f7848b40640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560bb328b8f0 0x560bb32a2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:39872/0 (socket says 172.21.15.2:39872) 2024-05-14T17:11:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.870+0000 7f7848b40640 1 -- 172.21.15.2:0/2020494238 learned_addr learned my addr 172.21.15.2:0/2020494238 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:11:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.871+0000 7f7849341640 1 -- 172.21.15.2:0/2020494238 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560bb328b8f0 msgr2=0x560bb32a2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.871+0000 7f7849341640 1 --2- 172.21.15.2:0/2020494238 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560bb328b8f0 0x560bb32a2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.871+0000 7f7849341640 1 -- 172.21.15.2:0/2020494238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bb328c210 msgr2=0x560bb3286f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.871+0000 7f7849341640 1 --2- 172.21.15.2:0/2020494238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bb328c210 0x560bb3286f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.871+0000 7f7849341640 1 -- 172.21.15.2:0/2020494238 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bb3287c30 con 0x560bb3296ad0 2024-05-14T17:11:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.871+0000 7f7849341640 1 --2- 172.21.15.2:0/2020494238 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560bb3296ad0 0x560bb32ae390 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f783c0af5e0 tx=0x7f783c0ae330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fad1cbcddc0c06e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:44.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.871+0000 7f782a7fc640 1 -- 172.21.15.2:0/2020494238 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783c0b2c60 con 0x560bb3296ad0 2024-05-14T17:11:44.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.871+0000 7f782a7fc640 1 -- 172.21.15.2:0/2020494238 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f783c0af340 con 0x560bb3296ad0 2024-05-14T17:11:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.871+0000 7f782a7fc640 1 -- 172.21.15.2:0/2020494238 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783c004ca0 con 0x560bb3296ad0 2024-05-14T17:11:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.872+0000 7f784db98740 1 -- 172.21.15.2:0/2020494238 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560bb3296ad0 msgr2=0x560bb32ae390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.872+0000 7f784db98740 1 --2- 172.21.15.2:0/2020494238 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560bb3296ad0 0x560bb32ae390 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f783c0af5e0 tx=0x7f783c0ae330 comp rx=0 tx=0).stop 2024-05-14T17:11:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.872+0000 7f784db98740 1 -- 172.21.15.2:0/2020494238 shutdown_connections 2024-05-14T17:11:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.872+0000 7f784db98740 1 --2- 172.21.15.2:0/2020494238 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560bb3296ad0 0x560bb32ae390 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.872+0000 7f784db98740 1 --2- 172.21.15.2:0/2020494238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bb328c210 0x560bb3286f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:44.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.873+0000 7f784db98740 1 --2- 172.21.15.2:0/2020494238 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560bb328b8f0 0x560bb32a2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:44.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.873+0000 7f784db98740 1 -- 172.21.15.2:0/2020494238 >> 172.21.15.2:0/2020494238 conn(0x560bb3287e00 msgr2=0x560bb3286870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.874+0000 7f784db98740 1 -- 172.21.15.2:0/2020494238 shutdown_connections 2024-05-14T17:11:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.874+0000 7f784db98740 1 -- 172.21.15.2:0/2020494238 wait complete. 2024-05-14T17:11:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.875+0000 7f784db98740 1 -- start start 2024-05-14T17:11:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.875+0000 7f784db98740 1 Processor -- start 2024-05-14T17:11:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.875+0000 7f784db98740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560bb328b8f0 0x560bb329b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.875+0000 7f784db98740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560bb328c210 0x560bb329bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.875+0000 7f784db98740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bb3296ad0 0x560bb329c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.875+0000 7f784db98740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f78441aaad0 con 0x560bb328b8f0 2024-05-14T17:11:44.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.875+0000 7f784db98740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f78441abf40 con 0x560bb3296ad0 2024-05-14T17:11:45.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.875+0000 7f784db98740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f78441ab1a0 con 0x560bb328c210 2024-05-14T17:11:45.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.876+0000 7f784db98740 10 client.0 _ll_register_callbacks cb 0x7f783c005400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:45.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.876+0000 7f7848b40640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560bb328b8f0 0x560bb329b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.876+0000 7f7848b40640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560bb328b8f0 0x560bb329b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38446/0 (socket says 172.21.15.2:38446) 2024-05-14T17:11:45.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.876+0000 7f7849341640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bb3296ad0 0x560bb329c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.876+0000 7f7848b40640 1 -- 172.21.15.2:0/3282196097 learned_addr learned my addr 172.21.15.2:0/3282196097 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:11:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.876+0000 7f783bfff640 1 --2- 172.21.15.2:0/3282196097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560bb328c210 0x560bb329bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.876+0000 7f7848b40640 1 -- 172.21.15.2:0/3282196097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560bb328c210 msgr2=0x560bb329bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.876+0000 7f7848b40640 1 --2- 172.21.15.2:0/3282196097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560bb328c210 0x560bb329bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.876+0000 7f7848b40640 1 -- 172.21.15.2:0/3282196097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bb3296ad0 msgr2=0x560bb329c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.876+0000 7f7848b40640 1 --2- 172.21.15.2:0/3282196097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bb3296ad0 0x560bb329c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.876+0000 7f7848b40640 1 -- 172.21.15.2:0/3282196097 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bb330f0e0 con 0x560bb328b8f0 2024-05-14T17:11:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.876+0000 7f7849341640 1 --2- 172.21.15.2:0/3282196097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bb3296ad0 0x560bb329c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:11:45.514 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.876+0000 7f7848b40640 1 --2- 172.21.15.2:0/3282196097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560bb328b8f0 0x560bb329b960 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f7840000c00 tx=0x7f784002ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dbd332ac9e121c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.876+0000 7f7823fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:11:45.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.876+0000 7f7823fff640 1 -- 172.21.15.2:0/3282196097 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7840035040 con 0x560bb328b8f0 2024-05-14T17:11:45.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.876+0000 7f7823fff640 1 -- 172.21.15.2:0/3282196097 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8557+0+0 (secure 0 0 0) 0x7f7840031d60 con 0x560bb328b8f0 2024-05-14T17:11:45.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.876+0000 7f784db98740 1 -- 172.21.15.2:0/3282196097 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f783c0af340 con 0x560bb328b8f0 2024-05-14T17:11:45.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.877+0000 7f7823fff640 1 -- 172.21.15.2:0/3282196097 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f7840060050 con 0x560bb328b8f0 2024-05-14T17:11:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.877+0000 7f784db98740 10 client.16878 fetch_fsmap learned FSMap version 51 2024-05-14T17:11:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.877+0000 7f784db98740 1 -- 172.21.15.2:0/3282196097 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x560bb33103f0 con 0x560bb328b8f0 2024-05-14T17:11:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.877+0000 7f7823fff640 1 -- 172.21.15.2:0/3282196097 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7840005040 con 0x560bb328b8f0 2024-05-14T17:11:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.877+0000 7f7823fff640 1 -- 172.21.15.2:0/3282196097 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 51) ==== 8354+0+0 (secure 0 0 0) 0x7f784003e7d0 con 0x560bb328b8f0 2024-05-14T17:11:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.877+0000 7f784db98740 10 client.16878 fetch_fsmap finished waiting for FSMap version 51 2024-05-14T17:11:45.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.877+0000 7f784db98740 10 client.16878 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:11:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.877+0000 7f784db98740 20 client.16878 populate_metadata read hostname 'smithi002' 2024-05-14T17:11:45.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.877+0000 7f784db98740 1 --2- 172.21.15.2:0/3282196097 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x560bb3310750 0x560bb3312b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:46.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.877+0000 7f784db98740 4 client.16878 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:11:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.877+0000 7f784db98740 1 -- 172.21.15.2:0/3282196097 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f783c0af340 con 0x560bb3310750 2024-05-14T17:11:46.090 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.877+0000 7f783bfff640 1 --2- 172.21.15.2:0/3282196097 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x560bb3310750 0x560bb3312b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.878+0000 7f7823fff640 10 client.16878 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:11:46.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.878+0000 7f783bfff640 1 --2- 172.21.15.2:0/3282196097 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x560bb3310750 0x560bb3312b70 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8560916af87dddd server_cookie=70355186b8231fb8 in_seq=0 out_seq=0 2024-05-14T17:11:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.890+0000 7f7823fff640 1 -- 172.21.15.2:0/3282196097 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f783c0af340 con 0x560bb3310750 2024-05-14T17:11:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.890+0000 7f7823fff640 10 client.16878 handle_command_reply: tid=0 2024-05-14T17:11:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.890+0000 7f784db98740 1 client.16878 shutdown 2024-05-14T17:11:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.890+0000 7f784db98740 1 -- 172.21.15.2:0/3282196097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560bb328b8f0 msgr2=0x560bb329b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.890+0000 7f784db98740 1 --2- 172.21.15.2:0/3282196097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560bb328b8f0 0x560bb329b960 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f7840000c00 tx=0x7f784002ab90 comp rx=0 tx=0).stop 2024-05-14T17:11:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.890+0000 7f784db98740 1 -- 172.21.15.2:0/3282196097 shutdown_connections 2024-05-14T17:11:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.890+0000 7f784db98740 1 --2- 172.21.15.2:0/3282196097 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x560bb3310750 0x560bb3312b70 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.890+0000 7f784db98740 1 --2- 172.21.15.2:0/3282196097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bb3296ad0 0x560bb329c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:46.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.890+0000 7f784db98740 1 --2- 172.21.15.2:0/3282196097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560bb328c210 0x560bb329bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:46.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.890+0000 7f784db98740 1 --2- 172.21.15.2:0/3282196097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560bb328b8f0 0x560bb329b960 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.890+0000 7f7823fff640 0 client.16878 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:11:46.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.890+0000 7f784db98740 1 -- 172.21.15.2:0/3282196097 >> 172.21.15.2:0/3282196097 conn(0x560bb3287e00 msgr2=0x560bb3286870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:46.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.890+0000 7f784db98740 1 -- 172.21.15.2:0/3282196097 shutdown_connections 2024-05-14T17:11:46.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.890+0000 7f784db98740 1 -- 172.21.15.2:0/3282196097 wait complete. 2024-05-14T17:11:46.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.890+0000 7f784db98740 20 client.16878 trim_cache size 0 max 16384 2024-05-14T17:11:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:11:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:11:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:11:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.891+0000 7f784ada4640 1 -- 172.21.15.2:0/82032345 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f782c0b2a90 msgr2=0x7f782c0b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.891+0000 7f784ada4640 1 --2- 172.21.15.2:0/82032345 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f782c0b2a90 0x7f782c0b4f50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7834009810 tx=0x7f7834009c30 comp rx=0 tx=0).stop 2024-05-14T17:11:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.891+0000 7f784ada4640 1 -- 172.21.15.2:0/82032345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7844108c50 msgr2=0x7f784419da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:46.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.891+0000 7f784ada4640 1 --2- 172.21.15.2:0/82032345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7844108c50 0x7f784419da00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f783c009fd0 tx=0x7f783c00ef10 comp rx=0 tx=0).stop 2024-05-14T17:11:46.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.891+0000 7f784ada4640 1 -- 172.21.15.2:0/82032345 shutdown_connections 2024-05-14T17:11:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.891+0000 7f784ada4640 1 --2- 172.21.15.2:0/82032345 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f782c0b2a90 0x7f782c0b4f50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.891+0000 7f784ada4640 1 --2- 172.21.15.2:0/82032345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7844108c50 0x7f784419da00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.891+0000 7f784ada4640 1 --2- 172.21.15.2:0/82032345 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f784406bc40 0x7f7844199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.891+0000 7f784ada4640 1 --2- 172.21.15.2:0/82032345 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f784406b280 0x7f7844199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.891+0000 7f784ada4640 1 -- 172.21.15.2:0/82032345 >> 172.21.15.2:0/82032345 conn(0x7f7844077750 msgr2=0x7f7844100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.891+0000 7f784ada4640 1 -- 172.21.15.2:0/82032345 shutdown_connections 2024-05-14T17:11:47.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:11.891+0000 7f784ada4640 1 -- 172.21.15.2:0/82032345 wait complete. 2024-05-14T17:11:47.233 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:10:12 smithi002 ceph-mon[31538]: pgmap v485: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 29 MiB/s wr, 1.84k op/s 2024-05-14T17:11:47.274 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:10:12 smithi002 ceph-mon[31538]: osdmap e99: 12 total, 12 up, 12 in 2024-05-14T17:11:47.323 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:10:14 smithi002 ceph-mon[31538]: pgmap v487: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 30 MiB/s wr, 1.96k op/s 2024-05-14T17:11:47.358 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 900.0 s, 1977.4 tps, lat 50.883 ms stddev 56.361 2024-05-14T17:11:47.391 INFO:teuthology.orchestra.run.smithi002.stdout:transaction type: 2024-05-14T17:11:47.424 INFO:teuthology.orchestra.run.smithi002.stdout:scaling factor: 32 2024-05-14T17:11:47.458 INFO:teuthology.orchestra.run.smithi002.stdout:query mode: simple 2024-05-14T17:11:47.491 INFO:teuthology.orchestra.run.smithi002.stdout:number of clients: 100 2024-05-14T17:11:47.524 INFO:teuthology.orchestra.run.smithi002.stdout:number of threads: 4 2024-05-14T17:11:47.624 INFO:teuthology.orchestra.run.smithi002.stdout:duration: 900 s 2024-05-14T17:11:47.658 INFO:teuthology.orchestra.run.smithi002.stdout:number of transactions actually processed: 1558276 2024-05-14T17:11:47.691 INFO:teuthology.orchestra.run.smithi002.stdout:latency average = 57.751 ms 2024-05-14T17:11:47.724 INFO:teuthology.orchestra.run.smithi002.stdout:latency stddev = 1212.323 ms 2024-05-14T17:11:47.758 INFO:teuthology.orchestra.run.smithi002.stdout:tps = 1731.222920 (including connections establishing) 2024-05-14T17:11:47.791 INFO:teuthology.orchestra.run.smithi002.stdout:tps = 1731.228351 (excluding connections establishing) 2024-05-14T17:11:47.842 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:10:16 smithi002 ceph-mon[31538]: pgmap v488: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 31 MiB/s wr, 1.99k op/s 2024-05-14T17:11:47.877 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2024-05-14T17:11:47.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:11:38 smithi123 ceph-mon[35971]: pgmap v531: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 KiB/s wr, 11 op/s 2024-05-14T17:11:47.941 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:11:40 smithi123 ceph-mon[35971]: pgmap v532: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-05-14T17:11:48.005 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:11:42 smithi123 ceph-mon[35971]: pgmap v533: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-14T17:11:48.041 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:11:44 smithi123 ceph-mon[35971]: pgmap v534: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 26 KiB/s wr, 14 op/s 2024-05-14T17:11:48.075 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:11:46 smithi123 ceph-mon[35971]: pgmap v535: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-05-14T17:11:48.108 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:11:38 smithi151 ceph-mon[36066]: pgmap v531: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 KiB/s wr, 11 op/s 2024-05-14T17:11:48.142 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:11:40 smithi151 ceph-mon[36066]: pgmap v532: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-05-14T17:11:48.175 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:11:42 smithi151 ceph-mon[36066]: pgmap v533: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-14T17:11:48.208 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:11:44 smithi151 ceph-mon[36066]: pgmap v534: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 26 KiB/s wr, 14 op/s 2024-05-14T17:11:48.242 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:11:46 smithi151 ceph-mon[36066]: pgmap v535: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-05-14T17:11:48.275 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:10:18 smithi002 ceph-mon[31538]: pgmap v489: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 27 MiB/s wr, 1.77k op/s 2024-05-14T17:11:48.308 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:10:20 smithi002 ceph-mon[31538]: pgmap v490: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.3 MiB/s wr, 741 op/s 2024-05-14T17:11:48.342 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:10:22 smithi002 ceph-mon[31538]: pgmap v491: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.3 MiB/s wr, 741 op/s 2024-05-14T17:11:48.383 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:10:24 smithi002 ceph-mon[31538]: pgmap v492: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.9 MiB/s wr, 715 op/s 2024-05-14T17:11:48.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.016+0000 7f1c19c1d640 1 -- 172.21.15.2:0/2860193087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c14108a40 msgr2=0x7f1c14108e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.016+0000 7f1c19c1d640 1 --2- 172.21.15.2:0/2860193087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c14108a40 0x7f1c14108e20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1c04009900 tx=0x7f1c0402a3f0 comp rx=0 tx=0).stop 2024-05-14T17:11:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.016+0000 7f1c19c1d640 1 -- 172.21.15.2:0/2860193087 shutdown_connections 2024-05-14T17:11:48.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.016+0000 7f1c19c1d640 1 --2- 172.21.15.2:0/2860193087 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c141033f0 0x7f1c1410f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.016+0000 7f1c19c1d640 1 --2- 172.21.15.2:0/2860193087 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c14102a30 0x7f1c14102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:48.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.016+0000 7f1c19c1d640 1 --2- 172.21.15.2:0/2860193087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c14108a40 0x7f1c14108e20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.016+0000 7f1c19c1d640 1 -- 172.21.15.2:0/2860193087 >> 172.21.15.2:0/2860193087 conn(0x7f1c140fe500 msgr2=0x7f1c14100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.016+0000 7f1c19c1d640 1 -- 172.21.15.2:0/2860193087 shutdown_connections 2024-05-14T17:11:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.016+0000 7f1c19c1d640 1 -- 172.21.15.2:0/2860193087 wait complete. 2024-05-14T17:11:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.017+0000 7f1c19c1d640 1 Processor -- start 2024-05-14T17:11:48.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.017+0000 7f1c19c1d640 1 -- start start 2024-05-14T17:11:48.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.017+0000 7f1c19c1d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c14102a30 0x7f1c14196f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.017+0000 7f1c19c1d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c141033f0 0x7f1c14197490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:48.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.017+0000 7f1c19c1d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c14108a40 0x7f1c1419b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.017+0000 7f1c19c1d640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1c14110b10 con 0x7f1c14108a40 2024-05-14T17:11:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.017+0000 7f1c19c1d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1c14110990 con 0x7f1c141033f0 2024-05-14T17:11:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.017+0000 7f1c19c1d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1c14110c90 con 0x7f1c14102a30 2024-05-14T17:11:49.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.017+0000 7f1c137fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c14102a30 0x7f1c14196f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.017+0000 7f1c0bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c141033f0 0x7f1c14197490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.017+0000 7f1c13fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c14108a40 0x7f1c1419b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.017+0000 7f1c137fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c14102a30 0x7f1c14196f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.2:38274/0 (socket says 172.21.15.2:38274) 2024-05-14T17:11:49.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.017+0000 7f1c137fe640 1 -- 172.21.15.2:0/4290240961 learned_addr learned my addr 172.21.15.2:0/4290240961 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:11:49.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.017+0000 7f1c0bfff640 1 -- 172.21.15.2:0/4290240961 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c14102a30 msgr2=0x7f1c14196f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.017+0000 7f1c0bfff640 1 --2- 172.21.15.2:0/4290240961 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c14102a30 0x7f1c14196f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.017+0000 7f1c0bfff640 1 -- 172.21.15.2:0/4290240961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c14108a40 msgr2=0x7f1c1419b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:49.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.017+0000 7f1c0bfff640 1 --2- 172.21.15.2:0/4290240961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c14108a40 0x7f1c1419b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.018+0000 7f1c0bfff640 1 -- 172.21.15.2:0/4290240961 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c1419bef0 con 0x7f1c141033f0 2024-05-14T17:11:49.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.018+0000 7f1c0bfff640 1 --2- 172.21.15.2:0/4290240961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c141033f0 0x7f1c14197490 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc00e9f0 tx=0x7f1bfc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.018+0000 7f1c117fa640 1 -- 172.21.15.2:0/4290240961 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc00ccc0 con 0x7f1c141033f0 2024-05-14T17:11:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.018+0000 7f1c117fa640 1 -- 172.21.15.2:0/4290240961 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bfc005020 con 0x7f1c141033f0 2024-05-14T17:11:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.018+0000 7f1c19c1d640 1 -- 172.21.15.2:0/4290240961 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c1419c1e0 con 0x7f1c141033f0 2024-05-14T17:11:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.018+0000 7f1c117fa640 1 -- 172.21.15.2:0/4290240961 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc004a00 con 0x7f1c141033f0 2024-05-14T17:11:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.018+0000 7f1c19c1d640 1 -- 172.21.15.2:0/4290240961 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c141a3a70 con 0x7f1c141033f0 2024-05-14T17:11:49.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.021+0000 7f1c1ca0b740 1 Processor -- start 2024-05-14T17:11:49.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.021+0000 7f1c1ca0b740 1 -- start start 2024-05-14T17:11:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.021+0000 7f1c117fa640 1 -- 172.21.15.2:0/4290240961 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f1bfc004ba0 con 0x7f1c141033f0 2024-05-14T17:11:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.021+0000 7f1c1ca0b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587ac3618f0 0x5587ac361cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.021+0000 7f1c1ca0b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587ac362210 0x5587ac36c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.021+0000 7f1c1ca0b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587ac36cad0 0x5587ac36eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.021+0000 7f1c1ca0b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5587ac2d4c30 con 0x5587ac362210 2024-05-14T17:11:49.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.021+0000 7f1c1ca0b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5587ac2c47d0 con 0x5587ac3618f0 2024-05-14T17:11:49.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.021+0000 7f1c1ca0b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5587ac2c8810 con 0x5587ac36cad0 2024-05-14T17:11:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.021+0000 7f1c13fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587ac3618f0 0x5587ac361cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.021+0000 7f1c137fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587ac362210 0x5587ac36c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.021+0000 7f1c13fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587ac3618f0 0x5587ac361cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:37576/0 (socket says 172.21.15.2:37576) 2024-05-14T17:11:50.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.021+0000 7f1c13fff640 1 -- 172.21.15.2:0/1190747437 learned_addr learned my addr 172.21.15.2:0/1190747437 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:11:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.021+0000 7f1c0bfff640 1 --2- 172.21.15.2:0/1190747437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587ac36cad0 0x5587ac36eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.022+0000 7f1c13fff640 1 -- 172.21.15.2:0/1190747437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587ac36cad0 msgr2=0x5587ac36eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.022+0000 7f1c13fff640 1 --2- 172.21.15.2:0/1190747437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587ac36cad0 0x5587ac36eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.022+0000 7f1c13fff640 1 -- 172.21.15.2:0/1190747437 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587ac362210 msgr2=0x5587ac36c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:50.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.022+0000 7f1c13fff640 1 --2- 172.21.15.2:0/1190747437 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587ac362210 0x5587ac36c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:50.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.022+0000 7f1c13fff640 1 -- 172.21.15.2:0/1190747437 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587abfd9210 con 0x5587ac3618f0 2024-05-14T17:11:50.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.022+0000 7f1c13fff640 1 --2- 172.21.15.2:0/1190747437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587ac3618f0 0x5587ac361cd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c013410 tx=0x7f1c0c0138d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4771d80de8af4b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:50.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.022+0000 7f1c09ffb640 1 -- 172.21.15.2:0/1190747437 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c01c020 con 0x5587ac3618f0 2024-05-14T17:11:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.022+0000 7f1c09ffb640 1 -- 172.21.15.2:0/1190747437 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c0c018b30 con 0x5587ac3618f0 2024-05-14T17:11:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.022+0000 7f1c09ffb640 1 -- 172.21.15.2:0/1190747437 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c017b90 con 0x5587ac3618f0 2024-05-14T17:11:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.022+0000 7f1c117fa640 1 --2- 172.21.15.2:0/4290240961 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f1c000b2a10 0x7f1c000b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:50.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.022+0000 7f1c137fe640 1 --2- 172.21.15.2:0/4290240961 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f1c000b2a10 0x7f1c000b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:50.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.022+0000 7f1c117fa640 1 -- 172.21.15.2:0/4290240961 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f1bfc0e25e0 con 0x7f1c141033f0 2024-05-14T17:11:50.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.023+0000 7f1c1ca0b740 1 -- 172.21.15.2:0/1190747437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587ac3618f0 msgr2=0x5587ac361cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:50.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.023+0000 7f1c137fe640 1 --2- 172.21.15.2:0/4290240961 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f1c000b2a10 0x7f1c000b4ed0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1c04005ec0 tx=0x7f1c04005a20 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:50.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.023+0000 7f1c1ca0b740 1 --2- 172.21.15.2:0/1190747437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587ac3618f0 0x5587ac361cd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c013410 tx=0x7f1c0c0138d0 comp rx=0 tx=0).stop 2024-05-14T17:11:50.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.023+0000 7f1c1ca0b740 1 -- 172.21.15.2:0/1190747437 shutdown_connections 2024-05-14T17:11:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.023+0000 7f1c1ca0b740 1 --2- 172.21.15.2:0/1190747437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587ac36cad0 0x5587ac36eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:50.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.023+0000 7f1c1ca0b740 1 --2- 172.21.15.2:0/1190747437 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587ac362210 0x5587ac36c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:50.811 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.023+0000 7f1c1ca0b740 1 --2- 172.21.15.2:0/1190747437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587ac3618f0 0x5587ac361cd0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.023+0000 7f1c1ca0b740 1 -- 172.21.15.2:0/1190747437 >> 172.21.15.2:0/1190747437 conn(0x5587ac35de00 msgr2=0x5587ac35e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.023+0000 7f1c1ca0b740 1 -- 172.21.15.2:0/1190747437 shutdown_connections 2024-05-14T17:11:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.023+0000 7f1c1ca0b740 1 -- 172.21.15.2:0/1190747437 wait complete. 2024-05-14T17:11:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.023+0000 7f1c1ca0b740 1 -- start start 2024-05-14T17:11:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.024+0000 7f1c1ca0b740 1 Processor -- start 2024-05-14T17:11:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.024+0000 7f1c1ca0b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587ac3618f0 0x5587ac36c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.024+0000 7f1c1ca0b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587ac362210 0x5587ac369730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.024+0000 7f1c1ca0b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587ac36cad0 0x5587ac369c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.024+0000 7f1c1ca0b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5587abfaa790 con 0x5587ac3618f0 2024-05-14T17:11:51.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.024+0000 7f1c1ca0b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5587ac280890 con 0x5587ac36cad0 2024-05-14T17:11:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.024+0000 7f1c1ca0b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5587ac36f7a0 con 0x5587ac362210 2024-05-14T17:11:51.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.024+0000 7f1c1ca0b740 10 client.0 _ll_register_callbacks cb 0x5587ac283d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.024+0000 7f1c137fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587ac3618f0 0x5587ac36c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:51.378 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.024+0000 7f1c137fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587ac3618f0 0x5587ac36c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58284/0 (socket says 172.21.15.2:58284) 2024-05-14T17:11:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.024+0000 7f1c137fe640 1 -- 172.21.15.2:0/544737357 learned_addr learned my addr 172.21.15.2:0/544737357 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:11:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.024+0000 7f1c13fff640 1 --2- 172.21.15.2:0/544737357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587ac36cad0 0x5587ac369c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:51.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.024+0000 7f1c0bfff640 1 --2- 172.21.15.2:0/544737357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587ac362210 0x5587ac369730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:51.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.024+0000 7f1c137fe640 1 -- 172.21.15.2:0/544737357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587ac362210 msgr2=0x5587ac369730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.024+0000 7f1c137fe640 1 --2- 172.21.15.2:0/544737357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587ac362210 0x5587ac369730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:51.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.024+0000 7f1c137fe640 1 -- 172.21.15.2:0/544737357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587ac36cad0 msgr2=0x5587ac369c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.024+0000 7f1c137fe640 1 --2- 172.21.15.2:0/544737357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587ac36cad0 0x5587ac369c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.024+0000 7f1c137fe640 1 -- 172.21.15.2:0/544737357 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587ac0a5620 con 0x5587ac3618f0 2024-05-14T17:11:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.024+0000 7f1c13fff640 1 --2- 172.21.15.2:0/544737357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587ac36cad0 0x5587ac369c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:11:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.025+0000 7f1c137fe640 1 --2- 172.21.15.2:0/544737357 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587ac3618f0 0x5587ac36c440 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f1c04009a20 tx=0x7f1c04009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e25e8861d0357d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.025+0000 7f1bf2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:11:51.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.025+0000 7f1bf2ffd640 1 -- 172.21.15.2:0/544737357 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0404d070 con 0x5587ac3618f0 2024-05-14T17:11:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.025+0000 7f1bf2ffd640 1 -- 172.21.15.2:0/544737357 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f1c04005460 con 0x5587ac3618f0 2024-05-14T17:11:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.025+0000 7f1c1ca0b740 1 -- 172.21.15.2:0/544737357 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c0c018b30 con 0x5587ac3618f0 2024-05-14T17:11:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.025+0000 7f1bf2ffd640 1 -- 172.21.15.2:0/544737357 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f1c04059050 con 0x5587ac3618f0 2024-05-14T17:11:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.025+0000 7f1c1ca0b740 10 client.16887 fetch_fsmap learned FSMap version 51 2024-05-14T17:11:52.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.025+0000 7f1c1ca0b740 1 -- 172.21.15.2:0/544737357 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5587ac37b1c0 con 0x5587ac3618f0 2024-05-14T17:11:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.025+0000 7f1bf2ffd640 1 -- 172.21.15.2:0/544737357 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c0c018b30 con 0x5587ac3618f0 2024-05-14T17:11:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.025+0000 7f1bf2ffd640 1 -- 172.21.15.2:0/544737357 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 51) ==== 8354+0+0 (secure 0 0 0) 0x7f1c040038c0 con 0x5587ac3618f0 2024-05-14T17:11:52.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.025+0000 7f1c1ca0b740 10 client.16887 fetch_fsmap finished waiting for FSMap version 51 2024-05-14T17:11:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.025+0000 7f1c1ca0b740 10 client.16887 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:11:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.025+0000 7f1c1ca0b740 20 client.16887 populate_metadata read hostname 'smithi002' 2024-05-14T17:11:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.025+0000 7f1c1ca0b740 1 --2- 172.21.15.2:0/544737357 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5587ac37c6f0 0x5587ac37eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.025+0000 7f1c1ca0b740 4 client.16887 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:11:52.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.025+0000 7f1c1ca0b740 1 -- 172.21.15.2:0/544737357 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587ac2d6b80 con 0x5587ac37c6f0 2024-05-14T17:11:52.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.026+0000 7f1c0bfff640 1 --2- 172.21.15.2:0/544737357 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5587ac37c6f0 0x5587ac37eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.026+0000 7f1c0bfff640 1 --2- 172.21.15.2:0/544737357 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5587ac37c6f0 0x5587ac37eae0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32c264d125c8cec7 server_cookie=135d3f29d04715a8 in_seq=0 out_seq=0 2024-05-14T17:11:52.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.026+0000 7f1bf2ffd640 10 client.16887 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:11:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.027+0000 7f1bf2ffd640 1 -- 172.21.15.2:0/544737357 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5587ac2d6b80 con 0x5587ac37c6f0 2024-05-14T17:11:52.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.027+0000 7f1bf2ffd640 10 client.16887 handle_command_reply: tid=0 2024-05-14T17:11:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.027+0000 7f1c1ca0b740 1 client.16887 shutdown 2024-05-14T17:11:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.027+0000 7f1c1ca0b740 1 -- 172.21.15.2:0/544737357 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587ac3618f0 msgr2=0x5587ac36c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:52.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.027+0000 7f1c1ca0b740 1 --2- 172.21.15.2:0/544737357 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587ac3618f0 0x5587ac36c440 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f1c04009a20 tx=0x7f1c04009260 comp rx=0 tx=0).stop 2024-05-14T17:11:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.027+0000 7f1c1ca0b740 1 -- 172.21.15.2:0/544737357 shutdown_connections 2024-05-14T17:11:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.027+0000 7f1c1ca0b740 1 --2- 172.21.15.2:0/544737357 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5587ac37c6f0 0x5587ac37eae0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.027+0000 7f1c1ca0b740 1 --2- 172.21.15.2:0/544737357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587ac36cad0 0x5587ac369c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:52.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.027+0000 7f1bf2ffd640 0 client.16887 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:11:52.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.027+0000 7f1c1ca0b740 1 --2- 172.21.15.2:0/544737357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587ac362210 0x5587ac369730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.027+0000 7f1c1ca0b740 1 --2- 172.21.15.2:0/544737357 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587ac3618f0 0x5587ac36c440 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:52.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.027+0000 7f1c1ca0b740 1 -- 172.21.15.2:0/544737357 >> 172.21.15.2:0/544737357 conn(0x5587ac35de00 msgr2=0x5587ac35d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.027+0000 7f1c1ca0b740 1 -- 172.21.15.2:0/544737357 shutdown_connections 2024-05-14T17:11:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.027+0000 7f1c1ca0b740 1 -- 172.21.15.2:0/544737357 wait complete. 2024-05-14T17:11:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.027+0000 7f1c1ca0b740 20 client.16887 trim_cache size 0 max 16384 2024-05-14T17:11:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.043+0000 7f1c1ca0b740 1 Processor -- start 2024-05-14T17:11:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.043+0000 7f1c1ca0b740 1 -- start start 2024-05-14T17:11:53.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.043+0000 7f1c1ca0b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587ac3618f0 0x5587ac364ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:53.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.043+0000 7f1c1ca0b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587ac362210 0x5587ac36bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:53.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.043+0000 7f1c1ca0b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587ac36cad0 0x5587ac3639b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:53.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.043+0000 7f1c1ca0b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1c141aec40 con 0x5587ac36cad0 2024-05-14T17:11:53.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.043+0000 7f1c1ca0b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1c141ac4b0 con 0x5587ac3618f0 2024-05-14T17:11:53.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.043+0000 7f1c1ca0b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1c141aa320 con 0x5587ac362210 2024-05-14T17:11:53.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.043+0000 7f1c13fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587ac36cad0 0x5587ac3639b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:53.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.043+0000 7f1c13fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587ac36cad0 0x5587ac3639b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58290/0 (socket says 172.21.15.2:58290) 2024-05-14T17:11:53.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.043+0000 7f1c13fff640 1 -- 172.21.15.2:0/1318606310 learned_addr learned my addr 172.21.15.2:0/1318606310 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:11:53.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.043+0000 7f1c0bfff640 1 --2- 172.21.15.2:0/1318606310 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587ac362210 0x5587ac36bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:53.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.043+0000 7f1c137fe640 1 --2- 172.21.15.2:0/1318606310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587ac3618f0 0x5587ac364ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:53.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.043+0000 7f1c13fff640 1 -- 172.21.15.2:0/1318606310 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587ac362210 msgr2=0x5587ac36bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.043+0000 7f1c13fff640 1 --2- 172.21.15.2:0/1318606310 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587ac362210 0x5587ac36bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.043+0000 7f1c13fff640 1 -- 172.21.15.2:0/1318606310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587ac3618f0 msgr2=0x5587ac364ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.043+0000 7f1c13fff640 1 --2- 172.21.15.2:0/1318606310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587ac3618f0 0x5587ac364ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.043+0000 7f1c13fff640 1 -- 172.21.15.2:0/1318606310 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587ac35dc30 con 0x5587ac36cad0 2024-05-14T17:11:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.043+0000 7f1c13fff640 1 --2- 172.21.15.2:0/1318606310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587ac36cad0 0x5587ac3639b0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c013410 tx=0x7f1c0c013310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea273351bea1f74 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.044+0000 7f1c097fa640 1 -- 172.21.15.2:0/1318606310 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c01c040 con 0x5587ac36cad0 2024-05-14T17:11:53.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.044+0000 7f1c097fa640 1 -- 172.21.15.2:0/1318606310 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c0c017d00 con 0x5587ac36cad0 2024-05-14T17:11:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.044+0000 7f1c097fa640 1 -- 172.21.15.2:0/1318606310 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c0189f0 con 0x5587ac36cad0 2024-05-14T17:11:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.044+0000 7f1c1ca0b740 1 -- 172.21.15.2:0/1318606310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587ac36cad0 msgr2=0x5587ac3639b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.044+0000 7f1c1ca0b740 1 --2- 172.21.15.2:0/1318606310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587ac36cad0 0x5587ac3639b0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c013410 tx=0x7f1c0c013310 comp rx=0 tx=0).stop 2024-05-14T17:11:53.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.044+0000 7f1c1ca0b740 1 -- 172.21.15.2:0/1318606310 shutdown_connections 2024-05-14T17:11:53.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.044+0000 7f1c1ca0b740 1 --2- 172.21.15.2:0/1318606310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587ac36cad0 0x5587ac3639b0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.044+0000 7f1c1ca0b740 1 --2- 172.21.15.2:0/1318606310 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587ac362210 0x5587ac36bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.044+0000 7f1c1ca0b740 1 --2- 172.21.15.2:0/1318606310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587ac3618f0 0x5587ac364ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:53.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.044+0000 7f1c1ca0b740 1 -- 172.21.15.2:0/1318606310 >> 172.21.15.2:0/1318606310 conn(0x5587ac35de00 msgr2=0x5587ac3632a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:53.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.044+0000 7f1c1ca0b740 1 -- 172.21.15.2:0/1318606310 shutdown_connections 2024-05-14T17:11:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.044+0000 7f1c1ca0b740 1 -- 172.21.15.2:0/1318606310 wait complete. 2024-05-14T17:11:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.045+0000 7f1c1ca0b740 1 -- start start 2024-05-14T17:11:54.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.045+0000 7f1c1ca0b740 1 Processor -- start 2024-05-14T17:11:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.045+0000 7f1c1ca0b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587ac3618f0 0x5587ac3664a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.045+0000 7f1c1ca0b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587ac362210 0x5587ac3669e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.045+0000 7f1c1ca0b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587ac36cad0 0x5587ac366f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.045+0000 7f1c1ca0b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1c141aa970 con 0x5587ac362210 2024-05-14T17:11:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.045+0000 7f1c1ca0b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1c141abde0 con 0x5587ac36cad0 2024-05-14T17:11:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.045+0000 7f1c1ca0b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1c141ab040 con 0x5587ac3618f0 2024-05-14T17:11:54.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.045+0000 7f1c1ca0b740 10 client.0 _ll_register_callbacks cb 0x7f1c04031430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:11:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.045+0000 7f1c0bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587ac362210 0x5587ac3669e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.045+0000 7f1c137fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587ac3618f0 0x5587ac3664a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:54.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.045+0000 7f1c0bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587ac362210 0x5587ac3669e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58294/0 (socket says 172.21.15.2:58294) 2024-05-14T17:11:54.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.045+0000 7f1c0bfff640 1 -- 172.21.15.2:0/295234676 learned_addr learned my addr 172.21.15.2:0/295234676 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:11:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.046+0000 7f1c13fff640 1 --2- 172.21.15.2:0/295234676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587ac36cad0 0x5587ac366f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.046+0000 7f1c0bfff640 1 -- 172.21.15.2:0/295234676 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587ac3618f0 msgr2=0x5587ac3664a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.046+0000 7f1c0bfff640 1 --2- 172.21.15.2:0/295234676 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587ac3618f0 0x5587ac3664a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:54.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.046+0000 7f1c0bfff640 1 -- 172.21.15.2:0/295234676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587ac36cad0 msgr2=0x5587ac366f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:54.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.046+0000 7f1c0bfff640 1 --2- 172.21.15.2:0/295234676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587ac36cad0 0x5587ac366f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.046+0000 7f1c0bfff640 1 -- 172.21.15.2:0/295234676 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587ac3e5f60 con 0x5587ac362210 2024-05-14T17:11:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.046+0000 7f1c137fe640 1 --2- 172.21.15.2:0/295234676 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587ac3618f0 0x5587ac3664a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:11:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.046+0000 7f1c0bfff640 1 --2- 172.21.15.2:0/295234676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587ac362210 0x5587ac3669e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc004fd0 tx=0x7f1bfc0a5040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b33fc79f73acd6a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:54.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.046+0000 7f1bf2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:11:54.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.046+0000 7f1bf2ffd640 1 -- 172.21.15.2:0/295234676 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc01c040 con 0x5587ac362210 2024-05-14T17:11:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.046+0000 7f1bf2ffd640 1 -- 172.21.15.2:0/295234676 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f1bfc0054f0 con 0x5587ac362210 2024-05-14T17:11:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.046+0000 7f1c1ca0b740 1 -- 172.21.15.2:0/295234676 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c0c017d00 con 0x5587ac362210 2024-05-14T17:11:55.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.046+0000 7f1bf2ffd640 1 -- 172.21.15.2:0/295234676 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f1bfc0a91b0 con 0x5587ac362210 2024-05-14T17:11:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.046+0000 7f1c1ca0b740 10 client.16893 fetch_fsmap learned FSMap version 51 2024-05-14T17:11:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.046+0000 7f1c1ca0b740 1 -- 172.21.15.2:0/295234676 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5587ac3e71e0 con 0x5587ac362210 2024-05-14T17:11:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.046+0000 7f1bf2ffd640 1 -- 172.21.15.2:0/295234676 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c0c017d00 con 0x5587ac362210 2024-05-14T17:11:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.046+0000 7f1bf2ffd640 1 -- 172.21.15.2:0/295234676 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 51) ==== 8354+0+0 (secure 0 0 0) 0x7f1bfc0a5af0 con 0x5587ac362210 2024-05-14T17:11:55.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.047+0000 7f1c1ca0b740 10 client.16893 fetch_fsmap finished waiting for FSMap version 51 2024-05-14T17:11:55.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.047+0000 7f1c1ca0b740 10 client.16893 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:11:55.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.047+0000 7f1c1ca0b740 20 client.16893 populate_metadata read hostname 'smithi002' 2024-05-14T17:11:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.047+0000 7f1c1ca0b740 1 --2- 172.21.15.2:0/295234676 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5587ac3e7970 0x5587ac3e9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.047+0000 7f1c1ca0b740 4 client.16893 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:11:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.047+0000 7f1c1ca0b740 1 -- 172.21.15.2:0/295234676 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5587ac2d0360 con 0x5587ac3e7970 2024-05-14T17:11:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.047+0000 7f1c137fe640 1 --2- 172.21.15.2:0/295234676 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5587ac3e7970 0x5587ac3e9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.047+0000 7f1bf2ffd640 10 client.16893 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:11:55.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.047+0000 7f1c137fe640 1 --2- 172.21.15.2:0/295234676 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5587ac3e7970 0x5587ac3e9d90 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e8d77a4191ed894 server_cookie=767e51d96467b729 in_seq=0 out_seq=0 2024-05-14T17:11:55.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.060+0000 7f1bf2ffd640 1 -- 172.21.15.2:0/295234676 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5587ac2d0360 con 0x5587ac3e7970 2024-05-14T17:11:55.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.060+0000 7f1bf2ffd640 10 client.16893 handle_command_reply: tid=0 2024-05-14T17:11:55.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.061+0000 7f1c1ca0b740 1 client.16893 shutdown 2024-05-14T17:11:55.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.061+0000 7f1c1ca0b740 1 -- 172.21.15.2:0/295234676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587ac362210 msgr2=0x5587ac3669e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.061+0000 7f1c1ca0b740 1 --2- 172.21.15.2:0/295234676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587ac362210 0x5587ac3669e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc004fd0 tx=0x7f1bfc0a5040 comp rx=0 tx=0).stop 2024-05-14T17:11:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.061+0000 7f1c1ca0b740 1 -- 172.21.15.2:0/295234676 shutdown_connections 2024-05-14T17:11:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.061+0000 7f1c1ca0b740 1 --2- 172.21.15.2:0/295234676 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5587ac3e7970 0x5587ac3e9d90 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:55.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.061+0000 7f1c1ca0b740 1 --2- 172.21.15.2:0/295234676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5587ac36cad0 0x5587ac366f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.061+0000 7f1c1ca0b740 1 --2- 172.21.15.2:0/295234676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587ac362210 0x5587ac3669e0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.061+0000 7f1c1ca0b740 1 --2- 172.21.15.2:0/295234676 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5587ac3618f0 0x5587ac3664a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.061+0000 7f1c1ca0b740 1 -- 172.21.15.2:0/295234676 >> 172.21.15.2:0/295234676 conn(0x5587ac35de00 msgr2=0x5587ac35d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:55.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.061+0000 7f1bf2ffd640 0 client.16893 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:11:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.061+0000 7f1c1ca0b740 1 -- 172.21.15.2:0/295234676 shutdown_connections 2024-05-14T17:11:55.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.061+0000 7f1c1ca0b740 1 -- 172.21.15.2:0/295234676 wait complete. 2024-05-14T17:11:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.061+0000 7f1c1ca0b740 20 client.16893 trim_cache size 0 max 16384 2024-05-14T17:11:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:11:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:11:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:11:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.062+0000 7f1c19c1d640 1 -- 172.21.15.2:0/4290240961 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f1c000b2a10 msgr2=0x7f1c000b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.062+0000 7f1c19c1d640 1 --2- 172.21.15.2:0/4290240961 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f1c000b2a10 0x7f1c000b4ed0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1c04005ec0 tx=0x7f1c04005a20 comp rx=0 tx=0).stop 2024-05-14T17:11:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.062+0000 7f1c19c1d640 1 -- 172.21.15.2:0/4290240961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c141033f0 msgr2=0x7f1c14197490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:56.283 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.062+0000 7f1c19c1d640 1 --2- 172.21.15.2:0/4290240961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c141033f0 0x7f1c14197490 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc00e9f0 tx=0x7f1bfc00eeb0 comp rx=0 tx=0).stop 2024-05-14T17:11:56.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.062+0000 7f1c19c1d640 1 -- 172.21.15.2:0/4290240961 shutdown_connections 2024-05-14T17:11:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.062+0000 7f1c19c1d640 1 --2- 172.21.15.2:0/4290240961 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f1c000b2a10 0x7f1c000b4ed0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:56.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.062+0000 7f1c19c1d640 1 --2- 172.21.15.2:0/4290240961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c14108a40 0x7f1c1419b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.062+0000 7f1c19c1d640 1 --2- 172.21.15.2:0/4290240961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c141033f0 0x7f1c14197490 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:56.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.062+0000 7f1c19c1d640 1 --2- 172.21.15.2:0/4290240961 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c14102a30 0x7f1c14196f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.062+0000 7f1c19c1d640 1 -- 172.21.15.2:0/4290240961 >> 172.21.15.2:0/4290240961 conn(0x7f1c140fe500 msgr2=0x7f1c140feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:56.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.062+0000 7f1c19c1d640 1 -- 172.21.15.2:0/4290240961 shutdown_connections 2024-05-14T17:11:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:25.062+0000 7f1c19c1d640 1 -- 172.21.15.2:0/4290240961 wait complete. 2024-05-14T17:11:56.593 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:10:26 smithi002 ceph-mon[31538]: pgmap v493: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 3.8 MiB/s wr, 387 op/s 2024-05-14T17:11:56.625 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:10:28 smithi002 ceph-mon[31538]: pgmap v494: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 3.3 MiB/s wr, 341 op/s 2024-05-14T17:11:56.667 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:10:30 smithi002 ceph-mon[31538]: pgmap v495: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 3.2 MiB/s wr, 340 op/s 2024-05-14T17:11:56.700 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:10:32 smithi002 ceph-mon[31538]: pgmap v496: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 818 KiB/s wr, 109 op/s 2024-05-14T17:11:56.733 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:10:34 smithi002 ceph-mon[31538]: pgmap v497: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-05-14T17:11:56.767 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:10:36 smithi002 ceph-mon[31538]: pgmap v498: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 750 KiB/s wr, 102 op/s 2024-05-14T17:11:56.801 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:11:48 smithi123 ceph-mon[35971]: pgmap v536: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-05-14T17:11:56.834 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:11:50 smithi123 ceph-mon[35971]: pgmap v537: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-05-14T17:11:56.867 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:11:52 smithi123 ceph-mon[35971]: pgmap v538: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 29 KiB/s wr, 10 op/s 2024-05-14T17:11:56.900 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:11:54 smithi123 ceph-mon[35971]: pgmap v539: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 KiB/s wr, 14 op/s 2024-05-14T17:11:56.933 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:11:54 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:11:56.967 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:11:48 smithi151 ceph-mon[36066]: pgmap v536: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-05-14T17:11:57.000 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:11:50 smithi151 ceph-mon[36066]: pgmap v537: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-05-14T17:11:57.034 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:11:52 smithi151 ceph-mon[36066]: pgmap v538: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 29 KiB/s wr, 10 op/s 2024-05-14T17:11:57.068 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:11:54 smithi151 ceph-mon[36066]: pgmap v539: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 KiB/s wr, 14 op/s 2024-05-14T17:11:57.101 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:11:54 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:11:57.136 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:11:56 smithi151 ceph-mon[36066]: pgmap v540: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 34 KiB/s wr, 11 op/s 2024-05-14T17:11:57.170 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:11:56 smithi123 ceph-mon[35971]: pgmap v540: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 34 KiB/s wr, 11 op/s 2024-05-14T17:11:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.159+0000 7f6fcce37640 1 -- 172.21.15.2:0/502999082 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6fc8102a60 msgr2=0x7f6fc8102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.159+0000 7f6fcce37640 1 --2- 172.21.15.2:0/502999082 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6fc8102a60 0x7f6fc8102ee0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0009930 tx=0x7f6fb002a420 comp rx=0 tx=0).stop 2024-05-14T17:11:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.159+0000 7f6fcce37640 1 -- 172.21.15.2:0/502999082 shutdown_connections 2024-05-14T17:11:57.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.159+0000 7f6fcce37640 1 --2- 172.21.15.2:0/502999082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fc8103420 0x7f6fc810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:57.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.159+0000 7f6fcce37640 1 --2- 172.21.15.2:0/502999082 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6fc8102a60 0x7f6fc8102ee0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.159+0000 7f6fcce37640 1 --2- 172.21.15.2:0/502999082 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6fc8108a70 0x7f6fc8108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.159+0000 7f6fcce37640 1 -- 172.21.15.2:0/502999082 >> 172.21.15.2:0/502999082 conn(0x7f6fc8079ee0 msgr2=0x7f6fc807a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:57.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.159+0000 7f6fcce37640 1 -- 172.21.15.2:0/502999082 shutdown_connections 2024-05-14T17:11:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.159+0000 7f6fcce37640 1 -- 172.21.15.2:0/502999082 wait complete. 2024-05-14T17:11:57.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.160+0000 7f6fcce37640 1 Processor -- start 2024-05-14T17:11:57.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.160+0000 7f6fcce37640 1 -- start start 2024-05-14T17:11:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.160+0000 7f6fcce37640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6fc8102a60 0x7f6fc81991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:57.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.160+0000 7f6fcce37640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6fc8103420 0x7f6fc8199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.160+0000 7f6fcce37640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fc8108a70 0x7f6fc819dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:57.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.160+0000 7f6fcce37640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc8112d30 con 0x7f6fc8103420 2024-05-14T17:11:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.160+0000 7f6fcce37640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc8112bb0 con 0x7f6fc8108a70 2024-05-14T17:11:57.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.160+0000 7f6fcce37640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc8112eb0 con 0x7f6fc8102a60 2024-05-14T17:11:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.160+0000 7f6fc5d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6fc8103420 0x7f6fc8199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:57.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.160+0000 7f6fc5d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6fc8103420 0x7f6fc8199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33866/0 (socket says 172.21.15.2:33866) 2024-05-14T17:11:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.160+0000 7f6fc5d9b640 1 -- 172.21.15.2:0/1851909841 learned_addr learned my addr 172.21.15.2:0/1851909841 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:11:58.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.160+0000 7f6fc659c640 1 --2- 172.21.15.2:0/1851909841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6fc8102a60 0x7f6fc81991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.160+0000 7f6fc6d9d640 1 --2- 172.21.15.2:0/1851909841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fc8108a70 0x7f6fc819dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.160+0000 7f6fc5d9b640 1 -- 172.21.15.2:0/1851909841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6fc8102a60 msgr2=0x7f6fc81991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:58.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.160+0000 7f6fc5d9b640 1 --2- 172.21.15.2:0/1851909841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6fc8102a60 0x7f6fc81991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.160+0000 7f6fc5d9b640 1 -- 172.21.15.2:0/1851909841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fc8108a70 msgr2=0x7f6fc819dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:58.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.160+0000 7f6fc5d9b640 1 --2- 172.21.15.2:0/1851909841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fc8108a70 0x7f6fc819dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.160+0000 7f6fc5d9b640 1 -- 172.21.15.2:0/1851909841 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fc819e1e0 con 0x7f6fc8103420 2024-05-14T17:11:58.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.160+0000 7f6fc6d9d640 1 --2- 172.21.15.2:0/1851909841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fc8108a70 0x7f6fc819dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:11:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.161+0000 7f6fc5d9b640 1 --2- 172.21.15.2:0/1851909841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6fc8103420 0x7f6fc8199730 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0005940 tx=0x7f6fb00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:58.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.161+0000 7f6fbf7fe640 1 -- 172.21.15.2:0/1851909841 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb003f030 con 0x7f6fc8103420 2024-05-14T17:11:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.161+0000 7f6fcce37640 1 -- 172.21.15.2:0/1851909841 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fc819e470 con 0x7f6fc8103420 2024-05-14T17:11:58.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.161+0000 7f6fbf7fe640 1 -- 172.21.15.2:0/1851909841 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fb0003cb0 con 0x7f6fc8103420 2024-05-14T17:11:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.161+0000 7f6fbf7fe640 1 -- 172.21.15.2:0/1851909841 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0035070 con 0x7f6fc8103420 2024-05-14T17:11:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.161+0000 7f6fcce37640 1 -- 172.21.15.2:0/1851909841 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fc81a5d60 con 0x7f6fc8103420 2024-05-14T17:11:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.163+0000 7f6fbf7fe640 1 -- 172.21.15.2:0/1851909841 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f6fb0042030 con 0x7f6fc8103420 2024-05-14T17:11:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.164+0000 7f6fcfc2b740 1 Processor -- start 2024-05-14T17:11:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.164+0000 7f6fcfc2b740 1 -- start start 2024-05-14T17:11:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.164+0000 7f6fcfc2b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c1dde8f0 0x55f7c1ddecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.164+0000 7f6fcfc2b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7c1ddf210 0x55f7c1de9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.164+0000 7f6fcfc2b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7c1de9ad0 0x55f7c1debec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.164+0000 7f6fcfc2b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f7c1d51c30 con 0x55f7c1de9ad0 2024-05-14T17:11:58.819 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.164+0000 7f6fcfc2b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f7c1d417d0 con 0x55f7c1ddf210 2024-05-14T17:11:58.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.164+0000 7f6fcfc2b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f7c1d45810 con 0x55f7c1dde8f0 2024-05-14T17:11:58.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.164+0000 7f6fc5d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7c1de9ad0 0x55f7c1debec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.165+0000 7f6fc5d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7c1de9ad0 0x55f7c1debec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33870/0 (socket says 172.21.15.2:33870) 2024-05-14T17:11:58.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.165+0000 7f6fc5d9b640 1 -- 172.21.15.2:0/2755944582 learned_addr learned my addr 172.21.15.2:0/2755944582 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:11:58.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.165+0000 7f6fc659c640 1 --2- 172.21.15.2:0/2755944582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7c1ddf210 0x55f7c1de9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.165+0000 7f6fc6d9d640 1 --2- 172.21.15.2:0/2755944582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c1dde8f0 0x55f7c1ddecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.165+0000 7f6fc5d9b640 1 -- 172.21.15.2:0/2755944582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c1dde8f0 msgr2=0x55f7c1ddecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.165+0000 7f6fc5d9b640 1 --2- 172.21.15.2:0/2755944582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c1dde8f0 0x55f7c1ddecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.165+0000 7f6fc5d9b640 1 -- 172.21.15.2:0/2755944582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7c1ddf210 msgr2=0x55f7c1de9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:11:59.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.165+0000 7f6fc5d9b640 1 --2- 172.21.15.2:0/2755944582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7c1ddf210 0x55f7c1de9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.165+0000 7f6fbf7fe640 1 --2- 172.21.15.2:0/1851909841 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f6fb40b2a10 0x7f6fb40b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.165+0000 7f6fc5d9b640 1 -- 172.21.15.2:0/2755944582 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7c1a56210 con 0x55f7c1de9ad0 2024-05-14T17:11:59.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.165+0000 7f6fc6d9d640 1 --2- 172.21.15.2:0/2755944582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c1dde8f0 0x55f7c1ddecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:11:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.165+0000 7f6fc6d9d640 1 --2- 172.21.15.2:0/1851909841 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f6fb40b2a10 0x7f6fb40b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:11:59.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.165+0000 7f6fc5d9b640 1 --2- 172.21.15.2:0/2755944582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7c1de9ad0 0x55f7c1debec0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f6fb01001c0 tx=0x7f6fb00c6520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4079326127d17bc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.165+0000 7f6fbf7fe640 1 -- 172.21.15.2:0/1851909841 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f6fb0102b90 con 0x7f6fc8103420 2024-05-14T17:11:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.165+0000 7f6fbcff9640 1 -- 172.21.15.2:0/2755944582 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0049070 con 0x55f7c1de9ad0 2024-05-14T17:11:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.165+0000 7f6fbcff9640 1 -- 172.21.15.2:0/2755944582 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fb00d5ac0 con 0x55f7c1de9ad0 2024-05-14T17:11:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.165+0000 7f6fbcff9640 1 -- 172.21.15.2:0/2755944582 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0107040 con 0x55f7c1de9ad0 2024-05-14T17:11:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.166+0000 7f6fcfc2b740 1 -- 172.21.15.2:0/2755944582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7c1de9ad0 msgr2=0x55f7c1debec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:11:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.166+0000 7f6fc6d9d640 1 --2- 172.21.15.2:0/1851909841 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f6fb40b2a10 0x7f6fb40b4ed0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8005fd0 tx=0x7f6fb8009500 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:11:59.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.166+0000 7f6fcfc2b740 1 --2- 172.21.15.2:0/2755944582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7c1de9ad0 0x55f7c1debec0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f6fb01001c0 tx=0x7f6fb00c6520 comp rx=0 tx=0).stop 2024-05-14T17:11:59.628 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.166+0000 7f6fcfc2b740 1 -- 172.21.15.2:0/2755944582 shutdown_connections 2024-05-14T17:11:59.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.166+0000 7f6fcfc2b740 1 --2- 172.21.15.2:0/2755944582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7c1de9ad0 0x55f7c1debec0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.166+0000 7f6fcfc2b740 1 --2- 172.21.15.2:0/2755944582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7c1ddf210 0x55f7c1de9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.166+0000 7f6fcfc2b740 1 --2- 172.21.15.2:0/2755944582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c1dde8f0 0x55f7c1ddecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:11:59.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.166+0000 7f6fcfc2b740 1 -- 172.21.15.2:0/2755944582 >> 172.21.15.2:0/2755944582 conn(0x55f7c1ddae00 msgr2=0x55f7c1ddb1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:11:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.166+0000 7f6fcfc2b740 1 -- 172.21.15.2:0/2755944582 shutdown_connections 2024-05-14T17:11:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.166+0000 7f6fcfc2b740 1 -- 172.21.15.2:0/2755944582 wait complete. 2024-05-14T17:11:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.167+0000 7f6fcfc2b740 1 -- start start 2024-05-14T17:11:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.167+0000 7f6fcfc2b740 1 Processor -- start 2024-05-14T17:11:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.167+0000 7f6fcfc2b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c1dde8f0 0x55f7c1de94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:11:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.167+0000 7f6fcfc2b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7c1ddf210 0x55f7c1de6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.167+0000 7f6fcfc2b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7c1de9ad0 0x55f7c1de6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:00.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.167+0000 7f6fcfc2b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f7c1a27790 con 0x55f7c1ddf210 2024-05-14T17:12:00.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.167+0000 7f6fcfc2b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f7c1cfd890 con 0x55f7c1de9ad0 2024-05-14T17:12:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.167+0000 7f6fcfc2b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f7c1dec4c0 con 0x55f7c1dde8f0 2024-05-14T17:12:00.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.167+0000 7f6fcfc2b740 10 client.0 _ll_register_callbacks cb 0x55f7c1d00d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:12:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.167+0000 7f6fc5d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7c1ddf210 0x55f7c1de6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.167+0000 7f6fc5d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7c1ddf210 0x55f7c1de6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33884/0 (socket says 172.21.15.2:33884) 2024-05-14T17:12:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.167+0000 7f6fc5d9b640 1 -- 172.21.15.2:0/1742349873 learned_addr learned my addr 172.21.15.2:0/1742349873 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:12:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.167+0000 7f6fc6d9d640 1 --2- 172.21.15.2:0/1742349873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7c1de9ad0 0x55f7c1de6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:00.320 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.167+0000 7f6fc659c640 1 --2- 172.21.15.2:0/1742349873 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c1dde8f0 0x55f7c1de94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:00.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.167+0000 7f6fc5d9b640 1 -- 172.21.15.2:0/1742349873 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c1dde8f0 msgr2=0x55f7c1de94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.167+0000 7f6fc5d9b640 1 --2- 172.21.15.2:0/1742349873 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c1dde8f0 0x55f7c1de94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.167+0000 7f6fc5d9b640 1 -- 172.21.15.2:0/1742349873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7c1de9ad0 msgr2=0x55f7c1de6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:00.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.167+0000 7f6fc5d9b640 1 --2- 172.21.15.2:0/1742349873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7c1de9ad0 0x55f7c1de6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:00.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.167+0000 7f6fc5d9b640 1 -- 172.21.15.2:0/1742349873 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7c1b22620 con 0x55f7c1ddf210 2024-05-14T17:12:00.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.167+0000 7f6fc6d9d640 1 --2- 172.21.15.2:0/1742349873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7c1de9ad0 0x55f7c1de6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:12:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.168+0000 7f6fc5d9b640 1 --2- 172.21.15.2:0/1742349873 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7c1ddf210 0x55f7c1de6790 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6fb003d040 tx=0x7f6fb0106a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5944e9d6157d99 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:00.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.168+0000 7f6fadffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:12:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.168+0000 7f6fadffb640 1 -- 172.21.15.2:0/1742349873 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0049050 con 0x55f7c1ddf210 2024-05-14T17:12:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.168+0000 7f6fadffb640 1 -- 172.21.15.2:0/1742349873 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f6fb0100500 con 0x55f7c1ddf210 2024-05-14T17:12:00.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.168+0000 7f6fcfc2b740 1 -- 172.21.15.2:0/1742349873 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f7c1d53b80 con 0x55f7c1ddf210 2024-05-14T17:12:00.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.168+0000 7f6fadffb640 1 -- 172.21.15.2:0/1742349873 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f6fb00c9be0 con 0x55f7c1ddf210 2024-05-14T17:12:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.168+0000 7f6fcfc2b740 10 client.16905 fetch_fsmap learned FSMap version 51 2024-05-14T17:12:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.168+0000 7f6fcfc2b740 1 -- 172.21.15.2:0/1742349873 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55f7c1df7f40 con 0x55f7c1ddf210 2024-05-14T17:12:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.168+0000 7f6fadffb640 1 -- 172.21.15.2:0/1742349873 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f7c1d53b80 con 0x55f7c1ddf210 2024-05-14T17:12:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.168+0000 7f6fadffb640 1 -- 172.21.15.2:0/1742349873 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 51) ==== 8354+0+0 (secure 0 0 0) 0x7f6fb00ca070 con 0x55f7c1ddf210 2024-05-14T17:12:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.168+0000 7f6fcfc2b740 10 client.16905 fetch_fsmap finished waiting for FSMap version 51 2024-05-14T17:12:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.168+0000 7f6fcfc2b740 10 client.16905 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:12:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.168+0000 7f6fcfc2b740 20 client.16905 populate_metadata read hostname 'smithi002' 2024-05-14T17:12:01.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.168+0000 7f6fcfc2b740 1 --2- 172.21.15.2:0/1742349873 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55f7c1df9360 0x55f7c1dfb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.168+0000 7f6fcfc2b740 4 client.16905 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:12:01.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.168+0000 7f6fcfc2b740 1 -- 172.21.15.2:0/1742349873 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7c1d4d360 con 0x55f7c1df9360 2024-05-14T17:12:01.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.169+0000 7f6fc659c640 1 --2- 172.21.15.2:0/1742349873 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55f7c1df9360 0x55f7c1dfb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.169+0000 7f6fadffb640 10 client.16905 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:12:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.169+0000 7f6fc659c640 1 --2- 172.21.15.2:0/1742349873 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55f7c1df9360 0x55f7c1dfb7a0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b213c74d98b708c1 server_cookie=eddff975df69443f in_seq=0 out_seq=0 2024-05-14T17:12:01.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.170+0000 7f6fadffb640 1 -- 172.21.15.2:0/1742349873 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f7c1d4d360 con 0x55f7c1df9360 2024-05-14T17:12:01.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.170+0000 7f6fadffb640 10 client.16905 handle_command_reply: tid=0 2024-05-14T17:12:01.330 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.170+0000 7f6fcfc2b740 1 client.16905 shutdown 2024-05-14T17:12:01.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.170+0000 7f6fcfc2b740 1 -- 172.21.15.2:0/1742349873 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7c1ddf210 msgr2=0x55f7c1de6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.170+0000 7f6fcfc2b740 1 --2- 172.21.15.2:0/1742349873 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7c1ddf210 0x55f7c1de6790 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6fb003d040 tx=0x7f6fb0106a10 comp rx=0 tx=0).stop 2024-05-14T17:12:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.170+0000 7f6fcfc2b740 1 -- 172.21.15.2:0/1742349873 shutdown_connections 2024-05-14T17:12:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.170+0000 7f6fcfc2b740 1 --2- 172.21.15.2:0/1742349873 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55f7c1df9360 0x55f7c1dfb7a0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.170+0000 7f6fcfc2b740 1 --2- 172.21.15.2:0/1742349873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7c1de9ad0 0x55f7c1de6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.170+0000 7f6fcfc2b740 1 --2- 172.21.15.2:0/1742349873 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7c1ddf210 0x55f7c1de6790 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.170+0000 7f6fadffb640 0 client.16905 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:12:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.170+0000 7f6fcfc2b740 1 --2- 172.21.15.2:0/1742349873 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c1dde8f0 0x55f7c1de94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:01.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.170+0000 7f6fcfc2b740 1 -- 172.21.15.2:0/1742349873 >> 172.21.15.2:0/1742349873 conn(0x55f7c1ddae00 msgr2=0x55f7c1dda100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.170+0000 7f6fcfc2b740 1 -- 172.21.15.2:0/1742349873 shutdown_connections 2024-05-14T17:12:01.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.170+0000 7f6fcfc2b740 1 -- 172.21.15.2:0/1742349873 wait complete. 2024-05-14T17:12:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.170+0000 7f6fcfc2b740 20 client.16905 trim_cache size 0 max 16384 2024-05-14T17:12:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.185+0000 7f6fcfc2b740 1 Processor -- start 2024-05-14T17:12:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.185+0000 7f6fcfc2b740 1 -- start start 2024-05-14T17:12:01.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.186+0000 7f6fcfc2b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7c1dde8f0 0x55f7c1e01070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.186+0000 7f6fcfc2b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7c1ddf210 0x55f7c1e015b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:01.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.186+0000 7f6fcfc2b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c1de9ad0 0x55f7c1df5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:01.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.186+0000 7f6fcfc2b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc81b0fa0 con 0x55f7c1ddf210 2024-05-14T17:12:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.186+0000 7f6fcfc2b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc81ae810 con 0x55f7c1dde8f0 2024-05-14T17:12:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.186+0000 7f6fcfc2b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc81ac680 con 0x55f7c1de9ad0 2024-05-14T17:12:02.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.186+0000 7f6fc5d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7c1ddf210 0x55f7c1e015b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.186+0000 7f6fc5d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7c1ddf210 0x55f7c1e015b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33900/0 (socket says 172.21.15.2:33900) 2024-05-14T17:12:02.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.186+0000 7f6fc5d9b640 1 -- 172.21.15.2:0/2929580712 learned_addr learned my addr 172.21.15.2:0/2929580712 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:12:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.186+0000 7f6fc659c640 1 --2- 172.21.15.2:0/2929580712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7c1dde8f0 0x55f7c1e01070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.186+0000 7f6fc6d9d640 1 --2- 172.21.15.2:0/2929580712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c1de9ad0 0x55f7c1df5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:02.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.186+0000 7f6fc5d9b640 1 -- 172.21.15.2:0/2929580712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c1de9ad0 msgr2=0x55f7c1df5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:02.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.186+0000 7f6fc5d9b640 1 --2- 172.21.15.2:0/2929580712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c1de9ad0 0x55f7c1df5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:02.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.186+0000 7f6fc5d9b640 1 -- 172.21.15.2:0/2929580712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7c1dde8f0 msgr2=0x55f7c1e01070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.186+0000 7f6fc5d9b640 1 --2- 172.21.15.2:0/2929580712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7c1dde8f0 0x55f7c1e01070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.186+0000 7f6fc5d9b640 1 -- 172.21.15.2:0/2929580712 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7c1de7110 con 0x55f7c1ddf210 2024-05-14T17:12:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.186+0000 7f6fc5d9b640 1 --2- 172.21.15.2:0/2929580712 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7c1ddf210 0x55f7c1e015b0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0009900 tx=0x7f6fb0106a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2917cafb8dc00ca6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.186+0000 7f6faffff640 1 -- 172.21.15.2:0/2929580712 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0049070 con 0x55f7c1ddf210 2024-05-14T17:12:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.186+0000 7f6faffff640 1 -- 172.21.15.2:0/2929580712 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fb00d2c70 con 0x55f7c1ddf210 2024-05-14T17:12:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.186+0000 7f6faffff640 1 -- 172.21.15.2:0/2929580712 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0003e60 con 0x55f7c1ddf210 2024-05-14T17:12:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.187+0000 7f6fcfc2b740 1 -- 172.21.15.2:0/2929580712 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7c1ddf210 msgr2=0x55f7c1e015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:02.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.187+0000 7f6fcfc2b740 1 --2- 172.21.15.2:0/2929580712 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7c1ddf210 0x55f7c1e015b0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0009900 tx=0x7f6fb0106a80 comp rx=0 tx=0).stop 2024-05-14T17:12:02.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.187+0000 7f6fcfc2b740 1 -- 172.21.15.2:0/2929580712 shutdown_connections 2024-05-14T17:12:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.187+0000 7f6fcfc2b740 1 --2- 172.21.15.2:0/2929580712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c1de9ad0 0x55f7c1df5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.187+0000 7f6fcfc2b740 1 --2- 172.21.15.2:0/2929580712 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7c1ddf210 0x55f7c1e015b0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.187+0000 7f6fcfc2b740 1 --2- 172.21.15.2:0/2929580712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7c1dde8f0 0x55f7c1e01070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:02.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.187+0000 7f6fcfc2b740 1 -- 172.21.15.2:0/2929580712 >> 172.21.15.2:0/2929580712 conn(0x55f7c1ddae00 msgr2=0x55f7c1df0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.187+0000 7f6fcfc2b740 1 -- 172.21.15.2:0/2929580712 shutdown_connections 2024-05-14T17:12:02.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.187+0000 7f6fcfc2b740 1 -- 172.21.15.2:0/2929580712 wait complete. 2024-05-14T17:12:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.187+0000 7f6fcfc2b740 1 -- start start 2024-05-14T17:12:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.187+0000 7f6fcfc2b740 1 Processor -- start 2024-05-14T17:12:02.939 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.188+0000 7f6fcfc2b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7c1dde8f0 0x55f7c1e602f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:02.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.188+0000 7f6fcfc2b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c1ddf210 0x55f7c1e60830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.188+0000 7f6fcfc2b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7c1de9ad0 0x55f7c1e5a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.188+0000 7f6fcfc2b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc81accd0 con 0x55f7c1de9ad0 2024-05-14T17:12:03.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.188+0000 7f6fcfc2b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc81ae140 con 0x55f7c1dde8f0 2024-05-14T17:12:03.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.188+0000 7f6fcfc2b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc81ad3a0 con 0x55f7c1ddf210 2024-05-14T17:12:03.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.188+0000 7f6fcfc2b740 10 client.0 _ll_register_callbacks cb 0x7f6fb0106b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:12:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.188+0000 7f6fc6d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7c1de9ad0 0x55f7c1e5a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.188+0000 7f6fc6d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7c1de9ad0 0x55f7c1e5a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33908/0 (socket says 172.21.15.2:33908) 2024-05-14T17:12:03.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.188+0000 7f6fc6d9d640 1 -- 172.21.15.2:0/1117333335 learned_addr learned my addr 172.21.15.2:0/1117333335 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:12:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.188+0000 7f6fc5d9b640 1 --2- 172.21.15.2:0/1117333335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c1ddf210 0x55f7c1e60830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.188+0000 7f6fc659c640 1 --2- 172.21.15.2:0/1117333335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7c1dde8f0 0x55f7c1e602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.188+0000 7f6fc6d9d640 1 -- 172.21.15.2:0/1117333335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c1ddf210 msgr2=0x55f7c1e60830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.188+0000 7f6fc6d9d640 1 --2- 172.21.15.2:0/1117333335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c1ddf210 0x55f7c1e60830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:03.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.188+0000 7f6fc6d9d640 1 -- 172.21.15.2:0/1117333335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7c1dde8f0 msgr2=0x55f7c1e602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.188+0000 7f6fc6d9d640 1 --2- 172.21.15.2:0/1117333335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7c1dde8f0 0x55f7c1e602f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:03.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.188+0000 7f6fc6d9d640 1 -- 172.21.15.2:0/1117333335 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7c1e5adf0 con 0x55f7c1de9ad0 2024-05-14T17:12:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.188+0000 7f6fc6d9d640 1 --2- 172.21.15.2:0/1117333335 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7c1de9ad0 0x55f7c1e5a700 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8011be0 tx=0x7f6fb8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfbcc1cd03c2554 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.188+0000 7f6fadffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:12:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.188+0000 7f6fadffb640 1 -- 172.21.15.2:0/1117333335 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb8004b20 con 0x55f7c1de9ad0 2024-05-14T17:12:03.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.189+0000 7f6fadffb640 1 -- 172.21.15.2:0/1117333335 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f6fb8004cc0 con 0x55f7c1de9ad0 2024-05-14T17:12:03.665 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.189+0000 7f6fcfc2b740 1 -- 172.21.15.2:0/1117333335 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fb00d2c70 con 0x55f7c1de9ad0 2024-05-14T17:12:03.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.189+0000 7f6fadffb640 1 -- 172.21.15.2:0/1117333335 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f6fb8035050 con 0x55f7c1de9ad0 2024-05-14T17:12:03.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.189+0000 7f6fcfc2b740 10 client.16911 fetch_fsmap learned FSMap version 51 2024-05-14T17:12:03.765 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.189+0000 7f6fcfc2b740 1 -- 172.21.15.2:0/1117333335 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55f7c1e67b60 con 0x55f7c1de9ad0 2024-05-14T17:12:03.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.189+0000 7f6fadffb640 1 -- 172.21.15.2:0/1117333335 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fb00d2c70 con 0x55f7c1de9ad0 2024-05-14T17:12:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.189+0000 7f6fadffb640 1 -- 172.21.15.2:0/1117333335 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 51) ==== 8354+0+0 (secure 0 0 0) 0x7f6fb8005460 con 0x55f7c1de9ad0 2024-05-14T17:12:03.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.189+0000 7f6fcfc2b740 10 client.16911 fetch_fsmap finished waiting for FSMap version 51 2024-05-14T17:12:03.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.189+0000 7f6fcfc2b740 10 client.16911 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:12:03.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.189+0000 7f6fcfc2b740 20 client.16911 populate_metadata read hostname 'smithi002' 2024-05-14T17:12:03.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.189+0000 7f6fcfc2b740 1 --2- 172.21.15.2:0/1117333335 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55f7c1e683c0 0x55f7c1e6a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.189+0000 7f6fcfc2b740 4 client.16911 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:12:04.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.189+0000 7f6fcfc2b740 1 -- 172.21.15.2:0/1117333335 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f7c1cc75f0 con 0x55f7c1e683c0 2024-05-14T17:12:04.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.189+0000 7f6fc659c640 1 --2- 172.21.15.2:0/1117333335 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55f7c1e683c0 0x55f7c1e6a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.190+0000 7f6fc659c640 1 --2- 172.21.15.2:0/1117333335 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55f7c1e683c0 0x55f7c1e6a7e0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce7f944c79c767eb server_cookie=797f4638bfbb3c59 in_seq=0 out_seq=0 2024-05-14T17:12:04.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.190+0000 7f6fadffb640 10 client.16911 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:12:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.205+0000 7f6fadffb640 1 -- 172.21.15.2:0/1117333335 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f7c1cc75f0 con 0x55f7c1e683c0 2024-05-14T17:12:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.205+0000 7f6fadffb640 10 client.16911 handle_command_reply: tid=0 2024-05-14T17:12:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.205+0000 7f6fcfc2b740 1 client.16911 shutdown 2024-05-14T17:12:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.206+0000 7f6fcfc2b740 1 -- 172.21.15.2:0/1117333335 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7c1de9ad0 msgr2=0x55f7c1e5a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.206+0000 7f6fcfc2b740 1 --2- 172.21.15.2:0/1117333335 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7c1de9ad0 0x55f7c1e5a700 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8011be0 tx=0x7f6fb8004a40 comp rx=0 tx=0).stop 2024-05-14T17:12:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.206+0000 7f6fcfc2b740 1 -- 172.21.15.2:0/1117333335 shutdown_connections 2024-05-14T17:12:04.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.206+0000 7f6fcfc2b740 1 --2- 172.21.15.2:0/1117333335 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55f7c1e683c0 0x55f7c1e6a7e0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.206+0000 7f6fcfc2b740 1 --2- 172.21.15.2:0/1117333335 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f7c1de9ad0 0x55f7c1e5a700 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.206+0000 7f6fcfc2b740 1 --2- 172.21.15.2:0/1117333335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f7c1ddf210 0x55f7c1e60830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.206+0000 7f6fcfc2b740 1 --2- 172.21.15.2:0/1117333335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7c1dde8f0 0x55f7c1e602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.206+0000 7f6fcfc2b740 1 -- 172.21.15.2:0/1117333335 >> 172.21.15.2:0/1117333335 conn(0x55f7c1ddae00 msgr2=0x55f7c1df1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.206+0000 7f6fadffb640 0 client.16911 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:12:04.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.206+0000 7f6fcfc2b740 1 -- 172.21.15.2:0/1117333335 shutdown_connections 2024-05-14T17:12:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.206+0000 7f6fcfc2b740 1 -- 172.21.15.2:0/1117333335 wait complete. 2024-05-14T17:12:04.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.206+0000 7f6fcfc2b740 20 client.16911 trim_cache size 0 max 16384 2024-05-14T17:12:04.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:12:04.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:12:04.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:12:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.207+0000 7f6fcce37640 1 -- 172.21.15.2:0/1851909841 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f6fb40b2a10 msgr2=0x7f6fb40b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.207+0000 7f6fcce37640 1 --2- 172.21.15.2:0/1851909841 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f6fb40b2a10 0x7f6fb40b4ed0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8005fd0 tx=0x7f6fb8009500 comp rx=0 tx=0).stop 2024-05-14T17:12:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.207+0000 7f6fcce37640 1 -- 172.21.15.2:0/1851909841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6fc8103420 msgr2=0x7f6fc8199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:05.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.207+0000 7f6fcce37640 1 --2- 172.21.15.2:0/1851909841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6fc8103420 0x7f6fc8199730 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0005940 tx=0x7f6fb00042f0 comp rx=0 tx=0).stop 2024-05-14T17:12:05.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.207+0000 7f6fcce37640 1 -- 172.21.15.2:0/1851909841 shutdown_connections 2024-05-14T17:12:05.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.207+0000 7f6fcce37640 1 --2- 172.21.15.2:0/1851909841 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f6fb40b2a10 0x7f6fb40b4ed0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.207+0000 7f6fcce37640 1 --2- 172.21.15.2:0/1851909841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fc8108a70 0x7f6fc819dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.207+0000 7f6fcce37640 1 --2- 172.21.15.2:0/1851909841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6fc8103420 0x7f6fc8199730 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:05.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.207+0000 7f6fcce37640 1 --2- 172.21.15.2:0/1851909841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6fc8102a60 0x7f6fc81991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:05.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.207+0000 7f6fcce37640 1 -- 172.21.15.2:0/1851909841 >> 172.21.15.2:0/1851909841 conn(0x7f6fc8079ee0 msgr2=0x7f6fc8100340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:05.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.207+0000 7f6fcce37640 1 -- 172.21.15.2:0/1851909841 shutdown_connections 2024-05-14T17:12:05.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:38.207+0000 7f6fcce37640 1 -- 172.21.15.2:0/1851909841 wait complete. 2024-05-14T17:12:05.351 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:10:38 smithi002 ceph-mon[31538]: pgmap v499: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 767 KiB/s wr, 104 op/s 2024-05-14T17:12:05.383 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:10:40 smithi002 ceph-mon[31538]: pgmap v500: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 1021 KiB/s wr, 140 op/s 2024-05-14T17:12:05.416 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:10:42 smithi002 ceph-mon[31538]: pgmap v501: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 721 KiB/s wr, 96 op/s 2024-05-14T17:12:05.450 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:10:43 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:12:05.483 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:10:44 smithi002 ceph-mon[31538]: pgmap v502: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 1.8 MiB/s wr, 173 op/s 2024-05-14T17:12:05.517 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:10:46 smithi002 ceph-mon[31538]: pgmap v503: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 1.5 MiB/s wr, 134 op/s 2024-05-14T17:12:05.550 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:10:48 smithi002 ceph-mon[31538]: pgmap v504: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 1.5 MiB/s wr, 129 op/s 2024-05-14T17:12:05.583 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:10:50 smithi002 ceph-mon[31538]: pgmap v505: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 1.4 MiB/s wr, 126 op/s 2024-05-14T17:12:05.618 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:11:58 smithi123 ceph-mon[35971]: pgmap v541: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 34 KiB/s wr, 11 op/s 2024-05-14T17:12:05.650 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:12:00 smithi123 ceph-mon[35971]: pgmap v542: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 53 KiB/s wr, 17 op/s 2024-05-14T17:12:05.683 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:12:02 smithi123 ceph-mon[35971]: pgmap v543: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 30 KiB/s wr, 10 op/s 2024-05-14T17:12:05.727 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:12:04 smithi123 ceph-mon[35971]: pgmap v544: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 75 KiB/s wr, 30 op/s 2024-05-14T17:12:05.758 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:11:58 smithi151 ceph-mon[36066]: pgmap v541: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 34 KiB/s wr, 11 op/s 2024-05-14T17:12:05.800 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:12:00 smithi151 ceph-mon[36066]: pgmap v542: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 53 KiB/s wr, 17 op/s 2024-05-14T17:12:05.833 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:12:02 smithi151 ceph-mon[36066]: pgmap v543: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 30 KiB/s wr, 10 op/s 2024-05-14T17:12:05.878 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:12:04 smithi151 ceph-mon[36066]: pgmap v544: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 75 KiB/s wr, 30 op/s 2024-05-14T17:12:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.308+0000 7f6b002c8640 1 -- 172.21.15.2:0/64509519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6af8103410 msgr2=0x7f6af810f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.308+0000 7f6b002c8640 1 --2- 172.21.15.2:0/64509519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6af8103410 0x7f6af810f920 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f6af0009cd0 tx=0x7f6af002a8e0 comp rx=0 tx=0).stop 2024-05-14T17:12:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.309+0000 7f6b002c8640 1 -- 172.21.15.2:0/64509519 shutdown_connections 2024-05-14T17:12:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.309+0000 7f6b002c8640 1 --2- 172.21.15.2:0/64509519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6af8103410 0x7f6af810f920 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.309+0000 7f6b002c8640 1 --2- 172.21.15.2:0/64509519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6af8102a50 0x7f6af8102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.309+0000 7f6b002c8640 1 --2- 172.21.15.2:0/64509519 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6af8108a60 0x7f6af8108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.309+0000 7f6b002c8640 1 -- 172.21.15.2:0/64509519 >> 172.21.15.2:0/64509519 conn(0x7f6af80fe520 msgr2=0x7f6af8100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.309+0000 7f6b002c8640 1 -- 172.21.15.2:0/64509519 shutdown_connections 2024-05-14T17:12:06.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.309+0000 7f6b002c8640 1 -- 172.21.15.2:0/64509519 wait complete. 2024-05-14T17:12:06.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.309+0000 7f6b002c8640 1 Processor -- start 2024-05-14T17:12:06.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.309+0000 7f6b002c8640 1 -- start start 2024-05-14T17:12:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.309+0000 7f6b002c8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6af8102a50 0x7f6af8074d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:06.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.310+0000 7f6b002c8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6af8103410 0x7f6af8075290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.310+0000 7f6b002c8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6af8108a60 0x7f6af80757d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.310+0000 7f6b002c8640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6af8112cf0 con 0x7f6af8108a60 2024-05-14T17:12:06.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.310+0000 7f6b002c8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6af8112b70 con 0x7f6af8103410 2024-05-14T17:12:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.310+0000 7f6b002c8640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6af8112e70 con 0x7f6af8102a50 2024-05-14T17:12:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.310+0000 7f6afe865640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6af8108a60 0x7f6af80757d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:06.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.310+0000 7f6afe865640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6af8108a60 0x7f6af80757d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60986/0 (socket says 172.21.15.2:60986) 2024-05-14T17:12:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.310+0000 7f6afe865640 1 -- 172.21.15.2:0/2958010502 learned_addr learned my addr 172.21.15.2:0/2958010502 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:12:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.310+0000 7f6afe064640 1 --2- 172.21.15.2:0/2958010502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6af8102a50 0x7f6af8074d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.310+0000 7f6afd863640 1 --2- 172.21.15.2:0/2958010502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6af8103410 0x7f6af8075290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:06.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.310+0000 7f6afe865640 1 -- 172.21.15.2:0/2958010502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6af8102a50 msgr2=0x7f6af8074d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:06.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.310+0000 7f6afe865640 1 --2- 172.21.15.2:0/2958010502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6af8102a50 0x7f6af8074d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.310+0000 7f6afe865640 1 -- 172.21.15.2:0/2958010502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6af8103410 msgr2=0x7f6af8075290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.310+0000 7f6afe865640 1 --2- 172.21.15.2:0/2958010502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6af8103410 0x7f6af8075290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.310+0000 7f6afe865640 1 -- 172.21.15.2:0/2958010502 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af81a7610 con 0x7f6af8108a60 2024-05-14T17:12:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.310+0000 7f6afd863640 1 --2- 172.21.15.2:0/2958010502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6af8103410 0x7f6af8075290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:12:06.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.310+0000 7f6afe865640 1 --2- 172.21.15.2:0/2958010502 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6af8108a60 0x7f6af80757d0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f6af0033040 tx=0x7f6af0003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.310+0000 7f6aef7fe640 1 -- 172.21.15.2:0/2958010502 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af0004040 con 0x7f6af8108a60 2024-05-14T17:12:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.311+0000 7f6aef7fe640 1 -- 172.21.15.2:0/2958010502 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6af0004690 con 0x7f6af8108a60 2024-05-14T17:12:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.311+0000 7f6b002c8640 1 -- 172.21.15.2:0/2958010502 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6af81a78a0 con 0x7f6af8108a60 2024-05-14T17:12:07.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.311+0000 7f6aef7fe640 1 -- 172.21.15.2:0/2958010502 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af0030e20 con 0x7f6af8108a60 2024-05-14T17:12:07.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.311+0000 7f6b002c8640 1 -- 172.21.15.2:0/2958010502 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6af81a7df0 con 0x7f6af8108a60 2024-05-14T17:12:07.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.313+0000 7f6b02fe6740 1 Processor -- start 2024-05-14T17:12:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.313+0000 7f6aef7fe640 1 -- 172.21.15.2:0/2958010502 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f6af00305a0 con 0x7f6af8108a60 2024-05-14T17:12:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.314+0000 7f6b02fe6740 1 -- start start 2024-05-14T17:12:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.314+0000 7f6b02fe6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55618f1ac8f0 0x55618f1accd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:07.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.314+0000 7f6b02fe6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55618f1ad210 0x55618f1b7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:07.401 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.314+0000 7f6b02fe6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55618f1b7ad0 0x55618f1b9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:07.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.314+0000 7f6b02fe6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55618f11fc30 con 0x55618f1ad210 2024-05-14T17:12:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.314+0000 7f6b02fe6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55618f10f7d0 con 0x55618f1b7ad0 2024-05-14T17:12:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.314+0000 7f6b02fe6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55618f113810 con 0x55618f1ac8f0 2024-05-14T17:12:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.314+0000 7f6afe064640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55618f1ad210 0x55618f1b7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.314+0000 7f6afe064640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55618f1ad210 0x55618f1b7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60994/0 (socket says 172.21.15.2:60994) 2024-05-14T17:12:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.314+0000 7f6afe064640 1 -- 172.21.15.2:0/892629878 learned_addr learned my addr 172.21.15.2:0/892629878 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:12:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.314+0000 7f6afd863640 1 --2- 172.21.15.2:0/892629878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55618f1b7ad0 0x55618f1b9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.314+0000 7f6afd863640 1 --2- 172.21.15.2:0/892629878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55618f1ac8f0 0x55618f1accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.315+0000 7f6afe064640 1 -- 172.21.15.2:0/892629878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55618f1ac8f0 msgr2=0x55618f1accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.315+0000 7f6afe064640 1 --2- 172.21.15.2:0/892629878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55618f1ac8f0 0x55618f1accd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:07.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.315+0000 7f6afe064640 1 -- 172.21.15.2:0/892629878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55618f1b7ad0 msgr2=0x55618f1b9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.315+0000 7f6afe064640 1 --2- 172.21.15.2:0/892629878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55618f1b7ad0 0x55618f1b9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:07.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.315+0000 7f6afe064640 1 -- 172.21.15.2:0/892629878 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55618ee24210 con 0x55618f1ad210 2024-05-14T17:12:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.315+0000 7f6afd863640 1 --2- 172.21.15.2:0/892629878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55618f1b7ad0 0x55618f1b9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:12:08.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.315+0000 7f6aef7fe640 1 --2- 172.21.15.2:0/2958010502 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f6ae40b2a10 0x7f6ae40b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:08.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.315+0000 7f6afd863640 1 --2- 172.21.15.2:0/2958010502 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f6ae40b2a10 0x7f6ae40b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.315+0000 7f6afe064640 1 --2- 172.21.15.2:0/892629878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55618f1ad210 0x55618f1b7590 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f6af400bc90 tx=0x7f6af40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1031a8a817df26b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:08.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.315+0000 7f6aef7fe640 1 -- 172.21.15.2:0/2958010502 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f6af002e030 con 0x7f6af8108a60 2024-05-14T17:12:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.315+0000 7f6aecbfe640 1 -- 172.21.15.2:0/892629878 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af401c480 con 0x55618f1ad210 2024-05-14T17:12:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.315+0000 7f6aecbfe640 1 -- 172.21.15.2:0/892629878 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6af401cad0 con 0x55618f1ad210 2024-05-14T17:12:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.315+0000 7f6aecbfe640 1 -- 172.21.15.2:0/892629878 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af4011a70 con 0x55618f1ad210 2024-05-14T17:12:08.285 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.316+0000 7f6b02fe6740 1 -- 172.21.15.2:0/892629878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55618f1ad210 msgr2=0x55618f1b7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.316+0000 7f6b02fe6740 1 --2- 172.21.15.2:0/892629878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55618f1ad210 0x55618f1b7590 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f6af400bc90 tx=0x7f6af40107a0 comp rx=0 tx=0).stop 2024-05-14T17:12:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.316+0000 7f6afd863640 1 --2- 172.21.15.2:0/2958010502 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f6ae40b2a10 0x7f6ae40b4ed0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x55618f1aa220 tx=0x7f6ae8009680 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.316+0000 7f6b02fe6740 1 -- 172.21.15.2:0/892629878 shutdown_connections 2024-05-14T17:12:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.316+0000 7f6b02fe6740 1 --2- 172.21.15.2:0/892629878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55618f1b7ad0 0x55618f1b9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.316+0000 7f6b02fe6740 1 --2- 172.21.15.2:0/892629878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55618f1ad210 0x55618f1b7590 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:08.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.316+0000 7f6b02fe6740 1 --2- 172.21.15.2:0/892629878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55618f1ac8f0 0x55618f1accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:08.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.316+0000 7f6b02fe6740 1 -- 172.21.15.2:0/892629878 >> 172.21.15.2:0/892629878 conn(0x55618f1a8e00 msgr2=0x55618f1a91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:08.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.316+0000 7f6b02fe6740 1 -- 172.21.15.2:0/892629878 shutdown_connections 2024-05-14T17:12:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.316+0000 7f6b02fe6740 1 -- 172.21.15.2:0/892629878 wait complete. 2024-05-14T17:12:08.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.316+0000 7f6b02fe6740 1 -- start start 2024-05-14T17:12:08.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.316+0000 7f6b02fe6740 1 Processor -- start 2024-05-14T17:12:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.317+0000 7f6b02fe6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55618f1ac8f0 0x55618f1b7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.317+0000 7f6b02fe6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55618f1ad210 0x55618f1b4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.317+0000 7f6b02fe6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55618f1b7ad0 0x55618f1b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:08.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.317+0000 7f6b02fe6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55618edf5790 con 0x55618f1ad210 2024-05-14T17:12:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.317+0000 7f6b02fe6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55618f0cb890 con 0x55618f1b7ad0 2024-05-14T17:12:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.317+0000 7f6b02fe6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55618f1ba7a0 con 0x55618f1ac8f0 2024-05-14T17:12:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.317+0000 7f6b02fe6740 10 client.0 _ll_register_callbacks cb 0x55618f0ced60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:12:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.317+0000 7f6afd863640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55618f1ad210 0x55618f1b4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.317+0000 7f6afd863640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55618f1ad210 0x55618f1b4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60998/0 (socket says 172.21.15.2:60998) 2024-05-14T17:12:09.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.317+0000 7f6afd863640 1 -- 172.21.15.2:0/1711266363 learned_addr learned my addr 172.21.15.2:0/1711266363 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:12:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.317+0000 7f6afe064640 1 --2- 172.21.15.2:0/1711266363 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55618f1ac8f0 0x55618f1b7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.317+0000 7f6afe865640 1 --2- 172.21.15.2:0/1711266363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55618f1b7ad0 0x55618f1b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:09.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.317+0000 7f6afd863640 1 -- 172.21.15.2:0/1711266363 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55618f1ac8f0 msgr2=0x55618f1b7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:09.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.317+0000 7f6afd863640 1 --2- 172.21.15.2:0/1711266363 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55618f1ac8f0 0x55618f1b7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.317+0000 7f6afd863640 1 -- 172.21.15.2:0/1711266363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55618f1b7ad0 msgr2=0x55618f1b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.317+0000 7f6afd863640 1 --2- 172.21.15.2:0/1711266363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55618f1b7ad0 0x55618f1b4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.317+0000 7f6afd863640 1 -- 172.21.15.2:0/1711266363 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55618eef0620 con 0x55618f1ad210 2024-05-14T17:12:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.317+0000 7f6afe865640 1 --2- 172.21.15.2:0/1711266363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55618f1b7ad0 0x55618f1b4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:12:09.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.317+0000 7f6afd863640 1 --2- 172.21.15.2:0/1711266363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55618f1ad210 0x55618f1b4720 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8012c30 tx=0x7f6ae8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d3d89a4d13b11e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.317+0000 7f6ae1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:12:09.503 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.318+0000 7f6ae1ffb640 1 -- 172.21.15.2:0/1711266363 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8029480 con 0x55618f1ad210 2024-05-14T17:12:09.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.318+0000 7f6ae1ffb640 1 -- 172.21.15.2:0/1711266363 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f6ae8029620 con 0x55618f1ad210 2024-05-14T17:12:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.318+0000 7f6b02fe6740 1 -- 172.21.15.2:0/1711266363 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6af401cad0 con 0x55618f1ad210 2024-05-14T17:12:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.318+0000 7f6ae1ffb640 1 -- 172.21.15.2:0/1711266363 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f6ae8029820 con 0x55618f1ad210 2024-05-14T17:12:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.318+0000 7f6b02fe6740 10 client.16923 fetch_fsmap learned FSMap version 52 2024-05-14T17:12:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.318+0000 7f6b02fe6740 1 -- 172.21.15.2:0/1711266363 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55618f1c62b0 con 0x55618f1ad210 2024-05-14T17:12:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.318+0000 7f6ae1ffb640 1 -- 172.21.15.2:0/1711266363 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6af401cad0 con 0x55618f1ad210 2024-05-14T17:12:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.318+0000 7f6ae1ffb640 1 -- 172.21.15.2:0/1711266363 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 52) ==== 8346+0+0 (secure 0 0 0) 0x7f6ae801e4a0 con 0x55618f1ad210 2024-05-14T17:12:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.318+0000 7f6b02fe6740 10 client.16923 fetch_fsmap finished waiting for FSMap version 52 2024-05-14T17:12:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.318+0000 7f6b02fe6740 10 client.16923 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:12:09.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.318+0000 7f6b02fe6740 20 client.16923 populate_metadata read hostname 'smithi002' 2024-05-14T17:12:09.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.318+0000 7f6b02fe6740 1 --2- 172.21.15.2:0/1711266363 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55618f1c77c0 0x55618f1c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:09.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.318+0000 7f6b02fe6740 4 client.16923 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:12:09.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.318+0000 7f6b02fe6740 1 -- 172.21.15.2:0/1711266363 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55618f121b80 con 0x55618f1c77c0 2024-05-14T17:12:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.319+0000 7f6afe064640 1 --2- 172.21.15.2:0/1711266363 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55618f1c77c0 0x55618f1c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.319+0000 7f6afe064640 1 --2- 172.21.15.2:0/1711266363 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55618f1c77c0 0x55618f1c9c10 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2953eadce371a893 server_cookie=2df480ede6a7c13c in_seq=0 out_seq=0 2024-05-14T17:12:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.319+0000 7f6ae1ffb640 10 client.16923 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:12:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.320+0000 7f6ae1ffb640 1 -- 172.21.15.2:0/1711266363 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55618f121b80 con 0x55618f1c77c0 2024-05-14T17:12:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.320+0000 7f6ae1ffb640 10 client.16923 handle_command_reply: tid=0 2024-05-14T17:12:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.320+0000 7f6b02fe6740 1 client.16923 shutdown 2024-05-14T17:12:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.320+0000 7f6b02fe6740 1 -- 172.21.15.2:0/1711266363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55618f1ad210 msgr2=0x55618f1b4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.320+0000 7f6b02fe6740 1 --2- 172.21.15.2:0/1711266363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55618f1ad210 0x55618f1b4720 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8012c30 tx=0x7f6ae8004730 comp rx=0 tx=0).stop 2024-05-14T17:12:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.320+0000 7f6b02fe6740 1 -- 172.21.15.2:0/1711266363 shutdown_connections 2024-05-14T17:12:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.320+0000 7f6b02fe6740 1 --2- 172.21.15.2:0/1711266363 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55618f1c77c0 0x55618f1c9c10 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.320+0000 7f6b02fe6740 1 --2- 172.21.15.2:0/1711266363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55618f1b7ad0 0x55618f1b4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:10.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.320+0000 7f6b02fe6740 1 --2- 172.21.15.2:0/1711266363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55618f1ad210 0x55618f1b4720 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:10.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.320+0000 7f6ae1ffb640 0 client.16923 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:12:10.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.320+0000 7f6b02fe6740 1 --2- 172.21.15.2:0/1711266363 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55618f1ac8f0 0x55618f1b7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:10.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.320+0000 7f6b02fe6740 1 -- 172.21.15.2:0/1711266363 >> 172.21.15.2:0/1711266363 conn(0x55618f1a8e00 msgr2=0x55618f1a8100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:10.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.320+0000 7f6b02fe6740 1 -- 172.21.15.2:0/1711266363 shutdown_connections 2024-05-14T17:12:10.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.320+0000 7f6b02fe6740 1 -- 172.21.15.2:0/1711266363 wait complete. 2024-05-14T17:12:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.320+0000 7f6b02fe6740 20 client.16923 trim_cache size 0 max 16384 2024-05-14T17:12:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.336+0000 7f6b02fe6740 1 Processor -- start 2024-05-14T17:12:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.336+0000 7f6b02fe6740 1 -- start start 2024-05-14T17:12:10.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.336+0000 7f6b02fe6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55618f1ac8f0 0x55618f1c3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:10.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.336+0000 7f6b02fe6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55618f1ad210 0x55618f1b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.336+0000 7f6b02fe6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55618f1b7ad0 0x55618f1b3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.336+0000 7f6b02fe6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6af81b3010 con 0x55618f1b7ad0 2024-05-14T17:12:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.336+0000 7f6b02fe6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6af81b08e0 con 0x55618f1ac8f0 2024-05-14T17:12:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.336+0000 7f6b02fe6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6af81ae750 con 0x55618f1ad210 2024-05-14T17:12:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.336+0000 7f6afe865640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55618f1b7ad0 0x55618f1b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.336+0000 7f6afe865640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55618f1b7ad0 0x55618f1b3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32782/0 (socket says 172.21.15.2:32782) 2024-05-14T17:12:11.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.336+0000 7f6afe865640 1 -- 172.21.15.2:0/2142465364 learned_addr learned my addr 172.21.15.2:0/2142465364 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:12:11.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.336+0000 7f6afe064640 1 --2- 172.21.15.2:0/2142465364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55618f1ac8f0 0x55618f1c3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:11.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.336+0000 7f6afe865640 1 -- 172.21.15.2:0/2142465364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55618f1ad210 msgr2=0x55618f1b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.336+0000 7f6afe865640 1 --2- 172.21.15.2:0/2142465364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55618f1ad210 0x55618f1b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:11.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.336+0000 7f6afe865640 1 -- 172.21.15.2:0/2142465364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55618f1ac8f0 msgr2=0x55618f1c3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.336+0000 7f6afe865640 1 --2- 172.21.15.2:0/2142465364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55618f1ac8f0 0x55618f1c3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.336+0000 7f6afe865640 1 -- 172.21.15.2:0/2142465364 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55618f1ad040 con 0x55618f1b7ad0 2024-05-14T17:12:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.336+0000 7f6afe064640 1 --2- 172.21.15.2:0/2142465364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55618f1ac8f0 0x55618f1c3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:12:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.336+0000 7f6afe865640 1 --2- 172.21.15.2:0/2142465364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55618f1b7ad0 0x55618f1b3530 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f6af00c8300 tx=0x7f6af002a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b580ae5d52e206 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:11.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.336+0000 7f6ae3fff640 1 -- 172.21.15.2:0/2142465364 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af00caba0 con 0x55618f1b7ad0 2024-05-14T17:12:11.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.337+0000 7f6ae3fff640 1 -- 172.21.15.2:0/2142465364 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6af0036070 con 0x55618f1b7ad0 2024-05-14T17:12:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.337+0000 7f6ae3fff640 1 -- 172.21.15.2:0/2142465364 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af00c7320 con 0x55618f1b7ad0 2024-05-14T17:12:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.337+0000 7f6b02fe6740 1 -- 172.21.15.2:0/2142465364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55618f1b7ad0 msgr2=0x55618f1b3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.337+0000 7f6b02fe6740 1 --2- 172.21.15.2:0/2142465364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55618f1b7ad0 0x55618f1b3530 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f6af00c8300 tx=0x7f6af002a8e0 comp rx=0 tx=0).stop 2024-05-14T17:12:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.337+0000 7f6b02fe6740 1 -- 172.21.15.2:0/2142465364 shutdown_connections 2024-05-14T17:12:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.337+0000 7f6b02fe6740 1 --2- 172.21.15.2:0/2142465364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55618f1b7ad0 0x55618f1b3530 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:11.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.337+0000 7f6b02fe6740 1 --2- 172.21.15.2:0/2142465364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55618f1ad210 0x55618f1b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.337+0000 7f6b02fe6740 1 --2- 172.21.15.2:0/2142465364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55618f1ac8f0 0x55618f1c3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.337+0000 7f6b02fe6740 1 -- 172.21.15.2:0/2142465364 >> 172.21.15.2:0/2142465364 conn(0x55618f1a8e00 msgr2=0x55618f1a91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.337+0000 7f6b02fe6740 1 -- 172.21.15.2:0/2142465364 shutdown_connections 2024-05-14T17:12:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.337+0000 7f6b02fe6740 1 -- 172.21.15.2:0/2142465364 wait complete. 2024-05-14T17:12:11.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.337+0000 7f6b02fe6740 1 -- start start 2024-05-14T17:12:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.338+0000 7f6b02fe6740 1 Processor -- start 2024-05-14T17:12:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.338+0000 7f6b02fe6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55618f1ac8f0 0x55618f1c0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.338+0000 7f6b02fe6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55618f1ad210 0x55618f1bd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:12.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.338+0000 7f6b02fe6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55618f1b7ad0 0x55618f1bdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:12.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.338+0000 7f6b02fe6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6af81aeda0 con 0x55618f1ad210 2024-05-14T17:12:12.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.338+0000 7f6b02fe6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6af81b0210 con 0x55618f1ac8f0 2024-05-14T17:12:12.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.338+0000 7f6b02fe6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6af81af470 con 0x55618f1b7ad0 2024-05-14T17:12:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.338+0000 7f6b02fe6740 10 client.0 _ll_register_callbacks cb 0x7f6ae80128b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:12:12.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.338+0000 7f6afd863640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55618f1ad210 0x55618f1bd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:12.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.338+0000 7f6afd863640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55618f1ad210 0x55618f1bd4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32786/0 (socket says 172.21.15.2:32786) 2024-05-14T17:12:12.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.338+0000 7f6afd863640 1 -- 172.21.15.2:0/789590853 learned_addr learned my addr 172.21.15.2:0/789590853 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:12:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.338+0000 7f6afe064640 1 --2- 172.21.15.2:0/789590853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55618f1ac8f0 0x55618f1c0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.338+0000 7f6afe865640 1 --2- 172.21.15.2:0/789590853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55618f1b7ad0 0x55618f1bdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.338+0000 7f6afd863640 1 -- 172.21.15.2:0/789590853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55618f1b7ad0 msgr2=0x55618f1bdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:12.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.338+0000 7f6afd863640 1 --2- 172.21.15.2:0/789590853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55618f1b7ad0 0x55618f1bdbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:12.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.338+0000 7f6afd863640 1 -- 172.21.15.2:0/789590853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55618f1ac8f0 msgr2=0x55618f1c0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.338+0000 7f6afd863640 1 --2- 172.21.15.2:0/789590853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55618f1ac8f0 0x55618f1c0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.338+0000 7f6afd863640 1 -- 172.21.15.2:0/789590853 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55618f2310e0 con 0x55618f1ad210 2024-05-14T17:12:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.338+0000 7f6afd863640 1 --2- 172.21.15.2:0/789590853 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55618f1ad210 0x55618f1bd4a0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8012c00 tx=0x7f6ae8009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a2633f43bca553 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.338+0000 7f6ae1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:12:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.339+0000 7f6ae1ffb640 1 -- 172.21.15.2:0/789590853 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8029bc0 con 0x55618f1ad210 2024-05-14T17:12:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.339+0000 7f6b02fe6740 1 -- 172.21.15.2:0/789590853 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6af0036070 con 0x55618f1ad210 2024-05-14T17:12:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.339+0000 7f6ae1ffb640 1 -- 172.21.15.2:0/789590853 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f6ae8029d60 con 0x55618f1ad210 2024-05-14T17:12:12.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.339+0000 7f6ae1ffb640 1 -- 172.21.15.2:0/789590853 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f6ae8042050 con 0x55618f1ad210 2024-05-14T17:12:12.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.339+0000 7f6b02fe6740 10 client.16929 fetch_fsmap learned FSMap version 52 2024-05-14T17:12:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.339+0000 7f6b02fe6740 1 -- 172.21.15.2:0/789590853 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55618f232220 con 0x55618f1ad210 2024-05-14T17:12:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.339+0000 7f6ae1ffb640 1 -- 172.21.15.2:0/789590853 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6af0036070 con 0x55618f1ad210 2024-05-14T17:12:13.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.339+0000 7f6ae1ffb640 1 -- 172.21.15.2:0/789590853 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 52) ==== 8346+0+0 (secure 0 0 0) 0x7f6ae8004730 con 0x55618f1ad210 2024-05-14T17:12:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.339+0000 7f6b02fe6740 10 client.16929 fetch_fsmap finished waiting for FSMap version 52 2024-05-14T17:12:13.139 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.339+0000 7f6b02fe6740 10 client.16929 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:12:13.180 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.339+0000 7f6b02fe6740 20 client.16929 populate_metadata read hostname 'smithi002' 2024-05-14T17:12:13.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.339+0000 7f6b02fe6740 1 --2- 172.21.15.2:0/789590853 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55618f232a30 0x55618f234e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:13.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.339+0000 7f6b02fe6740 4 client.16929 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:12:13.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.339+0000 7f6b02fe6740 1 -- 172.21.15.2:0/789590853 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55618f11b360 con 0x55618f232a30 2024-05-14T17:12:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.339+0000 7f6afe064640 1 --2- 172.21.15.2:0/789590853 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55618f232a30 0x55618f234e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.340+0000 7f6afe064640 1 --2- 172.21.15.2:0/789590853 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55618f232a30 0x55618f234e50 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=addb7c1811f4e676 server_cookie=a87430e4269d87e3 in_seq=0 out_seq=0 2024-05-14T17:12:13.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.340+0000 7f6ae1ffb640 10 client.16929 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:12:13.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.356+0000 7f6ae1ffb640 1 -- 172.21.15.2:0/789590853 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55618f11b360 con 0x55618f232a30 2024-05-14T17:12:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.356+0000 7f6ae1ffb640 10 client.16929 handle_command_reply: tid=0 2024-05-14T17:12:13.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.356+0000 7f6b02fe6740 1 client.16929 shutdown 2024-05-14T17:12:13.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.356+0000 7f6b02fe6740 1 -- 172.21.15.2:0/789590853 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55618f1ad210 msgr2=0x55618f1bd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:13.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.356+0000 7f6b02fe6740 1 --2- 172.21.15.2:0/789590853 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55618f1ad210 0x55618f1bd4a0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8012c00 tx=0x7f6ae8009d30 comp rx=0 tx=0).stop 2024-05-14T17:12:13.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.356+0000 7f6b02fe6740 1 -- 172.21.15.2:0/789590853 shutdown_connections 2024-05-14T17:12:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.356+0000 7f6b02fe6740 1 --2- 172.21.15.2:0/789590853 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55618f232a30 0x55618f234e50 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.356+0000 7f6b02fe6740 1 --2- 172.21.15.2:0/789590853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55618f1b7ad0 0x55618f1bdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.356+0000 7f6b02fe6740 1 --2- 172.21.15.2:0/789590853 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55618f1ad210 0x55618f1bd4a0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:14.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.356+0000 7f6ae1ffb640 0 client.16929 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:12:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.356+0000 7f6b02fe6740 1 --2- 172.21.15.2:0/789590853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55618f1ac8f0 0x55618f1c0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.356+0000 7f6b02fe6740 1 -- 172.21.15.2:0/789590853 >> 172.21.15.2:0/789590853 conn(0x55618f1a8e00 msgr2=0x55618f1b2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.356+0000 7f6b02fe6740 1 -- 172.21.15.2:0/789590853 shutdown_connections 2024-05-14T17:12:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.356+0000 7f6b02fe6740 1 -- 172.21.15.2:0/789590853 wait complete. 2024-05-14T17:12:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.356+0000 7f6b02fe6740 20 client.16929 trim_cache size 0 max 16384 2024-05-14T17:12:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:12:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:12:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:12:14.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.357+0000 7f6b002c8640 1 -- 172.21.15.2:0/2958010502 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f6ae40b2a10 msgr2=0x7f6ae40b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:14.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.357+0000 7f6b002c8640 1 --2- 172.21.15.2:0/2958010502 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f6ae40b2a10 0x7f6ae40b4ed0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x55618f1aa220 tx=0x7f6ae8009680 comp rx=0 tx=0).stop 2024-05-14T17:12:14.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.357+0000 7f6b002c8640 1 -- 172.21.15.2:0/2958010502 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6af8108a60 msgr2=0x7f6af80757d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.357+0000 7f6b002c8640 1 --2- 172.21.15.2:0/2958010502 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6af8108a60 0x7f6af80757d0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f6af0033040 tx=0x7f6af0003970 comp rx=0 tx=0).stop 2024-05-14T17:12:14.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.357+0000 7f6b002c8640 1 -- 172.21.15.2:0/2958010502 shutdown_connections 2024-05-14T17:12:14.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.357+0000 7f6b002c8640 1 --2- 172.21.15.2:0/2958010502 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f6ae40b2a10 0x7f6ae40b4ed0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:14.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.357+0000 7f6b002c8640 1 --2- 172.21.15.2:0/2958010502 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6af8108a60 0x7f6af80757d0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:14.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.357+0000 7f6b002c8640 1 --2- 172.21.15.2:0/2958010502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6af8103410 0x7f6af8075290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.357+0000 7f6b002c8640 1 --2- 172.21.15.2:0/2958010502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6af8102a50 0x7f6af8074d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.357+0000 7f6b002c8640 1 -- 172.21.15.2:0/2958010502 >> 172.21.15.2:0/2958010502 conn(0x7f6af80fe520 msgr2=0x7f6af80fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.357+0000 7f6b002c8640 1 -- 172.21.15.2:0/2958010502 shutdown_connections 2024-05-14T17:12:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:10:51.357+0000 7f6b002c8640 1 -- 172.21.15.2:0/2958010502 wait complete. 2024-05-14T17:12:15.008 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:10:52 smithi002 ceph-mon[31538]: pgmap v506: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.1 MiB/s wr, 82 op/s 2024-05-14T17:12:15.048 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:10:54 smithi002 ceph-mon[31538]: pgmap v507: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 1.2 MiB/s wr, 89 op/s 2024-05-14T17:12:15.090 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:10:56 smithi002 ceph-mon[31538]: pgmap v508: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 52 KiB/s wr, 13 op/s 2024-05-14T17:12:15.132 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:10:58 smithi002 ceph-mon[31538]: pgmap v509: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 199 KiB/s wr, 27 op/s 2024-05-14T17:12:15.173 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:11:00 smithi002 ceph-mon[31538]: pgmap v510: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-05-14T17:12:15.215 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:11:02 smithi002 ceph-mon[31538]: pgmap v511: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-05-14T17:12:15.260 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:12:05 smithi123 ceph-mon[35971]: pgmap v545: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 64 KiB/s wr, 26 op/s 2024-05-14T17:12:15.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:12:08 smithi123 ceph-mon[35971]: pgmap v546: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 64 KiB/s wr, 26 op/s 2024-05-14T17:12:15.340 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:12:09 smithi123 ceph-mon[35971]: osdmap e102: 12 total, 12 up, 12 in 2024-05-14T17:12:15.384 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:12:10 smithi123 ceph-mon[35971]: pgmap v548: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 55 KiB/s wr, 27 op/s 2024-05-14T17:12:15.423 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:12:11 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:12:15.465 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:12:11 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:12:15.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:12:11 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:12:15.554 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:12:11 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:12:15.594 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:12:12 smithi123 ceph-mon[35971]: pgmap v549: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 55 KiB/s wr, 27 op/s 2024-05-14T17:12:15.632 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:12:14 smithi123 ceph-mon[35971]: pgmap v550: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 9 op/s 2024-05-14T17:12:15.690 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:12:05 smithi151 ceph-mon[36066]: pgmap v545: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 64 KiB/s wr, 26 op/s 2024-05-14T17:12:15.732 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:12:08 smithi151 ceph-mon[36066]: pgmap v546: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 64 KiB/s wr, 26 op/s 2024-05-14T17:12:15.774 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:12:09 smithi151 ceph-mon[36066]: osdmap e102: 12 total, 12 up, 12 in 2024-05-14T17:12:15.815 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:12:10 smithi151 ceph-mon[36066]: pgmap v548: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 55 KiB/s wr, 27 op/s 2024-05-14T17:12:15.857 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:12:11 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:12:15.899 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:12:11 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:12:15.940 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:12:11 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:12:15.982 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:12:11 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:12:16.032 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:12:12 smithi151 ceph-mon[36066]: pgmap v549: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 55 KiB/s wr, 27 op/s 2024-05-14T17:12:16.074 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:12:14 smithi151 ceph-mon[36066]: pgmap v550: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 9 op/s 2024-05-14T17:12:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.450+0000 7f9e42396640 1 -- 172.21.15.2:0/2092602435 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e3c108b10 msgr2=0x7f9e3c108ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:16.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.450+0000 7f9e42396640 1 --2- 172.21.15.2:0/2092602435 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e3c108b10 0x7f9e3c108ef0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c009900 tx=0x7f9e2c02a480 comp rx=0 tx=0).stop 2024-05-14T17:12:16.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.450+0000 7f9e42396640 1 -- 172.21.15.2:0/2092602435 shutdown_connections 2024-05-14T17:12:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.450+0000 7f9e42396640 1 --2- 172.21.15.2:0/2092602435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e3c06bc00 0x7f9e3c10f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.450+0000 7f9e42396640 1 --2- 172.21.15.2:0/2092602435 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e3c06b260 0x7f9e3c06b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.450+0000 7f9e42396640 1 --2- 172.21.15.2:0/2092602435 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e3c108b10 0x7f9e3c108ef0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:16.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.451+0000 7f9e42396640 1 -- 172.21.15.2:0/2092602435 >> 172.21.15.2:0/2092602435 conn(0x7f9e3c0788c0 msgr2=0x7f9e3c078cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:16.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.451+0000 7f9e42396640 1 -- 172.21.15.2:0/2092602435 shutdown_connections 2024-05-14T17:12:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.451+0000 7f9e42396640 1 -- 172.21.15.2:0/2092602435 wait complete. 2024-05-14T17:12:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.451+0000 7f9e42396640 1 Processor -- start 2024-05-14T17:12:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.451+0000 7f9e42396640 1 -- start start 2024-05-14T17:12:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.452+0000 7f9e42396640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e3c06b260 0x7f9e3c199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.452+0000 7f9e42396640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e3c06bc00 0x7f9e3c199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.452+0000 7f9e42396640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e3c108b10 0x7f9e3c19d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.452+0000 7f9e42396640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3c112d90 con 0x7f9e3c06bc00 2024-05-14T17:12:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.452+0000 7f9e42396640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3c112c10 con 0x7f9e3c108b10 2024-05-14T17:12:16.799 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.452+0000 7f9e42396640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3c112f10 con 0x7f9e3c06b260 2024-05-14T17:12:16.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.452+0000 7f9e3b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e3c06bc00 0x7f9e3c199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.452+0000 7f9e3b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e3c06bc00 0x7f9e3c199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54504/0 (socket says 172.21.15.2:54504) 2024-05-14T17:12:16.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.452+0000 7f9e3b7fe640 1 -- 172.21.15.2:0/3940737337 learned_addr learned my addr 172.21.15.2:0/3940737337 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:12:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.452+0000 7f9e3bfff640 1 --2- 172.21.15.2:0/3940737337 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e3c06b260 0x7f9e3c199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:17.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.452+0000 7f9e40933640 1 --2- 172.21.15.2:0/3940737337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e3c108b10 0x7f9e3c19d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.452+0000 7f9e3b7fe640 1 -- 172.21.15.2:0/3940737337 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e3c06b260 msgr2=0x7f9e3c199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.452+0000 7f9e3b7fe640 1 --2- 172.21.15.2:0/3940737337 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e3c06b260 0x7f9e3c199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.452+0000 7f9e3b7fe640 1 -- 172.21.15.2:0/3940737337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e3c108b10 msgr2=0x7f9e3c19d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.452+0000 7f9e3b7fe640 1 --2- 172.21.15.2:0/3940737337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e3c108b10 0x7f9e3c19d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.452+0000 7f9e3b7fe640 1 -- 172.21.15.2:0/3940737337 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e3c19e0d0 con 0x7f9e3c06bc00 2024-05-14T17:12:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.452+0000 7f9e40933640 1 --2- 172.21.15.2:0/3940737337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e3c108b10 0x7f9e3c19d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:12:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.452+0000 7f9e3b7fe640 1 --2- 172.21.15.2:0/3940737337 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e3c06bc00 0x7f9e3c199670 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f9e3000c970 tx=0x7f9e3000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.452+0000 7f9e397fa640 1 -- 172.21.15.2:0/3940737337 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e30007bb0 con 0x7f9e3c06bc00 2024-05-14T17:12:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.452+0000 7f9e397fa640 1 -- 172.21.15.2:0/3940737337 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e30007d50 con 0x7f9e3c06bc00 2024-05-14T17:12:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.452+0000 7f9e42396640 1 -- 172.21.15.2:0/3940737337 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e3c19e3c0 con 0x7f9e3c06bc00 2024-05-14T17:12:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.452+0000 7f9e397fa640 1 -- 172.21.15.2:0/3940737337 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3000f450 con 0x7f9e3c06bc00 2024-05-14T17:12:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.453+0000 7f9e42396640 1 -- 172.21.15.2:0/3940737337 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e3c1a5c50 con 0x7f9e3c06bc00 2024-05-14T17:12:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.456+0000 7f9e45188740 1 Processor -- start 2024-05-14T17:12:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.456+0000 7f9e45188740 1 -- start start 2024-05-14T17:12:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.456+0000 7f9e397fa640 1 -- 172.21.15.2:0/3940737337 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f9e30004f20 con 0x7f9e3c06bc00 2024-05-14T17:12:17.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.456+0000 7f9e45188740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56335bdf08f0 0x56335bdf0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:17.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.456+0000 7f9e45188740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335bdf1210 0x56335bdfb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.456+0000 7f9e45188740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56335bdfbad0 0x56335bdfdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.456+0000 7f9e45188740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56335bd63c30 con 0x56335bdf1210 2024-05-14T17:12:17.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.456+0000 7f9e45188740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56335bd537d0 con 0x56335bdf08f0 2024-05-14T17:12:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.456+0000 7f9e45188740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56335bd57810 con 0x56335bdfbad0 2024-05-14T17:12:17.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.456+0000 7f9e3bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335bdf1210 0x56335bdfb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.456+0000 7f9e3bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335bdf1210 0x56335bdfb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54508/0 (socket says 172.21.15.2:54508) 2024-05-14T17:12:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.456+0000 7f9e3bfff640 1 -- 172.21.15.2:0/1512712670 learned_addr learned my addr 172.21.15.2:0/1512712670 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:12:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.456+0000 7f9e40933640 1 --2- 172.21.15.2:0/1512712670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56335bdf08f0 0x56335bdf0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.456+0000 7f9e3b7fe640 1 --2- 172.21.15.2:0/1512712670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56335bdfbad0 0x56335bdfdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.456+0000 7f9e3bfff640 1 -- 172.21.15.2:0/1512712670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56335bdfbad0 msgr2=0x56335bdfdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:18.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.456+0000 7f9e3bfff640 1 --2- 172.21.15.2:0/1512712670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56335bdfbad0 0x56335bdfdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.456+0000 7f9e3bfff640 1 -- 172.21.15.2:0/1512712670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56335bdf08f0 msgr2=0x56335bdf0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:18.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.456+0000 7f9e3bfff640 1 --2- 172.21.15.2:0/1512712670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56335bdf08f0 0x56335bdf0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:18.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.456+0000 7f9e3bfff640 1 -- 172.21.15.2:0/1512712670 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56335ba68210 con 0x56335bdf1210 2024-05-14T17:12:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.457+0000 7f9e397fa640 1 --2- 172.21.15.2:0/3940737337 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f9e240b2a10 0x7f9e240b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.457+0000 7f9e3bfff640 1 --2- 172.21.15.2:0/1512712670 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335bdf1210 0x56335bdfb590 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c009900 tx=0x7f9e2c004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595a7c7321d87624 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.457+0000 7f9e40933640 1 --2- 172.21.15.2:0/3940737337 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f9e240b2a10 0x7f9e240b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:18.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.457+0000 7f9e227fc640 1 -- 172.21.15.2:0/1512712670 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c035040 con 0x56335bdf1210 2024-05-14T17:12:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.457+0000 7f9e397fa640 1 -- 172.21.15.2:0/3940737337 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8557+0+0 (secure 0 0 0) 0x7f9e300e2340 con 0x7f9e3c06bc00 2024-05-14T17:12:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.457+0000 7f9e227fc640 1 -- 172.21.15.2:0/1512712670 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e2c004740 con 0x56335bdf1210 2024-05-14T17:12:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.457+0000 7f9e227fc640 1 -- 172.21.15.2:0/1512712670 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c003850 con 0x56335bdf1210 2024-05-14T17:12:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.458+0000 7f9e45188740 1 -- 172.21.15.2:0/1512712670 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335bdf1210 msgr2=0x56335bdfb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:18.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.458+0000 7f9e40933640 1 --2- 172.21.15.2:0/3940737337 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f9e240b2a10 0x7f9e240b4ed0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9e34005fd0 tx=0x7f9e34009700 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.458+0000 7f9e45188740 1 --2- 172.21.15.2:0/1512712670 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335bdf1210 0x56335bdfb590 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c009900 tx=0x7f9e2c004310 comp rx=0 tx=0).stop 2024-05-14T17:12:18.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.458+0000 7f9e45188740 1 -- 172.21.15.2:0/1512712670 shutdown_connections 2024-05-14T17:12:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.458+0000 7f9e45188740 1 --2- 172.21.15.2:0/1512712670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56335bdfbad0 0x56335bdfdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.458+0000 7f9e45188740 1 --2- 172.21.15.2:0/1512712670 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335bdf1210 0x56335bdfb590 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.458+0000 7f9e45188740 1 --2- 172.21.15.2:0/1512712670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56335bdf08f0 0x56335bdf0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.458+0000 7f9e45188740 1 -- 172.21.15.2:0/1512712670 >> 172.21.15.2:0/1512712670 conn(0x56335bdece00 msgr2=0x56335bded1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:18.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.458+0000 7f9e45188740 1 -- 172.21.15.2:0/1512712670 shutdown_connections 2024-05-14T17:12:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.458+0000 7f9e45188740 1 -- 172.21.15.2:0/1512712670 wait complete. 2024-05-14T17:12:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.458+0000 7f9e45188740 1 -- start start 2024-05-14T17:12:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.458+0000 7f9e45188740 1 Processor -- start 2024-05-14T17:12:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.458+0000 7f9e45188740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56335bdf08f0 0x56335bdfb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.458+0000 7f9e45188740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335bdf1210 0x56335bdf8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:19.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.459+0000 7f9e45188740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56335bdfbad0 0x56335bdf8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.459+0000 7f9e45188740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56335ba39790 con 0x56335bdf1210 2024-05-14T17:12:19.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.459+0000 7f9e45188740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56335bd0f890 con 0x56335bdf08f0 2024-05-14T17:12:19.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.459+0000 7f9e45188740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56335bdfe740 con 0x56335bdfbad0 2024-05-14T17:12:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.459+0000 7f9e45188740 10 client.0 _ll_register_callbacks cb 0x56335bd12d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:12:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.459+0000 7f9e3b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335bdf1210 0x56335bdf8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.459+0000 7f9e3b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335bdf1210 0x56335bdf8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54522/0 (socket says 172.21.15.2:54522) 2024-05-14T17:12:19.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.459+0000 7f9e3b7fe640 1 -- 172.21.15.2:0/576196295 learned_addr learned my addr 172.21.15.2:0/576196295 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:12:19.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.459+0000 7f9e40933640 1 --2- 172.21.15.2:0/576196295 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56335bdfbad0 0x56335bdf8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:19.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.459+0000 7f9e3bfff640 1 --2- 172.21.15.2:0/576196295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56335bdf08f0 0x56335bdfb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:19.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.459+0000 7f9e3b7fe640 1 -- 172.21.15.2:0/576196295 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56335bdfbad0 msgr2=0x56335bdf8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:19.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.459+0000 7f9e3b7fe640 1 --2- 172.21.15.2:0/576196295 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56335bdfbad0 0x56335bdf8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:19.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.459+0000 7f9e3b7fe640 1 -- 172.21.15.2:0/576196295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56335bdf08f0 msgr2=0x56335bdfb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:19.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.459+0000 7f9e3b7fe640 1 --2- 172.21.15.2:0/576196295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56335bdf08f0 0x56335bdfb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:19.488 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.459+0000 7f9e3b7fe640 1 -- 172.21.15.2:0/576196295 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56335bb34620 con 0x56335bdf1210 2024-05-14T17:12:19.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.459+0000 7f9e3b7fe640 1 --2- 172.21.15.2:0/576196295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335bdf1210 0x56335bdf8760 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9e3000c940 tx=0x7f9e300048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acdcd1c6772ff562 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.459+0000 7f9e1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:12:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.459+0000 7f9e1b7fe640 1 -- 172.21.15.2:0/576196295 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3001d070 con 0x56335bdf1210 2024-05-14T17:12:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.459+0000 7f9e1b7fe640 1 -- 172.21.15.2:0/576196295 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8557+0+0 (secure 0 0 0) 0x7f9e300dfcf0 con 0x56335bdf1210 2024-05-14T17:12:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.459+0000 7f9e45188740 1 -- 172.21.15.2:0/576196295 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e2c004740 con 0x56335bdf1210 2024-05-14T17:12:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.459+0000 7f9e1b7fe640 1 -- 172.21.15.2:0/576196295 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f9e300a9110 con 0x56335bdf1210 2024-05-14T17:12:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.459+0000 7f9e45188740 10 client.16941 fetch_fsmap learned FSMap version 52 2024-05-14T17:12:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.459+0000 7f9e45188740 1 -- 172.21.15.2:0/576196295 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x56335be0a1a0 con 0x56335bdf1210 2024-05-14T17:12:19.788 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.460+0000 7f9e1b7fe640 1 -- 172.21.15.2:0/576196295 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e2c004740 con 0x56335bdf1210 2024-05-14T17:12:19.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.460+0000 7f9e1b7fe640 1 -- 172.21.15.2:0/576196295 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 52) ==== 8346+0+0 (secure 0 0 0) 0x7f9e300a5890 con 0x56335bdf1210 2024-05-14T17:12:19.855 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.460+0000 7f9e45188740 10 client.16941 fetch_fsmap finished waiting for FSMap version 52 2024-05-14T17:12:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.460+0000 7f9e45188740 10 client.16941 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:12:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.460+0000 7f9e45188740 20 client.16941 populate_metadata read hostname 'smithi002' 2024-05-14T17:12:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.460+0000 7f9e45188740 1 --2- 172.21.15.2:0/576196295 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56335be0b700 0x56335be0db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.460+0000 7f9e45188740 4 client.16941 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:12:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.460+0000 7f9e45188740 1 -- 172.21.15.2:0/576196295 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56335bd65b80 con 0x56335be0b700 2024-05-14T17:12:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.460+0000 7f9e3bfff640 1 --2- 172.21.15.2:0/576196295 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56335be0b700 0x56335be0db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.460+0000 7f9e1b7fe640 10 client.16941 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:12:20.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.460+0000 7f9e3bfff640 1 --2- 172.21.15.2:0/576196295 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56335be0b700 0x56335be0db30 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14d506f88825845a server_cookie=907a32e9cb62d562 in_seq=0 out_seq=0 2024-05-14T17:12:20.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.461+0000 7f9e1b7fe640 1 -- 172.21.15.2:0/576196295 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56335bd65b80 con 0x56335be0b700 2024-05-14T17:12:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.461+0000 7f9e1b7fe640 10 client.16941 handle_command_reply: tid=0 2024-05-14T17:12:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.461+0000 7f9e45188740 1 client.16941 shutdown 2024-05-14T17:12:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.461+0000 7f9e45188740 1 -- 172.21.15.2:0/576196295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335bdf1210 msgr2=0x56335bdf8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:20.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.461+0000 7f9e45188740 1 --2- 172.21.15.2:0/576196295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335bdf1210 0x56335bdf8760 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9e3000c940 tx=0x7f9e300048f0 comp rx=0 tx=0).stop 2024-05-14T17:12:20.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.461+0000 7f9e45188740 1 -- 172.21.15.2:0/576196295 shutdown_connections 2024-05-14T17:12:20.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.461+0000 7f9e45188740 1 --2- 172.21.15.2:0/576196295 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56335be0b700 0x56335be0db30 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:20.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.461+0000 7f9e45188740 1 --2- 172.21.15.2:0/576196295 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56335bdfbad0 0x56335bdf8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.461+0000 7f9e1b7fe640 0 client.16941 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:12:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.461+0000 7f9e45188740 1 --2- 172.21.15.2:0/576196295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335bdf1210 0x56335bdf8760 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.461+0000 7f9e45188740 1 --2- 172.21.15.2:0/576196295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56335bdf08f0 0x56335bdfb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:20.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.461+0000 7f9e45188740 1 -- 172.21.15.2:0/576196295 >> 172.21.15.2:0/576196295 conn(0x56335bdece00 msgr2=0x56335bdec100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.462+0000 7f9e45188740 1 -- 172.21.15.2:0/576196295 shutdown_connections 2024-05-14T17:12:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.462+0000 7f9e45188740 1 -- 172.21.15.2:0/576196295 wait complete. 2024-05-14T17:12:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.462+0000 7f9e45188740 20 client.16941 trim_cache size 0 max 16384 2024-05-14T17:12:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.477+0000 7f9e45188740 1 Processor -- start 2024-05-14T17:12:20.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.477+0000 7f9e45188740 1 -- start start 2024-05-14T17:12:20.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.477+0000 7f9e45188740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335bdf08f0 0x56335be07610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.477+0000 7f9e45188740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56335bdf1210 0x56335bde7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.477+0000 7f9e45188740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56335bdfbad0 0x56335bdfafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.477+0000 7f9e45188740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3c1b0ee0 con 0x56335bdf08f0 2024-05-14T17:12:20.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.477+0000 7f9e45188740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3c1ae7b0 con 0x56335bdf1210 2024-05-14T17:12:20.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.477+0000 7f9e45188740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3c1ac620 con 0x56335bdfbad0 2024-05-14T17:12:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.477+0000 7f9e3bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335bdf08f0 0x56335be07610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.477+0000 7f9e3bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335bdf08f0 0x56335be07610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54532/0 (socket says 172.21.15.2:54532) 2024-05-14T17:12:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.477+0000 7f9e3bfff640 1 -- 172.21.15.2:0/3430449401 learned_addr learned my addr 172.21.15.2:0/3430449401 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:12:21.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.477+0000 7f9e3b7fe640 1 --2- 172.21.15.2:0/3430449401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56335bdf1210 0x56335bde7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.477+0000 7f9e3bfff640 1 -- 172.21.15.2:0/3430449401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56335bdfbad0 msgr2=0x56335bdfafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:21.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.477+0000 7f9e3bfff640 1 --2- 172.21.15.2:0/3430449401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56335bdfbad0 0x56335bdfafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:21.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.477+0000 7f9e3bfff640 1 -- 172.21.15.2:0/3430449401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56335bdf1210 msgr2=0x56335bde7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:21.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.477+0000 7f9e3bfff640 1 --2- 172.21.15.2:0/3430449401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56335bdf1210 0x56335bde7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:21.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.477+0000 7f9e3bfff640 1 -- 172.21.15.2:0/3430449401 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56335bdf9070 con 0x56335bdf08f0 2024-05-14T17:12:21.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.477+0000 7f9e3b7fe640 1 --2- 172.21.15.2:0/3430449401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56335bdf1210 0x56335bde7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:12:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.478+0000 7f9e3bfff640 1 --2- 172.21.15.2:0/3430449401 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335bdf08f0 0x56335be07610 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x56335bdf1e90 tx=0x7f9e2c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c333a18bdaf96c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.478+0000 7f9e21ffb640 1 -- 172.21.15.2:0/3430449401 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c035070 con 0x56335bdf08f0 2024-05-14T17:12:21.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.478+0000 7f9e21ffb640 1 -- 172.21.15.2:0/3430449401 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e2c0046c0 con 0x56335bdf08f0 2024-05-14T17:12:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.478+0000 7f9e21ffb640 1 -- 172.21.15.2:0/3430449401 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c0508c0 con 0x56335bdf08f0 2024-05-14T17:12:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.478+0000 7f9e45188740 1 -- 172.21.15.2:0/3430449401 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335bdf08f0 msgr2=0x56335be07610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.478+0000 7f9e45188740 1 --2- 172.21.15.2:0/3430449401 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335bdf08f0 0x56335be07610 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x56335bdf1e90 tx=0x7f9e2c005fd0 comp rx=0 tx=0).stop 2024-05-14T17:12:21.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.478+0000 7f9e45188740 1 -- 172.21.15.2:0/3430449401 shutdown_connections 2024-05-14T17:12:21.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.478+0000 7f9e45188740 1 --2- 172.21.15.2:0/3430449401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56335bdfbad0 0x56335bdfafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.478+0000 7f9e45188740 1 --2- 172.21.15.2:0/3430449401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56335bdf1210 0x56335bde7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.478+0000 7f9e45188740 1 --2- 172.21.15.2:0/3430449401 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335bdf08f0 0x56335be07610 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.478+0000 7f9e45188740 1 -- 172.21.15.2:0/3430449401 >> 172.21.15.2:0/3430449401 conn(0x56335bdece00 msgr2=0x56335bdf3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.478+0000 7f9e45188740 1 -- 172.21.15.2:0/3430449401 shutdown_connections 2024-05-14T17:12:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.478+0000 7f9e45188740 1 -- 172.21.15.2:0/3430449401 wait complete. 2024-05-14T17:12:21.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.479+0000 7f9e45188740 1 -- start start 2024-05-14T17:12:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.479+0000 7f9e45188740 1 Processor -- start 2024-05-14T17:12:21.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.479+0000 7f9e45188740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335bdf08f0 0x56335be03bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.479+0000 7f9e45188740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56335bdf1210 0x56335bdfef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.479+0000 7f9e45188740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56335bdfbad0 0x56335bdff490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.479+0000 7f9e45188740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3c1acc70 con 0x56335bdf08f0 2024-05-14T17:12:22.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.479+0000 7f9e45188740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3c1ae0e0 con 0x56335bdfbad0 2024-05-14T17:12:22.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.479+0000 7f9e45188740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3c1ad340 con 0x56335bdf1210 2024-05-14T17:12:22.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.479+0000 7f9e45188740 10 client.0 _ll_register_callbacks cb 0x7f9e300a54c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:12:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.479+0000 7f9e3bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335bdf08f0 0x56335be03bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:22.157 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.479+0000 7f9e3bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335bdf08f0 0x56335be03bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54538/0 (socket says 172.21.15.2:54538) 2024-05-14T17:12:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.479+0000 7f9e3bfff640 1 -- 172.21.15.2:0/3446078172 learned_addr learned my addr 172.21.15.2:0/3446078172 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:12:22.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.480+0000 7f9e3b7fe640 1 --2- 172.21.15.2:0/3446078172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56335bdf1210 0x56335bdfef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.480+0000 7f9e40933640 1 --2- 172.21.15.2:0/3446078172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56335bdfbad0 0x56335bdff490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.480+0000 7f9e3bfff640 1 -- 172.21.15.2:0/3446078172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56335bdf1210 msgr2=0x56335bdfef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.480+0000 7f9e3bfff640 1 --2- 172.21.15.2:0/3446078172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56335bdf1210 0x56335bdfef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.480+0000 7f9e3bfff640 1 -- 172.21.15.2:0/3446078172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56335bdfbad0 msgr2=0x56335bdff490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:22.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.480+0000 7f9e3bfff640 1 --2- 172.21.15.2:0/3446078172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56335bdfbad0 0x56335bdff490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.480+0000 7f9e3bfff640 1 -- 172.21.15.2:0/3446078172 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56335bdff9d0 con 0x56335bdf08f0 2024-05-14T17:12:22.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.480+0000 7f9e3b7fe640 1 --2- 172.21.15.2:0/3446078172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56335bdf1210 0x56335bdfef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:12:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.480+0000 7f9e40933640 1 --2- 172.21.15.2:0/3446078172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56335bdfbad0 0x56335bdff490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:12:22.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.480+0000 7f9e3bfff640 1 --2- 172.21.15.2:0/3446078172 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335bdf08f0 0x56335be03bb0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c02aa70 tx=0x7f9e2c0507f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5cbbc704e54d1b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:22.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.480+0000 7f9e1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:12:22.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.480+0000 7f9e1b7fe640 1 -- 172.21.15.2:0/3446078172 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c035040 con 0x56335bdf08f0 2024-05-14T17:12:22.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.480+0000 7f9e1b7fe640 1 -- 172.21.15.2:0/3446078172 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8557+0+0 (secure 0 0 0) 0x7f9e2c004ef0 con 0x56335bdf08f0 2024-05-14T17:12:22.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.480+0000 7f9e45188740 1 -- 172.21.15.2:0/3446078172 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e2c0046c0 con 0x56335bdf08f0 2024-05-14T17:12:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.480+0000 7f9e1b7fe640 1 -- 172.21.15.2:0/3446078172 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f9e2c066050 con 0x56335bdf08f0 2024-05-14T17:12:22.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.480+0000 7f9e45188740 10 client.16947 fetch_fsmap learned FSMap version 52 2024-05-14T17:12:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.480+0000 7f9e45188740 1 -- 172.21.15.2:0/3446078172 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x56335be75c40 con 0x56335bdf08f0 2024-05-14T17:12:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.480+0000 7f9e1b7fe640 1 -- 172.21.15.2:0/3446078172 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e2c0046c0 con 0x56335bdf08f0 2024-05-14T17:12:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.481+0000 7f9e1b7fe640 1 -- 172.21.15.2:0/3446078172 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 52) ==== 8346+0+0 (secure 0 0 0) 0x7f9e2c061030 con 0x56335bdf08f0 2024-05-14T17:12:22.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.481+0000 7f9e45188740 10 client.16947 fetch_fsmap finished waiting for FSMap version 52 2024-05-14T17:12:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.481+0000 7f9e45188740 10 client.16947 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:12:22.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.481+0000 7f9e45188740 20 client.16947 populate_metadata read hostname 'smithi002' 2024-05-14T17:12:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.481+0000 7f9e45188740 1 --2- 172.21.15.2:0/3446078172 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56335be75e80 0x56335be782a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:23.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.481+0000 7f9e45188740 4 client.16947 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:12:23.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.481+0000 7f9e45188740 1 -- 172.21.15.2:0/3446078172 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56335bd5f360 con 0x56335be75e80 2024-05-14T17:12:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.481+0000 7f9e3b7fe640 1 --2- 172.21.15.2:0/3446078172 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56335be75e80 0x56335be782a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.481+0000 7f9e3b7fe640 1 --2- 172.21.15.2:0/3446078172 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56335be75e80 0x56335be782a0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e810d1cbb05bd74 server_cookie=f6be354aa84497ea in_seq=0 out_seq=0 2024-05-14T17:12:23.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.481+0000 7f9e1b7fe640 10 client.16947 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:12:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.492+0000 7f9e1b7fe640 1 -- 172.21.15.2:0/3446078172 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56335bd5f360 con 0x56335be75e80 2024-05-14T17:12:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.492+0000 7f9e1b7fe640 10 client.16947 handle_command_reply: tid=0 2024-05-14T17:12:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.492+0000 7f9e45188740 1 client.16947 shutdown 2024-05-14T17:12:23.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.492+0000 7f9e45188740 1 -- 172.21.15.2:0/3446078172 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335bdf08f0 msgr2=0x56335be03bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.492+0000 7f9e45188740 1 --2- 172.21.15.2:0/3446078172 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335bdf08f0 0x56335be03bb0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c02aa70 tx=0x7f9e2c0507f0 comp rx=0 tx=0).stop 2024-05-14T17:12:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.493+0000 7f9e45188740 1 -- 172.21.15.2:0/3446078172 shutdown_connections 2024-05-14T17:12:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.493+0000 7f9e45188740 1 --2- 172.21.15.2:0/3446078172 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56335be75e80 0x56335be782a0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.493+0000 7f9e45188740 1 --2- 172.21.15.2:0/3446078172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56335bdfbad0 0x56335bdff490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:23.634 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.493+0000 7f9e1b7fe640 0 client.16947 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:12:23.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.493+0000 7f9e45188740 1 --2- 172.21.15.2:0/3446078172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56335bdf1210 0x56335bdfef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:23.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.493+0000 7f9e45188740 1 --2- 172.21.15.2:0/3446078172 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56335bdf08f0 0x56335be03bb0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:23.734 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.493+0000 7f9e45188740 1 -- 172.21.15.2:0/3446078172 >> 172.21.15.2:0/3446078172 conn(0x56335bdece00 msgr2=0x56335bdec100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.493+0000 7f9e45188740 1 -- 172.21.15.2:0/3446078172 shutdown_connections 2024-05-14T17:12:23.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.493+0000 7f9e45188740 1 -- 172.21.15.2:0/3446078172 wait complete. 2024-05-14T17:12:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.493+0000 7f9e45188740 20 client.16947 trim_cache size 0 max 16384 2024-05-14T17:12:23.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:12:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:12:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:12:24.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.493+0000 7f9e42396640 1 -- 172.21.15.2:0/3940737337 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f9e240b2a10 msgr2=0x7f9e240b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.493+0000 7f9e42396640 1 --2- 172.21.15.2:0/3940737337 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f9e240b2a10 0x7f9e240b4ed0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9e34005fd0 tx=0x7f9e34009700 comp rx=0 tx=0).stop 2024-05-14T17:12:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.493+0000 7f9e42396640 1 -- 172.21.15.2:0/3940737337 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e3c06bc00 msgr2=0x7f9e3c199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:24.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.493+0000 7f9e42396640 1 --2- 172.21.15.2:0/3940737337 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e3c06bc00 0x7f9e3c199670 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f9e3000c970 tx=0x7f9e3000ce30 comp rx=0 tx=0).stop 2024-05-14T17:12:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.493+0000 7f9e42396640 1 -- 172.21.15.2:0/3940737337 shutdown_connections 2024-05-14T17:12:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.493+0000 7f9e42396640 1 --2- 172.21.15.2:0/3940737337 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f9e240b2a10 0x7f9e240b4ed0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.493+0000 7f9e42396640 1 --2- 172.21.15.2:0/3940737337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e3c108b10 0x7f9e3c19d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:24.284 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.493+0000 7f9e42396640 1 --2- 172.21.15.2:0/3940737337 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e3c06bc00 0x7f9e3c199670 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:24.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.493+0000 7f9e42396640 1 --2- 172.21.15.2:0/3940737337 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e3c06b260 0x7f9e3c199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.493+0000 7f9e42396640 1 -- 172.21.15.2:0/3940737337 >> 172.21.15.2:0/3940737337 conn(0x7f9e3c0788c0 msgr2=0x7f9e3c100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:24.384 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.493+0000 7f9e42396640 1 -- 172.21.15.2:0/3940737337 shutdown_connections 2024-05-14T17:12:24.418 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:04.494+0000 7f9e42396640 1 -- 172.21.15.2:0/3940737337 wait complete. 2024-05-14T17:12:24.477 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:11:04 smithi002 ceph-mon[31538]: pgmap v512: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-05-14T17:12:24.515 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:11:04 smithi002 ceph-mon[31538]: osdmap e100: 12 total, 12 up, 12 in 2024-05-14T17:12:24.560 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:11:06 smithi002 ceph-mon[31538]: pgmap v514: 97 pgs: 97 active+clean; 4.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-05-14T17:12:24.593 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:11:08 smithi002 ceph-mon[31538]: pgmap v515: 97 pgs: 97 active+clean; 4.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-05-14T17:12:24.626 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:11:10 smithi002 ceph-mon[31538]: pgmap v516: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 89 KiB/s wr, 32 op/s 2024-05-14T17:12:24.660 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:11:10 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:12:24.710 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:11:10 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:12:24.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:11:10 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:12:24.776 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:11:10 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:12:24.810 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:11:12 smithi002 ceph-mon[31538]: pgmap v517: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 89 KiB/s wr, 32 op/s 2024-05-14T17:12:24.843 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:11:12 smithi002 ceph-mon[31538]: osdmap e101: 12 total, 12 up, 12 in 2024-05-14T17:12:24.877 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:11:14 smithi002 ceph-mon[31538]: pgmap v519: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.0 MiB/s wr, 691 op/s 2024-05-14T17:12:24.910 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:11:16 smithi002 ceph-mon[31538]: pgmap v520: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.8 MiB/s wr, 774 op/s 2024-05-14T17:12:24.944 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:12:16 smithi123 ceph-mon[35971]: pgmap v551: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 9 op/s 2024-05-14T17:12:24.977 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:12:18 smithi123 ceph-mon[35971]: osdmap e103: 12 total, 12 up, 12 in 2024-05-14T17:12:25.010 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:12:18 smithi123 ceph-mon[35971]: pgmap v553: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 7 op/s 2024-05-14T17:12:25.043 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:12:20 smithi123 ceph-mon[35971]: pgmap v554: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 9 op/s 2024-05-14T17:12:25.089 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:12:22 smithi123 ceph-mon[35971]: pgmap v555: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 9 op/s 2024-05-14T17:12:25.139 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:12:24 smithi123 ceph-mon[35971]: pgmap v556: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 KiB/s wr, 7 op/s 2024-05-14T17:12:25.191 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:12:16 smithi151 ceph-mon[36066]: pgmap v551: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 9 op/s 2024-05-14T17:12:25.227 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:12:18 smithi151 ceph-mon[36066]: osdmap e103: 12 total, 12 up, 12 in 2024-05-14T17:12:25.260 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:12:18 smithi151 ceph-mon[36066]: pgmap v553: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 7 op/s 2024-05-14T17:12:25.327 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:12:20 smithi151 ceph-mon[36066]: pgmap v554: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 9 op/s 2024-05-14T17:12:25.360 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:12:22 smithi151 ceph-mon[36066]: pgmap v555: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 9 op/s 2024-05-14T17:12:25.394 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:12:24 smithi151 ceph-mon[36066]: pgmap v556: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 KiB/s wr, 7 op/s 2024-05-14T17:12:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.594+0000 7fb7a7fda640 1 -- 172.21.15.2:0/1681037991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7a0103d20 msgr2=0x7fb7a0104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.594+0000 7fb7a7fda640 1 --2- 172.21.15.2:0/1681037991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7a0103d20 0x7fb7a0104100 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb79c009ca0 tx=0x7fb79c02a950 comp rx=0 tx=0).stop 2024-05-14T17:12:25.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.594+0000 7fb7a7fda640 1 -- 172.21.15.2:0/1681037991 shutdown_connections 2024-05-14T17:12:25.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.594+0000 7fb7a7fda640 1 --2- 172.21.15.2:0/1681037991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7a010d060 0x7fb7a010f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:25.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.594+0000 7fb7a7fda640 1 --2- 172.21.15.2:0/1681037991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7a0104640 0x7fb7a010cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:25.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.594+0000 7fb7a7fda640 1 --2- 172.21.15.2:0/1681037991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7a0103d20 0x7fb7a0104100 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.594+0000 7fb7a7fda640 1 -- 172.21.15.2:0/1681037991 >> 172.21.15.2:0/1681037991 conn(0x7fb7a0078ad0 msgr2=0x7fb7a0078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.594+0000 7fb7a7fda640 1 -- 172.21.15.2:0/1681037991 shutdown_connections 2024-05-14T17:12:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.594+0000 7fb7a7fda640 1 -- 172.21.15.2:0/1681037991 wait complete. 2024-05-14T17:12:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.594+0000 7fb7a7fda640 1 Processor -- start 2024-05-14T17:12:25.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.595+0000 7fb7a7fda640 1 -- start start 2024-05-14T17:12:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.595+0000 7fb7a7fda640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7a0103d20 0x7fb7a0199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.595+0000 7fb7a7fda640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7a0104640 0x7fb7a01997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.595+0000 7fb7a7fda640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7a010d060 0x7fb7a019dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:25.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.595+0000 7fb7a7fda640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a0112d10 con 0x7fb7a0104640 2024-05-14T17:12:25.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.595+0000 7fb7a7fda640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a0112b90 con 0x7fb7a010d060 2024-05-14T17:12:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.595+0000 7fb7a7fda640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a0112e90 con 0x7fb7a0103d20 2024-05-14T17:12:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.595+0000 7fb7a5575640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7a0104640 0x7fb7a01997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:26.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.595+0000 7fb7a5575640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7a0104640 0x7fb7a01997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57432/0 (socket says 172.21.15.2:57432) 2024-05-14T17:12:26.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.595+0000 7fb7a5575640 1 -- 172.21.15.2:0/120347227 learned_addr learned my addr 172.21.15.2:0/120347227 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:12:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.595+0000 7fb7a5d76640 1 --2- 172.21.15.2:0/120347227 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7a0103d20 0x7fb7a0199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.595+0000 7fb7a6577640 1 --2- 172.21.15.2:0/120347227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7a010d060 0x7fb7a019dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.595+0000 7fb7a5575640 1 -- 172.21.15.2:0/120347227 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7a0103d20 msgr2=0x7fb7a0199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.595+0000 7fb7a5575640 1 --2- 172.21.15.2:0/120347227 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7a0103d20 0x7fb7a0199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:26.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.595+0000 7fb7a5575640 1 -- 172.21.15.2:0/120347227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7a010d060 msgr2=0x7fb7a019dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:26.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.595+0000 7fb7a5575640 1 --2- 172.21.15.2:0/120347227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7a010d060 0x7fb7a019dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:26.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.595+0000 7fb7a5575640 1 -- 172.21.15.2:0/120347227 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7a019e210 con 0x7fb7a0104640 2024-05-14T17:12:26.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.596+0000 7fb7a5575640 1 --2- 172.21.15.2:0/120347227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7a0104640 0x7fb7a01997b0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb79000e9f0 tx=0x7fb79000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:26.453 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.596+0000 7fb796ffd640 1 -- 172.21.15.2:0/120347227 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79000cd80 con 0x7fb7a0104640 2024-05-14T17:12:26.503 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.596+0000 7fb796ffd640 1 -- 172.21.15.2:0/120347227 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7900054d0 con 0x7fb7a0104640 2024-05-14T17:12:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.596+0000 7fb7a7fda640 1 -- 172.21.15.2:0/120347227 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7a019e500 con 0x7fb7a0104640 2024-05-14T17:12:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.596+0000 7fb796ffd640 1 -- 172.21.15.2:0/120347227 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb790010690 con 0x7fb7a0104640 2024-05-14T17:12:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.596+0000 7fb7a7fda640 1 -- 172.21.15.2:0/120347227 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7a01a5d90 con 0x7fb7a0104640 2024-05-14T17:12:26.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.598+0000 7fb7aadc6740 1 Processor -- start 2024-05-14T17:12:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.599+0000 7fb796ffd640 1 -- 172.21.15.2:0/120347227 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fb790004e90 con 0x7fb7a0104640 2024-05-14T17:12:26.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.599+0000 7fb7aadc6740 1 -- start start 2024-05-14T17:12:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.599+0000 7fb7aadc6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560393e3c8f0 0x560393e3ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:26.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.599+0000 7fb7aadc6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560393e3d210 0x560393e47590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:26.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.599+0000 7fb7aadc6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560393e47ad0 0x560393e49ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.599+0000 7fb7aadc6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x560393dafc30 con 0x560393e47ad0 2024-05-14T17:12:26.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.599+0000 7fb7aadc6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560393d9f7d0 con 0x560393e3c8f0 2024-05-14T17:12:26.912 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.599+0000 7fb7aadc6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560393da3810 con 0x560393e3d210 2024-05-14T17:12:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.599+0000 7fb7a5575640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560393e47ad0 0x560393e49ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:26.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.599+0000 7fb7a5575640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560393e47ad0 0x560393e49ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57446/0 (socket says 172.21.15.2:57446) 2024-05-14T17:12:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.599+0000 7fb7a5575640 1 -- 172.21.15.2:0/2646883388 learned_addr learned my addr 172.21.15.2:0/2646883388 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:12:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.599+0000 7fb7a5d76640 1 --2- 172.21.15.2:0/2646883388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560393e3d210 0x560393e47590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.599+0000 7fb7a6577640 1 --2- 172.21.15.2:0/2646883388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560393e3c8f0 0x560393e3ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.599+0000 7fb7a5575640 1 -- 172.21.15.2:0/2646883388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560393e3d210 msgr2=0x560393e47590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.599+0000 7fb7a5575640 1 --2- 172.21.15.2:0/2646883388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560393e3d210 0x560393e47590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.599+0000 7fb7a5575640 1 -- 172.21.15.2:0/2646883388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560393e3c8f0 msgr2=0x560393e3ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.599+0000 7fb7a5575640 1 --2- 172.21.15.2:0/2646883388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560393e3c8f0 0x560393e3ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.599+0000 7fb7a5575640 1 -- 172.21.15.2:0/2646883388 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560393ab4210 con 0x560393e47ad0 2024-05-14T17:12:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.600+0000 7fb796ffd640 1 --2- 172.21.15.2:0/120347227 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb78c0b2a90 0x7fb78c0b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.600+0000 7fb7a5575640 1 --2- 172.21.15.2:0/2646883388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560393e47ad0 0x560393e49ec0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb7900e0050 tx=0x7fb7900e6a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eabcc59873cea32 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:27.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.600+0000 7fb7a6577640 1 --2- 172.21.15.2:0/120347227 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb78c0b2a90 0x7fb78c0b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.600+0000 7fb796ffd640 1 -- 172.21.15.2:0/120347227 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7fb7900e2990 con 0x7fb7a0104640 2024-05-14T17:12:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.600+0000 7fb78bfff640 1 -- 172.21.15.2:0/2646883388 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7900ed040 con 0x560393e47ad0 2024-05-14T17:12:27.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.600+0000 7fb78bfff640 1 -- 172.21.15.2:0/2646883388 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7900a9780 con 0x560393e47ad0 2024-05-14T17:12:27.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.600+0000 7fb78bfff640 1 -- 172.21.15.2:0/2646883388 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7900a93e0 con 0x560393e47ad0 2024-05-14T17:12:27.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.601+0000 7fb7aadc6740 1 -- 172.21.15.2:0/2646883388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560393e47ad0 msgr2=0x560393e49ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:27.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.601+0000 7fb7a6577640 1 --2- 172.21.15.2:0/120347227 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb78c0b2a90 0x7fb78c0b4f50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb79800b6c0 tx=0x7fb79800b2e0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:27.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.601+0000 7fb7aadc6740 1 --2- 172.21.15.2:0/2646883388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560393e47ad0 0x560393e49ec0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb7900e0050 tx=0x7fb7900e6a20 comp rx=0 tx=0).stop 2024-05-14T17:12:27.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.601+0000 7fb7aadc6740 1 -- 172.21.15.2:0/2646883388 shutdown_connections 2024-05-14T17:12:27.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.601+0000 7fb7aadc6740 1 --2- 172.21.15.2:0/2646883388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560393e47ad0 0x560393e49ec0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.601+0000 7fb7aadc6740 1 --2- 172.21.15.2:0/2646883388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560393e3d210 0x560393e47590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.601+0000 7fb7aadc6740 1 --2- 172.21.15.2:0/2646883388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560393e3c8f0 0x560393e3ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:27.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.601+0000 7fb7aadc6740 1 -- 172.21.15.2:0/2646883388 >> 172.21.15.2:0/2646883388 conn(0x560393e38e00 msgr2=0x560393e391e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.601+0000 7fb7aadc6740 1 -- 172.21.15.2:0/2646883388 shutdown_connections 2024-05-14T17:12:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.601+0000 7fb7aadc6740 1 -- 172.21.15.2:0/2646883388 wait complete. 2024-05-14T17:12:27.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.601+0000 7fb7aadc6740 1 -- start start 2024-05-14T17:12:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.601+0000 7fb7aadc6740 1 Processor -- start 2024-05-14T17:12:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.602+0000 7fb7aadc6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560393e3c8f0 0x560393e473a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.602+0000 7fb7aadc6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560393e3d210 0x560393e44700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.602+0000 7fb7aadc6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560393e47ad0 0x560393e44c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.602+0000 7fb7aadc6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x560393a85790 con 0x560393e3c8f0 2024-05-14T17:12:28.180 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.602+0000 7fb7aadc6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560393d5b890 con 0x560393e47ad0 2024-05-14T17:12:28.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.602+0000 7fb7aadc6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560393e4a740 con 0x560393e3d210 2024-05-14T17:12:28.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.602+0000 7fb7aadc6740 10 client.0 _ll_register_callbacks cb 0x560393d5ed60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:12:28.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.602+0000 7fb7a5d76640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560393e3c8f0 0x560393e473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:28.384 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.602+0000 7fb7a5d76640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560393e3c8f0 0x560393e473a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57454/0 (socket says 172.21.15.2:57454) 2024-05-14T17:12:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.602+0000 7fb7a5d76640 1 -- 172.21.15.2:0/2112513962 learned_addr learned my addr 172.21.15.2:0/2112513962 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:12:28.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.602+0000 7fb7a5575640 1 --2- 172.21.15.2:0/2112513962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560393e3d210 0x560393e44700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.602+0000 7fb7a6577640 1 --2- 172.21.15.2:0/2112513962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560393e47ad0 0x560393e44c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.602+0000 7fb7a5d76640 1 -- 172.21.15.2:0/2112513962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560393e3d210 msgr2=0x560393e44700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.602+0000 7fb7a5d76640 1 --2- 172.21.15.2:0/2112513962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560393e3d210 0x560393e44700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.602+0000 7fb7a5d76640 1 -- 172.21.15.2:0/2112513962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560393e47ad0 msgr2=0x560393e44c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:28.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.602+0000 7fb7a5d76640 1 --2- 172.21.15.2:0/2112513962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560393e47ad0 0x560393e44c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:28.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.602+0000 7fb7a5d76640 1 -- 172.21.15.2:0/2112513962 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560393b80620 con 0x560393e3c8f0 2024-05-14T17:12:28.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.603+0000 7fb7a5d76640 1 --2- 172.21.15.2:0/2112513962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560393e3c8f0 0x560393e473a0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb79c02ae70 tx=0x7fb79c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70ceac0b5dc959c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.603+0000 7fb7897fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:12:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.603+0000 7fb7897fa640 1 -- 172.21.15.2:0/2112513962 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79c04a060 con 0x560393e3c8f0 2024-05-14T17:12:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.603+0000 7fb7897fa640 1 -- 172.21.15.2:0/2112513962 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7fb79c02fa70 con 0x560393e3c8f0 2024-05-14T17:12:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.603+0000 7fb7aadc6740 1 -- 172.21.15.2:0/2112513962 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7900a9780 con 0x560393e3c8f0 2024-05-14T17:12:28.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.603+0000 7fb7897fa640 1 -- 172.21.15.2:0/2112513962 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fb79c034040 con 0x560393e3c8f0 2024-05-14T17:12:28.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.603+0000 7fb7aadc6740 10 client.16959 fetch_fsmap learned FSMap version 52 2024-05-14T17:12:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.603+0000 7fb7aadc6740 1 -- 172.21.15.2:0/2112513962 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x560393e56140 con 0x560393e3c8f0 2024-05-14T17:12:28.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.603+0000 7fb7897fa640 1 -- 172.21.15.2:0/2112513962 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7900a9780 con 0x560393e3c8f0 2024-05-14T17:12:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.603+0000 7fb7897fa640 1 -- 172.21.15.2:0/2112513962 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 52) ==== 8346+0+0 (secure 0 0 0) 0x7fb79c045460 con 0x560393e3c8f0 2024-05-14T17:12:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.603+0000 7fb7aadc6740 10 client.16959 fetch_fsmap finished waiting for FSMap version 52 2024-05-14T17:12:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.603+0000 7fb7aadc6740 10 client.16959 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:12:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.603+0000 7fb7aadc6740 20 client.16959 populate_metadata read hostname 'smithi002' 2024-05-14T17:12:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.603+0000 7fb7aadc6740 1 --2- 172.21.15.2:0/2112513962 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x560393e57650 0x560393e59ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:29.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.603+0000 7fb7aadc6740 4 client.16959 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:12:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.603+0000 7fb7aadc6740 1 -- 172.21.15.2:0/2112513962 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560393db1b80 con 0x560393e57650 2024-05-14T17:12:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.604+0000 7fb7a5575640 1 --2- 172.21.15.2:0/2112513962 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x560393e57650 0x560393e59ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:29.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.604+0000 7fb7897fa640 10 client.16959 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:12:29.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.604+0000 7fb7a5575640 1 --2- 172.21.15.2:0/2112513962 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x560393e57650 0x560393e59ad0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2c47e54b974c822 server_cookie=ba098056a36e6a2b in_seq=0 out_seq=0 2024-05-14T17:12:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.605+0000 7fb7897fa640 1 -- 172.21.15.2:0/2112513962 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560393db1b80 con 0x560393e57650 2024-05-14T17:12:29.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.605+0000 7fb7897fa640 10 client.16959 handle_command_reply: tid=0 2024-05-14T17:12:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.605+0000 7fb7aadc6740 1 client.16959 shutdown 2024-05-14T17:12:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.605+0000 7fb7aadc6740 1 -- 172.21.15.2:0/2112513962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560393e3c8f0 msgr2=0x560393e473a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.605+0000 7fb7aadc6740 1 --2- 172.21.15.2:0/2112513962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560393e3c8f0 0x560393e473a0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb79c02ae70 tx=0x7fb79c003ba0 comp rx=0 tx=0).stop 2024-05-14T17:12:29.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.605+0000 7fb7aadc6740 1 -- 172.21.15.2:0/2112513962 shutdown_connections 2024-05-14T17:12:29.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.606+0000 7fb7aadc6740 1 --2- 172.21.15.2:0/2112513962 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x560393e57650 0x560393e59ad0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:29.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.606+0000 7fb7aadc6740 1 --2- 172.21.15.2:0/2112513962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560393e47ad0 0x560393e44c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.606+0000 7fb7aadc6740 1 --2- 172.21.15.2:0/2112513962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560393e3d210 0x560393e44700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:29.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.606+0000 7fb7aadc6740 1 --2- 172.21.15.2:0/2112513962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560393e3c8f0 0x560393e473a0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:29.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.606+0000 7fb7897fa640 0 client.16959 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:12:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.606+0000 7fb7aadc6740 1 -- 172.21.15.2:0/2112513962 >> 172.21.15.2:0/2112513962 conn(0x560393e38e00 msgr2=0x560393e38100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.606+0000 7fb7aadc6740 1 -- 172.21.15.2:0/2112513962 shutdown_connections 2024-05-14T17:12:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.606+0000 7fb7aadc6740 1 -- 172.21.15.2:0/2112513962 wait complete. 2024-05-14T17:12:29.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.606+0000 7fb7aadc6740 20 client.16959 trim_cache size 0 max 16384 2024-05-14T17:12:29.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.621+0000 7fb7aadc6740 1 Processor -- start 2024-05-14T17:12:29.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.621+0000 7fb7aadc6740 1 -- start start 2024-05-14T17:12:29.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.621+0000 7fb7aadc6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560393e3c8f0 0x560393e53610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:30.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.621+0000 7fb7aadc6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560393e3d210 0x560393e4fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:30.090 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.621+0000 7fb7aadc6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560393e47ad0 0x560393e3fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.621+0000 7fb7aadc6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a01b0f70 con 0x560393e3d210 2024-05-14T17:12:30.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.621+0000 7fb7aadc6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a01ae840 con 0x560393e47ad0 2024-05-14T17:12:30.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.621+0000 7fb7aadc6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a01ac6b0 con 0x560393e3c8f0 2024-05-14T17:12:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.621+0000 7fb7a5575640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560393e3d210 0x560393e4fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.621+0000 7fb7a5575640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560393e3d210 0x560393e4fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57462/0 (socket says 172.21.15.2:57462) 2024-05-14T17:12:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.621+0000 7fb7a5575640 1 -- 172.21.15.2:0/1356565882 learned_addr learned my addr 172.21.15.2:0/1356565882 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:12:30.382 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.622+0000 7fb7a5575640 1 -- 172.21.15.2:0/1356565882 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560393e3c8f0 msgr2=0x560393e53610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.622+0000 7fb7a5575640 1 --2- 172.21.15.2:0/1356565882 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560393e3c8f0 0x560393e53610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.622+0000 7fb7a5575640 1 -- 172.21.15.2:0/1356565882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560393e47ad0 msgr2=0x560393e3fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.622+0000 7fb7a6577640 1 --2- 172.21.15.2:0/1356565882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560393e47ad0 0x560393e3fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.622+0000 7fb7a5575640 1 --2- 172.21.15.2:0/1356565882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560393e47ad0 0x560393e3fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.622+0000 7fb7a5575640 1 -- 172.21.15.2:0/1356565882 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560393e42c50 con 0x560393e3d210 2024-05-14T17:12:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.622+0000 7fb7a5575640 1 --2- 172.21.15.2:0/1356565882 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560393e3d210 0x560393e4fd70 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fb79000e9c0 tx=0x7fb790004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6550de8fab7acb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.622+0000 7fb78b7fe640 1 -- 172.21.15.2:0/1356565882 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7900ed040 con 0x560393e3d210 2024-05-14T17:12:30.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.622+0000 7fb78b7fe640 1 -- 172.21.15.2:0/1356565882 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7900ea070 con 0x560393e3d210 2024-05-14T17:12:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.622+0000 7fb78b7fe640 1 -- 172.21.15.2:0/1356565882 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb790005180 con 0x560393e3d210 2024-05-14T17:12:30.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.622+0000 7fb7aadc6740 1 -- 172.21.15.2:0/1356565882 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560393e3d210 msgr2=0x560393e4fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.622+0000 7fb7aadc6740 1 --2- 172.21.15.2:0/1356565882 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560393e3d210 0x560393e4fd70 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fb79000e9c0 tx=0x7fb790004a90 comp rx=0 tx=0).stop 2024-05-14T17:12:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.622+0000 7fb7aadc6740 1 -- 172.21.15.2:0/1356565882 shutdown_connections 2024-05-14T17:12:31.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.623+0000 7fb7aadc6740 1 --2- 172.21.15.2:0/1356565882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560393e47ad0 0x560393e3fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:31.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.623+0000 7fb7aadc6740 1 --2- 172.21.15.2:0/1356565882 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560393e3d210 0x560393e4fd70 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.623+0000 7fb7aadc6740 1 --2- 172.21.15.2:0/1356565882 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560393e3c8f0 0x560393e53610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.623+0000 7fb7aadc6740 1 -- 172.21.15.2:0/1356565882 >> 172.21.15.2:0/1356565882 conn(0x560393e38e00 msgr2=0x560393e3f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:31.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.623+0000 7fb7aadc6740 1 -- 172.21.15.2:0/1356565882 shutdown_connections 2024-05-14T17:12:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.623+0000 7fb7aadc6740 1 -- 172.21.15.2:0/1356565882 wait complete. 2024-05-14T17:12:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.623+0000 7fb7aadc6740 1 -- start start 2024-05-14T17:12:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.623+0000 7fb7aadc6740 1 Processor -- start 2024-05-14T17:12:31.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.623+0000 7fb7aadc6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560393e3c8f0 0x560393e40970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.623+0000 7fb7aadc6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560393e3d210 0x560393e40eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.623+0000 7fb7aadc6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560393e47ad0 0x560393e413f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:31.383 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.623+0000 7fb7aadc6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a01acd00 con 0x560393e3c8f0 2024-05-14T17:12:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.623+0000 7fb7aadc6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a01ae170 con 0x560393e47ad0 2024-05-14T17:12:31.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.623+0000 7fb7aadc6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a01ad3d0 con 0x560393e3d210 2024-05-14T17:12:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.623+0000 7fb7aadc6740 10 client.0 _ll_register_callbacks cb 0x7fb79c009950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:12:31.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.623+0000 7fb7a5d76640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560393e3c8f0 0x560393e40970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.624+0000 7fb7a5d76640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560393e3c8f0 0x560393e40970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57468/0 (socket says 172.21.15.2:57468) 2024-05-14T17:12:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.624+0000 7fb7a5d76640 1 -- 172.21.15.2:0/1222718815 learned_addr learned my addr 172.21.15.2:0/1222718815 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:12:31.666 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.624+0000 7fb7a5d76640 1 -- 172.21.15.2:0/1222718815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560393e3d210 msgr2=0x560393e40eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.624+0000 7fb7a5d76640 1 --2- 172.21.15.2:0/1222718815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560393e3d210 0x560393e40eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.624+0000 7fb7a5d76640 1 -- 172.21.15.2:0/1222718815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560393e47ad0 msgr2=0x560393e413f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.624+0000 7fb7a5d76640 1 --2- 172.21.15.2:0/1222718815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560393e47ad0 0x560393e413f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:31.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.624+0000 7fb7a5d76640 1 -- 172.21.15.2:0/1222718815 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560393e41b30 con 0x560393e3c8f0 2024-05-14T17:12:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.624+0000 7fb7a5d76640 1 --2- 172.21.15.2:0/1222718815 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560393e3c8f0 0x560393e40970 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fb79c009bf0 tx=0x7fb79c045d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c901970ceac4af6e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.624+0000 7fb7897fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:12:31.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.624+0000 7fb7897fa640 1 -- 172.21.15.2:0/1222718815 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79c04a040 con 0x560393e3c8f0 2024-05-14T17:12:31.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.624+0000 7fb7897fa640 1 -- 172.21.15.2:0/1222718815 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7fb79c02fc80 con 0x560393e3c8f0 2024-05-14T17:12:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.624+0000 7fb7aadc6740 1 -- 172.21.15.2:0/1222718815 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7900ea070 con 0x560393e3c8f0 2024-05-14T17:12:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.624+0000 7fb7897fa640 1 -- 172.21.15.2:0/1222718815 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fb79c005bd0 con 0x560393e3c8f0 2024-05-14T17:12:32.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.624+0000 7fb7aadc6740 10 client.16965 fetch_fsmap learned FSMap version 52 2024-05-14T17:12:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.624+0000 7fb7aadc6740 1 -- 172.21.15.2:0/1222718815 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x560393ec64d0 con 0x560393e3c8f0 2024-05-14T17:12:32.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.624+0000 7fb7897fa640 1 -- 172.21.15.2:0/1222718815 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7900ea070 con 0x560393e3c8f0 2024-05-14T17:12:32.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.625+0000 7fb7897fa640 1 -- 172.21.15.2:0/1222718815 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 52) ==== 8346+0+0 (secure 0 0 0) 0x7fb79c030440 con 0x560393e3c8f0 2024-05-14T17:12:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.625+0000 7fb7aadc6740 10 client.16965 fetch_fsmap finished waiting for FSMap version 52 2024-05-14T17:12:32.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.625+0000 7fb7aadc6740 10 client.16965 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:12:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.625+0000 7fb7aadc6740 20 client.16965 populate_metadata read hostname 'smithi002' 2024-05-14T17:12:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.625+0000 7fb7aadc6740 1 --2- 172.21.15.2:0/1222718815 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x560393ec6d30 0x560393ec9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.625+0000 7fb7aadc6740 4 client.16965 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:12:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.625+0000 7fb7aadc6740 1 -- 172.21.15.2:0/1222718815 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560393e44fd0 con 0x560393ec6d30 2024-05-14T17:12:32.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.625+0000 7fb7a5575640 1 --2- 172.21.15.2:0/1222718815 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x560393ec6d30 0x560393ec9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.625+0000 7fb7a5575640 1 --2- 172.21.15.2:0/1222718815 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x560393ec6d30 0x560393ec9190 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea8a1031f236072b server_cookie=81ea0631ced36f9c in_seq=0 out_seq=0 2024-05-14T17:12:32.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.625+0000 7fb7897fa640 10 client.16965 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:12:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.636+0000 7fb7897fa640 1 -- 172.21.15.2:0/1222718815 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560393e44fd0 con 0x560393ec6d30 2024-05-14T17:12:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.636+0000 7fb7897fa640 10 client.16965 handle_command_reply: tid=0 2024-05-14T17:12:32.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.636+0000 7fb7aadc6740 1 client.16965 shutdown 2024-05-14T17:12:32.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.636+0000 7fb7aadc6740 1 -- 172.21.15.2:0/1222718815 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560393e3c8f0 msgr2=0x560393e40970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:32.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.636+0000 7fb7aadc6740 1 --2- 172.21.15.2:0/1222718815 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560393e3c8f0 0x560393e40970 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fb79c009bf0 tx=0x7fb79c045d00 comp rx=0 tx=0).stop 2024-05-14T17:12:32.734 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.636+0000 7fb7aadc6740 1 -- 172.21.15.2:0/1222718815 shutdown_connections 2024-05-14T17:12:32.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.636+0000 7fb7aadc6740 1 --2- 172.21.15.2:0/1222718815 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x560393ec6d30 0x560393ec9190 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.636+0000 7fb7aadc6740 1 --2- 172.21.15.2:0/1222718815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560393e47ad0 0x560393e413f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:32.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.636+0000 7fb7aadc6740 1 --2- 172.21.15.2:0/1222718815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560393e3d210 0x560393e40eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.636+0000 7fb7aadc6740 1 --2- 172.21.15.2:0/1222718815 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560393e3c8f0 0x560393e40970 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.636+0000 7fb7897fa640 0 client.16965 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:12:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.636+0000 7fb7aadc6740 1 -- 172.21.15.2:0/1222718815 >> 172.21.15.2:0/1222718815 conn(0x560393e38e00 msgr2=0x560393e3eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.636+0000 7fb7aadc6740 1 -- 172.21.15.2:0/1222718815 shutdown_connections 2024-05-14T17:12:33.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.636+0000 7fb7aadc6740 1 -- 172.21.15.2:0/1222718815 wait complete. 2024-05-14T17:12:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.636+0000 7fb7aadc6740 20 client.16965 trim_cache size 0 max 16384 2024-05-14T17:12:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:12:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:12:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:12:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.637+0000 7fb7a7fda640 1 -- 172.21.15.2:0/120347227 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb78c0b2a90 msgr2=0x7fb78c0b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:33.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.637+0000 7fb7a7fda640 1 --2- 172.21.15.2:0/120347227 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb78c0b2a90 0x7fb78c0b4f50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb79800b6c0 tx=0x7fb79800b2e0 comp rx=0 tx=0).stop 2024-05-14T17:12:33.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.637+0000 7fb7a7fda640 1 -- 172.21.15.2:0/120347227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7a0104640 msgr2=0x7fb7a01997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:33.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.637+0000 7fb7a7fda640 1 --2- 172.21.15.2:0/120347227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7a0104640 0x7fb7a01997b0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb79000e9f0 tx=0x7fb79000edf0 comp rx=0 tx=0).stop 2024-05-14T17:12:33.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.637+0000 7fb7a7fda640 1 -- 172.21.15.2:0/120347227 shutdown_connections 2024-05-14T17:12:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.637+0000 7fb7a7fda640 1 --2- 172.21.15.2:0/120347227 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb78c0b2a90 0x7fb78c0b4f50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.637+0000 7fb7a7fda640 1 --2- 172.21.15.2:0/120347227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7a010d060 0x7fb7a019dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:33.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.637+0000 7fb7a7fda640 1 --2- 172.21.15.2:0/120347227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7a0104640 0x7fb7a01997b0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.637+0000 7fb7a7fda640 1 --2- 172.21.15.2:0/120347227 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7a0103d20 0x7fb7a0199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:33.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.637+0000 7fb7a7fda640 1 -- 172.21.15.2:0/120347227 >> 172.21.15.2:0/120347227 conn(0x7fb7a0078ad0 msgr2=0x7fb7a00feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:33.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.637+0000 7fb7a7fda640 1 -- 172.21.15.2:0/120347227 shutdown_connections 2024-05-14T17:12:33.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:17.637+0000 7fb7a7fda640 1 -- 172.21.15.2:0/120347227 wait complete. 2024-05-14T17:12:33.711 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:11:18 smithi002 ceph-mon[31538]: pgmap v521: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.8 MiB/s wr, 774 op/s 2024-05-14T17:12:33.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:11:19 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:12:33.777 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:11:20 smithi002 ceph-mon[31538]: pgmap v522: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 7.8 MiB/s wr, 774 op/s 2024-05-14T17:12:33.827 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:11:22 smithi002 ceph-mon[31538]: pgmap v523: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 7.8 MiB/s wr, 774 op/s 2024-05-14T17:12:33.884 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:11:24 smithi002 ceph-mon[31538]: pgmap v524: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.9 MiB/s wr, 691 op/s 2024-05-14T17:12:33.918 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:11:26 smithi002 ceph-mon[31538]: pgmap v525: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 846 KiB/s wr, 88 op/s 2024-05-14T17:12:33.952 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:11:28 smithi002 ceph-mon[31538]: pgmap v526: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 44 KiB/s wr, 9 op/s 2024-05-14T17:12:33.988 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:12:26 smithi123 ceph-mon[35971]: pgmap v557: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 KiB/s wr, 7 op/s 2024-05-14T17:12:34.019 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:12:28 smithi123 ceph-mon[35971]: pgmap v558: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.6 KiB/s wr, 6 op/s 2024-05-14T17:12:34.052 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:12:30 smithi123 ceph-mon[35971]: pgmap v559: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 11 op/s 2024-05-14T17:12:34.085 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:12:32 smithi123 ceph-mon[35971]: pgmap v560: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-05-14T17:12:34.119 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:12:26 smithi151 ceph-mon[36066]: pgmap v557: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 KiB/s wr, 7 op/s 2024-05-14T17:12:34.161 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:12:28 smithi151 ceph-mon[36066]: pgmap v558: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.6 KiB/s wr, 6 op/s 2024-05-14T17:12:34.204 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:12:30 smithi151 ceph-mon[36066]: pgmap v559: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 11 op/s 2024-05-14T17:12:34.235 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:12:32 smithi151 ceph-mon[36066]: pgmap v560: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-05-14T17:12:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.738+0000 7fb70d872640 1 -- 172.21.15.2:0/2244884586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb708101980 msgr2=0x7fb708101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:34.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.738+0000 7fb70d872640 1 --2- 172.21.15.2:0/2244884586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb708101980 0x7fb708101e00 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0009930 tx=0x7fb6f002a420 comp rx=0 tx=0).stop 2024-05-14T17:12:34.352 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.738+0000 7fb70d872640 1 -- 172.21.15.2:0/2244884586 shutdown_connections 2024-05-14T17:12:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.738+0000 7fb70d872640 1 --2- 172.21.15.2:0/2244884586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb708102340 0x7fb70810e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:34.419 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.738+0000 7fb70d872640 1 --2- 172.21.15.2:0/2244884586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb708101980 0x7fb708101e00 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.738+0000 7fb70d872640 1 --2- 172.21.15.2:0/2244884586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb708107940 0x7fb708107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.738+0000 7fb70d872640 1 -- 172.21.15.2:0/2244884586 >> 172.21.15.2:0/2244884586 conn(0x7fb708078ad0 msgr2=0x7fb708078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.738+0000 7fb70d872640 1 -- 172.21.15.2:0/2244884586 shutdown_connections 2024-05-14T17:12:34.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.738+0000 7fb70d872640 1 -- 172.21.15.2:0/2244884586 wait complete. 2024-05-14T17:12:34.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.739+0000 7fb70d872640 1 Processor -- start 2024-05-14T17:12:34.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.739+0000 7fb70d872640 1 -- start start 2024-05-14T17:12:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.739+0000 7fb70d872640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb708101980 0x7fb708199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.739+0000 7fb70d872640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb708102340 0x7fb7081996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.739+0000 7fb70d872640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb708107940 0x7fb70819da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.739+0000 7fb70d872640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb708111bd0 con 0x7fb708107940 2024-05-14T17:12:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.739+0000 7fb70d872640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb708111a50 con 0x7fb708101980 2024-05-14T17:12:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.739+0000 7fb70d872640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb708111d50 con 0x7fb708102340 2024-05-14T17:12:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.739+0000 7fb7077fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb708107940 0x7fb70819da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.739+0000 7fb7077fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb708107940 0x7fb70819da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56324/0 (socket says 172.21.15.2:56324) 2024-05-14T17:12:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.739+0000 7fb7077fe640 1 -- 172.21.15.2:0/3352466682 learned_addr learned my addr 172.21.15.2:0/3352466682 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:12:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.739+0000 7fb706ffd640 1 --2- 172.21.15.2:0/3352466682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb708101980 0x7fb708199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.740+0000 7fb7067fc640 1 --2- 172.21.15.2:0/3352466682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb708102340 0x7fb7081996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.740+0000 7fb7077fe640 1 -- 172.21.15.2:0/3352466682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb708102340 msgr2=0x7fb7081996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.740+0000 7fb7077fe640 1 --2- 172.21.15.2:0/3352466682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb708102340 0x7fb7081996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.740+0000 7fb7077fe640 1 -- 172.21.15.2:0/3352466682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb708101980 msgr2=0x7fb708199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.740+0000 7fb7077fe640 1 --2- 172.21.15.2:0/3352466682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb708101980 0x7fb708199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.740+0000 7fb7077fe640 1 -- 172.21.15.2:0/3352466682 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb70819e170 con 0x7fb708107940 2024-05-14T17:12:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.740+0000 7fb7067fc640 1 --2- 172.21.15.2:0/3352466682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb708102340 0x7fb7081996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:12:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.740+0000 7fb7077fe640 1 --2- 172.21.15.2:0/3352466682 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb708107940 0x7fb70819da40 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fb70000eaa0 tx=0x7fb70000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:35.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.740+0000 7fb70c870640 1 -- 172.21.15.2:0/3352466682 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb700004d20 con 0x7fb708107940 2024-05-14T17:12:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.740+0000 7fb70c870640 1 -- 172.21.15.2:0/3352466682 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb700005020 con 0x7fb708107940 2024-05-14T17:12:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.740+0000 7fb70d872640 1 -- 172.21.15.2:0/3352466682 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb70819e460 con 0x7fb708107940 2024-05-14T17:12:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.740+0000 7fb70c870640 1 -- 172.21.15.2:0/3352466682 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7000116b0 con 0x7fb708107940 2024-05-14T17:12:35.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.740+0000 7fb70d872640 1 -- 172.21.15.2:0/3352466682 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7081a5cf0 con 0x7fb708107940 2024-05-14T17:12:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.743+0000 7fb71058d740 1 Processor -- start 2024-05-14T17:12:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.743+0000 7fb70c870640 1 -- 172.21.15.2:0/3352466682 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fb7000051d0 con 0x7fb708107940 2024-05-14T17:12:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.743+0000 7fb71058d740 1 -- start start 2024-05-14T17:12:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.743+0000 7fb71058d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ab98e8f0 0x5574ab98ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.743+0000 7fb71058d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574ab98f210 0x5574ab999590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.743+0000 7fb71058d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574ab999ad0 0x5574ab99bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:35.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.743+0000 7fb71058d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5574ab901c30 con 0x5574ab999ad0 2024-05-14T17:12:35.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.743+0000 7fb71058d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5574ab8f17d0 con 0x5574ab98e8f0 2024-05-14T17:12:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.743+0000 7fb71058d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5574ab8f5810 con 0x5574ab98f210 2024-05-14T17:12:35.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.743+0000 7fb7067fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574ab999ad0 0x5574ab99bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.743+0000 7fb7067fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574ab999ad0 0x5574ab99bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56328/0 (socket says 172.21.15.2:56328) 2024-05-14T17:12:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.743+0000 7fb7067fc640 1 -- 172.21.15.2:0/1569313197 learned_addr learned my addr 172.21.15.2:0/1569313197 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:12:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.743+0000 7fb7067fc640 1 --2- 172.21.15.2:0/1569313197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ab98e8f0 0x5574ab98ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:36.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.743+0000 7fb706ffd640 1 --2- 172.21.15.2:0/1569313197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574ab98f210 0x5574ab999590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:36.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.744+0000 7fb7067fc640 1 -- 172.21.15.2:0/1569313197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574ab98f210 msgr2=0x5574ab999590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:36.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.744+0000 7fb7067fc640 1 --2- 172.21.15.2:0/1569313197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574ab98f210 0x5574ab999590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.744+0000 7fb7067fc640 1 -- 172.21.15.2:0/1569313197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ab98e8f0 msgr2=0x5574ab98ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.744+0000 7fb7067fc640 1 --2- 172.21.15.2:0/1569313197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ab98e8f0 0x5574ab98ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.744+0000 7fb7067fc640 1 -- 172.21.15.2:0/1569313197 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574ab606210 con 0x5574ab999ad0 2024-05-14T17:12:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.744+0000 7fb70c870640 1 --2- 172.21.15.2:0/3352466682 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb6f40b2a10 0x7fb6f40b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.744+0000 7fb7067fc640 1 --2- 172.21.15.2:0/1569313197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574ab999ad0 0x5574ab99bec0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fb6f00057c0 tx=0x7fb6f00058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce0e7198f4518514 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.744+0000 7fb706ffd640 1 --2- 172.21.15.2:0/3352466682 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb6f40b2a10 0x7fb6f40b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.744+0000 7fb70c870640 1 -- 172.21.15.2:0/3352466682 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7fb7000e2840 con 0x7fb708107940 2024-05-14T17:12:36.388 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.744+0000 7fb6fdffb640 1 -- 172.21.15.2:0/1569313197 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0035040 con 0x5574ab999ad0 2024-05-14T17:12:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.744+0000 7fb6fdffb640 1 -- 172.21.15.2:0/1569313197 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6f00035f0 con 0x5574ab999ad0 2024-05-14T17:12:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.744+0000 7fb6fdffb640 1 -- 172.21.15.2:0/1569313197 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f00044e0 con 0x5574ab999ad0 2024-05-14T17:12:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.745+0000 7fb71058d740 1 -- 172.21.15.2:0/1569313197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574ab999ad0 msgr2=0x5574ab99bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.745+0000 7fb706ffd640 1 --2- 172.21.15.2:0/3352466682 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb6f40b2a10 0x7fb6f40b4ed0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x5574ab98fe90 tx=0x7fb6f800a2d0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:36.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.745+0000 7fb71058d740 1 --2- 172.21.15.2:0/1569313197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574ab999ad0 0x5574ab99bec0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fb6f00057c0 tx=0x7fb6f00058d0 comp rx=0 tx=0).stop 2024-05-14T17:12:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.745+0000 7fb71058d740 1 -- 172.21.15.2:0/1569313197 shutdown_connections 2024-05-14T17:12:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.745+0000 7fb71058d740 1 --2- 172.21.15.2:0/1569313197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574ab999ad0 0x5574ab99bec0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:36.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.745+0000 7fb71058d740 1 --2- 172.21.15.2:0/1569313197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574ab98f210 0x5574ab999590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:36.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.745+0000 7fb71058d740 1 --2- 172.21.15.2:0/1569313197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ab98e8f0 0x5574ab98ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:36.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.745+0000 7fb71058d740 1 -- 172.21.15.2:0/1569313197 >> 172.21.15.2:0/1569313197 conn(0x5574ab98ae00 msgr2=0x5574ab98b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.745+0000 7fb71058d740 1 -- 172.21.15.2:0/1569313197 shutdown_connections 2024-05-14T17:12:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.745+0000 7fb71058d740 1 -- 172.21.15.2:0/1569313197 wait complete. 2024-05-14T17:12:36.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.745+0000 7fb71058d740 1 -- start start 2024-05-14T17:12:36.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.746+0000 7fb71058d740 1 Processor -- start 2024-05-14T17:12:36.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.746+0000 7fb71058d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574ab98e8f0 0x5574ab9993d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:36.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.746+0000 7fb71058d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ab98f210 0x5574ab9966c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:36.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.746+0000 7fb71058d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574ab999ad0 0x5574ab996c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.746+0000 7fb71058d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5574ab5d7790 con 0x5574ab98e8f0 2024-05-14T17:12:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.746+0000 7fb71058d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5574ab8ad890 con 0x5574ab98f210 2024-05-14T17:12:37.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.746+0000 7fb71058d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5574ab99c740 con 0x5574ab999ad0 2024-05-14T17:12:37.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.746+0000 7fb71058d740 10 client.0 _ll_register_callbacks cb 0x5574ab8b0d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:12:37.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.746+0000 7fb706ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574ab98e8f0 0x5574ab9993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.746+0000 7fb706ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574ab98e8f0 0x5574ab9993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56342/0 (socket says 172.21.15.2:56342) 2024-05-14T17:12:37.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.746+0000 7fb706ffd640 1 -- 172.21.15.2:0/3242749316 learned_addr learned my addr 172.21.15.2:0/3242749316 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:12:37.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.746+0000 7fb7077fe640 1 --2- 172.21.15.2:0/3242749316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574ab999ad0 0x5574ab996c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.746+0000 7fb7067fc640 1 --2- 172.21.15.2:0/3242749316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ab98f210 0x5574ab9966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.746+0000 7fb706ffd640 1 -- 172.21.15.2:0/3242749316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574ab999ad0 msgr2=0x5574ab996c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:37.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.746+0000 7fb706ffd640 1 --2- 172.21.15.2:0/3242749316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574ab999ad0 0x5574ab996c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:37.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.746+0000 7fb706ffd640 1 -- 172.21.15.2:0/3242749316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ab98f210 msgr2=0x5574ab9966c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:37.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.746+0000 7fb706ffd640 1 --2- 172.21.15.2:0/3242749316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ab98f210 0x5574ab9966c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.746+0000 7fb706ffd640 1 -- 172.21.15.2:0/3242749316 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574ab6d2620 con 0x5574ab98e8f0 2024-05-14T17:12:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.746+0000 7fb7067fc640 1 --2- 172.21.15.2:0/3242749316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ab98f210 0x5574ab9966c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:12:37.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.747+0000 7fb706ffd640 1 --2- 172.21.15.2:0/3242749316 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574ab98e8f0 0x5574ab9993d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fb6f801cc10 tx=0x7fb6f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69750f78e2c31865 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:37.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.747+0000 7fb6e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:12:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.747+0000 7fb6e6ffd640 1 -- 172.21.15.2:0/3242749316 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f802c060 con 0x5574ab98e8f0 2024-05-14T17:12:37.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.747+0000 7fb6e6ffd640 1 -- 172.21.15.2:0/3242749316 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7fb6f8016030 con 0x5574ab98e8f0 2024-05-14T17:12:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.747+0000 7fb71058d740 1 -- 172.21.15.2:0/3242749316 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6f00035f0 con 0x5574ab98e8f0 2024-05-14T17:12:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.747+0000 7fb6e6ffd640 1 -- 172.21.15.2:0/3242749316 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fb6f801cd00 con 0x5574ab98e8f0 2024-05-14T17:12:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.747+0000 7fb71058d740 10 client.16977 fetch_fsmap learned FSMap version 53 2024-05-14T17:12:37.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.747+0000 7fb71058d740 1 -- 172.21.15.2:0/3242749316 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x5574ab9a8200 con 0x5574ab98e8f0 2024-05-14T17:12:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.747+0000 7fb6e6ffd640 1 -- 172.21.15.2:0/3242749316 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6f00035f0 con 0x5574ab98e8f0 2024-05-14T17:12:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.747+0000 7fb6e6ffd640 1 -- 172.21.15.2:0/3242749316 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 53) ==== 8342+0+0 (secure 0 0 0) 0x7fb6f801a710 con 0x5574ab98e8f0 2024-05-14T17:12:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.747+0000 7fb71058d740 10 client.16977 fetch_fsmap finished waiting for FSMap version 53 2024-05-14T17:12:37.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.747+0000 7fb71058d740 10 client.16977 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:12:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.747+0000 7fb71058d740 20 client.16977 populate_metadata read hostname 'smithi002' 2024-05-14T17:12:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.747+0000 7fb71058d740 1 --2- 172.21.15.2:0/3242749316 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5574ab9a9730 0x5574ab9abb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.747+0000 7fb71058d740 4 client.16977 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:12:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.747+0000 7fb71058d740 1 -- 172.21.15.2:0/3242749316 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574ab903b80 con 0x5574ab9a9730 2024-05-14T17:12:38.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.748+0000 7fb7067fc640 1 --2- 172.21.15.2:0/3242749316 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5574ab9a9730 0x5574ab9abb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:38.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.748+0000 7fb7067fc640 1 --2- 172.21.15.2:0/3242749316 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5574ab9a9730 0x5574ab9abb80 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4fd21211922e180 server_cookie=8d97cf949d1d5b35 in_seq=0 out_seq=0 2024-05-14T17:12:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.748+0000 7fb6e6ffd640 10 client.16977 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:12:38.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.749+0000 7fb6e6ffd640 1 -- 172.21.15.2:0/3242749316 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5574ab903b80 con 0x5574ab9a9730 2024-05-14T17:12:38.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.749+0000 7fb6e6ffd640 10 client.16977 handle_command_reply: tid=0 2024-05-14T17:12:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.749+0000 7fb71058d740 1 client.16977 shutdown 2024-05-14T17:12:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.749+0000 7fb71058d740 1 -- 172.21.15.2:0/3242749316 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574ab98e8f0 msgr2=0x5574ab9993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.749+0000 7fb71058d740 1 --2- 172.21.15.2:0/3242749316 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574ab98e8f0 0x5574ab9993d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fb6f801cc10 tx=0x7fb6f8004730 comp rx=0 tx=0).stop 2024-05-14T17:12:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.749+0000 7fb71058d740 1 -- 172.21.15.2:0/3242749316 shutdown_connections 2024-05-14T17:12:38.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.749+0000 7fb71058d740 1 --2- 172.21.15.2:0/3242749316 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5574ab9a9730 0x5574ab9abb80 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.749+0000 7fb71058d740 1 --2- 172.21.15.2:0/3242749316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574ab999ad0 0x5574ab996c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:38.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.749+0000 7fb71058d740 1 --2- 172.21.15.2:0/3242749316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ab98f210 0x5574ab9966c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.749+0000 7fb6e6ffd640 0 client.16977 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:12:38.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.749+0000 7fb71058d740 1 --2- 172.21.15.2:0/3242749316 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574ab98e8f0 0x5574ab9993d0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.749+0000 7fb71058d740 1 -- 172.21.15.2:0/3242749316 >> 172.21.15.2:0/3242749316 conn(0x5574ab98ae00 msgr2=0x5574ab98a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.749+0000 7fb71058d740 1 -- 172.21.15.2:0/3242749316 shutdown_connections 2024-05-14T17:12:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.749+0000 7fb71058d740 1 -- 172.21.15.2:0/3242749316 wait complete. 2024-05-14T17:12:38.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.749+0000 7fb71058d740 20 client.16977 trim_cache size 0 max 16384 2024-05-14T17:12:38.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.765+0000 7fb71058d740 1 Processor -- start 2024-05-14T17:12:38.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.765+0000 7fb71058d740 1 -- start start 2024-05-14T17:12:38.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.765+0000 7fb71058d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ab98e8f0 0x5574ab9a1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:38.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.765+0000 7fb71058d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574ab98f210 0x5574ab998f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:38.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.765+0000 7fb71058d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574ab999ad0 0x5574ab989f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.765+0000 7fb71058d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb7081b0eb0 con 0x5574ab999ad0 2024-05-14T17:12:38.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.765+0000 7fb71058d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb7081ae780 con 0x5574ab98e8f0 2024-05-14T17:12:38.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.765+0000 7fb71058d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb7081ac5f0 con 0x5574ab98f210 2024-05-14T17:12:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.765+0000 7fb7077fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574ab999ad0 0x5574ab989f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:39.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.765+0000 7fb7077fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574ab999ad0 0x5574ab989f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56356/0 (socket says 172.21.15.2:56356) 2024-05-14T17:12:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.765+0000 7fb7077fe640 1 -- 172.21.15.2:0/911964067 learned_addr learned my addr 172.21.15.2:0/911964067 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:12:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.765+0000 7fb706ffd640 1 --2- 172.21.15.2:0/911964067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ab98e8f0 0x5574ab9a1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:39.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.765+0000 7fb7067fc640 1 --2- 172.21.15.2:0/911964067 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574ab98f210 0x5574ab998f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:39.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.765+0000 7fb7077fe640 1 -- 172.21.15.2:0/911964067 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574ab98f210 msgr2=0x5574ab998f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.765+0000 7fb7077fe640 1 --2- 172.21.15.2:0/911964067 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574ab98f210 0x5574ab998f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.765+0000 7fb7077fe640 1 -- 172.21.15.2:0/911964067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ab98e8f0 msgr2=0x5574ab9a1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:39.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.765+0000 7fb7077fe640 1 --2- 172.21.15.2:0/911964067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ab98e8f0 0x5574ab9a1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.765+0000 7fb7077fe640 1 -- 172.21.15.2:0/911964067 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574ab98ac30 con 0x5574ab999ad0 2024-05-14T17:12:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.765+0000 7fb706ffd640 1 --2- 172.21.15.2:0/911964067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ab98e8f0 0x5574ab9a1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:12:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.765+0000 7fb7077fe640 1 --2- 172.21.15.2:0/911964067 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574ab999ad0 0x5574ab989f80 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fb7000a5b50 tx=0x7fb7000a53b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c9672c6603a7882 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.766+0000 7fb6fd7fa640 1 -- 172.21.15.2:0/911964067 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7000c3070 con 0x5574ab999ad0 2024-05-14T17:12:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.766+0000 7fb6fd7fa640 1 -- 172.21.15.2:0/911964067 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7000aae50 con 0x5574ab999ad0 2024-05-14T17:12:39.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.766+0000 7fb6fd7fa640 1 -- 172.21.15.2:0/911964067 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70001f070 con 0x5574ab999ad0 2024-05-14T17:12:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.766+0000 7fb71058d740 1 -- 172.21.15.2:0/911964067 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574ab999ad0 msgr2=0x5574ab989f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.766+0000 7fb71058d740 1 --2- 172.21.15.2:0/911964067 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574ab999ad0 0x5574ab989f80 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fb7000a5b50 tx=0x7fb7000a53b0 comp rx=0 tx=0).stop 2024-05-14T17:12:39.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.766+0000 7fb71058d740 1 -- 172.21.15.2:0/911964067 shutdown_connections 2024-05-14T17:12:39.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.766+0000 7fb71058d740 1 --2- 172.21.15.2:0/911964067 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574ab999ad0 0x5574ab989f80 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:39.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.766+0000 7fb71058d740 1 --2- 172.21.15.2:0/911964067 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574ab98f210 0x5574ab998f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.766+0000 7fb71058d740 1 --2- 172.21.15.2:0/911964067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ab98e8f0 0x5574ab9a1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.766+0000 7fb71058d740 1 -- 172.21.15.2:0/911964067 >> 172.21.15.2:0/911964067 conn(0x5574ab98ae00 msgr2=0x5574ab989870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:39.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.766+0000 7fb71058d740 1 -- 172.21.15.2:0/911964067 shutdown_connections 2024-05-14T17:12:39.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.766+0000 7fb71058d740 1 -- 172.21.15.2:0/911964067 wait complete. 2024-05-14T17:12:39.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.767+0000 7fb71058d740 1 -- start start 2024-05-14T17:12:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.767+0000 7fb71058d740 1 Processor -- start 2024-05-14T17:12:39.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.767+0000 7fb71058d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574ab98e8f0 0x5574aba0c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:40.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.767+0000 7fb71058d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574ab98f210 0x5574aba0c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.767+0000 7fb71058d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ab999ad0 0x5574aba170d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.767+0000 7fb71058d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb7081acc40 con 0x5574ab98e8f0 2024-05-14T17:12:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.767+0000 7fb71058d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb7081ae0b0 con 0x5574ab999ad0 2024-05-14T17:12:40.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.767+0000 7fb71058d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb7081ad310 con 0x5574ab98f210 2024-05-14T17:12:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.767+0000 7fb71058d740 10 client.0 _ll_register_callbacks cb 0x7fb6f801c890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:12:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.767+0000 7fb706ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574ab98e8f0 0x5574aba0c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.767+0000 7fb706ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574ab98e8f0 0x5574aba0c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56366/0 (socket says 172.21.15.2:56366) 2024-05-14T17:12:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.767+0000 7fb706ffd640 1 -- 172.21.15.2:0/1136495439 learned_addr learned my addr 172.21.15.2:0/1136495439 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:12:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.767+0000 7fb7077fe640 1 --2- 172.21.15.2:0/1136495439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ab999ad0 0x5574aba170d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.767+0000 7fb706ffd640 1 -- 172.21.15.2:0/1136495439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574ab98f210 msgr2=0x5574aba0c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.767+0000 7fb706ffd640 1 --2- 172.21.15.2:0/1136495439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574ab98f210 0x5574aba0c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:40.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.768+0000 7fb706ffd640 1 -- 172.21.15.2:0/1136495439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ab999ad0 msgr2=0x5574aba170d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:40.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.768+0000 7fb706ffd640 1 --2- 172.21.15.2:0/1136495439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ab999ad0 0x5574aba170d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.768+0000 7fb706ffd640 1 -- 172.21.15.2:0/1136495439 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574aba17610 con 0x5574ab98e8f0 2024-05-14T17:12:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.768+0000 7fb706ffd640 1 --2- 172.21.15.2:0/1136495439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574ab98e8f0 0x5574aba0c240 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8017040 tx=0x7fb6f8010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc18e14a5aa3e6f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.768+0000 7fb6e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:12:40.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.768+0000 7fb6e6ffd640 1 -- 172.21.15.2:0/1136495439 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f803f060 con 0x5574ab98e8f0 2024-05-14T17:12:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.768+0000 7fb6e6ffd640 1 -- 172.21.15.2:0/1136495439 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7fb6f800f7f0 con 0x5574ab98e8f0 2024-05-14T17:12:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.768+0000 7fb71058d740 1 -- 172.21.15.2:0/1136495439 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7000aae50 con 0x5574ab98e8f0 2024-05-14T17:12:40.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.768+0000 7fb6e6ffd640 1 -- 172.21.15.2:0/1136495439 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fb6f800a9f0 con 0x5574ab98e8f0 2024-05-14T17:12:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.768+0000 7fb71058d740 10 client.16983 fetch_fsmap learned FSMap version 53 2024-05-14T17:12:40.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.768+0000 7fb71058d740 1 -- 172.21.15.2:0/1136495439 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x5574aba18890 con 0x5574ab98e8f0 2024-05-14T17:12:40.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.768+0000 7fb6e6ffd640 1 -- 172.21.15.2:0/1136495439 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7000aae50 con 0x5574ab98e8f0 2024-05-14T17:12:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.768+0000 7fb6e6ffd640 1 -- 172.21.15.2:0/1136495439 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 53) ==== 8342+0+0 (secure 0 0 0) 0x7fb6f8039030 con 0x5574ab98e8f0 2024-05-14T17:12:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.768+0000 7fb71058d740 10 client.16983 fetch_fsmap finished waiting for FSMap version 53 2024-05-14T17:12:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.768+0000 7fb71058d740 10 client.16983 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:12:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.768+0000 7fb71058d740 20 client.16983 populate_metadata read hostname 'smithi002' 2024-05-14T17:12:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.768+0000 7fb71058d740 1 --2- 172.21.15.2:0/1136495439 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5574aba19010 0x5574aba1b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.769+0000 7fb71058d740 4 client.16983 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:12:41.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.769+0000 7fb71058d740 1 -- 172.21.15.2:0/1136495439 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5574ab8fd360 con 0x5574aba19010 2024-05-14T17:12:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.769+0000 7fb7067fc640 1 --2- 172.21.15.2:0/1136495439 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5574aba19010 0x5574aba1b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.769+0000 7fb6e6ffd640 10 client.16983 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:12:41.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.769+0000 7fb7067fc640 1 --2- 172.21.15.2:0/1136495439 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5574aba19010 0x5574aba1b470 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ff01b933985ec8e server_cookie=15ce21c8c8dff71f in_seq=0 out_seq=0 2024-05-14T17:12:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.781+0000 7fb6e6ffd640 1 -- 172.21.15.2:0/1136495439 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5574ab8fd360 con 0x5574aba19010 2024-05-14T17:12:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.781+0000 7fb6e6ffd640 10 client.16983 handle_command_reply: tid=0 2024-05-14T17:12:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.781+0000 7fb71058d740 1 client.16983 shutdown 2024-05-14T17:12:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.782+0000 7fb71058d740 1 -- 172.21.15.2:0/1136495439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574ab98e8f0 msgr2=0x5574aba0c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:41.384 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.782+0000 7fb71058d740 1 --2- 172.21.15.2:0/1136495439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574ab98e8f0 0x5574aba0c240 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8017040 tx=0x7fb6f8010d50 comp rx=0 tx=0).stop 2024-05-14T17:12:41.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.782+0000 7fb71058d740 1 -- 172.21.15.2:0/1136495439 shutdown_connections 2024-05-14T17:12:41.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.782+0000 7fb71058d740 1 --2- 172.21.15.2:0/1136495439 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5574aba19010 0x5574aba1b470 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:41.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.782+0000 7fb71058d740 1 --2- 172.21.15.2:0/1136495439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5574ab999ad0 0x5574aba170d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:41.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.782+0000 7fb6e6ffd640 0 client.16983 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:12:41.550 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.782+0000 7fb71058d740 1 --2- 172.21.15.2:0/1136495439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574ab98f210 0x5574aba0c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:41.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.782+0000 7fb71058d740 1 --2- 172.21.15.2:0/1136495439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5574ab98e8f0 0x5574aba0c240 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:41.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.782+0000 7fb71058d740 1 -- 172.21.15.2:0/1136495439 >> 172.21.15.2:0/1136495439 conn(0x5574ab98ae00 msgr2=0x5574ab98a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:41.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.782+0000 7fb71058d740 1 -- 172.21.15.2:0/1136495439 shutdown_connections 2024-05-14T17:12:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.782+0000 7fb71058d740 1 -- 172.21.15.2:0/1136495439 wait complete. 2024-05-14T17:12:41.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.782+0000 7fb71058d740 20 client.16983 trim_cache size 0 max 16384 2024-05-14T17:12:41.751 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:12:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:12:41.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:12:41.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.782+0000 7fb70d872640 1 -- 172.21.15.2:0/3352466682 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb6f40b2a10 msgr2=0x7fb6f40b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.782+0000 7fb70d872640 1 --2- 172.21.15.2:0/3352466682 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb6f40b2a10 0x7fb6f40b4ed0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x5574ab98fe90 tx=0x7fb6f800a2d0 comp rx=0 tx=0).stop 2024-05-14T17:12:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.782+0000 7fb70d872640 1 -- 172.21.15.2:0/3352466682 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb708107940 msgr2=0x7fb70819da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.782+0000 7fb70d872640 1 --2- 172.21.15.2:0/3352466682 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb708107940 0x7fb70819da40 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fb70000eaa0 tx=0x7fb70000eea0 comp rx=0 tx=0).stop 2024-05-14T17:12:42.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.782+0000 7fb70d872640 1 -- 172.21.15.2:0/3352466682 shutdown_connections 2024-05-14T17:12:42.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.782+0000 7fb70d872640 1 --2- 172.21.15.2:0/3352466682 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb6f40b2a10 0x7fb6f40b4ed0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.782+0000 7fb70d872640 1 --2- 172.21.15.2:0/3352466682 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb708107940 0x7fb70819da40 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.782+0000 7fb70d872640 1 --2- 172.21.15.2:0/3352466682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb708102340 0x7fb7081996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:42.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.782+0000 7fb70d872640 1 --2- 172.21.15.2:0/3352466682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb708101980 0x7fb708199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.783+0000 7fb70d872640 1 -- 172.21.15.2:0/3352466682 >> 172.21.15.2:0/3352466682 conn(0x7fb708078ad0 msgr2=0x7fb70806a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:42.226 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.783+0000 7fb70d872640 1 -- 172.21.15.2:0/3352466682 shutdown_connections 2024-05-14T17:12:42.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:30.783+0000 7fb70d872640 1 -- 172.21.15.2:0/3352466682 wait complete. 2024-05-14T17:12:42.294 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:11:30 smithi002 ceph-mon[31538]: pgmap v527: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 58 KiB/s wr, 14 op/s 2024-05-14T17:12:42.326 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:11:32 smithi002 ceph-mon[31538]: pgmap v528: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 9 op/s 2024-05-14T17:12:42.359 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:11:34 smithi002 ceph-mon[31538]: pgmap v529: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 60 KiB/s wr, 15 op/s 2024-05-14T17:12:42.393 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:11:36 smithi002 ceph-mon[31538]: pgmap v530: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 KiB/s wr, 11 op/s 2024-05-14T17:12:42.426 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:11:38 smithi002 ceph-mon[31538]: pgmap v531: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 KiB/s wr, 11 op/s 2024-05-14T17:12:42.484 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:11:40 smithi002 ceph-mon[31538]: pgmap v532: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-05-14T17:12:42.518 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:11:42 smithi002 ceph-mon[31538]: pgmap v533: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-14T17:12:42.554 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:12:34 smithi123 ceph-mon[35971]: pgmap v561: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-14T17:12:42.585 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:12:36 smithi123 ceph-mon[35971]: pgmap v562: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-05-14T17:12:42.618 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:12:38 smithi123 ceph-mon[35971]: pgmap v563: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-05-14T17:12:42.651 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:12:40 smithi123 ceph-mon[35971]: pgmap v564: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-05-14T17:12:42.685 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:12:34 smithi151 ceph-mon[36066]: pgmap v561: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-14T17:12:42.726 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:12:36 smithi151 ceph-mon[36066]: pgmap v562: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-05-14T17:12:42.793 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:12:38 smithi151 ceph-mon[36066]: pgmap v563: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-05-14T17:12:42.851 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:12:40 smithi151 ceph-mon[36066]: pgmap v564: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-05-14T17:12:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.884+0000 7f3af455f640 1 -- 172.21.15.2:0/234169791 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3aec107980 msgr2=0x7f3aec107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.884+0000 7f3af455f640 1 --2- 172.21.15.2:0/234169791 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3aec107980 0x7f3aec107d60 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8009930 tx=0x7f3ae802a420 comp rx=0 tx=0).stop 2024-05-14T17:12:42.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.884+0000 7f3af455f640 1 -- 172.21.15.2:0/234169791 shutdown_connections 2024-05-14T17:12:42.985 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.884+0000 7f3af455f640 1 --2- 172.21.15.2:0/234169791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aec102330 0x7f3aec10e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:43.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.884+0000 7f3af455f640 1 --2- 172.21.15.2:0/234169791 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3aec101970 0x7f3aec101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.884+0000 7f3af455f640 1 --2- 172.21.15.2:0/234169791 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3aec107980 0x7f3aec107d60 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.884+0000 7f3af455f640 1 -- 172.21.15.2:0/234169791 >> 172.21.15.2:0/234169791 conn(0x7f3aec077cd0 msgr2=0x7f3aec0780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:43.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.884+0000 7f3af455f640 1 -- 172.21.15.2:0/234169791 shutdown_connections 2024-05-14T17:12:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.884+0000 7f3af455f640 1 -- 172.21.15.2:0/234169791 wait complete. 2024-05-14T17:12:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.885+0000 7f3af455f640 1 Processor -- start 2024-05-14T17:12:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.885+0000 7f3af455f640 1 -- start start 2024-05-14T17:12:43.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.885+0000 7f3af455f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aec101970 0x7f3aec199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.885+0000 7f3af455f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3aec102330 0x7f3aec1996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:43.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.885+0000 7f3af455f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3aec107980 0x7f3aec19da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:43.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.885+0000 7f3af455f640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3aec112d40 con 0x7f3aec107980 2024-05-14T17:12:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.885+0000 7f3af455f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3aec112bc0 con 0x7f3aec101970 2024-05-14T17:12:43.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.885+0000 7f3af455f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3aec112ec0 con 0x7f3aec102330 2024-05-14T17:12:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.886+0000 7f3af2afc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3aec107980 0x7f3aec19da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:43.627 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.886+0000 7f3af2afc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3aec107980 0x7f3aec19da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39184/0 (socket says 172.21.15.2:39184) 2024-05-14T17:12:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.886+0000 7f3af2afc640 1 -- 172.21.15.2:0/3385402085 learned_addr learned my addr 172.21.15.2:0/3385402085 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:12:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.886+0000 7f3af22fb640 1 --2- 172.21.15.2:0/3385402085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aec101970 0x7f3aec199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.886+0000 7f3af1afa640 1 --2- 172.21.15.2:0/3385402085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3aec102330 0x7f3aec1996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.886+0000 7f3af2afc640 1 -- 172.21.15.2:0/3385402085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3aec102330 msgr2=0x7f3aec1996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.886+0000 7f3af2afc640 1 --2- 172.21.15.2:0/3385402085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3aec102330 0x7f3aec1996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.886+0000 7f3af2afc640 1 -- 172.21.15.2:0/3385402085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aec101970 msgr2=0x7f3aec199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:43.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.886+0000 7f3af2afc640 1 --2- 172.21.15.2:0/3385402085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aec101970 0x7f3aec199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:43.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.886+0000 7f3af2afc640 1 -- 172.21.15.2:0/3385402085 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aec19e170 con 0x7f3aec107980 2024-05-14T17:12:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.886+0000 7f3af2afc640 1 --2- 172.21.15.2:0/3385402085 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3aec107980 0x7f3aec19da40 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f3ae400eaa0 tx=0x7f3ae400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:43.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.886+0000 7f3ae37fe640 1 -- 172.21.15.2:0/3385402085 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae400cdb0 con 0x7f3aec107980 2024-05-14T17:12:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.886+0000 7f3ae37fe640 1 -- 172.21.15.2:0/3385402085 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ae4005020 con 0x7f3aec107980 2024-05-14T17:12:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.886+0000 7f3af455f640 1 -- 172.21.15.2:0/3385402085 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aec19e460 con 0x7f3aec107980 2024-05-14T17:12:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.886+0000 7f3ae37fe640 1 -- 172.21.15.2:0/3385402085 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae40116d0 con 0x7f3aec107980 2024-05-14T17:12:44.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.886+0000 7f3af455f640 1 -- 172.21.15.2:0/3385402085 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aec1a5cf0 con 0x7f3aec107980 2024-05-14T17:12:44.152 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.889+0000 7f3ae37fe640 1 -- 172.21.15.2:0/3385402085 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f3ae40051d0 con 0x7f3aec107980 2024-05-14T17:12:44.186 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.889+0000 7f3af727b740 1 Processor -- start 2024-05-14T17:12:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.890+0000 7f3af727b740 1 -- start start 2024-05-14T17:12:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.890+0000 7f3af727b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c34ba68f0 0x563c34ba6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.890+0000 7f3af727b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c34ba7210 0x563c34bb1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:44.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.890+0000 7f3af727b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c34bb1ad0 0x563c34bb3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.890+0000 7f3af727b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563c34b19c30 con 0x563c34bb1ad0 2024-05-14T17:12:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.890+0000 7f3af727b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563c34b097d0 con 0x563c34ba68f0 2024-05-14T17:12:44.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.890+0000 7f3af727b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563c34b0d810 con 0x563c34ba7210 2024-05-14T17:12:44.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.890+0000 7f3af1afa640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c34bb1ad0 0x563c34bb3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:44.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.890+0000 7f3af1afa640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c34bb1ad0 0x563c34bb3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39192/0 (socket says 172.21.15.2:39192) 2024-05-14T17:12:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.890+0000 7f3af1afa640 1 -- 172.21.15.2:0/1941439466 learned_addr learned my addr 172.21.15.2:0/1941439466 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:12:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.890+0000 7f3af22fb640 1 --2- 172.21.15.2:0/1941439466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c34ba7210 0x563c34bb1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:44.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.890+0000 7f3af1afa640 1 --2- 172.21.15.2:0/1941439466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c34ba68f0 0x563c34ba6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:44.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.890+0000 7f3af1afa640 1 -- 172.21.15.2:0/1941439466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c34ba7210 msgr2=0x563c34bb1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.890+0000 7f3af1afa640 1 --2- 172.21.15.2:0/1941439466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c34ba7210 0x563c34bb1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.890+0000 7f3af1afa640 1 -- 172.21.15.2:0/1941439466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c34ba68f0 msgr2=0x563c34ba6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:44.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.890+0000 7f3af1afa640 1 --2- 172.21.15.2:0/1941439466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c34ba68f0 0x563c34ba6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.890+0000 7f3af1afa640 1 -- 172.21.15.2:0/1941439466 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c3481e210 con 0x563c34bb1ad0 2024-05-14T17:12:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.890+0000 7f3ae37fe640 1 --2- 172.21.15.2:0/3385402085 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f3ad80b2a90 0x7f3ad80b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.891+0000 7f3af22fb640 1 --2- 172.21.15.2:0/3385402085 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f3ad80b2a90 0x7f3ad80b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:44.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.891+0000 7f3af1afa640 1 --2- 172.21.15.2:0/1941439466 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c34bb1ad0 0x563c34bb3ec0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f3adc0125c0 tx=0x7f3adc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eddf873e24d7e6f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.891+0000 7f3ae37fe640 1 -- 172.21.15.2:0/3385402085 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f3ae40e2670 con 0x7f3aec107980 2024-05-14T17:12:44.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.891+0000 7f3ae0bfe640 1 -- 172.21.15.2:0/1941439466 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3adc0106b0 con 0x563c34bb1ad0 2024-05-14T17:12:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.891+0000 7f3ae0bfe640 1 -- 172.21.15.2:0/1941439466 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3adc010d00 con 0x563c34bb1ad0 2024-05-14T17:12:45.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.891+0000 7f3ae0bfe640 1 -- 172.21.15.2:0/1941439466 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3adc01dc60 con 0x563c34bb1ad0 2024-05-14T17:12:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.892+0000 7f3af727b740 1 -- 172.21.15.2:0/1941439466 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c34bb1ad0 msgr2=0x563c34bb3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:45.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.892+0000 7f3af22fb640 1 --2- 172.21.15.2:0/3385402085 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f3ad80b2a90 0x7f3ad80b4f50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8009900 tx=0x7f3ae8005fd0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:45.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.892+0000 7f3af727b740 1 --2- 172.21.15.2:0/1941439466 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c34bb1ad0 0x563c34bb3ec0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f3adc0125c0 tx=0x7f3adc00b870 comp rx=0 tx=0).stop 2024-05-14T17:12:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.892+0000 7f3af727b740 1 -- 172.21.15.2:0/1941439466 shutdown_connections 2024-05-14T17:12:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.892+0000 7f3af727b740 1 --2- 172.21.15.2:0/1941439466 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c34bb1ad0 0x563c34bb3ec0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.892+0000 7f3af727b740 1 --2- 172.21.15.2:0/1941439466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c34ba7210 0x563c34bb1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:45.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.892+0000 7f3af727b740 1 --2- 172.21.15.2:0/1941439466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c34ba68f0 0x563c34ba6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.892+0000 7f3af727b740 1 -- 172.21.15.2:0/1941439466 >> 172.21.15.2:0/1941439466 conn(0x563c34ba2e00 msgr2=0x563c34ba31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:45.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.892+0000 7f3af727b740 1 -- 172.21.15.2:0/1941439466 shutdown_connections 2024-05-14T17:12:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.892+0000 7f3af727b740 1 -- 172.21.15.2:0/1941439466 wait complete. 2024-05-14T17:12:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.892+0000 7f3af727b740 1 -- start start 2024-05-14T17:12:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.892+0000 7f3af727b740 1 Processor -- start 2024-05-14T17:12:45.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.893+0000 7f3af727b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c34ba68f0 0x563c34bb1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.893+0000 7f3af727b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c34ba7210 0x563c34bae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.893+0000 7f3af727b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c34bb1ad0 0x563c34baec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.893+0000 7f3af727b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x563c347ef790 con 0x563c34bb1ad0 2024-05-14T17:12:45.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.893+0000 7f3af727b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563c34ac5890 con 0x563c34ba7210 2024-05-14T17:12:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.893+0000 7f3af727b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563c34bb47a0 con 0x563c34ba68f0 2024-05-14T17:12:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.893+0000 7f3af727b740 10 client.0 _ll_register_callbacks cb 0x563c34ac8d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:12:45.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.893+0000 7f3af2afc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c34bb1ad0 0x563c34baec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.893+0000 7f3af2afc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c34bb1ad0 0x563c34baec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39194/0 (socket says 172.21.15.2:39194) 2024-05-14T17:12:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.893+0000 7f3af2afc640 1 -- 172.21.15.2:0/1922005231 learned_addr learned my addr 172.21.15.2:0/1922005231 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:12:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.893+0000 7f3af22fb640 1 --2- 172.21.15.2:0/1922005231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c34ba68f0 0x563c34bb1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:45.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.893+0000 7f3af1afa640 1 --2- 172.21.15.2:0/1922005231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c34ba7210 0x563c34bae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:45.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.893+0000 7f3af2afc640 1 -- 172.21.15.2:0/1922005231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c34ba68f0 msgr2=0x563c34bb1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:45.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.893+0000 7f3af2afc640 1 --2- 172.21.15.2:0/1922005231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c34ba68f0 0x563c34bb1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:45.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.893+0000 7f3af2afc640 1 -- 172.21.15.2:0/1922005231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c34ba7210 msgr2=0x563c34bae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.893+0000 7f3af2afc640 1 --2- 172.21.15.2:0/1922005231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c34ba7210 0x563c34bae720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.893+0000 7f3af2afc640 1 -- 172.21.15.2:0/1922005231 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c348ea620 con 0x563c34bb1ad0 2024-05-14T17:12:46.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.893+0000 7f3af1afa640 1 --2- 172.21.15.2:0/1922005231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c34ba7210 0x563c34bae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:12:46.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.893+0000 7f3af2afc640 1 --2- 172.21.15.2:0/1922005231 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c34bb1ad0 0x563c34baec80 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f3ae40dfe20 tx=0x7f3ae40a95d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b26bccf2c06b89 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:46.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.893+0000 7f3ad5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:12:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.893+0000 7f3ad5ffb640 1 -- 172.21.15.2:0/1922005231 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae401f040 con 0x563c34bb1ad0 2024-05-14T17:12:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.893+0000 7f3ad5ffb640 1 -- 172.21.15.2:0/1922005231 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f3ae40cc0d0 con 0x563c34bb1ad0 2024-05-14T17:12:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.893+0000 7f3af727b740 1 -- 172.21.15.2:0/1922005231 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3adc010d00 con 0x563c34bb1ad0 2024-05-14T17:12:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.894+0000 7f3ad5ffb640 1 -- 172.21.15.2:0/1922005231 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f3ae40b27b0 con 0x563c34bb1ad0 2024-05-14T17:12:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.894+0000 7f3af727b740 10 client.16995 fetch_fsmap learned FSMap version 53 2024-05-14T17:12:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.894+0000 7f3af727b740 1 -- 172.21.15.2:0/1922005231 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x563c34bc0220 con 0x563c34bb1ad0 2024-05-14T17:12:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.894+0000 7f3ad5ffb640 1 -- 172.21.15.2:0/1922005231 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3adc010d00 con 0x563c34bb1ad0 2024-05-14T17:12:46.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.894+0000 7f3ad5ffb640 1 -- 172.21.15.2:0/1922005231 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 53) ==== 8342+0+0 (secure 0 0 0) 0x7f3ae400f040 con 0x563c34bb1ad0 2024-05-14T17:12:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.894+0000 7f3af727b740 10 client.16995 fetch_fsmap finished waiting for FSMap version 53 2024-05-14T17:12:46.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.894+0000 7f3af727b740 10 client.16995 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:12:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.894+0000 7f3af727b740 20 client.16995 populate_metadata read hostname 'smithi002' 2024-05-14T17:12:46.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.894+0000 7f3af727b740 1 --2- 172.21.15.2:0/1922005231 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x563c34bc1780 0x563c34bc3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:46.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.894+0000 7f3af727b740 4 client.16995 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:12:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.894+0000 7f3af727b740 1 -- 172.21.15.2:0/1922005231 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c34b1bb80 con 0x563c34bc1780 2024-05-14T17:12:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.894+0000 7f3af22fb640 1 --2- 172.21.15.2:0/1922005231 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x563c34bc1780 0x563c34bc3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.895+0000 7f3af22fb640 1 --2- 172.21.15.2:0/1922005231 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x563c34bc1780 0x563c34bc3bb0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4670e5f82251b08a server_cookie=41f269489bf3d2b3 in_seq=0 out_seq=0 2024-05-14T17:12:46.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.895+0000 7f3ad5ffb640 10 client.16995 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:12:46.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.896+0000 7f3ad5ffb640 1 -- 172.21.15.2:0/1922005231 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563c34b1bb80 con 0x563c34bc1780 2024-05-14T17:12:46.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.896+0000 7f3ad5ffb640 10 client.16995 handle_command_reply: tid=0 2024-05-14T17:12:46.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.896+0000 7f3af727b740 1 client.16995 shutdown 2024-05-14T17:12:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.896+0000 7f3af727b740 1 -- 172.21.15.2:0/1922005231 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c34bb1ad0 msgr2=0x563c34baec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.896+0000 7f3af727b740 1 --2- 172.21.15.2:0/1922005231 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c34bb1ad0 0x563c34baec80 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f3ae40dfe20 tx=0x7f3ae40a95d0 comp rx=0 tx=0).stop 2024-05-14T17:12:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.896+0000 7f3af727b740 1 -- 172.21.15.2:0/1922005231 shutdown_connections 2024-05-14T17:12:47.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.896+0000 7f3af727b740 1 --2- 172.21.15.2:0/1922005231 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x563c34bc1780 0x563c34bc3bb0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.896+0000 7f3af727b740 1 --2- 172.21.15.2:0/1922005231 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c34bb1ad0 0x563c34baec80 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:47.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.896+0000 7f3ad5ffb640 0 client.16995 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:12:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.896+0000 7f3af727b740 1 --2- 172.21.15.2:0/1922005231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c34ba7210 0x563c34bae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.896+0000 7f3af727b740 1 --2- 172.21.15.2:0/1922005231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c34ba68f0 0x563c34bb1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.896+0000 7f3af727b740 1 -- 172.21.15.2:0/1922005231 >> 172.21.15.2:0/1922005231 conn(0x563c34ba2e00 msgr2=0x563c34ba2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:47.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.896+0000 7f3af727b740 1 -- 172.21.15.2:0/1922005231 shutdown_connections 2024-05-14T17:12:47.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.896+0000 7f3af727b740 1 -- 172.21.15.2:0/1922005231 wait complete. 2024-05-14T17:12:47.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.896+0000 7f3af727b740 20 client.16995 trim_cache size 0 max 16384 2024-05-14T17:12:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.911+0000 7f3af727b740 1 Processor -- start 2024-05-14T17:12:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.911+0000 7f3af727b740 1 -- start start 2024-05-14T17:12:47.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.912+0000 7f3af727b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c34ba68f0 0x563c34bc94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.912+0000 7f3af727b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c34ba7210 0x563c34bc9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.912+0000 7f3af727b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c34bb1ad0 0x563c34bab980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.912+0000 7f3af727b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3aec1b0ee0 con 0x563c34ba7210 2024-05-14T17:12:47.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.912+0000 7f3af727b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3aec1ae7b0 con 0x563c34bb1ad0 2024-05-14T17:12:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.912+0000 7f3af727b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3aec1ac620 con 0x563c34ba68f0 2024-05-14T17:12:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.912+0000 7f3af1afa640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c34ba7210 0x563c34bc9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.912+0000 7f3af1afa640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c34ba7210 0x563c34bc9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39210/0 (socket says 172.21.15.2:39210) 2024-05-14T17:12:47.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.912+0000 7f3af1afa640 1 -- 172.21.15.2:0/2753817219 learned_addr learned my addr 172.21.15.2:0/2753817219 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:12:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.912+0000 7f3af22fb640 1 --2- 172.21.15.2:0/2753817219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c34ba68f0 0x563c34bc94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:47.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.912+0000 7f3af2afc640 1 --2- 172.21.15.2:0/2753817219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c34bb1ad0 0x563c34bab980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:47.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.912+0000 7f3af1afa640 1 -- 172.21.15.2:0/2753817219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c34ba68f0 msgr2=0x563c34bc94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.912+0000 7f3af1afa640 1 --2- 172.21.15.2:0/2753817219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c34ba68f0 0x563c34bc94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.912+0000 7f3af1afa640 1 -- 172.21.15.2:0/2753817219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c34bb1ad0 msgr2=0x563c34bab980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:47.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.912+0000 7f3af1afa640 1 --2- 172.21.15.2:0/2753817219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c34bb1ad0 0x563c34bab980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.912+0000 7f3af1afa640 1 -- 172.21.15.2:0/2753817219 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c34ba7040 con 0x563c34ba7210 2024-05-14T17:12:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.912+0000 7f3af2afc640 1 --2- 172.21.15.2:0/2753817219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c34bb1ad0 0x563c34bab980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:12:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.912+0000 7f3af1afa640 1 --2- 172.21.15.2:0/2753817219 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c34ba7210 0x563c34bc9a00 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f3adc01e000 tx=0x7f3adc00b3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36f9aa2ca640f3b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:48.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.913+0000 7f3ad7fff640 1 -- 172.21.15.2:0/2753817219 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3adc0050c0 con 0x563c34ba7210 2024-05-14T17:12:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.913+0000 7f3ad7fff640 1 -- 172.21.15.2:0/2753817219 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3adc02b070 con 0x563c34ba7210 2024-05-14T17:12:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.913+0000 7f3ad7fff640 1 -- 172.21.15.2:0/2753817219 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3adc003d90 con 0x563c34ba7210 2024-05-14T17:12:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.913+0000 7f3af727b740 1 -- 172.21.15.2:0/2753817219 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c34ba7210 msgr2=0x563c34bc9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.913+0000 7f3af727b740 1 --2- 172.21.15.2:0/2753817219 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c34ba7210 0x563c34bc9a00 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f3adc01e000 tx=0x7f3adc00b3d0 comp rx=0 tx=0).stop 2024-05-14T17:12:48.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.913+0000 7f3af727b740 1 -- 172.21.15.2:0/2753817219 shutdown_connections 2024-05-14T17:12:48.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.913+0000 7f3af727b740 1 --2- 172.21.15.2:0/2753817219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c34bb1ad0 0x563c34bab980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:48.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.913+0000 7f3af727b740 1 --2- 172.21.15.2:0/2753817219 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c34ba7210 0x563c34bc9a00 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:48.414 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.913+0000 7f3af727b740 1 --2- 172.21.15.2:0/2753817219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c34ba68f0 0x563c34bc94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:48.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.913+0000 7f3af727b740 1 -- 172.21.15.2:0/2753817219 >> 172.21.15.2:0/2753817219 conn(0x563c34ba2e00 msgr2=0x563c34bbc660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.913+0000 7f3af727b740 1 -- 172.21.15.2:0/2753817219 shutdown_connections 2024-05-14T17:12:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.913+0000 7f3af727b740 1 -- 172.21.15.2:0/2753817219 wait complete. 2024-05-14T17:12:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.913+0000 7f3af727b740 1 -- start start 2024-05-14T17:12:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.914+0000 7f3af727b740 1 Processor -- start 2024-05-14T17:12:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.914+0000 7f3af727b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c34ba68f0 0x563c34bb77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.914+0000 7f3af727b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c34ba7210 0x563c34bb7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.914+0000 7f3af727b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c34bb1ad0 0x563c34bb8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.914+0000 7f3af727b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3aec1acc70 con 0x563c34ba68f0 2024-05-14T17:12:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.914+0000 7f3af727b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3aec1ae0e0 con 0x563c34bb1ad0 2024-05-14T17:12:48.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.914+0000 7f3af727b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3aec1ad340 con 0x563c34ba7210 2024-05-14T17:12:48.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.914+0000 7f3af727b740 10 client.0 _ll_register_callbacks cb 0x7f3ae40a5190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:12:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.914+0000 7f3af22fb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c34ba68f0 0x563c34bb77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:48.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.914+0000 7f3af22fb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c34ba68f0 0x563c34bb77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39212/0 (socket says 172.21.15.2:39212) 2024-05-14T17:12:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.914+0000 7f3af22fb640 1 -- 172.21.15.2:0/3234949803 learned_addr learned my addr 172.21.15.2:0/3234949803 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:12:49.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.914+0000 7f3af1afa640 1 --2- 172.21.15.2:0/3234949803 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c34ba7210 0x563c34bb7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.914+0000 7f3af22fb640 1 -- 172.21.15.2:0/3234949803 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c34ba7210 msgr2=0x563c34bb7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.914+0000 7f3af22fb640 1 --2- 172.21.15.2:0/3234949803 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c34ba7210 0x563c34bb7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.914+0000 7f3af22fb640 1 -- 172.21.15.2:0/3234949803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c34bb1ad0 msgr2=0x563c34bb8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:49.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.914+0000 7f3af2afc640 1 --2- 172.21.15.2:0/3234949803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c34bb1ad0 0x563c34bb8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.914+0000 7f3af22fb640 1 --2- 172.21.15.2:0/3234949803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c34bb1ad0 0x563c34bb8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.914+0000 7f3af22fb640 1 -- 172.21.15.2:0/3234949803 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c34c2b2d0 con 0x563c34ba68f0 2024-05-14T17:12:49.332 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.914+0000 7f3af22fb640 1 --2- 172.21.15.2:0/3234949803 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c34ba68f0 0x563c34bb77d0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f3ae802a940 tx=0x7f3ae802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ecc548776584e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:49.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.914+0000 7f3ad5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:12:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.915+0000 7f3ad5ffb640 1 -- 172.21.15.2:0/3234949803 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8035040 con 0x563c34ba68f0 2024-05-14T17:12:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.915+0000 7f3ad5ffb640 1 -- 172.21.15.2:0/3234949803 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f3ae8003c40 con 0x563c34ba68f0 2024-05-14T17:12:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.915+0000 7f3af727b740 1 -- 172.21.15.2:0/3234949803 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3adc02b070 con 0x563c34ba68f0 2024-05-14T17:12:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.915+0000 7f3ad5ffb640 1 -- 172.21.15.2:0/3234949803 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f3ae8003e40 con 0x563c34ba68f0 2024-05-14T17:12:49.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.915+0000 7f3af727b740 10 client.17001 fetch_fsmap learned FSMap version 53 2024-05-14T17:12:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.915+0000 7f3af727b740 1 -- 172.21.15.2:0/3234949803 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x563c34c2c570 con 0x563c34ba68f0 2024-05-14T17:12:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.915+0000 7f3ad5ffb640 1 -- 172.21.15.2:0/3234949803 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3adc02b070 con 0x563c34ba68f0 2024-05-14T17:12:49.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.915+0000 7f3ad5ffb640 1 -- 172.21.15.2:0/3234949803 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 53) ==== 8342+0+0 (secure 0 0 0) 0x7f3ae8051ce0 con 0x563c34ba68f0 2024-05-14T17:12:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.915+0000 7f3af727b740 10 client.17001 fetch_fsmap finished waiting for FSMap version 53 2024-05-14T17:12:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.915+0000 7f3af727b740 10 client.17001 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:12:49.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.915+0000 7f3af727b740 20 client.17001 populate_metadata read hostname 'smithi002' 2024-05-14T17:12:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.915+0000 7f3af727b740 1 --2- 172.21.15.2:0/3234949803 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x563c34c2cc80 0x563c34c2f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:49.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.915+0000 7f3af727b740 4 client.17001 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:12:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.915+0000 7f3af727b740 1 -- 172.21.15.2:0/3234949803 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563c34b15360 con 0x563c34c2cc80 2024-05-14T17:12:49.965 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.915+0000 7f3af1afa640 1 --2- 172.21.15.2:0/3234949803 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x563c34c2cc80 0x563c34c2f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.916+0000 7f3ad5ffb640 10 client.17001 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:12:50.032 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.916+0000 7f3af1afa640 1 --2- 172.21.15.2:0/3234949803 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x563c34c2cc80 0x563c34c2f0c0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b874a26100e350e server_cookie=9ac29e97d36c0a3a in_seq=0 out_seq=0 2024-05-14T17:12:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.928+0000 7f3ad5ffb640 1 -- 172.21.15.2:0/3234949803 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563c34b15360 con 0x563c34c2cc80 2024-05-14T17:12:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.928+0000 7f3ad5ffb640 10 client.17001 handle_command_reply: tid=0 2024-05-14T17:12:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.928+0000 7f3af727b740 1 client.17001 shutdown 2024-05-14T17:12:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.928+0000 7f3af727b740 1 -- 172.21.15.2:0/3234949803 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c34ba68f0 msgr2=0x563c34bb77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.928+0000 7f3af727b740 1 --2- 172.21.15.2:0/3234949803 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c34ba68f0 0x563c34bb77d0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f3ae802a940 tx=0x7f3ae802a420 comp rx=0 tx=0).stop 2024-05-14T17:12:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.928+0000 7f3af727b740 1 -- 172.21.15.2:0/3234949803 shutdown_connections 2024-05-14T17:12:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.928+0000 7f3af727b740 1 --2- 172.21.15.2:0/3234949803 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x563c34c2cc80 0x563c34c2f0c0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.928+0000 7f3af727b740 1 --2- 172.21.15.2:0/3234949803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c34bb1ad0 0x563c34bb8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:50.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.928+0000 7f3af727b740 1 --2- 172.21.15.2:0/3234949803 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c34ba7210 0x563c34bb7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.928+0000 7f3ad5ffb640 0 client.17001 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:12:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.928+0000 7f3af727b740 1 --2- 172.21.15.2:0/3234949803 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c34ba68f0 0x563c34bb77d0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:50.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.928+0000 7f3af727b740 1 -- 172.21.15.2:0/3234949803 >> 172.21.15.2:0/3234949803 conn(0x563c34ba2e00 msgr2=0x563c34bbcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.928+0000 7f3af727b740 1 -- 172.21.15.2:0/3234949803 shutdown_connections 2024-05-14T17:12:50.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.928+0000 7f3af727b740 1 -- 172.21.15.2:0/3234949803 wait complete. 2024-05-14T17:12:50.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.928+0000 7f3af727b740 20 client.17001 trim_cache size 0 max 16384 2024-05-14T17:12:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:12:50.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:12:50.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:12:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.928+0000 7f3af455f640 1 -- 172.21.15.2:0/3385402085 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f3ad80b2a90 msgr2=0x7f3ad80b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.928+0000 7f3af455f640 1 --2- 172.21.15.2:0/3385402085 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f3ad80b2a90 0x7f3ad80b4f50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8009900 tx=0x7f3ae8005fd0 comp rx=0 tx=0).stop 2024-05-14T17:12:50.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.928+0000 7f3af455f640 1 -- 172.21.15.2:0/3385402085 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3aec107980 msgr2=0x7f3aec19da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.928+0000 7f3af455f640 1 --2- 172.21.15.2:0/3385402085 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3aec107980 0x7f3aec19da40 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f3ae400eaa0 tx=0x7f3ae400ef60 comp rx=0 tx=0).stop 2024-05-14T17:12:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.928+0000 7f3af455f640 1 -- 172.21.15.2:0/3385402085 shutdown_connections 2024-05-14T17:12:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.928+0000 7f3af455f640 1 --2- 172.21.15.2:0/3385402085 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f3ad80b2a90 0x7f3ad80b4f50 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:51.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.928+0000 7f3af455f640 1 --2- 172.21.15.2:0/3385402085 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3aec107980 0x7f3aec19da40 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.928+0000 7f3af455f640 1 --2- 172.21.15.2:0/3385402085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3aec102330 0x7f3aec1996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.928+0000 7f3af455f640 1 --2- 172.21.15.2:0/3385402085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3aec101970 0x7f3aec199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:51.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.928+0000 7f3af455f640 1 -- 172.21.15.2:0/3385402085 >> 172.21.15.2:0/3385402085 conn(0x7f3aec077cd0 msgr2=0x7f3aec0ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.929+0000 7f3af455f640 1 -- 172.21.15.2:0/3385402085 shutdown_connections 2024-05-14T17:12:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:43.929+0000 7f3af455f640 1 -- 172.21.15.2:0/3385402085 wait complete. 2024-05-14T17:12:51.242 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:12:42 smithi151 ceph-mon[36066]: pgmap v565: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 KiB/s wr, 8 op/s 2024-05-14T17:12:51.275 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:12:44 smithi151 ceph-mon[36066]: pgmap v566: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-05-14T17:12:51.308 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:12:46 smithi151 ceph-mon[36066]: pgmap v567: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-05-14T17:12:51.341 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:12:48 smithi151 ceph-mon[36066]: pgmap v568: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-05-14T17:12:51.375 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:12:42 smithi123 ceph-mon[35971]: pgmap v565: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 KiB/s wr, 8 op/s 2024-05-14T17:12:51.408 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:12:44 smithi123 ceph-mon[35971]: pgmap v566: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-05-14T17:12:51.441 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:12:46 smithi123 ceph-mon[35971]: pgmap v567: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-05-14T17:12:51.491 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:12:48 smithi123 ceph-mon[35971]: pgmap v568: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-05-14T17:12:51.534 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:11:44 smithi002 ceph-mon[31538]: pgmap v534: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 26 KiB/s wr, 14 op/s 2024-05-14T17:12:51.567 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:11:46 smithi002 ceph-mon[31538]: pgmap v535: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-05-14T17:12:51.600 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:11:48 smithi002 ceph-mon[31538]: pgmap v536: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-05-14T17:12:51.633 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:11:50 smithi002 ceph-mon[31538]: pgmap v537: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-05-14T17:12:51.667 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:11:52 smithi002 ceph-mon[31538]: pgmap v538: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 29 KiB/s wr, 10 op/s 2024-05-14T17:12:51.703 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:11:54 smithi002 ceph-mon[31538]: pgmap v539: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 KiB/s wr, 14 op/s 2024-05-14T17:12:51.733 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:11:54 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:12:51.768 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:12:50 smithi151 ceph-mon[36066]: pgmap v569: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-05-14T17:12:51.800 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:12:50 smithi123 ceph-mon[35971]: pgmap v569: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-05-14T17:12:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.032+0000 7f131ddf0640 1 -- 172.21.15.2:0/1460943223 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1318103d00 msgr2=0x7f13181040e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.032+0000 7f131ddf0640 1 --2- 172.21.15.2:0/1460943223 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1318103d00 0x7f13181040e0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f1308009900 tx=0x7f130802a480 comp rx=0 tx=0).stop 2024-05-14T17:12:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.032+0000 7f131ddf0640 1 -- 172.21.15.2:0/1460943223 shutdown_connections 2024-05-14T17:12:51.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.032+0000 7f131ddf0640 1 --2- 172.21.15.2:0/1460943223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f131810d0d0 0x7f131810f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.032+0000 7f131ddf0640 1 --2- 172.21.15.2:0/1460943223 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1318104620 0x7f131810cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.032+0000 7f131ddf0640 1 --2- 172.21.15.2:0/1460943223 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1318103d00 0x7f13181040e0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.032+0000 7f131ddf0640 1 -- 172.21.15.2:0/1460943223 >> 172.21.15.2:0/1460943223 conn(0x7f1318077cd0 msgr2=0x7f13180780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.032+0000 7f131ddf0640 1 -- 172.21.15.2:0/1460943223 shutdown_connections 2024-05-14T17:12:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.032+0000 7f131ddf0640 1 -- 172.21.15.2:0/1460943223 wait complete. 2024-05-14T17:12:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.032+0000 7f131ddf0640 1 Processor -- start 2024-05-14T17:12:52.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.033+0000 7f131ddf0640 1 -- start start 2024-05-14T17:12:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.033+0000 7f131ddf0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1318103d00 0x7f13181991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.033+0000 7f131ddf0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1318104620 0x7f1318199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:52.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.033+0000 7f131ddf0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f131810d0d0 0x7f131819da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:52.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.033+0000 7f131ddf0640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1318112d10 con 0x7f1318104620 2024-05-14T17:12:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.033+0000 7f131ddf0640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1318112b90 con 0x7f131810d0d0 2024-05-14T17:12:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.033+0000 7f131ddf0640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1318112e90 con 0x7f1318103d00 2024-05-14T17:12:52.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.033+0000 7f1316ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1318104620 0x7f1318199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.033+0000 7f1316ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1318104620 0x7f1318199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54020/0 (socket says 172.21.15.2:54020) 2024-05-14T17:12:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.033+0000 7f1316ffd640 1 -- 172.21.15.2:0/1934230137 learned_addr learned my addr 172.21.15.2:0/1934230137 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:12:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.033+0000 7f13177fe640 1 --2- 172.21.15.2:0/1934230137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1318103d00 0x7f13181991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:52.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.033+0000 7f1317fff640 1 --2- 172.21.15.2:0/1934230137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f131810d0d0 0x7f131819da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.033+0000 7f1316ffd640 1 -- 172.21.15.2:0/1934230137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1318103d00 msgr2=0x7f13181991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.033+0000 7f1316ffd640 1 --2- 172.21.15.2:0/1934230137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1318103d00 0x7f13181991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.033+0000 7f1316ffd640 1 -- 172.21.15.2:0/1934230137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f131810d0d0 msgr2=0x7f131819da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.033+0000 7f1316ffd640 1 --2- 172.21.15.2:0/1934230137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f131810d0d0 0x7f131819da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.033+0000 7f1316ffd640 1 -- 172.21.15.2:0/1934230137 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f131819e170 con 0x7f1318104620 2024-05-14T17:12:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.033+0000 7f1317fff640 1 --2- 172.21.15.2:0/1934230137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f131810d0d0 0x7f131819da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:12:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.034+0000 7f1316ffd640 1 --2- 172.21.15.2:0/1934230137 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1318104620 0x7f1318199710 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f130c00c920 tx=0x7f130c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.034+0000 7f1314ff9640 1 -- 172.21.15.2:0/1934230137 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130c007c70 con 0x7f1318104620 2024-05-14T17:12:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.034+0000 7f1314ff9640 1 -- 172.21.15.2:0/1934230137 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f130c007e10 con 0x7f1318104620 2024-05-14T17:12:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.034+0000 7f1314ff9640 1 -- 172.21.15.2:0/1934230137 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130c018820 con 0x7f1318104620 2024-05-14T17:12:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.034+0000 7f131ddf0640 1 -- 172.21.15.2:0/1934230137 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f131819e460 con 0x7f1318104620 2024-05-14T17:12:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.034+0000 7f131ddf0640 1 -- 172.21.15.2:0/1934230137 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13181a5cf0 con 0x7f1318104620 2024-05-14T17:12:53.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.037+0000 7f1314ff9640 1 -- 172.21.15.2:0/1934230137 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f130c00f450 con 0x7f1318104620 2024-05-14T17:12:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.037+0000 7f1320be7740 1 Processor -- start 2024-05-14T17:12:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.037+0000 7f1320be7740 1 -- start start 2024-05-14T17:12:53.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.037+0000 7f1320be7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617dae237f0 0x5617dae23bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:53.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.037+0000 7f1320be7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617dae24110 0x5617dae329a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.037+0000 7f1320be7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5617dae32ee0 0x5617dae352d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.037+0000 7f1320be7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5617dad96c30 con 0x5617dae24110 2024-05-14T17:12:53.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.037+0000 7f1320be7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5617dad867d0 con 0x5617dae237f0 2024-05-14T17:12:53.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.037+0000 7f1320be7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5617dad8a810 con 0x5617dae32ee0 2024-05-14T17:12:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.037+0000 7f13177fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617dae24110 0x5617dae329a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.037+0000 7f13177fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617dae24110 0x5617dae329a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54030/0 (socket says 172.21.15.2:54030) 2024-05-14T17:12:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.037+0000 7f13177fe640 1 -- 172.21.15.2:0/3864599678 learned_addr learned my addr 172.21.15.2:0/3864599678 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:12:53.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.037+0000 7f1317fff640 1 --2- 172.21.15.2:0/3864599678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617dae237f0 0x5617dae23bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.037+0000 7f1316ffd640 1 --2- 172.21.15.2:0/3864599678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5617dae32ee0 0x5617dae352d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.037+0000 7f13177fe640 1 -- 172.21.15.2:0/3864599678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5617dae32ee0 msgr2=0x5617dae352d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.037+0000 7f13177fe640 1 --2- 172.21.15.2:0/3864599678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5617dae32ee0 0x5617dae352d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.037+0000 7f13177fe640 1 -- 172.21.15.2:0/3864599678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617dae237f0 msgr2=0x5617dae23bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:53.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.038+0000 7f13177fe640 1 --2- 172.21.15.2:0/3864599678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617dae237f0 0x5617dae23bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.038+0000 7f13177fe640 1 -- 172.21.15.2:0/3864599678 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617daa9b210 con 0x5617dae24110 2024-05-14T17:12:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.038+0000 7f1314ff9640 1 --2- 172.21.15.2:0/1934230137 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f13000b2a10 0x7f13000b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:53.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.038+0000 7f13177fe640 1 --2- 172.21.15.2:0/3864599678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617dae24110 0x5617dae329a0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f1308009900 tx=0x7f1308032e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98d3559ad1e7781c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:53.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.038+0000 7f1317fff640 1 --2- 172.21.15.2:0/1934230137 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f13000b2a10 0x7f13000b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.038+0000 7f1314ff9640 1 -- 172.21.15.2:0/1934230137 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f130c0e2780 con 0x7f1318104620 2024-05-14T17:12:53.893 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.038+0000 7f12fdffb640 1 -- 172.21.15.2:0/3864599678 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1308040070 con 0x5617dae24110 2024-05-14T17:12:53.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.038+0000 7f12fdffb640 1 -- 172.21.15.2:0/3864599678 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13080316a0 con 0x5617dae24110 2024-05-14T17:12:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.038+0000 7f12fdffb640 1 -- 172.21.15.2:0/3864599678 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1308035040 con 0x5617dae24110 2024-05-14T17:12:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.039+0000 7f1320be7740 1 -- 172.21.15.2:0/3864599678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617dae24110 msgr2=0x5617dae329a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.039+0000 7f1317fff640 1 --2- 172.21.15.2:0/1934230137 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f13000b2a10 0x7f13000b4ed0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f131806c360 tx=0x7f1310009780 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:54.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.039+0000 7f1320be7740 1 --2- 172.21.15.2:0/3864599678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617dae24110 0x5617dae329a0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f1308009900 tx=0x7f1308032e60 comp rx=0 tx=0).stop 2024-05-14T17:12:54.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.039+0000 7f1320be7740 1 -- 172.21.15.2:0/3864599678 shutdown_connections 2024-05-14T17:12:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.039+0000 7f1320be7740 1 --2- 172.21.15.2:0/3864599678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5617dae32ee0 0x5617dae352d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.039+0000 7f1320be7740 1 --2- 172.21.15.2:0/3864599678 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617dae24110 0x5617dae329a0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.039+0000 7f1320be7740 1 --2- 172.21.15.2:0/3864599678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617dae237f0 0x5617dae23bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:54.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.039+0000 7f1320be7740 1 -- 172.21.15.2:0/3864599678 >> 172.21.15.2:0/3864599678 conn(0x5617dae1fe00 msgr2=0x5617dae201e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.039+0000 7f1320be7740 1 -- 172.21.15.2:0/3864599678 shutdown_connections 2024-05-14T17:12:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.039+0000 7f1320be7740 1 -- 172.21.15.2:0/3864599678 wait complete. 2024-05-14T17:12:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.039+0000 7f1320be7740 1 -- start start 2024-05-14T17:12:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.040+0000 7f1320be7740 1 Processor -- start 2024-05-14T17:12:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.040+0000 7f1320be7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5617dae237f0 0x5617dae32820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.040+0000 7f1320be7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617dae24110 0x5617dae2fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.040+0000 7f1320be7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617dae32ee0 0x5617dae30050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:54.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.040+0000 7f1320be7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5617daa6c790 con 0x5617dae24110 2024-05-14T17:12:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.040+0000 7f1320be7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5617dad42890 con 0x5617dae32ee0 2024-05-14T17:12:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.040+0000 7f1320be7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5617dae35b50 con 0x5617dae237f0 2024-05-14T17:12:54.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.040+0000 7f1320be7740 10 client.0 _ll_register_callbacks cb 0x5617dad45d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:12:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.040+0000 7f1316ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617dae24110 0x5617dae2fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:54.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.040+0000 7f1316ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617dae24110 0x5617dae2fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54032/0 (socket says 172.21.15.2:54032) 2024-05-14T17:12:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.040+0000 7f1316ffd640 1 -- 172.21.15.2:0/3876571095 learned_addr learned my addr 172.21.15.2:0/3876571095 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:12:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.040+0000 7f13177fe640 1 --2- 172.21.15.2:0/3876571095 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5617dae237f0 0x5617dae32820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:54.844 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.040+0000 7f1317fff640 1 --2- 172.21.15.2:0/3876571095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617dae32ee0 0x5617dae30050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:54.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.040+0000 7f1316ffd640 1 -- 172.21.15.2:0/3876571095 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5617dae237f0 msgr2=0x5617dae32820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.040+0000 7f1316ffd640 1 --2- 172.21.15.2:0/3876571095 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5617dae237f0 0x5617dae32820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.040+0000 7f1316ffd640 1 -- 172.21.15.2:0/3876571095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617dae32ee0 msgr2=0x5617dae30050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.040+0000 7f1316ffd640 1 --2- 172.21.15.2:0/3876571095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617dae32ee0 0x5617dae30050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:55.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.040+0000 7f1316ffd640 1 -- 172.21.15.2:0/3876571095 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617dab67620 con 0x5617dae24110 2024-05-14T17:12:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.040+0000 7f1316ffd640 1 --2- 172.21.15.2:0/3876571095 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617dae24110 0x5617dae2fb10 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f130c007590 tx=0x7f130c00fcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2b779fd4c796469 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:55.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.040+0000 7f12f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:12:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.041+0000 7f12f6ffd640 1 -- 172.21.15.2:0/3876571095 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130c0aa160 con 0x5617dae24110 2024-05-14T17:12:55.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.041+0000 7f12f6ffd640 1 -- 172.21.15.2:0/3876571095 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f130c0a5cf0 con 0x5617dae24110 2024-05-14T17:12:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.041+0000 7f1320be7740 1 -- 172.21.15.2:0/3876571095 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13080316a0 con 0x5617dae24110 2024-05-14T17:12:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.041+0000 7f12f6ffd640 1 -- 172.21.15.2:0/3876571095 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f130c0a9a60 con 0x5617dae24110 2024-05-14T17:12:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.041+0000 7f1320be7740 10 client.17013 fetch_fsmap learned FSMap version 54 2024-05-14T17:12:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.041+0000 7f1320be7740 1 -- 172.21.15.2:0/3876571095 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5617dae41680 con 0x5617dae24110 2024-05-14T17:12:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.041+0000 7f12f6ffd640 1 -- 172.21.15.2:0/3876571095 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13080316a0 con 0x5617dae24110 2024-05-14T17:12:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.041+0000 7f12f6ffd640 1 -- 172.21.15.2:0/3876571095 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 54) ==== 8338+0+0 (secure 0 0 0) 0x7f130c0b2430 con 0x5617dae24110 2024-05-14T17:12:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.041+0000 7f1320be7740 10 client.17013 fetch_fsmap finished waiting for FSMap version 54 2024-05-14T17:12:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.041+0000 7f1320be7740 10 client.17013 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:12:55.503 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.041+0000 7f1320be7740 20 client.17013 populate_metadata read hostname 'smithi002' 2024-05-14T17:12:55.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.041+0000 7f1320be7740 1 --2- 172.21.15.2:0/3876571095 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5617dae42b40 0x5617dae44f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:55.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.041+0000 7f1320be7740 4 client.17013 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:12:55.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.041+0000 7f1320be7740 1 -- 172.21.15.2:0/3876571095 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617dad98b80 con 0x5617dae42b40 2024-05-14T17:12:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.042+0000 7f13177fe640 1 --2- 172.21.15.2:0/3876571095 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5617dae42b40 0x5617dae44f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.042+0000 7f13177fe640 1 --2- 172.21.15.2:0/3876571095 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5617dae42b40 0x5617dae44f30 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d39794d3bb0cd11 server_cookie=d7a175f8f40df2a6 in_seq=0 out_seq=0 2024-05-14T17:12:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.042+0000 7f12f6ffd640 10 client.17013 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:12:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.043+0000 7f12f6ffd640 1 -- 172.21.15.2:0/3876571095 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5617dad98b80 con 0x5617dae42b40 2024-05-14T17:12:55.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.043+0000 7f12f6ffd640 10 client.17013 handle_command_reply: tid=0 2024-05-14T17:12:55.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.043+0000 7f1320be7740 1 client.17013 shutdown 2024-05-14T17:12:55.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.043+0000 7f1320be7740 1 -- 172.21.15.2:0/3876571095 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617dae24110 msgr2=0x5617dae2fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:55.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.043+0000 7f1320be7740 1 --2- 172.21.15.2:0/3876571095 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617dae24110 0x5617dae2fb10 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f130c007590 tx=0x7f130c00fcc0 comp rx=0 tx=0).stop 2024-05-14T17:12:55.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.043+0000 7f1320be7740 1 -- 172.21.15.2:0/3876571095 shutdown_connections 2024-05-14T17:12:55.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.043+0000 7f1320be7740 1 --2- 172.21.15.2:0/3876571095 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5617dae42b40 0x5617dae44f30 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.043+0000 7f1320be7740 1 --2- 172.21.15.2:0/3876571095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617dae32ee0 0x5617dae30050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.043+0000 7f1320be7740 1 --2- 172.21.15.2:0/3876571095 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617dae24110 0x5617dae2fb10 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.043+0000 7f12f6ffd640 0 client.17013 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:12:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.043+0000 7f1320be7740 1 --2- 172.21.15.2:0/3876571095 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5617dae237f0 0x5617dae32820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:56.180 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.043+0000 7f1320be7740 1 -- 172.21.15.2:0/3876571095 >> 172.21.15.2:0/3876571095 conn(0x5617dae1fe00 msgr2=0x5617dae25c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.043+0000 7f1320be7740 1 -- 172.21.15.2:0/3876571095 shutdown_connections 2024-05-14T17:12:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.043+0000 7f1320be7740 1 -- 172.21.15.2:0/3876571095 wait complete. 2024-05-14T17:12:56.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.043+0000 7f1320be7740 20 client.17013 trim_cache size 0 max 16384 2024-05-14T17:12:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.058+0000 7f1320be7740 1 Processor -- start 2024-05-14T17:12:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.058+0000 7f1320be7740 1 -- start start 2024-05-14T17:12:56.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.059+0000 7f1320be7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617dae237f0 0x5617dae2b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.059+0000 7f1320be7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5617dae24110 0x5617dae32350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:56.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.059+0000 7f1320be7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617dae32ee0 0x5617dae25aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.059+0000 7f1320be7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f13181aeb20 con 0x5617dae237f0 2024-05-14T17:12:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.059+0000 7f1320be7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f13181ac3f0 con 0x5617dae32ee0 2024-05-14T17:12:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.059+0000 7f1320be7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f13181aa280 con 0x5617dae24110 2024-05-14T17:12:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.059+0000 7f13177fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617dae237f0 0x5617dae2b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.059+0000 7f13177fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617dae237f0 0x5617dae2b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54046/0 (socket says 172.21.15.2:54046) 2024-05-14T17:12:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.059+0000 7f13177fe640 1 -- 172.21.15.2:0/487492700 learned_addr learned my addr 172.21.15.2:0/487492700 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:12:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.059+0000 7f13177fe640 1 -- 172.21.15.2:0/487492700 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5617dae24110 msgr2=0x5617dae32350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.059+0000 7f13177fe640 1 --2- 172.21.15.2:0/487492700 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5617dae24110 0x5617dae32350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:56.814 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.059+0000 7f13177fe640 1 -- 172.21.15.2:0/487492700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617dae32ee0 msgr2=0x5617dae25aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.059+0000 7f13177fe640 1 --2- 172.21.15.2:0/487492700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617dae32ee0 0x5617dae25aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.059+0000 7f13177fe640 1 -- 172.21.15.2:0/487492700 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617dae24650 con 0x5617dae237f0 2024-05-14T17:12:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.059+0000 7f13177fe640 1 --2- 172.21.15.2:0/487492700 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617dae237f0 0x5617dae2b1d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f1308009900 tx=0x7f1308005420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2f6c0eac00f6dd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.059+0000 7f12fd7fa640 1 -- 172.21.15.2:0/487492700 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1308035040 con 0x5617dae237f0 2024-05-14T17:12:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.059+0000 7f12fd7fa640 1 -- 172.21.15.2:0/487492700 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13080315c0 con 0x5617dae237f0 2024-05-14T17:12:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.059+0000 7f12fd7fa640 1 -- 172.21.15.2:0/487492700 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1308040040 con 0x5617dae237f0 2024-05-14T17:12:57.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.060+0000 7f1320be7740 1 -- 172.21.15.2:0/487492700 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617dae237f0 msgr2=0x5617dae2b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:57.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.060+0000 7f1320be7740 1 --2- 172.21.15.2:0/487492700 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617dae237f0 0x5617dae2b1d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f1308009900 tx=0x7f1308005420 comp rx=0 tx=0).stop 2024-05-14T17:12:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.060+0000 7f1320be7740 1 -- 172.21.15.2:0/487492700 shutdown_connections 2024-05-14T17:12:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.060+0000 7f1320be7740 1 --2- 172.21.15.2:0/487492700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617dae32ee0 0x5617dae25aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:57.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.060+0000 7f1320be7740 1 --2- 172.21.15.2:0/487492700 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5617dae24110 0x5617dae32350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:57.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.060+0000 7f1320be7740 1 --2- 172.21.15.2:0/487492700 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617dae237f0 0x5617dae2b1d0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.060+0000 7f1320be7740 1 -- 172.21.15.2:0/487492700 >> 172.21.15.2:0/487492700 conn(0x5617dae1fe00 msgr2=0x5617dae25390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:57.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.060+0000 7f1320be7740 1 -- 172.21.15.2:0/487492700 shutdown_connections 2024-05-14T17:12:57.357 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.060+0000 7f1320be7740 1 -- 172.21.15.2:0/487492700 wait complete. 2024-05-14T17:12:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.060+0000 7f1320be7740 1 -- start start 2024-05-14T17:12:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.061+0000 7f1320be7740 1 Processor -- start 2024-05-14T17:12:57.456 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.061+0000 7f1320be7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5617dae237f0 0x5617dae3b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:57.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.061+0000 7f1320be7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617dae24110 0x5617dae366e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.061+0000 7f1320be7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617dae32ee0 0x5617dae36c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.061+0000 7f1320be7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f13181aa8d0 con 0x5617dae32ee0 2024-05-14T17:12:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.061+0000 7f1320be7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f13181abd20 con 0x5617dae24110 2024-05-14T17:12:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.061+0000 7f1320be7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f13181aaf80 con 0x5617dae237f0 2024-05-14T17:12:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.061+0000 7f1320be7740 10 client.0 _ll_register_callbacks cb 0x7f130c0a5920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:12:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.061+0000 7f1317fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617dae32ee0 0x5617dae36c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.061+0000 7f1317fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617dae32ee0 0x5617dae36c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54062/0 (socket says 172.21.15.2:54062) 2024-05-14T17:12:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.061+0000 7f1317fff640 1 -- 172.21.15.2:0/1778884156 learned_addr learned my addr 172.21.15.2:0/1778884156 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:12:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.061+0000 7f13177fe640 1 --2- 172.21.15.2:0/1778884156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5617dae237f0 0x5617dae3b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:57.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.061+0000 7f1316ffd640 1 --2- 172.21.15.2:0/1778884156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617dae24110 0x5617dae366e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.061+0000 7f1317fff640 1 -- 172.21.15.2:0/1778884156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5617dae237f0 msgr2=0x5617dae3b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.061+0000 7f1317fff640 1 --2- 172.21.15.2:0/1778884156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5617dae237f0 0x5617dae3b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.061+0000 7f1317fff640 1 -- 172.21.15.2:0/1778884156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617dae24110 msgr2=0x5617dae366e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:12:57.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.061+0000 7f1317fff640 1 --2- 172.21.15.2:0/1778884156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617dae24110 0x5617dae366e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:58.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.061+0000 7f1317fff640 1 -- 172.21.15.2:0/1778884156 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617dae1fc30 con 0x5617dae32ee0 2024-05-14T17:12:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.061+0000 7f13177fe640 1 --2- 172.21.15.2:0/1778884156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5617dae237f0 0x5617dae3b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:12:58.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.061+0000 7f1317fff640 1 --2- 172.21.15.2:0/1778884156 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617dae32ee0 0x5617dae36c20 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f1310009fd0 tx=0x7f1310004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f4336096e16ea27 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:12:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.061+0000 7f12f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:12:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.062+0000 7f12f6ffd640 1 -- 172.21.15.2:0/1778884156 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1310004d20 con 0x5617dae32ee0 2024-05-14T17:12:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.062+0000 7f12f6ffd640 1 -- 172.21.15.2:0/1778884156 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f1310005020 con 0x5617dae32ee0 2024-05-14T17:12:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.062+0000 7f1320be7740 1 -- 172.21.15.2:0/1778884156 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13080315c0 con 0x5617dae32ee0 2024-05-14T17:12:58.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.062+0000 7f12f6ffd640 1 -- 172.21.15.2:0/1778884156 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f1310005220 con 0x5617dae32ee0 2024-05-14T17:12:58.283 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.062+0000 7f1320be7740 10 client.17019 fetch_fsmap learned FSMap version 54 2024-05-14T17:12:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.062+0000 7f1320be7740 1 -- 172.21.15.2:0/1778884156 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5617daead280 con 0x5617dae32ee0 2024-05-14T17:12:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.062+0000 7f12f6ffd640 1 -- 172.21.15.2:0/1778884156 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13080315c0 con 0x5617dae32ee0 2024-05-14T17:12:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.062+0000 7f12f6ffd640 1 -- 172.21.15.2:0/1778884156 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 54) ==== 8338+0+0 (secure 0 0 0) 0x7f1310028070 con 0x5617dae32ee0 2024-05-14T17:12:58.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.062+0000 7f1320be7740 10 client.17019 fetch_fsmap finished waiting for FSMap version 54 2024-05-14T17:12:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.062+0000 7f1320be7740 10 client.17019 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:12:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.062+0000 7f1320be7740 20 client.17019 populate_metadata read hostname 'smithi002' 2024-05-14T17:12:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.062+0000 7f1320be7740 1 --2- 172.21.15.2:0/1778884156 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5617daeadab0 0x5617daeafed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:12:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.062+0000 7f1320be7740 4 client.17019 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:12:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.062+0000 7f1320be7740 1 -- 172.21.15.2:0/1778884156 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5617dad92360 con 0x5617daeadab0 2024-05-14T17:12:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.063+0000 7f13177fe640 1 --2- 172.21.15.2:0/1778884156 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5617daeadab0 0x5617daeafed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:12:58.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.063+0000 7f13177fe640 1 --2- 172.21.15.2:0/1778884156 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5617daeadab0 0x5617daeafed0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7c295c910e7e05f server_cookie=fea430cb207214a9 in_seq=0 out_seq=0 2024-05-14T17:12:58.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.063+0000 7f12f6ffd640 10 client.17019 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:12:58.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.075+0000 7f12f6ffd640 1 -- 172.21.15.2:0/1778884156 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5617dad92360 con 0x5617daeadab0 2024-05-14T17:12:58.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.075+0000 7f12f6ffd640 10 client.17019 handle_command_reply: tid=0 2024-05-14T17:12:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.075+0000 7f1320be7740 1 client.17019 shutdown 2024-05-14T17:12:58.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.075+0000 7f1320be7740 1 -- 172.21.15.2:0/1778884156 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617dae32ee0 msgr2=0x5617dae36c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.075+0000 7f1320be7740 1 --2- 172.21.15.2:0/1778884156 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617dae32ee0 0x5617dae36c20 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f1310009fd0 tx=0x7f1310004ab0 comp rx=0 tx=0).stop 2024-05-14T17:12:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.076+0000 7f1320be7740 1 -- 172.21.15.2:0/1778884156 shutdown_connections 2024-05-14T17:12:59.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.076+0000 7f1320be7740 1 --2- 172.21.15.2:0/1778884156 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5617daeadab0 0x5617daeafed0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:59.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.076+0000 7f1320be7740 1 --2- 172.21.15.2:0/1778884156 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617dae32ee0 0x5617dae36c20 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.076+0000 7f1320be7740 1 --2- 172.21.15.2:0/1778884156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617dae24110 0x5617dae366e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.076+0000 7f1320be7740 1 --2- 172.21.15.2:0/1778884156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5617dae237f0 0x5617dae3b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.076+0000 7f12f6ffd640 0 client.17019 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:12:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.076+0000 7f1320be7740 1 -- 172.21.15.2:0/1778884156 >> 172.21.15.2:0/1778884156 conn(0x5617dae1fe00 msgr2=0x5617dae1f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.076+0000 7f1320be7740 1 -- 172.21.15.2:0/1778884156 shutdown_connections 2024-05-14T17:12:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.076+0000 7f1320be7740 1 -- 172.21.15.2:0/1778884156 wait complete. 2024-05-14T17:12:59.376 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.076+0000 7f1320be7740 20 client.17019 trim_cache size 0 max 16384 2024-05-14T17:12:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:12:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:12:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:12:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.076+0000 7f131ddf0640 1 -- 172.21.15.2:0/1934230137 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f13000b2a10 msgr2=0x7f13000b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.076+0000 7f131ddf0640 1 --2- 172.21.15.2:0/1934230137 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f13000b2a10 0x7f13000b4ed0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f131806c360 tx=0x7f1310009780 comp rx=0 tx=0).stop 2024-05-14T17:12:59.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.076+0000 7f131ddf0640 1 -- 172.21.15.2:0/1934230137 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1318104620 msgr2=0x7f1318199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:12:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.076+0000 7f131ddf0640 1 --2- 172.21.15.2:0/1934230137 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1318104620 0x7f1318199710 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f130c00c920 tx=0x7f130c00cde0 comp rx=0 tx=0).stop 2024-05-14T17:12:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.076+0000 7f131ddf0640 1 -- 172.21.15.2:0/1934230137 shutdown_connections 2024-05-14T17:12:59.693 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.076+0000 7f131ddf0640 1 --2- 172.21.15.2:0/1934230137 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f13000b2a10 0x7f13000b4ed0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:59.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.076+0000 7f131ddf0640 1 --2- 172.21.15.2:0/1934230137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f131810d0d0 0x7f131819da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.076+0000 7f131ddf0640 1 --2- 172.21.15.2:0/1934230137 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1318104620 0x7f1318199710 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:59.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.076+0000 7f131ddf0640 1 --2- 172.21.15.2:0/1934230137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1318103d00 0x7f13181991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:12:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.076+0000 7f131ddf0640 1 -- 172.21.15.2:0/1934230137 >> 172.21.15.2:0/1934230137 conn(0x7f1318077cd0 msgr2=0x7f13180ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:12:59.943 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.077+0000 7f131ddf0640 1 -- 172.21.15.2:0/1934230137 shutdown_connections 2024-05-14T17:12:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:11:57.077+0000 7f131ddf0640 1 -- 172.21.15.2:0/1934230137 wait complete. 2024-05-14T17:13:00.011 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:11:56 smithi002 ceph-mon[31538]: pgmap v540: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 34 KiB/s wr, 11 op/s 2024-05-14T17:13:00.043 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:11:58 smithi002 ceph-mon[31538]: pgmap v541: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 34 KiB/s wr, 11 op/s 2024-05-14T17:13:00.076 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:12:00 smithi002 ceph-mon[31538]: pgmap v542: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 53 KiB/s wr, 17 op/s 2024-05-14T17:13:00.109 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:12:02 smithi002 ceph-mon[31538]: pgmap v543: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 30 KiB/s wr, 10 op/s 2024-05-14T17:13:00.151 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:12:04 smithi002 ceph-mon[31538]: pgmap v544: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 75 KiB/s wr, 30 op/s 2024-05-14T17:13:00.184 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:12:05 smithi002 ceph-mon[31538]: pgmap v545: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 64 KiB/s wr, 26 op/s 2024-05-14T17:13:00.218 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:12:08 smithi002 ceph-mon[31538]: pgmap v546: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 64 KiB/s wr, 26 op/s 2024-05-14T17:13:00.251 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:12:09 smithi002 ceph-mon[31538]: osdmap e102: 12 total, 12 up, 12 in 2024-05-14T17:13:00.285 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:12:52 smithi151 ceph-mon[36066]: pgmap v570: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-05-14T17:13:00.368 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:12:54 smithi151 ceph-mon[36066]: pgmap v571: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 1.1 MiB/s wr, 139 op/s 2024-05-14T17:13:00.401 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:12:56 smithi151 ceph-mon[36066]: pgmap v572: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.7 MiB/s wr, 203 op/s 2024-05-14T17:13:00.435 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:12:58 smithi151 ceph-mon[36066]: pgmap v573: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.7 MiB/s wr, 203 op/s 2024-05-14T17:13:00.468 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:12:52 smithi123 ceph-mon[35971]: pgmap v570: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-05-14T17:13:00.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:12:54 smithi123 ceph-mon[35971]: pgmap v571: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 1.1 MiB/s wr, 139 op/s 2024-05-14T17:13:00.543 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:12:56 smithi123 ceph-mon[35971]: pgmap v572: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.7 MiB/s wr, 203 op/s 2024-05-14T17:13:00.577 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:12:58 smithi123 ceph-mon[35971]: pgmap v573: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.7 MiB/s wr, 203 op/s 2024-05-14T17:13:00.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.177+0000 7f33ebb1c640 1 -- 172.21.15.2:0/1165578564 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33e4101880 msgr2=0x7f33e4101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.177+0000 7f33ebb1c640 1 --2- 172.21.15.2:0/1165578564 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33e4101880 0x7f33e4101c60 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f33e0009900 tx=0x7f33e002a410 comp rx=0 tx=0).stop 2024-05-14T17:13:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.178+0000 7f33ebb1c640 1 -- 172.21.15.2:0/1165578564 shutdown_connections 2024-05-14T17:13:00.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.178+0000 7f33ebb1c640 1 --2- 172.21.15.2:0/1165578564 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33e410f430 0x7f33e4111820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.178+0000 7f33ebb1c640 1 --2- 172.21.15.2:0/1165578564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33e41021a0 0x7f33e410eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.178+0000 7f33ebb1c640 1 --2- 172.21.15.2:0/1165578564 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33e4101880 0x7f33e4101c60 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.178+0000 7f33ebb1c640 1 -- 172.21.15.2:0/1165578564 >> 172.21.15.2:0/1165578564 conn(0x7f33e4077cd0 msgr2=0x7f33e40780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.178+0000 7f33ebb1c640 1 -- 172.21.15.2:0/1165578564 shutdown_connections 2024-05-14T17:13:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.178+0000 7f33ebb1c640 1 -- 172.21.15.2:0/1165578564 wait complete. 2024-05-14T17:13:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.178+0000 7f33ebb1c640 1 Processor -- start 2024-05-14T17:13:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.178+0000 7f33ebb1c640 1 -- start start 2024-05-14T17:13:01.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.179+0000 7f33ebb1c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33e4101880 0x7f33e419b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:01.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.179+0000 7f33ebb1c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33e41021a0 0x7f33e419ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.179+0000 7f33ebb1c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33e410f430 0x7f33e419fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:01.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.179+0000 7f33ebb1c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f33e4114f40 con 0x7f33e410f430 2024-05-14T17:13:01.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.179+0000 7f33ebb1c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f33e4114dc0 con 0x7f33e41021a0 2024-05-14T17:13:01.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.179+0000 7f33ebb1c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f33e41150c0 con 0x7f33e4101880 2024-05-14T17:13:01.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.179+0000 7f33ea0b9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33e410f430 0x7f33e419fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:01.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.179+0000 7f33ea0b9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33e410f430 0x7f33e419fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54932/0 (socket says 172.21.15.2:54932) 2024-05-14T17:13:01.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.179+0000 7f33ea0b9640 1 -- 172.21.15.2:0/1576721454 learned_addr learned my addr 172.21.15.2:0/1576721454 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:13:01.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.179+0000 7f33e98b8640 1 --2- 172.21.15.2:0/1576721454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33e4101880 0x7f33e419b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.179+0000 7f33e90b7640 1 --2- 172.21.15.2:0/1576721454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33e41021a0 0x7f33e419ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.179+0000 7f33ea0b9640 1 -- 172.21.15.2:0/1576721454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33e4101880 msgr2=0x7f33e419b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.179+0000 7f33ea0b9640 1 --2- 172.21.15.2:0/1576721454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33e4101880 0x7f33e419b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.179+0000 7f33ea0b9640 1 -- 172.21.15.2:0/1576721454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33e41021a0 msgr2=0x7f33e419ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.179+0000 7f33ea0b9640 1 --2- 172.21.15.2:0/1576721454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33e41021a0 0x7f33e419ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.179+0000 7f33ea0b9640 1 -- 172.21.15.2:0/1576721454 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33e41a04d0 con 0x7f33e410f430 2024-05-14T17:13:01.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.179+0000 7f33ea0b9640 1 --2- 172.21.15.2:0/1576721454 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33e410f430 0x7f33e419fda0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f33dc009fd0 tx=0x7f33dc00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.179+0000 7f33daffd640 1 -- 172.21.15.2:0/1576721454 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc004d20 con 0x7f33e410f430 2024-05-14T17:13:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.180+0000 7f33daffd640 1 -- 172.21.15.2:0/1576721454 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33dc005020 con 0x7f33e410f430 2024-05-14T17:13:01.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.180+0000 7f33daffd640 1 -- 172.21.15.2:0/1576721454 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc0116b0 con 0x7f33e410f430 2024-05-14T17:13:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.180+0000 7f33ebb1c640 1 -- 172.21.15.2:0/1576721454 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33e41a07c0 con 0x7f33e410f430 2024-05-14T17:13:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.180+0000 7f33ebb1c640 1 -- 172.21.15.2:0/1576721454 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33e41a8050 con 0x7f33e410f430 2024-05-14T17:13:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.182+0000 7f33daffd640 1 -- 172.21.15.2:0/1576721454 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f33dc0051d0 con 0x7f33e410f430 2024-05-14T17:13:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.182+0000 7f33ee831740 1 Processor -- start 2024-05-14T17:13:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.183+0000 7f33ee831740 1 -- start start 2024-05-14T17:13:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.183+0000 7f33ee831740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea981618f0 0x55ea98161cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.183+0000 7f33ee831740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ea98162210 0x55ea9816c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.183+0000 7f33ee831740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea9816cad0 0x55ea9816eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:02.074 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.183+0000 7f33ee831740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ea980d4c30 con 0x55ea981618f0 2024-05-14T17:13:02.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.183+0000 7f33ee831740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ea980c47d0 con 0x55ea9816cad0 2024-05-14T17:13:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.183+0000 7f33ee831740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ea980c8810 con 0x55ea98162210 2024-05-14T17:13:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.183+0000 7f33e90b7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea981618f0 0x55ea98161cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.183+0000 7f33e90b7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea981618f0 0x55ea98161cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54946/0 (socket says 172.21.15.2:54946) 2024-05-14T17:13:02.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.183+0000 7f33e90b7640 1 -- 172.21.15.2:0/2817003802 learned_addr learned my addr 172.21.15.2:0/2817003802 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:13:02.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.183+0000 7f33e90b7640 1 --2- 172.21.15.2:0/2817003802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea9816cad0 0x55ea9816eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.183+0000 7f33e98b8640 1 --2- 172.21.15.2:0/2817003802 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ea98162210 0x55ea9816c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.183+0000 7f33daffd640 1 --2- 172.21.15.2:0/1576721454 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f33d00b2a10 0x7f33d00b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.184+0000 7f33e90b7640 1 -- 172.21.15.2:0/2817003802 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ea98162210 msgr2=0x55ea9816c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.184+0000 7f33ea0b9640 1 --2- 172.21.15.2:0/1576721454 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f33d00b2a10 0x7f33d00b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:02.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.184+0000 7f33e90b7640 1 --2- 172.21.15.2:0/2817003802 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ea98162210 0x55ea9816c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:02.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.184+0000 7f33e90b7640 1 -- 172.21.15.2:0/2817003802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea9816cad0 msgr2=0x55ea9816eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:02.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.184+0000 7f33e90b7640 1 --2- 172.21.15.2:0/2817003802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea9816cad0 0x55ea9816eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.184+0000 7f33e90b7640 1 -- 172.21.15.2:0/2817003802 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea97dd9210 con 0x55ea981618f0 2024-05-14T17:13:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.184+0000 7f33daffd640 1 -- 172.21.15.2:0/1576721454 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8557+0+0 (secure 0 0 0) 0x7f33dc0e2c10 con 0x7f33e410f430 2024-05-14T17:13:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.184+0000 7f33e90b7640 1 --2- 172.21.15.2:0/2817003802 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea981618f0 0x55ea98161cd0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f33d40058c0 tx=0x7f33d400b470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f7f1ffa681d3257 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.184+0000 7f33cffff640 1 -- 172.21.15.2:0/2817003802 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d400d8e0 con 0x55ea981618f0 2024-05-14T17:13:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.184+0000 7f33cffff640 1 -- 172.21.15.2:0/2817003802 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33d4011400 con 0x55ea981618f0 2024-05-14T17:13:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.184+0000 7f33cffff640 1 -- 172.21.15.2:0/2817003802 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d400f650 con 0x55ea981618f0 2024-05-14T17:13:02.853 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.185+0000 7f33ee831740 1 -- 172.21.15.2:0/2817003802 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea981618f0 msgr2=0x55ea98161cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.185+0000 7f33ee831740 1 --2- 172.21.15.2:0/2817003802 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea981618f0 0x55ea98161cd0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f33d40058c0 tx=0x7f33d400b470 comp rx=0 tx=0).stop 2024-05-14T17:13:02.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.185+0000 7f33ea0b9640 1 --2- 172.21.15.2:0/1576721454 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f33d00b2a10 0x7f33d00b4ed0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f33dc0a9bd0 tx=0x7f33dc0a4040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:02.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.185+0000 7f33ee831740 1 -- 172.21.15.2:0/2817003802 shutdown_connections 2024-05-14T17:13:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.185+0000 7f33ee831740 1 --2- 172.21.15.2:0/2817003802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea9816cad0 0x55ea9816eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.185+0000 7f33ee831740 1 --2- 172.21.15.2:0/2817003802 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ea98162210 0x55ea9816c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:03.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.185+0000 7f33ee831740 1 --2- 172.21.15.2:0/2817003802 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea981618f0 0x55ea98161cd0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:03.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.185+0000 7f33ee831740 1 -- 172.21.15.2:0/2817003802 >> 172.21.15.2:0/2817003802 conn(0x55ea9815de00 msgr2=0x55ea9815e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:03.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.185+0000 7f33ee831740 1 -- 172.21.15.2:0/2817003802 shutdown_connections 2024-05-14T17:13:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.185+0000 7f33ee831740 1 -- 172.21.15.2:0/2817003802 wait complete. 2024-05-14T17:13:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.185+0000 7f33ee831740 1 -- start start 2024-05-14T17:13:03.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.185+0000 7f33ee831740 1 Processor -- start 2024-05-14T17:13:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.186+0000 7f33ee831740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ea981618f0 0x55ea9816c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:03.312 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.186+0000 7f33ee831740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea98162210 0x55ea981696b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:03.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.186+0000 7f33ee831740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea9816cad0 0x55ea98169bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:03.396 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.186+0000 7f33ee831740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ea97daa790 con 0x55ea98162210 2024-05-14T17:13:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.186+0000 7f33ee831740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ea98080890 con 0x55ea9816cad0 2024-05-14T17:13:03.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.186+0000 7f33ee831740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ea9816f740 con 0x55ea981618f0 2024-05-14T17:13:03.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.186+0000 7f33ee831740 10 client.0 _ll_register_callbacks cb 0x55ea98083d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:13:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.186+0000 7f33e98b8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea98162210 0x55ea981696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.186+0000 7f33e98b8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea98162210 0x55ea981696b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54952/0 (socket says 172.21.15.2:54952) 2024-05-14T17:13:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.186+0000 7f33e98b8640 1 -- 172.21.15.2:0/2508320225 learned_addr learned my addr 172.21.15.2:0/2508320225 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:13:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.186+0000 7f33e90b7640 1 --2- 172.21.15.2:0/2508320225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea9816cad0 0x55ea98169bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:03.676 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.186+0000 7f33e90b7640 1 --2- 172.21.15.2:0/2508320225 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ea981618f0 0x55ea9816c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:03.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.186+0000 7f33e98b8640 1 -- 172.21.15.2:0/2508320225 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ea981618f0 msgr2=0x55ea9816c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:03.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.186+0000 7f33e98b8640 1 --2- 172.21.15.2:0/2508320225 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ea981618f0 0x55ea9816c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:03.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.186+0000 7f33e98b8640 1 -- 172.21.15.2:0/2508320225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea9816cad0 msgr2=0x55ea98169bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:03.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.186+0000 7f33e98b8640 1 --2- 172.21.15.2:0/2508320225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea9816cad0 0x55ea98169bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.186+0000 7f33e98b8640 1 -- 172.21.15.2:0/2508320225 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea97ea5620 con 0x55ea98162210 2024-05-14T17:13:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.186+0000 7f33e90b7640 1 --2- 172.21.15.2:0/2508320225 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ea981618f0 0x55ea9816c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:13:03.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.187+0000 7f33e98b8640 1 --2- 172.21.15.2:0/2508320225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea98162210 0x55ea981696b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f33e002a930 tx=0x7f33e00317a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c20b3e915847a071 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.187+0000 7f33cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:13:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.187+0000 7f33cd7fa640 1 -- 172.21.15.2:0/2508320225 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e0035040 con 0x55ea98162210 2024-05-14T17:13:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.187+0000 7f33cd7fa640 1 -- 172.21.15.2:0/2508320225 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8557+0+0 (secure 0 0 0) 0x7f33e0004040 con 0x55ea98162210 2024-05-14T17:13:04.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.187+0000 7f33ee831740 1 -- 172.21.15.2:0/2508320225 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33d4011400 con 0x55ea98162210 2024-05-14T17:13:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.187+0000 7f33cd7fa640 1 -- 172.21.15.2:0/2508320225 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f33e0004240 con 0x55ea98162210 2024-05-14T17:13:04.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.187+0000 7f33ee831740 10 client.17031 fetch_fsmap learned FSMap version 54 2024-05-14T17:13:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.187+0000 7f33ee831740 1 -- 172.21.15.2:0/2508320225 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55ea9817b100 con 0x55ea98162210 2024-05-14T17:13:04.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.187+0000 7f33cd7fa640 1 -- 172.21.15.2:0/2508320225 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33d4011400 con 0x55ea98162210 2024-05-14T17:13:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.187+0000 7f33cd7fa640 1 -- 172.21.15.2:0/2508320225 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 54) ==== 8338+0+0 (secure 0 0 0) 0x7f33e0004aa0 con 0x55ea98162210 2024-05-14T17:13:04.365 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.187+0000 7f33ee831740 10 client.17031 fetch_fsmap finished waiting for FSMap version 54 2024-05-14T17:13:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.187+0000 7f33ee831740 10 client.17031 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:13:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.187+0000 7f33ee831740 20 client.17031 populate_metadata read hostname 'smithi002' 2024-05-14T17:13:04.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.188+0000 7f33ee831740 1 --2- 172.21.15.2:0/2508320225 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ea9817c610 0x55ea9817ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:04.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.188+0000 7f33ee831740 4 client.17031 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:13:04.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.188+0000 7f33ee831740 1 -- 172.21.15.2:0/2508320225 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea980d6b80 con 0x55ea9817c610 2024-05-14T17:13:04.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.188+0000 7f33e90b7640 1 --2- 172.21.15.2:0/2508320225 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ea9817c610 0x55ea9817ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:04.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.188+0000 7f33e90b7640 1 --2- 172.21.15.2:0/2508320225 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ea9817c610 0x55ea9817ea00 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59020ba7c68fb876 server_cookie=efc95859d86d601d in_seq=0 out_seq=0 2024-05-14T17:13:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.188+0000 7f33cd7fa640 10 client.17031 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:13:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.189+0000 7f33cd7fa640 1 -- 172.21.15.2:0/2508320225 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ea980d6b80 con 0x55ea9817c610 2024-05-14T17:13:04.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.189+0000 7f33cd7fa640 10 client.17031 handle_command_reply: tid=0 2024-05-14T17:13:04.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.189+0000 7f33ee831740 1 client.17031 shutdown 2024-05-14T17:13:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.189+0000 7f33ee831740 1 -- 172.21.15.2:0/2508320225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea98162210 msgr2=0x55ea981696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.189+0000 7f33ee831740 1 --2- 172.21.15.2:0/2508320225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea98162210 0x55ea981696b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f33e002a930 tx=0x7f33e00317a0 comp rx=0 tx=0).stop 2024-05-14T17:13:04.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.189+0000 7f33ee831740 1 -- 172.21.15.2:0/2508320225 shutdown_connections 2024-05-14T17:13:04.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.189+0000 7f33ee831740 1 --2- 172.21.15.2:0/2508320225 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ea9817c610 0x55ea9817ea00 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.189+0000 7f33ee831740 1 --2- 172.21.15.2:0/2508320225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea9816cad0 0x55ea98169bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.189+0000 7f33ee831740 1 --2- 172.21.15.2:0/2508320225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea98162210 0x55ea981696b0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:05.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.189+0000 7f33ee831740 1 --2- 172.21.15.2:0/2508320225 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ea981618f0 0x55ea9816c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.189+0000 7f33cd7fa640 0 client.17031 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:13:05.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.189+0000 7f33ee831740 1 -- 172.21.15.2:0/2508320225 >> 172.21.15.2:0/2508320225 conn(0x55ea9815de00 msgr2=0x55ea9815d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.189+0000 7f33ee831740 1 -- 172.21.15.2:0/2508320225 shutdown_connections 2024-05-14T17:13:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.189+0000 7f33ee831740 1 -- 172.21.15.2:0/2508320225 wait complete. 2024-05-14T17:13:05.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.189+0000 7f33ee831740 20 client.17031 trim_cache size 0 max 16384 2024-05-14T17:13:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.205+0000 7f33ee831740 1 Processor -- start 2024-05-14T17:13:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.205+0000 7f33ee831740 1 -- start start 2024-05-14T17:13:05.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.205+0000 7f33ee831740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ea981618f0 0x55ea98178610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:05.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.205+0000 7f33ee831740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea98162210 0x55ea9816bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:05.414 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.205+0000 7f33ee831740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea9816cad0 0x55ea981842f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.205+0000 7f33ee831740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f33e41b31f0 con 0x55ea98162210 2024-05-14T17:13:05.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.205+0000 7f33ee831740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f33e41b0ac0 con 0x55ea9816cad0 2024-05-14T17:13:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.205+0000 7f33ee831740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f33e41ae930 con 0x55ea981618f0 2024-05-14T17:13:05.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.205+0000 7f33e98b8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea98162210 0x55ea9816bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.205+0000 7f33e98b8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea98162210 0x55ea9816bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54960/0 (socket says 172.21.15.2:54960) 2024-05-14T17:13:05.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.205+0000 7f33e98b8640 1 -- 172.21.15.2:0/1529393561 learned_addr learned my addr 172.21.15.2:0/1529393561 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:13:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.205+0000 7f33e90b7640 1 --2- 172.21.15.2:0/1529393561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea9816cad0 0x55ea981842f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:05.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.205+0000 7f33e98b8640 1 -- 172.21.15.2:0/1529393561 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ea981618f0 msgr2=0x55ea98178610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.205+0000 7f33e98b8640 1 --2- 172.21.15.2:0/1529393561 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ea981618f0 0x55ea98178610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.205+0000 7f33e98b8640 1 -- 172.21.15.2:0/1529393561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea9816cad0 msgr2=0x55ea981842f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.205+0000 7f33e98b8640 1 --2- 172.21.15.2:0/1529393561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea9816cad0 0x55ea981842f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.205+0000 7f33e98b8640 1 -- 172.21.15.2:0/1529393561 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea9815dc30 con 0x55ea98162210 2024-05-14T17:13:05.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.205+0000 7f33e98b8640 1 --2- 172.21.15.2:0/1529393561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea98162210 0x55ea9816bef0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f33e002a450 tx=0x7f33e0009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76157b0f203000b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.206+0000 7f33cf7fe640 1 -- 172.21.15.2:0/1529393561 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e0052070 con 0x55ea98162210 2024-05-14T17:13:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.206+0000 7f33cf7fe640 1 -- 172.21.15.2:0/1529393561 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33e00038c0 con 0x55ea98162210 2024-05-14T17:13:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.206+0000 7f33cf7fe640 1 -- 172.21.15.2:0/1529393561 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e0035040 con 0x55ea98162210 2024-05-14T17:13:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.206+0000 7f33ee831740 1 -- 172.21.15.2:0/1529393561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea98162210 msgr2=0x55ea9816bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:06.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.206+0000 7f33ee831740 1 --2- 172.21.15.2:0/1529393561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea98162210 0x55ea9816bef0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f33e002a450 tx=0x7f33e0009230 comp rx=0 tx=0).stop 2024-05-14T17:13:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.206+0000 7f33ee831740 1 -- 172.21.15.2:0/1529393561 shutdown_connections 2024-05-14T17:13:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.206+0000 7f33ee831740 1 --2- 172.21.15.2:0/1529393561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea9816cad0 0x55ea981842f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:06.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.206+0000 7f33ee831740 1 --2- 172.21.15.2:0/1529393561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea98162210 0x55ea9816bef0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:06.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.206+0000 7f33ee831740 1 --2- 172.21.15.2:0/1529393561 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ea981618f0 0x55ea98178610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.206+0000 7f33ee831740 1 -- 172.21.15.2:0/1529393561 >> 172.21.15.2:0/1529393561 conn(0x55ea9815de00 msgr2=0x55ea98163b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:06.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.206+0000 7f33ee831740 1 -- 172.21.15.2:0/1529393561 shutdown_connections 2024-05-14T17:13:06.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.206+0000 7f33ee831740 1 -- 172.21.15.2:0/1529393561 wait complete. 2024-05-14T17:13:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.206+0000 7f33ee831740 1 -- start start 2024-05-14T17:13:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.207+0000 7f33ee831740 1 Processor -- start 2024-05-14T17:13:06.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.207+0000 7f33ee831740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea981618f0 0x55ea98174fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:06.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.207+0000 7f33ee831740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ea98162210 0x55ea98170350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.207+0000 7f33ee831740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea9816cad0 0x55ea98170960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.207+0000 7f33ee831740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f33e41aef80 con 0x55ea981618f0 2024-05-14T17:13:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.207+0000 7f33ee831740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f33e41b03f0 con 0x55ea9816cad0 2024-05-14T17:13:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.207+0000 7f33ee831740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f33e41af650 con 0x55ea98162210 2024-05-14T17:13:06.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.207+0000 7f33ee831740 10 client.0 _ll_register_callbacks cb 0x7f33e00314a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:13:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.207+0000 7f33e90b7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea981618f0 0x55ea98174fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:06.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.207+0000 7f33e90b7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea981618f0 0x55ea98174fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54972/0 (socket says 172.21.15.2:54972) 2024-05-14T17:13:06.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.207+0000 7f33e90b7640 1 -- 172.21.15.2:0/4056129050 learned_addr learned my addr 172.21.15.2:0/4056129050 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:13:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.207+0000 7f33e98b8640 1 --2- 172.21.15.2:0/4056129050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ea98162210 0x55ea98170350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.207+0000 7f33e90b7640 1 --2- 172.21.15.2:0/4056129050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea9816cad0 0x55ea98170960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.207+0000 7f33e90b7640 1 -- 172.21.15.2:0/4056129050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ea98162210 msgr2=0x55ea98170350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.207+0000 7f33e90b7640 1 --2- 172.21.15.2:0/4056129050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ea98162210 0x55ea98170350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:06.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.207+0000 7f33e90b7640 1 -- 172.21.15.2:0/4056129050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea9816cad0 msgr2=0x55ea98170960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:07.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.207+0000 7f33e90b7640 1 --2- 172.21.15.2:0/4056129050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea9816cad0 0x55ea98170960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.207+0000 7f33e90b7640 1 -- 172.21.15.2:0/4056129050 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea98171160 con 0x55ea981618f0 2024-05-14T17:13:07.075 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.208+0000 7f33e90b7640 1 --2- 172.21.15.2:0/4056129050 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea981618f0 0x55ea98174fe0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f33d4011eb0 tx=0x7f33d400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe61e947a2c3d64 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:07.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.208+0000 7f33cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:13:07.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.208+0000 7f33cd7fa640 1 -- 172.21.15.2:0/4056129050 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d402b070 con 0x55ea981618f0 2024-05-14T17:13:07.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.208+0000 7f33cd7fa640 1 -- 172.21.15.2:0/4056129050 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8557+0+0 (secure 0 0 0) 0x7f33d4004030 con 0x55ea981618f0 2024-05-14T17:13:07.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.208+0000 7f33ee831740 1 -- 172.21.15.2:0/4056129050 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33e00038c0 con 0x55ea981618f0 2024-05-14T17:13:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.208+0000 7f33cd7fa640 1 -- 172.21.15.2:0/4056129050 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f33d4005b90 con 0x55ea981618f0 2024-05-14T17:13:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.208+0000 7f33ee831740 10 client.17037 fetch_fsmap learned FSMap version 54 2024-05-14T17:13:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.208+0000 7f33ee831740 1 -- 172.21.15.2:0/4056129050 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55ea981e6fb0 con 0x55ea981618f0 2024-05-14T17:13:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.208+0000 7f33cd7fa640 1 -- 172.21.15.2:0/4056129050 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33e00038c0 con 0x55ea981618f0 2024-05-14T17:13:07.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.208+0000 7f33cd7fa640 1 -- 172.21.15.2:0/4056129050 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 54) ==== 8338+0+0 (secure 0 0 0) 0x7f33d402e020 con 0x55ea981618f0 2024-05-14T17:13:07.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.208+0000 7f33ee831740 10 client.17037 fetch_fsmap finished waiting for FSMap version 54 2024-05-14T17:13:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.208+0000 7f33ee831740 10 client.17037 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:13:07.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.208+0000 7f33ee831740 20 client.17037 populate_metadata read hostname 'smithi002' 2024-05-14T17:13:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.208+0000 7f33ee831740 1 --2- 172.21.15.2:0/4056129050 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ea981e7590 0x55ea981e9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.208+0000 7f33ee831740 4 client.17037 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:13:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.208+0000 7f33ee831740 1 -- 172.21.15.2:0/4056129050 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ea980d0360 con 0x55ea981e7590 2024-05-14T17:13:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.209+0000 7f33e98b8640 1 --2- 172.21.15.2:0/4056129050 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ea981e7590 0x55ea981e9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.209+0000 7f33e98b8640 1 --2- 172.21.15.2:0/4056129050 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ea981e7590 0x55ea981e9980 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31d9126d7d2e1295 server_cookie=7a7faae23d5b12df in_seq=0 out_seq=0 2024-05-14T17:13:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.209+0000 7f33cd7fa640 10 client.17037 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:13:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.219+0000 7f33cd7fa640 1 -- 172.21.15.2:0/4056129050 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ea980d0360 con 0x55ea981e7590 2024-05-14T17:13:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.219+0000 7f33cd7fa640 10 client.17037 handle_command_reply: tid=0 2024-05-14T17:13:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.219+0000 7f33ee831740 1 client.17037 shutdown 2024-05-14T17:13:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.220+0000 7f33ee831740 1 -- 172.21.15.2:0/4056129050 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea981618f0 msgr2=0x55ea98174fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:07.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.220+0000 7f33ee831740 1 --2- 172.21.15.2:0/4056129050 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea981618f0 0x55ea98174fe0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f33d4011eb0 tx=0x7f33d400b8a0 comp rx=0 tx=0).stop 2024-05-14T17:13:07.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.220+0000 7f33ee831740 1 -- 172.21.15.2:0/4056129050 shutdown_connections 2024-05-14T17:13:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.220+0000 7f33ee831740 1 --2- 172.21.15.2:0/4056129050 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ea981e7590 0x55ea981e9980 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.220+0000 7f33ee831740 1 --2- 172.21.15.2:0/4056129050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea9816cad0 0x55ea98170960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.220+0000 7f33ee831740 1 --2- 172.21.15.2:0/4056129050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ea98162210 0x55ea98170350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:08.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.220+0000 7f33cd7fa640 0 client.17037 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:13:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.220+0000 7f33ee831740 1 --2- 172.21.15.2:0/4056129050 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ea981618f0 0x55ea98174fe0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:08.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.220+0000 7f33ee831740 1 -- 172.21.15.2:0/4056129050 >> 172.21.15.2:0/4056129050 conn(0x55ea9815de00 msgr2=0x55ea9815c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.220+0000 7f33ee831740 1 -- 172.21.15.2:0/4056129050 shutdown_connections 2024-05-14T17:13:08.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.220+0000 7f33ee831740 1 -- 172.21.15.2:0/4056129050 wait complete. 2024-05-14T17:13:08.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.220+0000 7f33ee831740 20 client.17037 trim_cache size 0 max 16384 2024-05-14T17:13:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:13:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:13:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:13:08.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.221+0000 7f33ebb1c640 1 -- 172.21.15.2:0/1576721454 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f33d00b2a10 msgr2=0x7f33d00b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.221+0000 7f33ebb1c640 1 --2- 172.21.15.2:0/1576721454 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f33d00b2a10 0x7f33d00b4ed0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f33dc0a9bd0 tx=0x7f33dc0a4040 comp rx=0 tx=0).stop 2024-05-14T17:13:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.221+0000 7f33ebb1c640 1 -- 172.21.15.2:0/1576721454 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33e410f430 msgr2=0x7f33e419fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.221+0000 7f33ebb1c640 1 --2- 172.21.15.2:0/1576721454 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33e410f430 0x7f33e419fda0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f33dc009fd0 tx=0x7f33dc00ee60 comp rx=0 tx=0).stop 2024-05-14T17:13:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.221+0000 7f33ebb1c640 1 -- 172.21.15.2:0/1576721454 shutdown_connections 2024-05-14T17:13:08.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.221+0000 7f33ebb1c640 1 --2- 172.21.15.2:0/1576721454 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f33d00b2a10 0x7f33d00b4ed0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:08.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.221+0000 7f33ebb1c640 1 --2- 172.21.15.2:0/1576721454 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33e410f430 0x7f33e419fda0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.221+0000 7f33ebb1c640 1 --2- 172.21.15.2:0/1576721454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33e41021a0 0x7f33e419ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.221+0000 7f33ebb1c640 1 --2- 172.21.15.2:0/1576721454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33e4101880 0x7f33e419b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.221+0000 7f33ebb1c640 1 -- 172.21.15.2:0/1576721454 >> 172.21.15.2:0/1576721454 conn(0x7f33e4077cd0 msgr2=0x7f33e4110880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.221+0000 7f33ebb1c640 1 -- 172.21.15.2:0/1576721454 shutdown_connections 2024-05-14T17:13:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:10.221+0000 7f33ebb1c640 1 -- 172.21.15.2:0/1576721454 wait complete. 2024-05-14T17:13:08.894 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:12:10 smithi002 ceph-mon[31538]: pgmap v548: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 55 KiB/s wr, 27 op/s 2024-05-14T17:13:08.926 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:12:11 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:13:08.960 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:12:11 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:13:08.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:12:11 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:13:09.044 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:12:11 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:13:09.106 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:12:12 smithi002 ceph-mon[31538]: pgmap v549: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 55 KiB/s wr, 27 op/s 2024-05-14T17:13:09.135 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:12:14 smithi002 ceph-mon[31538]: pgmap v550: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 9 op/s 2024-05-14T17:13:09.168 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:12:16 smithi002 ceph-mon[31538]: pgmap v551: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 9 op/s 2024-05-14T17:13:09.201 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:12:18 smithi002 ceph-mon[31538]: osdmap e103: 12 total, 12 up, 12 in 2024-05-14T17:13:09.235 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:12:18 smithi002 ceph-mon[31538]: pgmap v553: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 7 op/s 2024-05-14T17:13:09.268 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:12:20 smithi002 ceph-mon[31538]: pgmap v554: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 9 op/s 2024-05-14T17:13:09.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:12:22 smithi002 ceph-mon[31538]: pgmap v555: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 9 op/s 2024-05-14T17:13:09.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:13:00 smithi151 ceph-mon[36066]: pgmap v574: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 5.1 MiB/s wr, 578 op/s 2024-05-14T17:13:09.401 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:13:02 smithi151 ceph-mon[36066]: pgmap v575: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 5.1 MiB/s wr, 575 op/s 2024-05-14T17:13:09.435 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:13:03 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:13:09.468 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:13:04 smithi151 ceph-mon[36066]: pgmap v576: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 5.2 MiB/s wr, 602 op/s 2024-05-14T17:13:09.502 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:13:04 smithi151 ceph-mon[36066]: osdmap e104: 12 total, 12 up, 12 in 2024-05-14T17:13:09.535 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:13:06 smithi151 ceph-mon[36066]: pgmap v578: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 4.2 MiB/s wr, 485 op/s 2024-05-14T17:13:09.568 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:13:08 smithi151 ceph-mon[36066]: pgmap v579: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 4.2 MiB/s wr, 485 op/s 2024-05-14T17:13:09.602 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:13:00 smithi123 ceph-mon[35971]: pgmap v574: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 5.1 MiB/s wr, 578 op/s 2024-05-14T17:13:09.635 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:13:02 smithi123 ceph-mon[35971]: pgmap v575: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 5.1 MiB/s wr, 575 op/s 2024-05-14T17:13:09.668 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:13:03 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:13:09.702 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:13:04 smithi123 ceph-mon[35971]: pgmap v576: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 5.2 MiB/s wr, 602 op/s 2024-05-14T17:13:09.735 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:13:04 smithi123 ceph-mon[35971]: osdmap e104: 12 total, 12 up, 12 in 2024-05-14T17:13:09.768 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:13:06 smithi123 ceph-mon[35971]: pgmap v578: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 4.2 MiB/s wr, 485 op/s 2024-05-14T17:13:09.802 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:13:08 smithi123 ceph-mon[35971]: pgmap v579: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 4.2 MiB/s wr, 485 op/s 2024-05-14T17:13:09.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.321+0000 7f83c4d46640 1 -- 172.21.15.2:0/332265466 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83c0103420 msgr2=0x7f83c010f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.321+0000 7f83c4d46640 1 --2- 172.21.15.2:0/332265466 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83c0103420 0x7f83c010f930 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f83b8009cd0 tx=0x7f83b802a8e0 comp rx=0 tx=0).stop 2024-05-14T17:13:09.919 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.322+0000 7f83c4d46640 1 -- 172.21.15.2:0/332265466 shutdown_connections 2024-05-14T17:13:09.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.322+0000 7f83c4d46640 1 --2- 172.21.15.2:0/332265466 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83c0103420 0x7f83c010f930 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.322+0000 7f83c4d46640 1 --2- 172.21.15.2:0/332265466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83c0102a60 0x7f83c0102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.322+0000 7f83c4d46640 1 --2- 172.21.15.2:0/332265466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83c0108a70 0x7f83c0108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.322+0000 7f83c4d46640 1 -- 172.21.15.2:0/332265466 >> 172.21.15.2:0/332265466 conn(0x7f83c0079ee0 msgr2=0x7f83c007a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.322+0000 7f83c4d46640 1 -- 172.21.15.2:0/332265466 shutdown_connections 2024-05-14T17:13:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.322+0000 7f83c4d46640 1 -- 172.21.15.2:0/332265466 wait complete. 2024-05-14T17:13:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.322+0000 7f83c4d46640 1 Processor -- start 2024-05-14T17:13:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.323+0000 7f83c4d46640 1 -- start start 2024-05-14T17:13:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.323+0000 7f83c4d46640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83c0102a60 0x7f83c0074d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.323+0000 7f83c4d46640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83c0103420 0x7f83c0075290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.323+0000 7f83c4d46640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83c0108a70 0x7f83c00757d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.323+0000 7f83c4d46640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f83c0112d30 con 0x7f83c0102a60 2024-05-14T17:13:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.323+0000 7f83c4d46640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f83c0112bb0 con 0x7f83c0103420 2024-05-14T17:13:10.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.323+0000 7f83c4d46640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f83c0112eb0 con 0x7f83c0108a70 2024-05-14T17:13:10.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.323+0000 7f83be59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83c0102a60 0x7f83c0074d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:10.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.323+0000 7f83be59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83c0102a60 0x7f83c0074d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47436/0 (socket says 172.21.15.2:47436) 2024-05-14T17:13:10.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.323+0000 7f83be59c640 1 -- 172.21.15.2:0/3582740760 learned_addr learned my addr 172.21.15.2:0/3582740760 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:13:10.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.323+0000 7f83b5d9b640 1 --2- 172.21.15.2:0/3582740760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83c0103420 0x7f83c0075290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.323+0000 7f83bed9d640 1 --2- 172.21.15.2:0/3582740760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83c0108a70 0x7f83c00757d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:10.636 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.323+0000 7f83be59c640 1 -- 172.21.15.2:0/3582740760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83c0108a70 msgr2=0x7f83c00757d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.323+0000 7f83be59c640 1 --2- 172.21.15.2:0/3582740760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83c0108a70 0x7f83c00757d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.323+0000 7f83be59c640 1 -- 172.21.15.2:0/3582740760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83c0103420 msgr2=0x7f83c0075290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.323+0000 7f83be59c640 1 --2- 172.21.15.2:0/3582740760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83c0103420 0x7f83c0075290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.323+0000 7f83be59c640 1 -- 172.21.15.2:0/3582740760 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c01a7690 con 0x7f83c0102a60 2024-05-14T17:13:10.803 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.324+0000 7f83be59c640 1 --2- 172.21.15.2:0/3582740760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83c0102a60 0x7f83c0074d50 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f83b000d950 tx=0x7f83b000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.324+0000 7f83b7fff640 1 -- 172.21.15.2:0/3582740760 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b0017070 con 0x7f83c0102a60 2024-05-14T17:13:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.324+0000 7f83b7fff640 1 -- 172.21.15.2:0/3582740760 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83b0009c30 con 0x7f83c0102a60 2024-05-14T17:13:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.324+0000 7f83c4d46640 1 -- 172.21.15.2:0/3582740760 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83c01a7980 con 0x7f83c0102a60 2024-05-14T17:13:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.324+0000 7f83b7fff640 1 -- 172.21.15.2:0/3582740760 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b000f620 con 0x7f83c0102a60 2024-05-14T17:13:10.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.324+0000 7f83c4d46640 1 -- 172.21.15.2:0/3582740760 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83c01a7ed0 con 0x7f83c0102a60 2024-05-14T17:13:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.327+0000 7f83b7fff640 1 -- 172.21.15.2:0/3582740760 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f83b0010460 con 0x7f83c0102a60 2024-05-14T17:13:11.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.327+0000 7f83c7a5d740 1 Processor -- start 2024-05-14T17:13:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.327+0000 7f83c7a5d740 1 -- start start 2024-05-14T17:13:11.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.327+0000 7f83c7a5d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556509d8c8f0 0x556509d8ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.327+0000 7f83c7a5d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556509d8d210 0x556509d97590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.327+0000 7f83c7a5d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556509d97ad0 0x556509d99ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.327+0000 7f83c7a5d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x556509cffc30 con 0x556509d8c8f0 2024-05-14T17:13:11.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.327+0000 7f83c7a5d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556509cef7d0 con 0x556509d8d210 2024-05-14T17:13:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.327+0000 7f83c7a5d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556509cf3810 con 0x556509d97ad0 2024-05-14T17:13:11.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.327+0000 7f83bed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556509d8c8f0 0x556509d8ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.328+0000 7f83bed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556509d8c8f0 0x556509d8ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47452/0 (socket says 172.21.15.2:47452) 2024-05-14T17:13:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.328+0000 7f83bed9d640 1 -- 172.21.15.2:0/2333246558 learned_addr learned my addr 172.21.15.2:0/2333246558 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:13:11.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.328+0000 7f83be59c640 1 --2- 172.21.15.2:0/2333246558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556509d8d210 0x556509d97590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:11.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.328+0000 7f83b5d9b640 1 --2- 172.21.15.2:0/2333246558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556509d97ad0 0x556509d99ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.328+0000 7f83bed9d640 1 -- 172.21.15.2:0/2333246558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556509d97ad0 msgr2=0x556509d99ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.328+0000 7f83bed9d640 1 --2- 172.21.15.2:0/2333246558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556509d97ad0 0x556509d99ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.328+0000 7f83bed9d640 1 -- 172.21.15.2:0/2333246558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556509d8d210 msgr2=0x556509d97590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.328+0000 7f83bed9d640 1 --2- 172.21.15.2:0/2333246558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556509d8d210 0x556509d97590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:11.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.328+0000 7f83bed9d640 1 -- 172.21.15.2:0/2333246558 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556509a04210 con 0x556509d8c8f0 2024-05-14T17:13:11.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.328+0000 7f83be59c640 1 --2- 172.21.15.2:0/2333246558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556509d8d210 0x556509d97590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:13:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.328+0000 7f83b7fff640 1 --2- 172.21.15.2:0/3582740760 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f83ac0b2a10 0x7f83ac0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.328+0000 7f83bed9d640 1 --2- 172.21.15.2:0/2333246558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556509d8c8f0 0x556509d8ccd0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f83b8009cd0 tx=0x7f83b8009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=133c11b7cc6e9c92 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.328+0000 7f83be59c640 1 --2- 172.21.15.2:0/3582740760 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f83ac0b2a10 0x7f83ac0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:11.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.328+0000 7f83b4d99640 1 -- 172.21.15.2:0/2333246558 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b8049070 con 0x556509d8c8f0 2024-05-14T17:13:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.328+0000 7f83b7fff640 1 -- 172.21.15.2:0/3582740760 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8513+0+0 (secure 0 0 0) 0x7f83b00e2150 con 0x7f83c0102a60 2024-05-14T17:13:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.328+0000 7f83b4d99640 1 -- 172.21.15.2:0/2333246558 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83b8036040 con 0x556509d8c8f0 2024-05-14T17:13:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.328+0000 7f83b4d99640 1 -- 172.21.15.2:0/2333246558 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b8003e60 con 0x556509d8c8f0 2024-05-14T17:13:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.329+0000 7f83c7a5d740 1 -- 172.21.15.2:0/2333246558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556509d8c8f0 msgr2=0x556509d8ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:11.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.329+0000 7f83c7a5d740 1 --2- 172.21.15.2:0/2333246558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556509d8c8f0 0x556509d8ccd0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f83b8009cd0 tx=0x7f83b8009a20 comp rx=0 tx=0).stop 2024-05-14T17:13:12.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.329+0000 7f83be59c640 1 --2- 172.21.15.2:0/3582740760 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f83ac0b2a10 0x7f83ac0b4ed0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f83b00df950 tx=0x7f83b0010cb0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.329+0000 7f83c7a5d740 1 -- 172.21.15.2:0/2333246558 shutdown_connections 2024-05-14T17:13:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.329+0000 7f83c7a5d740 1 --2- 172.21.15.2:0/2333246558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556509d97ad0 0x556509d99ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:12.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.329+0000 7f83c7a5d740 1 --2- 172.21.15.2:0/2333246558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556509d8d210 0x556509d97590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.329+0000 7f83c7a5d740 1 --2- 172.21.15.2:0/2333246558 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556509d8c8f0 0x556509d8ccd0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:12.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.329+0000 7f83c7a5d740 1 -- 172.21.15.2:0/2333246558 >> 172.21.15.2:0/2333246558 conn(0x556509d88e00 msgr2=0x556509d891e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:12.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.329+0000 7f83c7a5d740 1 -- 172.21.15.2:0/2333246558 shutdown_connections 2024-05-14T17:13:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.329+0000 7f83c7a5d740 1 -- 172.21.15.2:0/2333246558 wait complete. 2024-05-14T17:13:12.396 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.330+0000 7f83c7a5d740 1 -- start start 2024-05-14T17:13:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.330+0000 7f83c7a5d740 1 Processor -- start 2024-05-14T17:13:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.330+0000 7f83c7a5d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556509d8c8f0 0x556509d97440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:12.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.330+0000 7f83c7a5d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556509d8d210 0x556509d94730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.330+0000 7f83c7a5d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556509d97ad0 0x556509d94c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.330+0000 7f83c7a5d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5565099d5790 con 0x556509d8d210 2024-05-14T17:13:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.330+0000 7f83c7a5d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556509cab890 con 0x556509d97ad0 2024-05-14T17:13:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.330+0000 7f83c7a5d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556509d9a740 con 0x556509d8c8f0 2024-05-14T17:13:12.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.330+0000 7f83c7a5d740 10 client.0 _ll_register_callbacks cb 0x556509caed60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:13:12.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.330+0000 7f83b5d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556509d8d210 0x556509d94730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.330+0000 7f83b5d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556509d8d210 0x556509d94730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47468/0 (socket says 172.21.15.2:47468) 2024-05-14T17:13:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.330+0000 7f83b5d9b640 1 -- 172.21.15.2:0/3617243683 learned_addr learned my addr 172.21.15.2:0/3617243683 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:13:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.330+0000 7f83bed9d640 1 --2- 172.21.15.2:0/3617243683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556509d8c8f0 0x556509d97440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:12.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.330+0000 7f83bed9d640 1 --2- 172.21.15.2:0/3617243683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556509d97ad0 0x556509d94c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:12.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.330+0000 7f83b5d9b640 1 -- 172.21.15.2:0/3617243683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556509d8c8f0 msgr2=0x556509d97440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.330+0000 7f83b5d9b640 1 --2- 172.21.15.2:0/3617243683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556509d8c8f0 0x556509d97440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:12.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.330+0000 7f83b5d9b640 1 -- 172.21.15.2:0/3617243683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556509d97ad0 msgr2=0x556509d94c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:13.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.330+0000 7f83b5d9b640 1 --2- 172.21.15.2:0/3617243683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556509d97ad0 0x556509d94c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.330+0000 7f83b5d9b640 1 -- 172.21.15.2:0/3617243683 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556509ad0620 con 0x556509d8d210 2024-05-14T17:13:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.330+0000 7f83bed9d640 1 --2- 172.21.15.2:0/3617243683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556509d97ad0 0x556509d94c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:13:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.331+0000 7f83b5d9b640 1 --2- 172.21.15.2:0/3617243683 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556509d8d210 0x556509d94730 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f83a800bd70 tx=0x7f83a80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d356b794f705b957 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:13.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.331+0000 7f83a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:13:13.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.331+0000 7f83a5ffb640 1 -- 172.21.15.2:0/3617243683 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a8004220 con 0x556509d8d210 2024-05-14T17:13:13.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.331+0000 7f83a5ffb640 1 -- 172.21.15.2:0/3617243683 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8513+0+0 (secure 0 0 0) 0x7f83a80043c0 con 0x556509d8d210 2024-05-14T17:13:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.331+0000 7f83c7a5d740 1 -- 172.21.15.2:0/3617243683 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83b8036040 con 0x556509d8d210 2024-05-14T17:13:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.331+0000 7f83a5ffb640 1 -- 172.21.15.2:0/3617243683 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f83a8012660 con 0x556509d8d210 2024-05-14T17:13:13.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.331+0000 7f83c7a5d740 10 client.17049 fetch_fsmap learned FSMap version 54 2024-05-14T17:13:13.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.331+0000 7f83c7a5d740 1 -- 172.21.15.2:0/3617243683 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x556509da6130 con 0x556509d8d210 2024-05-14T17:13:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.331+0000 7f83a5ffb640 1 -- 172.21.15.2:0/3617243683 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83b8036040 con 0x556509d8d210 2024-05-14T17:13:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.331+0000 7f83a5ffb640 1 -- 172.21.15.2:0/3617243683 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 54) ==== 8338+0+0 (secure 0 0 0) 0x7f83a8003a80 con 0x556509d8d210 2024-05-14T17:13:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.331+0000 7f83c7a5d740 10 client.17049 fetch_fsmap finished waiting for FSMap version 54 2024-05-14T17:13:13.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.331+0000 7f83c7a5d740 10 client.17049 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:13:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.331+0000 7f83c7a5d740 20 client.17049 populate_metadata read hostname 'smithi002' 2024-05-14T17:13:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.331+0000 7f83c7a5d740 1 --2- 172.21.15.2:0/3617243683 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x556509da7640 0x556509da9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.331+0000 7f83c7a5d740 4 client.17049 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:13:13.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.331+0000 7f83c7a5d740 1 -- 172.21.15.2:0/3617243683 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556509d01b80 con 0x556509da7640 2024-05-14T17:13:13.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.332+0000 7f83bed9d640 1 --2- 172.21.15.2:0/3617243683 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x556509da7640 0x556509da9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:13.715 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.332+0000 7f83a5ffb640 10 client.17049 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:13:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.332+0000 7f83bed9d640 1 --2- 172.21.15.2:0/3617243683 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x556509da7640 0x556509da9a30 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=533f2286deec6c03 server_cookie=316e2921a8e7906 in_seq=0 out_seq=0 2024-05-14T17:13:13.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.333+0000 7f83a5ffb640 1 -- 172.21.15.2:0/3617243683 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556509d01b80 con 0x556509da7640 2024-05-14T17:13:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.333+0000 7f83a5ffb640 10 client.17049 handle_command_reply: tid=0 2024-05-14T17:13:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.333+0000 7f83c7a5d740 1 client.17049 shutdown 2024-05-14T17:13:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.333+0000 7f83c7a5d740 1 -- 172.21.15.2:0/3617243683 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556509d8d210 msgr2=0x556509d94730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:13.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.333+0000 7f83c7a5d740 1 --2- 172.21.15.2:0/3617243683 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556509d8d210 0x556509d94730 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f83a800bd70 tx=0x7f83a80117b0 comp rx=0 tx=0).stop 2024-05-14T17:13:13.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.333+0000 7f83c7a5d740 1 -- 172.21.15.2:0/3617243683 shutdown_connections 2024-05-14T17:13:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.333+0000 7f83c7a5d740 1 --2- 172.21.15.2:0/3617243683 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x556509da7640 0x556509da9a30 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.333+0000 7f83c7a5d740 1 --2- 172.21.15.2:0/3617243683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556509d97ad0 0x556509d94c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.333+0000 7f83a5ffb640 0 client.17049 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:13:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.333+0000 7f83c7a5d740 1 --2- 172.21.15.2:0/3617243683 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556509d8d210 0x556509d94730 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.333+0000 7f83c7a5d740 1 --2- 172.21.15.2:0/3617243683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556509d8c8f0 0x556509d97440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:14.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.333+0000 7f83c7a5d740 1 -- 172.21.15.2:0/3617243683 >> 172.21.15.2:0/3617243683 conn(0x556509d88e00 msgr2=0x556509d88100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.333+0000 7f83c7a5d740 1 -- 172.21.15.2:0/3617243683 shutdown_connections 2024-05-14T17:13:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.333+0000 7f83c7a5d740 1 -- 172.21.15.2:0/3617243683 wait complete. 2024-05-14T17:13:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.333+0000 7f83c7a5d740 20 client.17049 trim_cache size 0 max 16384 2024-05-14T17:13:14.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.348+0000 7f83c7a5d740 1 Processor -- start 2024-05-14T17:13:14.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.349+0000 7f83c7a5d740 1 -- start start 2024-05-14T17:13:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.349+0000 7f83c7a5d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556509d8c8f0 0x556509da3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:14.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.349+0000 7f83c7a5d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556509d8d210 0x556509d96f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:14.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.349+0000 7f83c7a5d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556509d97ad0 0x556509daf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:14.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.349+0000 7f83c7a5d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f83c01b3110 con 0x556509d97ad0 2024-05-14T17:13:14.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.349+0000 7f83c7a5d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f83c01b09e0 con 0x556509d8d210 2024-05-14T17:13:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.349+0000 7f83c7a5d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f83c01ae850 con 0x556509d8c8f0 2024-05-14T17:13:14.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.349+0000 7f83bed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556509d97ad0 0x556509daf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:14.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.349+0000 7f83bed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556509d97ad0 0x556509daf300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47480/0 (socket says 172.21.15.2:47480) 2024-05-14T17:13:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.349+0000 7f83bed9d640 1 -- 172.21.15.2:0/3282949205 learned_addr learned my addr 172.21.15.2:0/3282949205 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:13:14.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.349+0000 7f83bed9d640 1 --2- 172.21.15.2:0/3282949205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556509d8c8f0 0x556509da3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.349+0000 7f83bed9d640 1 -- 172.21.15.2:0/3282949205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556509d8c8f0 msgr2=0x556509da3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:14.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.349+0000 7f83bed9d640 1 --2- 172.21.15.2:0/3282949205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556509d8c8f0 0x556509da3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:14.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.349+0000 7f83bed9d640 1 -- 172.21.15.2:0/3282949205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556509d8d210 msgr2=0x556509d96f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:14.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.349+0000 7f83bed9d640 1 --2- 172.21.15.2:0/3282949205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556509d8d210 0x556509d96f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.349+0000 7f83bed9d640 1 -- 172.21.15.2:0/3282949205 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556509d95000 con 0x556509d97ad0 2024-05-14T17:13:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.349+0000 7f83bed9d640 1 --2- 172.21.15.2:0/3282949205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556509d97ad0 0x556509daf300 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f83b8047040 tx=0x7f83b802a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a2be9d22959c1da server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.349+0000 7f83a7fff640 1 -- 172.21.15.2:0/3282949205 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b8049040 con 0x556509d97ad0 2024-05-14T17:13:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.350+0000 7f83a7fff640 1 -- 172.21.15.2:0/3282949205 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83b8036070 con 0x556509d97ad0 2024-05-14T17:13:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.350+0000 7f83a7fff640 1 -- 172.21.15.2:0/3282949205 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b804f8b0 con 0x556509d97ad0 2024-05-14T17:13:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.350+0000 7f83c7a5d740 1 -- 172.21.15.2:0/3282949205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556509d97ad0 msgr2=0x556509daf300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:15.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.350+0000 7f83c7a5d740 1 --2- 172.21.15.2:0/3282949205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556509d97ad0 0x556509daf300 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f83b8047040 tx=0x7f83b802a8e0 comp rx=0 tx=0).stop 2024-05-14T17:13:15.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.350+0000 7f83c7a5d740 1 -- 172.21.15.2:0/3282949205 shutdown_connections 2024-05-14T17:13:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.350+0000 7f83c7a5d740 1 --2- 172.21.15.2:0/3282949205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556509d97ad0 0x556509daf300 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:15.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.350+0000 7f83c7a5d740 1 --2- 172.21.15.2:0/3282949205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556509d8d210 0x556509d96f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.350+0000 7f83c7a5d740 1 --2- 172.21.15.2:0/3282949205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556509d8c8f0 0x556509da3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.350+0000 7f83c7a5d740 1 -- 172.21.15.2:0/3282949205 >> 172.21.15.2:0/3282949205 conn(0x556509d88e00 msgr2=0x556509d8eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.350+0000 7f83c7a5d740 1 -- 172.21.15.2:0/3282949205 shutdown_connections 2024-05-14T17:13:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.350+0000 7f83c7a5d740 1 -- 172.21.15.2:0/3282949205 wait complete. 2024-05-14T17:13:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.350+0000 7f83c7a5d740 1 -- start start 2024-05-14T17:13:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.351+0000 7f83c7a5d740 1 Processor -- start 2024-05-14T17:13:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.351+0000 7f83c7a5d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556509d8c8f0 0x556509d9b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:15.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.351+0000 7f83c7a5d740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556509d8d210 0x556509d9ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.351+0000 7f83c7a5d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556509d97ad0 0x556509d9bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.351+0000 7f83c7a5d740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f83c01aeea0 con 0x556509d8d210 2024-05-14T17:13:15.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.351+0000 7f83c7a5d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f83c01b0310 con 0x556509d8c8f0 2024-05-14T17:13:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.351+0000 7f83c7a5d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f83c01af570 con 0x556509d97ad0 2024-05-14T17:13:15.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.351+0000 7f83c7a5d740 10 client.0 _ll_register_callbacks cb 0x7f83a800b9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:13:15.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.351+0000 7f83b5d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556509d8d210 0x556509d9ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.351+0000 7f83b5d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556509d8d210 0x556509d9ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47490/0 (socket says 172.21.15.2:47490) 2024-05-14T17:13:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.351+0000 7f83b5d9b640 1 -- 172.21.15.2:0/3659887743 learned_addr learned my addr 172.21.15.2:0/3659887743 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:13:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.351+0000 7f83bed9d640 1 --2- 172.21.15.2:0/3659887743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556509d8c8f0 0x556509d9b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:15.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.351+0000 7f83bed9d640 1 --2- 172.21.15.2:0/3659887743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556509d97ad0 0x556509d9bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.351+0000 7f83b5d9b640 1 -- 172.21.15.2:0/3659887743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556509d97ad0 msgr2=0x556509d9bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:15.991 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.351+0000 7f83b5d9b640 1 --2- 172.21.15.2:0/3659887743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556509d97ad0 0x556509d9bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.351+0000 7f83b5d9b640 1 -- 172.21.15.2:0/3659887743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556509d8c8f0 msgr2=0x556509d9b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.351+0000 7f83b5d9b640 1 --2- 172.21.15.2:0/3659887743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556509d8c8f0 0x556509d9b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.351+0000 7f83b5d9b640 1 -- 172.21.15.2:0/3659887743 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556509e10ee0 con 0x556509d8d210 2024-05-14T17:13:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.351+0000 7f83bed9d640 1 --2- 172.21.15.2:0/3659887743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556509d97ad0 0x556509d9bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:13:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.351+0000 7f83b5d9b640 1 --2- 172.21.15.2:0/3659887743 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556509d8d210 0x556509d9ba90 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f83a8006010 tx=0x7f83a80042b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7faddc42dffb07e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.351+0000 7f83a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:13:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.352+0000 7f83a5ffb640 1 -- 172.21.15.2:0/3659887743 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a8029070 con 0x556509d8d210 2024-05-14T17:13:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.352+0000 7f83a5ffb640 1 -- 172.21.15.2:0/3659887743 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8513+0+0 (secure 0 0 0) 0x7f83a8005450 con 0x556509d8d210 2024-05-14T17:13:16.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.352+0000 7f83c7a5d740 1 -- 172.21.15.2:0/3659887743 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83b8036070 con 0x556509d8d210 2024-05-14T17:13:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.352+0000 7f83a5ffb640 1 -- 172.21.15.2:0/3659887743 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f83a80044a0 con 0x556509d8d210 2024-05-14T17:13:16.416 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.352+0000 7f83c7a5d740 10 client.17055 fetch_fsmap learned FSMap version 54 2024-05-14T17:13:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.352+0000 7f83c7a5d740 1 -- 172.21.15.2:0/3659887743 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x556509e121b0 con 0x556509d8d210 2024-05-14T17:13:16.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.352+0000 7f83a5ffb640 1 -- 172.21.15.2:0/3659887743 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83b8036070 con 0x556509d8d210 2024-05-14T17:13:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.352+0000 7f83a5ffb640 1 -- 172.21.15.2:0/3659887743 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 54) ==== 8338+0+0 (secure 0 0 0) 0x7f83a8036030 con 0x556509d8d210 2024-05-14T17:13:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.352+0000 7f83c7a5d740 10 client.17055 fetch_fsmap finished waiting for FSMap version 54 2024-05-14T17:13:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.352+0000 7f83c7a5d740 10 client.17055 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:13:16.625 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.352+0000 7f83c7a5d740 20 client.17055 populate_metadata read hostname 'smithi002' 2024-05-14T17:13:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.352+0000 7f83c7a5d740 1 --2- 172.21.15.2:0/3659887743 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x556509e12950 0x556509e14d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.352+0000 7f83c7a5d740 4 client.17055 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:13:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.352+0000 7f83c7a5d740 1 -- 172.21.15.2:0/3659887743 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556509cfb360 con 0x556509e12950 2024-05-14T17:13:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.352+0000 7f83bed9d640 1 --2- 172.21.15.2:0/3659887743 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x556509e12950 0x556509e14d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.353+0000 7f83bed9d640 1 --2- 172.21.15.2:0/3659887743 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x556509e12950 0x556509e14d70 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38f4754b0eb00666 server_cookie=e7af94f952003ac3 in_seq=0 out_seq=0 2024-05-14T17:13:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.353+0000 7f83a5ffb640 10 client.17055 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:13:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.365+0000 7f83a5ffb640 1 -- 172.21.15.2:0/3659887743 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556509cfb360 con 0x556509e12950 2024-05-14T17:13:16.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.365+0000 7f83a5ffb640 10 client.17055 handle_command_reply: tid=0 2024-05-14T17:13:16.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.365+0000 7f83c7a5d740 1 client.17055 shutdown 2024-05-14T17:13:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.365+0000 7f83c7a5d740 1 -- 172.21.15.2:0/3659887743 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556509d8d210 msgr2=0x556509d9ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.365+0000 7f83c7a5d740 1 --2- 172.21.15.2:0/3659887743 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556509d8d210 0x556509d9ba90 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f83a8006010 tx=0x7f83a80042b0 comp rx=0 tx=0).stop 2024-05-14T17:13:17.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.365+0000 7f83c7a5d740 1 -- 172.21.15.2:0/3659887743 shutdown_connections 2024-05-14T17:13:17.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.365+0000 7f83c7a5d740 1 --2- 172.21.15.2:0/3659887743 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x556509e12950 0x556509e14d70 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:17.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.365+0000 7f83c7a5d740 1 --2- 172.21.15.2:0/3659887743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556509d97ad0 0x556509d9bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.365+0000 7f83a5ffb640 0 client.17055 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:13:17.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.365+0000 7f83c7a5d740 1 --2- 172.21.15.2:0/3659887743 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556509d8d210 0x556509d9ba90 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.365+0000 7f83c7a5d740 1 --2- 172.21.15.2:0/3659887743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556509d8c8f0 0x556509d9b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:17.267 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.365+0000 7f83c7a5d740 1 -- 172.21.15.2:0/3659887743 >> 172.21.15.2:0/3659887743 conn(0x556509d88e00 msgr2=0x556509d8f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:17.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.365+0000 7f83c7a5d740 1 -- 172.21.15.2:0/3659887743 shutdown_connections 2024-05-14T17:13:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.365+0000 7f83c7a5d740 1 -- 172.21.15.2:0/3659887743 wait complete. 2024-05-14T17:13:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.365+0000 7f83c7a5d740 20 client.17055 trim_cache size 0 max 16384 2024-05-14T17:13:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:13:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:13:17.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:13:17.551 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.366+0000 7f83c4d46640 1 -- 172.21.15.2:0/3582740760 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f83ac0b2a10 msgr2=0x7f83ac0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.366+0000 7f83c4d46640 1 --2- 172.21.15.2:0/3582740760 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f83ac0b2a10 0x7f83ac0b4ed0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f83b00df950 tx=0x7f83b0010cb0 comp rx=0 tx=0).stop 2024-05-14T17:13:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.366+0000 7f83c4d46640 1 -- 172.21.15.2:0/3582740760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83c0102a60 msgr2=0x7f83c0074d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:17.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.366+0000 7f83c4d46640 1 --2- 172.21.15.2:0/3582740760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83c0102a60 0x7f83c0074d50 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f83b000d950 tx=0x7f83b000de10 comp rx=0 tx=0).stop 2024-05-14T17:13:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.366+0000 7f83c4d46640 1 -- 172.21.15.2:0/3582740760 shutdown_connections 2024-05-14T17:13:17.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.366+0000 7f83c4d46640 1 --2- 172.21.15.2:0/3582740760 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f83ac0b2a10 0x7f83ac0b4ed0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.366+0000 7f83c4d46640 1 --2- 172.21.15.2:0/3582740760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83c0108a70 0x7f83c00757d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.366+0000 7f83c4d46640 1 --2- 172.21.15.2:0/3582740760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83c0103420 0x7f83c0075290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:17.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.366+0000 7f83c4d46640 1 --2- 172.21.15.2:0/3582740760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f83c0102a60 0x7f83c0074d50 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.366+0000 7f83c4d46640 1 -- 172.21.15.2:0/3582740760 >> 172.21.15.2:0/3582740760 conn(0x7f83c0079ee0 msgr2=0x7f83c01002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:17.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.366+0000 7f83c4d46640 1 -- 172.21.15.2:0/3582740760 shutdown_connections 2024-05-14T17:13:17.943 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:23.366+0000 7f83c4d46640 1 -- 172.21.15.2:0/3582740760 wait complete. 2024-05-14T17:13:17.977 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:12:24 smithi002 ceph-mon[31538]: pgmap v556: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 KiB/s wr, 7 op/s 2024-05-14T17:13:18.010 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:12:26 smithi002 ceph-mon[31538]: pgmap v557: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 KiB/s wr, 7 op/s 2024-05-14T17:13:18.047 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:12:28 smithi002 ceph-mon[31538]: pgmap v558: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.6 KiB/s wr, 6 op/s 2024-05-14T17:13:18.093 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:12:30 smithi002 ceph-mon[31538]: pgmap v559: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 11 op/s 2024-05-14T17:13:18.126 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:12:32 smithi002 ceph-mon[31538]: pgmap v560: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-05-14T17:13:18.160 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:12:34 smithi002 ceph-mon[31538]: pgmap v561: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-14T17:13:18.196 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:13:10 smithi123 ceph-mon[35971]: pgmap v580: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 116 KiB/s wr, 42 op/s 2024-05-14T17:13:18.227 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:13:11 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:13:18.260 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:13:12 smithi123 ceph-mon[35971]: pgmap v581: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 116 KiB/s wr, 42 op/s 2024-05-14T17:13:18.293 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:13:12 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:13:18.327 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:13:12 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:13:18.360 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:13:12 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:13:18.393 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:13:12 smithi123 ceph-mon[35971]: osdmap e105: 12 total, 12 up, 12 in 2024-05-14T17:13:18.427 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:13:14 smithi123 ceph-mon[35971]: pgmap v583: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 11 op/s 2024-05-14T17:13:18.460 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:13:16 smithi123 ceph-mon[35971]: pgmap v584: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-05-14T17:13:18.494 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:13:10 smithi151 ceph-mon[36066]: pgmap v580: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 116 KiB/s wr, 42 op/s 2024-05-14T17:13:18.527 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:13:11 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:13:18.560 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:13:12 smithi151 ceph-mon[36066]: pgmap v581: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 116 KiB/s wr, 42 op/s 2024-05-14T17:13:18.668 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:13:12 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:13:18.727 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:13:12 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:13:18.760 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:13:12 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:13:18.794 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:13:12 smithi151 ceph-mon[36066]: osdmap e105: 12 total, 12 up, 12 in 2024-05-14T17:13:18.827 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:13:14 smithi151 ceph-mon[36066]: pgmap v583: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 11 op/s 2024-05-14T17:13:18.860 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:13:16 smithi151 ceph-mon[36066]: pgmap v584: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-05-14T17:13:18.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.468+0000 7f1d6a904640 1 -- 172.21.15.2:0/3636428572 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d6410d040 msgr2=0x7f1d6410f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.468+0000 7f1d6a904640 1 --2- 172.21.15.2:0/3636428572 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d6410d040 0x7f1d6410f4b0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c009ca0 tx=0x7f1d5c02a950 comp rx=0 tx=0).stop 2024-05-14T17:13:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.468+0000 7f1d6a904640 1 -- 172.21.15.2:0/3636428572 shutdown_connections 2024-05-14T17:13:19.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.468+0000 7f1d6a904640 1 --2- 172.21.15.2:0/3636428572 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d6410d040 0x7f1d6410f4b0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.468+0000 7f1d6a904640 1 --2- 172.21.15.2:0/3636428572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d64104620 0x7f1d6410cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.468+0000 7f1d6a904640 1 --2- 172.21.15.2:0/3636428572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d64103d00 0x7f1d641040e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.468+0000 7f1d6a904640 1 -- 172.21.15.2:0/3636428572 >> 172.21.15.2:0/3636428572 conn(0x7f1d64078ad0 msgr2=0x7f1d64078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.468+0000 7f1d6a904640 1 -- 172.21.15.2:0/3636428572 shutdown_connections 2024-05-14T17:13:19.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.468+0000 7f1d6a904640 1 -- 172.21.15.2:0/3636428572 wait complete. 2024-05-14T17:13:19.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.468+0000 7f1d6a904640 1 Processor -- start 2024-05-14T17:13:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.469+0000 7f1d6a904640 1 -- start start 2024-05-14T17:13:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.469+0000 7f1d6a904640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d64103d00 0x7f1d64101950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:19.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.469+0000 7f1d6a904640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d64104620 0x7f1d64101e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:19.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.469+0000 7f1d6a904640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d6410d040 0x7f1d641023d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:19.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.469+0000 7f1d6a904640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1d64112d10 con 0x7f1d64103d00 2024-05-14T17:13:19.452 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.469+0000 7f1d6a904640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1d64112b90 con 0x7f1d6410d040 2024-05-14T17:13:19.486 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.469+0000 7f1d6a904640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1d64112e90 con 0x7f1d64104620 2024-05-14T17:13:19.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.469+0000 7f1d63fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d64103d00 0x7f1d64101950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.469+0000 7f1d63fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d64103d00 0x7f1d64101950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58294/0 (socket says 172.21.15.2:58294) 2024-05-14T17:13:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.469+0000 7f1d63fff640 1 -- 172.21.15.2:0/2736052961 learned_addr learned my addr 172.21.15.2:0/2736052961 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:13:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.469+0000 7f1d68ea1640 1 --2- 172.21.15.2:0/2736052961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d6410d040 0x7f1d641023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.469+0000 7f1d637fe640 1 --2- 172.21.15.2:0/2736052961 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d64104620 0x7f1d64101e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.469+0000 7f1d63fff640 1 -- 172.21.15.2:0/2736052961 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d64104620 msgr2=0x7f1d64101e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.469+0000 7f1d63fff640 1 --2- 172.21.15.2:0/2736052961 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d64104620 0x7f1d64101e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.469+0000 7f1d63fff640 1 -- 172.21.15.2:0/2736052961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d6410d040 msgr2=0x7f1d641023d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.469+0000 7f1d63fff640 1 --2- 172.21.15.2:0/2736052961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d6410d040 0x7f1d641023d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.469+0000 7f1d63fff640 1 -- 172.21.15.2:0/2736052961 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d641a7610 con 0x7f1d64103d00 2024-05-14T17:13:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.469+0000 7f1d637fe640 1 --2- 172.21.15.2:0/2736052961 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d64104620 0x7f1d64101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:13:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.470+0000 7f1d63fff640 1 --2- 172.21.15.2:0/2736052961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d64103d00 0x7f1d64101950 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f1d5400d770 tx=0x7f1d5400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.470+0000 7f1d617fa640 1 -- 172.21.15.2:0/2736052961 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d54017070 con 0x7f1d64103d00 2024-05-14T17:13:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.470+0000 7f1d617fa640 1 -- 172.21.15.2:0/2736052961 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d540054d0 con 0x7f1d64103d00 2024-05-14T17:13:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.470+0000 7f1d6a904640 1 -- 172.21.15.2:0/2736052961 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d641a7900 con 0x7f1d64103d00 2024-05-14T17:13:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.470+0000 7f1d617fa640 1 -- 172.21.15.2:0/2736052961 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d54004c30 con 0x7f1d64103d00 2024-05-14T17:13:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.470+0000 7f1d6a904640 1 -- 172.21.15.2:0/2736052961 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d641a7e50 con 0x7f1d64103d00 2024-05-14T17:13:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.473+0000 7f1d6d617740 1 Processor -- start 2024-05-14T17:13:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.473+0000 7f1d617fa640 1 -- 172.21.15.2:0/2736052961 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f1d54004dd0 con 0x7f1d64103d00 2024-05-14T17:13:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.473+0000 7f1d6d617740 1 -- start start 2024-05-14T17:13:20.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.473+0000 7f1d6d617740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b027298f0 0x561b02729cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.473+0000 7f1d6d617740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b0272a210 0x561b02734590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:20.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.473+0000 7f1d6d617740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b02734ad0 0x561b02736ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.473+0000 7f1d6d617740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x561b0269cc30 con 0x561b02734ad0 2024-05-14T17:13:20.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.473+0000 7f1d6d617740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561b0268c7d0 con 0x561b0272a210 2024-05-14T17:13:20.395 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.473+0000 7f1d6d617740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561b02690810 con 0x561b027298f0 2024-05-14T17:13:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.473+0000 7f1d637fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b02734ad0 0x561b02736ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.473+0000 7f1d637fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b02734ad0 0x561b02736ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58296/0 (socket says 172.21.15.2:58296) 2024-05-14T17:13:20.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.473+0000 7f1d637fe640 1 -- 172.21.15.2:0/601754905 learned_addr learned my addr 172.21.15.2:0/601754905 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:13:20.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.473+0000 7f1d68ea1640 1 --2- 172.21.15.2:0/601754905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b027298f0 0x561b02729cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:20.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.473+0000 7f1d63fff640 1 --2- 172.21.15.2:0/601754905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b0272a210 0x561b02734590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.474+0000 7f1d637fe640 1 -- 172.21.15.2:0/601754905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b027298f0 msgr2=0x561b02729cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:20.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.474+0000 7f1d637fe640 1 --2- 172.21.15.2:0/601754905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b027298f0 0x561b02729cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:20.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.474+0000 7f1d637fe640 1 -- 172.21.15.2:0/601754905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b0272a210 msgr2=0x561b02734590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:20.729 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.474+0000 7f1d637fe640 1 --2- 172.21.15.2:0/601754905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b0272a210 0x561b02734590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.474+0000 7f1d637fe640 1 -- 172.21.15.2:0/601754905 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b023a1210 con 0x561b02734ad0 2024-05-14T17:13:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.474+0000 7f1d68ea1640 1 --2- 172.21.15.2:0/601754905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b027298f0 0x561b02729cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:13:20.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.474+0000 7f1d63fff640 1 --2- 172.21.15.2:0/601754905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b0272a210 0x561b02734590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:13:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.474+0000 7f1d617fa640 1 --2- 172.21.15.2:0/2736052961 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f1d4c0b2ac0 0x7f1d4c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.474+0000 7f1d637fe640 1 --2- 172.21.15.2:0/601754905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b02734ad0 0x561b02736ec0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f1d58009c50 tx=0x7f1d58010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc19719da48223ee server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.474+0000 7f1d68ea1640 1 --2- 172.21.15.2:0/2736052961 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f1d4c0b2ac0 0x7f1d4c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:20.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.474+0000 7f1d617fa640 1 -- 172.21.15.2:0/2736052961 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8513+0+0 (secure 0 0 0) 0x7f1d540e2110 con 0x7f1d64103d00 2024-05-14T17:13:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.474+0000 7f1d4a7fc640 1 -- 172.21.15.2:0/601754905 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58004030 con 0x561b02734ad0 2024-05-14T17:13:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.474+0000 7f1d4a7fc640 1 -- 172.21.15.2:0/601754905 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d580041d0 con 0x561b02734ad0 2024-05-14T17:13:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.474+0000 7f1d4a7fc640 1 -- 172.21.15.2:0/601754905 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d580126b0 con 0x561b02734ad0 2024-05-14T17:13:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.475+0000 7f1d6d617740 1 -- 172.21.15.2:0/601754905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b02734ad0 msgr2=0x561b02736ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:21.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.475+0000 7f1d68ea1640 1 --2- 172.21.15.2:0/2736052961 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f1d4c0b2ac0 0x7f1d4c0b4f80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c004af0 tx=0x7f1d5c004a40 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:21.196 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.475+0000 7f1d6d617740 1 --2- 172.21.15.2:0/601754905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b02734ad0 0x561b02736ec0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f1d58009c50 tx=0x7f1d58010740 comp rx=0 tx=0).stop 2024-05-14T17:13:21.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.475+0000 7f1d6d617740 1 -- 172.21.15.2:0/601754905 shutdown_connections 2024-05-14T17:13:21.263 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.475+0000 7f1d6d617740 1 --2- 172.21.15.2:0/601754905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b02734ad0 0x561b02736ec0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:21.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.475+0000 7f1d6d617740 1 --2- 172.21.15.2:0/601754905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b0272a210 0x561b02734590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.475+0000 7f1d6d617740 1 --2- 172.21.15.2:0/601754905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b027298f0 0x561b02729cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.475+0000 7f1d6d617740 1 -- 172.21.15.2:0/601754905 >> 172.21.15.2:0/601754905 conn(0x561b02725e00 msgr2=0x561b027261e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.475+0000 7f1d6d617740 1 -- 172.21.15.2:0/601754905 shutdown_connections 2024-05-14T17:13:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.475+0000 7f1d6d617740 1 -- 172.21.15.2:0/601754905 wait complete. 2024-05-14T17:13:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.475+0000 7f1d6d617740 1 -- start start 2024-05-14T17:13:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.475+0000 7f1d6d617740 1 Processor -- start 2024-05-14T17:13:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.476+0000 7f1d6d617740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b027298f0 0x561b02734420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.476+0000 7f1d6d617740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b0272a210 0x561b02731710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:21.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.476+0000 7f1d6d617740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b02734ad0 0x561b02731c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.476+0000 7f1d6d617740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x561b02372790 con 0x561b0272a210 2024-05-14T17:13:21.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.476+0000 7f1d6d617740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561b02648890 con 0x561b02734ad0 2024-05-14T17:13:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.476+0000 7f1d6d617740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561b027377a0 con 0x561b027298f0 2024-05-14T17:13:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.476+0000 7f1d6d617740 10 client.0 _ll_register_callbacks cb 0x561b0264bd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:13:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.476+0000 7f1d637fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b0272a210 0x561b02731710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.476+0000 7f1d637fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b0272a210 0x561b02731710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58312/0 (socket says 172.21.15.2:58312) 2024-05-14T17:13:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.476+0000 7f1d637fe640 1 -- 172.21.15.2:0/580975644 learned_addr learned my addr 172.21.15.2:0/580975644 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:13:21.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.476+0000 7f1d68ea1640 1 --2- 172.21.15.2:0/580975644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b02734ad0 0x561b02731c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:21.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.476+0000 7f1d63fff640 1 --2- 172.21.15.2:0/580975644 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b027298f0 0x561b02734420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:21.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.476+0000 7f1d637fe640 1 -- 172.21.15.2:0/580975644 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b027298f0 msgr2=0x561b02734420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.476+0000 7f1d637fe640 1 --2- 172.21.15.2:0/580975644 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b027298f0 0x561b02734420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:22.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.476+0000 7f1d637fe640 1 -- 172.21.15.2:0/580975644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b02734ad0 msgr2=0x561b02731c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:22.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.476+0000 7f1d637fe640 1 --2- 172.21.15.2:0/580975644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b02734ad0 0x561b02731c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.476+0000 7f1d637fe640 1 -- 172.21.15.2:0/580975644 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b0246d620 con 0x561b0272a210 2024-05-14T17:13:22.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.476+0000 7f1d68ea1640 1 --2- 172.21.15.2:0/580975644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b02734ad0 0x561b02731c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:13:22.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.477+0000 7f1d637fe640 1 --2- 172.21.15.2:0/580975644 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b0272a210 0x561b02731710 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f1d58003d70 tx=0x7f1d58005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c47769c265a63217 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:22.230 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.477+0000 7f1d437fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:13:22.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.477+0000 7f1d437fe640 1 -- 172.21.15.2:0/580975644 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58004380 con 0x561b0272a210 2024-05-14T17:13:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.477+0000 7f1d437fe640 1 -- 172.21.15.2:0/580975644 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8513+0+0 (secure 0 0 0) 0x7f1d58009d40 con 0x561b0272a210 2024-05-14T17:13:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.477+0000 7f1d6d617740 1 -- 172.21.15.2:0/580975644 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d580041d0 con 0x561b0272a210 2024-05-14T17:13:22.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.477+0000 7f1d437fe640 1 -- 172.21.15.2:0/580975644 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f1d58005050 con 0x561b0272a210 2024-05-14T17:13:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.477+0000 7f1d6d617740 10 client.17067 fetch_fsmap learned FSMap version 54 2024-05-14T17:13:22.514 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.477+0000 7f1d6d617740 1 -- 172.21.15.2:0/580975644 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x561b027431c0 con 0x561b0272a210 2024-05-14T17:13:22.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.477+0000 7f1d437fe640 1 -- 172.21.15.2:0/580975644 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d580041d0 con 0x561b0272a210 2024-05-14T17:13:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.477+0000 7f1d437fe640 1 -- 172.21.15.2:0/580975644 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 54) ==== 8338+0+0 (secure 0 0 0) 0x7f1d5800f040 con 0x561b0272a210 2024-05-14T17:13:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.477+0000 7f1d6d617740 10 client.17067 fetch_fsmap finished waiting for FSMap version 54 2024-05-14T17:13:22.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.477+0000 7f1d6d617740 10 client.17067 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:13:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.477+0000 7f1d6d617740 20 client.17067 populate_metadata read hostname 'smithi002' 2024-05-14T17:13:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.477+0000 7f1d6d617740 1 --2- 172.21.15.2:0/580975644 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x561b02744620 0x561b02746a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:22.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.477+0000 7f1d6d617740 4 client.17067 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:13:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.477+0000 7f1d6d617740 1 -- 172.21.15.2:0/580975644 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b0269eb80 con 0x561b02744620 2024-05-14T17:13:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.478+0000 7f1d63fff640 1 --2- 172.21.15.2:0/580975644 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x561b02744620 0x561b02746a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:22.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.478+0000 7f1d63fff640 1 --2- 172.21.15.2:0/580975644 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x561b02744620 0x561b02746a10 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ba3d2a19224894e server_cookie=c4d15b3e0638744c in_seq=0 out_seq=0 2024-05-14T17:13:22.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.478+0000 7f1d437fe640 10 client.17067 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:13:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.479+0000 7f1d437fe640 1 -- 172.21.15.2:0/580975644 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561b0269eb80 con 0x561b02744620 2024-05-14T17:13:22.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.479+0000 7f1d437fe640 10 client.17067 handle_command_reply: tid=0 2024-05-14T17:13:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.479+0000 7f1d6d617740 1 client.17067 shutdown 2024-05-14T17:13:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.479+0000 7f1d6d617740 1 -- 172.21.15.2:0/580975644 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b0272a210 msgr2=0x561b02731710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.479+0000 7f1d6d617740 1 --2- 172.21.15.2:0/580975644 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b0272a210 0x561b02731710 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f1d58003d70 tx=0x7f1d58005d90 comp rx=0 tx=0).stop 2024-05-14T17:13:23.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.479+0000 7f1d6d617740 1 -- 172.21.15.2:0/580975644 shutdown_connections 2024-05-14T17:13:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.479+0000 7f1d6d617740 1 --2- 172.21.15.2:0/580975644 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x561b02744620 0x561b02746a10 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.479+0000 7f1d6d617740 1 --2- 172.21.15.2:0/580975644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b02734ad0 0x561b02731c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.479+0000 7f1d6d617740 1 --2- 172.21.15.2:0/580975644 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b0272a210 0x561b02731710 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:23.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.479+0000 7f1d6d617740 1 --2- 172.21.15.2:0/580975644 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b027298f0 0x561b02734420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.479+0000 7f1d437fe640 0 client.17067 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:13:23.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.479+0000 7f1d6d617740 1 -- 172.21.15.2:0/580975644 >> 172.21.15.2:0/580975644 conn(0x561b02725e00 msgr2=0x561b02725100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.479+0000 7f1d6d617740 1 -- 172.21.15.2:0/580975644 shutdown_connections 2024-05-14T17:13:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.479+0000 7f1d6d617740 1 -- 172.21.15.2:0/580975644 wait complete. 2024-05-14T17:13:23.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.479+0000 7f1d6d617740 20 client.17067 trim_cache size 0 max 16384 2024-05-14T17:13:23.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.495+0000 7f1d6d617740 1 Processor -- start 2024-05-14T17:13:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.495+0000 7f1d6d617740 1 -- start start 2024-05-14T17:13:23.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.495+0000 7f1d6d617740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b027298f0 0x561b02740600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.495+0000 7f1d6d617740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b0272a210 0x561b02733f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.495+0000 7f1d6d617740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b02734ad0 0x561b0273c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.495+0000 7f1d6d617740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1d641b3060 con 0x561b0272a210 2024-05-14T17:13:23.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.495+0000 7f1d6d617740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1d641b0930 con 0x561b027298f0 2024-05-14T17:13:23.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.495+0000 7f1d6d617740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1d641ae7a0 con 0x561b02734ad0 2024-05-14T17:13:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.495+0000 7f1d637fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b0272a210 0x561b02733f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:23.840 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.495+0000 7f1d637fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b0272a210 0x561b02733f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58322/0 (socket says 172.21.15.2:58322) 2024-05-14T17:13:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.495+0000 7f1d637fe640 1 -- 172.21.15.2:0/3971164848 learned_addr learned my addr 172.21.15.2:0/3971164848 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:13:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.495+0000 7f1d68ea1640 1 --2- 172.21.15.2:0/3971164848 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b02734ad0 0x561b0273c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:23.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.495+0000 7f1d63fff640 1 --2- 172.21.15.2:0/3971164848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b027298f0 0x561b02740600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.495+0000 7f1d637fe640 1 -- 172.21.15.2:0/3971164848 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b02734ad0 msgr2=0x561b0273c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.495+0000 7f1d637fe640 1 --2- 172.21.15.2:0/3971164848 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b02734ad0 0x561b0273c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:24.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.495+0000 7f1d637fe640 1 -- 172.21.15.2:0/3971164848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b027298f0 msgr2=0x561b02740600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:24.115 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.495+0000 7f1d637fe640 1 --2- 172.21.15.2:0/3971164848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b027298f0 0x561b02740600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:24.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.495+0000 7f1d637fe640 1 -- 172.21.15.2:0/3971164848 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b02720510 con 0x561b0272a210 2024-05-14T17:13:24.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.495+0000 7f1d63fff640 1 --2- 172.21.15.2:0/3971164848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b027298f0 0x561b02740600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:13:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.496+0000 7f1d637fe640 1 --2- 172.21.15.2:0/3971164848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b0272a210 0x561b02733f50 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f1d58003d40 tx=0x7f1d58009690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f95ba7d3fb682d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.496+0000 7f1d49ffb640 1 -- 172.21.15.2:0/3971164848 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58009d40 con 0x561b0272a210 2024-05-14T17:13:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.496+0000 7f1d49ffb640 1 -- 172.21.15.2:0/3971164848 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d58005a70 con 0x561b0272a210 2024-05-14T17:13:24.338 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.496+0000 7f1d49ffb640 1 -- 172.21.15.2:0/3971164848 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58004680 con 0x561b0272a210 2024-05-14T17:13:24.384 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.496+0000 7f1d6d617740 1 -- 172.21.15.2:0/3971164848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b0272a210 msgr2=0x561b02733f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.496+0000 7f1d6d617740 1 --2- 172.21.15.2:0/3971164848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b0272a210 0x561b02733f50 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f1d58003d40 tx=0x7f1d58009690 comp rx=0 tx=0).stop 2024-05-14T17:13:24.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.496+0000 7f1d6d617740 1 -- 172.21.15.2:0/3971164848 shutdown_connections 2024-05-14T17:13:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.496+0000 7f1d6d617740 1 --2- 172.21.15.2:0/3971164848 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b02734ad0 0x561b0273c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:24.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.496+0000 7f1d6d617740 1 --2- 172.21.15.2:0/3971164848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b0272a210 0x561b02733f50 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.496+0000 7f1d6d617740 1 --2- 172.21.15.2:0/3971164848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b027298f0 0x561b02740600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:24.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.496+0000 7f1d6d617740 1 -- 172.21.15.2:0/3971164848 >> 172.21.15.2:0/3971164848 conn(0x561b02725e00 msgr2=0x561b0273fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.496+0000 7f1d6d617740 1 -- 172.21.15.2:0/3971164848 shutdown_connections 2024-05-14T17:13:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.496+0000 7f1d6d617740 1 -- 172.21.15.2:0/3971164848 wait complete. 2024-05-14T17:13:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.497+0000 7f1d6d617740 1 -- start start 2024-05-14T17:13:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.497+0000 7f1d6d617740 1 Processor -- start 2024-05-14T17:13:24.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.497+0000 7f1d6d617740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b027298f0 0x561b0273ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:24.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.497+0000 7f1d6d617740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b0272a210 0x561b0273a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:24.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.497+0000 7f1d6d617740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b02734ad0 0x561b0273a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.497+0000 7f1d6d617740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1d641aedf0 con 0x561b02734ad0 2024-05-14T17:13:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.497+0000 7f1d6d617740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1d641b0260 con 0x561b027298f0 2024-05-14T17:13:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.497+0000 7f1d6d617740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1d641af4c0 con 0x561b0272a210 2024-05-14T17:13:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.497+0000 7f1d6d617740 10 client.0 _ll_register_callbacks cb 0x7f1d58003a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:13:25.033 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.497+0000 7f1d68ea1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b02734ad0 0x561b0273a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:25.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.497+0000 7f1d68ea1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b02734ad0 0x561b0273a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58330/0 (socket says 172.21.15.2:58330) 2024-05-14T17:13:25.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.497+0000 7f1d68ea1640 1 -- 172.21.15.2:0/3092824997 learned_addr learned my addr 172.21.15.2:0/3092824997 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:13:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.498+0000 7f1d63fff640 1 --2- 172.21.15.2:0/3092824997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b027298f0 0x561b0273ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:25.250 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.498+0000 7f1d637fe640 1 --2- 172.21.15.2:0/3092824997 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b0272a210 0x561b0273a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.498+0000 7f1d68ea1640 1 -- 172.21.15.2:0/3092824997 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b0272a210 msgr2=0x561b0273a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:25.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.498+0000 7f1d68ea1640 1 --2- 172.21.15.2:0/3092824997 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b0272a210 0x561b0273a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:25.358 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.498+0000 7f1d68ea1640 1 -- 172.21.15.2:0/3092824997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b027298f0 msgr2=0x561b0273ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.498+0000 7f1d68ea1640 1 --2- 172.21.15.2:0/3092824997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b027298f0 0x561b0273ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.498+0000 7f1d68ea1640 1 -- 172.21.15.2:0/3092824997 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b0273abd0 con 0x561b02734ad0 2024-05-14T17:13:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.498+0000 7f1d63fff640 1 --2- 172.21.15.2:0/3092824997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b027298f0 0x561b0273ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:13:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.498+0000 7f1d637fe640 1 --2- 172.21.15.2:0/3092824997 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b0272a210 0x561b0273a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:13:25.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.498+0000 7f1d68ea1640 1 --2- 172.21.15.2:0/3092824997 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b02734ad0 0x561b0273a610 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c009ca0 tx=0x7f1d5c02a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f764a2ca04e163 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:25.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.498+0000 7f1d437fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:13:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.498+0000 7f1d437fe640 1 -- 172.21.15.2:0/3092824997 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c034040 con 0x561b02734ad0 2024-05-14T17:13:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.498+0000 7f1d437fe640 1 -- 172.21.15.2:0/3092824997 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8513+0+0 (secure 0 0 0) 0x7f1d5c041030 con 0x561b02734ad0 2024-05-14T17:13:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.498+0000 7f1d6d617740 1 -- 172.21.15.2:0/3092824997 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d58005a70 con 0x561b02734ad0 2024-05-14T17:13:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.498+0000 7f1d437fe640 1 -- 172.21.15.2:0/3092824997 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f1d5c004040 con 0x561b02734ad0 2024-05-14T17:13:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.498+0000 7f1d6d617740 10 client.17073 fetch_fsmap learned FSMap version 54 2024-05-14T17:13:25.834 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.498+0000 7f1d6d617740 1 -- 172.21.15.2:0/3092824997 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x561b027aeb70 con 0x561b02734ad0 2024-05-14T17:13:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.498+0000 7f1d437fe640 1 -- 172.21.15.2:0/3092824997 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d58005a70 con 0x561b02734ad0 2024-05-14T17:13:25.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.499+0000 7f1d437fe640 1 -- 172.21.15.2:0/3092824997 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 54) ==== 8338+0+0 (secure 0 0 0) 0x7f1d5c030740 con 0x561b02734ad0 2024-05-14T17:13:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.499+0000 7f1d6d617740 10 client.17073 fetch_fsmap finished waiting for FSMap version 54 2024-05-14T17:13:25.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.499+0000 7f1d6d617740 10 client.17073 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:13:26.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.499+0000 7f1d6d617740 20 client.17073 populate_metadata read hostname 'smithi002' 2024-05-14T17:13:26.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.499+0000 7f1d6d617740 1 --2- 172.21.15.2:0/3092824997 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x561b027af3c0 0x561b027b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.499+0000 7f1d6d617740 4 client.17073 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:13:26.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.499+0000 7f1d6d617740 1 -- 172.21.15.2:0/3092824997 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561b02698360 con 0x561b027af3c0 2024-05-14T17:13:26.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.499+0000 7f1d63fff640 1 --2- 172.21.15.2:0/3092824997 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x561b027af3c0 0x561b027b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.499+0000 7f1d63fff640 1 --2- 172.21.15.2:0/3092824997 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x561b027af3c0 0x561b027b17e0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ef882e25ed18775 server_cookie=3063d15490f974dc in_seq=0 out_seq=0 2024-05-14T17:13:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.499+0000 7f1d437fe640 10 client.17073 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:13:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.508+0000 7f1d437fe640 1 -- 172.21.15.2:0/3092824997 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561b02698360 con 0x561b027af3c0 2024-05-14T17:13:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.508+0000 7f1d437fe640 10 client.17073 handle_command_reply: tid=0 2024-05-14T17:13:26.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.508+0000 7f1d6d617740 1 client.17073 shutdown 2024-05-14T17:13:26.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.508+0000 7f1d6d617740 1 -- 172.21.15.2:0/3092824997 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b02734ad0 msgr2=0x561b0273a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:26.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.508+0000 7f1d6d617740 1 --2- 172.21.15.2:0/3092824997 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b02734ad0 0x561b0273a610 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c009ca0 tx=0x7f1d5c02a950 comp rx=0 tx=0).stop 2024-05-14T17:13:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.508+0000 7f1d6d617740 1 -- 172.21.15.2:0/3092824997 shutdown_connections 2024-05-14T17:13:26.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.508+0000 7f1d6d617740 1 --2- 172.21.15.2:0/3092824997 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x561b027af3c0 0x561b027b17e0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:26.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.508+0000 7f1d6d617740 1 --2- 172.21.15.2:0/3092824997 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561b02734ad0 0x561b0273a610 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.508+0000 7f1d6d617740 1 --2- 172.21.15.2:0/3092824997 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561b0272a210 0x561b0273a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.508+0000 7f1d437fe640 0 client.17073 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:13:26.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.508+0000 7f1d6d617740 1 --2- 172.21.15.2:0/3092824997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b027298f0 0x561b0273ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:26.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.508+0000 7f1d6d617740 1 -- 172.21.15.2:0/3092824997 >> 172.21.15.2:0/3092824997 conn(0x561b02725e00 msgr2=0x561b02740600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:26.668 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.509+0000 7f1d6d617740 1 -- 172.21.15.2:0/3092824997 shutdown_connections 2024-05-14T17:13:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.509+0000 7f1d6d617740 1 -- 172.21.15.2:0/3092824997 wait complete. 2024-05-14T17:13:26.735 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.509+0000 7f1d6d617740 20 client.17073 trim_cache size 0 max 16384 2024-05-14T17:13:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:13:26.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:13:26.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:13:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.509+0000 7f1d6a904640 1 -- 172.21.15.2:0/2736052961 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f1d4c0b2ac0 msgr2=0x7f1d4c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.509+0000 7f1d6a904640 1 --2- 172.21.15.2:0/2736052961 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f1d4c0b2ac0 0x7f1d4c0b4f80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c004af0 tx=0x7f1d5c004a40 comp rx=0 tx=0).stop 2024-05-14T17:13:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.509+0000 7f1d6a904640 1 -- 172.21.15.2:0/2736052961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d64103d00 msgr2=0x7f1d64101950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:27.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.509+0000 7f1d6a904640 1 --2- 172.21.15.2:0/2736052961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d64103d00 0x7f1d64101950 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f1d5400d770 tx=0x7f1d5400dc30 comp rx=0 tx=0).stop 2024-05-14T17:13:27.060 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.509+0000 7f1d6a904640 1 -- 172.21.15.2:0/2736052961 shutdown_connections 2024-05-14T17:13:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.509+0000 7f1d6a904640 1 --2- 172.21.15.2:0/2736052961 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f1d4c0b2ac0 0x7f1d4c0b4f80 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.509+0000 7f1d6a904640 1 --2- 172.21.15.2:0/2736052961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d6410d040 0x7f1d641023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:27.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.509+0000 7f1d6a904640 1 --2- 172.21.15.2:0/2736052961 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d64104620 0x7f1d64101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.509+0000 7f1d6a904640 1 --2- 172.21.15.2:0/2736052961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d64103d00 0x7f1d64101950 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.509+0000 7f1d6a904640 1 -- 172.21.15.2:0/2736052961 >> 172.21.15.2:0/2736052961 conn(0x7f1d64078ad0 msgr2=0x7f1d640fe980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:27.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.509+0000 7f1d6a904640 1 -- 172.21.15.2:0/2736052961 shutdown_connections 2024-05-14T17:13:27.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:36.509+0000 7f1d6a904640 1 -- 172.21.15.2:0/2736052961 wait complete. 2024-05-14T17:13:27.328 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:12:36 smithi002 ceph-mon[31538]: pgmap v562: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-05-14T17:13:27.360 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:12:38 smithi002 ceph-mon[31538]: pgmap v563: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-05-14T17:13:27.394 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:12:40 smithi002 ceph-mon[31538]: pgmap v564: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-05-14T17:13:27.427 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:12:42 smithi002 ceph-mon[31538]: pgmap v565: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 16 KiB/s wr, 8 op/s 2024-05-14T17:13:27.460 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:12:44 smithi002 ceph-mon[31538]: pgmap v566: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-05-14T17:13:27.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:12:46 smithi002 ceph-mon[31538]: pgmap v567: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-05-14T17:13:27.527 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:12:48 smithi002 ceph-mon[31538]: pgmap v568: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-05-14T17:13:27.571 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:13:18 smithi123 ceph-mon[35971]: pgmap v585: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-05-14T17:13:27.602 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:13:20 smithi123 ceph-mon[35971]: pgmap v586: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-05-14T17:13:27.635 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:13:22 smithi123 ceph-mon[35971]: pgmap v587: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-05-14T17:13:27.669 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:13:24 smithi123 ceph-mon[35971]: pgmap v588: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 36 KiB/s wr, 14 op/s 2024-05-14T17:13:27.702 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:13:26 smithi123 ceph-mon[35971]: pgmap v589: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 40 KiB/s wr, 10 op/s 2024-05-14T17:13:27.736 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:13:18 smithi151 ceph-mon[36066]: pgmap v585: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-05-14T17:13:27.769 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:13:20 smithi151 ceph-mon[36066]: pgmap v586: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-05-14T17:13:27.802 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:13:22 smithi151 ceph-mon[36066]: pgmap v587: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-05-14T17:13:27.836 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:13:24 smithi151 ceph-mon[36066]: pgmap v588: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 36 KiB/s wr, 14 op/s 2024-05-14T17:13:27.869 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:13:26 smithi151 ceph-mon[36066]: pgmap v589: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 40 KiB/s wr, 10 op/s 2024-05-14T17:13:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.611+0000 7ff83ce74640 1 -- 172.21.15.2:0/2941042611 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff838101970 msgr2=0x7ff838101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.611+0000 7ff83ce74640 1 --2- 172.21.15.2:0/2941042611 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff838101970 0x7ff838101df0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7ff82c009930 tx=0x7ff82c02a420 comp rx=0 tx=0).stop 2024-05-14T17:13:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.611+0000 7ff83ce74640 1 -- 172.21.15.2:0/2941042611 shutdown_connections 2024-05-14T17:13:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.611+0000 7ff83ce74640 1 --2- 172.21.15.2:0/2941042611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff838102330 0x7ff83810e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:28.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.611+0000 7ff83ce74640 1 --2- 172.21.15.2:0/2941042611 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff838101970 0x7ff838101df0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.611+0000 7ff83ce74640 1 --2- 172.21.15.2:0/2941042611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff838107980 0x7ff838107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.611+0000 7ff83ce74640 1 -- 172.21.15.2:0/2941042611 >> 172.21.15.2:0/2941042611 conn(0x7ff838077cd0 msgr2=0x7ff8380780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.611+0000 7ff83ce74640 1 -- 172.21.15.2:0/2941042611 shutdown_connections 2024-05-14T17:13:28.228 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.611+0000 7ff83ce74640 1 -- 172.21.15.2:0/2941042611 wait complete. 2024-05-14T17:13:28.261 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.612+0000 7ff83ce74640 1 Processor -- start 2024-05-14T17:13:28.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.612+0000 7ff83ce74640 1 -- start start 2024-05-14T17:13:28.328 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.612+0000 7ff83ce74640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff838101970 0x7ff838198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:28.361 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.612+0000 7ff83ce74640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff838102330 0x7ff838198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:28.394 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.612+0000 7ff83ce74640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff838107980 0x7ff83819c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:28.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.612+0000 7ff83ce74640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff838111c40 con 0x7ff838101970 2024-05-14T17:13:28.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.612+0000 7ff83ce74640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff838111ac0 con 0x7ff838102330 2024-05-14T17:13:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.612+0000 7ff83ce74640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff838111dc0 con 0x7ff838107980 2024-05-14T17:13:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.612+0000 7ff83659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff838101970 0x7ff838198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.612+0000 7ff83659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff838101970 0x7ff838198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57516/0 (socket says 172.21.15.2:57516) 2024-05-14T17:13:28.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.612+0000 7ff83659c640 1 -- 172.21.15.2:0/2877406784 learned_addr learned my addr 172.21.15.2:0/2877406784 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:13:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.612+0000 7ff835d9b640 1 --2- 172.21.15.2:0/2877406784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff838102330 0x7ff838198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.612+0000 7ff836d9d640 1 --2- 172.21.15.2:0/2877406784 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff838107980 0x7ff83819c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:28.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.612+0000 7ff83659c640 1 -- 172.21.15.2:0/2877406784 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff838107980 msgr2=0x7ff83819c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.612+0000 7ff83659c640 1 --2- 172.21.15.2:0/2877406784 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff838107980 0x7ff83819c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:28.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.612+0000 7ff83659c640 1 -- 172.21.15.2:0/2877406784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff838102330 msgr2=0x7ff838198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.612+0000 7ff83659c640 1 --2- 172.21.15.2:0/2877406784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff838102330 0x7ff838198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.612+0000 7ff83659c640 1 -- 172.21.15.2:0/2877406784 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff83819d110 con 0x7ff838101970 2024-05-14T17:13:29.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.612+0000 7ff836d9d640 1 --2- 172.21.15.2:0/2877406784 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff838107980 0x7ff83819c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:13:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.613+0000 7ff83659c640 1 --2- 172.21.15.2:0/2877406784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff838101970 0x7ff838198120 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ff82800b4a0 tx=0x7ff82800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.613+0000 7ff8277fe640 1 -- 172.21.15.2:0/2877406784 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff828014070 con 0x7ff838101970 2024-05-14T17:13:29.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.613+0000 7ff8277fe640 1 -- 172.21.15.2:0/2877406784 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff828010ac0 con 0x7ff838101970 2024-05-14T17:13:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.613+0000 7ff83ce74640 1 -- 172.21.15.2:0/2877406784 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff83819d400 con 0x7ff838101970 2024-05-14T17:13:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.613+0000 7ff8277fe640 1 -- 172.21.15.2:0/2877406784 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff828017c30 con 0x7ff838101970 2024-05-14T17:13:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.613+0000 7ff83ce74640 1 -- 172.21.15.2:0/2877406784 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8381a4c90 con 0x7ff838101970 2024-05-14T17:13:29.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.615+0000 7ff8277fe640 1 -- 172.21.15.2:0/2877406784 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7ff828017070 con 0x7ff838101970 2024-05-14T17:13:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.616+0000 7ff83fb8b740 1 Processor -- start 2024-05-14T17:13:29.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.616+0000 7ff83fb8b740 1 -- start start 2024-05-14T17:13:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.616+0000 7ff83fb8b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5611e08aa750 0x5611e08b3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.616+0000 7ff83fb8b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611e08aab30 0x5611e08aaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.616+0000 7ff83fb8b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5611e08be370 0x5611e08c0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:29.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.616+0000 7ff83fb8b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5611e081dc30 con 0x5611e08aa750 2024-05-14T17:13:29.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.616+0000 7ff83fb8b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5611e080d7d0 con 0x5611e08aab30 2024-05-14T17:13:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.616+0000 7ff83fb8b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5611e0811810 con 0x5611e08be370 2024-05-14T17:13:29.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.616+0000 7ff836d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5611e08aa750 0x5611e08b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.616+0000 7ff836d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5611e08aa750 0x5611e08b3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57532/0 (socket says 172.21.15.2:57532) 2024-05-14T17:13:29.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.616+0000 7ff836d9d640 1 -- 172.21.15.2:0/3111935949 learned_addr learned my addr 172.21.15.2:0/3111935949 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:13:29.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.617+0000 7ff83659c640 1 --2- 172.21.15.2:0/3111935949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611e08aab30 0x5611e08aaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.617+0000 7ff8277fe640 1 --2- 172.21.15.2:0/2877406784 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ff8200b2a10 0x7ff8200b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.617+0000 7ff835d9b640 1 --2- 172.21.15.2:0/3111935949 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5611e08be370 0x5611e08c0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.617+0000 7ff836d9d640 1 -- 172.21.15.2:0/3111935949 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5611e08be370 msgr2=0x5611e08c0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.617+0000 7ff836d9d640 1 --2- 172.21.15.2:0/3111935949 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5611e08be370 0x5611e08c0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.617+0000 7ff836d9d640 1 -- 172.21.15.2:0/3111935949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611e08aab30 msgr2=0x5611e08aaf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.617+0000 7ff836d9d640 1 --2- 172.21.15.2:0/3111935949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611e08aab30 0x5611e08aaf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.617+0000 7ff8277fe640 1 -- 172.21.15.2:0/2877406784 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8513+0+0 (secure 0 0 0) 0x7ff8280e18b0 con 0x7ff838101970 2024-05-14T17:13:30.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.617+0000 7ff836d9d640 1 -- 172.21.15.2:0/3111935949 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611e0522210 con 0x5611e08aa750 2024-05-14T17:13:30.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.617+0000 7ff83659c640 1 --2- 172.21.15.2:0/3111935949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611e08aab30 0x5611e08aaf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:13:30.079 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.617+0000 7ff836d9d640 1 --2- 172.21.15.2:0/2877406784 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ff8200b2a10 0x7ff8200b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:30.113 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.617+0000 7ff836d9d640 1 --2- 172.21.15.2:0/3111935949 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5611e08aa750 0x5611e08b3580 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7ff83000e3a0 tx=0x7ff83000e860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e223d23eb2f47baa server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:30.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.618+0000 7ff824ff9640 1 -- 172.21.15.2:0/3111935949 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff830012dc0 con 0x5611e08aa750 2024-05-14T17:13:30.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.618+0000 7ff824ff9640 1 -- 172.21.15.2:0/3111935949 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff83001bd90 con 0x5611e08aa750 2024-05-14T17:13:30.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.618+0000 7ff824ff9640 1 -- 172.21.15.2:0/3111935949 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83001ab50 con 0x5611e08aa750 2024-05-14T17:13:30.246 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.618+0000 7ff836d9d640 1 --2- 172.21.15.2:0/2877406784 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ff8200b2a10 0x7ff8200b4ed0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff83001f000 tx=0x7ff830020000 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:30.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.618+0000 7ff83fb8b740 1 -- 172.21.15.2:0/3111935949 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5611e08aa750 msgr2=0x5611e08b3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:30.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.618+0000 7ff83fb8b740 1 --2- 172.21.15.2:0/3111935949 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5611e08aa750 0x5611e08b3580 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7ff83000e3a0 tx=0x7ff83000e860 comp rx=0 tx=0).stop 2024-05-14T17:13:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.619+0000 7ff83fb8b740 1 -- 172.21.15.2:0/3111935949 shutdown_connections 2024-05-14T17:13:30.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.619+0000 7ff83fb8b740 1 --2- 172.21.15.2:0/3111935949 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5611e08be370 0x5611e08c0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:30.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.619+0000 7ff83fb8b740 1 --2- 172.21.15.2:0/3111935949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611e08aab30 0x5611e08aaf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.619+0000 7ff83fb8b740 1 --2- 172.21.15.2:0/3111935949 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5611e08aa750 0x5611e08b3580 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.619+0000 7ff83fb8b740 1 -- 172.21.15.2:0/3111935949 >> 172.21.15.2:0/3111935949 conn(0x5611e08a6e00 msgr2=0x5611e08a71e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.619+0000 7ff83fb8b740 1 -- 172.21.15.2:0/3111935949 shutdown_connections 2024-05-14T17:13:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.619+0000 7ff83fb8b740 1 -- 172.21.15.2:0/3111935949 wait complete. 2024-05-14T17:13:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.619+0000 7ff83fb8b740 1 -- start start 2024-05-14T17:13:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.619+0000 7ff83fb8b740 1 Processor -- start 2024-05-14T17:13:30.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.619+0000 7ff83fb8b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611e08aa750 0x5611e08b3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.619+0000 7ff83fb8b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5611e08aab30 0x5611e08b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.619+0000 7ff83fb8b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5611e08be370 0x5611e08b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:30.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.619+0000 7ff83fb8b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5611e04f3790 con 0x5611e08be370 2024-05-14T17:13:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.619+0000 7ff83fb8b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5611e07c9890 con 0x5611e08aa750 2024-05-14T17:13:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.619+0000 7ff83fb8b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5611e08c0f80 con 0x5611e08aab30 2024-05-14T17:13:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.620+0000 7ff83fb8b740 10 client.0 _ll_register_callbacks cb 0x5611e07ccd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:13:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.620+0000 7ff836d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5611e08be370 0x5611e08b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.620+0000 7ff836d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5611e08be370 0x5611e08b0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57542/0 (socket says 172.21.15.2:57542) 2024-05-14T17:13:30.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.620+0000 7ff836d9d640 1 -- 172.21.15.2:0/2277130797 learned_addr learned my addr 172.21.15.2:0/2277130797 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:13:31.014 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.620+0000 7ff83659c640 1 --2- 172.21.15.2:0/2277130797 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611e08aa750 0x5611e08b3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.620+0000 7ff835d9b640 1 --2- 172.21.15.2:0/2277130797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5611e08aab30 0x5611e08b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.620+0000 7ff836d9d640 1 -- 172.21.15.2:0/2277130797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5611e08aab30 msgr2=0x5611e08b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.620+0000 7ff836d9d640 1 --2- 172.21.15.2:0/2277130797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5611e08aab30 0x5611e08b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.620+0000 7ff836d9d640 1 -- 172.21.15.2:0/2277130797 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611e08aa750 msgr2=0x5611e08b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.620+0000 7ff836d9d640 1 --2- 172.21.15.2:0/2277130797 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611e08aa750 0x5611e08b3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.620+0000 7ff836d9d640 1 -- 172.21.15.2:0/2277130797 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611e05ee620 con 0x5611e08be370 2024-05-14T17:13:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.620+0000 7ff836d9d640 1 --2- 172.21.15.2:0/2277130797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5611e08be370 0x5611e08b0cf0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7ff83000ef50 tx=0x7ff8300094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea47ba74ed298745 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.620+0000 7ff81dffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:13:31.364 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.621+0000 7ff81dffb640 1 -- 172.21.15.2:0/2277130797 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83001ae70 con 0x5611e08be370 2024-05-14T17:13:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.621+0000 7ff81dffb640 1 -- 172.21.15.2:0/2277130797 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8513+0+0 (secure 0 0 0) 0x7ff830021050 con 0x5611e08be370 2024-05-14T17:13:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.621+0000 7ff83fb8b740 1 -- 172.21.15.2:0/2277130797 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff83001bd90 con 0x5611e08be370 2024-05-14T17:13:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.621+0000 7ff81dffb640 1 -- 172.21.15.2:0/2277130797 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7ff83001aae0 con 0x5611e08be370 2024-05-14T17:13:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.621+0000 7ff83fb8b740 10 client.17085 fetch_fsmap learned FSMap version 54 2024-05-14T17:13:31.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.621+0000 7ff83fb8b740 1 -- 172.21.15.2:0/2277130797 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5611e08cca20 con 0x5611e08be370 2024-05-14T17:13:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.621+0000 7ff81dffb640 1 -- 172.21.15.2:0/2277130797 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff83001bd90 con 0x5611e08be370 2024-05-14T17:13:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.621+0000 7ff81dffb640 1 -- 172.21.15.2:0/2277130797 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 54) ==== 8338+0+0 (secure 0 0 0) 0x7ff83000f040 con 0x5611e08be370 2024-05-14T17:13:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.621+0000 7ff83fb8b740 10 client.17085 fetch_fsmap finished waiting for FSMap version 54 2024-05-14T17:13:31.664 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.621+0000 7ff83fb8b740 10 client.17085 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:13:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.621+0000 7ff83fb8b740 20 client.17085 populate_metadata read hostname 'smithi002' 2024-05-14T17:13:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.621+0000 7ff83fb8b740 1 --2- 172.21.15.2:0/2277130797 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5611e08cde80 0x5611e08d0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:31.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.621+0000 7ff83fb8b740 4 client.17085 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:13:31.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.621+0000 7ff83fb8b740 1 -- 172.21.15.2:0/2277130797 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611e081fb80 con 0x5611e08cde80 2024-05-14T17:13:31.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.622+0000 7ff83659c640 1 --2- 172.21.15.2:0/2277130797 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5611e08cde80 0x5611e08d0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:31.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.622+0000 7ff83659c640 1 --2- 172.21.15.2:0/2277130797 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5611e08cde80 0x5611e08d0270 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0874658e01cf50e server_cookie=2df0b085f1e84b28 in_seq=0 out_seq=0 2024-05-14T17:13:31.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.622+0000 7ff81dffb640 10 client.17085 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:13:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.623+0000 7ff81dffb640 1 -- 172.21.15.2:0/2277130797 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5611e081fb80 con 0x5611e08cde80 2024-05-14T17:13:32.048 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.623+0000 7ff81dffb640 10 client.17085 handle_command_reply: tid=0 2024-05-14T17:13:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.623+0000 7ff83fb8b740 1 client.17085 shutdown 2024-05-14T17:13:32.123 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.623+0000 7ff83fb8b740 1 -- 172.21.15.2:0/2277130797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5611e08be370 msgr2=0x5611e08b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.623+0000 7ff83fb8b740 1 --2- 172.21.15.2:0/2277130797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5611e08be370 0x5611e08b0cf0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7ff83000ef50 tx=0x7ff8300094f0 comp rx=0 tx=0).stop 2024-05-14T17:13:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.623+0000 7ff83fb8b740 1 -- 172.21.15.2:0/2277130797 shutdown_connections 2024-05-14T17:13:32.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.623+0000 7ff83fb8b740 1 --2- 172.21.15.2:0/2277130797 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5611e08cde80 0x5611e08d0270 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:32.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.623+0000 7ff83fb8b740 1 --2- 172.21.15.2:0/2277130797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5611e08be370 0x5611e08b0cf0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:32.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.623+0000 7ff83fb8b740 1 --2- 172.21.15.2:0/2277130797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5611e08aab30 0x5611e08b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:32.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.623+0000 7ff81dffb640 0 client.17085 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:13:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.623+0000 7ff83fb8b740 1 --2- 172.21.15.2:0/2277130797 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611e08aa750 0x5611e08b3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:32.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.623+0000 7ff83fb8b740 1 -- 172.21.15.2:0/2277130797 >> 172.21.15.2:0/2277130797 conn(0x5611e08a6e00 msgr2=0x5611e08a6100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.623+0000 7ff83fb8b740 1 -- 172.21.15.2:0/2277130797 shutdown_connections 2024-05-14T17:13:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.623+0000 7ff83fb8b740 1 -- 172.21.15.2:0/2277130797 wait complete. 2024-05-14T17:13:32.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.623+0000 7ff83fb8b740 20 client.17085 trim_cache size 0 max 16384 2024-05-14T17:13:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.639+0000 7ff83fb8b740 1 Processor -- start 2024-05-14T17:13:32.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.639+0000 7ff83fb8b740 1 -- start start 2024-05-14T17:13:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.639+0000 7ff83fb8b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5611e08aa750 0x5611e08b0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.639+0000 7ff83fb8b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611e08aab30 0x5611e08b2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.639+0000 7ff83fb8b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5611e08be370 0x5611e08bf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.639+0000 7ff83fb8b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff8381aedd0 con 0x5611e08aa750 2024-05-14T17:13:32.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.639+0000 7ff83fb8b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff8381ac6a0 con 0x5611e08aab30 2024-05-14T17:13:32.941 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.639+0000 7ff83fb8b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff8381aa510 con 0x5611e08be370 2024-05-14T17:13:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.639+0000 7ff83659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5611e08aa750 0x5611e08b0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.639+0000 7ff83659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5611e08aa750 0x5611e08b0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57548/0 (socket says 172.21.15.2:57548) 2024-05-14T17:13:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.639+0000 7ff83659c640 1 -- 172.21.15.2:0/29604930 learned_addr learned my addr 172.21.15.2:0/29604930 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:13:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.639+0000 7ff835d9b640 1 --2- 172.21.15.2:0/29604930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611e08aab30 0x5611e08b2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:33.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.639+0000 7ff836d9d640 1 --2- 172.21.15.2:0/29604930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5611e08be370 0x5611e08bf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:33.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.639+0000 7ff83659c640 1 -- 172.21.15.2:0/29604930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5611e08be370 msgr2=0x5611e08bf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:33.283 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.639+0000 7ff83659c640 1 --2- 172.21.15.2:0/29604930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5611e08be370 0x5611e08bf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:33.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.639+0000 7ff83659c640 1 -- 172.21.15.2:0/29604930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611e08aab30 msgr2=0x5611e08b2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.639+0000 7ff83659c640 1 --2- 172.21.15.2:0/29604930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611e08aab30 0x5611e08b2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:33.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.639+0000 7ff83659c640 1 -- 172.21.15.2:0/29604930 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611e08aeca0 con 0x5611e08aa750 2024-05-14T17:13:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.639+0000 7ff836d9d640 1 --2- 172.21.15.2:0/29604930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5611e08be370 0x5611e08bf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:13:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.639+0000 7ff835d9b640 1 --2- 172.21.15.2:0/29604930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611e08aab30 0x5611e08b2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:13:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.640+0000 7ff83659c640 1 --2- 172.21.15.2:0/29604930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5611e08aa750 0x5611e08b0030 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ff8280a89a0 tx=0x7ff828010f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f37ed4124ce34a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:33.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.640+0000 7ff81ffff640 1 -- 172.21.15.2:0/29604930 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82801e040 con 0x5611e08aa750 2024-05-14T17:13:33.691 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.640+0000 7ff81ffff640 1 -- 172.21.15.2:0/29604930 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8280a93a0 con 0x5611e08aa750 2024-05-14T17:13:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.640+0000 7ff81ffff640 1 -- 172.21.15.2:0/29604930 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8280df280 con 0x5611e08aa750 2024-05-14T17:13:33.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.640+0000 7ff83fb8b740 1 -- 172.21.15.2:0/29604930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5611e08aa750 msgr2=0x5611e08b0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.640+0000 7ff83fb8b740 1 --2- 172.21.15.2:0/29604930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5611e08aa750 0x5611e08b0030 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ff8280a89a0 tx=0x7ff828010f90 comp rx=0 tx=0).stop 2024-05-14T17:13:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.640+0000 7ff83fb8b740 1 -- 172.21.15.2:0/29604930 shutdown_connections 2024-05-14T17:13:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.640+0000 7ff83fb8b740 1 --2- 172.21.15.2:0/29604930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5611e08be370 0x5611e08bf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:34.008 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.640+0000 7ff83fb8b740 1 --2- 172.21.15.2:0/29604930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611e08aab30 0x5611e08b2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:34.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.640+0000 7ff83fb8b740 1 --2- 172.21.15.2:0/29604930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5611e08aa750 0x5611e08b0030 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.640+0000 7ff83fb8b740 1 -- 172.21.15.2:0/29604930 >> 172.21.15.2:0/29604930 conn(0x5611e08a6e00 msgr2=0x5611e08be9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:34.158 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.641+0000 7ff83fb8b740 1 -- 172.21.15.2:0/29604930 shutdown_connections 2024-05-14T17:13:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.641+0000 7ff83fb8b740 1 -- 172.21.15.2:0/29604930 wait complete. 2024-05-14T17:13:34.250 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.641+0000 7ff83fb8b740 1 -- start start 2024-05-14T17:13:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.641+0000 7ff83fb8b740 1 Processor -- start 2024-05-14T17:13:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.641+0000 7ff83fb8b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5611e08aa750 0x5611e08c66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.641+0000 7ff83fb8b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611e08aab30 0x5611e08c3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:34.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.641+0000 7ff83fb8b740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5611e08be370 0x5611e08c3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.641+0000 7ff83fb8b740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff8381aab60 con 0x5611e08be370 2024-05-14T17:13:34.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.641+0000 7ff83fb8b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff8381abfd0 con 0x5611e08aab30 2024-05-14T17:13:34.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.641+0000 7ff83fb8b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff8381ab230 con 0x5611e08aa750 2024-05-14T17:13:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.641+0000 7ff83fb8b740 10 client.0 _ll_register_callbacks cb 0x7ff82c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:13:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.641+0000 7ff836d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5611e08be370 0x5611e08c3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:34.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.641+0000 7ff836d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5611e08be370 0x5611e08c3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57552/0 (socket says 172.21.15.2:57552) 2024-05-14T17:13:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.641+0000 7ff836d9d640 1 -- 172.21.15.2:0/3660055550 learned_addr learned my addr 172.21.15.2:0/3660055550 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:13:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.641+0000 7ff835d9b640 1 --2- 172.21.15.2:0/3660055550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611e08aab30 0x5611e08c3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.642+0000 7ff83659c640 1 --2- 172.21.15.2:0/3660055550 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5611e08aa750 0x5611e08c66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:34.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.642+0000 7ff836d9d640 1 -- 172.21.15.2:0/3660055550 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5611e08aa750 msgr2=0x5611e08c66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.642+0000 7ff836d9d640 1 --2- 172.21.15.2:0/3660055550 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5611e08aa750 0x5611e08c66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.642+0000 7ff836d9d640 1 -- 172.21.15.2:0/3660055550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611e08aab30 msgr2=0x5611e08c3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.642+0000 7ff836d9d640 1 --2- 172.21.15.2:0/3660055550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611e08aab30 0x5611e08c3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:35.177 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.642+0000 7ff836d9d640 1 -- 172.21.15.2:0/3660055550 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611e08c47b0 con 0x5611e08be370 2024-05-14T17:13:35.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.642+0000 7ff836d9d640 1 --2- 172.21.15.2:0/3660055550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5611e08be370 0x5611e08c3fb0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff830014000 tx=0x7ff830009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e593bf34f793f42 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.642+0000 7ff81dffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:13:35.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.642+0000 7ff81dffb640 1 -- 172.21.15.2:0/3660055550 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83003f060 con 0x5611e08be370 2024-05-14T17:13:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.642+0000 7ff81dffb640 1 -- 172.21.15.2:0/3660055550 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8513+0+0 (secure 0 0 0) 0x7ff830021040 con 0x5611e08be370 2024-05-14T17:13:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.642+0000 7ff83fb8b740 1 -- 172.21.15.2:0/3660055550 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8280a93a0 con 0x5611e08be370 2024-05-14T17:13:35.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.642+0000 7ff81dffb640 1 -- 172.21.15.2:0/3660055550 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7ff83001a720 con 0x5611e08be370 2024-05-14T17:13:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.642+0000 7ff83fb8b740 10 client.17091 fetch_fsmap learned FSMap version 54 2024-05-14T17:13:35.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.642+0000 7ff83fb8b740 1 -- 172.21.15.2:0/3660055550 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5611e0938380 con 0x5611e08be370 2024-05-14T17:13:35.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.642+0000 7ff81dffb640 1 -- 172.21.15.2:0/3660055550 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8280a93a0 con 0x5611e08be370 2024-05-14T17:13:35.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.643+0000 7ff81dffb640 1 -- 172.21.15.2:0/3660055550 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 54) ==== 8338+0+0 (secure 0 0 0) 0x7ff83001b860 con 0x5611e08be370 2024-05-14T17:13:35.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.643+0000 7ff83fb8b740 10 client.17091 fetch_fsmap finished waiting for FSMap version 54 2024-05-14T17:13:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.643+0000 7ff83fb8b740 10 client.17091 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:13:35.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.643+0000 7ff83fb8b740 20 client.17091 populate_metadata read hostname 'smithi002' 2024-05-14T17:13:35.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.643+0000 7ff83fb8b740 1 --2- 172.21.15.2:0/3660055550 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5611e0938a10 0x5611e093ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.643+0000 7ff83fb8b740 4 client.17091 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:13:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.643+0000 7ff83fb8b740 1 -- 172.21.15.2:0/3660055550 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5611e0819360 con 0x5611e0938a10 2024-05-14T17:13:36.069 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.643+0000 7ff83659c640 1 --2- 172.21.15.2:0/3660055550 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5611e0938a10 0x5611e093ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.643+0000 7ff81dffb640 10 client.17091 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:13:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.643+0000 7ff83659c640 1 --2- 172.21.15.2:0/3660055550 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5611e0938a10 0x5611e093ae30 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afdbc020c8d92e1e server_cookie=5ad7c6f72d318dd3 in_seq=0 out_seq=0 2024-05-14T17:13:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.653+0000 7ff81dffb640 1 -- 172.21.15.2:0/3660055550 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5611e0819360 con 0x5611e0938a10 2024-05-14T17:13:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.653+0000 7ff81dffb640 10 client.17091 handle_command_reply: tid=0 2024-05-14T17:13:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.653+0000 7ff83fb8b740 1 client.17091 shutdown 2024-05-14T17:13:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.653+0000 7ff83fb8b740 1 -- 172.21.15.2:0/3660055550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5611e08be370 msgr2=0x5611e08c3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.653+0000 7ff83fb8b740 1 --2- 172.21.15.2:0/3660055550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5611e08be370 0x5611e08c3fb0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff830014000 tx=0x7ff830009a50 comp rx=0 tx=0).stop 2024-05-14T17:13:36.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.653+0000 7ff83fb8b740 1 -- 172.21.15.2:0/3660055550 shutdown_connections 2024-05-14T17:13:36.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.653+0000 7ff83fb8b740 1 --2- 172.21.15.2:0/3660055550 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5611e0938a10 0x5611e093ae30 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.653+0000 7ff83fb8b740 1 --2- 172.21.15.2:0/3660055550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5611e08be370 0x5611e08c3fb0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.653+0000 7ff83fb8b740 1 --2- 172.21.15.2:0/3660055550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611e08aab30 0x5611e08c3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:36.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.653+0000 7ff83fb8b740 1 --2- 172.21.15.2:0/3660055550 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5611e08aa750 0x5611e08c66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:36.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.653+0000 7ff81dffb640 0 client.17091 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:13:36.785 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.653+0000 7ff83fb8b740 1 -- 172.21.15.2:0/3660055550 >> 172.21.15.2:0/3660055550 conn(0x5611e08a6e00 msgr2=0x5611e08bf200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:36.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.653+0000 7ff83fb8b740 1 -- 172.21.15.2:0/3660055550 shutdown_connections 2024-05-14T17:13:36.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.653+0000 7ff83fb8b740 1 -- 172.21.15.2:0/3660055550 wait complete. 2024-05-14T17:13:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.653+0000 7ff83fb8b740 20 client.17091 trim_cache size 0 max 16384 2024-05-14T17:13:36.986 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:13:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:13:37.086 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:13:37.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.653+0000 7ff83ce74640 1 -- 172.21.15.2:0/2877406784 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ff8200b2a10 msgr2=0x7ff8200b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.653+0000 7ff83ce74640 1 --2- 172.21.15.2:0/2877406784 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ff8200b2a10 0x7ff8200b4ed0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff83001f000 tx=0x7ff830020000 comp rx=0 tx=0).stop 2024-05-14T17:13:37.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.654+0000 7ff83ce74640 1 -- 172.21.15.2:0/2877406784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff838101970 msgr2=0x7ff838198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.654+0000 7ff83ce74640 1 --2- 172.21.15.2:0/2877406784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff838101970 0x7ff838198120 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ff82800b4a0 tx=0x7ff82800b960 comp rx=0 tx=0).stop 2024-05-14T17:13:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.654+0000 7ff83ce74640 1 -- 172.21.15.2:0/2877406784 shutdown_connections 2024-05-14T17:13:37.419 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.654+0000 7ff83ce74640 1 --2- 172.21.15.2:0/2877406784 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ff8200b2a10 0x7ff8200b4ed0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.654+0000 7ff83ce74640 1 --2- 172.21.15.2:0/2877406784 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff838107980 0x7ff83819c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.654+0000 7ff83ce74640 1 --2- 172.21.15.2:0/2877406784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff838102330 0x7ff838198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.654+0000 7ff83ce74640 1 --2- 172.21.15.2:0/2877406784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff838101970 0x7ff838198120 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:37.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.654+0000 7ff83ce74640 1 -- 172.21.15.2:0/2877406784 >> 172.21.15.2:0/2877406784 conn(0x7ff838077cd0 msgr2=0x7ff8380ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:37.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.654+0000 7ff83ce74640 1 -- 172.21.15.2:0/2877406784 shutdown_connections 2024-05-14T17:13:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:12:49.654+0000 7ff83ce74640 1 -- 172.21.15.2:0/2877406784 wait complete. 2024-05-14T17:13:37.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:12:50 smithi002 ceph-mon[31538]: pgmap v569: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-05-14T17:13:37.855 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:12:52 smithi002 ceph-mon[31538]: pgmap v570: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-05-14T17:13:37.903 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:12:54 smithi002 ceph-mon[31538]: pgmap v571: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 1.1 MiB/s wr, 139 op/s 2024-05-14T17:13:37.953 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:12:56 smithi002 ceph-mon[31538]: pgmap v572: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.7 MiB/s wr, 203 op/s 2024-05-14T17:13:38.004 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:13:28 smithi123 ceph-mon[35971]: pgmap v590: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 40 KiB/s wr, 10 op/s 2024-05-14T17:13:38.095 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:13:29 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:13:38.145 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:13:30 smithi123 ceph-mon[35971]: pgmap v591: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-05-14T17:13:38.195 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:13:32 smithi123 ceph-mon[35971]: pgmap v592: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 25 KiB/s wr, 5 op/s 2024-05-14T17:13:38.245 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:13:34 smithi123 ceph-mon[35971]: pgmap v593: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 49 KiB/s wr, 12 op/s 2024-05-14T17:13:38.295 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:13:36 smithi123 ceph-mon[35971]: pgmap v594: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 44 KiB/s wr, 10 op/s 2024-05-14T17:13:38.345 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:13:28 smithi151 ceph-mon[36066]: pgmap v590: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 40 KiB/s wr, 10 op/s 2024-05-14T17:13:38.395 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:13:29 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:13:38.445 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:13:30 smithi151 ceph-mon[36066]: pgmap v591: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-05-14T17:13:38.495 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:13:32 smithi151 ceph-mon[36066]: pgmap v592: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 25 KiB/s wr, 5 op/s 2024-05-14T17:13:38.561 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:13:34 smithi151 ceph-mon[36066]: pgmap v593: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 49 KiB/s wr, 12 op/s 2024-05-14T17:13:38.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:13:36 smithi151 ceph-mon[36066]: pgmap v594: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 44 KiB/s wr, 10 op/s 2024-05-14T17:13:38.663 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-05-14T17:13:38.713 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:12:58 smithi002 ceph-mon[31538]: pgmap v573: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.7 MiB/s wr, 203 op/s 2024-05-14T17:13:38.762 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:13:00 smithi002 ceph-mon[31538]: pgmap v574: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 5.1 MiB/s wr, 578 op/s 2024-05-14T17:13:38.825 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.754+0000 7fc647baf640 1 -- 172.21.15.2:0/3262859331 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc640101ce0 msgr2=0x7fc640105d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:38.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.754+0000 7fc647baf640 1 --2- 172.21.15.2:0/3262859331 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc640101ce0 0x7fc640105d20 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc630009980 tx=0x7fc63002c620 comp rx=0 tx=0).stop 2024-05-14T17:13:38.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.755+0000 7fc647baf640 1 -- 172.21.15.2:0/3262859331 shutdown_connections 2024-05-14T17:13:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.755+0000 7fc647baf640 1 --2- 172.21.15.2:0/3262859331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc640106400 0x7fc64010dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:39.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.755+0000 7fc647baf640 1 --2- 172.21.15.2:0/3262859331 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc640101ce0 0x7fc640105d20 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.755+0000 7fc647baf640 1 --2- 172.21.15.2:0/3262859331 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc640101330 0x7fc640101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.755+0000 7fc647baf640 1 -- 172.21.15.2:0/3262859331 >> 172.21.15.2:0/3262859331 conn(0x7fc640077ee0 msgr2=0x7fc6400ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:39.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.755+0000 7fc647baf640 1 -- 172.21.15.2:0/3262859331 shutdown_connections 2024-05-14T17:13:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.755+0000 7fc647baf640 1 -- 172.21.15.2:0/3262859331 wait complete. 2024-05-14T17:13:39.304 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.755+0000 7fc647baf640 1 Processor -- start 2024-05-14T17:13:39.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.755+0000 7fc647baf640 1 -- start start 2024-05-14T17:13:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.755+0000 7fc647baf640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc640101330 0x7fc640196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.755+0000 7fc647baf640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc640106400 0x7fc6401974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.756+0000 7fc647baf640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc64019b850 0x7fc64019bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.756+0000 7fc647baf640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc640110ad0 con 0x7fc640101330 2024-05-14T17:13:39.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.756+0000 7fc647baf640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc640110950 con 0x7fc64019b850 2024-05-14T17:13:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.756+0000 7fc647baf640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc640110c50 con 0x7fc640106400 2024-05-14T17:13:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.756+0000 7fc64594b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc640101330 0x7fc640196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:39.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.756+0000 7fc64594b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc640101330 0x7fc640196f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54434/0 (socket says 172.21.15.2:54434) 2024-05-14T17:13:39.821 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.756+0000 7fc64594b640 1 -- 172.21.15.2:0/1644257413 learned_addr learned my addr 172.21.15.2:0/1644257413 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:13:39.871 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.756+0000 7fc64614c640 1 --2- 172.21.15.2:0/1644257413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc64019b850 0x7fc64019bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:39.921 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.756+0000 7fc64514a640 1 --2- 172.21.15.2:0/1644257413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc640106400 0x7fc6401974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:39.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.756+0000 7fc64594b640 1 -- 172.21.15.2:0/1644257413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc640106400 msgr2=0x7fc6401974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:40.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.756+0000 7fc64594b640 1 --2- 172.21.15.2:0/1644257413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc640106400 0x7fc6401974d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.756+0000 7fc64594b640 1 -- 172.21.15.2:0/1644257413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc64019b850 msgr2=0x7fc64019bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.756+0000 7fc64594b640 1 --2- 172.21.15.2:0/1644257413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc64019b850 0x7fc64019bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.756+0000 7fc64594b640 1 -- 172.21.15.2:0/1644257413 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc64019c340 con 0x7fc640101330 2024-05-14T17:13:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.756+0000 7fc64594b640 1 --2- 172.21.15.2:0/1644257413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc640101330 0x7fc640196f90 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc63c00d950 tx=0x7fc63c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.756+0000 7fc636ffd640 1 -- 172.21.15.2:0/1644257413 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c017030 con 0x7fc640101330 2024-05-14T17:13:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.756+0000 7fc636ffd640 1 -- 172.21.15.2:0/1644257413 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc63c004020 con 0x7fc640101330 2024-05-14T17:13:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.756+0000 7fc647baf640 1 -- 172.21.15.2:0/1644257413 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc64019c630 con 0x7fc640101330 2024-05-14T17:13:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.756+0000 7fc636ffd640 1 -- 172.21.15.2:0/1644257413 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c0047a0 con 0x7fc640101330 2024-05-14T17:13:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.756+0000 7fc647baf640 1 -- 172.21.15.2:0/1644257413 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6401a3df0 con 0x7fc640101330 2024-05-14T17:13:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.759+0000 7fc64a99e740 1 Processor -- start 2024-05-14T17:13:40.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.759+0000 7fc64a99e740 1 -- start start 2024-05-14T17:13:40.646 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.759+0000 7fc636ffd640 1 -- 172.21.15.2:0/1644257413 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fc63c009a30 con 0x7fc640101330 2024-05-14T17:13:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.760+0000 7fc64a99e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629498568f0 0x562949856cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.760+0000 7fc64a99e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562949857210 0x562949861590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.760+0000 7fc64a99e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562949861ad0 0x562949863ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.760+0000 7fc64a99e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5629497c9c30 con 0x562949857210 2024-05-14T17:13:40.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.760+0000 7fc64a99e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5629497b97d0 con 0x5629498568f0 2024-05-14T17:13:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.760+0000 7fc64a99e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5629497bd810 con 0x562949861ad0 2024-05-14T17:13:40.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.760+0000 7fc64594b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562949857210 0x562949861590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:41.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.760+0000 7fc64594b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562949857210 0x562949861590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54440/0 (socket says 172.21.15.2:54440) 2024-05-14T17:13:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.760+0000 7fc64514a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562949861ad0 0x562949863ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.760+0000 7fc64594b640 1 -- 172.21.15.2:0/2394597134 learned_addr learned my addr 172.21.15.2:0/2394597134 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:13:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.760+0000 7fc64614c640 1 --2- 172.21.15.2:0/2394597134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629498568f0 0x562949856cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.760+0000 7fc64514a640 1 -- 172.21.15.2:0/2394597134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629498568f0 msgr2=0x562949856cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.760+0000 7fc64514a640 1 --2- 172.21.15.2:0/2394597134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629498568f0 0x562949856cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:41.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.760+0000 7fc64514a640 1 -- 172.21.15.2:0/2394597134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562949857210 msgr2=0x562949861590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:41.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.760+0000 7fc64514a640 1 --2- 172.21.15.2:0/2394597134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562949857210 0x562949861590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.760+0000 7fc64514a640 1 -- 172.21.15.2:0/2394597134 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629494ce210 con 0x562949861ad0 2024-05-14T17:13:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.760+0000 7fc64594b640 1 --2- 172.21.15.2:0/2394597134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562949857210 0x562949861590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:13:41.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.761+0000 7fc636ffd640 1 --2- 172.21.15.2:0/1644257413 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc62c0b2a10 0x7fc62c0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.761+0000 7fc64514a640 1 --2- 172.21.15.2:0/2394597134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562949861ad0 0x562949863ec0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc630006010 tx=0x7fc630034bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32588e095e48a230 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.761+0000 7fc64614c640 1 --2- 172.21.15.2:0/1644257413 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc62c0b2a10 0x7fc62c0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:41.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.761+0000 7fc62bfff640 1 -- 172.21.15.2:0/2394597134 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc630041070 con 0x562949861ad0 2024-05-14T17:13:41.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.761+0000 7fc636ffd640 1 -- 172.21.15.2:0/1644257413 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8513+0+0 (secure 0 0 0) 0x7fc63c0e2070 con 0x7fc640101330 2024-05-14T17:13:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.761+0000 7fc62bfff640 1 -- 172.21.15.2:0/2394597134 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6300044d0 con 0x562949861ad0 2024-05-14T17:13:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.761+0000 7fc62bfff640 1 -- 172.21.15.2:0/2394597134 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc630037070 con 0x562949861ad0 2024-05-14T17:13:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.762+0000 7fc64614c640 1 --2- 172.21.15.2:0/1644257413 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc62c0b2a10 0x7fc62c0b4ed0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc638005fd0 tx=0x7fc638009600 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:42.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.762+0000 7fc64a99e740 1 -- 172.21.15.2:0/2394597134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562949861ad0 msgr2=0x562949863ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:42.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.762+0000 7fc64a99e740 1 --2- 172.21.15.2:0/2394597134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562949861ad0 0x562949863ec0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc630006010 tx=0x7fc630034bd0 comp rx=0 tx=0).stop 2024-05-14T17:13:42.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.762+0000 7fc64a99e740 1 -- 172.21.15.2:0/2394597134 shutdown_connections 2024-05-14T17:13:42.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.762+0000 7fc64a99e740 1 --2- 172.21.15.2:0/2394597134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562949861ad0 0x562949863ec0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.762+0000 7fc64a99e740 1 --2- 172.21.15.2:0/2394597134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562949857210 0x562949861590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:42.305 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.762+0000 7fc64a99e740 1 --2- 172.21.15.2:0/2394597134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629498568f0 0x562949856cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.762+0000 7fc64a99e740 1 -- 172.21.15.2:0/2394597134 >> 172.21.15.2:0/2394597134 conn(0x562949852e00 msgr2=0x5629498531e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.762+0000 7fc64a99e740 1 -- 172.21.15.2:0/2394597134 shutdown_connections 2024-05-14T17:13:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.762+0000 7fc64a99e740 1 -- 172.21.15.2:0/2394597134 wait complete. 2024-05-14T17:13:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.762+0000 7fc64a99e740 1 -- start start 2024-05-14T17:13:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.762+0000 7fc64a99e740 1 Processor -- start 2024-05-14T17:13:42.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.763+0000 7fc64a99e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629498568f0 0x562949861460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:42.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.763+0000 7fc64a99e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562949857210 0x56294985e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:42.731 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.763+0000 7fc64a99e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562949861ad0 0x56294985ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.763+0000 7fc64a99e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56294949f790 con 0x562949861ad0 2024-05-14T17:13:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.763+0000 7fc64a99e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562949775890 con 0x5629498568f0 2024-05-14T17:13:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.763+0000 7fc64a99e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562949864740 con 0x562949857210 2024-05-14T17:13:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.763+0000 7fc64a99e740 10 client.0 _ll_register_callbacks cb 0x562949778d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:13:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.763+0000 7fc64614c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562949861ad0 0x56294985ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.763+0000 7fc64614c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562949861ad0 0x56294985ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54452/0 (socket says 172.21.15.2:54452) 2024-05-14T17:13:43.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.763+0000 7fc64614c640 1 -- 172.21.15.2:0/4157181270 learned_addr learned my addr 172.21.15.2:0/4157181270 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:13:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.763+0000 7fc64514a640 1 --2- 172.21.15.2:0/4157181270 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562949857210 0x56294985e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.763+0000 7fc64594b640 1 --2- 172.21.15.2:0/4157181270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629498568f0 0x562949861460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.763+0000 7fc64614c640 1 -- 172.21.15.2:0/4157181270 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562949857210 msgr2=0x56294985e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:43.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.763+0000 7fc64614c640 1 --2- 172.21.15.2:0/4157181270 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562949857210 0x56294985e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.763+0000 7fc64614c640 1 -- 172.21.15.2:0/4157181270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629498568f0 msgr2=0x562949861460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.763+0000 7fc64614c640 1 --2- 172.21.15.2:0/4157181270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629498568f0 0x562949861460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.763+0000 7fc64614c640 1 -- 172.21.15.2:0/4157181270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56294959a620 con 0x562949861ad0 2024-05-14T17:13:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.764+0000 7fc64614c640 1 --2- 172.21.15.2:0/4157181270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562949861ad0 0x56294985ecd0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fc638011d40 tx=0x7fc638004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=830958c5a9230b8b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:43.548 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.764+0000 7fc6297fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:13:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.764+0000 7fc6297fa640 1 -- 172.21.15.2:0/4157181270 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc638004b20 con 0x562949861ad0 2024-05-14T17:13:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.764+0000 7fc6297fa640 1 -- 172.21.15.2:0/4157181270 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8513+0+0 (secure 0 0 0) 0x7fc638004cc0 con 0x562949861ad0 2024-05-14T17:13:43.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.764+0000 7fc64a99e740 1 -- 172.21.15.2:0/4157181270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6300044d0 con 0x562949861ad0 2024-05-14T17:13:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.764+0000 7fc6297fa640 1 -- 172.21.15.2:0/4157181270 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fc638005020 con 0x562949861ad0 2024-05-14T17:13:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.764+0000 7fc64a99e740 10 client.17103 fetch_fsmap learned FSMap version 55 2024-05-14T17:13:43.873 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.764+0000 7fc64a99e740 1 -- 172.21.15.2:0/4157181270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x562949870230 con 0x562949861ad0 2024-05-14T17:13:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.764+0000 7fc6297fa640 1 -- 172.21.15.2:0/4157181270 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6300044d0 con 0x562949861ad0 2024-05-14T17:13:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.764+0000 7fc6297fa640 1 -- 172.21.15.2:0/4157181270 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 55) ==== 8342+0+0 (secure 0 0 0) 0x7fc63800f040 con 0x562949861ad0 2024-05-14T17:13:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.764+0000 7fc64a99e740 10 client.17103 fetch_fsmap finished waiting for FSMap version 55 2024-05-14T17:13:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.764+0000 7fc64a99e740 10 client.17103 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:13:44.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.764+0000 7fc64a99e740 20 client.17103 populate_metadata read hostname 'smithi002' 2024-05-14T17:13:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.764+0000 7fc64a99e740 1 --2- 172.21.15.2:0/4157181270 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x562949871760 0x562949873b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:44.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.764+0000 7fc64a99e740 4 client.17103 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:13:44.290 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.764+0000 7fc64a99e740 1 -- 172.21.15.2:0/4157181270 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629497cbb80 con 0x562949871760 2024-05-14T17:13:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.765+0000 7fc64594b640 1 --2- 172.21.15.2:0/4157181270 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x562949871760 0x562949873b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.765+0000 7fc6297fa640 10 client.17103 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:13:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.765+0000 7fc64594b640 1 --2- 172.21.15.2:0/4157181270 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x562949871760 0x562949873b50 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c4f919ce684d3ab server_cookie=8b95fbdf1940462f in_seq=0 out_seq=0 2024-05-14T17:13:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.766+0000 7fc6297fa640 1 -- 172.21.15.2:0/4157181270 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5629497cbb80 con 0x562949871760 2024-05-14T17:13:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.766+0000 7fc6297fa640 10 client.17103 handle_command_reply: tid=0 2024-05-14T17:13:44.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.766+0000 7fc64a99e740 1 client.17103 shutdown 2024-05-14T17:13:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.766+0000 7fc64a99e740 1 -- 172.21.15.2:0/4157181270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562949861ad0 msgr2=0x56294985ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:44.707 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.766+0000 7fc64a99e740 1 --2- 172.21.15.2:0/4157181270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562949861ad0 0x56294985ecd0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fc638011d40 tx=0x7fc638004a40 comp rx=0 tx=0).stop 2024-05-14T17:13:44.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.766+0000 7fc64a99e740 1 -- 172.21.15.2:0/4157181270 shutdown_connections 2024-05-14T17:13:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.766+0000 7fc64a99e740 1 --2- 172.21.15.2:0/4157181270 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x562949871760 0x562949873b50 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.766+0000 7fc64a99e740 1 --2- 172.21.15.2:0/4157181270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562949861ad0 0x56294985ecd0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.766+0000 7fc64a99e740 1 --2- 172.21.15.2:0/4157181270 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562949857210 0x56294985e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.766+0000 7fc6297fa640 0 client.17103 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:13:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.766+0000 7fc64a99e740 1 --2- 172.21.15.2:0/4157181270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5629498568f0 0x562949861460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.766+0000 7fc64a99e740 1 -- 172.21.15.2:0/4157181270 >> 172.21.15.2:0/4157181270 conn(0x562949852e00 msgr2=0x562949852100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.766+0000 7fc64a99e740 1 -- 172.21.15.2:0/4157181270 shutdown_connections 2024-05-14T17:13:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.766+0000 7fc64a99e740 1 -- 172.21.15.2:0/4157181270 wait complete. 2024-05-14T17:13:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.766+0000 7fc64a99e740 20 client.17103 trim_cache size 0 max 16384 2024-05-14T17:13:45.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.782+0000 7fc64a99e740 1 Processor -- start 2024-05-14T17:13:45.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.782+0000 7fc64a99e740 1 -- start start 2024-05-14T17:13:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.782+0000 7fc64a99e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629498568f0 0x562949869d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.782+0000 7fc64a99e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562949857210 0x562949860f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.782+0000 7fc64a99e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562949861ad0 0x562949879490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.782+0000 7fc64a99e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc6401aec20 con 0x5629498568f0 2024-05-14T17:13:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.782+0000 7fc64a99e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc6401ac4c0 con 0x562949861ad0 2024-05-14T17:13:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.782+0000 7fc64a99e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc6401aa330 con 0x562949857210 2024-05-14T17:13:45.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.782+0000 7fc64594b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629498568f0 0x562949869d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:45.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.782+0000 7fc64594b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629498568f0 0x562949869d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54466/0 (socket says 172.21.15.2:54466) 2024-05-14T17:13:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.782+0000 7fc64594b640 1 -- 172.21.15.2:0/1973722446 learned_addr learned my addr 172.21.15.2:0/1973722446 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:13:45.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.782+0000 7fc64514a640 1 --2- 172.21.15.2:0/1973722446 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562949857210 0x562949860f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.782+0000 7fc64614c640 1 --2- 172.21.15.2:0/1973722446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562949861ad0 0x562949879490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.782+0000 7fc64594b640 1 -- 172.21.15.2:0/1973722446 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562949857210 msgr2=0x562949860f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.782+0000 7fc64594b640 1 --2- 172.21.15.2:0/1973722446 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562949857210 0x562949860f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.782+0000 7fc64594b640 1 -- 172.21.15.2:0/1973722446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562949861ad0 msgr2=0x562949879490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.782+0000 7fc64594b640 1 --2- 172.21.15.2:0/1973722446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562949861ad0 0x562949879490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.782+0000 7fc64594b640 1 -- 172.21.15.2:0/1973722446 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562949873ec0 con 0x5629498568f0 2024-05-14T17:13:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.782+0000 7fc64594b640 1 --2- 172.21.15.2:0/1973722446 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629498568f0 0x562949869d70 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc63c0df870 tx=0x7fc63c0e5bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=421b839330498be6 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.783+0000 7fc62b7fe640 1 -- 172.21.15.2:0/1973722446 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c0a9120 con 0x5629498568f0 2024-05-14T17:13:46.187 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.783+0000 7fc62b7fe640 1 -- 172.21.15.2:0/1973722446 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc63c0a8e60 con 0x5629498568f0 2024-05-14T17:13:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.783+0000 7fc62b7fe640 1 -- 172.21.15.2:0/1973722446 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c0a8ac0 con 0x5629498568f0 2024-05-14T17:13:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.783+0000 7fc64a99e740 1 -- 172.21.15.2:0/1973722446 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629498568f0 msgr2=0x562949869d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.783+0000 7fc64a99e740 1 --2- 172.21.15.2:0/1973722446 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629498568f0 0x562949869d70 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc63c0df870 tx=0x7fc63c0e5bc0 comp rx=0 tx=0).stop 2024-05-14T17:13:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.783+0000 7fc64a99e740 1 -- 172.21.15.2:0/1973722446 shutdown_connections 2024-05-14T17:13:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.783+0000 7fc64a99e740 1 --2- 172.21.15.2:0/1973722446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562949861ad0 0x562949879490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.783+0000 7fc64a99e740 1 --2- 172.21.15.2:0/1973722446 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562949857210 0x562949860f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.783+0000 7fc64a99e740 1 --2- 172.21.15.2:0/1973722446 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629498568f0 0x562949869d70 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.783+0000 7fc64a99e740 1 -- 172.21.15.2:0/1973722446 >> 172.21.15.2:0/1973722446 conn(0x562949852e00 msgr2=0x562949852100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.783+0000 7fc64a99e740 1 -- 172.21.15.2:0/1973722446 shutdown_connections 2024-05-14T17:13:46.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.783+0000 7fc64a99e740 1 -- 172.21.15.2:0/1973722446 wait complete. 2024-05-14T17:13:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.784+0000 7fc64a99e740 1 -- start start 2024-05-14T17:13:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.784+0000 7fc64a99e740 1 Processor -- start 2024-05-14T17:13:46.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.784+0000 7fc64a99e740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629498568f0 0x562949868690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:46.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.784+0000 7fc64a99e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562949857210 0x562949864a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:46.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.784+0000 7fc64a99e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562949861ad0 0x562949864f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.784+0000 7fc64a99e740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc6401aa980 con 0x5629498568f0 2024-05-14T17:13:47.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.784+0000 7fc64a99e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc6401abdf0 con 0x562949861ad0 2024-05-14T17:13:47.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.784+0000 7fc64a99e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc6401ab050 con 0x562949857210 2024-05-14T17:13:47.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.784+0000 7fc64a99e740 10 client.0 _ll_register_callbacks cb 0x7fc6380119f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:13:47.234 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.784+0000 7fc64594b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629498568f0 0x562949868690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.784+0000 7fc64594b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629498568f0 0x562949868690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54478/0 (socket says 172.21.15.2:54478) 2024-05-14T17:13:47.334 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.784+0000 7fc64594b640 1 -- 172.21.15.2:0/3053164545 learned_addr learned my addr 172.21.15.2:0/3053164545 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:13:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.784+0000 7fc64514a640 1 --2- 172.21.15.2:0/3053164545 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562949857210 0x562949864a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:47.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.785+0000 7fc64614c640 1 --2- 172.21.15.2:0/3053164545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562949861ad0 0x562949864f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.785+0000 7fc64594b640 1 -- 172.21.15.2:0/3053164545 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562949857210 msgr2=0x562949864a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:47.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.785+0000 7fc64594b640 1 --2- 172.21.15.2:0/3053164545 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562949857210 0x562949864a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.785+0000 7fc64594b640 1 -- 172.21.15.2:0/3053164545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562949861ad0 msgr2=0x562949864f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.785+0000 7fc64594b640 1 --2- 172.21.15.2:0/3053164545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562949861ad0 0x562949864f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:47.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.785+0000 7fc64594b640 1 -- 172.21.15.2:0/3053164545 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562949865780 con 0x5629498568f0 2024-05-14T17:13:47.809 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.785+0000 7fc64594b640 1 --2- 172.21.15.2:0/3053164545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629498568f0 0x562949868690 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fc63c0a65c0 tx=0x7fc63c009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83833d74ac64b39e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:47.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.785+0000 7fc6297fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:13:47.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.785+0000 7fc6297fa640 1 -- 172.21.15.2:0/3053164545 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c0a8ca0 con 0x5629498568f0 2024-05-14T17:13:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.785+0000 7fc6297fa640 1 -- 172.21.15.2:0/3053164545 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8513+0+0 (secure 0 0 0) 0x7fc63c0dfa40 con 0x5629498568f0 2024-05-14T17:13:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.785+0000 7fc64a99e740 1 -- 172.21.15.2:0/3053164545 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc63c0a8e60 con 0x5629498568f0 2024-05-14T17:13:48.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.785+0000 7fc6297fa640 1 -- 172.21.15.2:0/3053164545 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fc63c0a9a50 con 0x5629498568f0 2024-05-14T17:13:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.785+0000 7fc64a99e740 10 client.17109 fetch_fsmap learned FSMap version 55 2024-05-14T17:13:48.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.785+0000 7fc64a99e740 1 -- 172.21.15.2:0/3053164545 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5629498e4750 con 0x5629498568f0 2024-05-14T17:13:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.785+0000 7fc6297fa640 1 -- 172.21.15.2:0/3053164545 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc63c0a8e60 con 0x5629498568f0 2024-05-14T17:13:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.785+0000 7fc6297fa640 1 -- 172.21.15.2:0/3053164545 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 55) ==== 8342+0+0 (secure 0 0 0) 0x7fc63c0e41a0 con 0x5629498568f0 2024-05-14T17:13:48.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.785+0000 7fc64a99e740 10 client.17109 fetch_fsmap finished waiting for FSMap version 55 2024-05-14T17:13:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.785+0000 7fc64a99e740 10 client.17109 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:13:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.785+0000 7fc64a99e740 20 client.17109 populate_metadata read hostname 'smithi002' 2024-05-14T17:13:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.785+0000 7fc64a99e740 1 --2- 172.21.15.2:0/3053164545 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5629498e4e70 0x5629498e7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.785+0000 7fc64a99e740 4 client.17109 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:13:48.710 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.785+0000 7fc64a99e740 1 -- 172.21.15.2:0/3053164545 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5629497c5360 con 0x5629498e4e70 2024-05-14T17:13:48.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.786+0000 7fc64514a640 1 --2- 172.21.15.2:0/3053164545 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5629498e4e70 0x5629498e7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.786+0000 7fc64514a640 1 --2- 172.21.15.2:0/3053164545 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5629498e4e70 0x5629498e7260 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=896d62b23321f5f6 server_cookie=ffc0facdae23c206 in_seq=0 out_seq=0 2024-05-14T17:13:48.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.786+0000 7fc6297fa640 10 client.17109 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:13:48.910 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.814+0000 7fc6297fa640 1 -- 172.21.15.2:0/3053164545 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5629497c5360 con 0x5629498e4e70 2024-05-14T17:13:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.814+0000 7fc6297fa640 10 client.17109 handle_command_reply: tid=0 2024-05-14T17:13:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.814+0000 7fc64a99e740 1 client.17109 shutdown 2024-05-14T17:13:49.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.814+0000 7fc64a99e740 1 -- 172.21.15.2:0/3053164545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629498568f0 msgr2=0x562949868690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.814+0000 7fc64a99e740 1 --2- 172.21.15.2:0/3053164545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629498568f0 0x562949868690 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fc63c0a65c0 tx=0x7fc63c009f50 comp rx=0 tx=0).stop 2024-05-14T17:13:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.814+0000 7fc64a99e740 1 -- 172.21.15.2:0/3053164545 shutdown_connections 2024-05-14T17:13:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.814+0000 7fc64a99e740 1 --2- 172.21.15.2:0/3053164545 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5629498e4e70 0x5629498e7260 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.814+0000 7fc64a99e740 1 --2- 172.21.15.2:0/3053164545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562949861ad0 0x562949864f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:49.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.814+0000 7fc64a99e740 1 --2- 172.21.15.2:0/3053164545 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562949857210 0x562949864a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.814+0000 7fc6297fa640 0 client.17109 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:13:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.814+0000 7fc64a99e740 1 --2- 172.21.15.2:0/3053164545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5629498568f0 0x562949868690 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.814+0000 7fc64a99e740 1 -- 172.21.15.2:0/3053164545 >> 172.21.15.2:0/3053164545 conn(0x562949852e00 msgr2=0x562949868d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.814+0000 7fc64a99e740 1 -- 172.21.15.2:0/3053164545 shutdown_connections 2024-05-14T17:13:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.814+0000 7fc64a99e740 1 -- 172.21.15.2:0/3053164545 wait complete. 2024-05-14T17:13:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.814+0000 7fc64a99e740 20 client.17109 trim_cache size 0 max 16384 2024-05-14T17:13:49.669 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:13:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:13:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:13:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.815+0000 7fc647baf640 1 -- 172.21.15.2:0/1644257413 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc62c0b2a10 msgr2=0x7fc62c0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.815+0000 7fc647baf640 1 --2- 172.21.15.2:0/1644257413 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc62c0b2a10 0x7fc62c0b4ed0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc638005fd0 tx=0x7fc638009600 comp rx=0 tx=0).stop 2024-05-14T17:13:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.815+0000 7fc647baf640 1 -- 172.21.15.2:0/1644257413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc640101330 msgr2=0x7fc640196f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:49.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.815+0000 7fc647baf640 1 --2- 172.21.15.2:0/1644257413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc640101330 0x7fc640196f90 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc63c00d950 tx=0x7fc63c00de10 comp rx=0 tx=0).stop 2024-05-14T17:13:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.815+0000 7fc647baf640 1 -- 172.21.15.2:0/1644257413 shutdown_connections 2024-05-14T17:13:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.815+0000 7fc647baf640 1 --2- 172.21.15.2:0/1644257413 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fc62c0b2a10 0x7fc62c0b4ed0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.815+0000 7fc647baf640 1 --2- 172.21.15.2:0/1644257413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc64019b850 0x7fc64019bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:50.177 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.815+0000 7fc647baf640 1 --2- 172.21.15.2:0/1644257413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc640106400 0x7fc6401974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.815+0000 7fc647baf640 1 --2- 172.21.15.2:0/1644257413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc640101330 0x7fc640196f90 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:50.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.815+0000 7fc647baf640 1 -- 172.21.15.2:0/1644257413 >> 172.21.15.2:0/1644257413 conn(0x7fc640077ee0 msgr2=0x7fc640104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:50.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.815+0000 7fc647baf640 1 -- 172.21.15.2:0/1644257413 shutdown_connections 2024-05-14T17:13:50.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:02.815+0000 7fc647baf640 1 -- 172.21.15.2:0/1644257413 wait complete. 2024-05-14T17:13:50.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:13:38 smithi123 ceph-mon[35971]: pgmap v595: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 9 op/s 2024-05-14T17:13:50.477 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:13:40 smithi123 ceph-mon[35971]: pgmap v596: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 KiB/s wr, 13 op/s 2024-05-14T17:13:50.528 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:13:42 smithi123 ceph-mon[35971]: pgmap v597: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 10 op/s 2024-05-14T17:13:50.578 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:13:44 smithi123 ceph-mon[35971]: pgmap v598: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 70 KiB/s wr, 16 op/s 2024-05-14T17:13:50.644 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:13:46 smithi123 ceph-mon[35971]: pgmap v599: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 46 KiB/s wr, 10 op/s 2024-05-14T17:13:50.694 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:13:48 smithi123 ceph-mon[35971]: pgmap v600: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 46 KiB/s wr, 10 op/s 2024-05-14T17:13:50.745 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:13:38 smithi151 ceph-mon[36066]: pgmap v595: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 9 op/s 2024-05-14T17:13:50.794 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:13:40 smithi151 ceph-mon[36066]: pgmap v596: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 KiB/s wr, 13 op/s 2024-05-14T17:13:50.844 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:13:42 smithi151 ceph-mon[36066]: pgmap v597: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 10 op/s 2024-05-14T17:13:50.894 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:13:44 smithi151 ceph-mon[36066]: pgmap v598: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 70 KiB/s wr, 16 op/s 2024-05-14T17:13:50.945 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:13:46 smithi151 ceph-mon[36066]: pgmap v599: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 46 KiB/s wr, 10 op/s 2024-05-14T17:13:50.995 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:13:48 smithi151 ceph-mon[36066]: pgmap v600: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 46 KiB/s wr, 10 op/s 2024-05-14T17:13:51.045 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:13:02 smithi002 ceph-mon[31538]: pgmap v575: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 5.1 MiB/s wr, 575 op/s 2024-05-14T17:13:51.094 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:13:03 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:13:51.146 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:13:04 smithi002 ceph-mon[31538]: pgmap v576: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 5.2 MiB/s wr, 602 op/s 2024-05-14T17:13:51.211 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:13:04 smithi002 ceph-mon[31538]: osdmap e104: 12 total, 12 up, 12 in 2024-05-14T17:13:51.261 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:13:06 smithi002 ceph-mon[31538]: pgmap v578: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 4.2 MiB/s wr, 485 op/s 2024-05-14T17:13:51.311 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:13:08 smithi002 ceph-mon[31538]: pgmap v579: 97 pgs: 97 active+clean; 4.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 4.2 MiB/s wr, 485 op/s 2024-05-14T17:13:51.361 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:13:10 smithi002 ceph-mon[31538]: pgmap v580: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 116 KiB/s wr, 42 op/s 2024-05-14T17:13:51.411 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:13:11 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:13:51.478 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:13:12 smithi002 ceph-mon[31538]: pgmap v581: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 116 KiB/s wr, 42 op/s 2024-05-14T17:13:51.528 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:13:12 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:13:51.578 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:13:12 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:13:51.628 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:13:12 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:13:51.678 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:13:12 smithi002 ceph-mon[31538]: osdmap e105: 12 total, 12 up, 12 in 2024-05-14T17:13:51.728 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:13:14 smithi002 ceph-mon[31538]: pgmap v583: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 11 op/s 2024-05-14T17:13:51.780 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:13:50 smithi123 ceph-mon[35971]: pgmap v601: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 46 KiB/s wr, 13 op/s 2024-05-14T17:13:51.829 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:13:50 smithi151 ceph-mon[36066]: pgmap v601: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 46 KiB/s wr, 13 op/s 2024-05-14T17:13:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.915+0000 7f9568ebd640 1 -- 172.21.15.2:0/2158977154 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f956410f430 msgr2=0x7f9564111820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:51.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.915+0000 7f9568ebd640 1 --2- 172.21.15.2:0/2158977154 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f956410f430 0x7f9564111820 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f955c009ca0 tx=0x7f955c02a8c0 comp rx=0 tx=0).stop 2024-05-14T17:13:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.915+0000 7f9568ebd640 1 -- 172.21.15.2:0/2158977154 shutdown_connections 2024-05-14T17:13:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.915+0000 7f9568ebd640 1 --2- 172.21.15.2:0/2158977154 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f956410f430 0x7f9564111820 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:52.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.915+0000 7f9568ebd640 1 --2- 172.21.15.2:0/2158977154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95641021a0 0x7f956410eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:52.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.915+0000 7f9568ebd640 1 --2- 172.21.15.2:0/2158977154 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9564101880 0x7f9564101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:52.179 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.915+0000 7f9568ebd640 1 -- 172.21.15.2:0/2158977154 >> 172.21.15.2:0/2158977154 conn(0x7f9564077cd0 msgr2=0x7f95640780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.915+0000 7f9568ebd640 1 -- 172.21.15.2:0/2158977154 shutdown_connections 2024-05-14T17:13:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.915+0000 7f9568ebd640 1 -- 172.21.15.2:0/2158977154 wait complete. 2024-05-14T17:13:52.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.916+0000 7f9568ebd640 1 Processor -- start 2024-05-14T17:13:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.916+0000 7f9568ebd640 1 -- start start 2024-05-14T17:13:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.916+0000 7f9568ebd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9564101880 0x7f956419b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.916+0000 7f9568ebd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95641021a0 0x7f956419b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:52.545 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.916+0000 7f9568ebd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f956410f430 0x7f956419fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:52.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.916+0000 7f9568ebd640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9564114f40 con 0x7f9564101880 2024-05-14T17:13:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.916+0000 7f9568ebd640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9564114dc0 con 0x7f956410f430 2024-05-14T17:13:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.916+0000 7f9568ebd640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f95641150c0 con 0x7f95641021a0 2024-05-14T17:13:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.916+0000 7f956259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9564101880 0x7f956419b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.916+0000 7f956259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9564101880 0x7f956419b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57878/0 (socket says 172.21.15.2:57878) 2024-05-14T17:13:52.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.916+0000 7f956259c640 1 -- 172.21.15.2:0/3277605374 learned_addr learned my addr 172.21.15.2:0/3277605374 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:13:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.916+0000 7f9561d9b640 1 --2- 172.21.15.2:0/3277605374 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95641021a0 0x7f956419b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:52.952 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.916+0000 7f9562d9d640 1 --2- 172.21.15.2:0/3277605374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f956410f430 0x7f956419fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.916+0000 7f956259c640 1 -- 172.21.15.2:0/3277605374 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95641021a0 msgr2=0x7f956419b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:53.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.916+0000 7f956259c640 1 --2- 172.21.15.2:0/3277605374 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95641021a0 0x7f956419b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.916+0000 7f956259c640 1 -- 172.21.15.2:0/3277605374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f956410f430 msgr2=0x7f956419fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:53.163 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.916+0000 7f956259c640 1 --2- 172.21.15.2:0/3277605374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f956410f430 0x7f956419fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:53.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.916+0000 7f956259c640 1 -- 172.21.15.2:0/3277605374 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95641a02b0 con 0x7f9564101880 2024-05-14T17:13:53.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.917+0000 7f956259c640 1 --2- 172.21.15.2:0/3277605374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9564101880 0x7f956419b2c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f955400c890 tx=0x7f955400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.917+0000 7f95537fe640 1 -- 172.21.15.2:0/3277605374 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9554004220 con 0x7f9564101880 2024-05-14T17:13:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.917+0000 7f9568ebd640 1 -- 172.21.15.2:0/3277605374 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95641a05a0 con 0x7f9564101880 2024-05-14T17:13:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.917+0000 7f95537fe640 1 -- 172.21.15.2:0/3277605374 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95540043c0 con 0x7f9564101880 2024-05-14T17:13:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.917+0000 7f9568ebd640 1 -- 172.21.15.2:0/3277605374 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95641a7e30 con 0x7f9564101880 2024-05-14T17:13:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.917+0000 7f95537fe640 1 -- 172.21.15.2:0/3277605374 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955401d060 con 0x7f9564101880 2024-05-14T17:13:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.920+0000 7f95537fe640 1 -- 172.21.15.2:0/3277605374 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f95540079b0 con 0x7f9564101880 2024-05-14T17:13:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.920+0000 7f956bbd2740 1 Processor -- start 2024-05-14T17:13:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.920+0000 7f956bbd2740 1 -- start start 2024-05-14T17:13:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.920+0000 7f956bbd2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564dd7f1e8f0 0x564dd7f1ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.920+0000 7f956bbd2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564dd7f1f210 0x564dd7f29590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:53.847 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.920+0000 7f956bbd2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564dd7f29ad0 0x564dd7f2bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:53.896 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.920+0000 7f956bbd2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x564dd7e91c30 con 0x564dd7f1f210 2024-05-14T17:13:53.955 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.920+0000 7f956bbd2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564dd7e817d0 con 0x564dd7f29ad0 2024-05-14T17:13:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.920+0000 7f956bbd2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564dd7e85810 con 0x564dd7f1e8f0 2024-05-14T17:13:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.920+0000 7f956259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564dd7f1f210 0x564dd7f29590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.921+0000 7f956259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564dd7f1f210 0x564dd7f29590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57890/0 (socket says 172.21.15.2:57890) 2024-05-14T17:13:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.921+0000 7f956259c640 1 -- 172.21.15.2:0/1649646900 learned_addr learned my addr 172.21.15.2:0/1649646900 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:13:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.921+0000 7f9562d9d640 1 --2- 172.21.15.2:0/1649646900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564dd7f1e8f0 0x564dd7f1ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.921+0000 7f9561d9b640 1 --2- 172.21.15.2:0/1649646900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564dd7f29ad0 0x564dd7f2bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.921+0000 7f95537fe640 1 --2- 172.21.15.2:0/3277605374 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f954c0b2a10 0x7f954c0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.921+0000 7f956259c640 1 -- 172.21.15.2:0/1649646900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564dd7f1e8f0 msgr2=0x564dd7f1ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.921+0000 7f956259c640 1 --2- 172.21.15.2:0/1649646900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564dd7f1e8f0 0x564dd7f1ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:54.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.921+0000 7f956259c640 1 -- 172.21.15.2:0/1649646900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564dd7f29ad0 msgr2=0x564dd7f2bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.921+0000 7f956259c640 1 --2- 172.21.15.2:0/1649646900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564dd7f29ad0 0x564dd7f2bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:54.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.921+0000 7f956259c640 1 -- 172.21.15.2:0/1649646900 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564dd7b96210 con 0x564dd7f1f210 2024-05-14T17:13:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.921+0000 7f9562d9d640 1 --2- 172.21.15.2:0/3277605374 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f954c0b2a10 0x7f954c0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:54.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.921+0000 7f95537fe640 1 -- 172.21.15.2:0/3277605374 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8513+0+0 (secure 0 0 0) 0x7f95540e27e0 con 0x7f9564101880 2024-05-14T17:13:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.921+0000 7f956259c640 1 --2- 172.21.15.2:0/1649646900 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564dd7f1f210 0x564dd7f29590 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f95540dfea0 tx=0x7f955406efb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e248ed99baadf51e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.921+0000 7f9550ff9640 1 -- 172.21.15.2:0/1649646900 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95540a9410 con 0x564dd7f1f210 2024-05-14T17:13:54.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.921+0000 7f9550ff9640 1 -- 172.21.15.2:0/1649646900 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9554005290 con 0x564dd7f1f210 2024-05-14T17:13:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.922+0000 7f9550ff9640 1 -- 172.21.15.2:0/1649646900 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95540a5250 con 0x564dd7f1f210 2024-05-14T17:13:54.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.922+0000 7f9562d9d640 1 --2- 172.21.15.2:0/3277605374 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f954c0b2a10 0x7f954c0b4ed0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f955c005cc0 tx=0x7f955c0095c0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.922+0000 7f956bbd2740 1 -- 172.21.15.2:0/1649646900 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564dd7f1f210 msgr2=0x564dd7f29590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:55.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.922+0000 7f956bbd2740 1 --2- 172.21.15.2:0/1649646900 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564dd7f1f210 0x564dd7f29590 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f95540dfea0 tx=0x7f955406efb0 comp rx=0 tx=0).stop 2024-05-14T17:13:55.130 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.922+0000 7f956bbd2740 1 -- 172.21.15.2:0/1649646900 shutdown_connections 2024-05-14T17:13:55.180 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.922+0000 7f956bbd2740 1 --2- 172.21.15.2:0/1649646900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564dd7f29ad0 0x564dd7f2bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.922+0000 7f956bbd2740 1 --2- 172.21.15.2:0/1649646900 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564dd7f1f210 0x564dd7f29590 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.922+0000 7f956bbd2740 1 --2- 172.21.15.2:0/1649646900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564dd7f1e8f0 0x564dd7f1ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.922+0000 7f956bbd2740 1 -- 172.21.15.2:0/1649646900 >> 172.21.15.2:0/1649646900 conn(0x564dd7f1ae00 msgr2=0x564dd7f1b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.922+0000 7f956bbd2740 1 -- 172.21.15.2:0/1649646900 shutdown_connections 2024-05-14T17:13:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.922+0000 7f956bbd2740 1 -- 172.21.15.2:0/1649646900 wait complete. 2024-05-14T17:13:55.481 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.923+0000 7f956bbd2740 1 -- start start 2024-05-14T17:13:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.923+0000 7f956bbd2740 1 Processor -- start 2024-05-14T17:13:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.923+0000 7f956bbd2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564dd7f1e8f0 0x564dd7f29460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:55.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.923+0000 7f956bbd2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564dd7f1f210 0x564dd7f26750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.923+0000 7f956bbd2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564dd7f29ad0 0x564dd7f26cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:55.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.923+0000 7f956bbd2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x564dd7b67790 con 0x564dd7f1f210 2024-05-14T17:13:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.923+0000 7f956bbd2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564dd7e3d890 con 0x564dd7f29ad0 2024-05-14T17:13:55.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.923+0000 7f956bbd2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564dd7f2c740 con 0x564dd7f1e8f0 2024-05-14T17:13:55.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.923+0000 7f956bbd2740 10 client.0 _ll_register_callbacks cb 0x564dd7e40d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:13:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.923+0000 7f9561d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564dd7f1f210 0x564dd7f26750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.923+0000 7f9561d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564dd7f1f210 0x564dd7f26750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57894/0 (socket says 172.21.15.2:57894) 2024-05-14T17:13:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.923+0000 7f956259c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564dd7f1e8f0 0x564dd7f29460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.923+0000 7f9561d9b640 1 -- 172.21.15.2:0/200361307 learned_addr learned my addr 172.21.15.2:0/200361307 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:13:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.923+0000 7f9562d9d640 1 --2- 172.21.15.2:0/200361307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564dd7f29ad0 0x564dd7f26cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.924+0000 7f9561d9b640 1 -- 172.21.15.2:0/200361307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564dd7f1e8f0 msgr2=0x564dd7f29460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:56.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.924+0000 7f9561d9b640 1 --2- 172.21.15.2:0/200361307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564dd7f1e8f0 0x564dd7f29460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.924+0000 7f9561d9b640 1 -- 172.21.15.2:0/200361307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564dd7f29ad0 msgr2=0x564dd7f26cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:56.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.924+0000 7f9561d9b640 1 --2- 172.21.15.2:0/200361307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564dd7f29ad0 0x564dd7f26cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:56.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.924+0000 7f9561d9b640 1 -- 172.21.15.2:0/200361307 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564dd7c62620 con 0x564dd7f1f210 2024-05-14T17:13:56.448 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.924+0000 7f9561d9b640 1 --2- 172.21.15.2:0/200361307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564dd7f1f210 0x564dd7f26750 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f955800cdb0 tx=0x7f95580039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7ec536e59086ce server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.924+0000 7f9549ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:13:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.924+0000 7f9549ffb640 1 -- 172.21.15.2:0/200361307 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9558011910 con 0x564dd7f1f210 2024-05-14T17:13:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.924+0000 7f9549ffb640 1 -- 172.21.15.2:0/200361307 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8513+0+0 (secure 0 0 0) 0x7f9558011ab0 con 0x564dd7f1f210 2024-05-14T17:13:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.924+0000 7f956bbd2740 1 -- 172.21.15.2:0/200361307 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564dd7e93b80 con 0x564dd7f1f210 2024-05-14T17:13:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.924+0000 7f9549ffb640 1 -- 172.21.15.2:0/200361307 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f9558011cb0 con 0x564dd7f1f210 2024-05-14T17:13:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.924+0000 7f956bbd2740 10 client.17121 fetch_fsmap learned FSMap version 55 2024-05-14T17:13:56.798 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.924+0000 7f956bbd2740 1 -- 172.21.15.2:0/200361307 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x564dd7f38220 con 0x564dd7f1f210 2024-05-14T17:13:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.924+0000 7f9549ffb640 1 -- 172.21.15.2:0/200361307 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564dd7e93b80 con 0x564dd7f1f210 2024-05-14T17:13:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.925+0000 7f9549ffb640 1 -- 172.21.15.2:0/200361307 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 55) ==== 8342+0+0 (secure 0 0 0) 0x7f9558018070 con 0x564dd7f1f210 2024-05-14T17:13:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.925+0000 7f956bbd2740 10 client.17121 fetch_fsmap finished waiting for FSMap version 55 2024-05-14T17:13:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.925+0000 7f956bbd2740 10 client.17121 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:13:57.049 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.925+0000 7f956bbd2740 20 client.17121 populate_metadata read hostname 'smithi002' 2024-05-14T17:13:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.925+0000 7f956bbd2740 1 --2- 172.21.15.2:0/200361307 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564dd7f397a0 0x564dd7f3bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.925+0000 7f956bbd2740 4 client.17121 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:13:57.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.925+0000 7f956bbd2740 1 -- 172.21.15.2:0/200361307 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564dd7e8d360 con 0x564dd7f397a0 2024-05-14T17:13:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.925+0000 7f956259c640 1 --2- 172.21.15.2:0/200361307 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564dd7f397a0 0x564dd7f3bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:57.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.925+0000 7f956259c640 1 --2- 172.21.15.2:0/200361307 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564dd7f397a0 0x564dd7f3bb90 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea03b6c20b9bdc41 server_cookie=118af2aaac57b86e in_seq=0 out_seq=0 2024-05-14T17:13:57.374 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.925+0000 7f9549ffb640 10 client.17121 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:13:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.926+0000 7f9549ffb640 1 -- 172.21.15.2:0/200361307 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564dd7e8d360 con 0x564dd7f397a0 2024-05-14T17:13:57.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.926+0000 7f9549ffb640 10 client.17121 handle_command_reply: tid=0 2024-05-14T17:13:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.926+0000 7f956bbd2740 1 client.17121 shutdown 2024-05-14T17:13:57.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.926+0000 7f956bbd2740 1 -- 172.21.15.2:0/200361307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564dd7f1f210 msgr2=0x564dd7f26750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.926+0000 7f956bbd2740 1 --2- 172.21.15.2:0/200361307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564dd7f1f210 0x564dd7f26750 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f955800cdb0 tx=0x7f95580039c0 comp rx=0 tx=0).stop 2024-05-14T17:13:57.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.926+0000 7f956bbd2740 1 -- 172.21.15.2:0/200361307 shutdown_connections 2024-05-14T17:13:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.926+0000 7f956bbd2740 1 --2- 172.21.15.2:0/200361307 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564dd7f397a0 0x564dd7f3bb90 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:57.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.926+0000 7f956bbd2740 1 --2- 172.21.15.2:0/200361307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564dd7f29ad0 0x564dd7f26cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.926+0000 7f956bbd2740 1 --2- 172.21.15.2:0/200361307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564dd7f1f210 0x564dd7f26750 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:57.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.926+0000 7f9549ffb640 0 client.17121 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:13:57.932 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.926+0000 7f956bbd2740 1 --2- 172.21.15.2:0/200361307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564dd7f1e8f0 0x564dd7f29460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:57.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.926+0000 7f956bbd2740 1 -- 172.21.15.2:0/200361307 >> 172.21.15.2:0/200361307 conn(0x564dd7f1ae00 msgr2=0x564dd7f1a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:58.033 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.926+0000 7f956bbd2740 1 -- 172.21.15.2:0/200361307 shutdown_connections 2024-05-14T17:13:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.926+0000 7f956bbd2740 1 -- 172.21.15.2:0/200361307 wait complete. 2024-05-14T17:13:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.927+0000 7f956bbd2740 20 client.17121 trim_cache size 0 max 16384 2024-05-14T17:13:58.191 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.942+0000 7f956bbd2740 1 Processor -- start 2024-05-14T17:13:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.942+0000 7f956bbd2740 1 -- start start 2024-05-14T17:13:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.942+0000 7f956bbd2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564dd7f1e8f0 0x564dd7f20b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.942+0000 7f956bbd2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564dd7f1f210 0x564dd7f35610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.942+0000 7f956bbd2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564dd7f29ad0 0x564dd7f28f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:58.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.942+0000 7f956bbd2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f95641b3180 con 0x564dd7f29ad0 2024-05-14T17:13:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.942+0000 7f956bbd2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f95641b0a20 con 0x564dd7f1f210 2024-05-14T17:13:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.942+0000 7f956bbd2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f95641ae890 con 0x564dd7f1e8f0 2024-05-14T17:13:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.942+0000 7f9562d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564dd7f29ad0 0x564dd7f28f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.942+0000 7f9562d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564dd7f29ad0 0x564dd7f28f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57904/0 (socket says 172.21.15.2:57904) 2024-05-14T17:13:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.942+0000 7f9562d9d640 1 -- 172.21.15.2:0/3049302931 learned_addr learned my addr 172.21.15.2:0/3049302931 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:13:58.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.942+0000 7f9561d9b640 1 --2- 172.21.15.2:0/3049302931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564dd7f1f210 0x564dd7f35610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.942+0000 7f956259c640 1 --2- 172.21.15.2:0/3049302931 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564dd7f1e8f0 0x564dd7f20b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:13:58.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.942+0000 7f9562d9d640 1 -- 172.21.15.2:0/3049302931 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564dd7f1e8f0 msgr2=0x564dd7f20b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:58.900 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.942+0000 7f9562d9d640 1 --2- 172.21.15.2:0/3049302931 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564dd7f1e8f0 0x564dd7f20b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:58.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.943+0000 7f9562d9d640 1 -- 172.21.15.2:0/3049302931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564dd7f1f210 msgr2=0x564dd7f35610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:13:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.943+0000 7f9562d9d640 1 --2- 172.21.15.2:0/3049302931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564dd7f1f210 0x564dd7f35610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:59.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.943+0000 7f9562d9d640 1 -- 172.21.15.2:0/3049302931 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564dd7f27040 con 0x564dd7f29ad0 2024-05-14T17:13:59.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.943+0000 7f9562d9d640 1 --2- 172.21.15.2:0/3049302931 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564dd7f29ad0 0x564dd7f28f90 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f955c009ca0 tx=0x7f955c041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83db477852f7637b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:13:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.943+0000 7f954bfff640 1 -- 172.21.15.2:0/3049302931 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955c003970 con 0x564dd7f29ad0 2024-05-14T17:13:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.943+0000 7f954bfff640 1 -- 172.21.15.2:0/3049302931 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f955c036040 con 0x564dd7f29ad0 2024-05-14T17:13:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.943+0000 7f954bfff640 1 -- 172.21.15.2:0/3049302931 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955c02f8c0 con 0x564dd7f29ad0 2024-05-14T17:13:59.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.943+0000 7f956bbd2740 1 -- 172.21.15.2:0/3049302931 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564dd7f29ad0 msgr2=0x564dd7f28f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:13:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.943+0000 7f956bbd2740 1 --2- 172.21.15.2:0/3049302931 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564dd7f29ad0 0x564dd7f28f90 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f955c009ca0 tx=0x7f955c041000 comp rx=0 tx=0).stop 2024-05-14T17:13:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.943+0000 7f956bbd2740 1 -- 172.21.15.2:0/3049302931 shutdown_connections 2024-05-14T17:13:59.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.943+0000 7f956bbd2740 1 --2- 172.21.15.2:0/3049302931 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564dd7f29ad0 0x564dd7f28f90 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.943+0000 7f956bbd2740 1 --2- 172.21.15.2:0/3049302931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564dd7f1f210 0x564dd7f35610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.943+0000 7f956bbd2740 1 --2- 172.21.15.2:0/3049302931 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564dd7f1e8f0 0x564dd7f20b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:13:59.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.943+0000 7f956bbd2740 1 -- 172.21.15.2:0/3049302931 >> 172.21.15.2:0/3049302931 conn(0x564dd7f1ae00 msgr2=0x564dd7f1b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:13:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.944+0000 7f956bbd2740 1 -- 172.21.15.2:0/3049302931 shutdown_connections 2024-05-14T17:13:59.742 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.944+0000 7f956bbd2740 1 -- 172.21.15.2:0/3049302931 wait complete. 2024-05-14T17:13:59.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.944+0000 7f956bbd2740 1 -- start start 2024-05-14T17:13:59.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.944+0000 7f956bbd2740 1 Processor -- start 2024-05-14T17:13:59.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.944+0000 7f956bbd2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564dd7f1e8f0 0x564dd7f2cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.944+0000 7f956bbd2740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564dd7f1f210 0x564dd7f2d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:13:59.983 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.944+0000 7f956bbd2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564dd7f29ad0 0x564dd7f9c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.944+0000 7f956bbd2740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f95641aeee0 con 0x564dd7f1f210 2024-05-14T17:14:00.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.944+0000 7f956bbd2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f95641b0350 con 0x564dd7f1e8f0 2024-05-14T17:14:00.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.944+0000 7f956bbd2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f95641af5b0 con 0x564dd7f29ad0 2024-05-14T17:14:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.944+0000 7f956bbd2740 10 client.0 _ll_register_callbacks cb 0x7f955800c9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:14:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.944+0000 7f9561d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564dd7f1f210 0x564dd7f2d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.944+0000 7f9561d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564dd7f1f210 0x564dd7f2d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57908/0 (socket says 172.21.15.2:57908) 2024-05-14T17:14:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.944+0000 7f9561d9b640 1 -- 172.21.15.2:0/675242605 learned_addr learned my addr 172.21.15.2:0/675242605 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:14:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.944+0000 7f956259c640 1 --2- 172.21.15.2:0/675242605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564dd7f1e8f0 0x564dd7f2cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:00.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.944+0000 7f9562d9d640 1 --2- 172.21.15.2:0/675242605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564dd7f29ad0 0x564dd7f9c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.945+0000 7f9561d9b640 1 -- 172.21.15.2:0/675242605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564dd7f29ad0 msgr2=0x564dd7f9c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.945+0000 7f9561d9b640 1 --2- 172.21.15.2:0/675242605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564dd7f29ad0 0x564dd7f9c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:00.484 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.945+0000 7f9561d9b640 1 -- 172.21.15.2:0/675242605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564dd7f1e8f0 msgr2=0x564dd7f2cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.945+0000 7f9561d9b640 1 --2- 172.21.15.2:0/675242605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564dd7f1e8f0 0x564dd7f2cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.945+0000 7f9561d9b640 1 -- 172.21.15.2:0/675242605 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564dd7f9c600 con 0x564dd7f1f210 2024-05-14T17:14:00.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.945+0000 7f9562d9d640 1 --2- 172.21.15.2:0/675242605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564dd7f29ad0 0x564dd7f9c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:14:00.651 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.945+0000 7f9561d9b640 1 --2- 172.21.15.2:0/675242605 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564dd7f1f210 0x564dd7f2d4b0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f9558004220 tx=0x7f95580052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6bda5d3deb13a83 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:00.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.945+0000 7f9549ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:14:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.945+0000 7f9549ffb640 1 -- 172.21.15.2:0/675242605 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9558022910 con 0x564dd7f1f210 2024-05-14T17:14:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.945+0000 7f9549ffb640 1 -- 172.21.15.2:0/675242605 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8513+0+0 (secure 0 0 0) 0x7f9558022ab0 con 0x564dd7f1f210 2024-05-14T17:14:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.945+0000 7f956bbd2740 1 -- 172.21.15.2:0/675242605 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f955c036040 con 0x564dd7f1f210 2024-05-14T17:14:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.945+0000 7f9549ffb640 1 -- 172.21.15.2:0/675242605 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f9558022cb0 con 0x564dd7f1f210 2024-05-14T17:14:00.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.945+0000 7f956bbd2740 10 client.17127 fetch_fsmap learned FSMap version 55 2024-05-14T17:14:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.945+0000 7f956bbd2740 1 -- 172.21.15.2:0/675242605 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x564dd7fa3990 con 0x564dd7f1f210 2024-05-14T17:14:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.945+0000 7f9549ffb640 1 -- 172.21.15.2:0/675242605 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f955c036040 con 0x564dd7f1f210 2024-05-14T17:14:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.946+0000 7f9549ffb640 1 -- 172.21.15.2:0/675242605 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 55) ==== 8342+0+0 (secure 0 0 0) 0x7f9558018030 con 0x564dd7f1f210 2024-05-14T17:14:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.946+0000 7f956bbd2740 10 client.17127 fetch_fsmap finished waiting for FSMap version 55 2024-05-14T17:14:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.946+0000 7f956bbd2740 10 client.17127 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:14:01.193 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.946+0000 7f956bbd2740 20 client.17127 populate_metadata read hostname 'smithi002' 2024-05-14T17:14:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.946+0000 7f956bbd2740 1 --2- 172.21.15.2:0/675242605 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564dd7fa40c0 0x564dd7fa64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.946+0000 7f956bbd2740 4 client.17127 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:14:01.318 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.946+0000 7f956bbd2740 1 -- 172.21.15.2:0/675242605 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564dd7e075f0 con 0x564dd7fa40c0 2024-05-14T17:14:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.946+0000 7f956259c640 1 --2- 172.21.15.2:0/675242605 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564dd7fa40c0 0x564dd7fa64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.946+0000 7f956259c640 1 --2- 172.21.15.2:0/675242605 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564dd7fa40c0 0x564dd7fa64e0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=356f19284bfdb951 server_cookie=1f50f949fcf94f2a in_seq=0 out_seq=0 2024-05-14T17:14:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.946+0000 7f9549ffb640 10 client.17127 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:14:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.962+0000 7f9549ffb640 1 -- 172.21.15.2:0/675242605 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564dd7e075f0 con 0x564dd7fa40c0 2024-05-14T17:14:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.962+0000 7f9549ffb640 10 client.17127 handle_command_reply: tid=0 2024-05-14T17:14:01.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.962+0000 7f956bbd2740 1 client.17127 shutdown 2024-05-14T17:14:01.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.962+0000 7f956bbd2740 1 -- 172.21.15.2:0/675242605 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564dd7f1f210 msgr2=0x564dd7f2d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.962+0000 7f956bbd2740 1 --2- 172.21.15.2:0/675242605 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564dd7f1f210 0x564dd7f2d4b0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f9558004220 tx=0x7f95580052f0 comp rx=0 tx=0).stop 2024-05-14T17:14:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.962+0000 7f956bbd2740 1 -- 172.21.15.2:0/675242605 shutdown_connections 2024-05-14T17:14:01.735 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.962+0000 7f956bbd2740 1 --2- 172.21.15.2:0/675242605 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564dd7fa40c0 0x564dd7fa64e0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.962+0000 7f956bbd2740 1 --2- 172.21.15.2:0/675242605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564dd7f29ad0 0x564dd7f9c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.962+0000 7f956bbd2740 1 --2- 172.21.15.2:0/675242605 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564dd7f1f210 0x564dd7f2d4b0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:01.868 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.962+0000 7f956bbd2740 1 --2- 172.21.15.2:0/675242605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564dd7f1e8f0 0x564dd7f2cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:01.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.962+0000 7f9549ffb640 0 client.17127 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:14:01.960 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.962+0000 7f956bbd2740 1 -- 172.21.15.2:0/675242605 >> 172.21.15.2:0/675242605 conn(0x564dd7f1ae00 msgr2=0x564dd7f1a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:02.002 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.962+0000 7f956bbd2740 1 -- 172.21.15.2:0/675242605 shutdown_connections 2024-05-14T17:14:02.043 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.962+0000 7f956bbd2740 1 -- 172.21.15.2:0/675242605 wait complete. 2024-05-14T17:14:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.962+0000 7f956bbd2740 20 client.17127 trim_cache size 0 max 16384 2024-05-14T17:14:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:14:02.169 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:14:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:14:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.962+0000 7f9568ebd640 1 -- 172.21.15.2:0/3277605374 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f954c0b2a10 msgr2=0x7f954c0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.962+0000 7f9568ebd640 1 --2- 172.21.15.2:0/3277605374 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f954c0b2a10 0x7f954c0b4ed0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f955c005cc0 tx=0x7f955c0095c0 comp rx=0 tx=0).stop 2024-05-14T17:14:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.962+0000 7f9568ebd640 1 -- 172.21.15.2:0/3277605374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9564101880 msgr2=0x7f956419b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.962+0000 7f9568ebd640 1 --2- 172.21.15.2:0/3277605374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9564101880 0x7f956419b2c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f955400c890 tx=0x7f955400cd50 comp rx=0 tx=0).stop 2024-05-14T17:14:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.962+0000 7f9568ebd640 1 -- 172.21.15.2:0/3277605374 shutdown_connections 2024-05-14T17:14:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.962+0000 7f9568ebd640 1 --2- 172.21.15.2:0/3277605374 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f954c0b2a10 0x7f954c0b4ed0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:02.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.962+0000 7f9568ebd640 1 --2- 172.21.15.2:0/3277605374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f956410f430 0x7f956419fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.962+0000 7f9568ebd640 1 --2- 172.21.15.2:0/3277605374 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95641021a0 0x7f956419b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.962+0000 7f9568ebd640 1 --2- 172.21.15.2:0/3277605374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9564101880 0x7f956419b2c0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.962+0000 7f9568ebd640 1 -- 172.21.15.2:0/3277605374 >> 172.21.15.2:0/3277605374 conn(0x7f9564077cd0 msgr2=0x7f9564110740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.962+0000 7f9568ebd640 1 -- 172.21.15.2:0/3277605374 shutdown_connections 2024-05-14T17:14:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:15.962+0000 7f9568ebd640 1 -- 172.21.15.2:0/3277605374 wait complete. 2024-05-14T17:14:02.779 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:13:16 smithi002 ceph-mon[31538]: pgmap v584: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-05-14T17:14:02.820 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:13:18 smithi002 ceph-mon[31538]: pgmap v585: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-05-14T17:14:02.861 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:13:20 smithi002 ceph-mon[31538]: pgmap v586: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-05-14T17:14:02.902 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:13:22 smithi002 ceph-mon[31538]: pgmap v587: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-05-14T17:14:02.944 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:13:24 smithi002 ceph-mon[31538]: pgmap v588: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 36 KiB/s wr, 14 op/s 2024-05-14T17:14:02.986 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:13:26 smithi002 ceph-mon[31538]: pgmap v589: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 40 KiB/s wr, 10 op/s 2024-05-14T17:14:03.027 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:13:28 smithi002 ceph-mon[31538]: pgmap v590: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 40 KiB/s wr, 10 op/s 2024-05-14T17:14:03.070 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:13:52 smithi123 ceph-mon[35971]: pgmap v602: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-05-14T17:14:03.111 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:13:54 smithi123 ceph-mon[35971]: pgmap v603: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 11 op/s 2024-05-14T17:14:03.158 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:13:56 smithi123 ceph-mon[35971]: pgmap v604: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1023 B/s wr, 6 op/s 2024-05-14T17:14:03.227 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:13:58 smithi123 ceph-mon[35971]: pgmap v605: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 1023 B/s wr, 6 op/s 2024-05-14T17:14:03.269 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:14:00 smithi123 ceph-mon[35971]: pgmap v606: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 10 KiB/s wr, 9 op/s 2024-05-14T17:14:03.311 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:13:52 smithi151 ceph-mon[36066]: pgmap v602: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-05-14T17:14:03.353 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:13:54 smithi151 ceph-mon[36066]: pgmap v603: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 11 op/s 2024-05-14T17:14:03.394 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:13:56 smithi151 ceph-mon[36066]: pgmap v604: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1023 B/s wr, 6 op/s 2024-05-14T17:14:03.436 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:13:58 smithi151 ceph-mon[36066]: pgmap v605: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 1023 B/s wr, 6 op/s 2024-05-14T17:14:03.478 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:14:00 smithi151 ceph-mon[36066]: pgmap v606: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 10 KiB/s wr, 9 op/s 2024-05-14T17:14:03.521 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:14:02 smithi123 ceph-mon[35971]: pgmap v607: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 6 op/s 2024-05-14T17:14:03.561 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:14:02 smithi151 ceph-mon[36066]: pgmap v607: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 6 op/s 2024-05-14T17:14:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.065+0000 7f9eead3f640 1 -- 172.21.15.2:0/2044951790 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ee4101970 msgr2=0x7f9ee4101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:03.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.065+0000 7f9eead3f640 1 --2- 172.21.15.2:0/2044951790 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ee4101970 0x7f9ee4101df0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0009930 tx=0x7f9ed002a420 comp rx=0 tx=0).stop 2024-05-14T17:14:03.686 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.065+0000 7f9eead3f640 1 -- 172.21.15.2:0/2044951790 shutdown_connections 2024-05-14T17:14:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.065+0000 7f9eead3f640 1 --2- 172.21.15.2:0/2044951790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ee4102330 0x7f9ee410e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:03.794 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.065+0000 7f9eead3f640 1 --2- 172.21.15.2:0/2044951790 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ee4101970 0x7f9ee4101df0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.065+0000 7f9eead3f640 1 --2- 172.21.15.2:0/2044951790 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ee4107980 0x7f9ee4107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.065+0000 7f9eead3f640 1 -- 172.21.15.2:0/2044951790 >> 172.21.15.2:0/2044951790 conn(0x7f9ee4077cd0 msgr2=0x7f9ee40780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.065+0000 7f9eead3f640 1 -- 172.21.15.2:0/2044951790 shutdown_connections 2024-05-14T17:14:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.065+0000 7f9eead3f640 1 -- 172.21.15.2:0/2044951790 wait complete. 2024-05-14T17:14:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.065+0000 7f9eead3f640 1 Processor -- start 2024-05-14T17:14:04.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.066+0000 7f9eead3f640 1 -- start start 2024-05-14T17:14:04.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.066+0000 7f9eead3f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ee4101970 0x7f9ee4199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.066+0000 7f9eead3f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ee4102330 0x7f9ee41997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:04.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.066+0000 7f9eead3f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ee4107980 0x7f9ee419db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:04.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.066+0000 7f9eead3f640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee4111c40 con 0x7f9ee4102330 2024-05-14T17:14:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.066+0000 7f9eead3f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee4111ac0 con 0x7f9ee4101970 2024-05-14T17:14:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.066+0000 7f9eead3f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee4111dc0 con 0x7f9ee4107980 2024-05-14T17:14:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.066+0000 7f9ee3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ee4102330 0x7f9ee41997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:04.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.066+0000 7f9ee3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ee4102330 0x7f9ee41997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60556/0 (socket says 172.21.15.2:60556) 2024-05-14T17:14:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.066+0000 7f9ee3fff640 1 -- 172.21.15.2:0/1294432266 learned_addr learned my addr 172.21.15.2:0/1294432266 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:14:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.066+0000 7f9ee8adb640 1 --2- 172.21.15.2:0/1294432266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ee4101970 0x7f9ee4199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:04.620 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.066+0000 7f9ee92dc640 1 --2- 172.21.15.2:0/1294432266 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ee4107980 0x7f9ee419db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:04.662 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.066+0000 7f9ee3fff640 1 -- 172.21.15.2:0/1294432266 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ee4107980 msgr2=0x7f9ee419db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.066+0000 7f9ee3fff640 1 --2- 172.21.15.2:0/1294432266 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ee4107980 0x7f9ee419db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:04.753 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.066+0000 7f9ee3fff640 1 -- 172.21.15.2:0/1294432266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ee4101970 msgr2=0x7f9ee4199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.066+0000 7f9ee3fff640 1 --2- 172.21.15.2:0/1294432266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ee4101970 0x7f9ee4199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:04.837 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.066+0000 7f9ee3fff640 1 -- 172.21.15.2:0/1294432266 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee419e250 con 0x7f9ee4102330 2024-05-14T17:14:04.878 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.067+0000 7f9ee3fff640 1 --2- 172.21.15.2:0/1294432266 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ee4102330 0x7f9ee41997a0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0005940 tx=0x7f9ed00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.067+0000 7f9ee1ffb640 1 -- 172.21.15.2:0/1294432266 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed003f030 con 0x7f9ee4102330 2024-05-14T17:14:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.067+0000 7f9ee1ffb640 1 -- 172.21.15.2:0/1294432266 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ed0003cb0 con 0x7f9ee4102330 2024-05-14T17:14:05.004 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.067+0000 7f9eead3f640 1 -- 172.21.15.2:0/1294432266 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ee419e4e0 con 0x7f9ee4102330 2024-05-14T17:14:05.045 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.067+0000 7f9ee1ffb640 1 -- 172.21.15.2:0/1294432266 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0035070 con 0x7f9ee4102330 2024-05-14T17:14:05.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.067+0000 7f9eead3f640 1 -- 172.21.15.2:0/1294432266 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ee41a5dd0 con 0x7f9ee4102330 2024-05-14T17:14:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.070+0000 7f9eeda58740 1 Processor -- start 2024-05-14T17:14:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.070+0000 7f9eeda58740 1 -- start start 2024-05-14T17:14:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.070+0000 7f9ee1ffb640 1 -- 172.21.15.2:0/1294432266 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f9ed0042030 con 0x7f9ee4102330 2024-05-14T17:14:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.070+0000 7f9eeda58740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646a3f127f0 0x5646a3f12bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.070+0000 7f9eeda58740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a3f13110 0x5646a3f219a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.070+0000 7f9eeda58740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5646a3f21ee0 0x5646a3f242d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.070+0000 7f9eeda58740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5646a3e85c30 con 0x5646a3f21ee0 2024-05-14T17:14:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.070+0000 7f9eeda58740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5646a3e757d0 con 0x5646a3f13110 2024-05-14T17:14:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.070+0000 7f9eeda58740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5646a3e79810 con 0x5646a3f127f0 2024-05-14T17:14:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.070+0000 7f9ee3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5646a3f21ee0 0x5646a3f242d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.070+0000 7f9ee3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5646a3f21ee0 0x5646a3f242d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60572/0 (socket says 172.21.15.2:60572) 2024-05-14T17:14:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.070+0000 7f9ee3fff640 1 -- 172.21.15.2:0/206681289 learned_addr learned my addr 172.21.15.2:0/206681289 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:14:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.071+0000 7f9ee8adb640 1 --2- 172.21.15.2:0/206681289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a3f13110 0x5646a3f219a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.071+0000 7f9ee92dc640 1 --2- 172.21.15.2:0/206681289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646a3f127f0 0x5646a3f12bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.071+0000 7f9ee3fff640 1 -- 172.21.15.2:0/206681289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646a3f127f0 msgr2=0x5646a3f12bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.071+0000 7f9ee3fff640 1 --2- 172.21.15.2:0/206681289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646a3f127f0 0x5646a3f12bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.071+0000 7f9ee3fff640 1 -- 172.21.15.2:0/206681289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a3f13110 msgr2=0x5646a3f219a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:05.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.071+0000 7f9ee3fff640 1 --2- 172.21.15.2:0/206681289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a3f13110 0x5646a3f219a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.071+0000 7f9ee3fff640 1 -- 172.21.15.2:0/206681289 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646a3b8a210 con 0x5646a3f21ee0 2024-05-14T17:14:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.071+0000 7f9ee92dc640 1 --2- 172.21.15.2:0/206681289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646a3f127f0 0x5646a3f12bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:14:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.071+0000 7f9ee1ffb640 1 --2- 172.21.15.2:0/1294432266 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f9ed40b2a10 0x7f9ed40b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.071+0000 7f9ee3fff640 1 --2- 172.21.15.2:0/206681289 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5646a3f21ee0 0x5646a3f242d0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0100460 tx=0x7f9ed010b0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a58be56073ac75 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.071+0000 7f9ee92dc640 1 --2- 172.21.15.2:0/1294432266 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f9ed40b2a10 0x7f9ed40b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:06.054 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.071+0000 7f9eceffd640 1 -- 172.21.15.2:0/206681289 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed00c5ff0 con 0x5646a3f21ee0 2024-05-14T17:14:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.071+0000 7f9ee1ffb640 1 -- 172.21.15.2:0/1294432266 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8513+0+0 (secure 0 0 0) 0x7f9ed0102da0 con 0x7f9ee4102330 2024-05-14T17:14:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.071+0000 7f9eceffd640 1 -- 172.21.15.2:0/206681289 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ed0110070 con 0x5646a3f21ee0 2024-05-14T17:14:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.071+0000 7f9eceffd640 1 -- 172.21.15.2:0/206681289 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed00c98a0 con 0x5646a3f21ee0 2024-05-14T17:14:06.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.072+0000 7f9ee92dc640 1 --2- 172.21.15.2:0/1294432266 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f9ed40b2a10 0x7f9ed40b4ed0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9edc005fd0 tx=0x7f9edc009500 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.072+0000 7f9eeda58740 1 -- 172.21.15.2:0/206681289 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5646a3f21ee0 msgr2=0x5646a3f242d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:06.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.072+0000 7f9eeda58740 1 --2- 172.21.15.2:0/206681289 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5646a3f21ee0 0x5646a3f242d0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0100460 tx=0x7f9ed010b0e0 comp rx=0 tx=0).stop 2024-05-14T17:14:06.288 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.072+0000 7f9eeda58740 1 -- 172.21.15.2:0/206681289 shutdown_connections 2024-05-14T17:14:06.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.072+0000 7f9eeda58740 1 --2- 172.21.15.2:0/206681289 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5646a3f21ee0 0x5646a3f242d0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:06.396 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.072+0000 7f9eeda58740 1 --2- 172.21.15.2:0/206681289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a3f13110 0x5646a3f219a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.072+0000 7f9eeda58740 1 --2- 172.21.15.2:0/206681289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646a3f127f0 0x5646a3f12bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:06.485 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.072+0000 7f9eeda58740 1 -- 172.21.15.2:0/206681289 >> 172.21.15.2:0/206681289 conn(0x5646a3f0ee00 msgr2=0x5646a3f0f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:06.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.072+0000 7f9eeda58740 1 -- 172.21.15.2:0/206681289 shutdown_connections 2024-05-14T17:14:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.072+0000 7f9eeda58740 1 -- 172.21.15.2:0/206681289 wait complete. 2024-05-14T17:14:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.073+0000 7f9eeda58740 1 -- start start 2024-05-14T17:14:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.073+0000 7f9eeda58740 1 Processor -- start 2024-05-14T17:14:06.688 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.073+0000 7f9eeda58740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5646a3f127f0 0x5646a3f21810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.073+0000 7f9eeda58740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646a3f13110 0x5646a3f1eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:06.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.073+0000 7f9eeda58740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a3f21ee0 0x5646a3f1f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.073+0000 7f9eeda58740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5646a3b5b790 con 0x5646a3f127f0 2024-05-14T17:14:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.073+0000 7f9eeda58740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5646a3e31890 con 0x5646a3f21ee0 2024-05-14T17:14:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.073+0000 7f9eeda58740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5646a3f248d0 con 0x5646a3f13110 2024-05-14T17:14:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.073+0000 7f9eeda58740 10 client.0 _ll_register_callbacks cb 0x5646a3e34d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:14:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.073+0000 7f9ee8adb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5646a3f127f0 0x5646a3f21810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.073+0000 7f9ee8adb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5646a3f127f0 0x5646a3f21810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60578/0 (socket says 172.21.15.2:60578) 2024-05-14T17:14:07.005 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.073+0000 7f9ee8adb640 1 -- 172.21.15.2:0/2980825999 learned_addr learned my addr 172.21.15.2:0/2980825999 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:14:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.074+0000 7f9ee92dc640 1 --2- 172.21.15.2:0/2980825999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a3f21ee0 0x5646a3f1f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.074+0000 7f9ee3fff640 1 --2- 172.21.15.2:0/2980825999 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646a3f13110 0x5646a3f1eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.074+0000 7f9ee8adb640 1 -- 172.21.15.2:0/2980825999 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646a3f13110 msgr2=0x5646a3f1eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.074+0000 7f9ee8adb640 1 --2- 172.21.15.2:0/2980825999 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646a3f13110 0x5646a3f1eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.074+0000 7f9ee8adb640 1 -- 172.21.15.2:0/2980825999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a3f21ee0 msgr2=0x5646a3f1f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.074+0000 7f9ee8adb640 1 --2- 172.21.15.2:0/2980825999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a3f21ee0 0x5646a3f1f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.074+0000 7f9ee8adb640 1 -- 172.21.15.2:0/2980825999 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646a3c56620 con 0x5646a3f127f0 2024-05-14T17:14:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.074+0000 7f9ee8adb640 1 --2- 172.21.15.2:0/2980825999 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5646a3f127f0 0x5646a3f21810 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f9ed800b990 tx=0x7f9ed800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f79d8815304320e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:07.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.074+0000 7f9ec3fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:14:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.074+0000 7f9ec3fff640 1 -- 172.21.15.2:0/2980825999 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed8003e00 con 0x5646a3f127f0 2024-05-14T17:14:07.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.074+0000 7f9ec3fff640 1 -- 172.21.15.2:0/2980825999 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8513+0+0 (secure 0 0 0) 0x7f9ed8005020 con 0x5646a3f127f0 2024-05-14T17:14:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.074+0000 7f9eeda58740 1 -- 172.21.15.2:0/2980825999 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ed0110070 con 0x5646a3f127f0 2024-05-14T17:14:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.074+0000 7f9ec3fff640 1 -- 172.21.15.2:0/2980825999 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f9ed8005220 con 0x5646a3f127f0 2024-05-14T17:14:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.074+0000 7f9eeda58740 10 client.17139 fetch_fsmap learned FSMap version 56 2024-05-14T17:14:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.074+0000 7f9eeda58740 1 -- 172.21.15.2:0/2980825999 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x5646a3f303e0 con 0x5646a3f127f0 2024-05-14T17:14:07.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.075+0000 7f9ec3fff640 1 -- 172.21.15.2:0/2980825999 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ed0110070 con 0x5646a3f127f0 2024-05-14T17:14:07.655 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.075+0000 7f9ec3fff640 1 -- 172.21.15.2:0/2980825999 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 56) ==== 8346+0+0 (secure 0 0 0) 0x7f9ed801b590 con 0x5646a3f127f0 2024-05-14T17:14:07.689 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.075+0000 7f9eeda58740 10 client.17139 fetch_fsmap finished waiting for FSMap version 56 2024-05-14T17:14:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.075+0000 7f9eeda58740 10 client.17139 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:14:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.075+0000 7f9eeda58740 20 client.17139 populate_metadata read hostname 'smithi002' 2024-05-14T17:14:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.075+0000 7f9eeda58740 1 --2- 172.21.15.2:0/2980825999 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5646a3f318f0 0x5646a3f33d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.075+0000 7f9eeda58740 4 client.17139 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:14:07.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.075+0000 7f9eeda58740 1 -- 172.21.15.2:0/2980825999 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646a3e87b80 con 0x5646a3f318f0 2024-05-14T17:14:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.075+0000 7f9ee3fff640 1 --2- 172.21.15.2:0/2980825999 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5646a3f318f0 0x5646a3f33d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.076+0000 7f9ec3fff640 10 client.17139 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:14:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.076+0000 7f9ee3fff640 1 --2- 172.21.15.2:0/2980825999 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5646a3f318f0 0x5646a3f33d40 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f785893561d3d9ff server_cookie=af0c892783d19caa in_seq=0 out_seq=0 2024-05-14T17:14:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.076+0000 7f9ec3fff640 1 -- 172.21.15.2:0/2980825999 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5646a3e87b80 con 0x5646a3f318f0 2024-05-14T17:14:08.039 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.076+0000 7f9ec3fff640 10 client.17139 handle_command_reply: tid=0 2024-05-14T17:14:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.076+0000 7f9eeda58740 1 client.17139 shutdown 2024-05-14T17:14:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.077+0000 7f9eeda58740 1 -- 172.21.15.2:0/2980825999 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5646a3f127f0 msgr2=0x5646a3f21810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.077+0000 7f9eeda58740 1 --2- 172.21.15.2:0/2980825999 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5646a3f127f0 0x5646a3f21810 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f9ed800b990 tx=0x7f9ed800be50 comp rx=0 tx=0).stop 2024-05-14T17:14:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.077+0000 7f9eeda58740 1 -- 172.21.15.2:0/2980825999 shutdown_connections 2024-05-14T17:14:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.077+0000 7f9eeda58740 1 --2- 172.21.15.2:0/2980825999 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5646a3f318f0 0x5646a3f33d40 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:08.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.077+0000 7f9eeda58740 1 --2- 172.21.15.2:0/2980825999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a3f21ee0 0x5646a3f1f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:08.289 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.077+0000 7f9eeda58740 1 --2- 172.21.15.2:0/2980825999 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646a3f13110 0x5646a3f1eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.077+0000 7f9ec3fff640 0 client.17139 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:14:08.356 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.077+0000 7f9eeda58740 1 --2- 172.21.15.2:0/2980825999 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5646a3f127f0 0x5646a3f21810 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:08.406 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.077+0000 7f9eeda58740 1 -- 172.21.15.2:0/2980825999 >> 172.21.15.2:0/2980825999 conn(0x5646a3f0ee00 msgr2=0x5646a3f14bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.077+0000 7f9eeda58740 1 -- 172.21.15.2:0/2980825999 shutdown_connections 2024-05-14T17:14:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.077+0000 7f9eeda58740 1 -- 172.21.15.2:0/2980825999 wait complete. 2024-05-14T17:14:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.077+0000 7f9eeda58740 20 client.17139 trim_cache size 0 max 16384 2024-05-14T17:14:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.092+0000 7f9eeda58740 1 Processor -- start 2024-05-14T17:14:08.581 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.092+0000 7f9eeda58740 1 -- start start 2024-05-14T17:14:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.092+0000 7f9eeda58740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a3f127f0 0x5646a3f21340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.092+0000 7f9eeda58740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5646a3f13110 0x5646a3f14a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:08.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.092+0000 7f9eeda58740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646a3f21ee0 0x5646a3f14f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.092+0000 7f9eeda58740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41aed00 con 0x5646a3f13110 2024-05-14T17:14:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.092+0000 7f9eeda58740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41ac5a0 con 0x5646a3f127f0 2024-05-14T17:14:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.092+0000 7f9eeda58740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41aa430 con 0x5646a3f21ee0 2024-05-14T17:14:08.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.093+0000 7f9ee3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5646a3f13110 0x5646a3f14a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.093+0000 7f9ee3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5646a3f13110 0x5646a3f14a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60590/0 (socket says 172.21.15.2:60590) 2024-05-14T17:14:08.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.093+0000 7f9ee3fff640 1 -- 172.21.15.2:0/1724971142 learned_addr learned my addr 172.21.15.2:0/1724971142 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:14:09.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.093+0000 7f9ee92dc640 1 --2- 172.21.15.2:0/1724971142 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646a3f21ee0 0x5646a3f14f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.093+0000 7f9ee8adb640 1 --2- 172.21.15.2:0/1724971142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a3f127f0 0x5646a3f21340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:09.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.093+0000 7f9ee3fff640 1 -- 172.21.15.2:0/1724971142 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646a3f21ee0 msgr2=0x5646a3f14f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.093+0000 7f9ee3fff640 1 --2- 172.21.15.2:0/1724971142 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646a3f21ee0 0x5646a3f14f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.093+0000 7f9ee3fff640 1 -- 172.21.15.2:0/1724971142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a3f127f0 msgr2=0x5646a3f21340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:09.207 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.093+0000 7f9ee3fff640 1 --2- 172.21.15.2:0/1724971142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a3f127f0 0x5646a3f21340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.093+0000 7f9ee3fff640 1 -- 172.21.15.2:0/1724971142 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646a3f13650 con 0x5646a3f13110 2024-05-14T17:14:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.093+0000 7f9ee3fff640 1 --2- 172.21.15.2:0/1724971142 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5646a3f13110 0x5646a3f14a50 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0000c00 tx=0x7f9ed0106400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8c3eb6c2a1d5d94 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:09.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.093+0000 7f9ece7fc640 1 -- 172.21.15.2:0/1724971142 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0049040 con 0x5646a3f13110 2024-05-14T17:14:09.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.093+0000 7f9ece7fc640 1 -- 172.21.15.2:0/1724971142 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ed003b460 con 0x5646a3f13110 2024-05-14T17:14:09.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.093+0000 7f9ece7fc640 1 -- 172.21.15.2:0/1724971142 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed00c9a80 con 0x5646a3f13110 2024-05-14T17:14:09.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.094+0000 7f9eeda58740 1 -- 172.21.15.2:0/1724971142 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5646a3f13110 msgr2=0x5646a3f14a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.094+0000 7f9eeda58740 1 --2- 172.21.15.2:0/1724971142 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5646a3f13110 0x5646a3f14a50 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0000c00 tx=0x7f9ed0106400 comp rx=0 tx=0).stop 2024-05-14T17:14:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.094+0000 7f9eeda58740 1 -- 172.21.15.2:0/1724971142 shutdown_connections 2024-05-14T17:14:09.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.094+0000 7f9eeda58740 1 --2- 172.21.15.2:0/1724971142 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646a3f21ee0 0x5646a3f14f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:09.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.094+0000 7f9eeda58740 1 --2- 172.21.15.2:0/1724971142 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5646a3f13110 0x5646a3f14a50 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.094+0000 7f9eeda58740 1 --2- 172.21.15.2:0/1724971142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a3f127f0 0x5646a3f21340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.094+0000 7f9eeda58740 1 -- 172.21.15.2:0/1724971142 >> 172.21.15.2:0/1724971142 conn(0x5646a3f0ee00 msgr2=0x5646a3f14340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.094+0000 7f9eeda58740 1 -- 172.21.15.2:0/1724971142 shutdown_connections 2024-05-14T17:14:09.699 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.094+0000 7f9eeda58740 1 -- 172.21.15.2:0/1724971142 wait complete. 2024-05-14T17:14:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.094+0000 7f9eeda58740 1 -- start start 2024-05-14T17:14:09.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.094+0000 7f9eeda58740 1 Processor -- start 2024-05-14T17:14:09.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.095+0000 7f9eeda58740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5646a3f127f0 0x5646a3f257e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:09.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.095+0000 7f9eeda58740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a3f13110 0x5646a3f25d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:09.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.095+0000 7f9eeda58740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646a3f21ee0 0x5646a3f26260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:09.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.095+0000 7f9eeda58740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41aaa80 con 0x5646a3f127f0 2024-05-14T17:14:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.095+0000 7f9eeda58740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41abed0 con 0x5646a3f13110 2024-05-14T17:14:09.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.095+0000 7f9eeda58740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41ab130 con 0x5646a3f21ee0 2024-05-14T17:14:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.095+0000 7f9eeda58740 10 client.0 _ll_register_callbacks cb 0x7f9ed80045b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:14:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.095+0000 7f9ee8adb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5646a3f127f0 0x5646a3f257e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.095+0000 7f9ee8adb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5646a3f127f0 0x5646a3f257e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60596/0 (socket says 172.21.15.2:60596) 2024-05-14T17:14:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.095+0000 7f9ee8adb640 1 -- 172.21.15.2:0/560983830 learned_addr learned my addr 172.21.15.2:0/560983830 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:14:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.095+0000 7f9ee3fff640 1 --2- 172.21.15.2:0/560983830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a3f13110 0x5646a3f25d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.095+0000 7f9ee92dc640 1 --2- 172.21.15.2:0/560983830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646a3f21ee0 0x5646a3f26260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.095+0000 7f9ee8adb640 1 -- 172.21.15.2:0/560983830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646a3f21ee0 msgr2=0x5646a3f26260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:10.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.095+0000 7f9ee8adb640 1 --2- 172.21.15.2:0/560983830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646a3f21ee0 0x5646a3f26260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.095+0000 7f9ee8adb640 1 -- 172.21.15.2:0/560983830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a3f13110 msgr2=0x5646a3f25d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.095+0000 7f9ee8adb640 1 --2- 172.21.15.2:0/560983830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a3f13110 0x5646a3f25d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:10.399 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.095+0000 7f9ee8adb640 1 -- 172.21.15.2:0/560983830 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646a3f9b2b0 con 0x5646a3f127f0 2024-05-14T17:14:10.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.095+0000 7f9ee92dc640 1 --2- 172.21.15.2:0/560983830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646a3f21ee0 0x5646a3f26260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:14:10.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.096+0000 7f9ee8adb640 1 --2- 172.21.15.2:0/560983830 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5646a3f127f0 0x5646a3f257e0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f9ed801b960 tx=0x7f9ed800b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e0f4b9e241f6ecf server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:10.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.096+0000 7f9ec3fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:14:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.096+0000 7f9ec3fff640 1 -- 172.21.15.2:0/560983830 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed8031030 con 0x5646a3f127f0 2024-05-14T17:14:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.096+0000 7f9ec3fff640 1 -- 172.21.15.2:0/560983830 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8513+0+0 (secure 0 0 0) 0x7f9ed8017040 con 0x5646a3f127f0 2024-05-14T17:14:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.096+0000 7f9eeda58740 1 -- 172.21.15.2:0/560983830 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5646a3e81360 con 0x5646a3f127f0 2024-05-14T17:14:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.096+0000 7f9ec3fff640 1 -- 172.21.15.2:0/560983830 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f9ed8000e60 con 0x5646a3f127f0 2024-05-14T17:14:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.096+0000 7f9eeda58740 10 client.17145 fetch_fsmap learned FSMap version 56 2024-05-14T17:14:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.096+0000 7f9eeda58740 1 -- 172.21.15.2:0/560983830 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x5646a3f9c5a0 con 0x5646a3f127f0 2024-05-14T17:14:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.096+0000 7f9ec3fff640 1 -- 172.21.15.2:0/560983830 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5646a3e81360 con 0x5646a3f127f0 2024-05-14T17:14:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.096+0000 7f9ec3fff640 1 -- 172.21.15.2:0/560983830 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 56) ==== 8346+0+0 (secure 0 0 0) 0x7f9ed80118f0 con 0x5646a3f127f0 2024-05-14T17:14:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.096+0000 7f9eeda58740 10 client.17145 fetch_fsmap finished waiting for FSMap version 56 2024-05-14T17:14:10.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.096+0000 7f9eeda58740 10 client.17145 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:14:10.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.096+0000 7f9eeda58740 20 client.17145 populate_metadata read hostname 'smithi002' 2024-05-14T17:14:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.096+0000 7f9eeda58740 1 --2- 172.21.15.2:0/560983830 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5646a3f9cd10 0x5646a3f9f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.096+0000 7f9eeda58740 4 client.17145 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:14:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.096+0000 7f9eeda58740 1 -- 172.21.15.2:0/560983830 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5646a3dfb5f0 con 0x5646a3f9cd10 2024-05-14T17:14:11.025 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.097+0000 7f9ee3fff640 1 --2- 172.21.15.2:0/560983830 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5646a3f9cd10 0x5646a3f9f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.097+0000 7f9ec3fff640 10 client.17145 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:14:11.175 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.097+0000 7f9ee3fff640 1 --2- 172.21.15.2:0/560983830 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5646a3f9cd10 0x5646a3f9f130 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59bb4d3c48ababa1 server_cookie=b0269709aa74471f in_seq=0 out_seq=0 2024-05-14T17:14:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.107+0000 7f9ec3fff640 1 -- 172.21.15.2:0/560983830 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5646a3dfb5f0 con 0x5646a3f9cd10 2024-05-14T17:14:11.242 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.107+0000 7f9ec3fff640 10 client.17145 handle_command_reply: tid=0 2024-05-14T17:14:11.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.107+0000 7f9eeda58740 1 client.17145 shutdown 2024-05-14T17:14:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.107+0000 7f9eeda58740 1 -- 172.21.15.2:0/560983830 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5646a3f127f0 msgr2=0x5646a3f257e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:11.342 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.107+0000 7f9eeda58740 1 --2- 172.21.15.2:0/560983830 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5646a3f127f0 0x5646a3f257e0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f9ed801b960 tx=0x7f9ed800b400 comp rx=0 tx=0).stop 2024-05-14T17:14:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.107+0000 7f9eeda58740 1 -- 172.21.15.2:0/560983830 shutdown_connections 2024-05-14T17:14:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.107+0000 7f9eeda58740 1 --2- 172.21.15.2:0/560983830 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5646a3f9cd10 0x5646a3f9f130 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.107+0000 7f9eeda58740 1 --2- 172.21.15.2:0/560983830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646a3f21ee0 0x5646a3f26260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.107+0000 7f9ec3fff640 0 client.17145 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:14:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.107+0000 7f9eeda58740 1 --2- 172.21.15.2:0/560983830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646a3f13110 0x5646a3f25d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.107+0000 7f9eeda58740 1 --2- 172.21.15.2:0/560983830 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5646a3f127f0 0x5646a3f257e0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:11.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.107+0000 7f9eeda58740 1 -- 172.21.15.2:0/560983830 >> 172.21.15.2:0/560983830 conn(0x5646a3f0ee00 msgr2=0x5646a3f14bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:11.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.107+0000 7f9eeda58740 1 -- 172.21.15.2:0/560983830 shutdown_connections 2024-05-14T17:14:11.659 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.107+0000 7f9eeda58740 1 -- 172.21.15.2:0/560983830 wait complete. 2024-05-14T17:14:11.692 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.107+0000 7f9eeda58740 20 client.17145 trim_cache size 0 max 16384 2024-05-14T17:14:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:14:11.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:14:11.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:14:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.108+0000 7f9eead3f640 1 -- 172.21.15.2:0/1294432266 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f9ed40b2a10 msgr2=0x7f9ed40b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.108+0000 7f9eead3f640 1 --2- 172.21.15.2:0/1294432266 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f9ed40b2a10 0x7f9ed40b4ed0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9edc005fd0 tx=0x7f9edc009500 comp rx=0 tx=0).stop 2024-05-14T17:14:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.108+0000 7f9eead3f640 1 -- 172.21.15.2:0/1294432266 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ee4102330 msgr2=0x7f9ee41997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.108+0000 7f9eead3f640 1 --2- 172.21.15.2:0/1294432266 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ee4102330 0x7f9ee41997a0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0005940 tx=0x7f9ed00042f0 comp rx=0 tx=0).stop 2024-05-14T17:14:11.992 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.108+0000 7f9eead3f640 1 -- 172.21.15.2:0/1294432266 shutdown_connections 2024-05-14T17:14:12.026 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.108+0000 7f9eead3f640 1 --2- 172.21.15.2:0/1294432266 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f9ed40b2a10 0x7f9ed40b4ed0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.108+0000 7f9eead3f640 1 --2- 172.21.15.2:0/1294432266 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ee4107980 0x7f9ee419db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.108+0000 7f9eead3f640 1 --2- 172.21.15.2:0/1294432266 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ee4102330 0x7f9ee41997a0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.108+0000 7f9eead3f640 1 --2- 172.21.15.2:0/1294432266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ee4101970 0x7f9ee4199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.108+0000 7f9eead3f640 1 -- 172.21.15.2:0/1294432266 >> 172.21.15.2:0/1294432266 conn(0x7f9ee4077cd0 msgr2=0x7f9ee406b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:12.201 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.108+0000 7f9eead3f640 1 -- 172.21.15.2:0/1294432266 shutdown_connections 2024-05-14T17:14:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:29.108+0000 7f9eead3f640 1 -- 172.21.15.2:0/1294432266 wait complete. 2024-05-14T17:14:12.286 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:13:29 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:14:12.318 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:13:30 smithi002 ceph-mon[31538]: pgmap v591: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-05-14T17:14:12.351 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:13:32 smithi002 ceph-mon[31538]: pgmap v592: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 25 KiB/s wr, 5 op/s 2024-05-14T17:14:12.384 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:13:34 smithi002 ceph-mon[31538]: pgmap v593: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 49 KiB/s wr, 12 op/s 2024-05-14T17:14:12.418 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:13:36 smithi002 ceph-mon[31538]: pgmap v594: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 44 KiB/s wr, 10 op/s 2024-05-14T17:14:12.451 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:13:38 smithi002 ceph-mon[31538]: pgmap v595: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 9 op/s 2024-05-14T17:14:12.485 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:13:40 smithi002 ceph-mon[31538]: pgmap v596: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 KiB/s wr, 13 op/s 2024-05-14T17:14:12.520 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:14:04 smithi123 ceph-mon[35971]: pgmap v608: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 55 KiB/s wr, 26 op/s 2024-05-14T17:14:12.568 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:14:06 smithi123 ceph-mon[35971]: pgmap v609: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 55 KiB/s wr, 23 op/s 2024-05-14T17:14:12.601 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:14:08 smithi123 ceph-mon[35971]: pgmap v610: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 55 KiB/s wr, 23 op/s 2024-05-14T17:14:12.637 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:14:09 smithi123 ceph-mon[35971]: osdmap e106: 12 total, 12 up, 12 in 2024-05-14T17:14:12.668 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:14:10 smithi123 ceph-mon[35971]: pgmap v612: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-05-14T17:14:12.701 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:14:04 smithi151 ceph-mon[36066]: pgmap v608: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 55 KiB/s wr, 26 op/s 2024-05-14T17:14:12.735 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:14:06 smithi151 ceph-mon[36066]: pgmap v609: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 55 KiB/s wr, 23 op/s 2024-05-14T17:14:12.774 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:14:08 smithi151 ceph-mon[36066]: pgmap v610: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 55 KiB/s wr, 23 op/s 2024-05-14T17:14:12.820 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:14:09 smithi151 ceph-mon[36066]: osdmap e106: 12 total, 12 up, 12 in 2024-05-14T17:14:12.860 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:14:10 smithi151 ceph-mon[36066]: pgmap v612: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-05-14T17:14:12.906 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.209+0000 7ffa3cdc9640 1 -- 172.21.15.2:0/318476809 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa38101980 msgr2=0x7ffa38101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.209+0000 7ffa3cdc9640 1 --2- 172.21.15.2:0/318476809 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa38101980 0x7ffa38101e00 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ffa20009930 tx=0x7ffa2002a420 comp rx=0 tx=0).stop 2024-05-14T17:14:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.210+0000 7ffa3cdc9640 1 -- 172.21.15.2:0/318476809 shutdown_connections 2024-05-14T17:14:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.210+0000 7ffa3cdc9640 1 --2- 172.21.15.2:0/318476809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa38102340 0x7ffa3810e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.210+0000 7ffa3cdc9640 1 --2- 172.21.15.2:0/318476809 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa38101980 0x7ffa38101e00 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:13.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.210+0000 7ffa3cdc9640 1 --2- 172.21.15.2:0/318476809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa38107940 0x7ffa38107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:13.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.210+0000 7ffa3cdc9640 1 -- 172.21.15.2:0/318476809 >> 172.21.15.2:0/318476809 conn(0x7ffa38078ad0 msgr2=0x7ffa38078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.210+0000 7ffa3cdc9640 1 -- 172.21.15.2:0/318476809 shutdown_connections 2024-05-14T17:14:13.177 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.210+0000 7ffa3cdc9640 1 -- 172.21.15.2:0/318476809 wait complete. 2024-05-14T17:14:13.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.210+0000 7ffa3cdc9640 1 Processor -- start 2024-05-14T17:14:13.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.211+0000 7ffa3cdc9640 1 -- start start 2024-05-14T17:14:13.277 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.211+0000 7ffa3cdc9640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa38101980 0x7ffa380fe910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.211+0000 7ffa3cdc9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa38102340 0x7ffa380fee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:13.344 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.211+0000 7ffa3cdc9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa38107940 0x7ffa380ff390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:13.377 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.211+0000 7ffa3cdc9640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ffa38111c00 con 0x7ffa38107940 2024-05-14T17:14:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.211+0000 7ffa3cdc9640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffa38111a80 con 0x7ffa38101980 2024-05-14T17:14:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.211+0000 7ffa3cdc9640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffa38111d80 con 0x7ffa38102340 2024-05-14T17:14:13.477 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.211+0000 7ffa36d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa38107940 0x7ffa380ff390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.211+0000 7ffa36d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa38107940 0x7ffa380ff390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49142/0 (socket says 172.21.15.2:49142) 2024-05-14T17:14:13.544 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.211+0000 7ffa36d9d640 1 -- 172.21.15.2:0/2786610330 learned_addr learned my addr 172.21.15.2:0/2786610330 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:14:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.211+0000 7ffa3659c640 1 --2- 172.21.15.2:0/2786610330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa38101980 0x7ffa380fe910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:13.610 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.211+0000 7ffa2dd9b640 1 --2- 172.21.15.2:0/2786610330 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa38102340 0x7ffa380fee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.212+0000 7ffa36d9d640 1 -- 172.21.15.2:0/2786610330 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa38102340 msgr2=0x7ffa380fee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:13.685 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.212+0000 7ffa36d9d640 1 --2- 172.21.15.2:0/2786610330 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa38102340 0x7ffa380fee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.212+0000 7ffa36d9d640 1 -- 172.21.15.2:0/2786610330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa38101980 msgr2=0x7ffa380fe910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.212+0000 7ffa36d9d640 1 --2- 172.21.15.2:0/2786610330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa38101980 0x7ffa380fe910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.212+0000 7ffa36d9d640 1 -- 172.21.15.2:0/2786610330 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa381abb50 con 0x7ffa38107940 2024-05-14T17:14:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.212+0000 7ffa36d9d640 1 --2- 172.21.15.2:0/2786610330 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa38107940 0x7ffa380ff390 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ffa3000eaa0 tx=0x7ffa3000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.212+0000 7ffa2ffff640 1 -- 172.21.15.2:0/2786610330 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa30004d20 con 0x7ffa38107940 2024-05-14T17:14:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.212+0000 7ffa3cdc9640 1 -- 172.21.15.2:0/2786610330 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa381abd80 con 0x7ffa38107940 2024-05-14T17:14:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.212+0000 7ffa2ffff640 1 -- 172.21.15.2:0/2786610330 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa30005020 con 0x7ffa38107940 2024-05-14T17:14:13.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.212+0000 7ffa2ffff640 1 -- 172.21.15.2:0/2786610330 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa300116d0 con 0x7ffa38107940 2024-05-14T17:14:14.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.212+0000 7ffa3cdc9640 1 -- 172.21.15.2:0/2786610330 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa381ac250 con 0x7ffa38107940 2024-05-14T17:14:14.061 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.215+0000 7ffa2ffff640 1 -- 172.21.15.2:0/2786610330 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7ffa3000caf0 con 0x7ffa38107940 2024-05-14T17:14:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.215+0000 7ffa3fbb6740 1 Processor -- start 2024-05-14T17:14:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.215+0000 7ffa3fbb6740 1 -- start start 2024-05-14T17:14:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.215+0000 7ffa3fbb6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578a60d08f0 0x5578a60d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:14.194 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.215+0000 7ffa3fbb6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578a60d1210 0x5578a60db590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:14.227 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.215+0000 7ffa3fbb6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578a60dbad0 0x5578a60ddec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:14.269 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.215+0000 7ffa3fbb6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5578a6043c30 con 0x5578a60d08f0 2024-05-14T17:14:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.215+0000 7ffa3fbb6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5578a60337d0 con 0x5578a60dbad0 2024-05-14T17:14:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.215+0000 7ffa3fbb6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5578a6037810 con 0x5578a60d1210 2024-05-14T17:14:14.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.215+0000 7ffa2dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578a60d08f0 0x5578a60d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:14.403 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.215+0000 7ffa2dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578a60d08f0 0x5578a60d0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49154/0 (socket says 172.21.15.2:49154) 2024-05-14T17:14:14.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.215+0000 7ffa2dd9b640 1 -- 172.21.15.2:0/306797596 learned_addr learned my addr 172.21.15.2:0/306797596 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:14:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.216+0000 7ffa3659c640 1 --2- 172.21.15.2:0/306797596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578a60d1210 0x5578a60db590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:14.503 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.216+0000 7ffa2dd9b640 1 --2- 172.21.15.2:0/306797596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578a60dbad0 0x5578a60ddec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:14.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.216+0000 7ffa2dd9b640 1 -- 172.21.15.2:0/306797596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578a60d1210 msgr2=0x5578a60db590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:14.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.216+0000 7ffa2dd9b640 1 --2- 172.21.15.2:0/306797596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578a60d1210 0x5578a60db590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.216+0000 7ffa2dd9b640 1 -- 172.21.15.2:0/306797596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578a60dbad0 msgr2=0x5578a60ddec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:14.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.216+0000 7ffa2dd9b640 1 --2- 172.21.15.2:0/306797596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578a60dbad0 0x5578a60ddec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:14.703 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.216+0000 7ffa2ffff640 1 --2- 172.21.15.2:0/2786610330 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ffa240b2a10 0x7ffa240b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.216+0000 7ffa2dd9b640 1 -- 172.21.15.2:0/306797596 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578a5d48210 con 0x5578a60d08f0 2024-05-14T17:14:14.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.216+0000 7ffa3659c640 1 --2- 172.21.15.2:0/2786610330 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ffa240b2a10 0x7ffa240b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.216+0000 7ffa2ffff640 1 -- 172.21.15.2:0/2786610330 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8513+0+0 (secure 0 0 0) 0x7ffa300e26d0 con 0x7ffa38107940 2024-05-14T17:14:14.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.216+0000 7ffa2dd9b640 1 --2- 172.21.15.2:0/306797596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578a60d08f0 0x5578a60d0cd0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7ffa20009900 tx=0x7ffa20005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c99a50ae3e1041 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:14.920 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.216+0000 7ffa2cd99640 1 -- 172.21.15.2:0/306797596 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa20044030 con 0x5578a60d08f0 2024-05-14T17:14:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.216+0000 7ffa2cd99640 1 -- 172.21.15.2:0/306797596 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa20031670 con 0x5578a60d08f0 2024-05-14T17:14:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.216+0000 7ffa2cd99640 1 -- 172.21.15.2:0/306797596 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa20035040 con 0x5578a60d08f0 2024-05-14T17:14:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.217+0000 7ffa3fbb6740 1 -- 172.21.15.2:0/306797596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578a60d08f0 msgr2=0x5578a60d0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:15.070 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.217+0000 7ffa3659c640 1 --2- 172.21.15.2:0/2786610330 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ffa240b2a10 0x7ffa240b4ed0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x5578a60d1e90 tx=0x7ffa2800a2d0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.217+0000 7ffa3fbb6740 1 --2- 172.21.15.2:0/306797596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578a60d08f0 0x5578a60d0cd0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7ffa20009900 tx=0x7ffa20005fd0 comp rx=0 tx=0).stop 2024-05-14T17:14:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.217+0000 7ffa3fbb6740 1 -- 172.21.15.2:0/306797596 shutdown_connections 2024-05-14T17:14:15.170 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.217+0000 7ffa3fbb6740 1 --2- 172.21.15.2:0/306797596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578a60dbad0 0x5578a60ddec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.217+0000 7ffa3fbb6740 1 --2- 172.21.15.2:0/306797596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578a60d1210 0x5578a60db590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.217+0000 7ffa3fbb6740 1 --2- 172.21.15.2:0/306797596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578a60d08f0 0x5578a60d0cd0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:15.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.217+0000 7ffa3fbb6740 1 -- 172.21.15.2:0/306797596 >> 172.21.15.2:0/306797596 conn(0x5578a60cce00 msgr2=0x5578a60cd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:15.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.217+0000 7ffa3fbb6740 1 -- 172.21.15.2:0/306797596 shutdown_connections 2024-05-14T17:14:15.353 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.217+0000 7ffa3fbb6740 1 -- 172.21.15.2:0/306797596 wait complete. 2024-05-14T17:14:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.217+0000 7ffa3fbb6740 1 -- start start 2024-05-14T17:14:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.218+0000 7ffa3fbb6740 1 Processor -- start 2024-05-14T17:14:15.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.218+0000 7ffa3fbb6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578a60d08f0 0x5578a60db3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:15.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.218+0000 7ffa3fbb6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578a60d1210 0x5578a60d8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:15.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.218+0000 7ffa3fbb6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578a60dbad0 0x5578a60d8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.218+0000 7ffa3fbb6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5578a5d19790 con 0x5578a60d08f0 2024-05-14T17:14:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.218+0000 7ffa3fbb6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5578a5fef890 con 0x5578a60dbad0 2024-05-14T17:14:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.218+0000 7ffa3fbb6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5578a60de740 con 0x5578a60d1210 2024-05-14T17:14:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.218+0000 7ffa3fbb6740 10 client.0 _ll_register_callbacks cb 0x5578a5ff2d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:14:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.218+0000 7ffa3659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578a60d08f0 0x5578a60db3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.218+0000 7ffa3659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578a60d08f0 0x5578a60db3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49162/0 (socket says 172.21.15.2:49162) 2024-05-14T17:14:15.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.218+0000 7ffa3659c640 1 -- 172.21.15.2:0/1805195595 learned_addr learned my addr 172.21.15.2:0/1805195595 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:14:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.218+0000 7ffa2dd9b640 1 --2- 172.21.15.2:0/1805195595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578a60d1210 0x5578a60d8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.218+0000 7ffa36d9d640 1 --2- 172.21.15.2:0/1805195595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578a60dbad0 0x5578a60d8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.218+0000 7ffa3659c640 1 -- 172.21.15.2:0/1805195595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578a60d1210 msgr2=0x5578a60d8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.218+0000 7ffa3659c640 1 --2- 172.21.15.2:0/1805195595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578a60d1210 0x5578a60d8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.218+0000 7ffa3659c640 1 -- 172.21.15.2:0/1805195595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578a60dbad0 msgr2=0x5578a60d8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.218+0000 7ffa3659c640 1 --2- 172.21.15.2:0/1805195595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578a60dbad0 0x5578a60d8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.218+0000 7ffa3659c640 1 -- 172.21.15.2:0/1805195595 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578a5e14620 con 0x5578a60d08f0 2024-05-14T17:14:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.219+0000 7ffa3659c640 1 --2- 172.21.15.2:0/1805195595 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578a60d08f0 0x5578a60db3a0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7ffa2801cc10 tx=0x7ffa28004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad65bc452ee2c9c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.219+0000 7ffa1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:14:16.187 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.219+0000 7ffa1dffb640 1 -- 172.21.15.2:0/1805195595 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2802c060 con 0x5578a60d08f0 2024-05-14T17:14:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.219+0000 7ffa1dffb640 1 -- 172.21.15.2:0/1805195595 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8513+0+0 (secure 0 0 0) 0x7ffa28016030 con 0x5578a60d08f0 2024-05-14T17:14:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.219+0000 7ffa3fbb6740 1 -- 172.21.15.2:0/1805195595 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa20031670 con 0x5578a60d08f0 2024-05-14T17:14:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.219+0000 7ffa1dffb640 1 -- 172.21.15.2:0/1805195595 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7ffa2801cd00 con 0x5578a60d08f0 2024-05-14T17:14:16.321 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.219+0000 7ffa3fbb6740 10 client.17157 fetch_fsmap learned FSMap version 56 2024-05-14T17:14:16.354 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.219+0000 7ffa3fbb6740 1 -- 172.21.15.2:0/1805195595 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x5578a60ea1b0 con 0x5578a60d08f0 2024-05-14T17:14:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.219+0000 7ffa1dffb640 1 -- 172.21.15.2:0/1805195595 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa20031670 con 0x5578a60d08f0 2024-05-14T17:14:16.421 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.219+0000 7ffa1dffb640 1 -- 172.21.15.2:0/1805195595 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 56) ==== 8346+0+0 (secure 0 0 0) 0x7ffa2801a6f0 con 0x5578a60d08f0 2024-05-14T17:14:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.219+0000 7ffa3fbb6740 10 client.17157 fetch_fsmap finished waiting for FSMap version 56 2024-05-14T17:14:16.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.219+0000 7ffa3fbb6740 10 client.17157 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:14:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.219+0000 7ffa3fbb6740 20 client.17157 populate_metadata read hostname 'smithi002' 2024-05-14T17:14:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.219+0000 7ffa3fbb6740 1 --2- 172.21.15.2:0/1805195595 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5578a60eb6c0 0x5578a60edab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.219+0000 7ffa3fbb6740 4 client.17157 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:14:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.219+0000 7ffa3fbb6740 1 -- 172.21.15.2:0/1805195595 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578a6045b80 con 0x5578a60eb6c0 2024-05-14T17:14:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.220+0000 7ffa2dd9b640 1 --2- 172.21.15.2:0/1805195595 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5578a60eb6c0 0x5578a60edab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.220+0000 7ffa2dd9b640 1 --2- 172.21.15.2:0/1805195595 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5578a60eb6c0 0x5578a60edab0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d33bb88c9ec64be server_cookie=df352c68bb1d0d7 in_seq=0 out_seq=0 2024-05-14T17:14:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.220+0000 7ffa1dffb640 10 client.17157 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:14:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.221+0000 7ffa1dffb640 1 -- 172.21.15.2:0/1805195595 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5578a6045b80 con 0x5578a60eb6c0 2024-05-14T17:14:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.221+0000 7ffa1dffb640 10 client.17157 handle_command_reply: tid=0 2024-05-14T17:14:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.221+0000 7ffa3fbb6740 1 client.17157 shutdown 2024-05-14T17:14:16.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.221+0000 7ffa3fbb6740 1 -- 172.21.15.2:0/1805195595 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578a60d08f0 msgr2=0x5578a60db3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:16.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.221+0000 7ffa3fbb6740 1 --2- 172.21.15.2:0/1805195595 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578a60d08f0 0x5578a60db3a0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7ffa2801cc10 tx=0x7ffa28004730 comp rx=0 tx=0).stop 2024-05-14T17:14:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.221+0000 7ffa3fbb6740 1 -- 172.21.15.2:0/1805195595 shutdown_connections 2024-05-14T17:14:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.221+0000 7ffa3fbb6740 1 --2- 172.21.15.2:0/1805195595 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5578a60eb6c0 0x5578a60edab0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:17.046 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.221+0000 7ffa1dffb640 0 client.17157 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:14:17.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.221+0000 7ffa3fbb6740 1 --2- 172.21.15.2:0/1805195595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578a60dbad0 0x5578a60d8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:17.125 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.221+0000 7ffa3fbb6740 1 --2- 172.21.15.2:0/1805195595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578a60d1210 0x5578a60d8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:17.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.221+0000 7ffa3fbb6740 1 --2- 172.21.15.2:0/1805195595 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578a60d08f0 0x5578a60db3a0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.221+0000 7ffa3fbb6740 1 -- 172.21.15.2:0/1805195595 >> 172.21.15.2:0/1805195595 conn(0x5578a60cce00 msgr2=0x5578a60cc100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.221+0000 7ffa3fbb6740 1 -- 172.21.15.2:0/1805195595 shutdown_connections 2024-05-14T17:14:17.296 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.221+0000 7ffa3fbb6740 1 -- 172.21.15.2:0/1805195595 wait complete. 2024-05-14T17:14:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.221+0000 7ffa3fbb6740 20 client.17157 trim_cache size 0 max 16384 2024-05-14T17:14:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.237+0000 7ffa3fbb6740 1 Processor -- start 2024-05-14T17:14:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.237+0000 7ffa3fbb6740 1 -- start start 2024-05-14T17:14:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.237+0000 7ffa3fbb6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578a60d08f0 0x5578a60e7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.237+0000 7ffa3fbb6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578a60d1210 0x5578a60f33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:17.511 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.237+0000 7ffa3fbb6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578a60dbad0 0x5578a60f38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:17.547 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.237+0000 7ffa3fbb6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ffa381b7410 con 0x5578a60d08f0 2024-05-14T17:14:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.237+0000 7ffa3fbb6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffa381b4cb0 con 0x5578a60d1210 2024-05-14T17:14:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.237+0000 7ffa3fbb6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffa381b2b20 con 0x5578a60dbad0 2024-05-14T17:14:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.237+0000 7ffa3659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578a60d08f0 0x5578a60e7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.237+0000 7ffa3659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578a60d08f0 0x5578a60e7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49176/0 (socket says 172.21.15.2:49176) 2024-05-14T17:14:17.714 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.237+0000 7ffa3659c640 1 -- 172.21.15.2:0/1447015445 learned_addr learned my addr 172.21.15.2:0/1447015445 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:14:17.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.237+0000 7ffa36d9d640 1 --2- 172.21.15.2:0/1447015445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578a60dbad0 0x5578a60f38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.238+0000 7ffa2dd9b640 1 --2- 172.21.15.2:0/1447015445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578a60d1210 0x5578a60f33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.238+0000 7ffa3659c640 1 -- 172.21.15.2:0/1447015445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578a60dbad0 msgr2=0x5578a60f38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.238+0000 7ffa3659c640 1 --2- 172.21.15.2:0/1447015445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578a60dbad0 0x5578a60f38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.238+0000 7ffa3659c640 1 -- 172.21.15.2:0/1447015445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578a60d1210 msgr2=0x5578a60f33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.238+0000 7ffa3659c640 1 --2- 172.21.15.2:0/1447015445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578a60d1210 0x5578a60f33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.238+0000 7ffa3659c640 1 -- 172.21.15.2:0/1447015445 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578a60d7640 con 0x5578a60d08f0 2024-05-14T17:14:18.030 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.238+0000 7ffa2dd9b640 1 --2- 172.21.15.2:0/1447015445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578a60d1210 0x5578a60f33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:14:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.238+0000 7ffa3659c640 1 --2- 172.21.15.2:0/1447015445 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578a60d08f0 0x5578a60e7640 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7ffa28017040 tx=0x7ffa28005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b2ea47319bb0c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.238+0000 7ffa1ffff640 1 -- 172.21.15.2:0/1447015445 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2800a9f0 con 0x5578a60d08f0 2024-05-14T17:14:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.238+0000 7ffa1ffff640 1 -- 172.21.15.2:0/1447015445 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa2803d070 con 0x5578a60d08f0 2024-05-14T17:14:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.238+0000 7ffa1ffff640 1 -- 172.21.15.2:0/1447015445 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2800fcc0 con 0x5578a60d08f0 2024-05-14T17:14:18.197 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.239+0000 7ffa3fbb6740 1 -- 172.21.15.2:0/1447015445 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578a60d08f0 msgr2=0x5578a60e7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:18.231 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.239+0000 7ffa3fbb6740 1 --2- 172.21.15.2:0/1447015445 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578a60d08f0 0x5578a60e7640 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7ffa28017040 tx=0x7ffa28005b70 comp rx=0 tx=0).stop 2024-05-14T17:14:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.239+0000 7ffa3fbb6740 1 -- 172.21.15.2:0/1447015445 shutdown_connections 2024-05-14T17:14:18.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.239+0000 7ffa3fbb6740 1 --2- 172.21.15.2:0/1447015445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578a60dbad0 0x5578a60f38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:18.339 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.239+0000 7ffa3fbb6740 1 --2- 172.21.15.2:0/1447015445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578a60d1210 0x5578a60f33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.239+0000 7ffa3fbb6740 1 --2- 172.21.15.2:0/1447015445 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578a60d08f0 0x5578a60e7640 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.239+0000 7ffa3fbb6740 1 -- 172.21.15.2:0/1447015445 >> 172.21.15.2:0/1447015445 conn(0x5578a60cce00 msgr2=0x5578a60cc100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:18.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.239+0000 7ffa3fbb6740 1 -- 172.21.15.2:0/1447015445 shutdown_connections 2024-05-14T17:14:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.239+0000 7ffa3fbb6740 1 -- 172.21.15.2:0/1447015445 wait complete. 2024-05-14T17:14:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.239+0000 7ffa3fbb6740 1 -- start start 2024-05-14T17:14:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.239+0000 7ffa3fbb6740 1 Processor -- start 2024-05-14T17:14:18.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.239+0000 7ffa3fbb6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578a60d08f0 0x5578a60e3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:18.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.239+0000 7ffa3fbb6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578a60d1210 0x5578a60e1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:18.673 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.239+0000 7ffa3fbb6740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578a60dbad0 0x5578a60e1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.239+0000 7ffa3fbb6740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ffa381b3170 con 0x5578a60dbad0 2024-05-14T17:14:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.239+0000 7ffa3fbb6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffa381b45e0 con 0x5578a60d1210 2024-05-14T17:14:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.239+0000 7ffa3fbb6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffa381b3840 con 0x5578a60d08f0 2024-05-14T17:14:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.239+0000 7ffa3fbb6740 10 client.0 _ll_register_callbacks cb 0x7ffa2801c890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:14:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.239+0000 7ffa36d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578a60dbad0 0x5578a60e1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.240+0000 7ffa36d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578a60dbad0 0x5578a60e1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:49190/0 (socket says 172.21.15.2:49190) 2024-05-14T17:14:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.240+0000 7ffa36d9d640 1 -- 172.21.15.2:0/443100071 learned_addr learned my addr 172.21.15.2:0/443100071 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:14:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.240+0000 7ffa3659c640 1 --2- 172.21.15.2:0/443100071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578a60d08f0 0x5578a60e3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.240+0000 7ffa2dd9b640 1 --2- 172.21.15.2:0/443100071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578a60d1210 0x5578a60e1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.240+0000 7ffa36d9d640 1 -- 172.21.15.2:0/443100071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578a60d08f0 msgr2=0x5578a60e3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.240+0000 7ffa36d9d640 1 --2- 172.21.15.2:0/443100071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578a60d08f0 0x5578a60e3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.240+0000 7ffa36d9d640 1 -- 172.21.15.2:0/443100071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578a60d1210 msgr2=0x5578a60e1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.240+0000 7ffa36d9d640 1 --2- 172.21.15.2:0/443100071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578a60d1210 0x5578a60e1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.240+0000 7ffa36d9d640 1 -- 172.21.15.2:0/443100071 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578a60e2180 con 0x5578a60dbad0 2024-05-14T17:14:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.240+0000 7ffa36d9d640 1 --2- 172.21.15.2:0/443100071 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578a60dbad0 0x5578a60e1980 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7ffa3000efd0 tx=0x7ffa300a9250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd2af525aa050ba server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:19.273 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.240+0000 7ffa1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:14:19.306 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.240+0000 7ffa1dffb640 1 -- 172.21.15.2:0/443100071 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3001f040 con 0x5578a60dbad0 2024-05-14T17:14:19.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.240+0000 7ffa1dffb640 1 -- 172.21.15.2:0/443100071 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8513+0+0 (secure 0 0 0) 0x7ffa300a5fa0 con 0x5578a60dbad0 2024-05-14T17:14:19.373 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.240+0000 7ffa3fbb6740 1 -- 172.21.15.2:0/443100071 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa2803d070 con 0x5578a60dbad0 2024-05-14T17:14:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.240+0000 7ffa1dffb640 1 -- 172.21.15.2:0/443100071 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7ffa300aa0b0 con 0x5578a60dbad0 2024-05-14T17:14:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.240+0000 7ffa3fbb6740 10 client.17163 fetch_fsmap learned FSMap version 56 2024-05-14T17:14:19.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.240+0000 7ffa3fbb6740 1 -- 172.21.15.2:0/443100071 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x5578a6155fb0 con 0x5578a60dbad0 2024-05-14T17:14:19.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.240+0000 7ffa1dffb640 1 -- 172.21.15.2:0/443100071 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa2803d070 con 0x5578a60dbad0 2024-05-14T17:14:19.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.241+0000 7ffa1dffb640 1 -- 172.21.15.2:0/443100071 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 56) ==== 8346+0+0 (secure 0 0 0) 0x7ffa300c6050 con 0x5578a60dbad0 2024-05-14T17:14:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.241+0000 7ffa3fbb6740 10 client.17163 fetch_fsmap finished waiting for FSMap version 56 2024-05-14T17:14:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.241+0000 7ffa3fbb6740 10 client.17163 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:14:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.241+0000 7ffa3fbb6740 20 client.17163 populate_metadata read hostname 'smithi002' 2024-05-14T17:14:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.241+0000 7ffa3fbb6740 1 --2- 172.21.15.2:0/443100071 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5578a60eb6c0 0x5578a61586f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.241+0000 7ffa3fbb6740 4 client.17163 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:14:19.800 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.241+0000 7ffa3fbb6740 1 -- 172.21.15.2:0/443100071 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5578a603f360 con 0x5578a60eb6c0 2024-05-14T17:14:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.241+0000 7ffa3659c640 1 --2- 172.21.15.2:0/443100071 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5578a60eb6c0 0x5578a61586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:19.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.241+0000 7ffa3659c640 1 --2- 172.21.15.2:0/443100071 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5578a60eb6c0 0x5578a61586f0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cb776d60eda37be server_cookie=e5aa61741322453f in_seq=0 out_seq=0 2024-05-14T17:14:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.241+0000 7ffa1dffb640 10 client.17163 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:14:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.252+0000 7ffa1dffb640 1 -- 172.21.15.2:0/443100071 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5578a603f360 con 0x5578a60eb6c0 2024-05-14T17:14:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.252+0000 7ffa1dffb640 10 client.17163 handle_command_reply: tid=0 2024-05-14T17:14:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.252+0000 7ffa3fbb6740 1 client.17163 shutdown 2024-05-14T17:14:20.092 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.252+0000 7ffa3fbb6740 1 -- 172.21.15.2:0/443100071 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578a60dbad0 msgr2=0x5578a60e1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.252+0000 7ffa3fbb6740 1 --2- 172.21.15.2:0/443100071 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578a60dbad0 0x5578a60e1980 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7ffa3000efd0 tx=0x7ffa300a9250 comp rx=0 tx=0).stop 2024-05-14T17:14:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.252+0000 7ffa3fbb6740 1 -- 172.21.15.2:0/443100071 shutdown_connections 2024-05-14T17:14:20.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.252+0000 7ffa3fbb6740 1 --2- 172.21.15.2:0/443100071 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x5578a60eb6c0 0x5578a61586f0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.252+0000 7ffa3fbb6740 1 --2- 172.21.15.2:0/443100071 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5578a60dbad0 0x5578a60e1980 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.252+0000 7ffa1dffb640 0 client.17163 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:14:20.351 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.252+0000 7ffa3fbb6740 1 --2- 172.21.15.2:0/443100071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578a60d1210 0x5578a60e1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.252+0000 7ffa3fbb6740 1 --2- 172.21.15.2:0/443100071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5578a60d08f0 0x5578a60e3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.252+0000 7ffa3fbb6740 1 -- 172.21.15.2:0/443100071 >> 172.21.15.2:0/443100071 conn(0x5578a60cce00 msgr2=0x5578a60e6630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:20.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.252+0000 7ffa3fbb6740 1 -- 172.21.15.2:0/443100071 shutdown_connections 2024-05-14T17:14:20.517 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.252+0000 7ffa3fbb6740 1 -- 172.21.15.2:0/443100071 wait complete. 2024-05-14T17:14:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.252+0000 7ffa3fbb6740 20 client.17163 trim_cache size 0 max 16384 2024-05-14T17:14:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:14:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:14:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:14:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.253+0000 7ffa3cdc9640 1 -- 172.21.15.2:0/2786610330 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ffa240b2a10 msgr2=0x7ffa240b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.253+0000 7ffa3cdc9640 1 --2- 172.21.15.2:0/2786610330 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ffa240b2a10 0x7ffa240b4ed0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x5578a60d1e90 tx=0x7ffa2800a2d0 comp rx=0 tx=0).stop 2024-05-14T17:14:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.253+0000 7ffa3cdc9640 1 -- 172.21.15.2:0/2786610330 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa38107940 msgr2=0x7ffa380ff390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.253+0000 7ffa3cdc9640 1 --2- 172.21.15.2:0/2786610330 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa38107940 0x7ffa380ff390 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ffa3000eaa0 tx=0x7ffa3000eea0 comp rx=0 tx=0).stop 2024-05-14T17:14:20.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.253+0000 7ffa3cdc9640 1 -- 172.21.15.2:0/2786610330 shutdown_connections 2024-05-14T17:14:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.253+0000 7ffa3cdc9640 1 --2- 172.21.15.2:0/2786610330 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ffa240b2a10 0x7ffa240b4ed0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:21.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.253+0000 7ffa3cdc9640 1 --2- 172.21.15.2:0/2786610330 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa38107940 0x7ffa380ff390 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.253+0000 7ffa3cdc9640 1 --2- 172.21.15.2:0/2786610330 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa38102340 0x7ffa380fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.253+0000 7ffa3cdc9640 1 --2- 172.21.15.2:0/2786610330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa38101980 0x7ffa380fe910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.253+0000 7ffa3cdc9640 1 -- 172.21.15.2:0/2786610330 >> 172.21.15.2:0/2786610330 conn(0x7ffa38078ad0 msgr2=0x7ffa3806a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.253+0000 7ffa3cdc9640 1 -- 172.21.15.2:0/2786610330 shutdown_connections 2024-05-14T17:14:21.276 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:42.253+0000 7ffa3cdc9640 1 -- 172.21.15.2:0/2786610330 wait complete. 2024-05-14T17:14:21.319 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:13:42 smithi002 ceph-mon[31538]: pgmap v597: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 10 op/s 2024-05-14T17:14:21.360 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:13:44 smithi002 ceph-mon[31538]: pgmap v598: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 70 KiB/s wr, 16 op/s 2024-05-14T17:14:21.401 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:13:46 smithi002 ceph-mon[31538]: pgmap v599: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 46 KiB/s wr, 10 op/s 2024-05-14T17:14:21.443 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:13:48 smithi002 ceph-mon[31538]: pgmap v600: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 46 KiB/s wr, 10 op/s 2024-05-14T17:14:21.485 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:13:50 smithi002 ceph-mon[31538]: pgmap v601: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 46 KiB/s wr, 13 op/s 2024-05-14T17:14:21.527 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:13:52 smithi002 ceph-mon[31538]: pgmap v602: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 9 op/s 2024-05-14T17:14:21.568 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:13:54 smithi002 ceph-mon[31538]: pgmap v603: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 11 op/s 2024-05-14T17:14:21.612 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:14:12 smithi123 ceph-mon[35971]: pgmap v613: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-05-14T17:14:21.652 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:14:12 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:14:21.702 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:14:13 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:14:21.744 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:14:13 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:14:21.785 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:14:13 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:14:21.827 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:14:14 smithi123 ceph-mon[35971]: pgmap v614: 97 pgs: 97 active+clean; 3.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-05-14T17:14:21.869 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:14:16 smithi123 ceph-mon[35971]: pgmap v615: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-05-14T17:14:21.953 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:14:18 smithi123 ceph-mon[35971]: osdmap e107: 12 total, 12 up, 12 in 2024-05-14T17:14:21.999 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:14:18 smithi123 ceph-mon[35971]: pgmap v617: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 43 KiB/s wr, 14 op/s 2024-05-14T17:14:22.036 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:14:20 smithi123 ceph-mon[35971]: pgmap v618: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 38 KiB/s wr, 16 op/s 2024-05-14T17:14:22.078 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:14:12 smithi151 ceph-mon[36066]: pgmap v613: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-05-14T17:14:22.119 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:14:12 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:14:22.161 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:14:13 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:14:22.203 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:14:13 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:14:22.244 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:14:13 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:14:22.278 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:14:14 smithi151 ceph-mon[36066]: pgmap v614: 97 pgs: 97 active+clean; 3.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-05-14T17:14:22.319 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:14:16 smithi151 ceph-mon[36066]: pgmap v615: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-05-14T17:14:22.361 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:14:18 smithi151 ceph-mon[36066]: osdmap e107: 12 total, 12 up, 12 in 2024-05-14T17:14:22.411 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:14:18 smithi151 ceph-mon[36066]: pgmap v617: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 43 KiB/s wr, 14 op/s 2024-05-14T17:14:22.453 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:14:20 smithi151 ceph-mon[36066]: pgmap v618: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 38 KiB/s wr, 16 op/s 2024-05-14T17:14:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.354+0000 7fb0ba4c8640 1 -- 172.21.15.2:0/3332585107 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0b4108a70 msgr2=0x7fb0b4108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.354+0000 7fb0ba4c8640 1 --2- 172.21.15.2:0/3332585107 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0b4108a70 0x7fb0b4108e50 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4009930 tx=0x7fb0a402a420 comp rx=0 tx=0).stop 2024-05-14T17:14:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.354+0000 7fb0ba4c8640 1 -- 172.21.15.2:0/3332585107 shutdown_connections 2024-05-14T17:14:22.611 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.354+0000 7fb0ba4c8640 1 --2- 172.21.15.2:0/3332585107 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0b4103420 0x7fb0b410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:22.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.354+0000 7fb0ba4c8640 1 --2- 172.21.15.2:0/3332585107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0b4102a60 0x7fb0b4102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.354+0000 7fb0ba4c8640 1 --2- 172.21.15.2:0/3332585107 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0b4108a70 0x7fb0b4108e50 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:22.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.354+0000 7fb0ba4c8640 1 -- 172.21.15.2:0/3332585107 >> 172.21.15.2:0/3332585107 conn(0x7fb0b4079ee0 msgr2=0x7fb0b407a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.354+0000 7fb0ba4c8640 1 -- 172.21.15.2:0/3332585107 shutdown_connections 2024-05-14T17:14:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.354+0000 7fb0ba4c8640 1 -- 172.21.15.2:0/3332585107 wait complete. 2024-05-14T17:14:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.355+0000 7fb0ba4c8640 1 Processor -- start 2024-05-14T17:14:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.355+0000 7fb0ba4c8640 1 -- start start 2024-05-14T17:14:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.355+0000 7fb0ba4c8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0b4102a60 0x7fb0b41991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:22.987 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.355+0000 7fb0ba4c8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0b4103420 0x7fb0b4199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.355+0000 7fb0ba4c8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0b4108a70 0x7fb0b419dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.355+0000 7fb0ba4c8640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b4112d30 con 0x7fb0b4102a60 2024-05-14T17:14:23.120 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.355+0000 7fb0ba4c8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b4112bb0 con 0x7fb0b4108a70 2024-05-14T17:14:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.355+0000 7fb0ba4c8640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b4112eb0 con 0x7fb0b4103420 2024-05-14T17:14:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.355+0000 7fb0b3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0b4102a60 0x7fb0b41991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:23.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.355+0000 7fb0b3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0b4102a60 0x7fb0b41991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58802/0 (socket says 172.21.15.2:58802) 2024-05-14T17:14:23.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.355+0000 7fb0b3fff640 1 -- 172.21.15.2:0/2015692099 learned_addr learned my addr 172.21.15.2:0/2015692099 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:14:23.343 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.356+0000 7fb0b8a65640 1 --2- 172.21.15.2:0/2015692099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0b4108a70 0x7fb0b419dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.356+0000 7fb0b37fe640 1 --2- 172.21.15.2:0/2015692099 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0b4103420 0x7fb0b4199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.356+0000 7fb0b3fff640 1 -- 172.21.15.2:0/2015692099 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0b4103420 msgr2=0x7fb0b4199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:23.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.356+0000 7fb0b3fff640 1 --2- 172.21.15.2:0/2015692099 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0b4103420 0x7fb0b4199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.356+0000 7fb0b3fff640 1 -- 172.21.15.2:0/2015692099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0b4108a70 msgr2=0x7fb0b419dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.356+0000 7fb0b3fff640 1 --2- 172.21.15.2:0/2015692099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0b4108a70 0x7fb0b419dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:23.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.356+0000 7fb0b3fff640 1 -- 172.21.15.2:0/2015692099 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b419e1e0 con 0x7fb0b4102a60 2024-05-14T17:14:23.679 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.356+0000 7fb0b37fe640 1 --2- 172.21.15.2:0/2015692099 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0b4103420 0x7fb0b4199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:14:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.356+0000 7fb0b3fff640 1 --2- 172.21.15.2:0/2015692099 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0b4102a60 0x7fb0b41991f0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4004770 tx=0x7fb0a40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.356+0000 7fb0b17fa640 1 -- 172.21.15.2:0/2015692099 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a403f070 con 0x7fb0b4102a60 2024-05-14T17:14:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.356+0000 7fb0b17fa640 1 -- 172.21.15.2:0/2015692099 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0a403b460 con 0x7fb0b4102a60 2024-05-14T17:14:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.356+0000 7fb0ba4c8640 1 -- 172.21.15.2:0/2015692099 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0b419e470 con 0x7fb0b4102a60 2024-05-14T17:14:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.356+0000 7fb0b17fa640 1 -- 172.21.15.2:0/2015692099 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4035070 con 0x7fb0b4102a60 2024-05-14T17:14:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.356+0000 7fb0ba4c8640 1 -- 172.21.15.2:0/2015692099 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0b41a5d60 con 0x7fb0b4102a60 2024-05-14T17:14:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.359+0000 7fb0bd1e5740 1 Processor -- start 2024-05-14T17:14:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.359+0000 7fb0b17fa640 1 -- 172.21.15.2:0/2015692099 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fb0a403b8d0 con 0x7fb0b4102a60 2024-05-14T17:14:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.359+0000 7fb0bd1e5740 1 -- start start 2024-05-14T17:14:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.359+0000 7fb0bd1e5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56137c7157d0 0x56137c715bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.359+0000 7fb0bd1e5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56137c7160f0 0x56137c7249a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:24.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.359+0000 7fb0bd1e5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56137c724ee0 0x56137c7272d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:24.287 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.359+0000 7fb0bd1e5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56137c688c30 con 0x56137c724ee0 2024-05-14T17:14:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.359+0000 7fb0bd1e5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56137c6787d0 con 0x56137c7157d0 2024-05-14T17:14:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.359+0000 7fb0bd1e5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56137c67c810 con 0x56137c7160f0 2024-05-14T17:14:24.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.359+0000 7fb0b37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56137c724ee0 0x56137c7272d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.360+0000 7fb0b37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56137c724ee0 0x56137c7272d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58816/0 (socket says 172.21.15.2:58816) 2024-05-14T17:14:24.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.360+0000 7fb0b37fe640 1 -- 172.21.15.2:0/2362147551 learned_addr learned my addr 172.21.15.2:0/2362147551 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:14:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.360+0000 7fb0b3fff640 1 --2- 172.21.15.2:0/2362147551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56137c7160f0 0x56137c7249a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:24.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.360+0000 7fb0b8a65640 1 --2- 172.21.15.2:0/2362147551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56137c7157d0 0x56137c715bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.360+0000 7fb0b37fe640 1 -- 172.21.15.2:0/2362147551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56137c7160f0 msgr2=0x56137c7249a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:24.671 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.360+0000 7fb0b37fe640 1 --2- 172.21.15.2:0/2362147551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56137c7160f0 0x56137c7249a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.360+0000 7fb0b37fe640 1 -- 172.21.15.2:0/2362147551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56137c7157d0 msgr2=0x56137c715bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:24.755 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.360+0000 7fb0b37fe640 1 --2- 172.21.15.2:0/2362147551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56137c7157d0 0x56137c715bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.360+0000 7fb0b37fe640 1 -- 172.21.15.2:0/2362147551 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56137c38d210 con 0x56137c724ee0 2024-05-14T17:14:24.838 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.360+0000 7fb0b17fa640 1 --2- 172.21.15.2:0/2015692099 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb09c0b2a90 0x7fb09c0b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.360+0000 7fb0b37fe640 1 --2- 172.21.15.2:0/2362147551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56137c724ee0 0x56137c7272d0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8009c50 tx=0x7fb0a800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f6bb0e110d02dc server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:24.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.360+0000 7fb0b8a65640 1 --2- 172.21.15.2:0/2015692099 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb09c0b2a90 0x7fb09c0b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:24.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.360+0000 7fb0b17fa640 1 -- 172.21.15.2:0/2015692099 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8513+0+0 (secure 0 0 0) 0x7fb0a4102660 con 0x7fb0b4102a60 2024-05-14T17:14:25.021 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.360+0000 7fb09a7fc640 1 -- 172.21.15.2:0/2362147551 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8016070 con 0x56137c724ee0 2024-05-14T17:14:25.063 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.360+0000 7fb09a7fc640 1 -- 172.21.15.2:0/2362147551 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0a8011920 con 0x56137c724ee0 2024-05-14T17:14:25.105 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.360+0000 7fb09a7fc640 1 -- 172.21.15.2:0/2362147551 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a80107f0 con 0x56137c724ee0 2024-05-14T17:14:25.146 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.361+0000 7fb0bd1e5740 1 -- 172.21.15.2:0/2362147551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56137c724ee0 msgr2=0x56137c7272d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:25.188 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.361+0000 7fb0b8a65640 1 --2- 172.21.15.2:0/2015692099 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb09c0b2a90 0x7fb09c0b4f50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac00ae40 tx=0x7fb0ac00a820 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.361+0000 7fb0bd1e5740 1 --2- 172.21.15.2:0/2362147551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56137c724ee0 0x56137c7272d0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8009c50 tx=0x7fb0a800e6c0 comp rx=0 tx=0).stop 2024-05-14T17:14:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.361+0000 7fb0bd1e5740 1 -- 172.21.15.2:0/2362147551 shutdown_connections 2024-05-14T17:14:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.361+0000 7fb0bd1e5740 1 --2- 172.21.15.2:0/2362147551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56137c724ee0 0x56137c7272d0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:25.363 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.361+0000 7fb0bd1e5740 1 --2- 172.21.15.2:0/2362147551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56137c7160f0 0x56137c7249a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:25.405 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.361+0000 7fb0bd1e5740 1 --2- 172.21.15.2:0/2362147551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56137c7157d0 0x56137c715bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.361+0000 7fb0bd1e5740 1 -- 172.21.15.2:0/2362147551 >> 172.21.15.2:0/2362147551 conn(0x56137c711e00 msgr2=0x56137c7121e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.361+0000 7fb0bd1e5740 1 -- 172.21.15.2:0/2362147551 shutdown_connections 2024-05-14T17:14:25.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.361+0000 7fb0bd1e5740 1 -- 172.21.15.2:0/2362147551 wait complete. 2024-05-14T17:14:25.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.361+0000 7fb0bd1e5740 1 -- start start 2024-05-14T17:14:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.362+0000 7fb0bd1e5740 1 Processor -- start 2024-05-14T17:14:25.663 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.362+0000 7fb0bd1e5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56137c7157d0 0x56137c724870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.362+0000 7fb0bd1e5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56137c7160f0 0x56137c721b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.362+0000 7fb0bd1e5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56137c724ee0 0x56137c7220e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.362+0000 7fb0bd1e5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56137c35e790 con 0x56137c724ee0 2024-05-14T17:14:25.830 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.362+0000 7fb0bd1e5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56137c634890 con 0x56137c7157d0 2024-05-14T17:14:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.362+0000 7fb0bd1e5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56137c727bb0 con 0x56137c7160f0 2024-05-14T17:14:25.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.362+0000 7fb0bd1e5740 10 client.0 _ll_register_callbacks cb 0x56137c637d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:14:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.362+0000 7fb0b8a65640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56137c724ee0 0x56137c7220e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:26.014 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.362+0000 7fb0b8a65640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56137c724ee0 0x56137c7220e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58830/0 (socket says 172.21.15.2:58830) 2024-05-14T17:14:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.362+0000 7fb0b8a65640 1 -- 172.21.15.2:0/3842944464 learned_addr learned my addr 172.21.15.2:0/3842944464 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:14:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.362+0000 7fb0b3fff640 1 --2- 172.21.15.2:0/3842944464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56137c7157d0 0x56137c724870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.362+0000 7fb0b37fe640 1 --2- 172.21.15.2:0/3842944464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56137c7160f0 0x56137c721b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.363+0000 7fb0b8a65640 1 -- 172.21.15.2:0/3842944464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56137c7160f0 msgr2=0x56137c721b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.363+0000 7fb0b8a65640 1 --2- 172.21.15.2:0/3842944464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56137c7160f0 0x56137c721b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.363+0000 7fb0b8a65640 1 -- 172.21.15.2:0/3842944464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56137c7157d0 msgr2=0x56137c724870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.363+0000 7fb0b8a65640 1 --2- 172.21.15.2:0/3842944464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56137c7157d0 0x56137c724870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:26.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.363+0000 7fb0b8a65640 1 -- 172.21.15.2:0/3842944464 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56137c459620 con 0x56137c724ee0 2024-05-14T17:14:26.389 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.363+0000 7fb0b37fe640 1 --2- 172.21.15.2:0/3842944464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56137c7160f0 0x56137c721b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:14:26.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.363+0000 7fb0b8a65640 1 --2- 172.21.15.2:0/3842944464 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56137c724ee0 0x56137c7220e0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac011e10 tx=0x7fb0ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c917606b1741f2b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:26.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.363+0000 7fb0937fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:14:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.363+0000 7fb0937fe640 1 -- 172.21.15.2:0/3842944464 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac01e420 con 0x56137c724ee0 2024-05-14T17:14:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.363+0000 7fb0937fe640 1 -- 172.21.15.2:0/3842944464 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8513+0+0 (secure 0 0 0) 0x7fb0ac01e5c0 con 0x56137c724ee0 2024-05-14T17:14:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.363+0000 7fb0bd1e5740 1 -- 172.21.15.2:0/3842944464 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0a8011920 con 0x56137c724ee0 2024-05-14T17:14:26.639 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.363+0000 7fb0937fe640 1 -- 172.21.15.2:0/3842944464 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fb0ac01e7c0 con 0x56137c724ee0 2024-05-14T17:14:26.681 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.363+0000 7fb0bd1e5740 10 client.17175 fetch_fsmap learned FSMap version 56 2024-05-14T17:14:26.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.363+0000 7fb0bd1e5740 1 -- 172.21.15.2:0/3842944464 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x56137c733730 con 0x56137c724ee0 2024-05-14T17:14:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.363+0000 7fb0937fe640 1 -- 172.21.15.2:0/3842944464 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0a8011920 con 0x56137c724ee0 2024-05-14T17:14:26.806 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.364+0000 7fb0937fe640 1 -- 172.21.15.2:0/3842944464 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 56) ==== 8346+0+0 (secure 0 0 0) 0x7fb0ac02fb00 con 0x56137c724ee0 2024-05-14T17:14:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.364+0000 7fb0bd1e5740 10 client.17175 fetch_fsmap finished waiting for FSMap version 56 2024-05-14T17:14:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.364+0000 7fb0bd1e5740 10 client.17175 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:14:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.364+0000 7fb0bd1e5740 20 client.17175 populate_metadata read hostname 'smithi002' 2024-05-14T17:14:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.364+0000 7fb0bd1e5740 1 --2- 172.21.15.2:0/3842944464 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56137c734c60 0x56137c7370b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.364+0000 7fb0bd1e5740 4 client.17175 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:14:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.364+0000 7fb0bd1e5740 1 -- 172.21.15.2:0/3842944464 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56137c68ab80 con 0x56137c734c60 2024-05-14T17:14:27.186 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.364+0000 7fb0b3fff640 1 --2- 172.21.15.2:0/3842944464 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56137c734c60 0x56137c7370b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.364+0000 7fb0b3fff640 1 --2- 172.21.15.2:0/3842944464 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56137c734c60 0x56137c7370b0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe22f55959ab8f30 server_cookie=124e1ad2c24e0e6c in_seq=0 out_seq=0 2024-05-14T17:14:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.364+0000 7fb0937fe640 10 client.17175 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:14:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.365+0000 7fb0937fe640 1 -- 172.21.15.2:0/3842944464 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56137c68ab80 con 0x56137c734c60 2024-05-14T17:14:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.365+0000 7fb0937fe640 10 client.17175 handle_command_reply: tid=0 2024-05-14T17:14:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.365+0000 7fb0bd1e5740 1 client.17175 shutdown 2024-05-14T17:14:27.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.365+0000 7fb0bd1e5740 1 -- 172.21.15.2:0/3842944464 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56137c724ee0 msgr2=0x56137c7220e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.365+0000 7fb0bd1e5740 1 --2- 172.21.15.2:0/3842944464 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56137c724ee0 0x56137c7220e0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac011e10 tx=0x7fb0ac004a40 comp rx=0 tx=0).stop 2024-05-14T17:14:27.548 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.365+0000 7fb0bd1e5740 1 -- 172.21.15.2:0/3842944464 shutdown_connections 2024-05-14T17:14:27.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.365+0000 7fb0bd1e5740 1 --2- 172.21.15.2:0/3842944464 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56137c734c60 0x56137c7370b0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:27.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.365+0000 7fb0bd1e5740 1 --2- 172.21.15.2:0/3842944464 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56137c724ee0 0x56137c7220e0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.365+0000 7fb0bd1e5740 1 --2- 172.21.15.2:0/3842944464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56137c7160f0 0x56137c721b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:27.732 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.365+0000 7fb0937fe640 0 client.17175 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:14:27.774 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.365+0000 7fb0bd1e5740 1 --2- 172.21.15.2:0/3842944464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56137c7157d0 0x56137c724870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.365+0000 7fb0bd1e5740 1 -- 172.21.15.2:0/3842944464 >> 172.21.15.2:0/3842944464 conn(0x56137c711e00 msgr2=0x56137c717c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.365+0000 7fb0bd1e5740 1 -- 172.21.15.2:0/3842944464 shutdown_connections 2024-05-14T17:14:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.365+0000 7fb0bd1e5740 1 -- 172.21.15.2:0/3842944464 wait complete. 2024-05-14T17:14:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.366+0000 7fb0bd1e5740 20 client.17175 trim_cache size 0 max 16384 2024-05-14T17:14:28.042 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.381+0000 7fb0bd1e5740 1 Processor -- start 2024-05-14T17:14:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.381+0000 7fb0bd1e5740 1 -- start start 2024-05-14T17:14:28.140 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.381+0000 7fb0bd1e5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56137c7157d0 0x56137c7243a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:28.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.381+0000 7fb0bd1e5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56137c7160f0 0x56137c73c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:28.215 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.381+0000 7fb0bd1e5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56137c724ee0 0x56137c73cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.381+0000 7fb0bd1e5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b41aed90 con 0x56137c7157d0 2024-05-14T17:14:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.381+0000 7fb0bd1e5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b41ac630 con 0x56137c724ee0 2024-05-14T17:14:28.340 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.381+0000 7fb0bd1e5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b41aa4a0 con 0x56137c7160f0 2024-05-14T17:14:28.382 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.381+0000 7fb0b3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56137c7157d0 0x56137c7243a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:28.424 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.381+0000 7fb0b3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56137c7157d0 0x56137c7243a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58840/0 (socket says 172.21.15.2:58840) 2024-05-14T17:14:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.381+0000 7fb0b3fff640 1 -- 172.21.15.2:0/3359221560 learned_addr learned my addr 172.21.15.2:0/3359221560 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:14:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.382+0000 7fb0b37fe640 1 --2- 172.21.15.2:0/3359221560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56137c7160f0 0x56137c73c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.382+0000 7fb0b8a65640 1 --2- 172.21.15.2:0/3359221560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56137c724ee0 0x56137c73cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.382+0000 7fb0b3fff640 1 -- 172.21.15.2:0/3359221560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56137c7160f0 msgr2=0x56137c73c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.382+0000 7fb0b3fff640 1 --2- 172.21.15.2:0/3359221560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56137c7160f0 0x56137c73c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.382+0000 7fb0b3fff640 1 -- 172.21.15.2:0/3359221560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56137c724ee0 msgr2=0x56137c73cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.382+0000 7fb0b3fff640 1 --2- 172.21.15.2:0/3359221560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56137c724ee0 0x56137c73cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.382+0000 7fb0b3fff640 1 -- 172.21.15.2:0/3359221560 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56137c715f20 con 0x56137c7157d0 2024-05-14T17:14:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.382+0000 7fb0b3fff640 1 --2- 172.21.15.2:0/3359221560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56137c7157d0 0x56137c7243a0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fb0a40c59c0 tx=0x7fb0a4104b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c21d9e8144b65a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:28.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.382+0000 7fb099ffb640 1 -- 172.21.15.2:0/3359221560 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a40afe60 con 0x56137c7157d0 2024-05-14T17:14:28.924 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.382+0000 7fb099ffb640 1 -- 172.21.15.2:0/3359221560 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0a40c9450 con 0x56137c7157d0 2024-05-14T17:14:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.382+0000 7fb099ffb640 1 -- 172.21.15.2:0/3359221560 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4048040 con 0x56137c7157d0 2024-05-14T17:14:29.000 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.383+0000 7fb0bd1e5740 1 -- 172.21.15.2:0/3359221560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56137c7157d0 msgr2=0x56137c7243a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:29.041 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.383+0000 7fb0bd1e5740 1 --2- 172.21.15.2:0/3359221560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56137c7157d0 0x56137c7243a0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fb0a40c59c0 tx=0x7fb0a4104b70 comp rx=0 tx=0).stop 2024-05-14T17:14:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.383+0000 7fb0bd1e5740 1 -- 172.21.15.2:0/3359221560 shutdown_connections 2024-05-14T17:14:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.383+0000 7fb0bd1e5740 1 --2- 172.21.15.2:0/3359221560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56137c724ee0 0x56137c73cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:29.166 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.383+0000 7fb0bd1e5740 1 --2- 172.21.15.2:0/3359221560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56137c7160f0 0x56137c73c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:29.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.383+0000 7fb0bd1e5740 1 --2- 172.21.15.2:0/3359221560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56137c7157d0 0x56137c7243a0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.383+0000 7fb0bd1e5740 1 -- 172.21.15.2:0/3359221560 >> 172.21.15.2:0/3359221560 conn(0x56137c711e00 msgr2=0x56137c711100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.383+0000 7fb0bd1e5740 1 -- 172.21.15.2:0/3359221560 shutdown_connections 2024-05-14T17:14:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.383+0000 7fb0bd1e5740 1 -- 172.21.15.2:0/3359221560 wait complete. 2024-05-14T17:14:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.383+0000 7fb0bd1e5740 1 -- start start 2024-05-14T17:14:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.383+0000 7fb0bd1e5740 1 Processor -- start 2024-05-14T17:14:29.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.384+0000 7fb0bd1e5740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56137c7157d0 0x56137c729080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:29.518 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.384+0000 7fb0bd1e5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56137c7160f0 0x56137c7295c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:29.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.384+0000 7fb0bd1e5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56137c724ee0 0x56137c79e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:29.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.384+0000 7fb0bd1e5740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b41aaaf0 con 0x56137c7157d0 2024-05-14T17:14:29.641 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.384+0000 7fb0bd1e5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b41abf60 con 0x56137c7160f0 2024-05-14T17:14:29.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.384+0000 7fb0bd1e5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b41ab1c0 con 0x56137c724ee0 2024-05-14T17:14:29.725 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.384+0000 7fb0bd1e5740 10 client.0 _ll_register_callbacks cb 0x7fb0ac011ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:14:29.767 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.384+0000 7fb0b3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56137c7157d0 0x56137c729080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.384+0000 7fb0b3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56137c7157d0 0x56137c729080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58856/0 (socket says 172.21.15.2:58856) 2024-05-14T17:14:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.384+0000 7fb0b3fff640 1 -- 172.21.15.2:0/2815486315 learned_addr learned my addr 172.21.15.2:0/2815486315 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:14:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.384+0000 7fb0b8a65640 1 --2- 172.21.15.2:0/2815486315 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56137c724ee0 0x56137c79e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.384+0000 7fb0b37fe640 1 --2- 172.21.15.2:0/2815486315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56137c7160f0 0x56137c7295c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.384+0000 7fb0b3fff640 1 -- 172.21.15.2:0/2815486315 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56137c724ee0 msgr2=0x56137c79e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.384+0000 7fb0b3fff640 1 --2- 172.21.15.2:0/2815486315 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56137c724ee0 0x56137c79e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:30.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.384+0000 7fb0b3fff640 1 -- 172.21.15.2:0/2815486315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56137c7160f0 msgr2=0x56137c7295c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:30.100 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.384+0000 7fb0b3fff640 1 --2- 172.21.15.2:0/2815486315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56137c7160f0 0x56137c7295c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:30.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.384+0000 7fb0b3fff640 1 -- 172.21.15.2:0/2815486315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56137c79ea70 con 0x56137c7157d0 2024-05-14T17:14:30.214 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.384+0000 7fb0b8a65640 1 --2- 172.21.15.2:0/2815486315 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56137c724ee0 0x56137c79e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:14:30.252 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.384+0000 7fb0b3fff640 1 --2- 172.21.15.2:0/2815486315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56137c7157d0 0x56137c729080 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4005940 tx=0x7fb0a402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f1a3485e674224a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.384+0000 7fb0937fe640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:14:30.334 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.384+0000 7fb0937fe640 1 -- 172.21.15.2:0/2815486315 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a40afe60 con 0x56137c7157d0 2024-05-14T17:14:30.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.384+0000 7fb0937fe640 1 -- 172.21.15.2:0/2815486315 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8513+0+0 (secure 0 0 0) 0x7fb0a40c5bd0 con 0x56137c7157d0 2024-05-14T17:14:30.417 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.384+0000 7fb0bd1e5740 1 -- 172.21.15.2:0/2815486315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0a40c9450 con 0x56137c7157d0 2024-05-14T17:14:30.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.385+0000 7fb0937fe640 1 -- 172.21.15.2:0/2815486315 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fb0a4048040 con 0x56137c7157d0 2024-05-14T17:14:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.385+0000 7fb0bd1e5740 10 client.17181 fetch_fsmap learned FSMap version 56 2024-05-14T17:14:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.385+0000 7fb0bd1e5740 1 -- 172.21.15.2:0/2815486315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x56137c79fc80 con 0x56137c7157d0 2024-05-14T17:14:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.385+0000 7fb0937fe640 1 -- 172.21.15.2:0/2815486315 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0a40c9450 con 0x56137c7157d0 2024-05-14T17:14:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.385+0000 7fb0937fe640 1 -- 172.21.15.2:0/2815486315 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 56) ==== 8346+0+0 (secure 0 0 0) 0x7fb0a40d2310 con 0x56137c7157d0 2024-05-14T17:14:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.385+0000 7fb0bd1e5740 10 client.17181 fetch_fsmap finished waiting for FSMap version 56 2024-05-14T17:14:30.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.385+0000 7fb0bd1e5740 10 client.17181 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:14:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.385+0000 7fb0bd1e5740 20 client.17181 populate_metadata read hostname 'smithi002' 2024-05-14T17:14:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.385+0000 7fb0bd1e5740 1 --2- 172.21.15.2:0/2815486315 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56137c7a02f0 0x56137c7a26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.385+0000 7fb0bd1e5740 4 client.17181 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:14:30.967 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.385+0000 7fb0bd1e5740 1 -- 172.21.15.2:0/2815486315 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56137c684360 con 0x56137c7a02f0 2024-05-14T17:14:31.009 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.385+0000 7fb0b37fe640 1 --2- 172.21.15.2:0/2815486315 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56137c7a02f0 0x56137c7a26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.386+0000 7fb0937fe640 10 client.17181 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:14:31.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.386+0000 7fb0b37fe640 1 --2- 172.21.15.2:0/2815486315 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56137c7a02f0 0x56137c7a26e0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad6f63a9223508f1 server_cookie=49402fc6050172f8 in_seq=0 out_seq=0 2024-05-14T17:14:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.393+0000 7fb0937fe640 1 -- 172.21.15.2:0/2815486315 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56137c684360 con 0x56137c7a02f0 2024-05-14T17:14:31.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.393+0000 7fb0937fe640 10 client.17181 handle_command_reply: tid=0 2024-05-14T17:14:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.393+0000 7fb0bd1e5740 1 client.17181 shutdown 2024-05-14T17:14:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.393+0000 7fb0bd1e5740 1 -- 172.21.15.2:0/2815486315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56137c7157d0 msgr2=0x56137c729080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:31.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.393+0000 7fb0bd1e5740 1 --2- 172.21.15.2:0/2815486315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56137c7157d0 0x56137c729080 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4005940 tx=0x7fb0a402a420 comp rx=0 tx=0).stop 2024-05-14T17:14:31.368 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.393+0000 7fb0bd1e5740 1 -- 172.21.15.2:0/2815486315 shutdown_connections 2024-05-14T17:14:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.393+0000 7fb0bd1e5740 1 --2- 172.21.15.2:0/2815486315 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56137c7a02f0 0x56137c7a26e0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.393+0000 7fb0bd1e5740 1 --2- 172.21.15.2:0/2815486315 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56137c724ee0 0x56137c79e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.393+0000 7fb0bd1e5740 1 --2- 172.21.15.2:0/2815486315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56137c7160f0 0x56137c7295c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.393+0000 7fb0bd1e5740 1 --2- 172.21.15.2:0/2815486315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56137c7157d0 0x56137c729080 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:31.577 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.393+0000 7fb0937fe640 0 client.17181 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:14:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.393+0000 7fb0bd1e5740 1 -- 172.21.15.2:0/2815486315 >> 172.21.15.2:0/2815486315 conn(0x56137c711e00 msgr2=0x56137c710870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.393+0000 7fb0bd1e5740 1 -- 172.21.15.2:0/2815486315 shutdown_connections 2024-05-14T17:14:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.393+0000 7fb0bd1e5740 1 -- 172.21.15.2:0/2815486315 wait complete. 2024-05-14T17:14:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.393+0000 7fb0bd1e5740 20 client.17181 trim_cache size 0 max 16384 2024-05-14T17:14:31.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:14:31.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:14:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:14:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.394+0000 7fb0ba4c8640 1 -- 172.21.15.2:0/2015692099 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb09c0b2a90 msgr2=0x7fb09c0b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:31.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.394+0000 7fb0ba4c8640 1 --2- 172.21.15.2:0/2015692099 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb09c0b2a90 0x7fb09c0b4f50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac00ae40 tx=0x7fb0ac00a820 comp rx=0 tx=0).stop 2024-05-14T17:14:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.394+0000 7fb0ba4c8640 1 -- 172.21.15.2:0/2015692099 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0b4102a60 msgr2=0x7fb0b41991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:32.052 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.394+0000 7fb0ba4c8640 1 --2- 172.21.15.2:0/2015692099 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0b4102a60 0x7fb0b41991f0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4004770 tx=0x7fb0a40042f0 comp rx=0 tx=0).stop 2024-05-14T17:14:32.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.394+0000 7fb0ba4c8640 1 -- 172.21.15.2:0/2015692099 shutdown_connections 2024-05-14T17:14:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.394+0000 7fb0ba4c8640 1 --2- 172.21.15.2:0/2015692099 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb09c0b2a90 0x7fb09c0b4f50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.394+0000 7fb0ba4c8640 1 --2- 172.21.15.2:0/2015692099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0b4108a70 0x7fb0b419dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.394+0000 7fb0ba4c8640 1 --2- 172.21.15.2:0/2015692099 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0b4103420 0x7fb0b4199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.394+0000 7fb0ba4c8640 1 --2- 172.21.15.2:0/2015692099 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0b4102a60 0x7fb0b41991f0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.394+0000 7fb0ba4c8640 1 -- 172.21.15.2:0/2015692099 >> 172.21.15.2:0/2015692099 conn(0x7fb0b4079ee0 msgr2=0x7fb0b4100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:32.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.394+0000 7fb0ba4c8640 1 -- 172.21.15.2:0/2015692099 shutdown_connections 2024-05-14T17:14:32.411 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:13:55.394+0000 7fb0ba4c8640 1 -- 172.21.15.2:0/2015692099 wait complete. 2024-05-14T17:14:32.457 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:13:56 smithi002 ceph-mon[31538]: pgmap v604: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1023 B/s wr, 6 op/s 2024-05-14T17:14:32.494 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:13:58 smithi002 ceph-mon[31538]: pgmap v605: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 1023 B/s wr, 6 op/s 2024-05-14T17:14:32.539 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:14:00 smithi002 ceph-mon[31538]: pgmap v606: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 10 KiB/s wr, 9 op/s 2024-05-14T17:14:32.577 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:14:02 smithi002 ceph-mon[31538]: pgmap v607: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 KiB/s wr, 6 op/s 2024-05-14T17:14:32.627 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:14:04 smithi002 ceph-mon[31538]: pgmap v608: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 55 KiB/s wr, 26 op/s 2024-05-14T17:14:32.661 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:14:06 smithi002 ceph-mon[31538]: pgmap v609: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 55 KiB/s wr, 23 op/s 2024-05-14T17:14:32.697 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:14:22 smithi151 ceph-mon[36066]: pgmap v619: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 38 KiB/s wr, 16 op/s 2024-05-14T17:14:32.728 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:14:24 smithi151 ceph-mon[36066]: pgmap v620: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-05-14T17:14:32.761 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:14:26 smithi151 ceph-mon[36066]: pgmap v621: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-05-14T17:14:32.794 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:14:28 smithi151 ceph-mon[36066]: pgmap v622: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 26 KiB/s wr, 13 op/s 2024-05-14T17:14:32.828 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:14:30 smithi151 ceph-mon[36066]: pgmap v623: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-05-14T17:14:32.861 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:14:22 smithi123 ceph-mon[35971]: pgmap v619: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 38 KiB/s wr, 16 op/s 2024-05-14T17:14:32.894 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:14:24 smithi123 ceph-mon[35971]: pgmap v620: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-05-14T17:14:32.928 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:14:26 smithi123 ceph-mon[35971]: pgmap v621: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-05-14T17:14:32.978 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:14:28 smithi123 ceph-mon[35971]: pgmap v622: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 26 KiB/s wr, 13 op/s 2024-05-14T17:14:33.011 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:14:30 smithi123 ceph-mon[35971]: pgmap v623: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-05-14T17:14:33.046 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:14:32 smithi123 ceph-mon[35971]: pgmap v624: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 KiB/s wr, 10 op/s 2024-05-14T17:14:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.496+0000 7fb0cdd12640 1 -- 172.21.15.2:0/3594113551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0c8103d00 msgr2=0x7fb0c81040e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.496+0000 7fb0cdd12640 1 --2- 172.21.15.2:0/3594113551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0c8103d00 0x7fb0c81040e0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8009900 tx=0x7fb0b802a480 comp rx=0 tx=0).stop 2024-05-14T17:14:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.496+0000 7fb0cdd12640 1 -- 172.21.15.2:0/3594113551 shutdown_connections 2024-05-14T17:14:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.496+0000 7fb0cdd12640 1 --2- 172.21.15.2:0/3594113551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0c810d0d0 0x7fb0c810f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.496+0000 7fb0cdd12640 1 --2- 172.21.15.2:0/3594113551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0c8104620 0x7fb0c810cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:33.245 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.496+0000 7fb0cdd12640 1 --2- 172.21.15.2:0/3594113551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0c8103d00 0x7fb0c81040e0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:33.278 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.496+0000 7fb0cdd12640 1 -- 172.21.15.2:0/3594113551 >> 172.21.15.2:0/3594113551 conn(0x7fb0c8077cd0 msgr2=0x7fb0c80780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:33.311 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.496+0000 7fb0cdd12640 1 -- 172.21.15.2:0/3594113551 shutdown_connections 2024-05-14T17:14:33.345 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.496+0000 7fb0cdd12640 1 -- 172.21.15.2:0/3594113551 wait complete. 2024-05-14T17:14:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.497+0000 7fb0cdd12640 1 Processor -- start 2024-05-14T17:14:33.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.497+0000 7fb0cdd12640 1 -- start start 2024-05-14T17:14:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.498+0000 7fb0cdd12640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0c8103d00 0x7fb0c8199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:33.478 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.498+0000 7fb0cdd12640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0c8104620 0x7fb0c8199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.498+0000 7fb0cdd12640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0c810d0d0 0x7fb0c819dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.498+0000 7fb0cdd12640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c8112d70 con 0x7fb0c8103d00 2024-05-14T17:14:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.498+0000 7fb0cdd12640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c8112bf0 con 0x7fb0c810d0d0 2024-05-14T17:14:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.498+0000 7fb0cdd12640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c8112ef0 con 0x7fb0c8104620 2024-05-14T17:14:33.674 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.498+0000 7fb0c77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0c8103d00 0x7fb0c8199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:33.712 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.498+0000 7fb0c77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0c8103d00 0x7fb0c8199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45102/0 (socket says 172.21.15.2:45102) 2024-05-14T17:14:33.757 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.498+0000 7fb0c77fe640 1 -- 172.21.15.2:0/1329732256 learned_addr learned my addr 172.21.15.2:0/1329732256 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:14:33.795 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.498+0000 7fb0c6ffd640 1 --2- 172.21.15.2:0/1329732256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0c8104620 0x7fb0c8199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.498+0000 7fb0c7fff640 1 --2- 172.21.15.2:0/1329732256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0c810d0d0 0x7fb0c819dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.498+0000 7fb0c77fe640 1 -- 172.21.15.2:0/1329732256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0c8104620 msgr2=0x7fb0c8199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.498+0000 7fb0c77fe640 1 --2- 172.21.15.2:0/1329732256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0c8104620 0x7fb0c8199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.498+0000 7fb0c77fe640 1 -- 172.21.15.2:0/1329732256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0c810d0d0 msgr2=0x7fb0c819dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:33.962 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.498+0000 7fb0c77fe640 1 --2- 172.21.15.2:0/1329732256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0c810d0d0 0x7fb0c819dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:33.995 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.498+0000 7fb0c77fe640 1 -- 172.21.15.2:0/1329732256 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c819e200 con 0x7fb0c8103d00 2024-05-14T17:14:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.498+0000 7fb0c7fff640 1 --2- 172.21.15.2:0/1329732256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0c810d0d0 0x7fb0c819dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:14:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.498+0000 7fb0c77fe640 1 --2- 172.21.15.2:0/1329732256 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0c8103d00 0x7fb0c8199210 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8005940 tx=0x7fb0b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:34.104 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.498+0000 7fb0c4ff9640 1 -- 172.21.15.2:0/1329732256 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b803f030 con 0x7fb0c8103d00 2024-05-14T17:14:34.137 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.498+0000 7fb0cdd12640 1 -- 172.21.15.2:0/1329732256 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0c819e490 con 0x7fb0c8103d00 2024-05-14T17:14:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.499+0000 7fb0c4ff9640 1 -- 172.21.15.2:0/1329732256 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0b8003cb0 con 0x7fb0c8103d00 2024-05-14T17:14:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.499+0000 7fb0c4ff9640 1 -- 172.21.15.2:0/1329732256 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8035070 con 0x7fb0c8103d00 2024-05-14T17:14:34.262 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.499+0000 7fb0cdd12640 1 -- 172.21.15.2:0/1329732256 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0c81a5d80 con 0x7fb0c8103d00 2024-05-14T17:14:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.501+0000 7fb0d0a1f740 1 Processor -- start 2024-05-14T17:14:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.501+0000 7fb0d0a1f740 1 -- start start 2024-05-14T17:14:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.501+0000 7fb0c4ff9640 1 -- 172.21.15.2:0/1329732256 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fb0b8042030 con 0x7fb0c8103d00 2024-05-14T17:14:34.412 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.501+0000 7fb0d0a1f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e50a257f0 0x556e50a25bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:34.446 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.501+0000 7fb0d0a1f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e50a26110 0x556e50a349a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:34.488 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.501+0000 7fb0d0a1f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e50a34ee0 0x556e50a372d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.501+0000 7fb0d0a1f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x556e50998c30 con 0x556e50a26110 2024-05-14T17:14:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.501+0000 7fb0d0a1f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556e509887d0 con 0x556e50a257f0 2024-05-14T17:14:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.501+0000 7fb0d0a1f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556e5098c810 con 0x556e50a34ee0 2024-05-14T17:14:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.501+0000 7fb0c77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e50a26110 0x556e50a349a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:34.654 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.502+0000 7fb0c77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e50a26110 0x556e50a349a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45104/0 (socket says 172.21.15.2:45104) 2024-05-14T17:14:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.502+0000 7fb0c77fe640 1 -- 172.21.15.2:0/1430768672 learned_addr learned my addr 172.21.15.2:0/1430768672 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:14:34.746 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.502+0000 7fb0c7fff640 1 --2- 172.21.15.2:0/1430768672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e50a257f0 0x556e50a25bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.502+0000 7fb0c6ffd640 1 --2- 172.21.15.2:0/1430768672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e50a34ee0 0x556e50a372d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.502+0000 7fb0c77fe640 1 -- 172.21.15.2:0/1430768672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e50a34ee0 msgr2=0x556e50a372d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.502+0000 7fb0c77fe640 1 --2- 172.21.15.2:0/1430768672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e50a34ee0 0x556e50a372d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:34.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.502+0000 7fb0c77fe640 1 -- 172.21.15.2:0/1430768672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e50a257f0 msgr2=0x556e50a25bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.502+0000 7fb0c77fe640 1 --2- 172.21.15.2:0/1430768672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e50a257f0 0x556e50a25bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.502+0000 7fb0c77fe640 1 -- 172.21.15.2:0/1430768672 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e5069d210 con 0x556e50a26110 2024-05-14T17:14:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.502+0000 7fb0c77fe640 1 --2- 172.21.15.2:0/1430768672 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e50a26110 0x556e50a349a0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8100520 tx=0x7fb0b810b1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36699f2fc6fe4efd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.502+0000 7fb0c4ff9640 1 --2- 172.21.15.2:0/1329732256 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb0b40b2a10 0x7fb0b40b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:35.047 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.502+0000 7fb0c7fff640 1 --2- 172.21.15.2:0/1329732256 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb0b40b2a10 0x7fb0b40b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.502+0000 7fb0bdffb640 1 -- 172.21.15.2:0/1430768672 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b80c6000 con 0x556e50a26110 2024-05-14T17:14:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.503+0000 7fb0c4ff9640 1 -- 172.21.15.2:0/1329732256 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8513+0+0 (secure 0 0 0) 0x7fb0b8102e60 con 0x7fb0c8103d00 2024-05-14T17:14:35.147 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.503+0000 7fb0bdffb640 1 -- 172.21.15.2:0/1430768672 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0b8110070 con 0x556e50a26110 2024-05-14T17:14:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.503+0000 7fb0bdffb640 1 -- 172.21.15.2:0/1430768672 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b80c8c50 con 0x556e50a26110 2024-05-14T17:14:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.503+0000 7fb0d0a1f740 1 -- 172.21.15.2:0/1430768672 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e50a26110 msgr2=0x556e50a349a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.503+0000 7fb0c7fff640 1 --2- 172.21.15.2:0/1329732256 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb0b40b2a10 0x7fb0b40b4ed0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb0c806c360 tx=0x7fb0c00098a0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.503+0000 7fb0d0a1f740 1 --2- 172.21.15.2:0/1430768672 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e50a26110 0x556e50a349a0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8100520 tx=0x7fb0b810b1a0 comp rx=0 tx=0).stop 2024-05-14T17:14:35.347 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.503+0000 7fb0d0a1f740 1 -- 172.21.15.2:0/1430768672 shutdown_connections 2024-05-14T17:14:35.380 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.503+0000 7fb0d0a1f740 1 --2- 172.21.15.2:0/1430768672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e50a34ee0 0x556e50a372d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.503+0000 7fb0d0a1f740 1 --2- 172.21.15.2:0/1430768672 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e50a26110 0x556e50a349a0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.503+0000 7fb0d0a1f740 1 --2- 172.21.15.2:0/1430768672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e50a257f0 0x556e50a25bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.503+0000 7fb0d0a1f740 1 -- 172.21.15.2:0/1430768672 >> 172.21.15.2:0/1430768672 conn(0x556e50a21e00 msgr2=0x556e50a221e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:35.515 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.503+0000 7fb0d0a1f740 1 -- 172.21.15.2:0/1430768672 shutdown_connections 2024-05-14T17:14:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.503+0000 7fb0d0a1f740 1 -- 172.21.15.2:0/1430768672 wait complete. 2024-05-14T17:14:35.597 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.504+0000 7fb0d0a1f740 1 -- start start 2024-05-14T17:14:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.504+0000 7fb0d0a1f740 1 Processor -- start 2024-05-14T17:14:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.504+0000 7fb0d0a1f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e50a257f0 0x556e50a34820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:35.697 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.504+0000 7fb0d0a1f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e50a26110 0x556e50a31b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:35.730 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.504+0000 7fb0d0a1f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e50a34ee0 0x556e50a32050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:35.764 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.504+0000 7fb0d0a1f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x556e5066e790 con 0x556e50a257f0 2024-05-14T17:14:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.504+0000 7fb0d0a1f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556e50944890 con 0x556e50a34ee0 2024-05-14T17:14:35.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.504+0000 7fb0d0a1f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556e50a378d0 con 0x556e50a26110 2024-05-14T17:14:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.504+0000 7fb0d0a1f740 10 client.0 _ll_register_callbacks cb 0x556e50947d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:14:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.504+0000 7fb0c77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e50a257f0 0x556e50a34820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.504+0000 7fb0c77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e50a257f0 0x556e50a34820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45116/0 (socket says 172.21.15.2:45116) 2024-05-14T17:14:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.504+0000 7fb0c77fe640 1 -- 172.21.15.2:0/1845736175 learned_addr learned my addr 172.21.15.2:0/1845736175 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:14:36.056 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.505+0000 7fb0c7fff640 1 --2- 172.21.15.2:0/1845736175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e50a34ee0 0x556e50a32050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.505+0000 7fb0c6ffd640 1 --2- 172.21.15.2:0/1845736175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e50a26110 0x556e50a31b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.505+0000 7fb0c77fe640 1 -- 172.21.15.2:0/1845736175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e50a26110 msgr2=0x556e50a31b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.505+0000 7fb0c77fe640 1 --2- 172.21.15.2:0/1845736175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e50a26110 0x556e50a31b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.505+0000 7fb0c77fe640 1 -- 172.21.15.2:0/1845736175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e50a34ee0 msgr2=0x556e50a32050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:36.223 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.505+0000 7fb0c77fe640 1 --2- 172.21.15.2:0/1845736175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e50a34ee0 0x556e50a32050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.505+0000 7fb0c77fe640 1 -- 172.21.15.2:0/1845736175 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e50769620 con 0x556e50a257f0 2024-05-14T17:14:36.314 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.505+0000 7fb0c7fff640 1 --2- 172.21.15.2:0/1845736175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e50a34ee0 0x556e50a32050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:14:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.505+0000 7fb0c77fe640 1 --2- 172.21.15.2:0/1845736175 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e50a257f0 0x556e50a34820 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80c9e40 tx=0x7fb0b803b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3529ffc7dc0801a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.505+0000 7fb0a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:14:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.505+0000 7fb0a6ffd640 1 -- 172.21.15.2:0/1845736175 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8049070 con 0x556e50a257f0 2024-05-14T17:14:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.505+0000 7fb0a6ffd640 1 -- 172.21.15.2:0/1845736175 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8513+0+0 (secure 0 0 0) 0x7fb0b80c63d0 con 0x556e50a257f0 2024-05-14T17:14:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.505+0000 7fb0d0a1f740 1 -- 172.21.15.2:0/1845736175 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0b8110070 con 0x556e50a257f0 2024-05-14T17:14:36.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.505+0000 7fb0a6ffd640 1 -- 172.21.15.2:0/1845736175 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fb0b80c98b0 con 0x556e50a257f0 2024-05-14T17:14:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.505+0000 7fb0d0a1f740 10 client.17193 fetch_fsmap learned FSMap version 56 2024-05-14T17:14:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.505+0000 7fb0d0a1f740 1 -- 172.21.15.2:0/1845736175 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x556e50a43350 con 0x556e50a257f0 2024-05-14T17:14:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.506+0000 7fb0a6ffd640 1 -- 172.21.15.2:0/1845736175 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0b8110070 con 0x556e50a257f0 2024-05-14T17:14:36.690 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.506+0000 7fb0a6ffd640 1 -- 172.21.15.2:0/1845736175 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 56) ==== 8346+0+0 (secure 0 0 0) 0x7fb0b80ce150 con 0x556e50a257f0 2024-05-14T17:14:36.723 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.506+0000 7fb0d0a1f740 10 client.17193 fetch_fsmap finished waiting for FSMap version 56 2024-05-14T17:14:36.756 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.506+0000 7fb0d0a1f740 10 client.17193 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:14:36.790 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.506+0000 7fb0d0a1f740 20 client.17193 populate_metadata read hostname 'smithi002' 2024-05-14T17:14:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.506+0000 7fb0d0a1f740 1 --2- 172.21.15.2:0/1845736175 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x556e50a446a0 0x556e50a46a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:36.856 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.506+0000 7fb0d0a1f740 4 client.17193 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:14:36.890 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.506+0000 7fb0d0a1f740 1 -- 172.21.15.2:0/1845736175 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e5099ab80 con 0x556e50a446a0 2024-05-14T17:14:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.506+0000 7fb0c6ffd640 1 --2- 172.21.15.2:0/1845736175 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x556e50a446a0 0x556e50a46a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:36.956 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.507+0000 7fb0a6ffd640 10 client.17193 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:14:36.990 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.507+0000 7fb0c6ffd640 1 --2- 172.21.15.2:0/1845736175 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x556e50a446a0 0x556e50a46a90 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4a2af0ab69ca4c8 server_cookie=10ec888dee413b1 in_seq=0 out_seq=0 2024-05-14T17:14:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.507+0000 7fb0a6ffd640 1 -- 172.21.15.2:0/1845736175 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556e5099ab80 con 0x556e50a446a0 2024-05-14T17:14:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.507+0000 7fb0a6ffd640 10 client.17193 handle_command_reply: tid=0 2024-05-14T17:14:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.507+0000 7fb0d0a1f740 1 client.17193 shutdown 2024-05-14T17:14:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.507+0000 7fb0d0a1f740 1 -- 172.21.15.2:0/1845736175 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e50a257f0 msgr2=0x556e50a34820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:37.165 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.507+0000 7fb0d0a1f740 1 --2- 172.21.15.2:0/1845736175 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e50a257f0 0x556e50a34820 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80c9e40 tx=0x7fb0b803b460 comp rx=0 tx=0).stop 2024-05-14T17:14:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.508+0000 7fb0d0a1f740 1 -- 172.21.15.2:0/1845736175 shutdown_connections 2024-05-14T17:14:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.508+0000 7fb0d0a1f740 1 --2- 172.21.15.2:0/1845736175 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x556e50a446a0 0x556e50a46a90 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.282 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.508+0000 7fb0d0a1f740 1 --2- 172.21.15.2:0/1845736175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e50a34ee0 0x556e50a32050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.315 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.508+0000 7fb0d0a1f740 1 --2- 172.21.15.2:0/1845736175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e50a26110 0x556e50a31b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.349 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.508+0000 7fb0a6ffd640 0 client.17193 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:14:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.508+0000 7fb0d0a1f740 1 --2- 172.21.15.2:0/1845736175 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e50a257f0 0x556e50a34820 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:37.415 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.508+0000 7fb0d0a1f740 1 -- 172.21.15.2:0/1845736175 >> 172.21.15.2:0/1845736175 conn(0x556e50a21e00 msgr2=0x556e50a27c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:37.449 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.508+0000 7fb0d0a1f740 1 -- 172.21.15.2:0/1845736175 shutdown_connections 2024-05-14T17:14:37.482 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.508+0000 7fb0d0a1f740 1 -- 172.21.15.2:0/1845736175 wait complete. 2024-05-14T17:14:37.516 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.508+0000 7fb0d0a1f740 20 client.17193 trim_cache size 0 max 16384 2024-05-14T17:14:37.551 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.523+0000 7fb0d0a1f740 1 Processor -- start 2024-05-14T17:14:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.523+0000 7fb0d0a1f740 1 -- start start 2024-05-14T17:14:37.649 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.524+0000 7fb0d0a1f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e50a257f0 0x556e50a2d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:37.682 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.524+0000 7fb0d0a1f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e50a26110 0x556e50a27af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.524+0000 7fb0d0a1f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e50a34ee0 0x556e50a34350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:37.749 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.524+0000 7fb0d0a1f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c81aedb0 con 0x556e50a257f0 2024-05-14T17:14:37.782 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.524+0000 7fb0d0a1f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c81ac650 con 0x556e50a26110 2024-05-14T17:14:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.524+0000 7fb0d0a1f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c81aa4c0 con 0x556e50a34ee0 2024-05-14T17:14:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.524+0000 7fb0c77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e50a257f0 0x556e50a2d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:37.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.524+0000 7fb0c77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e50a257f0 0x556e50a2d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45120/0 (socket says 172.21.15.2:45120) 2024-05-14T17:14:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.524+0000 7fb0c77fe640 1 -- 172.21.15.2:0/1999494684 learned_addr learned my addr 172.21.15.2:0/1999494684 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:14:37.949 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.524+0000 7fb0c77fe640 1 -- 172.21.15.2:0/1999494684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e50a34ee0 msgr2=0x556e50a34350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.524+0000 7fb0c7fff640 1 --2- 172.21.15.2:0/1999494684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e50a34ee0 0x556e50a34350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:38.020 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.524+0000 7fb0c6ffd640 1 --2- 172.21.15.2:0/1999494684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e50a26110 0x556e50a27af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.524+0000 7fb0c77fe640 1 --2- 172.21.15.2:0/1999494684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e50a34ee0 0x556e50a34350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:38.091 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.524+0000 7fb0c77fe640 1 -- 172.21.15.2:0/1999494684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e50a26110 msgr2=0x556e50a27af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.524+0000 7fb0c77fe640 1 --2- 172.21.15.2:0/1999494684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e50a26110 0x556e50a27af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:38.174 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.524+0000 7fb0c77fe640 1 -- 172.21.15.2:0/1999494684 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e50a21c30 con 0x556e50a257f0 2024-05-14T17:14:38.208 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.524+0000 7fb0c7fff640 1 --2- 172.21.15.2:0/1999494684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e50a34ee0 0x556e50a34350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:14:38.241 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.524+0000 7fb0c77fe640 1 --2- 172.21.15.2:0/1999494684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e50a257f0 0x556e50a2d1d0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8108040 tx=0x7fb0b81005d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b28e338de17f5564 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.524+0000 7fb0bd7fa640 1 -- 172.21.15.2:0/1999494684 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8049030 con 0x556e50a257f0 2024-05-14T17:14:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.524+0000 7fb0bd7fa640 1 -- 172.21.15.2:0/1999494684 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0b8110070 con 0x556e50a257f0 2024-05-14T17:14:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.524+0000 7fb0bd7fa640 1 -- 172.21.15.2:0/1999494684 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b80c9a90 con 0x556e50a257f0 2024-05-14T17:14:38.375 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.525+0000 7fb0d0a1f740 1 -- 172.21.15.2:0/1999494684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e50a257f0 msgr2=0x556e50a2d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.525+0000 7fb0d0a1f740 1 --2- 172.21.15.2:0/1999494684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e50a257f0 0x556e50a2d1d0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8108040 tx=0x7fb0b81005d0 comp rx=0 tx=0).stop 2024-05-14T17:14:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.525+0000 7fb0d0a1f740 1 -- 172.21.15.2:0/1999494684 shutdown_connections 2024-05-14T17:14:38.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.525+0000 7fb0d0a1f740 1 --2- 172.21.15.2:0/1999494684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e50a34ee0 0x556e50a34350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.525+0000 7fb0d0a1f740 1 --2- 172.21.15.2:0/1999494684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e50a26110 0x556e50a27af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:38.541 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.525+0000 7fb0d0a1f740 1 --2- 172.21.15.2:0/1999494684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e50a257f0 0x556e50a2d1d0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:38.575 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.525+0000 7fb0d0a1f740 1 -- 172.21.15.2:0/1999494684 >> 172.21.15.2:0/1999494684 conn(0x556e50a21e00 msgr2=0x556e50a273e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:38.616 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.525+0000 7fb0d0a1f740 1 -- 172.21.15.2:0/1999494684 shutdown_connections 2024-05-14T17:14:38.650 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.525+0000 7fb0d0a1f740 1 -- 172.21.15.2:0/1999494684 wait complete. 2024-05-14T17:14:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.525+0000 7fb0d0a1f740 1 -- start start 2024-05-14T17:14:38.750 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.525+0000 7fb0d0a1f740 1 Processor -- start 2024-05-14T17:14:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.526+0000 7fb0d0a1f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e50a257f0 0x556e50a38650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.526+0000 7fb0d0a1f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e50a26110 0x556e50a38b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.526+0000 7fb0d0a1f740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e50a34ee0 0x556e50a390d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.526+0000 7fb0d0a1f740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c81aab10 con 0x556e50a34ee0 2024-05-14T17:14:38.917 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.526+0000 7fb0d0a1f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c81abf80 con 0x556e50a257f0 2024-05-14T17:14:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.526+0000 7fb0d0a1f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c81ab1e0 con 0x556e50a26110 2024-05-14T17:14:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.526+0000 7fb0d0a1f740 10 client.0 _ll_register_callbacks cb 0x7fb0b8004570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:14:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.526+0000 7fb0c7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e50a34ee0 0x556e50a390d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.526+0000 7fb0c7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e50a34ee0 0x556e50a390d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45130/0 (socket says 172.21.15.2:45130) 2024-05-14T17:14:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.526+0000 7fb0c7fff640 1 -- 172.21.15.2:0/853504961 learned_addr learned my addr 172.21.15.2:0/853504961 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:14:39.117 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.526+0000 7fb0c77fe640 1 --2- 172.21.15.2:0/853504961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e50a257f0 0x556e50a38650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.526+0000 7fb0c6ffd640 1 --2- 172.21.15.2:0/853504961 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e50a26110 0x556e50a38b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.526+0000 7fb0c7fff640 1 -- 172.21.15.2:0/853504961 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e50a26110 msgr2=0x556e50a38b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:39.217 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.526+0000 7fb0c7fff640 1 --2- 172.21.15.2:0/853504961 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e50a26110 0x556e50a38b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.526+0000 7fb0c7fff640 1 -- 172.21.15.2:0/853504961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e50a257f0 msgr2=0x556e50a38650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:14:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.526+0000 7fb0c7fff640 1 --2- 172.21.15.2:0/853504961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e50a257f0 0x556e50a38650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.526+0000 7fb0c7fff640 1 -- 172.21.15.2:0/853504961 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e50a26650 con 0x556e50a34ee0 2024-05-14T17:14:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.526+0000 7fb0c7fff640 1 --2- 172.21.15.2:0/853504961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e50a34ee0 0x556e50a390d0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0010a20 tx=0x7fb0c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d1cc2dec21fb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:14:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.526+0000 7fb0a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:14:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.527+0000 7fb0a6ffd640 1 -- 172.21.15.2:0/853504961 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c002c060 con 0x556e50a34ee0 2024-05-14T17:14:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.527+0000 7fb0a6ffd640 1 -- 172.21.15.2:0/853504961 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8513+0+0 (secure 0 0 0) 0x7fb0c001eb20 con 0x556e50a34ee0 2024-05-14T17:14:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.527+0000 7fb0d0a1f740 1 -- 172.21.15.2:0/853504961 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0b8110070 con 0x556e50a34ee0 2024-05-14T17:14:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.527+0000 7fb0a6ffd640 1 -- 172.21.15.2:0/853504961 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fb0c001ed20 con 0x556e50a34ee0 2024-05-14T17:14:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.527+0000 7fb0d0a1f740 10 client.17199 fetch_fsmap learned FSMap version 56 2024-05-14T17:14:39.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.527+0000 7fb0d0a1f740 1 -- 172.21.15.2:0/853504961 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x556e50aaee80 con 0x556e50a34ee0 2024-05-14T17:14:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.527+0000 7fb0a6ffd640 1 -- 172.21.15.2:0/853504961 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0b8110070 con 0x556e50a34ee0 2024-05-14T17:14:39.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.527+0000 7fb0a6ffd640 1 -- 172.21.15.2:0/853504961 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 56) ==== 8346+0+0 (secure 0 0 0) 0x7fb0c0026070 con 0x556e50a34ee0 2024-05-14T17:14:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.527+0000 7fb0d0a1f740 10 client.17199 fetch_fsmap finished waiting for FSMap version 56 2024-05-14T17:14:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.527+0000 7fb0d0a1f740 10 client.17199 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:14:39.762 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.527+0000 7fb0d0a1f740 20 client.17199 populate_metadata read hostname 'smithi002' 2024-05-14T17:14:39.801 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.527+0000 7fb0d0a1f740 1 --2- 172.21.15.2:0/853504961 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x556e50aaf660 0x556e50ab1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:14:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.527+0000 7fb0d0a1f740 4 client.17199 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:14:39.876 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.527+0000 7fb0d0a1f740 1 -- 172.21.15.2:0/853504961 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556e50994360 con 0x556e50aaf660 2024-05-14T17:14:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.528+0000 7fb0c77fe640 1 --2- 172.21.15.2:0/853504961 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x556e50aaf660 0x556e50ab1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:14:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.528+0000 7fb0c77fe640 1 --2- 172.21.15.2:0/853504961 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x556e50aaf660 0x556e50ab1a80 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b874930cf90b00f server_cookie=effa317a65a7c54 in_seq=0 out_seq=0 2024-05-14T17:14:40.010 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.528+0000 7fb0a6ffd640 10 client.17199 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:14:40.044 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.538+0000 7fb0a6ffd640 1 -- 172.21.15.2:0/853504961 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556e50994360 con 0x556e50aaf660 2024-05-14T17:14:40.077 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.538+0000 7fb0a6ffd640 10 client.17199 handle_command_reply: tid=0 2024-05-14T17:14:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.538+0000 7fb0d0a1f740 1 client.17199 shutdown 2024-05-14T17:14:40.144 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.538+0000 7fb0d0a1f740 1 -- 172.21.15.2:0/853504961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e50a34ee0 msgr2=0x556e50a390d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.538+0000 7fb0d0a1f740 1 --2- 172.21.15.2:0/853504961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e50a34ee0 0x556e50a390d0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0010a20 tx=0x7fb0c0004a40 comp rx=0 tx=0).stop 2024-05-14T17:14:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.538+0000 7fb0d0a1f740 1 -- 172.21.15.2:0/853504961 shutdown_connections 2024-05-14T17:14:40.319 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.538+0000 7fb0d0a1f740 1 --2- 172.21.15.2:0/853504961 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x556e50aaf660 0x556e50ab1a80 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.538+0000 7fb0d0a1f740 1 --2- 172.21.15.2:0/853504961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556e50a34ee0 0x556e50a390d0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.538+0000 7fb0a6ffd640 0 client.17199 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:14:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.538+0000 7fb0d0a1f740 1 --2- 172.21.15.2:0/853504961 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e50a26110 0x556e50a38b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.538+0000 7fb0d0a1f740 1 --2- 172.21.15.2:0/853504961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556e50a257f0 0x556e50a38650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:40.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.538+0000 7fb0d0a1f740 1 -- 172.21.15.2:0/853504961 >> 172.21.15.2:0/853504961 conn(0x556e50a21e00 msgr2=0x556e50a21100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:40.552 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.538+0000 7fb0d0a1f740 1 -- 172.21.15.2:0/853504961 shutdown_connections 2024-05-14T17:14:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.538+0000 7fb0d0a1f740 1 -- 172.21.15.2:0/853504961 wait complete. 2024-05-14T17:14:40.619 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.538+0000 7fb0d0a1f740 20 client.17199 trim_cache size 0 max 16384 2024-05-14T17:14:40.653 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:14:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:14:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:14:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.539+0000 7fb0cdd12640 1 -- 172.21.15.2:0/1329732256 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb0b40b2a10 msgr2=0x7fb0b40b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:40.802 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.539+0000 7fb0cdd12640 1 --2- 172.21.15.2:0/1329732256 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb0b40b2a10 0x7fb0b40b4ed0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb0c806c360 tx=0x7fb0c00098a0 comp rx=0 tx=0).stop 2024-05-14T17:14:40.836 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.539+0000 7fb0cdd12640 1 -- 172.21.15.2:0/1329732256 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0c8103d00 msgr2=0x7fb0c8199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:14:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.539+0000 7fb0cdd12640 1 --2- 172.21.15.2:0/1329732256 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0c8103d00 0x7fb0c8199210 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8005940 tx=0x7fb0b80042f0 comp rx=0 tx=0).stop 2024-05-14T17:14:40.903 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.539+0000 7fb0cdd12640 1 -- 172.21.15.2:0/1329732256 shutdown_connections 2024-05-14T17:14:40.936 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.539+0000 7fb0cdd12640 1 --2- 172.21.15.2:0/1329732256 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb0b40b2a10 0x7fb0b40b4ed0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:40.969 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.539+0000 7fb0cdd12640 1 --2- 172.21.15.2:0/1329732256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0c810d0d0 0x7fb0c819dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.539+0000 7fb0cdd12640 1 --2- 172.21.15.2:0/1329732256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0c8104620 0x7fb0c8199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:41.036 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.539+0000 7fb0cdd12640 1 --2- 172.21.15.2:0/1329732256 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb0c8103d00 0x7fb0c8199210 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:14:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.539+0000 7fb0cdd12640 1 -- 172.21.15.2:0/1329732256 >> 172.21.15.2:0/1329732256 conn(0x7fb0c8077cd0 msgr2=0x7fb0c80fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:14:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.539+0000 7fb0cdd12640 1 -- 172.21.15.2:0/1329732256 shutdown_connections 2024-05-14T17:14:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:08.539+0000 7fb0cdd12640 1 -- 172.21.15.2:0/1329732256 wait complete. 2024-05-14T17:14:41.196 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:14:08 smithi002 ceph-mon[31538]: pgmap v610: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 55 KiB/s wr, 23 op/s 2024-05-14T17:14:41.228 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:14:09 smithi002 ceph-mon[31538]: osdmap e106: 12 total, 12 up, 12 in 2024-05-14T17:14:41.261 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:14:10 smithi002 ceph-mon[31538]: pgmap v612: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-05-14T17:14:41.296 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:14:33 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:14:41.328 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:14:34 smithi123 ceph-mon[35971]: pgmap v625: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 50 KiB/s wr, 15 op/s 2024-05-14T17:14:41.361 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:14:36 smithi123 ceph-mon[35971]: pgmap v626: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 29 KiB/s wr, 8 op/s 2024-05-14T17:14:41.394 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:14:38 smithi123 ceph-mon[35971]: pgmap v627: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 29 KiB/s wr, 7 op/s 2024-05-14T17:14:41.428 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:14:40 smithi123 ceph-mon[35971]: pgmap v628: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 41 KiB/s wr, 18 op/s 2024-05-14T17:14:41.462 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:14:32 smithi151 ceph-mon[36066]: pgmap v624: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 KiB/s wr, 10 op/s 2024-05-14T17:14:41.495 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:14:33 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:14:41.528 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:14:34 smithi151 ceph-mon[36066]: pgmap v625: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 50 KiB/s wr, 15 op/s 2024-05-14T17:14:41.561 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:14:36 smithi151 ceph-mon[36066]: pgmap v626: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 29 KiB/s wr, 8 op/s 2024-05-14T17:14:41.595 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:14:38 smithi151 ceph-mon[36066]: pgmap v627: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 29 KiB/s wr, 7 op/s 2024-05-14T17:14:41.645 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:14:40 smithi151 ceph-mon[36066]: pgmap v628: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 41 KiB/s wr, 18 op/s 2024-05-14T17:14:41.682 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/: 2024-05-14T17:14:41.713 INFO:teuthology.orchestra.run.smithi002.stdout:total 0 2024-05-14T17:14:41.745 INFO:teuthology.orchestra.run.smithi002.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 16:55 . 2024-05-14T17:14:41.778 INFO:teuthology.orchestra.run.smithi002.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 May 14 16:55 .. 2024-05-14T17:14:41.812 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 14 17:12 data 2024-05-14T17:14:41.845 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:14:41.878 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data: 2024-05-14T17:14:41.912 INFO:teuthology.orchestra.run.smithi002.stdout:total 36 2024-05-14T17:14:41.945 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 14 17:12 . 2024-05-14T17:14:41.978 INFO:teuthology.orchestra.run.smithi002.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 16:55 .. 2024-05-14T17:14:42.012 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 May 14 16:55 base 2024-05-14T17:14:42.045 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 May 14 16:55 current_logfiles 2024-05-14T17:14:42.078 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 May 14 16:55 global 2024-05-14T17:14:42.112 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 16:55 log 2024-05-14T17:14:42.145 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 pg_commit_ts 2024-05-14T17:14:42.188 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 pg_dynshmem 2024-05-14T17:14:42.220 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 May 14 16:55 pg_hba.conf 2024-05-14T17:14:42.254 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 May 14 16:55 pg_ident.conf 2024-05-14T17:14:42.287 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 17:12 pg_logical 2024-05-14T17:14:42.320 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 14 16:55 pg_multixact 2024-05-14T17:14:42.354 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 pg_notify 2024-05-14T17:14:42.387 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 pg_replslot 2024-05-14T17:14:42.420 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 pg_serial 2024-05-14T17:14:42.454 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 pg_snapshots 2024-05-14T17:14:42.487 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 17:12 pg_stat 2024-05-14T17:14:42.529 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 17:12 pg_stat_tmp 2024-05-14T17:14:42.596 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 17:12 pg_subtrans 2024-05-14T17:14:42.629 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 pg_tblspc 2024-05-14T17:14:42.662 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 pg_twophase 2024-05-14T17:14:42.696 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 16:55 PG_VERSION 2024-05-14T17:14:42.729 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 May 14 17:12 pg_wal 2024-05-14T17:14:42.787 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 14 17:03 pg_xact 2024-05-14T17:14:42.821 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 May 14 16:55 postgresql.auto.conf 2024-05-14T17:14:42.854 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 May 14 16:55 postgresql.conf 2024-05-14T17:14:42.887 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 May 14 16:55 postmaster.opts 2024-05-14T17:14:42.921 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:14:42.954 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/base: 2024-05-14T17:14:42.987 INFO:teuthology.orchestra.run.smithi002.stdout:total 0 2024-05-14T17:14:43.021 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 May 14 16:55 . 2024-05-14T17:14:43.054 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 14 17:12 .. 2024-05-14T17:14:43.088 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 May 14 16:55 1 2024-05-14T17:14:43.121 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 May 14 16:55 13493 2024-05-14T17:14:43.154 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 May 14 17:00 13494 2024-05-14T17:14:43.188 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 pgsql_tmp 2024-05-14T17:14:43.221 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:14:43.254 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/base/1: 2024-05-14T17:14:43.288 INFO:teuthology.orchestra.run.smithi002.stdout:total 7777 2024-05-14T17:14:43.329 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 May 14 16:55 . 2024-05-14T17:14:43.363 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 May 14 16:55 .. 2024-05-14T17:14:43.396 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 112 2024-05-14T17:14:43.429 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 113 2024-05-14T17:14:43.463 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 14 16:55 1247 2024-05-14T17:14:43.497 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 1247_fsm 2024-05-14T17:14:43.530 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1247_vm 2024-05-14T17:14:43.563 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 14 16:55 1249 2024-05-14T17:14:43.638 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 1249_fsm 2024-05-14T17:14:43.671 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1249_vm 2024-05-14T17:14:43.705 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 14 16:55 1255 2024-05-14T17:14:43.755 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 1255_fsm 2024-05-14T17:14:43.788 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1255_vm 2024-05-14T17:14:43.821 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 14 16:55 1259 2024-05-14T17:14:43.858 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 1259_fsm 2024-05-14T17:14:43.896 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1259_vm 2024-05-14T17:14:43.963 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 14 16:55 13344 2024-05-14T17:14:43.996 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 13344_fsm 2024-05-14T17:14:44.030 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13344_vm 2024-05-14T17:14:44.063 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 13346 2024-05-14T17:14:44.097 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13348 2024-05-14T17:14:44.130 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13349 2024-05-14T17:14:44.175 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 13349_fsm 2024-05-14T17:14:44.213 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13349_vm 2024-05-14T17:14:44.247 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 13351 2024-05-14T17:14:44.280 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13353 2024-05-14T17:14:44.313 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13354 2024-05-14T17:14:44.347 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 13354_fsm 2024-05-14T17:14:44.380 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13354_vm 2024-05-14T17:14:44.414 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 13356 2024-05-14T17:14:44.447 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13358 2024-05-14T17:14:44.480 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13359 2024-05-14T17:14:44.522 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 13359_fsm 2024-05-14T17:14:44.555 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13359_vm 2024-05-14T17:14:44.589 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 13361 2024-05-14T17:14:44.622 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13363 2024-05-14T17:14:44.655 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 1417 2024-05-14T17:14:44.689 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 1418 2024-05-14T17:14:44.722 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 174 2024-05-14T17:14:44.755 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 175 2024-05-14T17:14:44.789 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2187 2024-05-14T17:14:44.822 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2224 2024-05-14T17:14:44.856 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2328 2024-05-14T17:14:44.889 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2336 2024-05-14T17:14:44.922 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2337 2024-05-14T17:14:44.956 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2579 2024-05-14T17:14:44.989 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2600 2024-05-14T17:14:45.023 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2600_fsm 2024-05-14T17:14:45.056 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2600_vm 2024-05-14T17:14:45.114 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2601 2024-05-14T17:14:45.147 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2601_fsm 2024-05-14T17:14:45.181 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2601_vm 2024-05-14T17:14:45.215 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 14 16:55 2602 2024-05-14T17:14:45.248 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2602_fsm 2024-05-14T17:14:45.281 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2602_vm 2024-05-14T17:14:45.315 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2603 2024-05-14T17:14:45.360 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2603_fsm 2024-05-14T17:14:45.398 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2603_vm 2024-05-14T17:14:45.431 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2604 2024-05-14T17:14:45.465 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2605 2024-05-14T17:14:45.498 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2605_fsm 2024-05-14T17:14:45.532 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2605_vm 2024-05-14T17:14:45.565 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2606 2024-05-14T17:14:45.598 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2606_fsm 2024-05-14T17:14:45.640 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2606_vm 2024-05-14T17:14:45.673 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2607 2024-05-14T17:14:45.707 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2607_fsm 2024-05-14T17:14:45.740 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2607_vm 2024-05-14T17:14:45.773 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 14 16:55 2608 2024-05-14T17:14:45.807 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2608_fsm 2024-05-14T17:14:45.840 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2608_vm 2024-05-14T17:14:45.873 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 14 16:55 2609 2024-05-14T17:14:45.907 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2609_fsm 2024-05-14T17:14:45.940 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2609_vm 2024-05-14T17:14:45.974 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2610 2024-05-14T17:14:46.007 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2610_fsm 2024-05-14T17:14:46.040 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2610_vm 2024-05-14T17:14:46.074 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2611 2024-05-14T17:14:46.107 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2612 2024-05-14T17:14:46.149 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2612_fsm 2024-05-14T17:14:46.224 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2612_vm 2024-05-14T17:14:46.257 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2613 2024-05-14T17:14:46.290 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2615 2024-05-14T17:14:46.324 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2615_fsm 2024-05-14T17:14:46.357 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2615_vm 2024-05-14T17:14:46.390 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2616 2024-05-14T17:14:46.424 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2616_fsm 2024-05-14T17:14:46.457 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2616_vm 2024-05-14T17:14:46.490 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 14 16:55 2617 2024-05-14T17:14:46.524 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2617_fsm 2024-05-14T17:14:46.557 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2617_vm 2024-05-14T17:14:46.591 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 14 16:55 2618 2024-05-14T17:14:46.624 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2618_fsm 2024-05-14T17:14:46.658 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2618_vm 2024-05-14T17:14:46.691 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 14 16:55 2619 2024-05-14T17:14:46.724 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2619_fsm 2024-05-14T17:14:46.807 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2619_vm 2024-05-14T17:14:46.841 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2620 2024-05-14T17:14:46.874 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2650 2024-05-14T17:14:46.907 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2651 2024-05-14T17:14:46.941 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2652 2024-05-14T17:14:46.974 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:55 2653 2024-05-14T17:14:47.008 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:55 2654 2024-05-14T17:14:47.041 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2655 2024-05-14T17:14:47.074 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2656 2024-05-14T17:14:47.107 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2657 2024-05-14T17:14:47.141 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 14 16:55 2658 2024-05-14T17:14:47.175 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 14 16:55 2659 2024-05-14T17:14:47.208 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2660 2024-05-14T17:14:47.241 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2661 2024-05-14T17:14:47.275 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2662 2024-05-14T17:14:47.308 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 14 16:55 2663 2024-05-14T17:14:47.349 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2664 2024-05-14T17:14:47.383 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2665 2024-05-14T17:14:47.416 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2666 2024-05-14T17:14:47.450 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2667 2024-05-14T17:14:47.483 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2668 2024-05-14T17:14:47.516 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2669 2024-05-14T17:14:47.550 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2670 2024-05-14T17:14:47.583 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 14 16:55 2673 2024-05-14T17:14:47.616 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 14 16:55 2674 2024-05-14T17:14:47.650 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 14 16:55 2675 2024-05-14T17:14:47.683 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2678 2024-05-14T17:14:47.717 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2679 2024-05-14T17:14:47.750 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2680 2024-05-14T17:14:47.783 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2681 2024-05-14T17:14:47.816 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2682 2024-05-14T17:14:47.850 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2683 2024-05-14T17:14:47.908 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2684 2024-05-14T17:14:47.942 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2685 2024-05-14T17:14:47.975 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2686 2024-05-14T17:14:48.008 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2687 2024-05-14T17:14:48.042 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:55 2688 2024-05-14T17:14:48.094 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 14 16:55 2689 2024-05-14T17:14:48.125 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 14 16:55 2690 2024-05-14T17:14:48.158 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 14 16:55 2691 2024-05-14T17:14:48.192 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2692 2024-05-14T17:14:48.225 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2693 2024-05-14T17:14:48.259 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2696 2024-05-14T17:14:48.292 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2699 2024-05-14T17:14:48.350 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2701 2024-05-14T17:14:48.421 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2702 2024-05-14T17:14:48.479 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2703 2024-05-14T17:14:48.517 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:55 2704 2024-05-14T17:14:48.567 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2753 2024-05-14T17:14:48.601 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2753_fsm 2024-05-14T17:14:48.634 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2753_vm 2024-05-14T17:14:48.667 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2754 2024-05-14T17:14:48.700 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2755 2024-05-14T17:14:48.734 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2756 2024-05-14T17:14:48.781 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2757 2024-05-14T17:14:48.832 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2830 2024-05-14T17:14:48.867 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2831 2024-05-14T17:14:48.901 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2832 2024-05-14T17:14:48.934 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2833 2024-05-14T17:14:48.967 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2834 2024-05-14T17:14:49.001 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2835 2024-05-14T17:14:49.034 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2836 2024-05-14T17:14:49.068 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2837 2024-05-14T17:14:49.101 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 14 16:55 2838 2024-05-14T17:14:49.143 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2838_fsm 2024-05-14T17:14:49.176 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2838_vm 2024-05-14T17:14:49.209 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2839 2024-05-14T17:14:49.243 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2840 2024-05-14T17:14:49.276 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2840_fsm 2024-05-14T17:14:49.309 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2840_vm 2024-05-14T17:14:49.343 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2841 2024-05-14T17:14:49.376 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2995 2024-05-14T17:14:49.409 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2996 2024-05-14T17:14:49.443 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3079 2024-05-14T17:14:49.476 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3079_fsm 2024-05-14T17:14:49.510 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3079_vm 2024-05-14T17:14:49.543 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3080 2024-05-14T17:14:49.576 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3081 2024-05-14T17:14:49.612 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:55 3085 2024-05-14T17:14:49.661 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3118 2024-05-14T17:14:49.735 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3119 2024-05-14T17:14:49.768 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 14 16:55 3164 2024-05-14T17:14:49.818 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3256 2024-05-14T17:14:49.851 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3257 2024-05-14T17:14:49.885 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3258 2024-05-14T17:14:49.918 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3350 2024-05-14T17:14:49.952 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3351 2024-05-14T17:14:49.985 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3379 2024-05-14T17:14:50.036 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3380 2024-05-14T17:14:50.068 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3381 2024-05-14T17:14:50.102 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3394 2024-05-14T17:14:50.135 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3394_fsm 2024-05-14T17:14:50.168 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3394_vm 2024-05-14T17:14:50.202 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3395 2024-05-14T17:14:50.235 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3429 2024-05-14T17:14:50.268 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3430 2024-05-14T17:14:50.318 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3431 2024-05-14T17:14:50.352 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3433 2024-05-14T17:14:50.385 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3439 2024-05-14T17:14:50.419 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3440 2024-05-14T17:14:50.452 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:55 3455 2024-05-14T17:14:50.485 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 14 16:55 3456 2024-05-14T17:14:50.519 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3456_fsm 2024-05-14T17:14:50.552 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3456_vm 2024-05-14T17:14:50.585 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3466 2024-05-14T17:14:50.619 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3467 2024-05-14T17:14:50.652 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3468 2024-05-14T17:14:50.686 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3501 2024-05-14T17:14:50.719 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3502 2024-05-14T17:14:50.752 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3503 2024-05-14T17:14:50.785 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3534 2024-05-14T17:14:50.819 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3541 2024-05-14T17:14:50.860 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3541_fsm 2024-05-14T17:14:50.901 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3541_vm 2024-05-14T17:14:50.935 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3542 2024-05-14T17:14:50.969 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3574 2024-05-14T17:14:51.002 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3575 2024-05-14T17:14:51.036 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3576 2024-05-14T17:14:51.069 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3596 2024-05-14T17:14:51.102 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3597 2024-05-14T17:14:51.136 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3598 2024-05-14T17:14:51.169 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3599 2024-05-14T17:14:51.202 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3600 2024-05-14T17:14:51.236 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3600_fsm 2024-05-14T17:14:51.269 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3600_vm 2024-05-14T17:14:51.303 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3601 2024-05-14T17:14:51.336 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3601_fsm 2024-05-14T17:14:51.369 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3601_vm 2024-05-14T17:14:51.419 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3602 2024-05-14T17:14:51.464 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3602_fsm 2024-05-14T17:14:51.494 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3602_vm 2024-05-14T17:14:51.528 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3603 2024-05-14T17:14:51.561 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3603_fsm 2024-05-14T17:14:51.594 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3603_vm 2024-05-14T17:14:51.628 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3604 2024-05-14T17:14:51.661 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3605 2024-05-14T17:14:51.695 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3606 2024-05-14T17:14:51.728 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3607 2024-05-14T17:14:51.769 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3608 2024-05-14T17:14:51.803 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 3609 2024-05-14T17:14:51.836 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3712 2024-05-14T17:14:51.870 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3764 2024-05-14T17:14:51.903 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3764_fsm 2024-05-14T17:14:51.937 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3764_vm 2024-05-14T17:14:51.978 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3766 2024-05-14T17:14:52.012 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3767 2024-05-14T17:14:52.049 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3997 2024-05-14T17:14:52.110 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4143 2024-05-14T17:14:52.147 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4144 2024-05-14T17:14:52.178 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4145 2024-05-14T17:14:52.212 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4146 2024-05-14T17:14:52.245 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4147 2024-05-14T17:14:52.278 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4148 2024-05-14T17:14:52.312 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4149 2024-05-14T17:14:52.345 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4150 2024-05-14T17:14:52.379 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4151 2024-05-14T17:14:52.412 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4152 2024-05-14T17:14:52.445 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4153 2024-05-14T17:14:52.478 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4154 2024-05-14T17:14:52.512 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4155 2024-05-14T17:14:52.562 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4156 2024-05-14T17:14:52.595 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4157 2024-05-14T17:14:52.628 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4158 2024-05-14T17:14:52.662 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4159 2024-05-14T17:14:52.699 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4160 2024-05-14T17:14:52.737 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4161 2024-05-14T17:14:52.771 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4162 2024-05-14T17:14:52.804 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4163 2024-05-14T17:14:52.854 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4164 2024-05-14T17:14:52.887 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4165 2024-05-14T17:14:52.921 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4166 2024-05-14T17:14:52.954 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4167 2024-05-14T17:14:52.987 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4168 2024-05-14T17:14:53.021 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4169 2024-05-14T17:14:53.054 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4170 2024-05-14T17:14:53.087 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4171 2024-05-14T17:14:53.137 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4172 2024-05-14T17:14:53.171 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4173 2024-05-14T17:14:53.204 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4174 2024-05-14T17:14:53.237 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 5002 2024-05-14T17:14:53.271 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 548 2024-05-14T17:14:53.313 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 549 2024-05-14T17:14:53.356 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 6102 2024-05-14T17:14:53.388 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 6104 2024-05-14T17:14:53.421 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 6106 2024-05-14T17:14:53.454 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 6110 2024-05-14T17:14:53.488 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 6111 2024-05-14T17:14:53.521 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 6112 2024-05-14T17:14:53.554 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 6113 2024-05-14T17:14:53.588 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 6117 2024-05-14T17:14:53.621 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 826 2024-05-14T17:14:53.654 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 827 2024-05-14T17:14:53.704 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 828 2024-05-14T17:14:53.738 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 14 16:55 pg_filenode.map 2024-05-14T17:14:53.771 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 16:55 PG_VERSION 2024-05-14T17:14:53.804 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:14:53.838 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-05-14T17:14:53.871 INFO:teuthology.orchestra.run.smithi002.stdout:total 7777 2024-05-14T17:14:53.905 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 May 14 16:55 . 2024-05-14T17:14:53.938 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 May 14 16:55 .. 2024-05-14T17:14:53.972 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 112 2024-05-14T17:14:54.005 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 113 2024-05-14T17:14:54.038 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 14 16:55 1247 2024-05-14T17:14:54.072 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 1247_fsm 2024-05-14T17:14:54.105 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1247_vm 2024-05-14T17:14:54.138 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 14 16:55 1249 2024-05-14T17:14:54.171 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 1249_fsm 2024-05-14T17:14:54.205 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1249_vm 2024-05-14T17:14:54.255 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 14 16:55 1255 2024-05-14T17:14:54.289 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 1255_fsm 2024-05-14T17:14:54.322 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1255_vm 2024-05-14T17:14:54.355 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 14 16:55 1259 2024-05-14T17:14:54.389 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 1259_fsm 2024-05-14T17:14:54.422 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1259_vm 2024-05-14T17:14:54.455 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 14 16:55 13344 2024-05-14T17:14:54.496 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 13344_fsm 2024-05-14T17:14:54.534 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13344_vm 2024-05-14T17:14:54.564 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 13346 2024-05-14T17:14:54.597 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13348 2024-05-14T17:14:54.631 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13349 2024-05-14T17:14:54.664 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 13349_fsm 2024-05-14T17:14:54.697 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13349_vm 2024-05-14T17:14:54.731 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 13351 2024-05-14T17:14:54.764 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13353 2024-05-14T17:14:54.814 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13354 2024-05-14T17:14:54.847 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 13354_fsm 2024-05-14T17:14:54.881 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13354_vm 2024-05-14T17:14:54.914 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 13356 2024-05-14T17:14:54.948 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13358 2024-05-14T17:14:54.981 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13359 2024-05-14T17:14:55.014 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 13359_fsm 2024-05-14T17:14:55.048 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13359_vm 2024-05-14T17:14:55.081 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 13361 2024-05-14T17:14:55.130 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13363 2024-05-14T17:14:55.164 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 1417 2024-05-14T17:14:55.198 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 1418 2024-05-14T17:14:55.231 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 174 2024-05-14T17:14:55.264 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 175 2024-05-14T17:14:55.298 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2187 2024-05-14T17:14:55.331 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2224 2024-05-14T17:14:55.381 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2328 2024-05-14T17:14:55.415 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2336 2024-05-14T17:14:55.448 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2337 2024-05-14T17:14:55.481 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2579 2024-05-14T17:14:55.514 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2600 2024-05-14T17:14:55.548 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2600_fsm 2024-05-14T17:14:55.581 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2600_vm 2024-05-14T17:14:55.615 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2601 2024-05-14T17:14:55.648 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2601_fsm 2024-05-14T17:14:55.691 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2601_vm 2024-05-14T17:14:55.723 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 14 16:55 2602 2024-05-14T17:14:55.756 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2602_fsm 2024-05-14T17:14:55.790 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2602_vm 2024-05-14T17:14:55.823 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2603 2024-05-14T17:14:55.932 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2603_fsm 2024-05-14T17:14:55.965 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2603_vm 2024-05-14T17:14:56.023 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2604 2024-05-14T17:14:56.057 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2605 2024-05-14T17:14:56.090 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2605_fsm 2024-05-14T17:14:56.124 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2605_vm 2024-05-14T17:14:56.157 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2606 2024-05-14T17:14:56.190 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2606_fsm 2024-05-14T17:14:56.223 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2606_vm 2024-05-14T17:14:56.257 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2607 2024-05-14T17:14:56.315 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2607_fsm 2024-05-14T17:14:56.348 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2607_vm 2024-05-14T17:14:56.382 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 14 16:55 2608 2024-05-14T17:14:56.430 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2608_fsm 2024-05-14T17:14:56.465 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2608_vm 2024-05-14T17:14:56.498 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 14 16:55 2609 2024-05-14T17:14:56.532 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2609_fsm 2024-05-14T17:14:56.569 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2609_vm 2024-05-14T17:14:56.624 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2610 2024-05-14T17:14:56.657 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2610_fsm 2024-05-14T17:14:56.690 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2610_vm 2024-05-14T17:14:56.724 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2611 2024-05-14T17:14:56.757 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2612 2024-05-14T17:14:56.790 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2612_fsm 2024-05-14T17:14:56.824 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2612_vm 2024-05-14T17:14:56.857 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2613 2024-05-14T17:14:56.891 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2615 2024-05-14T17:14:56.932 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2615_fsm 2024-05-14T17:14:56.966 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2615_vm 2024-05-14T17:14:56.999 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2616 2024-05-14T17:14:57.032 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2616_fsm 2024-05-14T17:14:57.066 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2616_vm 2024-05-14T17:14:57.099 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 14 16:55 2617 2024-05-14T17:14:57.133 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2617_fsm 2024-05-14T17:14:57.174 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2617_vm 2024-05-14T17:14:57.208 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 14 16:55 2618 2024-05-14T17:14:57.241 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2618_fsm 2024-05-14T17:14:57.274 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2618_vm 2024-05-14T17:14:57.308 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 14 16:55 2619 2024-05-14T17:14:57.341 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2619_fsm 2024-05-14T17:14:57.374 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2619_vm 2024-05-14T17:14:57.408 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2620 2024-05-14T17:14:57.441 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2650 2024-05-14T17:14:57.474 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2651 2024-05-14T17:14:57.508 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2652 2024-05-14T17:14:57.549 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:55 2653 2024-05-14T17:14:57.583 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:55 2654 2024-05-14T17:14:57.616 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2655 2024-05-14T17:14:57.650 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2656 2024-05-14T17:14:57.683 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2657 2024-05-14T17:14:57.733 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 14 16:55 2658 2024-05-14T17:14:57.766 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 14 16:55 2659 2024-05-14T17:14:57.800 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2660 2024-05-14T17:14:57.833 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2661 2024-05-14T17:14:57.866 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2662 2024-05-14T17:14:57.900 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 14 16:55 2663 2024-05-14T17:14:57.933 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2664 2024-05-14T17:14:57.966 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2665 2024-05-14T17:14:58.000 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2666 2024-05-14T17:14:58.033 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2667 2024-05-14T17:14:58.067 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2668 2024-05-14T17:14:58.100 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2669 2024-05-14T17:14:58.133 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2670 2024-05-14T17:14:58.167 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 14 16:55 2673 2024-05-14T17:14:58.200 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 14 16:55 2674 2024-05-14T17:14:58.233 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 14 16:55 2675 2024-05-14T17:14:58.275 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2678 2024-05-14T17:14:58.308 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2679 2024-05-14T17:14:58.344 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2680 2024-05-14T17:14:58.378 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2681 2024-05-14T17:14:58.411 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2682 2024-05-14T17:14:58.445 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2683 2024-05-14T17:14:58.478 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2684 2024-05-14T17:14:58.512 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2685 2024-05-14T17:14:58.545 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2686 2024-05-14T17:14:58.578 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2687 2024-05-14T17:14:58.612 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:55 2688 2024-05-14T17:14:58.645 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 14 16:55 2689 2024-05-14T17:14:58.678 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 14 16:55 2690 2024-05-14T17:14:58.712 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 14 16:55 2691 2024-05-14T17:14:58.745 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2692 2024-05-14T17:14:58.778 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2693 2024-05-14T17:14:58.845 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2696 2024-05-14T17:14:58.878 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2699 2024-05-14T17:14:58.912 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2701 2024-05-14T17:14:58.945 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2702 2024-05-14T17:14:58.978 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2703 2024-05-14T17:14:59.012 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:55 2704 2024-05-14T17:14:59.045 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2753 2024-05-14T17:14:59.079 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2753_fsm 2024-05-14T17:14:59.112 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2753_vm 2024-05-14T17:14:59.145 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2754 2024-05-14T17:14:59.179 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2755 2024-05-14T17:14:59.212 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2756 2024-05-14T17:14:59.245 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2757 2024-05-14T17:14:59.279 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2830 2024-05-14T17:14:59.312 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2831 2024-05-14T17:14:59.354 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2832 2024-05-14T17:14:59.412 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2833 2024-05-14T17:14:59.446 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2834 2024-05-14T17:14:59.479 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2835 2024-05-14T17:14:59.513 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2836 2024-05-14T17:14:59.571 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2837 2024-05-14T17:14:59.604 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 14 16:55 2838 2024-05-14T17:14:59.637 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2838_fsm 2024-05-14T17:14:59.671 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2838_vm 2024-05-14T17:14:59.704 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2839 2024-05-14T17:14:59.738 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2840 2024-05-14T17:14:59.771 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2840_fsm 2024-05-14T17:14:59.805 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2840_vm 2024-05-14T17:14:59.838 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2841 2024-05-14T17:14:59.871 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2995 2024-05-14T17:14:59.905 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2996 2024-05-14T17:14:59.938 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3079 2024-05-14T17:14:59.988 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3079_fsm 2024-05-14T17:15:00.022 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3079_vm 2024-05-14T17:15:00.055 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3080 2024-05-14T17:15:00.088 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3081 2024-05-14T17:15:00.122 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:55 3085 2024-05-14T17:15:00.155 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3118 2024-05-14T17:15:00.188 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3119 2024-05-14T17:15:00.222 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 14 16:55 3164 2024-05-14T17:15:00.274 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3256 2024-05-14T17:15:00.314 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3257 2024-05-14T17:15:00.347 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3258 2024-05-14T17:15:00.380 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3350 2024-05-14T17:15:00.414 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3351 2024-05-14T17:15:00.447 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3379 2024-05-14T17:15:00.481 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3380 2024-05-14T17:15:00.522 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3381 2024-05-14T17:15:00.572 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3394 2024-05-14T17:15:00.614 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3394_fsm 2024-05-14T17:15:00.659 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3394_vm 2024-05-14T17:15:00.697 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3395 2024-05-14T17:15:00.731 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3429 2024-05-14T17:15:00.764 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3430 2024-05-14T17:15:00.797 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3431 2024-05-14T17:15:00.831 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3433 2024-05-14T17:15:00.870 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3439 2024-05-14T17:15:00.906 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3440 2024-05-14T17:15:00.939 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:55 3455 2024-05-14T17:15:00.973 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 14 16:55 3456 2024-05-14T17:15:01.006 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3456_fsm 2024-05-14T17:15:01.039 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3456_vm 2024-05-14T17:15:01.072 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3466 2024-05-14T17:15:01.106 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3467 2024-05-14T17:15:01.148 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3468 2024-05-14T17:15:01.188 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3501 2024-05-14T17:15:01.223 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3502 2024-05-14T17:15:01.256 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3503 2024-05-14T17:15:01.289 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3534 2024-05-14T17:15:01.323 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3541 2024-05-14T17:15:01.356 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3541_fsm 2024-05-14T17:15:01.390 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3541_vm 2024-05-14T17:15:01.423 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3542 2024-05-14T17:15:01.456 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3574 2024-05-14T17:15:01.490 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3575 2024-05-14T17:15:01.523 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3576 2024-05-14T17:15:01.556 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3596 2024-05-14T17:15:01.590 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3597 2024-05-14T17:15:01.623 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3598 2024-05-14T17:15:01.656 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3599 2024-05-14T17:15:01.706 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3600 2024-05-14T17:15:01.740 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3600_fsm 2024-05-14T17:15:01.788 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3600_vm 2024-05-14T17:15:01.823 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3601 2024-05-14T17:15:01.876 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3601_fsm 2024-05-14T17:15:01.915 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3601_vm 2024-05-14T17:15:01.948 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3602 2024-05-14T17:15:01.982 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3602_fsm 2024-05-14T17:15:02.015 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3602_vm 2024-05-14T17:15:02.048 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3603 2024-05-14T17:15:02.082 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3603_fsm 2024-05-14T17:15:02.129 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3603_vm 2024-05-14T17:15:02.165 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3604 2024-05-14T17:15:02.199 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3605 2024-05-14T17:15:02.232 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3606 2024-05-14T17:15:02.265 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3607 2024-05-14T17:15:02.307 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3608 2024-05-14T17:15:02.340 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 3609 2024-05-14T17:15:02.407 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3712 2024-05-14T17:15:02.440 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3764 2024-05-14T17:15:02.481 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3764_fsm 2024-05-14T17:15:02.516 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3764_vm 2024-05-14T17:15:02.549 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3766 2024-05-14T17:15:02.582 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3767 2024-05-14T17:15:02.616 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3997 2024-05-14T17:15:02.649 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4143 2024-05-14T17:15:02.682 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4144 2024-05-14T17:15:02.716 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4145 2024-05-14T17:15:02.749 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4146 2024-05-14T17:15:02.783 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4147 2024-05-14T17:15:02.816 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4148 2024-05-14T17:15:02.849 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4149 2024-05-14T17:15:02.891 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4150 2024-05-14T17:15:02.924 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4151 2024-05-14T17:15:02.958 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4152 2024-05-14T17:15:02.991 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4153 2024-05-14T17:15:03.047 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4154 2024-05-14T17:15:03.087 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4155 2024-05-14T17:15:03.131 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4156 2024-05-14T17:15:03.166 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4157 2024-05-14T17:15:03.200 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4158 2024-05-14T17:15:03.233 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4159 2024-05-14T17:15:03.266 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4160 2024-05-14T17:15:03.300 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4161 2024-05-14T17:15:03.333 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4162 2024-05-14T17:15:03.366 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4163 2024-05-14T17:15:03.400 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4164 2024-05-14T17:15:03.433 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4165 2024-05-14T17:15:03.475 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4166 2024-05-14T17:15:03.508 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4167 2024-05-14T17:15:03.542 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4168 2024-05-14T17:15:03.575 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4169 2024-05-14T17:15:03.608 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4170 2024-05-14T17:15:03.647 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4171 2024-05-14T17:15:03.686 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4172 2024-05-14T17:15:03.725 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4173 2024-05-14T17:15:03.758 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4174 2024-05-14T17:15:03.792 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 5002 2024-05-14T17:15:03.825 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 548 2024-05-14T17:15:03.859 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 549 2024-05-14T17:15:03.892 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 6102 2024-05-14T17:15:03.925 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 6104 2024-05-14T17:15:03.959 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 6106 2024-05-14T17:15:04.000 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 6110 2024-05-14T17:15:04.042 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 6111 2024-05-14T17:15:04.075 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 6112 2024-05-14T17:15:04.109 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 6113 2024-05-14T17:15:04.142 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 6117 2024-05-14T17:15:04.175 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 826 2024-05-14T17:15:04.209 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 827 2024-05-14T17:15:04.242 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 828 2024-05-14T17:15:04.285 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 14 16:55 pg_filenode.map 2024-05-14T17:15:04.317 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 16:55 PG_VERSION 2024-05-14T17:15:04.351 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:15:04.384 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-05-14T17:15:04.417 INFO:teuthology.orchestra.run.smithi002.stdout:total 596894 2024-05-14T17:15:04.451 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 May 14 17:00 . 2024-05-14T17:15:04.484 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 May 14 16:55 .. 2024-05-14T17:15:04.517 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 112 2024-05-14T17:15:04.559 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 113 2024-05-14T17:15:04.634 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 14 16:55 1247 2024-05-14T17:15:04.668 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 1247_fsm 2024-05-14T17:15:04.701 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1247_vm 2024-05-14T17:15:04.739 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 14 16:55 1249 2024-05-14T17:15:04.776 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 1249_fsm 2024-05-14T17:15:04.809 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1249_vm 2024-05-14T17:15:04.895 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 14 16:55 1255 2024-05-14T17:15:04.940 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 1255_fsm 2024-05-14T17:15:04.982 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1255_vm 2024-05-14T17:15:05.074 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 14 17:12 1259 2024-05-14T17:15:05.110 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 1259_fsm 2024-05-14T17:15:05.143 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1259_vm 2024-05-14T17:15:05.176 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 14 16:55 13344 2024-05-14T17:15:05.210 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 13344_fsm 2024-05-14T17:15:05.243 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13344_vm 2024-05-14T17:15:05.276 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 13346 2024-05-14T17:15:05.324 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13348 2024-05-14T17:15:05.365 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13349 2024-05-14T17:15:05.477 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 13349_fsm 2024-05-14T17:15:05.510 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13349_vm 2024-05-14T17:15:05.543 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 13351 2024-05-14T17:15:05.577 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13353 2024-05-14T17:15:05.610 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13354 2024-05-14T17:15:05.644 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 13354_fsm 2024-05-14T17:15:05.677 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13354_vm 2024-05-14T17:15:05.733 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 13356 2024-05-14T17:15:05.769 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13358 2024-05-14T17:15:05.802 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13359 2024-05-14T17:15:05.844 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 13359_fsm 2024-05-14T17:15:05.877 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13359_vm 2024-05-14T17:15:05.910 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 13361 2024-05-14T17:15:05.944 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 13363 2024-05-14T17:15:05.994 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 1417 2024-05-14T17:15:06.051 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 1418 2024-05-14T17:15:06.085 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 443482112 May 14 17:12 16396 2024-05-14T17:15:06.119 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 May 14 17:12 16396_fsm 2024-05-14T17:15:06.152 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:59 16396_vm 2024-05-14T17:15:06.185 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 647168 May 14 17:12 16397 2024-05-14T17:15:06.219 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 17:12 16397_fsm 2024-05-14T17:15:06.252 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 17:12 16397_vm 2024-05-14T17:15:06.286 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 2187264 May 14 17:12 16399 2024-05-14T17:15:06.326 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 17:12 16399_fsm 2024-05-14T17:15:06.361 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 17:12 16399_vm 2024-05-14T17:15:06.401 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 17:12 16400 2024-05-14T17:15:06.436 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 14 17:12 16402 2024-05-14T17:15:06.469 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 May 14 17:12 16404 2024-05-14T17:15:06.511 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 84492288 May 14 17:12 16406 2024-05-14T17:15:06.544 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 17:12 16406_fsm 2024-05-14T17:15:06.586 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 17:09 16406_vm 2024-05-14T17:15:06.619 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 174 2024-05-14T17:15:06.653 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 175 2024-05-14T17:15:06.686 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2187 2024-05-14T17:15:06.719 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2224 2024-05-14T17:15:06.753 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2328 2024-05-14T17:15:06.786 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2336 2024-05-14T17:15:06.819 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2337 2024-05-14T17:15:06.853 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2579 2024-05-14T17:15:06.886 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2600 2024-05-14T17:15:06.928 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2600_fsm 2024-05-14T17:15:06.970 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2600_vm 2024-05-14T17:15:07.011 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2601 2024-05-14T17:15:07.053 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2601_fsm 2024-05-14T17:15:07.087 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2601_vm 2024-05-14T17:15:07.128 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 14 16:55 2602 2024-05-14T17:15:07.178 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2602_fsm 2024-05-14T17:15:07.212 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2602_vm 2024-05-14T17:15:07.253 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2603 2024-05-14T17:15:07.287 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2603_fsm 2024-05-14T17:15:07.328 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2603_vm 2024-05-14T17:15:07.370 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2604 2024-05-14T17:15:07.412 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2605 2024-05-14T17:15:07.454 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2605_fsm 2024-05-14T17:15:07.496 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2605_vm 2024-05-14T17:15:07.537 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2606 2024-05-14T17:15:07.579 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2606_fsm 2024-05-14T17:15:07.621 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2606_vm 2024-05-14T17:15:07.662 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2607 2024-05-14T17:15:07.704 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2607_fsm 2024-05-14T17:15:07.746 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2607_vm 2024-05-14T17:15:07.788 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 14 16:55 2608 2024-05-14T17:15:07.846 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2608_fsm 2024-05-14T17:15:07.888 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2608_vm 2024-05-14T17:15:07.929 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 14 16:55 2609 2024-05-14T17:15:07.971 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2609_fsm 2024-05-14T17:15:08.013 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2609_vm 2024-05-14T17:15:08.054 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2610 2024-05-14T17:15:08.096 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2610_fsm 2024-05-14T17:15:08.138 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2610_vm 2024-05-14T17:15:08.180 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2611 2024-05-14T17:15:08.221 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2612 2024-05-14T17:15:08.263 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2612_fsm 2024-05-14T17:15:08.305 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2612_vm 2024-05-14T17:15:08.346 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2613 2024-05-14T17:15:08.388 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2615 2024-05-14T17:15:08.430 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2615_fsm 2024-05-14T17:15:08.471 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2615_vm 2024-05-14T17:15:08.521 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2616 2024-05-14T17:15:08.563 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2616_fsm 2024-05-14T17:15:08.605 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2616_vm 2024-05-14T17:15:08.646 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 14 16:55 2617 2024-05-14T17:15:08.688 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2617_fsm 2024-05-14T17:15:08.730 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2617_vm 2024-05-14T17:15:08.772 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 14 16:55 2618 2024-05-14T17:15:08.813 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2618_fsm 2024-05-14T17:15:08.872 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2618_vm 2024-05-14T17:15:08.913 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 188416 May 14 17:12 2619 2024-05-14T17:15:08.955 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 17:07 2619_fsm 2024-05-14T17:15:08.997 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 17:04 2619_vm 2024-05-14T17:15:09.030 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2620 2024-05-14T17:15:09.066 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2650 2024-05-14T17:15:09.105 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2651 2024-05-14T17:15:09.147 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2652 2024-05-14T17:15:09.197 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:55 2653 2024-05-14T17:15:09.239 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:55 2654 2024-05-14T17:15:09.280 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2655 2024-05-14T17:15:09.322 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2656 2024-05-14T17:15:09.364 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2657 2024-05-14T17:15:09.405 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 14 16:55 2658 2024-05-14T17:15:09.439 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 14 16:55 2659 2024-05-14T17:15:09.480 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2660 2024-05-14T17:15:09.522 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2661 2024-05-14T17:15:09.564 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2662 2024-05-14T17:15:09.597 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 14 16:55 2663 2024-05-14T17:15:09.630 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2664 2024-05-14T17:15:09.664 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2665 2024-05-14T17:15:09.697 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2666 2024-05-14T17:15:09.739 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2667 2024-05-14T17:15:09.781 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2668 2024-05-14T17:15:09.831 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2669 2024-05-14T17:15:09.872 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2670 2024-05-14T17:15:09.914 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 14 16:55 2673 2024-05-14T17:15:09.956 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 14 16:55 2674 2024-05-14T17:15:09.998 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 14 16:55 2675 2024-05-14T17:15:10.039 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2678 2024-05-14T17:15:10.081 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2679 2024-05-14T17:15:10.123 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2680 2024-05-14T17:15:10.164 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2681 2024-05-14T17:15:10.199 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2682 2024-05-14T17:15:10.239 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2683 2024-05-14T17:15:10.281 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2684 2024-05-14T17:15:10.323 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2685 2024-05-14T17:15:10.364 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2686 2024-05-14T17:15:10.406 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2687 2024-05-14T17:15:10.456 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:55 2688 2024-05-14T17:15:10.548 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 14 16:55 2689 2024-05-14T17:15:10.590 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 14 16:55 2690 2024-05-14T17:15:10.631 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 14 16:55 2691 2024-05-14T17:15:10.673 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2692 2024-05-14T17:15:10.715 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2693 2024-05-14T17:15:10.756 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 17:08 2696 2024-05-14T17:15:10.798 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2699 2024-05-14T17:15:10.840 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2701 2024-05-14T17:15:10.881 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2702 2024-05-14T17:15:10.923 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2703 2024-05-14T17:15:10.965 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:55 2704 2024-05-14T17:15:11.007 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2753 2024-05-14T17:15:11.048 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2753_fsm 2024-05-14T17:15:11.090 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2753_vm 2024-05-14T17:15:11.132 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2754 2024-05-14T17:15:11.173 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2755 2024-05-14T17:15:11.232 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2756 2024-05-14T17:15:11.274 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 2757 2024-05-14T17:15:11.315 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2830 2024-05-14T17:15:11.357 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2831 2024-05-14T17:15:11.398 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2832 2024-05-14T17:15:11.432 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2833 2024-05-14T17:15:11.474 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2834 2024-05-14T17:15:11.515 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2835 2024-05-14T17:15:11.557 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2836 2024-05-14T17:15:11.599 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2837 2024-05-14T17:15:11.640 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 14 16:55 2838 2024-05-14T17:15:11.682 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2838_fsm 2024-05-14T17:15:11.724 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2838_vm 2024-05-14T17:15:11.766 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2839 2024-05-14T17:15:11.807 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2840 2024-05-14T17:15:11.849 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2840_fsm 2024-05-14T17:15:11.999 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2840_vm 2024-05-14T17:15:12.042 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2841 2024-05-14T17:15:12.090 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2995 2024-05-14T17:15:12.129 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2996 2024-05-14T17:15:12.166 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3079 2024-05-14T17:15:12.199 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3079_fsm 2024-05-14T17:15:12.241 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3079_vm 2024-05-14T17:15:12.282 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3080 2024-05-14T17:15:12.349 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3081 2024-05-14T17:15:12.391 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:55 3085 2024-05-14T17:15:12.432 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3118 2024-05-14T17:15:12.466 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3119 2024-05-14T17:15:12.508 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 14 16:55 3164 2024-05-14T17:15:12.541 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3256 2024-05-14T17:15:12.583 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3257 2024-05-14T17:15:12.624 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3258 2024-05-14T17:15:12.683 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3350 2024-05-14T17:15:12.724 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3351 2024-05-14T17:15:12.776 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3379 2024-05-14T17:15:12.818 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3380 2024-05-14T17:15:12.860 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3381 2024-05-14T17:15:12.900 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3394 2024-05-14T17:15:12.941 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3394_fsm 2024-05-14T17:15:12.983 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3394_vm 2024-05-14T17:15:13.025 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3395 2024-05-14T17:15:13.067 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3429 2024-05-14T17:15:13.108 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3430 2024-05-14T17:15:13.150 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3431 2024-05-14T17:15:13.183 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3433 2024-05-14T17:15:13.225 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3439 2024-05-14T17:15:13.267 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3440 2024-05-14T17:15:13.308 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 14 16:55 3455 2024-05-14T17:15:13.367 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 14 16:55 3456 2024-05-14T17:15:13.408 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3456_fsm 2024-05-14T17:15:13.450 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3456_vm 2024-05-14T17:15:13.494 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3466 2024-05-14T17:15:13.546 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3467 2024-05-14T17:15:13.594 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3468 2024-05-14T17:15:13.634 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3501 2024-05-14T17:15:13.675 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3502 2024-05-14T17:15:13.717 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3503 2024-05-14T17:15:13.759 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3534 2024-05-14T17:15:13.800 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3541 2024-05-14T17:15:13.842 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3541_fsm 2024-05-14T17:15:13.951 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3541_vm 2024-05-14T17:15:13.992 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3542 2024-05-14T17:15:14.034 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3574 2024-05-14T17:15:14.075 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3575 2024-05-14T17:15:14.134 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3576 2024-05-14T17:15:14.176 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3596 2024-05-14T17:15:14.217 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3597 2024-05-14T17:15:14.259 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3598 2024-05-14T17:15:14.301 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3599 2024-05-14T17:15:14.344 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3600 2024-05-14T17:15:14.384 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3600_fsm 2024-05-14T17:15:14.426 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3600_vm 2024-05-14T17:15:14.468 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3601 2024-05-14T17:15:14.509 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3601_fsm 2024-05-14T17:15:14.551 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3601_vm 2024-05-14T17:15:14.592 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3602 2024-05-14T17:15:14.634 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3602_fsm 2024-05-14T17:15:14.676 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3602_vm 2024-05-14T17:15:14.718 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3603 2024-05-14T17:15:14.759 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3603_fsm 2024-05-14T17:15:14.809 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3603_vm 2024-05-14T17:15:14.851 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3604 2024-05-14T17:15:14.893 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3605 2024-05-14T17:15:14.934 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3606 2024-05-14T17:15:14.976 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3607 2024-05-14T17:15:15.018 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3608 2024-05-14T17:15:15.060 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 14 16:55 3609 2024-05-14T17:15:15.109 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3712 2024-05-14T17:15:15.152 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3764 2024-05-14T17:15:15.193 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 3764_fsm 2024-05-14T17:15:15.235 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3764_vm 2024-05-14T17:15:15.276 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3766 2024-05-14T17:15:15.318 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 3767 2024-05-14T17:15:15.377 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3997 2024-05-14T17:15:15.418 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4143 2024-05-14T17:15:15.460 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4144 2024-05-14T17:15:15.518 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4145 2024-05-14T17:15:15.560 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4146 2024-05-14T17:15:15.602 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4147 2024-05-14T17:15:15.643 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4148 2024-05-14T17:15:15.685 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4149 2024-05-14T17:15:15.727 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4150 2024-05-14T17:15:15.803 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4151 2024-05-14T17:15:15.872 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4152 2024-05-14T17:15:15.910 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4153 2024-05-14T17:15:15.952 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4154 2024-05-14T17:15:15.994 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4155 2024-05-14T17:15:16.035 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4156 2024-05-14T17:15:16.077 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4157 2024-05-14T17:15:16.119 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4158 2024-05-14T17:15:16.160 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4159 2024-05-14T17:15:16.202 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4160 2024-05-14T17:15:16.252 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4161 2024-05-14T17:15:16.294 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4162 2024-05-14T17:15:16.336 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4163 2024-05-14T17:15:16.377 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4164 2024-05-14T17:15:16.419 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4165 2024-05-14T17:15:16.461 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4166 2024-05-14T17:15:16.503 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4167 2024-05-14T17:15:16.544 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4168 2024-05-14T17:15:16.586 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4169 2024-05-14T17:15:16.628 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4170 2024-05-14T17:15:16.670 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4171 2024-05-14T17:15:16.711 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4172 2024-05-14T17:15:16.753 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4173 2024-05-14T17:15:16.795 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4174 2024-05-14T17:15:16.837 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 5002 2024-05-14T17:15:16.878 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 548 2024-05-14T17:15:16.945 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 549 2024-05-14T17:15:16.986 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 6102 2024-05-14T17:15:17.028 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 6104 2024-05-14T17:15:17.078 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 6106 2024-05-14T17:15:17.128 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 6110 2024-05-14T17:15:17.178 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 6111 2024-05-14T17:15:17.220 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 6112 2024-05-14T17:15:17.270 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 6113 2024-05-14T17:15:17.312 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 6117 2024-05-14T17:15:17.353 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 826 2024-05-14T17:15:17.403 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 827 2024-05-14T17:15:17.453 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 828 2024-05-14T17:15:17.495 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 14 16:55 pg_filenode.map 2024-05-14T17:15:17.545 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 151596 May 14 16:55 pg_internal.init 2024-05-14T17:15:17.595 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 16:55 PG_VERSION 2024-05-14T17:15:17.645 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:15:17.695 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2024-05-14T17:15:17.745 INFO:teuthology.orchestra.run.smithi002.stdout:total 0 2024-05-14T17:15:17.787 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 . 2024-05-14T17:15:17.829 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 May 14 16:55 .. 2024-05-14T17:15:17.870 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:15:17.912 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/global: 2024-05-14T17:15:17.962 INFO:teuthology.orchestra.run.smithi002.stdout:total 560 2024-05-14T17:15:18.012 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 May 14 16:55 . 2024-05-14T17:15:18.062 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 14 17:12 .. 2024-05-14T17:15:18.112 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1213 2024-05-14T17:15:18.162 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 1213_fsm 2024-05-14T17:15:18.214 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1213_vm 2024-05-14T17:15:18.262 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1214 2024-05-14T17:15:18.312 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 1214_fsm 2024-05-14T17:15:18.362 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1214_vm 2024-05-14T17:15:18.471 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 1232 2024-05-14T17:15:18.504 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 1233 2024-05-14T17:15:18.588 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1260 2024-05-14T17:15:18.625 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 1260_fsm 2024-05-14T17:15:18.671 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1260_vm 2024-05-14T17:15:18.721 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1261 2024-05-14T17:15:18.763 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 1261_fsm 2024-05-14T17:15:18.804 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1261_vm 2024-05-14T17:15:18.863 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1262 2024-05-14T17:15:18.896 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 1262_fsm 2024-05-14T17:15:18.929 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 1262_vm 2024-05-14T17:15:18.997 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2396 2024-05-14T17:15:19.042 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 14 16:55 2396_fsm 2024-05-14T17:15:19.099 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2396_vm 2024-05-14T17:15:19.146 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2397 2024-05-14T17:15:19.188 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2671 2024-05-14T17:15:19.230 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2672 2024-05-14T17:15:19.327 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2676 2024-05-14T17:15:19.454 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2677 2024-05-14T17:15:19.488 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2694 2024-05-14T17:15:19.568 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2695 2024-05-14T17:15:19.604 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2697 2024-05-14T17:15:19.638 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 14 16:55 2698 2024-05-14T17:15:19.671 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2846 2024-05-14T17:15:19.704 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2847 2024-05-14T17:15:19.738 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2964 2024-05-14T17:15:19.771 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2965 2024-05-14T17:15:19.805 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 2966 2024-05-14T17:15:19.838 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 2967 2024-05-14T17:15:19.871 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 3592 2024-05-14T17:15:19.909 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 3593 2024-05-14T17:15:19.946 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4060 2024-05-14T17:15:19.980 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4061 2024-05-14T17:15:20.021 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4175 2024-05-14T17:15:20.055 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4176 2024-05-14T17:15:20.088 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4177 2024-05-14T17:15:20.121 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4178 2024-05-14T17:15:20.155 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4181 2024-05-14T17:15:20.197 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4182 2024-05-14T17:15:20.230 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4183 2024-05-14T17:15:20.272 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4184 2024-05-14T17:15:20.314 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 4185 2024-05-14T17:15:20.347 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 4186 2024-05-14T17:15:20.389 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 6000 2024-05-14T17:15:20.422 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 6001 2024-05-14T17:15:20.464 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 6002 2024-05-14T17:15:20.505 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 6100 2024-05-14T17:15:20.539 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 6114 2024-05-14T17:15:20.580 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 6115 2024-05-14T17:15:20.631 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 17:12 pg_control 2024-05-14T17:15:20.672 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 14 16:55 pg_filenode.map 2024-05-14T17:15:20.714 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 23144 May 14 16:55 pg_internal.init 2024-05-14T17:15:20.756 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:15:20.797 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/log: 2024-05-14T17:15:20.839 INFO:teuthology.orchestra.run.smithi002.stdout:total 2 2024-05-14T17:15:20.872 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 16:55 . 2024-05-14T17:15:20.906 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 14 17:12 .. 2024-05-14T17:15:20.947 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1249 May 14 17:12 postgresql-Tue.log 2024-05-14T17:15:20.981 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:15:21.031 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-05-14T17:15:21.064 INFO:teuthology.orchestra.run.smithi002.stdout:total 0 2024-05-14T17:15:21.097 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 . 2024-05-14T17:15:21.131 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 14 17:12 .. 2024-05-14T17:15:21.164 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:15:21.198 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-05-14T17:15:21.248 INFO:teuthology.orchestra.run.smithi002.stdout:total 0 2024-05-14T17:15:21.281 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 . 2024-05-14T17:15:21.314 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 14 17:12 .. 2024-05-14T17:15:21.348 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:15:21.381 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-05-14T17:15:21.414 INFO:teuthology.orchestra.run.smithi002.stdout:total 1 2024-05-14T17:15:21.448 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 17:12 . 2024-05-14T17:15:21.481 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 14 17:12 .. 2024-05-14T17:15:21.515 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 mappings 2024-05-14T17:15:21.548 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 14 17:12 replorigin_checkpoint 2024-05-14T17:15:21.582 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 snapshots 2024-05-14T17:15:21.615 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:15:21.648 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-05-14T17:15:21.681 INFO:teuthology.orchestra.run.smithi002.stdout:total 0 2024-05-14T17:15:21.715 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 . 2024-05-14T17:15:21.749 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 17:12 .. 2024-05-14T17:15:21.790 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:15:21.823 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-05-14T17:15:21.857 INFO:teuthology.orchestra.run.smithi002.stdout:total 0 2024-05-14T17:15:21.965 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 . 2024-05-14T17:15:21.998 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 17:12 .. 2024-05-14T17:15:22.031 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:15:22.065 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-05-14T17:15:22.098 INFO:teuthology.orchestra.run.smithi002.stdout:total 0 2024-05-14T17:15:22.132 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 14 16:55 . 2024-05-14T17:15:22.165 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 14 17:12 .. 2024-05-14T17:15:22.198 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 16:55 members 2024-05-14T17:15:22.232 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 16:55 offsets 2024-05-14T17:15:22.265 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:15:22.298 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-05-14T17:15:22.357 INFO:teuthology.orchestra.run.smithi002.stdout:total 8 2024-05-14T17:15:22.390 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 16:55 . 2024-05-14T17:15:22.432 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 14 16:55 .. 2024-05-14T17:15:22.476 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:55 0000 2024-05-14T17:15:22.515 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:15:22.548 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-05-14T17:15:22.582 INFO:teuthology.orchestra.run.smithi002.stdout:total 8 2024-05-14T17:15:22.615 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 16:55 . 2024-05-14T17:15:22.649 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 14 16:55 .. 2024-05-14T17:15:22.682 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 14 16:59 0000 2024-05-14T17:15:22.728 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:15:22.765 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-05-14T17:15:22.799 INFO:teuthology.orchestra.run.smithi002.stdout:total 0 2024-05-14T17:15:22.832 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 . 2024-05-14T17:15:22.865 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 14 17:12 .. 2024-05-14T17:15:22.899 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:15:22.932 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-05-14T17:15:22.966 INFO:teuthology.orchestra.run.smithi002.stdout:total 0 2024-05-14T17:15:23.024 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 . 2024-05-14T17:15:23.058 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 14 17:12 .. 2024-05-14T17:15:23.099 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:15:23.133 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-05-14T17:15:23.166 INFO:teuthology.orchestra.run.smithi002.stdout:total 0 2024-05-14T17:15:23.199 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 . 2024-05-14T17:15:23.233 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 14 17:12 .. 2024-05-14T17:15:23.266 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:15:23.299 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-05-14T17:15:23.332 INFO:teuthology.orchestra.run.smithi002.stdout:total 0 2024-05-14T17:15:23.366 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 . 2024-05-14T17:15:23.399 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 14 17:12 .. 2024-05-14T17:15:23.433 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:15:23.466 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-05-14T17:15:23.499 INFO:teuthology.orchestra.run.smithi002.stdout:total 15 2024-05-14T17:15:23.533 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 14 17:12 . 2024-05-14T17:15:23.574 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 14 17:12 .. 2024-05-14T17:15:23.608 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1952 May 14 17:12 db_0.stat 2024-05-14T17:15:23.648 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 10979 May 14 17:12 db_13494.stat 2024-05-14T17:15:23.691 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1151 May 14 17:12 global.stat 2024-05-14T17:15:23.725 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:15:23.775 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-05-14T17:15:23.816 INFO:teuthology.orchestra.run.smithi002.stdout:total 0 2024-05-14T17:15:23.850 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 17:12 . 2024-05-14T17:15:23.883 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 14 17:12 .. 2024-05-14T17:15:23.916 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:15:23.994 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-05-14T17:15:24.033 INFO:teuthology.orchestra.run.smithi002.stdout:total 208 2024-05-14T17:15:24.067 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 May 14 17:12 . 2024-05-14T17:15:24.100 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 14 17:12 .. 2024-05-14T17:15:24.133 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 212992 May 14 17:12 0017 2024-05-14T17:15:24.167 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:15:24.225 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-05-14T17:15:24.258 INFO:teuthology.orchestra.run.smithi002.stdout:total 0 2024-05-14T17:15:24.333 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 . 2024-05-14T17:15:24.379 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 14 17:12 .. 2024-05-14T17:15:24.417 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:15:24.458 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-05-14T17:15:24.492 INFO:teuthology.orchestra.run.smithi002.stdout:total 0 2024-05-14T17:15:24.525 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 . 2024-05-14T17:15:24.558 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 14 17:12 .. 2024-05-14T17:15:24.592 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:15:24.625 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-05-14T17:15:24.659 INFO:teuthology.orchestra.run.smithi002.stdout:total 1048576 2024-05-14T17:15:24.692 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 May 14 17:12 . 2024-05-14T17:15:24.725 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 14 17:12 .. 2024-05-14T17:15:24.758 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:12 000000010000000000000097 2024-05-14T17:15:24.792 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:05 000000010000000000000098 2024-05-14T17:15:24.834 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:08 000000010000000000000099 2024-05-14T17:15:24.867 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:08 00000001000000000000009A 2024-05-14T17:15:24.900 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:07 00000001000000000000009B 2024-05-14T17:15:24.942 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:03 00000001000000000000009C 2024-05-14T17:15:24.975 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:03 00000001000000000000009D 2024-05-14T17:15:25.043 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:03 00000001000000000000009E 2024-05-14T17:15:25.084 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:08 00000001000000000000009F 2024-05-14T17:15:25.117 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:04 0000000100000000000000A0 2024-05-14T17:15:25.151 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:03 0000000100000000000000A1 2024-05-14T17:15:25.184 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:04 0000000100000000000000A2 2024-05-14T17:15:25.217 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:04 0000000100000000000000A3 2024-05-14T17:15:25.251 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:03 0000000100000000000000A4 2024-05-14T17:15:25.284 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:04 0000000100000000000000A5 2024-05-14T17:15:25.317 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:03 0000000100000000000000A6 2024-05-14T17:15:25.426 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:03 0000000100000000000000A7 2024-05-14T17:15:25.459 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:04 0000000100000000000000A8 2024-05-14T17:15:25.501 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:05 0000000100000000000000A9 2024-05-14T17:15:25.534 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:04 0000000100000000000000AA 2024-05-14T17:15:25.567 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:03 0000000100000000000000AB 2024-05-14T17:15:25.601 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:04 0000000100000000000000AC 2024-05-14T17:15:25.634 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:03 0000000100000000000000AD 2024-05-14T17:15:25.671 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:03 0000000100000000000000AE 2024-05-14T17:15:25.709 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:04 0000000100000000000000AF 2024-05-14T17:15:25.743 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:03 0000000100000000000000B0 2024-05-14T17:15:25.776 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:04 0000000100000000000000B1 2024-05-14T17:15:25.818 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:03 0000000100000000000000B2 2024-05-14T17:15:25.851 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:03 0000000100000000000000B3 2024-05-14T17:15:25.885 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:04 0000000100000000000000B4 2024-05-14T17:15:25.918 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:03 0000000100000000000000B5 2024-05-14T17:15:25.951 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:05 0000000100000000000000B6 2024-05-14T17:15:25.984 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:07 0000000100000000000000B7 2024-05-14T17:15:26.018 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:03 0000000100000000000000B8 2024-05-14T17:15:26.093 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:09 0000000100000000000000B9 2024-05-14T17:15:26.126 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:08 0000000100000000000000BA 2024-05-14T17:15:26.160 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:09 0000000100000000000000BB 2024-05-14T17:15:26.193 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:08 0000000100000000000000BC 2024-05-14T17:15:26.226 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:08 0000000100000000000000BD 2024-05-14T17:15:26.260 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:08 0000000100000000000000BE 2024-05-14T17:15:26.293 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:09 0000000100000000000000BF 2024-05-14T17:15:26.335 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:10 0000000100000000000000C0 2024-05-14T17:15:26.368 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:08 0000000100000000000000C1 2024-05-14T17:15:26.401 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:09 0000000100000000000000C2 2024-05-14T17:15:26.435 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:09 0000000100000000000000C3 2024-05-14T17:15:26.468 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:08 0000000100000000000000C4 2024-05-14T17:15:26.502 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:09 0000000100000000000000C5 2024-05-14T17:15:26.535 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:10 0000000100000000000000C6 2024-05-14T17:15:26.568 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:08 0000000100000000000000C7 2024-05-14T17:15:26.602 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:08 0000000100000000000000C8 2024-05-14T17:15:26.660 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:09 0000000100000000000000C9 2024-05-14T17:15:26.693 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:08 0000000100000000000000CA 2024-05-14T17:15:26.727 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:08 0000000100000000000000CB 2024-05-14T17:15:26.760 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:09 0000000100000000000000CC 2024-05-14T17:15:26.794 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:09 0000000100000000000000CD 2024-05-14T17:15:26.827 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:09 0000000100000000000000CE 2024-05-14T17:15:26.860 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:09 0000000100000000000000CF 2024-05-14T17:15:26.899 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:08 0000000100000000000000D0 2024-05-14T17:15:26.936 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:09 0000000100000000000000D1 2024-05-14T17:15:26.977 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:08 0000000100000000000000D2 2024-05-14T17:15:27.010 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:09 0000000100000000000000D3 2024-05-14T17:15:27.044 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:08 0000000100000000000000D4 2024-05-14T17:15:27.077 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:09 0000000100000000000000D5 2024-05-14T17:15:27.110 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 14 17:09 0000000100000000000000D6 2024-05-14T17:15:27.144 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 archive_status 2024-05-14T17:15:27.177 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:15:27.227 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-05-14T17:15:27.260 INFO:teuthology.orchestra.run.smithi002.stdout:total 0 2024-05-14T17:15:27.294 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 14 16:55 . 2024-05-14T17:15:27.327 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 May 14 17:12 .. 2024-05-14T17:15:27.360 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:15:27.394 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-05-14T17:15:27.427 INFO:teuthology.orchestra.run.smithi002.stdout:total 384 2024-05-14T17:15:27.477 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 2 May 14 17:03 . 2024-05-14T17:15:27.511 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 May 14 17:12 .. 2024-05-14T17:15:27.554 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 262144 May 14 17:03 0000 2024-05-14T17:15:27.594 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 May 14 17:12 0001 2024-05-14T17:15:27.628 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-05-14T17:15:27.673 INFO:tasks.quiescer:joining Quiescers 2024-05-14T17:15:27.694 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-05-14T17:15:27.728 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:15:27.763 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:14:12 smithi002 ceph-mon[31538]: pgmap v613: 97 pgs: 97 active+clean; 4.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-05-14T17:15:27.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:14:12 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:15:27.836 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:14:13 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:15:27.869 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:14:13 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:15:27.903 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:14:13 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:15:27.936 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:14:14 smithi002 ceph-mon[31538]: pgmap v614: 97 pgs: 97 active+clean; 3.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-05-14T17:15:27.969 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:14:16 smithi002 ceph-mon[31538]: pgmap v615: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-05-14T17:15:28.003 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:14:18 smithi002 ceph-mon[31538]: osdmap e107: 12 total, 12 up, 12 in 2024-05-14T17:15:28.036 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:14:18 smithi002 ceph-mon[31538]: pgmap v617: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 43 KiB/s wr, 14 op/s 2024-05-14T17:15:28.069 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:14:20 smithi002 ceph-mon[31538]: pgmap v618: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 38 KiB/s wr, 16 op/s 2024-05-14T17:15:28.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:14:42 smithi151 ceph-mon[36066]: pgmap v629: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-05-14T17:15:28.145 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:14:44 smithi151 ceph-mon[36066]: pgmap v630: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-05-14T17:15:28.178 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:14:46 smithi151 ceph-mon[36066]: pgmap v631: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-05-14T17:15:28.211 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:14:48 smithi151 ceph-mon[36066]: pgmap v632: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-05-14T17:15:28.244 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:14:50 smithi151 ceph-mon[36066]: pgmap v633: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-05-14T17:15:28.278 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:14:52 smithi151 ceph-mon[36066]: pgmap v634: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.6 KiB/s wr, 7 op/s 2024-05-14T17:15:28.311 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:14:54 smithi151 ceph-mon[36066]: pgmap v635: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-14T17:15:28.361 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:14:56 smithi151 ceph-mon[36066]: pgmap v636: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 7.8 KiB/s wr, 8 op/s 2024-05-14T17:15:28.395 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:14:58 smithi151 ceph-mon[36066]: pgmap v637: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.8 KiB/s wr, 7 op/s 2024-05-14T17:15:28.428 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:00 smithi151 ceph-mon[36066]: pgmap v638: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.0 KiB/s wr, 10 op/s 2024-05-14T17:15:28.461 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:02 smithi151 ceph-mon[36066]: pgmap v639: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.4 KiB/s wr, 6 op/s 2024-05-14T17:15:28.496 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: *** Caught signal (Segmentation fault) ** 2024-05-14T17:15:28.528 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: in thread 7f5e01d81700 thread_name:md_log_replay 2024-05-14T17:15:28.561 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev) 2024-05-14T17:15:28.595 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f5e13143d20] 2024-05-14T17:15:28.632 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: 2: /usr/bin/ceph-mds(+0x54ac02) [0x559a6d74dc02] 2024-05-14T17:15:28.692 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x559a6d7513b3] 2024-05-14T17:15:28.728 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: 4: (MDLog::_replay_thread()+0x617) [0x559a6d7ac197] 2024-05-14T17:15:28.787 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: 5: (MDLog::ReplayThread::entry()+0x11) [0x559a6d7b5a11] 2024-05-14T17:15:28.853 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f5e131391ca] 2024-05-14T17:15:28.887 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: 7: clone() 2024-05-14T17:15:28.920 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: 2024-05-14T17:15:04.258+0000 7f5e01d81700 -1 *** Caught signal (Segmentation fault) ** 2024-05-14T17:15:28.953 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: in thread 7f5e01d81700 thread_name:md_log_replay 2024-05-14T17:15:29.003 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: 2024-05-14T17:15:29.037 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev) 2024-05-14T17:15:29.070 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f5e13143d20] 2024-05-14T17:15:29.103 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: 2: /usr/bin/ceph-mds(+0x54ac02) [0x559a6d74dc02] 2024-05-14T17:15:29.137 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x559a6d7513b3] 2024-05-14T17:15:29.170 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: 4: (MDLog::_replay_thread()+0x617) [0x559a6d7ac197] 2024-05-14T17:15:29.204 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: 5: (MDLog::ReplayThread::entry()+0x11) [0x559a6d7b5a11] 2024-05-14T17:15:29.237 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f5e131391ca] 2024-05-14T17:15:29.271 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: 7: clone() 2024-05-14T17:15:29.304 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-14T17:15:29.337 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: 2024-05-14T17:15:29.371 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: 0> 2024-05-14T17:15:04.258+0000 7f5e01d81700 -1 *** Caught signal (Segmentation fault) ** 2024-05-14T17:15:29.404 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: in thread 7f5e01d81700 thread_name:md_log_replay 2024-05-14T17:15:29.437 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: 2024-05-14T17:15:29.471 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev) 2024-05-14T17:15:29.504 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f5e13143d20] 2024-05-14T17:15:29.546 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: 2: /usr/bin/ceph-mds(+0x54ac02) [0x559a6d74dc02] 2024-05-14T17:15:29.579 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x559a6d7513b3] 2024-05-14T17:15:29.613 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: 4: (MDLog::_replay_thread()+0x617) [0x559a6d7ac197] 2024-05-14T17:15:29.654 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: 5: (MDLog::ReplayThread::entry()+0x11) [0x559a6d7b5a11] 2024-05-14T17:15:29.688 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f5e131391ca] 2024-05-14T17:15:29.721 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: 7: clone() 2024-05-14T17:15:29.755 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-14T17:15:29.788 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: 2024-05-14T17:15:29.821 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: 0> 2024-05-14T17:15:04.258+0000 7f5e01d81700 -1 *** Caught signal (Segmentation fault) ** 2024-05-14T17:15:29.855 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: in thread 7f5e01d81700 thread_name:md_log_replay 2024-05-14T17:15:29.888 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: 2024-05-14T17:15:29.921 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev) 2024-05-14T17:15:29.968 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f5e13143d20] 2024-05-14T17:15:30.004 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: 2: /usr/bin/ceph-mds(+0x54ac02) [0x559a6d74dc02] 2024-05-14T17:15:30.038 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x559a6d7513b3] 2024-05-14T17:15:30.071 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: 4: (MDLog::_replay_thread()+0x617) [0x559a6d7ac197] 2024-05-14T17:15:30.113 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: 5: (MDLog::ReplayThread::entry()+0x11) [0x559a6d7b5a11] 2024-05-14T17:15:30.147 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f5e131391ca] 2024-05-14T17:15:30.180 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: 7: clone() 2024-05-14T17:15:30.213 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-14T17:15:30.258 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[67256]: 2024-05-14T17:15:30.297 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:04 smithi151 podman[74684]: 2024-05-14 17:15:04.31507551 +0000 UTC m=+0.020060299 container died d2272915ee3cd763c5b443d25b6efff74fe08e614e01b71fbc434cf1bfd0ab95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-14T17:15:30.331 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:04 smithi151 ceph-mon[36066]: pgmap v640: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 56 KiB/s wr, 26 op/s 2024-05-14T17:15:30.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:04 smithi151 ceph-mon[36066]: osdmap e108: 12 total, 12 up, 12 in 2024-05-14T17:15:30.399 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:05 smithi151 podman[74684]: 2024-05-14 17:15:05.213307718 +0000 UTC m=+0.918292497 container remove d2272915ee3cd763c5b443d25b6efff74fe08e614e01b71fbc434cf1bfd0ab95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux ) 2024-05-14T17:15:30.430 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:05 smithi151 systemd[1]: ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.i.service: Main process exited, code=exited, status=139/n/a 2024-05-14T17:15:30.463 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:05 smithi151 systemd[1]: ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.i.service: Failed with result 'exit-code'. 2024-05-14T17:15:30.497 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:05 smithi151 systemd[1]: ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.i.service: Consumed 3.824s CPU time. 2024-05-14T17:15:30.536 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:06 smithi151 ceph-mon[36066]: pgmap v642: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 60 KiB/s wr, 27 op/s 2024-05-14T17:15:30.579 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:08 smithi151 ceph-mon[36066]: pgmap v643: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 60 KiB/s wr, 27 op/s 2024-05-14T17:15:30.622 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:10 smithi151 ceph-mon[36066]: pgmap v644: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 60 KiB/s wr, 26 op/s 2024-05-14T17:15:30.655 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:12 smithi151 ceph-mon[36066]: pgmap v645: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 60 KiB/s wr, 26 op/s 2024-05-14T17:15:30.713 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:12 smithi151 ceph-mon[36066]: osdmap e109: 12 total, 12 up, 12 in 2024-05-14T17:15:30.747 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:13 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:15:30.780 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:14 smithi151 ceph-mon[36066]: pgmap v647: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 517 B/s wr, 4 op/s 2024-05-14T17:15:30.814 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:14 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:15:30.847 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:14 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:15:30.880 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:14 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:15:30.914 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:14 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:15:30.947 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:14 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:15:30.980 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:14 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:15:31.015 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:14:42 smithi123 ceph-mon[35971]: pgmap v629: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-05-14T17:15:31.047 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:14:44 smithi123 ceph-mon[35971]: pgmap v630: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-05-14T17:15:31.081 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:14:46 smithi123 ceph-mon[35971]: pgmap v631: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-05-14T17:15:31.114 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:14:48 smithi123 ceph-mon[35971]: pgmap v632: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-05-14T17:15:31.147 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:14:50 smithi123 ceph-mon[35971]: pgmap v633: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-05-14T17:15:31.191 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:14:52 smithi123 ceph-mon[35971]: pgmap v634: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.6 KiB/s wr, 7 op/s 2024-05-14T17:15:31.247 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:14:54 smithi123 ceph-mon[35971]: pgmap v635: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-14T17:15:31.314 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:14:56 smithi123 ceph-mon[35971]: pgmap v636: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 7.8 KiB/s wr, 8 op/s 2024-05-14T17:15:31.347 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:14:58 smithi123 ceph-mon[35971]: pgmap v637: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.8 KiB/s wr, 7 op/s 2024-05-14T17:15:31.381 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:00 smithi123 ceph-mon[35971]: pgmap v638: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.0 KiB/s wr, 10 op/s 2024-05-14T17:15:31.414 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:02 smithi123 ceph-mon[35971]: pgmap v639: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.4 KiB/s wr, 6 op/s 2024-05-14T17:15:31.447 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:04 smithi123 ceph-mon[35971]: pgmap v640: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 56 KiB/s wr, 26 op/s 2024-05-14T17:15:31.497 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:04 smithi123 ceph-mon[35971]: osdmap e108: 12 total, 12 up, 12 in 2024-05-14T17:15:31.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:06 smithi123 ceph-mon[35971]: pgmap v642: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 60 KiB/s wr, 27 op/s 2024-05-14T17:15:31.589 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:08 smithi123 ceph-mon[35971]: pgmap v643: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 60 KiB/s wr, 27 op/s 2024-05-14T17:15:31.623 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:10 smithi123 ceph-mon[35971]: pgmap v644: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 60 KiB/s wr, 26 op/s 2024-05-14T17:15:31.656 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:12 smithi123 ceph-mon[35971]: pgmap v645: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 60 KiB/s wr, 26 op/s 2024-05-14T17:15:31.689 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:12 smithi123 ceph-mon[35971]: osdmap e109: 12 total, 12 up, 12 in 2024-05-14T17:15:31.722 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:13 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:15:31.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:14 smithi123 ceph-mon[35971]: pgmap v647: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 517 B/s wr, 4 op/s 2024-05-14T17:15:31.789 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:14 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:15:31.872 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:14 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:15:31.940 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:14 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:15:32.003 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:14 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:15:32.039 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:14 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:15:32.073 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:14 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:15:32.106 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:16 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:15:32.139 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:16 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:15:32.173 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:16 smithi123 ceph-mon[35971]: pgmap v648: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 511 B/s wr, 4 op/s 2024-05-14T17:15:32.206 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:16 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:15:32.247 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:16 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:15:32.281 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:16 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:15:32.315 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:16 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:15:32.348 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:16 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:15:32.381 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:16 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:15:32.415 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:16 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:15:32.448 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:16 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:15:32.481 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:16 smithi123 ceph-mon[35971]: daemon mds.i restarted 2024-05-14T17:15:32.515 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:16 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:15:32.565 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:16 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:15:32.636 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:16 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:15:32.673 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:16 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:15:32.707 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:16 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:15:32.740 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:16 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:15:32.773 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:16 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:15:32.807 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:16 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:15:32.840 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:16 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:15:32.873 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:16 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:15:32.907 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:16 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:15:32.940 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:16 smithi123 ceph-mon[35971]: osdmap e110: 12 total, 12 up, 12 in 2024-05-14T17:15:32.974 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:17 smithi123 ceph-mon[35971]: Adjusting osd_memory_target on smithi123 to 176.2M 2024-05-14T17:15:33.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:17 smithi123 ceph-mon[35971]: Unable to set osd_memory_target on smithi123 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-05-14T17:15:33.040 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:17 smithi123 ceph-mon[35971]: Adjusting osd_memory_target on smithi002 to 177.2M 2024-05-14T17:15:33.074 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:17 smithi123 ceph-mon[35971]: Unable to set osd_memory_target on smithi002 to 185879142: error parsing value: Value '185879142' is below minimum 939524096 2024-05-14T17:15:33.107 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:17 smithi123 ceph-mon[35971]: Adjusting osd_memory_target on smithi151 to 176.2M 2024-05-14T17:15:33.148 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:17 smithi123 ceph-mon[35971]: Unable to set osd_memory_target on smithi151 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-05-14T17:15:33.182 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:17 smithi123 ceph-mon[35971]: pgmap v649: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 588 B/s wr, 5 op/s 2024-05-14T17:15:33.215 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:17 smithi123 ceph-mon[35971]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-05-14T17:15:33.249 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:17 smithi123 ceph-mon[35971]: mds.? [v2:172.21.15.151:6838/239059007,v1:172.21.15.151:6839/239059007] up:boot 2024-05-14T17:15:33.282 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:17 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T17:15:33.315 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:17 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T17:15:33.349 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:17 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:15:33.382 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:19 smithi123 ceph-mon[35971]: pgmap v651: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-14T17:15:33.416 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:21 smithi123 ceph-mon[35971]: pgmap v652: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 8 op/s 2024-05-14T17:15:33.449 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:23 smithi123 ceph-mon[35971]: pgmap v653: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.7 KiB/s wr, 5 op/s 2024-05-14T17:15:33.482 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:25 smithi123 ceph-mon[35971]: pgmap v654: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.0 KiB/s wr, 9 op/s 2024-05-14T17:15:33.516 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:27 smithi123 ceph-mon[35971]: pgmap v655: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.8 KiB/s wr, 8 op/s 2024-05-14T17:15:33.549 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:29 smithi123 ceph-mon[35971]: pgmap v656: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.5 KiB/s wr, 8 op/s 2024-05-14T17:15:33.583 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:15 smithi151 systemd[1]: ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.i.service: Scheduled restart job, restart counter is at 1. 2024-05-14T17:15:33.616 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:15 smithi151 systemd[1]: Stopped Ceph mds.i for ed40717a-1210-11ef-bc98-c7b262605968. 2024-05-14T17:15:33.649 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:15 smithi151 systemd[1]: ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.i.service: Consumed 3.824s CPU time. 2024-05-14T17:15:33.699 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:15 smithi151 systemd[1]: Starting Ceph mds.i for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T17:15:33.733 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:15 smithi151 podman[75223]: 2024-05-14 17:15:15.738145043 +0000 UTC m=+0.017870696 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:15:33.771 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:15 smithi151 podman[75223]: 2024-05-14 17:15:15.862803703 +0000 UTC m=+0.142529333 container create aa6f716b2e2b78f1ccbd848e9bf8c734214ae968aa83e147ef220730820a9bde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T17:15:33.808 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:16 smithi151 podman[75223]: 2024-05-14 17:15:16.133278044 +0000 UTC m=+0.413003702 container init aa6f716b2e2b78f1ccbd848e9bf8c734214ae968aa83e147ef220730820a9bde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:15:33.841 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:16 smithi151 podman[75223]: 2024-05-14 17:15:16.136917193 +0000 UTC m=+0.416642841 container start aa6f716b2e2b78f1ccbd848e9bf8c734214ae968aa83e147ef220730820a9bde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:15:33.874 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:16 smithi151 ceph-mds[75315]: -- 172.21.15.151:0/672715228 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5562bd179a40 con 0x5562bd14c800 2024-05-14T17:15:33.907 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:16 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[75299]: starting mds.i at 2024-05-14T17:15:33.949 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:16 smithi151 bash[75223]: aa6f716b2e2b78f1ccbd848e9bf8c734214ae968aa83e147ef220730820a9bde 2024-05-14T17:15:33.983 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:15:16 smithi151 systemd[1]: Started Ceph mds.i for ed40717a-1210-11ef-bc98-c7b262605968. 2024-05-14T17:15:34.019 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:16 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:15:34.054 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:16 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:15:34.091 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:16 smithi151 ceph-mon[36066]: pgmap v648: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 511 B/s wr, 4 op/s 2024-05-14T17:15:34.126 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:16 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:15:34.166 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:16 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:15:34.200 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:16 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:15:34.233 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:16 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:15:34.291 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:16 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:15:34.325 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:16 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:15:34.390 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:16 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:15:34.429 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:16 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:15:34.466 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:16 smithi151 ceph-mon[36066]: daemon mds.i restarted 2024-05-14T17:15:34.500 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:16 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:15:34.533 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:16 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:15:34.566 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:16 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:15:34.600 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:16 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:15:34.633 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:16 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:15:34.666 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:16 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:15:34.723 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:16 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:15:34.758 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:16 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:15:34.792 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:16 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:15:34.825 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:16 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:15:34.858 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:16 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:15:34.925 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:16 smithi151 ceph-mon[36066]: osdmap e110: 12 total, 12 up, 12 in 2024-05-14T17:15:34.983 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:17 smithi151 ceph-mon[36066]: Adjusting osd_memory_target on smithi123 to 176.2M 2024-05-14T17:15:35.017 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:17 smithi151 ceph-mon[36066]: Unable to set osd_memory_target on smithi123 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-05-14T17:15:35.050 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:17 smithi151 ceph-mon[36066]: Adjusting osd_memory_target on smithi002 to 177.2M 2024-05-14T17:15:35.083 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:17 smithi151 ceph-mon[36066]: Unable to set osd_memory_target on smithi002 to 185879142: error parsing value: Value '185879142' is below minimum 939524096 2024-05-14T17:15:35.117 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:17 smithi151 ceph-mon[36066]: Adjusting osd_memory_target on smithi151 to 176.2M 2024-05-14T17:15:35.150 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:17 smithi151 ceph-mon[36066]: Unable to set osd_memory_target on smithi151 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-05-14T17:15:35.184 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:17 smithi151 ceph-mon[36066]: pgmap v649: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 588 B/s wr, 5 op/s 2024-05-14T17:15:35.217 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:17 smithi151 ceph-mon[36066]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-05-14T17:15:35.250 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:17 smithi151 ceph-mon[36066]: mds.? [v2:172.21.15.151:6838/239059007,v1:172.21.15.151:6839/239059007] up:boot 2024-05-14T17:15:35.284 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:17 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T17:15:35.317 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:17 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T17:15:35.350 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:17 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:15:35.384 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:19 smithi151 ceph-mon[36066]: pgmap v651: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-14T17:15:35.430 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:21 smithi151 ceph-mon[36066]: pgmap v652: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 8 op/s 2024-05-14T17:15:35.467 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:23 smithi151 ceph-mon[36066]: pgmap v653: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.7 KiB/s wr, 5 op/s 2024-05-14T17:15:35.517 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:25 smithi151 ceph-mon[36066]: pgmap v654: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.0 KiB/s wr, 9 op/s 2024-05-14T17:15:35.550 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:27 smithi151 ceph-mon[36066]: pgmap v655: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.8 KiB/s wr, 8 op/s 2024-05-14T17:15:35.584 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:29 smithi151 ceph-mon[36066]: pgmap v656: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.5 KiB/s wr, 8 op/s 2024-05-14T17:15:35.617 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:31 smithi151 ceph-mon[36066]: pgmap v657: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T17:15:35.690 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:33 smithi151 ceph-mon[36066]: pgmap v658: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-14T17:15:35.726 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:31 smithi123 ceph-mon[35971]: pgmap v657: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T17:15:35.760 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:33 smithi123 ceph-mon[35971]: pgmap v658: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-14T17:15:35.792 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:34 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:15:35.828 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:34 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:15:35.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:35 smithi151 ceph-mon[36066]: pgmap v659: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.9 KiB/s wr, 8 op/s 2024-05-14T17:15:35.898 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.640+0000 7f8a1befc640 1 -- 172.21.15.2:0/1370248147 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a14102340 msgr2=0x7f8a1410e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.640+0000 7f8a1befc640 1 --2- 172.21.15.2:0/1370248147 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a14102340 0x7f8a1410e800 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f8a04009cd0 tx=0x7f8a0402a8e0 comp rx=0 tx=0).stop 2024-05-14T17:15:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.640+0000 7f8a1befc640 1 -- 172.21.15.2:0/1370248147 shutdown_connections 2024-05-14T17:15:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.640+0000 7f8a1befc640 1 --2- 172.21.15.2:0/1370248147 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a14102340 0x7f8a1410e800 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.640+0000 7f8a1befc640 1 --2- 172.21.15.2:0/1370248147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a14101980 0x7f8a14101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:36.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.640+0000 7f8a1befc640 1 --2- 172.21.15.2:0/1370248147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a14107940 0x7f8a14107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.640+0000 7f8a1befc640 1 -- 172.21.15.2:0/1370248147 >> 172.21.15.2:0/1370248147 conn(0x7f8a14078ad0 msgr2=0x7f8a14078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:36.143 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.640+0000 7f8a1befc640 1 -- 172.21.15.2:0/1370248147 shutdown_connections 2024-05-14T17:15:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.640+0000 7f8a1befc640 1 -- 172.21.15.2:0/1370248147 wait complete. 2024-05-14T17:15:36.209 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.641+0000 7f8a1befc640 1 Processor -- start 2024-05-14T17:15:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.641+0000 7f8a1befc640 1 -- start start 2024-05-14T17:15:36.324 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.641+0000 7f8a1befc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a14101980 0x7f8a14199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.641+0000 7f8a1befc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a14102340 0x7f8a141997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.641+0000 7f8a1befc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a14107940 0x7f8a1419db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.641+0000 7f8a1befc640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8a14111c00 con 0x7f8a14102340 2024-05-14T17:15:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.641+0000 7f8a1befc640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8a14111a80 con 0x7f8a14101980 2024-05-14T17:15:36.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.641+0000 7f8a1befc640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8a14111d80 con 0x7f8a14107940 2024-05-14T17:15:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.641+0000 7f8a19497640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a14102340 0x7f8a141997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.641+0000 7f8a19497640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a14102340 0x7f8a141997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34026/0 (socket says 172.21.15.2:34026) 2024-05-14T17:15:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.641+0000 7f8a19497640 1 -- 172.21.15.2:0/3914982192 learned_addr learned my addr 172.21.15.2:0/3914982192 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.641+0000 7f8a19c98640 1 --2- 172.21.15.2:0/3914982192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a14101980 0x7f8a14199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.641+0000 7f8a1a499640 1 --2- 172.21.15.2:0/3914982192 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a14107940 0x7f8a1419db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:36.696 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.641+0000 7f8a19497640 1 -- 172.21.15.2:0/3914982192 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a14107940 msgr2=0x7f8a1419db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.641+0000 7f8a19497640 1 --2- 172.21.15.2:0/3914982192 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a14107940 0x7f8a1419db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.641+0000 7f8a19497640 1 -- 172.21.15.2:0/3914982192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a14101980 msgr2=0x7f8a14199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.641+0000 7f8a19497640 1 --2- 172.21.15.2:0/3914982192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a14101980 0x7f8a14199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:36.885 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.641+0000 7f8a19497640 1 -- 172.21.15.2:0/3914982192 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a1419e260 con 0x7f8a14102340 2024-05-14T17:15:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.641+0000 7f8a1a499640 1 --2- 172.21.15.2:0/3914982192 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a14107940 0x7f8a1419db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:15:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.642+0000 7f8a19497640 1 --2- 172.21.15.2:0/3914982192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a14102340 0x7f8a141997b0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f8a0800d950 tx=0x7f8a0800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.642+0000 7f8a12ffd640 1 -- 172.21.15.2:0/3914982192 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a08017070 con 0x7f8a14102340 2024-05-14T17:15:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.642+0000 7f8a12ffd640 1 -- 172.21.15.2:0/3914982192 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a08009c30 con 0x7f8a14102340 2024-05-14T17:15:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.642+0000 7f8a1befc640 1 -- 172.21.15.2:0/3914982192 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a1419e550 con 0x7f8a14102340 2024-05-14T17:15:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.642+0000 7f8a12ffd640 1 -- 172.21.15.2:0/3914982192 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0800f620 con 0x7f8a14102340 2024-05-14T17:15:37.135 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.642+0000 7f8a1befc640 1 -- 172.21.15.2:0/3914982192 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a141a5de0 con 0x7f8a14102340 2024-05-14T17:15:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.645+0000 7f8a1ec12740 1 Processor -- start 2024-05-14T17:15:37.202 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.645+0000 7f8a12ffd640 1 -- 172.21.15.2:0/3914982192 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f8a08010460 con 0x7f8a14102340 2024-05-14T17:15:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.645+0000 7f8a1ec12740 1 -- start start 2024-05-14T17:15:37.268 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.645+0000 7f8a1ec12740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddef17a8f0 0x55ddef17acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.645+0000 7f8a1ec12740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddef17b210 0x55ddef185590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.360 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.645+0000 7f8a1ec12740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddef185ad0 0x55ddef187ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.393 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.645+0000 7f8a1ec12740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ddef0edc30 con 0x55ddef185ad0 2024-05-14T17:15:37.427 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.645+0000 7f8a1ec12740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ddef0dd7d0 con 0x55ddef17a8f0 2024-05-14T17:15:37.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.645+0000 7f8a1ec12740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ddef0e1810 con 0x55ddef17b210 2024-05-14T17:15:37.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.645+0000 7f8a19497640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddef185ad0 0x55ddef187ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.645+0000 7f8a1a499640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddef17a8f0 0x55ddef17acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.646+0000 7f8a1a499640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddef17a8f0 0x55ddef17acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:47904/0 (socket says 172.21.15.2:47904) 2024-05-14T17:15:37.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.646+0000 7f8a19497640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddef185ad0 0x55ddef187ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34040/0 (socket says 172.21.15.2:34040) 2024-05-14T17:15:37.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.646+0000 7f8a1a499640 1 -- 172.21.15.2:0/95444935 learned_addr learned my addr 172.21.15.2:0/95444935 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:37.428 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.646+0000 7f8a19c98640 1 --2- 172.21.15.2:0/95444935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddef17b210 0x55ddef185590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.646+0000 7f8a1a499640 1 -- 172.21.15.2:0/95444935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddef17b210 msgr2=0x55ddef185590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.646+0000 7f8a1a499640 1 --2- 172.21.15.2:0/95444935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddef17b210 0x55ddef185590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.646+0000 7f8a1a499640 1 -- 172.21.15.2:0/95444935 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddef185ad0 msgr2=0x55ddef187ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.646+0000 7f8a1a499640 1 --2- 172.21.15.2:0/95444935 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddef185ad0 0x55ddef187ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.646+0000 7f8a1a499640 1 -- 172.21.15.2:0/95444935 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddeedf2210 con 0x55ddef17a8f0 2024-05-14T17:15:37.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.646+0000 7f8a19c98640 1 --2- 172.21.15.2:0/95444935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddef17b210 0x55ddef185590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:15:37.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.646+0000 7f8a12ffd640 1 --2- 172.21.15.2:0/3914982192 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f89fc0b2a10 0x7f89fc0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.646+0000 7f8a19c98640 1 --2- 172.21.15.2:0/3914982192 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f89fc0b2a10 0x7f89fc0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.646+0000 7f8a1a499640 1 --2- 172.21.15.2:0/95444935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddef17a8f0 0x55ddef17acd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8a04005e30 tx=0x7f8a04009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b92cb56567c22af9 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:37.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.646+0000 7f8a12ffd640 1 -- 172.21.15.2:0/3914982192 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8513+0+0 (secure 0 0 0) 0x7f8a080e2300 con 0x7f8a14102340 2024-05-14T17:15:37.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.646+0000 7f89fbfff640 1 -- 172.21.15.2:0/95444935 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a04031850 con 0x55ddef17a8f0 2024-05-14T17:15:37.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.646+0000 7f89fbfff640 1 -- 172.21.15.2:0/95444935 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a04036040 con 0x55ddef17a8f0 2024-05-14T17:15:37.429 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.646+0000 7f89fbfff640 1 -- 172.21.15.2:0/95444935 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a04004460 con 0x55ddef17a8f0 2024-05-14T17:15:37.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.647+0000 7f8a1ec12740 1 -- 172.21.15.2:0/95444935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddef17a8f0 msgr2=0x55ddef17acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.647+0000 7f8a19c98640 1 --2- 172.21.15.2:0/3914982192 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f89fc0b2a10 0x7f89fc0b4ed0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x55ddef17be90 tx=0x7f8a0c005f90 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:37.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.647+0000 7f8a1ec12740 1 --2- 172.21.15.2:0/95444935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddef17a8f0 0x55ddef17acd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8a04005e30 tx=0x7f8a04009a20 comp rx=0 tx=0).stop 2024-05-14T17:15:37.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.647+0000 7f8a1ec12740 1 -- 172.21.15.2:0/95444935 shutdown_connections 2024-05-14T17:15:37.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.647+0000 7f8a1ec12740 1 --2- 172.21.15.2:0/95444935 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddef185ad0 0x55ddef187ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.647+0000 7f8a1ec12740 1 --2- 172.21.15.2:0/95444935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddef17b210 0x55ddef185590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.647+0000 7f8a1ec12740 1 --2- 172.21.15.2:0/95444935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddef17a8f0 0x55ddef17acd0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.647+0000 7f8a1ec12740 1 -- 172.21.15.2:0/95444935 >> 172.21.15.2:0/95444935 conn(0x55ddef176e00 msgr2=0x55ddef1771e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:37.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.647+0000 7f8a1ec12740 1 -- 172.21.15.2:0/95444935 shutdown_connections 2024-05-14T17:15:37.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.647+0000 7f8a1ec12740 1 -- 172.21.15.2:0/95444935 wait complete. 2024-05-14T17:15:37.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.648+0000 7f8a1ec12740 1 -- start start 2024-05-14T17:15:37.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.648+0000 7f8a1ec12740 1 Processor -- start 2024-05-14T17:15:37.430 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.648+0000 7f8a1ec12740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddef17a8f0 0x55ddef185490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.648+0000 7f8a1ec12740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddef17b210 0x55ddef182780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.648+0000 7f8a1ec12740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddef185ad0 0x55ddef182d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.648+0000 7f8a1ec12740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55ddeedc3790 con 0x55ddef17b210 2024-05-14T17:15:37.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.648+0000 7f8a1ec12740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ddef099890 con 0x55ddef17a8f0 2024-05-14T17:15:37.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.648+0000 7f8a1ec12740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ddef188740 con 0x55ddef185ad0 2024-05-14T17:15:37.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.648+0000 7f8a1ec12740 10 client.0 _ll_register_callbacks cb 0x55ddef09cd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:15:37.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.648+0000 7f8a19497640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddef17b210 0x55ddef182780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.648+0000 7f8a19497640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddef17b210 0x55ddef182780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34052/0 (socket says 172.21.15.2:34052) 2024-05-14T17:15:37.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.648+0000 7f8a19497640 1 -- 172.21.15.2:0/85233239 learned_addr learned my addr 172.21.15.2:0/85233239 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:37.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.649+0000 7f8a19c98640 1 --2- 172.21.15.2:0/85233239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddef17a8f0 0x55ddef185490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.649+0000 7f8a1a499640 1 --2- 172.21.15.2:0/85233239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddef185ad0 0x55ddef182d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.649+0000 7f8a19497640 1 -- 172.21.15.2:0/85233239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddef185ad0 msgr2=0x55ddef182d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.649+0000 7f8a19497640 1 --2- 172.21.15.2:0/85233239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddef185ad0 0x55ddef182d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.431 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.649+0000 7f8a19497640 1 -- 172.21.15.2:0/85233239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddef17a8f0 msgr2=0x55ddef185490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.649+0000 7f8a19497640 1 --2- 172.21.15.2:0/85233239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddef17a8f0 0x55ddef185490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.649+0000 7f8a19497640 1 -- 172.21.15.2:0/85233239 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddeeebe620 con 0x55ddef17b210 2024-05-14T17:15:37.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.649+0000 7f8a19c98640 1 --2- 172.21.15.2:0/85233239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddef17a8f0 0x55ddef185490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:15:37.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.649+0000 7f8a1a499640 1 --2- 172.21.15.2:0/85233239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddef185ad0 0x55ddef182d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:15:37.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.649+0000 7f8a19497640 1 --2- 172.21.15.2:0/85233239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddef17b210 0x55ddef182780 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f8a080dfab0 tx=0x7f8a080e5a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a44e9520350930fa server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:37.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.649+0000 7f89f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:15:37.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.649+0000 7f89f97fa640 1 -- 172.21.15.2:0/85233239 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a08020030 con 0x55ddef17b210 2024-05-14T17:15:37.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.649+0000 7f89f97fa640 1 -- 172.21.15.2:0/85233239 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8513+0+0 (secure 0 0 0) 0x7f8a080a5870 con 0x55ddef17b210 2024-05-14T17:15:37.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.649+0000 7f8a1ec12740 1 -- 172.21.15.2:0/85233239 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a04036040 con 0x55ddef17b210 2024-05-14T17:15:37.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.649+0000 7f89f97fa640 1 -- 172.21.15.2:0/85233239 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f8a080a8d50 con 0x55ddef17b210 2024-05-14T17:15:37.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.649+0000 7f8a1ec12740 10 client.17211 fetch_fsmap learned FSMap version 56 2024-05-14T17:15:37.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.649+0000 7f8a1ec12740 1 -- 172.21.15.2:0/85233239 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x55ddef194250 con 0x55ddef17b210 2024-05-14T17:15:37.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.650+0000 7f89f97fa640 1 -- 172.21.15.2:0/85233239 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a04036040 con 0x55ddef17b210 2024-05-14T17:15:37.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.650+0000 7f89f97fa640 1 -- 172.21.15.2:0/85233239 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 56) ==== 8346+0+0 (secure 0 0 0) 0x7f8a0808fb00 con 0x55ddef17b210 2024-05-14T17:15:37.432 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.650+0000 7f8a1ec12740 10 client.17211 fetch_fsmap finished waiting for FSMap version 56 2024-05-14T17:15:37.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.650+0000 7f8a1ec12740 10 client.17211 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:15:37.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.650+0000 7f8a1ec12740 20 client.17211 populate_metadata read hostname 'smithi002' 2024-05-14T17:15:37.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.650+0000 7f8a1ec12740 1 --2- 172.21.15.2:0/85233239 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ddef195760 0x55ddef197b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.650+0000 7f8a1ec12740 4 client.17211 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:15:37.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.650+0000 7f8a1ec12740 1 -- 172.21.15.2:0/85233239 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddef0efb80 con 0x55ddef195760 2024-05-14T17:15:37.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.650+0000 7f8a19c98640 1 --2- 172.21.15.2:0/85233239 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ddef195760 0x55ddef197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.651+0000 7f8a19c98640 1 --2- 172.21.15.2:0/85233239 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ddef195760 0x55ddef197b50 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57554b146414919b server_cookie=99c29f6189d35ff2 in_seq=0 out_seq=0 2024-05-14T17:15:37.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.651+0000 7f89f97fa640 10 client.17211 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:15:37.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.651+0000 7f89f97fa640 1 -- 172.21.15.2:0/85233239 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ddef0efb80 con 0x55ddef195760 2024-05-14T17:15:37.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.651+0000 7f89f97fa640 10 client.17211 handle_command_reply: tid=0 2024-05-14T17:15:37.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.651+0000 7f8a1ec12740 1 client.17211 shutdown 2024-05-14T17:15:37.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.651+0000 7f8a1ec12740 1 -- 172.21.15.2:0/85233239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddef17b210 msgr2=0x55ddef182780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.651+0000 7f8a1ec12740 1 --2- 172.21.15.2:0/85233239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddef17b210 0x55ddef182780 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f8a080dfab0 tx=0x7f8a080e5a40 comp rx=0 tx=0).stop 2024-05-14T17:15:37.433 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.652+0000 7f8a1ec12740 1 -- 172.21.15.2:0/85233239 shutdown_connections 2024-05-14T17:15:37.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.652+0000 7f8a1ec12740 1 --2- 172.21.15.2:0/85233239 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ddef195760 0x55ddef197b50 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.652+0000 7f8a1ec12740 1 --2- 172.21.15.2:0/85233239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddef185ad0 0x55ddef182d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.652+0000 7f8a1ec12740 1 --2- 172.21.15.2:0/85233239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddef17b210 0x55ddef182780 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.652+0000 7f8a1ec12740 1 --2- 172.21.15.2:0/85233239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddef17a8f0 0x55ddef185490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.652+0000 7f89f97fa640 0 client.17211 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:15:37.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.652+0000 7f8a1ec12740 1 -- 172.21.15.2:0/85233239 >> 172.21.15.2:0/85233239 conn(0x55ddef176e00 msgr2=0x55ddef176100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:37.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.652+0000 7f8a1ec12740 1 -- 172.21.15.2:0/85233239 shutdown_connections 2024-05-14T17:15:37.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.652+0000 7f8a1ec12740 1 -- 172.21.15.2:0/85233239 wait complete. 2024-05-14T17:15:37.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.652+0000 7f8a1ec12740 20 client.17211 trim_cache size 0 max 16384 2024-05-14T17:15:37.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.667+0000 7f8a1ec12740 1 Processor -- start 2024-05-14T17:15:37.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.667+0000 7f8a1ec12740 1 -- start start 2024-05-14T17:15:37.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.667+0000 7f8a1ec12740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddef17a8f0 0x55ddef184fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.667+0000 7f8a1ec12740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddef17b210 0x55ddef17c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.434 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.667+0000 7f8a1ec12740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddef185ad0 0x55ddef19d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.667+0000 7f8a1ec12740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8a141b0df0 con 0x55ddef17a8f0 2024-05-14T17:15:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.667+0000 7f8a1ec12740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8a141ae690 con 0x55ddef185ad0 2024-05-14T17:15:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.667+0000 7f8a1ec12740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8a141ac500 con 0x55ddef17b210 2024-05-14T17:15:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.667+0000 7f8a19c98640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddef17a8f0 0x55ddef184fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.668+0000 7f8a19c98640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddef17a8f0 0x55ddef184fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34064/0 (socket says 172.21.15.2:34064) 2024-05-14T17:15:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.668+0000 7f8a19c98640 1 -- 172.21.15.2:0/3135901760 learned_addr learned my addr 172.21.15.2:0/3135901760 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.668+0000 7f8a1a499640 1 --2- 172.21.15.2:0/3135901760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddef185ad0 0x55ddef19d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.668+0000 7f8a19c98640 1 -- 172.21.15.2:0/3135901760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddef17b210 msgr2=0x55ddef17c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.668+0000 7f8a19c98640 1 --2- 172.21.15.2:0/3135901760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddef17b210 0x55ddef17c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.668+0000 7f8a19c98640 1 -- 172.21.15.2:0/3135901760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddef185ad0 msgr2=0x55ddef19d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.668+0000 7f8a19c98640 1 --2- 172.21.15.2:0/3135901760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddef185ad0 0x55ddef19d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.668+0000 7f8a19c98640 1 -- 172.21.15.2:0/3135901760 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddef180cd0 con 0x55ddef17a8f0 2024-05-14T17:15:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.668+0000 7f8a1a499640 1 --2- 172.21.15.2:0/3135901760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddef185ad0 0x55ddef19d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:15:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.668+0000 7f8a19c98640 1 --2- 172.21.15.2:0/3135901760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddef17a8f0 0x55ddef184fc0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c013cc0 tx=0x7f8a0c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a83a2840e36b9f8a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.668+0000 7f89fb7fe640 1 -- 172.21.15.2:0/3135901760 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c005020 con 0x55ddef17a8f0 2024-05-14T17:15:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.668+0000 7f89fb7fe640 1 -- 172.21.15.2:0/3135901760 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a0c0047a0 con 0x55ddef17a8f0 2024-05-14T17:15:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.668+0000 7f89fb7fe640 1 -- 172.21.15.2:0/3135901760 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c02a670 con 0x55ddef17a8f0 2024-05-14T17:15:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.669+0000 7f8a1ec12740 1 -- 172.21.15.2:0/3135901760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddef17a8f0 msgr2=0x55ddef184fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.669+0000 7f8a1ec12740 1 --2- 172.21.15.2:0/3135901760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddef17a8f0 0x55ddef184fc0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c013cc0 tx=0x7f8a0c016000 comp rx=0 tx=0).stop 2024-05-14T17:15:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.669+0000 7f8a1ec12740 1 -- 172.21.15.2:0/3135901760 shutdown_connections 2024-05-14T17:15:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.669+0000 7f8a1ec12740 1 --2- 172.21.15.2:0/3135901760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddef185ad0 0x55ddef19d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.669+0000 7f8a1ec12740 1 --2- 172.21.15.2:0/3135901760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddef17b210 0x55ddef17c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.669+0000 7f8a1ec12740 1 --2- 172.21.15.2:0/3135901760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddef17a8f0 0x55ddef184fc0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.669+0000 7f8a1ec12740 1 -- 172.21.15.2:0/3135901760 >> 172.21.15.2:0/3135901760 conn(0x55ddef176e00 msgr2=0x55ddef17c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.669+0000 7f8a1ec12740 1 -- 172.21.15.2:0/3135901760 shutdown_connections 2024-05-14T17:15:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.669+0000 7f8a1ec12740 1 -- 172.21.15.2:0/3135901760 wait complete. 2024-05-14T17:15:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.669+0000 7f8a1ec12740 1 -- start start 2024-05-14T17:15:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.669+0000 7f8a1ec12740 1 Processor -- start 2024-05-14T17:15:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.670+0000 7f8a1ec12740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddef17a8f0 0x55ddef189860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.670+0000 7f8a1ec12740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddef17b210 0x55ddef189da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.670+0000 7f8a1ec12740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddef185ad0 0x55ddef1fefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.670+0000 7f8a1ec12740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8a141acb50 con 0x55ddef17b210 2024-05-14T17:15:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.670+0000 7f8a1ec12740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8a141adfc0 con 0x55ddef185ad0 2024-05-14T17:15:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.670+0000 7f8a1ec12740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8a141ad220 con 0x55ddef17a8f0 2024-05-14T17:15:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.670+0000 7f8a1ec12740 10 client.0 _ll_register_callbacks cb 0x7f8a080a5570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:15:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.670+0000 7f8a19497640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddef17b210 0x55ddef189da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.670+0000 7f8a19497640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddef17b210 0x55ddef189da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34066/0 (socket says 172.21.15.2:34066) 2024-05-14T17:15:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.670+0000 7f8a19497640 1 -- 172.21.15.2:0/1009222391 learned_addr learned my addr 172.21.15.2:0/1009222391 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.670+0000 7f8a19c98640 1 --2- 172.21.15.2:0/1009222391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddef17a8f0 0x55ddef189860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.670+0000 7f8a1a499640 1 --2- 172.21.15.2:0/1009222391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddef185ad0 0x55ddef1fefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.670+0000 7f8a19497640 1 -- 172.21.15.2:0/1009222391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddef17a8f0 msgr2=0x55ddef189860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.670+0000 7f8a19497640 1 --2- 172.21.15.2:0/1009222391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddef17a8f0 0x55ddef189860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.670+0000 7f8a19497640 1 -- 172.21.15.2:0/1009222391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddef185ad0 msgr2=0x55ddef1fefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.670+0000 7f8a19497640 1 --2- 172.21.15.2:0/1009222391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddef185ad0 0x55ddef1fefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.670+0000 7f8a19497640 1 -- 172.21.15.2:0/1009222391 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddef18a670 con 0x55ddef17b210 2024-05-14T17:15:37.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.670+0000 7f8a1a499640 1 --2- 172.21.15.2:0/1009222391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddef185ad0 0x55ddef1fefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:15:37.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.670+0000 7f8a19497640 1 --2- 172.21.15.2:0/1009222391 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddef17b210 0x55ddef189da0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f8a0808ff40 tx=0x7f8a080a5a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f50b55a5a3079c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:37.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.670+0000 7f89f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:15:37.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.671+0000 7f89f97fa640 1 -- 172.21.15.2:0/1009222391 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a08020070 con 0x55ddef17b210 2024-05-14T17:15:37.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.671+0000 7f89f97fa640 1 -- 172.21.15.2:0/1009222391 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8513+0+0 (secure 0 0 0) 0x7f8a080a8a50 con 0x55ddef17b210 2024-05-14T17:15:37.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.671+0000 7f8a1ec12740 1 -- 172.21.15.2:0/1009222391 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a0c0047a0 con 0x55ddef17b210 2024-05-14T17:15:37.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.671+0000 7f89f97fa640 1 -- 172.21.15.2:0/1009222391 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f8a0806d050 con 0x55ddef17b210 2024-05-14T17:15:37.438 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.671+0000 7f8a1ec12740 10 client.17217 fetch_fsmap learned FSMap version 56 2024-05-14T17:15:37.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.671+0000 7f8a1ec12740 1 -- 172.21.15.2:0/1009222391 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x55ddef200560 con 0x55ddef17b210 2024-05-14T17:15:37.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.671+0000 7f89f97fa640 1 -- 172.21.15.2:0/1009222391 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a0c0047a0 con 0x55ddef17b210 2024-05-14T17:15:37.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.671+0000 7f89f97fa640 1 -- 172.21.15.2:0/1009222391 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 56) ==== 8346+0+0 (secure 0 0 0) 0x7f8a08010750 con 0x55ddef17b210 2024-05-14T17:15:37.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.671+0000 7f8a1ec12740 10 client.17217 fetch_fsmap finished waiting for FSMap version 56 2024-05-14T17:15:37.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.671+0000 7f8a1ec12740 10 client.17217 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:15:37.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.671+0000 7f8a1ec12740 20 client.17217 populate_metadata read hostname 'smithi002' 2024-05-14T17:15:37.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.671+0000 7f8a1ec12740 1 --2- 172.21.15.2:0/1009222391 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ddef200a90 0x55ddef202eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.671+0000 7f8a1ec12740 4 client.17217 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:15:37.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.671+0000 7f8a1ec12740 1 -- 172.21.15.2:0/1009222391 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ddef0e9360 con 0x55ddef200a90 2024-05-14T17:15:37.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.671+0000 7f8a19c98640 1 --2- 172.21.15.2:0/1009222391 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ddef200a90 0x55ddef202eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.672+0000 7f8a19c98640 1 --2- 172.21.15.2:0/1009222391 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ddef200a90 0x55ddef202eb0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=962df2f66c9ec458 server_cookie=32d54fe0d989f17b in_seq=0 out_seq=0 2024-05-14T17:15:37.439 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.672+0000 7f89f97fa640 10 client.17217 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:15:37.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.689+0000 7f89f97fa640 1 -- 172.21.15.2:0/1009222391 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ddef0e9360 con 0x55ddef200a90 2024-05-14T17:15:37.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.689+0000 7f89f97fa640 10 client.17217 handle_command_reply: tid=0 2024-05-14T17:15:37.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.689+0000 7f8a1ec12740 1 client.17217 shutdown 2024-05-14T17:15:37.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.689+0000 7f8a1ec12740 1 -- 172.21.15.2:0/1009222391 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddef17b210 msgr2=0x55ddef189da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.689+0000 7f8a1ec12740 1 --2- 172.21.15.2:0/1009222391 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddef17b210 0x55ddef189da0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f8a0808ff40 tx=0x7f8a080a5a80 comp rx=0 tx=0).stop 2024-05-14T17:15:37.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.689+0000 7f8a1ec12740 1 -- 172.21.15.2:0/1009222391 shutdown_connections 2024-05-14T17:15:37.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.689+0000 7f8a1ec12740 1 --2- 172.21.15.2:0/1009222391 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55ddef200a90 0x55ddef202eb0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.689+0000 7f8a1ec12740 1 --2- 172.21.15.2:0/1009222391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddef185ad0 0x55ddef1fefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.689+0000 7f8a1ec12740 1 --2- 172.21.15.2:0/1009222391 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddef17b210 0x55ddef189da0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.689+0000 7f8a1ec12740 1 --2- 172.21.15.2:0/1009222391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ddef17a8f0 0x55ddef189860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.689+0000 7f89f97fa640 0 client.17217 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:15:37.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.689+0000 7f8a1ec12740 1 -- 172.21.15.2:0/1009222391 >> 172.21.15.2:0/1009222391 conn(0x55ddef176e00 msgr2=0x55ddef17cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:37.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.689+0000 7f8a1ec12740 1 -- 172.21.15.2:0/1009222391 shutdown_connections 2024-05-14T17:15:37.440 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.689+0000 7f8a1ec12740 1 -- 172.21.15.2:0/1009222391 wait complete. 2024-05-14T17:15:37.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.689+0000 7f8a1ec12740 20 client.17217 trim_cache size 0 max 16384 2024-05-14T17:15:37.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:15:37.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:15:37.441 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:15:37.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.690+0000 7f8a1befc640 1 -- 172.21.15.2:0/3914982192 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f89fc0b2a10 msgr2=0x7f89fc0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.442 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.690+0000 7f8a1befc640 1 --2- 172.21.15.2:0/3914982192 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f89fc0b2a10 0x7f89fc0b4ed0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x55ddef17be90 tx=0x7f8a0c005f90 comp rx=0 tx=0).stop 2024-05-14T17:15:37.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.690+0000 7f8a1befc640 1 -- 172.21.15.2:0/3914982192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a14102340 msgr2=0x7f8a141997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.690+0000 7f8a1befc640 1 --2- 172.21.15.2:0/3914982192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a14102340 0x7f8a141997b0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f8a0800d950 tx=0x7f8a0800de10 comp rx=0 tx=0).stop 2024-05-14T17:15:37.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.690+0000 7f8a1befc640 1 -- 172.21.15.2:0/3914982192 shutdown_connections 2024-05-14T17:15:37.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.690+0000 7f8a1befc640 1 --2- 172.21.15.2:0/3914982192 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f89fc0b2a10 0x7f89fc0b4ed0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.690+0000 7f8a1befc640 1 --2- 172.21.15.2:0/3914982192 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a14107940 0x7f8a1419db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.690+0000 7f8a1befc640 1 --2- 172.21.15.2:0/3914982192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a14102340 0x7f8a141997b0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.690+0000 7f8a1befc640 1 --2- 172.21.15.2:0/3914982192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a14101980 0x7f8a14199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.690+0000 7f8a1befc640 1 -- 172.21.15.2:0/3914982192 >> 172.21.15.2:0/3914982192 conn(0x7f8a14078ad0 msgr2=0x7f8a1406a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:37.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.690+0000 7f8a1befc640 1 -- 172.21.15.2:0/3914982192 shutdown_connections 2024-05-14T17:15:37.443 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:21.690+0000 7f8a1befc640 1 -- 172.21.15.2:0/3914982192 wait complete. 2024-05-14T17:15:37.444 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:35 smithi123 ceph-mon[35971]: pgmap v659: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.9 KiB/s wr, 8 op/s 2024-05-14T17:15:37.444 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:14:22 smithi002 ceph-mon[31538]: pgmap v619: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 38 KiB/s wr, 16 op/s 2024-05-14T17:15:37.445 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:14:24 smithi002 ceph-mon[31538]: pgmap v620: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-05-14T17:15:37.445 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:14:26 smithi002 ceph-mon[31538]: pgmap v621: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-05-14T17:15:37.445 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:14:28 smithi002 ceph-mon[31538]: pgmap v622: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 26 KiB/s wr, 13 op/s 2024-05-14T17:15:37.445 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:14:30 smithi002 ceph-mon[31538]: pgmap v623: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-05-14T17:15:37.445 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:14:32 smithi002 ceph-mon[31538]: pgmap v624: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 KiB/s wr, 10 op/s 2024-05-14T17:15:37.445 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:14:33 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:15:37.458 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.790+0000 7fe19a146640 1 -- 172.21.15.2:0/3605154705 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe194101980 msgr2=0x7fe194101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.790+0000 7fe19a146640 1 --2- 172.21.15.2:0/3605154705 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe194101980 0x7fe194101e00 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fe188009930 tx=0x7fe18802a420 comp rx=0 tx=0).stop 2024-05-14T17:15:37.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.791+0000 7fe19a146640 1 -- 172.21.15.2:0/3605154705 shutdown_connections 2024-05-14T17:15:37.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.791+0000 7fe19a146640 1 --2- 172.21.15.2:0/3605154705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe194102340 0x7fe19410e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.791+0000 7fe19a146640 1 --2- 172.21.15.2:0/3605154705 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe194101980 0x7fe194101e00 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.791+0000 7fe19a146640 1 --2- 172.21.15.2:0/3605154705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe194107940 0x7fe194107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.791+0000 7fe19a146640 1 -- 172.21.15.2:0/3605154705 >> 172.21.15.2:0/3605154705 conn(0x7fe194078ad0 msgr2=0x7fe194078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:37.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.791+0000 7fe19a146640 1 -- 172.21.15.2:0/3605154705 shutdown_connections 2024-05-14T17:15:37.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.791+0000 7fe19a146640 1 -- 172.21.15.2:0/3605154705 wait complete. 2024-05-14T17:15:37.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.791+0000 7fe19a146640 1 Processor -- start 2024-05-14T17:15:37.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.792+0000 7fe19a146640 1 -- start start 2024-05-14T17:15:37.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.792+0000 7fe19a146640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe194101980 0x7fe194199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.459 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.792+0000 7fe19a146640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe194102340 0x7fe194199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.792+0000 7fe19a146640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe194107940 0x7fe19419dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.792+0000 7fe19a146640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe194112d00 con 0x7fe194102340 2024-05-14T17:15:37.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.792+0000 7fe19a146640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe194112b80 con 0x7fe194107940 2024-05-14T17:15:37.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.792+0000 7fe19a146640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe194112e80 con 0x7fe194101980 2024-05-14T17:15:37.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.792+0000 7fe192ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe194102340 0x7fe194199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.792+0000 7fe192ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe194102340 0x7fe194199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48416/0 (socket says 172.21.15.2:48416) 2024-05-14T17:15:37.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.792+0000 7fe192ffd640 1 -- 172.21.15.2:0/740409012 learned_addr learned my addr 172.21.15.2:0/740409012 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:37.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.792+0000 7fe193fff640 1 --2- 172.21.15.2:0/740409012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe194107940 0x7fe19419dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.792+0000 7fe1937fe640 1 --2- 172.21.15.2:0/740409012 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe194101980 0x7fe194199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.792+0000 7fe192ffd640 1 -- 172.21.15.2:0/740409012 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe194101980 msgr2=0x7fe194199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.792+0000 7fe192ffd640 1 --2- 172.21.15.2:0/740409012 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe194101980 0x7fe194199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.792+0000 7fe192ffd640 1 -- 172.21.15.2:0/740409012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe194107940 msgr2=0x7fe19419dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.792+0000 7fe192ffd640 1 --2- 172.21.15.2:0/740409012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe194107940 0x7fe19419dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.460 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.792+0000 7fe192ffd640 1 -- 172.21.15.2:0/740409012 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe19419e1f0 con 0x7fe194102340 2024-05-14T17:15:37.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.793+0000 7fe192ffd640 1 --2- 172.21.15.2:0/740409012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe194102340 0x7fe194199740 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fe188004770 tx=0x7fe1880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:37.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.793+0000 7fe190ff9640 1 -- 172.21.15.2:0/740409012 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18803f070 con 0x7fe194102340 2024-05-14T17:15:37.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.793+0000 7fe190ff9640 1 -- 172.21.15.2:0/740409012 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe18803b460 con 0x7fe194102340 2024-05-14T17:15:37.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.793+0000 7fe19a146640 1 -- 172.21.15.2:0/740409012 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe19419e480 con 0x7fe194102340 2024-05-14T17:15:37.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.793+0000 7fe190ff9640 1 -- 172.21.15.2:0/740409012 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe188035070 con 0x7fe194102340 2024-05-14T17:15:37.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.793+0000 7fe19a146640 1 -- 172.21.15.2:0/740409012 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1941a5d70 con 0x7fe194102340 2024-05-14T17:15:37.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.795+0000 7fe19ce65740 1 Processor -- start 2024-05-14T17:15:37.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.795+0000 7fe19ce65740 1 -- start start 2024-05-14T17:15:37.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.795+0000 7fe190ff9640 1 -- 172.21.15.2:0/740409012 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fe18803b8d0 con 0x7fe194102340 2024-05-14T17:15:37.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.796+0000 7fe19ce65740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556647c8b8f0 0x556647c8bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.796+0000 7fe19ce65740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556647c8c210 0x556647c96590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.796+0000 7fe19ce65740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556647c96ad0 0x556647c98ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.796+0000 7fe19ce65740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x556647bfec30 con 0x556647c8c210 2024-05-14T17:15:37.461 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.796+0000 7fe19ce65740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556647bee7d0 con 0x556647c8b8f0 2024-05-14T17:15:37.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.796+0000 7fe19ce65740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556647bf2810 con 0x556647c96ad0 2024-05-14T17:15:37.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.796+0000 7fe1937fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556647c8c210 0x556647c96590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.796+0000 7fe1937fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556647c8c210 0x556647c96590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48426/0 (socket says 172.21.15.2:48426) 2024-05-14T17:15:37.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.796+0000 7fe1937fe640 1 -- 172.21.15.2:0/3056018103 learned_addr learned my addr 172.21.15.2:0/3056018103 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:37.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.796+0000 7fe193fff640 1 --2- 172.21.15.2:0/3056018103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556647c8b8f0 0x556647c8bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.796+0000 7fe192ffd640 1 --2- 172.21.15.2:0/3056018103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556647c96ad0 0x556647c98ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.796+0000 7fe1937fe640 1 -- 172.21.15.2:0/3056018103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556647c96ad0 msgr2=0x556647c98ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.796+0000 7fe1937fe640 1 --2- 172.21.15.2:0/3056018103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556647c96ad0 0x556647c98ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.796+0000 7fe1937fe640 1 -- 172.21.15.2:0/3056018103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556647c8b8f0 msgr2=0x556647c8bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.796+0000 7fe1937fe640 1 --2- 172.21.15.2:0/3056018103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556647c8b8f0 0x556647c8bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.796+0000 7fe1937fe640 1 -- 172.21.15.2:0/3056018103 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556647903210 con 0x556647c8c210 2024-05-14T17:15:37.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.796+0000 7fe193fff640 1 --2- 172.21.15.2:0/3056018103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556647c8b8f0 0x556647c8bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:15:37.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.797+0000 7fe1937fe640 1 --2- 172.21.15.2:0/3056018103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556647c8c210 0x556647c96590 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fe18400bc90 tx=0x7fe1840107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=708076004186194a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:37.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.797+0000 7fe190ff9640 1 --2- 172.21.15.2:0/740409012 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fe17c0b2a10 0x7fe17c0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.462 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.797+0000 7fe179ffb640 1 -- 172.21.15.2:0/3056018103 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18401c480 con 0x556647c8c210 2024-05-14T17:15:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.797+0000 7fe193fff640 1 --2- 172.21.15.2:0/740409012 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fe17c0b2a10 0x7fe17c0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.797+0000 7fe179ffb640 1 -- 172.21.15.2:0/3056018103 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe18401cad0 con 0x556647c8c210 2024-05-14T17:15:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.797+0000 7fe179ffb640 1 -- 172.21.15.2:0/3056018103 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe184011ae0 con 0x556647c8c210 2024-05-14T17:15:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.797+0000 7fe190ff9640 1 -- 172.21.15.2:0/740409012 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8513+0+0 (secure 0 0 0) 0x7fe1881028e0 con 0x7fe194102340 2024-05-14T17:15:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.798+0000 7fe19ce65740 1 -- 172.21.15.2:0/3056018103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556647c8c210 msgr2=0x556647c96590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.798+0000 7fe193fff640 1 --2- 172.21.15.2:0/740409012 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fe17c0b2a10 0x7fe17c0b4ed0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe18c005fd0 tx=0x7fe18c0096a0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.798+0000 7fe19ce65740 1 --2- 172.21.15.2:0/3056018103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556647c8c210 0x556647c96590 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fe18400bc90 tx=0x7fe1840107a0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.798+0000 7fe19ce65740 1 -- 172.21.15.2:0/3056018103 shutdown_connections 2024-05-14T17:15:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.798+0000 7fe19ce65740 1 --2- 172.21.15.2:0/3056018103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556647c96ad0 0x556647c98ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.798+0000 7fe19ce65740 1 --2- 172.21.15.2:0/3056018103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556647c8c210 0x556647c96590 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.798+0000 7fe19ce65740 1 --2- 172.21.15.2:0/3056018103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556647c8b8f0 0x556647c8bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.798+0000 7fe19ce65740 1 -- 172.21.15.2:0/3056018103 >> 172.21.15.2:0/3056018103 conn(0x556647c87e00 msgr2=0x556647c881e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:37.463 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.798+0000 7fe19ce65740 1 -- 172.21.15.2:0/3056018103 shutdown_connections 2024-05-14T17:15:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.798+0000 7fe19ce65740 1 -- 172.21.15.2:0/3056018103 wait complete. 2024-05-14T17:15:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.798+0000 7fe19ce65740 1 -- start start 2024-05-14T17:15:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.798+0000 7fe19ce65740 1 Processor -- start 2024-05-14T17:15:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.799+0000 7fe19ce65740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556647c8b8f0 0x556647c96430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.799+0000 7fe19ce65740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556647c8c210 0x556647c93720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.799+0000 7fe19ce65740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556647c96ad0 0x556647c93c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.799+0000 7fe19ce65740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x5566478d4790 con 0x556647c8b8f0 2024-05-14T17:15:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.799+0000 7fe19ce65740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556647baa890 con 0x556647c96ad0 2024-05-14T17:15:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.799+0000 7fe19ce65740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556647c997a0 con 0x556647c8c210 2024-05-14T17:15:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.799+0000 7fe19ce65740 10 client.0 _ll_register_callbacks cb 0x556647badd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:15:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.799+0000 7fe1937fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556647c8b8f0 0x556647c96430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.799+0000 7fe1937fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556647c8b8f0 0x556647c96430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48432/0 (socket says 172.21.15.2:48432) 2024-05-14T17:15:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.799+0000 7fe1937fe640 1 -- 172.21.15.2:0/2232332233 learned_addr learned my addr 172.21.15.2:0/2232332233 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:37.464 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.799+0000 7fe192ffd640 1 --2- 172.21.15.2:0/2232332233 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556647c8c210 0x556647c93720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.799+0000 7fe193fff640 1 --2- 172.21.15.2:0/2232332233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556647c96ad0 0x556647c93c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.799+0000 7fe1937fe640 1 -- 172.21.15.2:0/2232332233 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556647c8c210 msgr2=0x556647c93720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.799+0000 7fe1937fe640 1 --2- 172.21.15.2:0/2232332233 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556647c8c210 0x556647c93720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.799+0000 7fe1937fe640 1 -- 172.21.15.2:0/2232332233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556647c96ad0 msgr2=0x556647c93c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.799+0000 7fe1937fe640 1 --2- 172.21.15.2:0/2232332233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556647c96ad0 0x556647c93c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.799+0000 7fe1937fe640 1 -- 172.21.15.2:0/2232332233 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566479cf620 con 0x556647c8b8f0 2024-05-14T17:15:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.799+0000 7fe192ffd640 1 --2- 172.21.15.2:0/2232332233 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556647c8c210 0x556647c93720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:15:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.799+0000 7fe1937fe640 1 --2- 172.21.15.2:0/2232332233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556647c8b8f0 0x556647c96430 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fe184011700 tx=0x7fe18400b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=798ee075402ac766 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.799+0000 7fe172ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:15:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.799+0000 7fe172ffd640 1 -- 172.21.15.2:0/2232332233 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18401c680 con 0x556647c8b8f0 2024-05-14T17:15:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.799+0000 7fe172ffd640 1 -- 172.21.15.2:0/2232332233 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8513+0+0 (secure 0 0 0) 0x7fe18401c820 con 0x556647c8b8f0 2024-05-14T17:15:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.800+0000 7fe19ce65740 1 -- 172.21.15.2:0/2232332233 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe18401cad0 con 0x556647c8b8f0 2024-05-14T17:15:37.465 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.800+0000 7fe172ffd640 1 -- 172.21.15.2:0/2232332233 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fe184011860 con 0x556647c8b8f0 2024-05-14T17:15:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.800+0000 7fe19ce65740 10 client.17229 fetch_fsmap learned FSMap version 57 2024-05-14T17:15:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.800+0000 7fe19ce65740 1 -- 172.21.15.2:0/2232332233 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x556647ca51c0 con 0x556647c8b8f0 2024-05-14T17:15:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.800+0000 7fe172ffd640 1 -- 172.21.15.2:0/2232332233 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe18401cad0 con 0x556647c8b8f0 2024-05-14T17:15:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.800+0000 7fe172ffd640 1 -- 172.21.15.2:0/2232332233 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 57) ==== 8350+0+0 (secure 0 0 0) 0x7fe1840046c0 con 0x556647c8b8f0 2024-05-14T17:15:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.800+0000 7fe19ce65740 10 client.17229 fetch_fsmap finished waiting for FSMap version 57 2024-05-14T17:15:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.800+0000 7fe19ce65740 10 client.17229 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:15:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.800+0000 7fe19ce65740 20 client.17229 populate_metadata read hostname 'smithi002' 2024-05-14T17:15:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.800+0000 7fe19ce65740 1 --2- 172.21.15.2:0/2232332233 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x556647ca65c0 0x556647ca89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.800+0000 7fe19ce65740 4 client.17229 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:15:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.800+0000 7fe19ce65740 1 -- 172.21.15.2:0/2232332233 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556647c00b80 con 0x556647ca65c0 2024-05-14T17:15:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.800+0000 7fe192ffd640 1 --2- 172.21.15.2:0/2232332233 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x556647ca65c0 0x556647ca89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.801+0000 7fe192ffd640 1 --2- 172.21.15.2:0/2232332233 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x556647ca65c0 0x556647ca89b0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d908f942543f569 server_cookie=a9737c1393e29324 in_seq=0 out_seq=0 2024-05-14T17:15:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.801+0000 7fe172ffd640 10 client.17229 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:15:37.466 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.802+0000 7fe172ffd640 1 -- 172.21.15.2:0/2232332233 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556647c00b80 con 0x556647ca65c0 2024-05-14T17:15:37.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.802+0000 7fe172ffd640 10 client.17229 handle_command_reply: tid=0 2024-05-14T17:15:37.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.802+0000 7fe19ce65740 1 client.17229 shutdown 2024-05-14T17:15:37.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.802+0000 7fe19ce65740 1 -- 172.21.15.2:0/2232332233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556647c8b8f0 msgr2=0x556647c96430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.802+0000 7fe19ce65740 1 --2- 172.21.15.2:0/2232332233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556647c8b8f0 0x556647c96430 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fe184011700 tx=0x7fe18400b970 comp rx=0 tx=0).stop 2024-05-14T17:15:37.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.802+0000 7fe19ce65740 1 -- 172.21.15.2:0/2232332233 shutdown_connections 2024-05-14T17:15:37.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.802+0000 7fe19ce65740 1 --2- 172.21.15.2:0/2232332233 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x556647ca65c0 0x556647ca89b0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.802+0000 7fe19ce65740 1 --2- 172.21.15.2:0/2232332233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556647c96ad0 0x556647c93c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.802+0000 7fe19ce65740 1 --2- 172.21.15.2:0/2232332233 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556647c8c210 0x556647c93720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.802+0000 7fe172ffd640 0 client.17229 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:15:37.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.802+0000 7fe19ce65740 1 --2- 172.21.15.2:0/2232332233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556647c8b8f0 0x556647c96430 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.802+0000 7fe19ce65740 1 -- 172.21.15.2:0/2232332233 >> 172.21.15.2:0/2232332233 conn(0x556647c87e00 msgr2=0x556647c87100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:37.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.802+0000 7fe19ce65740 1 -- 172.21.15.2:0/2232332233 shutdown_connections 2024-05-14T17:15:37.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.802+0000 7fe19ce65740 1 -- 172.21.15.2:0/2232332233 wait complete. 2024-05-14T17:15:37.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.802+0000 7fe19ce65740 20 client.17229 trim_cache size 0 max 16384 2024-05-14T17:15:37.467 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.817+0000 7fe19ce65740 1 Processor -- start 2024-05-14T17:15:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.817+0000 7fe19ce65740 1 -- start start 2024-05-14T17:15:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.818+0000 7fe19ce65740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556647c8b8f0 0x556647ca2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.818+0000 7fe19ce65740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556647c8c210 0x556647c95f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.818+0000 7fe19ce65740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556647c96ad0 0x556647cadfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.818+0000 7fe19ce65740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe1941b1020 con 0x556647c96ad0 2024-05-14T17:15:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.818+0000 7fe19ce65740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe1941ae890 con 0x556647c8b8f0 2024-05-14T17:15:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.818+0000 7fe19ce65740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe1941ac700 con 0x556647c8c210 2024-05-14T17:15:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.818+0000 7fe193fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556647c96ad0 0x556647cadfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.818+0000 7fe193fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556647c96ad0 0x556647cadfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48448/0 (socket says 172.21.15.2:48448) 2024-05-14T17:15:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.818+0000 7fe193fff640 1 -- 172.21.15.2:0/985786434 learned_addr learned my addr 172.21.15.2:0/985786434 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.818+0000 7fe192ffd640 1 --2- 172.21.15.2:0/985786434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556647c8c210 0x556647c95f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.818+0000 7fe1937fe640 1 --2- 172.21.15.2:0/985786434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556647c8b8f0 0x556647ca2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.468 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.818+0000 7fe193fff640 1 -- 172.21.15.2:0/985786434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556647c8c210 msgr2=0x556647c95f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.818+0000 7fe193fff640 1 --2- 172.21.15.2:0/985786434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556647c8c210 0x556647c95f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.818+0000 7fe193fff640 1 -- 172.21.15.2:0/985786434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556647c8b8f0 msgr2=0x556647ca2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.818+0000 7fe193fff640 1 --2- 172.21.15.2:0/985786434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556647c8b8f0 0x556647ca2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.818+0000 7fe193fff640 1 -- 172.21.15.2:0/985786434 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556647c91c70 con 0x556647c96ad0 2024-05-14T17:15:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.818+0000 7fe1937fe640 1 --2- 172.21.15.2:0/985786434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556647c8b8f0 0x556647ca2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:15:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.818+0000 7fe193fff640 1 --2- 172.21.15.2:0/985786434 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556647c96ad0 0x556647cadfb0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fe18c003a20 tx=0x7fe18c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a409d1c47147353 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.819+0000 7fe1797fa640 1 -- 172.21.15.2:0/985786434 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c01ee30 con 0x556647c96ad0 2024-05-14T17:15:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.819+0000 7fe1797fa640 1 -- 172.21.15.2:0/985786434 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe18c01e0d0 con 0x556647c96ad0 2024-05-14T17:15:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.819+0000 7fe1797fa640 1 -- 172.21.15.2:0/985786434 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c01c400 con 0x556647c96ad0 2024-05-14T17:15:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.819+0000 7fe19ce65740 1 -- 172.21.15.2:0/985786434 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556647c96ad0 msgr2=0x556647cadfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.819+0000 7fe19ce65740 1 --2- 172.21.15.2:0/985786434 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556647c96ad0 0x556647cadfb0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fe18c003a20 tx=0x7fe18c016000 comp rx=0 tx=0).stop 2024-05-14T17:15:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.819+0000 7fe19ce65740 1 -- 172.21.15.2:0/985786434 shutdown_connections 2024-05-14T17:15:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.819+0000 7fe19ce65740 1 --2- 172.21.15.2:0/985786434 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556647c96ad0 0x556647cadfb0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.819+0000 7fe19ce65740 1 --2- 172.21.15.2:0/985786434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556647c8c210 0x556647c95f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.819+0000 7fe19ce65740 1 --2- 172.21.15.2:0/985786434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556647c8b8f0 0x556647ca2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.469 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.819+0000 7fe19ce65740 1 -- 172.21.15.2:0/985786434 >> 172.21.15.2:0/985786434 conn(0x556647c87e00 msgr2=0x556647c8db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.819+0000 7fe19ce65740 1 -- 172.21.15.2:0/985786434 shutdown_connections 2024-05-14T17:15:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.819+0000 7fe19ce65740 1 -- 172.21.15.2:0/985786434 wait complete. 2024-05-14T17:15:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.820+0000 7fe19ce65740 1 -- start start 2024-05-14T17:15:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.820+0000 7fe19ce65740 1 Processor -- start 2024-05-14T17:15:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.820+0000 7fe19ce65740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556647c8b8f0 0x556647c9efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.820+0000 7fe19ce65740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556647c8c210 0x556647c9a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.820+0000 7fe19ce65740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556647c96ad0 0x556647c9a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.820+0000 7fe19ce65740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe1941acd50 con 0x556647c8c210 2024-05-14T17:15:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.820+0000 7fe19ce65740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe1941ae1c0 con 0x556647c96ad0 2024-05-14T17:15:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.820+0000 7fe19ce65740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe1941ad420 con 0x556647c8b8f0 2024-05-14T17:15:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.820+0000 7fe19ce65740 10 client.0 _ll_register_callbacks cb 0x7fe18400bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:15:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.820+0000 7fe192ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556647c8c210 0x556647c9a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.470 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.820+0000 7fe192ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556647c8c210 0x556647c9a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48452/0 (socket says 172.21.15.2:48452) 2024-05-14T17:15:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.820+0000 7fe192ffd640 1 -- 172.21.15.2:0/3316640165 learned_addr learned my addr 172.21.15.2:0/3316640165 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.820+0000 7fe193fff640 1 --2- 172.21.15.2:0/3316640165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556647c96ad0 0x556647c9a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.820+0000 7fe1937fe640 1 --2- 172.21.15.2:0/3316640165 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556647c8b8f0 0x556647c9efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.821+0000 7fe192ffd640 1 -- 172.21.15.2:0/3316640165 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556647c8b8f0 msgr2=0x556647c9efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.821+0000 7fe192ffd640 1 --2- 172.21.15.2:0/3316640165 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556647c8b8f0 0x556647c9efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.821+0000 7fe192ffd640 1 -- 172.21.15.2:0/3316640165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556647c96ad0 msgr2=0x556647c9a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.821+0000 7fe192ffd640 1 --2- 172.21.15.2:0/3316640165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556647c96ad0 0x556647c9a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.821+0000 7fe192ffd640 1 -- 172.21.15.2:0/3316640165 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556647c9b100 con 0x556647c8c210 2024-05-14T17:15:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.821+0000 7fe192ffd640 1 --2- 172.21.15.2:0/3316640165 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556647c8c210 0x556647c9a350 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fe188000c00 tx=0x7fe188108550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c7aff3f2c2cac51 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.821+0000 7fe172ffd640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:15:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.821+0000 7fe172ffd640 1 -- 172.21.15.2:0/3316640165 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe188048040 con 0x556647c8c210 2024-05-14T17:15:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.821+0000 7fe172ffd640 1 -- 172.21.15.2:0/3316640165 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8513+0+0 (secure 0 0 0) 0x7fe1880c5f00 con 0x556647c8c210 2024-05-14T17:15:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.821+0000 7fe19ce65740 1 -- 172.21.15.2:0/3316640165 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe18c01e0d0 con 0x556647c8c210 2024-05-14T17:15:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.821+0000 7fe172ffd640 1 -- 172.21.15.2:0/3316640165 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fe1880c95c0 con 0x556647c8c210 2024-05-14T17:15:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.821+0000 7fe19ce65740 10 client.17235 fetch_fsmap learned FSMap version 57 2024-05-14T17:15:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.821+0000 7fe19ce65740 1 -- 172.21.15.2:0/3316640165 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x556647d10be0 con 0x556647c8c210 2024-05-14T17:15:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.821+0000 7fe172ffd640 1 -- 172.21.15.2:0/3316640165 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe18c01e0d0 con 0x556647c8c210 2024-05-14T17:15:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.821+0000 7fe172ffd640 1 -- 172.21.15.2:0/3316640165 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 57) ==== 8350+0+0 (secure 0 0 0) 0x7fe1880d2040 con 0x556647c8c210 2024-05-14T17:15:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.821+0000 7fe19ce65740 10 client.17235 fetch_fsmap finished waiting for FSMap version 57 2024-05-14T17:15:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.821+0000 7fe19ce65740 10 client.17235 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:15:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.821+0000 7fe19ce65740 20 client.17235 populate_metadata read hostname 'smithi002' 2024-05-14T17:15:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.821+0000 7fe19ce65740 1 --2- 172.21.15.2:0/3316640165 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x556647d11310 0x556647d13730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.821+0000 7fe19ce65740 4 client.17235 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:15:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.821+0000 7fe19ce65740 1 -- 172.21.15.2:0/3316640165 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556647bfa360 con 0x556647d11310 2024-05-14T17:15:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.822+0000 7fe1937fe640 1 --2- 172.21.15.2:0/3316640165 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x556647d11310 0x556647d13730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.822+0000 7fe1937fe640 1 --2- 172.21.15.2:0/3316640165 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x556647d11310 0x556647d13730 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a8e7c7c96e45007 server_cookie=73207b7d6696a91d in_seq=0 out_seq=0 2024-05-14T17:15:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.822+0000 7fe172ffd640 10 client.17235 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:15:37.472 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.848+0000 7fe172ffd640 1 -- 172.21.15.2:0/3316640165 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556647bfa360 con 0x556647d11310 2024-05-14T17:15:37.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.849+0000 7fe172ffd640 10 client.17235 handle_command_reply: tid=0 2024-05-14T17:15:37.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.849+0000 7fe19ce65740 1 client.17235 shutdown 2024-05-14T17:15:37.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.849+0000 7fe19ce65740 1 -- 172.21.15.2:0/3316640165 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556647c8c210 msgr2=0x556647c9a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.849+0000 7fe19ce65740 1 --2- 172.21.15.2:0/3316640165 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556647c8c210 0x556647c9a350 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fe188000c00 tx=0x7fe188108550 comp rx=0 tx=0).stop 2024-05-14T17:15:37.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.849+0000 7fe19ce65740 1 -- 172.21.15.2:0/3316640165 shutdown_connections 2024-05-14T17:15:37.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.849+0000 7fe19ce65740 1 --2- 172.21.15.2:0/3316640165 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x556647d11310 0x556647d13730 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.849+0000 7fe19ce65740 1 --2- 172.21.15.2:0/3316640165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556647c96ad0 0x556647c9a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.849+0000 7fe172ffd640 0 client.17235 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:15:37.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.849+0000 7fe19ce65740 1 --2- 172.21.15.2:0/3316640165 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556647c8c210 0x556647c9a350 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.849+0000 7fe19ce65740 1 --2- 172.21.15.2:0/3316640165 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556647c8b8f0 0x556647c9efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.849+0000 7fe19ce65740 1 -- 172.21.15.2:0/3316640165 >> 172.21.15.2:0/3316640165 conn(0x556647c87e00 msgr2=0x556647c8d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:37.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.849+0000 7fe19ce65740 1 -- 172.21.15.2:0/3316640165 shutdown_connections 2024-05-14T17:15:37.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.849+0000 7fe19ce65740 1 -- 172.21.15.2:0/3316640165 wait complete. 2024-05-14T17:15:37.473 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.849+0000 7fe19ce65740 20 client.17235 trim_cache size 0 max 16384 2024-05-14T17:15:37.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:15:37.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:15:37.474 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:15:37.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.850+0000 7fe19a146640 1 -- 172.21.15.2:0/740409012 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fe17c0b2a10 msgr2=0x7fe17c0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.850+0000 7fe19a146640 1 --2- 172.21.15.2:0/740409012 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fe17c0b2a10 0x7fe17c0b4ed0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe18c005fd0 tx=0x7fe18c0096a0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.850+0000 7fe19a146640 1 -- 172.21.15.2:0/740409012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe194102340 msgr2=0x7fe194199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.850+0000 7fe19a146640 1 --2- 172.21.15.2:0/740409012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe194102340 0x7fe194199740 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fe188004770 tx=0x7fe1880042f0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.850+0000 7fe19a146640 1 -- 172.21.15.2:0/740409012 shutdown_connections 2024-05-14T17:15:37.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.850+0000 7fe19a146640 1 --2- 172.21.15.2:0/740409012 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fe17c0b2a10 0x7fe17c0b4ed0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.850+0000 7fe19a146640 1 --2- 172.21.15.2:0/740409012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe194107940 0x7fe19419dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.850+0000 7fe19a146640 1 --2- 172.21.15.2:0/740409012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe194102340 0x7fe194199740 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.850+0000 7fe19a146640 1 --2- 172.21.15.2:0/740409012 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe194101980 0x7fe194199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.475 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.850+0000 7fe19a146640 1 -- 172.21.15.2:0/740409012 >> 172.21.15.2:0/740409012 conn(0x7fe194078ad0 msgr2=0x7fe194104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:37.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.850+0000 7fe19a146640 1 -- 172.21.15.2:0/740409012 shutdown_connections 2024-05-14T17:15:37.476 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:34.850+0000 7fe19a146640 1 -- 172.21.15.2:0/740409012 wait complete. 2024-05-14T17:15:37.476 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:14:34 smithi002 ceph-mon[31538]: pgmap v625: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 50 KiB/s wr, 15 op/s 2024-05-14T17:15:37.477 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:14:36 smithi002 ceph-mon[31538]: pgmap v626: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 29 KiB/s wr, 8 op/s 2024-05-14T17:15:37.477 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:14:38 smithi002 ceph-mon[31538]: pgmap v627: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 29 KiB/s wr, 7 op/s 2024-05-14T17:15:37.477 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:14:40 smithi002 ceph-mon[31538]: pgmap v628: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 41 KiB/s wr, 18 op/s 2024-05-14T17:15:37.477 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:14:42 smithi002 ceph-mon[31538]: pgmap v629: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-05-14T17:15:37.477 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:14:44 smithi002 ceph-mon[31538]: pgmap v630: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-05-14T17:15:37.477 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:14:46 smithi002 ceph-mon[31538]: pgmap v631: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-05-14T17:15:37.490 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.949+0000 7fb868859640 1 -- 172.21.15.2:0/1194703192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb864102330 msgr2=0x7fb86410e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.949+0000 7fb868859640 1 --2- 172.21.15.2:0/1194703192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb864102330 0x7fb86410e840 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fb85c009cd0 tx=0x7fb85c02a8e0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.949+0000 7fb868859640 1 -- 172.21.15.2:0/1194703192 shutdown_connections 2024-05-14T17:15:37.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.949+0000 7fb868859640 1 --2- 172.21.15.2:0/1194703192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb864102330 0x7fb86410e840 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.949+0000 7fb868859640 1 --2- 172.21.15.2:0/1194703192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb864101970 0x7fb864101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.949+0000 7fb868859640 1 --2- 172.21.15.2:0/1194703192 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb864107980 0x7fb864107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.949+0000 7fb868859640 1 -- 172.21.15.2:0/1194703192 >> 172.21.15.2:0/1194703192 conn(0x7fb864077cd0 msgr2=0x7fb8640780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:37.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.950+0000 7fb868859640 1 -- 172.21.15.2:0/1194703192 shutdown_connections 2024-05-14T17:15:37.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.950+0000 7fb868859640 1 -- 172.21.15.2:0/1194703192 wait complete. 2024-05-14T17:15:37.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.950+0000 7fb868859640 1 Processor -- start 2024-05-14T17:15:37.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.950+0000 7fb868859640 1 -- start start 2024-05-14T17:15:37.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.951+0000 7fb868859640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb864101970 0x7fb864199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.951+0000 7fb868859640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb864102330 0x7fb8641996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.491 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.951+0000 7fb868859640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb864107980 0x7fb86419da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.951+0000 7fb868859640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb864111c10 con 0x7fb864102330 2024-05-14T17:15:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.951+0000 7fb868859640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb864111a90 con 0x7fb864101970 2024-05-14T17:15:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.951+0000 7fb868859640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb864111d90 con 0x7fb864107980 2024-05-14T17:15:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.951+0000 7fb861d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb864102330 0x7fb8641996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.951+0000 7fb861d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb864102330 0x7fb8641996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50724/0 (socket says 172.21.15.2:50724) 2024-05-14T17:15:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.951+0000 7fb861d9b640 1 -- 172.21.15.2:0/3785028894 learned_addr learned my addr 172.21.15.2:0/3785028894 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.951+0000 7fb862d9d640 1 --2- 172.21.15.2:0/3785028894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb864107980 0x7fb86419da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.951+0000 7fb86259c640 1 --2- 172.21.15.2:0/3785028894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb864101970 0x7fb864199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.951+0000 7fb861d9b640 1 -- 172.21.15.2:0/3785028894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb864107980 msgr2=0x7fb86419da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.951+0000 7fb861d9b640 1 --2- 172.21.15.2:0/3785028894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb864107980 0x7fb86419da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.951+0000 7fb861d9b640 1 -- 172.21.15.2:0/3785028894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb864101970 msgr2=0x7fb864199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.951+0000 7fb861d9b640 1 --2- 172.21.15.2:0/3785028894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb864101970 0x7fb864199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.951+0000 7fb861d9b640 1 -- 172.21.15.2:0/3785028894 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb86419e150 con 0x7fb864102330 2024-05-14T17:15:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.951+0000 7fb861d9b640 1 --2- 172.21.15.2:0/3785028894 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb864102330 0x7fb8641996a0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fb85800e9f0 tx=0x7fb85800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.952+0000 7fb8537fe640 1 -- 172.21.15.2:0/3785028894 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85800cd80 con 0x7fb864102330 2024-05-14T17:15:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.952+0000 7fb8537fe640 1 -- 172.21.15.2:0/3785028894 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8580108e0 con 0x7fb864102330 2024-05-14T17:15:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.952+0000 7fb868859640 1 -- 172.21.15.2:0/3785028894 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb86419e440 con 0x7fb864102330 2024-05-14T17:15:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.952+0000 7fb8537fe640 1 -- 172.21.15.2:0/3785028894 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb858018980 con 0x7fb864102330 2024-05-14T17:15:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.952+0000 7fb868859640 1 -- 172.21.15.2:0/3785028894 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8641a5cd0 con 0x7fb864102330 2024-05-14T17:15:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.954+0000 7fb86b575740 1 Processor -- start 2024-05-14T17:15:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.954+0000 7fb8537fe640 1 -- 172.21.15.2:0/3785028894 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fb858010430 con 0x7fb864102330 2024-05-14T17:15:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.954+0000 7fb86b575740 1 -- start start 2024-05-14T17:15:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.955+0000 7fb86b575740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3af939810 0x55c3af939bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.955+0000 7fb86b575740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3af93a130 0x55c3af9489a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.955+0000 7fb86b575740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3af948ee0 0x55c3af94b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.955+0000 7fb86b575740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c3af8acc30 con 0x55c3af93a130 2024-05-14T17:15:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.955+0000 7fb86b575740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c3af89c7d0 con 0x55c3af939810 2024-05-14T17:15:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.955+0000 7fb86b575740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c3af8a0810 con 0x55c3af948ee0 2024-05-14T17:15:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.955+0000 7fb86259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3af93a130 0x55c3af9489a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.955+0000 7fb86259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3af93a130 0x55c3af9489a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50738/0 (socket says 172.21.15.2:50738) 2024-05-14T17:15:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.955+0000 7fb86259c640 1 -- 172.21.15.2:0/1620054027 learned_addr learned my addr 172.21.15.2:0/1620054027 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.955+0000 7fb862d9d640 1 --2- 172.21.15.2:0/1620054027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3af939810 0x55c3af939bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.955+0000 7fb861d9b640 1 --2- 172.21.15.2:0/1620054027 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3af948ee0 0x55c3af94b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.955+0000 7fb8537fe640 1 --2- 172.21.15.2:0/3785028894 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb84c0b2a10 0x7fb84c0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.955+0000 7fb86259c640 1 -- 172.21.15.2:0/1620054027 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3af948ee0 msgr2=0x55c3af94b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.955+0000 7fb86259c640 1 --2- 172.21.15.2:0/1620054027 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3af948ee0 0x55c3af94b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.955+0000 7fb86259c640 1 -- 172.21.15.2:0/1620054027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3af939810 msgr2=0x55c3af939bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.955+0000 7fb86259c640 1 --2- 172.21.15.2:0/1620054027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3af939810 0x55c3af939bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.955+0000 7fb86259c640 1 -- 172.21.15.2:0/1620054027 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3af5b1210 con 0x55c3af93a130 2024-05-14T17:15:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.955+0000 7fb862d9d640 1 --2- 172.21.15.2:0/3785028894 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb84c0b2a10 0x7fb84c0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.956+0000 7fb8537fe640 1 -- 172.21.15.2:0/3785028894 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8513+0+0 (secure 0 0 0) 0x7fb8580e2a00 con 0x7fb864102330 2024-05-14T17:15:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.956+0000 7fb86259c640 1 --2- 172.21.15.2:0/1620054027 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3af93a130 0x55c3af9489a0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fb85400dc30 tx=0x7fb85400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0df73c958da054e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.956+0000 7fb850ff9640 1 -- 172.21.15.2:0/1620054027 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb854011440 con 0x55c3af93a130 2024-05-14T17:15:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.956+0000 7fb850ff9640 1 -- 172.21.15.2:0/1620054027 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb854017070 con 0x55c3af93a130 2024-05-14T17:15:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.956+0000 7fb850ff9640 1 -- 172.21.15.2:0/1620054027 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb854010860 con 0x55c3af93a130 2024-05-14T17:15:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.957+0000 7fb86b575740 1 -- 172.21.15.2:0/1620054027 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3af93a130 msgr2=0x55c3af9489a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.957+0000 7fb862d9d640 1 --2- 172.21.15.2:0/3785028894 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb84c0b2a10 0x7fb84c0b4ed0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb85c005e10 tx=0x7fb85c0095c0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.957+0000 7fb86b575740 1 --2- 172.21.15.2:0/1620054027 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3af93a130 0x55c3af9489a0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fb85400dc30 tx=0x7fb85400f730 comp rx=0 tx=0).stop 2024-05-14T17:15:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.957+0000 7fb86b575740 1 -- 172.21.15.2:0/1620054027 shutdown_connections 2024-05-14T17:15:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.957+0000 7fb86b575740 1 --2- 172.21.15.2:0/1620054027 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3af948ee0 0x55c3af94b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.957+0000 7fb86b575740 1 --2- 172.21.15.2:0/1620054027 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3af93a130 0x55c3af9489a0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.957+0000 7fb86b575740 1 --2- 172.21.15.2:0/1620054027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3af939810 0x55c3af939bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.957+0000 7fb86b575740 1 -- 172.21.15.2:0/1620054027 >> 172.21.15.2:0/1620054027 conn(0x55c3af935e00 msgr2=0x55c3af9361e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.957+0000 7fb86b575740 1 -- 172.21.15.2:0/1620054027 shutdown_connections 2024-05-14T17:15:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.957+0000 7fb86b575740 1 -- 172.21.15.2:0/1620054027 wait complete. 2024-05-14T17:15:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.957+0000 7fb86b575740 1 -- start start 2024-05-14T17:15:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.957+0000 7fb86b575740 1 Processor -- start 2024-05-14T17:15:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.958+0000 7fb86b575740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3af939810 0x55c3af948830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.958+0000 7fb86b575740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3af93a130 0x55c3af945b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.958+0000 7fb86b575740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3af948ee0 0x55c3af946080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.958+0000 7fb86b575740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55c3af582790 con 0x55c3af93a130 2024-05-14T17:15:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.958+0000 7fb86b575740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c3af858890 con 0x55c3af939810 2024-05-14T17:15:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.958+0000 7fb86b575740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c3af94bbb0 con 0x55c3af948ee0 2024-05-14T17:15:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.958+0000 7fb86b575740 10 client.0 _ll_register_callbacks cb 0x55c3af85bd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:15:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.958+0000 7fb861d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3af93a130 0x55c3af945b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.958+0000 7fb861d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3af93a130 0x55c3af945b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50744/0 (socket says 172.21.15.2:50744) 2024-05-14T17:15:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.958+0000 7fb861d9b640 1 -- 172.21.15.2:0/1145651210 learned_addr learned my addr 172.21.15.2:0/1145651210 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.958+0000 7fb862d9d640 1 --2- 172.21.15.2:0/1145651210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3af948ee0 0x55c3af946080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.958+0000 7fb86259c640 1 --2- 172.21.15.2:0/1145651210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3af939810 0x55c3af948830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.958+0000 7fb861d9b640 1 -- 172.21.15.2:0/1145651210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3af948ee0 msgr2=0x55c3af946080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.958+0000 7fb861d9b640 1 --2- 172.21.15.2:0/1145651210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3af948ee0 0x55c3af946080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.958+0000 7fb861d9b640 1 -- 172.21.15.2:0/1145651210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3af939810 msgr2=0x55c3af948830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.958+0000 7fb861d9b640 1 --2- 172.21.15.2:0/1145651210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3af939810 0x55c3af948830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.958+0000 7fb861d9b640 1 -- 172.21.15.2:0/1145651210 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3af67d620 con 0x55c3af93a130 2024-05-14T17:15:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.958+0000 7fb86259c640 1 --2- 172.21.15.2:0/1145651210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3af939810 0x55c3af948830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:15:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.958+0000 7fb861d9b640 1 --2- 172.21.15.2:0/1145651210 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3af93a130 0x55c3af945b20 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fb85800c6a0 tx=0x7fb858000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50c3eeb11bc8eb9b server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.958+0000 7fb849ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:15:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.958+0000 7fb849ffb640 1 -- 172.21.15.2:0/1145651210 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8580ec040 con 0x55c3af93a130 2024-05-14T17:15:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.958+0000 7fb849ffb640 1 -- 172.21.15.2:0/1145651210 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8513+0+0 (secure 0 0 0) 0x7fb8580a8f40 con 0x55c3af93a130 2024-05-14T17:15:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.958+0000 7fb86b575740 1 -- 172.21.15.2:0/1145651210 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb854017070 con 0x55c3af93a130 2024-05-14T17:15:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.959+0000 7fb849ffb640 1 -- 172.21.15.2:0/1145651210 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fb8580a9450 con 0x55c3af93a130 2024-05-14T17:15:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.959+0000 7fb86b575740 10 client.17247 fetch_fsmap learned FSMap version 57 2024-05-14T17:15:37.497 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.959+0000 7fb86b575740 1 -- 172.21.15.2:0/1145651210 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55c3af957670 con 0x55c3af93a130 2024-05-14T17:15:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.959+0000 7fb849ffb640 1 -- 172.21.15.2:0/1145651210 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb854017070 con 0x55c3af93a130 2024-05-14T17:15:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.959+0000 7fb849ffb640 1 -- 172.21.15.2:0/1145651210 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 57) ==== 8350+0+0 (secure 0 0 0) 0x7fb8580a5650 con 0x55c3af93a130 2024-05-14T17:15:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.959+0000 7fb86b575740 10 client.17247 fetch_fsmap finished waiting for FSMap version 57 2024-05-14T17:15:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.959+0000 7fb86b575740 10 client.17247 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:15:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.959+0000 7fb86b575740 20 client.17247 populate_metadata read hostname 'smithi002' 2024-05-14T17:15:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.959+0000 7fb86b575740 1 --2- 172.21.15.2:0/1145651210 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55c3af958b80 0x55c3af95af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.959+0000 7fb86b575740 4 client.17247 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:15:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.959+0000 7fb86b575740 1 -- 172.21.15.2:0/1145651210 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3af8aeb80 con 0x55c3af958b80 2024-05-14T17:15:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.959+0000 7fb86259c640 1 --2- 172.21.15.2:0/1145651210 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55c3af958b80 0x55c3af95af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.960+0000 7fb86259c640 1 --2- 172.21.15.2:0/1145651210 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55c3af958b80 0x55c3af95af70 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20505453725f7d6f server_cookie=9004007bad6a38d in_seq=0 out_seq=0 2024-05-14T17:15:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.960+0000 7fb849ffb640 10 client.17247 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:15:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.962+0000 7fb849ffb640 1 -- 172.21.15.2:0/1145651210 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c3af8aeb80 con 0x55c3af958b80 2024-05-14T17:15:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.962+0000 7fb849ffb640 10 client.17247 handle_command_reply: tid=0 2024-05-14T17:15:37.498 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.962+0000 7fb86b575740 1 client.17247 shutdown 2024-05-14T17:15:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.962+0000 7fb86b575740 1 -- 172.21.15.2:0/1145651210 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3af93a130 msgr2=0x55c3af945b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.962+0000 7fb86b575740 1 --2- 172.21.15.2:0/1145651210 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3af93a130 0x55c3af945b20 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fb85800c6a0 tx=0x7fb858000bc0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.962+0000 7fb86b575740 1 -- 172.21.15.2:0/1145651210 shutdown_connections 2024-05-14T17:15:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.962+0000 7fb86b575740 1 --2- 172.21.15.2:0/1145651210 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55c3af958b80 0x55c3af95af70 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.962+0000 7fb86b575740 1 --2- 172.21.15.2:0/1145651210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3af948ee0 0x55c3af946080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.962+0000 7fb86b575740 1 --2- 172.21.15.2:0/1145651210 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3af93a130 0x55c3af945b20 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.962+0000 7fb849ffb640 0 client.17247 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:15:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.962+0000 7fb86b575740 1 --2- 172.21.15.2:0/1145651210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3af939810 0x55c3af948830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.962+0000 7fb86b575740 1 -- 172.21.15.2:0/1145651210 >> 172.21.15.2:0/1145651210 conn(0x55c3af935e00 msgr2=0x55c3af93bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.962+0000 7fb86b575740 1 -- 172.21.15.2:0/1145651210 shutdown_connections 2024-05-14T17:15:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.962+0000 7fb86b575740 1 -- 172.21.15.2:0/1145651210 wait complete. 2024-05-14T17:15:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.962+0000 7fb86b575740 20 client.17247 trim_cache size 0 max 16384 2024-05-14T17:15:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.976+0000 7fb86b575740 1 Processor -- start 2024-05-14T17:15:37.499 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.976+0000 7fb86b575740 1 -- start start 2024-05-14T17:15:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.977+0000 7fb86b575740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3af939810 0x55c3af954a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.977+0000 7fb86b575740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3af93a130 0x55c3af948360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.977+0000 7fb86b575740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3af948ee0 0x55c3af960880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.977+0000 7fb86b575740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb8641aedb0 con 0x55c3af93a130 2024-05-14T17:15:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.977+0000 7fb86b575740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb8641ac620 con 0x55c3af939810 2024-05-14T17:15:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.977+0000 7fb86b575740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb8641aa490 con 0x55c3af948ee0 2024-05-14T17:15:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.977+0000 7fb861d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3af93a130 0x55c3af948360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.977+0000 7fb861d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3af93a130 0x55c3af948360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50752/0 (socket says 172.21.15.2:50752) 2024-05-14T17:15:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.977+0000 7fb861d9b640 1 -- 172.21.15.2:0/4279946757 learned_addr learned my addr 172.21.15.2:0/4279946757 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.977+0000 7fb862d9d640 1 --2- 172.21.15.2:0/4279946757 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3af948ee0 0x55c3af960880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.977+0000 7fb86259c640 1 --2- 172.21.15.2:0/4279946757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3af939810 0x55c3af954a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.977+0000 7fb861d9b640 1 -- 172.21.15.2:0/4279946757 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3af948ee0 msgr2=0x55c3af960880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.977+0000 7fb861d9b640 1 --2- 172.21.15.2:0/4279946757 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3af948ee0 0x55c3af960880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.977+0000 7fb861d9b640 1 -- 172.21.15.2:0/4279946757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3af939810 msgr2=0x55c3af954a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.500 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.977+0000 7fb861d9b640 1 --2- 172.21.15.2:0/4279946757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3af939810 0x55c3af954a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.977+0000 7fb861d9b640 1 -- 172.21.15.2:0/4279946757 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3af939f60 con 0x55c3af93a130 2024-05-14T17:15:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.977+0000 7fb862d9d640 1 --2- 172.21.15.2:0/4279946757 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3af948ee0 0x55c3af960880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:15:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.977+0000 7fb861d9b640 1 --2- 172.21.15.2:0/4279946757 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3af93a130 0x55c3af948360 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fb8580a5d60 tx=0x7fb8580a96f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4be49ca7ec7b0ce server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.977+0000 7fb84bfff640 1 -- 172.21.15.2:0/4279946757 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8580ae0e0 con 0x55c3af93a130 2024-05-14T17:15:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.977+0000 7fb84bfff640 1 -- 172.21.15.2:0/4279946757 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8580c1050 con 0x55c3af93a130 2024-05-14T17:15:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.977+0000 7fb84bfff640 1 -- 172.21.15.2:0/4279946757 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8580b2b90 con 0x55c3af93a130 2024-05-14T17:15:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.978+0000 7fb86b575740 1 -- 172.21.15.2:0/4279946757 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3af93a130 msgr2=0x55c3af948360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.978+0000 7fb86b575740 1 --2- 172.21.15.2:0/4279946757 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3af93a130 0x55c3af948360 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fb8580a5d60 tx=0x7fb8580a96f0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.978+0000 7fb86b575740 1 -- 172.21.15.2:0/4279946757 shutdown_connections 2024-05-14T17:15:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.978+0000 7fb86b575740 1 --2- 172.21.15.2:0/4279946757 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3af948ee0 0x55c3af960880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.978+0000 7fb86b575740 1 --2- 172.21.15.2:0/4279946757 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3af93a130 0x55c3af948360 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.978+0000 7fb86b575740 1 --2- 172.21.15.2:0/4279946757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3af939810 0x55c3af954a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.978+0000 7fb86b575740 1 -- 172.21.15.2:0/4279946757 >> 172.21.15.2:0/4279946757 conn(0x55c3af935e00 msgr2=0x55c3af9411d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:37.501 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.978+0000 7fb86b575740 1 -- 172.21.15.2:0/4279946757 shutdown_connections 2024-05-14T17:15:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.978+0000 7fb86b575740 1 -- 172.21.15.2:0/4279946757 wait complete. 2024-05-14T17:15:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.978+0000 7fb86b575740 1 -- start start 2024-05-14T17:15:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.978+0000 7fb86b575740 1 Processor -- start 2024-05-14T17:15:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.979+0000 7fb86b575740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3af939810 0x55c3af950e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.979+0000 7fb86b575740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3af93a130 0x55c3af951340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.979+0000 7fb86b575740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3af948ee0 0x55c3af94ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.979+0000 7fb86b575740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb8641aaae0 con 0x55c3af948ee0 2024-05-14T17:15:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.979+0000 7fb86b575740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb8641abf50 con 0x55c3af939810 2024-05-14T17:15:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.979+0000 7fb86b575740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb8641ab1b0 con 0x55c3af93a130 2024-05-14T17:15:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.979+0000 7fb86b575740 10 client.0 _ll_register_callbacks cb 0x7fb8580a5ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:15:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.979+0000 7fb862d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3af948ee0 0x55c3af94ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.979+0000 7fb862d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3af948ee0 0x55c3af94ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50764/0 (socket says 172.21.15.2:50764) 2024-05-14T17:15:37.502 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.979+0000 7fb862d9d640 1 -- 172.21.15.2:0/2648510309 learned_addr learned my addr 172.21.15.2:0/2648510309 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.979+0000 7fb861d9b640 1 --2- 172.21.15.2:0/2648510309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3af93a130 0x55c3af951340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.979+0000 7fb86259c640 1 --2- 172.21.15.2:0/2648510309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3af939810 0x55c3af950e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.979+0000 7fb862d9d640 1 -- 172.21.15.2:0/2648510309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3af93a130 msgr2=0x55c3af951340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.979+0000 7fb862d9d640 1 --2- 172.21.15.2:0/2648510309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3af93a130 0x55c3af951340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.979+0000 7fb862d9d640 1 -- 172.21.15.2:0/2648510309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3af939810 msgr2=0x55c3af950e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.979+0000 7fb862d9d640 1 --2- 172.21.15.2:0/2648510309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3af939810 0x55c3af950e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.979+0000 7fb862d9d640 1 -- 172.21.15.2:0/2648510309 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3af94e310 con 0x55c3af948ee0 2024-05-14T17:15:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.979+0000 7fb862d9d640 1 --2- 172.21.15.2:0/2648510309 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3af948ee0 0x55c3af94ddd0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fb85c009cd0 tx=0x7fb85c02a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fbf398667b8d35f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.979+0000 7fb849ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:15:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.979+0000 7fb849ffb640 1 -- 172.21.15.2:0/2648510309 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c0305a0 con 0x55c3af948ee0 2024-05-14T17:15:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.980+0000 7fb849ffb640 1 -- 172.21.15.2:0/2648510309 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8513+0+0 (secure 0 0 0) 0x7fb85c030740 con 0x55c3af948ee0 2024-05-14T17:15:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.980+0000 7fb86b575740 1 -- 172.21.15.2:0/2648510309 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8580c1050 con 0x55c3af948ee0 2024-05-14T17:15:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.980+0000 7fb849ffb640 1 -- 172.21.15.2:0/2648510309 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fb85c030940 con 0x55c3af948ee0 2024-05-14T17:15:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.980+0000 7fb86b575740 10 client.17253 fetch_fsmap learned FSMap version 57 2024-05-14T17:15:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.980+0000 7fb86b575740 1 -- 172.21.15.2:0/2648510309 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55c3af9c3b60 con 0x55c3af948ee0 2024-05-14T17:15:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.980+0000 7fb849ffb640 1 -- 172.21.15.2:0/2648510309 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8580c1050 con 0x55c3af948ee0 2024-05-14T17:15:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.980+0000 7fb849ffb640 1 -- 172.21.15.2:0/2648510309 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 57) ==== 8350+0+0 (secure 0 0 0) 0x7fb85c045070 con 0x55c3af948ee0 2024-05-14T17:15:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.980+0000 7fb86b575740 10 client.17253 fetch_fsmap finished waiting for FSMap version 57 2024-05-14T17:15:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.980+0000 7fb86b575740 10 client.17253 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:15:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.980+0000 7fb86b575740 20 client.17253 populate_metadata read hostname 'smithi002' 2024-05-14T17:15:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.980+0000 7fb86b575740 1 --2- 172.21.15.2:0/2648510309 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55c3af9c4300 0x55c3af9c6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.980+0000 7fb86b575740 4 client.17253 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:15:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.980+0000 7fb86b575740 1 -- 172.21.15.2:0/2648510309 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c3af8a8360 con 0x55c3af9c4300 2024-05-14T17:15:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.980+0000 7fb86259c640 1 --2- 172.21.15.2:0/2648510309 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55c3af9c4300 0x55c3af9c6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.981+0000 7fb86259c640 1 --2- 172.21.15.2:0/2648510309 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55c3af9c4300 0x55c3af9c6720 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd3688e899d59da1 server_cookie=58c0c14e4aa46dd4 in_seq=0 out_seq=0 2024-05-14T17:15:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.981+0000 7fb849ffb640 10 client.17253 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:15:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.991+0000 7fb849ffb640 1 -- 172.21.15.2:0/2648510309 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c3af8a8360 con 0x55c3af9c4300 2024-05-14T17:15:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.991+0000 7fb849ffb640 10 client.17253 handle_command_reply: tid=0 2024-05-14T17:15:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.991+0000 7fb86b575740 1 client.17253 shutdown 2024-05-14T17:15:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.991+0000 7fb86b575740 1 -- 172.21.15.2:0/2648510309 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3af948ee0 msgr2=0x55c3af94ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.991+0000 7fb86b575740 1 --2- 172.21.15.2:0/2648510309 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3af948ee0 0x55c3af94ddd0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fb85c009cd0 tx=0x7fb85c02a8e0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.991+0000 7fb86b575740 1 -- 172.21.15.2:0/2648510309 shutdown_connections 2024-05-14T17:15:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.991+0000 7fb86b575740 1 --2- 172.21.15.2:0/2648510309 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55c3af9c4300 0x55c3af9c6720 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.991+0000 7fb86b575740 1 --2- 172.21.15.2:0/2648510309 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3af948ee0 0x55c3af94ddd0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.991+0000 7fb86b575740 1 --2- 172.21.15.2:0/2648510309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c3af93a130 0x55c3af951340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.991+0000 7fb849ffb640 0 client.17253 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:15:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.991+0000 7fb86b575740 1 --2- 172.21.15.2:0/2648510309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c3af939810 0x55c3af950e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.991+0000 7fb86b575740 1 -- 172.21.15.2:0/2648510309 >> 172.21.15.2:0/2648510309 conn(0x55c3af935e00 msgr2=0x55c3af93b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.991+0000 7fb86b575740 1 -- 172.21.15.2:0/2648510309 shutdown_connections 2024-05-14T17:15:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.991+0000 7fb86b575740 1 -- 172.21.15.2:0/2648510309 wait complete. 2024-05-14T17:15:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.991+0000 7fb86b575740 20 client.17253 trim_cache size 0 max 16384 2024-05-14T17:15:37.505 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:15:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:15:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:15:37.506 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.991+0000 7fb868859640 1 -- 172.21.15.2:0/3785028894 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb84c0b2a10 msgr2=0x7fb84c0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.991+0000 7fb868859640 1 --2- 172.21.15.2:0/3785028894 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb84c0b2a10 0x7fb84c0b4ed0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb85c005e10 tx=0x7fb85c0095c0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.991+0000 7fb868859640 1 -- 172.21.15.2:0/3785028894 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb864102330 msgr2=0x7fb8641996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.991+0000 7fb868859640 1 --2- 172.21.15.2:0/3785028894 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb864102330 0x7fb8641996a0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fb85800e9f0 tx=0x7fb85800eeb0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.991+0000 7fb868859640 1 -- 172.21.15.2:0/3785028894 shutdown_connections 2024-05-14T17:15:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.991+0000 7fb868859640 1 --2- 172.21.15.2:0/3785028894 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb84c0b2a10 0x7fb84c0b4ed0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.991+0000 7fb868859640 1 --2- 172.21.15.2:0/3785028894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb864107980 0x7fb86419da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.991+0000 7fb868859640 1 --2- 172.21.15.2:0/3785028894 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb864102330 0x7fb8641996a0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.991+0000 7fb868859640 1 --2- 172.21.15.2:0/3785028894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb864101970 0x7fb864199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.991+0000 7fb868859640 1 -- 172.21.15.2:0/3785028894 >> 172.21.15.2:0/3785028894 conn(0x7fb864077cd0 msgr2=0x7fb86406b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.992+0000 7fb868859640 1 -- 172.21.15.2:0/3785028894 shutdown_connections 2024-05-14T17:15:37.507 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:14:47.992+0000 7fb868859640 1 -- 172.21.15.2:0/3785028894 wait complete. 2024-05-14T17:15:37.508 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:14:48 smithi002 ceph-mon[31538]: pgmap v632: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-05-14T17:15:37.508 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:14:50 smithi002 ceph-mon[31538]: pgmap v633: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 17 op/s 2024-05-14T17:15:37.508 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:14:52 smithi002 ceph-mon[31538]: pgmap v634: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.6 KiB/s wr, 7 op/s 2024-05-14T17:15:37.508 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:14:54 smithi002 ceph-mon[31538]: pgmap v635: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-14T17:15:37.508 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:14:56 smithi002 ceph-mon[31538]: pgmap v636: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 7.8 KiB/s wr, 8 op/s 2024-05-14T17:15:37.508 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:14:58 smithi002 ceph-mon[31538]: pgmap v637: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.8 KiB/s wr, 7 op/s 2024-05-14T17:15:37.508 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:00 smithi002 ceph-mon[31538]: pgmap v638: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.0 KiB/s wr, 10 op/s 2024-05-14T17:15:37.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.093+0000 7f95a8cf1640 1 -- 172.21.15.2:0/4062040512 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95a4102a60 msgr2=0x7f95a410e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.093+0000 7f95a8cf1640 1 --2- 172.21.15.2:0/4062040512 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95a4102a60 0x7f95a410e810 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f959c009ca0 tx=0x7f959c02a8c0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.093+0000 7f95a8cf1640 1 -- 172.21.15.2:0/4062040512 shutdown_connections 2024-05-14T17:15:37.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.093+0000 7f95a8cf1640 1 --2- 172.21.15.2:0/4062040512 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95a4102a60 0x7f95a410e810 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.521 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.093+0000 7f95a8cf1640 1 --2- 172.21.15.2:0/4062040512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95a41020a0 0x7f95a4102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.093+0000 7f95a8cf1640 1 --2- 172.21.15.2:0/4062040512 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95a4106980 0x7f95a4106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.093+0000 7f95a8cf1640 1 -- 172.21.15.2:0/4062040512 >> 172.21.15.2:0/4062040512 conn(0x7f95a4077e70 msgr2=0x7f95a40ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:37.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.093+0000 7f95a8cf1640 1 -- 172.21.15.2:0/4062040512 shutdown_connections 2024-05-14T17:15:37.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.093+0000 7f95a8cf1640 1 -- 172.21.15.2:0/4062040512 wait complete. 2024-05-14T17:15:37.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.094+0000 7f95a8cf1640 1 Processor -- start 2024-05-14T17:15:37.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.094+0000 7f95a8cf1640 1 -- start start 2024-05-14T17:15:37.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.094+0000 7f95a8cf1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95a41020a0 0x7f95a4198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.094+0000 7f95a8cf1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95a4102a60 0x7f95a4198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.094+0000 7f95a8cf1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95a4106980 0x7f95a419c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.094+0000 7f95a8cf1640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f95a4111be0 con 0x7f95a41020a0 2024-05-14T17:15:37.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.094+0000 7f95a8cf1640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f95a4111a60 con 0x7f95a4106980 2024-05-14T17:15:37.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.094+0000 7f95a8cf1640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f95a4111d60 con 0x7f95a4102a60 2024-05-14T17:15:37.522 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.095+0000 7f95a259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95a41020a0 0x7f95a4198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.095+0000 7f95a259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95a41020a0 0x7f95a4198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45458/0 (socket says 172.21.15.2:45458) 2024-05-14T17:15:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.095+0000 7f95a259c640 1 -- 172.21.15.2:0/507346684 learned_addr learned my addr 172.21.15.2:0/507346684 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.095+0000 7f95a1d9b640 1 --2- 172.21.15.2:0/507346684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95a4102a60 0x7f95a4198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.095+0000 7f95a2d9d640 1 --2- 172.21.15.2:0/507346684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95a4106980 0x7f95a419c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.095+0000 7f95a259c640 1 -- 172.21.15.2:0/507346684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95a4102a60 msgr2=0x7f95a4198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.095+0000 7f95a259c640 1 --2- 172.21.15.2:0/507346684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95a4102a60 0x7f95a4198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.095+0000 7f95a259c640 1 -- 172.21.15.2:0/507346684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95a4106980 msgr2=0x7f95a419c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.095+0000 7f95a259c640 1 --2- 172.21.15.2:0/507346684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95a4106980 0x7f95a419c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.095+0000 7f95a259c640 1 -- 172.21.15.2:0/507346684 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a419cfa0 con 0x7f95a41020a0 2024-05-14T17:15:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.095+0000 7f95a1d9b640 1 --2- 172.21.15.2:0/507346684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95a4102a60 0x7f95a4198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:15:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.095+0000 7f95a259c640 1 --2- 172.21.15.2:0/507346684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95a41020a0 0x7f95a4198000 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f959400d770 tx=0x7f959400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.095+0000 7f95937fe640 1 -- 172.21.15.2:0/507346684 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594017070 con 0x7f95a41020a0 2024-05-14T17:15:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.095+0000 7f95937fe640 1 -- 172.21.15.2:0/507346684 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95940054d0 con 0x7f95a41020a0 2024-05-14T17:15:37.523 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.095+0000 7f95a8cf1640 1 -- 172.21.15.2:0/507346684 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95a419d290 con 0x7f95a41020a0 2024-05-14T17:15:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.095+0000 7f95937fe640 1 -- 172.21.15.2:0/507346684 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594004d00 con 0x7f95a41020a0 2024-05-14T17:15:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.095+0000 7f95a8cf1640 1 -- 172.21.15.2:0/507346684 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95a41a4b20 con 0x7f95a41020a0 2024-05-14T17:15:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.099+0000 7f95aba0c740 1 Processor -- start 2024-05-14T17:15:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.099+0000 7f95aba0c740 1 -- start start 2024-05-14T17:15:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.099+0000 7f95937fe640 1 -- 172.21.15.2:0/507346684 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f9594004ea0 con 0x7f95a41020a0 2024-05-14T17:15:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.099+0000 7f95aba0c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560aaa2537d0 0x560aaa253bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.099+0000 7f95aba0c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560aaa2540f0 0x560aaa2607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.099+0000 7f95aba0c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560aaa260ce0 0x560aaa2630d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.099+0000 7f95aba0c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x560aaa1c6c30 con 0x560aaa2540f0 2024-05-14T17:15:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.099+0000 7f95aba0c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560aaa1b67d0 con 0x560aaa260ce0 2024-05-14T17:15:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.099+0000 7f95aba0c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560aaa1ba810 con 0x560aaa2537d0 2024-05-14T17:15:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.099+0000 7f95a259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560aaa2540f0 0x560aaa2607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.524 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.099+0000 7f95a259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560aaa2540f0 0x560aaa2607a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45474/0 (socket says 172.21.15.2:45474) 2024-05-14T17:15:37.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.099+0000 7f95a259c640 1 -- 172.21.15.2:0/3174211753 learned_addr learned my addr 172.21.15.2:0/3174211753 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:37.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.100+0000 7f95a2d9d640 1 --2- 172.21.15.2:0/3174211753 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560aaa2537d0 0x560aaa253bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.100+0000 7f95a1d9b640 1 --2- 172.21.15.2:0/3174211753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560aaa260ce0 0x560aaa2630d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.100+0000 7f95a259c640 1 -- 172.21.15.2:0/3174211753 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560aaa2537d0 msgr2=0x560aaa253bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.100+0000 7f95a259c640 1 --2- 172.21.15.2:0/3174211753 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560aaa2537d0 0x560aaa253bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.100+0000 7f95a259c640 1 -- 172.21.15.2:0/3174211753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560aaa260ce0 msgr2=0x560aaa2630d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.100+0000 7f95a259c640 1 --2- 172.21.15.2:0/3174211753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560aaa260ce0 0x560aaa2630d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.100+0000 7f95a259c640 1 -- 172.21.15.2:0/3174211753 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560aa9ecb210 con 0x560aaa2540f0 2024-05-14T17:15:37.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.100+0000 7f95a259c640 1 --2- 172.21.15.2:0/3174211753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560aaa2540f0 0x560aaa2607a0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f95940df710 tx=0x7f95940ea780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=391a03eb51876272 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:37.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.100+0000 7f95937fe640 1 --2- 172.21.15.2:0/507346684 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f958c0b2a10 0x7f958c0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.100+0000 7f9590ff9640 1 -- 172.21.15.2:0/3174211753 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95940f5070 con 0x560aaa2540f0 2024-05-14T17:15:37.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.100+0000 7f9590ff9640 1 -- 172.21.15.2:0/3174211753 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95940a8e70 con 0x560aaa2540f0 2024-05-14T17:15:37.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.100+0000 7f9590ff9640 1 -- 172.21.15.2:0/3174211753 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95940cbf80 con 0x560aaa2540f0 2024-05-14T17:15:37.525 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.100+0000 7f95a2d9d640 1 --2- 172.21.15.2:0/507346684 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f958c0b2a10 0x7f958c0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.101+0000 7f95937fe640 1 -- 172.21.15.2:0/507346684 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8513+0+0 (secure 0 0 0) 0x7f95940e2080 con 0x7f95a41020a0 2024-05-14T17:15:37.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.101+0000 7f95aba0c740 1 -- 172.21.15.2:0/3174211753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560aaa2540f0 msgr2=0x560aaa2607a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.101+0000 7f95a2d9d640 1 --2- 172.21.15.2:0/507346684 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f958c0b2a10 0x7f958c0b4ed0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f959c006010 tx=0x7f959c0093f0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:37.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.101+0000 7f95aba0c740 1 --2- 172.21.15.2:0/3174211753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560aaa2540f0 0x560aaa2607a0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f95940df710 tx=0x7f95940ea780 comp rx=0 tx=0).stop 2024-05-14T17:15:37.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.101+0000 7f95aba0c740 1 -- 172.21.15.2:0/3174211753 shutdown_connections 2024-05-14T17:15:37.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.101+0000 7f95aba0c740 1 --2- 172.21.15.2:0/3174211753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560aaa260ce0 0x560aaa2630d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.101+0000 7f95aba0c740 1 --2- 172.21.15.2:0/3174211753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560aaa2540f0 0x560aaa2607a0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.101+0000 7f95aba0c740 1 --2- 172.21.15.2:0/3174211753 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560aaa2537d0 0x560aaa253bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.101+0000 7f95aba0c740 1 -- 172.21.15.2:0/3174211753 >> 172.21.15.2:0/3174211753 conn(0x560aaa24fe00 msgr2=0x560aaa2501e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:37.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.101+0000 7f95aba0c740 1 -- 172.21.15.2:0/3174211753 shutdown_connections 2024-05-14T17:15:37.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.101+0000 7f95aba0c740 1 -- 172.21.15.2:0/3174211753 wait complete. 2024-05-14T17:15:37.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.102+0000 7f95aba0c740 1 -- start start 2024-05-14T17:15:37.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.102+0000 7f95aba0c740 1 Processor -- start 2024-05-14T17:15:37.526 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.102+0000 7f95aba0c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560aaa2537d0 0x560aaa260530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.102+0000 7f95aba0c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560aaa2540f0 0x560aaa25d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.102+0000 7f95aba0c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560aaa260ce0 0x560aaa25dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.102+0000 7f95aba0c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x560aa9e9c790 con 0x560aaa260ce0 2024-05-14T17:15:37.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.102+0000 7f95aba0c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560aaa172890 con 0x560aaa2540f0 2024-05-14T17:15:37.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.102+0000 7f95aba0c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560aaa2639b0 con 0x560aaa2537d0 2024-05-14T17:15:37.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.102+0000 7f95aba0c740 10 client.0 _ll_register_callbacks cb 0x560aaa175d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:15:37.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.102+0000 7f95a2d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560aaa260ce0 0x560aaa25dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.102+0000 7f95a2d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560aaa260ce0 0x560aaa25dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45476/0 (socket says 172.21.15.2:45476) 2024-05-14T17:15:37.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.102+0000 7f95a2d9d640 1 -- 172.21.15.2:0/3784621501 learned_addr learned my addr 172.21.15.2:0/3784621501 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:37.527 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.102+0000 7f95a1d9b640 1 --2- 172.21.15.2:0/3784621501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560aaa2540f0 0x560aaa25d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.102+0000 7f95a259c640 1 --2- 172.21.15.2:0/3784621501 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560aaa2537d0 0x560aaa260530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.103+0000 7f95a2d9d640 1 -- 172.21.15.2:0/3784621501 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560aaa2537d0 msgr2=0x560aaa260530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.103+0000 7f95a2d9d640 1 --2- 172.21.15.2:0/3784621501 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560aaa2537d0 0x560aaa260530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.103+0000 7f95a2d9d640 1 -- 172.21.15.2:0/3784621501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560aaa2540f0 msgr2=0x560aaa25d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.103+0000 7f95a2d9d640 1 --2- 172.21.15.2:0/3784621501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560aaa2540f0 0x560aaa25d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.103+0000 7f95a2d9d640 1 -- 172.21.15.2:0/3784621501 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560aa9f97620 con 0x560aaa260ce0 2024-05-14T17:15:37.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.103+0000 7f95a259c640 1 --2- 172.21.15.2:0/3784621501 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560aaa2537d0 0x560aaa260530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:15:37.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.103+0000 7f95a2d9d640 1 --2- 172.21.15.2:0/3784621501 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560aaa260ce0 0x560aaa25dd60 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f959c0316e0 tx=0x7f959c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c210d8c837d0595d server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:37.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.103+0000 7f9589ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:15:37.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.103+0000 7f9589ffb640 1 -- 172.21.15.2:0/3784621501 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c04e070 con 0x560aaa260ce0 2024-05-14T17:15:37.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.103+0000 7f9589ffb640 1 -- 172.21.15.2:0/3784621501 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8513+0+0 (secure 0 0 0) 0x7f959c0045c0 con 0x560aaa260ce0 2024-05-14T17:15:37.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.103+0000 7f95aba0c740 1 -- 172.21.15.2:0/3784621501 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95940a8e70 con 0x560aaa260ce0 2024-05-14T17:15:37.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.103+0000 7f9589ffb640 1 -- 172.21.15.2:0/3784621501 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f959c0047c0 con 0x560aaa260ce0 2024-05-14T17:15:37.528 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.103+0000 7f95aba0c740 10 client.17265 fetch_fsmap learned FSMap version 57 2024-05-14T17:15:37.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.103+0000 7f95aba0c740 1 -- 172.21.15.2:0/3784621501 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x560aaa26f3a0 con 0x560aaa260ce0 2024-05-14T17:15:37.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.103+0000 7f9589ffb640 1 -- 172.21.15.2:0/3784621501 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95940a8e70 con 0x560aaa260ce0 2024-05-14T17:15:37.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.104+0000 7f9589ffb640 1 -- 172.21.15.2:0/3784621501 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 57) ==== 8350+0+0 (secure 0 0 0) 0x7f959c043040 con 0x560aaa260ce0 2024-05-14T17:15:37.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.104+0000 7f95aba0c740 10 client.17265 fetch_fsmap finished waiting for FSMap version 57 2024-05-14T17:15:37.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.104+0000 7f95aba0c740 10 client.17265 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:15:37.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.104+0000 7f95aba0c740 20 client.17265 populate_metadata read hostname 'smithi002' 2024-05-14T17:15:37.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.104+0000 7f95aba0c740 1 --2- 172.21.15.2:0/3784621501 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x560aaa270980 0x560aaa272e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.104+0000 7f95aba0c740 4 client.17265 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:15:37.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.104+0000 7f95aba0c740 1 -- 172.21.15.2:0/3784621501 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560aaa1c8b80 con 0x560aaa270980 2024-05-14T17:15:37.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.104+0000 7f95a259c640 1 --2- 172.21.15.2:0/3784621501 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x560aaa270980 0x560aaa272e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.104+0000 7f9589ffb640 10 client.17265 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:15:37.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.104+0000 7f95a259c640 1 --2- 172.21.15.2:0/3784621501 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x560aaa270980 0x560aaa272e20 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b48d80e89a34df server_cookie=a0e15fa255183d0a in_seq=0 out_seq=0 2024-05-14T17:15:37.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.105+0000 7f9589ffb640 1 -- 172.21.15.2:0/3784621501 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560aaa1c8b80 con 0x560aaa270980 2024-05-14T17:15:37.529 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.105+0000 7f9589ffb640 10 client.17265 handle_command_reply: tid=0 2024-05-14T17:15:37.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.105+0000 7f95aba0c740 1 client.17265 shutdown 2024-05-14T17:15:37.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.105+0000 7f95aba0c740 1 -- 172.21.15.2:0/3784621501 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560aaa260ce0 msgr2=0x560aaa25dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.105+0000 7f95aba0c740 1 --2- 172.21.15.2:0/3784621501 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560aaa260ce0 0x560aaa25dd60 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f959c0316e0 tx=0x7f959c0095c0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.105+0000 7f95aba0c740 1 -- 172.21.15.2:0/3784621501 shutdown_connections 2024-05-14T17:15:37.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.105+0000 7f95aba0c740 1 --2- 172.21.15.2:0/3784621501 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x560aaa270980 0x560aaa272e20 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.105+0000 7f95aba0c740 1 --2- 172.21.15.2:0/3784621501 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560aaa260ce0 0x560aaa25dd60 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.105+0000 7f95aba0c740 1 --2- 172.21.15.2:0/3784621501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560aaa2540f0 0x560aaa25d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.105+0000 7f9589ffb640 0 client.17265 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:15:37.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.105+0000 7f95aba0c740 1 --2- 172.21.15.2:0/3784621501 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560aaa2537d0 0x560aaa260530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.105+0000 7f95aba0c740 1 -- 172.21.15.2:0/3784621501 >> 172.21.15.2:0/3784621501 conn(0x560aaa24fe00 msgr2=0x560aaa255bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:37.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.106+0000 7f95aba0c740 1 -- 172.21.15.2:0/3784621501 shutdown_connections 2024-05-14T17:15:37.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.106+0000 7f95aba0c740 1 -- 172.21.15.2:0/3784621501 wait complete. 2024-05-14T17:15:37.530 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.106+0000 7f95aba0c740 20 client.17265 trim_cache size 0 max 16384 2024-05-14T17:15:37.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.121+0000 7f95aba0c740 1 Processor -- start 2024-05-14T17:15:37.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.121+0000 7f95aba0c740 1 -- start start 2024-05-14T17:15:37.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.121+0000 7f95aba0c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560aaa2537d0 0x560aaa25c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.121+0000 7f95aba0c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560aaa2540f0 0x560aaa269080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.121+0000 7f95aba0c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560aaa260ce0 0x560aaa260060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.121+0000 7f95aba0c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f95a41aec70 con 0x560aaa2540f0 2024-05-14T17:15:37.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.121+0000 7f95aba0c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f95a41ac4e0 con 0x560aaa2537d0 2024-05-14T17:15:37.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.121+0000 7f95aba0c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f95a41aa350 con 0x560aaa260ce0 2024-05-14T17:15:37.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.121+0000 7f95a1d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560aaa2540f0 0x560aaa269080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.121+0000 7f95a1d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560aaa2540f0 0x560aaa269080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45488/0 (socket says 172.21.15.2:45488) 2024-05-14T17:15:37.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.121+0000 7f95a1d9b640 1 -- 172.21.15.2:0/3445664765 learned_addr learned my addr 172.21.15.2:0/3445664765 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:37.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.121+0000 7f95a259c640 1 --2- 172.21.15.2:0/3445664765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560aaa2537d0 0x560aaa25c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.121+0000 7f95a2d9d640 1 --2- 172.21.15.2:0/3445664765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560aaa260ce0 0x560aaa260060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.531 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.122+0000 7f95a1d9b640 1 -- 172.21.15.2:0/3445664765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560aaa260ce0 msgr2=0x560aaa260060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.122+0000 7f95a1d9b640 1 --2- 172.21.15.2:0/3445664765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560aaa260ce0 0x560aaa260060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.122+0000 7f95a1d9b640 1 -- 172.21.15.2:0/3445664765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560aaa2537d0 msgr2=0x560aaa25c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.122+0000 7f95a1d9b640 1 --2- 172.21.15.2:0/3445664765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560aaa2537d0 0x560aaa25c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.122+0000 7f95a1d9b640 1 -- 172.21.15.2:0/3445664765 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560aaa253f20 con 0x560aaa2540f0 2024-05-14T17:15:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.122+0000 7f95a1d9b640 1 --2- 172.21.15.2:0/3445664765 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560aaa2540f0 0x560aaa269080 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f95a41060d0 tx=0x7f9598012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906ed3df7676fccd server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.122+0000 7f958bfff640 1 -- 172.21.15.2:0/3445664765 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9598004030 con 0x560aaa2540f0 2024-05-14T17:15:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.122+0000 7f958bfff640 1 -- 172.21.15.2:0/3445664765 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95980041d0 con 0x560aaa2540f0 2024-05-14T17:15:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.122+0000 7f958bfff640 1 -- 172.21.15.2:0/3445664765 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959801c680 con 0x560aaa2540f0 2024-05-14T17:15:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.122+0000 7f95aba0c740 1 -- 172.21.15.2:0/3445664765 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560aaa2540f0 msgr2=0x560aaa269080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.123+0000 7f95aba0c740 1 --2- 172.21.15.2:0/3445664765 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560aaa2540f0 0x560aaa269080 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f95a41060d0 tx=0x7f9598012820 comp rx=0 tx=0).stop 2024-05-14T17:15:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.123+0000 7f95aba0c740 1 -- 172.21.15.2:0/3445664765 shutdown_connections 2024-05-14T17:15:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.123+0000 7f95aba0c740 1 --2- 172.21.15.2:0/3445664765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560aaa260ce0 0x560aaa260060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.123+0000 7f95aba0c740 1 --2- 172.21.15.2:0/3445664765 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560aaa2540f0 0x560aaa269080 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.123+0000 7f95aba0c740 1 --2- 172.21.15.2:0/3445664765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560aaa2537d0 0x560aaa25c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.123+0000 7f95aba0c740 1 -- 172.21.15.2:0/3445664765 >> 172.21.15.2:0/3445664765 conn(0x560aaa24fe00 msgr2=0x560aaa24e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.123+0000 7f95aba0c740 1 -- 172.21.15.2:0/3445664765 shutdown_connections 2024-05-14T17:15:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.123+0000 7f95aba0c740 1 -- 172.21.15.2:0/3445664765 wait complete. 2024-05-14T17:15:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.123+0000 7f95aba0c740 1 -- start start 2024-05-14T17:15:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.123+0000 7f95aba0c740 1 Processor -- start 2024-05-14T17:15:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.123+0000 7f95aba0c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560aaa2537d0 0x560aaa256510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.123+0000 7f95aba0c740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560aaa2540f0 0x560aaa256a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.123+0000 7f95aba0c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560aaa260ce0 0x560aaa256f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.123+0000 7f95aba0c740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f95a41aa9a0 con 0x560aaa2540f0 2024-05-14T17:15:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.123+0000 7f95aba0c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f95a41abe10 con 0x560aaa260ce0 2024-05-14T17:15:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.123+0000 7f95aba0c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f95a41ab070 con 0x560aaa2537d0 2024-05-14T17:15:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.123+0000 7f95aba0c740 10 client.0 _ll_register_callbacks cb 0x7f959c031f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:15:37.533 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.123+0000 7f95a1d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560aaa2540f0 0x560aaa256a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.123+0000 7f95a1d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560aaa2540f0 0x560aaa256a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45500/0 (socket says 172.21.15.2:45500) 2024-05-14T17:15:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.123+0000 7f95a1d9b640 1 -- 172.21.15.2:0/3580032372 learned_addr learned my addr 172.21.15.2:0/3580032372 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.123+0000 7f95a2d9d640 1 --2- 172.21.15.2:0/3580032372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560aaa260ce0 0x560aaa256f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.124+0000 7f95a259c640 1 --2- 172.21.15.2:0/3580032372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560aaa2537d0 0x560aaa256510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.124+0000 7f95a1d9b640 1 -- 172.21.15.2:0/3580032372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560aaa2537d0 msgr2=0x560aaa256510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.124+0000 7f95a1d9b640 1 --2- 172.21.15.2:0/3580032372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560aaa2537d0 0x560aaa256510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.124+0000 7f95a1d9b640 1 -- 172.21.15.2:0/3580032372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560aaa260ce0 msgr2=0x560aaa256f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.124+0000 7f95a1d9b640 1 --2- 172.21.15.2:0/3580032372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560aaa260ce0 0x560aaa256f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.124+0000 7f95a1d9b640 1 -- 172.21.15.2:0/3580032372 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560aaa2dc2c0 con 0x560aaa2540f0 2024-05-14T17:15:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.124+0000 7f95a1d9b640 1 --2- 172.21.15.2:0/3580032372 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560aaa2540f0 0x560aaa256a50 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f95980058c0 tx=0x7f95980045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61851e21cee78675 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.124+0000 7f9589ffb640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:15:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.124+0000 7f9589ffb640 1 -- 172.21.15.2:0/3580032372 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959800be70 con 0x560aaa2540f0 2024-05-14T17:15:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.124+0000 7f95aba0c740 1 -- 172.21.15.2:0/3580032372 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95980041d0 con 0x560aaa2540f0 2024-05-14T17:15:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.124+0000 7f9589ffb640 1 -- 172.21.15.2:0/3580032372 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8513+0+0 (secure 0 0 0) 0x7f9598018040 con 0x560aaa2540f0 2024-05-14T17:15:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.124+0000 7f9589ffb640 1 -- 172.21.15.2:0/3580032372 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f9598005260 con 0x560aaa2540f0 2024-05-14T17:15:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.124+0000 7f95aba0c740 10 client.17271 fetch_fsmap learned FSMap version 57 2024-05-14T17:15:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.124+0000 7f95aba0c740 1 -- 172.21.15.2:0/3580032372 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x560aaa2dd520 con 0x560aaa2540f0 2024-05-14T17:15:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.125+0000 7f9589ffb640 1 -- 172.21.15.2:0/3580032372 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95980041d0 con 0x560aaa2540f0 2024-05-14T17:15:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.125+0000 7f9589ffb640 1 -- 172.21.15.2:0/3580032372 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 57) ==== 8350+0+0 (secure 0 0 0) 0x7f9598004930 con 0x560aaa2540f0 2024-05-14T17:15:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.125+0000 7f95aba0c740 10 client.17271 fetch_fsmap finished waiting for FSMap version 57 2024-05-14T17:15:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.125+0000 7f95aba0c740 10 client.17271 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:15:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.125+0000 7f95aba0c740 20 client.17271 populate_metadata read hostname 'smithi002' 2024-05-14T17:15:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.125+0000 7f95aba0c740 1 --2- 172.21.15.2:0/3580032372 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x560aaa2ddbf0 0x560aaa2e0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.125+0000 7f95aba0c740 4 client.17271 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:15:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.125+0000 7f95aba0c740 1 -- 172.21.15.2:0/3580032372 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560aaa1c2360 con 0x560aaa2ddbf0 2024-05-14T17:15:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.125+0000 7f95a259c640 1 --2- 172.21.15.2:0/3580032372 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x560aaa2ddbf0 0x560aaa2e0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.126+0000 7f9589ffb640 10 client.17271 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:15:37.536 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:01.126+0000 7f95a259c640 1 --2- 172.21.15.2:0/3580032372 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x560aaa2ddbf0 0x560aaa2e0010 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17de56c22caaf5dc server_cookie=e4877ee2e51c5185 in_seq=0 out_seq=0 2024-05-14T17:15:37.536 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:02 smithi002 ceph-mon[31538]: pgmap v639: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.4 KiB/s wr, 6 op/s 2024-05-14T17:15:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:03.483+0000 7f9589ffb640 1 -- 172.21.15.2:0/3580032372 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560aaa1c2360 con 0x560aaa2ddbf0 2024-05-14T17:15:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:03.483+0000 7f9589ffb640 10 client.17271 handle_command_reply: tid=0 2024-05-14T17:15:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:03.483+0000 7f95aba0c740 1 client.17271 shutdown 2024-05-14T17:15:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:03.483+0000 7f95aba0c740 1 -- 172.21.15.2:0/3580032372 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560aaa2540f0 msgr2=0x560aaa256a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:03.483+0000 7f95aba0c740 1 --2- 172.21.15.2:0/3580032372 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560aaa2540f0 0x560aaa256a50 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f95980058c0 tx=0x7f95980045b0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:03.483+0000 7f95aba0c740 1 -- 172.21.15.2:0/3580032372 shutdown_connections 2024-05-14T17:15:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:03.483+0000 7f95aba0c740 1 --2- 172.21.15.2:0/3580032372 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x560aaa2ddbf0 0x560aaa2e0010 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:03.483+0000 7f95aba0c740 1 --2- 172.21.15.2:0/3580032372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560aaa260ce0 0x560aaa256f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:03.483+0000 7f95aba0c740 1 --2- 172.21.15.2:0/3580032372 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560aaa2540f0 0x560aaa256a50 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:03.483+0000 7f9589ffb640 0 client.17271 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:15:37.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:03.483+0000 7f95aba0c740 1 --2- 172.21.15.2:0/3580032372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560aaa2537d0 0x560aaa256510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:03.483+0000 7f95aba0c740 1 -- 172.21.15.2:0/3580032372 >> 172.21.15.2:0/3580032372 conn(0x560aaa24fe00 msgr2=0x560aaa24f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:37.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:03.483+0000 7f95aba0c740 1 -- 172.21.15.2:0/3580032372 shutdown_connections 2024-05-14T17:15:37.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:03.483+0000 7f95aba0c740 1 -- 172.21.15.2:0/3580032372 wait complete. 2024-05-14T17:15:37.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:03.483+0000 7f95aba0c740 20 client.17271 trim_cache size 0 max 16384 2024-05-14T17:15:37.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:15:37.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:15:37.538 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:15:37.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:03.484+0000 7f95a8cf1640 1 -- 172.21.15.2:0/507346684 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f958c0b2a10 msgr2=0x7f958c0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:03.484+0000 7f95a8cf1640 1 --2- 172.21.15.2:0/507346684 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f958c0b2a10 0x7f958c0b4ed0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f959c006010 tx=0x7f959c0093f0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:03.484+0000 7f95a8cf1640 1 -- 172.21.15.2:0/507346684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95a41020a0 msgr2=0x7f95a4198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:03.484+0000 7f95a8cf1640 1 --2- 172.21.15.2:0/507346684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95a41020a0 0x7f95a4198000 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f959400d770 tx=0x7f959400dc30 comp rx=0 tx=0).stop 2024-05-14T17:15:37.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:03.484+0000 7f95a8cf1640 1 -- 172.21.15.2:0/507346684 shutdown_connections 2024-05-14T17:15:37.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:03.484+0000 7f95a8cf1640 1 --2- 172.21.15.2:0/507346684 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f958c0b2a10 0x7f958c0b4ed0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.539 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:03.484+0000 7f95a8cf1640 1 --2- 172.21.15.2:0/507346684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95a4106980 0x7f95a419c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:03.484+0000 7f95a8cf1640 1 --2- 172.21.15.2:0/507346684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95a4102a60 0x7f95a4198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:03.484+0000 7f95a8cf1640 1 --2- 172.21.15.2:0/507346684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95a41020a0 0x7f95a4198000 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:03.484+0000 7f95a8cf1640 1 -- 172.21.15.2:0/507346684 >> 172.21.15.2:0/507346684 conn(0x7f95a4077e70 msgr2=0x7f95a4104830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:03.484+0000 7f95a8cf1640 1 -- 172.21.15.2:0/507346684 shutdown_connections 2024-05-14T17:15:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:03.484+0000 7f95a8cf1640 1 -- 172.21.15.2:0/507346684 wait complete. 2024-05-14T17:15:37.541 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:04 smithi002 ceph-mon[31538]: pgmap v640: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 56 KiB/s wr, 26 op/s 2024-05-14T17:15:37.541 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:04 smithi002 ceph-mon[31538]: osdmap e108: 12 total, 12 up, 12 in 2024-05-14T17:15:37.541 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:06 smithi002 ceph-mon[31538]: pgmap v642: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 60 KiB/s wr, 27 op/s 2024-05-14T17:15:37.541 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:08 smithi002 ceph-mon[31538]: pgmap v643: 97 pgs: 97 active+clean; 3.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 60 KiB/s wr, 27 op/s 2024-05-14T17:15:37.541 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:10 smithi002 ceph-mon[31538]: pgmap v644: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 60 KiB/s wr, 26 op/s 2024-05-14T17:15:37.541 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:12 smithi002 ceph-mon[31538]: pgmap v645: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 60 KiB/s wr, 26 op/s 2024-05-14T17:15:37.541 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:12 smithi002 ceph-mon[31538]: osdmap e109: 12 total, 12 up, 12 in 2024-05-14T17:15:37.541 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:13 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:15:37.541 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:14 smithi002 ceph-mon[31538]: pgmap v647: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 517 B/s wr, 4 op/s 2024-05-14T17:15:37.541 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:14 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:15:37.541 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:14 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:15:37.541 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:14 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:15:37.542 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:14 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:15:37.542 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:14 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:15:37.542 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:14 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:15:37.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.578+0000 7f4fc34bd640 1 -- 172.21.15.2:0/3174563090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4fbc108a40 msgr2=0x7f4fbc108e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.578+0000 7f4fc34bd640 1 --2- 172.21.15.2:0/3174563090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4fbc108a40 0x7f4fbc108e20 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8009900 tx=0x7f4fb802a410 comp rx=0 tx=0).stop 2024-05-14T17:15:37.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.578+0000 7f4fc34bd640 1 -- 172.21.15.2:0/3174563090 shutdown_connections 2024-05-14T17:15:37.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.578+0000 7f4fc34bd640 1 --2- 172.21.15.2:0/3174563090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fbc06bc20 0x7f4fbc10f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.578+0000 7f4fc34bd640 1 --2- 172.21.15.2:0/3174563090 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4fbc06b280 0x7f4fbc06b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.578+0000 7f4fc34bd640 1 --2- 172.21.15.2:0/3174563090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4fbc108a40 0x7f4fbc108e20 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.578+0000 7f4fc34bd640 1 -- 172.21.15.2:0/3174563090 >> 172.21.15.2:0/3174563090 conn(0x7f4fbc077750 msgr2=0x7f4fbc077b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:37.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.578+0000 7f4fc34bd640 1 -- 172.21.15.2:0/3174563090 shutdown_connections 2024-05-14T17:15:37.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.578+0000 7f4fc34bd640 1 -- 172.21.15.2:0/3174563090 wait complete. 2024-05-14T17:15:37.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.579+0000 7f4fc34bd640 1 Processor -- start 2024-05-14T17:15:37.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.579+0000 7f4fc34bd640 1 -- start start 2024-05-14T17:15:37.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.579+0000 7f4fc34bd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fbc06b280 0x7f4fbc199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.555 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.579+0000 7f4fc34bd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4fbc06bc20 0x7f4fbc199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.579+0000 7f4fc34bd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4fbc108a40 0x7f4fbc19d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.579+0000 7f4fc34bd640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc112cb0 con 0x7f4fbc108a40 2024-05-14T17:15:37.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.579+0000 7f4fc34bd640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc112b30 con 0x7f4fbc06b280 2024-05-14T17:15:37.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.579+0000 7f4fc34bd640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc112e30 con 0x7f4fbc06bc20 2024-05-14T17:15:37.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.579+0000 7f4fc1a5a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4fbc108a40 0x7f4fbc19d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.579+0000 7f4fc1a5a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4fbc108a40 0x7f4fbc19d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55676/0 (socket says 172.21.15.2:55676) 2024-05-14T17:15:37.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.579+0000 7f4fc1a5a640 1 -- 172.21.15.2:0/2488231346 learned_addr learned my addr 172.21.15.2:0/2488231346 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:37.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.579+0000 7f4fc0a58640 1 --2- 172.21.15.2:0/2488231346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4fbc06bc20 0x7f4fbc199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.579+0000 7f4fc1259640 1 --2- 172.21.15.2:0/2488231346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fbc06b280 0x7f4fbc199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.579+0000 7f4fc1a5a640 1 -- 172.21.15.2:0/2488231346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4fbc06bc20 msgr2=0x7f4fbc199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.580+0000 7f4fc1a5a640 1 --2- 172.21.15.2:0/2488231346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4fbc06bc20 0x7f4fbc199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.580+0000 7f4fc1a5a640 1 -- 172.21.15.2:0/2488231346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fbc06b280 msgr2=0x7f4fbc199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.556 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.580+0000 7f4fc1a5a640 1 --2- 172.21.15.2:0/2488231346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fbc06b280 0x7f4fbc199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.580+0000 7f4fc1a5a640 1 -- 172.21.15.2:0/2488231346 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fbc19e0b0 con 0x7f4fbc108a40 2024-05-14T17:15:37.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.580+0000 7f4fc1259640 1 --2- 172.21.15.2:0/2488231346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fbc06b280 0x7f4fbc199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:15:37.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.580+0000 7f4fc1a5a640 1 --2- 172.21.15.2:0/2488231346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4fbc108a40 0x7f4fbc19d980 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f4fb400eb10 tx=0x7f4fb400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:37.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.580+0000 7f4fb27fc640 1 -- 172.21.15.2:0/2488231346 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb400ce20 con 0x7f4fbc108a40 2024-05-14T17:15:37.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.580+0000 7f4fb27fc640 1 -- 172.21.15.2:0/2488231346 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fb4005020 con 0x7f4fbc108a40 2024-05-14T17:15:37.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.580+0000 7f4fc34bd640 1 -- 172.21.15.2:0/2488231346 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fbc19e3a0 con 0x7f4fbc108a40 2024-05-14T17:15:37.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.580+0000 7f4fb27fc640 1 -- 172.21.15.2:0/2488231346 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4010690 con 0x7f4fbc108a40 2024-05-14T17:15:37.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.580+0000 7f4fc34bd640 1 -- 172.21.15.2:0/2488231346 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fbc1a5c30 con 0x7f4fbc108a40 2024-05-14T17:15:37.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.583+0000 7f4fc61d1740 1 Processor -- start 2024-05-14T17:15:37.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.583+0000 7f4fb27fc640 1 -- 172.21.15.2:0/2488231346 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f4fb4005510 con 0x7f4fbc108a40 2024-05-14T17:15:37.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.583+0000 7f4fc61d1740 1 -- start start 2024-05-14T17:15:37.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.584+0000 7f4fc61d1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055fc008f0 0x56055fc00cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.584+0000 7f4fc61d1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56055fc01210 0x56055fc0b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.584+0000 7f4fc61d1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56055fc0bad0 0x56055fc0dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.557 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.584+0000 7f4fc61d1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56055fb73c30 con 0x56055fc0bad0 2024-05-14T17:15:37.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.584+0000 7f4fc61d1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56055fb637d0 con 0x56055fc01210 2024-05-14T17:15:37.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.584+0000 7f4fc61d1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56055fb67810 con 0x56055fc008f0 2024-05-14T17:15:37.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.584+0000 7f4fc1259640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56055fc01210 0x56055fc0b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.584+0000 7f4fc0a58640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055fc008f0 0x56055fc00cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.584+0000 7f4fc1259640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56055fc01210 0x56055fc0b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:48116/0 (socket says 172.21.15.2:48116) 2024-05-14T17:15:37.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.584+0000 7f4fc1259640 1 -- 172.21.15.2:0/3647758352 learned_addr learned my addr 172.21.15.2:0/3647758352 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:37.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.584+0000 7f4fc0a58640 1 --2- 172.21.15.2:0/3647758352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56055fc0bad0 0x56055fc0dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.584+0000 7f4fb27fc640 1 --2- 172.21.15.2:0/2488231346 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f4fa80b2a90 0x7f4fa80b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.584+0000 7f4fc0a58640 1 -- 172.21.15.2:0/3647758352 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055fc008f0 msgr2=0x56055fc00cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.584+0000 7f4fc0a58640 1 --2- 172.21.15.2:0/3647758352 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055fc008f0 0x56055fc00cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.584+0000 7f4fc0a58640 1 -- 172.21.15.2:0/3647758352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56055fc01210 msgr2=0x56055fc0b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.584+0000 7f4fc0a58640 1 --2- 172.21.15.2:0/3647758352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56055fc01210 0x56055fc0b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.558 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.584+0000 7f4fc0a58640 1 -- 172.21.15.2:0/3647758352 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56055f878210 con 0x56055fc0bad0 2024-05-14T17:15:37.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.584+0000 7f4fc1259640 1 --2- 172.21.15.2:0/3647758352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56055fc01210 0x56055fc0b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:15:37.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.584+0000 7f4fc1a5a640 1 --2- 172.21.15.2:0/2488231346 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f4fa80b2a90 0x7f4fa80b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.584+0000 7f4fb27fc640 1 -- 172.21.15.2:0/2488231346 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8469+0+0 (secure 0 0 0) 0x7f4fb40e2b50 con 0x7f4fbc108a40 2024-05-14T17:15:37.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.584+0000 7f4fc0a58640 1 --2- 172.21.15.2:0/3647758352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56055fc0bad0 0x56055fc0dec0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc108000 tx=0x7f4fac00a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=331d2b223381958e server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:37.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.585+0000 7f4fa77fe640 1 -- 172.21.15.2:0/3647758352 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac0110d0 con 0x56055fc0bad0 2024-05-14T17:15:37.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.585+0000 7f4fa77fe640 1 -- 172.21.15.2:0/3647758352 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fac011270 con 0x56055fc0bad0 2024-05-14T17:15:37.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.585+0000 7f4fa77fe640 1 -- 172.21.15.2:0/3647758352 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac0047a0 con 0x56055fc0bad0 2024-05-14T17:15:37.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.585+0000 7f4fc61d1740 1 -- 172.21.15.2:0/3647758352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56055fc0bad0 msgr2=0x56055fc0dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.585+0000 7f4fc1a5a640 1 --2- 172.21.15.2:0/2488231346 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f4fa80b2a90 0x7f4fa80b4f50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4fb40e0380 tx=0x7f4fb4005480 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:37.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.585+0000 7f4fc61d1740 1 --2- 172.21.15.2:0/3647758352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56055fc0bad0 0x56055fc0dec0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc108000 tx=0x7f4fac00a2d0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.585+0000 7f4fc61d1740 1 -- 172.21.15.2:0/3647758352 shutdown_connections 2024-05-14T17:15:37.559 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.585+0000 7f4fc61d1740 1 --2- 172.21.15.2:0/3647758352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56055fc0bad0 0x56055fc0dec0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.585+0000 7f4fc61d1740 1 --2- 172.21.15.2:0/3647758352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56055fc01210 0x56055fc0b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.585+0000 7f4fc61d1740 1 --2- 172.21.15.2:0/3647758352 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055fc008f0 0x56055fc00cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.585+0000 7f4fc61d1740 1 -- 172.21.15.2:0/3647758352 >> 172.21.15.2:0/3647758352 conn(0x56055fbfce00 msgr2=0x56055fbfd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.585+0000 7f4fc61d1740 1 -- 172.21.15.2:0/3647758352 shutdown_connections 2024-05-14T17:15:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.585+0000 7f4fc61d1740 1 -- 172.21.15.2:0/3647758352 wait complete. 2024-05-14T17:15:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.586+0000 7f4fc61d1740 1 -- start start 2024-05-14T17:15:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.586+0000 7f4fc61d1740 1 Processor -- start 2024-05-14T17:15:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.586+0000 7f4fc61d1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56055fc008f0 0x56055fc0b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.586+0000 7f4fc61d1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56055fc01210 0x56055fc08710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.586+0000 7f4fc61d1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055fc0bad0 0x56055fc08c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.586+0000 7f4fc61d1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x56055f849790 con 0x56055fc01210 2024-05-14T17:15:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.586+0000 7f4fc61d1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56055fb1f890 con 0x56055fc008f0 2024-05-14T17:15:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.586+0000 7f4fc61d1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56055fc0e740 con 0x56055fc0bad0 2024-05-14T17:15:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.586+0000 7f4fc61d1740 10 client.0 _ll_register_callbacks cb 0x56055fb22d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:15:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.586+0000 7f4fc1259640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56055fc01210 0x56055fc08710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.586+0000 7f4fc1259640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56055fc01210 0x56055fc08710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55702/0 (socket says 172.21.15.2:55702) 2024-05-14T17:15:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.586+0000 7f4fc1259640 1 -- 172.21.15.2:0/1113478005 learned_addr learned my addr 172.21.15.2:0/1113478005 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.587+0000 7f4fc0a58640 1 --2- 172.21.15.2:0/1113478005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055fc0bad0 0x56055fc08c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.587+0000 7f4fc0a58640 1 --2- 172.21.15.2:0/1113478005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56055fc008f0 0x56055fc0b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.587+0000 7f4fc1259640 1 -- 172.21.15.2:0/1113478005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055fc0bad0 msgr2=0x56055fc08c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.587+0000 7f4fc1259640 1 --2- 172.21.15.2:0/1113478005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055fc0bad0 0x56055fc08c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.587+0000 7f4fc1259640 1 -- 172.21.15.2:0/1113478005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56055fc008f0 msgr2=0x56055fc0b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.587+0000 7f4fc1259640 1 --2- 172.21.15.2:0/1113478005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56055fc008f0 0x56055fc0b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.587+0000 7f4fc1259640 1 -- 172.21.15.2:0/1113478005 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56055f944620 con 0x56055fc01210 2024-05-14T17:15:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.587+0000 7f4fc0a58640 1 --2- 172.21.15.2:0/1113478005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055fc0bad0 0x56055fc08c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:15:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.587+0000 7f4fc1259640 1 --2- 172.21.15.2:0/1113478005 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56055fc01210 0x56055fc08710 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8009230 tx=0x7f4fb8031760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=144efa57c8efbbec server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.587+0000 7f4fa4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:15:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.587+0000 7f4fa4ff9640 1 -- 172.21.15.2:0/1113478005 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8035040 con 0x56055fc01210 2024-05-14T17:15:37.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.587+0000 7f4fa4ff9640 1 -- 172.21.15.2:0/1113478005 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8469+0+0 (secure 0 0 0) 0x7f4fb8004240 con 0x56055fc01210 2024-05-14T17:15:37.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.587+0000 7f4fc61d1740 1 -- 172.21.15.2:0/1113478005 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fac011270 con 0x56055fc01210 2024-05-14T17:15:37.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.587+0000 7f4fa4ff9640 1 -- 172.21.15.2:0/1113478005 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f4fb8004440 con 0x56055fc01210 2024-05-14T17:15:37.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.587+0000 7f4fc61d1740 10 client.17289 fetch_fsmap learned FSMap version 57 2024-05-14T17:15:37.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.587+0000 7f4fc61d1740 1 -- 172.21.15.2:0/1113478005 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x56055fc1a1a0 con 0x56055fc01210 2024-05-14T17:15:37.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.587+0000 7f4fa4ff9640 1 -- 172.21.15.2:0/1113478005 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fac011270 con 0x56055fc01210 2024-05-14T17:15:37.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.588+0000 7f4fa4ff9640 1 -- 172.21.15.2:0/1113478005 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 57) ==== 8350+0+0 (secure 0 0 0) 0x7f4fb8004b80 con 0x56055fc01210 2024-05-14T17:15:37.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.588+0000 7f4fc61d1740 10 client.17289 fetch_fsmap finished waiting for FSMap version 57 2024-05-14T17:15:37.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.588+0000 7f4fc61d1740 10 client.17289 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:15:37.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.588+0000 7f4fc61d1740 20 client.17289 populate_metadata read hostname 'smithi002' 2024-05-14T17:15:37.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.588+0000 7f4fc61d1740 1 --2- 172.21.15.2:0/1113478005 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56055fc1b700 0x56055fc1db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.588+0000 7f4fc61d1740 4 client.17289 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:15:37.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.588+0000 7f4fc61d1740 1 -- 172.21.15.2:0/1113478005 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56055fb75b80 con 0x56055fc1b700 2024-05-14T17:15:37.562 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.588+0000 7f4fc0a58640 1 --2- 172.21.15.2:0/1113478005 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56055fc1b700 0x56055fc1db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.588+0000 7f4fa4ff9640 10 client.17289 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:15:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.588+0000 7f4fc0a58640 1 --2- 172.21.15.2:0/1113478005 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56055fc1b700 0x56055fc1db10 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9594dbe66993fcf server_cookie=24a40027037ca928 in_seq=0 out_seq=0 2024-05-14T17:15:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.589+0000 7f4fa4ff9640 1 -- 172.21.15.2:0/1113478005 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56055fb75b80 con 0x56055fc1b700 2024-05-14T17:15:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.589+0000 7f4fa4ff9640 10 client.17289 handle_command_reply: tid=0 2024-05-14T17:15:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.589+0000 7f4fc61d1740 1 client.17289 shutdown 2024-05-14T17:15:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.589+0000 7f4fc61d1740 1 -- 172.21.15.2:0/1113478005 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56055fc01210 msgr2=0x56055fc08710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.589+0000 7f4fc61d1740 1 --2- 172.21.15.2:0/1113478005 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56055fc01210 0x56055fc08710 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8009230 tx=0x7f4fb8031760 comp rx=0 tx=0).stop 2024-05-14T17:15:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.589+0000 7f4fc61d1740 1 -- 172.21.15.2:0/1113478005 shutdown_connections 2024-05-14T17:15:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.589+0000 7f4fc61d1740 1 --2- 172.21.15.2:0/1113478005 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56055fc1b700 0x56055fc1db10 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.589+0000 7f4fc61d1740 1 --2- 172.21.15.2:0/1113478005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055fc0bad0 0x56055fc08c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.589+0000 7f4fc61d1740 1 --2- 172.21.15.2:0/1113478005 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56055fc01210 0x56055fc08710 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.589+0000 7f4fa4ff9640 0 client.17289 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:15:37.563 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.589+0000 7f4fc61d1740 1 --2- 172.21.15.2:0/1113478005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56055fc008f0 0x56055fc0b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.589+0000 7f4fc61d1740 1 -- 172.21.15.2:0/1113478005 >> 172.21.15.2:0/1113478005 conn(0x56055fbfce00 msgr2=0x56055fbfc100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.589+0000 7f4fc61d1740 1 -- 172.21.15.2:0/1113478005 shutdown_connections 2024-05-14T17:15:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.589+0000 7f4fc61d1740 1 -- 172.21.15.2:0/1113478005 wait complete. 2024-05-14T17:15:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.590+0000 7f4fc61d1740 20 client.17289 trim_cache size 0 max 16384 2024-05-14T17:15:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.605+0000 7f4fc61d1740 1 Processor -- start 2024-05-14T17:15:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.605+0000 7f4fc61d1740 1 -- start start 2024-05-14T17:15:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.605+0000 7f4fc61d1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055fc008f0 0x56055fc23420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.605+0000 7f4fc61d1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56055fc01210 0x56055fc23960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.605+0000 7f4fc61d1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56055fc0bad0 0x56055fc05990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.605+0000 7f4fc61d1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc1b0e70 con 0x56055fc01210 2024-05-14T17:15:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.605+0000 7f4fc61d1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc1ae6e0 con 0x56055fc0bad0 2024-05-14T17:15:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.605+0000 7f4fc61d1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc1ac530 con 0x56055fc008f0 2024-05-14T17:15:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.605+0000 7f4fc1259640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56055fc01210 0x56055fc23960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.605+0000 7f4fc1259640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56055fc01210 0x56055fc23960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55714/0 (socket says 172.21.15.2:55714) 2024-05-14T17:15:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.605+0000 7f4fc1259640 1 -- 172.21.15.2:0/3546687326 learned_addr learned my addr 172.21.15.2:0/3546687326 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.605+0000 7f4fc0a58640 1 --2- 172.21.15.2:0/3546687326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56055fc0bad0 0x56055fc05990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.605+0000 7f4fc0a58640 1 --2- 172.21.15.2:0/3546687326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055fc008f0 0x56055fc23420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.605+0000 7f4fc1259640 1 -- 172.21.15.2:0/3546687326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055fc008f0 msgr2=0x56055fc23420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.606+0000 7f4fc1259640 1 --2- 172.21.15.2:0/3546687326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055fc008f0 0x56055fc23420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.606+0000 7f4fc1259640 1 -- 172.21.15.2:0/3546687326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56055fc0bad0 msgr2=0x56055fc05990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.606+0000 7f4fc1259640 1 --2- 172.21.15.2:0/3546687326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56055fc0bad0 0x56055fc05990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.606+0000 7f4fc1259640 1 -- 172.21.15.2:0/3546687326 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56055fc08fe0 con 0x56055fc01210 2024-05-14T17:15:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.606+0000 7f4fc0a58640 1 --2- 172.21.15.2:0/3546687326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055fc008f0 0x56055fc23420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:15:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.606+0000 7f4fc1259640 1 --2- 172.21.15.2:0/3546687326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56055fc01210 0x56055fc23960 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8009900 tx=0x7f4fb802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e71d675d5a683715 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.606+0000 7f4fa6ffd640 1 -- 172.21.15.2:0/3546687326 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8052070 con 0x56055fc01210 2024-05-14T17:15:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.606+0000 7f4fa6ffd640 1 -- 172.21.15.2:0/3546687326 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fb8004b70 con 0x56055fc01210 2024-05-14T17:15:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.606+0000 7f4fa6ffd640 1 -- 172.21.15.2:0/3546687326 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8035040 con 0x56055fc01210 2024-05-14T17:15:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.606+0000 7f4fc61d1740 1 -- 172.21.15.2:0/3546687326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56055fc01210 msgr2=0x56055fc23960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.606+0000 7f4fc61d1740 1 --2- 172.21.15.2:0/3546687326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56055fc01210 0x56055fc23960 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8009900 tx=0x7f4fb802a410 comp rx=0 tx=0).stop 2024-05-14T17:15:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.607+0000 7f4fc61d1740 1 -- 172.21.15.2:0/3546687326 shutdown_connections 2024-05-14T17:15:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.607+0000 7f4fc61d1740 1 --2- 172.21.15.2:0/3546687326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56055fc0bad0 0x56055fc05990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.607+0000 7f4fc61d1740 1 --2- 172.21.15.2:0/3546687326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56055fc01210 0x56055fc23960 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.607+0000 7f4fc61d1740 1 --2- 172.21.15.2:0/3546687326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055fc008f0 0x56055fc23420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.607+0000 7f4fc61d1740 1 -- 172.21.15.2:0/3546687326 >> 172.21.15.2:0/3546687326 conn(0x56055fbfce00 msgr2=0x56055fc16600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.607+0000 7f4fc61d1740 1 -- 172.21.15.2:0/3546687326 shutdown_connections 2024-05-14T17:15:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.607+0000 7f4fc61d1740 1 -- 172.21.15.2:0/3546687326 wait complete. 2024-05-14T17:15:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.607+0000 7f4fc61d1740 1 -- start start 2024-05-14T17:15:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.607+0000 7f4fc61d1740 1 Processor -- start 2024-05-14T17:15:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.607+0000 7f4fc61d1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055fc008f0 0x56055fc13840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.607+0000 7f4fc61d1740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56055fc01210 0x56055fc13d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.607+0000 7f4fc61d1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56055fc0bad0 0x56055fc11ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.607+0000 7f4fc61d1740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc1acba0 con 0x56055fc01210 2024-05-14T17:15:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.607+0000 7f4fc61d1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc1ae010 con 0x56055fc0bad0 2024-05-14T17:15:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.607+0000 7f4fc61d1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc1ad270 con 0x56055fc008f0 2024-05-14T17:15:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.607+0000 7f4fc61d1740 10 client.0 _ll_register_callbacks cb 0x7f4fb8032ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:15:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.607+0000 7f4fc1259640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56055fc01210 0x56055fc13d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.608+0000 7f4fc1259640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56055fc01210 0x56055fc13d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55726/0 (socket says 172.21.15.2:55726) 2024-05-14T17:15:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.608+0000 7f4fc1259640 1 -- 172.21.15.2:0/2232248919 learned_addr learned my addr 172.21.15.2:0/2232248919 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.608+0000 7f4fc0a58640 1 --2- 172.21.15.2:0/2232248919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055fc008f0 0x56055fc13840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.608+0000 7f4fc0a58640 1 --2- 172.21.15.2:0/2232248919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56055fc0bad0 0x56055fc11ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.608+0000 7f4fc1259640 1 -- 172.21.15.2:0/2232248919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055fc008f0 msgr2=0x56055fc13840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.608+0000 7f4fc1259640 1 --2- 172.21.15.2:0/2232248919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055fc008f0 0x56055fc13840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.608+0000 7f4fc1259640 1 -- 172.21.15.2:0/2232248919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56055fc0bad0 msgr2=0x56055fc11ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.608+0000 7f4fc1259640 1 --2- 172.21.15.2:0/2232248919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56055fc0bad0 0x56055fc11ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.608+0000 7f4fc1259640 1 -- 172.21.15.2:0/2232248919 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56055fc124c0 con 0x56055fc01210 2024-05-14T17:15:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.608+0000 7f4fc0a58640 1 --2- 172.21.15.2:0/2232248919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055fc008f0 0x56055fc13840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-14T17:15:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.608+0000 7f4fc1259640 1 --2- 172.21.15.2:0/2232248919 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56055fc01210 0x56055fc13d80 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8004f20 tx=0x7f4fb8003640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dec74f66635bc2a server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.608+0000 7f4fa4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:15:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.608+0000 7f4fa4ff9640 1 -- 172.21.15.2:0/2232248919 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8035070 con 0x56055fc01210 2024-05-14T17:15:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.608+0000 7f4fa4ff9640 1 -- 172.21.15.2:0/2232248919 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8469+0+0 (secure 0 0 0) 0x7f4fb80051a0 con 0x56055fc01210 2024-05-14T17:15:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.608+0000 7f4fc61d1740 1 -- 172.21.15.2:0/2232248919 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fb8004b70 con 0x56055fc01210 2024-05-14T17:15:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.608+0000 7f4fa4ff9640 1 -- 172.21.15.2:0/2232248919 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f4fb8052070 con 0x56055fc01210 2024-05-14T17:15:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.608+0000 7f4fc61d1740 10 client.17295 fetch_fsmap learned FSMap version 57 2024-05-14T17:15:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.608+0000 7f4fc61d1740 1 -- 172.21.15.2:0/2232248919 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x56055fc869f0 con 0x56055fc01210 2024-05-14T17:15:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.608+0000 7f4fa4ff9640 1 -- 172.21.15.2:0/2232248919 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fb8004b70 con 0x56055fc01210 2024-05-14T17:15:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.609+0000 7f4fa4ff9640 1 -- 172.21.15.2:0/2232248919 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 57) ==== 8350+0+0 (secure 0 0 0) 0x7f4fb8045470 con 0x56055fc01210 2024-05-14T17:15:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.609+0000 7f4fc61d1740 10 client.17295 fetch_fsmap finished waiting for FSMap version 57 2024-05-14T17:15:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.609+0000 7f4fc61d1740 10 client.17295 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:15:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.609+0000 7f4fc61d1740 20 client.17295 populate_metadata read hostname 'smithi002' 2024-05-14T17:15:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.609+0000 7f4fc61d1740 1 --2- 172.21.15.2:0/2232248919 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56055fc87050 0x56055fc89470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.609+0000 7f4fc61d1740 4 client.17295 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:15:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.609+0000 7f4fc61d1740 1 -- 172.21.15.2:0/2232248919 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56055fb6f360 con 0x56055fc87050 2024-05-14T17:15:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.609+0000 7f4fc0a58640 1 --2- 172.21.15.2:0/2232248919 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56055fc87050 0x56055fc89470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.609+0000 7f4fa4ff9640 10 client.17295 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:15:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.609+0000 7f4fc0a58640 1 --2- 172.21.15.2:0/2232248919 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56055fc87050 0x56055fc89470 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3bc7bb507ccba96 server_cookie=376695f78265bc2b in_seq=0 out_seq=0 2024-05-14T17:15:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.615+0000 7f4fa4ff9640 1 -- 172.21.15.2:0/2232248919 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56055fb6f360 con 0x56055fc87050 2024-05-14T17:15:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.615+0000 7f4fa4ff9640 10 client.17295 handle_command_reply: tid=0 2024-05-14T17:15:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.615+0000 7f4fc61d1740 1 client.17295 shutdown 2024-05-14T17:15:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.616+0000 7f4fc61d1740 1 -- 172.21.15.2:0/2232248919 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56055fc01210 msgr2=0x56055fc13d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.616+0000 7f4fc61d1740 1 --2- 172.21.15.2:0/2232248919 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56055fc01210 0x56055fc13d80 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8004f20 tx=0x7f4fb8003640 comp rx=0 tx=0).stop 2024-05-14T17:15:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.616+0000 7f4fc61d1740 1 -- 172.21.15.2:0/2232248919 shutdown_connections 2024-05-14T17:15:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.616+0000 7f4fc61d1740 1 --2- 172.21.15.2:0/2232248919 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x56055fc87050 0x56055fc89470 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.616+0000 7f4fc61d1740 1 --2- 172.21.15.2:0/2232248919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56055fc0bad0 0x56055fc11ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.616+0000 7f4fc61d1740 1 --2- 172.21.15.2:0/2232248919 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56055fc01210 0x56055fc13d80 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.616+0000 7f4fa4ff9640 0 client.17295 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:15:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.616+0000 7f4fc61d1740 1 --2- 172.21.15.2:0/2232248919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055fc008f0 0x56055fc13840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.616+0000 7f4fc61d1740 1 -- 172.21.15.2:0/2232248919 >> 172.21.15.2:0/2232248919 conn(0x56055fbfce00 msgr2=0x56055fc16e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.616+0000 7f4fc61d1740 1 -- 172.21.15.2:0/2232248919 shutdown_connections 2024-05-14T17:15:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.616+0000 7f4fc61d1740 1 -- 172.21.15.2:0/2232248919 wait complete. 2024-05-14T17:15:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.616+0000 7f4fc61d1740 20 client.17295 trim_cache size 0 max 16384 2024-05-14T17:15:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:15:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:15:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:15:37.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.616+0000 7f4fc34bd640 1 -- 172.21.15.2:0/2488231346 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f4fa80b2a90 msgr2=0x7f4fa80b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.616+0000 7f4fc34bd640 1 --2- 172.21.15.2:0/2488231346 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f4fa80b2a90 0x7f4fa80b4f50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4fb40e0380 tx=0x7f4fb4005480 comp rx=0 tx=0).stop 2024-05-14T17:15:37.571 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.616+0000 7f4fc34bd640 1 -- 172.21.15.2:0/2488231346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4fbc108a40 msgr2=0x7f4fbc19d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.616+0000 7f4fc34bd640 1 --2- 172.21.15.2:0/2488231346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4fbc108a40 0x7f4fbc19d980 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f4fb400eb10 tx=0x7f4fb400ef10 comp rx=0 tx=0).stop 2024-05-14T17:15:37.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.616+0000 7f4fc34bd640 1 -- 172.21.15.2:0/2488231346 shutdown_connections 2024-05-14T17:15:37.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.616+0000 7f4fc34bd640 1 --2- 172.21.15.2:0/2488231346 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f4fa80b2a90 0x7f4fa80b4f50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.616+0000 7f4fc34bd640 1 --2- 172.21.15.2:0/2488231346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4fbc108a40 0x7f4fbc19d980 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.616+0000 7f4fc34bd640 1 --2- 172.21.15.2:0/2488231346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4fbc06bc20 0x7f4fbc199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.616+0000 7f4fc34bd640 1 --2- 172.21.15.2:0/2488231346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fbc06b280 0x7f4fbc199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.616+0000 7f4fc34bd640 1 -- 172.21.15.2:0/2488231346 >> 172.21.15.2:0/2488231346 conn(0x7f4fbc077750 msgr2=0x7f4fbc100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:37.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.617+0000 7f4fc34bd640 1 -- 172.21.15.2:0/2488231346 shutdown_connections 2024-05-14T17:15:37.572 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:16.617+0000 7f4fc34bd640 1 -- 172.21.15.2:0/2488231346 wait complete. 2024-05-14T17:15:37.573 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:16 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:15:37.573 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:16 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:15:37.573 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:16 smithi002 ceph-mon[31538]: pgmap v648: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 511 B/s wr, 4 op/s 2024-05-14T17:15:37.573 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:16 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:15:37.573 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:16 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:15:37.574 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:16 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:15:37.574 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:16 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:15:37.574 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:16 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:15:37.574 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:16 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:15:37.574 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:16 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:15:37.574 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:16 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:15:37.574 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:16 smithi002 ceph-mon[31538]: daemon mds.i restarted 2024-05-14T17:15:37.574 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:16 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:15:37.574 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:16 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:15:37.574 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:16 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:15:37.574 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:16 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:15:37.574 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:16 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:15:37.574 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:16 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:15:37.574 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:16 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:15:37.575 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:16 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-14T17:15:37.575 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:16 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:15:37.575 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:16 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:15:37.575 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:16 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:15:37.575 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:16 smithi002 ceph-mon[31538]: osdmap e110: 12 total, 12 up, 12 in 2024-05-14T17:15:37.575 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:17 smithi002 ceph-mon[31538]: Adjusting osd_memory_target on smithi123 to 176.2M 2024-05-14T17:15:37.575 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:17 smithi002 ceph-mon[31538]: Unable to set osd_memory_target on smithi123 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-05-14T17:15:37.575 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:17 smithi002 ceph-mon[31538]: Adjusting osd_memory_target on smithi002 to 177.2M 2024-05-14T17:15:37.575 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:17 smithi002 ceph-mon[31538]: Unable to set osd_memory_target on smithi002 to 185879142: error parsing value: Value '185879142' is below minimum 939524096 2024-05-14T17:15:37.575 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:17 smithi002 ceph-mon[31538]: Adjusting osd_memory_target on smithi151 to 176.2M 2024-05-14T17:15:37.575 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:17 smithi002 ceph-mon[31538]: Unable to set osd_memory_target on smithi151 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-05-14T17:15:37.575 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:17 smithi002 ceph-mon[31538]: pgmap v649: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 588 B/s wr, 5 op/s 2024-05-14T17:15:37.575 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:17 smithi002 ceph-mon[31538]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-05-14T17:15:37.575 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:17 smithi002 ceph-mon[31538]: mds.? [v2:172.21.15.151:6838/239059007,v1:172.21.15.151:6839/239059007] up:boot 2024-05-14T17:15:37.575 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:17 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-05-14T17:15:37.576 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:17 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T17:15:37.576 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:17 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:15:37.576 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:19 smithi002 ceph-mon[31538]: pgmap v651: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-05-14T17:15:37.576 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:21 smithi002 ceph-mon[31538]: pgmap v652: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 8 op/s 2024-05-14T17:15:37.576 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:23 smithi002 ceph-mon[31538]: pgmap v653: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.7 KiB/s wr, 5 op/s 2024-05-14T17:15:37.576 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:25 smithi002 ceph-mon[31538]: pgmap v654: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.0 KiB/s wr, 9 op/s 2024-05-14T17:15:37.576 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:27 smithi002 ceph-mon[31538]: pgmap v655: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.8 KiB/s wr, 8 op/s 2024-05-14T17:15:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.715+0000 7f28596a7640 1 -- 172.21.15.2:0/3935663878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f285410d0d0 msgr2=0x7f285410f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.715+0000 7f28596a7640 1 --2- 172.21.15.2:0/3935663878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f285410d0d0 0x7f285410f500 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f284c009ca0 tx=0x7f284c02a950 comp rx=0 tx=0).stop 2024-05-14T17:15:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.716+0000 7f28596a7640 1 -- 172.21.15.2:0/3935663878 shutdown_connections 2024-05-14T17:15:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.716+0000 7f28596a7640 1 --2- 172.21.15.2:0/3935663878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f285410d0d0 0x7f285410f500 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.716+0000 7f28596a7640 1 --2- 172.21.15.2:0/3935663878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2854104620 0x7f285410cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.716+0000 7f28596a7640 1 --2- 172.21.15.2:0/3935663878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2854103d00 0x7f28541040e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.716+0000 7f28596a7640 1 -- 172.21.15.2:0/3935663878 >> 172.21.15.2:0/3935663878 conn(0x7f2854077cd0 msgr2=0x7f28540780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.716+0000 7f28596a7640 1 -- 172.21.15.2:0/3935663878 shutdown_connections 2024-05-14T17:15:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.716+0000 7f28596a7640 1 -- 172.21.15.2:0/3935663878 wait complete. 2024-05-14T17:15:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.716+0000 7f28596a7640 1 Processor -- start 2024-05-14T17:15:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.717+0000 7f28596a7640 1 -- start start 2024-05-14T17:15:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.717+0000 7f28596a7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2854103d00 0x7f2854199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.717+0000 7f28596a7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2854104620 0x7f2854199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.717+0000 7f28596a7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f285410d0d0 0x7f285419d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.717+0000 7f28596a7640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2854112d10 con 0x7f2854103d00 2024-05-14T17:15:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.717+0000 7f28596a7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2854112b90 con 0x7f285410d0d0 2024-05-14T17:15:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.717+0000 7f28596a7640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2854112e90 con 0x7f2854104620 2024-05-14T17:15:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.717+0000 7f2852ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2854103d00 0x7f2854199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.717+0000 7f2852ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2854103d00 0x7f2854199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45780/0 (socket says 172.21.15.2:45780) 2024-05-14T17:15:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.717+0000 7f2852ffd640 1 -- 172.21.15.2:0/1213591670 learned_addr learned my addr 172.21.15.2:0/1213591670 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.717+0000 7f28527fc640 1 --2- 172.21.15.2:0/1213591670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2854104620 0x7f2854199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.717+0000 7f28537fe640 1 --2- 172.21.15.2:0/1213591670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f285410d0d0 0x7f285419d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.717+0000 7f2852ffd640 1 -- 172.21.15.2:0/1213591670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2854104620 msgr2=0x7f2854199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.717+0000 7f2852ffd640 1 --2- 172.21.15.2:0/1213591670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2854104620 0x7f2854199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.717+0000 7f2852ffd640 1 -- 172.21.15.2:0/1213591670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f285410d0d0 msgr2=0x7f285419d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.717+0000 7f2852ffd640 1 --2- 172.21.15.2:0/1213591670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f285410d0d0 0x7f285419d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.717+0000 7f2852ffd640 1 -- 172.21.15.2:0/1213591670 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f285419e0d0 con 0x7f2854103d00 2024-05-14T17:15:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.717+0000 7f28537fe640 1 --2- 172.21.15.2:0/1213591670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f285410d0d0 0x7f285419d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:15:37.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.717+0000 7f2852ffd640 1 --2- 172.21.15.2:0/1213591670 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2854103d00 0x7f2854199130 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f284400e9f0 tx=0x7f284400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:37.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.718+0000 7f283bfff640 1 -- 172.21.15.2:0/1213591670 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284400cd80 con 0x7f2854103d00 2024-05-14T17:15:37.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.718+0000 7f28596a7640 1 -- 172.21.15.2:0/1213591670 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f285419e3c0 con 0x7f2854103d00 2024-05-14T17:15:37.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.718+0000 7f283bfff640 1 -- 172.21.15.2:0/1213591670 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28440108e0 con 0x7f2854103d00 2024-05-14T17:15:37.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.718+0000 7f283bfff640 1 -- 172.21.15.2:0/1213591670 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2844018980 con 0x7f2854103d00 2024-05-14T17:15:37.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.718+0000 7f28596a7640 1 -- 172.21.15.2:0/1213591670 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28541a5c50 con 0x7f2854103d00 2024-05-14T17:15:37.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.721+0000 7f285c3c7740 1 Processor -- start 2024-05-14T17:15:37.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.721+0000 7f285c3c7740 1 -- start start 2024-05-14T17:15:37.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.721+0000 7f283bfff640 1 -- 172.21.15.2:0/1213591670 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f2844010430 con 0x7f2854103d00 2024-05-14T17:15:37.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.721+0000 7f285c3c7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564de66ed8f0 0x564de66edcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.721+0000 7f285c3c7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564de66ee210 0x564de66f8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.721+0000 7f285c3c7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564de66f8ad0 0x564de66faec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.721+0000 7f285c3c7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x564de6660c30 con 0x564de66ed8f0 2024-05-14T17:15:37.587 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.721+0000 7f285c3c7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564de66507d0 con 0x564de66ee210 2024-05-14T17:15:37.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.721+0000 7f285c3c7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564de6654810 con 0x564de66f8ad0 2024-05-14T17:15:37.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.722+0000 7f28537fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564de66ed8f0 0x564de66edcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.722+0000 7f28537fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564de66ed8f0 0x564de66edcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45784/0 (socket says 172.21.15.2:45784) 2024-05-14T17:15:37.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.722+0000 7f28537fe640 1 -- 172.21.15.2:0/2901811597 learned_addr learned my addr 172.21.15.2:0/2901811597 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:37.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.722+0000 7f2852ffd640 1 --2- 172.21.15.2:0/2901811597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564de66ee210 0x564de66f8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.722+0000 7f28527fc640 1 --2- 172.21.15.2:0/2901811597 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564de66f8ad0 0x564de66faec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.722+0000 7f28537fe640 1 -- 172.21.15.2:0/2901811597 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564de66f8ad0 msgr2=0x564de66faec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.722+0000 7f28537fe640 1 --2- 172.21.15.2:0/2901811597 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564de66f8ad0 0x564de66faec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.722+0000 7f28537fe640 1 -- 172.21.15.2:0/2901811597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564de66ee210 msgr2=0x564de66f8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.722+0000 7f28537fe640 1 --2- 172.21.15.2:0/2901811597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564de66ee210 0x564de66f8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.722+0000 7f28537fe640 1 -- 172.21.15.2:0/2901811597 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564de6365210 con 0x564de66ed8f0 2024-05-14T17:15:37.588 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.722+0000 7f28527fc640 1 --2- 172.21.15.2:0/2901811597 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564de66f8ad0 0x564de66faec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:15:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.722+0000 7f28537fe640 1 --2- 172.21.15.2:0/2901811597 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564de66ed8f0 0x564de66edcd0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f284c02af40 tx=0x7f284c009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc86f5584e92a7c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.722+0000 7f283bfff640 1 --2- 172.21.15.2:0/1213591670 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f283c0b2a10 0x7f283c0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.722+0000 7f28397fa640 1 -- 172.21.15.2:0/2901811597 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c034050 con 0x564de66ed8f0 2024-05-14T17:15:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.722+0000 7f28397fa640 1 -- 172.21.15.2:0/2901811597 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f284c02fec0 con 0x564de66ed8f0 2024-05-14T17:15:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.723+0000 7f28397fa640 1 -- 172.21.15.2:0/2901811597 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c004280 con 0x564de66ed8f0 2024-05-14T17:15:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.723+0000 7f2852ffd640 1 --2- 172.21.15.2:0/1213591670 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f283c0b2a10 0x7f283c0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.723+0000 7f283bfff640 1 -- 172.21.15.2:0/1213591670 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8469+0+0 (secure 0 0 0) 0x7f28440e29c0 con 0x7f2854103d00 2024-05-14T17:15:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.723+0000 7f285c3c7740 1 -- 172.21.15.2:0/2901811597 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564de66ed8f0 msgr2=0x564de66edcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.723+0000 7f2852ffd640 1 --2- 172.21.15.2:0/1213591670 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f283c0b2a10 0x7f283c0b4ed0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f28440e01f0 tx=0x7f284400cf20 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.723+0000 7f285c3c7740 1 --2- 172.21.15.2:0/2901811597 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564de66ed8f0 0x564de66edcd0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f284c02af40 tx=0x7f284c009a20 comp rx=0 tx=0).stop 2024-05-14T17:15:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.723+0000 7f285c3c7740 1 -- 172.21.15.2:0/2901811597 shutdown_connections 2024-05-14T17:15:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.723+0000 7f285c3c7740 1 --2- 172.21.15.2:0/2901811597 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564de66f8ad0 0x564de66faec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.723+0000 7f285c3c7740 1 --2- 172.21.15.2:0/2901811597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564de66ee210 0x564de66f8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.723+0000 7f285c3c7740 1 --2- 172.21.15.2:0/2901811597 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564de66ed8f0 0x564de66edcd0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.723+0000 7f285c3c7740 1 -- 172.21.15.2:0/2901811597 >> 172.21.15.2:0/2901811597 conn(0x564de66e9e00 msgr2=0x564de66ea1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.723+0000 7f285c3c7740 1 -- 172.21.15.2:0/2901811597 shutdown_connections 2024-05-14T17:15:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.723+0000 7f285c3c7740 1 -- 172.21.15.2:0/2901811597 wait complete. 2024-05-14T17:15:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.724+0000 7f285c3c7740 1 -- start start 2024-05-14T17:15:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.724+0000 7f285c3c7740 1 Processor -- start 2024-05-14T17:15:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.724+0000 7f285c3c7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564de66ed8f0 0x564de66f8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.724+0000 7f285c3c7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564de66ee210 0x564de66f5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.724+0000 7f285c3c7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564de66f8ad0 0x564de66f5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.724+0000 7f285c3c7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x564de6336790 con 0x564de66ed8f0 2024-05-14T17:15:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.724+0000 7f285c3c7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564de660c890 con 0x564de66f8ad0 2024-05-14T17:15:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.724+0000 7f285c3c7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564de66fb740 con 0x564de66ee210 2024-05-14T17:15:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.724+0000 7f285c3c7740 10 client.0 _ll_register_callbacks cb 0x564de660fd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:15:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.724+0000 7f28537fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564de66ed8f0 0x564de66f8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.724+0000 7f28537fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564de66ed8f0 0x564de66f8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45792/0 (socket says 172.21.15.2:45792) 2024-05-14T17:15:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.724+0000 7f28537fe640 1 -- 172.21.15.2:0/4206384542 learned_addr learned my addr 172.21.15.2:0/4206384542 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.725+0000 7f28527fc640 1 --2- 172.21.15.2:0/4206384542 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564de66ee210 0x564de66f5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.725+0000 7f28537fe640 1 -- 172.21.15.2:0/4206384542 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564de66ee210 msgr2=0x564de66f5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.725+0000 7f28537fe640 1 --2- 172.21.15.2:0/4206384542 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564de66ee210 0x564de66f5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.725+0000 7f28537fe640 1 -- 172.21.15.2:0/4206384542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564de66f8ad0 msgr2=0x564de66f5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.725+0000 7f28537fe640 1 --2- 172.21.15.2:0/4206384542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564de66f8ad0 0x564de66f5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.725+0000 7f28537fe640 1 -- 172.21.15.2:0/4206384542 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564de6431620 con 0x564de66ed8f0 2024-05-14T17:15:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.725+0000 7f28537fe640 1 --2- 172.21.15.2:0/4206384542 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564de66ed8f0 0x564de66f8480 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f284c02af10 tx=0x7f284c02f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f34f8c300c58db server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.725+0000 7f2836ff5640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:15:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.725+0000 7f2836ff5640 1 -- 172.21.15.2:0/4206384542 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c003dc0 con 0x564de66ed8f0 2024-05-14T17:15:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.725+0000 7f2836ff5640 1 -- 172.21.15.2:0/4206384542 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8469+0+0 (secure 0 0 0) 0x7f284c004040 con 0x564de66ed8f0 2024-05-14T17:15:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.725+0000 7f285c3c7740 1 -- 172.21.15.2:0/4206384542 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f284c02fec0 con 0x564de66ed8f0 2024-05-14T17:15:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.725+0000 7f2836ff5640 1 -- 172.21.15.2:0/4206384542 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f284c059050 con 0x564de66ed8f0 2024-05-14T17:15:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.725+0000 7f285c3c7740 10 client.17307 fetch_fsmap learned FSMap version 59 2024-05-14T17:15:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.725+0000 7f285c3c7740 1 -- 172.21.15.2:0/4206384542 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x564de6707110 con 0x564de66ed8f0 2024-05-14T17:15:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.725+0000 7f2836ff5640 1 -- 172.21.15.2:0/4206384542 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f284c02fec0 con 0x564de66ed8f0 2024-05-14T17:15:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.726+0000 7f2836ff5640 1 -- 172.21.15.2:0/4206384542 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 59) ==== 8350+0+0 (secure 0 0 0) 0x7f284c041070 con 0x564de66ed8f0 2024-05-14T17:15:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.726+0000 7f285c3c7740 10 client.17307 fetch_fsmap finished waiting for FSMap version 59 2024-05-14T17:15:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.726+0000 7f285c3c7740 10 client.17307 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:15:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.726+0000 7f285c3c7740 20 client.17307 populate_metadata read hostname 'smithi002' 2024-05-14T17:15:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.726+0000 7f285c3c7740 1 --2- 172.21.15.2:0/4206384542 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564de6708460 0x564de670a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.726+0000 7f285c3c7740 4 client.17307 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:15:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.726+0000 7f285c3c7740 1 -- 172.21.15.2:0/4206384542 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564de6662b80 con 0x564de6708460 2024-05-14T17:15:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.726+0000 7f28527fc640 1 --2- 172.21.15.2:0/4206384542 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564de6708460 0x564de670a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.726+0000 7f2836ff5640 10 client.17307 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:15:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.726+0000 7f28527fc640 1 --2- 172.21.15.2:0/4206384542 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564de6708460 0x564de670a850 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9931e998d2fb058 server_cookie=71380595e50f1bbd in_seq=0 out_seq=0 2024-05-14T17:15:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.727+0000 7f2836ff5640 1 -- 172.21.15.2:0/4206384542 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564de6662b80 con 0x564de6708460 2024-05-14T17:15:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.727+0000 7f2836ff5640 10 client.17307 handle_command_reply: tid=0 2024-05-14T17:15:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.727+0000 7f285c3c7740 1 client.17307 shutdown 2024-05-14T17:15:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.727+0000 7f285c3c7740 1 -- 172.21.15.2:0/4206384542 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564de66ed8f0 msgr2=0x564de66f8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.727+0000 7f285c3c7740 1 --2- 172.21.15.2:0/4206384542 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564de66ed8f0 0x564de66f8480 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f284c02af10 tx=0x7f284c02f5d0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.727+0000 7f285c3c7740 1 -- 172.21.15.2:0/4206384542 shutdown_connections 2024-05-14T17:15:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.727+0000 7f285c3c7740 1 --2- 172.21.15.2:0/4206384542 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564de6708460 0x564de670a850 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.727+0000 7f285c3c7740 1 --2- 172.21.15.2:0/4206384542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564de66f8ad0 0x564de66f5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.727+0000 7f285c3c7740 1 --2- 172.21.15.2:0/4206384542 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564de66ee210 0x564de66f5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.727+0000 7f285c3c7740 1 --2- 172.21.15.2:0/4206384542 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564de66ed8f0 0x564de66f8480 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.727+0000 7f2836ff5640 0 client.17307 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:15:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.727+0000 7f285c3c7740 1 -- 172.21.15.2:0/4206384542 >> 172.21.15.2:0/4206384542 conn(0x564de66e9e00 msgr2=0x564de66e9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.727+0000 7f285c3c7740 1 -- 172.21.15.2:0/4206384542 shutdown_connections 2024-05-14T17:15:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.727+0000 7f285c3c7740 1 -- 172.21.15.2:0/4206384542 wait complete. 2024-05-14T17:15:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.727+0000 7f285c3c7740 20 client.17307 trim_cache size 0 max 16384 2024-05-14T17:15:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.743+0000 7f285c3c7740 1 Processor -- start 2024-05-14T17:15:37.594 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:29 smithi002 ceph-mon[31538]: pgmap v656: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.5 KiB/s wr, 8 op/s 2024-05-14T17:15:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.743+0000 7f285c3c7740 1 -- start start 2024-05-14T17:15:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.743+0000 7f285c3c7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564de66ed8f0 0x564de6704610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.743+0000 7f285c3c7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564de66ee210 0x564de66f7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.743+0000 7f285c3c7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564de66f8ad0 0x564de670feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.743+0000 7f285c3c7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f28541b0db0 con 0x564de66ee210 2024-05-14T17:15:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.743+0000 7f285c3c7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f28541ae650 con 0x564de66f8ad0 2024-05-14T17:15:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.743+0000 7f285c3c7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f28541ac4a0 con 0x564de66ed8f0 2024-05-14T17:15:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.743+0000 7f28527fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564de66ee210 0x564de66f7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.743+0000 7f28527fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564de66ee210 0x564de66f7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45804/0 (socket says 172.21.15.2:45804) 2024-05-14T17:15:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.743+0000 7f28527fc640 1 -- 172.21.15.2:0/1814366042 learned_addr learned my addr 172.21.15.2:0/1814366042 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.743+0000 7f28537fe640 1 --2- 172.21.15.2:0/1814366042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564de66ed8f0 0x564de6704610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.743+0000 7f28537fe640 1 --2- 172.21.15.2:0/1814366042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564de66f8ad0 0x564de670feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.743+0000 7f28527fc640 1 -- 172.21.15.2:0/1814366042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564de66ed8f0 msgr2=0x564de6704610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.743+0000 7f28527fc640 1 --2- 172.21.15.2:0/1814366042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564de66ed8f0 0x564de6704610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.743+0000 7f28527fc640 1 -- 172.21.15.2:0/1814366042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564de66f8ad0 msgr2=0x564de670feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.743+0000 7f28527fc640 1 --2- 172.21.15.2:0/1814366042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564de66f8ad0 0x564de670feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.743+0000 7f28527fc640 1 -- 172.21.15.2:0/1814366042 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564de66f6080 con 0x564de66ee210 2024-05-14T17:15:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.743+0000 7f28537fe640 1 --2- 172.21.15.2:0/1814366042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564de66f8ad0 0x564de670feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:15:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.744+0000 7f28527fc640 1 --2- 172.21.15.2:0/1814366042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564de66ee210 0x564de66f7fb0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x564de66eb220 tx=0x7f284800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b3be41f652228f server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.744+0000 7f2838ff9640 1 -- 172.21.15.2:0/1814366042 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2848003d20 con 0x564de66ee210 2024-05-14T17:15:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.744+0000 7f2838ff9640 1 -- 172.21.15.2:0/1814366042 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2848003ec0 con 0x564de66ee210 2024-05-14T17:15:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.744+0000 7f2838ff9640 1 -- 172.21.15.2:0/1814366042 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2848012990 con 0x564de66ee210 2024-05-14T17:15:37.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.744+0000 7f285c3c7740 1 -- 172.21.15.2:0/1814366042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564de66ee210 msgr2=0x564de66f7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.744+0000 7f285c3c7740 1 --2- 172.21.15.2:0/1814366042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564de66ee210 0x564de66f7fb0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x564de66eb220 tx=0x7f284800b390 comp rx=0 tx=0).stop 2024-05-14T17:15:37.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.744+0000 7f285c3c7740 1 -- 172.21.15.2:0/1814366042 shutdown_connections 2024-05-14T17:15:37.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.744+0000 7f285c3c7740 1 --2- 172.21.15.2:0/1814366042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564de66f8ad0 0x564de670feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.744+0000 7f285c3c7740 1 --2- 172.21.15.2:0/1814366042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564de66ee210 0x564de66f7fb0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.744+0000 7f285c3c7740 1 --2- 172.21.15.2:0/1814366042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564de66ed8f0 0x564de6704610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.744+0000 7f285c3c7740 1 -- 172.21.15.2:0/1814366042 >> 172.21.15.2:0/1814366042 conn(0x564de66e9e00 msgr2=0x564de66f0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:37.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.744+0000 7f285c3c7740 1 -- 172.21.15.2:0/1814366042 shutdown_connections 2024-05-14T17:15:37.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.744+0000 7f285c3c7740 1 -- 172.21.15.2:0/1814366042 wait complete. 2024-05-14T17:15:37.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.745+0000 7f285c3c7740 1 -- start start 2024-05-14T17:15:37.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.745+0000 7f285c3c7740 1 Processor -- start 2024-05-14T17:15:37.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.745+0000 7f285c3c7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564de66ed8f0 0x564de66fc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.602 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.745+0000 7f285c3c7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564de66ee210 0x564de66fcc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.745+0000 7f285c3c7740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564de66f8ad0 0x564de66fd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.745+0000 7f285c3c7740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f28541acb10 con 0x564de66f8ad0 2024-05-14T17:15:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.745+0000 7f285c3c7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f28541adf80 con 0x564de66ee210 2024-05-14T17:15:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.745+0000 7f285c3c7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f28541ad1e0 con 0x564de66ed8f0 2024-05-14T17:15:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.745+0000 7f285c3c7740 10 client.0 _ll_register_callbacks cb 0x7f284c009840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:15:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.745+0000 7f28537fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564de66f8ad0 0x564de66fd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.745+0000 7f28537fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564de66f8ad0 0x564de66fd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:45814/0 (socket says 172.21.15.2:45814) 2024-05-14T17:15:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.745+0000 7f28537fe640 1 -- 172.21.15.2:0/2505389024 learned_addr learned my addr 172.21.15.2:0/2505389024 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.745+0000 7f28527fc640 1 --2- 172.21.15.2:0/2505389024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564de66ee210 0x564de66fcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.745+0000 7f28537fe640 1 -- 172.21.15.2:0/2505389024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564de66ed8f0 msgr2=0x564de66fc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.745+0000 7f28537fe640 1 --2- 172.21.15.2:0/2505389024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564de66ed8f0 0x564de66fc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.745+0000 7f28537fe640 1 -- 172.21.15.2:0/2505389024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564de66ee210 msgr2=0x564de66fcc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.745+0000 7f28537fe640 1 --2- 172.21.15.2:0/2505389024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564de66ee210 0x564de66fcc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.746+0000 7f28537fe640 1 -- 172.21.15.2:0/2505389024 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564de6771c10 con 0x564de66f8ad0 2024-05-14T17:15:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.746+0000 7f28537fe640 1 --2- 172.21.15.2:0/2505389024 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564de66f8ad0 0x564de66fd140 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f284c043b40 tx=0x7f284c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed061eef259d0359 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.746+0000 7f2836ff5640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:15:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.746+0000 7f2836ff5640 1 -- 172.21.15.2:0/2505389024 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c034070 con 0x564de66f8ad0 2024-05-14T17:15:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.746+0000 7f2836ff5640 1 -- 172.21.15.2:0/2505389024 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8469+0+0 (secure 0 0 0) 0x7f284c02f640 con 0x564de66f8ad0 2024-05-14T17:15:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.746+0000 7f285c3c7740 1 -- 172.21.15.2:0/2505389024 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2848003ec0 con 0x564de66f8ad0 2024-05-14T17:15:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.746+0000 7f2836ff5640 1 -- 172.21.15.2:0/2505389024 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f284c043bc0 con 0x564de66f8ad0 2024-05-14T17:15:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.746+0000 7f285c3c7740 10 client.17313 fetch_fsmap learned FSMap version 59 2024-05-14T17:15:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.746+0000 7f285c3c7740 1 -- 172.21.15.2:0/2505389024 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x564de6772ee0 con 0x564de66f8ad0 2024-05-14T17:15:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.746+0000 7f2836ff5640 1 -- 172.21.15.2:0/2505389024 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2848003ec0 con 0x564de66f8ad0 2024-05-14T17:15:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.746+0000 7f2836ff5640 1 -- 172.21.15.2:0/2505389024 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 59) ==== 8350+0+0 (secure 0 0 0) 0x7f284c041040 con 0x564de66f8ad0 2024-05-14T17:15:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.746+0000 7f285c3c7740 10 client.17313 fetch_fsmap finished waiting for FSMap version 59 2024-05-14T17:15:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.746+0000 7f285c3c7740 10 client.17313 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:15:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.746+0000 7f285c3c7740 20 client.17313 populate_metadata read hostname 'smithi002' 2024-05-14T17:15:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.746+0000 7f285c3c7740 1 --2- 172.21.15.2:0/2505389024 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564de67735b0 0x564de67759d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.747+0000 7f285c3c7740 4 client.17313 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:15:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.747+0000 7f285c3c7740 1 -- 172.21.15.2:0/2505389024 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564de665c360 con 0x564de67735b0 2024-05-14T17:15:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.747+0000 7f28527fc640 1 --2- 172.21.15.2:0/2505389024 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564de67735b0 0x564de67759d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.747+0000 7f28527fc640 1 --2- 172.21.15.2:0/2505389024 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564de67735b0 0x564de67759d0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f49667eb2aab17f3 server_cookie=21cf2f71cb2b7149 in_seq=0 out_seq=0 2024-05-14T17:15:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.747+0000 7f2836ff5640 10 client.17313 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:15:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.752+0000 7f2836ff5640 1 -- 172.21.15.2:0/2505389024 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564de665c360 con 0x564de67735b0 2024-05-14T17:15:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.752+0000 7f2836ff5640 10 client.17313 handle_command_reply: tid=0 2024-05-14T17:15:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.752+0000 7f285c3c7740 1 client.17313 shutdown 2024-05-14T17:15:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.752+0000 7f285c3c7740 1 -- 172.21.15.2:0/2505389024 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564de66f8ad0 msgr2=0x564de66fd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.752+0000 7f285c3c7740 1 --2- 172.21.15.2:0/2505389024 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564de66f8ad0 0x564de66fd140 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f284c043b40 tx=0x7f284c000f40 comp rx=0 tx=0).stop 2024-05-14T17:15:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.752+0000 7f285c3c7740 1 -- 172.21.15.2:0/2505389024 shutdown_connections 2024-05-14T17:15:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.752+0000 7f285c3c7740 1 --2- 172.21.15.2:0/2505389024 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x564de67735b0 0x564de67759d0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.752+0000 7f285c3c7740 1 --2- 172.21.15.2:0/2505389024 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564de66f8ad0 0x564de66fd140 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.752+0000 7f285c3c7740 1 --2- 172.21.15.2:0/2505389024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564de66ee210 0x564de66fcc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.752+0000 7f285c3c7740 1 --2- 172.21.15.2:0/2505389024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564de66ed8f0 0x564de66fc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.752+0000 7f2836ff5640 0 client.17313 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:15:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.752+0000 7f285c3c7740 1 -- 172.21.15.2:0/2505389024 >> 172.21.15.2:0/2505389024 conn(0x564de66e9e00 msgr2=0x564de66ef210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.752+0000 7f285c3c7740 1 -- 172.21.15.2:0/2505389024 shutdown_connections 2024-05-14T17:15:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.752+0000 7f285c3c7740 1 -- 172.21.15.2:0/2505389024 wait complete. 2024-05-14T17:15:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.753+0000 7f285c3c7740 20 client.17313 trim_cache size 0 max 16384 2024-05-14T17:15:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:15:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:15:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:15:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.753+0000 7f28596a7640 1 -- 172.21.15.2:0/1213591670 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f283c0b2a10 msgr2=0x7f283c0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.753+0000 7f28596a7640 1 --2- 172.21.15.2:0/1213591670 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f283c0b2a10 0x7f283c0b4ed0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f28440e01f0 tx=0x7f284400cf20 comp rx=0 tx=0).stop 2024-05-14T17:15:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.753+0000 7f28596a7640 1 -- 172.21.15.2:0/1213591670 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2854103d00 msgr2=0x7f2854199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.753+0000 7f28596a7640 1 --2- 172.21.15.2:0/1213591670 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2854103d00 0x7f2854199130 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f284400e9f0 tx=0x7f284400eeb0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.753+0000 7f28596a7640 1 -- 172.21.15.2:0/1213591670 shutdown_connections 2024-05-14T17:15:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.753+0000 7f28596a7640 1 --2- 172.21.15.2:0/1213591670 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f283c0b2a10 0x7f283c0b4ed0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.753+0000 7f28596a7640 1 --2- 172.21.15.2:0/1213591670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f285410d0d0 0x7f285419d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.753+0000 7f28596a7640 1 --2- 172.21.15.2:0/1213591670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2854104620 0x7f2854199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.753+0000 7f28596a7640 1 --2- 172.21.15.2:0/1213591670 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2854103d00 0x7f2854199130 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.753+0000 7f28596a7640 1 -- 172.21.15.2:0/1213591670 >> 172.21.15.2:0/1213591670 conn(0x7f2854077cd0 msgr2=0x7f28540ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.753+0000 7f28596a7640 1 -- 172.21.15.2:0/1213591670 shutdown_connections 2024-05-14T17:15:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:29.753+0000 7f28596a7640 1 -- 172.21.15.2:0/1213591670 wait complete. 2024-05-14T17:15:37.609 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:31 smithi002 ceph-mon[31538]: pgmap v657: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-14T17:15:37.610 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:33 smithi002 ceph-mon[31538]: pgmap v658: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-14T17:15:37.610 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:34 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:15:37.610 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:35 smithi002 ceph-mon[31538]: pgmap v659: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.9 KiB/s wr, 8 op/s 2024-05-14T17:15:37.837 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:15:37.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:37 smithi151 ceph-mon[36066]: pgmap v660: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-05-14T17:15:37.910 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:37 smithi123 ceph-mon[35971]: pgmap v660: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-05-14T17:15:37.960 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:37 smithi002 ceph-mon[31538]: pgmap v660: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-05-14T17:15:39.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.567+0000 7f12fc365700 1 -- 172.21.15.2:0/3651065726 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12f4102b10 msgr2=0x7f12f41068d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:39.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.567+0000 7f12fc365700 1 --2- 172.21.15.2:0/3651065726 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12f4102b10 0x7f12f41068d0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f12e4009990 tx=0x7f12e4009ca0 comp rx=0 tx=0).stop 2024-05-14T17:15:39.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.567+0000 7f12fc365700 1 -- 172.21.15.2:0/3651065726 shutdown_connections 2024-05-14T17:15:39.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.567+0000 7f12fc365700 1 --2- 172.21.15.2:0/3651065726 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12f4107000 0x7f12f410eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:39.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.567+0000 7f12fc365700 1 --2- 172.21.15.2:0/3651065726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12f4102160 0x7f12f4102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:39.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.567+0000 7f12fc365700 1 --2- 172.21.15.2:0/3651065726 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12f4102b10 0x7f12f41068d0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:39.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.567+0000 7f12fc365700 1 -- 172.21.15.2:0/3651065726 >> 172.21.15.2:0/3651065726 conn(0x7f12f4077780 msgr2=0x7f12f4077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:39.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.567+0000 7f12fc365700 1 -- 172.21.15.2:0/3651065726 shutdown_connections 2024-05-14T17:15:39.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.567+0000 7f12fc365700 1 -- 172.21.15.2:0/3651065726 wait complete. 2024-05-14T17:15:39.568 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:39 smithi002 ceph-mon[31538]: pgmap v661: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-05-14T17:15:39.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.568+0000 7f12fc365700 1 Processor -- start 2024-05-14T17:15:39.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.568+0000 7f12fc365700 1 -- start start 2024-05-14T17:15:39.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.568+0000 7f12fc365700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12f4102160 0x7f12f419e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:39.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.568+0000 7f12fc365700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12f4102b10 0x7f12f419ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:39.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.568+0000 7f12fc365700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12f4107000 0x7f12f4198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:39.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.568+0000 7f12fc365700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f12f41126c0 con 0x7f12f4102160 2024-05-14T17:15:39.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.568+0000 7f12fc365700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f12f4112540 con 0x7f12f4107000 2024-05-14T17:15:39.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.568+0000 7f12fc365700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f12f4112840 con 0x7f12f4102b10 2024-05-14T17:15:39.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.568+0000 7f12fa101700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12f4102160 0x7f12f419e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:39.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.568+0000 7f12fa101700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12f4102160 0x7f12f419e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52290/0 (socket says 172.21.15.2:52290) 2024-05-14T17:15:39.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.568+0000 7f12fa101700 1 -- 172.21.15.2:0/1965791663 learned_addr learned my addr 172.21.15.2:0/1965791663 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:39.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.568+0000 7f12fa902700 1 --2- 172.21.15.2:0/1965791663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12f4107000 0x7f12f4198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:39.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.569+0000 7f12f9900700 1 --2- 172.21.15.2:0/1965791663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12f4102b10 0x7f12f419ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:39.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.569+0000 7f12fa101700 1 -- 172.21.15.2:0/1965791663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12f4102b10 msgr2=0x7f12f419ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:39.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.569+0000 7f12fa101700 1 --2- 172.21.15.2:0/1965791663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12f4102b10 0x7f12f419ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:39.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.569+0000 7f12fa101700 1 -- 172.21.15.2:0/1965791663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12f4107000 msgr2=0x7f12f4198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:39.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.569+0000 7f12fa101700 1 --2- 172.21.15.2:0/1965791663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12f4107000 0x7f12f4198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:39.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.569+0000 7f12fa101700 1 -- 172.21.15.2:0/1965791663 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12f41990c0 con 0x7f12f4102160 2024-05-14T17:15:39.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.569+0000 7f12fa101700 1 --2- 172.21.15.2:0/1965791663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12f4102160 0x7f12f419e7a0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f12f000c920 tx=0x7f12f000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:39.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.569+0000 7f12eb7fe700 1 -- 172.21.15.2:0/1965791663 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f0004a30 con 0x7f12f4102160 2024-05-14T17:15:39.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.569+0000 7f12fc365700 1 -- 172.21.15.2:0/1965791663 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12f41993b0 con 0x7f12f4102160 2024-05-14T17:15:39.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.569+0000 7f12eb7fe700 1 -- 172.21.15.2:0/1965791663 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12f00054d0 con 0x7f12f4102160 2024-05-14T17:15:39.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.569+0000 7f12eb7fe700 1 -- 172.21.15.2:0/1965791663 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f0007c70 con 0x7f12f4102160 2024-05-14T17:15:39.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.569+0000 7f12fc365700 1 -- 172.21.15.2:0/1965791663 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12f410f430 con 0x7f12f4102160 2024-05-14T17:15:39.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.571+0000 7f12fc365700 1 -- 172.21.15.2:0/1965791663 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12f406a230 con 0x7f12f4102160 2024-05-14T17:15:39.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.572+0000 7f12eb7fe700 1 -- 172.21.15.2:0/1965791663 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f12f0004c60 con 0x7f12f4102160 2024-05-14T17:15:39.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.573+0000 7f12eb7fe700 1 --2- 172.21.15.2:0/1965791663 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f12e00b29f0 0x7f12e00b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:39.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.573+0000 7f12f9900700 1 --2- 172.21.15.2:0/1965791663 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f12e00b29f0 0x7f12e00b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:39.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.573+0000 7f12eb7fe700 1 -- 172.21.15.2:0/1965791663 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8469+0+0 (secure 0 0 0) 0x7f12f00e24b0 con 0x7f12f4102160 2024-05-14T17:15:39.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.574+0000 7f12f9900700 1 --2- 172.21.15.2:0/1965791663 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f12e00b29f0 0x7f12e00b4eb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f12e40059d0 tx=0x7f12e4005960 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:39.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.578+0000 7f12eb7fe700 1 -- 172.21.15.2:0/1965791663 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f12f0014030 con 0x7f12f4102160 2024-05-14T17:15:39.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.843+0000 7f12fc365700 1 -- 172.21.15.2:0/1965791663 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f12f4199e90 con 0x7f12f4102160 2024-05-14T17:15:39.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.844+0000 7f12eb7fe700 1 -- 172.21.15.2:0/1965791663 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+11850 (secure 0 0 0) 0x7f12f00a9ec0 con 0x7f12f4102160 2024-05-14T17:15:39.844 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:15:39.844 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":60,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2275696923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2275696923},{"type":"v1","addr":"172.21.15.2:6835","nonce":2275696923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/716718761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":716718761},{"type":"v1","addr":"172.21.15.2:6839","nonce":716718761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/3644815179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3644815179},{"type":"v1","addr":"172.21.15.2:6841","nonce":3644815179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1880336731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1880336731},{"type":"v1","addr":"172.21.15.151:6833","nonce":1880336731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2415605548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2415605548},{"type":"v1","addr":"172.21.15.2:6837","nonce":2415605548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2635167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2635167},{"type":"v1","addr":"172.21.15.123:6833","nonce":2635167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":60,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:51:34.106919+0000","modified":"2024-05-14T17:15:33.480957+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24455,"mds_2":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24415":{"gid":24415,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/762044339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":762044339},{"type":"v1","addr":"172.21.15.151:6835","nonce":762044339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/132043151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":132043151},{"type":"v1","addr":"172.21.15.151:6837","nonce":132043151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.123:6835/1198670013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1198670013},{"type":"v1","addr":"172.21.15.123:6835","nonce":1198670013}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/333426645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":333426645},{"type":"v1","addr":"172.21.15.123:6837","nonce":333426645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/580275877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":580275877},{"type":"v1","addr":"172.21.15.123:6839","nonce":580275877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26800":{"gid":26800,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/239059007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":239059007},{"type":"v1","addr":"172.21.15.151:6839","nonce":239059007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24455,24479,24431]},"id":1}]} 2024-05-14T17:15:39.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.846+0000 7f12fc365700 1 -- 172.21.15.2:0/1965791663 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f12e00b29f0 msgr2=0x7f12e00b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:39.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.846+0000 7f12fc365700 1 --2- 172.21.15.2:0/1965791663 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f12e00b29f0 0x7f12e00b4eb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f12e40059d0 tx=0x7f12e4005960 comp rx=0 tx=0).stop 2024-05-14T17:15:39.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.846+0000 7f12fc365700 1 -- 172.21.15.2:0/1965791663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12f4102160 msgr2=0x7f12f419e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:39.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.846+0000 7f12fc365700 1 --2- 172.21.15.2:0/1965791663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12f4102160 0x7f12f419e7a0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f12f000c920 tx=0x7f12f000cc30 comp rx=0 tx=0).stop 2024-05-14T17:15:39.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.847+0000 7f12fc365700 1 -- 172.21.15.2:0/1965791663 shutdown_connections 2024-05-14T17:15:39.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.847+0000 7f12fc365700 1 --2- 172.21.15.2:0/1965791663 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f12e00b29f0 0x7f12e00b4eb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:39.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.847+0000 7f12fc365700 1 --2- 172.21.15.2:0/1965791663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12f4107000 0x7f12f4198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:39.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.847+0000 7f12fc365700 1 --2- 172.21.15.2:0/1965791663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12f4102160 0x7f12f419e7a0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:39.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.847+0000 7f12fc365700 1 --2- 172.21.15.2:0/1965791663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12f4102b10 0x7f12f419ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:39.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.847+0000 7f12fc365700 1 -- 172.21.15.2:0/1965791663 >> 172.21.15.2:0/1965791663 conn(0x7f12f4077780 msgr2=0x7f12f41002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:39.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.847+0000 7f12fc365700 1 -- 172.21.15.2:0/1965791663 shutdown_connections 2024-05-14T17:15:39.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:39.847+0000 7f12fc365700 1 -- 172.21.15.2:0/1965791663 wait complete. 2024-05-14T17:15:39.847 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 60 2024-05-14T17:15:39.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:39 smithi151 ceph-mon[36066]: pgmap v661: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-05-14T17:15:39.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:39 smithi123 ceph-mon[35971]: pgmap v661: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-05-14T17:15:40.474 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24479 quiesce db --cancel --all' 2024-05-14T17:15:40.474 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24479 quiesce db --cancel --all 2024-05-14T17:15:40.730 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:15:40.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:40 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/1965791663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:15:40.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:40 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/1965791663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:15:40.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:40 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/1965791663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:15:41.766 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:41 smithi002 ceph-mon[31538]: pgmap v662: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-05-14T17:15:41.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:41 smithi151 ceph-mon[36066]: pgmap v662: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-05-14T17:15:41.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:41 smithi123 ceph-mon[35971]: pgmap v662: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-05-14T17:15:42.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.543+0000 7fcbb520b700 1 -- 172.21.15.2:0/3488331293 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcbb0102160 msgr2=0x7fcbb0102540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:42.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.543+0000 7fcbb520b700 1 --2- 172.21.15.2:0/3488331293 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcbb0102160 0x7fcbb0102540 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009990 tx=0x7fcba0009ca0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.543+0000 7fcbb520b700 1 -- 172.21.15.2:0/3488331293 shutdown_connections 2024-05-14T17:15:42.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.543+0000 7fcbb520b700 1 --2- 172.21.15.2:0/3488331293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbb0107000 0x7fcbb010eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.543+0000 7fcbb520b700 1 --2- 172.21.15.2:0/3488331293 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcbb0102160 0x7fcbb0102540 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.543+0000 7fcbb520b700 1 --2- 172.21.15.2:0/3488331293 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcbb0102b10 0x7fcbb01068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.543+0000 7fcbb520b700 1 -- 172.21.15.2:0/3488331293 >> 172.21.15.2:0/3488331293 conn(0x7fcbb0077780 msgr2=0x7fcbb0077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:42.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.543+0000 7fcbb520b700 1 -- 172.21.15.2:0/3488331293 shutdown_connections 2024-05-14T17:15:42.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.543+0000 7fcbb520b700 1 -- 172.21.15.2:0/3488331293 wait complete. 2024-05-14T17:15:42.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.544+0000 7fcbb520b700 1 Processor -- start 2024-05-14T17:15:42.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.544+0000 7fcbb520b700 1 -- start start 2024-05-14T17:15:42.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.544+0000 7fcbb520b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbb0102160 0x7fcbb019f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:42.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.544+0000 7fcbb520b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcbb0102b10 0x7fcbb019fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:42.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.544+0000 7fcbb520b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcbb0107000 0x7fcbb0199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:42.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.544+0000 7fcbb520b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb01137f0 con 0x7fcbb0102b10 2024-05-14T17:15:42.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.544+0000 7fcbb520b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb0113670 con 0x7fcbb0102160 2024-05-14T17:15:42.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.544+0000 7fcbb520b700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb0113970 con 0x7fcbb0107000 2024-05-14T17:15:42.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.544+0000 7fcbae59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcbb0102b10 0x7fcbb019fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:42.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.544+0000 7fcbae59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcbb0102b10 0x7fcbb019fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52302/0 (socket says 172.21.15.2:52302) 2024-05-14T17:15:42.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.544+0000 7fcbae59c700 1 -- 172.21.15.2:0/2328936909 learned_addr learned my addr 172.21.15.2:0/2328936909 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:42.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.544+0000 7fcbaf59e700 1 --2- 172.21.15.2:0/2328936909 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcbb0107000 0x7fcbb0199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:42.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.544+0000 7fcbaed9d700 1 --2- 172.21.15.2:0/2328936909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbb0102160 0x7fcbb019f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:42.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.545+0000 7fcbae59c700 1 -- 172.21.15.2:0/2328936909 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcbb0107000 msgr2=0x7fcbb0199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:42.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.545+0000 7fcbae59c700 1 --2- 172.21.15.2:0/2328936909 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcbb0107000 0x7fcbb0199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.545+0000 7fcbae59c700 1 -- 172.21.15.2:0/2328936909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbb0102160 msgr2=0x7fcbb019f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:42.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.545+0000 7fcbae59c700 1 --2- 172.21.15.2:0/2328936909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbb0102160 0x7fcbb019f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.545+0000 7fcbae59c700 1 -- 172.21.15.2:0/2328936909 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb019a230 con 0x7fcbb0102b10 2024-05-14T17:15:42.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.545+0000 7fcbae59c700 1 --2- 172.21.15.2:0/2328936909 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcbb0102b10 0x7fcbb019fe50 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fcba400d950 tx=0x7fcba400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:42.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.545+0000 7fcb97fff700 1 -- 172.21.15.2:0/2328936909 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba4017070 con 0x7fcbb0102b10 2024-05-14T17:15:42.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.545+0000 7fcb97fff700 1 -- 172.21.15.2:0/2328936909 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcba400fd40 con 0x7fcbb0102b10 2024-05-14T17:15:42.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.545+0000 7fcbb520b700 1 -- 172.21.15.2:0/2328936909 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbb019a520 con 0x7fcbb0102b10 2024-05-14T17:15:42.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.545+0000 7fcb97fff700 1 -- 172.21.15.2:0/2328936909 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba400e3b0 con 0x7fcbb0102b10 2024-05-14T17:15:42.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.545+0000 7fcbb520b700 1 -- 172.21.15.2:0/2328936909 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbb01a67e0 con 0x7fcbb0102b10 2024-05-14T17:15:42.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.547+0000 7fcb97fff700 1 -- 172.21.15.2:0/2328936909 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fcba401a070 con 0x7fcbb0102b10 2024-05-14T17:15:42.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.548+0000 7fcb97fff700 1 --2- 172.21.15.2:0/2328936909 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcb980b29f0 0x7fcb980b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:42.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.548+0000 7fcbaed9d700 1 --2- 172.21.15.2:0/2328936909 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcb980b29f0 0x7fcb980b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:42.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.549+0000 7fcb97fff700 1 -- 172.21.15.2:0/2328936909 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8469+0+0 (secure 0 0 0) 0x7fcba40e1de0 con 0x7fcbb0102b10 2024-05-14T17:15:42.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.549+0000 7fcbbfaa1b80 1 Processor -- start 2024-05-14T17:15:42.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.549+0000 7fcbbfaa1b80 1 -- start start 2024-05-14T17:15:42.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.549+0000 7fcbbfaa1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5504e390 0x55fc5504e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:42.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.549+0000 7fcbbfaa1b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5504ecb0 0x55fc55049c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:42.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.549+0000 7fcbaed9d700 1 --2- 172.21.15.2:0/2328936909 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcb980b29f0 0x7fcb980b4eb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcba00098e0 tx=0x7fcba0005960 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:42.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.549+0000 7fcbbfaa1b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc5504a230 0x55fc5504a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:42.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.549+0000 7fcbbfaa1b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55fc54fa7de0 con 0x55fc5504ecb0 2024-05-14T17:15:42.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.549+0000 7fcbbfaa1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fc54f97980 con 0x55fc5504e390 2024-05-14T17:15:42.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.549+0000 7fcbbfaa1b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55fc54f9b9c0 con 0x55fc5504a230 2024-05-14T17:15:42.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.549+0000 7fcbae59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5504ecb0 0x55fc55049c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:42.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.549+0000 7fcbae59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5504ecb0 0x55fc55049c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52304/0 (socket says 172.21.15.2:52304) 2024-05-14T17:15:42.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.549+0000 7fcbae59c700 1 -- 172.21.15.2:0/4232232412 learned_addr learned my addr 172.21.15.2:0/4232232412 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:42.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.549+0000 7fcbaed9d700 1 --2- 172.21.15.2:0/4232232412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5504e390 0x55fc5504e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:42.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.549+0000 7fcbaf59e700 1 --2- 172.21.15.2:0/4232232412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc5504a230 0x55fc5504a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:42.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.549+0000 7fcbae59c700 1 -- 172.21.15.2:0/4232232412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc5504a230 msgr2=0x55fc5504a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:42.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.549+0000 7fcbae59c700 1 --2- 172.21.15.2:0/4232232412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc5504a230 0x55fc5504a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.549+0000 7fcbae59c700 1 -- 172.21.15.2:0/4232232412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5504e390 msgr2=0x55fc5504e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:42.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.549+0000 7fcbae59c700 1 --2- 172.21.15.2:0/4232232412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5504e390 0x55fc5504e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.549+0000 7fcbae59c700 1 -- 172.21.15.2:0/4232232412 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc54f41020 con 0x55fc5504ecb0 2024-05-14T17:15:42.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.549+0000 7fcbaf59e700 1 --2- 172.21.15.2:0/4232232412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc5504a230 0x55fc5504a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:15:42.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.550+0000 7fcbae59c700 1 --2- 172.21.15.2:0/4232232412 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5504ecb0 0x55fc55049c10 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fcba4005920 tx=0x7fcba4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fda6c009a682e07 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:42.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.550+0000 7fcb957fa700 1 -- 172.21.15.2:0/4232232412 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba4021070 con 0x55fc5504ecb0 2024-05-14T17:15:42.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.550+0000 7fcb957fa700 1 -- 172.21.15.2:0/4232232412 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcba40a90f0 con 0x55fc5504ecb0 2024-05-14T17:15:42.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.550+0000 7fcb957fa700 1 -- 172.21.15.2:0/4232232412 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba40a95b0 con 0x55fc5504ecb0 2024-05-14T17:15:42.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.550+0000 7fcbbfaa1b80 1 -- 172.21.15.2:0/4232232412 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5504ecb0 msgr2=0x55fc55049c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:42.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.550+0000 7fcbbfaa1b80 1 --2- 172.21.15.2:0/4232232412 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5504ecb0 0x55fc55049c10 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fcba4005920 tx=0x7fcba4010000 comp rx=0 tx=0).stop 2024-05-14T17:15:42.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.551+0000 7fcbbfaa1b80 1 -- 172.21.15.2:0/4232232412 shutdown_connections 2024-05-14T17:15:42.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.551+0000 7fcbbfaa1b80 1 --2- 172.21.15.2:0/4232232412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc5504a230 0x55fc5504a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.551+0000 7fcbbfaa1b80 1 --2- 172.21.15.2:0/4232232412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5504e390 0x55fc5504e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.551+0000 7fcbbfaa1b80 1 --2- 172.21.15.2:0/4232232412 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5504ecb0 0x55fc55049c10 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.551+0000 7fcbbfaa1b80 1 -- 172.21.15.2:0/4232232412 >> 172.21.15.2:0/4232232412 conn(0x55fc54f76e20 msgr2=0x55fc55044410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:42.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.551+0000 7fcbbfaa1b80 1 -- 172.21.15.2:0/4232232412 shutdown_connections 2024-05-14T17:15:42.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.551+0000 7fcbbfaa1b80 1 -- 172.21.15.2:0/4232232412 wait complete. 2024-05-14T17:15:42.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.551+0000 7fcbbfaa1b80 1 -- start start 2024-05-14T17:15:42.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.551+0000 7fcbbfaa1b80 1 Processor -- start 2024-05-14T17:15:42.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.551+0000 7fcbbfaa1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5504a230 0x55fc55055a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:42.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.551+0000 7fcbbfaa1b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc5504e390 0x55fc55052e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:42.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.551+0000 7fcbbfaa1b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5504ecb0 0x55fc55053340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:42.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.551+0000 7fcbbfaa1b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55fc54e9e010 con 0x55fc5504ecb0 2024-05-14T17:15:42.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.551+0000 7fcbbfaa1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fc54effe10 con 0x55fc5504a230 2024-05-14T17:15:42.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.551+0000 7fcbbfaa1b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55fc55056790 con 0x55fc5504e390 2024-05-14T17:15:42.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.552+0000 7fcbbfaa1b80 10 client.0 _ll_register_callbacks cb 0x55fc54f329c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:15:42.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.552+0000 7fcbaf59e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5504ecb0 0x55fc55053340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:42.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.552+0000 7fcbaf59e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5504ecb0 0x55fc55053340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52316/0 (socket says 172.21.15.2:52316) 2024-05-14T17:15:42.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.552+0000 7fcbaf59e700 1 -- 172.21.15.2:0/2017438007 learned_addr learned my addr 172.21.15.2:0/2017438007 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:42.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.552+0000 7fcbaed9d700 1 --2- 172.21.15.2:0/2017438007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5504a230 0x55fc55055a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:42.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.552+0000 7fcbae59c700 1 --2- 172.21.15.2:0/2017438007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc5504e390 0x55fc55052e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:42.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.552+0000 7fcbaf59e700 1 -- 172.21.15.2:0/2017438007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc5504e390 msgr2=0x55fc55052e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:42.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.552+0000 7fcbaf59e700 1 --2- 172.21.15.2:0/2017438007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc5504e390 0x55fc55052e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.552+0000 7fcbaf59e700 1 -- 172.21.15.2:0/2017438007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5504a230 msgr2=0x55fc55055a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:42.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.552+0000 7fcbaf59e700 1 --2- 172.21.15.2:0/2017438007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5504a230 0x55fc55055a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.552+0000 7fcbaf59e700 1 -- 172.21.15.2:0/2017438007 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc54ee7940 con 0x55fc5504ecb0 2024-05-14T17:15:42.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.552+0000 7fcbaf59e700 1 --2- 172.21.15.2:0/2017438007 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5504ecb0 0x55fc55053340 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fcba8008e60 tx=0x7fcba8010430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d1ee577fa5c8546 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:42.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.552+0000 7fcb92ff5700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:15:42.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.552+0000 7fcb92ff5700 1 -- 172.21.15.2:0/2017438007 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba8010bc0 con 0x55fc5504ecb0 2024-05-14T17:15:42.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.552+0000 7fcb92ff5700 1 -- 172.21.15.2:0/2017438007 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8469+0+0 (secure 0 0 0) 0x7fcba8010d60 con 0x55fc5504ecb0 2024-05-14T17:15:42.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.552+0000 7fcbbfaa1b80 1 -- 172.21.15.2:0/2017438007 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcba40a90f0 con 0x55fc5504ecb0 2024-05-14T17:15:42.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.552+0000 7fcb92ff5700 1 -- 172.21.15.2:0/2017438007 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fcba8023050 con 0x55fc5504ecb0 2024-05-14T17:15:42.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.552+0000 7fcbbfaa1b80 10 client.17331 fetch_fsmap learned FSMap version 60 2024-05-14T17:15:42.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.552+0000 7fcbbfaa1b80 1 -- 172.21.15.2:0/2017438007 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55fc54f517d0 con 0x55fc5504ecb0 2024-05-14T17:15:42.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.553+0000 7fcb92ff5700 1 -- 172.21.15.2:0/2017438007 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcba40a90f0 con 0x55fc5504ecb0 2024-05-14T17:15:42.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.553+0000 7fcb92ff5700 1 -- 172.21.15.2:0/2017438007 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 60) ==== 8346+0+0 (secure 0 0 0) 0x7fcba801e550 con 0x55fc5504ecb0 2024-05-14T17:15:42.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.553+0000 7fcbbfaa1b80 10 client.17331 fetch_fsmap finished waiting for FSMap version 60 2024-05-14T17:15:42.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.553+0000 7fcbbfaa1b80 10 client.17331 resolve_mds: validated gid 24479 aka daemon mds.b 2024-05-14T17:15:42.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.553+0000 7fcbbfaa1b80 20 client.17331 populate_metadata read hostname 'smithi002' 2024-05-14T17:15:42.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.553+0000 7fcbbfaa1b80 1 --2- 172.21.15.2:0/2017438007 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55fc550632d0 0x55fc550656c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:42.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.553+0000 7fcbbfaa1b80 4 client.17331 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:15:42.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.553+0000 7fcbbfaa1b80 1 -- 172.21.15.2:0/2017438007 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc54fa9d30 con 0x55fc550632d0 2024-05-14T17:15:42.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.553+0000 7fcbaed9d700 1 --2- 172.21.15.2:0/2017438007 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55fc550632d0 0x55fc550656c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:42.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.554+0000 7fcbaed9d700 1 --2- 172.21.15.2:0/2017438007 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55fc550632d0 0x55fc550656c0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42380b2a51eb02a9 server_cookie=90e58d7d59c47ef3 in_seq=0 out_seq=0 2024-05-14T17:15:42.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.554+0000 7fcb92ff5700 10 client.17331 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:15:42.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.554+0000 7fcb92ff5700 1 -- 172.21.15.2:0/2017438007 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fc54fa9d30 con 0x55fc550632d0 2024-05-14T17:15:42.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.554+0000 7fcb92ff5700 10 client.17331 handle_command_reply: tid=0 2024-05-14T17:15:42.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.554+0000 7fcbbfaa1b80 1 client.17331 shutdown 2024-05-14T17:15:42.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.554+0000 7fcbbfaa1b80 1 -- 172.21.15.2:0/2017438007 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5504ecb0 msgr2=0x55fc55053340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:42.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.554+0000 7fcbbfaa1b80 1 --2- 172.21.15.2:0/2017438007 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5504ecb0 0x55fc55053340 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fcba8008e60 tx=0x7fcba8010430 comp rx=0 tx=0).stop 2024-05-14T17:15:42.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.554+0000 7fcbbfaa1b80 1 -- 172.21.15.2:0/2017438007 shutdown_connections 2024-05-14T17:15:42.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.554+0000 7fcbbfaa1b80 1 --2- 172.21.15.2:0/2017438007 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55fc550632d0 0x55fc550656c0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.554+0000 7fcbbfaa1b80 1 --2- 172.21.15.2:0/2017438007 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5504ecb0 0x55fc55053340 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.554+0000 7fcbbfaa1b80 1 --2- 172.21.15.2:0/2017438007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5504a230 0x55fc55055a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.554+0000 7fcbbfaa1b80 1 --2- 172.21.15.2:0/2017438007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc5504e390 0x55fc55052e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.554+0000 7fcbbfaa1b80 1 -- 172.21.15.2:0/2017438007 >> 172.21.15.2:0/2017438007 conn(0x55fc54f76e20 msgr2=0x55fc5503ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:42.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.555+0000 7fcb92ff5700 0 client.17331 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:15:42.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.555+0000 7fcbbfaa1b80 1 -- 172.21.15.2:0/2017438007 shutdown_connections 2024-05-14T17:15:42.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.555+0000 7fcbbfaa1b80 1 -- 172.21.15.2:0/2017438007 wait complete. 2024-05-14T17:15:42.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.555+0000 7fcbbfaa1b80 20 client.17331 trim_cache size 0 max 16384 2024-05-14T17:15:42.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.575+0000 7fcbbfaa1b80 1 Processor -- start 2024-05-14T17:15:42.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.575+0000 7fcbbfaa1b80 1 -- start start 2024-05-14T17:15:42.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.576+0000 7fcbbfaa1b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5504a230 0x55fc550552a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:42.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.576+0000 7fcbbfaa1b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc5504e390 0x55fc5503f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:42.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.576+0000 7fcbbfaa1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5504ecb0 0x55fc55051e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:42.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.576+0000 7fcbbfaa1b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb01b0720 con 0x55fc5504a230 2024-05-14T17:15:42.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.576+0000 7fcbbfaa1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb01adfb0 con 0x55fc5504ecb0 2024-05-14T17:15:42.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.576+0000 7fcbbfaa1b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb01abe00 con 0x55fc5504e390 2024-05-14T17:15:42.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.576+0000 7fcbaed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5504a230 0x55fc550552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:42.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.576+0000 7fcbaed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5504a230 0x55fc550552a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52324/0 (socket says 172.21.15.2:52324) 2024-05-14T17:15:42.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.576+0000 7fcbaed9d700 1 -- 172.21.15.2:0/4291753475 learned_addr learned my addr 172.21.15.2:0/4291753475 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:42.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.576+0000 7fcbae59c700 1 --2- 172.21.15.2:0/4291753475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc5504e390 0x55fc5503f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:42.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.576+0000 7fcbaf59e700 1 --2- 172.21.15.2:0/4291753475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5504ecb0 0x55fc55051e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:42.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.576+0000 7fcbaed9d700 1 -- 172.21.15.2:0/4291753475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc5504e390 msgr2=0x55fc5503f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:42.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.576+0000 7fcbaed9d700 1 --2- 172.21.15.2:0/4291753475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc5504e390 0x55fc5503f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.576+0000 7fcbaed9d700 1 -- 172.21.15.2:0/4291753475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5504ecb0 msgr2=0x55fc55051e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:42.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.576+0000 7fcbaed9d700 1 --2- 172.21.15.2:0/4291753475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5504ecb0 0x55fc55051e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.576+0000 7fcbaed9d700 1 -- 172.21.15.2:0/4291753475 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc5503f580 con 0x55fc5504a230 2024-05-14T17:15:42.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.576+0000 7fcbaed9d700 1 --2- 172.21.15.2:0/4291753475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5504a230 0x55fc550552a0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fcba0000c00 tx=0x7fcba0023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d4009a959aa902 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:42.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.576+0000 7fcb94ff9700 1 -- 172.21.15.2:0/4291753475 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba0024ae0 con 0x55fc5504a230 2024-05-14T17:15:42.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.577+0000 7fcb94ff9700 1 -- 172.21.15.2:0/4291753475 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcba003b070 con 0x55fc5504a230 2024-05-14T17:15:42.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.577+0000 7fcb94ff9700 1 -- 172.21.15.2:0/4291753475 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba000f460 con 0x55fc5504a230 2024-05-14T17:15:42.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.577+0000 7fcbbfaa1b80 1 -- 172.21.15.2:0/4291753475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5504a230 msgr2=0x55fc550552a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:42.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.577+0000 7fcbbfaa1b80 1 --2- 172.21.15.2:0/4291753475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5504a230 0x55fc550552a0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fcba0000c00 tx=0x7fcba0023000 comp rx=0 tx=0).stop 2024-05-14T17:15:42.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.577+0000 7fcbbfaa1b80 1 -- 172.21.15.2:0/4291753475 shutdown_connections 2024-05-14T17:15:42.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.577+0000 7fcbbfaa1b80 1 --2- 172.21.15.2:0/4291753475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5504ecb0 0x55fc55051e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.577+0000 7fcbbfaa1b80 1 --2- 172.21.15.2:0/4291753475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5504a230 0x55fc550552a0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.577+0000 7fcbbfaa1b80 1 --2- 172.21.15.2:0/4291753475 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc5504e390 0x55fc5503f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.577+0000 7fcbbfaa1b80 1 -- 172.21.15.2:0/4291753475 >> 172.21.15.2:0/4291753475 conn(0x55fc54f76e20 msgr2=0x55fc550516f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:42.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.577+0000 7fcbbfaa1b80 1 -- 172.21.15.2:0/4291753475 shutdown_connections 2024-05-14T17:15:42.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.577+0000 7fcbbfaa1b80 1 -- 172.21.15.2:0/4291753475 wait complete. 2024-05-14T17:15:42.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.578+0000 7fcbbfaa1b80 1 -- start start 2024-05-14T17:15:42.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.578+0000 7fcbbfaa1b80 1 Processor -- start 2024-05-14T17:15:42.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.578+0000 7fcbbfaa1b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc5504a230 0x55fc55058a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:42.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.578+0000 7fcbbfaa1b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5504e390 0x55fc55058f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:42.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.578+0000 7fcbbfaa1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5504ecb0 0x55fc550594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:42.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.578+0000 7fcbbfaa1b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb01ac470 con 0x55fc5504e390 2024-05-14T17:15:42.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.578+0000 7fcbbfaa1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb01ad8e0 con 0x55fc5504ecb0 2024-05-14T17:15:42.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.578+0000 7fcbbfaa1b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb01acb40 con 0x55fc5504a230 2024-05-14T17:15:42.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.578+0000 7fcbbfaa1b80 10 client.0 _ll_register_callbacks cb 0x55fc54f329c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:15:42.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.578+0000 7fcbae59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5504e390 0x55fc55058f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:42.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.578+0000 7fcbae59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5504e390 0x55fc55058f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52334/0 (socket says 172.21.15.2:52334) 2024-05-14T17:15:42.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.578+0000 7fcbae59c700 1 -- 172.21.15.2:0/2034461448 learned_addr learned my addr 172.21.15.2:0/2034461448 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:42.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.578+0000 7fcbaf59e700 1 --2- 172.21.15.2:0/2034461448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5504ecb0 0x55fc550594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:42.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.578+0000 7fcbaed9d700 1 --2- 172.21.15.2:0/2034461448 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc5504a230 0x55fc55058a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:42.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.578+0000 7fcbae59c700 1 -- 172.21.15.2:0/2034461448 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc5504a230 msgr2=0x55fc55058a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:42.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.578+0000 7fcbae59c700 1 --2- 172.21.15.2:0/2034461448 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc5504a230 0x55fc55058a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.578+0000 7fcbae59c700 1 -- 172.21.15.2:0/2034461448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5504ecb0 msgr2=0x55fc550594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:42.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.578+0000 7fcbae59c700 1 --2- 172.21.15.2:0/2034461448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5504ecb0 0x55fc550594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.578+0000 7fcbae59c700 1 -- 172.21.15.2:0/2034461448 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc550cdd00 con 0x55fc5504e390 2024-05-14T17:15:42.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.579+0000 7fcbae59c700 1 --2- 172.21.15.2:0/2034461448 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5504e390 0x55fc55058f60 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fcba40a8df0 tx=0x7fcba4005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ecb5c3a172f18c server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:42.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.579+0000 7fcb92ff5700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:15:42.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.579+0000 7fcb92ff5700 1 -- 172.21.15.2:0/2034461448 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba406d060 con 0x55fc5504e390 2024-05-14T17:15:42.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.579+0000 7fcb92ff5700 1 -- 172.21.15.2:0/2034461448 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8469+0+0 (secure 0 0 0) 0x7fcba40a9560 con 0x55fc5504e390 2024-05-14T17:15:42.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.579+0000 7fcbbfaa1b80 1 -- 172.21.15.2:0/2034461448 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcba003b070 con 0x55fc5504e390 2024-05-14T17:15:42.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.579+0000 7fcb92ff5700 1 -- 172.21.15.2:0/2034461448 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fcba4021040 con 0x55fc5504e390 2024-05-14T17:15:42.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.579+0000 7fcbbfaa1b80 10 client.17337 fetch_fsmap learned FSMap version 60 2024-05-14T17:15:42.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.579+0000 7fcbbfaa1b80 1 -- 172.21.15.2:0/2034461448 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55fc550cefe0 con 0x55fc5504e390 2024-05-14T17:15:42.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.579+0000 7fcb92ff5700 1 -- 172.21.15.2:0/2034461448 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcba003b070 con 0x55fc5504e390 2024-05-14T17:15:42.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.579+0000 7fcb92ff5700 1 -- 172.21.15.2:0/2034461448 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 60) ==== 8346+0+0 (secure 0 0 0) 0x7fcba40b1ac0 con 0x55fc5504e390 2024-05-14T17:15:42.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.579+0000 7fcbbfaa1b80 10 client.17337 fetch_fsmap finished waiting for FSMap version 60 2024-05-14T17:15:42.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.579+0000 7fcbbfaa1b80 10 client.17337 resolve_mds: validated gid 24479 aka daemon mds.b 2024-05-14T17:15:42.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.579+0000 7fcbbfaa1b80 20 client.17337 populate_metadata read hostname 'smithi002' 2024-05-14T17:15:42.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.579+0000 7fcbbfaa1b80 1 --2- 172.21.15.2:0/2034461448 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55fc550cf2c0 0x55fc550d16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:42.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.579+0000 7fcbbfaa1b80 4 client.17337 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-05-14T17:15:42.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.579+0000 7fcbbfaa1b80 1 -- 172.21.15.2:0/2034461448 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55fc54fa3510 con 0x55fc550cf2c0 2024-05-14T17:15:42.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.580+0000 7fcbaed9d700 1 --2- 172.21.15.2:0/2034461448 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55fc550cf2c0 0x55fc550d16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:42.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.580+0000 7fcbaed9d700 1 --2- 172.21.15.2:0/2034461448 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55fc550cf2c0 0x55fc550d16b0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42ef30ddbf26c3c8 server_cookie=425ab452987fa202 in_seq=0 out_seq=0 2024-05-14T17:15:42.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.580+0000 7fcb92ff5700 10 client.17337 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:15:42.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.580+0000 7fcb92ff5700 1 -- 172.21.15.2:0/2034461448 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55fc54fa3510 con 0x55fc550cf2c0 2024-05-14T17:15:42.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.580+0000 7fcb92ff5700 10 client.17337 handle_command_reply: tid=0 2024-05-14T17:15:42.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.580+0000 7fcbbfaa1b80 1 client.17337 shutdown 2024-05-14T17:15:42.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.581+0000 7fcbbfaa1b80 1 -- 172.21.15.2:0/2034461448 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5504e390 msgr2=0x55fc55058f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:42.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.581+0000 7fcbbfaa1b80 1 --2- 172.21.15.2:0/2034461448 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5504e390 0x55fc55058f60 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fcba40a8df0 tx=0x7fcba4005d50 comp rx=0 tx=0).stop 2024-05-14T17:15:42.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.581+0000 7fcbbfaa1b80 1 -- 172.21.15.2:0/2034461448 shutdown_connections 2024-05-14T17:15:42.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.581+0000 7fcbbfaa1b80 1 --2- 172.21.15.2:0/2034461448 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55fc550cf2c0 0x55fc550d16b0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.581+0000 7fcbbfaa1b80 1 --2- 172.21.15.2:0/2034461448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5504ecb0 0x55fc550594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.581+0000 7fcbbfaa1b80 1 --2- 172.21.15.2:0/2034461448 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc5504a230 0x55fc55058a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.581+0000 7fcbbfaa1b80 1 --2- 172.21.15.2:0/2034461448 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc5504e390 0x55fc55058f60 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.581+0000 7fcbbfaa1b80 1 -- 172.21.15.2:0/2034461448 >> 172.21.15.2:0/2034461448 conn(0x55fc54f76e20 msgr2=0x55fc55051f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:42.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.581+0000 7fcb92ff5700 0 client.17337 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:15:42.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.581+0000 7fcbbfaa1b80 1 -- 172.21.15.2:0/2034461448 shutdown_connections 2024-05-14T17:15:42.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.581+0000 7fcbbfaa1b80 1 -- 172.21.15.2:0/2034461448 wait complete. 2024-05-14T17:15:42.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.581+0000 7fcbbfaa1b80 20 client.17337 trim_cache size 0 max 16384 2024-05-14T17:15:42.589 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T17:15:42.589 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 60, 2024-05-14T17:15:42.589 INFO:teuthology.orchestra.run.smithi002.stdout: "leader": 24479, 2024-05-14T17:15:42.589 INFO:teuthology.orchestra.run.smithi002.stdout: "set_version": 35, 2024-05-14T17:15:42.589 INFO:teuthology.orchestra.run.smithi002.stdout: "sets": {} 2024-05-14T17:15:42.589 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T17:15:42.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.581+0000 7fcbb520b700 1 -- 172.21.15.2:0/2328936909 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcb980b29f0 msgr2=0x7fcb980b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:42.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.581+0000 7fcbb520b700 1 --2- 172.21.15.2:0/2328936909 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcb980b29f0 0x7fcb980b4eb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcba00098e0 tx=0x7fcba0005960 comp rx=0 tx=0).stop 2024-05-14T17:15:42.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.581+0000 7fcbb520b700 1 -- 172.21.15.2:0/2328936909 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcbb0102b10 msgr2=0x7fcbb019fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:42.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.581+0000 7fcbb520b700 1 --2- 172.21.15.2:0/2328936909 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcbb0102b10 0x7fcbb019fe50 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fcba400d950 tx=0x7fcba400dc60 comp rx=0 tx=0).stop 2024-05-14T17:15:42.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.581+0000 7fcbb520b700 1 -- 172.21.15.2:0/2328936909 shutdown_connections 2024-05-14T17:15:42.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.581+0000 7fcbb520b700 1 --2- 172.21.15.2:0/2328936909 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcb980b29f0 0x7fcb980b4eb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.581+0000 7fcbb520b700 1 --2- 172.21.15.2:0/2328936909 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcbb0107000 0x7fcbb0199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.581+0000 7fcbb520b700 1 --2- 172.21.15.2:0/2328936909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbb0102160 0x7fcbb019f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.581+0000 7fcbb520b700 1 --2- 172.21.15.2:0/2328936909 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcbb0102b10 0x7fcbb019fe50 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.581+0000 7fcbb520b700 1 -- 172.21.15.2:0/2328936909 >> 172.21.15.2:0/2328936909 conn(0x7fcbb0077780 msgr2=0x7fcbb01002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:42.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.582+0000 7fcbb520b700 1 -- 172.21.15.2:0/2328936909 shutdown_connections 2024-05-14T17:15:42.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:42.582+0000 7fcbb520b700 1 -- 172.21.15.2:0/2328936909 wait complete. 2024-05-14T17:15:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.848+0000 7fa9dec45640 1 -- 172.21.15.2:0/1419355620 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9d8103420 msgr2=0x7fa9d810f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.848+0000 7fa9dec45640 1 --2- 172.21.15.2:0/1419355620 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9d8103420 0x7fa9d810f930 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8009cd0 tx=0x7fa9c802a8e0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.848+0000 7fa9dec45640 1 -- 172.21.15.2:0/1419355620 shutdown_connections 2024-05-14T17:15:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.848+0000 7fa9dec45640 1 --2- 172.21.15.2:0/1419355620 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9d8103420 0x7fa9d810f930 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.848+0000 7fa9dec45640 1 --2- 172.21.15.2:0/1419355620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9d8102a60 0x7fa9d8102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.848+0000 7fa9dec45640 1 --2- 172.21.15.2:0/1419355620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9d8108a70 0x7fa9d8108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.848+0000 7fa9dec45640 1 -- 172.21.15.2:0/1419355620 >> 172.21.15.2:0/1419355620 conn(0x7fa9d80fe550 msgr2=0x7fa9d8100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.848+0000 7fa9dec45640 1 -- 172.21.15.2:0/1419355620 shutdown_connections 2024-05-14T17:15:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.848+0000 7fa9dec45640 1 -- 172.21.15.2:0/1419355620 wait complete. 2024-05-14T17:15:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.849+0000 7fa9dec45640 1 Processor -- start 2024-05-14T17:15:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.849+0000 7fa9dec45640 1 -- start start 2024-05-14T17:15:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.849+0000 7fa9dec45640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9d8102a60 0x7fa9d819b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:42.850 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.849+0000 7fa9dec45640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9d8103420 0x7fa9d819b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.849+0000 7fa9dec45640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9d8108a70 0x7fa9d819fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.849+0000 7fa9dec45640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d8075e20 con 0x7fa9d8103420 2024-05-14T17:15:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.849+0000 7fa9dec45640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d8075ca0 con 0x7fa9d8108a70 2024-05-14T17:15:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.849+0000 7fa9dec45640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d8075fa0 con 0x7fa9d8102a60 2024-05-14T17:15:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.849+0000 7fa9d7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9d8103420 0x7fa9d819b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.849+0000 7fa9d7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9d8103420 0x7fa9d819b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52344/0 (socket says 172.21.15.2:52344) 2024-05-14T17:15:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.849+0000 7fa9d7fff640 1 -- 172.21.15.2:0/2106588349 learned_addr learned my addr 172.21.15.2:0/2106588349 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.849+0000 7fa9dc9e1640 1 --2- 172.21.15.2:0/2106588349 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9d8102a60 0x7fa9d819b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.849+0000 7fa9dd1e2640 1 --2- 172.21.15.2:0/2106588349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9d8108a70 0x7fa9d819fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.849+0000 7fa9d7fff640 1 -- 172.21.15.2:0/2106588349 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9d8102a60 msgr2=0x7fa9d819b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.849+0000 7fa9d7fff640 1 --2- 172.21.15.2:0/2106588349 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9d8102a60 0x7fa9d819b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.851 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.849+0000 7fa9d7fff640 1 -- 172.21.15.2:0/2106588349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9d8108a70 msgr2=0x7fa9d819fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.849+0000 7fa9d7fff640 1 --2- 172.21.15.2:0/2106588349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9d8108a70 0x7fa9d819fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.849+0000 7fa9d7fff640 1 -- 172.21.15.2:0/2106588349 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d81a0280 con 0x7fa9d8103420 2024-05-14T17:15:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.849+0000 7fa9dc9e1640 1 --2- 172.21.15.2:0/2106588349 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9d8102a60 0x7fa9d819b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:15:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.850+0000 7fa9d7fff640 1 --2- 172.21.15.2:0/2106588349 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9d8103420 0x7fa9d819b7d0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00c920 tx=0x7fa9cc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.850+0000 7fa9d5ffb640 1 -- 172.21.15.2:0/2106588349 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc007c70 con 0x7fa9d8103420 2024-05-14T17:15:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.850+0000 7fa9d5ffb640 1 -- 172.21.15.2:0/2106588349 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9cc007e10 con 0x7fa9d8103420 2024-05-14T17:15:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.850+0000 7fa9dec45640 1 -- 172.21.15.2:0/2106588349 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9d81a0570 con 0x7fa9d8103420 2024-05-14T17:15:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.850+0000 7fa9d5ffb640 1 -- 172.21.15.2:0/2106588349 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc018820 con 0x7fa9d8103420 2024-05-14T17:15:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.850+0000 7fa9dec45640 1 -- 172.21.15.2:0/2106588349 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9d81a7e00 con 0x7fa9d8103420 2024-05-14T17:15:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.852+0000 7fa9d5ffb640 1 -- 172.21.15.2:0/2106588349 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fa9cc00f450 con 0x7fa9d8103420 2024-05-14T17:15:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.852+0000 7fa9e1a38740 1 Processor -- start 2024-05-14T17:15:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.852+0000 7fa9e1a38740 1 -- start start 2024-05-14T17:15:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.853+0000 7fa9e1a38740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2bea738f0 0x55f2bea73cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.853+0000 7fa9e1a38740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2bea74210 0x55f2bea7e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.853+0000 7fa9e1a38740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2bea7ead0 0x55f2bea80ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.853+0000 7fa9e1a38740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f2be9e6c30 con 0x55f2bea738f0 2024-05-14T17:15:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.853+0000 7fa9e1a38740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f2be9d67d0 con 0x55f2bea74210 2024-05-14T17:15:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.853+0000 7fa9e1a38740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f2be9da810 con 0x55f2bea7ead0 2024-05-14T17:15:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.853+0000 7fa9dd1e2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2bea738f0 0x55f2bea73cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.853+0000 7fa9dd1e2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2bea738f0 0x55f2bea73cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52358/0 (socket says 172.21.15.2:52358) 2024-05-14T17:15:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.853+0000 7fa9dd1e2640 1 -- 172.21.15.2:0/1513282588 learned_addr learned my addr 172.21.15.2:0/1513282588 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.853+0000 7fa9dc9e1640 1 --2- 172.21.15.2:0/1513282588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2bea74210 0x55f2bea7e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.853+0000 7fa9d5ffb640 1 --2- 172.21.15.2:0/2106588349 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fa9c00b2a90 0x7fa9c00b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.853+0000 7fa9d7fff640 1 --2- 172.21.15.2:0/1513282588 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2bea7ead0 0x55f2bea80ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.853+0000 7fa9dd1e2640 1 -- 172.21.15.2:0/1513282588 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2bea7ead0 msgr2=0x55f2bea80ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.853+0000 7fa9dd1e2640 1 --2- 172.21.15.2:0/1513282588 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2bea7ead0 0x55f2bea80ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.853+0000 7fa9dd1e2640 1 -- 172.21.15.2:0/1513282588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2bea74210 msgr2=0x55f2bea7e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.853+0000 7fa9dd1e2640 1 --2- 172.21.15.2:0/1513282588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2bea74210 0x55f2bea7e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.853+0000 7fa9dd1e2640 1 -- 172.21.15.2:0/1513282588 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2be6eb210 con 0x55f2bea738f0 2024-05-14T17:15:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.853+0000 7fa9d5ffb640 1 -- 172.21.15.2:0/2106588349 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8469+0+0 (secure 0 0 0) 0x7fa9cc0e26d0 con 0x7fa9d8103420 2024-05-14T17:15:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.853+0000 7fa9dd1e2640 1 --2- 172.21.15.2:0/2106588349 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fa9c00b2a90 0x7fa9c00b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.854+0000 7fa9dd1e2640 1 --2- 172.21.15.2:0/1513282588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2bea738f0 0x55f2bea73cd0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8009890 tx=0x7fa9c8031640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b30b31c97bb94597 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.854+0000 7fa9beffd640 1 -- 172.21.15.2:0/1513282588 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8042040 con 0x55f2bea738f0 2024-05-14T17:15:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.854+0000 7fa9beffd640 1 -- 172.21.15.2:0/1513282588 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9c8044070 con 0x55f2bea738f0 2024-05-14T17:15:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.854+0000 7fa9beffd640 1 -- 172.21.15.2:0/1513282588 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8030d40 con 0x55f2bea738f0 2024-05-14T17:15:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.854+0000 7fa9e1a38740 1 -- 172.21.15.2:0/1513282588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2bea738f0 msgr2=0x55f2bea73cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.854+0000 7fa9dd1e2640 1 --2- 172.21.15.2:0/2106588349 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fa9c00b2a90 0x7fa9c00b4f50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa9c804d000 tx=0x7fa9c804e000 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.854+0000 7fa9e1a38740 1 --2- 172.21.15.2:0/1513282588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2bea738f0 0x55f2bea73cd0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8009890 tx=0x7fa9c8031640 comp rx=0 tx=0).stop 2024-05-14T17:15:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.855+0000 7fa9e1a38740 1 -- 172.21.15.2:0/1513282588 shutdown_connections 2024-05-14T17:15:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.855+0000 7fa9e1a38740 1 --2- 172.21.15.2:0/1513282588 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2bea7ead0 0x55f2bea80ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.855+0000 7fa9e1a38740 1 --2- 172.21.15.2:0/1513282588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2bea74210 0x55f2bea7e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.855+0000 7fa9e1a38740 1 --2- 172.21.15.2:0/1513282588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2bea738f0 0x55f2bea73cd0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.855+0000 7fa9e1a38740 1 -- 172.21.15.2:0/1513282588 >> 172.21.15.2:0/1513282588 conn(0x55f2bea6fe00 msgr2=0x55f2bea701e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.855+0000 7fa9e1a38740 1 -- 172.21.15.2:0/1513282588 shutdown_connections 2024-05-14T17:15:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.855+0000 7fa9e1a38740 1 -- 172.21.15.2:0/1513282588 wait complete. 2024-05-14T17:15:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.855+0000 7fa9e1a38740 1 -- start start 2024-05-14T17:15:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.855+0000 7fa9e1a38740 1 Processor -- start 2024-05-14T17:15:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.855+0000 7fa9e1a38740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2bea738f0 0x55f2bea7e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.855+0000 7fa9e1a38740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2bea74210 0x55f2bea7b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.855+0000 7fa9e1a38740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2bea7ead0 0x55f2bea7bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.855+0000 7fa9e1a38740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x55f2be6bc790 con 0x55f2bea7ead0 2024-05-14T17:15:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.855+0000 7fa9e1a38740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f2be992890 con 0x55f2bea74210 2024-05-14T17:15:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.855+0000 7fa9e1a38740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f2bea817a0 con 0x55f2bea738f0 2024-05-14T17:15:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.855+0000 7fa9e1a38740 10 client.0 _ll_register_callbacks cb 0x55f2be995d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:15:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.856+0000 7fa9dd1e2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2bea7ead0 0x55f2bea7bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.856+0000 7fa9dd1e2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2bea7ead0 0x55f2bea7bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52364/0 (socket says 172.21.15.2:52364) 2024-05-14T17:15:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.856+0000 7fa9dd1e2640 1 -- 172.21.15.2:0/4127078101 learned_addr learned my addr 172.21.15.2:0/4127078101 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.856+0000 7fa9dc9e1640 1 --2- 172.21.15.2:0/4127078101 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2bea738f0 0x55f2bea7e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.856+0000 7fa9d7fff640 1 --2- 172.21.15.2:0/4127078101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2bea74210 0x55f2bea7b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.856+0000 7fa9dd1e2640 1 -- 172.21.15.2:0/4127078101 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2bea738f0 msgr2=0x55f2bea7e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.856+0000 7fa9dd1e2640 1 --2- 172.21.15.2:0/4127078101 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2bea738f0 0x55f2bea7e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.856+0000 7fa9dd1e2640 1 -- 172.21.15.2:0/4127078101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2bea74210 msgr2=0x55f2bea7b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.856+0000 7fa9dd1e2640 1 --2- 172.21.15.2:0/4127078101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2bea74210 0x55f2bea7b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.856+0000 7fa9dd1e2640 1 -- 172.21.15.2:0/4127078101 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2be7b7620 con 0x55f2bea7ead0 2024-05-14T17:15:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.856+0000 7fa9d7fff640 1 --2- 172.21.15.2:0/4127078101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2bea74210 0x55f2bea7b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:15:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.856+0000 7fa9dd1e2640 1 --2- 172.21.15.2:0/4127078101 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2bea7ead0 0x55f2bea7bc50 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8030e70 tx=0x7fa9c80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f52cf0f6de4fda06 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.856+0000 7fa9b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:15:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.856+0000 7fa9b7fff640 1 -- 172.21.15.2:0/4127078101 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c805b060 con 0x55f2bea7ead0 2024-05-14T17:15:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.856+0000 7fa9e1a38740 1 -- 172.21.15.2:0/4127078101 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9c8044070 con 0x55f2bea7ead0 2024-05-14T17:15:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.856+0000 7fa9b7fff640 1 -- 172.21.15.2:0/4127078101 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8469+0+0 (secure 0 0 0) 0x7fa9c8003e00 con 0x55f2bea7ead0 2024-05-14T17:15:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.856+0000 7fa9b7fff640 1 -- 172.21.15.2:0/4127078101 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fa9c8042070 con 0x55f2bea7ead0 2024-05-14T17:15:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.856+0000 7fa9e1a38740 10 client.17349 fetch_fsmap learned FSMap version 60 2024-05-14T17:15:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.856+0000 7fa9e1a38740 1 -- 172.21.15.2:0/4127078101 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55f2bea8d1e0 con 0x55f2bea7ead0 2024-05-14T17:15:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.857+0000 7fa9b7fff640 1 -- 172.21.15.2:0/4127078101 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9c8044070 con 0x55f2bea7ead0 2024-05-14T17:15:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.857+0000 7fa9b7fff640 1 -- 172.21.15.2:0/4127078101 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 60) ==== 8346+0+0 (secure 0 0 0) 0x7fa9c8030070 con 0x55f2bea7ead0 2024-05-14T17:15:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.857+0000 7fa9e1a38740 10 client.17349 fetch_fsmap finished waiting for FSMap version 60 2024-05-14T17:15:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.857+0000 7fa9e1a38740 10 client.17349 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:15:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.857+0000 7fa9e1a38740 20 client.17349 populate_metadata read hostname 'smithi002' 2024-05-14T17:15:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.857+0000 7fa9e1a38740 1 --2- 172.21.15.2:0/4127078101 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55f2bea8e530 0x55f2bea90920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.857+0000 7fa9e1a38740 4 client.17349 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-14T17:15:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.857+0000 7fa9e1a38740 1 -- 172.21.15.2:0/4127078101 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2be9e8b80 con 0x55f2bea8e530 2024-05-14T17:15:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.857+0000 7fa9dc9e1640 1 --2- 172.21.15.2:0/4127078101 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55f2bea8e530 0x55f2bea90920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.857+0000 7fa9dc9e1640 1 --2- 172.21.15.2:0/4127078101 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55f2bea8e530 0x55f2bea90920 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf75aef3108dfe78 server_cookie=7a966024e2ac7faf in_seq=0 out_seq=0 2024-05-14T17:15:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.857+0000 7fa9b7fff640 10 client.17349 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:15:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.858+0000 7fa9b7fff640 1 -- 172.21.15.2:0/4127078101 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f2be9e8b80 con 0x55f2bea8e530 2024-05-14T17:15:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.858+0000 7fa9b7fff640 10 client.17349 handle_command_reply: tid=0 2024-05-14T17:15:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.858+0000 7fa9e1a38740 1 client.17349 shutdown 2024-05-14T17:15:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.859+0000 7fa9e1a38740 1 -- 172.21.15.2:0/4127078101 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2bea7ead0 msgr2=0x55f2bea7bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.859+0000 7fa9e1a38740 1 --2- 172.21.15.2:0/4127078101 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2bea7ead0 0x55f2bea7bc50 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8030e70 tx=0x7fa9c80095c0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.859+0000 7fa9e1a38740 1 -- 172.21.15.2:0/4127078101 shutdown_connections 2024-05-14T17:15:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.859+0000 7fa9e1a38740 1 --2- 172.21.15.2:0/4127078101 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55f2bea8e530 0x55f2bea90920 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.859+0000 7fa9b7fff640 0 client.17349 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:15:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.859+0000 7fa9e1a38740 1 --2- 172.21.15.2:0/4127078101 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2bea7ead0 0x55f2bea7bc50 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.859+0000 7fa9e1a38740 1 --2- 172.21.15.2:0/4127078101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2bea74210 0x55f2bea7b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.859+0000 7fa9e1a38740 1 --2- 172.21.15.2:0/4127078101 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2bea738f0 0x55f2bea7e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.859+0000 7fa9e1a38740 1 -- 172.21.15.2:0/4127078101 >> 172.21.15.2:0/4127078101 conn(0x55f2bea6fe00 msgr2=0x55f2bea6f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.859+0000 7fa9e1a38740 1 -- 172.21.15.2:0/4127078101 shutdown_connections 2024-05-14T17:15:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.859+0000 7fa9e1a38740 1 -- 172.21.15.2:0/4127078101 wait complete. 2024-05-14T17:15:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.859+0000 7fa9e1a38740 20 client.17349 trim_cache size 0 max 16384 2024-05-14T17:15:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.874+0000 7fa9e1a38740 1 Processor -- start 2024-05-14T17:15:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.874+0000 7fa9e1a38740 1 -- start start 2024-05-14T17:15:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.874+0000 7fa9e1a38740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2bea738f0 0x55f2bea8a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.874+0000 7fa9e1a38740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2bea74210 0x55f2bea95f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.874+0000 7fa9e1a38740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2bea7ead0 0x55f2bea96490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.874+0000 7fa9e1a38740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d81b2ea0 con 0x55f2bea7ead0 2024-05-14T17:15:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.874+0000 7fa9e1a38740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d81b0770 con 0x55f2bea74210 2024-05-14T17:15:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.874+0000 7fa9e1a38740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d81ae5c0 con 0x55f2bea738f0 2024-05-14T17:15:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.874+0000 7fa9dd1e2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2bea7ead0 0x55f2bea96490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.874+0000 7fa9dd1e2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2bea7ead0 0x55f2bea96490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52370/0 (socket says 172.21.15.2:52370) 2024-05-14T17:15:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.874+0000 7fa9dd1e2640 1 -- 172.21.15.2:0/3985149386 learned_addr learned my addr 172.21.15.2:0/3985149386 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.874+0000 7fa9d7fff640 1 --2- 172.21.15.2:0/3985149386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2bea74210 0x55f2bea95f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.874+0000 7fa9dc9e1640 1 --2- 172.21.15.2:0/3985149386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2bea738f0 0x55f2bea8a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.875+0000 7fa9dd1e2640 1 -- 172.21.15.2:0/3985149386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2bea738f0 msgr2=0x55f2bea8a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.875+0000 7fa9dd1e2640 1 --2- 172.21.15.2:0/3985149386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2bea738f0 0x55f2bea8a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.875+0000 7fa9dd1e2640 1 -- 172.21.15.2:0/3985149386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2bea74210 msgr2=0x55f2bea95f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.875+0000 7fa9dd1e2640 1 --2- 172.21.15.2:0/3985149386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2bea74210 0x55f2bea95f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.875+0000 7fa9dd1e2640 1 -- 172.21.15.2:0/3985149386 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2bea730f0 con 0x55f2bea7ead0 2024-05-14T17:15:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.875+0000 7fa9d7fff640 1 --2- 172.21.15.2:0/3985149386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2bea74210 0x55f2bea95f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:15:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.875+0000 7fa9dd1e2640 1 --2- 172.21.15.2:0/3985149386 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2bea7ead0 0x55f2bea96490 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8009890 tx=0x7fa9c802a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ed4024466848d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.875+0000 7fa9be7fc640 1 -- 172.21.15.2:0/3985149386 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c805b030 con 0x55f2bea7ead0 2024-05-14T17:15:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.875+0000 7fa9be7fc640 1 -- 172.21.15.2:0/3985149386 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9c8036040 con 0x55f2bea7ead0 2024-05-14T17:15:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.875+0000 7fa9be7fc640 1 -- 172.21.15.2:0/3985149386 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8048520 con 0x55f2bea7ead0 2024-05-14T17:15:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.875+0000 7fa9e1a38740 1 -- 172.21.15.2:0/3985149386 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2bea7ead0 msgr2=0x55f2bea96490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.876+0000 7fa9e1a38740 1 --2- 172.21.15.2:0/3985149386 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2bea7ead0 0x55f2bea96490 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8009890 tx=0x7fa9c802a8e0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.876+0000 7fa9e1a38740 1 -- 172.21.15.2:0/3985149386 shutdown_connections 2024-05-14T17:15:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.876+0000 7fa9e1a38740 1 --2- 172.21.15.2:0/3985149386 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2bea7ead0 0x55f2bea96490 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.876+0000 7fa9e1a38740 1 --2- 172.21.15.2:0/3985149386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2bea74210 0x55f2bea95f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.876+0000 7fa9e1a38740 1 --2- 172.21.15.2:0/3985149386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2bea738f0 0x55f2bea8a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.876+0000 7fa9e1a38740 1 -- 172.21.15.2:0/3985149386 >> 172.21.15.2:0/3985149386 conn(0x55f2bea6fe00 msgr2=0x55f2bea75b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.876+0000 7fa9e1a38740 1 -- 172.21.15.2:0/3985149386 shutdown_connections 2024-05-14T17:15:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.876+0000 7fa9e1a38740 1 -- 172.21.15.2:0/3985149386 wait complete. 2024-05-14T17:15:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.876+0000 7fa9e1a38740 1 -- start start 2024-05-14T17:15:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.876+0000 7fa9e1a38740 1 Processor -- start 2024-05-14T17:15:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.876+0000 7fa9e1a38740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2bea738f0 0x55f2bea870f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.876+0000 7fa9e1a38740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2bea74210 0x55f2bea82460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.876+0000 7fa9e1a38740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2bea7ead0 0x55f2bea82b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.876+0000 7fa9e1a38740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d81aec30 con 0x55f2bea7ead0 2024-05-14T17:15:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.876+0000 7fa9e1a38740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d81b00a0 con 0x55f2bea738f0 2024-05-14T17:15:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.876+0000 7fa9e1a38740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d81af300 con 0x55f2bea74210 2024-05-14T17:15:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.876+0000 7fa9e1a38740 10 client.0 _ll_register_callbacks cb 0x7fa9c8031d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-14T17:15:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.877+0000 7fa9dd1e2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2bea7ead0 0x55f2bea82b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.877+0000 7fa9dd1e2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2bea7ead0 0x55f2bea82b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52380/0 (socket says 172.21.15.2:52380) 2024-05-14T17:15:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.877+0000 7fa9dd1e2640 1 -- 172.21.15.2:0/1941914181 learned_addr learned my addr 172.21.15.2:0/1941914181 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:42.882 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.877+0000 7fa9dc9e1640 1 --2- 172.21.15.2:0/1941914181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2bea738f0 0x55f2bea870f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.877+0000 7fa9d7fff640 1 --2- 172.21.15.2:0/1941914181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2bea74210 0x55f2bea82460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.877+0000 7fa9dd1e2640 1 -- 172.21.15.2:0/1941914181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2bea74210 msgr2=0x55f2bea82460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.877+0000 7fa9dd1e2640 1 --2- 172.21.15.2:0/1941914181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2bea74210 0x55f2bea82460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.877+0000 7fa9dd1e2640 1 -- 172.21.15.2:0/1941914181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2bea738f0 msgr2=0x55f2bea870f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-14T17:15:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.877+0000 7fa9dd1e2640 1 --2- 172.21.15.2:0/1941914181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2bea738f0 0x55f2bea870f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.877+0000 7fa9dd1e2640 1 -- 172.21.15.2:0/1941914181 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2beaf7b00 con 0x55f2bea7ead0 2024-05-14T17:15:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.877+0000 7fa9d7fff640 1 --2- 172.21.15.2:0/1941914181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2bea74210 0x55f2bea82460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:15:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.877+0000 7fa9dd1e2640 1 --2- 172.21.15.2:0/1941914181 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2bea7ead0 0x55f2bea82b50 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fa9c80317f0 tx=0x7fa9c80044b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c30e552cb7bdbcee server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.877+0000 7fa9b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-05-14T17:15:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.877+0000 7fa9b7fff640 1 -- 172.21.15.2:0/1941914181 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c805b040 con 0x55f2bea7ead0 2024-05-14T17:15:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.877+0000 7fa9b7fff640 1 -- 172.21.15.2:0/1941914181 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8469+0+0 (secure 0 0 0) 0x7fa9c8052440 con 0x55f2bea7ead0 2024-05-14T17:15:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.877+0000 7fa9e1a38740 1 -- 172.21.15.2:0/1941914181 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9c8036040 con 0x55f2bea7ead0 2024-05-14T17:15:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.877+0000 7fa9b7fff640 1 -- 172.21.15.2:0/1941914181 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fa9c8052640 con 0x55f2bea7ead0 2024-05-14T17:15:42.883 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.877+0000 7fa9e1a38740 10 client.17355 fetch_fsmap learned FSMap version 60 2024-05-14T17:15:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.877+0000 7fa9e1a38740 1 -- 172.21.15.2:0/1941914181 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55f2beaf8cb0 con 0x55f2bea7ead0 2024-05-14T17:15:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.878+0000 7fa9b7fff640 1 -- 172.21.15.2:0/1941914181 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9c8036040 con 0x55f2bea7ead0 2024-05-14T17:15:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.878+0000 7fa9b7fff640 1 -- 172.21.15.2:0/1941914181 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 60) ==== 8346+0+0 (secure 0 0 0) 0x7fa9c8030050 con 0x55f2bea7ead0 2024-05-14T17:15:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.878+0000 7fa9e1a38740 10 client.17355 fetch_fsmap finished waiting for FSMap version 60 2024-05-14T17:15:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.878+0000 7fa9e1a38740 10 client.17355 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-14T17:15:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.878+0000 7fa9e1a38740 20 client.17355 populate_metadata read hostname 'smithi002' 2024-05-14T17:15:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.878+0000 7fa9e1a38740 1 --2- 172.21.15.2:0/1941914181 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55f2bea8e530 0x55f2beafb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.878+0000 7fa9e1a38740 4 client.17355 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-14T17:15:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.878+0000 7fa9e1a38740 1 -- 172.21.15.2:0/1941914181 --> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f2be9e2360 con 0x55f2bea8e530 2024-05-14T17:15:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.878+0000 7fa9dc9e1640 1 --2- 172.21.15.2:0/1941914181 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55f2bea8e530 0x55f2beafb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.878+0000 7fa9b7fff640 10 client.17355 ms_handle_connect on v2:172.21.15.123:6838/580275877 2024-05-14T17:15:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.878+0000 7fa9dc9e1640 1 --2- 172.21.15.2:0/1941914181 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55f2bea8e530 0x55f2beafb3d0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4449761b54ad4172 server_cookie=9aef3c1c34ec6035 in_seq=0 out_seq=0 2024-05-14T17:15:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.883+0000 7fa9b7fff640 1 -- 172.21.15.2:0/1941914181 <== mds.0 v2:172.21.15.123:6838/580275877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f2be9e2360 con 0x55f2bea8e530 2024-05-14T17:15:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.883+0000 7fa9b7fff640 10 client.17355 handle_command_reply: tid=0 2024-05-14T17:15:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.883+0000 7fa9e1a38740 1 client.17355 shutdown 2024-05-14T17:15:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.883+0000 7fa9e1a38740 1 -- 172.21.15.2:0/1941914181 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2bea7ead0 msgr2=0x55f2bea82b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.883+0000 7fa9e1a38740 1 --2- 172.21.15.2:0/1941914181 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2bea7ead0 0x55f2bea82b50 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fa9c80317f0 tx=0x7fa9c80044b0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.883+0000 7fa9e1a38740 1 -- 172.21.15.2:0/1941914181 shutdown_connections 2024-05-14T17:15:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.883+0000 7fa9e1a38740 1 --2- 172.21.15.2:0/1941914181 >> [v2:172.21.15.123:6838/580275877,v1:172.21.15.123:6839/580275877] conn(0x55f2bea8e530 0x55f2beafb3d0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.883+0000 7fa9e1a38740 1 --2- 172.21.15.2:0/1941914181 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f2bea7ead0 0x55f2bea82b50 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.883+0000 7fa9e1a38740 1 --2- 172.21.15.2:0/1941914181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f2bea74210 0x55f2bea82460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.883+0000 7fa9b7fff640 0 client.17355 ms_handle_reset on v2:172.21.15.123:6838/580275877 2024-05-14T17:15:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.883+0000 7fa9e1a38740 1 --2- 172.21.15.2:0/1941914181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2bea738f0 0x55f2bea870f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.883+0000 7fa9e1a38740 1 -- 172.21.15.2:0/1941914181 >> 172.21.15.2:0/1941914181 conn(0x55f2bea6fe00 msgr2=0x55f2bea7cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.883+0000 7fa9e1a38740 1 -- 172.21.15.2:0/1941914181 shutdown_connections 2024-05-14T17:15:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.883+0000 7fa9e1a38740 1 -- 172.21.15.2:0/1941914181 wait complete. 2024-05-14T17:15:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.883+0000 7fa9e1a38740 20 client.17355 trim_cache size 0 max 16384 2024-05-14T17:15:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:{ 2024-05-14T17:15:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout: "return_code": 0 2024-05-14T17:15:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stdout:} 2024-05-14T17:15:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.884+0000 7fa9dec45640 1 -- 172.21.15.2:0/2106588349 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fa9c00b2a90 msgr2=0x7fa9c00b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.884+0000 7fa9dec45640 1 --2- 172.21.15.2:0/2106588349 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fa9c00b2a90 0x7fa9c00b4f50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa9c804d000 tx=0x7fa9c804e000 comp rx=0 tx=0).stop 2024-05-14T17:15:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.884+0000 7fa9dec45640 1 -- 172.21.15.2:0/2106588349 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9d8103420 msgr2=0x7fa9d819b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.884+0000 7fa9dec45640 1 --2- 172.21.15.2:0/2106588349 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9d8103420 0x7fa9d819b7d0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00c920 tx=0x7fa9cc00cde0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.884+0000 7fa9dec45640 1 -- 172.21.15.2:0/2106588349 shutdown_connections 2024-05-14T17:15:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.884+0000 7fa9dec45640 1 --2- 172.21.15.2:0/2106588349 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fa9c00b2a90 0x7fa9c00b4f50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.884+0000 7fa9dec45640 1 --2- 172.21.15.2:0/2106588349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9d8108a70 0x7fa9d819fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.884+0000 7fa9dec45640 1 --2- 172.21.15.2:0/2106588349 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9d8103420 0x7fa9d819b7d0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.884+0000 7fa9dec45640 1 --2- 172.21.15.2:0/2106588349 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9d8102a60 0x7fa9d819b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.884+0000 7fa9dec45640 1 -- 172.21.15.2:0/2106588349 >> 172.21.15.2:0/2106588349 conn(0x7fa9d80fe550 msgr2=0x7fa9d8106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.884+0000 7fa9dec45640 1 -- 172.21.15.2:0/2106588349 shutdown_connections 2024-05-14T17:15:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi002.front.sepia.ceph.com.smithi002.stderr:2024-05-14T17:15:42.884+0000 7fa9dec45640 1 -- 172.21.15.2:0/2106588349 wait complete. 2024-05-14T17:15:43.214 ERROR:teuthology.run_tasks:Manager failed: quiescer Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/qa/tasks/quiescer.py", line 419, in task stop_all_quiescers(ctx.ceph[cluster_name].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/qa/tasks/quiescer.py", line 378, in stop_all_quiescers raise RuntimeError(f"error during quiesce thrashing: {thrasher.exception}") RuntimeError: error during quiesce thrashing: Error releasing set 'e0c7677c': 1 (EPERM) 2024-05-14T17:15:43.215 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-05-14T17:15:43.227 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi002.front.sepia.ceph.com 2024-05-14T17:15:43.235 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-05-14T17:15:43.277 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-05-14T17:15:43.277 INFO:teuthology.task.exec:Executing custom commands... 2024-05-14T17:15:43.277 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi002.front.sepia.ceph.com 2024-05-14T17:15:43.277 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-05-14T17:15:43.308 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-05-14T17:15:43.376 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-14T17:15:43.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.523+0000 7f91c2672640 1 -- 172.21.15.2:0/601102796 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f91bc102660 msgr2=0x7f91bc10e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:43.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.523+0000 7f91c2672640 1 --2- 172.21.15.2:0/601102796 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f91bc102660 0x7f91bc10e770 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f91b4009ca0 tx=0x7f91b402a8c0 comp rx=0 tx=0).stop 2024-05-14T17:15:43.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.523+0000 7f91c2672640 1 -- 172.21.15.2:0/601102796 shutdown_connections 2024-05-14T17:15:43.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.523+0000 7f91c2672640 1 --2- 172.21.15.2:0/601102796 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f91bc102660 0x7f91bc10e770 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:43.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.523+0000 7f91c2672640 1 --2- 172.21.15.2:0/601102796 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91bc101ca0 0x7f91bc102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:43.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.524+0000 7f91c2672640 1 --2- 172.21.15.2:0/601102796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91bc1068f0 0x7f91bc106cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:43.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.524+0000 7f91c2672640 1 -- 172.21.15.2:0/601102796 >> 172.21.15.2:0/601102796 conn(0x7f91bc077750 msgr2=0x7f91bc077b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:43.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.524+0000 7f91c2672640 1 -- 172.21.15.2:0/601102796 shutdown_connections 2024-05-14T17:15:43.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.524+0000 7f91c2672640 1 -- 172.21.15.2:0/601102796 wait complete. 2024-05-14T17:15:43.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.524+0000 7f91c2672640 1 Processor -- start 2024-05-14T17:15:43.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.524+0000 7f91c2672640 1 -- start start 2024-05-14T17:15:43.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.525+0000 7f91c2672640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91bc101ca0 0x7f91bc199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:43.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.525+0000 7f91c2672640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f91bc102660 0x7f91bc19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:43.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.525+0000 7f91c2672640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91bc1068f0 0x7f91bc1a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:43.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.525+0000 7f91c2672640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f91bc111bd0 con 0x7f91bc102660 2024-05-14T17:15:43.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.525+0000 7f91c2672640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f91bc111a50 con 0x7f91bc1068f0 2024-05-14T17:15:43.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.525+0000 7f91c2672640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f91bc111d50 con 0x7f91bc101ca0 2024-05-14T17:15:43.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.525+0000 7f91bb7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f91bc102660 0x7f91bc19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:43.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.525+0000 7f91bb7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f91bc102660 0x7f91bc19a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52392/0 (socket says 172.21.15.2:52392) 2024-05-14T17:15:43.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.525+0000 7f91bb7fe640 1 -- 172.21.15.2:0/93345602 learned_addr learned my addr 172.21.15.2:0/93345602 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:43.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.525+0000 7f91bbfff640 1 --2- 172.21.15.2:0/93345602 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91bc101ca0 0x7f91bc199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:43.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.525+0000 7f91c0c0f640 1 --2- 172.21.15.2:0/93345602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91bc1068f0 0x7f91bc1a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:43.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.525+0000 7f91bb7fe640 1 -- 172.21.15.2:0/93345602 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91bc101ca0 msgr2=0x7f91bc199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:43.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.525+0000 7f91bb7fe640 1 --2- 172.21.15.2:0/93345602 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91bc101ca0 0x7f91bc199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:43.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.525+0000 7f91bb7fe640 1 -- 172.21.15.2:0/93345602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91bc1068f0 msgr2=0x7f91bc1a4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:43.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.525+0000 7f91bb7fe640 1 --2- 172.21.15.2:0/93345602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91bc1068f0 0x7f91bc1a4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:43.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.525+0000 7f91bb7fe640 1 -- 172.21.15.2:0/93345602 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91bc1a5510 con 0x7f91bc102660 2024-05-14T17:15:43.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.525+0000 7f91c0c0f640 1 --2- 172.21.15.2:0/93345602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91bc1068f0 0x7f91bc1a4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:15:43.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.525+0000 7f91bb7fe640 1 --2- 172.21.15.2:0/93345602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f91bc102660 0x7f91bc19a490 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f91b000d770 tx=0x7f91b000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:43.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.525+0000 7f91b97fa640 1 -- 172.21.15.2:0/93345602 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b0017070 con 0x7f91bc102660 2024-05-14T17:15:43.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.525+0000 7f91b97fa640 1 -- 172.21.15.2:0/93345602 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91b00054d0 con 0x7f91bc102660 2024-05-14T17:15:43.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.525+0000 7f91b97fa640 1 -- 172.21.15.2:0/93345602 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b0004c30 con 0x7f91bc102660 2024-05-14T17:15:43.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.525+0000 7f91c2672640 1 -- 172.21.15.2:0/93345602 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91bc1a56e0 con 0x7f91bc102660 2024-05-14T17:15:43.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.525+0000 7f91c2672640 1 -- 172.21.15.2:0/93345602 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91bc1a5b50 con 0x7f91bc102660 2024-05-14T17:15:43.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.526+0000 7f91c2672640 1 -- 172.21.15.2:0/93345602 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91bc103050 con 0x7f91bc102660 2024-05-14T17:15:43.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.528+0000 7f91b97fa640 1 -- 172.21.15.2:0/93345602 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f91b0004dd0 con 0x7f91bc102660 2024-05-14T17:15:43.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.528+0000 7f91b97fa640 1 --2- 172.21.15.2:0/93345602 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f91a40b2a90 0x7f91a40b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:43.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.529+0000 7f91bbfff640 1 --2- 172.21.15.2:0/93345602 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f91a40b2a90 0x7f91a40b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:43.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.529+0000 7f91b97fa640 1 -- 172.21.15.2:0/93345602 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8469+0+0 (secure 0 0 0) 0x7f91b00e2510 con 0x7f91bc102660 2024-05-14T17:15:43.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.529+0000 7f91bbfff640 1 --2- 172.21.15.2:0/93345602 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f91a40b2a90 0x7f91a40b4f50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f91ac0099e0 tx=0x7f91ac005ed0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:43.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.532+0000 7f91b97fa640 1 -- 172.21.15.2:0/93345602 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f91b00a9fa0 con 0x7f91bc102660 2024-05-14T17:15:43.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:43 smithi002 ceph-mon[31538]: pgmap v663: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-05-14T17:15:43.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.747+0000 7f91c2672640 1 -- 172.21.15.2:0/93345602 --> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f91bc062e90 con 0x7f91a40b2a90 2024-05-14T17:15:43.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.755+0000 7f91b97fa640 1 -- 172.21.15.2:0/93345602 <== mgr.15120 v2:172.21.15.2:6800/1339485805 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7f91bc062e90 con 0x7f91a40b2a90 2024-05-14T17:15:43.755 INFO:teuthology.orchestra.run.smithi002.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-14T00:00:00", "created": "2024-05-14T16:54:42", "first": "2024-05-14T16:55:00", "last": "2024-05-14T17:15:00", "last_pruned": "2024-05-14T17:15:00", "created_count": 21, "pruned_count": 14, "active": true} 2024-05-14T17:15:43.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.757+0000 7f91c2672640 1 -- 172.21.15.2:0/93345602 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f91a40b2a90 msgr2=0x7f91a40b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:43.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.757+0000 7f91c2672640 1 --2- 172.21.15.2:0/93345602 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f91a40b2a90 0x7f91a40b4f50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f91ac0099e0 tx=0x7f91ac005ed0 comp rx=0 tx=0).stop 2024-05-14T17:15:43.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.757+0000 7f91c2672640 1 -- 172.21.15.2:0/93345602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f91bc102660 msgr2=0x7f91bc19a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:43.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.757+0000 7f91c2672640 1 --2- 172.21.15.2:0/93345602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f91bc102660 0x7f91bc19a490 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f91b000d770 tx=0x7f91b000dc30 comp rx=0 tx=0).stop 2024-05-14T17:15:43.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.757+0000 7f91c2672640 1 -- 172.21.15.2:0/93345602 shutdown_connections 2024-05-14T17:15:43.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.757+0000 7f91c2672640 1 --2- 172.21.15.2:0/93345602 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f91a40b2a90 0x7f91a40b4f50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:43.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.757+0000 7f91c2672640 1 --2- 172.21.15.2:0/93345602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91bc1068f0 0x7f91bc1a4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:43.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.757+0000 7f91c2672640 1 --2- 172.21.15.2:0/93345602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f91bc102660 0x7f91bc19a490 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:43.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.758+0000 7f91c2672640 1 --2- 172.21.15.2:0/93345602 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91bc101ca0 0x7f91bc199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:43.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.758+0000 7f91c2672640 1 -- 172.21.15.2:0/93345602 >> 172.21.15.2:0/93345602 conn(0x7f91bc077750 msgr2=0x7f91bc0ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:43.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.758+0000 7f91c2672640 1 -- 172.21.15.2:0/93345602 shutdown_connections 2024-05-14T17:15:43.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.758+0000 7f91c2672640 1 -- 172.21.15.2:0/93345602 wait complete. 2024-05-14T17:15:43.768 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-14T17:15:43.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:43 smithi151 ceph-mon[36066]: pgmap v663: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-05-14T17:15:43.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.875+0000 7f2349ea0640 1 -- 172.21.15.2:0/3306244856 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2344108a70 msgr2=0x7f2344108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:43.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.875+0000 7f2349ea0640 1 --2- 172.21.15.2:0/3306244856 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2344108a70 0x7f2344108e50 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f2334009930 tx=0x7f233402a420 comp rx=0 tx=0).stop 2024-05-14T17:15:43.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.875+0000 7f2349ea0640 1 -- 172.21.15.2:0/3306244856 shutdown_connections 2024-05-14T17:15:43.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.875+0000 7f2349ea0640 1 --2- 172.21.15.2:0/3306244856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2344103420 0x7f234410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:43.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.875+0000 7f2349ea0640 1 --2- 172.21.15.2:0/3306244856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2344102a60 0x7f2344102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:43.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.875+0000 7f2349ea0640 1 --2- 172.21.15.2:0/3306244856 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2344108a70 0x7f2344108e50 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:43.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.875+0000 7f2349ea0640 1 -- 172.21.15.2:0/3306244856 >> 172.21.15.2:0/3306244856 conn(0x7f2344079ee0 msgr2=0x7f234407a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:43.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.875+0000 7f2349ea0640 1 -- 172.21.15.2:0/3306244856 shutdown_connections 2024-05-14T17:15:43.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.875+0000 7f2349ea0640 1 -- 172.21.15.2:0/3306244856 wait complete. 2024-05-14T17:15:43.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.876+0000 7f2349ea0640 1 Processor -- start 2024-05-14T17:15:43.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.876+0000 7f2349ea0640 1 -- start start 2024-05-14T17:15:43.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.876+0000 7f2349ea0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2344102a60 0x7f23441991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:43.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.876+0000 7f2349ea0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2344103420 0x7f2344199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:43.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.876+0000 7f2349ea0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2344108a70 0x7f234419da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:43.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.876+0000 7f2349ea0640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2344112d30 con 0x7f2344108a70 2024-05-14T17:15:43.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.876+0000 7f2349ea0640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2344112bb0 con 0x7f2344102a60 2024-05-14T17:15:43.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.876+0000 7f2349ea0640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2344112eb0 con 0x7f2344103420 2024-05-14T17:15:43.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.876+0000 7f2343fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2344108a70 0x7f234419da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:43.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.876+0000 7f2343fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2344108a70 0x7f234419da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34508/0 (socket says 172.21.15.2:34508) 2024-05-14T17:15:43.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.876+0000 7f2343fff640 1 -- 172.21.15.2:0/300474420 learned_addr learned my addr 172.21.15.2:0/300474420 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:43.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.876+0000 7f23437fe640 1 --2- 172.21.15.2:0/300474420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2344102a60 0x7f23441991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:43.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.877+0000 7f2343fff640 1 -- 172.21.15.2:0/300474420 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2344103420 msgr2=0x7f2344199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:43.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.877+0000 7f2342ffd640 1 --2- 172.21.15.2:0/300474420 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2344103420 0x7f2344199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:43.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.877+0000 7f2343fff640 1 --2- 172.21.15.2:0/300474420 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2344103420 0x7f2344199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:43.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.877+0000 7f2343fff640 1 -- 172.21.15.2:0/300474420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2344102a60 msgr2=0x7f23441991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:43.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.877+0000 7f2343fff640 1 --2- 172.21.15.2:0/300474420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2344102a60 0x7f23441991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:43.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.877+0000 7f2343fff640 1 -- 172.21.15.2:0/300474420 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f234419e1c0 con 0x7f2344108a70 2024-05-14T17:15:43.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.877+0000 7f23437fe640 1 --2- 172.21.15.2:0/300474420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2344102a60 0x7f23441991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:15:43.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.877+0000 7f2343fff640 1 --2- 172.21.15.2:0/300474420 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2344108a70 0x7f234419da90 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f233c00cab0 tx=0x7f233c00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:43.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.877+0000 7f2340ff9640 1 -- 172.21.15.2:0/300474420 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233c00ae20 con 0x7f2344108a70 2024-05-14T17:15:43.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.877+0000 7f2340ff9640 1 -- 172.21.15.2:0/300474420 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f233c00f900 con 0x7f2344108a70 2024-05-14T17:15:43.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.877+0000 7f2349ea0640 1 -- 172.21.15.2:0/300474420 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f234419e4b0 con 0x7f2344108a70 2024-05-14T17:15:43.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.877+0000 7f2340ff9640 1 -- 172.21.15.2:0/300474420 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233c0189f0 con 0x7f2344108a70 2024-05-14T17:15:43.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.877+0000 7f2349ea0640 1 -- 172.21.15.2:0/300474420 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23441a5d40 con 0x7f2344108a70 2024-05-14T17:15:43.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.878+0000 7f2349ea0640 1 -- 172.21.15.2:0/300474420 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23440507e0 con 0x7f2344108a70 2024-05-14T17:15:43.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.880+0000 7f2340ff9640 1 -- 172.21.15.2:0/300474420 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f233c00fab0 con 0x7f2344108a70 2024-05-14T17:15:43.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.880+0000 7f2340ff9640 1 --2- 172.21.15.2:0/300474420 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f232c0b2a90 0x7f232c0b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:43.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.881+0000 7f23437fe640 1 --2- 172.21.15.2:0/300474420 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f232c0b2a90 0x7f232c0b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:43.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.881+0000 7f2340ff9640 1 -- 172.21.15.2:0/300474420 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8469+0+0 (secure 0 0 0) 0x7f233c0e2ea0 con 0x7f2344108a70 2024-05-14T17:15:43.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.881+0000 7f23437fe640 1 --2- 172.21.15.2:0/300474420 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f232c0b2a90 0x7f232c0b4f50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2334000c00 tx=0x7f2334031d10 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:43.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:43.884+0000 7f2340ff9640 1 -- 172.21.15.2:0/300474420 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f233c014030 con 0x7f2344108a70 2024-05-14T17:15:43.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:43 smithi123 ceph-mon[35971]: pgmap v663: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-05-14T17:15:44.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.098+0000 7f2349ea0640 1 -- 172.21.15.2:0/300474420 --> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f2344062e90 con 0x7f232c0b2a90 2024-05-14T17:15:44.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.106+0000 7f2340ff9640 1 -- 172.21.15.2:0/300474420 <== mgr.15120 v2:172.21.15.2:6800/1339485805 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f2344062e90 con 0x7f232c0b2a90 2024-05-14T17:15:44.105 INFO:teuthology.orchestra.run.smithi002.stdout:/ 1m 6m3h 2024-05-14T17:15:44.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.108+0000 7f2349ea0640 1 -- 172.21.15.2:0/300474420 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f232c0b2a90 msgr2=0x7f232c0b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:44.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.108+0000 7f2349ea0640 1 --2- 172.21.15.2:0/300474420 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f232c0b2a90 0x7f232c0b4f50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2334000c00 tx=0x7f2334031d10 comp rx=0 tx=0).stop 2024-05-14T17:15:44.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.108+0000 7f2349ea0640 1 -- 172.21.15.2:0/300474420 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2344108a70 msgr2=0x7f234419da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:44.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.108+0000 7f2349ea0640 1 --2- 172.21.15.2:0/300474420 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2344108a70 0x7f234419da90 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f233c00cab0 tx=0x7f233c00cf70 comp rx=0 tx=0).stop 2024-05-14T17:15:44.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.108+0000 7f2349ea0640 1 -- 172.21.15.2:0/300474420 shutdown_connections 2024-05-14T17:15:44.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.108+0000 7f2349ea0640 1 --2- 172.21.15.2:0/300474420 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f232c0b2a90 0x7f232c0b4f50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:44.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.108+0000 7f2349ea0640 1 --2- 172.21.15.2:0/300474420 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2344108a70 0x7f234419da90 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:44.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.108+0000 7f2349ea0640 1 --2- 172.21.15.2:0/300474420 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2344103420 0x7f2344199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:44.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.108+0000 7f2349ea0640 1 --2- 172.21.15.2:0/300474420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2344102a60 0x7f23441991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:44.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.108+0000 7f2349ea0640 1 -- 172.21.15.2:0/300474420 >> 172.21.15.2:0/300474420 conn(0x7f2344079ee0 msgr2=0x7f2344100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:44.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.108+0000 7f2349ea0640 1 -- 172.21.15.2:0/300474420 shutdown_connections 2024-05-14T17:15:44.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.108+0000 7f2349ea0640 1 -- 172.21.15.2:0/300474420 wait complete. 2024-05-14T17:15:44.118 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-14T17:15:44.129 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-14T17:15:44.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.234+0000 7f8a5636a640 1 -- 172.21.15.2:0/1487740884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a50104650 msgr2=0x7f8a5010cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:44.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.234+0000 7f8a5636a640 1 --2- 172.21.15.2:0/1487740884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a50104650 0x7f8a5010cb30 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f8a44009900 tx=0x7f8a4402a480 comp rx=0 tx=0).stop 2024-05-14T17:15:44.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.234+0000 7f8a5636a640 1 -- 172.21.15.2:0/1487740884 shutdown_connections 2024-05-14T17:15:44.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.234+0000 7f8a5636a640 1 --2- 172.21.15.2:0/1487740884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a5010d070 0x7f8a5010f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:44.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.234+0000 7f8a5636a640 1 --2- 172.21.15.2:0/1487740884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a50104650 0x7f8a5010cb30 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:44.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.234+0000 7f8a5636a640 1 --2- 172.21.15.2:0/1487740884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a50103d30 0x7f8a50104110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:44.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.234+0000 7f8a5636a640 1 -- 172.21.15.2:0/1487740884 >> 172.21.15.2:0/1487740884 conn(0x7f8a50078ab0 msgr2=0x7f8a50078ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:44.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.234+0000 7f8a5636a640 1 -- 172.21.15.2:0/1487740884 shutdown_connections 2024-05-14T17:15:44.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.234+0000 7f8a5636a640 1 -- 172.21.15.2:0/1487740884 wait complete. 2024-05-14T17:15:44.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.235+0000 7f8a5636a640 1 Processor -- start 2024-05-14T17:15:44.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.235+0000 7f8a5636a640 1 -- start start 2024-05-14T17:15:44.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.235+0000 7f8a5636a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a50103d30 0x7f8a501992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:44.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.235+0000 7f8a5636a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a50104650 0x7f8a50199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:44.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.235+0000 7f8a5636a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a5010d070 0x7f8a5019dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:44.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.235+0000 7f8a5636a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8a50112cf0 con 0x7f8a5010d070 2024-05-14T17:15:44.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.235+0000 7f8a5636a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8a50112b70 con 0x7f8a50104650 2024-05-14T17:15:44.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.235+0000 7f8a5636a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8a50112e70 con 0x7f8a50103d30 2024-05-14T17:15:44.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.235+0000 7f8a54907640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a5010d070 0x7f8a5019dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:44.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.235+0000 7f8a54907640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a5010d070 0x7f8a5019dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34522/0 (socket says 172.21.15.2:34522) 2024-05-14T17:15:44.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.235+0000 7f8a54907640 1 -- 172.21.15.2:0/1569225546 learned_addr learned my addr 172.21.15.2:0/1569225546 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:44.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.235+0000 7f8a4f7fe640 1 --2- 172.21.15.2:0/1569225546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a50104650 0x7f8a50199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:44.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.235+0000 7f8a4ffff640 1 --2- 172.21.15.2:0/1569225546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a50103d30 0x7f8a501992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:44.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.236+0000 7f8a54907640 1 -- 172.21.15.2:0/1569225546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a50103d30 msgr2=0x7f8a501992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:44.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.236+0000 7f8a54907640 1 --2- 172.21.15.2:0/1569225546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a50103d30 0x7f8a501992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:44.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.236+0000 7f8a54907640 1 -- 172.21.15.2:0/1569225546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a50104650 msgr2=0x7f8a50199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:44.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.236+0000 7f8a54907640 1 --2- 172.21.15.2:0/1569225546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a50104650 0x7f8a50199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:44.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.236+0000 7f8a54907640 1 -- 172.21.15.2:0/1569225546 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a5019e2e0 con 0x7f8a5010d070 2024-05-14T17:15:44.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.236+0000 7f8a54907640 1 --2- 172.21.15.2:0/1569225546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a5010d070 0x7f8a5019dbb0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f8a4800cab0 tx=0x7f8a4800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:44.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.236+0000 7f8a4d7fa640 1 -- 172.21.15.2:0/1569225546 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4800adc0 con 0x7f8a5010d070 2024-05-14T17:15:44.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.236+0000 7f8a4d7fa640 1 -- 172.21.15.2:0/1569225546 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a4800f900 con 0x7f8a5010d070 2024-05-14T17:15:44.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.236+0000 7f8a5636a640 1 -- 172.21.15.2:0/1569225546 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a5019e5d0 con 0x7f8a5010d070 2024-05-14T17:15:44.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.236+0000 7f8a4d7fa640 1 -- 172.21.15.2:0/1569225546 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a480189a0 con 0x7f8a5010d070 2024-05-14T17:15:44.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.236+0000 7f8a5636a640 1 -- 172.21.15.2:0/1569225546 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a501a5e60 con 0x7f8a5010d070 2024-05-14T17:15:44.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.237+0000 7f8a5636a640 1 -- 172.21.15.2:0/1569225546 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a50069cc0 con 0x7f8a5010d070 2024-05-14T17:15:44.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.238+0000 7f8a4d7fa640 1 -- 172.21.15.2:0/1569225546 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f8a4800f450 con 0x7f8a5010d070 2024-05-14T17:15:44.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.239+0000 7f8a4d7fa640 1 --2- 172.21.15.2:0/1569225546 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f8a380b2960 0x7f8a380b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:44.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.240+0000 7f8a4ffff640 1 --2- 172.21.15.2:0/1569225546 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f8a380b2960 0x7f8a380b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:44.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.240+0000 7f8a4d7fa640 1 -- 172.21.15.2:0/1569225546 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8469+0+0 (secure 0 0 0) 0x7f8a480e29e0 con 0x7f8a5010d070 2024-05-14T17:15:44.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.240+0000 7f8a4ffff640 1 --2- 172.21.15.2:0/1569225546 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f8a380b2960 0x7f8a380b4e20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8a400058c0 tx=0x7f8a4000a4c0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:44.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.243+0000 7f8a4d7fa640 1 -- 172.21.15.2:0/1569225546 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8a48054df0 con 0x7f8a5010d070 2024-05-14T17:15:44.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.436+0000 7f8a5636a640 1 -- 172.21.15.2:0/1569225546 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8a5019a640 con 0x7f8a5010d070 2024-05-14T17:15:44.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.438+0000 7f8a4d7fa640 1 -- 172.21.15.2:0/1569225546 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) ==== 74+0+22972 (secure 0 0 0) 0x7f8a5019a640 con 0x7f8a5010d070 2024-05-14T17:15:44.438 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:15:44.438 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":110,"fsid":"ed40717a-1210-11ef-bc98-c7b262605968","created":"2024-05-14T16:43:09.415298+0000","modified":"2024-05-14T17:15:16.166651+0000","last_up_change":"2024-05-14T16:50:29.787654+0000","last_in_change":"2024-05-14T16:50:09.287932+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T16:46:38.090903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T16:51:27.296955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T16:51:31.103019+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":36,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"153f20ec-5bcf-4786-9ede-2e512a1e3d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6803","nonce":2654237820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6805","nonce":2654237820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6809","nonce":2654237820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6807","nonce":2654237820}]},"public_addr":"172.21.15.2:6803/2654237820","cluster_addr":"172.21.15.2:6805/2654237820","heartbeat_back_addr":"172.21.15.2:6809/2654237820","heartbeat_front_addr":"172.21.15.2:6807/2654237820","state":["exists","up"]},{"osd":1,"uuid":"0e95419f-9a9d-4f7a-b55d-ecb54971918a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6801","nonce":2668156992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6803","nonce":2668156992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6807","nonce":2668156992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6805","nonce":2668156992}]},"public_addr":"172.21.15.123:6801/2668156992","cluster_addr":"172.21.15.123:6803/2668156992","heartbeat_back_addr":"172.21.15.123:6807/2668156992","heartbeat_front_addr":"172.21.15.123:6805/2668156992","state":["exists","up"]},{"osd":2,"uuid":"b6ce7ac7-be6c-4549-a356-59cb81b5b492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6801","nonce":2160189636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6803","nonce":2160189636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6807","nonce":2160189636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6805","nonce":2160189636}]},"public_addr":"172.21.15.151:6801/2160189636","cluster_addr":"172.21.15.151:6803/2160189636","heartbeat_back_addr":"172.21.15.151:6807/2160189636","heartbeat_front_addr":"172.21.15.151:6805/2160189636","state":["exists","up"]},{"osd":3,"uuid":"2fd592bf-c212-411a-afbe-a7c498e5ac93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6811","nonce":3451424073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6813","nonce":3451424073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6817","nonce":3451424073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6815","nonce":3451424073}]},"public_addr":"172.21.15.2:6811/3451424073","cluster_addr":"172.21.15.2:6813/3451424073","heartbeat_back_addr":"172.21.15.2:6817/3451424073","heartbeat_front_addr":"172.21.15.2:6815/3451424073","state":["exists","up"]},{"osd":4,"uuid":"af1905e7-e036-47c6-be59-b085bc41fede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6809","nonce":2101935762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6811","nonce":2101935762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6815","nonce":2101935762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6813","nonce":2101935762}]},"public_addr":"172.21.15.123:6809/2101935762","cluster_addr":"172.21.15.123:6811/2101935762","heartbeat_back_addr":"172.21.15.123:6815/2101935762","heartbeat_front_addr":"172.21.15.123:6813/2101935762","state":["exists","up"]},{"osd":5,"uuid":"9c166916-af78-4cc2-bd14-d1220b6e7d9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6809","nonce":3695961886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6811","nonce":3695961886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6815","nonce":3695961886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6813","nonce":3695961886}]},"public_addr":"172.21.15.151:6809/3695961886","cluster_addr":"172.21.15.151:6811/3695961886","heartbeat_back_addr":"172.21.15.151:6815/3695961886","heartbeat_front_addr":"172.21.15.151:6813/3695961886","state":["exists","up"]},{"osd":6,"uuid":"60e9bf66-2338-41e8-9280-398d4e3b322f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6819","nonce":2711893600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6821","nonce":2711893600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6825","nonce":2711893600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6823","nonce":2711893600}]},"public_addr":"172.21.15.2:6819/2711893600","cluster_addr":"172.21.15.2:6821/2711893600","heartbeat_back_addr":"172.21.15.2:6825/2711893600","heartbeat_front_addr":"172.21.15.2:6823/2711893600","state":["exists","up"]},{"osd":7,"uuid":"c93e07a1-e941-493e-8468-33429df2dff0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6817","nonce":3634155507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6819","nonce":3634155507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6823","nonce":3634155507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6821","nonce":3634155507}]},"public_addr":"172.21.15.123:6817/3634155507","cluster_addr":"172.21.15.123:6819/3634155507","heartbeat_back_addr":"172.21.15.123:6823/3634155507","heartbeat_front_addr":"172.21.15.123:6821/3634155507","state":["exists","up"]},{"osd":8,"uuid":"1155fb58-d614-4f45-91e2-5da3a93e7d9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6817","nonce":4271107010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6819","nonce":4271107010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6823","nonce":4271107010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6821","nonce":4271107010}]},"public_addr":"172.21.15.151:6817/4271107010","cluster_addr":"172.21.15.151:6819/4271107010","heartbeat_back_addr":"172.21.15.151:6823/4271107010","heartbeat_front_addr":"172.21.15.151:6821/4271107010","state":["exists","up"]},{"osd":9,"uuid":"82b34e3b-0017-4ca4-8f86-0993ca78ce7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6827","nonce":3479431021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6829","nonce":3479431021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6833","nonce":3479431021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6831","nonce":3479431021}]},"public_addr":"172.21.15.2:6827/3479431021","cluster_addr":"172.21.15.2:6829/3479431021","heartbeat_back_addr":"172.21.15.2:6833/3479431021","heartbeat_front_addr":"172.21.15.2:6831/3479431021","state":["exists","up"]},{"osd":10,"uuid":"32e2ace9-eb5d-4258-a2ac-49b782be2b5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6825","nonce":4266374435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6827","nonce":4266374435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6831","nonce":4266374435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6829","nonce":4266374435}]},"public_addr":"172.21.15.123:6825/4266374435","cluster_addr":"172.21.15.123:6827/4266374435","heartbeat_back_addr":"172.21.15.123:6831/4266374435","heartbeat_front_addr":"172.21.15.123:6829/4266374435","state":["exists","up"]},{"osd":11,"uuid":"6e3c2ead-1af6-4bda-87cd-12770c81fa67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6825","nonce":4129194698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6827","nonce":4129194698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6831","nonce":4129194698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6829","nonce":4129194698}]},"public_addr":"172.21.15.151:6825/4129194698","cluster_addr":"172.21.15.151:6827/4129194698","heartbeat_back_addr":"172.21.15.151:6831/4129194698","heartbeat_front_addr":"172.21.15.151:6829/4129194698","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:45:46.540023+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:46:10.845186+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:46:34.283112+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:47:00.072467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:47:25.315644+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:47:49.748688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:48:16.964258+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:48:42.875515+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:49:08.596500+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:49:35.450943+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:50:01.293368+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:50:27.793713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/2907436778":"2024-05-15T16:54:41.187591+0000","172.21.15.2:0/2111644395":"2024-05-15T16:54:41.187591+0000","172.21.15.2:6801/1498936028":"2024-05-15T16:54:41.187591+0000","172.21.15.2:6800/1498936028":"2024-05-15T16:54:41.187591+0000","172.21.15.2:0/156580230":"2024-05-15T16:54:41.187591+0000","172.21.15.2:0/4267809161":"2024-05-15T16:44:11.998214+0000","172.21.15.2:0/4090012382":"2024-05-15T16:54:41.187591+0000","172.21.15.2:6801/3582988344":"2024-05-15T16:44:11.998214+0000","172.21.15.2:6800/2574015037":"2024-05-15T16:43:43.407184+0000","172.21.15.2:0/2100732571":"2024-05-15T16:44:11.998214+0000","172.21.15.2:0/2954199324":"2024-05-15T16:43:43.407184+0000","172.21.15.2:6801/2574015037":"2024-05-15T16:43:43.407184+0000","172.21.15.2:0/3171550944":"2024-05-15T16:43:43.407184+0000","172.21.15.2:0/1679761136":"2024-05-15T16:44:11.998214+0000","172.21.15.2:0/1746766591":"2024-05-15T16:43:43.407184+0000","172.21.15.2:6800/3582988344":"2024-05-15T16:44:11.998214+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T17:15:44.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.440+0000 7f8a5636a640 1 -- 172.21.15.2:0/1569225546 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f8a380b2960 msgr2=0x7f8a380b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:44.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.440+0000 7f8a5636a640 1 --2- 172.21.15.2:0/1569225546 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f8a380b2960 0x7f8a380b4e20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8a400058c0 tx=0x7f8a4000a4c0 comp rx=0 tx=0).stop 2024-05-14T17:15:44.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.440+0000 7f8a5636a640 1 -- 172.21.15.2:0/1569225546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a5010d070 msgr2=0x7f8a5019dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:44.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.440+0000 7f8a5636a640 1 --2- 172.21.15.2:0/1569225546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a5010d070 0x7f8a5019dbb0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f8a4800cab0 tx=0x7f8a4800cf70 comp rx=0 tx=0).stop 2024-05-14T17:15:44.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.440+0000 7f8a5636a640 1 -- 172.21.15.2:0/1569225546 shutdown_connections 2024-05-14T17:15:44.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.440+0000 7f8a5636a640 1 --2- 172.21.15.2:0/1569225546 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f8a380b2960 0x7f8a380b4e20 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:44.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.440+0000 7f8a5636a640 1 --2- 172.21.15.2:0/1569225546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a5010d070 0x7f8a5019dbb0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:44.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.440+0000 7f8a5636a640 1 --2- 172.21.15.2:0/1569225546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a50104650 0x7f8a50199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:44.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.440+0000 7f8a5636a640 1 --2- 172.21.15.2:0/1569225546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a50103d30 0x7f8a501992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:44.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.440+0000 7f8a5636a640 1 -- 172.21.15.2:0/1569225546 >> 172.21.15.2:0/1569225546 conn(0x7f8a50078ab0 msgr2=0x7f8a500ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:44.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.440+0000 7f8a5636a640 1 -- 172.21.15.2:0/1569225546 shutdown_connections 2024-05-14T17:15:44.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.440+0000 7f8a5636a640 1 -- 172.21.15.2:0/1569225546 wait complete. 2024-05-14T17:15:44.451 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-14T16:46:38.090903+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-14T16:51:27.296955+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-14T16:51:31.103019+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 36, 'snap_epoch': 108, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-14T17:15:44.452 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-14T17:15:44.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.561+0000 7fbf5fa23640 1 -- 172.21.15.2:0/25831433 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf581033e0 msgr2=0x7fbf5810f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:44.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.561+0000 7fbf5fa23640 1 --2- 172.21.15.2:0/25831433 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf581033e0 0x7fbf5810f8f0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fbf48009cd0 tx=0x7fbf4802a8e0 comp rx=0 tx=0).stop 2024-05-14T17:15:44.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.561+0000 7fbf5fa23640 1 -- 172.21.15.2:0/25831433 shutdown_connections 2024-05-14T17:15:44.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.561+0000 7fbf5fa23640 1 --2- 172.21.15.2:0/25831433 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf581033e0 0x7fbf5810f8f0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:44.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.561+0000 7fbf5fa23640 1 --2- 172.21.15.2:0/25831433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf58102a20 0x7fbf58102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:44.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.561+0000 7fbf5fa23640 1 --2- 172.21.15.2:0/25831433 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf58108a30 0x7fbf58108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:44.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.561+0000 7fbf5fa23640 1 -- 172.21.15.2:0/25831433 >> 172.21.15.2:0/25831433 conn(0x7fbf580fe490 msgr2=0x7fbf581008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:44.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.561+0000 7fbf5fa23640 1 -- 172.21.15.2:0/25831433 shutdown_connections 2024-05-14T17:15:44.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.561+0000 7fbf5fa23640 1 -- 172.21.15.2:0/25831433 wait complete. 2024-05-14T17:15:44.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.561+0000 7fbf5fa23640 1 Processor -- start 2024-05-14T17:15:44.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.561+0000 7fbf5fa23640 1 -- start start 2024-05-14T17:15:44.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.562+0000 7fbf5fa23640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf58102a20 0x7fbf58199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:44.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.562+0000 7fbf5fa23640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf581033e0 0x7fbf581997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:44.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.562+0000 7fbf5fa23640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf58108a30 0x7fbf5819db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:44.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.562+0000 7fbf5fa23640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbf58112cf0 con 0x7fbf58108a30 2024-05-14T17:15:44.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.562+0000 7fbf5fa23640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbf58112b70 con 0x7fbf581033e0 2024-05-14T17:15:44.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.562+0000 7fbf5fa23640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbf58112e70 con 0x7fbf58102a20 2024-05-14T17:15:44.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.562+0000 7fbf5dfc0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf58108a30 0x7fbf5819db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:44.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.562+0000 7fbf5dfc0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf58108a30 0x7fbf5819db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34544/0 (socket says 172.21.15.2:34544) 2024-05-14T17:15:44.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.562+0000 7fbf5dfc0640 1 -- 172.21.15.2:0/3219333750 learned_addr learned my addr 172.21.15.2:0/3219333750 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:44.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.562+0000 7fbf5cfbe640 1 --2- 172.21.15.2:0/3219333750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf581033e0 0x7fbf581997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:44.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.562+0000 7fbf5d7bf640 1 --2- 172.21.15.2:0/3219333750 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf58102a20 0x7fbf58199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:44.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.562+0000 7fbf5dfc0640 1 -- 172.21.15.2:0/3219333750 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf58102a20 msgr2=0x7fbf58199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:44.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.562+0000 7fbf5dfc0640 1 --2- 172.21.15.2:0/3219333750 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf58102a20 0x7fbf58199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:44.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.562+0000 7fbf5dfc0640 1 -- 172.21.15.2:0/3219333750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf581033e0 msgr2=0x7fbf581997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:44.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.562+0000 7fbf5dfc0640 1 --2- 172.21.15.2:0/3219333750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf581033e0 0x7fbf581997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:44.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.562+0000 7fbf5dfc0640 1 -- 172.21.15.2:0/3219333750 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf5819e280 con 0x7fbf58108a30 2024-05-14T17:15:44.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.562+0000 7fbf5cfbe640 1 --2- 172.21.15.2:0/3219333750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf581033e0 0x7fbf581997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:15:44.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.562+0000 7fbf5dfc0640 1 --2- 172.21.15.2:0/3219333750 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf58108a30 0x7fbf5819db50 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fbf48005cc0 tx=0x7fbf48003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:44.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.562+0000 7fbf567fc640 1 -- 172.21.15.2:0/3219333750 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf48004450 con 0x7fbf58108a30 2024-05-14T17:15:44.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.562+0000 7fbf567fc640 1 -- 172.21.15.2:0/3219333750 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf48031c90 con 0x7fbf58108a30 2024-05-14T17:15:44.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.562+0000 7fbf5fa23640 1 -- 172.21.15.2:0/3219333750 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf5819e510 con 0x7fbf58108a30 2024-05-14T17:15:44.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.562+0000 7fbf567fc640 1 -- 172.21.15.2:0/3219333750 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4803b400 con 0x7fbf58108a30 2024-05-14T17:15:44.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.563+0000 7fbf5fa23640 1 -- 172.21.15.2:0/3219333750 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf581a5e00 con 0x7fbf58108a30 2024-05-14T17:15:44.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.563+0000 7fbf5fa23640 1 -- 172.21.15.2:0/3219333750 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf58069cc0 con 0x7fbf58108a30 2024-05-14T17:15:44.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.565+0000 7fbf567fc640 1 -- 172.21.15.2:0/3219333750 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fbf48004680 con 0x7fbf58108a30 2024-05-14T17:15:44.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.565+0000 7fbf567fc640 1 --2- 172.21.15.2:0/3219333750 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fbf400b29e0 0x7fbf400b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:44.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.566+0000 7fbf5d7bf640 1 --2- 172.21.15.2:0/3219333750 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fbf400b29e0 0x7fbf400b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:44.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.566+0000 7fbf567fc640 1 -- 172.21.15.2:0/3219333750 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8469+0+0 (secure 0 0 0) 0x7fbf4802e030 con 0x7fbf58108a30 2024-05-14T17:15:44.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.566+0000 7fbf5d7bf640 1 --2- 172.21.15.2:0/3219333750 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fbf400b29e0 0x7fbf400b4ea0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbf50005fd0 tx=0x7fbf5000b510 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:44.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.569+0000 7fbf567fc640 1 -- 172.21.15.2:0/3219333750 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbf480cb670 con 0x7fbf58108a30 2024-05-14T17:15:44.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:44 smithi002 ceph-mon[31538]: from='client.17361 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:15:44.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:44 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/1569225546' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:15:44.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.760+0000 7fbf5fa23640 1 -- 172.21.15.2:0/3219333750 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbf5802dba0 con 0x7fbf58108a30 2024-05-14T17:15:44.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.760+0000 7fbf567fc640 1 -- 172.21.15.2:0/3219333750 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) ==== 93+0+10 (secure 0 0 0) 0x7fbf480caf70 con 0x7fbf58108a30 2024-05-14T17:15:44.759 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2024-05-14T17:15:44.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.762+0000 7fbf5fa23640 1 -- 172.21.15.2:0/3219333750 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fbf400b29e0 msgr2=0x7fbf400b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:44.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.762+0000 7fbf5fa23640 1 --2- 172.21.15.2:0/3219333750 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fbf400b29e0 0x7fbf400b4ea0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbf50005fd0 tx=0x7fbf5000b510 comp rx=0 tx=0).stop 2024-05-14T17:15:44.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.762+0000 7fbf5fa23640 1 -- 172.21.15.2:0/3219333750 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf58108a30 msgr2=0x7fbf5819db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:44.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.762+0000 7fbf5fa23640 1 --2- 172.21.15.2:0/3219333750 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf58108a30 0x7fbf5819db50 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fbf48005cc0 tx=0x7fbf48003c20 comp rx=0 tx=0).stop 2024-05-14T17:15:44.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.762+0000 7fbf5fa23640 1 -- 172.21.15.2:0/3219333750 shutdown_connections 2024-05-14T17:15:44.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.762+0000 7fbf5fa23640 1 --2- 172.21.15.2:0/3219333750 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fbf400b29e0 0x7fbf400b4ea0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:44.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.762+0000 7fbf5fa23640 1 --2- 172.21.15.2:0/3219333750 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf58108a30 0x7fbf5819db50 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:44.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.762+0000 7fbf5fa23640 1 --2- 172.21.15.2:0/3219333750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf581033e0 0x7fbf581997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:44.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.762+0000 7fbf5fa23640 1 --2- 172.21.15.2:0/3219333750 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf58102a20 0x7fbf58199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:44.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.762+0000 7fbf5fa23640 1 -- 172.21.15.2:0/3219333750 >> 172.21.15.2:0/3219333750 conn(0x7fbf580fe490 msgr2=0x7fbf580fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:44.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.762+0000 7fbf5fa23640 1 -- 172.21.15.2:0/3219333750 shutdown_connections 2024-05-14T17:15:44.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.762+0000 7fbf5fa23640 1 -- 172.21.15.2:0/3219333750 wait complete. 2024-05-14T17:15:44.773 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-14T17:15:44.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:44 smithi151 ceph-mon[36066]: from='client.17361 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:15:44.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:44 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/1569225546' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:15:44.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.881+0000 7fdbfa9af640 1 -- 172.21.15.2:0/1866217301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbf4107930 msgr2=0x7fdbf4107d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:44.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.881+0000 7fdbfa9af640 1 --2- 172.21.15.2:0/1866217301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbf4107930 0x7fdbf4107d10 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4009930 tx=0x7fdbe402a420 comp rx=0 tx=0).stop 2024-05-14T17:15:44.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.881+0000 7fdbfa9af640 1 -- 172.21.15.2:0/1866217301 shutdown_connections 2024-05-14T17:15:44.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.881+0000 7fdbfa9af640 1 --2- 172.21.15.2:0/1866217301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbf41022e0 0x7fdbf410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:44.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.881+0000 7fdbfa9af640 1 --2- 172.21.15.2:0/1866217301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdbf4101920 0x7fdbf4101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:44.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.881+0000 7fdbfa9af640 1 --2- 172.21.15.2:0/1866217301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbf4107930 0x7fdbf4107d10 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:44.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.881+0000 7fdbfa9af640 1 -- 172.21.15.2:0/1866217301 >> 172.21.15.2:0/1866217301 conn(0x7fdbf4078a50 msgr2=0x7fdbf4078e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:44.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.881+0000 7fdbfa9af640 1 -- 172.21.15.2:0/1866217301 shutdown_connections 2024-05-14T17:15:44.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.881+0000 7fdbfa9af640 1 -- 172.21.15.2:0/1866217301 wait complete. 2024-05-14T17:15:44.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.881+0000 7fdbfa9af640 1 Processor -- start 2024-05-14T17:15:44.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.881+0000 7fdbfa9af640 1 -- start start 2024-05-14T17:15:44.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.882+0000 7fdbfa9af640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbf4101920 0x7fdbf41992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:44.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.882+0000 7fdbfa9af640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbf41022e0 0x7fdbf4199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:44.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.882+0000 7fdbfa9af640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdbf4107930 0x7fdbf419db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:44.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.882+0000 7fdbfa9af640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf4111bf0 con 0x7fdbf41022e0 2024-05-14T17:15:44.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.882+0000 7fdbfa9af640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf4111a70 con 0x7fdbf4101920 2024-05-14T17:15:44.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.882+0000 7fdbfa9af640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf4111d70 con 0x7fdbf4107930 2024-05-14T17:15:44.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.882+0000 7fdbf37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbf41022e0 0x7fdbf4199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:44.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.882+0000 7fdbf37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbf41022e0 0x7fdbf4199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34554/0 (socket says 172.21.15.2:34554) 2024-05-14T17:15:44.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.882+0000 7fdbf37fe640 1 -- 172.21.15.2:0/2641455778 learned_addr learned my addr 172.21.15.2:0/2641455778 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:44.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.882+0000 7fdbf3fff640 1 --2- 172.21.15.2:0/2641455778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbf4101920 0x7fdbf41992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:44.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.882+0000 7fdbf8f4c640 1 --2- 172.21.15.2:0/2641455778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdbf4107930 0x7fdbf419db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:44.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.882+0000 7fdbf37fe640 1 -- 172.21.15.2:0/2641455778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdbf4107930 msgr2=0x7fdbf419db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:44.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.882+0000 7fdbf37fe640 1 --2- 172.21.15.2:0/2641455778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdbf4107930 0x7fdbf419db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:44.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.882+0000 7fdbf37fe640 1 -- 172.21.15.2:0/2641455778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbf4101920 msgr2=0x7fdbf41992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:44.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.882+0000 7fdbf37fe640 1 --2- 172.21.15.2:0/2641455778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbf4101920 0x7fdbf41992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:44.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.882+0000 7fdbf37fe640 1 -- 172.21.15.2:0/2641455778 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf419e260 con 0x7fdbf41022e0 2024-05-14T17:15:44.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.882+0000 7fdbf37fe640 1 --2- 172.21.15.2:0/2641455778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbf41022e0 0x7fdbf4199800 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fdbe800d770 tx=0x7fdbe800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:44.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.883+0000 7fdbf17fa640 1 -- 172.21.15.2:0/2641455778 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8017070 con 0x7fdbf41022e0 2024-05-14T17:15:44.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.883+0000 7fdbf17fa640 1 -- 172.21.15.2:0/2641455778 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbe80054d0 con 0x7fdbf41022e0 2024-05-14T17:15:44.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.883+0000 7fdbfa9af640 1 -- 172.21.15.2:0/2641455778 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbf419e550 con 0x7fdbf41022e0 2024-05-14T17:15:44.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.883+0000 7fdbf17fa640 1 -- 172.21.15.2:0/2641455778 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8004c30 con 0x7fdbf41022e0 2024-05-14T17:15:44.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.883+0000 7fdbfa9af640 1 -- 172.21.15.2:0/2641455778 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbf41a5de0 con 0x7fdbf41022e0 2024-05-14T17:15:44.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.884+0000 7fdbfa9af640 1 -- 172.21.15.2:0/2641455778 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbf4069870 con 0x7fdbf41022e0 2024-05-14T17:15:44.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.886+0000 7fdbf17fa640 1 -- 172.21.15.2:0/2641455778 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fdbe8004dd0 con 0x7fdbf41022e0 2024-05-14T17:15:44.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.887+0000 7fdbf17fa640 1 --2- 172.21.15.2:0/2641455778 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fdbdc0b2960 0x7fdbdc0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:44.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.887+0000 7fdbf3fff640 1 --2- 172.21.15.2:0/2641455778 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fdbdc0b2960 0x7fdbdc0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:44.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.887+0000 7fdbf17fa640 1 -- 172.21.15.2:0/2641455778 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8469+0+0 (secure 0 0 0) 0x7fdbe80e2590 con 0x7fdbf41022e0 2024-05-14T17:15:44.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.887+0000 7fdbf3fff640 1 --2- 172.21.15.2:0/2641455778 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fdbdc0b2960 0x7fdbdc0b4e20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4000c00 tx=0x7fdbe4032bc0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:44.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:44.890+0000 7fdbf17fa640 1 -- 172.21.15.2:0/2641455778 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdbe80a9f40 con 0x7fdbf41022e0 2024-05-14T17:15:44.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:44 smithi123 ceph-mon[35971]: from='client.17361 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:15:44.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:44 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/1569225546' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:15:45.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.083+0000 7fdbfa9af640 1 -- 172.21.15.2:0/2641455778 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdbf410a910 con 0x7fdbf41022e0 2024-05-14T17:15:45.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.083+0000 7fdbf17fa640 1 -- 172.21.15.2:0/2641455778 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) ==== 104+0+11 (secure 0 0 0) 0x7fdbe80a9840 con 0x7fdbf41022e0 2024-05-14T17:15:45.083 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2024-05-14T17:15:45.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.085+0000 7fdbfa9af640 1 -- 172.21.15.2:0/2641455778 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fdbdc0b2960 msgr2=0x7fdbdc0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:45.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.085+0000 7fdbfa9af640 1 --2- 172.21.15.2:0/2641455778 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fdbdc0b2960 0x7fdbdc0b4e20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4000c00 tx=0x7fdbe4032bc0 comp rx=0 tx=0).stop 2024-05-14T17:15:45.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.085+0000 7fdbfa9af640 1 -- 172.21.15.2:0/2641455778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbf41022e0 msgr2=0x7fdbf4199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:45.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.085+0000 7fdbfa9af640 1 --2- 172.21.15.2:0/2641455778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbf41022e0 0x7fdbf4199800 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fdbe800d770 tx=0x7fdbe800dc30 comp rx=0 tx=0).stop 2024-05-14T17:15:45.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.085+0000 7fdbfa9af640 1 -- 172.21.15.2:0/2641455778 shutdown_connections 2024-05-14T17:15:45.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.085+0000 7fdbfa9af640 1 --2- 172.21.15.2:0/2641455778 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fdbdc0b2960 0x7fdbdc0b4e20 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:45.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.085+0000 7fdbfa9af640 1 --2- 172.21.15.2:0/2641455778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdbf4107930 0x7fdbf419db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:45.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.085+0000 7fdbfa9af640 1 --2- 172.21.15.2:0/2641455778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbf41022e0 0x7fdbf4199800 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:45.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.085+0000 7fdbfa9af640 1 --2- 172.21.15.2:0/2641455778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdbf4101920 0x7fdbf41992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:45.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.085+0000 7fdbfa9af640 1 -- 172.21.15.2:0/2641455778 >> 172.21.15.2:0/2641455778 conn(0x7fdbf4078a50 msgr2=0x7fdbf4069fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:45.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.085+0000 7fdbfa9af640 1 -- 172.21.15.2:0/2641455778 shutdown_connections 2024-05-14T17:15:45.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.085+0000 7fdbfa9af640 1 -- 172.21.15.2:0/2641455778 wait complete. 2024-05-14T17:15:45.096 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-14T17:15:45.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.204+0000 7f0786ff7640 1 -- 172.21.15.2:0/1894633933 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f078010d010 msgr2=0x7f078010f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:45.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.204+0000 7f0786ff7640 1 --2- 172.21.15.2:0/1894633933 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f078010d010 0x7f078010f4b0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f0778009ca0 tx=0x7f077802a9e0 comp rx=0 tx=0).stop 2024-05-14T17:15:45.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.204+0000 7f0786ff7640 1 -- 172.21.15.2:0/1894633933 shutdown_connections 2024-05-14T17:15:45.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.204+0000 7f0786ff7640 1 --2- 172.21.15.2:0/1894633933 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f078010d010 0x7f078010f4b0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:45.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.204+0000 7f0786ff7640 1 --2- 172.21.15.2:0/1894633933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07801045f0 0x7f078010cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:45.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.204+0000 7f0786ff7640 1 --2- 172.21.15.2:0/1894633933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0780103cd0 0x7f07801040b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:45.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.204+0000 7f0786ff7640 1 -- 172.21.15.2:0/1894633933 >> 172.21.15.2:0/1894633933 conn(0x7f0780078a50 msgr2=0x7f0780078e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:45.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.204+0000 7f0786ff7640 1 -- 172.21.15.2:0/1894633933 shutdown_connections 2024-05-14T17:15:45.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.204+0000 7f0786ff7640 1 -- 172.21.15.2:0/1894633933 wait complete. 2024-05-14T17:15:45.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.205+0000 7f0786ff7640 1 Processor -- start 2024-05-14T17:15:45.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.205+0000 7f0786ff7640 1 -- start start 2024-05-14T17:15:45.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.205+0000 7f0786ff7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0780103cd0 0x7f0780101920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:45.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.205+0000 7f0786ff7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07801045f0 0x7f0780101e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:45.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.205+0000 7f0786ff7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f078010d010 0x7f07801023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:45.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.205+0000 7f0786ff7640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0780112d10 con 0x7f0780103cd0 2024-05-14T17:15:45.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.205+0000 7f0786ff7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0780112b90 con 0x7f07801045f0 2024-05-14T17:15:45.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.205+0000 7f0786ff7640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0780112e90 con 0x7f078010d010 2024-05-14T17:15:45.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.205+0000 7f0784d93640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0780103cd0 0x7f0780101920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:45.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.205+0000 7f0784d93640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0780103cd0 0x7f0780101920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34572/0 (socket says 172.21.15.2:34572) 2024-05-14T17:15:45.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.205+0000 7f0784d93640 1 -- 172.21.15.2:0/4026311418 learned_addr learned my addr 172.21.15.2:0/4026311418 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:45.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.205+0000 7f0785594640 1 --2- 172.21.15.2:0/4026311418 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f078010d010 0x7f07801023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:45.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.206+0000 7f0777fff640 1 --2- 172.21.15.2:0/4026311418 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07801045f0 0x7f0780101e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:45.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.206+0000 7f0784d93640 1 -- 172.21.15.2:0/4026311418 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f078010d010 msgr2=0x7f07801023a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:45.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.206+0000 7f0784d93640 1 --2- 172.21.15.2:0/4026311418 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f078010d010 0x7f07801023a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:45.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.206+0000 7f0784d93640 1 -- 172.21.15.2:0/4026311418 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07801045f0 msgr2=0x7f0780101e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:45.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.206+0000 7f0784d93640 1 --2- 172.21.15.2:0/4026311418 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07801045f0 0x7f0780101e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:45.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.206+0000 7f0784d93640 1 -- 172.21.15.2:0/4026311418 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07801a7710 con 0x7f0780103cd0 2024-05-14T17:15:45.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.206+0000 7f0784d93640 1 --2- 172.21.15.2:0/4026311418 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0780103cd0 0x7f0780101920 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f077c00d950 tx=0x7f077c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:45.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.206+0000 7f0775ffb640 1 -- 172.21.15.2:0/4026311418 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077c017030 con 0x7f0780103cd0 2024-05-14T17:15:45.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.206+0000 7f0775ffb640 1 -- 172.21.15.2:0/4026311418 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f077c004020 con 0x7f0780103cd0 2024-05-14T17:15:45.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.206+0000 7f0786ff7640 1 -- 172.21.15.2:0/4026311418 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07801a7a00 con 0x7f0780103cd0 2024-05-14T17:15:45.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.206+0000 7f0775ffb640 1 -- 172.21.15.2:0/4026311418 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077c0047a0 con 0x7f0780103cd0 2024-05-14T17:15:45.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.206+0000 7f0786ff7640 1 -- 172.21.15.2:0/4026311418 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07801a7fa0 con 0x7f0780103cd0 2024-05-14T17:15:45.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.207+0000 7f0786ff7640 1 -- 172.21.15.2:0/4026311418 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0780069870 con 0x7f0780103cd0 2024-05-14T17:15:45.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.208+0000 7f0775ffb640 1 -- 172.21.15.2:0/4026311418 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f077c009a30 con 0x7f0780103cd0 2024-05-14T17:15:45.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.209+0000 7f0775ffb640 1 --2- 172.21.15.2:0/4026311418 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f07680b29e0 0x7f07680b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:45.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.210+0000 7f0777fff640 1 --2- 172.21.15.2:0/4026311418 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f07680b29e0 0x7f07680b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:45.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.210+0000 7f0775ffb640 1 -- 172.21.15.2:0/4026311418 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8469+0+0 (secure 0 0 0) 0x7f077c0e1ff0 con 0x7f0780103cd0 2024-05-14T17:15:45.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.210+0000 7f0777fff640 1 --2- 172.21.15.2:0/4026311418 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f07680b29e0 0x7f07680b4ea0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f07801033b0 tx=0x7f0770005f90 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:45.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.214+0000 7f0775ffb640 1 -- 172.21.15.2:0/4026311418 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f077c0a9a00 con 0x7f0780103cd0 2024-05-14T17:15:45.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.406+0000 7f0786ff7640 1 -- 172.21.15.2:0/4026311418 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f07801a8230 con 0x7f0780103cd0 2024-05-14T17:15:45.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.407+0000 7f0775ffb640 1 -- 172.21.15.2:0/4026311418 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) ==== 100+0+11 (secure 0 0 0) 0x7f077c0a9300 con 0x7f0780103cd0 2024-05-14T17:15:45.407 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2024-05-14T17:15:45.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.409+0000 7f0786ff7640 1 -- 172.21.15.2:0/4026311418 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f07680b29e0 msgr2=0x7f07680b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:45.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.409+0000 7f0786ff7640 1 --2- 172.21.15.2:0/4026311418 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f07680b29e0 0x7f07680b4ea0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f07801033b0 tx=0x7f0770005f90 comp rx=0 tx=0).stop 2024-05-14T17:15:45.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.409+0000 7f0786ff7640 1 -- 172.21.15.2:0/4026311418 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0780103cd0 msgr2=0x7f0780101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:45.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.409+0000 7f0786ff7640 1 --2- 172.21.15.2:0/4026311418 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0780103cd0 0x7f0780101920 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f077c00d950 tx=0x7f077c00de10 comp rx=0 tx=0).stop 2024-05-14T17:15:45.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.409+0000 7f0786ff7640 1 -- 172.21.15.2:0/4026311418 shutdown_connections 2024-05-14T17:15:45.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.409+0000 7f0786ff7640 1 --2- 172.21.15.2:0/4026311418 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f07680b29e0 0x7f07680b4ea0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:45.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.409+0000 7f0786ff7640 1 --2- 172.21.15.2:0/4026311418 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f078010d010 0x7f07801023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:45.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.409+0000 7f0786ff7640 1 --2- 172.21.15.2:0/4026311418 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07801045f0 0x7f0780101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:45.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.409+0000 7f0786ff7640 1 --2- 172.21.15.2:0/4026311418 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0780103cd0 0x7f0780101920 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:45.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.409+0000 7f0786ff7640 1 -- 172.21.15.2:0/4026311418 >> 172.21.15.2:0/4026311418 conn(0x7f0780078a50 msgr2=0x7f07800fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:45.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.409+0000 7f0786ff7640 1 -- 172.21.15.2:0/4026311418 shutdown_connections 2024-05-14T17:15:45.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.409+0000 7f0786ff7640 1 -- 172.21.15.2:0/4026311418 wait complete. 2024-05-14T17:15:45.420 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-14T17:15:45.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.528+0000 7f39df4cc640 1 -- 172.21.15.2:0/2962308343 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f39d81022e0 msgr2=0x7f39d810e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:45.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.528+0000 7f39df4cc640 1 --2- 172.21.15.2:0/2962308343 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f39d81022e0 0x7f39d810e7f0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f39d0009930 tx=0x7f39d002a420 comp rx=0 tx=0).stop 2024-05-14T17:15:45.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.528+0000 7f39df4cc640 1 -- 172.21.15.2:0/2962308343 shutdown_connections 2024-05-14T17:15:45.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.528+0000 7f39df4cc640 1 --2- 172.21.15.2:0/2962308343 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f39d81022e0 0x7f39d810e7f0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:45.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.528+0000 7f39df4cc640 1 --2- 172.21.15.2:0/2962308343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39d8101920 0x7f39d8101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:45.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.528+0000 7f39df4cc640 1 --2- 172.21.15.2:0/2962308343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39d8107930 0x7f39d8107d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:45.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.528+0000 7f39df4cc640 1 -- 172.21.15.2:0/2962308343 >> 172.21.15.2:0/2962308343 conn(0x7f39d8077cd0 msgr2=0x7f39d80780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:15:45.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.528+0000 7f39df4cc640 1 -- 172.21.15.2:0/2962308343 shutdown_connections 2024-05-14T17:15:45.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.528+0000 7f39df4cc640 1 -- 172.21.15.2:0/2962308343 wait complete. 2024-05-14T17:15:45.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.529+0000 7f39df4cc640 1 Processor -- start 2024-05-14T17:15:45.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.529+0000 7f39df4cc640 1 -- start start 2024-05-14T17:15:45.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.529+0000 7f39df4cc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f39d8101920 0x7f39d819a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:45.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.529+0000 7f39df4cc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39d81022e0 0x7f39d819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:45.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.529+0000 7f39df4cc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39d8107930 0x7f39d819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:45.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.529+0000 7f39df4cc640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f39d8111bf0 con 0x7f39d8101920 2024-05-14T17:15:45.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.529+0000 7f39df4cc640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f39d8111a70 con 0x7f39d81022e0 2024-05-14T17:15:45.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.529+0000 7f39df4cc640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f39d8111d70 con 0x7f39d8107930 2024-05-14T17:15:45.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.529+0000 7f39dd268640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f39d8101920 0x7f39d819a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:45.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.529+0000 7f39dd268640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f39d8101920 0x7f39d819a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34588/0 (socket says 172.21.15.2:34588) 2024-05-14T17:15:45.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.529+0000 7f39dd268640 1 -- 172.21.15.2:0/1378451415 learned_addr learned my addr 172.21.15.2:0/1378451415 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:15:45.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.529+0000 7f39dca67640 1 --2- 172.21.15.2:0/1378451415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39d81022e0 0x7f39d819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:45.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.529+0000 7f39dda69640 1 --2- 172.21.15.2:0/1378451415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39d8107930 0x7f39d819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:45.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.529+0000 7f39dd268640 1 -- 172.21.15.2:0/1378451415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39d8107930 msgr2=0x7f39d819abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:45.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.529+0000 7f39dd268640 1 --2- 172.21.15.2:0/1378451415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39d8107930 0x7f39d819abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:45.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.529+0000 7f39dd268640 1 -- 172.21.15.2:0/1378451415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39d81022e0 msgr2=0x7f39d819a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:15:45.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.529+0000 7f39dd268640 1 --2- 172.21.15.2:0/1378451415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39d81022e0 0x7f39d819a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:15:45.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.529+0000 7f39dd268640 1 -- 172.21.15.2:0/1378451415 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d81a5500 con 0x7f39d8101920 2024-05-14T17:15:45.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.529+0000 7f39dca67640 1 --2- 172.21.15.2:0/1378451415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39d81022e0 0x7f39d819a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-14T17:15:45.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.529+0000 7f39dd268640 1 --2- 172.21.15.2:0/1378451415 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f39d8101920 0x7f39d819a150 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f39d400ea80 tx=0x7f39d400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:45.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.530+0000 7f39ce7fc640 1 -- 172.21.15.2:0/1378451415 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d400ce40 con 0x7f39d8101920 2024-05-14T17:15:45.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.530+0000 7f39ce7fc640 1 -- 172.21.15.2:0/1378451415 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39d4005020 con 0x7f39d8101920 2024-05-14T17:15:45.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.530+0000 7f39ce7fc640 1 -- 172.21.15.2:0/1378451415 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d40116d0 con 0x7f39d8101920 2024-05-14T17:15:45.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.530+0000 7f39df4cc640 1 -- 172.21.15.2:0/1378451415 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39d81a57f0 con 0x7f39d8101920 2024-05-14T17:15:45.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.530+0000 7f39df4cc640 1 -- 172.21.15.2:0/1378451415 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39d81a5d40 con 0x7f39d8101920 2024-05-14T17:15:45.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.531+0000 7f39df4cc640 1 -- 172.21.15.2:0/1378451415 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39d8069cc0 con 0x7f39d8101920 2024-05-14T17:15:45.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.532+0000 7f39ce7fc640 1 -- 172.21.15.2:0/1378451415 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f39d40054d0 con 0x7f39d8101920 2024-05-14T17:15:45.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.533+0000 7f39ce7fc640 1 --2- 172.21.15.2:0/1378451415 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f39c40b29e0 0x7f39c40b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:15:45.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.533+0000 7f39dca67640 1 --2- 172.21.15.2:0/1378451415 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f39c40b29e0 0x7f39c40b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:15:45.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.533+0000 7f39ce7fc640 1 -- 172.21.15.2:0/1378451415 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8469+0+0 (secure 0 0 0) 0x7f39d40e28f0 con 0x7f39d8101920 2024-05-14T17:15:45.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.533+0000 7f39dca67640 1 --2- 172.21.15.2:0/1378451415 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f39c40b29e0 0x7f39c40b4ea0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f39c8005d00 tx=0x7f39c8009550 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:15:45.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.537+0000 7f39ce7fc640 1 -- 172.21.15.2:0/1378451415 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f39d4016030 con 0x7f39d8101920 2024-05-14T17:15:45.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:45 smithi002 ceph-mon[31538]: from='client.17367 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:15:45.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:45 smithi002 ceph-mon[31538]: pgmap v664: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.9 KiB/s wr, 8 op/s 2024-05-14T17:15:45.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:45 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/3219333750' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-14T17:15:45.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:45 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/2641455778' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-14T17:15:45.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:45 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/4026311418' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-14T17:15:45.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.784+0000 7f39df4cc640 1 -- 172.21.15.2:0/1378451415 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f39d8109900 con 0x7f39d8101920 2024-05-14T17:15:45.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.793+0000 7f39ce7fc640 1 -- 172.21.15.2:0/1378451415 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+816163 (secure 0 0 0) 0x7f39d40aa2a0 con 0x7f39d8101920 2024-05-14T17:15:45.804 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:15:45.804 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T17:15:45.804 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 26, 2024-05-14T17:15:45.804 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.804 INFO:teuthology.orchestra.run.smithi002.stdout: "active_gid": 15120, 2024-05-14T17:15:45.804 INFO:teuthology.orchestra.run.smithi002.stdout: "active_name": "x", 2024-05-14T17:15:45.804 INFO:teuthology.orchestra.run.smithi002.stdout: "active_addrs": { 2024-05-14T17:15:45.804 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-05-14T17:15:45.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:45.804 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-05-14T17:15:45.804 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6800", 2024-05-14T17:15:45.804 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1339485805 2024-05-14T17:15:45.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:45.805 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-05-14T17:15:45.805 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6801", 2024-05-14T17:15:45.805 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1339485805 2024-05-14T17:15:45.805 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:45.805 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-05-14T17:15:45.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.805 INFO:teuthology.orchestra.run.smithi002.stdout: "active_addr": "172.21.15.2:6801/1339485805", 2024-05-14T17:15:45.805 INFO:teuthology.orchestra.run.smithi002.stdout: "active_change": "2024-05-14T16:54:41.187794+0000", 2024-05-14T17:15:45.805 INFO:teuthology.orchestra.run.smithi002.stdout: "active_mgr_features": 4540701547738038271, 2024-05-14T17:15:45.805 INFO:teuthology.orchestra.run.smithi002.stdout: "available": true, 2024-05-14T17:15:45.805 INFO:teuthology.orchestra.run.smithi002.stdout: "standbys": [ 2024-05-14T17:15:45.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:45.805 INFO:teuthology.orchestra.run.smithi002.stdout: "gid": 24871, 2024-05-14T17:15:45.806 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "z", 2024-05-14T17:15:45.806 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_features": 4540701547738038271, 2024-05-14T17:15:45.806 INFO:teuthology.orchestra.run.smithi002.stdout: "available_modules": [ 2024-05-14T17:15:45.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:45.806 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "alerts", 2024-05-14T17:15:45.806 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:45.806 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:45.806 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:45.806 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-05-14T17:15:45.806 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-05-14T17:15:45.806 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:45.806 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.806 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.806 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-05-14T17:15:45.806 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.807 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.807 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.807 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How frequently to reexamine health status", 2024-05-14T17:15:45.807 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.807 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.807 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.807 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:45.807 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:45.807 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.807 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.807 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.807 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:45.807 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.807 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.807 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:45.808 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:45.808 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:45.808 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:45.808 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:45.808 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:45.808 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:45.808 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:45.808 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.808 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.808 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.808 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.808 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:45.808 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:45.808 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.808 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.810 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:45.810 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.810 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.810 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:45.810 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:45.810 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:45.810 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:45.810 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:45.810 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:45.810 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:45.810 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:45.810 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.810 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.810 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.810 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_destination": { 2024-05-14T17:15:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_destination", 2024-05-14T17:15:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Email address to send alerts to", 2024-05-14T17:15:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_from_name": { 2024-05-14T17:15:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_from_name", 2024-05-14T17:15:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "Ceph", 2024-05-14T17:15:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Email From: name", 2024-05-14T17:15:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_host": { 2024-05-14T17:15:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_host", 2024-05-14T17:15:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP server", 2024-05-14T17:15:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_password": { 2024-05-14T17:15:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_password", 2024-05-14T17:15:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Password to authenticate with", 2024-05-14T17:15:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_port": { 2024-05-14T17:15:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_port", 2024-05-14T17:15:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "465", 2024-05-14T17:15:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP port", 2024-05-14T17:15:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_sender": { 2024-05-14T17:15:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_sender", 2024-05-14T17:15:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP envelope sender", 2024-05-14T17:15:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_ssl": { 2024-05-14T17:15:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_ssl", 2024-05-14T17:15:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-14T17:15:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_user": { 2024-05-14T17:15:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_user", 2024-05-14T17:15:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "User to authenticate as", 2024-05-14T17:15:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:45.819 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:45.819 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:45.819 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:45.819 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.819 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:45.819 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.819 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.819 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.819 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.819 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.819 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.819 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.819 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:45.819 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:45.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:45.820 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "balancer", 2024-05-14T17:15:45.820 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:45.820 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:45.820 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:45.820 INFO:teuthology.orchestra.run.smithi002.stdout: "active": { 2024-05-14T17:15:45.820 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "active", 2024-05-14T17:15:45.820 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.820 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.820 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.820 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:45.820 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.820 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.820 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.820 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically balance PGs across cluster", 2024-05-14T17:15:45.820 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.820 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.820 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.821 INFO:teuthology.orchestra.run.smithi002.stdout: "begin_time": { 2024-05-14T17:15:45.821 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "begin_time", 2024-05-14T17:15:45.821 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.821 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.821 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.821 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0000", 2024-05-14T17:15:45.821 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.821 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.821 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.821 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "beginning time of day to automatically balance", 2024-05-14T17:15:45.821 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-14T17:15:45.821 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.821 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.821 INFO:teuthology.orchestra.run.smithi002.stdout: "begin_weekday": { 2024-05-14T17:15:45.822 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "begin_weekday", 2024-05-14T17:15:45.822 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-05-14T17:15:45.822 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.822 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.822 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:45.822 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0", 2024-05-14T17:15:45.822 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "6", 2024-05-14T17:15:45.822 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.822 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-14T17:15:45.822 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-14T17:15:45.822 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.822 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.822 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_max_iterations": { 2024-05-14T17:15:45.822 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_max_iterations", 2024-05-14T17:15:45.822 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-05-14T17:15:45.823 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.823 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.823 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "25", 2024-05-14T17:15:45.823 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-05-14T17:15:45.823 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "250", 2024-05-14T17:15:45.823 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.823 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-14T17:15:45.823 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.823 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.823 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.823 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_metrics": { 2024-05-14T17:15:45.823 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_metrics", 2024-05-14T17:15:45.823 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.823 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.824 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.824 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "pgs,objects,bytes", 2024-05-14T17:15:45.824 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.824 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.824 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.824 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-14T17:15:45.824 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-14T17:15:45.824 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.824 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.824 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_step": { 2024-05-14T17:15:45.824 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_step", 2024-05-14T17:15:45.825 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T17:15:45.825 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.825 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.825 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0.5", 2024-05-14T17:15:45.825 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0.001", 2024-05-14T17:15:45.825 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "0.999", 2024-05-14T17:15:45.825 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.825 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "aggressiveness of optimization", 2024-05-14T17:15:45.825 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-14T17:15:45.825 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.825 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.825 INFO:teuthology.orchestra.run.smithi002.stdout: "end_time": { 2024-05-14T17:15:45.825 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "end_time", 2024-05-14T17:15:45.826 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.826 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.826 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.826 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "2359", 2024-05-14T17:15:45.826 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.826 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.826 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.826 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "ending time of day to automatically balance", 2024-05-14T17:15:45.826 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-14T17:15:45.826 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.826 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.826 INFO:teuthology.orchestra.run.smithi002.stdout: "end_weekday": { 2024-05-14T17:15:45.827 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "end_weekday", 2024-05-14T17:15:45.827 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-05-14T17:15:45.827 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.827 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.827 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:45.827 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0", 2024-05-14T17:15:45.827 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "6", 2024-05-14T17:15:45.827 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.827 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-14T17:15:45.827 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-14T17:15:45.827 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.827 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.827 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:45.827 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:45.827 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.828 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.828 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.828 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:45.828 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.828 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.828 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:45.828 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:45.828 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:45.828 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:45.828 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:45.828 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:45.828 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:45.828 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:45.829 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.829 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.829 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.829 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.829 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:45.829 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:45.829 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.829 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.829 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.829 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:45.829 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.829 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.829 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.830 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.830 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.830 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.830 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.830 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:45.830 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:45.830 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.830 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.830 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.830 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:45.830 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.830 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.830 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:45.831 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:45.831 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:45.831 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:45.831 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:45.831 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:45.831 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:45.831 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:45.831 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.831 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.831 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.831 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.831 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:45.831 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:45.831 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.831 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.832 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.832 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:45.832 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.832 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.832 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.832 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.832 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.832 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.832 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.832 INFO:teuthology.orchestra.run.smithi002.stdout: "min_score": { 2024-05-14T17:15:45.832 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "min_score", 2024-05-14T17:15:45.832 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T17:15:45.832 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.833 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.833 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:45.833 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.833 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.833 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.833 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-14T17:15:45.833 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.833 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.833 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.833 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": { 2024-05-14T17:15:45.833 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mode", 2024-05-14T17:15:45.833 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.833 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.833 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.834 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "upmap", 2024-05-14T17:15:45.834 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.834 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.834 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:45.834 INFO:teuthology.orchestra.run.smithi002.stdout: "crush-compat", 2024-05-14T17:15:45.834 INFO:teuthology.orchestra.run.smithi002.stdout: "none", 2024-05-14T17:15:45.834 INFO:teuthology.orchestra.run.smithi002.stdout: "read", 2024-05-14T17:15:45.834 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap", 2024-05-14T17:15:45.834 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap-read" 2024-05-14T17:15:45.834 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:45.834 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Balancer mode", 2024-05-14T17:15:45.834 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.834 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.834 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_ids": { 2024-05-14T17:15:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pool_ids", 2024-05-14T17:15:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-14T17:15:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-05-14T17:15:45.836 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-05-14T17:15:45.836 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:45.836 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.836 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.836 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-05-14T17:15:45.836 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.836 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.836 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.836 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-14T17:15:45.836 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.836 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.836 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.836 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:45.837 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:45.837 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:45.837 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:45.837 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.837 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:45.837 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.837 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.837 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.837 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.837 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.837 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.837 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.837 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap_max_deviation": { 2024-05-14T17:15:45.837 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "upmap_max_deviation", 2024-05-14T17:15:45.837 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:45.838 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.838 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.838 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-05-14T17:15:45.838 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-05-14T17:15:45.838 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.838 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.838 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-14T17:15:45.838 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-14T17:15:45.838 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.838 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.838 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap_max_optimizations": { 2024-05-14T17:15:45.838 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "upmap_max_optimizations", 2024-05-14T17:15:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-05-14T17:15:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T17:15:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-14T17:15:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:45.840 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cephadm", 2024-05-14T17:15:45.840 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:45.840 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:45.840 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:45.840 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_down_multiplier": { 2024-05-14T17:15:45.840 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_down_multiplier", 2024-05-14T17:15:45.840 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T17:15:45.840 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.840 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.840 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3.0", 2024-05-14T17:15:45.840 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.840 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.840 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.841 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-14T17:15:45.841 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.841 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.841 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.841 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_refresh_rate": { 2024-05-14T17:15:45.841 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_refresh_rate", 2024-05-14T17:15:45.841 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:45.841 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.841 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.841 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "20", 2024-05-14T17:15:45.842 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.842 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.842 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.842 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-14T17:15:45.842 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.842 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.842 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.842 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_starting_port": { 2024-05-14T17:15:45.842 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_starting_port", 2024-05-14T17:15:45.842 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:45.842 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.842 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.842 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "4721", 2024-05-14T17:15:45.843 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.843 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.843 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.843 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-14T17:15:45.843 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.843 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.843 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.843 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_ptrace": { 2024-05-14T17:15:45.843 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_ptrace", 2024-05-14T17:15:45.843 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.843 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.843 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.843 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:45.843 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-14T17:15:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-14T17:15:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: "autotune_interval": { 2024-05-14T17:15:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "autotune_interval", 2024-05-14T17:15:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-05-14T17:15:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.845 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.845 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-14T17:15:45.845 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.845 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.845 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.845 INFO:teuthology.orchestra.run.smithi002.stdout: "autotune_memory_target_ratio": { 2024-05-14T17:15:45.845 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "autotune_memory_target_ratio", 2024-05-14T17:15:45.845 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T17:15:45.845 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.845 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.845 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0.7", 2024-05-14T17:15:45.845 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.846 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.846 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.846 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-14T17:15:45.846 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.846 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.846 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.846 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm_log_destination": { 2024-05-14T17:15:45.846 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cephadm_log_destination", 2024-05-14T17:15:45.846 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.846 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.846 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.846 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:45.847 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.847 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.847 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:45.847 INFO:teuthology.orchestra.run.smithi002.stdout: "file", 2024-05-14T17:15:45.847 INFO:teuthology.orchestra.run.smithi002.stdout: "file,syslog", 2024-05-14T17:15:45.847 INFO:teuthology.orchestra.run.smithi002.stdout: "syslog" 2024-05-14T17:15:45.847 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:45.847 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-14T17:15:45.847 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.847 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.847 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.847 INFO:teuthology.orchestra.run.smithi002.stdout: "cgroups_split": { 2024-05-14T17:15:45.847 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cgroups_split", 2024-05-14T17:15:45.847 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.848 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.848 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.848 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:45.848 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.848 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.848 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.848 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-14T17:15:45.848 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.848 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.848 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.848 INFO:teuthology.orchestra.run.smithi002.stdout: "config_checks_enabled": { 2024-05-14T17:15:45.848 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_checks_enabled", 2024-05-14T17:15:45.848 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.848 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-14T17:15:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: "config_dashboard": { 2024-05-14T17:15:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_dashboard", 2024-05-14T17:15:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-14T17:15:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_alertmanager": { 2024-05-14T17:15:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_alertmanager", 2024-05-14T17:15:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-14T17:15:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-05-14T17:15:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_base": { 2024-05-14T17:15:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_base", 2024-05-14T17:15:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-14T17:15:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.852 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.852 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Container image name, without the tag", 2024-05-14T17:15:45.852 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.852 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.852 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.852 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_elasticsearch": { 2024-05-14T17:15:45.852 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_elasticsearch", 2024-05-14T17:15:45.852 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.852 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.852 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.852 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-14T17:15:45.852 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.852 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.853 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.853 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "elasticsearch container image", 2024-05-14T17:15:45.853 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.853 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.853 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.853 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_grafana": { 2024-05-14T17:15:45.853 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_grafana", 2024-05-14T17:15:45.853 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.853 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.853 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.853 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-14T17:15:45.853 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.854 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.854 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.854 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-05-14T17:15:45.854 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.854 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.854 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.854 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_haproxy": { 2024-05-14T17:15:45.854 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_haproxy", 2024-05-14T17:15:45.854 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.854 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.854 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.854 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-14T17:15:45.854 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.855 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.855 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.855 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "HAproxy container image", 2024-05-14T17:15:45.855 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.855 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.855 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.855 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_agent": { 2024-05-14T17:15:45.855 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_agent", 2024-05-14T17:15:45.855 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.855 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.855 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.855 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-14T17:15:45.855 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.855 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.856 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.856 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger agent container image", 2024-05-14T17:15:45.856 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.856 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.856 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.856 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_collector": { 2024-05-14T17:15:45.856 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_collector", 2024-05-14T17:15:45.856 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.856 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.856 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.856 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-14T17:15:45.856 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.856 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.856 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.856 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger collector container image", 2024-05-14T17:15:45.857 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.857 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.857 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.857 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_query": { 2024-05-14T17:15:45.857 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_query", 2024-05-14T17:15:45.857 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.857 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.857 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.857 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-14T17:15:45.857 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.857 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.857 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.857 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger query container image", 2024-05-14T17:15:45.857 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.858 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.858 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.858 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_keepalived": { 2024-05-14T17:15:45.858 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_keepalived", 2024-05-14T17:15:45.858 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.858 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.858 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.858 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-14T17:15:45.858 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.858 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.858 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.858 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Keepalived container image", 2024-05-14T17:15:45.858 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.858 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.859 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.859 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_loki": { 2024-05-14T17:15:45.859 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_loki", 2024-05-14T17:15:45.859 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.859 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.859 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.859 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-14T17:15:45.859 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.859 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.859 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.859 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Loki container image", 2024-05-14T17:15:45.859 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.859 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.859 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.860 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_node_exporter": { 2024-05-14T17:15:45.860 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_node_exporter", 2024-05-14T17:15:45.860 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.860 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.860 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.860 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-14T17:15:45.860 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.860 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.860 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.860 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-05-14T17:15:45.860 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.860 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.860 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.860 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.860 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_nvmeof": { 2024-05-14T17:15:45.860 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_nvmeof", 2024-05-14T17:15:45.860 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-14T17:15:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Nvme-of container image", 2024-05-14T17:15:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_prometheus": { 2024-05-14T17:15:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_prometheus", 2024-05-14T17:15:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-14T17:15:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-05-14T17:15:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_promtail": { 2024-05-14T17:15:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_promtail", 2024-05-14T17:15:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.863 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.863 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-14T17:15:45.863 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.863 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.863 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.863 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Promtail container image", 2024-05-14T17:15:45.863 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.863 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.863 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.863 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_samba": { 2024-05-14T17:15:45.863 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_samba", 2024-05-14T17:15:45.863 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.863 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.863 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.864 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-14T17:15:45.864 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.864 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.864 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.864 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Samba/SMB container image", 2024-05-14T17:15:45.864 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.864 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.864 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.864 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_snmp_gateway": { 2024-05-14T17:15:45.864 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_snmp_gateway", 2024-05-14T17:15:45.864 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.864 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.864 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.865 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-14T17:15:45.865 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.865 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.865 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.865 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SNMP Gateway container image", 2024-05-14T17:15:45.865 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.865 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.865 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.865 INFO:teuthology.orchestra.run.smithi002.stdout: "container_init": { 2024-05-14T17:15:45.865 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_init", 2024-05-14T17:15:45.865 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.865 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.865 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.865 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:45.866 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.866 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.866 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.866 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Run podman/docker with `--init`", 2024-05-14T17:15:45.866 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.866 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.866 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.866 INFO:teuthology.orchestra.run.smithi002.stdout: "daemon_cache_timeout": { 2024-05-14T17:15:45.866 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "daemon_cache_timeout", 2024-05-14T17:15:45.866 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:45.866 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.866 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.866 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-05-14T17:15:45.866 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.867 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.867 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.867 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-14T17:15:45.867 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.867 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.867 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.867 INFO:teuthology.orchestra.run.smithi002.stdout: "default_cephadm_command_timeout": { 2024-05-14T17:15:45.867 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "default_cephadm_command_timeout", 2024-05-14T17:15:45.867 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:45.867 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.867 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.867 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "900", 2024-05-14T17:15:45.867 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.867 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.868 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.868 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-14T17:15:45.868 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.868 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.868 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.868 INFO:teuthology.orchestra.run.smithi002.stdout: "default_registry": { 2024-05-14T17:15:45.868 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "default_registry", 2024-05-14T17:15:45.868 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.868 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.868 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.868 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io", 2024-05-14T17:15:45.868 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.868 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.868 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.868 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-14T17:15:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: "device_cache_timeout": { 2024-05-14T17:15:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_cache_timeout", 2024-05-14T17:15:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1800", 2024-05-14T17:15:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache device inventory", 2024-05-14T17:15:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.870 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.870 INFO:teuthology.orchestra.run.smithi002.stdout: "device_enhanced_scan": { 2024-05-14T17:15:45.870 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_enhanced_scan", 2024-05-14T17:15:45.870 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.870 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.870 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.870 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:45.870 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.870 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.870 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.870 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-14T17:15:45.870 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.870 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: "facts_cache_timeout": { 2024-05-14T17:15:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "facts_cache_timeout", 2024-05-14T17:15:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-05-14T17:15:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache host facts data", 2024-05-14T17:15:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: "grafana_dashboards_path": { 2024-05-14T17:15:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "grafana_dashboards_path", 2024-05-14T17:15:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-14T17:15:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-14T17:15:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: "host_check_interval": { 2024-05-14T17:15:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "host_check_interval", 2024-05-14T17:15:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-05-14T17:15:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to perform a host check", 2024-05-14T17:15:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: "hw_monitoring": { 2024-05-14T17:15:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "hw_monitoring", 2024-05-14T17:15:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-14T17:15:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: "inventory_list_all": { 2024-05-14T17:15:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "inventory_list_all", 2024-05-14T17:15:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-14T17:15:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: "log_refresh_metadata": { 2024-05-14T17:15:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_refresh_metadata", 2024-05-14T17:15:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-14T17:15:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-14T17:15:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.879 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:45.879 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:45.879 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.879 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.879 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.879 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:45.879 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.879 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.879 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:45.879 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:45.879 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:45.879 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:45.879 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:45.880 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:45.880 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:45.880 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:45.880 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.880 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.880 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.880 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.880 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:45.880 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:45.880 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.880 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.880 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-14T17:15:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-14T17:15:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.882 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:45.882 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.882 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.882 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.882 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-14T17:15:45.882 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.882 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.882 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.882 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.882 INFO:teuthology.orchestra.run.smithi002.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-14T17:15:45.882 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-14T17:15:45.882 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.882 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.882 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.882 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "*", 2024-05-14T17:15:45.883 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.883 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.883 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.883 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-14T17:15:45.883 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.883 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.883 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.883 INFO:teuthology.orchestra.run.smithi002.stdout: "max_count_per_host": { 2024-05-14T17:15:45.883 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_count_per_host", 2024-05-14T17:15:45.883 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:45.883 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.883 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.883 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T17:15:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "max number of daemons per service per host", 2024-05-14T17:15:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "max_osd_draining_count": { 2024-05-14T17:15:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_osd_draining_count", 2024-05-14T17:15:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T17:15:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-14T17:15:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "migration_current": { 2024-05-14T17:15:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "migration_current", 2024-05-14T17:15:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "internal - do not modify", 2024-05-14T17:15:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": { 2024-05-14T17:15:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mode", 2024-05-14T17:15:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "root", 2024-05-14T17:15:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm-package", 2024-05-14T17:15:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "root" 2024-05-14T17:15:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "mode for remote execution of cephadm", 2024-05-14T17:15:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "oob_default_addr": { 2024-05-14T17:15:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "oob_default_addr", 2024-05-14T17:15:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "169.254.1.1", 2024-05-14T17:15:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-14T17:15:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "prometheus_alerts_path": { 2024-05-14T17:15:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus_alerts_path", 2024-05-14T17:15:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-14T17:15:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-14T17:15:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_insecure": { 2024-05-14T17:15:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_insecure", 2024-05-14T17:15:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-14T17:15:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_password": { 2024-05-14T17:15:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_password", 2024-05-14T17:15:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-14T17:15:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_url": { 2024-05-14T17:15:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_url", 2024-05-14T17:15:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-14T17:15:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_username": { 2024-05-14T17:15:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_username", 2024-05-14T17:15:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-14T17:15:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "secure_monitoring_stack": { 2024-05-14T17:15:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "secure_monitoring_stack", 2024-05-14T17:15:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-14T17:15:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "service_discovery_port": { 2024-05-14T17:15:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "service_discovery_port", 2024-05-14T17:15:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8765", 2024-05-14T17:15:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "cephadm service discovery port", 2024-05-14T17:15:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "ssh_config_file": { 2024-05-14T17:15:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssh_config_file", 2024-05-14T17:15:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-14T17:15:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "use_agent": { 2024-05-14T17:15:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "use_agent", 2024-05-14T17:15:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-14T17:15:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "use_repo_digest": { 2024-05-14T17:15:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "use_repo_digest", 2024-05-14T17:15:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-14T17:15:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_failed_host_check": { 2024-05-14T17:15:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_failed_host_check", 2024-05-14T17:15:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if the host check fails", 2024-05-14T17:15:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_stray_daemons": { 2024-05-14T17:15:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_stray_daemons", 2024-05-14T17:15:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-14T17:15:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_stray_hosts": { 2024-05-14T17:15:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_stray_hosts", 2024-05-14T17:15:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-14T17:15:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crash", 2024-05-14T17:15:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "retain_interval": { 2024-05-14T17:15:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "retain_interval", 2024-05-14T17:15:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "31536000", 2024-05-14T17:15:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-14T17:15:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_recent_interval": { 2024-05-14T17:15:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_recent_interval", 2024-05-14T17:15:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1209600", 2024-05-14T17:15:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-14T17:15:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "dashboard", 2024-05-14T17:15:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-14T17:15:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-14T17:15:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T17:15:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "ALERTMANAGER_API_HOST": { 2024-05-14T17:15:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-14T17:15:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-14T17:15:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-14T17:15:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "AUDIT_API_ENABLED": { 2024-05-14T17:15:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "AUDIT_API_ENABLED", 2024-05-14T17:15:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-14T17:15:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-14T17:15:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "ENABLE_BROWSABLE_API": { 2024-05-14T17:15:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-14T17:15:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-14T17:15:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-14T17:15:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-14T17:15:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-14T17:15:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-14T17:15:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-14T17:15:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-14T17:15:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-14T17:15:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-14T17:15:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-14T17:15:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-14T17:15:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-14T17:15:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-14T17:15:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-14T17:15:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-14T17:15:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-14T17:15:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_PASSWORD": { 2024-05-14T17:15:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-14T17:15:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2024-05-14T17:15:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-14T17:15:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-14T17:15:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_URL": { 2024-05-14T17:15:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_URL", 2024-05-14T17:15:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_USERNAME": { 2024-05-14T17:15:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-14T17:15:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2024-05-14T17:15:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-14T17:15:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-14T17:15:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-14T17:15:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-14T17:15:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-14T17:15:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-14T17:15:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-14T17:15:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-14T17:15:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-14T17:15:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-14T17:15:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "[]", 2024-05-14T17:15:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "MULTICLUSTER_CONFIG": { 2024-05-14T17:15:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-14T17:15:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "{}", 2024-05-14T17:15:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "PROMETHEUS_API_HOST": { 2024-05-14T17:15:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-14T17:15:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-14T17:15:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-14T17:15:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-14T17:15:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-14T17:15:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-14T17:15:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-14T17:15:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-14T17:15:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-14T17:15:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-14T17:15:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-14T17:15:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-14T17:15:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-14T17:15:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-14T17:15:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-14T17:15:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-14T17:15:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-14T17:15:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_ENABLED": { 2024-05-14T17:15:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-14T17:15:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-14T17:15:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-14T17:15:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-14T17:15:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-14T17:15:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-14T17:15:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T17:15:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-14T17:15:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-14T17:15:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8", 2024-05-14T17:15:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-14T17:15:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-14T17:15:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "45", 2024-05-14T17:15:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_ACCESS_KEY": { 2024-05-14T17:15:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-14T17:15:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-14T17:15:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-14T17:15:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2024-05-14T17:15:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_SECRET_KEY": { 2024-05-14T17:15:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-14T17:15:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_SSL_VERIFY": { 2024-05-14T17:15:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-14T17:15:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "UNSAFE_TLS_v1_2": { 2024-05-14T17:15:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-14T17:15:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-14T17:15:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-14T17:15:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-14T17:15:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-14T17:15:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T17:15:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-14T17:15:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-14T17:15:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-05-14T17:15:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "cross_origin_url": { 2024-05-14T17:15:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cross_origin_url", 2024-05-14T17:15:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "crt_file": { 2024-05-14T17:15:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crt_file", 2024-05-14T17:15:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "debug": { 2024-05-14T17:15:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "debug", 2024-05-14T17:15:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable/disable debug options", 2024-05-14T17:15:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "jwt_token_ttl": { 2024-05-14T17:15:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "jwt_token_ttl", 2024-05-14T17:15:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "28800", 2024-05-14T17:15:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "key_file": { 2024-05-14T17:15:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "key_file", 2024-05-14T17:15:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "motd": { 2024-05-14T17:15:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "motd", 2024-05-14T17:15:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "The message of the day", 2024-05-14T17:15:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "redirect_resolve_ip_addr": { 2024-05-14T17:15:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "redirect_resolve_ip_addr", 2024-05-14T17:15:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2024-05-14T17:15:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2024-05-14T17:15:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "::", 2024-05-14T17:15:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.967 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.968 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.968 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.968 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2024-05-14T17:15:45.968 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2024-05-14T17:15:45.968 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:45.968 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.968 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.968 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8080", 2024-05-14T17:15:45.968 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.968 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.968 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.968 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.968 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.968 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.969 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.969 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:45.969 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:45.969 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:45.969 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:45.969 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.969 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:45.969 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.969 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.969 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.969 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.969 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.969 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.970 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.970 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl": { 2024-05-14T17:15:45.970 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl", 2024-05-14T17:15:45.970 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.970 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.970 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.970 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:45.970 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.970 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.970 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.970 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.970 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.970 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.971 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.971 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl_server_port": { 2024-05-14T17:15:45.971 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl_server_port", 2024-05-14T17:15:45.971 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:45.971 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.971 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.971 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8443", 2024-05-14T17:15:45.971 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.971 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.971 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.971 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.971 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.971 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.971 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.972 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_behaviour": { 2024-05-14T17:15:45.972 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_behaviour", 2024-05-14T17:15:45.972 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.972 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.972 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.972 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "redirect", 2024-05-14T17:15:45.972 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.972 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.972 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:45.972 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:45.972 INFO:teuthology.orchestra.run.smithi002.stdout: "redirect" 2024-05-14T17:15:45.972 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:45.972 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.972 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.973 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.973 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.973 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_error_status_code": { 2024-05-14T17:15:45.973 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_error_status_code", 2024-05-14T17:15:45.973 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:45.973 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.973 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.973 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "500", 2024-05-14T17:15:45.973 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "400", 2024-05-14T17:15:45.973 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "599", 2024-05-14T17:15:45.973 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.973 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.973 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.973 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.973 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.974 INFO:teuthology.orchestra.run.smithi002.stdout: "url_prefix": { 2024-05-14T17:15:45.974 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "url_prefix", 2024-05-14T17:15:45.974 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.974 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.974 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.974 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:45.974 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.974 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.974 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.974 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.974 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.974 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.974 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.974 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:45.975 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:45.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:45.975 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "devicehealth", 2024-05-14T17:15:45.975 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:45.975 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:45.975 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:45.975 INFO:teuthology.orchestra.run.smithi002.stdout: "enable_monitoring": { 2024-05-14T17:15:45.975 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enable_monitoring", 2024-05-14T17:15:45.975 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.975 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.975 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.975 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:45.975 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "monitor device health metrics", 2024-05-14T17:15:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.977 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:45.977 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:45.977 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:45.977 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:45.977 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:45.977 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:45.977 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:45.977 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:45.977 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.977 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.977 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.977 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.977 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:45.977 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:45.978 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.978 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.978 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.978 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:45.978 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.978 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.978 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.978 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.978 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.978 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.978 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.978 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:45.978 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:45.979 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.979 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.979 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.979 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:45.979 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.979 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.979 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:45.979 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:45.979 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:45.979 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:45.979 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:45.979 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:45.979 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:45.979 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:45.980 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.980 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.980 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.980 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.980 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:45.980 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:45.980 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.980 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.980 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.980 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:45.980 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.980 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.980 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.981 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.981 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.981 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.981 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.981 INFO:teuthology.orchestra.run.smithi002.stdout: "mark_out_threshold": { 2024-05-14T17:15:45.981 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mark_out_threshold", 2024-05-14T17:15:45.981 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:45.981 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.981 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.981 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "2419200", 2024-05-14T17:15:45.981 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.981 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.981 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.982 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-14T17:15:45.982 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.982 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.982 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.982 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": { 2024-05-14T17:15:45.982 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pool_name", 2024-05-14T17:15:45.982 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.982 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.982 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.982 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "device_health_metrics", 2024-05-14T17:15:45.982 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.982 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.982 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.983 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-14T17:15:45.983 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.983 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.983 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.983 INFO:teuthology.orchestra.run.smithi002.stdout: "retention_period": { 2024-05-14T17:15:45.983 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "retention_period", 2024-05-14T17:15:45.983 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:45.983 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.983 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.983 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15552000", 2024-05-14T17:15:45.983 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.984 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.984 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.984 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long to retain device health metrics", 2024-05-14T17:15:45.984 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.984 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.984 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.984 INFO:teuthology.orchestra.run.smithi002.stdout: "scrape_frequency": { 2024-05-14T17:15:45.984 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "scrape_frequency", 2024-05-14T17:15:45.984 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:45.984 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.984 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.984 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "86400", 2024-05-14T17:15:45.984 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-14T17:15:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "self_heal": { 2024-05-14T17:15:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "self_heal", 2024-05-14T17:15:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:45.985 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.986 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.986 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.986 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-14T17:15:45.986 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.986 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.986 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.986 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-05-14T17:15:45.986 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-05-14T17:15:45.986 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:45.986 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.986 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.986 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-05-14T17:15:45.986 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.986 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.987 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.987 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to wake up and check device health", 2024-05-14T17:15:45.987 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.987 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.987 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.987 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:45.987 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:45.987 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:45.987 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:45.987 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.987 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:45.988 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.988 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.988 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.988 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.988 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.988 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.988 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.988 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_threshold": { 2024-05-14T17:15:45.988 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_threshold", 2024-05-14T17:15:45.988 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:45.988 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.988 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.988 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "7257600", 2024-05-14T17:15:45.988 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.989 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.989 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.989 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-14T17:15:45.989 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.989 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.989 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.989 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:45.989 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:45.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:45.989 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "diskprediction_local", 2024-05-14T17:15:45.989 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:45.989 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:45.989 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:45.989 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:45.990 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:45.990 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.990 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.990 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.990 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:45.990 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.990 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.990 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:45.990 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:45.990 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:45.990 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:45.990 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:45.991 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:45.991 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:45.991 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:45.991 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.991 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.991 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.991 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.991 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:45.991 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:45.991 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.991 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.991 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.991 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:45.992 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.992 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.992 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.992 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.992 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.992 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.992 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.992 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:45.992 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:45.992 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.992 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.992 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.992 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:45.993 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.993 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.993 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:45.993 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:45.993 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:45.993 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:45.993 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:45.993 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:45.993 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:45.993 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:45.993 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.993 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.993 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.993 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.994 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:45.994 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:45.994 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:45.994 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.994 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.994 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:45.994 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.994 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.994 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.994 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.994 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.994 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.994 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.995 INFO:teuthology.orchestra.run.smithi002.stdout: "predict_interval": { 2024-05-14T17:15:45.995 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "predict_interval", 2024-05-14T17:15:45.995 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.995 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.995 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.995 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "86400", 2024-05-14T17:15:45.995 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.995 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.995 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.995 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.995 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.995 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.995 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.995 INFO:teuthology.orchestra.run.smithi002.stdout: "predictor_model": { 2024-05-14T17:15:45.996 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "predictor_model", 2024-05-14T17:15:45.996 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.996 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.996 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.996 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "prophetstor", 2024-05-14T17:15:45.996 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.996 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.996 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.996 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.996 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.996 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.996 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.996 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-05-14T17:15:45.996 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-05-14T17:15:45.997 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:45.997 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.997 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.997 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-05-14T17:15:45.997 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.997 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.997 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.997 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.997 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.997 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.997 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.997 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:45.997 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:45.997 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:45.998 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:45.998 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:45.998 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:45.998 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.998 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.998 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.998 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:45.998 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.998 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.998 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.998 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:45.998 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:45.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:45.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:45.998 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "influx", 2024-05-14T17:15:45.998 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": false, 2024-05-14T17:15:45.999 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "influxdb python module not found", 2024-05-14T17:15:45.999 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:45.999 INFO:teuthology.orchestra.run.smithi002.stdout: "batch_size": { 2024-05-14T17:15:45.999 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "batch_size", 2024-05-14T17:15:45.999 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:45.999 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:45.999 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:45.999 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5000", 2024-05-14T17:15:45.999 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:45.999 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:45.999 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:45.999 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-14T17:15:45.999 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:45.999 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:45.999 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:45.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.000 INFO:teuthology.orchestra.run.smithi002.stdout: "database": { 2024-05-14T17:15:46.000 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "database", 2024-05-14T17:15:46.000 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.000 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.000 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.000 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "ceph", 2024-05-14T17:15:46.000 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.000 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.000 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.000 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-14T17:15:46.000 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.000 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.000 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.000 INFO:teuthology.orchestra.run.smithi002.stdout: "hostname": { 2024-05-14T17:15:46.001 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "hostname", 2024-05-14T17:15:46.001 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.001 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.001 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.001 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.001 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.001 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.001 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.001 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB server hostname", 2024-05-14T17:15:46.001 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.001 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.001 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.001 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-05-14T17:15:46.001 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-05-14T17:15:46.002 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.002 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.002 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.002 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "30", 2024-05-14T17:15:46.002 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "5", 2024-05-14T17:15:46.002 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.002 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.002 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-14T17:15:46.002 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.002 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.002 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.002 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.002 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.003 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.004 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.005 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.006 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.006 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.006 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.006 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.006 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.006 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.006 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.006 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.006 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.006 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.006 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.006 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.006 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.006 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.007 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.007 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.007 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.007 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.007 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.007 INFO:teuthology.orchestra.run.smithi002.stdout: "password": { 2024-05-14T17:15:46.007 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "password", 2024-05-14T17:15:46.007 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.007 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.007 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.007 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.007 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.007 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.007 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "password of InfluxDB server user", 2024-05-14T17:15:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: "port": { 2024-05-14T17:15:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "port", 2024-05-14T17:15:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8086", 2024-05-14T17:15:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB server port", 2024-05-14T17:15:46.008 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.009 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.009 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.009 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.009 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.009 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.009 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.009 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.009 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.009 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.009 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.009 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.009 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.009 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.009 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.009 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.010 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl": { 2024-05-14T17:15:46.010 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl", 2024-05-14T17:15:46.010 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.010 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.010 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.010 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "false", 2024-05-14T17:15:46.010 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.010 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.010 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.010 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-14T17:15:46.010 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.010 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.010 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.011 INFO:teuthology.orchestra.run.smithi002.stdout: "threads": { 2024-05-14T17:15:46.011 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "threads", 2024-05-14T17:15:46.011 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.011 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.011 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.011 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-05-14T17:15:46.011 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-05-14T17:15:46.011 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "32", 2024-05-14T17:15:46.011 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.011 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-14T17:15:46.011 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.011 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.011 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: "username": { 2024-05-14T17:15:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "username", 2024-05-14T17:15:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "username of InfluxDB server user", 2024-05-14T17:15:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: "verify_ssl": { 2024-05-14T17:15:46.012 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "verify_ssl", 2024-05-14T17:15:46.013 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.013 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.013 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.013 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "true", 2024-05-14T17:15:46.013 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.013 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.013 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.013 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-14T17:15:46.013 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.013 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.013 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.013 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.013 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.013 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "insights", 2024-05-14T17:15:46.014 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.014 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.014 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.014 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.014 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.014 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.014 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.014 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.014 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.014 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.014 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.014 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.014 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.015 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.015 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.015 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.015 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.015 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.015 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.015 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.015 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.015 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.015 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.015 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.015 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.016 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.017 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.018 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.018 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.018 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.018 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.018 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.018 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.018 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.018 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.018 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.018 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.018 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.018 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.018 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "iostat", 2024-05-14T17:15:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.020 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.021 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.022 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.022 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.022 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.022 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.022 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.022 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.022 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.022 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.022 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.022 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.022 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.022 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.022 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.022 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.023 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.023 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.023 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.023 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.023 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.023 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.023 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.023 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.023 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.023 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.023 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.023 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.023 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.023 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.023 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.024 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.024 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.024 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.024 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.024 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.024 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.024 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.024 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.024 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.024 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.024 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.024 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.024 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "k8sevents", 2024-05-14T17:15:46.025 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_event_retention_days": { 2024-05-14T17:15:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ceph_event_retention_days", 2024-05-14T17:15:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "7", 2024-05-14T17:15:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-14T17:15:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.026 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.027 INFO:teuthology.orchestra.run.smithi002.stdout: "config_check_secs": { 2024-05-14T17:15:46.027 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_check_secs", 2024-05-14T17:15:46.027 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.027 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.027 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.027 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T17:15:46.027 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "10", 2024-05-14T17:15:46.027 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.027 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.027 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-14T17:15:46.027 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.027 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.027 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.028 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.029 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.029 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.029 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.029 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.029 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.029 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.029 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.029 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.029 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.029 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.029 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.029 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.029 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.029 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.030 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.030 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.030 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.030 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.030 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.030 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.030 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.030 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.030 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.030 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.030 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.030 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.030 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.031 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.031 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.031 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.031 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.031 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.031 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.031 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.031 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.031 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.031 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.031 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.031 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.031 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.031 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.032 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.032 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.032 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.032 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.032 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.032 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.032 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.032 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.032 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.032 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.032 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.032 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.032 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.033 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.034 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "localpool", 2024-05-14T17:15:46.034 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.034 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.034 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.034 INFO:teuthology.orchestra.run.smithi002.stdout: "failure_domain": { 2024-05-14T17:15:46.034 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "failure_domain", 2024-05-14T17:15:46.034 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.034 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.034 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.034 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "host", 2024-05-14T17:15:46.034 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.034 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.035 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.035 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "failure domain for any created local pool", 2024-05-14T17:15:46.035 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-14T17:15:46.035 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.035 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.035 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.035 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.035 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.035 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.035 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.035 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.035 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.035 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.035 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.036 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.037 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.037 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.037 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.037 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.037 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.037 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.037 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.037 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.037 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.037 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.037 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.037 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.037 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.037 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.038 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.038 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.038 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.038 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.038 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.038 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.038 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.038 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.038 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.038 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.038 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.038 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.038 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.038 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.038 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.039 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.039 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.039 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.039 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.039 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.039 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.039 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.039 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.039 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.039 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.039 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.039 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.039 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.039 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.040 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": { 2024-05-14T17:15:46.040 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "min_size", 2024-05-14T17:15:46.040 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.040 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.040 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.040 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.040 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.040 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.040 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.040 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default min_size for any created local pool", 2024-05-14T17:15:46.040 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-14T17:15:46.040 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.040 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.041 INFO:teuthology.orchestra.run.smithi002.stdout: "num_rep": { 2024-05-14T17:15:46.041 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "num_rep", 2024-05-14T17:15:46.041 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.041 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.041 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.041 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3", 2024-05-14T17:15:46.041 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.041 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.041 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.041 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default replica count for any created local pool", 2024-05-14T17:15:46.041 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.041 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.042 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.042 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": { 2024-05-14T17:15:46.042 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pg_num", 2024-05-14T17:15:46.042 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.042 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.042 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.042 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "128", 2024-05-14T17:15:46.042 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.042 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.042 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.042 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default pg_num for any created local pool", 2024-05-14T17:15:46.042 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.042 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.042 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.043 INFO:teuthology.orchestra.run.smithi002.stdout: "prefix": { 2024-05-14T17:15:46.043 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prefix", 2024-05-14T17:15:46.043 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.043 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.043 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.043 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.043 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.043 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.043 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.043 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "name prefix for any created local pool", 2024-05-14T17:15:46.043 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.043 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.043 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.044 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.044 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.044 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.044 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.044 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.044 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.044 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.044 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.044 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.044 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.044 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.044 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.044 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.045 INFO:teuthology.orchestra.run.smithi002.stdout: "subtree": { 2024-05-14T17:15:46.045 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "subtree", 2024-05-14T17:15:46.045 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.045 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.045 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.045 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "rack", 2024-05-14T17:15:46.045 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.045 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.045 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.045 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-14T17:15:46.045 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-14T17:15:46.045 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.045 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.045 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.046 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.046 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mds_autoscaler", 2024-05-14T17:15:46.046 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.046 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.046 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.046 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.046 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.046 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.046 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.046 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.046 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.046 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.047 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.047 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.047 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.047 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.047 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.047 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.047 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.047 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.047 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.047 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.047 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.047 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.047 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.047 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.048 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.048 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.048 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.048 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.048 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.048 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.048 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.048 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.048 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.048 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.048 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.048 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.048 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.049 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.049 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.049 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.049 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.049 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.049 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.049 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.049 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.049 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.049 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.049 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.049 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.049 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.049 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.049 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.050 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.050 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.050 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.050 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.050 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.050 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.050 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.050 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.050 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.050 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.050 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.050 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.050 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.051 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.051 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.051 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.051 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.051 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.051 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.051 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.051 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.051 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.051 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.051 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.051 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.051 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.051 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.052 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.052 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.052 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.052 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.052 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.052 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mirroring", 2024-05-14T17:15:46.052 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.052 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.052 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.052 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.052 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.052 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.053 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.053 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.053 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.053 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.053 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.053 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.053 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.053 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.053 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.053 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.053 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.053 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.053 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.053 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.054 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.054 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.054 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.054 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.054 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.054 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.054 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.054 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.054 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.054 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.054 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.054 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.055 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.055 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.055 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.055 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.055 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.055 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.055 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.055 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.055 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.055 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.055 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.055 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.055 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.056 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.056 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.056 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.056 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.056 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.056 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.056 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.056 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.056 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.056 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.056 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.056 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.056 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.056 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.057 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.057 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.057 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.057 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.057 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.057 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.057 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.057 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.057 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.057 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.057 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.057 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.057 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.058 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.058 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.058 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.058 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.058 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.058 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.058 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.058 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.058 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.058 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.058 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.058 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.058 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "nfs", 2024-05-14T17:15:46.059 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.059 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.059 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.059 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.059 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.059 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.059 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.059 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.059 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.059 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.059 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.059 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.059 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.059 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.060 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.060 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.060 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.060 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.060 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.060 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.060 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.060 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.060 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.060 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.060 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.060 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.060 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.060 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.061 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.061 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.061 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.061 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.061 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.061 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.061 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.061 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.061 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.061 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.061 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.061 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.061 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.061 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.062 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.062 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.062 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.062 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.062 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.062 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.062 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.062 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.062 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.062 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.062 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.062 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.062 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.062 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.063 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.063 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.063 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.063 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.063 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.063 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.063 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.063 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.063 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.063 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.063 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.063 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.063 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.064 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.064 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.064 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.064 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.064 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.064 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.064 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.064 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.064 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.064 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.064 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.064 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.064 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.064 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.064 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.065 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "orchestrator", 2024-05-14T17:15:46.065 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.065 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.065 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.065 INFO:teuthology.orchestra.run.smithi002.stdout: "fail_fs": { 2024-05-14T17:15:46.065 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "fail_fs", 2024-05-14T17:15:46.065 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.065 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.065 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.065 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.065 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.065 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.066 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.066 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-14T17:15:46.066 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.066 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.066 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.066 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.066 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.066 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.066 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.066 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.066 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.066 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.066 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.067 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.067 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.067 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.067 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.067 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.067 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.067 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.067 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.067 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.067 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.067 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.067 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.067 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.067 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.068 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.068 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.068 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.068 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.068 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.068 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.068 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.068 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.068 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.068 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.068 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.068 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.068 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.069 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.069 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.069 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.069 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.069 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.069 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.069 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.069 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.069 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.069 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.069 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.069 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.069 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.069 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.069 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.069 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.070 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.070 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.070 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.070 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.070 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.070 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.070 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.070 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.070 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.070 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.070 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.070 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.070 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.070 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.071 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.071 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator": { 2024-05-14T17:15:46.071 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "orchestrator", 2024-05-14T17:15:46.071 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.071 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.071 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.071 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.071 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.071 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.071 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.071 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm", 2024-05-14T17:15:46.071 INFO:teuthology.orchestra.run.smithi002.stdout: "rook", 2024-05-14T17:15:46.071 INFO:teuthology.orchestra.run.smithi002.stdout: "test_orchestrator" 2024-05-14T17:15:46.071 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.072 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Orchestrator backend", 2024-05-14T17:15:46.072 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.072 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.072 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.072 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.072 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.072 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.072 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.072 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.072 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.072 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.072 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.072 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.072 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.073 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.073 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.073 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.073 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.073 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.073 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "osd_perf_query", 2024-05-14T17:15:46.073 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.073 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.073 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.073 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.073 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.073 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.073 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.074 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.074 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.074 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.074 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.074 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.074 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.074 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.074 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.074 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.074 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.074 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.074 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.074 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.074 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.074 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.075 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.075 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.075 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.075 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.075 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.075 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.075 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.075 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.075 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.075 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.075 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.075 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.075 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.076 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.076 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.076 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.076 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.076 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.076 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.076 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.076 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.076 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.076 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.076 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.076 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.076 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.076 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.077 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.077 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.077 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.077 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.077 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.077 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.077 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.077 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.077 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.077 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.077 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.077 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.077 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.077 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.078 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.078 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.078 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.078 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.078 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.078 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.078 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.078 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.078 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.078 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.078 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.078 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.078 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.079 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.079 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.079 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.079 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.079 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.079 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.079 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.079 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.079 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "osd_support", 2024-05-14T17:15:46.079 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.079 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.079 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.079 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.080 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.080 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.080 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.080 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.080 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.080 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.080 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.080 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.080 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.080 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.080 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.080 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.080 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.080 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.081 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.081 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.081 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.081 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.081 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.081 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.081 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.081 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.081 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.081 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.081 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.081 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.082 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.082 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.082 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.082 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.082 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.082 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.082 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.082 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.082 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.082 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.082 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.082 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.082 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.083 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.083 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.083 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.083 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.083 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.083 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.083 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.083 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.083 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.083 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.083 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.083 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.083 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.084 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.084 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.084 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.084 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.084 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.084 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.084 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.084 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.084 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.084 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.084 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.084 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.084 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.084 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.085 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.085 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.085 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.085 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.085 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.085 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.085 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.085 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.085 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.085 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.085 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.085 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.085 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.085 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.086 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pg_autoscaler", 2024-05-14T17:15:46.086 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.086 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.086 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.086 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.086 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.086 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.086 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.086 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.086 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.086 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.086 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.086 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.086 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.087 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.087 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.087 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.087 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.087 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.087 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.087 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.087 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.087 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.087 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.087 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.087 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.087 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.088 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.088 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.088 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.088 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.088 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.088 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.088 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.088 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.088 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.088 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.088 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.088 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.088 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.088 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.088 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.089 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.089 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.089 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.089 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.089 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.089 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.089 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.089 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.089 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.089 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.089 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.089 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.089 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.089 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.089 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.090 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.090 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.090 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.090 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.090 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.090 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.090 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.090 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.090 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.090 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.090 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.090 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.090 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.091 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.091 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-05-14T17:15:46.091 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-05-14T17:15:46.091 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.091 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.091 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.091 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-05-14T17:15:46.091 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.091 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.091 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.091 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.091 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.091 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.092 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.092 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.092 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.092 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.092 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.092 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.092 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.092 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.092 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.092 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.092 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.092 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.092 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.093 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.093 INFO:teuthology.orchestra.run.smithi002.stdout: "threshold": { 2024-05-14T17:15:46.093 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "threshold", 2024-05-14T17:15:46.093 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T17:15:46.093 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.093 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.093 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3.0", 2024-05-14T17:15:46.093 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1.0", 2024-05-14T17:15:46.093 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.093 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.093 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "scaling threshold", 2024-05-14T17:15:46.093 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-14T17:15:46.093 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.094 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.094 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.094 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.094 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.094 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.094 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "progress", 2024-05-14T17:15:46.094 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.094 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.094 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.094 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_pg_recovery_event": { 2024-05-14T17:15:46.094 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_pg_recovery_event", 2024-05-14T17:15:46.094 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.094 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.094 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.095 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.095 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.095 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.095 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.095 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-14T17:15:46.095 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.095 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.095 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.095 INFO:teuthology.orchestra.run.smithi002.stdout: "enabled": { 2024-05-14T17:15:46.095 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enabled", 2024-05-14T17:15:46.095 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.095 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.096 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.096 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.096 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.096 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.096 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.096 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.096 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.096 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.096 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.096 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.096 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.096 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.097 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.097 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.097 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.097 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.097 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.097 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.097 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.097 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.097 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.097 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.098 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.098 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.098 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.098 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.098 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.098 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.098 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.098 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.098 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.098 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.098 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.098 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.098 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.098 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.099 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.099 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.099 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.099 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.099 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.099 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.099 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.099 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.099 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.099 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.099 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.099 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.099 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.099 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.100 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.100 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.100 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.100 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.100 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.100 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.100 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.100 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.100 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.100 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.100 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.100 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.100 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.101 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.101 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.101 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.101 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.101 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.101 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.101 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.101 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.101 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.101 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.101 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.101 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.101 INFO:teuthology.orchestra.run.smithi002.stdout: "max_completed_events": { 2024-05-14T17:15:46.101 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_completed_events", 2024-05-14T17:15:46.102 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.102 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.102 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.102 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "50", 2024-05-14T17:15:46.102 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.102 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.102 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.102 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "number of past completed events to remember", 2024-05-14T17:15:46.102 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.102 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.102 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.102 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.102 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-05-14T17:15:46.103 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-05-14T17:15:46.103 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.103 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.103 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.103 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-05-14T17:15:46.103 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.103 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.103 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.103 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long the module is going to sleep", 2024-05-14T17:15:46.103 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.103 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.103 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.103 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.103 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.103 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.103 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.104 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.104 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.104 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.104 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.104 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.104 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.104 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.104 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.104 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.104 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.104 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.104 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.104 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus", 2024-05-14T17:15:46.105 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.105 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.105 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.105 INFO:teuthology.orchestra.run.smithi002.stdout: "cache": { 2024-05-14T17:15:46.105 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cache", 2024-05-14T17:15:46.105 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.105 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.105 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.105 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.105 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.105 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.105 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.105 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.105 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.105 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.106 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.106 INFO:teuthology.orchestra.run.smithi002.stdout: "exclude_perf_counters": { 2024-05-14T17:15:46.106 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "exclude_perf_counters", 2024-05-14T17:15:46.106 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.106 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.106 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.106 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.106 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.106 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.106 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.106 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-14T17:15:46.106 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-14T17:15:46.106 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.107 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.107 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.107 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.107 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.107 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.107 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.107 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.107 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.107 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.107 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.107 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.107 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.107 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.107 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.108 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.108 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.108 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.108 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.108 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.108 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.108 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.108 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.108 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.109 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.109 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.109 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.109 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.109 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.109 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.109 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.109 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.109 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.109 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.109 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.109 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.109 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.109 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.109 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.110 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.110 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.110 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.110 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.110 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.110 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.110 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.110 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.110 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.110 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.110 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.110 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.110 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.110 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.110 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.111 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.111 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.111 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.111 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.111 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.111 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.111 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.111 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.111 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.111 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.111 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.111 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.111 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.111 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.112 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_stats_pools": { 2024-05-14T17:15:46.112 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_stats_pools", 2024-05-14T17:15:46.112 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.112 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.112 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.112 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.112 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.112 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.112 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.112 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.112 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.112 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.112 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.112 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-14T17:15:46.113 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-14T17:15:46.113 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.113 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.113 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.113 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "300", 2024-05-14T17:15:46.113 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.113 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.113 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.113 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.113 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.113 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.113 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.113 INFO:teuthology.orchestra.run.smithi002.stdout: "scrape_interval": { 2024-05-14T17:15:46.113 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "scrape_interval", 2024-05-14T17:15:46.113 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T17:15:46.114 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.114 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.114 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15.0", 2024-05-14T17:15:46.114 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.114 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.114 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.114 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.114 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.114 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.114 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.114 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2024-05-14T17:15:46.114 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2024-05-14T17:15:46.115 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.115 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.115 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.115 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "::", 2024-05-14T17:15:46.115 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.115 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.115 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.115 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-14T17:15:46.115 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.115 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.115 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.115 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2024-05-14T17:15:46.115 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2024-05-14T17:15:46.115 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.116 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.116 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.116 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "9283", 2024-05-14T17:15:46.116 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.116 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.116 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.116 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-14T17:15:46.116 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.116 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.116 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.116 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.116 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.116 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.116 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.116 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.117 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.117 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.117 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.117 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.117 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.117 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.117 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.117 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.117 INFO:teuthology.orchestra.run.smithi002.stdout: "stale_cache_strategy": { 2024-05-14T17:15:46.117 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "stale_cache_strategy", 2024-05-14T17:15:46.117 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.117 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.117 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.117 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "log", 2024-05-14T17:15:46.117 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.118 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.118 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.118 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.118 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.118 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.118 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.118 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_behaviour": { 2024-05-14T17:15:46.118 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_behaviour", 2024-05-14T17:15:46.118 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.118 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.118 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.118 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "default", 2024-05-14T17:15:46.118 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.118 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.119 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.119 INFO:teuthology.orchestra.run.smithi002.stdout: "default", 2024-05-14T17:15:46.119 INFO:teuthology.orchestra.run.smithi002.stdout: "error" 2024-05-14T17:15:46.119 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.119 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.119 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.119 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.119 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.119 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_error_status_code": { 2024-05-14T17:15:46.119 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_error_status_code", 2024-05-14T17:15:46.119 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.119 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.119 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.119 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "500", 2024-05-14T17:15:46.119 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "400", 2024-05-14T17:15:46.120 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "599", 2024-05-14T17:15:46.120 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.120 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.120 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.120 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.120 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.120 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.120 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.120 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_support", 2024-05-14T17:15:46.120 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.120 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.120 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.120 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.121 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.121 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.121 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.121 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.121 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.121 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.121 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.121 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.121 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.121 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.121 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.121 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.121 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.121 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.121 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.122 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.122 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.122 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.122 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.122 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.122 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.122 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.122 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.122 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.122 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.122 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.122 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.122 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.123 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.123 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.123 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.123 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.123 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.123 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.123 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.123 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.123 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.123 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.123 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.123 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.123 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.123 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.124 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.124 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.124 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.124 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.124 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.124 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.124 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.124 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.124 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.124 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.124 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.124 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.124 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.125 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.125 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.125 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.125 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.125 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.125 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.125 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.125 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.125 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.125 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.125 INFO:teuthology.orchestra.run.smithi002.stdout: "max_concurrent_snap_create": { 2024-05-14T17:15:46.125 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_concurrent_snap_create", 2024-05-14T17:15:46.125 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.126 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.126 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.126 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T17:15:46.126 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.126 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.126 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.126 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.126 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.126 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.126 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.126 INFO:teuthology.orchestra.run.smithi002.stdout: "mirror_snapshot_schedule": { 2024-05-14T17:15:46.126 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mirror_snapshot_schedule", 2024-05-14T17:15:46.126 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.126 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.127 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.127 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.127 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.127 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.127 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.127 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.127 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.127 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.127 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.127 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.127 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.127 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.127 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.127 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.128 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.128 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.128 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.128 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.128 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.128 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.128 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.128 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.128 INFO:teuthology.orchestra.run.smithi002.stdout: "trash_purge_schedule": { 2024-05-14T17:15:46.128 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "trash_purge_schedule", 2024-05-14T17:15:46.128 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.128 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.128 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.129 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.129 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.129 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.129 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.129 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.129 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.129 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.129 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.129 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.129 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.129 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "restful", 2024-05-14T17:15:46.129 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.129 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.129 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.130 INFO:teuthology.orchestra.run.smithi002.stdout: "enable_auth": { 2024-05-14T17:15:46.130 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enable_auth", 2024-05-14T17:15:46.130 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.130 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.130 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.130 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.130 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.130 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.130 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.130 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.130 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.130 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.130 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.131 INFO:teuthology.orchestra.run.smithi002.stdout: "key_file": { 2024-05-14T17:15:46.131 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "key_file", 2024-05-14T17:15:46.131 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.131 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.131 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.131 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.131 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.131 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.131 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.131 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.131 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.131 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.131 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.131 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.132 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.132 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.132 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.132 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.132 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.132 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.132 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.132 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.132 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.132 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.132 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.132 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.132 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.132 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.132 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.132 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.133 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.133 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.133 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.133 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.133 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.133 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.133 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.133 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.133 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.133 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.133 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.133 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.133 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.133 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.134 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.134 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.134 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.134 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.134 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.134 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.134 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.134 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.134 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.134 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.134 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.134 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.134 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.135 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.135 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.135 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.135 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.135 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.135 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.135 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.135 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.135 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.135 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.135 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.135 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.135 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.136 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.136 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.136 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.136 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.136 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.136 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.136 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.136 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.136 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.136 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2024-05-14T17:15:46.136 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2024-05-14T17:15:46.136 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.136 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.136 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.136 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.137 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.137 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.137 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.137 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.137 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.137 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.137 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.137 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2024-05-14T17:15:46.137 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2024-05-14T17:15:46.137 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.137 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.137 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.137 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.137 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.138 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.138 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.138 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.138 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.138 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.138 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.138 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.138 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.138 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.138 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.138 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.138 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.138 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.139 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.139 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.139 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.139 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.139 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.139 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.139 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.139 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.139 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rgw", 2024-05-14T17:15:46.139 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.139 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.139 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.139 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.139 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.140 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.140 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.140 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.140 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.140 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.140 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.140 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.140 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.140 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.140 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.140 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.140 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.140 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.140 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.140 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.140 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.141 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.141 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.141 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.141 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.141 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.141 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.141 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.141 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.141 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.141 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.141 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.141 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.141 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.142 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.142 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.142 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.142 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.142 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.142 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.142 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.142 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.142 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.142 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.142 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.142 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.142 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.142 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.143 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.143 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.143 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.143 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.143 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.143 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.143 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.143 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.143 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.143 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.143 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.143 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.143 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.143 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.144 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.144 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.144 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.144 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.144 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.144 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.144 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.144 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.144 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.144 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.144 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.144 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.144 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.144 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.145 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.145 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.145 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.145 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.145 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.145 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.145 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.145 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.145 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rook", 2024-05-14T17:15:46.145 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.145 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.145 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.145 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.146 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.146 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.146 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.146 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.146 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.146 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.146 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.146 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.146 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.146 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.146 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.146 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.146 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.146 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.147 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.147 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.147 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.147 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.147 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.147 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.147 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.147 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.147 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.147 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.147 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.147 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.147 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.148 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.148 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.148 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.148 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.148 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.148 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.148 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.148 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.148 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.148 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.148 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.148 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.148 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.149 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.149 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.149 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.149 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.149 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.149 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.149 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.149 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.149 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.149 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.149 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.149 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.149 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.150 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.150 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.150 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.150 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.150 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.150 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.150 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.150 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.150 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.150 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.150 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.150 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.150 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.151 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.151 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.151 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.151 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.151 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.151 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.151 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.151 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.151 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.151 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.151 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.151 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.151 INFO:teuthology.orchestra.run.smithi002.stdout: "storage_class": { 2024-05-14T17:15:46.151 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "storage_class", 2024-05-14T17:15:46.152 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.152 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.152 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.152 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "local", 2024-05-14T17:15:46.152 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.152 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.152 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.152 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-14T17:15:46.152 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.152 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.152 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.152 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.152 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.153 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "selftest", 2024-05-14T17:15:46.153 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.153 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.153 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.153 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.153 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.153 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.153 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.153 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.153 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.153 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.153 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.153 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.153 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.154 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.154 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.154 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.154 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.154 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.154 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.154 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.154 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.154 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.154 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.154 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.154 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.154 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.155 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.155 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.155 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.155 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.155 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.155 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.155 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.155 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.155 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.155 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.155 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.155 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.155 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.155 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.155 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.156 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.156 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.156 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.156 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.156 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.156 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.156 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.156 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.156 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.156 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.156 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.156 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.156 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.156 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.156 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.157 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.157 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.157 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.157 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.157 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.157 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.157 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.157 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.157 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.157 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.157 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.157 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.157 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.157 INFO:teuthology.orchestra.run.smithi002.stdout: "roption1": { 2024-05-14T17:15:46.158 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "roption1", 2024-05-14T17:15:46.158 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.158 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.158 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.158 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.158 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.158 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.158 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.158 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.158 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.158 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.158 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.158 INFO:teuthology.orchestra.run.smithi002.stdout: "roption2": { 2024-05-14T17:15:46.158 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "roption2", 2024-05-14T17:15:46.159 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.159 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.159 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.159 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "xyz", 2024-05-14T17:15:46.159 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.159 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.159 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.159 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.159 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.159 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.159 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.159 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption1": { 2024-05-14T17:15:46.159 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption1", 2024-05-14T17:15:46.159 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.159 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.160 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.160 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.160 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.160 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.160 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.160 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.160 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.160 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.160 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.160 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption2": { 2024-05-14T17:15:46.160 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption2", 2024-05-14T17:15:46.160 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.160 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.160 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.161 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.161 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.161 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.161 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.161 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.161 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.161 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.161 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.161 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption3": { 2024-05-14T17:15:46.161 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption3", 2024-05-14T17:15:46.161 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T17:15:46.161 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.161 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.161 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.161 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.162 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.162 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.162 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.162 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.162 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.162 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.162 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption4": { 2024-05-14T17:15:46.162 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption4", 2024-05-14T17:15:46.162 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.162 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.162 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.162 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.162 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.163 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.163 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.163 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.163 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.163 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.163 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.163 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption5": { 2024-05-14T17:15:46.163 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption5", 2024-05-14T17:15:46.163 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.163 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.163 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.163 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.163 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.163 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.163 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.164 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.164 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.164 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.164 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.164 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption6": { 2024-05-14T17:15:46.164 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption6", 2024-05-14T17:15:46.164 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.164 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.164 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.164 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.164 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.164 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.164 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.164 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.165 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.165 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.165 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.165 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption7": { 2024-05-14T17:15:46.165 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption7", 2024-05-14T17:15:46.165 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.165 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.165 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.165 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.165 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-05-14T17:15:46.165 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "42", 2024-05-14T17:15:46.165 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.165 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.166 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.166 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.166 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.166 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.166 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.166 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.166 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.166 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.166 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.166 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.166 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.166 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.166 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.167 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.167 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.167 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.167 INFO:teuthology.orchestra.run.smithi002.stdout: "testkey": { 2024-05-14T17:15:46.167 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testkey", 2024-05-14T17:15:46.167 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.167 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.167 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.167 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.167 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.167 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.167 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.167 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.167 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.168 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.168 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.168 INFO:teuthology.orchestra.run.smithi002.stdout: "testlkey": { 2024-05-14T17:15:46.168 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testlkey", 2024-05-14T17:15:46.168 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.168 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.168 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.168 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.168 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.168 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.168 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.168 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.168 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.168 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.169 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.169 INFO:teuthology.orchestra.run.smithi002.stdout: "testnewline": { 2024-05-14T17:15:46.169 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testnewline", 2024-05-14T17:15:46.169 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.169 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.169 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.169 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.169 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.169 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.169 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.169 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.169 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.169 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.169 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.169 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.170 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.170 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smb", 2024-05-14T17:15:46.170 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.170 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.170 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.170 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.170 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.170 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.170 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.170 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.170 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.170 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.170 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.171 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.171 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.171 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.171 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.171 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.171 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.171 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.171 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.171 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.171 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.171 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.171 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.171 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.171 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.172 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.172 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.172 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.172 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.172 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.172 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.172 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.172 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.172 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.172 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.172 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.172 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.172 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.172 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.173 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.173 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.173 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.173 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.173 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.173 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.173 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.173 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.173 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.173 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.173 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.173 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.173 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.174 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.174 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.174 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.174 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.174 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.174 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.174 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.174 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.174 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.174 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.174 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.174 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.174 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.175 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.175 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.175 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.175 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.175 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.175 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.175 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.175 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.175 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.175 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.175 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.175 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.175 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.176 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.176 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.176 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.176 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.176 INFO:teuthology.orchestra.run.smithi002.stdout: "update_orchestration": { 2024-05-14T17:15:46.176 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "update_orchestration", 2024-05-14T17:15:46.176 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.176 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.176 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.176 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.176 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.176 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.176 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.176 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-14T17:15:46.177 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.177 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.177 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.177 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.177 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.177 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snap_schedule", 2024-05-14T17:15:46.177 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.177 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.177 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.177 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_m_granularity": { 2024-05-14T17:15:46.177 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_m_granularity", 2024-05-14T17:15:46.177 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.178 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.178 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.178 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.178 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.178 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.178 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.178 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow minute scheduled snapshots", 2024-05-14T17:15:46.178 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.178 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.178 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.178 INFO:teuthology.orchestra.run.smithi002.stdout: "dump_on_update": { 2024-05-14T17:15:46.178 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "dump_on_update", 2024-05-14T17:15:46.178 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.179 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.179 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.179 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.179 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.179 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.179 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.179 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "dump database to debug log on update", 2024-05-14T17:15:46.179 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.179 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.179 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.179 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.179 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.179 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.179 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.180 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.180 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.180 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.180 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.180 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.180 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.180 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.180 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.180 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.180 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.180 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.180 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.180 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.180 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.181 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.181 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.181 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.181 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.181 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.181 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.181 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.181 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.181 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.181 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.181 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.181 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.181 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.181 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.182 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.182 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.182 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.182 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.182 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.182 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.182 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.182 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.182 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.182 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.182 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.182 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.182 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.183 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.183 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.183 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.183 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.183 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.183 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.183 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.183 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.183 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.183 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.183 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.183 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.183 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.184 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.184 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.184 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.184 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.184 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.184 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.184 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.184 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.184 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.184 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.184 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.184 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.184 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.184 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.185 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.185 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.185 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.185 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.185 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.185 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.185 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.185 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.185 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.185 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "stats", 2024-05-14T17:15:46.185 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.185 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.185 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.186 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.186 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.186 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.186 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.186 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.186 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.186 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.186 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.186 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.186 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.186 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.186 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.186 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.186 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.187 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.187 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.187 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.187 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.187 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.187 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.187 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.187 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.187 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.187 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.187 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.187 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.187 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.188 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.188 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.188 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.188 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.188 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.188 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.188 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.188 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.188 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.188 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.188 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.188 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.188 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.188 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.188 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.189 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.189 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.189 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.189 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.189 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.189 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.189 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.189 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.189 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.189 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.189 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.189 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.189 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.189 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.190 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.190 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.190 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.190 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.190 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.190 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.190 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.190 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.190 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.190 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.190 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.190 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.190 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.191 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.191 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.191 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.191 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.191 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.191 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.191 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.191 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.191 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.191 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.191 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.191 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.192 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "status", 2024-05-14T17:15:46.192 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.192 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.192 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.192 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.192 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.192 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.192 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.192 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.192 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.192 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.192 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.192 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.192 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.193 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.194 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.195 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.195 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.195 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.195 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.195 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.195 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.195 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.195 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.195 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.195 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.195 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.195 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.195 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.195 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.196 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.196 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.196 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.196 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.196 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.196 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.196 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.196 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.196 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.196 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.196 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.196 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.196 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.196 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.197 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.197 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.197 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.197 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.197 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.197 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.197 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.197 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.197 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.197 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.197 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.197 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.197 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.197 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.198 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.198 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.198 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "telegraf", 2024-05-14T17:15:46.198 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.198 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.198 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.198 INFO:teuthology.orchestra.run.smithi002.stdout: "address": { 2024-05-14T17:15:46.198 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "address", 2024-05-14T17:15:46.198 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.198 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.198 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.198 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-14T17:15:46.199 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.199 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.199 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.199 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.199 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.199 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.199 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.199 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-05-14T17:15:46.199 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-05-14T17:15:46.199 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.199 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.199 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.199 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15", 2024-05-14T17:15:46.199 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.200 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.200 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.200 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.200 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.200 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.200 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.200 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.200 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.200 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.200 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.200 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.200 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.200 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.201 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.201 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.201 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.201 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.201 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.201 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.201 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.201 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.201 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.201 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.201 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.201 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.201 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.201 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.202 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.202 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.202 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.202 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.202 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.202 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.202 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.202 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.202 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.202 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.202 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.202 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.202 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.203 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.203 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.203 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.203 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.203 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.203 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.203 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.203 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.203 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.203 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.203 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.203 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.203 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.203 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.203 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.204 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.204 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.204 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.204 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.204 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.204 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.204 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.204 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.204 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.204 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.204 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.204 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.204 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.204 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.204 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.205 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.205 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.205 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.205 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.205 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.205 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.205 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.205 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.205 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.205 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.205 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.205 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.205 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.205 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.205 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.206 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.206 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.206 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "telemetry", 2024-05-14T17:15:46.206 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.206 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.206 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.206 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_basic": { 2024-05-14T17:15:46.206 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_basic", 2024-05-14T17:15:46.206 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.206 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.206 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.206 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.206 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.207 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.207 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.207 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-14T17:15:46.207 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.207 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.207 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.207 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_crash": { 2024-05-14T17:15:46.207 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_crash", 2024-05-14T17:15:46.207 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.207 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.207 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.207 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.207 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.207 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.207 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.208 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-14T17:15:46.208 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.208 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.208 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.208 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_device": { 2024-05-14T17:15:46.208 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_device", 2024-05-14T17:15:46.208 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.208 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.208 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.208 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.208 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.208 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.209 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.209 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-14T17:15:46.209 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.209 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.209 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.209 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_ident": { 2024-05-14T17:15:46.209 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_ident", 2024-05-14T17:15:46.209 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.209 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.209 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.209 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.209 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.209 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.209 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.209 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-14T17:15:46.210 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.210 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.210 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.210 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_perf": { 2024-05-14T17:15:46.210 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_perf", 2024-05-14T17:15:46.210 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.210 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.210 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.210 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.210 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.210 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.210 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.210 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-14T17:15:46.210 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.211 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.211 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.211 INFO:teuthology.orchestra.run.smithi002.stdout: "contact": { 2024-05-14T17:15:46.211 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "contact", 2024-05-14T17:15:46.211 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.211 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.211 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.211 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.211 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.211 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.211 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.211 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.211 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.211 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.211 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.212 INFO:teuthology.orchestra.run.smithi002.stdout: "description": { 2024-05-14T17:15:46.212 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "description", 2024-05-14T17:15:46.212 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.212 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.212 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.212 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.212 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.212 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.212 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.212 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.212 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.212 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.212 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.213 INFO:teuthology.orchestra.run.smithi002.stdout: "device_url": { 2024-05-14T17:15:46.213 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_url", 2024-05-14T17:15:46.213 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.213 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.213 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.213 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-14T17:15:46.213 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.213 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.213 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.213 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.213 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.213 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.213 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.213 INFO:teuthology.orchestra.run.smithi002.stdout: "enabled": { 2024-05-14T17:15:46.214 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enabled", 2024-05-14T17:15:46.214 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.214 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.214 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.214 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.214 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.214 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.214 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.214 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.214 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.214 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.214 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.214 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-05-14T17:15:46.214 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-05-14T17:15:46.215 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.215 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.215 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.215 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "24", 2024-05-14T17:15:46.215 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "8", 2024-05-14T17:15:46.215 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.215 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.215 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.215 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.215 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.215 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.215 INFO:teuthology.orchestra.run.smithi002.stdout: "last_opt_revision": { 2024-05-14T17:15:46.215 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "last_opt_revision", 2024-05-14T17:15:46.215 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.215 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.216 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.216 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1", 2024-05-14T17:15:46.216 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.216 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.216 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.216 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.216 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.216 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.216 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.216 INFO:teuthology.orchestra.run.smithi002.stdout: "leaderboard": { 2024-05-14T17:15:46.216 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "leaderboard", 2024-05-14T17:15:46.216 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.216 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.216 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.217 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.217 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.217 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.217 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.217 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.217 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.217 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.217 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.217 INFO:teuthology.orchestra.run.smithi002.stdout: "leaderboard_description": { 2024-05-14T17:15:46.217 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "leaderboard_description", 2024-05-14T17:15:46.217 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.217 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.217 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.218 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.218 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.218 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.218 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.218 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.218 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.218 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.218 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.218 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.218 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.218 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.218 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.218 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.218 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.219 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.219 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.219 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.219 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.219 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.219 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.219 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.219 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.219 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.219 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.219 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.219 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.219 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.219 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.219 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.220 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.220 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.220 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.220 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.220 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.220 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.220 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.220 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.220 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.220 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.220 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.220 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.220 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.220 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.221 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.221 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.221 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.221 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.221 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.221 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.221 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.221 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.221 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.221 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.221 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.221 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.221 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.221 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.221 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.222 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.222 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.222 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.222 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.222 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.222 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.222 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.222 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.222 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.222 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.222 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.222 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.222 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.223 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.223 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.223 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.223 INFO:teuthology.orchestra.run.smithi002.stdout: "organization": { 2024-05-14T17:15:46.223 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "organization", 2024-05-14T17:15:46.223 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.223 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.223 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.223 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.223 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.223 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.223 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.223 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.223 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.224 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.224 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.224 INFO:teuthology.orchestra.run.smithi002.stdout: "proxy": { 2024-05-14T17:15:46.224 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "proxy", 2024-05-14T17:15:46.224 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.224 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.224 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.224 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.224 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.224 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.224 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.224 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.224 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.224 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.224 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.225 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.225 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.225 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.225 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.225 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.225 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.225 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.225 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.225 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.225 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.225 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.225 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.226 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.226 INFO:teuthology.orchestra.run.smithi002.stdout: "url": { 2024-05-14T17:15:46.226 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "url", 2024-05-14T17:15:46.226 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.226 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.226 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.226 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-14T17:15:46.226 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.226 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.226 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.226 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.226 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.226 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.227 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.227 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.227 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.227 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "test_orchestrator", 2024-05-14T17:15:46.227 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.227 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.227 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.227 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.227 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.227 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.227 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.227 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.227 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.228 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.228 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.228 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.228 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.228 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.228 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.228 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.228 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.228 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.228 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.228 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.228 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.228 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.228 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.229 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.229 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.229 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.229 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.229 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.229 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.229 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.229 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.229 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.229 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.229 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.229 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.229 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.230 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.230 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.230 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.230 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.230 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.230 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.230 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.230 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.230 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.230 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.230 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.230 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.230 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.231 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.231 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.231 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.231 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.231 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.231 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.231 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.231 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.231 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.231 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.231 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.231 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.231 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.231 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.232 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.232 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.232 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.232 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.232 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.232 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.232 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.232 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.232 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.232 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.232 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.232 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.232 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.232 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.233 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.233 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.233 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.233 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.233 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.233 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.233 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.233 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "volumes", 2024-05-14T17:15:46.233 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.233 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.233 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.233 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.234 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.234 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.234 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.234 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.234 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.234 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.234 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.234 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.234 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.234 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.234 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.234 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.234 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.234 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.235 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.235 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.235 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.235 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.235 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.235 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.235 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.235 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.235 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.235 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.235 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.235 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.235 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.236 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.236 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.236 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.236 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.236 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.236 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.236 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.236 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.236 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.236 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.236 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.236 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.236 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.236 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.237 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.237 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.237 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.237 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.237 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.237 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.237 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.237 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.237 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.237 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.237 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.237 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.237 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.237 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.237 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.238 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.238 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.238 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.238 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.238 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.238 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.238 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.238 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.238 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.238 INFO:teuthology.orchestra.run.smithi002.stdout: "max_concurrent_clones": { 2024-05-14T17:15:46.238 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_concurrent_clones", 2024-05-14T17:15:46.238 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.238 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.239 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.239 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "4", 2024-05-14T17:15:46.239 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.239 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.239 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.239 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-14T17:15:46.239 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.239 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.239 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.239 INFO:teuthology.orchestra.run.smithi002.stdout: "periodic_async_work": { 2024-05-14T17:15:46.239 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "periodic_async_work", 2024-05-14T17:15:46.239 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.239 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.240 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.240 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.240 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.240 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.240 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.240 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Periodically check for async work", 2024-05-14T17:15:46.240 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.240 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.240 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.240 INFO:teuthology.orchestra.run.smithi002.stdout: "snapshot_clone_delay": { 2024-05-14T17:15:46.240 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snapshot_clone_delay", 2024-05-14T17:15:46.240 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.240 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.240 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.240 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.241 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.241 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.241 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.241 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-14T17:15:46.241 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.241 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.241 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.241 INFO:teuthology.orchestra.run.smithi002.stdout: "snapshot_clone_no_wait": { 2024-05-14T17:15:46.241 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snapshot_clone_no_wait", 2024-05-14T17:15:46.241 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.241 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.241 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.241 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.242 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.242 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.242 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.242 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-14T17:15:46.242 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.242 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.242 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.242 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.242 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.242 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.242 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.242 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.242 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.243 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.243 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.243 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.243 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.243 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.243 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.243 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.243 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.243 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.243 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix", 2024-05-14T17:15:46.243 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.243 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.243 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.244 INFO:teuthology.orchestra.run.smithi002.stdout: "discovery_interval": { 2024-05-14T17:15:46.244 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "discovery_interval", 2024-05-14T17:15:46.244 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-05-14T17:15:46.244 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.244 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.244 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "100", 2024-05-14T17:15:46.244 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.244 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.244 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.244 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.244 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.244 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.244 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.245 INFO:teuthology.orchestra.run.smithi002.stdout: "identifier": { 2024-05-14T17:15:46.245 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "identifier", 2024-05-14T17:15:46.245 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.245 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.245 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.245 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.245 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.245 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.245 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.245 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.245 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.245 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.245 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.246 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-05-14T17:15:46.246 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-05-14T17:15:46.246 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.246 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.246 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.246 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-05-14T17:15:46.246 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.246 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.246 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.246 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.246 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.246 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.246 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.247 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.247 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.247 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.247 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.247 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.247 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.247 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.247 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.247 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.247 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.247 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.247 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.247 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.247 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.248 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.248 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.248 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.248 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.248 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.248 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.248 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.248 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.248 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.248 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.248 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.248 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.248 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.248 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.249 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.249 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.249 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.249 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.249 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.249 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.249 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.249 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.249 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.249 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.249 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.249 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.249 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.250 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.250 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.250 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.250 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.250 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.250 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.250 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.250 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.250 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.250 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.250 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.250 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.250 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.250 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.251 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.251 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.251 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.251 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.251 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.251 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.251 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.251 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.251 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.251 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.251 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.251 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.251 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.251 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.252 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.252 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.252 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.252 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.252 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.252 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.252 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.252 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.252 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.252 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.252 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_host": { 2024-05-14T17:15:46.252 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_host", 2024-05-14T17:15:46.252 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.253 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.253 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.253 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.253 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.253 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.253 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.253 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.253 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.253 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.253 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.253 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_port": { 2024-05-14T17:15:46.253 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_port", 2024-05-14T17:15:46.253 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.254 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.254 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.254 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10051", 2024-05-14T17:15:46.254 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.254 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.254 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.254 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.254 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.254 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.254 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.254 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_sender": { 2024-05-14T17:15:46.255 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_sender", 2024-05-14T17:15:46.255 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.255 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.255 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.255 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-14T17:15:46.255 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.255 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.255 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.255 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.255 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.255 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.255 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.255 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.255 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.255 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.255 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-05-14T17:15:46.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.256 INFO:teuthology.orchestra.run.smithi002.stdout: "gid": 24875, 2024-05-14T17:15:46.256 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "y", 2024-05-14T17:15:46.256 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_features": 4540701547738038271, 2024-05-14T17:15:46.256 INFO:teuthology.orchestra.run.smithi002.stdout: "available_modules": [ 2024-05-14T17:15:46.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.256 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "alerts", 2024-05-14T17:15:46.256 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.256 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.256 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.256 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-05-14T17:15:46.256 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-05-14T17:15:46.256 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.256 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.257 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.257 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-05-14T17:15:46.257 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.257 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.257 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.257 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How frequently to reexamine health status", 2024-05-14T17:15:46.257 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.257 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.257 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.257 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.257 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.257 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.257 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.258 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.258 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.258 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.258 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.258 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.258 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.258 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.258 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.258 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.258 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.258 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.258 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.258 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.258 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.258 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.259 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.259 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.259 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.259 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.259 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.259 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.259 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.259 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.259 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.259 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.259 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.259 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.259 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.259 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.260 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.260 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.260 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.260 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.260 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.260 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.260 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.260 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.260 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.260 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.260 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.260 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.260 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.261 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.261 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.261 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.261 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.261 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.261 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.261 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.261 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.261 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.261 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.261 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.261 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.261 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.261 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.262 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.262 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.262 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.262 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.262 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.262 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.262 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_destination": { 2024-05-14T17:15:46.262 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_destination", 2024-05-14T17:15:46.262 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.262 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.262 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.262 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.262 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.263 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.263 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.263 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Email address to send alerts to", 2024-05-14T17:15:46.263 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.263 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.263 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.263 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_from_name": { 2024-05-14T17:15:46.263 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_from_name", 2024-05-14T17:15:46.263 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.263 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.263 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.263 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "Ceph", 2024-05-14T17:15:46.263 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.263 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.264 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.264 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Email From: name", 2024-05-14T17:15:46.264 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.264 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.264 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.264 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_host": { 2024-05-14T17:15:46.264 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_host", 2024-05-14T17:15:46.264 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.264 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.264 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.264 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.264 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.264 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.264 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.265 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP server", 2024-05-14T17:15:46.265 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.265 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.265 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.265 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_password": { 2024-05-14T17:15:46.265 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_password", 2024-05-14T17:15:46.265 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.265 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.265 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.265 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.265 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.265 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.266 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.266 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Password to authenticate with", 2024-05-14T17:15:46.266 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.266 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.266 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.266 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_port": { 2024-05-14T17:15:46.266 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_port", 2024-05-14T17:15:46.266 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.266 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.266 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.266 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "465", 2024-05-14T17:15:46.266 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.266 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.267 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.267 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP port", 2024-05-14T17:15:46.267 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.267 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.267 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.267 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_sender": { 2024-05-14T17:15:46.267 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_sender", 2024-05-14T17:15:46.267 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.267 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.267 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.267 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.267 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.267 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.268 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.268 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP envelope sender", 2024-05-14T17:15:46.268 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.268 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.268 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.268 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_ssl": { 2024-05-14T17:15:46.268 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_ssl", 2024-05-14T17:15:46.268 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.268 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.268 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.268 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.268 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.268 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.269 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.269 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-14T17:15:46.269 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.269 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.269 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.269 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_user": { 2024-05-14T17:15:46.269 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_user", 2024-05-14T17:15:46.269 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.269 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.269 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.269 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.269 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.269 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.270 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.270 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "User to authenticate as", 2024-05-14T17:15:46.270 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.270 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.270 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.270 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.270 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.270 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.270 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.270 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.270 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.270 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.271 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.271 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.271 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.271 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.271 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.271 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.271 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.271 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.271 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "balancer", 2024-05-14T17:15:46.271 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.271 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.271 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.272 INFO:teuthology.orchestra.run.smithi002.stdout: "active": { 2024-05-14T17:15:46.272 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "active", 2024-05-14T17:15:46.272 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.272 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.272 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.272 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.272 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.272 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.272 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.272 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically balance PGs across cluster", 2024-05-14T17:15:46.272 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.272 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.272 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.273 INFO:teuthology.orchestra.run.smithi002.stdout: "begin_time": { 2024-05-14T17:15:46.273 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "begin_time", 2024-05-14T17:15:46.273 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.273 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.273 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.273 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0000", 2024-05-14T17:15:46.273 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.273 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.273 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.273 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "beginning time of day to automatically balance", 2024-05-14T17:15:46.273 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-14T17:15:46.273 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.273 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.273 INFO:teuthology.orchestra.run.smithi002.stdout: "begin_weekday": { 2024-05-14T17:15:46.274 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "begin_weekday", 2024-05-14T17:15:46.274 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-05-14T17:15:46.274 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.274 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.274 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.274 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0", 2024-05-14T17:15:46.274 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "6", 2024-05-14T17:15:46.274 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.274 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-14T17:15:46.274 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-14T17:15:46.274 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.274 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.274 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_max_iterations": { 2024-05-14T17:15:46.275 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_max_iterations", 2024-05-14T17:15:46.275 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-05-14T17:15:46.275 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.275 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.275 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "25", 2024-05-14T17:15:46.275 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-05-14T17:15:46.275 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "250", 2024-05-14T17:15:46.275 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.275 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-14T17:15:46.275 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.275 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.275 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.275 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_metrics": { 2024-05-14T17:15:46.276 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_metrics", 2024-05-14T17:15:46.276 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.276 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.276 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.276 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "pgs,objects,bytes", 2024-05-14T17:15:46.276 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.276 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.276 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.276 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-14T17:15:46.276 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-14T17:15:46.276 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.276 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.276 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_step": { 2024-05-14T17:15:46.277 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_step", 2024-05-14T17:15:46.277 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T17:15:46.277 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.277 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.277 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0.5", 2024-05-14T17:15:46.277 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0.001", 2024-05-14T17:15:46.277 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "0.999", 2024-05-14T17:15:46.277 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.277 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "aggressiveness of optimization", 2024-05-14T17:15:46.277 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-14T17:15:46.277 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.277 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.277 INFO:teuthology.orchestra.run.smithi002.stdout: "end_time": { 2024-05-14T17:15:46.277 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "end_time", 2024-05-14T17:15:46.278 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.278 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.278 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.278 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "2359", 2024-05-14T17:15:46.278 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.278 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.278 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.278 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "ending time of day to automatically balance", 2024-05-14T17:15:46.278 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-14T17:15:46.278 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.278 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.279 INFO:teuthology.orchestra.run.smithi002.stdout: "end_weekday": { 2024-05-14T17:15:46.279 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "end_weekday", 2024-05-14T17:15:46.279 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-05-14T17:15:46.279 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.279 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.279 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.279 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0", 2024-05-14T17:15:46.279 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "6", 2024-05-14T17:15:46.279 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.279 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-14T17:15:46.279 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-14T17:15:46.279 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.279 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.280 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.280 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.280 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.280 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.280 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.280 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.280 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.280 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.280 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.280 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.280 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.280 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.280 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.280 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.280 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.281 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.281 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.281 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.281 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.281 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.281 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.281 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.281 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.281 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.281 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.281 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.281 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.281 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.282 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.282 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.282 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.282 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.282 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.282 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.282 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.282 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.282 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.282 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.282 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.282 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.283 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.283 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.283 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.283 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.283 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.283 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.283 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.283 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.283 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.283 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.283 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.283 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.283 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.284 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.284 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.284 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.284 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.284 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.284 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.284 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.284 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.284 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.284 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.284 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.284 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.284 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.285 INFO:teuthology.orchestra.run.smithi002.stdout: "min_score": { 2024-05-14T17:15:46.285 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "min_score", 2024-05-14T17:15:46.285 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T17:15:46.285 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.285 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.285 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.285 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.285 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.285 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.285 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-14T17:15:46.285 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.285 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.285 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.286 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": { 2024-05-14T17:15:46.286 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mode", 2024-05-14T17:15:46.286 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.286 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.286 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.286 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "upmap", 2024-05-14T17:15:46.286 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.286 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.286 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.286 INFO:teuthology.orchestra.run.smithi002.stdout: "crush-compat", 2024-05-14T17:15:46.286 INFO:teuthology.orchestra.run.smithi002.stdout: "none", 2024-05-14T17:15:46.286 INFO:teuthology.orchestra.run.smithi002.stdout: "read", 2024-05-14T17:15:46.286 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap", 2024-05-14T17:15:46.287 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap-read" 2024-05-14T17:15:46.287 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.287 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Balancer mode", 2024-05-14T17:15:46.287 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.287 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.287 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.287 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_ids": { 2024-05-14T17:15:46.287 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pool_ids", 2024-05-14T17:15:46.287 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.287 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.287 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.287 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.287 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.287 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.288 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.288 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-14T17:15:46.288 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.288 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.288 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.288 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-05-14T17:15:46.288 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-05-14T17:15:46.288 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.288 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.288 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.288 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-05-14T17:15:46.288 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.288 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.288 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.289 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-14T17:15:46.289 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.289 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.289 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.289 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.289 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.289 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.289 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.289 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.289 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.289 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.289 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.289 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.290 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.290 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.290 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.290 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.290 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap_max_deviation": { 2024-05-14T17:15:46.290 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "upmap_max_deviation", 2024-05-14T17:15:46.290 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.290 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.290 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.290 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-05-14T17:15:46.290 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-05-14T17:15:46.290 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.290 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.290 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-14T17:15:46.291 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-14T17:15:46.291 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.291 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.291 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap_max_optimizations": { 2024-05-14T17:15:46.291 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "upmap_max_optimizations", 2024-05-14T17:15:46.291 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-05-14T17:15:46.291 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.291 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.291 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T17:15:46.291 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.291 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.291 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.292 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-14T17:15:46.292 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.292 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.292 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.292 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.292 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.292 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cephadm", 2024-05-14T17:15:46.292 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.292 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.292 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.292 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_down_multiplier": { 2024-05-14T17:15:46.293 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_down_multiplier", 2024-05-14T17:15:46.293 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T17:15:46.293 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.293 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.293 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3.0", 2024-05-14T17:15:46.293 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.293 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.293 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.293 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-14T17:15:46.293 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.293 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.293 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.293 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_refresh_rate": { 2024-05-14T17:15:46.294 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_refresh_rate", 2024-05-14T17:15:46.294 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.294 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.294 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.294 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "20", 2024-05-14T17:15:46.294 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.294 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.294 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.294 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-14T17:15:46.294 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.294 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.294 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.294 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_starting_port": { 2024-05-14T17:15:46.295 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_starting_port", 2024-05-14T17:15:46.295 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.295 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.295 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.295 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "4721", 2024-05-14T17:15:46.295 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.295 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.295 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.295 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-14T17:15:46.295 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.295 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.295 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.296 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_ptrace": { 2024-05-14T17:15:46.296 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_ptrace", 2024-05-14T17:15:46.296 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.296 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.296 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.296 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.296 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.296 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.296 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.296 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-14T17:15:46.296 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-14T17:15:46.296 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.296 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.296 INFO:teuthology.orchestra.run.smithi002.stdout: "autotune_interval": { 2024-05-14T17:15:46.297 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "autotune_interval", 2024-05-14T17:15:46.297 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.297 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.297 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.297 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-05-14T17:15:46.297 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.297 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.297 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.297 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-14T17:15:46.297 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.297 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.297 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.297 INFO:teuthology.orchestra.run.smithi002.stdout: "autotune_memory_target_ratio": { 2024-05-14T17:15:46.297 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "autotune_memory_target_ratio", 2024-05-14T17:15:46.298 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T17:15:46.298 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.298 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.298 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0.7", 2024-05-14T17:15:46.298 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.298 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.298 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.298 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-14T17:15:46.298 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.298 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.298 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.298 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm_log_destination": { 2024-05-14T17:15:46.298 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cephadm_log_destination", 2024-05-14T17:15:46.299 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.299 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.299 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.299 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.299 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.299 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.299 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.299 INFO:teuthology.orchestra.run.smithi002.stdout: "file", 2024-05-14T17:15:46.299 INFO:teuthology.orchestra.run.smithi002.stdout: "file,syslog", 2024-05-14T17:15:46.299 INFO:teuthology.orchestra.run.smithi002.stdout: "syslog" 2024-05-14T17:15:46.299 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.299 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-14T17:15:46.299 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.299 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.299 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: "cgroups_split": { 2024-05-14T17:15:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cgroups_split", 2024-05-14T17:15:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-14T17:15:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.301 INFO:teuthology.orchestra.run.smithi002.stdout: "config_checks_enabled": { 2024-05-14T17:15:46.301 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_checks_enabled", 2024-05-14T17:15:46.301 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.301 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.301 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.301 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.301 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.301 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.301 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.301 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-14T17:15:46.301 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.301 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.301 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.302 INFO:teuthology.orchestra.run.smithi002.stdout: "config_dashboard": { 2024-05-14T17:15:46.302 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_dashboard", 2024-05-14T17:15:46.302 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.302 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.302 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.302 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.302 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.302 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.302 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.302 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-14T17:15:46.302 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.302 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.302 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.303 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_alertmanager": { 2024-05-14T17:15:46.303 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_alertmanager", 2024-05-14T17:15:46.303 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.303 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.303 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.303 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-14T17:15:46.303 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.303 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.303 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.303 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-05-14T17:15:46.303 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.304 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.304 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.304 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_base": { 2024-05-14T17:15:46.304 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_base", 2024-05-14T17:15:46.304 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.304 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.304 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.304 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-14T17:15:46.304 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.304 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.304 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.304 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Container image name, without the tag", 2024-05-14T17:15:46.305 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.305 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.305 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.305 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_elasticsearch": { 2024-05-14T17:15:46.305 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_elasticsearch", 2024-05-14T17:15:46.305 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.305 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.305 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.305 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-14T17:15:46.305 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.305 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.305 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.305 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "elasticsearch container image", 2024-05-14T17:15:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_grafana": { 2024-05-14T17:15:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_grafana", 2024-05-14T17:15:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-14T17:15:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.306 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.307 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-05-14T17:15:46.307 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.307 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.307 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.307 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_haproxy": { 2024-05-14T17:15:46.307 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_haproxy", 2024-05-14T17:15:46.307 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.307 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.307 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.307 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-14T17:15:46.307 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.307 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.307 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.308 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "HAproxy container image", 2024-05-14T17:15:46.308 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.308 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.308 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.308 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_agent": { 2024-05-14T17:15:46.308 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_agent", 2024-05-14T17:15:46.308 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.308 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.308 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.308 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-14T17:15:46.308 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.308 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.308 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger agent container image", 2024-05-14T17:15:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_collector": { 2024-05-14T17:15:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_collector", 2024-05-14T17:15:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-14T17:15:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.309 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger collector container image", 2024-05-14T17:15:46.310 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.310 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.310 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.310 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_query": { 2024-05-14T17:15:46.310 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_query", 2024-05-14T17:15:46.310 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.310 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.310 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.310 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-14T17:15:46.310 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.310 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.310 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.310 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger query container image", 2024-05-14T17:15:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_keepalived": { 2024-05-14T17:15:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_keepalived", 2024-05-14T17:15:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-14T17:15:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.311 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Keepalived container image", 2024-05-14T17:15:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_loki": { 2024-05-14T17:15:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_loki", 2024-05-14T17:15:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-14T17:15:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Loki container image", 2024-05-14T17:15:46.312 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.313 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.313 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.313 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_node_exporter": { 2024-05-14T17:15:46.313 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_node_exporter", 2024-05-14T17:15:46.313 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.313 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.313 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.313 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-14T17:15:46.313 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.313 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.313 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.313 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-05-14T17:15:46.313 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.314 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.314 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.314 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_nvmeof": { 2024-05-14T17:15:46.314 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_nvmeof", 2024-05-14T17:15:46.314 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.314 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.314 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.314 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-14T17:15:46.314 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.314 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.314 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.314 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Nvme-of container image", 2024-05-14T17:15:46.314 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.315 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.315 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.315 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_prometheus": { 2024-05-14T17:15:46.315 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_prometheus", 2024-05-14T17:15:46.315 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.315 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.315 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.315 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-14T17:15:46.315 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.315 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.315 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.315 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-05-14T17:15:46.315 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_promtail": { 2024-05-14T17:15:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_promtail", 2024-05-14T17:15:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-14T17:15:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Promtail container image", 2024-05-14T17:15:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.316 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_samba": { 2024-05-14T17:15:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_samba", 2024-05-14T17:15:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-14T17:15:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Samba/SMB container image", 2024-05-14T17:15:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_snmp_gateway": { 2024-05-14T17:15:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_snmp_gateway", 2024-05-14T17:15:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-14T17:15:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SNMP Gateway container image", 2024-05-14T17:15:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.318 INFO:teuthology.orchestra.run.smithi002.stdout: "container_init": { 2024-05-14T17:15:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_init", 2024-05-14T17:15:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Run podman/docker with `--init`", 2024-05-14T17:15:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: "daemon_cache_timeout": { 2024-05-14T17:15:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "daemon_cache_timeout", 2024-05-14T17:15:46.319 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-05-14T17:15:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-14T17:15:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: "default_cephadm_command_timeout": { 2024-05-14T17:15:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "default_cephadm_command_timeout", 2024-05-14T17:15:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.320 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "900", 2024-05-14T17:15:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-14T17:15:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: "default_registry": { 2024-05-14T17:15:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "default_registry", 2024-05-14T17:15:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.321 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io", 2024-05-14T17:15:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-14T17:15:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: "device_cache_timeout": { 2024-05-14T17:15:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_cache_timeout", 2024-05-14T17:15:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.322 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1800", 2024-05-14T17:15:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache device inventory", 2024-05-14T17:15:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: "device_enhanced_scan": { 2024-05-14T17:15:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_enhanced_scan", 2024-05-14T17:15:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.323 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-14T17:15:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: "facts_cache_timeout": { 2024-05-14T17:15:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "facts_cache_timeout", 2024-05-14T17:15:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-05-14T17:15:46.324 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache host facts data", 2024-05-14T17:15:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: "grafana_dashboards_path": { 2024-05-14T17:15:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "grafana_dashboards_path", 2024-05-14T17:15:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-14T17:15:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.325 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.326 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-14T17:15:46.326 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.326 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.326 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.326 INFO:teuthology.orchestra.run.smithi002.stdout: "host_check_interval": { 2024-05-14T17:15:46.326 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "host_check_interval", 2024-05-14T17:15:46.326 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.326 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.326 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.326 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-05-14T17:15:46.326 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to perform a host check", 2024-05-14T17:15:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: "hw_monitoring": { 2024-05-14T17:15:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "hw_monitoring", 2024-05-14T17:15:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.327 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-14T17:15:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: "inventory_list_all": { 2024-05-14T17:15:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "inventory_list_all", 2024-05-14T17:15:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-14T17:15:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.328 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.329 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.330 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.330 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.330 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.330 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.330 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.330 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.330 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.330 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.330 INFO:teuthology.orchestra.run.smithi002.stdout: "log_refresh_metadata": { 2024-05-14T17:15:46.330 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_refresh_metadata", 2024-05-14T17:15:46.330 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.330 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.330 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.331 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.331 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.331 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.331 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.331 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-14T17:15:46.331 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.331 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.331 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.331 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.331 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.331 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.331 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.331 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.331 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.331 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.331 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.332 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.332 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-14T17:15:46.332 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.332 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.332 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.332 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.332 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.332 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.332 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.332 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.332 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.332 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.332 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.332 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.332 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.333 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.333 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.333 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.333 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.333 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.333 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.333 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.333 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.333 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.333 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.333 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.333 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.333 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.333 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.333 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.333 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.334 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.334 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.334 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.334 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.334 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.334 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.334 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.334 INFO:teuthology.orchestra.run.smithi002.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-14T17:15:46.334 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-14T17:15:46.334 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.334 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.334 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.334 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.334 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.335 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.335 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.335 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-14T17:15:46.335 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.335 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.335 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.335 INFO:teuthology.orchestra.run.smithi002.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-14T17:15:46.335 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-14T17:15:46.335 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.335 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.335 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.335 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "*", 2024-05-14T17:15:46.335 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.335 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.336 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.336 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-14T17:15:46.336 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.336 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.336 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.336 INFO:teuthology.orchestra.run.smithi002.stdout: "max_count_per_host": { 2024-05-14T17:15:46.336 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_count_per_host", 2024-05-14T17:15:46.336 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.336 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.336 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.336 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T17:15:46.336 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.336 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.336 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.337 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "max number of daemons per service per host", 2024-05-14T17:15:46.337 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.337 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.337 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.337 INFO:teuthology.orchestra.run.smithi002.stdout: "max_osd_draining_count": { 2024-05-14T17:15:46.337 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_osd_draining_count", 2024-05-14T17:15:46.337 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.337 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.337 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.337 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T17:15:46.337 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.337 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.337 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.337 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-14T17:15:46.338 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.338 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.338 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.338 INFO:teuthology.orchestra.run.smithi002.stdout: "migration_current": { 2024-05-14T17:15:46.338 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "migration_current", 2024-05-14T17:15:46.338 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.338 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.338 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.338 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.338 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.338 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.338 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.338 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "internal - do not modify", 2024-05-14T17:15:46.338 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.339 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.339 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.339 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": { 2024-05-14T17:15:46.339 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mode", 2024-05-14T17:15:46.339 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.339 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.339 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.339 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "root", 2024-05-14T17:15:46.339 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.339 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.339 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.339 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm-package", 2024-05-14T17:15:46.339 INFO:teuthology.orchestra.run.smithi002.stdout: "root" 2024-05-14T17:15:46.339 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.340 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "mode for remote execution of cephadm", 2024-05-14T17:15:46.340 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.340 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.340 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.340 INFO:teuthology.orchestra.run.smithi002.stdout: "oob_default_addr": { 2024-05-14T17:15:46.340 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "oob_default_addr", 2024-05-14T17:15:46.340 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.340 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.340 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.340 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "169.254.1.1", 2024-05-14T17:15:46.340 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.340 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.340 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.340 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-14T17:15:46.340 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.341 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.341 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.341 INFO:teuthology.orchestra.run.smithi002.stdout: "prometheus_alerts_path": { 2024-05-14T17:15:46.341 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus_alerts_path", 2024-05-14T17:15:46.341 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.341 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.341 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.341 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-14T17:15:46.341 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.341 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.341 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.341 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-14T17:15:46.341 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.341 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.341 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.342 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_insecure": { 2024-05-14T17:15:46.342 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_insecure", 2024-05-14T17:15:46.342 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.342 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.342 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.342 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.342 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.342 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.342 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.342 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-14T17:15:46.342 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.342 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.343 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.343 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_password": { 2024-05-14T17:15:46.343 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_password", 2024-05-14T17:15:46.343 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.343 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.343 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.343 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.343 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.343 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.343 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.343 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-14T17:15:46.343 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.343 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.343 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.344 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_url": { 2024-05-14T17:15:46.344 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_url", 2024-05-14T17:15:46.344 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.344 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.344 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.344 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.344 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.344 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.344 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.344 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-14T17:15:46.344 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.344 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.344 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.345 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_username": { 2024-05-14T17:15:46.345 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_username", 2024-05-14T17:15:46.345 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.345 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.345 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.345 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.345 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.345 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.345 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.345 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-14T17:15:46.345 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.345 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.345 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.345 INFO:teuthology.orchestra.run.smithi002.stdout: "secure_monitoring_stack": { 2024-05-14T17:15:46.346 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "secure_monitoring_stack", 2024-05-14T17:15:46.346 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.346 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.346 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.346 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.346 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.346 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.346 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.346 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-14T17:15:46.346 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.346 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.346 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.347 INFO:teuthology.orchestra.run.smithi002.stdout: "service_discovery_port": { 2024-05-14T17:15:46.347 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "service_discovery_port", 2024-05-14T17:15:46.347 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.347 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.347 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.347 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8765", 2024-05-14T17:15:46.347 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.347 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.347 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.347 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "cephadm service discovery port", 2024-05-14T17:15:46.347 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.347 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.347 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.347 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.348 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.348 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.348 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.348 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.348 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.348 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.348 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.348 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.348 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.348 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.348 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.348 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.348 INFO:teuthology.orchestra.run.smithi002.stdout: "ssh_config_file": { 2024-05-14T17:15:46.349 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssh_config_file", 2024-05-14T17:15:46.349 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.349 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.349 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.349 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.349 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.349 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.349 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.349 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-14T17:15:46.349 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.349 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.349 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.349 INFO:teuthology.orchestra.run.smithi002.stdout: "use_agent": { 2024-05-14T17:15:46.349 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "use_agent", 2024-05-14T17:15:46.350 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.350 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.350 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.350 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.350 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.350 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.350 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.350 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-14T17:15:46.350 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.350 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.350 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.350 INFO:teuthology.orchestra.run.smithi002.stdout: "use_repo_digest": { 2024-05-14T17:15:46.350 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "use_repo_digest", 2024-05-14T17:15:46.351 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.351 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.351 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.351 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.351 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.351 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.351 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.351 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-14T17:15:46.351 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.351 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.351 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.351 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_failed_host_check": { 2024-05-14T17:15:46.351 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_failed_host_check", 2024-05-14T17:15:46.351 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.352 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.352 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.352 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.352 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.352 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.352 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.352 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if the host check fails", 2024-05-14T17:15:46.352 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.352 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.352 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.352 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_stray_daemons": { 2024-05-14T17:15:46.352 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_stray_daemons", 2024-05-14T17:15:46.352 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.353 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.353 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.353 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.353 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.353 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.353 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.353 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-14T17:15:46.353 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.353 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.353 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.353 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_stray_hosts": { 2024-05-14T17:15:46.353 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_stray_hosts", 2024-05-14T17:15:46.353 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.353 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.354 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.354 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.354 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.354 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.354 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.354 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-14T17:15:46.354 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.354 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.354 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.354 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.354 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.354 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crash", 2024-05-14T17:15:46.355 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.355 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.355 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.355 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.355 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.355 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.355 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.355 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.355 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.355 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.355 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.355 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.355 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.355 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.356 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.356 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.356 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.356 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.356 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.356 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.356 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.356 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.356 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.356 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.356 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.356 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.356 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.357 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.357 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.357 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.357 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.357 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.357 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.357 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.357 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.357 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.357 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.357 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.357 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.357 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.357 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.358 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.358 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.358 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.358 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.358 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.358 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.358 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.358 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.358 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.358 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.358 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.358 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.358 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.358 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.358 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.359 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.359 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.359 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.359 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.359 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.359 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.359 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.359 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.359 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.359 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.359 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.359 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.359 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.360 INFO:teuthology.orchestra.run.smithi002.stdout: "retain_interval": { 2024-05-14T17:15:46.360 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "retain_interval", 2024-05-14T17:15:46.360 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.360 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.360 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.360 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "31536000", 2024-05-14T17:15:46.360 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.360 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.360 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.360 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-14T17:15:46.360 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.360 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.360 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.361 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.361 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.361 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.361 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.361 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.361 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.361 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.361 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.361 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.361 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.361 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.361 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.361 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.361 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_recent_interval": { 2024-05-14T17:15:46.362 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_recent_interval", 2024-05-14T17:15:46.362 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.362 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.362 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.362 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1209600", 2024-05-14T17:15:46.362 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.362 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.362 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.362 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-14T17:15:46.362 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.362 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.362 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.362 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.362 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.363 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "dashboard", 2024-05-14T17:15:46.363 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.363 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.363 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.363 INFO:teuthology.orchestra.run.smithi002.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-14T17:15:46.363 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-14T17:15:46.363 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.363 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.363 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.363 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T17:15:46.363 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.363 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.364 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.364 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.364 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.364 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.364 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.364 INFO:teuthology.orchestra.run.smithi002.stdout: "ALERTMANAGER_API_HOST": { 2024-05-14T17:15:46.364 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-14T17:15:46.364 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.364 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.364 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.364 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.364 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.364 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.364 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.365 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.365 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.365 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.365 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.365 INFO:teuthology.orchestra.run.smithi002.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-14T17:15:46.365 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-14T17:15:46.365 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.365 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.365 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.365 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.365 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.365 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.365 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.366 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.366 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.366 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.366 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.366 INFO:teuthology.orchestra.run.smithi002.stdout: "AUDIT_API_ENABLED": { 2024-05-14T17:15:46.366 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "AUDIT_API_ENABLED", 2024-05-14T17:15:46.366 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.366 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.366 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.366 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.366 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.366 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.366 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.367 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.367 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.367 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.367 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.367 INFO:teuthology.orchestra.run.smithi002.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-14T17:15:46.367 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-14T17:15:46.367 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.367 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.367 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.367 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.367 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.367 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.367 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.368 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.368 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.368 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.368 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.368 INFO:teuthology.orchestra.run.smithi002.stdout: "ENABLE_BROWSABLE_API": { 2024-05-14T17:15:46.368 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-14T17:15:46.368 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.368 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.368 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.368 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.368 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.368 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.369 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.369 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.369 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.369 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.369 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.369 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-14T17:15:46.369 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-14T17:15:46.369 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.369 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.369 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.369 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.369 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.369 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.370 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.370 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.370 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.370 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.370 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.370 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-14T17:15:46.370 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-14T17:15:46.370 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.370 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.370 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.370 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.370 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.370 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.371 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.371 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.371 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.371 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.371 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.371 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-14T17:15:46.371 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-14T17:15:46.371 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.371 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.371 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.371 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.371 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.371 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.371 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.372 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.372 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.372 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.372 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.372 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-14T17:15:46.372 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-14T17:15:46.372 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.372 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.372 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.372 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.372 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.372 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.372 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.372 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.373 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.373 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.373 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.373 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-14T17:15:46.373 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-14T17:15:46.373 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.373 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.373 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.373 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.373 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.373 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.373 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.373 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.373 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.374 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.374 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.374 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-14T17:15:46.374 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-14T17:15:46.374 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.374 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.374 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.374 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.374 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.374 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.374 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.374 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.375 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.375 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.375 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.375 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-14T17:15:46.375 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-14T17:15:46.375 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.375 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.375 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.375 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.375 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.375 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.375 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.375 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.375 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.376 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.376 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.376 INFO:teuthology.orchestra.run.smithi002.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-14T17:15:46.376 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-14T17:15:46.376 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.376 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.376 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.376 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.376 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.376 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.376 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.376 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.376 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.377 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.377 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.377 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_PASSWORD": { 2024-05-14T17:15:46.377 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-14T17:15:46.377 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.377 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.377 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.377 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2024-05-14T17:15:46.377 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.377 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.377 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.377 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.377 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.377 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.378 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.378 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-14T17:15:46.378 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-14T17:15:46.378 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.378 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.378 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.378 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.378 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.378 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.378 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.378 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.378 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.378 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.379 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.379 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_URL": { 2024-05-14T17:15:46.379 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_URL", 2024-05-14T17:15:46.379 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.379 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.379 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.379 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.379 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.379 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.379 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.379 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.379 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.379 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.380 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.380 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_USERNAME": { 2024-05-14T17:15:46.380 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-14T17:15:46.380 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.380 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.380 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.380 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2024-05-14T17:15:46.380 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.380 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.380 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.380 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.380 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.380 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.381 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.381 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-14T17:15:46.381 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-14T17:15:46.381 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.381 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.381 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.381 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.381 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.381 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.381 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.381 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.381 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.381 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.381 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.382 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-14T17:15:46.382 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-14T17:15:46.382 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.382 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.382 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.382 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.382 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.382 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.382 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.382 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.382 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.382 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.383 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.383 INFO:teuthology.orchestra.run.smithi002.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-14T17:15:46.383 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-14T17:15:46.383 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.383 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.383 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.383 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.383 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.383 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.383 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.383 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.383 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.383 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.384 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.384 INFO:teuthology.orchestra.run.smithi002.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-14T17:15:46.384 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-14T17:15:46.384 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.384 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.384 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.384 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.384 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.384 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.384 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.384 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.384 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.384 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.385 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.385 INFO:teuthology.orchestra.run.smithi002.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-14T17:15:46.385 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-14T17:15:46.385 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.385 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.385 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.385 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "[]", 2024-05-14T17:15:46.385 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.385 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.385 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.385 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.385 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.385 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.386 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.386 INFO:teuthology.orchestra.run.smithi002.stdout: "MULTICLUSTER_CONFIG": { 2024-05-14T17:15:46.386 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-14T17:15:46.386 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.386 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.386 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.386 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "{}", 2024-05-14T17:15:46.386 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.386 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.386 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.386 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.386 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.386 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.387 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.387 INFO:teuthology.orchestra.run.smithi002.stdout: "PROMETHEUS_API_HOST": { 2024-05-14T17:15:46.387 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-14T17:15:46.387 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.387 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.387 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.387 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.387 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.387 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.387 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.387 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.387 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.387 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.388 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.388 INFO:teuthology.orchestra.run.smithi002.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-14T17:15:46.388 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-14T17:15:46.388 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.388 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.388 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.388 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.388 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.388 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.388 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.388 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.388 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.388 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.388 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.389 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-14T17:15:46.389 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-14T17:15:46.389 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.389 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.389 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.389 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.389 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.389 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.389 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.389 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.389 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.389 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.389 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.390 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-14T17:15:46.390 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-14T17:15:46.390 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.390 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.390 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.390 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.390 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.390 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.390 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.390 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.390 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.390 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.391 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.391 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-14T17:15:46.391 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-14T17:15:46.391 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.391 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.391 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.391 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.391 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.391 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.391 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.391 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.391 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.391 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.392 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.392 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-14T17:15:46.392 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-14T17:15:46.392 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.392 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.392 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.392 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.392 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.392 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.392 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.392 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.392 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.392 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.392 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.393 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-14T17:15:46.393 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-14T17:15:46.393 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.393 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.393 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.393 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.393 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.393 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.393 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.393 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.393 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.393 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.393 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.394 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-14T17:15:46.394 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-14T17:15:46.394 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.394 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.394 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.394 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.394 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.394 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.394 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.394 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.394 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.394 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.394 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.395 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-14T17:15:46.395 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-14T17:15:46.395 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.395 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.395 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.395 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.395 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.395 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.395 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.395 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.395 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.395 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.395 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.396 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_ENABLED": { 2024-05-14T17:15:46.396 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-14T17:15:46.396 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.396 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.396 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.396 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.396 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.396 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.396 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.396 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.396 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.396 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.396 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.397 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-14T17:15:46.397 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-14T17:15:46.397 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.397 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.397 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.397 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-14T17:15:46.397 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.397 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.397 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.397 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.397 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.397 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.397 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.398 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-14T17:15:46.398 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-14T17:15:46.398 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.398 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.398 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.398 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T17:15:46.398 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.398 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.398 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.398 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.398 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.398 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.398 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.399 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-14T17:15:46.399 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-14T17:15:46.399 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.399 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.399 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.399 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8", 2024-05-14T17:15:46.399 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.399 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.399 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.399 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.399 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.399 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.399 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.399 INFO:teuthology.orchestra.run.smithi002.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-14T17:15:46.400 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-14T17:15:46.400 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.400 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.400 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.400 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "45", 2024-05-14T17:15:46.400 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.400 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.400 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.400 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.400 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.400 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.400 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.400 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_ACCESS_KEY": { 2024-05-14T17:15:46.400 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-14T17:15:46.400 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.401 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.401 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.401 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.401 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.401 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.401 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.401 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.401 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.401 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.401 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.401 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-14T17:15:46.401 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-14T17:15:46.401 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.402 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.402 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.402 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2024-05-14T17:15:46.402 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.402 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.402 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.402 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.402 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.402 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.402 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.402 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_SECRET_KEY": { 2024-05-14T17:15:46.402 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-14T17:15:46.402 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.402 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.403 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.403 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.403 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.403 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.403 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.403 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.403 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.403 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.403 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.403 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_SSL_VERIFY": { 2024-05-14T17:15:46.403 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-14T17:15:46.403 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.403 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.404 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.404 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.404 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.404 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.404 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.404 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.404 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.404 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.404 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.404 INFO:teuthology.orchestra.run.smithi002.stdout: "UNSAFE_TLS_v1_2": { 2024-05-14T17:15:46.404 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-14T17:15:46.404 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.404 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.404 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.404 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.405 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.405 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.405 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.405 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.405 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.405 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.405 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.405 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-14T17:15:46.405 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-14T17:15:46.405 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.405 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.405 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.406 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.406 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.406 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.406 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.406 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.406 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.406 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.406 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.406 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-14T17:15:46.406 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-14T17:15:46.406 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.406 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.406 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.406 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T17:15:46.407 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.407 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.407 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.407 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.407 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.407 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.407 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.407 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-14T17:15:46.407 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-14T17:15:46.407 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.407 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.407 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.407 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-05-14T17:15:46.407 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.408 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.408 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.408 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.408 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.408 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.408 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.408 INFO:teuthology.orchestra.run.smithi002.stdout: "cross_origin_url": { 2024-05-14T17:15:46.408 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cross_origin_url", 2024-05-14T17:15:46.408 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.408 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.408 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.408 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.408 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.408 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.409 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.409 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.409 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.409 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.409 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.409 INFO:teuthology.orchestra.run.smithi002.stdout: "crt_file": { 2024-05-14T17:15:46.409 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crt_file", 2024-05-14T17:15:46.409 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.409 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.409 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.409 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.409 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.409 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.410 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.410 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.410 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.410 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.410 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.410 INFO:teuthology.orchestra.run.smithi002.stdout: "debug": { 2024-05-14T17:15:46.410 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "debug", 2024-05-14T17:15:46.410 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.410 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.410 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.410 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.410 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.411 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.411 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.411 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable/disable debug options", 2024-05-14T17:15:46.411 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.411 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.411 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.411 INFO:teuthology.orchestra.run.smithi002.stdout: "jwt_token_ttl": { 2024-05-14T17:15:46.411 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "jwt_token_ttl", 2024-05-14T17:15:46.411 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.411 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.411 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.411 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "28800", 2024-05-14T17:15:46.411 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.412 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.412 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.412 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.412 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.412 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.412 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.412 INFO:teuthology.orchestra.run.smithi002.stdout: "key_file": { 2024-05-14T17:15:46.412 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "key_file", 2024-05-14T17:15:46.412 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.412 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.412 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.412 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.412 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.413 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.413 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.413 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.413 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.413 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.413 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.413 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.413 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.413 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.413 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.413 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.413 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.413 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.415 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.415 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.415 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.415 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.415 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.415 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.415 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.415 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.415 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.415 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.415 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.415 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.416 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.416 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.416 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.416 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.416 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.416 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.416 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.416 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.416 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.416 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.416 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.416 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.416 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.417 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.417 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.417 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.417 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.417 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.417 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.417 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.417 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.417 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.417 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.417 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.417 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.417 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.418 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.418 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.418 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.418 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.418 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.418 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.418 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.418 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.418 INFO:teuthology.orchestra.run.smithi002.stdout: "motd": { 2024-05-14T17:15:46.418 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "motd", 2024-05-14T17:15:46.418 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.418 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.419 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.419 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.419 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.419 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.419 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.419 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "The message of the day", 2024-05-14T17:15:46.419 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.419 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.419 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.419 INFO:teuthology.orchestra.run.smithi002.stdout: "redirect_resolve_ip_addr": { 2024-05-14T17:15:46.419 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "redirect_resolve_ip_addr", 2024-05-14T17:15:46.419 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.419 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.419 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.420 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.420 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.420 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.420 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.420 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.420 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.420 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.420 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.420 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2024-05-14T17:15:46.420 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2024-05-14T17:15:46.420 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.420 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.420 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.420 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "::", 2024-05-14T17:15:46.421 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.421 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.421 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.421 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.421 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.421 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.421 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.421 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2024-05-14T17:15:46.421 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2024-05-14T17:15:46.421 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.421 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.421 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.421 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8080", 2024-05-14T17:15:46.421 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.422 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.422 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.422 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.422 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.422 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.422 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.422 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.422 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.422 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.422 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.422 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.422 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.422 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.423 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.423 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.423 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.423 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.423 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.423 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.423 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl": { 2024-05-14T17:15:46.423 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl", 2024-05-14T17:15:46.423 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.423 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.423 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.423 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.423 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.423 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.424 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.424 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.424 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.424 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.424 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.424 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl_server_port": { 2024-05-14T17:15:46.424 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl_server_port", 2024-05-14T17:15:46.424 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.424 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.424 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.424 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8443", 2024-05-14T17:15:46.424 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.424 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.424 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.425 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.425 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.425 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.425 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.425 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_behaviour": { 2024-05-14T17:15:46.425 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_behaviour", 2024-05-14T17:15:46.425 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.425 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.425 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.425 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "redirect", 2024-05-14T17:15:46.425 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.425 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.425 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.426 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.426 INFO:teuthology.orchestra.run.smithi002.stdout: "redirect" 2024-05-14T17:15:46.426 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.426 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.426 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.426 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.426 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.426 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_error_status_code": { 2024-05-14T17:15:46.426 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_error_status_code", 2024-05-14T17:15:46.426 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.426 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.426 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.427 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "500", 2024-05-14T17:15:46.427 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "400", 2024-05-14T17:15:46.427 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "599", 2024-05-14T17:15:46.427 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.427 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.427 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.427 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.427 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.427 INFO:teuthology.orchestra.run.smithi002.stdout: "url_prefix": { 2024-05-14T17:15:46.427 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "url_prefix", 2024-05-14T17:15:46.427 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.427 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.427 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.428 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.428 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.428 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.428 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.428 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.428 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.428 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.428 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.428 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.428 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.428 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "devicehealth", 2024-05-14T17:15:46.428 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.429 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.429 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.429 INFO:teuthology.orchestra.run.smithi002.stdout: "enable_monitoring": { 2024-05-14T17:15:46.429 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enable_monitoring", 2024-05-14T17:15:46.429 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.429 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.429 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.429 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.429 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.429 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.429 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.429 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "monitor device health metrics", 2024-05-14T17:15:46.429 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.429 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.430 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.430 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.430 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.430 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.430 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.430 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.430 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.430 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.430 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.430 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.430 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.430 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.431 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.431 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.431 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.431 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.431 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.431 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.431 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.431 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.431 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.431 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.431 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.431 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.431 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.432 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.432 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.432 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.432 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.432 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.432 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.432 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.432 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.432 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.432 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.432 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.432 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.432 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.432 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.433 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.433 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.433 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.433 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.433 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.433 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.433 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.433 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.433 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.433 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.433 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.433 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.433 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.434 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.434 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.434 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.434 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.434 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.434 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.434 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.434 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.434 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.434 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.434 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.434 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.434 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.435 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.435 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.435 INFO:teuthology.orchestra.run.smithi002.stdout: "mark_out_threshold": { 2024-05-14T17:15:46.435 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mark_out_threshold", 2024-05-14T17:15:46.435 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.435 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.435 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.435 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "2419200", 2024-05-14T17:15:46.435 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.435 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.435 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.435 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-14T17:15:46.435 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.435 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.436 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.436 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": { 2024-05-14T17:15:46.436 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pool_name", 2024-05-14T17:15:46.436 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.436 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.436 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.436 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "device_health_metrics", 2024-05-14T17:15:46.436 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.436 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.436 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.436 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-14T17:15:46.436 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.436 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.436 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.437 INFO:teuthology.orchestra.run.smithi002.stdout: "retention_period": { 2024-05-14T17:15:46.437 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "retention_period", 2024-05-14T17:15:46.437 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.437 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.437 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.437 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15552000", 2024-05-14T17:15:46.437 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.437 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.437 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.437 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long to retain device health metrics", 2024-05-14T17:15:46.437 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.437 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.437 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.438 INFO:teuthology.orchestra.run.smithi002.stdout: "scrape_frequency": { 2024-05-14T17:15:46.438 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "scrape_frequency", 2024-05-14T17:15:46.438 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.438 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.438 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.438 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "86400", 2024-05-14T17:15:46.438 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.438 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.438 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.438 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-14T17:15:46.438 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.438 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.438 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.438 INFO:teuthology.orchestra.run.smithi002.stdout: "self_heal": { 2024-05-14T17:15:46.439 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "self_heal", 2024-05-14T17:15:46.439 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.439 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.439 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.439 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.439 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.439 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.439 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.439 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-14T17:15:46.439 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.439 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.439 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.439 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-05-14T17:15:46.439 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-05-14T17:15:46.440 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.440 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.440 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.440 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-05-14T17:15:46.440 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.440 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.440 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.440 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to wake up and check device health", 2024-05-14T17:15:46.440 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.440 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.440 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.440 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.440 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.441 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.441 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.441 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.441 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.441 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.441 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.441 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.441 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.441 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.441 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.441 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.442 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_threshold": { 2024-05-14T17:15:46.442 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_threshold", 2024-05-14T17:15:46.442 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.442 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.442 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.442 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "7257600", 2024-05-14T17:15:46.442 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.442 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.442 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.442 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-14T17:15:46.442 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.442 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.442 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.442 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.443 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.443 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "diskprediction_local", 2024-05-14T17:15:46.443 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.443 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.443 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.443 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.443 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.443 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.443 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.443 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.443 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.443 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.444 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.444 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.444 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.444 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.444 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.444 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.444 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.444 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.444 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.444 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.444 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.444 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.444 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.444 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.445 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.445 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.445 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.445 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.445 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.445 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.445 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.445 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.445 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.445 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.445 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.445 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.445 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.445 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.446 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.446 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.446 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.446 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.446 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.446 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.446 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.446 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.446 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.446 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.446 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.446 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.446 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.446 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.446 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.447 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.447 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.447 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.447 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.447 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.447 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.447 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.447 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.447 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.447 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.447 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.447 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.447 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.447 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.448 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.448 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.448 INFO:teuthology.orchestra.run.smithi002.stdout: "predict_interval": { 2024-05-14T17:15:46.448 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "predict_interval", 2024-05-14T17:15:46.448 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.448 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.448 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.448 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "86400", 2024-05-14T17:15:46.448 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.448 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.448 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.448 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.448 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.449 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.449 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.449 INFO:teuthology.orchestra.run.smithi002.stdout: "predictor_model": { 2024-05-14T17:15:46.449 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "predictor_model", 2024-05-14T17:15:46.449 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.449 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.449 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.449 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "prophetstor", 2024-05-14T17:15:46.449 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.449 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.449 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.449 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.449 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.449 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.450 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.450 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-05-14T17:15:46.450 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-05-14T17:15:46.450 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.450 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.450 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.450 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-05-14T17:15:46.450 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.450 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.450 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.450 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.450 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.450 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.451 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.451 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.451 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.451 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.451 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.451 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.451 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.451 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.451 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.451 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.451 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.451 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.451 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.451 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.452 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.452 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.452 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "influx", 2024-05-14T17:15:46.452 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": false, 2024-05-14T17:15:46.452 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "influxdb python module not found", 2024-05-14T17:15:46.452 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.452 INFO:teuthology.orchestra.run.smithi002.stdout: "batch_size": { 2024-05-14T17:15:46.452 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "batch_size", 2024-05-14T17:15:46.452 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.452 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.452 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.452 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5000", 2024-05-14T17:15:46.453 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.453 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.453 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.453 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-14T17:15:46.453 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.453 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.453 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.453 INFO:teuthology.orchestra.run.smithi002.stdout: "database": { 2024-05-14T17:15:46.453 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "database", 2024-05-14T17:15:46.453 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.453 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.453 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.453 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "ceph", 2024-05-14T17:15:46.453 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.454 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.454 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.454 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-14T17:15:46.454 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.454 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.454 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.454 INFO:teuthology.orchestra.run.smithi002.stdout: "hostname": { 2024-05-14T17:15:46.454 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "hostname", 2024-05-14T17:15:46.454 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.454 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.454 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.454 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.454 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.455 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.455 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.455 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB server hostname", 2024-05-14T17:15:46.455 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.455 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.455 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.455 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-05-14T17:15:46.455 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-05-14T17:15:46.455 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.455 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.455 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.455 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "30", 2024-05-14T17:15:46.455 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "5", 2024-05-14T17:15:46.456 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.456 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.456 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-14T17:15:46.456 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.456 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.456 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.456 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.456 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.456 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.456 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.456 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.456 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.456 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.457 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.457 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.457 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.457 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.457 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.457 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.457 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.457 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.457 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.457 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.457 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.457 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.457 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.457 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.458 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.458 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.458 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.458 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.458 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.458 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.458 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.458 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.458 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.458 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.458 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.458 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.458 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.458 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.459 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.459 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.459 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.459 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.459 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.459 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.459 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.459 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.459 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.459 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.459 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.459 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.459 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.459 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.460 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.460 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.460 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.460 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.460 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.460 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.460 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.460 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.460 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.460 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.460 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.460 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.460 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.461 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.461 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.461 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.461 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.461 INFO:teuthology.orchestra.run.smithi002.stdout: "password": { 2024-05-14T17:15:46.461 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "password", 2024-05-14T17:15:46.461 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.461 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.461 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.461 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.461 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.461 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.461 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.461 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "password of InfluxDB server user", 2024-05-14T17:15:46.461 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.462 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.462 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.462 INFO:teuthology.orchestra.run.smithi002.stdout: "port": { 2024-05-14T17:15:46.462 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "port", 2024-05-14T17:15:46.462 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.462 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.462 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.462 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8086", 2024-05-14T17:15:46.462 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.462 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.462 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.462 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB server port", 2024-05-14T17:15:46.462 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.463 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.463 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.463 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.463 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.463 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.463 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.463 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.463 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.463 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.463 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.463 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.463 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.463 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.464 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.464 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.464 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl": { 2024-05-14T17:15:46.464 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl", 2024-05-14T17:15:46.464 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.464 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.464 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.464 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "false", 2024-05-14T17:15:46.464 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.464 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.464 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.464 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-14T17:15:46.465 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.465 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.465 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.465 INFO:teuthology.orchestra.run.smithi002.stdout: "threads": { 2024-05-14T17:15:46.465 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "threads", 2024-05-14T17:15:46.465 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.465 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.465 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.465 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-05-14T17:15:46.465 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-05-14T17:15:46.465 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "32", 2024-05-14T17:15:46.465 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.465 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-14T17:15:46.465 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.466 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.466 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.466 INFO:teuthology.orchestra.run.smithi002.stdout: "username": { 2024-05-14T17:15:46.466 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "username", 2024-05-14T17:15:46.466 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.466 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.466 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.466 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.466 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.466 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.466 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.466 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "username of InfluxDB server user", 2024-05-14T17:15:46.466 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.467 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.467 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.467 INFO:teuthology.orchestra.run.smithi002.stdout: "verify_ssl": { 2024-05-14T17:15:46.467 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "verify_ssl", 2024-05-14T17:15:46.467 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.467 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.467 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.467 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "true", 2024-05-14T17:15:46.467 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.467 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.467 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.467 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-14T17:15:46.468 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.468 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.468 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.468 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.468 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.468 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "insights", 2024-05-14T17:15:46.468 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.468 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.468 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.468 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.468 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.469 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.469 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.469 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.469 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.469 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.469 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.469 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.469 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.469 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.469 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.469 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.469 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.469 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.469 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.470 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.470 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.470 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.470 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.470 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.470 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.470 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.470 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.470 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.470 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.470 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.470 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.471 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.471 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.471 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.471 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.471 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.471 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.471 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.471 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.471 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.471 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.471 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.471 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.472 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.472 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.472 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.472 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.472 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.472 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.472 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.472 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.472 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.472 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.472 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.472 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.472 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.473 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.473 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.473 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.473 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.473 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.473 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.473 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.473 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.473 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.473 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.473 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.473 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.473 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.474 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.474 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.474 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.474 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.474 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.474 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.474 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.474 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.474 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.474 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.474 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.474 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.474 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.474 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.475 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.475 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "iostat", 2024-05-14T17:15:46.475 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.475 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.475 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.475 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.475 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.475 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.475 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.475 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.475 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.475 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.476 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.476 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.476 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.476 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.476 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.476 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.476 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.476 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.476 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.476 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.476 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.476 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.476 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.477 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.477 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.477 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.477 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.477 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.477 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.477 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.477 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.477 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.477 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.477 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.477 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.477 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.478 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.478 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.478 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.478 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.478 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.478 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.478 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.478 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.478 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.478 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.478 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.478 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.478 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.478 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.479 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.479 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.479 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.479 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.479 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.479 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.479 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.479 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.479 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.479 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.479 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.479 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.479 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.480 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.480 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.480 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.480 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.480 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.480 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.480 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.480 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.480 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.480 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.480 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.480 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.480 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.480 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.481 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.481 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.481 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.481 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.481 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.481 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.481 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.481 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "k8sevents", 2024-05-14T17:15:46.481 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.481 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.481 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.482 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_event_retention_days": { 2024-05-14T17:15:46.482 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ceph_event_retention_days", 2024-05-14T17:15:46.482 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.482 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.482 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.482 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "7", 2024-05-14T17:15:46.482 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.482 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.482 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.482 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-14T17:15:46.482 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.482 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.482 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.483 INFO:teuthology.orchestra.run.smithi002.stdout: "config_check_secs": { 2024-05-14T17:15:46.483 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_check_secs", 2024-05-14T17:15:46.483 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.483 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.483 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.483 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T17:15:46.483 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "10", 2024-05-14T17:15:46.483 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.483 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.483 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-14T17:15:46.483 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.483 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.483 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.484 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.484 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.484 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.484 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.484 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.484 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.484 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.484 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.484 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.484 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.484 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.484 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.484 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.484 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.484 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.485 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.485 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.485 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.485 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.485 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.485 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.485 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.485 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.485 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.485 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.485 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.485 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.486 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.486 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.486 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.486 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.486 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.486 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.486 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.486 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.486 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.486 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.486 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.486 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.486 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.487 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.487 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.487 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.487 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.487 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.487 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.487 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.487 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.487 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.487 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.487 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.487 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.487 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.488 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.488 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.488 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.488 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.488 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.488 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.488 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.488 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.488 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.488 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.488 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.488 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.488 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.489 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.489 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.489 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.489 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.489 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.489 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.489 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.489 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.489 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.489 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.489 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.489 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.489 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.489 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.490 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.490 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "localpool", 2024-05-14T17:15:46.490 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.490 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.490 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.490 INFO:teuthology.orchestra.run.smithi002.stdout: "failure_domain": { 2024-05-14T17:15:46.490 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "failure_domain", 2024-05-14T17:15:46.490 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.490 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.490 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.490 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "host", 2024-05-14T17:15:46.490 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.491 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.491 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.491 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "failure domain for any created local pool", 2024-05-14T17:15:46.491 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-14T17:15:46.491 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.491 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.491 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.491 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.491 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.491 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.491 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.491 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.491 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.492 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.492 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.492 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.492 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.492 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.492 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.492 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.492 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.492 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.492 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.492 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.492 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.492 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.493 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.493 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.493 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.493 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.493 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.493 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.493 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.493 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.493 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.493 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.493 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.493 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.493 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.494 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.494 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.494 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.494 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.494 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.494 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.494 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.494 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.494 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.494 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.494 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.494 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.494 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.494 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.495 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.495 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.495 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.495 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.495 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.495 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.495 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.495 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.495 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.495 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.495 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.495 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.495 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.496 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.496 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.496 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.496 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.496 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.496 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.496 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": { 2024-05-14T17:15:46.496 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "min_size", 2024-05-14T17:15:46.496 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.496 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.496 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.496 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.496 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.497 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.497 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.497 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default min_size for any created local pool", 2024-05-14T17:15:46.497 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-14T17:15:46.497 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.497 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.497 INFO:teuthology.orchestra.run.smithi002.stdout: "num_rep": { 2024-05-14T17:15:46.497 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "num_rep", 2024-05-14T17:15:46.497 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.497 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.497 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.497 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3", 2024-05-14T17:15:46.497 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.498 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.498 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.498 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default replica count for any created local pool", 2024-05-14T17:15:46.498 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.498 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.498 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.498 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": { 2024-05-14T17:15:46.498 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pg_num", 2024-05-14T17:15:46.498 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.498 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.498 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.498 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "128", 2024-05-14T17:15:46.498 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.499 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.499 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.499 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default pg_num for any created local pool", 2024-05-14T17:15:46.499 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.499 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.499 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.499 INFO:teuthology.orchestra.run.smithi002.stdout: "prefix": { 2024-05-14T17:15:46.499 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prefix", 2024-05-14T17:15:46.499 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.499 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.499 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.499 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.499 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.500 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.500 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.500 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "name prefix for any created local pool", 2024-05-14T17:15:46.500 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.500 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.500 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.500 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.500 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.500 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.500 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.500 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.500 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.500 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.500 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.501 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.501 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.501 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.501 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.501 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.501 INFO:teuthology.orchestra.run.smithi002.stdout: "subtree": { 2024-05-14T17:15:46.501 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "subtree", 2024-05-14T17:15:46.501 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.501 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.501 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.501 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "rack", 2024-05-14T17:15:46.501 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.501 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.502 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.502 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-14T17:15:46.502 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-14T17:15:46.502 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.502 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.502 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.502 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.502 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mds_autoscaler", 2024-05-14T17:15:46.502 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.502 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.502 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.502 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.503 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.503 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.503 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.503 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.503 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.503 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.503 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.503 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.503 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.503 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.503 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.503 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.503 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.503 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.504 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.504 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.504 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.504 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.504 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.504 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.504 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.504 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.504 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.504 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.504 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.504 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.504 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.505 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.505 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.505 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.505 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.505 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.505 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.505 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.505 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.505 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.505 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.505 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.505 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.505 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.505 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.506 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.506 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.506 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.506 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.506 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.506 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.506 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.506 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.506 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.506 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.506 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.506 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.507 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.507 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.507 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.507 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.507 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.507 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.507 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.507 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.507 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.507 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.507 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.507 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.507 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.507 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.508 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.508 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.508 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.508 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.508 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.508 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.508 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.508 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.508 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.508 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.508 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.508 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.508 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.509 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mirroring", 2024-05-14T17:15:46.509 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.509 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.509 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.509 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.509 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.509 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.509 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.509 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.509 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.509 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.509 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.509 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.510 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.510 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.510 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.510 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.510 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.510 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.510 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.510 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.510 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.510 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.510 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.510 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.510 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.511 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.511 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.511 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.511 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.511 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.511 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.511 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.511 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.511 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.511 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.511 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.511 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.511 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.511 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.512 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.512 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.512 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.512 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.512 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.512 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.512 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.512 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.512 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.512 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.512 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.512 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.512 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.512 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.513 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.513 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.513 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.513 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.513 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.513 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.513 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.513 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.513 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.513 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.513 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.513 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.513 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.514 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.514 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.514 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.514 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.514 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.514 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.514 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.514 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.514 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.514 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.514 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.514 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.514 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.515 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.515 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.515 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.515 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.515 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.515 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "nfs", 2024-05-14T17:15:46.515 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.515 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.515 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.515 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.515 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.515 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.516 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.516 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.516 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.516 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.516 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.516 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.516 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.516 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.516 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.516 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.516 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.516 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.516 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.516 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.517 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.517 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.517 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.517 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.517 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.517 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.517 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.517 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.517 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.517 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.517 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.517 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.517 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.517 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.518 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.518 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.518 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.518 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.518 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.518 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.518 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.518 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.518 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.518 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.518 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.518 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.518 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.519 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.519 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.519 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.519 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.519 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.519 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.519 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.519 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.519 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.519 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.519 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.519 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.519 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.520 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.520 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.520 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.520 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.520 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.520 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.520 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.520 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.520 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.520 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.520 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.520 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.520 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.520 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.521 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.521 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.521 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.521 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.521 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.521 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.521 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.521 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.521 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.521 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.521 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "orchestrator", 2024-05-14T17:15:46.521 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.521 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.522 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.522 INFO:teuthology.orchestra.run.smithi002.stdout: "fail_fs": { 2024-05-14T17:15:46.522 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "fail_fs", 2024-05-14T17:15:46.522 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.522 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.522 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.522 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.522 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.522 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.522 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.522 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-14T17:15:46.522 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.522 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.523 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.523 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.523 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.523 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.523 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.523 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.523 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.523 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.523 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.523 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.523 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.523 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.523 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.524 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.524 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.524 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.524 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.524 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.524 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.524 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.524 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.524 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.524 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.524 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.524 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.524 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.525 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.525 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.525 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.525 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.525 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.525 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.525 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.525 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.525 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.525 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.525 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.525 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.525 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.526 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.526 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.526 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.526 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.526 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.526 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.526 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.526 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.526 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.526 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.526 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.526 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.526 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.526 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.527 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.527 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.527 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.527 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.527 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.527 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.527 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.527 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.527 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.527 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.527 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.527 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.527 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.528 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.528 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator": { 2024-05-14T17:15:46.528 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "orchestrator", 2024-05-14T17:15:46.528 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.528 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.528 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.528 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.528 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.528 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.528 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.528 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm", 2024-05-14T17:15:46.528 INFO:teuthology.orchestra.run.smithi002.stdout: "rook", 2024-05-14T17:15:46.528 INFO:teuthology.orchestra.run.smithi002.stdout: "test_orchestrator" 2024-05-14T17:15:46.528 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.529 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Orchestrator backend", 2024-05-14T17:15:46.529 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.529 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.529 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.529 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.529 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.529 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.529 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.529 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.529 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.529 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.529 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.529 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.529 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.530 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.530 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.530 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.530 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.530 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.530 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "osd_perf_query", 2024-05-14T17:15:46.530 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.530 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.530 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.530 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.530 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.530 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.531 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.531 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.531 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.531 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.531 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.531 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.531 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.531 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.531 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.531 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.531 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.531 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.531 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.531 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.532 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.532 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.532 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.532 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.532 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.532 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.532 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.532 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.532 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.532 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.532 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.532 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.532 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.532 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.533 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.533 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.533 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.533 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.533 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.533 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.533 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.533 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.533 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.533 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.533 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.533 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.533 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.533 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.534 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.534 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.534 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.534 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.534 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.534 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.534 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.534 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.534 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.534 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.534 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.534 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.534 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.535 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.535 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.535 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.535 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.535 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.535 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.535 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.535 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.535 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.535 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.535 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.535 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.535 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.536 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.536 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.536 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.536 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.536 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.536 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.536 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.536 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.536 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.536 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.536 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "osd_support", 2024-05-14T17:15:46.536 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.536 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.536 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.537 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.537 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.537 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.537 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.537 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.537 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.537 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.537 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.537 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.537 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.537 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.537 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.537 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.537 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.537 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.538 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.538 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.538 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.538 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.538 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.538 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.538 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.538 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.538 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.538 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.538 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.538 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.539 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.539 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.539 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.539 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.539 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.539 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.539 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.539 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.539 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.539 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.539 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.539 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.539 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.539 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.539 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.540 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.540 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.540 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.540 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.540 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.540 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.540 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.540 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.540 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.540 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.540 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.540 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.540 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.541 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.541 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.541 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.541 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.541 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.541 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.541 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.541 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.541 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.541 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.541 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.541 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.541 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.542 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.542 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.542 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.542 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.542 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.542 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.542 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.542 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.542 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.542 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.542 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.542 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.542 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.543 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pg_autoscaler", 2024-05-14T17:15:46.543 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.543 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.543 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.543 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.543 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.543 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.543 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.543 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.543 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.543 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.543 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.543 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.543 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.543 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.544 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.544 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.544 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.544 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.544 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.544 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.544 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.544 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.544 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.544 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.544 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.544 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.544 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.544 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.545 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.545 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.545 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.545 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.545 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.545 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.545 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.545 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.545 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.545 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.545 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.545 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.545 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.546 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.546 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.546 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.546 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.546 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.546 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.546 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.546 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.546 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.546 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.546 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.546 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.546 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.546 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.546 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.547 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.547 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.547 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.547 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.547 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.547 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.547 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.547 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.547 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.547 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.547 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.547 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.547 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.547 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-05-14T17:15:46.548 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-05-14T17:15:46.548 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.548 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.548 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.548 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-05-14T17:15:46.548 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.548 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.548 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.548 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.548 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.548 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.548 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.548 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.548 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.548 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.549 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.549 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.549 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.549 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.549 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.549 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.549 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.549 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.549 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.549 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.549 INFO:teuthology.orchestra.run.smithi002.stdout: "threshold": { 2024-05-14T17:15:46.549 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "threshold", 2024-05-14T17:15:46.549 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T17:15:46.550 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.550 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.550 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3.0", 2024-05-14T17:15:46.550 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1.0", 2024-05-14T17:15:46.550 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.550 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.550 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "scaling threshold", 2024-05-14T17:15:46.550 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-14T17:15:46.550 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.550 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.550 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.550 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.550 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "progress", 2024-05-14T17:15:46.551 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.551 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.551 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.551 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_pg_recovery_event": { 2024-05-14T17:15:46.551 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_pg_recovery_event", 2024-05-14T17:15:46.551 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.551 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.551 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.551 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.551 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.551 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.551 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.551 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-14T17:15:46.551 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.552 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.552 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.552 INFO:teuthology.orchestra.run.smithi002.stdout: "enabled": { 2024-05-14T17:15:46.552 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enabled", 2024-05-14T17:15:46.552 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.552 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.552 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.552 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.552 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.552 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.552 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.552 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.552 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.553 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.553 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.553 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.553 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.553 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.553 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.553 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.553 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.553 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.553 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.553 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.553 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.553 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.553 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.554 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.554 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.554 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.554 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.554 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.554 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.554 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.554 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.554 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.554 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.554 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.554 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.554 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.555 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.555 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.555 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.555 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.555 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.555 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.555 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.555 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.555 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.555 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.555 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.555 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.555 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.555 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.556 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.556 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.556 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.556 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.556 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.556 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.556 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.556 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.556 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.556 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.556 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.556 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.556 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.556 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.557 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.557 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.557 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.557 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.557 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.557 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.557 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.557 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.557 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.557 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.557 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.557 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.558 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.558 INFO:teuthology.orchestra.run.smithi002.stdout: "max_completed_events": { 2024-05-14T17:15:46.558 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_completed_events", 2024-05-14T17:15:46.558 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.558 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.558 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.558 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "50", 2024-05-14T17:15:46.558 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.558 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.558 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.558 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "number of past completed events to remember", 2024-05-14T17:15:46.558 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.559 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.559 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.559 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-05-14T17:15:46.559 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-05-14T17:15:46.559 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.559 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.559 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.559 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-05-14T17:15:46.559 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.559 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.559 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.559 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long the module is going to sleep", 2024-05-14T17:15:46.559 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.559 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.559 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.560 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.560 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.560 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.560 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.560 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.560 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.560 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.560 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.560 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.560 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.560 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.560 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.560 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.560 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.561 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.561 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus", 2024-05-14T17:15:46.561 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.561 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.561 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.561 INFO:teuthology.orchestra.run.smithi002.stdout: "cache": { 2024-05-14T17:15:46.561 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cache", 2024-05-14T17:15:46.561 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.561 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.561 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.561 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.561 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.561 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.562 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.562 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.562 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.562 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.562 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.562 INFO:teuthology.orchestra.run.smithi002.stdout: "exclude_perf_counters": { 2024-05-14T17:15:46.562 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "exclude_perf_counters", 2024-05-14T17:15:46.562 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.562 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.562 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.562 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.562 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.562 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.563 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.563 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-14T17:15:46.563 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-14T17:15:46.563 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.563 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.563 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.563 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.563 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.563 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.563 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.563 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.563 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.563 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.563 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.564 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.564 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.564 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.564 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.564 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.564 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.564 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.564 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.564 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.564 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.564 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.564 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.564 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.564 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.564 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.565 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.565 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.565 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.565 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.565 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.565 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.565 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.565 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.565 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.565 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.565 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.565 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.565 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.565 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.565 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.566 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.566 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.566 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.566 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.566 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.566 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.566 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.566 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.566 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.566 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.566 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.566 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.566 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.566 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.567 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.567 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.567 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.567 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.567 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.567 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.567 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.567 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.567 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.567 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.567 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.567 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.567 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.568 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_stats_pools": { 2024-05-14T17:15:46.568 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_stats_pools", 2024-05-14T17:15:46.568 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.568 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.568 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.568 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.568 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.568 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.568 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.568 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.568 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.568 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.568 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.568 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-14T17:15:46.569 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-14T17:15:46.569 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.569 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.569 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.569 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "300", 2024-05-14T17:15:46.569 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.569 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.569 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.569 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.569 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.569 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.569 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.569 INFO:teuthology.orchestra.run.smithi002.stdout: "scrape_interval": { 2024-05-14T17:15:46.569 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "scrape_interval", 2024-05-14T17:15:46.570 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T17:15:46.570 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.570 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.570 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15.0", 2024-05-14T17:15:46.570 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.570 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.570 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.570 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.570 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.570 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.570 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.570 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2024-05-14T17:15:46.570 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2024-05-14T17:15:46.571 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.571 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.571 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.571 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "::", 2024-05-14T17:15:46.571 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.571 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.571 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.571 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-14T17:15:46.571 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.571 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.571 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.571 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2024-05-14T17:15:46.571 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2024-05-14T17:15:46.571 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.571 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.572 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.572 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "9283", 2024-05-14T17:15:46.572 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.572 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.572 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.572 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-14T17:15:46.572 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.572 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.572 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.572 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.572 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.572 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.572 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.572 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.573 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.573 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.573 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.573 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.573 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.573 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.573 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.573 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.573 INFO:teuthology.orchestra.run.smithi002.stdout: "stale_cache_strategy": { 2024-05-14T17:15:46.573 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "stale_cache_strategy", 2024-05-14T17:15:46.573 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.573 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.573 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.573 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "log", 2024-05-14T17:15:46.573 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.574 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.574 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.574 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.574 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.574 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.574 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.574 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_behaviour": { 2024-05-14T17:15:46.574 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_behaviour", 2024-05-14T17:15:46.574 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.574 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.574 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.574 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "default", 2024-05-14T17:15:46.575 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.575 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.575 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.575 INFO:teuthology.orchestra.run.smithi002.stdout: "default", 2024-05-14T17:15:46.575 INFO:teuthology.orchestra.run.smithi002.stdout: "error" 2024-05-14T17:15:46.575 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.575 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.575 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.575 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.575 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.575 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_error_status_code": { 2024-05-14T17:15:46.575 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_error_status_code", 2024-05-14T17:15:46.575 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.576 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.576 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.576 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "500", 2024-05-14T17:15:46.576 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "400", 2024-05-14T17:15:46.576 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "599", 2024-05-14T17:15:46.576 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.576 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.576 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.576 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.576 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.576 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.576 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.577 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_support", 2024-05-14T17:15:46.577 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.577 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.577 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.577 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.577 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.577 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.577 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.577 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.577 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.577 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.577 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.577 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.577 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.578 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.578 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.578 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.578 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.578 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.578 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.578 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.578 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.578 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.578 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.578 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.578 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.578 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.579 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.579 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.579 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.579 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.579 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.579 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.579 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.579 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.579 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.579 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.579 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.579 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.579 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.580 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.580 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.580 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.580 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.580 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.580 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.580 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.580 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.580 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.580 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.580 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.580 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.580 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.580 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.580 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.581 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.581 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.581 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.581 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.581 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.581 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.581 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.581 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.581 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.581 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.581 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.581 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.581 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.582 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.582 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.582 INFO:teuthology.orchestra.run.smithi002.stdout: "max_concurrent_snap_create": { 2024-05-14T17:15:46.582 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_concurrent_snap_create", 2024-05-14T17:15:46.582 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.582 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.582 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.582 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T17:15:46.582 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.582 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.582 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.582 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.583 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.583 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.583 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.583 INFO:teuthology.orchestra.run.smithi002.stdout: "mirror_snapshot_schedule": { 2024-05-14T17:15:46.583 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mirror_snapshot_schedule", 2024-05-14T17:15:46.583 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.583 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.583 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.583 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.583 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.583 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.583 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.583 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.583 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.584 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.584 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.584 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.584 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.584 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.584 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.584 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.584 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.584 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.584 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.584 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.584 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.584 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.585 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.585 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.585 INFO:teuthology.orchestra.run.smithi002.stdout: "trash_purge_schedule": { 2024-05-14T17:15:46.585 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "trash_purge_schedule", 2024-05-14T17:15:46.585 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.585 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.585 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.585 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.585 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.585 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.585 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.585 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.585 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.585 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.586 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.586 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.586 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.586 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "restful", 2024-05-14T17:15:46.586 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.586 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.586 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.586 INFO:teuthology.orchestra.run.smithi002.stdout: "enable_auth": { 2024-05-14T17:15:46.586 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enable_auth", 2024-05-14T17:15:46.586 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.586 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.586 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.587 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.587 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.587 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.587 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.587 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.587 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.587 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.587 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.587 INFO:teuthology.orchestra.run.smithi002.stdout: "key_file": { 2024-05-14T17:15:46.587 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "key_file", 2024-05-14T17:15:46.587 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.587 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.588 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.588 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.588 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.588 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.588 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.588 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.588 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.588 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.588 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.588 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.588 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.588 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.588 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.589 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.589 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.589 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.589 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.589 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.589 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.589 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.589 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.589 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.589 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.589 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.589 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.589 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.589 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.590 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.590 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.590 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.590 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.590 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.590 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.590 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.590 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.590 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.590 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.590 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.590 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.591 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.591 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.591 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.591 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.591 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.591 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.591 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.591 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.591 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.591 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.591 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.591 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.591 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.592 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.592 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.592 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.592 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.592 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.592 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.592 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.592 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.592 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.592 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.592 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.592 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.592 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.592 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.593 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.593 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.593 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.593 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.593 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.593 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.593 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.593 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.593 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.593 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2024-05-14T17:15:46.593 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2024-05-14T17:15:46.594 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.594 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.594 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.594 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.594 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.594 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.594 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.594 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.594 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.594 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.594 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.595 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2024-05-14T17:15:46.595 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2024-05-14T17:15:46.595 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.595 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.595 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.595 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.595 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.595 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.595 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.595 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.595 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.595 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.595 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.595 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.596 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.596 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.596 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.596 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.596 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.596 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.596 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.596 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.596 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.596 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.596 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.596 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.596 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.596 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.597 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rgw", 2024-05-14T17:15:46.597 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.597 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.597 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.597 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.597 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.597 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.597 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.597 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.597 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.597 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.597 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.597 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.598 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.598 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.598 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.598 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.598 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.598 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.598 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.598 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.598 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.598 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.598 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.598 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.598 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.598 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.599 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.599 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.599 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.599 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.599 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.599 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.599 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.599 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.599 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.599 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.599 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.599 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.599 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.600 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.600 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.600 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.600 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.600 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.600 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.600 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.600 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.600 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.600 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.600 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.600 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.600 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.600 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.600 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.601 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.601 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.601 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.601 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.601 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.601 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.601 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.601 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.601 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.601 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.601 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.601 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.601 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.601 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.602 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.602 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.602 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.602 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.602 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.602 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.602 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.602 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.602 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.602 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.602 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.602 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.602 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.602 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.603 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.603 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.603 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rook", 2024-05-14T17:15:46.603 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.603 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.603 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.603 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.603 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.603 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.603 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.603 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.603 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.604 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.604 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.604 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.604 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.604 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.604 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.604 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.604 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.604 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.604 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.604 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.604 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.604 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.604 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.605 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.605 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.605 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.605 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.605 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.605 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.605 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.605 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.605 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.605 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.605 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.605 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.605 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.605 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.605 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.606 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.606 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.606 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.606 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.606 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.606 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.606 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.606 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.606 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.606 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.606 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.606 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.606 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.606 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.607 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.607 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.607 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.607 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.607 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.607 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.607 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.607 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.607 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.607 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.607 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.607 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.607 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.607 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.608 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.608 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.608 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.608 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.608 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.608 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.608 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.608 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.608 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.608 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.608 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.608 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.608 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.608 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.608 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.608 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.609 INFO:teuthology.orchestra.run.smithi002.stdout: "storage_class": { 2024-05-14T17:15:46.609 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "storage_class", 2024-05-14T17:15:46.609 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.609 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.609 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.609 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "local", 2024-05-14T17:15:46.609 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.609 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.609 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.609 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-14T17:15:46.609 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.609 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.609 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.610 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.610 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.610 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "selftest", 2024-05-14T17:15:46.610 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.610 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.610 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.610 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.610 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.610 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.610 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.610 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.610 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.610 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.611 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.611 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.611 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.611 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.611 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.611 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.611 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.611 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.611 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.611 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.611 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.611 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.611 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.612 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.612 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.612 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.612 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.612 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.612 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.612 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.612 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.612 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.612 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.612 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.612 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.612 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.612 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.613 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.613 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.613 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.613 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.613 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.613 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.613 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.613 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.613 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.613 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.613 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.613 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.613 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.613 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.614 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.614 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.614 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.614 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.614 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.614 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.614 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.614 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.614 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.614 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.614 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.614 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.614 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.615 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.615 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.615 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.615 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.615 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.615 INFO:teuthology.orchestra.run.smithi002.stdout: "roption1": { 2024-05-14T17:15:46.615 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "roption1", 2024-05-14T17:15:46.615 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.615 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.615 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.615 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.615 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.615 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.616 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.616 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.616 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.616 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.616 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.616 INFO:teuthology.orchestra.run.smithi002.stdout: "roption2": { 2024-05-14T17:15:46.616 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "roption2", 2024-05-14T17:15:46.616 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.616 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.616 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.616 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "xyz", 2024-05-14T17:15:46.616 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.616 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.617 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.617 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.617 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.617 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.617 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.617 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption1": { 2024-05-14T17:15:46.617 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption1", 2024-05-14T17:15:46.617 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.617 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.617 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.617 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.617 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.617 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.617 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.618 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.618 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.618 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.618 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.618 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption2": { 2024-05-14T17:15:46.618 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption2", 2024-05-14T17:15:46.618 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.618 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.618 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.618 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.618 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.618 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.618 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.619 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.619 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.619 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.619 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.619 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption3": { 2024-05-14T17:15:46.619 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption3", 2024-05-14T17:15:46.619 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T17:15:46.619 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.619 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.619 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.619 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.619 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.619 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.620 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.620 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.620 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.620 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.620 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption4": { 2024-05-14T17:15:46.620 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption4", 2024-05-14T17:15:46.620 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.620 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.620 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.620 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.620 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.620 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.620 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.620 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.620 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.621 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.621 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.621 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption5": { 2024-05-14T17:15:46.621 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption5", 2024-05-14T17:15:46.621 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.621 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.621 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.621 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.621 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.621 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.621 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.621 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.621 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.621 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.622 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.622 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption6": { 2024-05-14T17:15:46.622 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption6", 2024-05-14T17:15:46.622 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.622 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.622 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.622 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.622 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.622 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.622 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.622 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.622 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.622 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.623 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.623 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption7": { 2024-05-14T17:15:46.623 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption7", 2024-05-14T17:15:46.623 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.623 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.623 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.623 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.623 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-05-14T17:15:46.623 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "42", 2024-05-14T17:15:46.623 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.623 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.623 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.623 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.623 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.624 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.624 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.624 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.624 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.624 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.624 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.624 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.624 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.624 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.624 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.624 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.624 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.624 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.624 INFO:teuthology.orchestra.run.smithi002.stdout: "testkey": { 2024-05-14T17:15:46.625 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testkey", 2024-05-14T17:15:46.625 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.625 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.625 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.625 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.625 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.625 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.625 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.625 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.625 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.625 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.625 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.625 INFO:teuthology.orchestra.run.smithi002.stdout: "testlkey": { 2024-05-14T17:15:46.626 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testlkey", 2024-05-14T17:15:46.626 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.626 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.626 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.626 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.626 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.626 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.626 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.626 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.626 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.626 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.626 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.626 INFO:teuthology.orchestra.run.smithi002.stdout: "testnewline": { 2024-05-14T17:15:46.626 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testnewline", 2024-05-14T17:15:46.627 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.627 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.627 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.627 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.627 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.627 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.627 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.627 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.627 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.627 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.627 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.627 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.627 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.628 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smb", 2024-05-14T17:15:46.628 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.628 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.628 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.628 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.628 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.628 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.628 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.628 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.628 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.628 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.628 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.628 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.628 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.628 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.629 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.630 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.630 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.630 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.630 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.630 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.630 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.630 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.630 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.630 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.630 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.630 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.630 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.630 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.631 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.631 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.631 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.631 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.631 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.631 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.631 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.631 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.631 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.631 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.631 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.631 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.631 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.631 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.632 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.632 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.632 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.632 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.632 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.632 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.632 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.632 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.632 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.632 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.632 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.632 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.632 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.632 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.634 INFO:teuthology.orchestra.run.smithi002.stdout: "update_orchestration": { 2024-05-14T17:15:46.634 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "update_orchestration", 2024-05-14T17:15:46.634 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.634 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.634 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.634 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.634 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.634 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.634 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.634 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-14T17:15:46.634 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.634 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.634 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snap_schedule", 2024-05-14T17:15:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_m_granularity": { 2024-05-14T17:15:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_m_granularity", 2024-05-14T17:15:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.635 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow minute scheduled snapshots", 2024-05-14T17:15:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: "dump_on_update": { 2024-05-14T17:15:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "dump_on_update", 2024-05-14T17:15:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.636 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "dump database to debug log on update", 2024-05-14T17:15:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.637 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.638 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.638 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.638 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.638 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.638 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.638 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.638 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.638 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.638 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.638 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.638 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.638 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.638 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.638 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.639 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.640 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.640 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.640 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.640 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.640 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.640 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.640 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.640 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.640 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.640 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.640 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.640 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.640 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.640 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.640 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.641 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.641 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.641 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.641 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.641 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.641 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.641 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.641 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.641 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.641 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.641 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.642 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.642 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.642 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.642 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.642 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.642 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.642 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.642 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.642 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.642 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.642 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.642 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.642 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.642 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.643 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "stats", 2024-05-14T17:15:46.643 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.643 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.643 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.643 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.643 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.643 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.643 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.643 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.643 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.643 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.643 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.643 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.643 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.643 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.644 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.644 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.644 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.644 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.644 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.644 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.644 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.644 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.644 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.644 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.644 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.644 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.644 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.645 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.645 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.645 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.645 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.645 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.645 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.645 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.645 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.645 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.645 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.645 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.645 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.646 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.646 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.646 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.646 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.646 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.646 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.646 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.646 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.646 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.646 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.646 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.646 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.646 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.647 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.648 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.648 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.648 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.648 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.648 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.648 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.648 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.648 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.648 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.648 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.648 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.648 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.648 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "status", 2024-05-14T17:15:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.649 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.650 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.650 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.650 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.650 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.650 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.650 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.650 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.650 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.650 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.650 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.650 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.650 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.650 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.650 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.650 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.650 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.651 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.651 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.651 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.651 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.651 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.651 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.651 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.651 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.651 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.651 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.651 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.651 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.651 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.652 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.652 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.652 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.652 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.652 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.652 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.652 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.652 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.652 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.652 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.652 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.652 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.652 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.652 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.653 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.653 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.653 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.653 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.653 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.653 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.653 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.653 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.653 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.653 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.653 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.653 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.653 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.653 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.654 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.654 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.654 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.654 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.654 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.654 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.654 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.654 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.654 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.654 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.654 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.654 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.654 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.654 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.655 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.655 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.655 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.655 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.655 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.655 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.655 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "telegraf", 2024-05-14T17:15:46.655 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.655 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.655 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.655 INFO:teuthology.orchestra.run.smithi002.stdout: "address": { 2024-05-14T17:15:46.655 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "address", 2024-05-14T17:15:46.655 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.656 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.656 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.656 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-14T17:15:46.656 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.656 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.656 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.656 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.656 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.656 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.656 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.656 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-05-14T17:15:46.656 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-05-14T17:15:46.656 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.656 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.656 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.657 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15", 2024-05-14T17:15:46.657 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.657 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.657 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.657 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.657 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.657 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.657 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.657 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.657 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.657 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.657 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.657 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.657 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.658 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.658 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.658 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.658 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.658 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.658 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.658 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.658 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.658 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.658 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.658 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.658 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.658 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.658 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.659 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.659 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.659 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.659 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.659 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.659 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.659 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.659 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.659 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.659 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.659 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.659 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.659 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.659 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.660 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.660 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.660 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.660 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.660 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.660 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.660 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.660 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.660 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.660 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.660 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.660 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.660 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.660 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.660 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.661 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.661 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.661 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.661 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.661 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.661 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.661 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.661 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.661 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.661 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.661 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.661 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.661 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.661 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.662 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.662 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.662 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.662 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.662 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.662 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.662 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.662 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.662 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.662 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.662 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.662 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.662 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.662 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.663 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.663 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.663 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.663 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.663 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "telemetry", 2024-05-14T17:15:46.663 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.663 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.663 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.663 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_basic": { 2024-05-14T17:15:46.663 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_basic", 2024-05-14T17:15:46.663 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.663 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.663 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-14T17:15:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_crash": { 2024-05-14T17:15:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_crash", 2024-05-14T17:15:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.664 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.665 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.665 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.665 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-14T17:15:46.665 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.665 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.665 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.665 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_device": { 2024-05-14T17:15:46.665 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_device", 2024-05-14T17:15:46.665 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.665 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.665 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.665 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.665 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.665 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.666 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.666 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-14T17:15:46.666 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.666 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.666 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.666 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_ident": { 2024-05-14T17:15:46.666 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_ident", 2024-05-14T17:15:46.666 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.666 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.666 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.666 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.666 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.666 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.666 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.667 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-14T17:15:46.667 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.667 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.667 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.667 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_perf": { 2024-05-14T17:15:46.667 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_perf", 2024-05-14T17:15:46.667 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.667 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.667 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.667 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.667 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.667 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.667 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.668 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-14T17:15:46.668 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.668 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.668 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.668 INFO:teuthology.orchestra.run.smithi002.stdout: "contact": { 2024-05-14T17:15:46.668 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "contact", 2024-05-14T17:15:46.668 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.668 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.668 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.668 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.668 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.668 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.668 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.669 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.669 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.669 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.669 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.669 INFO:teuthology.orchestra.run.smithi002.stdout: "description": { 2024-05-14T17:15:46.669 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "description", 2024-05-14T17:15:46.669 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.669 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.669 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.669 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.669 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.669 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.669 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.670 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.670 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.670 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.670 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.670 INFO:teuthology.orchestra.run.smithi002.stdout: "device_url": { 2024-05-14T17:15:46.670 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_url", 2024-05-14T17:15:46.670 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.670 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.670 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.670 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-14T17:15:46.670 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.670 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.670 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.671 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.671 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.671 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.671 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.671 INFO:teuthology.orchestra.run.smithi002.stdout: "enabled": { 2024-05-14T17:15:46.671 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enabled", 2024-05-14T17:15:46.671 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.671 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.671 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.671 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.671 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.671 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.671 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.671 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.671 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-05-14T17:15:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-05-14T17:15:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "24", 2024-05-14T17:15:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "8", 2024-05-14T17:15:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.672 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.673 INFO:teuthology.orchestra.run.smithi002.stdout: "last_opt_revision": { 2024-05-14T17:15:46.673 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "last_opt_revision", 2024-05-14T17:15:46.673 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.673 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.673 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.673 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1", 2024-05-14T17:15:46.673 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.673 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.673 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.673 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.673 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.673 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.673 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.673 INFO:teuthology.orchestra.run.smithi002.stdout: "leaderboard": { 2024-05-14T17:15:46.674 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "leaderboard", 2024-05-14T17:15:46.674 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.674 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.674 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.674 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.674 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.674 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.674 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.674 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.674 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.674 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.674 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.674 INFO:teuthology.orchestra.run.smithi002.stdout: "leaderboard_description": { 2024-05-14T17:15:46.675 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "leaderboard_description", 2024-05-14T17:15:46.675 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.675 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.675 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.675 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.675 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.675 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.675 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.675 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.675 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.675 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.675 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.675 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.675 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.676 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.677 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.677 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.677 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.677 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.677 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.677 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.677 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.677 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.677 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.677 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.677 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.677 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.677 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.677 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.677 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.678 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.678 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.678 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.678 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.678 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.678 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.678 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.678 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.678 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.678 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.678 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.678 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.678 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.678 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.678 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.679 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: "organization": { 2024-05-14T17:15:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "organization", 2024-05-14T17:15:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.680 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.681 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.681 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.681 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.681 INFO:teuthology.orchestra.run.smithi002.stdout: "proxy": { 2024-05-14T17:15:46.681 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "proxy", 2024-05-14T17:15:46.681 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.681 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.681 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.681 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.681 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.681 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.681 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.682 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.682 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.682 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.682 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.682 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.682 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.682 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.682 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.682 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.682 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.682 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.682 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.683 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.683 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.683 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.683 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.683 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.683 INFO:teuthology.orchestra.run.smithi002.stdout: "url": { 2024-05-14T17:15:46.683 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "url", 2024-05-14T17:15:46.683 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.683 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.683 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.683 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-14T17:15:46.683 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.684 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.684 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.684 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.684 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.684 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.684 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.684 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.684 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.684 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "test_orchestrator", 2024-05-14T17:15:46.684 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.684 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.685 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.685 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.685 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.685 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.685 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.685 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.685 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.685 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.685 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.685 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.685 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.685 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.685 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.685 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.685 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.686 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.686 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.686 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.686 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.686 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.686 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.686 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.686 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.686 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.686 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.686 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.686 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.686 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.687 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.687 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.687 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.687 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.687 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.687 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.687 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.687 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.687 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.687 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.687 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.687 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.688 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.688 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.688 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.688 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.688 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.688 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.688 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.688 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.688 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.688 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.688 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.688 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.688 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.688 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.689 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.689 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.689 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.689 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.689 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.689 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.689 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.689 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.689 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.689 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.689 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.689 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.689 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.690 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.690 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.690 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.690 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.690 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.690 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.690 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.690 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.690 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.690 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.690 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.690 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.690 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.690 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.691 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.691 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "volumes", 2024-05-14T17:15:46.691 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.691 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.691 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.691 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.691 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.691 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.691 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.691 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.691 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.691 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.692 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.692 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.692 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.692 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.692 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.692 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.692 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.692 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.692 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.692 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.692 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.692 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.692 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.693 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.693 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.693 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.693 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.693 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.693 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.693 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.693 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.693 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.693 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.693 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.693 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.693 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.694 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.694 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.694 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.694 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.694 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.694 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.694 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.694 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.694 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.694 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.694 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.694 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.694 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.695 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.695 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.695 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.695 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.695 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.695 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.695 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.695 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.695 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.695 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.695 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.695 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.696 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.696 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.696 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.696 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.696 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.696 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.696 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.696 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.696 INFO:teuthology.orchestra.run.smithi002.stdout: "max_concurrent_clones": { 2024-05-14T17:15:46.696 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_concurrent_clones", 2024-05-14T17:15:46.696 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.696 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.696 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.696 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "4", 2024-05-14T17:15:46.696 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.697 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.697 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.697 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-14T17:15:46.697 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.697 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.697 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.697 INFO:teuthology.orchestra.run.smithi002.stdout: "periodic_async_work": { 2024-05-14T17:15:46.697 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "periodic_async_work", 2024-05-14T17:15:46.697 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.697 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.697 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.697 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.697 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.698 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.698 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.698 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Periodically check for async work", 2024-05-14T17:15:46.698 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.698 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.698 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.698 INFO:teuthology.orchestra.run.smithi002.stdout: "snapshot_clone_delay": { 2024-05-14T17:15:46.698 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snapshot_clone_delay", 2024-05-14T17:15:46.698 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.698 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.698 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.698 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.699 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.699 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.699 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.699 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-14T17:15:46.699 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.699 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.699 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.699 INFO:teuthology.orchestra.run.smithi002.stdout: "snapshot_clone_no_wait": { 2024-05-14T17:15:46.699 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snapshot_clone_no_wait", 2024-05-14T17:15:46.699 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.699 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.699 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.699 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.699 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.700 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.700 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.700 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-14T17:15:46.700 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.700 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.700 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.700 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.700 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.700 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.700 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.700 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.700 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.700 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.701 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.701 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.701 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.701 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.701 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.701 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.701 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.701 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.701 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix", 2024-05-14T17:15:46.701 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.701 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.701 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.701 INFO:teuthology.orchestra.run.smithi002.stdout: "discovery_interval": { 2024-05-14T17:15:46.702 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "discovery_interval", 2024-05-14T17:15:46.702 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-05-14T17:15:46.702 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.702 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.702 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "100", 2024-05-14T17:15:46.702 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.702 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.702 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.702 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.702 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.702 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.702 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.702 INFO:teuthology.orchestra.run.smithi002.stdout: "identifier": { 2024-05-14T17:15:46.703 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "identifier", 2024-05-14T17:15:46.703 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.703 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.703 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.703 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.703 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.703 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.703 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.703 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.703 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.703 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.703 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.703 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-05-14T17:15:46.704 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-05-14T17:15:46.704 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.704 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.704 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.704 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-05-14T17:15:46.704 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.704 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.704 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.704 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.704 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.704 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.704 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.704 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.705 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.705 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.705 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.705 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.705 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.705 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.705 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.705 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.705 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.705 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.705 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.705 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.705 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.705 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.705 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.706 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.706 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.706 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.706 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.706 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.706 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.706 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.706 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.706 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.706 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.706 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.706 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.707 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.707 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.707 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.707 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.707 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.707 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.707 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.707 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.707 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.707 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.707 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.708 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.708 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.708 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.708 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.708 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.708 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.708 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.708 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.708 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.708 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.708 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.708 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.708 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.708 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.709 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.709 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.709 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.709 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.709 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.709 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.709 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.709 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.709 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.709 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.709 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.709 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.709 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.710 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.710 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.710 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.710 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.710 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.710 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.710 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.710 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.710 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.710 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.710 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.710 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.710 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.711 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_host": { 2024-05-14T17:15:46.711 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_host", 2024-05-14T17:15:46.711 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.711 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.711 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.711 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.711 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.711 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.711 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.711 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.711 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.711 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.711 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.712 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_port": { 2024-05-14T17:15:46.712 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_port", 2024-05-14T17:15:46.712 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.712 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.712 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.712 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10051", 2024-05-14T17:15:46.712 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.712 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.712 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.712 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.712 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.712 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.713 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.713 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_sender": { 2024-05-14T17:15:46.713 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_sender", 2024-05-14T17:15:46.713 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.713 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.713 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.713 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-14T17:15:46.713 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.713 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.713 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.713 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.713 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.713 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.713 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.714 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.714 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.714 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.714 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-05-14T17:15:46.714 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.714 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.714 INFO:teuthology.orchestra.run.smithi002.stdout: "modules": [ 2024-05-14T17:15:46.714 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm", 2024-05-14T17:15:46.714 INFO:teuthology.orchestra.run.smithi002.stdout: "dashboard", 2024-05-14T17:15:46.714 INFO:teuthology.orchestra.run.smithi002.stdout: "iostat", 2024-05-14T17:15:46.714 INFO:teuthology.orchestra.run.smithi002.stdout: "nfs", 2024-05-14T17:15:46.714 INFO:teuthology.orchestra.run.smithi002.stdout: "restful", 2024-05-14T17:15:46.715 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_schedule" 2024-05-14T17:15:46.715 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.715 INFO:teuthology.orchestra.run.smithi002.stdout: "available_modules": [ 2024-05-14T17:15:46.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.715 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "alerts", 2024-05-14T17:15:46.715 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.715 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.715 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.715 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-05-14T17:15:46.715 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-05-14T17:15:46.715 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.715 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.715 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.715 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-05-14T17:15:46.715 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.715 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.716 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.716 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How frequently to reexamine health status", 2024-05-14T17:15:46.716 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.716 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.716 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.716 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.716 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.716 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.716 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.716 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.716 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.716 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.716 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.716 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.716 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.717 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.717 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.717 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.717 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.717 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.717 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.717 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.717 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.717 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.717 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.717 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.717 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.717 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.717 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.717 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.718 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.718 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.718 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.718 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.718 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.718 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.718 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.718 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.718 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.718 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.718 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.718 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.718 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.718 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.719 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.719 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.719 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.719 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.719 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.719 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.719 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.719 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.719 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.719 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.719 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.719 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.719 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.719 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.719 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.719 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.720 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.720 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.720 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.720 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.720 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.720 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.720 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.720 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.720 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.720 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.720 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.720 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_destination": { 2024-05-14T17:15:46.720 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_destination", 2024-05-14T17:15:46.721 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.721 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.721 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.721 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.721 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.721 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.721 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.721 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Email address to send alerts to", 2024-05-14T17:15:46.721 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.721 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.721 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.721 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_from_name": { 2024-05-14T17:15:46.721 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_from_name", 2024-05-14T17:15:46.721 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.721 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.722 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.722 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "Ceph", 2024-05-14T17:15:46.722 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.722 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.722 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.722 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Email From: name", 2024-05-14T17:15:46.722 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.722 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.722 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.722 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_host": { 2024-05-14T17:15:46.722 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_host", 2024-05-14T17:15:46.722 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.722 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.723 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.723 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.723 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.723 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.723 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.723 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP server", 2024-05-14T17:15:46.723 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.723 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.723 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.723 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_password": { 2024-05-14T17:15:46.723 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_password", 2024-05-14T17:15:46.723 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.723 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.724 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.724 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.724 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.724 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.724 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.724 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Password to authenticate with", 2024-05-14T17:15:46.724 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.724 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.724 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.724 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_port": { 2024-05-14T17:15:46.724 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_port", 2024-05-14T17:15:46.724 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.724 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.724 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.724 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "465", 2024-05-14T17:15:46.725 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.725 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.725 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.725 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP port", 2024-05-14T17:15:46.725 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.725 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.725 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.725 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_sender": { 2024-05-14T17:15:46.725 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_sender", 2024-05-14T17:15:46.725 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.725 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.725 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.725 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.725 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.726 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.726 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.726 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP envelope sender", 2024-05-14T17:15:46.726 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.726 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.726 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.726 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_ssl": { 2024-05-14T17:15:46.726 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_ssl", 2024-05-14T17:15:46.726 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.726 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.726 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.726 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.726 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.726 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.727 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.727 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-14T17:15:46.727 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.727 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.727 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.727 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_user": { 2024-05-14T17:15:46.727 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_user", 2024-05-14T17:15:46.727 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.727 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.727 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.727 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.727 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.727 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.728 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.728 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "User to authenticate as", 2024-05-14T17:15:46.728 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.728 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.728 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.728 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.728 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.728 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.728 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.728 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.728 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.728 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.728 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.729 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.729 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.729 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.729 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.729 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.729 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.729 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.729 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "balancer", 2024-05-14T17:15:46.729 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.729 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.729 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.729 INFO:teuthology.orchestra.run.smithi002.stdout: "active": { 2024-05-14T17:15:46.729 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "active", 2024-05-14T17:15:46.729 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.730 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.730 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.730 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.730 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.730 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.730 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.730 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically balance PGs across cluster", 2024-05-14T17:15:46.730 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.730 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.730 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.730 INFO:teuthology.orchestra.run.smithi002.stdout: "begin_time": { 2024-05-14T17:15:46.730 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "begin_time", 2024-05-14T17:15:46.730 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.730 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.731 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.731 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0000", 2024-05-14T17:15:46.731 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.731 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.731 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.731 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "beginning time of day to automatically balance", 2024-05-14T17:15:46.731 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-14T17:15:46.731 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.731 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.731 INFO:teuthology.orchestra.run.smithi002.stdout: "begin_weekday": { 2024-05-14T17:15:46.731 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "begin_weekday", 2024-05-14T17:15:46.731 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-05-14T17:15:46.731 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.732 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.732 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.732 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0", 2024-05-14T17:15:46.732 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "6", 2024-05-14T17:15:46.732 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.732 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-14T17:15:46.732 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-14T17:15:46.732 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.732 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.732 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_max_iterations": { 2024-05-14T17:15:46.732 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_max_iterations", 2024-05-14T17:15:46.732 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-05-14T17:15:46.732 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.733 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.733 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "25", 2024-05-14T17:15:46.733 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-05-14T17:15:46.733 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "250", 2024-05-14T17:15:46.733 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.733 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-14T17:15:46.733 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.733 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.733 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.733 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_metrics": { 2024-05-14T17:15:46.733 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_metrics", 2024-05-14T17:15:46.733 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.733 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.734 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.734 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "pgs,objects,bytes", 2024-05-14T17:15:46.734 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.734 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.734 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.734 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-14T17:15:46.734 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-14T17:15:46.734 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.734 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.734 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_step": { 2024-05-14T17:15:46.734 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_step", 2024-05-14T17:15:46.734 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T17:15:46.735 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.735 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.735 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0.5", 2024-05-14T17:15:46.735 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0.001", 2024-05-14T17:15:46.735 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "0.999", 2024-05-14T17:15:46.735 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.735 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "aggressiveness of optimization", 2024-05-14T17:15:46.735 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-14T17:15:46.735 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.735 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.735 INFO:teuthology.orchestra.run.smithi002.stdout: "end_time": { 2024-05-14T17:15:46.735 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "end_time", 2024-05-14T17:15:46.735 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.736 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.736 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.736 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "2359", 2024-05-14T17:15:46.736 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.736 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.736 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.736 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "ending time of day to automatically balance", 2024-05-14T17:15:46.736 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-14T17:15:46.736 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.736 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.736 INFO:teuthology.orchestra.run.smithi002.stdout: "end_weekday": { 2024-05-14T17:15:46.736 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "end_weekday", 2024-05-14T17:15:46.736 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-05-14T17:15:46.737 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.737 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.737 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.737 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0", 2024-05-14T17:15:46.737 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "6", 2024-05-14T17:15:46.737 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.737 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-14T17:15:46.737 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-14T17:15:46.737 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.737 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.737 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.737 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.738 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.738 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.738 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.738 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.738 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.738 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.738 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.738 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.738 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.738 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.738 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.738 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.738 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.739 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.739 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.739 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.739 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.739 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.739 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.739 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.739 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.739 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.739 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.739 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.739 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.739 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.740 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.740 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.740 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.740 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.740 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.740 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.740 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.740 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.740 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.740 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.740 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.740 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.740 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.741 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.741 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.741 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.741 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.741 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.741 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.741 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.741 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.741 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.741 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.741 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.741 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.741 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.742 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.742 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.742 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.742 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.742 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.742 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.742 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.742 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.742 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.742 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.742 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.742 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.742 INFO:teuthology.orchestra.run.smithi002.stdout: "min_score": { 2024-05-14T17:15:46.743 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "min_score", 2024-05-14T17:15:46.743 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T17:15:46.743 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.743 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.743 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.743 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.743 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.743 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.743 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-14T17:15:46.743 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.743 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.743 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.743 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": { 2024-05-14T17:15:46.743 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mode", 2024-05-14T17:15:46.743 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.744 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.744 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.744 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "upmap", 2024-05-14T17:15:46.744 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.744 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.744 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.744 INFO:teuthology.orchestra.run.smithi002.stdout: "crush-compat", 2024-05-14T17:15:46.744 INFO:teuthology.orchestra.run.smithi002.stdout: "none", 2024-05-14T17:15:46.744 INFO:teuthology.orchestra.run.smithi002.stdout: "read", 2024-05-14T17:15:46.744 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap", 2024-05-14T17:15:46.744 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap-read" 2024-05-14T17:15:46.744 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.744 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Balancer mode", 2024-05-14T17:15:46.744 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.744 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.745 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.745 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_ids": { 2024-05-14T17:15:46.745 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pool_ids", 2024-05-14T17:15:46.745 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.745 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.745 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.745 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.745 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.745 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.745 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.745 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-14T17:15:46.745 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.745 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.745 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.746 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-05-14T17:15:46.746 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-05-14T17:15:46.746 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.746 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.746 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.746 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-05-14T17:15:46.746 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.746 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.746 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.746 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-14T17:15:46.746 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.746 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.747 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.747 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.747 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.747 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.747 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.747 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.747 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.747 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.747 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.747 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.747 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.747 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.747 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.747 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.748 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap_max_deviation": { 2024-05-14T17:15:46.748 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "upmap_max_deviation", 2024-05-14T17:15:46.748 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.748 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.748 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.748 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-05-14T17:15:46.748 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-05-14T17:15:46.748 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.748 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.748 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-14T17:15:46.748 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-14T17:15:46.748 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.748 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.748 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap_max_optimizations": { 2024-05-14T17:15:46.749 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "upmap_max_optimizations", 2024-05-14T17:15:46.749 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-05-14T17:15:46.749 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.749 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.749 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T17:15:46.749 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.749 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.749 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.749 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-14T17:15:46.749 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.749 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.749 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.749 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.749 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.750 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cephadm", 2024-05-14T17:15:46.750 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.750 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.750 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.750 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_down_multiplier": { 2024-05-14T17:15:46.750 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_down_multiplier", 2024-05-14T17:15:46.750 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T17:15:46.750 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.750 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.750 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3.0", 2024-05-14T17:15:46.750 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.750 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.750 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.751 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-14T17:15:46.751 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.751 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.751 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.751 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_refresh_rate": { 2024-05-14T17:15:46.751 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_refresh_rate", 2024-05-14T17:15:46.751 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.751 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.751 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.751 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "20", 2024-05-14T17:15:46.751 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.751 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.751 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.752 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-14T17:15:46.752 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.752 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.752 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.752 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_starting_port": { 2024-05-14T17:15:46.752 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_starting_port", 2024-05-14T17:15:46.752 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.752 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.752 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.752 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "4721", 2024-05-14T17:15:46.752 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.752 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.752 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.753 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-14T17:15:46.753 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.753 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.753 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.753 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_ptrace": { 2024-05-14T17:15:46.753 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_ptrace", 2024-05-14T17:15:46.753 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.753 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.753 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.753 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.753 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.753 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.753 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.753 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-14T17:15:46.754 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-14T17:15:46.754 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.754 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.754 INFO:teuthology.orchestra.run.smithi002.stdout: "autotune_interval": { 2024-05-14T17:15:46.754 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "autotune_interval", 2024-05-14T17:15:46.754 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.754 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.754 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.754 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-05-14T17:15:46.754 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.754 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.754 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.754 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-14T17:15:46.755 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.755 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.755 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.755 INFO:teuthology.orchestra.run.smithi002.stdout: "autotune_memory_target_ratio": { 2024-05-14T17:15:46.755 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "autotune_memory_target_ratio", 2024-05-14T17:15:46.755 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T17:15:46.755 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.755 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.755 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0.7", 2024-05-14T17:15:46.755 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.755 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.755 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.755 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-14T17:15:46.756 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.756 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.756 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.756 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm_log_destination": { 2024-05-14T17:15:46.756 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cephadm_log_destination", 2024-05-14T17:15:46.756 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.756 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.756 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.756 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.756 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.756 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.756 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.756 INFO:teuthology.orchestra.run.smithi002.stdout: "file", 2024-05-14T17:15:46.756 INFO:teuthology.orchestra.run.smithi002.stdout: "file,syslog", 2024-05-14T17:15:46.757 INFO:teuthology.orchestra.run.smithi002.stdout: "syslog" 2024-05-14T17:15:46.757 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.757 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-14T17:15:46.757 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.757 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.757 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.757 INFO:teuthology.orchestra.run.smithi002.stdout: "cgroups_split": { 2024-05-14T17:15:46.757 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cgroups_split", 2024-05-14T17:15:46.757 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.757 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.757 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.757 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.757 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.757 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.757 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.758 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-14T17:15:46.758 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.758 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.758 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.758 INFO:teuthology.orchestra.run.smithi002.stdout: "config_checks_enabled": { 2024-05-14T17:15:46.758 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_checks_enabled", 2024-05-14T17:15:46.758 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.758 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.758 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.758 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.758 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.758 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.758 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.759 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-14T17:15:46.759 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.759 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.759 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.759 INFO:teuthology.orchestra.run.smithi002.stdout: "config_dashboard": { 2024-05-14T17:15:46.759 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_dashboard", 2024-05-14T17:15:46.759 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.759 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.759 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.759 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.759 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.759 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.759 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.759 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-14T17:15:46.759 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.760 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.760 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.760 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_alertmanager": { 2024-05-14T17:15:46.760 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_alertmanager", 2024-05-14T17:15:46.760 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.760 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.760 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.760 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-14T17:15:46.760 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.760 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.760 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.760 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-05-14T17:15:46.760 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.760 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.761 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.761 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_base": { 2024-05-14T17:15:46.761 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_base", 2024-05-14T17:15:46.761 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.761 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.761 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.761 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-14T17:15:46.761 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.761 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.761 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.761 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Container image name, without the tag", 2024-05-14T17:15:46.761 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.761 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.761 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.762 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_elasticsearch": { 2024-05-14T17:15:46.762 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_elasticsearch", 2024-05-14T17:15:46.762 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.762 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.762 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.762 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-14T17:15:46.762 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.762 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.762 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.762 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "elasticsearch container image", 2024-05-14T17:15:46.762 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.762 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.762 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.762 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_grafana": { 2024-05-14T17:15:46.763 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_grafana", 2024-05-14T17:15:46.763 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.763 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.763 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.763 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-14T17:15:46.763 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.763 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.763 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.763 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-05-14T17:15:46.763 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.763 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.763 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.763 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_haproxy": { 2024-05-14T17:15:46.763 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_haproxy", 2024-05-14T17:15:46.763 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.764 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.764 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.764 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-14T17:15:46.764 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.764 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.764 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.764 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "HAproxy container image", 2024-05-14T17:15:46.764 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.764 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.764 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.764 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_agent": { 2024-05-14T17:15:46.764 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_agent", 2024-05-14T17:15:46.764 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.764 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.765 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.765 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-14T17:15:46.765 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.765 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.765 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.765 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger agent container image", 2024-05-14T17:15:46.765 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.765 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.765 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.765 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_collector": { 2024-05-14T17:15:46.765 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_collector", 2024-05-14T17:15:46.765 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.765 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.765 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.765 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-14T17:15:46.765 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.766 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.766 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.766 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger collector container image", 2024-05-14T17:15:46.766 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.766 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.766 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.766 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_query": { 2024-05-14T17:15:46.766 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_query", 2024-05-14T17:15:46.766 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.766 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.766 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.766 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-14T17:15:46.767 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.767 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.767 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.767 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger query container image", 2024-05-14T17:15:46.767 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.767 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.767 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.767 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_keepalived": { 2024-05-14T17:15:46.767 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_keepalived", 2024-05-14T17:15:46.767 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.767 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.767 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.767 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-14T17:15:46.767 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.767 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.768 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.768 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Keepalived container image", 2024-05-14T17:15:46.768 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.768 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.768 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.768 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_loki": { 2024-05-14T17:15:46.768 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_loki", 2024-05-14T17:15:46.768 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.768 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.768 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.768 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-14T17:15:46.768 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.768 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.768 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.768 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Loki container image", 2024-05-14T17:15:46.769 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.769 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.769 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.769 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_node_exporter": { 2024-05-14T17:15:46.769 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_node_exporter", 2024-05-14T17:15:46.769 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.769 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.769 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.769 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-14T17:15:46.769 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.769 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.769 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.769 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-05-14T17:15:46.770 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.770 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.770 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.770 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_nvmeof": { 2024-05-14T17:15:46.770 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_nvmeof", 2024-05-14T17:15:46.770 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.770 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.770 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.770 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-14T17:15:46.770 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.770 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.770 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.770 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Nvme-of container image", 2024-05-14T17:15:46.770 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.771 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.771 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.771 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_prometheus": { 2024-05-14T17:15:46.771 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_prometheus", 2024-05-14T17:15:46.771 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.771 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.771 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.771 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-14T17:15:46.771 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.771 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.771 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.772 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2024-05-14T17:15:46.772 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.772 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.772 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.772 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_promtail": { 2024-05-14T17:15:46.772 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_promtail", 2024-05-14T17:15:46.772 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.772 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.772 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.772 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-14T17:15:46.772 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.772 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.772 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.773 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Promtail container image", 2024-05-14T17:15:46.773 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.773 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.773 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.773 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_samba": { 2024-05-14T17:15:46.773 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_samba", 2024-05-14T17:15:46.773 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.773 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.773 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.773 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-14T17:15:46.773 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.773 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.773 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.773 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Samba/SMB container image", 2024-05-14T17:15:46.774 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.774 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.774 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.774 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_snmp_gateway": { 2024-05-14T17:15:46.774 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_snmp_gateway", 2024-05-14T17:15:46.774 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.774 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.774 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.774 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-14T17:15:46.774 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.774 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.774 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.774 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SNMP Gateway container image", 2024-05-14T17:15:46.775 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.775 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.775 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.775 INFO:teuthology.orchestra.run.smithi002.stdout: "container_init": { 2024-05-14T17:15:46.775 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_init", 2024-05-14T17:15:46.775 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.775 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.775 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.775 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.775 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.775 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.775 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.775 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Run podman/docker with `--init`", 2024-05-14T17:15:46.775 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.776 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.776 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.776 INFO:teuthology.orchestra.run.smithi002.stdout: "daemon_cache_timeout": { 2024-05-14T17:15:46.776 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "daemon_cache_timeout", 2024-05-14T17:15:46.776 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.776 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.776 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.776 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-05-14T17:15:46.776 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.776 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.776 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.776 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-14T17:15:46.776 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.776 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.777 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.777 INFO:teuthology.orchestra.run.smithi002.stdout: "default_cephadm_command_timeout": { 2024-05-14T17:15:46.777 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "default_cephadm_command_timeout", 2024-05-14T17:15:46.777 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.777 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.777 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.777 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "900", 2024-05-14T17:15:46.777 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.777 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.777 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.777 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-14T17:15:46.777 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.777 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.777 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.778 INFO:teuthology.orchestra.run.smithi002.stdout: "default_registry": { 2024-05-14T17:15:46.778 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "default_registry", 2024-05-14T17:15:46.778 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.778 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.778 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.778 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io", 2024-05-14T17:15:46.778 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.778 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.778 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.778 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-14T17:15:46.778 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.778 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.778 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.778 INFO:teuthology.orchestra.run.smithi002.stdout: "device_cache_timeout": { 2024-05-14T17:15:46.778 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_cache_timeout", 2024-05-14T17:15:46.779 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.779 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.779 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.779 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1800", 2024-05-14T17:15:46.779 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.779 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.779 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.779 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache device inventory", 2024-05-14T17:15:46.779 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.779 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.779 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.779 INFO:teuthology.orchestra.run.smithi002.stdout: "device_enhanced_scan": { 2024-05-14T17:15:46.779 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_enhanced_scan", 2024-05-14T17:15:46.779 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-14T17:15:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: "facts_cache_timeout": { 2024-05-14T17:15:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "facts_cache_timeout", 2024-05-14T17:15:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.780 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-05-14T17:15:46.781 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.781 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.781 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.781 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache host facts data", 2024-05-14T17:15:46.781 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.781 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.781 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.781 INFO:teuthology.orchestra.run.smithi002.stdout: "grafana_dashboards_path": { 2024-05-14T17:15:46.781 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "grafana_dashboards_path", 2024-05-14T17:15:46.781 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.781 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.781 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.781 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-14T17:15:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-14T17:15:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: "host_check_interval": { 2024-05-14T17:15:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "host_check_interval", 2024-05-14T17:15:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.782 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-05-14T17:15:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to perform a host check", 2024-05-14T17:15:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "hw_monitoring": { 2024-05-14T17:15:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "hw_monitoring", 2024-05-14T17:15:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.783 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-14T17:15:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "inventory_list_all": { 2024-05-14T17:15:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "inventory_list_all", 2024-05-14T17:15:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.784 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.785 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-14T17:15:46.785 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.785 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.785 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.785 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.785 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.785 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.785 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.785 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.785 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.785 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.785 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.786 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.786 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.786 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.786 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.786 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.786 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.786 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.786 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.786 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.786 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.786 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.786 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.786 INFO:teuthology.orchestra.run.smithi002.stdout: "log_refresh_metadata": { 2024-05-14T17:15:46.786 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_refresh_metadata", 2024-05-14T17:15:46.786 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-14T17:15:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.787 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.788 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.788 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.788 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.788 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.788 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.788 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-14T17:15:46.788 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.788 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.788 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.788 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.788 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.788 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.788 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.788 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.789 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.789 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.789 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.789 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.789 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.789 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.789 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.789 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.789 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.789 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.789 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.789 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.789 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.789 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.789 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.790 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.790 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.790 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.790 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.790 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.790 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.790 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.790 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.790 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.790 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.790 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.790 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.790 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-14T17:15:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-14T17:15:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-14T17:15:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.792 INFO:teuthology.orchestra.run.smithi002.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-14T17:15:46.792 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-14T17:15:46.792 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.792 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.792 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.792 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "*", 2024-05-14T17:15:46.792 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.792 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.792 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.792 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-14T17:15:46.792 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.792 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.792 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.792 INFO:teuthology.orchestra.run.smithi002.stdout: "max_count_per_host": { 2024-05-14T17:15:46.792 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_count_per_host", 2024-05-14T17:15:46.793 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.793 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.793 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.793 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T17:15:46.793 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.793 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.793 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.793 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "max number of daemons per service per host", 2024-05-14T17:15:46.793 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.793 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.793 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.793 INFO:teuthology.orchestra.run.smithi002.stdout: "max_osd_draining_count": { 2024-05-14T17:15:46.793 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_osd_draining_count", 2024-05-14T17:15:46.794 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.794 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.794 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.794 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T17:15:46.794 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.794 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.794 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.794 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-14T17:15:46.794 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.794 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.794 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.795 INFO:teuthology.orchestra.run.smithi002.stdout: "migration_current": { 2024-05-14T17:15:46.795 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "migration_current", 2024-05-14T17:15:46.795 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.795 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.795 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.795 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.795 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.795 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.795 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.795 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "internal - do not modify", 2024-05-14T17:15:46.795 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.795 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.795 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.796 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": { 2024-05-14T17:15:46.796 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mode", 2024-05-14T17:15:46.796 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.796 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.796 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.796 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "root", 2024-05-14T17:15:46.796 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.796 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.796 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.796 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm-package", 2024-05-14T17:15:46.796 INFO:teuthology.orchestra.run.smithi002.stdout: "root" 2024-05-14T17:15:46.796 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.796 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "mode for remote execution of cephadm", 2024-05-14T17:15:46.796 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.796 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.797 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.797 INFO:teuthology.orchestra.run.smithi002.stdout: "oob_default_addr": { 2024-05-14T17:15:46.797 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "oob_default_addr", 2024-05-14T17:15:46.797 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.797 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.797 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.797 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "169.254.1.1", 2024-05-14T17:15:46.797 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.797 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.797 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.797 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-14T17:15:46.797 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.797 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.797 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.798 INFO:teuthology.orchestra.run.smithi002.stdout: "prometheus_alerts_path": { 2024-05-14T17:15:46.798 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus_alerts_path", 2024-05-14T17:15:46.798 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.798 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.798 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.798 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-14T17:15:46.798 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.798 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.798 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.798 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-14T17:15:46.798 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.798 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.798 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.799 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_insecure": { 2024-05-14T17:15:46.799 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_insecure", 2024-05-14T17:15:46.799 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.799 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.799 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.799 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.799 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.799 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.799 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.799 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-14T17:15:46.799 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.799 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.799 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.800 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_password": { 2024-05-14T17:15:46.800 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_password", 2024-05-14T17:15:46.800 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.800 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.800 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.800 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.800 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.800 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.800 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.800 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-14T17:15:46.800 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.800 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.800 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.801 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_url": { 2024-05-14T17:15:46.801 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_url", 2024-05-14T17:15:46.801 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.801 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.801 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.801 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.801 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.801 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.801 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.801 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-14T17:15:46.801 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.801 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.801 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.801 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_username": { 2024-05-14T17:15:46.802 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_username", 2024-05-14T17:15:46.802 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.802 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.802 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.802 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.802 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.802 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.802 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.802 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-14T17:15:46.802 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.802 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.802 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.802 INFO:teuthology.orchestra.run.smithi002.stdout: "secure_monitoring_stack": { 2024-05-14T17:15:46.803 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "secure_monitoring_stack", 2024-05-14T17:15:46.803 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.803 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.803 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.803 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.803 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.803 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.803 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.803 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-14T17:15:46.803 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.803 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.803 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.803 INFO:teuthology.orchestra.run.smithi002.stdout: "service_discovery_port": { 2024-05-14T17:15:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "service_discovery_port", 2024-05-14T17:15:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8765", 2024-05-14T17:15:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "cephadm service discovery port", 2024-05-14T17:15:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.804 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.805 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.805 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.805 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.805 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.805 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.805 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.805 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.805 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.805 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.805 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.805 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.805 INFO:teuthology.orchestra.run.smithi002.stdout: "ssh_config_file": { 2024-05-14T17:15:46.806 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssh_config_file", 2024-05-14T17:15:46.806 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.806 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.806 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.806 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.806 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.806 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.806 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.806 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-14T17:15:46.806 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.806 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.806 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.806 INFO:teuthology.orchestra.run.smithi002.stdout: "use_agent": { 2024-05-14T17:15:46.807 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "use_agent", 2024-05-14T17:15:46.807 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.807 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.807 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.807 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.807 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.807 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.807 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.807 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-14T17:15:46.807 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.807 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.807 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.807 INFO:teuthology.orchestra.run.smithi002.stdout: "use_repo_digest": { 2024-05-14T17:15:46.808 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "use_repo_digest", 2024-05-14T17:15:46.808 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.808 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.808 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.808 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.808 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.808 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.808 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.808 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-14T17:15:46.808 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.808 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.808 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_failed_host_check": { 2024-05-14T17:15:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_failed_host_check", 2024-05-14T17:15:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if the host check fails", 2024-05-14T17:15:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_stray_daemons": { 2024-05-14T17:15:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_stray_daemons", 2024-05-14T17:15:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-14T17:15:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.811 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_stray_hosts": { 2024-05-14T17:15:46.811 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_stray_hosts", 2024-05-14T17:15:46.811 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.811 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.811 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.811 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.811 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.811 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.811 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.811 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-14T17:15:46.811 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.811 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.811 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.811 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.812 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.812 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crash", 2024-05-14T17:15:46.812 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.812 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.812 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.812 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.812 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.812 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.812 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.812 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.812 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.812 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.813 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.814 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.814 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.814 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.814 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.814 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.814 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.814 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.814 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.814 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.814 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.814 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.814 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.815 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.816 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: "retain_interval": { 2024-05-14T17:15:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "retain_interval", 2024-05-14T17:15:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "31536000", 2024-05-14T17:15:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-14T17:15:46.817 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.818 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_recent_interval": { 2024-05-14T17:15:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_recent_interval", 2024-05-14T17:15:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1209600", 2024-05-14T17:15:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-14T17:15:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.819 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "dashboard", 2024-05-14T17:15:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-14T17:15:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-14T17:15:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T17:15:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.820 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: "ALERTMANAGER_API_HOST": { 2024-05-14T17:15:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-14T17:15:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.821 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.822 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.822 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.822 INFO:teuthology.orchestra.run.smithi002.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-14T17:15:46.822 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-14T17:15:46.822 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.822 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.822 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.822 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.822 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.822 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.822 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.822 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.822 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.822 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.823 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.823 INFO:teuthology.orchestra.run.smithi002.stdout: "AUDIT_API_ENABLED": { 2024-05-14T17:15:46.823 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "AUDIT_API_ENABLED", 2024-05-14T17:15:46.823 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.823 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.823 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.823 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.823 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.823 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.823 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.823 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.823 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.823 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.823 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.824 INFO:teuthology.orchestra.run.smithi002.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-14T17:15:46.824 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-14T17:15:46.824 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.824 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.824 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.824 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.824 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.824 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.824 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.824 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.824 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.824 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: "ENABLE_BROWSABLE_API": { 2024-05-14T17:15:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-14T17:15:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.826 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-14T17:15:46.826 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-14T17:15:46.826 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.826 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.826 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.826 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.826 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.826 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.826 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.826 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.826 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.826 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.826 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-14T17:15:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-14T17:15:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.828 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-14T17:15:46.828 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-14T17:15:46.828 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.828 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.828 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.828 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.828 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.828 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.828 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.828 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.828 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.828 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.828 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.829 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-14T17:15:46.829 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-14T17:15:46.829 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.829 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.829 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.829 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.829 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.829 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.829 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.829 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.829 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.829 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.829 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.830 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-14T17:15:46.830 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-14T17:15:46.830 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.830 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.830 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.830 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.830 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.830 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.830 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.830 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.830 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.830 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.830 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.831 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-14T17:15:46.831 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-14T17:15:46.831 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.831 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.831 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.831 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.831 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.831 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.831 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.831 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.831 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.831 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.831 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.832 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-14T17:15:46.832 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-14T17:15:46.832 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.832 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.832 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.832 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.832 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.832 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.832 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.832 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.832 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.832 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.832 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.833 INFO:teuthology.orchestra.run.smithi002.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-14T17:15:46.833 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-14T17:15:46.833 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.833 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.833 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.833 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.833 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.833 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.833 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.833 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.833 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.833 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.833 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.834 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_PASSWORD": { 2024-05-14T17:15:46.834 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-14T17:15:46.834 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.834 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.834 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.834 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2024-05-14T17:15:46.834 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.834 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.834 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.834 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.834 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.834 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.834 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.835 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-14T17:15:46.835 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-14T17:15:46.835 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.835 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.835 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.835 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.835 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.835 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.835 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.835 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.835 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.835 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.835 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.836 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_URL": { 2024-05-14T17:15:46.836 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_URL", 2024-05-14T17:15:46.836 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.836 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.836 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.836 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.836 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.836 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.836 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.836 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.836 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.836 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.836 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_USERNAME": { 2024-05-14T17:15:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-14T17:15:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2024-05-14T17:15:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.838 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-14T17:15:46.838 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-14T17:15:46.838 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.838 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.838 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.838 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.838 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.838 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.838 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.838 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.838 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.838 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.838 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.838 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-14T17:15:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-14T17:15:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-14T17:15:46.839 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-14T17:15:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-14T17:15:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-14T17:15:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.840 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.841 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.841 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.841 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.841 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.841 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.841 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.841 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.841 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.841 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.841 INFO:teuthology.orchestra.run.smithi002.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-14T17:15:46.841 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-14T17:15:46.841 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.841 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.842 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.842 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "[]", 2024-05-14T17:15:46.842 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.842 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.842 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.842 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.842 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.842 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.842 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.842 INFO:teuthology.orchestra.run.smithi002.stdout: "MULTICLUSTER_CONFIG": { 2024-05-14T17:15:46.842 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-14T17:15:46.842 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.842 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "{}", 2024-05-14T17:15:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: "PROMETHEUS_API_HOST": { 2024-05-14T17:15:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-14T17:15:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.843 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.844 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.844 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.844 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.844 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.844 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.844 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.844 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.844 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.844 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.844 INFO:teuthology.orchestra.run.smithi002.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-14T17:15:46.844 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-14T17:15:46.844 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.844 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.845 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.845 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.845 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.845 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.845 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.845 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.845 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.845 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.845 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.845 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-14T17:15:46.845 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-14T17:15:46.845 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.845 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-14T17:15:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-14T17:15:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.846 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-14T17:15:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-14T17:15:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.847 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-14T17:15:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-14T17:15:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.848 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.849 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.849 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.849 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.849 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.849 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.849 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.849 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.849 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-14T17:15:46.849 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-14T17:15:46.849 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.849 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.849 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.849 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.850 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.850 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.850 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.850 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.850 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.850 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.850 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.850 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-14T17:15:46.850 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-14T17:15:46.850 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.850 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.850 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.850 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.851 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.851 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.851 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.851 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.851 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.851 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.851 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.851 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-14T17:15:46.851 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-14T17:15:46.851 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.851 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.851 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.851 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.851 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.852 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.852 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.852 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.852 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.852 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.852 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.852 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_ENABLED": { 2024-05-14T17:15:46.852 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-14T17:15:46.852 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.852 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.852 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.852 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.852 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.853 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.853 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.853 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.853 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.853 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.853 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.853 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-14T17:15:46.853 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-14T17:15:46.853 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.853 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.853 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.853 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-14T17:15:46.853 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.854 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.854 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.854 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.854 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.854 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.854 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.854 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-14T17:15:46.854 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-14T17:15:46.854 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.854 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.854 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.854 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T17:15:46.855 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.855 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.855 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.855 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.855 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.855 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.855 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.855 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-14T17:15:46.855 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-14T17:15:46.855 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.855 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.855 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.855 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8", 2024-05-14T17:15:46.856 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.856 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.856 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.856 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.856 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.856 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.856 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.856 INFO:teuthology.orchestra.run.smithi002.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-14T17:15:46.856 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-14T17:15:46.856 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.856 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.856 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.856 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "45", 2024-05-14T17:15:46.857 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.857 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.857 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.857 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.857 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.857 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.857 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.857 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_ACCESS_KEY": { 2024-05-14T17:15:46.857 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-14T17:15:46.857 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.857 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.857 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.857 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.858 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.858 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.858 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.858 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.858 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.858 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.858 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.858 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-14T17:15:46.858 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-14T17:15:46.858 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.858 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.858 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.859 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2024-05-14T17:15:46.859 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.859 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.859 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.859 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.859 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.859 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.859 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.859 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_SECRET_KEY": { 2024-05-14T17:15:46.859 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-14T17:15:46.859 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.859 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.860 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.860 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.860 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.860 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.860 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.860 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.860 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.860 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.860 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.860 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.860 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_SSL_VERIFY": { 2024-05-14T17:15:46.860 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-14T17:15:46.860 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.860 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.860 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.861 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.861 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.861 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.861 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.861 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.861 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.861 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.861 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.861 INFO:teuthology.orchestra.run.smithi002.stdout: "UNSAFE_TLS_v1_2": { 2024-05-14T17:15:46.861 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-14T17:15:46.861 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.861 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.861 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.862 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.862 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.862 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.862 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.862 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.862 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.862 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.862 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.862 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-14T17:15:46.862 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-14T17:15:46.862 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.862 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.863 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.863 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.863 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.863 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.863 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.863 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.863 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.863 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.863 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.863 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-14T17:15:46.863 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-14T17:15:46.863 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.863 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.864 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.864 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T17:15:46.864 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.864 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.864 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.864 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.864 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.864 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.864 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.864 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-14T17:15:46.864 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-14T17:15:46.865 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.865 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.865 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.865 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-05-14T17:15:46.865 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.865 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.865 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.865 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.865 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.865 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.865 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.865 INFO:teuthology.orchestra.run.smithi002.stdout: "cross_origin_url": { 2024-05-14T17:15:46.865 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cross_origin_url", 2024-05-14T17:15:46.865 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.866 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.866 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.866 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.866 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.866 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.866 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.866 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.866 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.866 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.866 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.866 INFO:teuthology.orchestra.run.smithi002.stdout: "crt_file": { 2024-05-14T17:15:46.866 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crt_file", 2024-05-14T17:15:46.866 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.866 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.867 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.867 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.867 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.867 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.867 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.867 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.867 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.867 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.867 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.867 INFO:teuthology.orchestra.run.smithi002.stdout: "debug": { 2024-05-14T17:15:46.867 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "debug", 2024-05-14T17:15:46.867 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.867 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.867 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.868 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.868 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.868 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.868 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.868 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable/disable debug options", 2024-05-14T17:15:46.868 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.868 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.868 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.868 INFO:teuthology.orchestra.run.smithi002.stdout: "jwt_token_ttl": { 2024-05-14T17:15:46.868 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "jwt_token_ttl", 2024-05-14T17:15:46.868 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.868 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.868 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.868 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "28800", 2024-05-14T17:15:46.868 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.869 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.869 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.869 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.869 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.869 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.869 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.869 INFO:teuthology.orchestra.run.smithi002.stdout: "key_file": { 2024-05-14T17:15:46.869 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "key_file", 2024-05-14T17:15:46.869 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.869 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.869 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.869 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.869 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.869 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.869 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.870 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.870 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.870 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.870 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.870 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.870 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.870 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.870 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.870 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.870 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.870 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.870 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.870 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.871 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.871 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.871 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.871 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.871 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.871 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.871 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.871 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.871 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.871 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.871 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.871 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.871 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.871 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.872 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.872 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.872 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.872 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.872 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.872 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.872 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.872 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.872 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.872 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.872 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.872 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.872 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.872 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.873 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.873 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.873 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.873 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.873 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.873 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.873 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.873 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.873 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.873 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.873 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.873 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.873 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.873 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.873 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.873 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.874 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.874 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.874 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.874 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.874 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.874 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.874 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.874 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.874 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.874 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.874 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.874 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.874 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.875 INFO:teuthology.orchestra.run.smithi002.stdout: "motd": { 2024-05-14T17:15:46.875 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "motd", 2024-05-14T17:15:46.875 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.875 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.875 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.875 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.875 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.875 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.875 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.875 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "The message of the day", 2024-05-14T17:15:46.875 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.875 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.875 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.876 INFO:teuthology.orchestra.run.smithi002.stdout: "redirect_resolve_ip_addr": { 2024-05-14T17:15:46.876 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "redirect_resolve_ip_addr", 2024-05-14T17:15:46.876 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.876 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.876 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.876 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.876 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.876 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.876 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.876 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.876 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.876 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.876 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.877 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2024-05-14T17:15:46.877 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2024-05-14T17:15:46.877 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.877 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.877 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.877 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "::", 2024-05-14T17:15:46.877 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.877 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.877 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.877 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.877 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.877 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.877 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.877 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2024-05-14T17:15:46.878 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2024-05-14T17:15:46.878 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.878 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.878 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.878 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8080", 2024-05-14T17:15:46.878 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.878 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.878 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.878 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.878 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.878 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.878 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.878 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.878 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.879 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.879 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.879 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.879 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.879 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.879 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.879 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.879 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.879 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.879 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.879 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.879 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl": { 2024-05-14T17:15:46.879 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl", 2024-05-14T17:15:46.880 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.880 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.880 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.880 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.880 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.880 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.880 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.880 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.880 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.880 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.880 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.880 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl_server_port": { 2024-05-14T17:15:46.880 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl_server_port", 2024-05-14T17:15:46.880 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.880 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.881 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.881 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8443", 2024-05-14T17:15:46.881 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.881 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.881 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.881 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.881 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.881 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.881 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.881 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_behaviour": { 2024-05-14T17:15:46.881 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_behaviour", 2024-05-14T17:15:46.881 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.881 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.882 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.882 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "redirect", 2024-05-14T17:15:46.882 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.882 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.882 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.882 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.882 INFO:teuthology.orchestra.run.smithi002.stdout: "redirect" 2024-05-14T17:15:46.882 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.882 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.882 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.882 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.882 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.882 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.882 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_error_status_code": { 2024-05-14T17:15:46.883 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_error_status_code", 2024-05-14T17:15:46.883 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.883 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.883 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.883 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "500", 2024-05-14T17:15:46.883 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "400", 2024-05-14T17:15:46.883 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "599", 2024-05-14T17:15:46.883 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.883 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.883 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.883 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.883 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.883 INFO:teuthology.orchestra.run.smithi002.stdout: "url_prefix": { 2024-05-14T17:15:46.884 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "url_prefix", 2024-05-14T17:15:46.884 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.884 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.884 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.884 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.884 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.884 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.884 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.884 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.884 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.884 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.884 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.884 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.884 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.885 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "devicehealth", 2024-05-14T17:15:46.885 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.885 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.885 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.885 INFO:teuthology.orchestra.run.smithi002.stdout: "enable_monitoring": { 2024-05-14T17:15:46.885 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enable_monitoring", 2024-05-14T17:15:46.885 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.885 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.885 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.885 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.885 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.885 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.885 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.885 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "monitor device health metrics", 2024-05-14T17:15:46.886 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.886 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.886 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.886 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.886 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.886 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.886 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.886 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.886 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.886 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.886 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.886 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.886 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.886 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.886 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.887 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.887 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.887 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.887 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.887 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.887 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.887 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.887 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.887 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.887 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.887 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.887 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.887 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.887 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.887 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.888 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.888 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.888 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.888 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.888 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.888 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.888 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.888 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.888 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.888 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.888 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.888 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.888 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.889 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.889 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.889 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.889 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.889 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.889 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.889 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.889 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.889 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.889 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.889 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.889 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.889 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.889 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.890 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.890 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.890 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.890 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.890 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.890 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.890 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.890 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.890 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.890 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.890 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.890 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.890 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.891 INFO:teuthology.orchestra.run.smithi002.stdout: "mark_out_threshold": { 2024-05-14T17:15:46.891 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mark_out_threshold", 2024-05-14T17:15:46.891 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.891 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.891 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.891 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "2419200", 2024-05-14T17:15:46.891 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.891 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.891 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.891 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-14T17:15:46.891 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.891 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.891 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.891 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": { 2024-05-14T17:15:46.892 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pool_name", 2024-05-14T17:15:46.892 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.892 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.892 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.892 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "device_health_metrics", 2024-05-14T17:15:46.892 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.892 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.892 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.892 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-14T17:15:46.892 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.892 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.892 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.892 INFO:teuthology.orchestra.run.smithi002.stdout: "retention_period": { 2024-05-14T17:15:46.893 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "retention_period", 2024-05-14T17:15:46.893 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.893 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.893 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.893 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15552000", 2024-05-14T17:15:46.893 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.893 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.893 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.893 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long to retain device health metrics", 2024-05-14T17:15:46.893 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.893 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.893 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.893 INFO:teuthology.orchestra.run.smithi002.stdout: "scrape_frequency": { 2024-05-14T17:15:46.893 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "scrape_frequency", 2024-05-14T17:15:46.894 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.894 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.894 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.894 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "86400", 2024-05-14T17:15:46.894 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.894 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.894 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.894 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-14T17:15:46.894 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.894 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.894 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.894 INFO:teuthology.orchestra.run.smithi002.stdout: "self_heal": { 2024-05-14T17:15:46.894 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "self_heal", 2024-05-14T17:15:46.895 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.895 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.895 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.895 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:46.895 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.895 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.895 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.895 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-14T17:15:46.895 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.895 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.895 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.895 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-05-14T17:15:46.896 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-05-14T17:15:46.896 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.896 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.896 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.896 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-05-14T17:15:46.896 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.896 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.896 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.896 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to wake up and check device health", 2024-05-14T17:15:46.896 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.896 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.896 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.896 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.897 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.897 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.897 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.897 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.897 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.897 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.897 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.897 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.897 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.897 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.897 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.897 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.897 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_threshold": { 2024-05-14T17:15:46.897 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_threshold", 2024-05-14T17:15:46.898 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.898 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.898 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.898 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "7257600", 2024-05-14T17:15:46.898 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.898 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.898 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.898 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-14T17:15:46.898 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.898 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.898 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.898 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.898 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.899 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "diskprediction_local", 2024-05-14T17:15:46.899 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.899 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.899 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.899 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.899 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.899 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.899 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.899 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.899 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.899 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.899 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.899 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.900 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.900 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.900 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.900 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.900 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.900 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.900 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.900 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.900 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.900 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.900 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.900 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.900 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.900 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.901 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.901 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.901 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.901 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.901 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.901 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.901 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.901 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.901 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.901 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.901 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.901 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.901 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.901 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.901 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.902 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.902 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.902 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.902 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.902 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.902 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.902 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.902 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.902 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.902 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.902 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.902 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.902 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.902 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.902 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.903 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.903 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.903 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.903 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.903 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.903 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.903 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.903 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.903 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.903 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.903 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.903 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.903 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.904 INFO:teuthology.orchestra.run.smithi002.stdout: "predict_interval": { 2024-05-14T17:15:46.904 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "predict_interval", 2024-05-14T17:15:46.904 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.904 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.904 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.904 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "86400", 2024-05-14T17:15:46.904 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.904 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.904 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.904 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.904 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.904 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.904 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.905 INFO:teuthology.orchestra.run.smithi002.stdout: "predictor_model": { 2024-05-14T17:15:46.905 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "predictor_model", 2024-05-14T17:15:46.905 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.905 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.905 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.905 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "prophetstor", 2024-05-14T17:15:46.905 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.905 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.905 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.905 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.905 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.905 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.905 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.905 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-05-14T17:15:46.906 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-05-14T17:15:46.906 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.906 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.906 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.906 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2024-05-14T17:15:46.906 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.906 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.906 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.906 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.906 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.906 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.906 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.906 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.907 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.907 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.907 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.907 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.907 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.907 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.907 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.907 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.907 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.907 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.907 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.907 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.907 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.907 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.908 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "influx", 2024-05-14T17:15:46.908 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": false, 2024-05-14T17:15:46.908 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "influxdb python module not found", 2024-05-14T17:15:46.908 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.908 INFO:teuthology.orchestra.run.smithi002.stdout: "batch_size": { 2024-05-14T17:15:46.908 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "batch_size", 2024-05-14T17:15:46.908 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.908 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.908 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.908 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5000", 2024-05-14T17:15:46.908 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.908 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.908 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.908 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-14T17:15:46.909 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.909 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.909 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.909 INFO:teuthology.orchestra.run.smithi002.stdout: "database": { 2024-05-14T17:15:46.909 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "database", 2024-05-14T17:15:46.909 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.909 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.909 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.909 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "ceph", 2024-05-14T17:15:46.909 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.909 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.909 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.910 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-14T17:15:46.910 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.910 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.910 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.910 INFO:teuthology.orchestra.run.smithi002.stdout: "hostname": { 2024-05-14T17:15:46.910 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "hostname", 2024-05-14T17:15:46.910 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.910 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.910 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.910 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.910 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.910 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.910 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.910 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB server hostname", 2024-05-14T17:15:46.911 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.911 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.911 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.911 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-05-14T17:15:46.911 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-05-14T17:15:46.911 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:46.911 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.911 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.911 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "30", 2024-05-14T17:15:46.911 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "5", 2024-05-14T17:15:46.911 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.911 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.911 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-14T17:15:46.911 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.912 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.912 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.912 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.912 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.912 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.913 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.913 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.913 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.913 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.913 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.913 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.913 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.913 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.913 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.914 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.914 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.914 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.914 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.914 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.914 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.914 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.914 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.914 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.914 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.914 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.914 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.915 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.915 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.915 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.915 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.915 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.915 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.915 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.915 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.915 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.915 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.915 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.915 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.915 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.915 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.916 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.916 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.916 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.916 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.916 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.916 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.916 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.916 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.916 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.916 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.916 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.916 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.916 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.916 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.917 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.917 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.917 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.917 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.917 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.917 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.917 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.917 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.917 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.917 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.917 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.917 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.917 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.917 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.918 INFO:teuthology.orchestra.run.smithi002.stdout: "password": { 2024-05-14T17:15:46.918 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "password", 2024-05-14T17:15:46.918 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.918 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.918 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.918 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.918 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.918 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.918 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.918 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "password of InfluxDB server user", 2024-05-14T17:15:46.918 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.918 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.918 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.919 INFO:teuthology.orchestra.run.smithi002.stdout: "port": { 2024-05-14T17:15:46.919 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "port", 2024-05-14T17:15:46.919 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.919 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.919 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.919 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8086", 2024-05-14T17:15:46.919 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.919 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.919 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.919 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB server port", 2024-05-14T17:15:46.919 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.919 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.919 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.920 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.920 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.920 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.920 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.920 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.920 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.920 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.920 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.920 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.920 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.920 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.920 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.920 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.921 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl": { 2024-05-14T17:15:46.921 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl", 2024-05-14T17:15:46.921 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.921 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.921 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.921 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "false", 2024-05-14T17:15:46.921 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.921 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.921 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.921 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-14T17:15:46.921 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.921 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.921 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.922 INFO:teuthology.orchestra.run.smithi002.stdout: "threads": { 2024-05-14T17:15:46.922 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "threads", 2024-05-14T17:15:46.922 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.922 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.922 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.922 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-05-14T17:15:46.922 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-05-14T17:15:46.922 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "32", 2024-05-14T17:15:46.922 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.922 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-14T17:15:46.922 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.922 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.922 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.923 INFO:teuthology.orchestra.run.smithi002.stdout: "username": { 2024-05-14T17:15:46.923 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "username", 2024-05-14T17:15:46.923 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.923 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.923 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.923 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.923 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.923 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.923 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.923 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "username of InfluxDB server user", 2024-05-14T17:15:46.923 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.923 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.924 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.924 INFO:teuthology.orchestra.run.smithi002.stdout: "verify_ssl": { 2024-05-14T17:15:46.924 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "verify_ssl", 2024-05-14T17:15:46.924 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.924 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.924 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.924 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "true", 2024-05-14T17:15:46.924 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.924 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.924 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.924 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-14T17:15:46.924 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.924 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.925 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.925 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.925 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.925 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "insights", 2024-05-14T17:15:46.925 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.925 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.925 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.925 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.925 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.925 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.926 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.926 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.926 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.926 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.926 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.926 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.926 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.926 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.926 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.926 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.926 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.926 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.926 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.927 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.927 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.927 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.927 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.927 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.927 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.927 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.927 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.927 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.927 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.927 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.927 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.927 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.927 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.928 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.928 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.928 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.928 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.928 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.928 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.928 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.928 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.928 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.928 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.928 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.928 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.928 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.928 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.929 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.929 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.929 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.929 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.929 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.929 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.929 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.929 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.929 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.929 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.929 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.929 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.929 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.929 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.929 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.930 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.930 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.930 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.930 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.930 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.930 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.930 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.930 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.930 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.930 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.930 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.930 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.930 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.931 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.931 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.931 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.931 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.931 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.931 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.931 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.931 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.931 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.931 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "iostat", 2024-05-14T17:15:46.931 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.932 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.932 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.932 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.932 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.932 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.932 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.932 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.932 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.932 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.932 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.932 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.932 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.932 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.932 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.932 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.932 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.933 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.933 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.933 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.933 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.933 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.933 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.933 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.933 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.933 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.933 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.933 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.933 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.933 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.933 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.933 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.934 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.934 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.934 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.934 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.934 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.934 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.934 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.934 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.934 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.934 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.934 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.934 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.935 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.935 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.935 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.935 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.935 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.935 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.935 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.935 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.935 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.935 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.935 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.935 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.935 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.936 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.936 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.936 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.936 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.936 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.936 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.936 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.936 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.936 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.936 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.936 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.936 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.936 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.936 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.937 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.937 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.937 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.937 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.937 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.937 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.937 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.937 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.937 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.937 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.937 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.937 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.937 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.938 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "k8sevents", 2024-05-14T17:15:46.938 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.938 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.938 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.938 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_event_retention_days": { 2024-05-14T17:15:46.938 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ceph_event_retention_days", 2024-05-14T17:15:46.938 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.938 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.938 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.938 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "7", 2024-05-14T17:15:46.938 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.938 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.939 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.939 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-14T17:15:46.939 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.939 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.939 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.939 INFO:teuthology.orchestra.run.smithi002.stdout: "config_check_secs": { 2024-05-14T17:15:46.939 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_check_secs", 2024-05-14T17:15:46.939 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.939 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.939 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:46.939 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T17:15:46.939 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "10", 2024-05-14T17:15:46.939 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.939 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.939 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-14T17:15:46.940 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.940 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.940 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.940 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.940 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.940 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.940 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.940 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.940 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.940 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.940 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.940 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.940 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.941 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.941 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.941 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.941 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.941 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.941 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.941 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.941 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.941 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.941 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.941 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:46.941 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:46.941 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.942 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.942 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.942 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.942 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.942 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.942 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.942 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.942 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.942 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.942 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.942 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:46.943 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:46.943 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.943 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.943 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.943 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:46.943 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.943 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.943 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.943 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.943 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.943 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.943 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.943 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.943 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:46.943 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:46.944 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.944 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.944 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.944 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.944 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:46.944 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:46.944 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:46.944 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.944 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.944 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:46.944 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.944 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.944 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.945 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.945 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.945 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.945 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.945 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:46.945 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:46.945 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:46.945 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:46.945 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.945 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:46.945 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.945 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.945 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.946 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:46.946 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:46.946 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.946 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.946 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.946 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:46.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:46.946 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "localpool", 2024-05-14T17:15:46.946 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:46.946 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:46.946 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:46.946 INFO:teuthology.orchestra.run.smithi002.stdout: "failure_domain": { 2024-05-14T17:15:46.946 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "failure_domain", 2024-05-14T17:15:46.947 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.947 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.947 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.947 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "host", 2024-05-14T17:15:46.947 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.947 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.947 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:46.947 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "failure domain for any created local pool", 2024-05-14T17:15:46.947 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-14T17:15:46.947 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:46.947 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:46.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:46.947 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:46.947 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:46.948 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:46.948 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:46.948 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:46.948 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:46.948 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:46.948 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:46.948 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:46.948 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:46.948 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:46.948 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:46.948 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:46.948 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:46.948 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:47.013 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:47.021 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:47.026 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:47.034 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:47.042 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:47.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:47.059 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:47.070 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:47.078 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:47.087 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:47.095 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:47.103 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:47.112 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:47.122 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:47.130 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:47.138 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:47.146 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:47.203 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:47.230 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:47.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:47.244 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:47.253 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:47.260 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:47.269 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:47.277 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:47.285 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:47.293 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:47.298 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:47.304 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:47.310 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:47.318 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:47.327 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:47.339 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:47.346 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:47.415 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:47.423 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:47.431 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:47.437 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:47.443 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:47.449 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:47.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:47.464 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:47.472 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:47.478 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:47.484 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:47.490 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:47.499 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:47.505 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:47.511 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:47.517 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:47.524 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:47.567 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:47.573 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:47.581 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:47.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:47.595 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": { 2024-05-14T17:15:47.601 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "min_size", 2024-05-14T17:15:47.607 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:47.614 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:47.622 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:47.628 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:47.635 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:47.641 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:47.648 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:47.656 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default min_size for any created local pool", 2024-05-14T17:15:47.661 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-14T17:15:47.669 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:47.675 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:47.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:47.766 INFO:teuthology.orchestra.run.smithi002.stdout: "num_rep": { 2024-05-14T17:15:47.772 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "num_rep", 2024-05-14T17:15:47.777 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:47.784 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:47.789 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:47.796 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3", 2024-05-14T17:15:47.803 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:47.809 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:47.815 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:47.821 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default replica count for any created local pool", 2024-05-14T17:15:47.830 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:47.839 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:47.847 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:47.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:47.865 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": { 2024-05-14T17:15:47.874 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pg_num", 2024-05-14T17:15:47.925 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:47.935 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:47.942 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:47.949 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "128", 2024-05-14T17:15:47.958 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:47.966 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:47.973 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:47.982 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default pg_num for any created local pool", 2024-05-14T17:15:47.990 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:47.999 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:48.008 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:48.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:48.029 INFO:teuthology.orchestra.run.smithi002.stdout: "prefix": { 2024-05-14T17:15:48.039 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prefix", 2024-05-14T17:15:48.047 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:48.057 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:48.065 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:48.090 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:48.112 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:48.121 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:48.130 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:48.138 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "name prefix for any created local pool", 2024-05-14T17:15:48.145 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:48.153 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:48.162 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:48.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:48.177 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:48.183 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:48.191 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:48.197 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:48.202 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:48.208 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:48.217 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:48.224 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:48.251 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:48.257 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:48.264 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:48.274 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:48.282 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:48.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:48.295 INFO:teuthology.orchestra.run.smithi002.stdout: "subtree": { 2024-05-14T17:15:48.302 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "subtree", 2024-05-14T17:15:48.310 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:48.316 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:48.324 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:48.333 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "rack", 2024-05-14T17:15:48.341 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:48.350 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:48.360 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:48.366 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-14T17:15:48.372 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-14T17:15:48.410 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:48.419 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:48.426 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:48.434 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:48.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:48.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:48.457 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mds_autoscaler", 2024-05-14T17:15:48.474 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:48.482 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:48.493 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:48.526 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:48.538 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:48.544 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:48.550 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:48.556 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:48.563 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:48.569 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:48.598 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:48.603 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:48.610 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:48.616 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:48.623 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:48.629 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:48.635 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:48.641 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:48.647 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:48.654 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:48.660 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:48.666 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:48.673 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:48.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:48.687 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:48.693 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:48.699 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:48.738 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:48.744 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:48.750 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:48.757 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:48.762 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:48.768 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:48.775 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:48.780 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:48.786 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:48.791 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:48.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:48.805 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:48.811 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:48.819 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:48.827 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:48.839 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:48.848 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:48.893 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:48.902 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:48.911 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:48.918 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:48.927 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:48.935 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:48.943 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:48.949 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:48.957 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:48.967 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:48.985 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:48.993 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:49.003 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:49.011 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:49.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:49.027 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:49.035 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:49.072 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:49.081 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:49.090 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:49.100 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:49.110 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:49.118 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:49.126 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:49.135 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:49.143 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:49.152 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:49.158 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:49.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:49.176 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:49.183 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:49.191 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:49.200 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:49.213 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:49.271 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:49.280 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:49.288 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:49.296 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:49.304 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:49.312 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:49.321 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:49.328 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:49.335 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:49.342 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:49.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:49.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:49.361 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mirroring", 2024-05-14T17:15:49.369 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:49.376 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:49.383 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:49.390 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:49.431 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:49.438 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:49.445 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:49.451 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:49.457 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:49.464 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:49.471 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:49.478 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:49.485 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:49.492 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:49.499 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:49.505 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:49.511 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:49.518 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:49.524 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:49.530 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:49.537 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:49.583 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:49.589 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:49.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:49.616 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:49.623 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:49.630 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:49.637 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:49.644 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:49.651 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:49.699 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:49.705 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:49.713 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:49.719 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:49.741 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:49.758 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:49.764 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:49.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:49.825 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:49.841 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:49.851 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:49.860 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:49.886 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:49.895 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:49.909 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:49.916 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:49.922 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:49.932 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:49.940 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:49.948 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:49.957 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:49.965 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:49.994 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:50.001 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:50.007 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:50.061 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:50.068 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:50.075 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:50.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:50.090 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:50.099 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:50.109 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:50.116 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:50.127 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:50.132 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:50.140 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:50.149 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:50.158 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:50.166 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:50.228 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:50.234 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:50.239 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:50.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:50.317 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:50.323 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:50.333 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:50.340 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:50.347 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:50.354 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:50.362 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:50.370 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:50.378 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:50.383 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:50.390 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:50.398 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:50.405 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:50.412 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:50.454 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:50.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:50.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:50.520 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "nfs", 2024-05-14T17:15:50.526 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:50.533 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:50.540 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:50.549 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:50.555 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:50.561 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:50.568 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:50.575 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:50.582 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:50.662 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:50.709 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:50.716 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:50.722 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:50.729 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:50.738 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:50.743 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:50.750 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:50.757 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:50.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:50.772 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:50.777 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:50.784 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:50.791 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:50.797 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:50.809 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:50.815 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:50.822 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:50.861 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:50.870 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:50.879 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:50.888 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:50.897 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:50.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:50.911 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:50.918 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:50.924 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:50.930 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:50.936 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:50.942 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:50.948 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:50.957 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:50.968 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:50.974 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:50.980 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:51.035 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:51.041 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:51.049 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:51.057 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:51.065 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:51.073 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:51.082 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:51.090 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:51.099 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:51.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:51.116 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:51.126 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:51.134 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:51.140 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:51.169 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:51.176 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:51.185 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:51.213 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:51.221 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:51.227 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:51.236 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:51.242 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:51.249 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:51.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:51.268 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:51.275 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:51.282 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:51.288 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:51.297 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:51.305 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:51.313 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:51.327 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:51.333 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:51.340 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:51.397 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:51.405 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:51.412 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:51.417 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:51.425 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:51.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:51.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:51.451 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "orchestrator", 2024-05-14T17:15:51.459 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:51.466 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:51.473 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:51.479 INFO:teuthology.orchestra.run.smithi002.stdout: "fail_fs": { 2024-05-14T17:15:51.487 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "fail_fs", 2024-05-14T17:15:51.494 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:51.508 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:51.516 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:51.525 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:51.573 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:51.580 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:51.587 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:51.593 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-14T17:15:51.600 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:51.607 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:51.614 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:51.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:51.630 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:51.637 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:51.645 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:51.652 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:51.658 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:51.664 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:51.692 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:51.700 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:51.708 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:51.781 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:51.788 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:51.795 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:51.800 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:51.807 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:51.811 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:51.818 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:51.824 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:51.831 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:51.837 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:51.845 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:51.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:51.858 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:51.865 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:51.901 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:51.908 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:51.914 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:51.935 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:51.955 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:51.963 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:51.974 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:51.983 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:51.991 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:52.000 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:52.011 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:52.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:52.029 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:52.038 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:52.047 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:52.055 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:52.063 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:52.076 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:52.084 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:52.092 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:52.140 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:52.148 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:52.156 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:52.165 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:52.173 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:52.181 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:52.187 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:52.192 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:52.197 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:52.208 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:52.214 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:52.221 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:52.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:52.239 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:52.287 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:52.294 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:52.303 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:52.346 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:52.354 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:52.362 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:52.370 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:52.380 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:52.388 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:52.395 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:52.401 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:52.407 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:52.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:52.421 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator": { 2024-05-14T17:15:52.428 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "orchestrator", 2024-05-14T17:15:52.435 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:52.441 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:52.476 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:52.483 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:52.488 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:52.553 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:52.559 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:52.565 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm", 2024-05-14T17:15:52.573 INFO:teuthology.orchestra.run.smithi002.stdout: "rook", 2024-05-14T17:15:52.581 INFO:teuthology.orchestra.run.smithi002.stdout: "test_orchestrator" 2024-05-14T17:15:52.588 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:52.597 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Orchestrator backend", 2024-05-14T17:15:52.605 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:52.613 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:52.619 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:52.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:52.634 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:52.642 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:52.649 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:52.663 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:52.669 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:52.676 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:52.724 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:52.731 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:52.739 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:52.746 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:52.755 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:52.761 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:52.767 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:52.774 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:52.781 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:52.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:52.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:52.800 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "osd_perf_query", 2024-05-14T17:15:52.806 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:52.815 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:52.830 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:52.836 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:52.842 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:52.896 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:52.903 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:52.910 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:52.927 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:52.934 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:52.943 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:52.950 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:52.961 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:52.970 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:52.980 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:52.987 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:52.995 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:53.003 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:53.011 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:53.028 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:53.037 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:53.045 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:53.096 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:53.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:53.112 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:53.119 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:53.127 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:53.137 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:53.145 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:53.176 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:53.185 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:53.194 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:53.201 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:53.207 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:53.212 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:53.223 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:53.257 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:53.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:53.269 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:53.311 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:53.317 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:53.326 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:53.334 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:53.342 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:53.351 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:53.360 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:53.368 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:53.378 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:53.386 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:53.394 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:53.405 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:53.413 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:53.422 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:53.449 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:53.457 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:53.485 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:53.538 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:53.544 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:53.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:53.559 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:53.566 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:53.572 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:53.580 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:53.586 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:53.592 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:53.598 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:53.605 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:53.613 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:53.621 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:53.628 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:53.649 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:53.680 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:53.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:53.731 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:53.738 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:53.745 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:53.753 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:53.761 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:53.768 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:53.774 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:53.779 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:53.784 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:53.792 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:53.800 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:53.809 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:53.817 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:53.827 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:53.858 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:53.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:53.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:53.928 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "osd_support", 2024-05-14T17:15:53.937 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:53.946 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:53.954 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:53.962 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:53.971 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:53.979 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:53.987 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:53.995 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:54.005 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:54.015 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:54.023 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:54.031 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:54.040 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:54.095 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:54.111 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:54.163 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:54.207 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:54.224 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:54.235 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:54.242 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:54.247 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:54.255 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:54.261 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:54.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:54.274 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:54.285 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:54.293 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:54.330 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:54.336 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:54.341 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:54.347 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:54.387 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:54.425 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:54.479 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:54.492 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:54.497 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:54.503 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:54.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:54.517 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:54.525 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:54.531 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:54.537 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:54.544 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:54.551 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:54.557 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:54.563 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:54.569 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:54.586 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:54.624 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:54.667 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:54.704 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:54.712 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:54.718 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:54.725 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:54.732 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:54.742 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:54.748 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:54.753 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:54.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:54.768 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:54.772 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:54.778 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:54.785 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:54.791 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:54.798 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:54.821 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:54.868 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:54.896 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:54.903 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:54.911 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:54.920 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:54.927 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:54.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:55.003 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:55.013 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:55.029 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:55.037 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:55.063 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:55.072 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:55.081 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:55.089 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:55.097 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:55.105 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:55.163 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:55.192 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:55.199 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:55.206 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:55.214 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:55.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:55.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:55.249 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pg_autoscaler", 2024-05-14T17:15:55.280 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:55.287 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:55.295 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:55.303 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:55.312 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:55.321 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:55.330 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:55.338 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:55.347 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:55.422 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:55.441 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:55.448 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:55.456 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:55.499 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:55.506 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:55.527 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:55.533 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:55.539 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:55.547 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:55.553 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:55.559 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:55.564 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:55.570 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:55.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:55.581 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:55.587 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:55.655 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:55.692 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:55.699 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:55.713 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:55.747 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:55.752 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:55.778 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:55.784 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:55.789 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:55.802 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:55.807 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:55.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:55.827 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:55.833 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:55.839 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:55.846 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:55.852 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:55.913 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:55.962 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:55.969 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:55.974 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:56.024 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:56.028 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:56.036 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:56.041 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:56.046 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:56.052 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:56.059 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:56.066 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:56.072 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:56.079 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:56.084 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:56.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:56.100 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:56.136 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:56.180 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:56.188 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:56.219 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:56.225 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:56.233 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:56.244 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:56.249 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:56.257 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:56.265 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:56.273 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:56.280 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:56.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:56.296 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-05-14T17:15:56.305 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-05-14T17:15:56.312 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:56.321 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:56.346 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:56.423 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-05-14T17:15:56.430 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:56.438 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:56.447 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:56.457 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:56.469 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:56.477 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:56.485 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:56.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:56.498 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:56.505 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:56.512 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:56.519 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:56.527 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:56.535 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:56.541 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:56.549 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:56.582 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:56.607 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:56.614 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:56.620 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:56.627 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:56.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:56.642 INFO:teuthology.orchestra.run.smithi002.stdout: "threshold": { 2024-05-14T17:15:56.649 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "threshold", 2024-05-14T17:15:56.654 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T17:15:56.660 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:56.668 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3.0", 2024-05-14T17:15:56.680 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1.0", 2024-05-14T17:15:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:56.692 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:56.698 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "scaling threshold", 2024-05-14T17:15:56.751 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-14T17:15:56.760 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:56.784 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:56.790 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:56.795 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:56.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:56.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:56.820 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "progress", 2024-05-14T17:15:56.826 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:56.832 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:56.837 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:56.843 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_pg_recovery_event": { 2024-05-14T17:15:56.848 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_pg_recovery_event", 2024-05-14T17:15:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:56.860 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:56.865 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:56.870 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:56.902 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:56.948 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:56.953 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:56.958 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-14T17:15:56.964 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:56.974 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:57.037 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:57.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:57.049 INFO:teuthology.orchestra.run.smithi002.stdout: "enabled": { 2024-05-14T17:15:57.064 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enabled", 2024-05-14T17:15:57.070 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:57.078 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:57.085 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:57.092 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:57.099 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:57.106 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:57.130 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:57.156 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:57.168 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:57.174 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:57.217 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:57.225 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:57.231 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:57.239 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:57.245 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:57.252 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:57.260 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:57.267 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:57.301 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:57.331 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:57.343 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:57.348 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:57.355 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:57.362 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:57.370 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:57.377 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:57.389 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:57.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:57.402 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:57.409 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:57.416 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:57.423 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:57.430 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:57.441 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:57.446 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:57.541 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:57.581 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:57.588 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:57.593 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:57.598 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:57.604 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:57.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:57.653 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:57.660 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:57.666 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:57.672 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:57.677 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:57.683 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:57.688 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:57.694 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:57.699 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:57.741 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:57.747 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:57.811 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:57.824 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:57.872 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:57.878 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:57.884 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:57.889 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:57.897 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:57.904 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:57.911 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:57.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:57.939 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:57.967 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:57.975 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:57.983 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:57.991 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:57.999 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:58.006 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:58.012 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:58.050 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:58.105 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:58.111 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:58.119 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:58.127 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:58.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:58.142 INFO:teuthology.orchestra.run.smithi002.stdout: "max_completed_events": { 2024-05-14T17:15:58.173 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_completed_events", 2024-05-14T17:15:58.179 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:58.185 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:58.191 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:58.218 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "50", 2024-05-14T17:15:58.224 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:58.232 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:58.239 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:58.246 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "number of past completed events to remember", 2024-05-14T17:15:58.287 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:58.293 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:58.316 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:58.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:58.329 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2024-05-14T17:15:58.387 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2024-05-14T17:15:58.407 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:15:58.412 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:58.417 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:58.423 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2024-05-14T17:15:58.428 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:58.435 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:58.486 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:58.491 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long the module is going to sleep", 2024-05-14T17:15:58.496 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:58.503 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:58.508 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:58.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:58.533 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:15:58.558 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:15:58.568 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:15:58.573 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:15:58.579 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:58.584 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:15:58.616 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:58.622 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:58.628 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:58.646 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:58.651 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:58.656 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:58.662 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:58.670 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:58.675 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:15:58.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:58.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:15:58.751 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus", 2024-05-14T17:15:58.774 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:15:58.781 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:15:58.788 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:15:58.794 INFO:teuthology.orchestra.run.smithi002.stdout: "cache": { 2024-05-14T17:15:58.802 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cache", 2024-05-14T17:15:58.827 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:58.832 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:58.839 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:15:58.855 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:58.861 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:58.870 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:58.876 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:58.884 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:58.891 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:58.900 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:58.932 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:58.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:58.967 INFO:teuthology.orchestra.run.smithi002.stdout: "exclude_perf_counters": { 2024-05-14T17:15:58.976 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "exclude_perf_counters", 2024-05-14T17:15:58.981 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:58.988 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:58.996 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:59.045 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:15:59.052 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:59.073 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:59.078 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:59.083 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-14T17:15:59.088 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-14T17:15:59.092 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:59.098 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:59.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:59.112 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:15:59.146 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:15:59.191 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:59.226 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:59.232 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:59.240 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:15:59.245 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:59.250 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:59.296 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:59.301 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:59.337 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:59.343 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:59.350 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:59.355 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:59.361 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:59.367 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:59.373 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:59.378 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:59.416 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:59.456 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:59.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:59.498 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:15:59.504 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:15:59.510 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:59.517 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:59.524 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:59.532 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:59.582 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:59.589 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:59.597 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:15:59.606 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:59.614 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:59.622 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:59.632 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:59.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:59.648 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:15:59.686 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:15:59.701 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:15:59.736 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:59.744 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:59.749 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:15:59.757 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:59.763 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:15:59.771 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:15:59.801 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:15:59.806 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:15:59.814 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:15:59.821 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:15:59.828 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:15:59.835 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:15:59.842 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:15:59.849 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:15:59.856 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:15:59.901 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:15:59.918 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:15:59.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:15:59.942 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:15:59.948 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:15:59.956 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:15:59.962 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:15:59.967 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:15:59.982 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:15:59.989 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:15:59.996 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:00.001 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:00.005 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:00.011 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:00.016 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:00.023 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:00.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:00.086 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_stats_pools": { 2024-05-14T17:16:00.102 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_stats_pools", 2024-05-14T17:16:00.131 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:00.137 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:00.143 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:00.150 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:16:00.156 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:00.163 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:00.182 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:00.191 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:00.197 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:00.204 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:00.210 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:00.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:00.223 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-14T17:16:00.229 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-14T17:16:00.259 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:16:00.288 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:00.303 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:00.321 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "300", 2024-05-14T17:16:00.327 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:00.336 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:00.345 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:00.351 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:00.357 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:00.382 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:00.389 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:00.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:00.402 INFO:teuthology.orchestra.run.smithi002.stdout: "scrape_interval": { 2024-05-14T17:16:00.408 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "scrape_interval", 2024-05-14T17:16:00.414 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T17:16:00.420 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:00.426 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:00.457 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15.0", 2024-05-14T17:16:00.517 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:00.529 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:00.538 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:00.544 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:00.552 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:00.561 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:00.569 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:00.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:00.599 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2024-05-14T17:16:00.607 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2024-05-14T17:16:00.615 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:00.622 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:00.631 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:00.641 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "::", 2024-05-14T17:16:00.649 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:00.658 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:00.735 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:00.802 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-14T17:16:00.815 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:00.830 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:00.838 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:00.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:00.857 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2024-05-14T17:16:00.866 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2024-05-14T17:16:00.874 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:16:00.891 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:00.899 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:00.907 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "9283", 2024-05-14T17:16:00.915 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:00.924 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:00.932 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:00.939 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-14T17:16:00.947 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:00.954 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:00.985 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:01.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:01.022 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:16:01.028 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:16:01.034 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:16:01.042 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:16:01.048 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:01.055 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:16:01.075 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:01.081 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:01.086 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:01.092 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:01.098 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:01.104 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:01.112 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:01.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:01.155 INFO:teuthology.orchestra.run.smithi002.stdout: "stale_cache_strategy": { 2024-05-14T17:16:01.169 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "stale_cache_strategy", 2024-05-14T17:16:01.183 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:01.194 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:01.199 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:01.205 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "log", 2024-05-14T17:16:01.211 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:01.216 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:01.223 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:01.230 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:01.236 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:01.243 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:01.249 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:01.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:01.263 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_behaviour": { 2024-05-14T17:16:01.269 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_behaviour", 2024-05-14T17:16:01.274 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:01.290 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:01.321 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:01.334 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "default", 2024-05-14T17:16:01.341 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:01.347 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:01.353 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:16:01.359 INFO:teuthology.orchestra.run.smithi002.stdout: "default", 2024-05-14T17:16:01.366 INFO:teuthology.orchestra.run.smithi002.stdout: "error" 2024-05-14T17:16:01.374 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:16:01.390 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:01.396 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:01.405 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:01.412 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:01.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:01.424 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_error_status_code": { 2024-05-14T17:16:01.432 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_error_status_code", 2024-05-14T17:16:01.438 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:16:01.483 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:01.510 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:01.517 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "500", 2024-05-14T17:16:01.533 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "400", 2024-05-14T17:16:01.539 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "599", 2024-05-14T17:16:01.548 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:01.555 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:01.563 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:01.570 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:01.590 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:01.599 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:01.609 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:01.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:01.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:16:01.635 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_support", 2024-05-14T17:16:01.643 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:16:01.651 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:16:01.660 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:16:01.710 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:16:01.720 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:16:01.733 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:01.742 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:01.750 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:01.762 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:16:01.769 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:01.775 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:01.791 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:16:01.797 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:16:01.804 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:16:01.810 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:16:01.819 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:16:01.825 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:16:01.831 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:16:01.837 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:16:01.902 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:01.927 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:01.932 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:01.951 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:01.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:01.962 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:16:01.968 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:16:01.973 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:01.979 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:01.999 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:02.004 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:16:02.010 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:02.016 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:02.022 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:02.028 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:02.033 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:02.038 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:02.098 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:02.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:02.143 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:16:02.159 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:16:02.167 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:02.171 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:02.223 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:02.231 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:16:02.239 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:02.282 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:02.291 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:16:02.300 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:16:02.309 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:16:02.317 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:16:02.334 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:16:02.341 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:16:02.349 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:16:02.357 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:16:02.385 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:02.395 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:02.418 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:02.426 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:02.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:02.444 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:16:02.452 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:16:02.461 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:02.475 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:02.482 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:02.490 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:16:02.498 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:02.506 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:02.514 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:02.522 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:02.529 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:02.584 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:02.619 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:02.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:02.635 INFO:teuthology.orchestra.run.smithi002.stdout: "max_concurrent_snap_create": { 2024-05-14T17:16:02.642 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_concurrent_snap_create", 2024-05-14T17:16:02.648 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:16:02.657 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:02.662 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:02.668 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2024-05-14T17:16:02.684 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:02.690 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:02.697 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:02.703 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:02.709 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:02.714 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:02.720 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:02.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:02.755 INFO:teuthology.orchestra.run.smithi002.stdout: "mirror_snapshot_schedule": { 2024-05-14T17:16:02.770 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mirror_snapshot_schedule", 2024-05-14T17:16:02.786 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:02.802 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:02.809 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:02.816 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:16:02.822 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:02.829 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:02.835 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:02.857 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:02.863 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:02.868 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:02.873 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:02.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:02.888 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:16:02.895 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:16:02.901 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:16:02.933 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:16:02.962 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:02.994 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:16:03.003 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:03.008 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:03.016 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:03.021 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:03.026 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:03.033 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:03.051 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:03.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:03.066 INFO:teuthology.orchestra.run.smithi002.stdout: "trash_purge_schedule": { 2024-05-14T17:16:03.073 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "trash_purge_schedule", 2024-05-14T17:16:03.082 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:03.089 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:03.096 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:03.102 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:16:03.107 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:03.134 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:03.149 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:03.165 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:03.172 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:03.179 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:03.187 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:03.194 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:03.202 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:03.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:03.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:16:03.232 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "restful", 2024-05-14T17:16:03.254 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:16:03.277 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:16:03.350 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:16:03.365 INFO:teuthology.orchestra.run.smithi002.stdout: "enable_auth": { 2024-05-14T17:16:03.410 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enable_auth", 2024-05-14T17:16:03.418 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:03.425 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:03.431 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:03.437 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:16:03.444 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:03.455 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:03.460 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:03.465 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:03.471 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:03.477 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:03.483 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:03.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:03.495 INFO:teuthology.orchestra.run.smithi002.stdout: "key_file": { 2024-05-14T17:16:03.503 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "key_file", 2024-05-14T17:16:03.529 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:03.538 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:03.579 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:03.587 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:16:03.594 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:03.602 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:03.610 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:03.618 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:03.625 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:03.633 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:03.641 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:03.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:03.657 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:16:03.665 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:16:03.674 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:03.682 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:03.691 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:03.729 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:16:03.735 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:03.743 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:03.776 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:16:03.786 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:16:03.797 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:16:03.805 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:16:03.813 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:16:03.821 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:16:03.830 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:16:03.838 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:16:03.846 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:03.854 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:03.862 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:03.869 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:03.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:03.887 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:16:03.921 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:16:03.927 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:03.934 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:03.955 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:03.964 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:16:03.973 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:03.980 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:03.987 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:03.995 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:04.003 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:04.012 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:04.026 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:04.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:04.042 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:16:04.050 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:16:04.059 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:04.067 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:04.086 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:04.091 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:16:04.115 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:04.121 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:04.127 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:16:04.133 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:16:04.140 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:16:04.146 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:16:04.152 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:16:04.161 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:16:04.168 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:16:04.175 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:16:04.184 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:04.191 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:04.198 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:04.205 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:04.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:04.238 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:16:04.245 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:16:04.317 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:04.324 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:04.330 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:04.337 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:16:04.343 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:04.350 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:04.356 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:04.362 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:04.367 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:04.373 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:04.379 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:04.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:04.397 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2024-05-14T17:16:04.423 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2024-05-14T17:16:04.434 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:04.490 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:04.497 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:04.521 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:16:04.530 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:04.537 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:04.546 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:04.555 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:04.562 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:04.570 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:04.578 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:04.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:04.596 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2024-05-14T17:16:04.604 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2024-05-14T17:16:04.614 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:04.620 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:04.627 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:04.689 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:16:04.727 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:04.733 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:04.766 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:04.774 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:04.782 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:04.791 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:04.799 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:04.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:04.814 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:16:04.822 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:16:04.831 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:16:04.839 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:16:04.847 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:04.856 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:16:04.865 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:04.872 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:04.889 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:04.922 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:04.927 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:04.952 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:04.961 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:04.968 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:04.978 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:04.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:04.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:16:05.000 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rgw", 2024-05-14T17:16:05.007 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:16:05.013 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:16:05.020 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:16:05.026 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:16:05.032 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:16:05.038 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:05.044 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:05.056 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:05.080 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:16:05.086 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:05.107 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:05.117 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:16:05.124 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:16:05.131 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:16:05.138 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:16:05.144 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:16:05.154 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:16:05.159 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:16:05.165 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:16:05.172 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:05.179 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:05.185 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:05.192 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:05.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:05.206 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:16:05.231 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:16:05.237 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:05.266 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:05.272 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:05.279 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:16:05.284 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:05.291 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:05.297 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:05.303 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:05.310 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:05.316 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:05.322 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:05.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:05.334 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:16:05.339 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:16:05.345 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:05.374 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:05.398 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:05.406 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:16:05.438 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:05.446 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:05.452 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:16:05.461 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:16:05.469 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:16:05.477 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:16:05.485 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:16:05.497 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:16:05.511 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:16:05.519 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:16:05.530 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:05.539 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:05.546 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:05.555 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:05.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:05.598 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:16:05.605 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:16:05.631 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:05.640 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:05.648 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:05.656 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:16:05.665 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:05.673 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:05.683 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:05.690 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:05.697 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:05.706 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:05.715 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:05.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:05.728 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:16:05.750 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:16:05.774 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:16:05.802 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:16:05.811 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:05.840 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:16:05.849 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:05.858 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:05.864 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:05.870 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:05.876 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:05.882 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:05.888 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:05.895 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:05.900 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:05.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:05.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:16:05.916 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rook", 2024-05-14T17:16:05.924 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:16:05.945 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:16:05.979 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:16:05.988 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:16:06.048 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:16:06.056 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:06.063 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:06.069 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:06.075 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:16:06.082 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:06.088 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:06.094 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:16:06.100 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:16:06.107 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:16:06.113 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:16:06.120 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:16:06.127 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:16:06.134 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:16:06.141 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:16:06.173 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:06.179 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:06.210 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:06.217 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:06.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:06.230 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:16:06.236 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:16:06.243 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:06.251 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:06.258 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:06.265 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:16:06.273 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:06.280 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:06.288 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:06.295 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:06.303 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:06.325 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:06.349 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:06.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:06.362 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:16:06.385 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:16:06.393 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:06.402 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:06.411 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:06.419 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:16:06.427 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:06.435 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:06.444 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:16:06.456 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:16:06.463 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:16:06.469 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:16:06.476 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:16:06.485 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:16:06.492 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:16:06.510 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:16:06.518 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:06.583 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:06.591 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:06.600 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:06.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:06.613 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:16:06.622 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:16:06.629 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:06.637 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:06.646 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:06.656 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:16:06.664 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:06.672 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:06.679 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:06.688 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:06.701 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:06.745 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:06.813 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:06.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:06.828 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:16:06.835 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:16:06.842 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:16:06.849 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:16:06.857 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:06.863 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:16:06.870 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:06.878 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:06.885 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:06.892 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:06.900 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:06.907 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:06.912 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:06.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:06.962 INFO:teuthology.orchestra.run.smithi002.stdout: "storage_class": { 2024-05-14T17:16:06.999 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "storage_class", 2024-05-14T17:16:07.006 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:07.012 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:07.018 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:07.025 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "local", 2024-05-14T17:16:07.031 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:07.037 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:07.044 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:07.051 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-14T17:16:07.057 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:07.064 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:07.070 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:07.077 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:07.084 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:07.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:07.102 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:16:07.149 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "selftest", 2024-05-14T17:16:07.242 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:16:07.251 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:16:07.259 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:16:07.266 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:16:07.273 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:16:07.281 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:07.289 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:07.297 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:07.306 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:16:07.315 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:07.323 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:07.332 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:16:07.339 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:16:07.347 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:16:07.355 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:16:07.369 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:16:07.400 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:16:07.443 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:16:07.452 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:16:07.460 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:07.467 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:07.475 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:07.483 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:07.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:07.499 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:16:07.507 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:16:07.515 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:07.523 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:07.531 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:07.540 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:16:07.548 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:07.556 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:07.569 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:07.624 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:07.673 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:07.681 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:07.689 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:07.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:07.705 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:16:07.715 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:16:07.720 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:07.729 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:07.737 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:07.743 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:16:07.751 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:07.763 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:07.772 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:16:07.796 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:16:07.804 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:16:07.819 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:16:07.849 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:16:07.922 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:16:07.929 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:16:07.937 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:16:07.945 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:07.952 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:07.960 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:07.969 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:07.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:07.987 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:16:07.995 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:16:08.001 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:08.006 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:08.013 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:08.052 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:16:08.065 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:08.089 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:08.139 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:08.160 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:08.172 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:08.186 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:08.195 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:08.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:08.215 INFO:teuthology.orchestra.run.smithi002.stdout: "roption1": { 2024-05-14T17:16:08.229 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "roption1", 2024-05-14T17:16:08.238 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:08.247 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:08.256 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:08.268 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:16:08.281 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:08.319 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:08.327 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:08.332 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:08.353 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:08.387 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:08.446 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:08.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:08.460 INFO:teuthology.orchestra.run.smithi002.stdout: "roption2": { 2024-05-14T17:16:08.467 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "roption2", 2024-05-14T17:16:08.474 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:08.481 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:08.489 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:08.497 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "xyz", 2024-05-14T17:16:08.504 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:08.511 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:08.521 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:08.531 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:08.562 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:08.582 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:08.591 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:08.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:08.671 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption1": { 2024-05-14T17:16:08.703 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption1", 2024-05-14T17:16:08.712 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:08.725 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:08.734 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:08.746 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:16:08.757 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:08.771 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:08.779 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:08.788 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:08.797 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:08.809 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:08.816 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:08.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:08.852 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption2": { 2024-05-14T17:16:08.861 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption2", 2024-05-14T17:16:08.880 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:16:08.912 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:08.939 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:08.945 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:16:08.953 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:08.959 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:08.965 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:08.973 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:08.979 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:08.989 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:09.002 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:09.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:09.024 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption3": { 2024-05-14T17:16:09.034 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption3", 2024-05-14T17:16:09.058 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2024-05-14T17:16:09.067 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:09.082 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:09.098 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:16:09.121 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:09.159 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:09.169 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:09.179 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:09.188 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:09.197 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:09.210 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:09.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:09.224 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption4": { 2024-05-14T17:16:09.232 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption4", 2024-05-14T17:16:09.239 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:09.244 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:09.252 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:09.274 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:16:09.280 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:09.287 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:09.307 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:09.330 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:09.395 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:09.404 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:09.413 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:09.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:09.437 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption5": { 2024-05-14T17:16:09.450 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption5", 2024-05-14T17:16:09.465 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:09.483 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:09.497 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:09.508 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:16:09.522 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:09.529 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:09.542 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:09.554 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:09.562 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:09.590 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:09.630 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:09.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:09.691 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption6": { 2024-05-14T17:16:09.731 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption6", 2024-05-14T17:16:09.738 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:09.746 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:09.753 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:09.765 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:16:09.774 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:09.781 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:09.788 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:09.795 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:09.801 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:09.809 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:09.821 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:09.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:09.872 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption7": { 2024-05-14T17:16:09.905 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption7", 2024-05-14T17:16:09.978 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:16:09.990 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:09.998 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:10.032 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:16:10.045 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2024-05-14T17:16:10.062 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "42", 2024-05-14T17:16:10.069 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:10.076 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:10.084 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:10.092 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:10.101 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:10.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:10.119 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:16:10.126 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:16:10.158 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:16:10.185 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:16:10.216 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:10.244 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:16:10.252 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:10.262 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:10.274 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:10.287 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:10.296 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:10.306 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:10.319 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:10.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:10.340 INFO:teuthology.orchestra.run.smithi002.stdout: "testkey": { 2024-05-14T17:16:10.348 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testkey", 2024-05-14T17:16:10.358 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:10.368 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:10.382 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:10.417 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:16:10.431 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:10.457 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:10.495 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:10.503 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:10.511 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:10.516 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:10.523 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:10.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:10.538 INFO:teuthology.orchestra.run.smithi002.stdout: "testlkey": { 2024-05-14T17:16:10.545 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testlkey", 2024-05-14T17:16:10.551 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:10.558 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:10.566 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:10.572 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:16:10.581 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:10.589 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:10.609 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:10.639 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:10.692 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:10.733 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:10.745 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:10.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:10.768 INFO:teuthology.orchestra.run.smithi002.stdout: "testnewline": { 2024-05-14T17:16:10.779 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testnewline", 2024-05-14T17:16:10.787 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:10.798 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:10.807 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:10.819 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:16:10.828 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:10.842 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:10.854 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:10.862 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:10.871 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:10.902 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:10.932 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:10.973 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:11.000 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:11.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:11.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:16:11.026 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smb", 2024-05-14T17:16:11.032 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:16:11.039 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:16:11.046 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:16:11.053 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:16:11.058 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:16:11.067 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:11.074 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:11.081 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:11.089 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:16:11.119 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:11.131 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:11.154 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:16:11.207 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:16:11.242 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:16:11.272 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:16:11.285 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:16:11.297 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:16:11.308 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:16:11.317 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:16:11.353 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:11.367 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:11.376 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:11.388 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:11.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:11.410 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:16:11.417 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:16:11.426 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:11.431 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:11.467 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:11.501 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:16:11.545 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:11.553 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:11.559 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:11.566 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:11.574 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:11.620 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:11.628 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:11.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:11.652 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:16:11.667 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:16:11.678 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:11.690 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:11.701 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:11.712 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:16:11.725 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:11.754 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:11.799 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:16:11.841 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:16:11.848 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:16:11.856 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:16:11.863 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:16:11.871 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:16:11.877 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:16:11.912 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:16:11.918 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:11.924 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:11.929 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:11.934 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:11.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:11.949 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:16:11.955 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:16:11.963 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:11.971 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:12.023 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:12.045 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:16:12.051 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:12.057 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:12.063 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:12.075 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:12.079 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:12.105 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:12.113 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:12.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:12.128 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:16:12.136 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:16:12.144 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:16:12.152 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:16:12.160 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:12.168 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:16:12.180 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:12.199 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:12.226 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:12.234 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:12.242 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:12.250 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:12.258 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:12.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:12.286 INFO:teuthology.orchestra.run.smithi002.stdout: "update_orchestration": { 2024-05-14T17:16:12.292 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "update_orchestration", 2024-05-14T17:16:12.299 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:12.307 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:12.314 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:12.321 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:16:12.329 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:12.336 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:12.342 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:12.355 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-14T17:16:12.384 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:12.409 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:12.416 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:12.423 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:12.430 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:12.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:12.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:16:12.466 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snap_schedule", 2024-05-14T17:16:12.472 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:16:12.477 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:16:12.483 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:16:12.489 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_m_granularity": { 2024-05-14T17:16:12.494 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_m_granularity", 2024-05-14T17:16:12.499 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:12.505 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:12.510 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:12.529 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:16:12.546 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:12.586 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:12.592 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:12.598 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow minute scheduled snapshots", 2024-05-14T17:16:12.604 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:12.611 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:12.637 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:12.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:12.651 INFO:teuthology.orchestra.run.smithi002.stdout: "dump_on_update": { 2024-05-14T17:16:12.657 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "dump_on_update", 2024-05-14T17:16:12.664 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:12.670 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:12.678 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:12.683 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:16:12.689 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:12.694 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:12.706 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:12.742 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "dump database to debug log on update", 2024-05-14T17:16:12.769 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:12.777 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:12.786 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:12.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:12.803 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:16:12.834 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:16:12.846 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:12.854 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:12.862 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:12.870 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:16:12.878 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:12.886 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:12.893 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:16:12.900 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:16:12.908 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:16:12.931 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:16:12.953 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:16:12.979 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:16:12.987 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:16:12.995 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:16:13.002 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:13.011 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:13.030 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:13.040 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:13.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:13.056 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:16:13.063 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:16:13.071 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:13.080 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:13.087 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:13.094 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:16:13.104 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:13.123 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:13.137 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:13.167 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:13.174 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:13.180 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:13.187 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:13.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:13.206 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:16:13.211 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:16:13.217 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:13.224 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:13.230 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:13.237 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:16:13.244 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:13.250 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:13.256 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:16:13.262 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:16:13.290 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:16:13.313 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:16:13.345 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:16:13.352 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:16:13.359 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:16:13.366 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:16:13.373 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:13.390 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:13.395 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:13.402 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:13.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:13.416 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:16:13.423 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:16:13.429 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:13.436 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:13.441 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:13.447 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:16:13.466 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:13.488 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:13.518 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:13.524 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:13.532 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:13.539 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:13.545 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:13.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:13.573 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:16:13.579 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:16:13.584 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:16:13.589 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:16:13.597 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:13.603 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:16:13.611 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:13.617 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:13.623 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:13.641 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:13.690 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:13.727 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:13.736 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:13.746 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:13.755 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:13.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:13.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:16:13.796 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "stats", 2024-05-14T17:16:13.804 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:16:13.811 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:16:13.820 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:16:13.827 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:16:13.837 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:16:13.845 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:13.853 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:13.861 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:13.876 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:16:13.906 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:13.943 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:13.955 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:16:13.967 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:16:13.976 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:16:13.984 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:16:13.999 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:16:14.007 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:16:14.015 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:16:14.043 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:16:14.051 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:14.061 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:14.068 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:14.077 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:14.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:14.095 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:16:14.109 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:16:14.132 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:14.162 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:14.171 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:14.179 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:16:14.187 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:14.195 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:14.207 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:14.215 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:14.222 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:14.230 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:14.239 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:14.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:14.253 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:16:14.260 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:16:14.268 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:14.277 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:14.285 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:14.315 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:16:14.353 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:14.379 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:14.385 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:16:14.392 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:16:14.398 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:16:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:16:14.423 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:16:14.430 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:16:14.437 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:16:14.443 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:16:14.452 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:14.458 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:14.465 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:14.471 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:14.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:14.485 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:16:14.505 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:16:14.530 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:14.537 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:14.543 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:14.549 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:16:14.554 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:14.583 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:14.588 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:14.595 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:14.601 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:14.608 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:14.614 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:14.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:14.628 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:16:14.633 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:16:14.639 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:16:14.652 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:16:14.670 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:14.698 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:16:14.703 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:14.709 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:14.714 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:14.720 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:14.759 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:14.773 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:14.782 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:14.790 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:14.797 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:14.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:14.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:16:14.819 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "status", 2024-05-14T17:16:14.827 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:16:14.834 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:16:14.869 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:16:14.896 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:16:14.933 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:16:14.941 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:14.946 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:14.955 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:14.963 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:16:14.977 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:14.985 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:14.994 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:16:15.002 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:16:15.011 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:16:15.019 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:16:15.025 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:16:15.032 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:16:15.038 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:16:15.045 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:16:15.061 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:15.096 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:15.145 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:15.152 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:15.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:15.169 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:16:15.174 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:16:15.186 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:15.192 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:15.198 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:15.203 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:16:15.208 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:15.214 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:15.219 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:15.225 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:15.231 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:15.237 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:15.261 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:15.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:15.334 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:16:15.340 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:16:15.347 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:15.353 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:15.359 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:15.391 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:16:15.397 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:15.404 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:15.409 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:16:15.416 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:16:15.422 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:16:15.428 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:16:15.434 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:16:15.441 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:16:15.446 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:16:15.470 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:16:15.486 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:15.525 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:15.531 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:15.539 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:15.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:15.550 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:16:15.583 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:16:15.594 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:15.602 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:15.610 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:15.618 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:16:15.627 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:15.636 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:15.646 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:15.656 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:15.664 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:15.679 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:15.705 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:15.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:15.756 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:16:15.764 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:16:15.772 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:16:15.779 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:16:15.804 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:15.814 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:16:15.822 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:15.831 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:15.839 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:15.848 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:15.869 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:15.876 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:15.886 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:15.891 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:15.916 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:15.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:15.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:16:15.973 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "telegraf", 2024-05-14T17:16:15.980 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:16:15.988 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:16:15.997 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:16:16.021 INFO:teuthology.orchestra.run.smithi002.stdout: "address": { 2024-05-14T17:16:16.030 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "address", 2024-05-14T17:16:16.038 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:16.046 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:16.055 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:16.063 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-14T17:16:16.071 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:16.086 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:16.096 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:16.104 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:16.141 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:16.188 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:16.216 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:16.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:16.228 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-05-14T17:16:16.235 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-05-14T17:16:16.242 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:16:16.263 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:16.270 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:16.281 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15", 2024-05-14T17:16:16.287 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:16.294 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:16.300 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:16.308 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:16.313 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:16.319 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:16.328 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:16.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:16.365 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:16:16.410 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:16:16.418 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:16.424 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:16.428 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:16.436 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:16:16.450 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:16.458 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:16.465 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:16:16.471 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:16:16.477 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:16:16.484 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:16:16.491 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:16:16.498 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:16:16.506 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:16:16.513 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:16:16.525 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:16.548 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:16.580 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:16.587 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:16.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:16.604 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:16:16.611 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:16:16.626 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:16.634 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:16.641 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:16.650 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:16:16.658 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:16.666 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:16.674 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:16.683 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:16.691 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:16.700 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:16.710 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:16.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:16.778 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:16:16.788 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:16:16.796 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:16.804 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:16.813 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:16.834 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:16:16.841 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:16.849 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:16.856 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:16:16.864 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:16:16.871 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:16:16.879 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:16:16.887 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:16:16.894 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:16:16.902 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:16:16.926 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:16:16.948 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:16.983 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:16.993 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:17.001 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:17.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:17.016 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:16:17.035 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:16:17.043 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:17.050 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:17.060 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:17.067 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:16:17.075 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:17.083 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:17.091 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:17.098 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:17.106 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:17.118 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:17.140 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:17.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:17.172 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:16:17.178 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:16:17.185 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:16:17.192 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:16:17.202 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:17.211 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:16:17.218 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:17.225 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:17.232 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:17.244 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:17.251 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:17.257 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:17.265 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:17.270 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:17.285 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:17.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:17.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:16:17.344 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "telemetry", 2024-05-14T17:16:17.350 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:16:17.356 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:16:17.362 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:16:17.369 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_basic": { 2024-05-14T17:16:17.375 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_basic", 2024-05-14T17:16:17.384 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:17.391 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:17.397 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:17.404 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:16:17.410 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:17.415 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:17.421 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:17.427 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-14T17:16:17.443 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:17.462 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:17.505 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:17.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:17.519 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_crash": { 2024-05-14T17:16:17.525 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_crash", 2024-05-14T17:16:17.531 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:17.544 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:17.556 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:17.562 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:16:17.590 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:17.597 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:17.603 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:17.610 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-14T17:16:17.617 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:17.633 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:17.639 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:17.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:17.686 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_device": { 2024-05-14T17:16:17.694 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_device", 2024-05-14T17:16:17.720 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:17.727 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:17.735 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:17.743 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:16:17.762 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:17.767 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:17.775 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:17.784 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-14T17:16:17.792 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:17.799 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:17.807 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:17.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:17.821 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_ident": { 2024-05-14T17:16:17.829 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_ident", 2024-05-14T17:16:17.844 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:17.869 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:17.896 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:17.902 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:16:17.908 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:17.914 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:17.922 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:17.929 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-14T17:16:17.936 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:17.944 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:17.953 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:17.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:17.970 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_perf": { 2024-05-14T17:16:17.978 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_perf", 2024-05-14T17:16:17.991 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:17.996 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:18.003 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:18.011 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:16:18.027 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:18.062 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:18.067 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:18.072 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-14T17:16:18.078 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:18.083 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:18.097 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:18.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: "contact": { 2024-05-14T17:16:18.118 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "contact", 2024-05-14T17:16:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:18.141 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:16:18.148 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:18.178 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:18.191 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:18.275 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:18.282 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:18.290 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:18.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:18.303 INFO:teuthology.orchestra.run.smithi002.stdout: "description": { 2024-05-14T17:16:18.329 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "description", 2024-05-14T17:16:18.337 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:18.344 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:18.351 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:18.357 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:16:18.364 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:18.371 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:18.378 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:18.385 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:18.392 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:18.420 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:18.466 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:18.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:18.547 INFO:teuthology.orchestra.run.smithi002.stdout: "device_url": { 2024-05-14T17:16:18.553 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_url", 2024-05-14T17:16:18.560 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:18.593 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-14T17:16:18.601 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:18.610 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:18.618 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:18.627 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:18.635 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:18.643 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:18.651 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:18.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:18.697 INFO:teuthology.orchestra.run.smithi002.stdout: "enabled": { 2024-05-14T17:16:18.725 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enabled", 2024-05-14T17:16:18.764 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:18.772 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:18.780 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:18.788 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:16:18.796 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:18.823 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:18.830 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:18.838 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:18.847 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:18.854 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:18.861 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:18.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:18.876 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-05-14T17:16:18.885 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-05-14T17:16:18.892 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:16:18.899 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:18.923 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:18.999 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "24", 2024-05-14T17:16:19.005 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "8", 2024-05-14T17:16:19.011 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:19.017 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:19.023 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:19.029 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:19.090 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:19.098 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:19.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:19.113 INFO:teuthology.orchestra.run.smithi002.stdout: "last_opt_revision": { 2024-05-14T17:16:19.119 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "last_opt_revision", 2024-05-14T17:16:19.125 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:16:19.131 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:19.138 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:19.144 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1", 2024-05-14T17:16:19.174 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:19.204 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:19.225 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:19.231 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:19.237 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:19.242 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:19.248 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:19.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:19.262 INFO:teuthology.orchestra.run.smithi002.stdout: "leaderboard": { 2024-05-14T17:16:19.273 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "leaderboard", 2024-05-14T17:16:19.279 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:19.285 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:19.291 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:19.297 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:16:19.303 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:19.308 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:19.315 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:19.320 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:19.342 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:19.423 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:19.429 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:19.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:19.444 INFO:teuthology.orchestra.run.smithi002.stdout: "leaderboard_description": { 2024-05-14T17:16:19.450 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "leaderboard_description", 2024-05-14T17:16:19.466 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:19.473 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:19.481 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:19.489 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:16:19.498 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:19.508 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:19.515 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:19.523 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:19.530 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:19.558 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:19.568 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:19.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:19.596 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:16:19.620 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:16:19.629 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:19.636 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:19.644 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:19.675 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:16:19.683 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:19.690 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:19.698 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:16:19.705 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:16:19.713 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:16:19.721 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:16:19.728 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:16:19.735 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:16:19.743 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:16:19.759 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:16:19.787 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:19.815 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:19.824 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:19.832 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:19.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:19.848 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:16:19.881 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:16:19.887 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:19.894 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:19.902 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:19.909 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:16:19.917 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:19.925 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:19.933 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:19.940 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:19.948 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:19.968 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:19.993 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:20.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:20.015 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:16:20.022 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:16:20.028 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:20.035 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:20.054 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:20.060 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:16:20.067 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:20.074 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:20.080 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:16:20.086 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:16:20.094 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:16:20.106 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:16:20.112 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:16:20.138 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:16:20.160 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:16:20.185 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:16:20.218 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:20.225 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:20.231 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:20.238 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:20.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:20.276 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:16:20.282 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:16:20.289 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:20.295 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:20.301 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:20.308 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:16:20.316 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:20.322 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:20.329 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:20.335 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:20.343 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:20.369 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:20.442 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:20.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:20.455 INFO:teuthology.orchestra.run.smithi002.stdout: "organization": { 2024-05-14T17:16:20.461 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "organization", 2024-05-14T17:16:20.467 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:20.479 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:20.485 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:20.491 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:16:20.497 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:20.504 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:20.510 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:20.516 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:20.523 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:20.530 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:20.538 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:20.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:20.565 INFO:teuthology.orchestra.run.smithi002.stdout: "proxy": { 2024-05-14T17:16:20.609 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "proxy", 2024-05-14T17:16:20.618 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:20.626 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:20.633 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:20.640 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:16:20.663 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:20.669 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:20.676 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:20.684 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:20.691 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:20.699 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:20.706 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:20.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:20.722 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:16:20.729 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:16:20.757 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:16:20.781 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:16:20.838 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:20.847 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:16:20.854 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:20.862 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:20.869 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:20.888 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:20.898 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:20.906 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:20.913 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:20.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:20.927 INFO:teuthology.orchestra.run.smithi002.stdout: "url": { 2024-05-14T17:16:20.935 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "url", 2024-05-14T17:16:20.946 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:20.951 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:20.956 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:20.961 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-14T17:16:20.981 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:21.047 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:21.053 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:21.058 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:21.063 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:21.068 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:21.081 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:21.096 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:21.105 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:21.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:21.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:16:21.162 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "test_orchestrator", 2024-05-14T17:16:21.168 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:16:21.174 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:16:21.179 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:16:21.183 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:16:21.192 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:16:21.223 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:21.246 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:21.253 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:21.259 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:16:21.273 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:21.278 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:21.284 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:16:21.288 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:16:21.294 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:16:21.317 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:16:21.324 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:16:21.329 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:16:21.336 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:16:21.341 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:16:21.348 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:21.355 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:21.360 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:21.382 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:21.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:21.473 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:16:21.480 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:16:21.518 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:21.525 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:21.533 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:21.540 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:16:21.548 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:21.556 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:21.564 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:21.574 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:21.581 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:21.588 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:21.604 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:21.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:21.626 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:16:21.683 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:16:21.716 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:21.725 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:21.734 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:21.770 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:16:21.782 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:21.794 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:21.804 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:16:21.815 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:16:21.826 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:16:21.838 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:16:21.849 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:16:21.860 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:16:21.868 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:16:21.874 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:16:21.883 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:21.891 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:21.967 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:22.001 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:22.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:22.043 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:16:22.049 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:16:22.057 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:22.063 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:22.069 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:22.077 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:16:22.084 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:22.094 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:22.100 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:22.108 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:22.117 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:22.133 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:22.147 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:22.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:22.187 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:16:22.213 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:16:22.241 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:16:22.252 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:16:22.271 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:22.283 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:16:22.293 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:22.305 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:22.317 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:22.325 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:22.334 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:22.347 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:22.358 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:22.369 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:22.376 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:22.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:22.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:16:22.420 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "volumes", 2024-05-14T17:16:22.444 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:16:22.475 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:16:22.484 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:16:22.493 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:16:22.501 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:16:22.508 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:22.515 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:22.521 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:22.532 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:16:22.542 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:22.549 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:22.582 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:16:22.590 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:16:22.598 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:16:22.604 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:16:22.611 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:16:22.627 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:16:22.681 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:16:22.712 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:16:22.721 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:22.733 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:22.747 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:22.759 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:22.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:22.785 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:16:22.794 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:16:22.802 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:22.813 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:22.824 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:22.833 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:16:22.847 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:22.856 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:22.867 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:22.892 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:22.947 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:22.987 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:22.995 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:23.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:23.013 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:16:23.020 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:16:23.028 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:23.038 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:23.047 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:23.052 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:16:23.060 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:23.065 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:23.072 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:16:23.078 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:16:23.086 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:16:23.093 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:16:23.127 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:16:23.198 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:16:23.247 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:16:23.264 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:16:23.274 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:23.281 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:23.293 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:23.305 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:23.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:23.322 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:16:23.331 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:16:23.343 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:23.356 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:23.364 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:23.377 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:16:23.385 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:23.396 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:23.427 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:23.475 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:23.496 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:23.507 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:23.513 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:23.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:23.527 INFO:teuthology.orchestra.run.smithi002.stdout: "max_concurrent_clones": { 2024-05-14T17:16:23.535 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_concurrent_clones", 2024-05-14T17:16:23.543 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:16:23.550 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:23.558 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:23.566 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "4", 2024-05-14T17:16:23.575 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:23.584 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:23.589 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:23.597 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-14T17:16:23.605 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:23.627 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:23.653 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:23.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:23.689 INFO:teuthology.orchestra.run.smithi002.stdout: "periodic_async_work": { 2024-05-14T17:16:23.702 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "periodic_async_work", 2024-05-14T17:16:23.714 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:23.724 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:23.733 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:23.742 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:16:23.751 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:23.765 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:23.777 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:23.789 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Periodically check for async work", 2024-05-14T17:16:23.802 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:23.816 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:23.828 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:23.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:23.876 INFO:teuthology.orchestra.run.smithi002.stdout: "snapshot_clone_delay": { 2024-05-14T17:16:23.950 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snapshot_clone_delay", 2024-05-14T17:16:23.973 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:16:23.982 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:23.989 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:23.997 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:16:24.005 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:24.019 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:24.028 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:24.037 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-14T17:16:24.052 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:24.063 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:24.075 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:24.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:24.101 INFO:teuthology.orchestra.run.smithi002.stdout: "snapshot_clone_no_wait": { 2024-05-14T17:16:24.110 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snapshot_clone_no_wait", 2024-05-14T17:16:24.119 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:24.152 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:24.192 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:24.215 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2024-05-14T17:16:24.228 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:24.237 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:24.244 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:24.255 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-14T17:16:24.263 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:24.271 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:24.278 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:24.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:24.295 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:16:24.304 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:16:24.312 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:16:24.320 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:16:24.329 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:24.337 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:16:24.352 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:24.402 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:24.415 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:24.425 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:24.438 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:24.450 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:24.480 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:24.492 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:24.503 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:24.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:24.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:16:24.535 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix", 2024-05-14T17:16:24.550 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2024-05-14T17:16:24.560 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2024-05-14T17:16:24.573 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2024-05-14T17:16:24.585 INFO:teuthology.orchestra.run.smithi002.stdout: "discovery_interval": { 2024-05-14T17:16:24.594 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "discovery_interval", 2024-05-14T17:16:24.627 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2024-05-14T17:16:24.670 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:24.684 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:24.693 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "100", 2024-05-14T17:16:24.698 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:24.706 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:24.714 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:24.721 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:24.729 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:24.737 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:24.745 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:24.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:24.759 INFO:teuthology.orchestra.run.smithi002.stdout: "identifier": { 2024-05-14T17:16:24.768 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "identifier", 2024-05-14T17:16:24.775 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:24.784 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:24.792 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:24.828 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:16:24.862 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:24.904 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:24.917 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:24.929 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:24.936 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:24.948 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:24.959 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:24.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:24.984 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2024-05-14T17:16:24.995 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2024-05-14T17:16:25.002 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2024-05-14T17:16:25.014 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:25.025 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:25.034 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2024-05-14T17:16:25.045 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:25.056 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:25.087 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:25.138 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:25.146 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:25.155 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:25.162 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:25.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:25.178 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2024-05-14T17:16:25.187 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2024-05-14T17:16:25.194 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:25.201 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:25.209 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:25.216 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:16:25.225 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:25.232 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:25.240 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:16:25.247 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:16:25.254 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:16:25.286 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:16:25.331 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:16:25.343 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:16:25.355 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:16:25.364 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:16:25.376 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:25.385 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:25.398 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:25.407 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:25.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:25.431 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2024-05-14T17:16:25.442 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2024-05-14T17:16:25.451 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:25.461 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:25.468 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:25.481 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:16:25.488 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:25.512 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:25.542 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:25.552 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:25.560 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:25.568 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:25.574 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:25.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:25.587 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2024-05-14T17:16:25.594 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2024-05-14T17:16:25.603 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:25.609 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:25.617 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:25.625 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2024-05-14T17:16:25.632 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:25.641 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:25.648 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2024-05-14T17:16:25.655 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2024-05-14T17:16:25.686 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2024-05-14T17:16:25.722 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2024-05-14T17:16:25.731 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2024-05-14T17:16:25.739 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2024-05-14T17:16:25.748 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2024-05-14T17:16:25.755 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:16:25.764 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:25.776 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:25.789 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:25.801 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:25.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:25.823 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2024-05-14T17:16:25.839 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2024-05-14T17:16:25.849 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2024-05-14T17:16:25.861 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:25.871 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:25.885 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2024-05-14T17:16:25.904 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:25.937 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:25.948 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:25.957 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:25.969 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:25.979 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:25.987 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:25.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:26.006 INFO:teuthology.orchestra.run.smithi002.stdout: "sqlite3_killpoint": { 2024-05-14T17:16:26.014 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sqlite3_killpoint", 2024-05-14T17:16:26.020 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:16:26.028 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "dev", 2024-05-14T17:16:26.035 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-05-14T17:16:26.042 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2024-05-14T17:16:26.051 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:26.059 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:26.067 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:26.101 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:26.132 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:26.140 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:26.148 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:26.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:26.164 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_host": { 2024-05-14T17:16:26.171 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_host", 2024-05-14T17:16:26.178 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:26.185 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:26.192 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:26.201 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2024-05-14T17:16:26.208 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:26.213 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:26.220 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:26.229 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:26.240 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:26.250 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:26.271 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:26.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:26.312 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_port": { 2024-05-14T17:16:26.324 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_port", 2024-05-14T17:16:26.334 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2024-05-14T17:16:26.345 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:26.355 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:26.361 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10051", 2024-05-14T17:16:26.369 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:26.377 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:26.391 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:26.405 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:26.413 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:26.420 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:26.432 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:26.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:26.447 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_sender": { 2024-05-14T17:16:26.471 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_sender", 2024-05-14T17:16:26.503 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2024-05-14T17:16:26.508 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2024-05-14T17:16:26.515 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2024-05-14T17:16:26.520 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-14T17:16:26.525 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2024-05-14T17:16:26.531 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2024-05-14T17:16:26.537 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2024-05-14T17:16:26.542 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2024-05-14T17:16:26.547 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2024-05-14T17:16:26.553 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2024-05-14T17:16:26.558 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2024-05-14T17:16:26.564 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:26.570 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:26.575 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:26.580 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:16:26.585 INFO:teuthology.orchestra.run.smithi002.stdout: "services": { 2024-05-14T17:16:26.600 INFO:teuthology.orchestra.run.smithi002.stdout: "dashboard": "https://172.21.15.2:8443/" 2024-05-14T17:16:26.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:26.627 INFO:teuthology.orchestra.run.smithi002.stdout: "always_on_modules": { 2024-05-14T17:16:26.633 INFO:teuthology.orchestra.run.smithi002.stdout: "octopus": [ 2024-05-14T17:16:26.640 INFO:teuthology.orchestra.run.smithi002.stdout: "balancer", 2024-05-14T17:16:26.645 INFO:teuthology.orchestra.run.smithi002.stdout: "crash", 2024-05-14T17:16:26.650 INFO:teuthology.orchestra.run.smithi002.stdout: "devicehealth", 2024-05-14T17:16:26.657 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator", 2024-05-14T17:16:26.664 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscaler", 2024-05-14T17:16:26.670 INFO:teuthology.orchestra.run.smithi002.stdout: "progress", 2024-05-14T17:16:26.678 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_support", 2024-05-14T17:16:26.683 INFO:teuthology.orchestra.run.smithi002.stdout: "status", 2024-05-14T17:16:26.690 INFO:teuthology.orchestra.run.smithi002.stdout: "telemetry", 2024-05-14T17:16:26.696 INFO:teuthology.orchestra.run.smithi002.stdout: "volumes" 2024-05-14T17:16:26.701 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:16:26.707 INFO:teuthology.orchestra.run.smithi002.stdout: "pacific": [ 2024-05-14T17:16:26.711 INFO:teuthology.orchestra.run.smithi002.stdout: "balancer", 2024-05-14T17:16:26.726 INFO:teuthology.orchestra.run.smithi002.stdout: "crash", 2024-05-14T17:16:26.752 INFO:teuthology.orchestra.run.smithi002.stdout: "devicehealth", 2024-05-14T17:16:26.758 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator", 2024-05-14T17:16:26.763 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscaler", 2024-05-14T17:16:26.768 INFO:teuthology.orchestra.run.smithi002.stdout: "progress", 2024-05-14T17:16:26.775 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_support", 2024-05-14T17:16:26.781 INFO:teuthology.orchestra.run.smithi002.stdout: "status", 2024-05-14T17:16:26.786 INFO:teuthology.orchestra.run.smithi002.stdout: "telemetry", 2024-05-14T17:16:26.793 INFO:teuthology.orchestra.run.smithi002.stdout: "volumes" 2024-05-14T17:16:26.798 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:16:26.804 INFO:teuthology.orchestra.run.smithi002.stdout: "quincy": [ 2024-05-14T17:16:26.810 INFO:teuthology.orchestra.run.smithi002.stdout: "balancer", 2024-05-14T17:16:26.816 INFO:teuthology.orchestra.run.smithi002.stdout: "crash", 2024-05-14T17:16:26.822 INFO:teuthology.orchestra.run.smithi002.stdout: "devicehealth", 2024-05-14T17:16:26.829 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator", 2024-05-14T17:16:26.835 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscaler", 2024-05-14T17:16:26.845 INFO:teuthology.orchestra.run.smithi002.stdout: "progress", 2024-05-14T17:16:26.860 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_support", 2024-05-14T17:16:26.892 INFO:teuthology.orchestra.run.smithi002.stdout: "status", 2024-05-14T17:16:26.898 INFO:teuthology.orchestra.run.smithi002.stdout: "telemetry", 2024-05-14T17:16:26.906 INFO:teuthology.orchestra.run.smithi002.stdout: "volumes" 2024-05-14T17:16:26.914 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:16:26.921 INFO:teuthology.orchestra.run.smithi002.stdout: "reef": [ 2024-05-14T17:16:26.928 INFO:teuthology.orchestra.run.smithi002.stdout: "balancer", 2024-05-14T17:16:26.936 INFO:teuthology.orchestra.run.smithi002.stdout: "crash", 2024-05-14T17:16:26.943 INFO:teuthology.orchestra.run.smithi002.stdout: "devicehealth", 2024-05-14T17:16:26.950 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator", 2024-05-14T17:16:26.957 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscaler", 2024-05-14T17:16:26.964 INFO:teuthology.orchestra.run.smithi002.stdout: "progress", 2024-05-14T17:16:26.971 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_support", 2024-05-14T17:16:26.979 INFO:teuthology.orchestra.run.smithi002.stdout: "status", 2024-05-14T17:16:26.986 INFO:teuthology.orchestra.run.smithi002.stdout: "telemetry", 2024-05-14T17:16:26.994 INFO:teuthology.orchestra.run.smithi002.stdout: "volumes" 2024-05-14T17:16:27.001 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-05-14T17:16:27.019 INFO:teuthology.orchestra.run.smithi002.stdout: "squid": [ 2024-05-14T17:16:27.043 INFO:teuthology.orchestra.run.smithi002.stdout: "balancer", 2024-05-14T17:16:27.050 INFO:teuthology.orchestra.run.smithi002.stdout: "crash", 2024-05-14T17:16:27.057 INFO:teuthology.orchestra.run.smithi002.stdout: "devicehealth", 2024-05-14T17:16:27.064 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator", 2024-05-14T17:16:27.071 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscaler", 2024-05-14T17:16:27.077 INFO:teuthology.orchestra.run.smithi002.stdout: "progress", 2024-05-14T17:16:27.084 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_support", 2024-05-14T17:16:27.091 INFO:teuthology.orchestra.run.smithi002.stdout: "status", 2024-05-14T17:16:27.098 INFO:teuthology.orchestra.run.smithi002.stdout: "telemetry", 2024-05-14T17:16:27.107 INFO:teuthology.orchestra.run.smithi002.stdout: "volumes" 2024-05-14T17:16:27.114 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-05-14T17:16:27.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:27.127 INFO:teuthology.orchestra.run.smithi002.stdout: "last_failure_osd_epoch": 81, 2024-05-14T17:16:27.134 INFO:teuthology.orchestra.run.smithi002.stdout: "active_clients": [ 2024-05-14T17:16:27.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:16:27.148 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "devicehealth", 2024-05-14T17:16:27.167 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-05-14T17:16:27.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:16:27.211 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-05-14T17:16:27.216 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2024-05-14T17:16:27.222 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1434957501 2024-05-14T17:16:27.227 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:27.233 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-05-14T17:16:27.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:27.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:16:27.252 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "libcephsqlite", 2024-05-14T17:16:27.259 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-05-14T17:16:27.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:16:27.270 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-05-14T17:16:27.276 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2024-05-14T17:16:27.283 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1122937930 2024-05-14T17:16:27.290 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:27.348 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-05-14T17:16:27.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:27.413 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:16:27.419 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_support", 2024-05-14T17:16:27.424 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-05-14T17:16:27.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:16:27.435 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-05-14T17:16:27.442 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2024-05-14T17:16:27.448 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3905855028 2024-05-14T17:16:27.457 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:27.463 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-05-14T17:16:27.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:27.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:16:27.484 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snap_schedule", 2024-05-14T17:16:27.490 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-05-14T17:16:27.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:16:27.504 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-05-14T17:16:27.509 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2024-05-14T17:16:27.523 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 203338902 2024-05-14T17:16:27.558 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:27.564 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-05-14T17:16:27.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:27.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:16:27.582 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snap_schedule", 2024-05-14T17:16:27.589 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-05-14T17:16:27.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:16:27.601 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-05-14T17:16:27.610 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2024-05-14T17:16:27.618 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 820400986 2024-05-14T17:16:27.625 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:27.634 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-05-14T17:16:27.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:27.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:16:27.657 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "volumes", 2024-05-14T17:16:27.666 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-05-14T17:16:27.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-05-14T17:16:27.699 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-05-14T17:16:27.708 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2024-05-14T17:16:27.717 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4208862962 2024-05-14T17:16:27.725 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:27.732 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-05-14T17:16:27.740 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:27.748 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-05-14T17:16:27.755 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T17:16:27.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.799+0000 7f39df4cc640 1 -- 172.21.15.2:0/1378451415 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f39c40b29e0 msgr2=0x7f39c40b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:27.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.799+0000 7f39df4cc640 1 --2- 172.21.15.2:0/1378451415 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f39c40b29e0 0x7f39c40b4ea0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f39c8005d00 tx=0x7f39c8009550 comp rx=0 tx=0).stop 2024-05-14T17:16:27.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.799+0000 7f39df4cc640 1 -- 172.21.15.2:0/1378451415 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f39d8101920 msgr2=0x7f39d819a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:16:27.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.799+0000 7f39df4cc640 1 --2- 172.21.15.2:0/1378451415 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f39d8101920 0x7f39d819a150 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f39d400ea80 tx=0x7f39d400ef40 comp rx=0 tx=0).stop 2024-05-14T17:16:27.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.799+0000 7f39df4cc640 1 -- 172.21.15.2:0/1378451415 shutdown_connections 2024-05-14T17:16:27.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.799+0000 7f39df4cc640 1 --2- 172.21.15.2:0/1378451415 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f39c40b29e0 0x7f39c40b4ea0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:27.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.799+0000 7f39df4cc640 1 --2- 172.21.15.2:0/1378451415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39d8107930 0x7f39d819abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:27.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.799+0000 7f39df4cc640 1 --2- 172.21.15.2:0/1378451415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39d81022e0 0x7f39d819a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:27.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.799+0000 7f39df4cc640 1 --2- 172.21.15.2:0/1378451415 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f39d8101920 0x7f39d819a150 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:16:27.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.799+0000 7f39df4cc640 1 -- 172.21.15.2:0/1378451415 >> 172.21.15.2:0/1378451415 conn(0x7f39d8077cd0 msgr2=0x7f39d80ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:16:27.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.799+0000 7f39df4cc640 1 -- 172.21.15.2:0/1378451415 shutdown_connections 2024-05-14T17:16:27.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:15:45.799+0000 7f39df4cc640 1 -- 172.21.15.2:0/1378451415 wait complete. 2024-05-14T17:16:27.925 DEBUG:tasks.check_counter:Getting stats from a 2024-05-14T17:16:27.933 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-14T17:16:27.948 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:45 smithi123 ceph-mon[35971]: from='client.17367 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:16:27.955 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:45 smithi123 ceph-mon[35971]: pgmap v664: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.9 KiB/s wr, 8 op/s 2024-05-14T17:16:27.963 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:45 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/3219333750' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-14T17:16:27.972 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:45 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/2641455778' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-14T17:16:27.978 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:45 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/4026311418' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-14T17:16:27.986 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:46 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/1378451415' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-14T17:16:27.994 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:47 smithi123 ceph-mon[35971]: pgmap v665: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-14T17:16:28.002 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:49 smithi123 ceph-mon[35971]: pgmap v666: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-05-14T17:16:28.010 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:51 smithi123 ceph-mon[35971]: pgmap v667: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-05-14T17:16:28.018 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:53 smithi123 ceph-mon[35971]: pgmap v668: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-05-14T17:16:28.026 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:55 smithi123 ceph-mon[35971]: pgmap v669: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 10 op/s 2024-05-14T17:16:28.049 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:57 smithi123 ceph-mon[35971]: pgmap v670: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.6 KiB/s wr, 7 op/s 2024-05-14T17:16:28.071 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:15:59 smithi123 ceph-mon[35971]: pgmap v671: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-05-14T17:16:28.079 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:16:01 smithi123 ceph-mon[35971]: pgmap v672: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:16:28.087 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:16:03 smithi123 ceph-mon[35971]: pgmap v673: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:16:28.095 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:16:05 smithi123 ceph-mon[35971]: pgmap v674: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-05-14T17:16:28.103 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:16:07 smithi123 ceph-mon[35971]: pgmap v675: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-05-14T17:16:28.111 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:16:09 smithi123 ceph-mon[35971]: pgmap v676: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-05-14T17:16:28.121 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:16:09 smithi123 ceph-mon[35971]: osdmap e111: 12 total, 12 up, 12 in 2024-05-14T17:16:28.129 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:16:11 smithi123 ceph-mon[35971]: pgmap v678: 97 pgs: 3 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 85 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-05-14T17:16:28.137 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:16:13 smithi123 ceph-mon[35971]: pgmap v679: 97 pgs: 3 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 85 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-05-14T17:16:28.145 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:16:15 smithi123 ceph-mon[35971]: pgmap v680: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:16:28.154 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:16:16 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:16:28.162 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:16:17 smithi123 ceph-mon[35971]: pgmap v681: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:16:28.171 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:16:17 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:16:28.178 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:16:17 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:16:28.186 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:16:17 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:16:28.193 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:16:17 smithi123 ceph-mon[35971]: osdmap e112: 12 total, 12 up, 12 in 2024-05-14T17:16:28.209 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:16:18 smithi123 ceph-mon[35971]: pgmap v682: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 115 B/s wr, 4 op/s 2024-05-14T17:16:28.245 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:16:18 smithi123 ceph-mon[35971]: pgmap v683: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 119 B/s wr, 4 op/s 2024-05-14T17:16:28.253 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:16:20 smithi123 ceph-mon[35971]: pgmap v685: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 299 B/s wr, 9 op/s 2024-05-14T17:16:28.260 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:16:22 smithi123 ceph-mon[35971]: pgmap v686: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 149 B/s wr, 4 op/s 2024-05-14T17:16:28.265 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:16:24 smithi123 ceph-mon[35971]: pgmap v687: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 149 B/s wr, 6 op/s 2024-05-14T17:16:28.272 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:16:26 smithi123 ceph-mon[35971]: pgmap v688: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 7 op/s 2024-05-14T17:16:28.282 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:45 smithi151 ceph-mon[36066]: from='client.17367 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-14T17:16:28.287 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:45 smithi151 ceph-mon[36066]: pgmap v664: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.9 KiB/s wr, 8 op/s 2024-05-14T17:16:28.294 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:45 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/3219333750' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-14T17:16:28.302 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:45 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/2641455778' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-14T17:16:28.316 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:45 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/4026311418' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-14T17:16:28.323 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:46 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/1378451415' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-14T17:16:28.329 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:47 smithi151 ceph-mon[36066]: pgmap v665: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-14T17:16:28.335 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:49 smithi151 ceph-mon[36066]: pgmap v666: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-05-14T17:16:28.341 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:51 smithi151 ceph-mon[36066]: pgmap v667: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-05-14T17:16:28.347 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:53 smithi151 ceph-mon[36066]: pgmap v668: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-05-14T17:16:28.356 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:55 smithi151 ceph-mon[36066]: pgmap v669: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 10 op/s 2024-05-14T17:16:28.389 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:57 smithi151 ceph-mon[36066]: pgmap v670: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.6 KiB/s wr, 7 op/s 2024-05-14T17:16:28.417 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:15:59 smithi151 ceph-mon[36066]: pgmap v671: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-05-14T17:16:28.424 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:16:01 smithi151 ceph-mon[36066]: pgmap v672: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:16:28.431 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:16:03 smithi151 ceph-mon[36066]: pgmap v673: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:16:28.437 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:16:05 smithi151 ceph-mon[36066]: pgmap v674: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-05-14T17:16:28.444 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:16:07 smithi151 ceph-mon[36066]: pgmap v675: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-05-14T17:16:28.451 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:16:09 smithi151 ceph-mon[36066]: pgmap v676: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-05-14T17:16:28.457 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:16:09 smithi151 ceph-mon[36066]: osdmap e111: 12 total, 12 up, 12 in 2024-05-14T17:16:28.463 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:16:11 smithi151 ceph-mon[36066]: pgmap v678: 97 pgs: 3 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 85 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-05-14T17:16:28.469 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:16:13 smithi151 ceph-mon[36066]: pgmap v679: 97 pgs: 3 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 85 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-05-14T17:16:28.475 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:16:15 smithi151 ceph-mon[36066]: pgmap v680: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:16:28.480 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:16:16 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:16:28.485 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:16:17 smithi151 ceph-mon[36066]: pgmap v681: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:16:28.491 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:16:17 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:16:28.499 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:16:17 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:16:28.505 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:16:17 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:16:28.512 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:16:17 smithi151 ceph-mon[36066]: osdmap e112: 12 total, 12 up, 12 in 2024-05-14T17:16:28.536 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:16:18 smithi151 ceph-mon[36066]: pgmap v682: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 115 B/s wr, 4 op/s 2024-05-14T17:16:28.564 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:16:18 smithi151 ceph-mon[36066]: pgmap v683: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 119 B/s wr, 4 op/s 2024-05-14T17:16:28.571 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:16:20 smithi151 ceph-mon[36066]: pgmap v685: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 299 B/s wr, 9 op/s 2024-05-14T17:16:28.577 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:16:22 smithi151 ceph-mon[36066]: pgmap v686: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 149 B/s wr, 4 op/s 2024-05-14T17:16:28.583 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:16:24 smithi151 ceph-mon[36066]: pgmap v687: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 149 B/s wr, 6 op/s 2024-05-14T17:16:28.589 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:16:26 smithi151 ceph-mon[36066]: pgmap v688: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 7 op/s 2024-05-14T17:16:28.598 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:46 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/1378451415' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-14T17:16:28.604 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:47 smithi002 ceph-mon[31538]: pgmap v665: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-14T17:16:28.611 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:49 smithi002 ceph-mon[31538]: pgmap v666: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-05-14T17:16:28.617 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:51 smithi002 ceph-mon[31538]: pgmap v667: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-05-14T17:16:28.623 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:53 smithi002 ceph-mon[31538]: pgmap v668: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-05-14T17:16:28.631 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:55 smithi002 ceph-mon[31538]: pgmap v669: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 10 op/s 2024-05-14T17:16:28.637 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:57 smithi002 ceph-mon[31538]: pgmap v670: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.6 KiB/s wr, 7 op/s 2024-05-14T17:16:28.643 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:15:59 smithi002 ceph-mon[31538]: pgmap v671: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-05-14T17:16:28.649 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:16:01 smithi002 ceph-mon[31538]: pgmap v672: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-14T17:16:28.657 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:16:03 smithi002 ceph-mon[31538]: pgmap v673: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:16:28.665 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:16:05 smithi002 ceph-mon[31538]: pgmap v674: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-05-14T17:16:28.686 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:16:07 smithi002 ceph-mon[31538]: pgmap v675: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-05-14T17:16:28.718 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:16:09 smithi002 ceph-mon[31538]: pgmap v676: 97 pgs: 97 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-05-14T17:16:28.727 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:16:09 smithi002 ceph-mon[31538]: osdmap e111: 12 total, 12 up, 12 in 2024-05-14T17:16:28.734 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:16:11 smithi002 ceph-mon[31538]: pgmap v678: 97 pgs: 3 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 85 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-05-14T17:16:28.742 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:16:13 smithi002 ceph-mon[31538]: pgmap v679: 97 pgs: 3 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 85 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-05-14T17:16:28.751 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:16:15 smithi002 ceph-mon[31538]: pgmap v680: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:16:28.759 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:16:16 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:16:28.767 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:16:17 smithi002 ceph-mon[31538]: pgmap v681: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:16:28.776 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:16:17 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:16:28.784 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:16:17 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:16:28.792 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:16:17 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:16:28.799 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:16:17 smithi002 ceph-mon[31538]: osdmap e112: 12 total, 12 up, 12 in 2024-05-14T17:16:28.808 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:16:18 smithi002 ceph-mon[31538]: pgmap v682: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 115 B/s wr, 4 op/s 2024-05-14T17:16:28.816 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:16:18 smithi002 ceph-mon[31538]: pgmap v683: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 119 B/s wr, 4 op/s 2024-05-14T17:16:28.824 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:16:20 smithi002 ceph-mon[31538]: pgmap v685: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 299 B/s wr, 9 op/s 2024-05-14T17:16:28.831 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:16:22 smithi002 ceph-mon[31538]: pgmap v686: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 149 B/s wr, 4 op/s 2024-05-14T17:16:28.840 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:16:24 smithi002 ceph-mon[31538]: pgmap v687: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 149 B/s wr, 6 op/s 2024-05-14T17:16:28.871 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:16:26 smithi002 ceph-mon[31538]: pgmap v688: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 7 op/s 2024-05-14T17:16:28.899 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:16:28 smithi151 ceph-mon[36066]: pgmap v689: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-14T17:16:28.911 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:16:28 smithi123 ceph-mon[35971]: pgmap v689: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-14T17:16:28.926 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:16:28 smithi002 ceph-mon[31538]: pgmap v689: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-14T17:16:29.161 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:16:30.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:16:30 smithi151 ceph-mon[36066]: pgmap v690: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 86 B/s wr, 7 op/s 2024-05-14T17:16:30.862 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:16:30 smithi002 ceph-mon[31538]: pgmap v690: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 86 B/s wr, 7 op/s 2024-05-14T17:16:30.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:16:30 smithi123 ceph-mon[35971]: pgmap v690: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 86 B/s wr, 7 op/s 2024-05-14T17:16:30.999 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T17:16:31.004 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T17:16:31.083 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 4, 2024-05-14T17:16:31.089 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 4, 2024-05-14T17:16:31.096 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 2944, 2024-05-14T17:16:31.103 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 448, 2024-05-14T17:16:31.109 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 4, 2024-05-14T17:16:31.115 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 0, 2024-05-14T17:16:31.122 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.001784067, 2024-05-14T17:16:31.191 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.000045953, 2024-05-14T17:16:31.284 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.000595960, 2024-05-14T17:16:31.290 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T17:16:31.298 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:16:31.304 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 3, 2024-05-14T17:16:31.312 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000828769, 2024-05-14T17:16:31.359 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000276256 2024-05-14T17:16:31.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:31.375 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:16:31.383 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-05-14T17:16:31.400 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-05-14T17:16:31.407 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-05-14T17:16:31.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:31.422 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-14T17:16:31.430 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 448 2024-05-14T17:16:31.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:31.446 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T17:16:31.470 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 3, 2024-05-14T17:16:31.502 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 308, 2024-05-14T17:16:31.509 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 480, 2024-05-14T17:16:31.517 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 290416, 2024-05-14T17:16:31.525 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 6, 2024-05-14T17:16:31.533 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 3, 2024-05-14T17:16:31.547 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.024095301, 2024-05-14T17:16:31.555 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.018083244, 2024-05-14T17:16:31.563 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.002166553, 2024-05-14T17:16:31.570 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T17:16:31.583 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:16:31.590 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 305, 2024-05-14T17:16:31.598 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.036479227, 2024-05-14T17:16:31.605 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000119604 2024-05-14T17:16:31.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:31.622 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:16:31.630 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-05-14T17:16:31.653 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-05-14T17:16:31.705 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-05-14T17:16:31.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:31.721 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-14T17:16:31.729 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 290416 2024-05-14T17:16:31.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:31.749 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T17:16:31.757 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 413, 2024-05-14T17:16:31.765 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 402, 2024-05-14T17:16:31.772 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 1765568, 2024-05-14T17:16:31.783 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 248736, 2024-05-14T17:16:31.790 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 2, 2024-05-14T17:16:31.797 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 1, 2024-05-14T17:16:31.803 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.073127708, 2024-05-14T17:16:31.808 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.025499977, 2024-05-14T17:16:31.815 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.046788333, 2024-05-14T17:16:31.821 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.009507009, 2024-05-14T17:16:31.837 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:16:31.880 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 401, 2024-05-14T17:16:31.887 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.018604820, 2024-05-14T17:16:31.892 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000046396 2024-05-14T17:16:31.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:31.907 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:16:31.914 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-05-14T17:16:31.921 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-05-14T17:16:31.927 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-05-14T17:16:31.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:31.942 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 1765568, 2024-05-14T17:16:31.948 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 248736 2024-05-14T17:16:31.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:31.961 INFO:teuthology.orchestra.run.smithi002.stdout: "cct": { 2024-05-14T17:16:31.966 INFO:teuthology.orchestra.run.smithi002.stdout: "total_workers": 0, 2024-05-14T17:16:31.972 INFO:teuthology.orchestra.run.smithi002.stdout: "unhealthy_workers": 0 2024-05-14T17:16:31.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:32.000 INFO:teuthology.orchestra.run.smithi002.stdout: "mempool": { 2024-05-14T17:16:32.023 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_bytes": 0, 2024-05-14T17:16:32.029 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_items": 0, 2024-05-14T17:16:32.034 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T17:16:32.041 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_items": 0, 2024-05-14T17:16:32.046 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T17:16:32.056 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_items": 0, 2024-05-14T17:16:32.062 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T17:16:32.068 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T17:16:32.074 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T17:16:32.084 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T17:16:32.090 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T17:16:32.097 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_items": 0, 2024-05-14T17:16:32.103 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T17:16:32.109 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T17:16:32.115 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_bytes": 0, 2024-05-14T17:16:32.123 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_items": 0, 2024-05-14T17:16:32.168 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_bytes": 0, 2024-05-14T17:16:32.205 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_items": 0, 2024-05-14T17:16:32.228 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T17:16:32.236 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T17:16:32.244 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T17:16:32.252 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T17:16:32.277 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T17:16:32.284 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_items": 0, 2024-05-14T17:16:32.290 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_bytes": 0, 2024-05-14T17:16:32.298 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_items": 0, 2024-05-14T17:16:32.309 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T17:16:32.317 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T17:16:32.323 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_bytes": 0, 2024-05-14T17:16:32.328 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_items": 0, 2024-05-14T17:16:32.334 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_bytes": 0, 2024-05-14T17:16:32.341 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_items": 0, 2024-05-14T17:16:32.350 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T17:16:32.378 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_items": 0, 2024-05-14T17:16:32.399 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T17:16:32.407 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_items": 0, 2024-05-14T17:16:32.416 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_bytes": 148714, 2024-05-14T17:16:32.427 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_items": 30, 2024-05-14T17:16:32.433 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_bytes": 0, 2024-05-14T17:16:32.444 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_items": 0, 2024-05-14T17:16:32.450 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_bytes": 0, 2024-05-14T17:16:32.456 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_items": 0, 2024-05-14T17:16:32.464 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_bytes": 0, 2024-05-14T17:16:32.476 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_items": 0, 2024-05-14T17:16:32.485 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_bytes": 0, 2024-05-14T17:16:32.493 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_items": 0, 2024-05-14T17:16:32.499 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_bytes": 0, 2024-05-14T17:16:32.505 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_items": 0, 2024-05-14T17:16:32.513 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T17:16:32.520 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_items": 0, 2024-05-14T17:16:32.549 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_bytes": 0, 2024-05-14T17:16:32.579 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_items": 0, 2024-05-14T17:16:32.587 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_bytes": 928, 2024-05-14T17:16:32.594 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_items": 2, 2024-05-14T17:16:32.601 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_bytes": 0, 2024-05-14T17:16:32.608 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_items": 0, 2024-05-14T17:16:32.628 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_bytes": 0, 2024-05-14T17:16:32.634 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_items": 0 2024-05-14T17:16:32.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:32.646 INFO:teuthology.orchestra.run.smithi002.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T17:16:32.660 INFO:teuthology.orchestra.run.smithi002.stdout: "val": 0, 2024-05-14T17:16:32.665 INFO:teuthology.orchestra.run.smithi002.stdout: "max": 104857600, 2024-05-14T17:16:32.674 INFO:teuthology.orchestra.run.smithi002.stdout: "get_started": 0, 2024-05-14T17:16:32.679 INFO:teuthology.orchestra.run.smithi002.stdout: "get": 416, 2024-05-14T17:16:32.686 INFO:teuthology.orchestra.run.smithi002.stdout: "get_sum": 1707220, 2024-05-14T17:16:32.692 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_fail": 0, 2024-05-14T17:16:32.698 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_success": 416, 2024-05-14T17:16:32.718 INFO:teuthology.orchestra.run.smithi002.stdout: "take": 0, 2024-05-14T17:16:32.766 INFO:teuthology.orchestra.run.smithi002.stdout: "take_sum": 0, 2024-05-14T17:16:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: "put": 416, 2024-05-14T17:16:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: "put_sum": 1707220, 2024-05-14T17:16:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: "wait": { 2024-05-14T17:16:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-05-14T17:16:32.803 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-05-14T17:16:32.809 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-05-14T17:16:32.816 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:32.823 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:32.835 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T17:16:32.844 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-05-14T17:16:32.850 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-05-14T17:16:32.857 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-05-14T17:16:32.863 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-05-14T17:16:32.869 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-05-14T17:16:32.875 DEBUG:tasks.check_counter:Getting stats from d 2024-05-14T17:16:32.892 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-14T17:16:32.932 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:16:32 smithi123 ceph-mon[35971]: pgmap v691: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:16:32.939 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:16:32 smithi151 ceph-mon[36066]: pgmap v691: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:16:32.975 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:16:32 smithi002 ceph-mon[31538]: pgmap v691: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:16:33.211 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:16:34.644 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:16:34 smithi002 ceph-mon[31538]: pgmap v692: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-05-14T17:16:34.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:16:34 smithi123 ceph-mon[35971]: pgmap v692: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-05-14T17:16:35.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:16:34 smithi151 ceph-mon[36066]: pgmap v692: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-05-14T17:16:35.215 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T17:16:35.222 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T17:16:35.237 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 0, 2024-05-14T17:16:35.244 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 2, 2024-05-14T17:16:35.250 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 0, 2024-05-14T17:16:35.256 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 0, 2024-05-14T17:16:35.263 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 4, 2024-05-14T17:16:35.270 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 0, 2024-05-14T17:16:35.276 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.001554726, 2024-05-14T17:16:35.313 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.000000000, 2024-05-14T17:16:35.389 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.000399214, 2024-05-14T17:16:35.395 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T17:16:35.401 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:16:35.407 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-05-14T17:16:35.416 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-05-14T17:16:35.470 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-05-14T17:16:35.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:35.482 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:16:35.487 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-05-14T17:16:35.502 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-05-14T17:16:35.508 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-05-14T17:16:35.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:35.520 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-14T17:16:35.527 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 0 2024-05-14T17:16:35.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:35.539 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T17:16:35.563 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 421, 2024-05-14T17:16:35.602 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 410, 2024-05-14T17:16:35.609 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 1767808, 2024-05-14T17:16:35.615 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 253376, 2024-05-14T17:16:35.623 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 2, 2024-05-14T17:16:35.631 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 1, 2024-05-14T17:16:35.654 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.071292233, 2024-05-14T17:16:35.660 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.021395861, 2024-05-14T17:16:35.666 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.047064431, 2024-05-14T17:16:35.673 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.010004968, 2024-05-14T17:16:35.685 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:16:35.692 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 409, 2024-05-14T17:16:35.698 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.023973072, 2024-05-14T17:16:35.706 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000058613 2024-05-14T17:16:35.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:35.718 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:16:35.724 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-05-14T17:16:35.738 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-05-14T17:16:35.762 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-05-14T17:16:35.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:35.774 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 1767808, 2024-05-14T17:16:35.780 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 253376 2024-05-14T17:16:35.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:35.804 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T17:16:35.813 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 7, 2024-05-14T17:16:35.820 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 315, 2024-05-14T17:16:35.828 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 3424, 2024-05-14T17:16:35.836 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 291744, 2024-05-14T17:16:35.844 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 6, 2024-05-14T17:16:35.853 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 3, 2024-05-14T17:16:35.863 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.025010523, 2024-05-14T17:16:35.870 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.018086682, 2024-05-14T17:16:35.878 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.002849173, 2024-05-14T17:16:35.886 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T17:16:35.901 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:16:35.980 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 313, 2024-05-14T17:16:35.989 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.024045317, 2024-05-14T17:16:35.998 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000076822 2024-05-14T17:16:36.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:36.014 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:16:36.038 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-05-14T17:16:36.046 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-05-14T17:16:36.057 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-05-14T17:16:36.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:36.079 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 3424, 2024-05-14T17:16:36.084 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 291744 2024-05-14T17:16:36.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:36.100 INFO:teuthology.orchestra.run.smithi002.stdout: "cct": { 2024-05-14T17:16:36.109 INFO:teuthology.orchestra.run.smithi002.stdout: "total_workers": 0, 2024-05-14T17:16:36.117 INFO:teuthology.orchestra.run.smithi002.stdout: "unhealthy_workers": 0 2024-05-14T17:16:36.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:36.143 INFO:teuthology.orchestra.run.smithi002.stdout: "mempool": { 2024-05-14T17:16:36.190 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_bytes": 0, 2024-05-14T17:16:36.199 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_items": 0, 2024-05-14T17:16:36.206 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T17:16:36.214 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_items": 0, 2024-05-14T17:16:36.222 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T17:16:36.233 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_items": 0, 2024-05-14T17:16:36.241 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T17:16:36.247 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T17:16:36.255 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T17:16:36.270 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T17:16:36.276 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T17:16:36.283 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_items": 0, 2024-05-14T17:16:36.291 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T17:16:36.299 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T17:16:36.306 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_bytes": 0, 2024-05-14T17:16:36.312 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_items": 0, 2024-05-14T17:16:36.331 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_bytes": 0, 2024-05-14T17:16:36.367 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_items": 0, 2024-05-14T17:16:36.375 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T17:16:36.381 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T17:16:36.387 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T17:16:36.393 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T17:16:36.417 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T17:16:36.424 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_items": 0, 2024-05-14T17:16:36.430 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_bytes": 0, 2024-05-14T17:16:36.437 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_items": 0, 2024-05-14T17:16:36.445 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T17:16:36.451 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T17:16:36.458 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_bytes": 0, 2024-05-14T17:16:36.465 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_items": 0, 2024-05-14T17:16:36.473 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_bytes": 0, 2024-05-14T17:16:36.478 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_items": 0, 2024-05-14T17:16:36.485 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T17:16:36.501 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_items": 0, 2024-05-14T17:16:36.577 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T17:16:36.583 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_items": 0, 2024-05-14T17:16:36.587 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_bytes": 149762, 2024-05-14T17:16:36.593 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_items": 32, 2024-05-14T17:16:36.599 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_bytes": 0, 2024-05-14T17:16:36.609 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_items": 0, 2024-05-14T17:16:36.614 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_bytes": 0, 2024-05-14T17:16:36.620 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_items": 0, 2024-05-14T17:16:36.626 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_bytes": 0, 2024-05-14T17:16:36.651 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_items": 0, 2024-05-14T17:16:36.657 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_bytes": 0, 2024-05-14T17:16:36.663 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_items": 0, 2024-05-14T17:16:36.669 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_bytes": 0, 2024-05-14T17:16:36.676 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_items": 0, 2024-05-14T17:16:36.684 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T17:16:36.690 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_items": 0, 2024-05-14T17:16:36.733 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_bytes": 0, 2024-05-14T17:16:36.788 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_items": 0, 2024-05-14T17:16:36.793 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_bytes": 928, 2024-05-14T17:16:36.802 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_items": 2, 2024-05-14T17:16:36.809 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_bytes": 0, 2024-05-14T17:16:36.816 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_items": 0, 2024-05-14T17:16:36.843 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_bytes": 0, 2024-05-14T17:16:36.853 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_items": 0 2024-05-14T17:16:36.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:36.868 INFO:teuthology.orchestra.run.smithi002.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T17:16:36.889 INFO:teuthology.orchestra.run.smithi002.stdout: "val": 0, 2024-05-14T17:16:36.900 INFO:teuthology.orchestra.run.smithi002.stdout: "max": 104857600, 2024-05-14T17:16:36.909 INFO:teuthology.orchestra.run.smithi002.stdout: "get_started": 0, 2024-05-14T17:16:36.917 INFO:teuthology.orchestra.run.smithi002.stdout: "get": 424, 2024-05-14T17:16:36.925 INFO:teuthology.orchestra.run.smithi002.stdout: "get_sum": 1708324, 2024-05-14T17:16:36.933 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_fail": 0, 2024-05-14T17:16:36.941 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_success": 424, 2024-05-14T17:16:36.967 INFO:teuthology.orchestra.run.smithi002.stdout: "take": 0, 2024-05-14T17:16:36.988 INFO:teuthology.orchestra.run.smithi002.stdout: "take_sum": 0, 2024-05-14T17:16:37.012 INFO:teuthology.orchestra.run.smithi002.stdout: "put": 424, 2024-05-14T17:16:37.019 INFO:teuthology.orchestra.run.smithi002.stdout: "put_sum": 1708324, 2024-05-14T17:16:37.027 INFO:teuthology.orchestra.run.smithi002.stdout: "wait": { 2024-05-14T17:16:37.034 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-05-14T17:16:37.050 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-05-14T17:16:37.056 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-05-14T17:16:37.064 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:37.074 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:37.090 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T17:16:37.099 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:16:36 smithi123 ceph-mon[35971]: pgmap v693: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T17:16:37.107 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-14T17:16:37.115 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-14T17:16:37.123 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-05-14T17:16:37.129 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-05-14T17:16:37.138 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-05-14T17:16:37.152 DEBUG:tasks.check_counter:Getting stats from g 2024-05-14T17:16:37.182 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-14T17:16:37.190 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:16:36 smithi002 ceph-mon[31538]: pgmap v693: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T17:16:37.198 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:16:36 smithi151 ceph-mon[36066]: pgmap v693: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T17:16:37.464 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:16:38.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:16:38 smithi123 ceph-mon[35971]: pgmap v694: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:16:38.951 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:16:38 smithi002 ceph-mon[31538]: pgmap v694: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:16:39.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:16:38 smithi151 ceph-mon[36066]: pgmap v694: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:16:39.369 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T17:16:39.376 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T17:16:39.414 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 417, 2024-05-14T17:16:39.424 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 406, 2024-05-14T17:16:39.435 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 1766720, 2024-05-14T17:16:39.443 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 250800, 2024-05-14T17:16:39.451 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 4, 2024-05-14T17:16:39.458 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 0, 2024-05-14T17:16:39.465 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.072772084, 2024-05-14T17:16:39.515 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.026728012, 2024-05-14T17:16:39.574 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.043122305, 2024-05-14T17:16:39.585 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.009159799, 2024-05-14T17:16:39.593 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:16:39.602 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 405, 2024-05-14T17:16:39.609 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.025356121, 2024-05-14T17:16:39.713 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000062607 2024-05-14T17:16:39.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:39.726 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:16:39.732 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-05-14T17:16:39.748 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-05-14T17:16:39.755 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-05-14T17:16:39.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:39.770 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 1766720, 2024-05-14T17:16:39.777 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 250800 2024-05-14T17:16:39.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:39.789 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T17:16:39.802 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 3, 2024-05-14T17:16:39.848 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 311, 2024-05-14T17:16:39.854 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 480, 2024-05-14T17:16:39.861 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 290944, 2024-05-14T17:16:39.869 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 6, 2024-05-14T17:16:39.877 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 3, 2024-05-14T17:16:39.893 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.024667292, 2024-05-14T17:16:39.900 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.018348515, 2024-05-14T17:16:39.906 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.002131379, 2024-05-14T17:16:39.913 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T17:16:39.923 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:16:39.928 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 308, 2024-05-14T17:16:39.935 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.026417995, 2024-05-14T17:16:39.941 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000085772 2024-05-14T17:16:39.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:39.956 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:16:39.962 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-05-14T17:16:39.977 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-05-14T17:16:39.998 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-05-14T17:16:40.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:40.012 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-14T17:16:40.018 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 290944 2024-05-14T17:16:40.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:40.051 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T17:16:40.057 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 4, 2024-05-14T17:16:40.063 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 4, 2024-05-14T17:16:40.072 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 2944, 2024-05-14T17:16:40.090 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 448, 2024-05-14T17:16:40.096 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 2, 2024-05-14T17:16:40.104 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 1, 2024-05-14T17:16:40.112 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.000647166, 2024-05-14T17:16:40.118 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.000034998, 2024-05-14T17:16:40.124 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.000580400, 2024-05-14T17:16:40.131 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T17:16:40.140 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:16:40.171 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 3, 2024-05-14T17:16:40.177 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000760789, 2024-05-14T17:16:40.188 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000253596 2024-05-14T17:16:40.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:40.202 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:16:40.210 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-05-14T17:16:40.218 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-05-14T17:16:40.227 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-05-14T17:16:40.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:40.248 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-14T17:16:40.255 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 448 2024-05-14T17:16:40.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:40.266 INFO:teuthology.orchestra.run.smithi002.stdout: "cct": { 2024-05-14T17:16:40.273 INFO:teuthology.orchestra.run.smithi002.stdout: "total_workers": 0, 2024-05-14T17:16:40.279 INFO:teuthology.orchestra.run.smithi002.stdout: "unhealthy_workers": 0 2024-05-14T17:16:40.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:40.335 INFO:teuthology.orchestra.run.smithi002.stdout: "mempool": { 2024-05-14T17:16:40.383 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_bytes": 0, 2024-05-14T17:16:40.392 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_items": 0, 2024-05-14T17:16:40.402 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T17:16:40.409 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_items": 0, 2024-05-14T17:16:40.418 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T17:16:40.469 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_items": 0, 2024-05-14T17:16:40.478 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T17:16:40.487 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T17:16:40.496 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T17:16:40.520 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T17:16:40.529 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T17:16:40.537 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_items": 0, 2024-05-14T17:16:40.545 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T17:16:40.554 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T17:16:40.561 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_bytes": 0, 2024-05-14T17:16:40.569 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_items": 0, 2024-05-14T17:16:40.601 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_bytes": 0, 2024-05-14T17:16:40.650 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_items": 0, 2024-05-14T17:16:40.656 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T17:16:40.664 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T17:16:40.672 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T17:16:40.681 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T17:16:40.690 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T17:16:40.698 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_items": 0, 2024-05-14T17:16:40.704 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_bytes": 0, 2024-05-14T17:16:40.711 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_items": 0, 2024-05-14T17:16:40.724 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T17:16:40.732 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T17:16:40.740 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_bytes": 0, 2024-05-14T17:16:40.749 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_items": 0, 2024-05-14T17:16:40.756 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_bytes": 0, 2024-05-14T17:16:40.766 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_items": 0, 2024-05-14T17:16:40.771 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T17:16:40.799 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_items": 0, 2024-05-14T17:16:40.823 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T17:16:40.829 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_items": 0, 2024-05-14T17:16:40.836 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_bytes": 148714, 2024-05-14T17:16:40.842 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_items": 30, 2024-05-14T17:16:40.849 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_bytes": 0, 2024-05-14T17:16:40.861 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_items": 0, 2024-05-14T17:16:40.868 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_bytes": 0, 2024-05-14T17:16:40.875 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_items": 0, 2024-05-14T17:16:40.882 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_bytes": 0, 2024-05-14T17:16:40.891 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_items": 0, 2024-05-14T17:16:40.898 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_bytes": 0, 2024-05-14T17:16:40.906 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_items": 0, 2024-05-14T17:16:40.920 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_bytes": 0, 2024-05-14T17:16:40.927 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_items": 0, 2024-05-14T17:16:40.934 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T17:16:40.940 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_items": 0, 2024-05-14T17:16:40.952 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_bytes": 0, 2024-05-14T17:16:40.988 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_items": 0, 2024-05-14T17:16:40.994 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_bytes": 928, 2024-05-14T17:16:41.000 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_items": 2, 2024-05-14T17:16:41.007 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_bytes": 0, 2024-05-14T17:16:41.014 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_items": 0, 2024-05-14T17:16:41.028 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_bytes": 0, 2024-05-14T17:16:41.033 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_items": 0 2024-05-14T17:16:41.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:41.049 INFO:teuthology.orchestra.run.smithi002.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T17:16:41.058 INFO:teuthology.orchestra.run.smithi002.stdout: "val": 0, 2024-05-14T17:16:41.065 INFO:teuthology.orchestra.run.smithi002.stdout: "max": 104857600, 2024-05-14T17:16:41.072 INFO:teuthology.orchestra.run.smithi002.stdout: "get_started": 0, 2024-05-14T17:16:41.079 INFO:teuthology.orchestra.run.smithi002.stdout: "get": 420, 2024-05-14T17:16:41.086 INFO:teuthology.orchestra.run.smithi002.stdout: "get_sum": 1707804, 2024-05-14T17:16:41.093 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_fail": 0, 2024-05-14T17:16:41.100 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_success": 420, 2024-05-14T17:16:41.116 INFO:teuthology.orchestra.run.smithi002.stdout: "take": 0, 2024-05-14T17:16:41.178 INFO:teuthology.orchestra.run.smithi002.stdout: "take_sum": 0, 2024-05-14T17:16:41.185 INFO:teuthology.orchestra.run.smithi002.stdout: "put": 420, 2024-05-14T17:16:41.193 INFO:teuthology.orchestra.run.smithi002.stdout: "put_sum": 1707804, 2024-05-14T17:16:41.199 INFO:teuthology.orchestra.run.smithi002.stdout: "wait": { 2024-05-14T17:16:41.205 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-05-14T17:16:41.228 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-05-14T17:16:41.233 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-05-14T17:16:41.240 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:41.247 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:41.283 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T17:16:41.290 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:16:40 smithi151 ceph-mon[36066]: pgmap v695: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:16:41.296 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:16:40 smithi123 ceph-mon[35971]: pgmap v695: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:16:41.304 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-05-14T17:16:41.311 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-05-14T17:16:41.320 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-05-14T17:16:41.328 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-05-14T17:16:41.341 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-05-14T17:16:41.365 DEBUG:tasks.check_counter:Getting stats from j 2024-05-14T17:16:41.421 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-14T17:16:41.434 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:16:40 smithi002 ceph-mon[31538]: pgmap v695: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:16:41.709 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:16:42.906 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:16:42 smithi002 ceph-mon[31538]: pgmap v696: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:16:42.913 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:16:42 smithi123 ceph-mon[35971]: pgmap v696: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:16:43.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:16:42 smithi151 ceph-mon[36066]: pgmap v696: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:16:43.504 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-05-14T17:16:43.513 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T17:16:43.521 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 0, 2024-05-14T17:16:43.529 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 2, 2024-05-14T17:16:43.537 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 0, 2024-05-14T17:16:43.544 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 0, 2024-05-14T17:16:43.551 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 4, 2024-05-14T17:16:43.559 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 0, 2024-05-14T17:16:43.566 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.001176547, 2024-05-14T17:16:43.628 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.000000000, 2024-05-14T17:16:43.708 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.000159704, 2024-05-14T17:16:43.717 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T17:16:43.725 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:16:43.734 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-05-14T17:16:43.743 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-05-14T17:16:43.787 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-05-14T17:16:43.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:43.801 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:16:43.808 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-05-14T17:16:43.822 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-05-14T17:16:43.830 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-05-14T17:16:43.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:43.844 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-14T17:16:43.850 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 0 2024-05-14T17:16:43.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:43.865 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T17:16:43.883 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 3, 2024-05-14T17:16:43.948 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 316, 2024-05-14T17:16:43.956 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 480, 2024-05-14T17:16:43.962 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 291824, 2024-05-14T17:16:43.970 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 6, 2024-05-14T17:16:43.976 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 3, 2024-05-14T17:16:43.986 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.024428642, 2024-05-14T17:16:43.991 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.018305790, 2024-05-14T17:16:43.997 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.002430558, 2024-05-14T17:16:44.004 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T17:16:44.016 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:16:44.025 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 313, 2024-05-14T17:16:44.033 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.020191395, 2024-05-14T17:16:44.040 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000064509 2024-05-14T17:16:44.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:44.053 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:16:44.059 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-05-14T17:16:44.074 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-05-14T17:16:44.135 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-05-14T17:16:44.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:44.149 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-14T17:16:44.155 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 291824 2024-05-14T17:16:44.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:44.178 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T17:16:44.184 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 428, 2024-05-14T17:16:44.191 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 415, 2024-05-14T17:16:44.199 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 1771568, 2024-05-14T17:16:44.219 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 255744, 2024-05-14T17:16:44.229 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 2, 2024-05-14T17:16:44.238 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 1, 2024-05-14T17:16:44.247 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.076262864, 2024-05-14T17:16:44.255 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.026449217, 2024-05-14T17:16:44.264 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.049407758, 2024-05-14T17:16:44.272 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.009518345, 2024-05-14T17:16:44.296 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:16:44.321 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 415, 2024-05-14T17:16:44.330 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.028200848, 2024-05-14T17:16:44.340 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000067953 2024-05-14T17:16:44.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:44.357 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:16:44.392 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-05-14T17:16:44.400 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-05-14T17:16:44.409 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-05-14T17:16:44.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:44.442 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 1771568, 2024-05-14T17:16:44.449 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 255744 2024-05-14T17:16:44.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:44.465 INFO:teuthology.orchestra.run.smithi002.stdout: "cct": { 2024-05-14T17:16:44.474 INFO:teuthology.orchestra.run.smithi002.stdout: "total_workers": 0, 2024-05-14T17:16:44.482 INFO:teuthology.orchestra.run.smithi002.stdout: "unhealthy_workers": 0 2024-05-14T17:16:44.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:44.513 INFO:teuthology.orchestra.run.smithi002.stdout: "mempool": { 2024-05-14T17:16:44.558 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_bytes": 0, 2024-05-14T17:16:44.567 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_items": 0, 2024-05-14T17:16:44.575 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T17:16:44.582 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_items": 0, 2024-05-14T17:16:44.590 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T17:16:44.612 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_items": 0, 2024-05-14T17:16:44.618 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T17:16:44.626 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T17:16:44.634 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T17:16:44.651 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T17:16:44.659 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T17:16:44.667 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_items": 0, 2024-05-14T17:16:44.675 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T17:16:44.681 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T17:16:44.689 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_bytes": 0, 2024-05-14T17:16:44.695 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_items": 0, 2024-05-14T17:16:44.707 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_bytes": 0, 2024-05-14T17:16:44.740 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_items": 0, 2024-05-14T17:16:44.746 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T17:16:44.752 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T17:16:44.760 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T17:16:44.766 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T17:16:44.821 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T17:16:44.827 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_items": 0, 2024-05-14T17:16:44.833 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_bytes": 0, 2024-05-14T17:16:44.845 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_items": 0, 2024-05-14T17:16:44.868 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T17:16:44.875 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T17:16:44.882 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_bytes": 0, 2024-05-14T17:16:44.889 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_items": 0, 2024-05-14T17:16:44.896 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_bytes": 0, 2024-05-14T17:16:44.903 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_items": 0, 2024-05-14T17:16:44.909 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T17:16:44.941 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_items": 0, 2024-05-14T17:16:44.985 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T17:16:44.992 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_items": 0, 2024-05-14T17:16:44.999 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_bytes": 148754, 2024-05-14T17:16:45.006 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_items": 31, 2024-05-14T17:16:45.013 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_bytes": 0, 2024-05-14T17:16:45.048 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_items": 0, 2024-05-14T17:16:45.057 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_bytes": 0, 2024-05-14T17:16:45.065 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_items": 0, 2024-05-14T17:16:45.070 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_bytes": 0, 2024-05-14T17:16:45.085 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_items": 0, 2024-05-14T17:16:45.094 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_bytes": 0, 2024-05-14T17:16:45.102 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_items": 0, 2024-05-14T17:16:45.112 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_bytes": 0, 2024-05-14T17:16:45.120 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_items": 0, 2024-05-14T17:16:45.129 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T17:16:45.138 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_items": 0, 2024-05-14T17:16:45.151 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_bytes": 0, 2024-05-14T17:16:45.182 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_items": 0, 2024-05-14T17:16:45.192 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_bytes": 928, 2024-05-14T17:16:45.202 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_items": 2, 2024-05-14T17:16:45.213 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_bytes": 0, 2024-05-14T17:16:45.221 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_items": 0, 2024-05-14T17:16:45.238 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_bytes": 0, 2024-05-14T17:16:45.245 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_items": 0 2024-05-14T17:16:45.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-05-14T17:16:45.261 INFO:teuthology.orchestra.run.smithi002.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T17:16:45.276 INFO:teuthology.orchestra.run.smithi002.stdout: "val": 0, 2024-05-14T17:16:45.285 INFO:teuthology.orchestra.run.smithi002.stdout: "max": 104857600, 2024-05-14T17:16:45.295 INFO:teuthology.orchestra.run.smithi002.stdout: "get_started": 0, 2024-05-14T17:16:45.304 INFO:teuthology.orchestra.run.smithi002.stdout: "get": 427, 2024-05-14T17:16:45.312 INFO:teuthology.orchestra.run.smithi002.stdout: "get_sum": 1708714, 2024-05-14T17:16:45.319 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_fail": 0, 2024-05-14T17:16:45.327 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_success": 427, 2024-05-14T17:16:45.342 INFO:teuthology.orchestra.run.smithi002.stdout: "take": 0, 2024-05-14T17:16:45.359 INFO:teuthology.orchestra.run.smithi002.stdout: "take_sum": 0, 2024-05-14T17:16:45.396 INFO:teuthology.orchestra.run.smithi002.stdout: "put": 427, 2024-05-14T17:16:45.405 INFO:teuthology.orchestra.run.smithi002.stdout: "put_sum": 1708714, 2024-05-14T17:16:45.413 INFO:teuthology.orchestra.run.smithi002.stdout: "wait": { 2024-05-14T17:16:45.422 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2024-05-14T17:16:45.449 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2024-05-14T17:16:45.458 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2024-05-14T17:16:45.467 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:45.476 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-05-14T17:16:45.485 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-05-14T17:16:45.494 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:16:44 smithi151 ceph-mon[36066]: pgmap v697: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-05-14T17:16:45.501 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:16:44 smithi123 ceph-mon[35971]: pgmap v697: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-05-14T17:16:45.511 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-14T17:16:45.520 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-14T17:16:45.528 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-05-14T17:16:45.534 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-05-14T17:16:45.559 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-05-14T17:16:45.590 DEBUG:tasks.check_counter:Getting stats from b 2024-05-14T17:16:45.599 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-14T17:16:45.608 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:16:44 smithi002 ceph-mon[31538]: pgmap v697: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-05-14T17:16:45.873 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T17:16:46.909 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:16:46 smithi123 ceph-mon[35971]: pgmap v698: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T17:16:46.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:16:46 smithi002 ceph-mon[31538]: pgmap v698: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T17:16:47.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:16:46 smithi151 ceph-mon[36066]: pgmap v698: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T17:16:47.777 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-05-14T17:16:47.784 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T17:16:47.819 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 6205, 2024-05-14T17:16:47.825 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 6421, 2024-05-14T17:16:47.832 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 2654351, 2024-05-14T17:16:47.839 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 20504025, 2024-05-14T17:16:47.845 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 239, 2024-05-14T17:16:47.852 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 6, 2024-05-14T17:16:47.860 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.592307332, 2024-05-14T17:16:47.909 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.221659984, 2024-05-14T17:16:47.984 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.263409918, 2024-05-14T17:16:47.992 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.092235038, 2024-05-14T17:16:47.998 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:16:48.005 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 6419, 2024-05-14T17:16:48.012 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.633225036, 2024-05-14T17:16:48.052 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000098648 2024-05-14T17:16:48.060 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:48.067 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:16:48.073 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 7384, 2024-05-14T17:16:48.085 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.001718397, 2024-05-14T17:16:48.092 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000232 2024-05-14T17:16:48.098 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:48.105 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 42544, 2024-05-14T17:16:48.113 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 1376 2024-05-14T17:16:48.120 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:48.129 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T17:16:48.151 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 8512, 2024-05-14T17:16:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 11271, 2024-05-14T17:16:48.185 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 3115942, 2024-05-14T17:16:48.192 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 21738805, 2024-05-14T17:16:48.199 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 17, 2024-05-14T17:16:48.205 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 5, 2024-05-14T17:16:48.231 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.719701462, 2024-05-14T17:16:48.239 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.268875665, 2024-05-14T17:16:48.246 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.392708075, 2024-05-14T17:16:48.254 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.150880823, 2024-05-14T17:16:48.269 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:16:48.276 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 11268, 2024-05-14T17:16:48.282 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 1.548151173, 2024-05-14T17:16:48.288 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000137393 2024-05-14T17:16:48.293 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:48.301 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:16:48.309 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 14299, 2024-05-14T17:16:48.318 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.001998759, 2024-05-14T17:16:48.329 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000139 2024-05-14T17:16:48.349 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:48.357 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 2471296, 2024-05-14T17:16:48.366 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 5556112 2024-05-14T17:16:48.375 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:48.401 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T17:16:48.409 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 7451, 2024-05-14T17:16:48.418 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 6548, 2024-05-14T17:16:48.427 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 5464008, 2024-05-14T17:16:48.444 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 8881800, 2024-05-14T17:16:48.451 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 10, 2024-05-14T17:16:48.459 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 7, 2024-05-14T17:16:48.468 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.713284044, 2024-05-14T17:16:48.476 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.267036700, 2024-05-14T17:16:48.484 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.317627261, 2024-05-14T17:16:48.493 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.155214976, 2024-05-14T17:16:48.511 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:16:48.540 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 6548, 2024-05-14T17:16:48.546 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.958801477, 2024-05-14T17:16:48.554 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000146426 2024-05-14T17:16:48.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:48.570 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:16:48.594 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 9950, 2024-05-14T17:16:48.600 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.001976943, 2024-05-14T17:16:48.609 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000198 2024-05-14T17:16:48.618 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:48.627 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 2693680, 2024-05-14T17:16:48.636 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 511600 2024-05-14T17:16:48.645 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:48.653 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-05-14T17:16:48.660 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 1, 2024-05-14T17:16:48.668 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-05-14T17:16:48.676 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:48.695 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-MDSRank": { 2024-05-14T17:16:48.726 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-05-14T17:16:48.734 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-05-14T17:16:48.743 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 4992, 2024-05-14T17:16:48.751 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 2.835909087, 2024-05-14T17:16:48.759 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000568090 2024-05-14T17:16:48.776 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-14T17:16:48.828 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:48.836 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-PurgeQueue": { 2024-05-14T17:16:48.844 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-05-14T17:16:48.861 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-05-14T17:16:48.868 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 190, 2024-05-14T17:16:48.877 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.006893104, 2024-05-14T17:16:48.883 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000036279 2024-05-14T17:16:48.890 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-14T17:16:48.896 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:48.902 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": { 2024-05-14T17:16:48.920 INFO:teuthology.orchestra.run.smithi123.stdout: "request": 2944, 2024-05-14T17:16:48.944 INFO:teuthology.orchestra.run.smithi123.stdout: "reply": 2611, 2024-05-14T17:16:48.951 INFO:teuthology.orchestra.run.smithi123.stdout: "reply_latency": { 2024-05-14T17:16:48.957 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2611, 2024-05-14T17:16:48.963 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 45.398156428, 2024-05-14T17:16:48.970 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.017387267 2024-05-14T17:16:48.995 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:49.001 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_reply": 0, 2024-05-14T17:16:49.008 INFO:teuthology.orchestra.run.smithi123.stdout: "forward": 333, 2024-05-14T17:16:49.014 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_complete": 8, 2024-05-14T17:16:49.028 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_keys": 0, 2024-05-14T17:16:49.033 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_commit": 1043, 2024-05-14T17:16:49.039 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_split": 4, 2024-05-14T17:16:49.046 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_merge": 0, 2024-05-14T17:16:49.052 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes": 453, 2024-05-14T17:16:49.058 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_top": 152, 2024-05-14T17:16:49.064 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_bottom": 291, 2024-05-14T17:16:49.120 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pin_tail": 10, 2024-05-14T17:16:49.156 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pinned": 129, 2024-05-14T17:16:49.162 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_expired": 2293, 2024-05-14T17:16:49.168 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_with_caps": 95, 2024-05-14T17:16:49.175 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": 95, 2024-05-14T17:16:49.180 INFO:teuthology.orchestra.run.smithi123.stdout: "subtrees": 12, 2024-05-14T17:16:49.193 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse": 3920, 2024-05-14T17:16:49.200 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_hit": 2932, 2024-05-14T17:16:49.207 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_forward": 330, 2024-05-14T17:16:49.212 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_discover": 14, 2024-05-14T17:16:49.221 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_dir_fetch": 0, 2024-05-14T17:16:49.228 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_remote_ino": 0, 2024-05-14T17:16:49.234 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_lock": 77, 2024-05-14T17:16:49.240 INFO:teuthology.orchestra.run.smithi123.stdout: "load_cent": 0, 2024-05-14T17:16:49.247 INFO:teuthology.orchestra.run.smithi123.stdout: "q": 0, 2024-05-14T17:16:49.253 INFO:teuthology.orchestra.run.smithi123.stdout: "exported": 40, 2024-05-14T17:16:49.261 INFO:teuthology.orchestra.run.smithi123.stdout: "exported_inodes": 2257, 2024-05-14T17:16:49.279 INFO:teuthology.orchestra.run.smithi123.stdout: "imported": 34, 2024-05-14T17:16:49.328 INFO:teuthology.orchestra.run.smithi123.stdout: "imported_inodes": 1468, 2024-05-14T17:16:49.336 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_dir_fetch": 0, 2024-05-14T17:16:49.345 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_backtrace_fetch": 0, 2024-05-14T17:16:49.354 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_peer_discover": 0, 2024-05-14T17:16:49.362 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rfiles": 1026, 2024-05-14T17:16:49.376 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rbytes": 1702131160, 2024-05-14T17:16:49.384 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rsnaps": 7, 2024-05-14T17:16:49.393 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_fetch": 0, 2024-05-14T17:16:49.400 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_set_tag": 0, 2024-05-14T17:16:49.463 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_repaired": 0, 2024-05-14T17:16:49.469 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_inotable_repaired": 0, 2024-05-14T17:16:49.477 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_inodes": 0, 2024-05-14T17:16:49.485 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_base_inodes": 0, 2024-05-14T17:16:49.494 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dirfrag_rstats": 0, 2024-05-14T17:16:49.502 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_file_inodes": 0, 2024-05-14T17:16:49.511 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_inode_file_caps": 34, 2024-05-14T17:16:49.529 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_revoke": 663, 2024-05-14T17:16:49.561 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_grant": 2026, 2024-05-14T17:16:49.566 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_trunc": 0, 2024-05-14T17:16:49.573 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-14T17:16:49.581 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-14T17:16:49.588 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps": 2048, 2024-05-14T17:16:49.602 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps_dirty": 1212, 2024-05-14T17:16:49.609 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_cap_release": 61, 2024-05-14T17:16:49.617 INFO:teuthology.orchestra.run.smithi123.stdout: "process_request_cap_release": 312 2024-05-14T17:16:49.624 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:49.646 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache": { 2024-05-14T17:16:49.654 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update": 0, 2024-05-14T17:16:49.662 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update_receipt": 0, 2024-05-14T17:16:49.670 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_try_discover": 0, 2024-05-14T17:16:49.678 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_send_discover": 46, 2024-05-14T17:16:49.687 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_handle_discover": 54, 2024-05-14T17:16:49.695 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays": 0, 2024-05-14T17:16:49.704 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_delayed": 0, 2024-05-14T17:16:49.725 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_enqueuing": 0, 2024-05-14T17:16:49.735 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_created": 104, 2024-05-14T17:16:49.741 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_enqueued": 61, 2024-05-14T17:16:49.749 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_reintegrated": 42, 2024-05-14T17:16:49.755 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_migrated": 7, 2024-05-14T17:16:49.768 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_processing": 0, 2024-05-14T17:16:49.776 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_enqueued": 0, 2024-05-14T17:16:49.782 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_prioritized": 0, 2024-05-14T17:16:49.788 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_started": 0, 2024-05-14T17:16:49.804 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_completed": 0, 2024-05-14T17:16:49.811 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_path": 9, 2024-05-14T17:16:49.819 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_inode": 2342, 2024-05-14T17:16:49.826 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_enqueue_scrub": 0, 2024-05-14T17:16:49.833 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_exportdir": 41, 2024-05-14T17:16:49.840 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_flush": 0, 2024-05-14T17:16:49.848 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragmentdir": 4, 2024-05-14T17:16:49.871 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragstats": 0, 2024-05-14T17:16:49.894 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_inodestats": 0 2024-05-14T17:16:49.901 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:49.908 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log": { 2024-05-14T17:16:49.914 INFO:teuthology.orchestra.run.smithi123.stdout: "evlrg": 0, 2024-05-14T17:16:49.920 INFO:teuthology.orchestra.run.smithi123.stdout: "evadd": 4334, 2024-05-14T17:16:49.927 INFO:teuthology.orchestra.run.smithi123.stdout: "evex": 4324, 2024-05-14T17:16:49.933 INFO:teuthology.orchestra.run.smithi123.stdout: "evtrm": 4324, 2024-05-14T17:16:49.940 INFO:teuthology.orchestra.run.smithi123.stdout: "ev": 10, 2024-05-14T17:16:49.947 INFO:teuthology.orchestra.run.smithi123.stdout: "evexg": 0, 2024-05-14T17:16:49.963 INFO:teuthology.orchestra.run.smithi123.stdout: "evexd": 1, 2024-05-14T17:16:49.969 INFO:teuthology.orchestra.run.smithi123.stdout: "segadd": 104, 2024-05-14T17:16:49.975 INFO:teuthology.orchestra.run.smithi123.stdout: "segex": 103, 2024-05-14T17:16:49.982 INFO:teuthology.orchestra.run.smithi123.stdout: "segtrm": 103, 2024-05-14T17:16:49.988 INFO:teuthology.orchestra.run.smithi123.stdout: "seg": 1, 2024-05-14T17:16:49.997 INFO:teuthology.orchestra.run.smithi123.stdout: "segmjr": 1, 2024-05-14T17:16:50.003 INFO:teuthology.orchestra.run.smithi123.stdout: "segexg": 0, 2024-05-14T17:16:50.013 INFO:teuthology.orchestra.run.smithi123.stdout: "segexd": 1, 2024-05-14T17:16:50.035 INFO:teuthology.orchestra.run.smithi123.stdout: "expos": 33803422, 2024-05-14T17:16:50.043 INFO:teuthology.orchestra.run.smithi123.stdout: "wrpos": 33868552, 2024-05-14T17:16:50.050 INFO:teuthology.orchestra.run.smithi123.stdout: "rdpos": 0, 2024-05-14T17:16:50.056 INFO:teuthology.orchestra.run.smithi123.stdout: "jlat": { 2024-05-14T17:16:50.061 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2000, 2024-05-14T17:16:50.069 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 13.266752437, 2024-05-14T17:16:50.075 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.006633376 2024-05-14T17:16:50.082 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:50.090 INFO:teuthology.orchestra.run.smithi123.stdout: "replayed": 0 2024-05-14T17:16:50.108 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:50.114 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_mem": { 2024-05-14T17:16:50.120 INFO:teuthology.orchestra.run.smithi123.stdout: "ino": 389, 2024-05-14T17:16:50.125 INFO:teuthology.orchestra.run.smithi123.stdout: "ino+": 2623, 2024-05-14T17:16:50.131 INFO:teuthology.orchestra.run.smithi123.stdout: "ino-": 2234, 2024-05-14T17:16:50.137 INFO:teuthology.orchestra.run.smithi123.stdout: "dir": 86, 2024-05-14T17:16:50.143 INFO:teuthology.orchestra.run.smithi123.stdout: "dir+": 150, 2024-05-14T17:16:50.164 INFO:teuthology.orchestra.run.smithi123.stdout: "dir-": 64, 2024-05-14T17:16:50.195 INFO:teuthology.orchestra.run.smithi123.stdout: "dn": 453, 2024-05-14T17:16:50.202 INFO:teuthology.orchestra.run.smithi123.stdout: "dn+": 3132, 2024-05-14T17:16:50.211 INFO:teuthology.orchestra.run.smithi123.stdout: "dn-": 2679, 2024-05-14T17:16:50.219 INFO:teuthology.orchestra.run.smithi123.stdout: "cap": 95, 2024-05-14T17:16:50.227 INFO:teuthology.orchestra.run.smithi123.stdout: "cap+": 1477, 2024-05-14T17:16:50.268 INFO:teuthology.orchestra.run.smithi123.stdout: "cap-": 1382, 2024-05-14T17:16:50.276 INFO:teuthology.orchestra.run.smithi123.stdout: "rss": 76652, 2024-05-14T17:16:50.286 INFO:teuthology.orchestra.run.smithi123.stdout: "heap": 231800 2024-05-14T17:16:50.294 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:50.308 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_server": { 2024-05-14T17:16:50.316 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_client_request": 3535, 2024-05-14T17:16:50.324 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_server_request": 228, 2024-05-14T17:16:50.333 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_request": 2944, 2024-05-14T17:16:50.339 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_session": 339, 2024-05-14T17:16:50.347 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_peer_request": 416, 2024-05-14T17:16:50.355 INFO:teuthology.orchestra.run.smithi123.stdout: "req_create_latency": { 2024-05-14T17:16:50.373 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1073, 2024-05-14T17:16:50.419 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 2.369880857, 2024-05-14T17:16:50.428 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.002208649 2024-05-14T17:16:50.434 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:50.442 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getattr_latency": { 2024-05-14T17:16:50.450 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 336, 2024-05-14T17:16:50.469 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.203039084, 2024-05-14T17:16:50.477 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000604282 2024-05-14T17:16:50.485 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:50.493 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getfilelock_latency": { 2024-05-14T17:16:50.508 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:16:50.517 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:16:50.529 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:16:50.539 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:50.546 INFO:teuthology.orchestra.run.smithi123.stdout: "req_link_latency": { 2024-05-14T17:16:50.553 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 42, 2024-05-14T17:16:50.561 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.110618007, 2024-05-14T17:16:50.571 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.002633762 2024-05-14T17:16:50.624 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:50.631 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookup_latency": { 2024-05-14T17:16:50.653 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 319, 2024-05-14T17:16:50.659 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 41.144777367, 2024-05-14T17:16:50.664 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.128980493 2024-05-14T17:16:50.678 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:50.683 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookuphash_latency": { 2024-05-14T17:16:50.688 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:16:50.695 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:16:50.715 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:16:50.722 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:50.728 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupino_latency": { 2024-05-14T17:16:50.736 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1, 2024-05-14T17:16:50.742 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000198814, 2024-05-14T17:16:50.748 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000198814 2024-05-14T17:16:50.755 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:50.763 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupname_latency": { 2024-05-14T17:16:50.794 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:16:50.800 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:16:50.806 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:16:50.812 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:50.818 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupparent_latency": { 2024-05-14T17:16:50.828 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:16:50.834 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:16:50.842 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:16:50.848 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:50.857 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupsnap_latency": { 2024-05-14T17:16:50.863 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:16:50.869 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:16:50.875 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:16:50.882 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:50.890 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:16:48 smithi151 ceph-mon[36066]: pgmap v699: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:16:50.895 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:16:48 smithi002 ceph-mon[31538]: pgmap v699: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:16:50.906 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lssnap_latency": { 2024-05-14T17:16:50.944 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 22, 2024-05-14T17:16:50.951 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.004195680, 2024-05-14T17:16:50.958 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000190712 2024-05-14T17:16:50.963 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:50.969 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mkdir_latency": { 2024-05-14T17:16:50.978 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 37, 2024-05-14T17:16:50.986 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.277192901, 2024-05-14T17:16:50.991 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.007491700 2024-05-14T17:16:50.998 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:51.007 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mknod_latency": { 2024-05-14T17:16:51.013 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:16:51.020 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:16:51.026 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:16:51.033 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:51.040 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mksnap_latency": { 2024-05-14T17:16:51.045 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 22, 2024-05-14T17:16:51.056 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.110660340, 2024-05-14T17:16:51.099 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.005030015 2024-05-14T17:16:51.107 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:51.116 INFO:teuthology.orchestra.run.smithi123.stdout: "req_open_latency": { 2024-05-14T17:16:51.123 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 513, 2024-05-14T17:16:51.131 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.177036559, 2024-05-14T17:16:51.145 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000345100 2024-05-14T17:16:51.152 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:51.161 INFO:teuthology.orchestra.run.smithi123.stdout: "req_readdir_latency": { 2024-05-14T17:16:51.169 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 57, 2024-05-14T17:16:51.183 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.103268509, 2024-05-14T17:16:51.189 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.001811728 2024-05-14T17:16:51.196 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:51.204 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rename_latency": { 2024-05-14T17:16:51.212 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 89, 2024-05-14T17:16:51.221 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.616050582, 2024-05-14T17:16:51.229 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.006921916 2024-05-14T17:16:51.248 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:51.298 INFO:teuthology.orchestra.run.smithi123.stdout: "req_renamesnap_latency": { 2024-05-14T17:16:51.305 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:16:51.313 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:16:51.321 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:16:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:51.337 INFO:teuthology.orchestra.run.smithi123.stdout: "req_snapdiff_latency": { 2024-05-14T17:16:51.344 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:16:51.351 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:16:51.358 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:16:51.366 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:51.371 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmdir_latency": { 2024-05-14T17:16:51.379 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2, 2024-05-14T17:16:51.387 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.009703563, 2024-05-14T17:16:51.396 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.004851781 2024-05-14T17:16:51.404 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:51.413 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmsnap_latency": { 2024-05-14T17:16:51.449 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 15, 2024-05-14T17:16:51.528 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.072987944, 2024-05-14T17:16:51.534 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.004865862 2024-05-14T17:16:51.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:51.546 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmxattr_latency": { 2024-05-14T17:16:51.553 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:16:51.586 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:16:51.593 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:16:51.599 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:51.605 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setattr_latency": { 2024-05-14T17:16:51.626 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1, 2024-05-14T17:16:51.632 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.001035087, 2024-05-14T17:16:51.639 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.001035087 2024-05-14T17:16:51.646 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:51.652 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setdirlayout_latency": { 2024-05-14T17:16:51.659 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:16:51.665 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:16:51.703 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:16:51.785 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:51.792 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setfilelock_latency": { 2024-05-14T17:16:51.799 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:16:51.806 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:16:51.812 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:16:51.821 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:51.827 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setlayout_latency": { 2024-05-14T17:16:51.833 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:16:51.839 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:16:51.850 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:16:51.859 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:51.865 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setxattr_latency": { 2024-05-14T17:16:51.871 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 7, 2024-05-14T17:16:51.878 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.021044704, 2024-05-14T17:16:51.884 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.003006386 2024-05-14T17:16:51.889 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:51.912 INFO:teuthology.orchestra.run.smithi123.stdout: "req_symlink_latency": { 2024-05-14T17:16:51.961 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:16:51.967 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:16:51.973 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:16:51.979 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:51.985 INFO:teuthology.orchestra.run.smithi123.stdout: "req_unlink_latency": { 2024-05-14T17:16:51.996 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 75, 2024-05-14T17:16:52.003 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.176466430, 2024-05-14T17:16:52.013 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.002352885 2024-05-14T17:16:52.020 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:52.027 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_revoke_eviction": 0, 2024-05-14T17:16:52.033 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_acquisition_throttle": 0, 2024-05-14T17:16:52.041 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getvxattr_latency": { 2024-05-14T17:16:52.049 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:16:52.058 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:16:52.066 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:16:52.073 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-14T17:16:52.087 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:52.147 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_sessions": { 2024-05-14T17:16:52.153 INFO:teuthology.orchestra.run.smithi123.stdout: "session_count": 2, 2024-05-14T17:16:52.160 INFO:teuthology.orchestra.run.smithi123.stdout: "session_add": 5, 2024-05-14T17:16:52.168 INFO:teuthology.orchestra.run.smithi123.stdout: "session_remove": 3, 2024-05-14T17:16:52.176 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_open": 2, 2024-05-14T17:16:52.188 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_stale": 0, 2024-05-14T17:16:52.195 INFO:teuthology.orchestra.run.smithi123.stdout: "total_load": 43, 2024-05-14T17:16:52.203 INFO:teuthology.orchestra.run.smithi123.stdout: "average_load": 21, 2024-05-14T17:16:52.211 INFO:teuthology.orchestra.run.smithi123.stdout: "avg_session_uptime": 1314, 2024-05-14T17:16:52.231 INFO:teuthology.orchestra.run.smithi123.stdout: "mdthresh_evicted": 0 2024-05-14T17:16:52.236 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:52.245 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-05-14T17:16:52.253 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 360, 2024-05-14T17:16:52.261 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 360, 2024-05-14T17:16:52.267 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T17:16:52.274 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-05-14T17:16:52.312 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T17:16:52.417 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-05-14T17:16:52.424 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T17:16:52.431 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T17:16:52.438 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T17:16:52.447 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T17:16:52.463 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T17:16:52.470 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-05-14T17:16:52.480 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T17:16:52.488 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T17:16:52.520 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-05-14T17:16:52.527 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-05-14T17:16:52.547 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-05-14T17:16:52.553 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-05-14T17:16:52.560 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T17:16:52.566 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T17:16:52.574 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T17:16:52.591 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T17:16:52.625 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T17:16:52.632 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-05-14T17:16:52.638 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-05-14T17:16:52.644 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-05-14T17:16:52.651 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T17:16:52.663 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T17:16:52.670 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-05-14T17:16:52.675 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-05-14T17:16:52.683 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-05-14T17:16:52.708 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-05-14T17:16:52.715 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T17:16:52.721 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-05-14T17:16:52.726 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T17:16:52.732 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-05-14T17:16:52.738 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 576485, 2024-05-14T17:16:52.745 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 546, 2024-05-14T17:16:52.758 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 88, 2024-05-14T17:16:52.800 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 1, 2024-05-14T17:16:52.809 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-05-14T17:16:52.817 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-05-14T17:16:52.826 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-05-14T17:16:52.834 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-05-14T17:16:52.842 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-05-14T17:16:52.851 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-05-14T17:16:52.861 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 8032, 2024-05-14T17:16:52.869 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 167, 2024-05-14T17:16:52.898 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T17:16:52.906 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-05-14T17:16:52.915 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-05-14T17:16:52.924 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-05-14T17:16:52.932 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 1537652, 2024-05-14T17:16:52.940 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 27498, 2024-05-14T17:16:52.949 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-05-14T17:16:52.962 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-05-14T17:16:52.975 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-05-14T17:16:53.016 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-05-14T17:16:53.025 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:53.034 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter": { 2024-05-14T17:16:53.042 INFO:teuthology.orchestra.run.smithi123.stdout: "op_active": 0, 2024-05-14T17:16:53.050 INFO:teuthology.orchestra.run.smithi123.stdout: "op_laggy": 0, 2024-05-14T17:16:53.059 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send": 5582, 2024-05-14T17:16:53.067 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send_bytes": 35664041, 2024-05-14T17:16:53.077 INFO:teuthology.orchestra.run.smithi123.stdout: "op_resend": 37, 2024-05-14T17:16:53.090 INFO:teuthology.orchestra.run.smithi123.stdout: "op_reply": 5582, 2024-05-14T17:16:53.097 INFO:teuthology.orchestra.run.smithi123.stdout: "op_latency": { 2024-05-14T17:16:53.105 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 5582, 2024-05-14T17:16:53.112 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 115.064291684, 2024-05-14T17:16:53.121 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.020613452 2024-05-14T17:16:53.129 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:53.137 INFO:teuthology.orchestra.run.smithi123.stdout: "op_inflight": 0, 2024-05-14T17:16:53.150 INFO:teuthology.orchestra.run.smithi123.stdout: "oplen_avg": { 2024-05-14T17:16:53.205 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 5582, 2024-05-14T17:16:53.212 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 10150 2024-05-14T17:16:53.220 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:53.228 INFO:teuthology.orchestra.run.smithi123.stdout: "op": 5582, 2024-05-14T17:16:53.237 INFO:teuthology.orchestra.run.smithi123.stdout: "op_r": 26, 2024-05-14T17:16:53.264 INFO:teuthology.orchestra.run.smithi123.stdout: "op_w": 5556, 2024-05-14T17:16:53.274 INFO:teuthology.orchestra.run.smithi123.stdout: "op_rmw": 0, 2024-05-14T17:16:53.283 INFO:teuthology.orchestra.run.smithi123.stdout: "op_pg": 0, 2024-05-14T17:16:53.291 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_stat": 964, 2024-05-14T17:16:53.299 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_create": 1183, 2024-05-14T17:16:53.306 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_read": 18, 2024-05-14T17:16:53.313 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_write": 2018, 2024-05-14T17:16:53.319 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writefull": 367, 2024-05-14T17:16:53.326 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writesame": 0, 2024-05-14T17:16:53.332 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_append": 0, 2024-05-14T17:16:53.341 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_zero": 0, 2024-05-14T17:16:53.354 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_truncate": 4, 2024-05-14T17:16:53.422 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_delete": 601, 2024-05-14T17:16:53.429 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_mapext": 0, 2024-05-14T17:16:53.435 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_sparse_read": 0, 2024-05-14T17:16:53.443 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_clonerange": 0, 2024-05-14T17:16:53.451 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_getxattr": 3, 2024-05-14T17:16:53.464 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_setxattr": 2366, 2024-05-14T17:16:53.472 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_cmpxattr": 0, 2024-05-14T17:16:53.478 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_rmxattr": 0, 2024-05-14T17:16:53.485 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_resetxattrs": 0, 2024-05-14T17:16:53.491 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_call": 0, 2024-05-14T17:16:53.499 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_watch": 0, 2024-05-14T17:16:53.505 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_notify": 0, 2024-05-14T17:16:53.513 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_src_cmpxattr": 0, 2024-05-14T17:16:53.520 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls": 0, 2024-05-14T17:16:53.527 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls_filter": 0, 2024-05-14T17:16:53.535 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_other": 0, 2024-05-14T17:16:53.551 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_active": 0, 2024-05-14T17:16:53.568 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_send": 0, 2024-05-14T17:16:53.575 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_resend": 0, 2024-05-14T17:16:53.581 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_ping": 0, 2024-05-14T17:16:53.589 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_active": 0, 2024-05-14T17:16:53.595 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_send": 0, 2024-05-14T17:16:53.606 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_resend": 0, 2024-05-14T17:16:53.612 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_active": 0, 2024-05-14T17:16:53.618 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_send": 0, 2024-05-14T17:16:53.625 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_resend": 0, 2024-05-14T17:16:53.641 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_active": 0, 2024-05-14T17:16:53.647 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_send": 0, 2024-05-14T17:16:53.653 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_resend": 0, 2024-05-14T17:16:53.659 INFO:teuthology.orchestra.run.smithi123.stdout: "command_active": 0, 2024-05-14T17:16:53.665 INFO:teuthology.orchestra.run.smithi123.stdout: "command_send": 0, 2024-05-14T17:16:53.672 INFO:teuthology.orchestra.run.smithi123.stdout: "command_resend": 0, 2024-05-14T17:16:53.679 INFO:teuthology.orchestra.run.smithi123.stdout: "map_epoch": 112, 2024-05-14T17:16:53.696 INFO:teuthology.orchestra.run.smithi123.stdout: "map_full": 0, 2024-05-14T17:16:53.763 INFO:teuthology.orchestra.run.smithi123.stdout: "map_inc": 34, 2024-05-14T17:16:53.770 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_sessions": 12, 2024-05-14T17:16:53.777 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_open": 12, 2024-05-14T17:16:53.785 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_close": 0, 2024-05-14T17:16:53.791 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_laggy": 0, 2024-05-14T17:16:53.798 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_wr": 2459, 2024-05-14T17:16:53.804 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_rd": 16, 2024-05-14T17:16:53.811 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_del": 151 2024-05-14T17:16:53.818 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:53.824 INFO:teuthology.orchestra.run.smithi123.stdout: "oft": { 2024-05-14T17:16:53.833 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_objs": 1, 2024-05-14T17:16:53.842 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_kv_pairs": 0, 2024-05-14T17:16:53.851 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_updates": 0, 2024-05-14T17:16:53.859 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_removes": 0 2024-05-14T17:16:53.867 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:53.876 INFO:teuthology.orchestra.run.smithi123.stdout: "osd-slow-ops": { 2024-05-14T17:16:53.893 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_ops_count": 2352 2024-05-14T17:16:53.972 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:53.980 INFO:teuthology.orchestra.run.smithi123.stdout: "purge_queue": { 2024-05-14T17:16:53.989 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops": 0, 2024-05-14T17:16:53.998 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops_high_water": 17, 2024-05-14T17:16:54.007 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing": 0, 2024-05-14T17:16:54.040 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_high_water": 10, 2024-05-14T17:16:54.047 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed_ops": 75, 2024-05-14T17:16:54.055 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed": 61, 2024-05-14T17:16:54.063 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_item_in_journal": 0 2024-05-14T17:16:54.074 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:54.085 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T17:16:54.091 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-05-14T17:16:54.098 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-05-14T17:16:54.107 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-05-14T17:16:54.116 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 22164, 2024-05-14T17:16:54.124 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 8809215, 2024-05-14T17:16:54.155 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-05-14T17:16:54.190 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 22164, 2024-05-14T17:16:54.200 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-05-14T17:16:54.210 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-05-14T17:16:54.218 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 22164, 2024-05-14T17:16:54.224 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 8809215, 2024-05-14T17:16:54.241 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-05-14T17:16:54.253 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:16:54.262 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:16:54.270 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:16:54.279 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-14T17:16:54.287 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:54.295 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_bytes": { 2024-05-14T17:16:54.303 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-05-14T17:16:54.311 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-05-14T17:16:54.320 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-05-14T17:16:54.328 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-05-14T17:16:54.356 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-05-14T17:16:54.375 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-05-14T17:16:54.458 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-05-14T17:16:54.466 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 5582, 2024-05-14T17:16:54.473 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 35670730, 2024-05-14T17:16:54.480 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 5582, 2024-05-14T17:16:54.508 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 35670730, 2024-05-14T17:16:54.515 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-05-14T17:16:54.523 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:16:54.530 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:16:54.580 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:16:54.586 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-14T17:16:54.593 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:54.599 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_ops": { 2024-05-14T17:16:54.607 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-05-14T17:16:54.614 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 1024, 2024-05-14T17:16:54.621 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-05-14T17:16:54.647 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-05-14T17:16:54.702 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-05-14T17:16:54.708 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-05-14T17:16:54.715 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-05-14T17:16:54.721 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 5582, 2024-05-14T17:16:54.729 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 5582, 2024-05-14T17:16:54.746 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 5582, 2024-05-14T17:16:54.753 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 5582, 2024-05-14T17:16:54.762 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-05-14T17:16:54.769 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:16:54.789 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:16:54.798 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:16:54.804 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-14T17:16:54.810 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:54.816 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle": { 2024-05-14T17:16:54.822 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-05-14T17:16:54.829 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-05-14T17:16:54.844 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-05-14T17:16:54.885 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 61, 2024-05-14T17:16:54.891 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 6817, 2024-05-14T17:16:54.897 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-05-14T17:16:54.904 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 61, 2024-05-14T17:16:54.911 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-05-14T17:16:54.925 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-05-14T17:16:54.931 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 18, 2024-05-14T17:16:54.937 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 6817, 2024-05-14T17:16:54.943 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-05-14T17:16:54.963 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:16:54.969 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:16:54.975 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:16:54.984 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-14T17:16:54.991 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:54.999 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle-0x5589c298c500": { 2024-05-14T17:16:55.007 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-05-14T17:16:55.031 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-05-14T17:16:55.069 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-05-14T17:16:55.077 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 4334, 2024-05-14T17:16:55.085 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 29674248, 2024-05-14T17:16:55.093 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-05-14T17:16:55.100 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 4334, 2024-05-14T17:16:55.109 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-05-14T17:16:55.118 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-05-14T17:16:55.126 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 2000, 2024-05-14T17:16:55.135 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 29674248, 2024-05-14T17:16:55.155 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-05-14T17:16:55.163 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:16:55.171 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:16:55.179 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:16:55.185 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-14T17:16:55.191 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-14T17:16:55.197 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-05-14T17:16:55.237 INFO:tasks.check_counter:Daemon mds.b mds.exported=40 2024-05-14T17:16:55.300 INFO:tasks.check_counter:Daemon mds.b mds.imported=34 2024-05-14T17:16:55.307 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=7 2024-05-14T17:16:55.313 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=22 2024-05-14T17:16:55.319 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=15 2024-05-14T17:16:55.325 DEBUG:tasks.check_counter:Getting stats from e 2024-05-14T17:16:55.337 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-14T17:16:55.343 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:16:48 smithi123 ceph-mon[35971]: pgmap v699: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:16:55.348 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:16:50 smithi123 ceph-mon[35971]: pgmap v700: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:16:55.353 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:16:52 smithi123 ceph-mon[35971]: pgmap v701: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:16:55.363 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:16:54 smithi123 ceph-mon[35971]: pgmap v702: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-05-14T17:16:55.371 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:16:50 smithi151 ceph-mon[36066]: pgmap v700: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:16:55.378 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:16:52 smithi151 ceph-mon[36066]: pgmap v701: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:16:55.384 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:16:54 smithi151 ceph-mon[36066]: pgmap v702: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-05-14T17:16:55.390 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:16:50 smithi002 ceph-mon[31538]: pgmap v700: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:16:55.396 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:16:52 smithi002 ceph-mon[31538]: pgmap v701: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:16:55.402 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:16:54 smithi002 ceph-mon[31538]: pgmap v702: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-05-14T17:16:55.625 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T17:16:56.906 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:16:56 smithi151 ceph-mon[36066]: pgmap v703: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T17:16:56.916 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:16:56 smithi123 ceph-mon[35971]: pgmap v703: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T17:16:56.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:16:56 smithi002 ceph-mon[31538]: pgmap v703: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T17:16:57.386 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-05-14T17:16:57.395 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T17:16:57.460 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 6458, 2024-05-14T17:16:57.465 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 6192, 2024-05-14T17:16:57.471 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 3522357, 2024-05-14T17:16:57.477 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 7576356, 2024-05-14T17:16:57.507 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 9, 2024-05-14T17:16:57.512 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 3, 2024-05-14T17:16:57.519 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.496520546, 2024-05-14T17:16:57.525 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.237889334, 2024-05-14T17:16:57.531 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.264260736, 2024-05-14T17:16:57.538 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.038986839, 2024-05-14T17:16:57.544 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:16:57.578 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 6192, 2024-05-14T17:16:57.607 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.336192805, 2024-05-14T17:16:57.660 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000054294 2024-05-14T17:16:57.667 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:57.672 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:16:57.677 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 9662, 2024-05-14T17:16:57.692 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.002001909, 2024-05-14T17:16:57.698 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000207 2024-05-14T17:16:57.703 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:57.710 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-14T17:16:57.722 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 0 2024-05-14T17:16:57.728 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:57.735 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T17:16:57.743 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 1200, 2024-05-14T17:16:57.751 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 1158, 2024-05-14T17:16:57.758 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 2178768, 2024-05-14T17:16:57.766 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 3746527, 2024-05-14T17:16:57.781 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 11, 2024-05-14T17:16:57.834 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 7, 2024-05-14T17:16:57.842 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.139557672, 2024-05-14T17:16:57.850 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.047838081, 2024-05-14T17:16:57.859 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.067096815, 2024-05-14T17:16:57.867 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.029577710, 2024-05-14T17:16:57.904 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:16:57.910 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1158, 2024-05-14T17:16:57.919 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.066073847, 2024-05-14T17:16:57.927 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000057058 2024-05-14T17:16:57.939 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:57.948 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:16:57.957 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 11, 2024-05-14T17:16:57.965 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000001590, 2024-05-14T17:16:57.974 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000144 2024-05-14T17:16:57.983 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:57.991 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 1981248, 2024-05-14T17:16:58.006 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 271408 2024-05-14T17:16:58.046 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:58.054 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T17:16:58.062 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 4263, 2024-05-14T17:16:58.071 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 6889, 2024-05-14T17:16:58.078 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 1587577, 2024-05-14T17:16:58.093 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 4706883, 2024-05-14T17:16:58.100 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 16, 2024-05-14T17:16:58.107 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 8, 2024-05-14T17:16:58.115 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.336610506, 2024-05-14T17:16:58.122 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.150821274, 2024-05-14T17:16:58.128 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.220532025, 2024-05-14T17:16:58.136 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.008333096, 2024-05-14T17:16:58.143 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:16:58.150 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 6884, 2024-05-14T17:16:58.157 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.419216771, 2024-05-14T17:16:58.164 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000060897 2024-05-14T17:16:58.178 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:58.211 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:16:58.217 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 8461, 2024-05-14T17:16:58.225 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.001968366, 2024-05-14T17:16:58.231 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000232 2024-05-14T17:16:58.238 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:58.252 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 1466688, 2024-05-14T17:16:58.259 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 4417568 2024-05-14T17:16:58.265 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:58.272 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-05-14T17:16:58.281 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 1, 2024-05-14T17:16:58.289 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-05-14T17:16:58.295 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:58.302 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-MDSRank": { 2024-05-14T17:16:58.308 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-05-14T17:16:58.315 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-05-14T17:16:58.322 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2276, 2024-05-14T17:16:58.336 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 1.738697290, 2024-05-14T17:16:58.394 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000763926 2024-05-14T17:16:58.400 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-14T17:16:58.406 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:58.413 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-PurgeQueue": { 2024-05-14T17:16:58.420 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-05-14T17:16:58.435 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-05-14T17:16:58.441 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 633, 2024-05-14T17:16:58.448 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.026949388, 2024-05-14T17:16:58.454 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000042574 2024-05-14T17:16:58.464 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-14T17:16:58.471 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:58.476 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": { 2024-05-14T17:16:58.484 INFO:teuthology.orchestra.run.smithi123.stdout: "request": 1740, 2024-05-14T17:16:58.490 INFO:teuthology.orchestra.run.smithi123.stdout: "reply": 1662, 2024-05-14T17:16:58.497 INFO:teuthology.orchestra.run.smithi123.stdout: "reply_latency": { 2024-05-14T17:16:58.502 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1662, 2024-05-14T17:16:58.515 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 64.133466855, 2024-05-14T17:16:58.547 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.038588126 2024-05-14T17:16:58.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:58.562 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_reply": 0, 2024-05-14T17:16:58.570 INFO:teuthology.orchestra.run.smithi123.stdout: "forward": 78, 2024-05-14T17:16:58.579 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_complete": 5, 2024-05-14T17:16:58.593 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_keys": 0, 2024-05-14T17:16:58.599 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_commit": 43, 2024-05-14T17:16:58.607 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_split": 4, 2024-05-14T17:16:58.615 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_merge": 0, 2024-05-14T17:16:58.640 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes": 791, 2024-05-14T17:16:58.648 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_top": 148, 2024-05-14T17:16:58.656 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_bottom": 632, 2024-05-14T17:16:58.664 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pin_tail": 11, 2024-05-14T17:16:58.672 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pinned": 254, 2024-05-14T17:16:58.680 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_expired": 1065, 2024-05-14T17:16:58.689 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_with_caps": 133, 2024-05-14T17:16:58.704 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": 133, 2024-05-14T17:16:58.766 INFO:teuthology.orchestra.run.smithi123.stdout: "subtrees": 31, 2024-05-14T17:16:58.774 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse": 3127, 2024-05-14T17:16:58.783 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_hit": 2797, 2024-05-14T17:16:58.791 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_forward": 65, 2024-05-14T17:16:58.799 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_discover": 37, 2024-05-14T17:16:58.810 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_dir_fetch": 2, 2024-05-14T17:16:58.817 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_remote_ino": 0, 2024-05-14T17:16:58.825 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_lock": 95, 2024-05-14T17:16:58.833 INFO:teuthology.orchestra.run.smithi123.stdout: "load_cent": 0, 2024-05-14T17:16:58.857 INFO:teuthology.orchestra.run.smithi123.stdout: "q": 0, 2024-05-14T17:16:58.864 INFO:teuthology.orchestra.run.smithi123.stdout: "exported": 31, 2024-05-14T17:16:58.869 INFO:teuthology.orchestra.run.smithi123.stdout: "exported_inodes": 1007, 2024-05-14T17:16:58.875 INFO:teuthology.orchestra.run.smithi123.stdout: "imported": 26, 2024-05-14T17:16:58.882 INFO:teuthology.orchestra.run.smithi123.stdout: "imported_inodes": 1388, 2024-05-14T17:16:58.889 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_dir_fetch": 0, 2024-05-14T17:16:58.896 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_backtrace_fetch": 0, 2024-05-14T17:16:58.912 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_peer_discover": 0, 2024-05-14T17:16:58.953 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rfiles": 1008, 2024-05-14T17:16:58.960 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rbytes": 1383322801, 2024-05-14T17:16:58.967 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rsnaps": 7, 2024-05-14T17:16:58.974 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_fetch": 0, 2024-05-14T17:16:58.979 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_set_tag": 0, 2024-05-14T17:16:58.995 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_repaired": 0, 2024-05-14T17:16:59.002 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_inotable_repaired": 0, 2024-05-14T17:16:59.008 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_inodes": 0, 2024-05-14T17:16:59.016 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_base_inodes": 0, 2024-05-14T17:16:59.041 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dirfrag_rstats": 0, 2024-05-14T17:16:59.048 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_file_inodes": 0, 2024-05-14T17:16:59.055 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_inode_file_caps": 6, 2024-05-14T17:16:59.062 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_revoke": 1131, 2024-05-14T17:16:59.069 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_grant": 1602, 2024-05-14T17:16:59.078 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_trunc": 3, 2024-05-14T17:16:59.084 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-14T17:16:59.104 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-14T17:16:59.163 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps": 1993, 2024-05-14T17:16:59.169 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps_dirty": 455, 2024-05-14T17:16:59.174 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_cap_release": 183, 2024-05-14T17:16:59.181 INFO:teuthology.orchestra.run.smithi123.stdout: "process_request_cap_release": 914 2024-05-14T17:16:59.188 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:59.212 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache": { 2024-05-14T17:16:59.218 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update": 0, 2024-05-14T17:16:59.225 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update_receipt": 0, 2024-05-14T17:16:59.232 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_try_discover": 0, 2024-05-14T17:16:59.266 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_send_discover": 45, 2024-05-14T17:16:59.275 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_handle_discover": 57, 2024-05-14T17:16:59.282 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays": 0, 2024-05-14T17:16:59.289 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_delayed": 0, 2024-05-14T17:16:59.295 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_enqueuing": 0, 2024-05-14T17:16:59.303 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_created": 210, 2024-05-14T17:16:59.310 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_enqueued": 198, 2024-05-14T17:16:59.343 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_reintegrated": 18, 2024-05-14T17:16:59.391 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_migrated": 8, 2024-05-14T17:16:59.398 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_processing": 0, 2024-05-14T17:16:59.404 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_enqueued": 0, 2024-05-14T17:16:59.410 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_prioritized": 0, 2024-05-14T17:16:59.417 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_started": 0, 2024-05-14T17:16:59.429 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_completed": 0, 2024-05-14T17:16:59.435 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_path": 9, 2024-05-14T17:16:59.441 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_inode": 5197, 2024-05-14T17:16:59.448 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_enqueue_scrub": 0, 2024-05-14T17:16:59.493 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_exportdir": 31, 2024-05-14T17:16:59.503 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_flush": 0, 2024-05-14T17:16:59.510 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragmentdir": 4, 2024-05-14T17:16:59.517 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragstats": 0, 2024-05-14T17:16:59.524 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_inodestats": 0 2024-05-14T17:16:59.531 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:59.541 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log": { 2024-05-14T17:16:59.558 INFO:teuthology.orchestra.run.smithi123.stdout: "evlrg": 0, 2024-05-14T17:16:59.630 INFO:teuthology.orchestra.run.smithi123.stdout: "evadd": 3214, 2024-05-14T17:16:59.637 INFO:teuthology.orchestra.run.smithi123.stdout: "evex": 0, 2024-05-14T17:16:59.645 INFO:teuthology.orchestra.run.smithi123.stdout: "evtrm": 0, 2024-05-14T17:16:59.652 INFO:teuthology.orchestra.run.smithi123.stdout: "ev": 3214, 2024-05-14T17:16:59.660 INFO:teuthology.orchestra.run.smithi123.stdout: "evexg": 0, 2024-05-14T17:16:59.709 INFO:teuthology.orchestra.run.smithi123.stdout: "evexd": 0, 2024-05-14T17:16:59.716 INFO:teuthology.orchestra.run.smithi123.stdout: "segadd": 6, 2024-05-14T17:16:59.723 INFO:teuthology.orchestra.run.smithi123.stdout: "segex": 0, 2024-05-14T17:16:59.730 INFO:teuthology.orchestra.run.smithi123.stdout: "segtrm": 0, 2024-05-14T17:16:59.764 INFO:teuthology.orchestra.run.smithi123.stdout: "seg": 6, 2024-05-14T17:16:59.773 INFO:teuthology.orchestra.run.smithi123.stdout: "segmjr": 2, 2024-05-14T17:16:59.780 INFO:teuthology.orchestra.run.smithi123.stdout: "segexg": 0, 2024-05-14T17:16:59.788 INFO:teuthology.orchestra.run.smithi123.stdout: "segexd": 0, 2024-05-14T17:16:59.798 INFO:teuthology.orchestra.run.smithi123.stdout: "expos": 4194304, 2024-05-14T17:16:59.805 INFO:teuthology.orchestra.run.smithi123.stdout: "wrpos": 11657249, 2024-05-14T17:16:59.813 INFO:teuthology.orchestra.run.smithi123.stdout: "rdpos": 0, 2024-05-14T17:16:59.830 INFO:teuthology.orchestra.run.smithi123.stdout: "jlat": { 2024-05-14T17:16:59.886 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1393, 2024-05-14T17:16:59.893 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 10.800803810, 2024-05-14T17:16:59.901 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.007753628 2024-05-14T17:16:59.908 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:59.915 INFO:teuthology.orchestra.run.smithi123.stdout: "replayed": 0 2024-05-14T17:16:59.929 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:16:59.935 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_mem": { 2024-05-14T17:16:59.943 INFO:teuthology.orchestra.run.smithi123.stdout: "ino": 770, 2024-05-14T17:16:59.949 INFO:teuthology.orchestra.run.smithi123.stdout: "ino+": 1944, 2024-05-14T17:16:59.985 INFO:teuthology.orchestra.run.smithi123.stdout: "ino-": 1174, 2024-05-14T17:16:59.992 INFO:teuthology.orchestra.run.smithi123.stdout: "dir": 96, 2024-05-14T17:16:59.999 INFO:teuthology.orchestra.run.smithi123.stdout: "dir+": 111, 2024-05-14T17:17:00.006 INFO:teuthology.orchestra.run.smithi123.stdout: "dir-": 15, 2024-05-14T17:17:00.014 INFO:teuthology.orchestra.run.smithi123.stdout: "dn": 791, 2024-05-14T17:17:00.024 INFO:teuthology.orchestra.run.smithi123.stdout: "dn+": 2074, 2024-05-14T17:17:00.032 INFO:teuthology.orchestra.run.smithi123.stdout: "dn-": 1283, 2024-05-14T17:17:00.047 INFO:teuthology.orchestra.run.smithi123.stdout: "cap": 133, 2024-05-14T17:17:00.099 INFO:teuthology.orchestra.run.smithi123.stdout: "cap+": 1688, 2024-05-14T17:17:00.105 INFO:teuthology.orchestra.run.smithi123.stdout: "cap-": 1555, 2024-05-14T17:17:00.112 INFO:teuthology.orchestra.run.smithi123.stdout: "rss": 67124, 2024-05-14T17:17:00.119 INFO:teuthology.orchestra.run.smithi123.stdout: "heap": 223608 2024-05-14T17:17:00.126 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:00.138 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_server": { 2024-05-14T17:17:00.144 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_client_request": 2304, 2024-05-14T17:17:00.149 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_server_request": 189, 2024-05-14T17:17:00.156 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_request": 1740, 2024-05-14T17:17:00.175 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_session": 64, 2024-05-14T17:17:00.182 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_peer_request": 433, 2024-05-14T17:17:00.187 INFO:teuthology.orchestra.run.smithi123.stdout: "req_create_latency": { 2024-05-14T17:17:00.194 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 196, 2024-05-14T17:17:00.212 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.192743142, 2024-05-14T17:17:00.220 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000983383 2024-05-14T17:17:00.227 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:00.243 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getattr_latency": { 2024-05-14T17:17:00.270 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 439, 2024-05-14T17:17:00.277 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 38.395295864, 2024-05-14T17:17:00.289 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.087460810 2024-05-14T17:17:00.296 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:00.303 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getfilelock_latency": { 2024-05-14T17:17:00.314 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:00.321 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:00.329 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:00.335 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:00.350 INFO:teuthology.orchestra.run.smithi123.stdout: "req_link_latency": { 2024-05-14T17:17:00.355 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 18, 2024-05-14T17:17:00.362 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.145152606, 2024-05-14T17:17:00.367 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.008064033 2024-05-14T17:17:00.373 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:00.378 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookup_latency": { 2024-05-14T17:17:00.385 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 484, 2024-05-14T17:17:00.398 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 4.424049028, 2024-05-14T17:17:00.426 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.009140597 2024-05-14T17:17:00.434 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:00.440 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookuphash_latency": { 2024-05-14T17:17:00.448 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:00.458 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:00.498 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:00.506 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:00.512 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupino_latency": { 2024-05-14T17:17:00.517 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1, 2024-05-14T17:17:00.525 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000164430, 2024-05-14T17:17:00.533 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000164430 2024-05-14T17:17:00.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:00.551 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupname_latency": { 2024-05-14T17:17:00.559 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:00.567 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:00.575 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:00.590 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:00.621 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupparent_latency": { 2024-05-14T17:17:00.631 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:00.639 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:00.648 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:00.656 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:00.673 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupsnap_latency": { 2024-05-14T17:17:00.682 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:00.690 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:00.699 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:00.716 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:00.726 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lssnap_latency": { 2024-05-14T17:17:00.739 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:16:58 smithi151 ceph-mon[36066]: pgmap v704: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:17:00.748 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:16:58 smithi002 ceph-mon[31538]: pgmap v704: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:17:00.759 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:00.768 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:00.778 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:00.822 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:00.881 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mkdir_latency": { 2024-05-14T17:17:00.889 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:00.896 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:00.902 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:00.909 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:00.923 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mknod_latency": { 2024-05-14T17:17:00.930 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:00.941 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:00.951 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:00.975 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:00.985 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mksnap_latency": { 2024-05-14T17:17:00.991 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:00.998 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:01.003 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:01.010 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:01.020 INFO:teuthology.orchestra.run.smithi123.stdout: "req_open_latency": { 2024-05-14T17:17:01.043 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 224, 2024-05-14T17:17:01.086 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.150506605, 2024-05-14T17:17:01.097 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000671904 2024-05-14T17:17:01.103 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:01.111 INFO:teuthology.orchestra.run.smithi123.stdout: "req_readdir_latency": { 2024-05-14T17:17:01.117 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 55, 2024-05-14T17:17:01.132 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.044150881, 2024-05-14T17:17:01.139 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000802743 2024-05-14T17:17:01.147 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:01.154 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rename_latency": { 2024-05-14T17:17:01.166 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 227, 2024-05-14T17:17:01.174 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 20.721996771, 2024-05-14T17:17:01.180 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.091286329 2024-05-14T17:17:01.186 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:01.191 INFO:teuthology.orchestra.run.smithi123.stdout: "req_renamesnap_latency": { 2024-05-14T17:17:01.201 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:01.211 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:01.258 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:01.320 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:01.328 INFO:teuthology.orchestra.run.smithi123.stdout: "req_snapdiff_latency": { 2024-05-14T17:17:01.338 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:01.349 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:01.364 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:01.384 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:01.391 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmdir_latency": { 2024-05-14T17:17:01.401 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:01.408 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:01.451 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:01.461 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:01.468 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmsnap_latency": { 2024-05-14T17:17:01.476 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:01.484 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:01.490 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:01.497 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:01.524 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmxattr_latency": { 2024-05-14T17:17:01.605 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:01.612 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:01.619 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:01.626 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:01.634 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setattr_latency": { 2024-05-14T17:17:01.649 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 3, 2024-05-14T17:17:01.656 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.019379071, 2024-05-14T17:17:01.663 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.006459690 2024-05-14T17:17:01.672 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:01.679 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setdirlayout_latency": { 2024-05-14T17:17:01.685 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:01.694 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:01.701 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:01.707 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:01.716 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setfilelock_latency": { 2024-05-14T17:17:01.728 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:01.750 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:01.787 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:01.796 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:01.810 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setlayout_latency": { 2024-05-14T17:17:01.819 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:01.829 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:01.867 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:01.880 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:01.892 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setxattr_latency": { 2024-05-14T17:17:01.900 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:01.912 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:01.919 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:01.934 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:01.943 INFO:teuthology.orchestra.run.smithi123.stdout: "req_symlink_latency": { 2024-05-14T17:17:01.950 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:01.960 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:01.967 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:01.983 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:02.022 INFO:teuthology.orchestra.run.smithi123.stdout: "req_unlink_latency": { 2024-05-14T17:17:02.031 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 15, 2024-05-14T17:17:02.042 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.040028457, 2024-05-14T17:17:02.049 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.002668563 2024-05-14T17:17:02.057 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:02.083 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_revoke_eviction": 0, 2024-05-14T17:17:02.095 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_acquisition_throttle": 0, 2024-05-14T17:17:02.104 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getvxattr_latency": { 2024-05-14T17:17:02.113 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:02.121 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:02.131 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:02.141 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-14T17:17:02.150 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:02.157 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_sessions": { 2024-05-14T17:17:02.164 INFO:teuthology.orchestra.run.smithi123.stdout: "session_count": 1, 2024-05-14T17:17:02.171 INFO:teuthology.orchestra.run.smithi123.stdout: "session_add": 3, 2024-05-14T17:17:02.201 INFO:teuthology.orchestra.run.smithi123.stdout: "session_remove": 2, 2024-05-14T17:17:02.252 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_open": 1, 2024-05-14T17:17:02.257 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_stale": 0, 2024-05-14T17:17:02.263 INFO:teuthology.orchestra.run.smithi123.stdout: "total_load": 124, 2024-05-14T17:17:02.270 INFO:teuthology.orchestra.run.smithi123.stdout: "average_load": 124, 2024-05-14T17:17:02.278 INFO:teuthology.orchestra.run.smithi123.stdout: "avg_session_uptime": 1338, 2024-05-14T17:17:02.318 INFO:teuthology.orchestra.run.smithi123.stdout: "mdthresh_evicted": 0 2024-05-14T17:17:02.329 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:02.341 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-05-14T17:17:02.352 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-05-14T17:17:02.363 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-05-14T17:17:02.373 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T17:17:02.383 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-05-14T17:17:02.391 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T17:17:02.409 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-05-14T17:17:02.420 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T17:17:02.430 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T17:17:02.443 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T17:17:02.474 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T17:17:02.481 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T17:17:02.490 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-05-14T17:17:02.501 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T17:17:02.513 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T17:17:02.534 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-05-14T17:17:02.546 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-05-14T17:17:02.554 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-05-14T17:17:02.567 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-05-14T17:17:02.580 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T17:17:02.588 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T17:17:02.597 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T17:17:02.606 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T17:17:02.615 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T17:17:02.624 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-05-14T17:17:02.634 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-05-14T17:17:02.645 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-05-14T17:17:02.658 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T17:17:02.667 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T17:17:02.675 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-05-14T17:17:02.682 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-05-14T17:17:02.690 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-05-14T17:17:02.709 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-05-14T17:17:02.715 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T17:17:02.724 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-05-14T17:17:02.733 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T17:17:02.741 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-05-14T17:17:02.747 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 529571, 2024-05-14T17:17:02.754 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 825, 2024-05-14T17:17:02.762 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2024-05-14T17:17:02.768 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2024-05-14T17:17:02.777 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-05-14T17:17:02.783 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-05-14T17:17:02.801 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-05-14T17:17:02.864 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-05-14T17:17:02.873 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-05-14T17:17:02.878 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-05-14T17:17:02.886 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 8032, 2024-05-14T17:17:02.893 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 167, 2024-05-14T17:17:02.902 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T17:17:02.909 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-05-14T17:17:02.917 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-05-14T17:17:02.924 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-05-14T17:17:02.947 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 2641184, 2024-05-14T17:17:02.955 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 34286, 2024-05-14T17:17:02.963 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-05-14T17:17:02.977 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-05-14T17:17:02.985 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-05-14T17:17:02.998 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-05-14T17:17:03.009 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:03.035 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter": { 2024-05-14T17:17:03.084 INFO:teuthology.orchestra.run.smithi123.stdout: "op_active": 0, 2024-05-14T17:17:03.094 INFO:teuthology.orchestra.run.smithi123.stdout: "op_laggy": 0, 2024-05-14T17:17:03.105 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send": 2109, 2024-05-14T17:17:03.119 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send_bytes": 7847319, 2024-05-14T17:17:03.126 INFO:teuthology.orchestra.run.smithi123.stdout: "op_resend": 33, 2024-05-14T17:17:03.136 INFO:teuthology.orchestra.run.smithi123.stdout: "op_reply": 2109, 2024-05-14T17:17:03.143 INFO:teuthology.orchestra.run.smithi123.stdout: "op_latency": { 2024-05-14T17:17:03.154 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2109, 2024-05-14T17:17:03.163 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 21.207507536, 2024-05-14T17:17:03.188 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.010055717 2024-05-14T17:17:03.195 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:03.207 INFO:teuthology.orchestra.run.smithi123.stdout: "op_inflight": 0, 2024-05-14T17:17:03.217 INFO:teuthology.orchestra.run.smithi123.stdout: "oplen_avg": { 2024-05-14T17:17:03.224 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2109, 2024-05-14T17:17:03.233 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 2343 2024-05-14T17:17:03.242 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:03.260 INFO:teuthology.orchestra.run.smithi123.stdout: "op": 2109, 2024-05-14T17:17:03.308 INFO:teuthology.orchestra.run.smithi123.stdout: "op_r": 77, 2024-05-14T17:17:03.320 INFO:teuthology.orchestra.run.smithi123.stdout: "op_w": 2032, 2024-05-14T17:17:03.332 INFO:teuthology.orchestra.run.smithi123.stdout: "op_rmw": 0, 2024-05-14T17:17:03.337 INFO:teuthology.orchestra.run.smithi123.stdout: "op_pg": 0, 2024-05-14T17:17:03.344 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_stat": 0, 2024-05-14T17:17:03.371 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_create": 10, 2024-05-14T17:17:03.380 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_read": 72, 2024-05-14T17:17:03.388 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_write": 1465, 2024-05-14T17:17:03.396 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writefull": 139, 2024-05-14T17:17:03.413 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writesame": 0, 2024-05-14T17:17:03.423 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_append": 0, 2024-05-14T17:17:03.431 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_zero": 0, 2024-05-14T17:17:03.440 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_truncate": 0, 2024-05-14T17:17:03.447 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_delete": 215, 2024-05-14T17:17:03.453 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_mapext": 0, 2024-05-14T17:17:03.461 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_sparse_read": 0, 2024-05-14T17:17:03.476 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_clonerange": 0, 2024-05-14T17:17:03.551 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_getxattr": 1, 2024-05-14T17:17:03.558 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_setxattr": 20, 2024-05-14T17:17:03.566 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_cmpxattr": 0, 2024-05-14T17:17:03.572 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_rmxattr": 0, 2024-05-14T17:17:03.579 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_resetxattrs": 0, 2024-05-14T17:17:03.645 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_call": 0, 2024-05-14T17:17:03.657 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_watch": 0, 2024-05-14T17:17:03.670 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_notify": 0, 2024-05-14T17:17:03.685 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_src_cmpxattr": 0, 2024-05-14T17:17:03.697 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls": 0, 2024-05-14T17:17:03.709 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls_filter": 0, 2024-05-14T17:17:03.722 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_other": 3, 2024-05-14T17:17:03.734 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_active": 0, 2024-05-14T17:17:03.743 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_send": 0, 2024-05-14T17:17:03.753 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_resend": 0, 2024-05-14T17:17:03.761 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_ping": 0, 2024-05-14T17:17:03.777 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_active": 0, 2024-05-14T17:17:03.812 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_send": 0, 2024-05-14T17:17:03.823 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_resend": 0, 2024-05-14T17:17:03.831 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_active": 0, 2024-05-14T17:17:03.841 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_send": 0, 2024-05-14T17:17:03.850 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_resend": 0, 2024-05-14T17:17:03.861 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_active": 0, 2024-05-14T17:17:03.869 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_send": 0, 2024-05-14T17:17:03.878 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_resend": 0, 2024-05-14T17:17:03.890 INFO:teuthology.orchestra.run.smithi123.stdout: "command_active": 0, 2024-05-14T17:17:03.899 INFO:teuthology.orchestra.run.smithi123.stdout: "command_send": 0, 2024-05-14T17:17:03.907 INFO:teuthology.orchestra.run.smithi123.stdout: "command_resend": 0, 2024-05-14T17:17:03.918 INFO:teuthology.orchestra.run.smithi123.stdout: "map_epoch": 112, 2024-05-14T17:17:03.928 INFO:teuthology.orchestra.run.smithi123.stdout: "map_full": 0, 2024-05-14T17:17:03.939 INFO:teuthology.orchestra.run.smithi123.stdout: "map_inc": 34, 2024-05-14T17:17:03.948 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_sessions": 12, 2024-05-14T17:17:03.965 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_open": 12, 2024-05-14T17:17:03.977 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_close": 0, 2024-05-14T17:17:04.073 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_laggy": 0, 2024-05-14T17:17:04.081 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_wr": 315, 2024-05-14T17:17:04.088 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_rd": 10, 2024-05-14T17:17:04.094 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_del": 93 2024-05-14T17:17:04.105 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:04.127 INFO:teuthology.orchestra.run.smithi123.stdout: "oft": { 2024-05-14T17:17:04.135 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_objs": 1, 2024-05-14T17:17:04.141 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_kv_pairs": 0, 2024-05-14T17:17:04.147 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_updates": 0, 2024-05-14T17:17:04.156 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_removes": 0 2024-05-14T17:17:04.162 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:04.170 INFO:teuthology.orchestra.run.smithi123.stdout: "osd-slow-ops": { 2024-05-14T17:17:04.177 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_ops_count": 5208 2024-05-14T17:17:04.185 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:04.192 INFO:teuthology.orchestra.run.smithi123.stdout: "purge_queue": { 2024-05-14T17:17:04.200 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops": 0, 2024-05-14T17:17:04.219 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops_high_water": 10, 2024-05-14T17:17:04.285 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing": 0, 2024-05-14T17:17:04.293 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_high_water": 10, 2024-05-14T17:17:04.306 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed_ops": 198, 2024-05-14T17:17:04.318 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed": 198, 2024-05-14T17:17:04.328 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_item_in_journal": 0 2024-05-14T17:17:04.355 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:04.366 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T17:17:04.377 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-05-14T17:17:04.388 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-05-14T17:17:04.400 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-05-14T17:17:04.408 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 11917, 2024-05-14T17:17:04.424 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 5993901, 2024-05-14T17:17:04.434 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-05-14T17:17:04.442 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 11917, 2024-05-14T17:17:04.452 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-05-14T17:17:04.459 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-05-14T17:17:04.469 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 11917, 2024-05-14T17:17:04.536 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 5993901, 2024-05-14T17:17:04.542 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-05-14T17:17:04.549 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:04.556 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:04.565 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:04.589 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-14T17:17:04.599 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:04.608 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_bytes": { 2024-05-14T17:17:04.615 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-05-14T17:17:04.625 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-05-14T17:17:04.630 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-05-14T17:17:04.639 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-05-14T17:17:04.645 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-05-14T17:17:04.653 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-05-14T17:17:04.659 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-05-14T17:17:04.666 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 2109, 2024-05-14T17:17:04.684 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 7867717, 2024-05-14T17:17:04.733 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 2109, 2024-05-14T17:17:04.741 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 7867717, 2024-05-14T17:17:04.749 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-05-14T17:17:04.757 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:04.765 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:04.804 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:04.814 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-14T17:17:04.830 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:04.836 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_ops": { 2024-05-14T17:17:04.857 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-05-14T17:17:04.868 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 1024, 2024-05-14T17:17:04.878 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-05-14T17:17:04.885 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-05-14T17:17:04.900 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-05-14T17:17:04.907 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-05-14T17:17:04.919 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-05-14T17:17:04.936 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 2109, 2024-05-14T17:17:04.963 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 2109, 2024-05-14T17:17:04.977 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 2109, 2024-05-14T17:17:04.985 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 2109, 2024-05-14T17:17:04.995 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-05-14T17:17:05.002 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:05.012 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:05.023 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:05.035 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-14T17:17:05.046 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:05.074 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle": { 2024-05-14T17:17:05.086 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-05-14T17:17:05.093 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-05-14T17:17:05.098 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-05-14T17:17:05.108 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 198, 2024-05-14T17:17:05.115 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 20478, 2024-05-14T17:17:05.123 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-05-14T17:17:05.137 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 198, 2024-05-14T17:17:05.167 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-05-14T17:17:05.174 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-05-14T17:17:05.181 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 72, 2024-05-14T17:17:05.194 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 20478, 2024-05-14T17:17:05.202 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-05-14T17:17:05.213 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:05.221 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:05.229 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:05.238 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-14T17:17:05.249 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:05.257 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle-0x55ec1561a500": { 2024-05-14T17:17:05.265 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-05-14T17:17:05.273 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-05-14T17:17:05.278 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-05-14T17:17:05.284 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 3214, 2024-05-14T17:17:05.293 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 7462945, 2024-05-14T17:17:05.303 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-05-14T17:17:05.342 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 3214, 2024-05-14T17:17:05.355 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-05-14T17:17:05.365 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-05-14T17:17:05.376 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 1393, 2024-05-14T17:17:05.385 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 7462945, 2024-05-14T17:17:05.400 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-05-14T17:17:05.413 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:05.425 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:05.438 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:05.457 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-14T17:17:05.473 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-14T17:17:05.483 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-05-14T17:17:05.493 INFO:tasks.check_counter:Daemon mds.e mds.exported=31 2024-05-14T17:17:05.504 INFO:tasks.check_counter:Daemon mds.e mds.imported=26 2024-05-14T17:17:05.512 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=7 2024-05-14T17:17:05.525 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-05-14T17:17:05.545 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-05-14T17:17:05.590 DEBUG:tasks.check_counter:Getting stats from h 2024-05-14T17:17:05.598 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-14T17:17:05.606 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:16:58 smithi123 ceph-mon[35971]: pgmap v704: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:17:05.612 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:00 smithi123 ceph-mon[35971]: pgmap v705: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:17:05.620 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:02 smithi123 ceph-mon[35971]: pgmap v706: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:17:05.629 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:03 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:17:05.640 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:04 smithi123 ceph-mon[35971]: pgmap v707: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-05-14T17:17:05.648 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:00 smithi151 ceph-mon[36066]: pgmap v705: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:17:05.657 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:02 smithi151 ceph-mon[36066]: pgmap v706: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:17:05.674 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:03 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:17:05.685 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:04 smithi151 ceph-mon[36066]: pgmap v707: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-05-14T17:17:05.692 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:00 smithi002 ceph-mon[31538]: pgmap v705: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:17:05.698 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:02 smithi002 ceph-mon[31538]: pgmap v706: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:17:05.703 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:03 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:17:05.708 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:04 smithi002 ceph-mon[31538]: pgmap v707: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-05-14T17:17:05.902 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T17:17:06.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:06 smithi002 ceph-mon[31538]: pgmap v708: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-05-14T17:17:07.070 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:06 smithi123 ceph-mon[35971]: pgmap v708: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-05-14T17:17:07.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:06 smithi151 ceph-mon[36066]: pgmap v708: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-05-14T17:17:07.679 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-05-14T17:17:07.690 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T17:17:07.703 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 3076, 2024-05-14T17:17:07.838 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 3049, 2024-05-14T17:17:07.851 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 1536510, 2024-05-14T17:17:07.861 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 8412860, 2024-05-14T17:17:07.868 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 17, 2024-05-14T17:17:07.922 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 5, 2024-05-14T17:17:07.932 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.209326448, 2024-05-14T17:17:07.940 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.085406413, 2024-05-14T17:17:07.949 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.089942495, 2024-05-14T17:17:07.958 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.037340126, 2024-05-14T17:17:07.968 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:17:07.977 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 3046, 2024-05-14T17:17:08.045 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.130455419, 2024-05-14T17:17:08.160 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000042828 2024-05-14T17:17:08.167 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:08.175 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:17:08.184 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2656, 2024-05-14T17:17:08.192 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000457493, 2024-05-14T17:17:08.209 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000172 2024-05-14T17:17:08.216 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:08.222 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 2512, 2024-05-14T17:17:08.229 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 2656 2024-05-14T17:17:08.249 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:08.258 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T17:17:08.267 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 8547, 2024-05-14T17:17:08.273 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 10402, 2024-05-14T17:17:08.281 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 3721104, 2024-05-14T17:17:08.290 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 6550546, 2024-05-14T17:17:08.300 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 11, 2024-05-14T17:17:08.333 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 7, 2024-05-14T17:17:08.400 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.604851804, 2024-05-14T17:17:08.408 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.296330305, 2024-05-14T17:17:08.420 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.381555058, 2024-05-14T17:17:08.428 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.008018917, 2024-05-14T17:17:08.436 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:17:08.450 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 10402, 2024-05-14T17:17:08.458 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.472985938, 2024-05-14T17:17:08.468 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000045470 2024-05-14T17:17:08.485 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:08.506 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:17:08.514 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 16306, 2024-05-14T17:17:08.520 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.002904565, 2024-05-14T17:17:08.527 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000178 2024-05-14T17:17:08.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:08.541 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 1455456, 2024-05-14T17:17:08.549 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 4076336 2024-05-14T17:17:08.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:08.610 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T17:17:08.619 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 779, 2024-05-14T17:17:08.627 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 771, 2024-05-14T17:17:08.635 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 2075802, 2024-05-14T17:17:08.643 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 475866, 2024-05-14T17:17:08.651 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 8, 2024-05-14T17:17:08.659 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 7, 2024-05-14T17:17:08.667 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.106885839, 2024-05-14T17:17:08.675 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.034192381, 2024-05-14T17:17:08.689 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.063302484, 2024-05-14T17:17:08.697 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.017566889, 2024-05-14T17:17:08.706 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:17:08.714 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 769, 2024-05-14T17:17:08.722 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.048577218, 2024-05-14T17:17:08.731 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000063169 2024-05-14T17:17:08.737 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:08.758 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:17:08.803 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 71, 2024-05-14T17:17:08.810 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000017565, 2024-05-14T17:17:08.817 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000247 2024-05-14T17:17:08.824 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:08.831 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 1996112, 2024-05-14T17:17:08.843 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 284272 2024-05-14T17:17:08.848 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:08.854 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-05-14T17:17:08.861 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 1, 2024-05-14T17:17:08.873 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-05-14T17:17:08.881 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:08.887 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-MDSRank": { 2024-05-14T17:17:08.893 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-05-14T17:17:08.900 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-05-14T17:17:08.906 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2947, 2024-05-14T17:17:08.912 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 1.529674687, 2024-05-14T17:17:08.929 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000519061 2024-05-14T17:17:08.996 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-14T17:17:09.003 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:09.008 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-PurgeQueue": { 2024-05-14T17:17:09.014 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-05-14T17:17:09.020 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-05-14T17:17:09.026 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 28, 2024-05-14T17:17:09.033 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000907235, 2024-05-14T17:17:09.039 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000032401 2024-05-14T17:17:09.046 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-14T17:17:09.056 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:09.063 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": { 2024-05-14T17:17:09.069 INFO:teuthology.orchestra.run.smithi123.stdout: "request": 1767, 2024-05-14T17:17:09.076 INFO:teuthology.orchestra.run.smithi123.stdout: "reply": 1709, 2024-05-14T17:17:09.084 INFO:teuthology.orchestra.run.smithi123.stdout: "reply_latency": { 2024-05-14T17:17:09.090 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1709, 2024-05-14T17:17:09.095 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 60.854519932, 2024-05-14T17:17:09.125 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.035608262 2024-05-14T17:17:09.183 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:09.189 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_reply": 0, 2024-05-14T17:17:09.196 INFO:teuthology.orchestra.run.smithi123.stdout: "forward": 58, 2024-05-14T17:17:09.203 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_complete": 8, 2024-05-14T17:17:09.208 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_keys": 0, 2024-05-14T17:17:09.239 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_commit": 27, 2024-05-14T17:17:09.246 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_split": 2, 2024-05-14T17:17:09.252 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_merge": 0, 2024-05-14T17:17:09.261 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes": 1021, 2024-05-14T17:17:09.273 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_top": 161, 2024-05-14T17:17:09.282 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_bottom": 784, 2024-05-14T17:17:09.290 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pin_tail": 76, 2024-05-14T17:17:09.297 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pinned": 653, 2024-05-14T17:17:09.306 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_expired": 910, 2024-05-14T17:17:09.314 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_with_caps": 191, 2024-05-14T17:17:09.322 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": 191, 2024-05-14T17:17:09.342 INFO:teuthology.orchestra.run.smithi123.stdout: "subtrees": 30, 2024-05-14T17:17:09.421 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse": 4028, 2024-05-14T17:17:09.430 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_hit": 3148, 2024-05-14T17:17:09.438 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_forward": 51, 2024-05-14T17:17:09.446 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_discover": 31, 2024-05-14T17:17:09.455 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_dir_fetch": 0, 2024-05-14T17:17:09.477 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_remote_ino": 0, 2024-05-14T17:17:09.486 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_lock": 260, 2024-05-14T17:17:09.494 INFO:teuthology.orchestra.run.smithi123.stdout: "load_cent": 0, 2024-05-14T17:17:09.502 INFO:teuthology.orchestra.run.smithi123.stdout: "q": 0, 2024-05-14T17:17:09.515 INFO:teuthology.orchestra.run.smithi123.stdout: "exported": 15, 2024-05-14T17:17:09.523 INFO:teuthology.orchestra.run.smithi123.stdout: "exported_inodes": 842, 2024-05-14T17:17:09.531 INFO:teuthology.orchestra.run.smithi123.stdout: "imported": 26, 2024-05-14T17:17:09.541 INFO:teuthology.orchestra.run.smithi123.stdout: "imported_inodes": 1250, 2024-05-14T17:17:09.552 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_dir_fetch": 0, 2024-05-14T17:17:09.559 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_backtrace_fetch": 0, 2024-05-14T17:17:09.568 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_peer_discover": 0, 2024-05-14T17:17:09.596 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rfiles": 1008, 2024-05-14T17:17:09.684 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rbytes": 1383322801, 2024-05-14T17:17:09.691 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rsnaps": 7, 2024-05-14T17:17:09.699 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_fetch": 0, 2024-05-14T17:17:09.707 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_set_tag": 0, 2024-05-14T17:17:09.715 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_repaired": 0, 2024-05-14T17:17:09.727 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_inotable_repaired": 0, 2024-05-14T17:17:09.736 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_inodes": 0, 2024-05-14T17:17:09.744 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_base_inodes": 0, 2024-05-14T17:17:09.751 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dirfrag_rstats": 0, 2024-05-14T17:17:09.774 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_file_inodes": 0, 2024-05-14T17:17:09.783 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_inode_file_caps": 13, 2024-05-14T17:17:09.791 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_revoke": 1162, 2024-05-14T17:17:09.801 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_grant": 1517, 2024-05-14T17:17:09.809 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_trunc": 0, 2024-05-14T17:17:09.818 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-14T17:17:09.826 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-14T17:17:09.838 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps": 1991, 2024-05-14T17:17:09.881 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps_dirty": 574, 2024-05-14T17:17:09.887 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_cap_release": 63, 2024-05-14T17:17:09.895 INFO:teuthology.orchestra.run.smithi123.stdout: "process_request_cap_release": 693 2024-05-14T17:17:09.902 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:09.908 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache": { 2024-05-14T17:17:09.919 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update": 0, 2024-05-14T17:17:09.926 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update_receipt": 0, 2024-05-14T17:17:09.932 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_try_discover": 0, 2024-05-14T17:17:09.941 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_send_discover": 69, 2024-05-14T17:17:09.966 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_handle_discover": 49, 2024-05-14T17:17:09.972 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays": 0, 2024-05-14T17:17:09.979 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_delayed": 0, 2024-05-14T17:17:09.986 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_enqueuing": 0, 2024-05-14T17:17:09.993 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_created": 166, 2024-05-14T17:17:10.000 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_enqueued": 7, 2024-05-14T17:17:10.006 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_reintegrated": 154, 2024-05-14T17:17:10.034 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_migrated": 16, 2024-05-14T17:17:10.062 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_processing": 0, 2024-05-14T17:17:10.069 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_enqueued": 0, 2024-05-14T17:17:10.075 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_prioritized": 0, 2024-05-14T17:17:10.082 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_started": 0, 2024-05-14T17:17:10.088 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_completed": 0, 2024-05-14T17:17:10.095 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_path": 9, 2024-05-14T17:17:10.101 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_inode": 2414, 2024-05-14T17:17:10.107 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_enqueue_scrub": 0, 2024-05-14T17:17:10.112 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_exportdir": 15, 2024-05-14T17:17:10.124 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_flush": 0, 2024-05-14T17:17:10.130 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragmentdir": 2, 2024-05-14T17:17:10.135 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragstats": 0, 2024-05-14T17:17:10.141 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_inodestats": 0 2024-05-14T17:17:10.147 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:10.152 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log": { 2024-05-14T17:17:10.158 INFO:teuthology.orchestra.run.smithi123.stdout: "evlrg": 0, 2024-05-14T17:17:10.172 INFO:teuthology.orchestra.run.smithi123.stdout: "evadd": 3289, 2024-05-14T17:17:10.209 INFO:teuthology.orchestra.run.smithi123.stdout: "evex": 0, 2024-05-14T17:17:10.216 INFO:teuthology.orchestra.run.smithi123.stdout: "evtrm": 0, 2024-05-14T17:17:10.223 INFO:teuthology.orchestra.run.smithi123.stdout: "ev": 3289, 2024-05-14T17:17:10.230 INFO:teuthology.orchestra.run.smithi123.stdout: "evexg": 0, 2024-05-14T17:17:10.237 INFO:teuthology.orchestra.run.smithi123.stdout: "evexd": 0, 2024-05-14T17:17:10.244 INFO:teuthology.orchestra.run.smithi123.stdout: "segadd": 6, 2024-05-14T17:17:10.251 INFO:teuthology.orchestra.run.smithi123.stdout: "segex": 0, 2024-05-14T17:17:10.256 INFO:teuthology.orchestra.run.smithi123.stdout: "segtrm": 0, 2024-05-14T17:17:10.264 INFO:teuthology.orchestra.run.smithi123.stdout: "seg": 6, 2024-05-14T17:17:10.274 INFO:teuthology.orchestra.run.smithi123.stdout: "segmjr": 2, 2024-05-14T17:17:10.283 INFO:teuthology.orchestra.run.smithi123.stdout: "segexg": 0, 2024-05-14T17:17:10.289 INFO:teuthology.orchestra.run.smithi123.stdout: "segexd": 0, 2024-05-14T17:17:10.295 INFO:teuthology.orchestra.run.smithi123.stdout: "expos": 4194304, 2024-05-14T17:17:10.304 INFO:teuthology.orchestra.run.smithi123.stdout: "wrpos": 11395771, 2024-05-14T17:17:10.309 INFO:teuthology.orchestra.run.smithi123.stdout: "rdpos": 0, 2024-05-14T17:17:10.315 INFO:teuthology.orchestra.run.smithi123.stdout: "jlat": { 2024-05-14T17:17:10.338 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1554, 2024-05-14T17:17:10.366 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 5.572238332, 2024-05-14T17:17:10.375 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.003585738 2024-05-14T17:17:10.386 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:10.395 INFO:teuthology.orchestra.run.smithi123.stdout: "replayed": 0 2024-05-14T17:17:10.403 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:10.413 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_mem": { 2024-05-14T17:17:10.421 INFO:teuthology.orchestra.run.smithi123.stdout: "ino": 733, 2024-05-14T17:17:10.431 INFO:teuthology.orchestra.run.smithi123.stdout: "ino+": 1571, 2024-05-14T17:17:10.439 INFO:teuthology.orchestra.run.smithi123.stdout: "ino-": 838, 2024-05-14T17:17:10.475 INFO:teuthology.orchestra.run.smithi123.stdout: "dir": 61, 2024-05-14T17:17:10.482 INFO:teuthology.orchestra.run.smithi123.stdout: "dir+": 98, 2024-05-14T17:17:10.490 INFO:teuthology.orchestra.run.smithi123.stdout: "dir-": 37, 2024-05-14T17:17:10.497 INFO:teuthology.orchestra.run.smithi123.stdout: "dn": 1021, 2024-05-14T17:17:10.505 INFO:teuthology.orchestra.run.smithi123.stdout: "dn+": 2028, 2024-05-14T17:17:10.512 INFO:teuthology.orchestra.run.smithi123.stdout: "dn-": 1007, 2024-05-14T17:17:10.519 INFO:teuthology.orchestra.run.smithi123.stdout: "cap": 191, 2024-05-14T17:17:10.535 INFO:teuthology.orchestra.run.smithi123.stdout: "cap+": 1046, 2024-05-14T17:17:10.563 INFO:teuthology.orchestra.run.smithi123.stdout: "cap-": 855, 2024-05-14T17:17:10.588 INFO:teuthology.orchestra.run.smithi123.stdout: "rss": 68580, 2024-05-14T17:17:10.599 INFO:teuthology.orchestra.run.smithi123.stdout: "heap": 223608 2024-05-14T17:17:10.608 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:10.616 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_server": { 2024-05-14T17:17:10.671 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_client_request": 2808, 2024-05-14T17:17:10.676 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_server_request": 428, 2024-05-14T17:17:10.685 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_request": 1767, 2024-05-14T17:17:10.693 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_session": 354, 2024-05-14T17:17:10.717 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_peer_request": 612, 2024-05-14T17:17:10.724 INFO:teuthology.orchestra.run.smithi123.stdout: "req_create_latency": { 2024-05-14T17:17:10.731 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 35, 2024-05-14T17:17:10.739 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.034820744, 2024-05-14T17:17:10.747 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000994878 2024-05-14T17:17:10.755 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:10.763 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getattr_latency": { 2024-05-14T17:17:10.781 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 324, 2024-05-14T17:17:10.821 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 58.068984188, 2024-05-14T17:17:10.827 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.179225259 2024-05-14T17:17:10.834 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:10.842 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getfilelock_latency": { 2024-05-14T17:17:10.848 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:10.918 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:10.925 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:10.932 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:10.939 INFO:teuthology.orchestra.run.smithi123.stdout: "req_link_latency": { 2024-05-14T17:17:10.951 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 154, 2024-05-14T17:17:10.958 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.351637367, 2024-05-14T17:17:10.964 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.002283359 2024-05-14T17:17:10.970 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:10.976 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookup_latency": { 2024-05-14T17:17:10.983 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 654, 2024-05-14T17:17:10.988 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.574236608, 2024-05-14T17:17:11.002 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000878037 2024-05-14T17:17:11.058 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:11.065 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookuphash_latency": { 2024-05-14T17:17:11.073 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:11.079 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:11.085 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:11.104 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:11.111 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupino_latency": { 2024-05-14T17:17:11.117 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:11.123 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:11.143 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:11.149 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:11.156 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupname_latency": { 2024-05-14T17:17:11.163 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:11.170 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:11.176 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:11.183 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:11.215 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupparent_latency": { 2024-05-14T17:17:11.279 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:11.286 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:11.293 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:11.300 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:11.307 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupsnap_latency": { 2024-05-14T17:17:11.339 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:11.347 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:11.356 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:11.365 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:11.419 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lssnap_latency": { 2024-05-14T17:17:11.429 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:08 smithi151 ceph-mon[36066]: pgmap v709: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-05-14T17:17:11.435 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:08 smithi151 ceph-mon[36066]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:17:11.443 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:09 smithi151 ceph-mon[36066]: osdmap e113: 12 total, 12 up, 12 in 2024-05-14T17:17:11.450 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:10 smithi151 ceph-mon[36066]: pgmap v711: 97 pgs: 8 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 85 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-05-14T17:17:11.458 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:08 smithi002 ceph-mon[31538]: pgmap v709: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-05-14T17:17:11.466 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:08 smithi002 ceph-mon[31538]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:17:11.482 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:09 smithi002 ceph-mon[31538]: osdmap e113: 12 total, 12 up, 12 in 2024-05-14T17:17:11.518 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:10 smithi002 ceph-mon[31538]: pgmap v711: 97 pgs: 8 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 85 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-05-14T17:17:11.527 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:11.535 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:11.543 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:11.551 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:11.562 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mkdir_latency": { 2024-05-14T17:17:11.570 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:11.578 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:11.586 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:11.602 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:11.610 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mknod_latency": { 2024-05-14T17:17:11.619 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:11.628 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:11.635 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:11.642 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:11.650 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mksnap_latency": { 2024-05-14T17:17:11.666 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:11.683 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:11.691 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:11.698 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:11.705 INFO:teuthology.orchestra.run.smithi123.stdout: "req_open_latency": { 2024-05-14T17:17:11.712 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 169, 2024-05-14T17:17:11.729 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.153993084, 2024-05-14T17:17:11.737 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000911201 2024-05-14T17:17:11.745 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:11.752 INFO:teuthology.orchestra.run.smithi123.stdout: "req_readdir_latency": { 2024-05-14T17:17:11.769 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 42, 2024-05-14T17:17:11.777 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.035872443, 2024-05-14T17:17:11.785 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000854105 2024-05-14T17:17:11.793 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:11.801 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rename_latency": { 2024-05-14T17:17:11.809 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 165, 2024-05-14T17:17:11.817 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 1.369665894, 2024-05-14T17:17:11.869 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.008301005 2024-05-14T17:17:11.901 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:11.909 INFO:teuthology.orchestra.run.smithi123.stdout: "req_renamesnap_latency": { 2024-05-14T17:17:11.916 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:11.922 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:11.927 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:11.938 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:11.944 INFO:teuthology.orchestra.run.smithi123.stdout: "req_snapdiff_latency": { 2024-05-14T17:17:11.950 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:11.956 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:11.969 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:11.977 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:11.983 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmdir_latency": { 2024-05-14T17:17:11.990 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:11.996 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:12.003 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:12.009 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:12.024 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmsnap_latency": { 2024-05-14T17:17:12.063 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:12.071 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:12.078 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:12.085 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:12.091 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmxattr_latency": { 2024-05-14T17:17:12.097 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:12.103 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:12.109 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:12.116 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:12.128 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setattr_latency": { 2024-05-14T17:17:12.133 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:12.140 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:12.147 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:12.154 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:12.160 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setdirlayout_latency": { 2024-05-14T17:17:12.166 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:12.211 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:12.235 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:12.241 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:12.246 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setfilelock_latency": { 2024-05-14T17:17:12.253 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:12.260 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:12.266 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:12.277 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:12.284 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setlayout_latency": { 2024-05-14T17:17:12.290 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:12.303 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:12.309 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:12.316 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:12.323 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setxattr_latency": { 2024-05-14T17:17:12.329 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:12.337 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:12.344 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:12.363 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:12.390 INFO:teuthology.orchestra.run.smithi123.stdout: "req_symlink_latency": { 2024-05-14T17:17:12.397 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:12.407 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:12.416 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:12.426 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:12.434 INFO:teuthology.orchestra.run.smithi123.stdout: "req_unlink_latency": { 2024-05-14T17:17:12.442 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 166, 2024-05-14T17:17:12.450 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.265309604, 2024-05-14T17:17:12.457 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.001598250 2024-05-14T17:17:12.466 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:12.473 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_revoke_eviction": 0, 2024-05-14T17:17:12.481 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_acquisition_throttle": 0, 2024-05-14T17:17:12.488 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getvxattr_latency": { 2024-05-14T17:17:12.497 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:12.505 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:12.513 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:12.529 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-14T17:17:12.556 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:12.564 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_sessions": { 2024-05-14T17:17:12.572 INFO:teuthology.orchestra.run.smithi123.stdout: "session_count": 2, 2024-05-14T17:17:12.580 INFO:teuthology.orchestra.run.smithi123.stdout: "session_add": 4, 2024-05-14T17:17:12.588 INFO:teuthology.orchestra.run.smithi123.stdout: "session_remove": 2, 2024-05-14T17:17:12.631 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_open": 2, 2024-05-14T17:17:12.638 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_stale": 0, 2024-05-14T17:17:12.645 INFO:teuthology.orchestra.run.smithi123.stdout: "total_load": 134, 2024-05-14T17:17:12.653 INFO:teuthology.orchestra.run.smithi123.stdout: "average_load": 67, 2024-05-14T17:17:12.666 INFO:teuthology.orchestra.run.smithi123.stdout: "avg_session_uptime": 1329, 2024-05-14T17:17:12.673 INFO:teuthology.orchestra.run.smithi123.stdout: "mdthresh_evicted": 0 2024-05-14T17:17:12.681 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:12.689 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-05-14T17:17:12.699 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-05-14T17:17:12.706 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-05-14T17:17:12.713 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T17:17:12.729 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-05-14T17:17:12.764 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T17:17:12.772 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-05-14T17:17:12.779 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T17:17:12.786 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T17:17:12.792 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T17:17:12.806 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T17:17:12.815 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T17:17:12.823 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-05-14T17:17:12.831 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T17:17:12.883 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T17:17:12.890 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-05-14T17:17:12.897 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-05-14T17:17:12.904 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-05-14T17:17:12.912 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-05-14T17:17:12.920 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T17:17:12.930 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T17:17:12.975 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T17:17:13.047 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T17:17:13.053 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T17:17:13.060 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-05-14T17:17:13.067 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-05-14T17:17:13.074 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-05-14T17:17:13.081 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T17:17:13.088 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T17:17:13.095 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-05-14T17:17:13.102 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-05-14T17:17:13.133 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-05-14T17:17:13.140 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-05-14T17:17:13.146 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T17:17:13.152 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-05-14T17:17:13.159 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T17:17:13.165 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-05-14T17:17:13.171 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 842206, 2024-05-14T17:17:13.182 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 901, 2024-05-14T17:17:13.207 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2024-05-14T17:17:13.214 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2024-05-14T17:17:13.220 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-05-14T17:17:13.225 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-05-14T17:17:13.231 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-05-14T17:17:13.239 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-05-14T17:17:13.245 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-05-14T17:17:13.249 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-05-14T17:17:13.254 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 8032, 2024-05-14T17:17:13.266 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 167, 2024-05-14T17:17:13.271 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T17:17:13.277 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-05-14T17:17:13.283 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-05-14T17:17:13.289 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-05-14T17:17:13.295 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 2857948, 2024-05-14T17:17:13.300 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 68178, 2024-05-14T17:17:13.315 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-05-14T17:17:13.337 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-05-14T17:17:13.343 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-05-14T17:17:13.349 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-05-14T17:17:13.356 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:13.362 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter": { 2024-05-14T17:17:13.373 INFO:teuthology.orchestra.run.smithi123.stdout: "op_active": 0, 2024-05-14T17:17:13.380 INFO:teuthology.orchestra.run.smithi123.stdout: "op_laggy": 0, 2024-05-14T17:17:13.387 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send": 1895, 2024-05-14T17:17:13.393 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send_bytes": 7440888, 2024-05-14T17:17:13.408 INFO:teuthology.orchestra.run.smithi123.stdout: "op_resend": 33, 2024-05-14T17:17:13.414 INFO:teuthology.orchestra.run.smithi123.stdout: "op_reply": 1895, 2024-05-14T17:17:13.420 INFO:teuthology.orchestra.run.smithi123.stdout: "op_latency": { 2024-05-14T17:17:13.425 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1895, 2024-05-14T17:17:13.432 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 8.188809636, 2024-05-14T17:17:13.437 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.004321271 2024-05-14T17:17:13.443 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:13.457 INFO:teuthology.orchestra.run.smithi123.stdout: "op_inflight": 0, 2024-05-14T17:17:13.480 INFO:teuthology.orchestra.run.smithi123.stdout: "oplen_avg": { 2024-05-14T17:17:13.487 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1895, 2024-05-14T17:17:13.493 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 2160 2024-05-14T17:17:13.498 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:13.503 INFO:teuthology.orchestra.run.smithi123.stdout: "op": 1895, 2024-05-14T17:17:13.515 INFO:teuthology.orchestra.run.smithi123.stdout: "op_r": 10, 2024-05-14T17:17:13.521 INFO:teuthology.orchestra.run.smithi123.stdout: "op_w": 1885, 2024-05-14T17:17:13.528 INFO:teuthology.orchestra.run.smithi123.stdout: "op_rmw": 0, 2024-05-14T17:17:13.537 INFO:teuthology.orchestra.run.smithi123.stdout: "op_pg": 0, 2024-05-14T17:17:13.559 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_stat": 0, 2024-05-14T17:17:13.568 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_create": 10, 2024-05-14T17:17:13.576 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_read": 2, 2024-05-14T17:17:13.583 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_write": 1556, 2024-05-14T17:17:13.593 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writefull": 73, 2024-05-14T17:17:13.601 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writesame": 0, 2024-05-14T17:17:13.609 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_append": 0, 2024-05-14T17:17:13.624 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_zero": 0, 2024-05-14T17:17:13.651 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_truncate": 1, 2024-05-14T17:17:13.660 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_delete": 21, 2024-05-14T17:17:13.668 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_mapext": 0, 2024-05-14T17:17:13.675 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_sparse_read": 0, 2024-05-14T17:17:13.683 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_clonerange": 0, 2024-05-14T17:17:13.691 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_getxattr": 1, 2024-05-14T17:17:13.699 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_setxattr": 20, 2024-05-14T17:17:13.708 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_cmpxattr": 0, 2024-05-14T17:17:13.717 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_rmxattr": 0, 2024-05-14T17:17:13.734 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_resetxattrs": 0, 2024-05-14T17:17:13.743 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_call": 0, 2024-05-14T17:17:13.749 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_watch": 0, 2024-05-14T17:17:13.758 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_notify": 0, 2024-05-14T17:17:13.766 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_src_cmpxattr": 0, 2024-05-14T17:17:13.774 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls": 0, 2024-05-14T17:17:13.782 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls_filter": 0, 2024-05-14T17:17:13.794 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_other": 0, 2024-05-14T17:17:13.827 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_active": 0, 2024-05-14T17:17:13.835 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_send": 0, 2024-05-14T17:17:13.843 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_resend": 0, 2024-05-14T17:17:13.851 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_ping": 0, 2024-05-14T17:17:13.859 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_active": 0, 2024-05-14T17:17:13.868 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_send": 0, 2024-05-14T17:17:13.876 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_resend": 0, 2024-05-14T17:17:13.884 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_active": 0, 2024-05-14T17:17:13.894 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_send": 0, 2024-05-14T17:17:13.909 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_resend": 0, 2024-05-14T17:17:13.915 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_active": 0, 2024-05-14T17:17:13.923 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_send": 0, 2024-05-14T17:17:13.931 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_resend": 0, 2024-05-14T17:17:13.939 INFO:teuthology.orchestra.run.smithi123.stdout: "command_active": 0, 2024-05-14T17:17:13.947 INFO:teuthology.orchestra.run.smithi123.stdout: "command_send": 0, 2024-05-14T17:17:13.955 INFO:teuthology.orchestra.run.smithi123.stdout: "command_resend": 0, 2024-05-14T17:17:13.970 INFO:teuthology.orchestra.run.smithi123.stdout: "map_epoch": 112, 2024-05-14T17:17:13.998 INFO:teuthology.orchestra.run.smithi123.stdout: "map_full": 0, 2024-05-14T17:17:14.006 INFO:teuthology.orchestra.run.smithi123.stdout: "map_inc": 34, 2024-05-14T17:17:14.014 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_sessions": 11, 2024-05-14T17:17:14.022 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_open": 11, 2024-05-14T17:17:14.029 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_close": 0, 2024-05-14T17:17:14.040 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_laggy": 0, 2024-05-14T17:17:14.048 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_wr": 385, 2024-05-14T17:17:14.057 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_rd": 16, 2024-05-14T17:17:14.065 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_del": 75 2024-05-14T17:17:14.076 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:14.084 INFO:teuthology.orchestra.run.smithi123.stdout: "oft": { 2024-05-14T17:17:14.093 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_objs": 1, 2024-05-14T17:17:14.099 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_kv_pairs": 0, 2024-05-14T17:17:14.108 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_updates": 0, 2024-05-14T17:17:14.116 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_removes": 0 2024-05-14T17:17:14.122 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:14.137 INFO:teuthology.orchestra.run.smithi123.stdout: "osd-slow-ops": { 2024-05-14T17:17:14.162 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_ops_count": 2424 2024-05-14T17:17:14.167 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:14.173 INFO:teuthology.orchestra.run.smithi123.stdout: "purge_queue": { 2024-05-14T17:17:14.178 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops": 0, 2024-05-14T17:17:14.183 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops_high_water": 6, 2024-05-14T17:17:14.190 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing": 0, 2024-05-14T17:17:14.195 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_high_water": 6, 2024-05-14T17:17:14.200 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed_ops": 7, 2024-05-14T17:17:14.205 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed": 7, 2024-05-14T17:17:14.218 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_item_in_journal": 0 2024-05-14T17:17:14.224 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:14.230 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T17:17:14.236 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-05-14T17:17:14.243 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-05-14T17:17:14.249 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-05-14T17:17:14.256 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 12398, 2024-05-14T17:17:14.270 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 5989170, 2024-05-14T17:17:14.293 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-05-14T17:17:14.301 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 12398, 2024-05-14T17:17:14.309 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-05-14T17:17:14.317 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-05-14T17:17:14.324 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 12398, 2024-05-14T17:17:14.332 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 5989170, 2024-05-14T17:17:14.339 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-05-14T17:17:14.344 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:14.349 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:14.360 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:14.367 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-14T17:17:14.374 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:14.380 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_bytes": { 2024-05-14T17:17:14.386 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-05-14T17:17:14.393 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-05-14T17:17:14.399 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-05-14T17:17:14.446 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-05-14T17:17:14.466 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-05-14T17:17:14.473 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-05-14T17:17:14.479 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-05-14T17:17:14.486 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 1895, 2024-05-14T17:17:14.492 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 7441587, 2024-05-14T17:17:14.499 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 1895, 2024-05-14T17:17:14.506 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 7441587, 2024-05-14T17:17:14.513 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-05-14T17:17:14.519 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:14.539 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:14.545 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:14.554 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-14T17:17:14.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:14.571 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_ops": { 2024-05-14T17:17:14.579 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-05-14T17:17:14.588 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 1024, 2024-05-14T17:17:14.604 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-05-14T17:17:14.628 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-05-14T17:17:14.637 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-05-14T17:17:14.646 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-05-14T17:17:14.655 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-05-14T17:17:14.663 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 1895, 2024-05-14T17:17:14.675 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 1895, 2024-05-14T17:17:14.680 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 1895, 2024-05-14T17:17:14.689 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 1895, 2024-05-14T17:17:14.698 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-05-14T17:17:14.711 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:14.719 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:14.725 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:14.733 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-14T17:17:14.741 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:14.749 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle": { 2024-05-14T17:17:14.757 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-05-14T17:17:14.771 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-05-14T17:17:14.795 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-05-14T17:17:14.811 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 7, 2024-05-14T17:17:14.819 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 827, 2024-05-14T17:17:14.827 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-05-14T17:17:14.836 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 7, 2024-05-14T17:17:14.845 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-05-14T17:17:14.853 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-05-14T17:17:14.861 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 2, 2024-05-14T17:17:14.868 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 827, 2024-05-14T17:17:14.878 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-05-14T17:17:14.886 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:14.892 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:14.900 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:14.908 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-14T17:17:14.917 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:14.925 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle-0x5599c5d6c500": { 2024-05-14T17:17:14.967 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-05-14T17:17:15.005 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-05-14T17:17:15.013 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-05-14T17:17:15.021 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 3289, 2024-05-14T17:17:15.029 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 7201467, 2024-05-14T17:17:15.037 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-05-14T17:17:15.045 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 3289, 2024-05-14T17:17:15.054 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-05-14T17:17:15.062 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-05-14T17:17:15.071 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 1554, 2024-05-14T17:17:15.086 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 7201467, 2024-05-14T17:17:15.095 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-05-14T17:17:15.102 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:15.113 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:15.121 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:15.126 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-14T17:17:15.133 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-14T17:17:15.163 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-05-14T17:17:15.204 INFO:tasks.check_counter:Daemon mds.h mds.exported=15 2024-05-14T17:17:15.211 INFO:tasks.check_counter:Daemon mds.h mds.imported=26 2024-05-14T17:17:15.217 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=7 2024-05-14T17:17:15.224 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-05-14T17:17:15.230 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-05-14T17:17:15.242 DEBUG:tasks.check_counter:Getting stats from k 2024-05-14T17:17:15.248 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-14T17:17:15.255 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:08 smithi123 ceph-mon[35971]: pgmap v709: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-05-14T17:17:15.263 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:08 smithi123 ceph-mon[35971]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-05-14T17:17:15.278 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:09 smithi123 ceph-mon[35971]: osdmap e113: 12 total, 12 up, 12 in 2024-05-14T17:17:15.284 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:10 smithi123 ceph-mon[35971]: pgmap v711: 97 pgs: 8 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 85 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-05-14T17:17:15.291 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:12 smithi123 ceph-mon[35971]: pgmap v712: 97 pgs: 8 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 85 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-05-14T17:17:15.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:14 smithi123 ceph-mon[35971]: pgmap v713: 97 pgs: 8 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 85 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-05-14T17:17:15.305 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:12 smithi151 ceph-mon[36066]: pgmap v712: 97 pgs: 8 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 85 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-05-14T17:17:15.311 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:14 smithi151 ceph-mon[36066]: pgmap v713: 97 pgs: 8 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 85 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-05-14T17:17:15.318 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:12 smithi002 ceph-mon[31538]: pgmap v712: 97 pgs: 8 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 85 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-05-14T17:17:15.334 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:14 smithi002 ceph-mon[31538]: pgmap v713: 97 pgs: 8 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 85 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-05-14T17:17:15.561 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.b/config 2024-05-14T17:17:16.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:16 smithi002 ceph-mon[31538]: pgmap v714: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:17:17.029 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:16 smithi123 ceph-mon[35971]: pgmap v714: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:17:17.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:16 smithi151 ceph-mon[36066]: pgmap v714: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:17:17.288 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-05-14T17:17:17.294 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T17:17:17.299 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 433, 2024-05-14T17:17:17.305 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 422, 2024-05-14T17:17:17.311 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 1771072, 2024-05-14T17:17:17.318 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 261040, 2024-05-14T17:17:17.326 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 4, 2024-05-14T17:17:17.435 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 0, 2024-05-14T17:17:17.442 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.073527818, 2024-05-14T17:17:17.449 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.023687273, 2024-05-14T17:17:17.456 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.045580648, 2024-05-14T17:17:17.465 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.010534731, 2024-05-14T17:17:17.482 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:17:17.490 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 421, 2024-05-14T17:17:17.593 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.019298953, 2024-05-14T17:17:17.645 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000045840 2024-05-14T17:17:17.653 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:17.662 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:17:17.675 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:17.683 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:17.692 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:17.701 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:17.711 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 1771072, 2024-05-14T17:17:17.719 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 261040 2024-05-14T17:17:17.742 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:17.750 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T17:17:17.758 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 7, 2024-05-14T17:17:17.767 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 326, 2024-05-14T17:17:17.775 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 3424, 2024-05-14T17:17:17.792 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 293680, 2024-05-14T17:17:17.846 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 6, 2024-05-14T17:17:17.853 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 3, 2024-05-14T17:17:17.870 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.019316143, 2024-05-14T17:17:17.878 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.013361850, 2024-05-14T17:17:17.885 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.002258508, 2024-05-14T17:17:17.892 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T17:17:17.900 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:17:17.908 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 324, 2024-05-14T17:17:17.915 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.035832975, 2024-05-14T17:17:17.923 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000110595 2024-05-14T17:17:17.933 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:17.950 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:17:17.959 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:17.966 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:17.973 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:17.981 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:17.992 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 3424, 2024-05-14T17:17:18.029 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 293680 2024-05-14T17:17:18.036 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:18.062 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T17:17:18.069 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 0, 2024-05-14T17:17:18.076 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 2, 2024-05-14T17:17:18.084 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 0, 2024-05-14T17:17:18.091 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 0, 2024-05-14T17:17:18.098 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 2, 2024-05-14T17:17:18.105 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 1, 2024-05-14T17:17:18.112 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.000235976, 2024-05-14T17:17:18.118 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.000000000, 2024-05-14T17:17:18.134 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.000145770, 2024-05-14T17:17:18.141 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T17:17:18.147 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:17:18.154 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:18.161 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:18.176 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:18.210 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:18.217 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:17:18.237 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:18.243 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:18.250 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:18.259 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:18.266 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-14T17:17:18.273 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 0 2024-05-14T17:17:18.280 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:18.286 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-05-14T17:17:18.294 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 0, 2024-05-14T17:17:18.309 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-05-14T17:17:18.315 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:18.321 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-05-14T17:17:18.327 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-05-14T17:17:18.337 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-05-14T17:17:18.351 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T17:17:18.370 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-05-14T17:17:18.376 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T17:17:18.417 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-05-14T17:17:18.424 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T17:17:18.431 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T17:17:18.438 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T17:17:18.445 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T17:17:18.451 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T17:17:18.456 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-05-14T17:17:18.462 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T17:17:18.468 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T17:17:18.493 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-05-14T17:17:18.498 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-05-14T17:17:18.506 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-05-14T17:17:18.511 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-05-14T17:17:18.518 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T17:17:18.526 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T17:17:18.550 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T17:17:18.556 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T17:17:18.571 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T17:17:18.582 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-05-14T17:17:18.589 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-05-14T17:17:18.598 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-05-14T17:17:18.606 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T17:17:18.615 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T17:17:18.624 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-05-14T17:17:18.631 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-05-14T17:17:18.640 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-05-14T17:17:18.685 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-05-14T17:17:18.691 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T17:17:18.698 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-05-14T17:17:18.708 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T17:17:18.717 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-05-14T17:17:18.735 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 148714, 2024-05-14T17:17:18.762 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 30, 2024-05-14T17:17:18.771 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2024-05-14T17:17:18.801 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2024-05-14T17:17:18.809 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-05-14T17:17:18.815 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-05-14T17:17:18.835 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-05-14T17:17:18.844 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-05-14T17:17:18.853 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-05-14T17:17:18.859 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-05-14T17:17:18.869 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 0, 2024-05-14T17:17:18.878 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 0, 2024-05-14T17:17:18.903 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T17:17:18.912 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-05-14T17:17:18.922 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-05-14T17:17:18.930 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-05-14T17:17:18.937 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 928, 2024-05-14T17:17:18.952 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 2, 2024-05-14T17:17:19.015 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-05-14T17:17:19.023 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-05-14T17:17:19.047 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-05-14T17:17:19.053 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-05-14T17:17:19.060 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-05-14T17:17:19.069 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T17:17:19.076 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-05-14T17:17:19.084 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-05-14T17:17:19.091 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-05-14T17:17:19.097 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 436, 2024-05-14T17:17:19.105 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 1709884, 2024-05-14T17:17:19.119 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-05-14T17:17:19.125 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 436, 2024-05-14T17:17:19.132 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-05-14T17:17:19.139 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-05-14T17:17:19.146 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 436, 2024-05-14T17:17:19.160 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 1709884, 2024-05-14T17:17:19.198 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-05-14T17:17:19.204 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-05-14T17:17:19.228 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-05-14T17:17:19.234 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-05-14T17:17:19.242 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-14T17:17:19.252 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-05-14T17:17:19.258 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-05-14T17:17:19.266 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:17 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:17:19.272 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:17 smithi123 ceph-mon[35971]: osdmap e114: 12 total, 12 up, 12 in 2024-05-14T17:17:19.279 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:17 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:17:19.286 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:17 smithi151 ceph-mon[36066]: osdmap e114: 12 total, 12 up, 12 in 2024-05-14T17:17:19.305 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:18 smithi151 ceph-mon[36066]: pgmap v715: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:17:19.311 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:18 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:17:19.321 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:18 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:17:19.328 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:18 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:17:19.336 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:17 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:17:19.352 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:17 smithi002 ceph-mon[31538]: osdmap e114: 12 total, 12 up, 12 in 2024-05-14T17:17:19.379 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:18 smithi002 ceph-mon[31538]: pgmap v715: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:17:19.387 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:18 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:17:19.405 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:18 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:17:19.412 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:18 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:17:19.419 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-05-14T17:17:19.427 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-05-14T17:17:19.433 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-05-14T17:17:19.440 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-05-14T17:17:19.448 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-05-14T17:17:19.455 DEBUG:tasks.check_counter:Getting stats from c 2024-05-14T17:17:19.464 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-14T17:17:19.489 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:18 smithi123 ceph-mon[35971]: pgmap v715: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-14T17:17:19.495 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:18 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:17:19.503 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:18 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:17:19.509 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:18 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:17:19.767 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T17:17:19.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:19 smithi151 ceph-mon[36066]: pgmap v717: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 226 B/s wr, 5 op/s 2024-05-14T17:17:19.873 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:19 smithi151 ceph-mon[36066]: pgmap v718: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 290 B/s wr, 6 op/s 2024-05-14T17:17:19.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:19 smithi002 ceph-mon[31538]: pgmap v717: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 226 B/s wr, 5 op/s 2024-05-14T17:17:20.003 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:19 smithi002 ceph-mon[31538]: pgmap v718: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 290 B/s wr, 6 op/s 2024-05-14T17:17:20.159 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:19 smithi123 ceph-mon[35971]: pgmap v717: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 226 B/s wr, 5 op/s 2024-05-14T17:17:20.166 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:19 smithi123 ceph-mon[35971]: pgmap v718: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 290 B/s wr, 6 op/s 2024-05-14T17:17:21.610 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-14T17:17:21.618 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T17:17:21.625 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 4786, 2024-05-14T17:17:21.631 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 4788, 2024-05-14T17:17:21.638 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 8542367, 2024-05-14T17:17:21.644 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 1521955, 2024-05-14T17:17:21.680 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 6, 2024-05-14T17:17:21.687 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 2, 2024-05-14T17:17:21.694 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.373365585, 2024-05-14T17:17:21.700 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.120553992, 2024-05-14T17:17:21.707 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.096126207, 2024-05-14T17:17:21.728 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.128749060, 2024-05-14T17:17:21.734 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:17:21.834 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 4786, 2024-05-14T17:17:21.905 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.171218191, 2024-05-14T17:17:21.912 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000035774 2024-05-14T17:17:21.919 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:21.946 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:17:21.953 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:21.959 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:21.966 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:21.974 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:21.981 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-14T17:17:21.997 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 0 2024-05-14T17:17:22.003 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:22.011 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T17:17:22.017 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 5, 2024-05-14T17:17:22.023 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 323, 2024-05-14T17:17:22.044 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 1036, 2024-05-14T17:17:22.110 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 814122, 2024-05-14T17:17:22.116 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 7, 2024-05-14T17:17:22.143 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 4, 2024-05-14T17:17:22.149 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.018279299, 2024-05-14T17:17:22.155 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.011931091, 2024-05-14T17:17:22.169 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.003162906, 2024-05-14T17:17:22.177 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000071563, 2024-05-14T17:17:22.183 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:17:22.191 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 320, 2024-05-14T17:17:22.199 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.029858399, 2024-05-14T17:17:22.206 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000093307 2024-05-14T17:17:22.240 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:22.248 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:17:22.254 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:22.262 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:22.270 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:22.278 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:22.286 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-14T17:17:22.306 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 813424 2024-05-14T17:17:22.335 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:22.343 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T17:17:22.351 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 515, 2024-05-14T17:17:22.361 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 463, 2024-05-14T17:17:22.370 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 1979100, 2024-05-14T17:17:22.378 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 267070, 2024-05-14T17:17:22.386 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 4, 2024-05-14T17:17:22.394 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 2, 2024-05-14T17:17:22.402 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.078410514, 2024-05-14T17:17:22.414 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.026814362, 2024-05-14T17:17:22.422 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.050024409, 2024-05-14T17:17:22.431 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.010307473, 2024-05-14T17:17:22.439 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:17:22.447 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 463, 2024-05-14T17:17:22.462 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.027590136, 2024-05-14T17:17:22.491 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000059589 2024-05-14T17:17:22.499 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:22.524 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:17:22.533 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:22.542 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:22.553 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:22.561 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:22.569 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 1978832, 2024-05-14T17:17:22.577 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 266752 2024-05-14T17:17:22.586 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:22.594 INFO:teuthology.orchestra.run.smithi151.stdout: "cct": { 2024-05-14T17:17:22.606 INFO:teuthology.orchestra.run.smithi151.stdout: "total_workers": 1, 2024-05-14T17:17:22.612 INFO:teuthology.orchestra.run.smithi151.stdout: "unhealthy_workers": 0 2024-05-14T17:17:22.620 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:22.628 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-MDSRank": { 2024-05-14T17:17:22.636 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-05-14T17:17:22.653 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-05-14T17:17:22.680 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 8066, 2024-05-14T17:17:22.688 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.204028807, 2024-05-14T17:17:22.712 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000025294 2024-05-14T17:17:22.722 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-14T17:17:22.731 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:22.739 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-PurgeQueue": { 2024-05-14T17:17:22.749 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-05-14T17:17:22.756 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-05-14T17:17:22.764 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:22.769 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:22.775 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:22.789 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-14T17:17:22.796 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:22.803 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": { 2024-05-14T17:17:22.809 INFO:teuthology.orchestra.run.smithi151.stdout: "request": 0, 2024-05-14T17:17:22.815 INFO:teuthology.orchestra.run.smithi151.stdout: "reply": 0, 2024-05-14T17:17:22.829 INFO:teuthology.orchestra.run.smithi151.stdout: "reply_latency": { 2024-05-14T17:17:22.888 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:22.895 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:22.919 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:22.925 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:22.932 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_reply": 0, 2024-05-14T17:17:22.939 INFO:teuthology.orchestra.run.smithi151.stdout: "forward": 0, 2024-05-14T17:17:22.946 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_complete": 0, 2024-05-14T17:17:22.952 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_keys": 0, 2024-05-14T17:17:22.958 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_commit": 0, 2024-05-14T17:17:22.965 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_split": 0, 2024-05-14T17:17:22.971 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_merge": 0, 2024-05-14T17:17:22.990 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes": 1050, 2024-05-14T17:17:23.011 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_top": 305, 2024-05-14T17:17:23.018 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_bottom": 745, 2024-05-14T17:17:23.025 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pin_tail": 0, 2024-05-14T17:17:23.034 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pinned": 613, 2024-05-14T17:17:23.046 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_expired": 0, 2024-05-14T17:17:23.069 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_with_caps": 0, 2024-05-14T17:17:23.077 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": 0, 2024-05-14T17:17:23.099 INFO:teuthology.orchestra.run.smithi151.stdout: "subtrees": 27, 2024-05-14T17:17:23.105 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse": 0, 2024-05-14T17:17:23.113 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_hit": 0, 2024-05-14T17:17:23.120 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_forward": 0, 2024-05-14T17:17:23.128 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_discover": 0, 2024-05-14T17:17:23.136 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_dir_fetch": 0, 2024-05-14T17:17:23.143 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_remote_ino": 0, 2024-05-14T17:17:23.150 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_lock": 0, 2024-05-14T17:17:23.156 INFO:teuthology.orchestra.run.smithi151.stdout: "load_cent": 0, 2024-05-14T17:17:23.180 INFO:teuthology.orchestra.run.smithi151.stdout: "q": 0, 2024-05-14T17:17:23.186 INFO:teuthology.orchestra.run.smithi151.stdout: "exported": 0, 2024-05-14T17:17:23.193 INFO:teuthology.orchestra.run.smithi151.stdout: "exported_inodes": 0, 2024-05-14T17:17:23.199 INFO:teuthology.orchestra.run.smithi151.stdout: "imported": 0, 2024-05-14T17:17:23.209 INFO:teuthology.orchestra.run.smithi151.stdout: "imported_inodes": 0, 2024-05-14T17:17:23.221 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_dir_fetch": 0, 2024-05-14T17:17:23.245 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_backtrace_fetch": 0, 2024-05-14T17:17:23.252 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_peer_discover": 0, 2024-05-14T17:17:23.274 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rfiles": 0, 2024-05-14T17:17:23.282 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rbytes": 0, 2024-05-14T17:17:23.290 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rsnaps": 0, 2024-05-14T17:17:23.297 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_fetch": 0, 2024-05-14T17:17:23.302 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_set_tag": 0, 2024-05-14T17:17:23.309 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_repaired": 0, 2024-05-14T17:17:23.316 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_inotable_repaired": 0, 2024-05-14T17:17:23.323 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_inodes": 0, 2024-05-14T17:17:23.331 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_base_inodes": 0, 2024-05-14T17:17:23.347 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dirfrag_rstats": 0, 2024-05-14T17:17:23.353 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_file_inodes": 0, 2024-05-14T17:17:23.359 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_inode_file_caps": 0, 2024-05-14T17:17:23.365 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_revoke": 0, 2024-05-14T17:17:23.371 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_grant": 0, 2024-05-14T17:17:23.380 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_trunc": 0, 2024-05-14T17:17:23.405 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-14T17:17:23.415 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-14T17:17:23.442 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps": 0, 2024-05-14T17:17:23.448 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps_dirty": 0, 2024-05-14T17:17:23.455 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_cap_release": 0, 2024-05-14T17:17:23.462 INFO:teuthology.orchestra.run.smithi151.stdout: "process_request_cap_release": 0 2024-05-14T17:17:23.470 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:23.477 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_cache": { 2024-05-14T17:17:23.483 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update": 0, 2024-05-14T17:17:23.491 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update_receipt": 0, 2024-05-14T17:17:23.500 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_try_discover": 0, 2024-05-14T17:17:23.523 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_send_discover": 0, 2024-05-14T17:17:23.534 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_handle_discover": 0, 2024-05-14T17:17:23.544 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays": 0, 2024-05-14T17:17:23.551 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_delayed": 0, 2024-05-14T17:17:23.558 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_enqueuing": 0, 2024-05-14T17:17:23.572 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_created": 0, 2024-05-14T17:17:23.602 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_enqueued": 0, 2024-05-14T17:17:23.611 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_reintegrated": 0, 2024-05-14T17:17:23.642 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_migrated": 0, 2024-05-14T17:17:23.648 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_processing": 0, 2024-05-14T17:17:23.656 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_enqueued": 0, 2024-05-14T17:17:23.666 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_prioritized": 0, 2024-05-14T17:17:23.674 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_started": 0, 2024-05-14T17:17:23.681 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_completed": 0, 2024-05-14T17:17:23.688 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_path": 0, 2024-05-14T17:17:23.697 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_inode": 0, 2024-05-14T17:17:23.704 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_enqueue_scrub": 0, 2024-05-14T17:17:23.731 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_exportdir": 0, 2024-05-14T17:17:23.743 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_flush": 0, 2024-05-14T17:17:23.753 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragmentdir": 0, 2024-05-14T17:17:23.760 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragstats": 0, 2024-05-14T17:17:23.769 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_inodestats": 0 2024-05-14T17:17:23.789 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:23.816 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_log": { 2024-05-14T17:17:23.829 INFO:teuthology.orchestra.run.smithi151.stdout: "evlrg": 0, 2024-05-14T17:17:23.850 INFO:teuthology.orchestra.run.smithi151.stdout: "evadd": 0, 2024-05-14T17:17:23.859 INFO:teuthology.orchestra.run.smithi151.stdout: "evex": 0, 2024-05-14T17:17:23.871 INFO:teuthology.orchestra.run.smithi151.stdout: "evtrm": 0, 2024-05-14T17:17:23.879 INFO:teuthology.orchestra.run.smithi151.stdout: "ev": 3289, 2024-05-14T17:17:23.888 INFO:teuthology.orchestra.run.smithi151.stdout: "evexg": 0, 2024-05-14T17:17:23.896 INFO:teuthology.orchestra.run.smithi151.stdout: "evexd": 0, 2024-05-14T17:17:23.904 INFO:teuthology.orchestra.run.smithi151.stdout: "segadd": 0, 2024-05-14T17:17:23.911 INFO:teuthology.orchestra.run.smithi151.stdout: "segex": 0, 2024-05-14T17:17:23.917 INFO:teuthology.orchestra.run.smithi151.stdout: "segtrm": 0, 2024-05-14T17:17:23.940 INFO:teuthology.orchestra.run.smithi151.stdout: "seg": 6, 2024-05-14T17:17:23.949 INFO:teuthology.orchestra.run.smithi151.stdout: "segmjr": 2, 2024-05-14T17:17:23.956 INFO:teuthology.orchestra.run.smithi151.stdout: "segexg": 0, 2024-05-14T17:17:23.964 INFO:teuthology.orchestra.run.smithi151.stdout: "segexd": 0, 2024-05-14T17:17:23.972 INFO:teuthology.orchestra.run.smithi151.stdout: "expos": 4194304, 2024-05-14T17:17:23.981 INFO:teuthology.orchestra.run.smithi151.stdout: "wrpos": 11395771, 2024-05-14T17:17:24.045 INFO:teuthology.orchestra.run.smithi151.stdout: "rdpos": 11395710, 2024-05-14T17:17:24.052 INFO:teuthology.orchestra.run.smithi151.stdout: "jlat": { 2024-05-14T17:17:24.074 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:24.083 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:24.095 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:24.105 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:24.112 INFO:teuthology.orchestra.run.smithi151.stdout: "replayed": 3289 2024-05-14T17:17:24.123 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:24.137 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_mem": { 2024-05-14T17:17:24.148 INFO:teuthology.orchestra.run.smithi151.stdout: "ino": 706, 2024-05-14T17:17:24.157 INFO:teuthology.orchestra.run.smithi151.stdout: "ino+": 1541, 2024-05-14T17:17:24.174 INFO:teuthology.orchestra.run.smithi151.stdout: "ino-": 835, 2024-05-14T17:17:24.188 INFO:teuthology.orchestra.run.smithi151.stdout: "dir": 38, 2024-05-14T17:17:24.197 INFO:teuthology.orchestra.run.smithi151.stdout: "dir+": 81, 2024-05-14T17:17:24.209 INFO:teuthology.orchestra.run.smithi151.stdout: "dir-": 43, 2024-05-14T17:17:24.216 INFO:teuthology.orchestra.run.smithi151.stdout: "dn": 1050, 2024-05-14T17:17:24.231 INFO:teuthology.orchestra.run.smithi151.stdout: "dn+": 1941, 2024-05-14T17:17:24.259 INFO:teuthology.orchestra.run.smithi151.stdout: "dn-": 891, 2024-05-14T17:17:24.268 INFO:teuthology.orchestra.run.smithi151.stdout: "cap": 0, 2024-05-14T17:17:24.285 INFO:teuthology.orchestra.run.smithi151.stdout: "cap+": 0, 2024-05-14T17:17:24.291 INFO:teuthology.orchestra.run.smithi151.stdout: "cap-": 0, 2024-05-14T17:17:24.301 INFO:teuthology.orchestra.run.smithi151.stdout: "rss": 56320, 2024-05-14T17:17:24.314 INFO:teuthology.orchestra.run.smithi151.stdout: "heap": 215416 2024-05-14T17:17:24.321 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:24.333 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_server": { 2024-05-14T17:17:24.342 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_client_request": 0, 2024-05-14T17:17:24.351 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_server_request": 0, 2024-05-14T17:17:24.360 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_request": 0, 2024-05-14T17:17:24.374 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_session": 0, 2024-05-14T17:17:24.385 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_peer_request": 0, 2024-05-14T17:17:24.391 INFO:teuthology.orchestra.run.smithi151.stdout: "req_create_latency": { 2024-05-14T17:17:24.397 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:24.407 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:24.423 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:24.449 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:24.458 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getattr_latency": { 2024-05-14T17:17:24.488 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:24.494 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:24.502 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:24.514 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:24.522 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getfilelock_latency": { 2024-05-14T17:17:24.528 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:24.537 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:24.543 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:24.551 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:24.565 INFO:teuthology.orchestra.run.smithi151.stdout: "req_link_latency": { 2024-05-14T17:17:24.577 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:24.584 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:24.592 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:24.605 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:24.614 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookup_latency": { 2024-05-14T17:17:24.632 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:24.639 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:24.652 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:24.659 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:24.665 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookuphash_latency": { 2024-05-14T17:17:24.672 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:24.680 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:24.687 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:24.694 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:24.700 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupino_latency": { 2024-05-14T17:17:24.709 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:24.732 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:24.738 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:24.746 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:24.753 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupname_latency": { 2024-05-14T17:17:24.761 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:24.773 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:24.813 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:24.821 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:24.835 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupparent_latency": { 2024-05-14T17:17:24.844 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:24.852 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:24.865 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:24.878 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:24.889 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupsnap_latency": { 2024-05-14T17:17:24.896 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:24.926 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:24.936 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:24.949 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:24.958 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lssnap_latency": { 2024-05-14T17:17:24.968 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:24.978 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:24.988 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:24.998 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:25.081 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mkdir_latency": { 2024-05-14T17:17:25.093 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:21 smithi123 ceph-mon[35971]: pgmap v719: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 145 B/s wr, 6 op/s 2024-05-14T17:17:25.102 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:23 smithi123 ceph-mon[35971]: pgmap v720: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 145 B/s wr, 4 op/s 2024-05-14T17:17:25.111 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:21 smithi002 ceph-mon[31538]: pgmap v719: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 145 B/s wr, 6 op/s 2024-05-14T17:17:25.126 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:23 smithi002 ceph-mon[31538]: pgmap v720: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 145 B/s wr, 4 op/s 2024-05-14T17:17:25.138 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:25.145 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:25.155 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:25.167 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:25.176 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mknod_latency": { 2024-05-14T17:17:25.186 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:25.200 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:25.207 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:25.214 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:25.221 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mksnap_latency": { 2024-05-14T17:17:25.232 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:25.248 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:25.293 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:25.301 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:25.311 INFO:teuthology.orchestra.run.smithi151.stdout: "req_open_latency": { 2024-05-14T17:17:25.318 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:25.325 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:25.337 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:25.346 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:25.355 INFO:teuthology.orchestra.run.smithi151.stdout: "req_readdir_latency": { 2024-05-14T17:17:25.369 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:25.377 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:25.387 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:25.409 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:25.416 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rename_latency": { 2024-05-14T17:17:25.422 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:25.428 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:25.435 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:25.457 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:25.482 INFO:teuthology.orchestra.run.smithi151.stdout: "req_renamesnap_latency": { 2024-05-14T17:17:25.489 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:25.513 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:25.522 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:25.531 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:25.554 INFO:teuthology.orchestra.run.smithi151.stdout: "req_snapdiff_latency": { 2024-05-14T17:17:25.564 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:25.573 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:25.583 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:25.593 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:25.605 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmdir_latency": { 2024-05-14T17:17:25.622 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:25.634 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:25.645 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:25.658 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:25.669 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmsnap_latency": { 2024-05-14T17:17:25.682 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:25.712 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:25.727 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:25.753 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:25.765 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmxattr_latency": { 2024-05-14T17:17:25.777 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:25.787 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:25.796 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:25.805 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:25.815 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setattr_latency": { 2024-05-14T17:17:25.823 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:25.833 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:25.848 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:25.856 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:25.864 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setdirlayout_latency": { 2024-05-14T17:17:25.875 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:25.884 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:25.907 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:25.967 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:25.974 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setfilelock_latency": { 2024-05-14T17:17:26.010 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:26.017 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:26.023 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:26.038 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:26.046 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setlayout_latency": { 2024-05-14T17:17:26.053 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:26.060 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:26.066 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:26.073 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:26.090 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setxattr_latency": { 2024-05-14T17:17:26.098 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:26.105 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:26.111 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:26.118 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:26.132 INFO:teuthology.orchestra.run.smithi151.stdout: "req_symlink_latency": { 2024-05-14T17:17:26.152 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:26.159 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:26.187 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:26.193 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:26.201 INFO:teuthology.orchestra.run.smithi151.stdout: "req_unlink_latency": { 2024-05-14T17:17:26.207 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:26.217 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:26.229 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:26.239 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:26.248 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_revoke_eviction": 0, 2024-05-14T17:17:26.258 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_acquisition_throttle": 0, 2024-05-14T17:17:26.285 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getvxattr_latency": { 2024-05-14T17:17:26.297 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:26.309 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:26.322 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:26.334 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-14T17:17:26.349 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:26.369 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_sessions": { 2024-05-14T17:17:26.381 INFO:teuthology.orchestra.run.smithi151.stdout: "session_count": 2, 2024-05-14T17:17:26.404 INFO:teuthology.orchestra.run.smithi151.stdout: "session_add": 4, 2024-05-14T17:17:26.415 INFO:teuthology.orchestra.run.smithi151.stdout: "session_remove": 2, 2024-05-14T17:17:26.431 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_open": 3, 2024-05-14T17:17:26.447 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_stale": 0, 2024-05-14T17:17:26.458 INFO:teuthology.orchestra.run.smithi151.stdout: "total_load": 0, 2024-05-14T17:17:26.468 INFO:teuthology.orchestra.run.smithi151.stdout: "average_load": 0, 2024-05-14T17:17:26.477 INFO:teuthology.orchestra.run.smithi151.stdout: "avg_session_uptime": 3396, 2024-05-14T17:17:26.485 INFO:teuthology.orchestra.run.smithi151.stdout: "mdthresh_evicted": 0 2024-05-14T17:17:26.494 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:26.515 INFO:teuthology.orchestra.run.smithi151.stdout: "mempool": { 2024-05-14T17:17:26.522 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_bytes": 0, 2024-05-14T17:17:26.534 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_items": 0, 2024-05-14T17:17:26.543 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T17:17:26.551 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_items": 0, 2024-05-14T17:17:26.566 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T17:17:26.605 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_items": 0, 2024-05-14T17:17:26.613 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T17:17:26.646 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T17:17:26.653 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T17:17:26.661 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T17:17:26.668 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T17:17:26.675 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_items": 0, 2024-05-14T17:17:26.683 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T17:17:26.691 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T17:17:26.697 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_bytes": 0, 2024-05-14T17:17:26.704 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_items": 0, 2024-05-14T17:17:26.724 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_bytes": 0, 2024-05-14T17:17:26.732 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_items": 0, 2024-05-14T17:17:26.741 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T17:17:26.747 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T17:17:26.754 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T17:17:26.779 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T17:17:26.814 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T17:17:26.821 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_items": 0, 2024-05-14T17:17:26.854 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_bytes": 0, 2024-05-14T17:17:26.862 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_items": 0, 2024-05-14T17:17:26.870 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T17:17:26.881 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T17:17:26.887 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_bytes": 0, 2024-05-14T17:17:26.894 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_items": 0, 2024-05-14T17:17:26.901 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_bytes": 0, 2024-05-14T17:17:26.907 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_items": 0, 2024-05-14T17:17:26.914 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T17:17:26.946 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_items": 0, 2024-05-14T17:17:26.953 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T17:17:26.970 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_items": 0, 2024-05-14T17:17:26.983 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_bytes": 215112, 2024-05-14T17:17:26.993 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_items": 922, 2024-05-14T17:17:27.004 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_bytes": 0, 2024-05-14T17:17:27.028 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_items": 0, 2024-05-14T17:17:27.035 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_bytes": 0, 2024-05-14T17:17:27.071 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_items": 0, 2024-05-14T17:17:27.081 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_bytes": 0, 2024-05-14T17:17:27.089 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_items": 0, 2024-05-14T17:17:27.099 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_bytes": 0, 2024-05-14T17:17:27.109 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_items": 0, 2024-05-14T17:17:27.123 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_bytes": 8032, 2024-05-14T17:17:27.135 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_items": 167, 2024-05-14T17:17:27.144 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T17:17:27.153 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_items": 0, 2024-05-14T17:17:27.175 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_bytes": 0, 2024-05-14T17:17:27.188 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_items": 0, 2024-05-14T17:17:27.195 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_bytes": 2601658, 2024-05-14T17:17:27.209 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_items": 55234, 2024-05-14T17:17:27.218 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_bytes": 0, 2024-05-14T17:17:27.229 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_items": 0, 2024-05-14T17:17:27.254 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_bytes": 0, 2024-05-14T17:17:27.265 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_items": 0 2024-05-14T17:17:27.297 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:27.305 INFO:teuthology.orchestra.run.smithi151.stdout: "objecter": { 2024-05-14T17:17:27.314 INFO:teuthology.orchestra.run.smithi151.stdout: "op_active": 0, 2024-05-14T17:17:27.338 INFO:teuthology.orchestra.run.smithi151.stdout: "op_laggy": 0, 2024-05-14T17:17:27.345 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send": 4789, 2024-05-14T17:17:27.352 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send_bytes": 21, 2024-05-14T17:17:27.359 INFO:teuthology.orchestra.run.smithi151.stdout: "op_resend": 3, 2024-05-14T17:17:27.366 INFO:teuthology.orchestra.run.smithi151.stdout: "op_reply": 4789, 2024-05-14T17:17:27.374 INFO:teuthology.orchestra.run.smithi151.stdout: "op_latency": { 2024-05-14T17:17:27.384 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 4789, 2024-05-14T17:17:27.391 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 3.361958532, 2024-05-14T17:17:27.398 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000702016 2024-05-14T17:17:27.404 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:27.411 INFO:teuthology.orchestra.run.smithi151.stdout: "op_inflight": 0, 2024-05-14T17:17:27.429 INFO:teuthology.orchestra.run.smithi151.stdout: "oplen_avg": { 2024-05-14T17:17:27.503 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 4789, 2024-05-14T17:17:27.512 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 4790 2024-05-14T17:17:27.538 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:27.546 INFO:teuthology.orchestra.run.smithi151.stdout: "op": 4789, 2024-05-14T17:17:27.554 INFO:teuthology.orchestra.run.smithi151.stdout: "op_r": 4789, 2024-05-14T17:17:27.573 INFO:teuthology.orchestra.run.smithi151.stdout: "op_w": 0, 2024-05-14T17:17:27.581 INFO:teuthology.orchestra.run.smithi151.stdout: "op_rmw": 0, 2024-05-14T17:17:27.589 INFO:teuthology.orchestra.run.smithi151.stdout: "op_pg": 0, 2024-05-14T17:17:27.596 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_stat": 2975, 2024-05-14T17:17:27.602 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_create": 0, 2024-05-14T17:17:27.609 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_read": 1812, 2024-05-14T17:17:27.626 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_write": 0, 2024-05-14T17:17:27.633 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writefull": 0, 2024-05-14T17:17:27.639 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writesame": 0, 2024-05-14T17:17:27.645 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_append": 0, 2024-05-14T17:17:27.655 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_zero": 0, 2024-05-14T17:17:27.671 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_truncate": 0, 2024-05-14T17:17:27.715 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_delete": 0, 2024-05-14T17:17:27.726 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_mapext": 0, 2024-05-14T17:17:27.738 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_sparse_read": 0, 2024-05-14T17:17:27.749 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_clonerange": 0, 2024-05-14T17:17:27.756 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_getxattr": 1, 2024-05-14T17:17:27.772 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_setxattr": 0, 2024-05-14T17:17:27.784 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_cmpxattr": 0, 2024-05-14T17:17:27.796 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_rmxattr": 0, 2024-05-14T17:17:27.803 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_resetxattrs": 0, 2024-05-14T17:17:27.814 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_call": 0, 2024-05-14T17:17:27.827 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_watch": 0, 2024-05-14T17:17:27.842 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_notify": 0, 2024-05-14T17:17:27.854 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_src_cmpxattr": 0, 2024-05-14T17:17:27.862 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls": 0, 2024-05-14T17:17:27.872 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls_filter": 0, 2024-05-14T17:17:27.882 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_other": 0, 2024-05-14T17:17:27.897 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_active": 0, 2024-05-14T17:17:27.926 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_send": 0, 2024-05-14T17:17:27.939 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_resend": 0, 2024-05-14T17:17:27.970 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_ping": 0, 2024-05-14T17:17:27.981 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_active": 0, 2024-05-14T17:17:27.994 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_send": 0, 2024-05-14T17:17:28.007 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_resend": 0, 2024-05-14T17:17:28.020 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_active": 0, 2024-05-14T17:17:28.028 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_send": 0, 2024-05-14T17:17:28.037 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_resend": 0, 2024-05-14T17:17:28.045 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_active": 0, 2024-05-14T17:17:28.056 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_send": 0, 2024-05-14T17:17:28.068 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_resend": 0, 2024-05-14T17:17:28.073 INFO:teuthology.orchestra.run.smithi151.stdout: "command_active": 0, 2024-05-14T17:17:28.082 INFO:teuthology.orchestra.run.smithi151.stdout: "command_send": 0, 2024-05-14T17:17:28.089 INFO:teuthology.orchestra.run.smithi151.stdout: "command_resend": 0, 2024-05-14T17:17:28.096 INFO:teuthology.orchestra.run.smithi151.stdout: "map_epoch": 114, 2024-05-14T17:17:28.105 INFO:teuthology.orchestra.run.smithi151.stdout: "map_full": 0, 2024-05-14T17:17:28.154 INFO:teuthology.orchestra.run.smithi151.stdout: "map_inc": 36, 2024-05-14T17:17:28.159 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_sessions": 4, 2024-05-14T17:17:28.180 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_open": 5, 2024-05-14T17:17:28.189 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_close": 1, 2024-05-14T17:17:28.196 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_laggy": 0, 2024-05-14T17:17:28.216 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_wr": 0, 2024-05-14T17:17:28.223 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_rd": 2, 2024-05-14T17:17:28.229 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_del": 0 2024-05-14T17:17:28.236 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:28.241 INFO:teuthology.orchestra.run.smithi151.stdout: "oft": { 2024-05-14T17:17:28.248 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_objs": 0, 2024-05-14T17:17:28.259 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_kv_pairs": 0, 2024-05-14T17:17:28.267 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_updates": 0, 2024-05-14T17:17:28.275 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_removes": 0 2024-05-14T17:17:28.281 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:28.288 INFO:teuthology.orchestra.run.smithi151.stdout: "osd-slow-ops": { 2024-05-14T17:17:28.306 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_ops_count": 0 2024-05-14T17:17:28.384 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:28.392 INFO:teuthology.orchestra.run.smithi151.stdout: "purge_queue": { 2024-05-14T17:17:28.423 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops": 0, 2024-05-14T17:17:28.436 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops_high_water": 0, 2024-05-14T17:17:28.449 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing": 0, 2024-05-14T17:17:28.476 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_high_water": 0, 2024-05-14T17:17:28.485 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed_ops": 0, 2024-05-14T17:17:28.493 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed": 0, 2024-05-14T17:17:28.504 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_item_in_journal": 0 2024-05-14T17:17:28.517 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:28.524 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T17:17:28.553 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-14T17:17:28.565 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-05-14T17:17:28.572 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-14T17:17:28.582 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 5302, 2024-05-14T17:17:28.589 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 10016375, 2024-05-14T17:17:28.605 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-14T17:17:28.673 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 5302, 2024-05-14T17:17:28.685 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-05-14T17:17:28.709 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-05-14T17:17:28.719 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 5302, 2024-05-14T17:17:28.732 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 10016375, 2024-05-14T17:17:28.772 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-14T17:17:28.781 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:28.787 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:28.793 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:28.800 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-14T17:17:28.808 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:28.820 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_bytes": { 2024-05-14T17:17:28.827 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-14T17:17:28.834 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-05-14T17:17:28.841 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-14T17:17:28.850 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-05-14T17:17:28.864 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-05-14T17:17:28.933 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-14T17:17:28.939 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-05-14T17:17:28.956 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 4789, 2024-05-14T17:17:28.963 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 7202393, 2024-05-14T17:17:28.970 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 4789, 2024-05-14T17:17:29.008 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 7202393, 2024-05-14T17:17:29.016 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-14T17:17:29.021 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:29.027 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:29.033 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:29.039 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-14T17:17:29.054 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:29.061 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_ops": { 2024-05-14T17:17:29.071 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-14T17:17:29.077 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1024, 2024-05-14T17:17:29.085 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-14T17:17:29.097 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-05-14T17:17:29.121 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-05-14T17:17:29.128 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-14T17:17:29.140 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-05-14T17:17:29.154 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 4789, 2024-05-14T17:17:29.163 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 4789, 2024-05-14T17:17:29.175 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 4789, 2024-05-14T17:17:29.183 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 4789, 2024-05-14T17:17:29.194 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-14T17:17:29.210 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:29.218 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:29.228 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:29.246 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-14T17:17:29.262 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:29.272 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle": { 2024-05-14T17:17:29.280 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-14T17:17:29.291 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-05-14T17:17:29.319 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-14T17:17:29.343 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-05-14T17:17:29.355 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-05-14T17:17:29.374 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-14T17:17:29.387 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-05-14T17:17:29.396 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-05-14T17:17:29.406 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-05-14T17:17:29.416 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 0, 2024-05-14T17:17:29.425 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 0, 2024-05-14T17:17:29.437 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-14T17:17:29.445 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:29.458 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:29.471 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:29.485 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-14T17:17:29.496 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:29.503 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle-0x5646884526e0": { 2024-05-14T17:17:29.511 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-14T17:17:29.519 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-05-14T17:17:29.550 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-14T17:17:29.557 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-05-14T17:17:29.583 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-05-14T17:17:29.590 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-14T17:17:29.597 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-05-14T17:17:29.604 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-05-14T17:17:29.610 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-05-14T17:17:29.617 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 0, 2024-05-14T17:17:29.624 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 0, 2024-05-14T17:17:29.634 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-14T17:17:29.640 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:29.655 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:29.661 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:29.668 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-14T17:17:29.676 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-14T17:17:29.685 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-14T17:17:29.695 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:21 smithi151 ceph-mon[36066]: pgmap v719: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 145 B/s wr, 6 op/s 2024-05-14T17:17:29.719 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-05-14T17:17:29.726 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-05-14T17:17:29.741 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-05-14T17:17:29.748 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-05-14T17:17:29.755 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-05-14T17:17:29.762 DEBUG:tasks.check_counter:Getting stats from f 2024-05-14T17:17:29.769 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-14T17:17:29.778 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:23 smithi151 ceph-mon[36066]: pgmap v720: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 145 B/s wr, 4 op/s 2024-05-14T17:17:29.786 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:25 smithi151 ceph-mon[36066]: pgmap v721: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 145 B/s wr, 8 op/s 2024-05-14T17:17:29.795 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:27 smithi151 ceph-mon[36066]: pgmap v722: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 115 B/s wr, 6 op/s 2024-05-14T17:17:29.805 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:25 smithi123 ceph-mon[35971]: pgmap v721: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 145 B/s wr, 8 op/s 2024-05-14T17:17:29.822 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:27 smithi123 ceph-mon[35971]: pgmap v722: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 115 B/s wr, 6 op/s 2024-05-14T17:17:29.832 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:25 smithi002 ceph-mon[31538]: pgmap v721: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 145 B/s wr, 8 op/s 2024-05-14T17:17:29.841 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:27 smithi002 ceph-mon[31538]: pgmap v722: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 115 B/s wr, 6 op/s 2024-05-14T17:17:29.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:29 smithi002 ceph-mon[31538]: pgmap v723: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-14T17:17:30.057 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T17:17:30.103 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:29 smithi151 ceph-mon[36066]: pgmap v723: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-14T17:17:30.159 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:29 smithi123 ceph-mon[35971]: pgmap v723: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-14T17:17:31.934 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:31 smithi151 ceph-mon[36066]: pgmap v724: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T17:17:31.962 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-14T17:17:31.969 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T17:17:31.976 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 1134, 2024-05-14T17:17:32.004 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 1136, 2024-05-14T17:17:32.010 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 4489507, 2024-05-14T17:17:32.017 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 360676, 2024-05-14T17:17:32.024 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 7, 2024-05-14T17:17:32.031 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 2, 2024-05-14T17:17:32.037 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.101356489, 2024-05-14T17:17:32.046 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.026914626, 2024-05-14T17:17:32.053 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.030464014, 2024-05-14T17:17:32.060 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.034324490, 2024-05-14T17:17:32.067 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:17:32.074 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 1134, 2024-05-14T17:17:32.094 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.043896242, 2024-05-14T17:17:32.175 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000038709 2024-05-14T17:17:32.181 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:32.219 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:17:32.227 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:32.232 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:32.247 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:32.253 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:32.260 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-14T17:17:32.267 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 0 2024-05-14T17:17:32.273 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:32.282 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T17:17:32.297 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 1467, 2024-05-14T17:17:32.303 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 1785, 2024-05-14T17:17:32.309 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 3640357, 2024-05-14T17:17:32.316 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 1192075, 2024-05-14T17:17:32.323 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 7, 2024-05-14T17:17:32.330 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 5, 2024-05-14T17:17:32.366 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.151523165, 2024-05-14T17:17:32.372 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.047258004, 2024-05-14T17:17:32.392 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.039360377, 2024-05-14T17:17:32.399 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.049818151, 2024-05-14T17:17:32.405 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:17:32.412 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 1783, 2024-05-14T17:17:32.421 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.082291257, 2024-05-14T17:17:32.430 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000046153 2024-05-14T17:17:32.438 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:32.446 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:17:32.454 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:32.471 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:32.479 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:32.488 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:32.495 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-05-14T17:17:32.503 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 727472 2024-05-14T17:17:32.511 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:32.552 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T17:17:32.561 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 2604, 2024-05-14T17:17:32.576 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 2554, 2024-05-14T17:17:32.583 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 2628906, 2024-05-14T17:17:32.593 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 933484, 2024-05-14T17:17:32.619 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 4, 2024-05-14T17:17:32.626 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 3, 2024-05-14T17:17:32.633 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.300641916, 2024-05-14T17:17:32.641 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.090518428, 2024-05-14T17:17:32.648 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.112807727, 2024-05-14T17:17:32.656 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.089681610, 2024-05-14T17:17:32.679 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:17:32.689 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 2553, 2024-05-14T17:17:32.698 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.120662801, 2024-05-14T17:17:32.706 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000047263 2024-05-14T17:17:32.716 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:32.736 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:17:32.767 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:32.775 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:32.818 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:32.826 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:32.835 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 1976976, 2024-05-14T17:17:32.860 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 268864 2024-05-14T17:17:32.869 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:32.878 INFO:teuthology.orchestra.run.smithi151.stdout: "cct": { 2024-05-14T17:17:32.886 INFO:teuthology.orchestra.run.smithi151.stdout: "total_workers": 1, 2024-05-14T17:17:32.894 INFO:teuthology.orchestra.run.smithi151.stdout: "unhealthy_workers": 0 2024-05-14T17:17:32.903 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:32.913 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-MDSRank": { 2024-05-14T17:17:32.921 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-05-14T17:17:32.929 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-05-14T17:17:32.937 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 7830, 2024-05-14T17:17:32.944 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.226783330, 2024-05-14T17:17:32.953 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000028963 2024-05-14T17:17:32.997 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-14T17:17:33.004 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:33.029 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-PurgeQueue": { 2024-05-14T17:17:33.035 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-05-14T17:17:33.041 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-05-14T17:17:33.077 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:33.084 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:33.091 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:33.098 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-14T17:17:33.105 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:33.112 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": { 2024-05-14T17:17:33.121 INFO:teuthology.orchestra.run.smithi151.stdout: "request": 0, 2024-05-14T17:17:33.128 INFO:teuthology.orchestra.run.smithi151.stdout: "reply": 0, 2024-05-14T17:17:33.135 INFO:teuthology.orchestra.run.smithi151.stdout: "reply_latency": { 2024-05-14T17:17:33.142 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:33.150 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:33.194 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:33.283 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:33.334 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_reply": 0, 2024-05-14T17:17:33.375 INFO:teuthology.orchestra.run.smithi151.stdout: "forward": 0, 2024-05-14T17:17:33.382 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_complete": 0, 2024-05-14T17:17:33.390 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_keys": 0, 2024-05-14T17:17:33.411 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_commit": 0, 2024-05-14T17:17:33.420 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_split": 0, 2024-05-14T17:17:33.427 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_merge": 0, 2024-05-14T17:17:33.433 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes": 971, 2024-05-14T17:17:33.441 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_top": 316, 2024-05-14T17:17:33.448 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_bottom": 655, 2024-05-14T17:17:33.483 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pin_tail": 0, 2024-05-14T17:17:33.492 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pinned": 519, 2024-05-14T17:17:33.500 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_expired": 0, 2024-05-14T17:17:33.508 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_with_caps": 0, 2024-05-14T17:17:33.517 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": 0, 2024-05-14T17:17:33.528 INFO:teuthology.orchestra.run.smithi151.stdout: "subtrees": 24, 2024-05-14T17:17:33.581 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse": 0, 2024-05-14T17:17:33.589 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_hit": 0, 2024-05-14T17:17:33.628 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_forward": 0, 2024-05-14T17:17:33.635 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_discover": 0, 2024-05-14T17:17:33.644 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_dir_fetch": 0, 2024-05-14T17:17:33.669 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_remote_ino": 0, 2024-05-14T17:17:33.678 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_lock": 0, 2024-05-14T17:17:33.688 INFO:teuthology.orchestra.run.smithi151.stdout: "load_cent": 0, 2024-05-14T17:17:33.696 INFO:teuthology.orchestra.run.smithi151.stdout: "q": 0, 2024-05-14T17:17:33.706 INFO:teuthology.orchestra.run.smithi151.stdout: "exported": 0, 2024-05-14T17:17:33.714 INFO:teuthology.orchestra.run.smithi151.stdout: "exported_inodes": 0, 2024-05-14T17:17:33.777 INFO:teuthology.orchestra.run.smithi151.stdout: "imported": 0, 2024-05-14T17:17:33.786 INFO:teuthology.orchestra.run.smithi151.stdout: "imported_inodes": 0, 2024-05-14T17:17:33.795 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_dir_fetch": 0, 2024-05-14T17:17:33.804 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_backtrace_fetch": 0, 2024-05-14T17:17:33.815 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_peer_discover": 0, 2024-05-14T17:17:33.847 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rfiles": 0, 2024-05-14T17:17:33.895 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rbytes": 0, 2024-05-14T17:17:33.906 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rsnaps": 0, 2024-05-14T17:17:33.946 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_fetch": 0, 2024-05-14T17:17:33.955 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_set_tag": 0, 2024-05-14T17:17:33.964 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_repaired": 0, 2024-05-14T17:17:33.971 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_inotable_repaired": 0, 2024-05-14T17:17:33.979 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_inodes": 0, 2024-05-14T17:17:33.986 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_base_inodes": 0, 2024-05-14T17:17:33.993 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dirfrag_rstats": 0, 2024-05-14T17:17:34.000 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_file_inodes": 0, 2024-05-14T17:17:34.008 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_inode_file_caps": 0, 2024-05-14T17:17:34.022 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_revoke": 0, 2024-05-14T17:17:34.030 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_grant": 0, 2024-05-14T17:17:34.037 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_trunc": 0, 2024-05-14T17:17:34.044 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-14T17:17:34.051 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-14T17:17:34.070 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps": 0, 2024-05-14T17:17:34.107 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps_dirty": 0, 2024-05-14T17:17:34.112 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_cap_release": 0, 2024-05-14T17:17:34.144 INFO:teuthology.orchestra.run.smithi151.stdout: "process_request_cap_release": 0 2024-05-14T17:17:34.149 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:34.156 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_cache": { 2024-05-14T17:17:34.163 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update": 0, 2024-05-14T17:17:34.171 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update_receipt": 0, 2024-05-14T17:17:34.177 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_try_discover": 0, 2024-05-14T17:17:34.185 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_send_discover": 0, 2024-05-14T17:17:34.192 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_handle_discover": 0, 2024-05-14T17:17:34.199 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays": 0, 2024-05-14T17:17:34.214 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_delayed": 0, 2024-05-14T17:17:34.221 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_enqueuing": 0, 2024-05-14T17:17:34.228 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_created": 0, 2024-05-14T17:17:34.234 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_enqueued": 0, 2024-05-14T17:17:34.241 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_reintegrated": 0, 2024-05-14T17:17:34.254 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_migrated": 0, 2024-05-14T17:17:34.328 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_processing": 0, 2024-05-14T17:17:34.334 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_enqueued": 0, 2024-05-14T17:17:34.381 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_prioritized": 0, 2024-05-14T17:17:34.386 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_started": 0, 2024-05-14T17:17:34.393 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_completed": 0, 2024-05-14T17:17:34.403 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_path": 0, 2024-05-14T17:17:34.409 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_inode": 0, 2024-05-14T17:17:34.416 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_enqueue_scrub": 0, 2024-05-14T17:17:34.422 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_exportdir": 0, 2024-05-14T17:17:34.429 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_flush": 0, 2024-05-14T17:17:34.435 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragmentdir": 0, 2024-05-14T17:17:34.448 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragstats": 0, 2024-05-14T17:17:34.454 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_inodestats": 0 2024-05-14T17:17:34.460 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:34.467 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_log": { 2024-05-14T17:17:34.473 INFO:teuthology.orchestra.run.smithi151.stdout: "evlrg": 0, 2024-05-14T17:17:34.487 INFO:teuthology.orchestra.run.smithi151.stdout: "evadd": 0, 2024-05-14T17:17:34.533 INFO:teuthology.orchestra.run.smithi151.stdout: "evex": 0, 2024-05-14T17:17:34.543 INFO:teuthology.orchestra.run.smithi151.stdout: "evtrm": 0, 2024-05-14T17:17:34.573 INFO:teuthology.orchestra.run.smithi151.stdout: "ev": 3214, 2024-05-14T17:17:34.578 INFO:teuthology.orchestra.run.smithi151.stdout: "evexg": 0, 2024-05-14T17:17:34.587 INFO:teuthology.orchestra.run.smithi151.stdout: "evexd": 0, 2024-05-14T17:17:34.600 INFO:teuthology.orchestra.run.smithi151.stdout: "segadd": 0, 2024-05-14T17:17:34.609 INFO:teuthology.orchestra.run.smithi151.stdout: "segex": 0, 2024-05-14T17:17:34.616 INFO:teuthology.orchestra.run.smithi151.stdout: "segtrm": 0, 2024-05-14T17:17:34.623 INFO:teuthology.orchestra.run.smithi151.stdout: "seg": 6, 2024-05-14T17:17:34.631 INFO:teuthology.orchestra.run.smithi151.stdout: "segmjr": 2, 2024-05-14T17:17:34.638 INFO:teuthology.orchestra.run.smithi151.stdout: "segexg": 0, 2024-05-14T17:17:34.665 INFO:teuthology.orchestra.run.smithi151.stdout: "segexd": 0, 2024-05-14T17:17:34.671 INFO:teuthology.orchestra.run.smithi151.stdout: "expos": 4194304, 2024-05-14T17:17:34.678 INFO:teuthology.orchestra.run.smithi151.stdout: "wrpos": 11657249, 2024-05-14T17:17:34.686 INFO:teuthology.orchestra.run.smithi151.stdout: "rdpos": 11656156, 2024-05-14T17:17:34.695 INFO:teuthology.orchestra.run.smithi151.stdout: "jlat": { 2024-05-14T17:17:34.704 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:34.713 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:34.735 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:34.773 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:34.782 INFO:teuthology.orchestra.run.smithi151.stdout: "replayed": 3214 2024-05-14T17:17:34.791 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:34.833 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_mem": { 2024-05-14T17:17:34.841 INFO:teuthology.orchestra.run.smithi151.stdout: "ino": 757, 2024-05-14T17:17:34.848 INFO:teuthology.orchestra.run.smithi151.stdout: "ino+": 1921, 2024-05-14T17:17:34.856 INFO:teuthology.orchestra.run.smithi151.stdout: "ino-": 1164, 2024-05-14T17:17:34.863 INFO:teuthology.orchestra.run.smithi151.stdout: "dir": 72, 2024-05-14T17:17:34.870 INFO:teuthology.orchestra.run.smithi151.stdout: "dir+": 108, 2024-05-14T17:17:34.898 INFO:teuthology.orchestra.run.smithi151.stdout: "dir-": 36, 2024-05-14T17:17:34.906 INFO:teuthology.orchestra.run.smithi151.stdout: "dn": 971, 2024-05-14T17:17:34.914 INFO:teuthology.orchestra.run.smithi151.stdout: "dn+": 2003, 2024-05-14T17:17:34.923 INFO:teuthology.orchestra.run.smithi151.stdout: "dn-": 1032, 2024-05-14T17:17:34.931 INFO:teuthology.orchestra.run.smithi151.stdout: "cap": 0, 2024-05-14T17:17:34.946 INFO:teuthology.orchestra.run.smithi151.stdout: "cap+": 0, 2024-05-14T17:17:34.954 INFO:teuthology.orchestra.run.smithi151.stdout: "cap-": 0, 2024-05-14T17:17:34.972 INFO:teuthology.orchestra.run.smithi151.stdout: "rss": 57936, 2024-05-14T17:17:35.017 INFO:teuthology.orchestra.run.smithi151.stdout: "heap": 215416 2024-05-14T17:17:35.026 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:35.035 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_server": { 2024-05-14T17:17:35.050 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_client_request": 0, 2024-05-14T17:17:35.058 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_server_request": 0, 2024-05-14T17:17:35.068 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_request": 0, 2024-05-14T17:17:35.077 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_session": 0, 2024-05-14T17:17:35.085 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_peer_request": 0, 2024-05-14T17:17:35.093 INFO:teuthology.orchestra.run.smithi151.stdout: "req_create_latency": { 2024-05-14T17:17:35.125 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:35.132 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:35.139 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:35.147 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:35.155 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getattr_latency": { 2024-05-14T17:17:35.163 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:35.169 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:35.198 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:35.234 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:35.240 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getfilelock_latency": { 2024-05-14T17:17:35.247 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:35.267 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:35.274 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:35.321 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:35.329 INFO:teuthology.orchestra.run.smithi151.stdout: "req_link_latency": { 2024-05-14T17:17:35.338 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:35.346 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:35.357 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:35.364 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:35.372 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookup_latency": { 2024-05-14T17:17:35.381 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:35.388 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:35.397 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:35.419 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:35.432 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookuphash_latency": { 2024-05-14T17:17:35.450 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:35.458 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:35.467 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:35.477 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:35.485 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupino_latency": { 2024-05-14T17:17:35.494 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:35.502 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:35.510 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:35.518 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:35.532 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupname_latency": { 2024-05-14T17:17:35.541 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:35.549 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:35.557 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:35.566 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:35.581 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupparent_latency": { 2024-05-14T17:17:35.589 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:35.628 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:35.652 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:35.657 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:35.663 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupsnap_latency": { 2024-05-14T17:17:35.670 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:35.677 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:35.683 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:35.691 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:35.698 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lssnap_latency": { 2024-05-14T17:17:35.705 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:35.719 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:35.727 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:35.734 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:35.741 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mkdir_latency": { 2024-05-14T17:17:35.748 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:35.756 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:35.784 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:35.791 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:35.820 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mknod_latency": { 2024-05-14T17:17:35.827 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:35.833 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:35.842 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:35.848 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:35.855 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mksnap_latency": { 2024-05-14T17:17:35.862 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:35.869 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:35.875 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:35.895 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:35.902 INFO:teuthology.orchestra.run.smithi151.stdout: "req_open_latency": { 2024-05-14T17:17:35.912 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:35.919 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:35.926 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:35.939 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:36.031 INFO:teuthology.orchestra.run.smithi151.stdout: "req_readdir_latency": { 2024-05-14T17:17:36.038 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:36.060 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:36.066 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:36.075 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:36.084 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rename_latency": { 2024-05-14T17:17:36.090 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:36.097 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:36.104 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:36.110 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:36.116 INFO:teuthology.orchestra.run.smithi151.stdout: "req_renamesnap_latency": { 2024-05-14T17:17:36.128 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:36.135 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:36.142 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:36.149 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:36.156 INFO:teuthology.orchestra.run.smithi151.stdout: "req_snapdiff_latency": { 2024-05-14T17:17:36.165 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:36.173 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:36.205 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:36.222 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:36.230 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmdir_latency": { 2024-05-14T17:17:36.239 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:36.251 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:36.257 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:36.264 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:36.272 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmsnap_latency": { 2024-05-14T17:17:36.280 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:36.289 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:36.303 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:36.312 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:36.321 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmxattr_latency": { 2024-05-14T17:17:36.330 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:36.339 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:36.348 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:36.357 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:36.377 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setattr_latency": { 2024-05-14T17:17:36.413 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:36.421 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:36.429 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:36.438 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:36.446 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setdirlayout_latency": { 2024-05-14T17:17:36.456 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:36.465 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:36.474 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:36.483 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:36.512 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setfilelock_latency": { 2024-05-14T17:17:36.522 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:36.531 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:36.537 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:36.546 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:36.565 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setlayout_latency": { 2024-05-14T17:17:36.597 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:36.606 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:36.622 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:36.630 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:36.638 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setxattr_latency": { 2024-05-14T17:17:36.668 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:36.677 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:36.685 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:36.694 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:36.704 INFO:teuthology.orchestra.run.smithi151.stdout: "req_symlink_latency": { 2024-05-14T17:17:36.712 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:36.721 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:36.730 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:36.739 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:36.747 INFO:teuthology.orchestra.run.smithi151.stdout: "req_unlink_latency": { 2024-05-14T17:17:36.756 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:36.765 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:36.801 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:36.811 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:36.837 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_revoke_eviction": 0, 2024-05-14T17:17:36.843 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_acquisition_throttle": 0, 2024-05-14T17:17:36.849 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getvxattr_latency": { 2024-05-14T17:17:36.859 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:36.865 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:36.871 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:36.877 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-14T17:17:36.882 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:36.888 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_sessions": { 2024-05-14T17:17:36.896 INFO:teuthology.orchestra.run.smithi151.stdout: "session_count": 1, 2024-05-14T17:17:36.903 INFO:teuthology.orchestra.run.smithi151.stdout: "session_add": 3, 2024-05-14T17:17:36.910 INFO:teuthology.orchestra.run.smithi151.stdout: "session_remove": 2, 2024-05-14T17:17:36.919 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_open": 3, 2024-05-14T17:17:36.926 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_stale": 0, 2024-05-14T17:17:36.940 INFO:teuthology.orchestra.run.smithi151.stdout: "total_load": 0, 2024-05-14T17:17:36.971 INFO:teuthology.orchestra.run.smithi151.stdout: "average_load": 0, 2024-05-14T17:17:36.979 INFO:teuthology.orchestra.run.smithi151.stdout: "avg_session_uptime": 5449, 2024-05-14T17:17:37.015 INFO:teuthology.orchestra.run.smithi151.stdout: "mdthresh_evicted": 0 2024-05-14T17:17:37.021 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:37.027 INFO:teuthology.orchestra.run.smithi151.stdout: "mempool": { 2024-05-14T17:17:37.051 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_bytes": 0, 2024-05-14T17:17:37.056 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_items": 0, 2024-05-14T17:17:37.065 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T17:17:37.073 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_items": 0, 2024-05-14T17:17:37.080 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T17:17:37.086 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_items": 0, 2024-05-14T17:17:37.104 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T17:17:37.112 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T17:17:37.118 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T17:17:37.126 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T17:17:37.132 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T17:17:37.149 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_items": 0, 2024-05-14T17:17:37.193 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T17:17:37.201 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T17:17:37.271 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_bytes": 0, 2024-05-14T17:17:37.278 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_items": 0, 2024-05-14T17:17:37.285 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_bytes": 0, 2024-05-14T17:17:37.319 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_items": 0, 2024-05-14T17:17:37.325 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T17:17:37.331 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T17:17:37.337 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T17:17:37.343 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T17:17:37.349 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T17:17:37.371 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_items": 0, 2024-05-14T17:17:37.378 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_bytes": 0, 2024-05-14T17:17:37.386 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_items": 0, 2024-05-14T17:17:37.396 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T17:17:37.410 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T17:17:37.424 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_bytes": 0, 2024-05-14T17:17:37.464 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_items": 0, 2024-05-14T17:17:37.473 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_bytes": 0, 2024-05-14T17:17:37.498 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_items": 0, 2024-05-14T17:17:37.509 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T17:17:37.518 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_items": 0, 2024-05-14T17:17:37.530 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T17:17:37.539 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_items": 0, 2024-05-14T17:17:37.548 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_bytes": 227087, 2024-05-14T17:17:37.556 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_items": 817, 2024-05-14T17:17:37.565 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_bytes": 0, 2024-05-14T17:17:37.575 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_items": 0, 2024-05-14T17:17:37.588 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_bytes": 0, 2024-05-14T17:17:37.599 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_items": 0, 2024-05-14T17:17:37.609 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_bytes": 0, 2024-05-14T17:17:37.617 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_items": 0, 2024-05-14T17:17:37.626 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_bytes": 0, 2024-05-14T17:17:37.635 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_items": 0, 2024-05-14T17:17:37.675 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_bytes": 8032, 2024-05-14T17:17:37.682 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_items": 167, 2024-05-14T17:17:37.705 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T17:17:37.714 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_items": 0, 2024-05-14T17:17:37.722 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_bytes": 0, 2024-05-14T17:17:37.739 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_items": 0, 2024-05-14T17:17:37.747 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_bytes": 2537074, 2024-05-14T17:17:37.755 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_items": 19737, 2024-05-14T17:17:37.764 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_bytes": 0, 2024-05-14T17:17:37.773 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_items": 0, 2024-05-14T17:17:37.787 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_bytes": 0, 2024-05-14T17:17:37.816 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_items": 0 2024-05-14T17:17:37.827 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:37.838 INFO:teuthology.orchestra.run.smithi151.stdout: "objecter": { 2024-05-14T17:17:37.852 INFO:teuthology.orchestra.run.smithi151.stdout: "op_active": 0, 2024-05-14T17:17:37.864 INFO:teuthology.orchestra.run.smithi151.stdout: "op_laggy": 0, 2024-05-14T17:17:37.876 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send": 4685, 2024-05-14T17:17:37.903 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send_bytes": 21, 2024-05-14T17:17:37.917 INFO:teuthology.orchestra.run.smithi151.stdout: "op_resend": 3, 2024-05-14T17:17:37.938 INFO:teuthology.orchestra.run.smithi151.stdout: "op_reply": 4685, 2024-05-14T17:17:37.952 INFO:teuthology.orchestra.run.smithi151.stdout: "op_latency": { 2024-05-14T17:17:37.976 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 4685, 2024-05-14T17:17:38.015 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 10.015877316, 2024-05-14T17:17:38.026 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.002137860 2024-05-14T17:17:38.038 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:38.053 INFO:teuthology.orchestra.run.smithi151.stdout: "op_inflight": 0, 2024-05-14T17:17:38.065 INFO:teuthology.orchestra.run.smithi151.stdout: "oplen_avg": { 2024-05-14T17:17:38.077 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 4685, 2024-05-14T17:17:38.134 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 4686 2024-05-14T17:17:38.143 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:38.158 INFO:teuthology.orchestra.run.smithi151.stdout: "op": 4685, 2024-05-14T17:17:38.168 INFO:teuthology.orchestra.run.smithi151.stdout: "op_r": 4685, 2024-05-14T17:17:38.181 INFO:teuthology.orchestra.run.smithi151.stdout: "op_w": 0, 2024-05-14T17:17:38.218 INFO:teuthology.orchestra.run.smithi151.stdout: "op_rmw": 0, 2024-05-14T17:17:38.276 INFO:teuthology.orchestra.run.smithi151.stdout: "op_pg": 0, 2024-05-14T17:17:38.287 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_stat": 2895, 2024-05-14T17:17:38.356 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_create": 0, 2024-05-14T17:17:38.365 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_read": 1788, 2024-05-14T17:17:38.375 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_write": 0, 2024-05-14T17:17:38.391 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writefull": 0, 2024-05-14T17:17:38.401 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writesame": 0, 2024-05-14T17:17:38.411 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_append": 0, 2024-05-14T17:17:38.421 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_zero": 0, 2024-05-14T17:17:38.429 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_truncate": 0, 2024-05-14T17:17:38.437 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_delete": 0, 2024-05-14T17:17:38.464 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_mapext": 0, 2024-05-14T17:17:38.472 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_sparse_read": 0, 2024-05-14T17:17:38.482 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_clonerange": 0, 2024-05-14T17:17:38.490 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_getxattr": 1, 2024-05-14T17:17:38.498 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_setxattr": 0, 2024-05-14T17:17:38.518 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_cmpxattr": 0, 2024-05-14T17:17:38.548 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_rmxattr": 0, 2024-05-14T17:17:38.555 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_resetxattrs": 0, 2024-05-14T17:17:38.572 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_call": 0, 2024-05-14T17:17:38.581 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_watch": 0, 2024-05-14T17:17:38.590 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_notify": 0, 2024-05-14T17:17:38.611 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_src_cmpxattr": 0, 2024-05-14T17:17:38.620 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls": 0, 2024-05-14T17:17:38.629 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls_filter": 0, 2024-05-14T17:17:38.637 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_other": 0, 2024-05-14T17:17:38.647 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_active": 0, 2024-05-14T17:17:38.657 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_send": 0, 2024-05-14T17:17:38.672 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_resend": 0, 2024-05-14T17:17:38.682 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_ping": 0, 2024-05-14T17:17:38.692 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_active": 0, 2024-05-14T17:17:38.700 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_send": 0, 2024-05-14T17:17:38.710 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_resend": 0, 2024-05-14T17:17:38.727 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_active": 0, 2024-05-14T17:17:38.785 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_send": 0, 2024-05-14T17:17:38.794 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_resend": 0, 2024-05-14T17:17:38.823 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_active": 0, 2024-05-14T17:17:38.832 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_send": 0, 2024-05-14T17:17:38.842 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_resend": 0, 2024-05-14T17:17:38.853 INFO:teuthology.orchestra.run.smithi151.stdout: "command_active": 0, 2024-05-14T17:17:38.860 INFO:teuthology.orchestra.run.smithi151.stdout: "command_send": 0, 2024-05-14T17:17:38.869 INFO:teuthology.orchestra.run.smithi151.stdout: "command_resend": 0, 2024-05-14T17:17:38.877 INFO:teuthology.orchestra.run.smithi151.stdout: "map_epoch": 114, 2024-05-14T17:17:38.888 INFO:teuthology.orchestra.run.smithi151.stdout: "map_full": 0, 2024-05-14T17:17:38.901 INFO:teuthology.orchestra.run.smithi151.stdout: "map_inc": 36, 2024-05-14T17:17:38.915 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_sessions": 6, 2024-05-14T17:17:38.927 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_open": 7, 2024-05-14T17:17:38.941 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_close": 1, 2024-05-14T17:17:38.955 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_laggy": 0, 2024-05-14T17:17:38.967 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_wr": 0, 2024-05-14T17:17:38.985 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_rd": 2, 2024-05-14T17:17:39.018 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_del": 0 2024-05-14T17:17:39.033 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:39.056 INFO:teuthology.orchestra.run.smithi151.stdout: "oft": { 2024-05-14T17:17:39.068 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_objs": 0, 2024-05-14T17:17:39.080 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_kv_pairs": 0, 2024-05-14T17:17:39.103 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_updates": 0, 2024-05-14T17:17:39.118 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_removes": 0 2024-05-14T17:17:39.129 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:39.141 INFO:teuthology.orchestra.run.smithi151.stdout: "osd-slow-ops": { 2024-05-14T17:17:39.154 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_ops_count": 0 2024-05-14T17:17:39.169 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:39.190 INFO:teuthology.orchestra.run.smithi151.stdout: "purge_queue": { 2024-05-14T17:17:39.201 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops": 0, 2024-05-14T17:17:39.213 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops_high_water": 0, 2024-05-14T17:17:39.225 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing": 0, 2024-05-14T17:17:39.235 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_high_water": 0, 2024-05-14T17:17:39.252 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed_ops": 0, 2024-05-14T17:17:39.320 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed": 0, 2024-05-14T17:17:39.331 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_item_in_journal": 0 2024-05-14T17:17:39.357 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:39.368 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T17:17:39.380 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-14T17:17:39.403 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-05-14T17:17:39.412 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-14T17:17:39.423 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 5201, 2024-05-14T17:17:39.437 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 10261567, 2024-05-14T17:17:39.452 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-14T17:17:39.466 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 5201, 2024-05-14T17:17:39.482 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-05-14T17:17:39.494 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-05-14T17:17:39.505 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 5201, 2024-05-14T17:17:39.514 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 10261567, 2024-05-14T17:17:39.526 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-14T17:17:39.544 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:39.581 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:39.589 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:39.615 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-14T17:17:39.625 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:39.633 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_bytes": { 2024-05-14T17:17:39.643 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-14T17:17:39.653 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-05-14T17:17:39.662 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-14T17:17:39.672 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-05-14T17:17:39.681 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-05-14T17:17:39.690 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-14T17:17:39.707 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-05-14T17:17:39.717 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 4685, 2024-05-14T17:17:39.726 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 7462950, 2024-05-14T17:17:39.735 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 4685, 2024-05-14T17:17:39.743 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 7462950, 2024-05-14T17:17:39.752 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-14T17:17:39.812 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:39.819 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:39.840 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:39.848 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-14T17:17:39.856 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:39.867 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_ops": { 2024-05-14T17:17:39.875 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-14T17:17:39.882 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1024, 2024-05-14T17:17:39.892 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-14T17:17:39.901 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-05-14T17:17:39.920 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-05-14T17:17:39.949 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-14T17:17:39.957 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-05-14T17:17:39.965 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 4685, 2024-05-14T17:17:39.974 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 4685, 2024-05-14T17:17:39.983 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 4685, 2024-05-14T17:17:39.994 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 4685, 2024-05-14T17:17:40.037 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-14T17:17:40.049 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:40.086 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:40.091 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:40.100 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-14T17:17:40.108 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:40.120 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle": { 2024-05-14T17:17:40.132 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-14T17:17:40.143 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-05-14T17:17:40.156 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-14T17:17:40.170 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-05-14T17:17:40.199 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-05-14T17:17:40.209 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-14T17:17:40.222 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-05-14T17:17:40.232 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-05-14T17:17:40.242 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-05-14T17:17:40.262 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 0, 2024-05-14T17:17:40.296 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 0, 2024-05-14T17:17:40.306 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-14T17:17:40.328 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:40.336 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:40.350 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:40.361 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-14T17:17:40.371 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:40.382 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle-0x564b863e46e0": { 2024-05-14T17:17:40.393 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-14T17:17:40.403 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-05-14T17:17:40.416 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-14T17:17:40.432 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-05-14T17:17:40.446 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-05-14T17:17:40.459 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-14T17:17:40.470 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-05-14T17:17:40.480 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-05-14T17:17:40.495 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-05-14T17:17:40.572 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 0, 2024-05-14T17:17:40.581 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 0, 2024-05-14T17:17:40.594 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-14T17:17:40.604 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:40.614 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:40.636 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:40.647 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-14T17:17:40.657 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-14T17:17:40.669 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-14T17:17:40.682 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:31 smithi123 ceph-mon[35971]: pgmap v724: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T17:17:40.691 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:33 smithi123 ceph-mon[35971]: pgmap v725: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:17:40.707 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:35 smithi123 ceph-mon[35971]: pgmap v726: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:17:40.717 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:37 smithi123 ceph-mon[35971]: pgmap v727: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:17:40.726 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:39 smithi123 ceph-mon[35971]: pgmap v728: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-05-14T17:17:40.735 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:31 smithi002 ceph-mon[31538]: pgmap v724: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T17:17:40.745 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:33 smithi002 ceph-mon[31538]: pgmap v725: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:17:40.762 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:35 smithi002 ceph-mon[31538]: pgmap v726: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:17:40.823 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:37 smithi002 ceph-mon[31538]: pgmap v727: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:17:40.833 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:39 smithi002 ceph-mon[31538]: pgmap v728: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-05-14T17:17:40.863 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-05-14T17:17:40.873 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-05-14T17:17:40.883 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-05-14T17:17:40.904 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-05-14T17:17:40.913 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-05-14T17:17:40.922 DEBUG:tasks.check_counter:Getting stats from i 2024-05-14T17:17:40.932 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-14T17:17:40.944 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:33 smithi151 ceph-mon[36066]: pgmap v725: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:17:40.953 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:35 smithi151 ceph-mon[36066]: pgmap v726: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:17:40.966 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:37 smithi151 ceph-mon[36066]: pgmap v727: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:17:40.975 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:39 smithi151 ceph-mon[36066]: pgmap v728: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-05-14T17:17:41.234 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T17:17:41.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:41 smithi002 ceph-mon[31538]: pgmap v729: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T17:17:42.103 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:41 smithi151 ceph-mon[36066]: pgmap v729: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T17:17:42.159 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:41 smithi123 ceph-mon[35971]: pgmap v729: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T17:17:43.172 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-14T17:17:43.179 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T17:17:43.255 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 150, 2024-05-14T17:17:43.261 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 152, 2024-05-14T17:17:43.270 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 54170, 2024-05-14T17:17:43.302 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 47710, 2024-05-14T17:17:43.309 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 6, 2024-05-14T17:17:43.320 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 2, 2024-05-14T17:17:43.329 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.021436546, 2024-05-14T17:17:43.339 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.006479210, 2024-05-14T17:17:43.351 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.005728084, 2024-05-14T17:17:43.370 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.006252207, 2024-05-14T17:17:43.380 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:17:43.398 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 150, 2024-05-14T17:17:43.405 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.010717114, 2024-05-14T17:17:43.414 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000071447 2024-05-14T17:17:43.447 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:43.555 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:17:43.565 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:43.594 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:43.605 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:43.615 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:43.628 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-14T17:17:43.639 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 0 2024-05-14T17:17:43.651 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:43.663 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T17:17:43.678 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 368, 2024-05-14T17:17:43.690 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 362, 2024-05-14T17:17:43.707 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 460479, 2024-05-14T17:17:43.717 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 125123, 2024-05-14T17:17:43.729 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 4, 2024-05-14T17:17:43.740 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 3, 2024-05-14T17:17:43.749 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.030675800, 2024-05-14T17:17:43.762 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.010956991, 2024-05-14T17:17:43.802 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.010549339, 2024-05-14T17:17:43.810 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.008078205, 2024-05-14T17:17:43.837 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:17:43.844 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 362, 2024-05-14T17:17:43.853 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.018744660, 2024-05-14T17:17:43.887 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000051780 2024-05-14T17:17:43.897 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:43.905 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:17:43.911 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:43.918 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:43.925 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:43.941 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:43.950 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 196512, 2024-05-14T17:17:43.958 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 27440 2024-05-14T17:17:43.967 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:43.975 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T17:17:43.984 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 4, 2024-05-14T17:17:44.082 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 36, 2024-05-14T17:17:44.090 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 2136, 2024-05-14T17:17:44.165 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 143092, 2024-05-14T17:17:44.173 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 5, 2024-05-14T17:17:44.181 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 4, 2024-05-14T17:17:44.188 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.003213459, 2024-05-14T17:17:44.195 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.001304253, 2024-05-14T17:17:44.204 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.001395455, 2024-05-14T17:17:44.214 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000050400, 2024-05-14T17:17:44.220 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:17:44.230 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 34, 2024-05-14T17:17:44.273 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.003535960, 2024-05-14T17:17:44.287 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000103998 2024-05-14T17:17:44.302 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:44.316 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:17:44.339 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:44.354 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:44.403 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:44.417 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:44.478 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-14T17:17:44.492 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 142128 2024-05-14T17:17:44.506 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:44.529 INFO:teuthology.orchestra.run.smithi151.stdout: "cct": { 2024-05-14T17:17:44.544 INFO:teuthology.orchestra.run.smithi151.stdout: "total_workers": 1, 2024-05-14T17:17:44.556 INFO:teuthology.orchestra.run.smithi151.stdout: "unhealthy_workers": 0 2024-05-14T17:17:44.570 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:44.583 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-MDSRank": { 2024-05-14T17:17:44.597 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-05-14T17:17:44.641 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-05-14T17:17:44.651 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 761, 2024-05-14T17:17:44.662 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.022414301, 2024-05-14T17:17:44.675 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000029453 2024-05-14T17:17:44.688 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-14T17:17:44.704 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:44.765 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-PurgeQueue": { 2024-05-14T17:17:44.775 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-05-14T17:17:44.797 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-05-14T17:17:44.806 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:44.813 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:44.829 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:44.840 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-14T17:17:44.850 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:44.859 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": { 2024-05-14T17:17:44.869 INFO:teuthology.orchestra.run.smithi151.stdout: "request": 0, 2024-05-14T17:17:44.878 INFO:teuthology.orchestra.run.smithi151.stdout: "reply": 0, 2024-05-14T17:17:44.900 INFO:teuthology.orchestra.run.smithi151.stdout: "reply_latency": { 2024-05-14T17:17:44.910 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:44.917 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:44.925 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:44.932 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:44.946 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_reply": 0, 2024-05-14T17:17:45.011 INFO:teuthology.orchestra.run.smithi151.stdout: "forward": 0, 2024-05-14T17:17:45.018 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_complete": 0, 2024-05-14T17:17:45.048 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_keys": 0, 2024-05-14T17:17:45.055 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_commit": 0, 2024-05-14T17:17:45.067 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_split": 0, 2024-05-14T17:17:45.080 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_merge": 0, 2024-05-14T17:17:45.089 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes": 9, 2024-05-14T17:17:45.100 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_top": 2, 2024-05-14T17:17:45.111 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_bottom": 7, 2024-05-14T17:17:45.122 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pin_tail": 0, 2024-05-14T17:17:45.133 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pinned": 9, 2024-05-14T17:17:45.158 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_expired": 0, 2024-05-14T17:17:45.170 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_with_caps": 0, 2024-05-14T17:17:45.183 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": 0, 2024-05-14T17:17:45.194 INFO:teuthology.orchestra.run.smithi151.stdout: "subtrees": 9, 2024-05-14T17:17:45.206 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse": 0, 2024-05-14T17:17:45.221 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_hit": 0, 2024-05-14T17:17:45.258 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_forward": 0, 2024-05-14T17:17:45.268 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_discover": 0, 2024-05-14T17:17:45.290 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_dir_fetch": 0, 2024-05-14T17:17:45.302 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_remote_ino": 0, 2024-05-14T17:17:45.312 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_lock": 0, 2024-05-14T17:17:45.332 INFO:teuthology.orchestra.run.smithi151.stdout: "load_cent": 0, 2024-05-14T17:17:45.342 INFO:teuthology.orchestra.run.smithi151.stdout: "q": 0, 2024-05-14T17:17:45.353 INFO:teuthology.orchestra.run.smithi151.stdout: "exported": 0, 2024-05-14T17:17:45.363 INFO:teuthology.orchestra.run.smithi151.stdout: "exported_inodes": 0, 2024-05-14T17:17:45.375 INFO:teuthology.orchestra.run.smithi151.stdout: "imported": 0, 2024-05-14T17:17:45.387 INFO:teuthology.orchestra.run.smithi151.stdout: "imported_inodes": 0, 2024-05-14T17:17:45.408 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_dir_fetch": 0, 2024-05-14T17:17:45.420 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_backtrace_fetch": 0, 2024-05-14T17:17:45.432 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_peer_discover": 0, 2024-05-14T17:17:45.439 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rfiles": 1026, 2024-05-14T17:17:45.446 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rbytes": 1702131160, 2024-05-14T17:17:45.463 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rsnaps": 7, 2024-05-14T17:17:45.496 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_fetch": 0, 2024-05-14T17:17:45.503 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_set_tag": 0, 2024-05-14T17:17:45.523 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_repaired": 0, 2024-05-14T17:17:45.531 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_inotable_repaired": 0, 2024-05-14T17:17:45.540 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_inodes": 0, 2024-05-14T17:17:45.548 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_base_inodes": 0, 2024-05-14T17:17:45.556 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dirfrag_rstats": 0, 2024-05-14T17:17:45.565 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_file_inodes": 0, 2024-05-14T17:17:45.576 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_inode_file_caps": 0, 2024-05-14T17:17:45.585 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_revoke": 0, 2024-05-14T17:17:45.593 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_grant": 0, 2024-05-14T17:17:45.601 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_trunc": 0, 2024-05-14T17:17:45.610 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-14T17:17:45.620 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-14T17:17:45.630 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps": 0, 2024-05-14T17:17:45.639 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps_dirty": 0, 2024-05-14T17:17:45.655 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_cap_release": 0, 2024-05-14T17:17:45.678 INFO:teuthology.orchestra.run.smithi151.stdout: "process_request_cap_release": 0 2024-05-14T17:17:45.690 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:45.718 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_cache": { 2024-05-14T17:17:45.728 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update": 0, 2024-05-14T17:17:45.738 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update_receipt": 0, 2024-05-14T17:17:45.772 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_try_discover": 0, 2024-05-14T17:17:45.784 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_send_discover": 0, 2024-05-14T17:17:45.799 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_handle_discover": 0, 2024-05-14T17:17:45.813 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays": 0, 2024-05-14T17:17:45.826 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_delayed": 0, 2024-05-14T17:17:45.838 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_enqueuing": 0, 2024-05-14T17:17:45.856 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_created": 0, 2024-05-14T17:17:45.870 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_enqueued": 0, 2024-05-14T17:17:45.884 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_reintegrated": 0, 2024-05-14T17:17:45.897 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_migrated": 0, 2024-05-14T17:17:45.909 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_processing": 0, 2024-05-14T17:17:45.922 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_enqueued": 0, 2024-05-14T17:17:45.982 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_prioritized": 0, 2024-05-14T17:17:45.994 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_started": 0, 2024-05-14T17:17:46.015 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_completed": 0, 2024-05-14T17:17:46.030 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_path": 0, 2024-05-14T17:17:46.039 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_inode": 0, 2024-05-14T17:17:46.072 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_enqueue_scrub": 0, 2024-05-14T17:17:46.085 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_exportdir": 0, 2024-05-14T17:17:46.098 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_flush": 0, 2024-05-14T17:17:46.107 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragmentdir": 0, 2024-05-14T17:17:46.121 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragstats": 0, 2024-05-14T17:17:46.134 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_inodestats": 0 2024-05-14T17:17:46.151 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:46.162 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_log": { 2024-05-14T17:17:46.176 INFO:teuthology.orchestra.run.smithi151.stdout: "evlrg": 0, 2024-05-14T17:17:46.190 INFO:teuthology.orchestra.run.smithi151.stdout: "evadd": 0, 2024-05-14T17:17:46.201 INFO:teuthology.orchestra.run.smithi151.stdout: "evex": 0, 2024-05-14T17:17:46.213 INFO:teuthology.orchestra.run.smithi151.stdout: "evtrm": 0, 2024-05-14T17:17:46.278 INFO:teuthology.orchestra.run.smithi151.stdout: "ev": 21, 2024-05-14T17:17:46.287 INFO:teuthology.orchestra.run.smithi151.stdout: "evexg": 0, 2024-05-14T17:17:46.299 INFO:teuthology.orchestra.run.smithi151.stdout: "evexd": 0, 2024-05-14T17:17:46.311 INFO:teuthology.orchestra.run.smithi151.stdout: "segadd": 0, 2024-05-14T17:17:46.320 INFO:teuthology.orchestra.run.smithi151.stdout: "segex": 0, 2024-05-14T17:17:46.339 INFO:teuthology.orchestra.run.smithi151.stdout: "segtrm": 0, 2024-05-14T17:17:46.346 INFO:teuthology.orchestra.run.smithi151.stdout: "seg": 2, 2024-05-14T17:17:46.354 INFO:teuthology.orchestra.run.smithi151.stdout: "segmjr": 3, 2024-05-14T17:17:46.364 INFO:teuthology.orchestra.run.smithi151.stdout: "segexg": 0, 2024-05-14T17:17:46.375 INFO:teuthology.orchestra.run.smithi151.stdout: "segexd": 0, 2024-05-14T17:17:46.383 INFO:teuthology.orchestra.run.smithi151.stdout: "expos": 33803422, 2024-05-14T17:17:46.392 INFO:teuthology.orchestra.run.smithi151.stdout: "wrpos": 33876826, 2024-05-14T17:17:46.400 INFO:teuthology.orchestra.run.smithi151.stdout: "rdpos": 33876714, 2024-05-14T17:17:46.409 INFO:teuthology.orchestra.run.smithi151.stdout: "jlat": { 2024-05-14T17:17:46.419 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:46.428 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:46.439 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:46.492 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:46.499 INFO:teuthology.orchestra.run.smithi151.stdout: "replayed": 21 2024-05-14T17:17:46.524 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:46.532 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_mem": { 2024-05-14T17:17:46.539 INFO:teuthology.orchestra.run.smithi151.stdout: "ino": 12, 2024-05-14T17:17:46.556 INFO:teuthology.orchestra.run.smithi151.stdout: "ino+": 12, 2024-05-14T17:17:46.565 INFO:teuthology.orchestra.run.smithi151.stdout: "ino-": 0, 2024-05-14T17:17:46.570 INFO:teuthology.orchestra.run.smithi151.stdout: "dir": 15, 2024-05-14T17:17:46.578 INFO:teuthology.orchestra.run.smithi151.stdout: "dir+": 15, 2024-05-14T17:17:46.586 INFO:teuthology.orchestra.run.smithi151.stdout: "dir-": 0, 2024-05-14T17:17:46.595 INFO:teuthology.orchestra.run.smithi151.stdout: "dn": 9, 2024-05-14T17:17:46.609 INFO:teuthology.orchestra.run.smithi151.stdout: "dn+": 9, 2024-05-14T17:17:46.617 INFO:teuthology.orchestra.run.smithi151.stdout: "dn-": 0, 2024-05-14T17:17:46.625 INFO:teuthology.orchestra.run.smithi151.stdout: "cap": 0, 2024-05-14T17:17:46.631 INFO:teuthology.orchestra.run.smithi151.stdout: "cap+": 0, 2024-05-14T17:17:46.638 INFO:teuthology.orchestra.run.smithi151.stdout: "cap-": 0, 2024-05-14T17:17:46.647 INFO:teuthology.orchestra.run.smithi151.stdout: "rss": 46900, 2024-05-14T17:17:46.714 INFO:teuthology.orchestra.run.smithi151.stdout: "heap": 215416 2024-05-14T17:17:46.720 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:46.755 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_server": { 2024-05-14T17:17:46.762 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_client_request": 0, 2024-05-14T17:17:46.768 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_server_request": 0, 2024-05-14T17:17:46.809 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_request": 0, 2024-05-14T17:17:46.817 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_session": 0, 2024-05-14T17:17:46.824 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_peer_request": 0, 2024-05-14T17:17:46.831 INFO:teuthology.orchestra.run.smithi151.stdout: "req_create_latency": { 2024-05-14T17:17:46.837 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:46.844 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:46.868 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:46.874 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:46.892 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getattr_latency": { 2024-05-14T17:17:46.901 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:46.907 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:46.914 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:46.945 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:46.953 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getfilelock_latency": { 2024-05-14T17:17:46.974 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:46.982 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:46.990 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:47.019 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:47.031 INFO:teuthology.orchestra.run.smithi151.stdout: "req_link_latency": { 2024-05-14T17:17:47.038 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:47.045 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:47.052 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:47.060 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:47.094 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookup_latency": { 2024-05-14T17:17:47.100 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:47.108 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:47.117 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:47.124 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:47.140 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookuphash_latency": { 2024-05-14T17:17:47.148 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:47.174 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:47.184 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:47.192 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:47.200 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupino_latency": { 2024-05-14T17:17:47.230 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:47.240 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:47.249 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:47.257 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:47.266 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupname_latency": { 2024-05-14T17:17:47.277 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:47.293 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:47.301 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:47.308 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:47.315 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupparent_latency": { 2024-05-14T17:17:47.322 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:47.332 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:47.348 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:47.355 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:47.367 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupsnap_latency": { 2024-05-14T17:17:47.375 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:47.383 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:47.415 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:47.424 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:47.433 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lssnap_latency": { 2024-05-14T17:17:47.441 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:47.450 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:47.458 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:47.477 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:47.485 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mkdir_latency": { 2024-05-14T17:17:47.494 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:47.502 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:47.512 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:47.524 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:47.573 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mknod_latency": { 2024-05-14T17:17:47.579 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:47.635 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:47.641 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:47.648 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:47.710 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mksnap_latency": { 2024-05-14T17:17:47.716 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:47.723 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:47.729 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:47.736 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:47.744 INFO:teuthology.orchestra.run.smithi151.stdout: "req_open_latency": { 2024-05-14T17:17:47.769 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:47.776 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:47.781 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:47.787 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:47.794 INFO:teuthology.orchestra.run.smithi151.stdout: "req_readdir_latency": { 2024-05-14T17:17:47.808 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:47.813 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:47.879 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:47.939 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:47.946 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rename_latency": { 2024-05-14T17:17:47.954 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:47.966 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:47.973 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:47.980 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:47.987 INFO:teuthology.orchestra.run.smithi151.stdout: "req_renamesnap_latency": { 2024-05-14T17:17:47.994 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:48.002 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:48.019 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:48.026 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:48.033 INFO:teuthology.orchestra.run.smithi151.stdout: "req_snapdiff_latency": { 2024-05-14T17:17:48.040 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:48.049 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:48.066 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:48.105 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:48.114 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmdir_latency": { 2024-05-14T17:17:48.135 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:48.143 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:48.149 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:48.157 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:48.167 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmsnap_latency": { 2024-05-14T17:17:48.176 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:48.185 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:48.194 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:48.203 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:48.219 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmxattr_latency": { 2024-05-14T17:17:48.228 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:48.237 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:48.246 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:48.255 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:48.266 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setattr_latency": { 2024-05-14T17:17:48.308 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:48.316 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:48.334 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:48.342 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:48.352 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setdirlayout_latency": { 2024-05-14T17:17:48.395 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:48.406 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:48.415 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:48.423 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:48.432 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setfilelock_latency": { 2024-05-14T17:17:48.441 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:48.469 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:48.478 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:48.487 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:48.496 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setlayout_latency": { 2024-05-14T17:17:48.505 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:48.517 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:48.555 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:48.564 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:48.586 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setxattr_latency": { 2024-05-14T17:17:48.594 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:48.600 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:48.607 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:48.615 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:48.622 INFO:teuthology.orchestra.run.smithi151.stdout: "req_symlink_latency": { 2024-05-14T17:17:48.631 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:48.637 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:48.643 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:48.652 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:48.663 INFO:teuthology.orchestra.run.smithi151.stdout: "req_unlink_latency": { 2024-05-14T17:17:48.671 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:48.679 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:48.692 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:48.701 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:48.769 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_revoke_eviction": 0, 2024-05-14T17:17:48.776 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_acquisition_throttle": 0, 2024-05-14T17:17:48.794 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getvxattr_latency": { 2024-05-14T17:17:48.801 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:48.809 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:48.816 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:48.823 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-14T17:17:48.830 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:48.836 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_sessions": { 2024-05-14T17:17:48.844 INFO:teuthology.orchestra.run.smithi151.stdout: "session_count": 2, 2024-05-14T17:17:48.852 INFO:teuthology.orchestra.run.smithi151.stdout: "session_add": 2, 2024-05-14T17:17:48.878 INFO:teuthology.orchestra.run.smithi151.stdout: "session_remove": 0, 2024-05-14T17:17:48.885 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_open": 0, 2024-05-14T17:17:48.891 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_stale": 0, 2024-05-14T17:17:48.899 INFO:teuthology.orchestra.run.smithi151.stdout: "total_load": 0, 2024-05-14T17:17:48.907 INFO:teuthology.orchestra.run.smithi151.stdout: "average_load": 0, 2024-05-14T17:17:48.968 INFO:teuthology.orchestra.run.smithi151.stdout: "avg_session_uptime": 2739, 2024-05-14T17:17:49.058 INFO:teuthology.orchestra.run.smithi151.stdout: "mdthresh_evicted": 0 2024-05-14T17:17:49.065 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:49.100 INFO:teuthology.orchestra.run.smithi151.stdout: "mempool": { 2024-05-14T17:17:49.107 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_bytes": 0, 2024-05-14T17:17:49.114 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_items": 0, 2024-05-14T17:17:49.125 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T17:17:49.131 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_items": 0, 2024-05-14T17:17:49.139 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T17:17:49.149 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_items": 0, 2024-05-14T17:17:49.169 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T17:17:49.176 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T17:17:49.195 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T17:17:49.201 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T17:17:49.209 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T17:17:49.216 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_items": 0, 2024-05-14T17:17:49.224 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T17:17:49.235 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T17:17:49.278 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_bytes": 0, 2024-05-14T17:17:49.284 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_items": 0, 2024-05-14T17:17:49.304 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_bytes": 0, 2024-05-14T17:17:49.312 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_items": 0, 2024-05-14T17:17:49.319 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T17:17:49.328 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T17:17:49.337 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T17:17:49.345 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T17:17:49.352 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T17:17:49.359 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_items": 0, 2024-05-14T17:17:49.365 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_bytes": 0, 2024-05-14T17:17:49.378 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_items": 0, 2024-05-14T17:17:49.385 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T17:17:49.392 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T17:17:49.399 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_bytes": 0, 2024-05-14T17:17:49.406 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_items": 0, 2024-05-14T17:17:49.426 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_bytes": 0, 2024-05-14T17:17:49.467 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_items": 0, 2024-05-14T17:17:49.476 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T17:17:49.497 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_items": 0, 2024-05-14T17:17:49.506 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T17:17:49.515 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_items": 0, 2024-05-14T17:17:49.525 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_bytes": 197740, 2024-05-14T17:17:49.532 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_items": 82, 2024-05-14T17:17:49.541 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_bytes": 0, 2024-05-14T17:17:49.549 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_items": 0, 2024-05-14T17:17:49.557 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_bytes": 0, 2024-05-14T17:17:49.566 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_items": 0, 2024-05-14T17:17:49.578 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_bytes": 0, 2024-05-14T17:17:49.584 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_items": 0, 2024-05-14T17:17:49.591 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_bytes": 0, 2024-05-14T17:17:49.602 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_items": 0, 2024-05-14T17:17:49.610 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_bytes": 8032, 2024-05-14T17:17:49.619 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_items": 167, 2024-05-14T17:17:49.689 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T17:17:49.698 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_items": 0, 2024-05-14T17:17:49.712 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_bytes": 0, 2024-05-14T17:17:49.722 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_items": 0, 2024-05-14T17:17:49.729 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_bytes": 103590, 2024-05-14T17:17:49.750 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_items": 833, 2024-05-14T17:17:49.758 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_bytes": 0, 2024-05-14T17:17:49.767 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_items": 0, 2024-05-14T17:17:49.776 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_bytes": 0, 2024-05-14T17:17:49.784 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_items": 0 2024-05-14T17:17:49.792 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:49.803 INFO:teuthology.orchestra.run.smithi151.stdout: "objecter": { 2024-05-14T17:17:49.812 INFO:teuthology.orchestra.run.smithi151.stdout: "op_active": 0, 2024-05-14T17:17:49.821 INFO:teuthology.orchestra.run.smithi151.stdout: "op_laggy": 0, 2024-05-14T17:17:49.829 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send": 460, 2024-05-14T17:17:49.836 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send_bytes": 26, 2024-05-14T17:17:49.867 INFO:teuthology.orchestra.run.smithi151.stdout: "op_resend": 4, 2024-05-14T17:17:49.902 INFO:teuthology.orchestra.run.smithi151.stdout: "op_reply": 460, 2024-05-14T17:17:49.910 INFO:teuthology.orchestra.run.smithi151.stdout: "op_latency": { 2024-05-14T17:17:49.921 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 460, 2024-05-14T17:17:49.928 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.242996949, 2024-05-14T17:17:49.935 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000528254 2024-05-14T17:17:49.951 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:49.961 INFO:teuthology.orchestra.run.smithi151.stdout: "op_inflight": 0, 2024-05-14T17:17:49.969 INFO:teuthology.orchestra.run.smithi151.stdout: "oplen_avg": { 2024-05-14T17:17:49.979 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 460, 2024-05-14T17:17:49.988 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 461 2024-05-14T17:17:49.997 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:50.004 INFO:teuthology.orchestra.run.smithi151.stdout: "op": 460, 2024-05-14T17:17:50.011 INFO:teuthology.orchestra.run.smithi151.stdout: "op_r": 460, 2024-05-14T17:17:50.018 INFO:teuthology.orchestra.run.smithi151.stdout: "op_w": 0, 2024-05-14T17:17:50.026 INFO:teuthology.orchestra.run.smithi151.stdout: "op_rmw": 0, 2024-05-14T17:17:50.033 INFO:teuthology.orchestra.run.smithi151.stdout: "op_pg": 0, 2024-05-14T17:17:50.042 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_stat": 294, 2024-05-14T17:17:50.095 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_create": 0, 2024-05-14T17:17:50.103 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_read": 163, 2024-05-14T17:17:50.119 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_write": 0, 2024-05-14T17:17:50.126 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writefull": 0, 2024-05-14T17:17:50.133 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writesame": 0, 2024-05-14T17:17:50.143 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_append": 0, 2024-05-14T17:17:50.150 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_zero": 0, 2024-05-14T17:17:50.156 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_truncate": 0, 2024-05-14T17:17:50.164 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_delete": 0, 2024-05-14T17:17:50.170 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_mapext": 0, 2024-05-14T17:17:50.175 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_sparse_read": 0, 2024-05-14T17:17:50.196 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_clonerange": 0, 2024-05-14T17:17:50.201 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_getxattr": 2, 2024-05-14T17:17:50.207 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_setxattr": 0, 2024-05-14T17:17:50.213 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_cmpxattr": 0, 2024-05-14T17:17:50.221 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_rmxattr": 0, 2024-05-14T17:17:50.234 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_resetxattrs": 0, 2024-05-14T17:17:50.254 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_call": 0, 2024-05-14T17:17:50.262 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_watch": 0, 2024-05-14T17:17:50.287 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_notify": 0, 2024-05-14T17:17:50.293 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_src_cmpxattr": 0, 2024-05-14T17:17:50.301 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls": 0, 2024-05-14T17:17:50.309 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls_filter": 0, 2024-05-14T17:17:50.317 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_other": 0, 2024-05-14T17:17:50.324 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_active": 0, 2024-05-14T17:17:50.331 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_send": 0, 2024-05-14T17:17:50.339 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_resend": 0, 2024-05-14T17:17:50.346 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_ping": 0, 2024-05-14T17:17:50.354 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_active": 0, 2024-05-14T17:17:50.361 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_send": 0, 2024-05-14T17:17:50.368 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_resend": 0, 2024-05-14T17:17:50.376 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_active": 0, 2024-05-14T17:17:50.383 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_send": 0, 2024-05-14T17:17:50.393 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_resend": 0, 2024-05-14T17:17:50.420 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_active": 0, 2024-05-14T17:17:50.428 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_send": 0, 2024-05-14T17:17:50.446 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_resend": 0, 2024-05-14T17:17:50.453 INFO:teuthology.orchestra.run.smithi151.stdout: "command_active": 0, 2024-05-14T17:17:50.467 INFO:teuthology.orchestra.run.smithi151.stdout: "command_send": 0, 2024-05-14T17:17:50.491 INFO:teuthology.orchestra.run.smithi151.stdout: "command_resend": 0, 2024-05-14T17:17:50.498 INFO:teuthology.orchestra.run.smithi151.stdout: "map_epoch": 114, 2024-05-14T17:17:50.505 INFO:teuthology.orchestra.run.smithi151.stdout: "map_full": 0, 2024-05-14T17:17:50.512 INFO:teuthology.orchestra.run.smithi151.stdout: "map_inc": 4, 2024-05-14T17:17:50.518 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_sessions": 6, 2024-05-14T17:17:50.525 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_open": 6, 2024-05-14T17:17:50.546 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_close": 0, 2024-05-14T17:17:50.553 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_laggy": 0, 2024-05-14T17:17:50.559 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_wr": 0, 2024-05-14T17:17:50.568 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_rd": 2, 2024-05-14T17:17:50.576 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_del": 0 2024-05-14T17:17:50.587 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:50.596 INFO:teuthology.orchestra.run.smithi151.stdout: "oft": { 2024-05-14T17:17:50.666 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_objs": 0, 2024-05-14T17:17:50.694 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_kv_pairs": 0, 2024-05-14T17:17:50.703 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_updates": 0, 2024-05-14T17:17:50.712 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_removes": 0 2024-05-14T17:17:50.735 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:50.741 INFO:teuthology.orchestra.run.smithi151.stdout: "osd-slow-ops": { 2024-05-14T17:17:50.750 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_ops_count": 0 2024-05-14T17:17:50.774 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:50.784 INFO:teuthology.orchestra.run.smithi151.stdout: "purge_queue": { 2024-05-14T17:17:50.793 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops": 0, 2024-05-14T17:17:50.822 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops_high_water": 0, 2024-05-14T17:17:50.830 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing": 0, 2024-05-14T17:17:50.838 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_high_water": 0, 2024-05-14T17:17:50.848 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed_ops": 0, 2024-05-14T17:17:50.856 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed": 0, 2024-05-14T17:17:50.870 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_item_in_journal": 0 2024-05-14T17:17:50.901 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:50.910 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T17:17:50.963 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-14T17:17:50.971 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-05-14T17:17:50.980 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-14T17:17:51.002 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 518, 2024-05-14T17:17:51.010 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 464332, 2024-05-14T17:17:51.019 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-14T17:17:51.026 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 518, 2024-05-14T17:17:51.033 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-05-14T17:17:51.041 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-05-14T17:17:51.050 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 518, 2024-05-14T17:17:51.058 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 464332, 2024-05-14T17:17:51.066 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-14T17:17:51.074 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:51.081 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:51.096 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:51.155 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-14T17:17:51.164 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:51.200 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_bytes": { 2024-05-14T17:17:51.252 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-14T17:17:51.260 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-05-14T17:17:51.277 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-14T17:17:51.282 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-05-14T17:17:51.289 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-05-14T17:17:51.296 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-14T17:17:51.303 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-05-14T17:17:51.309 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 460, 2024-05-14T17:17:51.317 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 185234, 2024-05-14T17:17:51.322 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 460, 2024-05-14T17:17:51.332 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 185234, 2024-05-14T17:17:51.337 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-14T17:17:51.344 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:51.354 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:51.378 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:51.385 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-14T17:17:51.406 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:51.413 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_ops": { 2024-05-14T17:17:51.419 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-14T17:17:51.435 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1024, 2024-05-14T17:17:51.442 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-14T17:17:51.448 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-05-14T17:17:51.454 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-05-14T17:17:51.460 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-14T17:17:51.467 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-05-14T17:17:51.475 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 460, 2024-05-14T17:17:51.481 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 460, 2024-05-14T17:17:51.488 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 460, 2024-05-14T17:17:51.496 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 460, 2024-05-14T17:17:51.505 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-14T17:17:51.513 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:51.526 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:51.547 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:51.559 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-14T17:17:51.568 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:51.576 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle": { 2024-05-14T17:17:51.594 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-14T17:17:51.602 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-05-14T17:17:51.612 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-14T17:17:51.619 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-05-14T17:17:51.627 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-05-14T17:17:51.634 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-14T17:17:51.651 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-05-14T17:17:51.662 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-05-14T17:17:51.668 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-05-14T17:17:51.677 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 0, 2024-05-14T17:17:51.686 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 0, 2024-05-14T17:17:51.697 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-14T17:17:51.703 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:51.766 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:51.799 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:51.807 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-14T17:17:51.815 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:51.822 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle-0x5562be0d66e0": { 2024-05-14T17:17:51.831 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-14T17:17:51.837 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-05-14T17:17:51.845 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-14T17:17:51.853 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-05-14T17:17:51.861 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-05-14T17:17:51.875 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-14T17:17:51.884 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-05-14T17:17:51.892 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-05-14T17:17:51.900 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-05-14T17:17:51.908 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 0, 2024-05-14T17:17:51.922 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 0, 2024-05-14T17:17:51.941 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-14T17:17:51.951 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:51.974 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:51.982 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:51.990 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-14T17:17:51.998 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-14T17:17:52.006 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-14T17:17:52.015 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:43 smithi151 ceph-mon[36066]: pgmap v730: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:17:52.026 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:43 smithi002 ceph-mon[31538]: pgmap v730: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:17:52.034 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:45 smithi002 ceph-mon[31538]: pgmap v731: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:17:52.043 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:47 smithi002 ceph-mon[31538]: pgmap v732: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:17:52.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:49 smithi002 ceph-mon[31538]: pgmap v733: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-05-14T17:17:52.061 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:43 smithi123 ceph-mon[35971]: pgmap v730: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:17:52.070 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:45 smithi123 ceph-mon[35971]: pgmap v731: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:17:52.079 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:47 smithi123 ceph-mon[35971]: pgmap v732: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:17:52.086 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:49 smithi123 ceph-mon[35971]: pgmap v733: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-05-14T17:17:52.098 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-05-14T17:17:52.129 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-05-14T17:17:52.136 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=7 2024-05-14T17:17:52.157 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-05-14T17:17:52.165 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-05-14T17:17:52.172 DEBUG:tasks.check_counter:Getting stats from l 2024-05-14T17:17:52.184 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-14T17:17:52.191 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:45 smithi151 ceph-mon[36066]: pgmap v731: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:17:52.199 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:47 smithi151 ceph-mon[36066]: pgmap v732: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:17:52.206 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:49 smithi151 ceph-mon[36066]: pgmap v733: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-05-14T17:17:52.213 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:51 smithi151 ceph-mon[36066]: pgmap v734: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T17:17:52.220 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:51 smithi123 ceph-mon[35971]: pgmap v734: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T17:17:52.243 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:51 smithi002 ceph-mon[31538]: pgmap v734: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T17:17:52.470 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.c/config 2024-05-14T17:17:53.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:53 smithi151 ceph-mon[36066]: pgmap v735: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:17:54.157 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-14T17:17:54.164 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-0": { 2024-05-14T17:17:54.172 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 448, 2024-05-14T17:17:54.257 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 435, 2024-05-14T17:17:54.264 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 1777632, 2024-05-14T17:17:54.368 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 268240, 2024-05-14T17:17:54.375 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 4, 2024-05-14T17:17:54.383 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 0, 2024-05-14T17:17:54.437 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.074905007, 2024-05-14T17:17:54.445 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.025545518, 2024-05-14T17:17:54.454 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.046417210, 2024-05-14T17:17:54.465 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.010123444, 2024-05-14T17:17:54.473 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:17:54.483 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 435, 2024-05-14T17:17:54.523 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.023919436, 2024-05-14T17:17:54.531 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000054987 2024-05-14T17:17:54.537 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:54.547 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:17:54.555 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:54.563 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:54.571 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:54.615 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:54.642 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 1777632, 2024-05-14T17:17:54.648 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 268240 2024-05-14T17:17:54.658 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:54.676 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-1": { 2024-05-14T17:17:54.681 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 3, 2024-05-14T17:17:54.688 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 332, 2024-05-14T17:17:54.695 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 480, 2024-05-14T17:17:54.701 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 294640, 2024-05-14T17:17:54.707 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 6, 2024-05-14T17:17:54.728 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 3, 2024-05-14T17:17:54.736 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.019058424, 2024-05-14T17:17:54.742 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.013587321, 2024-05-14T17:17:54.750 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.001757088, 2024-05-14T17:17:54.756 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T17:17:54.772 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:17:54.814 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 329, 2024-05-14T17:17:54.820 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.030981997, 2024-05-14T17:17:54.841 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000094170 2024-05-14T17:17:54.847 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:54.855 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:17:54.867 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:54.876 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:54.882 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:54.889 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:54.894 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-14T17:17:54.901 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 294640 2024-05-14T17:17:54.913 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:54.939 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-2": { 2024-05-14T17:17:54.948 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 0, 2024-05-14T17:17:54.958 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 2, 2024-05-14T17:17:54.966 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 0, 2024-05-14T17:17:54.981 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 0, 2024-05-14T17:17:55.026 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 2, 2024-05-14T17:17:55.037 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 1, 2024-05-14T17:17:55.058 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.000367897, 2024-05-14T17:17:55.070 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.000000000, 2024-05-14T17:17:55.082 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.000282354, 2024-05-14T17:17:55.096 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-14T17:17:55.109 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-05-14T17:17:55.124 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:55.136 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:55.148 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:55.162 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:55.176 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-05-14T17:17:55.188 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:55.200 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:55.213 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:55.222 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:55.239 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-14T17:17:55.298 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 0 2024-05-14T17:17:55.313 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:55.338 INFO:teuthology.orchestra.run.smithi151.stdout: "cct": { 2024-05-14T17:17:55.355 INFO:teuthology.orchestra.run.smithi151.stdout: "total_workers": 0, 2024-05-14T17:17:55.369 INFO:teuthology.orchestra.run.smithi151.stdout: "unhealthy_workers": 0 2024-05-14T17:17:55.383 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:55.396 INFO:teuthology.orchestra.run.smithi151.stdout: "mempool": { 2024-05-14T17:17:55.411 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_bytes": 0, 2024-05-14T17:17:55.429 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_items": 0, 2024-05-14T17:17:55.443 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_bytes": 0, 2024-05-14T17:17:55.457 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_items": 0, 2024-05-14T17:17:55.482 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_bytes": 0, 2024-05-14T17:17:55.498 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_items": 0, 2024-05-14T17:17:55.509 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-14T17:17:55.517 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_items": 0, 2024-05-14T17:17:55.526 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-14T17:17:55.539 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_items": 0, 2024-05-14T17:17:55.601 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_bytes": 0, 2024-05-14T17:17:55.612 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_items": 0, 2024-05-14T17:17:55.638 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-14T17:17:55.646 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_items": 0, 2024-05-14T17:17:55.657 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_bytes": 0, 2024-05-14T17:17:55.668 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_items": 0, 2024-05-14T17:17:55.677 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_bytes": 0, 2024-05-14T17:17:55.687 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_items": 0, 2024-05-14T17:17:55.694 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-14T17:17:55.704 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_items": 0, 2024-05-14T17:17:55.713 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-14T17:17:55.736 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_items": 0, 2024-05-14T17:17:55.744 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_bytes": 0, 2024-05-14T17:17:55.754 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_items": 0, 2024-05-14T17:17:55.762 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_bytes": 0, 2024-05-14T17:17:55.770 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_items": 0, 2024-05-14T17:17:55.789 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-14T17:17:55.869 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_items": 0, 2024-05-14T17:17:55.876 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_bytes": 0, 2024-05-14T17:17:55.900 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_items": 0, 2024-05-14T17:17:55.907 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_bytes": 0, 2024-05-14T17:17:55.915 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_items": 0, 2024-05-14T17:17:55.935 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_bytes": 0, 2024-05-14T17:17:55.943 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_items": 0, 2024-05-14T17:17:55.952 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_bytes": 0, 2024-05-14T17:17:55.962 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_items": 0, 2024-05-14T17:17:55.971 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_bytes": 148714, 2024-05-14T17:17:55.980 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_items": 30, 2024-05-14T17:17:56.020 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_bytes": 0, 2024-05-14T17:17:56.034 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_items": 0, 2024-05-14T17:17:56.050 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_bytes": 0, 2024-05-14T17:17:56.057 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_items": 0, 2024-05-14T17:17:56.070 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_bytes": 0, 2024-05-14T17:17:56.081 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_items": 0, 2024-05-14T17:17:56.153 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_bytes": 0, 2024-05-14T17:17:56.166 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_items": 0, 2024-05-14T17:17:56.204 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_bytes": 0, 2024-05-14T17:17:56.219 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_items": 0, 2024-05-14T17:17:56.233 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_bytes": 0, 2024-05-14T17:17:56.251 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_items": 0, 2024-05-14T17:17:56.265 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_bytes": 0, 2024-05-14T17:17:56.277 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_items": 0, 2024-05-14T17:17:56.288 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_bytes": 928, 2024-05-14T17:17:56.300 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_items": 2, 2024-05-14T17:17:56.312 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_bytes": 0, 2024-05-14T17:17:56.327 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_items": 0, 2024-05-14T17:17:56.343 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_bytes": 0, 2024-05-14T17:17:56.351 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_items": 0 2024-05-14T17:17:56.362 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-14T17:17:56.376 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-14T17:17:56.390 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-14T17:17:56.487 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-05-14T17:17:56.501 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-14T17:17:56.529 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 447, 2024-05-14T17:17:56.542 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 1711951, 2024-05-14T17:17:56.556 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-14T17:17:56.577 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 447, 2024-05-14T17:17:56.585 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-05-14T17:17:56.594 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-05-14T17:17:56.603 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 447, 2024-05-14T17:17:56.612 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 1711951, 2024-05-14T17:17:56.622 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-14T17:17:56.635 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-14T17:17:56.645 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-14T17:17:56.657 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-14T17:17:56.667 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-14T17:17:56.677 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-14T17:17:56.690 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-14T17:17:56.736 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:53 smithi002 ceph-mon[31538]: pgmap v735: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:17:56.748 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:55 smithi002 ceph-mon[31538]: pgmap v736: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:17:56.759 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:53 smithi123 ceph-mon[35971]: pgmap v735: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:17:56.768 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:55 smithi123 ceph-mon[35971]: pgmap v736: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:17:56.779 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-05-14T17:17:56.798 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-05-14T17:17:56.805 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-05-14T17:17:56.814 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-05-14T17:17:56.823 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-05-14T17:17:56.831 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-14T17:17:56.850 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-14T17:17:56.859 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T17:17:56.867 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T17:17:56.876 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:55 smithi151 ceph-mon[36066]: pgmap v736: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-14T17:17:56.908 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T17:17:56.915 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T17:17:56.969 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph osd blocklist ls 2024-05-14T17:17:57.271 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:17:57.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:57 smithi002 ceph-mon[31538]: pgmap v737: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:17:58.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:57 smithi151 ceph-mon[36066]: pgmap v737: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:17:58.159 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:57 smithi123 ceph-mon[35971]: pgmap v737: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:17:59.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.155+0000 7f2a10af7700 1 -- 172.21.15.2:0/1388033599 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a0c06c470 msgr2=0x7f2a0c1103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:59.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.155+0000 7f2a10af7700 1 --2- 172.21.15.2:0/1388033599 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a0c06c470 0x7f2a0c1103c0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f2a04009990 tx=0x7f2a04009ca0 comp rx=0 tx=0).stop 2024-05-14T17:17:59.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.155+0000 7f2a10af7700 1 -- 172.21.15.2:0/1388033599 shutdown_connections 2024-05-14T17:17:59.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.155+0000 7f2a10af7700 1 --2- 172.21.15.2:0/1388033599 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a0c06c470 0x7f2a0c1103c0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:59.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.155+0000 7f2a10af7700 1 --2- 172.21.15.2:0/1388033599 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a0c1095a0 0x7f2a0c109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:59.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.155+0000 7f2a10af7700 1 --2- 172.21.15.2:0/1388033599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a0c06bab0 0x7f2a0c06bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:59.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.155+0000 7f2a10af7700 1 -- 172.21.15.2:0/1388033599 >> 172.21.15.2:0/1388033599 conn(0x7f2a0c0793b0 msgr2=0x7f2a0c0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:17:59.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.155+0000 7f2a10af7700 1 -- 172.21.15.2:0/1388033599 shutdown_connections 2024-05-14T17:17:59.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.155+0000 7f2a10af7700 1 -- 172.21.15.2:0/1388033599 wait complete. 2024-05-14T17:17:59.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.156+0000 7f2a10af7700 1 Processor -- start 2024-05-14T17:17:59.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.156+0000 7f2a10af7700 1 -- start start 2024-05-14T17:17:59.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.156+0000 7f2a10af7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a0c06bab0 0x7f2a0c19f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:59.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.156+0000 7f2a10af7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a0c06c470 0x7f2a0c19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:59.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.156+0000 7f2a10af7700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a0c1095a0 0x7f2a0c1999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:59.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.157+0000 7f2a10af7700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c1136b0 con 0x7f2a0c06bab0 2024-05-14T17:17:59.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.157+0000 7f2a10af7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c113530 con 0x7f2a0c06c470 2024-05-14T17:17:59.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.157+0000 7f2a10af7700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c113830 con 0x7f2a0c1095a0 2024-05-14T17:17:59.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.157+0000 7f2a0a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a0c06bab0 0x7f2a0c19f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:59.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.157+0000 7f2a0a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a0c06bab0 0x7f2a0c19f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46986/0 (socket says 172.21.15.2:46986) 2024-05-14T17:17:59.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.157+0000 7f2a0a59c700 1 -- 172.21.15.2:0/2124889223 learned_addr learned my addr 172.21.15.2:0/2124889223 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:17:59.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.157+0000 7f2a09d9b700 1 --2- 172.21.15.2:0/2124889223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a0c06c470 0x7f2a0c19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:59.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.157+0000 7f2a0ad9d700 1 --2- 172.21.15.2:0/2124889223 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a0c1095a0 0x7f2a0c1999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:59.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.157+0000 7f2a0a59c700 1 -- 172.21.15.2:0/2124889223 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a0c1095a0 msgr2=0x7f2a0c1999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:59.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.157+0000 7f2a0a59c700 1 --2- 172.21.15.2:0/2124889223 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a0c1095a0 0x7f2a0c1999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:59.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.157+0000 7f2a0a59c700 1 -- 172.21.15.2:0/2124889223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a0c06c470 msgr2=0x7f2a0c19fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:17:59.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.157+0000 7f2a0a59c700 1 --2- 172.21.15.2:0/2124889223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a0c06c470 0x7f2a0c19fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:17:59.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.157+0000 7f2a0a59c700 1 -- 172.21.15.2:0/2124889223 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a0c19a130 con 0x7f2a0c06bab0 2024-05-14T17:17:59.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.157+0000 7f2a0ad9d700 1 --2- 172.21.15.2:0/2124889223 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a0c1095a0 0x7f2a0c1999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:17:59.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.157+0000 7f2a0a59c700 1 --2- 172.21.15.2:0/2124889223 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a0c06bab0 0x7f2a0c19f7c0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f29fc007f10 tx=0x7f29fc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:17:59.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.157+0000 7f2a037fe700 1 -- 172.21.15.2:0/2124889223 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc00ddd0 con 0x7f2a0c06bab0 2024-05-14T17:17:59.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.157+0000 7f2a10af7700 1 -- 172.21.15.2:0/2124889223 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a0c19a420 con 0x7f2a0c06bab0 2024-05-14T17:17:59.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.157+0000 7f2a037fe700 1 -- 172.21.15.2:0/2124889223 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29fc004bb0 con 0x7f2a0c06bab0 2024-05-14T17:17:59.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.157+0000 7f2a037fe700 1 -- 172.21.15.2:0/2124889223 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc0046a0 con 0x7f2a0c06bab0 2024-05-14T17:17:59.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.158+0000 7f2a10af7700 1 -- 172.21.15.2:0/2124889223 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a0c1a6690 con 0x7f2a0c06bab0 2024-05-14T17:17:59.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.158+0000 7f2a10af7700 1 -- 172.21.15.2:0/2124889223 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a0c104c60 con 0x7f2a0c06bab0 2024-05-14T17:17:59.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.160+0000 7f2a037fe700 1 -- 172.21.15.2:0/2124889223 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f29fc01c020 con 0x7f2a0c06bab0 2024-05-14T17:17:59.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.161+0000 7f2a037fe700 1 --2- 172.21.15.2:0/2124889223 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f29f80b29f0 0x7f29f80b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:17:59.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.161+0000 7f2a09d9b700 1 --2- 172.21.15.2:0/2124889223 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f29f80b29f0 0x7f29f80b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:17:59.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.161+0000 7f2a037fe700 1 -- 172.21.15.2:0/2124889223 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8513+0+0 (secure 0 0 0) 0x7f29fc0e2940 con 0x7f2a0c06bab0 2024-05-14T17:17:59.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.161+0000 7f2a09d9b700 1 --2- 172.21.15.2:0/2124889223 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f29f80b29f0 0x7f29f80b4eb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c079a20 tx=0x7f29f400b410 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:17:59.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.164+0000 7f2a037fe700 1 -- 172.21.15.2:0/2124889223 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f29fc023030 con 0x7f2a0c06bab0 2024-05-14T17:17:59.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.395+0000 7f2a10af7700 1 -- 172.21.15.2:0/2124889223 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2a0c02e560 con 0x7f2a0c06bab0 2024-05-14T17:17:59.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.395+0000 7f2a037fe700 1 -- 172.21.15.2:0/2124889223 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v114) ==== 81+0+929 (secure 0 0 0) 0x7f29fc010070 con 0x7f2a0c06bab0 2024-05-14T17:17:59.901 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/2907436778 2024-05-15T16:54:41.187591+0000 2024-05-14T17:17:59.911 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/2111644395 2024-05-15T16:54:41.187591+0000 2024-05-14T17:17:59.921 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/1498936028 2024-05-15T16:54:41.187591+0000 2024-05-14T17:17:59.940 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/1498936028 2024-05-15T16:54:41.187591+0000 2024-05-14T17:18:00.018 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/156580230 2024-05-15T16:54:41.187591+0000 2024-05-14T17:18:00.030 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/4267809161 2024-05-15T16:44:11.998214+0000 2024-05-14T17:18:00.050 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/4090012382 2024-05-15T16:54:41.187591+0000 2024-05-14T17:18:00.062 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/3582988344 2024-05-15T16:44:11.998214+0000 2024-05-14T17:18:00.073 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/2574015037 2024-05-15T16:43:43.407184+0000 2024-05-14T17:18:00.085 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/2100732571 2024-05-15T16:44:11.998214+0000 2024-05-14T17:18:00.094 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/2954199324 2024-05-15T16:43:43.407184+0000 2024-05-14T17:18:00.105 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/2574015037 2024-05-15T16:43:43.407184+0000 2024-05-14T17:18:00.115 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/3171550944 2024-05-15T16:43:43.407184+0000 2024-05-14T17:18:00.127 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/1679761136 2024-05-15T16:44:11.998214+0000 2024-05-14T17:18:00.135 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/1746766591 2024-05-15T16:43:43.407184+0000 2024-05-14T17:18:00.144 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/3582988344 2024-05-15T16:44:11.998214+0000 2024-05-14T17:18:00.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.398+0000 7f2a10af7700 1 -- 172.21.15.2:0/2124889223 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f29f80b29f0 msgr2=0x7f29f80b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:00.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.398+0000 7f2a10af7700 1 --2- 172.21.15.2:0/2124889223 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f29f80b29f0 0x7f29f80b4eb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c079a20 tx=0x7f29f400b410 comp rx=0 tx=0).stop 2024-05-14T17:18:00.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.398+0000 7f2a10af7700 1 -- 172.21.15.2:0/2124889223 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a0c06bab0 msgr2=0x7f2a0c19f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:00.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.398+0000 7f2a10af7700 1 --2- 172.21.15.2:0/2124889223 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a0c06bab0 0x7f2a0c19f7c0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f29fc007f10 tx=0x7f29fc00d3b0 comp rx=0 tx=0).stop 2024-05-14T17:18:00.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.398+0000 7f2a10af7700 1 -- 172.21.15.2:0/2124889223 shutdown_connections 2024-05-14T17:18:00.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.398+0000 7f2a10af7700 1 --2- 172.21.15.2:0/2124889223 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f29f80b29f0 0x7f29f80b4eb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:00.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.398+0000 7f2a10af7700 1 --2- 172.21.15.2:0/2124889223 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2a0c1095a0 0x7f2a0c1999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:00.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.398+0000 7f2a10af7700 1 --2- 172.21.15.2:0/2124889223 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a0c06bab0 0x7f2a0c19f7c0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:00.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.398+0000 7f2a10af7700 1 --2- 172.21.15.2:0/2124889223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a0c06c470 0x7f2a0c19fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:00.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.398+0000 7f2a10af7700 1 -- 172.21.15.2:0/2124889223 >> 172.21.15.2:0/2124889223 conn(0x7f2a0c0793b0 msgr2=0x7f2a0c1011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:00.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.398+0000 7f2a10af7700 1 -- 172.21.15.2:0/2124889223 shutdown_connections 2024-05-14T17:18:00.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:17:59.398+0000 7f2a10af7700 1 -- 172.21.15.2:0/2124889223 wait complete. 2024-05-14T17:18:00.322 INFO:teuthology.orchestra.run.smithi002.stderr:listed 16 entries 2024-05-14T17:18:00.331 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:59 smithi002 ceph-mon[31538]: pgmap v738: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-05-14T17:18:00.340 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:17:59 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/2124889223' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-14T17:18:00.351 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-14T17:18:00.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T17:18:00.371 DEBUG:teuthology.orchestra.run.smithi002:> sudo umount /tmp/cephfs 2024-05-14T17:18:00.387 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:59 smithi123 ceph-mon[35971]: pgmap v738: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-05-14T17:18:00.394 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:17:59 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/2124889223' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-14T17:18:00.402 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:59 smithi151 ceph-mon[36066]: pgmap v738: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-05-14T17:18:00.418 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:17:59 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/2124889223' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-14T17:18:00.557 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi002.front.sepia.ceph.com 2024-05-14T17:18:00.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T17:18:00.602 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2024-05-14T17:18:00.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T17:18:00.679 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list 2024-05-14T17:18:00.738 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-ns--tmp-cephfs (id: 0) 2024-05-14T17:18:00.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T17:18:00.765 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns delete ceph-ns--tmp-cephfs 2024-05-14T17:18:00.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T17:18:00.860 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link delete ceph-brx 2024-05-14T17:18:00.975 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-14T17:18:00.986 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-05-14T17:18:00.995 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-14T17:18:01.046 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19615 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/8e512064853a605d9d92c9e3dd4ba1fe8aadd54c11bab4f6d74c6ac0b204273f/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/HXBQE2KERQ2I2BLMSEPPIDKI5W:/var/lib/containers/storage/overlay/l/FC22OMOJNRZ7WUV7PACJKMBDII,upperdir=/var/lib/containers/storage/overlay/8e512064853a605d9d92c9e3dd4ba1fe8aadd54c11bab4f6d74c6ac0b204273f/diff,workdir=/var/lib/containers/storage/overlay/8e512064853a605d9d92c9e3dd4ba1fe8aadd54c11bab4f6d74c6ac0b204273f/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/878b491a7d4cb1d5b93cf168974cdac98e2784cf17bd30aca2d4b02a37383bb8/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/HXBQE2KERQ2I2BLMSEPPIDKI5W:/var/lib/containers/storage/overlay/l/FC22OMOJNRZ7WUV7PACJKMBDII,upperdir=/var/lib/containers/storage/overlay/878b491a7d4cb1d5b93cf168974cdac98e2784cf17bd30aca2d4b02a37383bb8/diff,workdir=/var/lib/containers/storage/overlay/878b491a7d4cb1d5b93cf168974cdac98e2784cf17bd30aca2d4b02a37383bb8/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,inode64 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/25ff6d2179bb51dfcfffab6805bedc028840427212a4ab2d223cde233d97043f/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/HXBQE2KERQ2I2BLMSEPPIDKI5W:/var/lib/containers/storage/overlay/l/FC22OMOJNRZ7WUV7PACJKMBDII,upperdir=/var/lib/containers/storage/overlay/25ff6d2179bb51dfcfffab6805bedc028840427212a4ab2d223cde233d97043f/diff,workdir=/var/lib/containers/storage/overlay/25ff6d2179bb51dfcfffab6805bedc028840427212a4ab2d223cde233d97043f/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/0c008a3bcf152a5c01361295997b8d18f77ff114dbcb03bc9d7ae7b7a56baa47/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/HXBQE2KERQ2I2BLMSEPPIDKI5W:/var/lib/containers/storage/overlay/l/FC22OMOJNRZ7WUV7PACJKMBDII,upperdir=/var/lib/containers/storage/overlay/0c008a3bcf152a5c01361295997b8d18f77ff114dbcb03bc9d7ae7b7a56baa47/diff,workdir=/var/lib/containers/storage/overlay/0c008a3bcf152a5c01361295997b8d18f77ff114dbcb03bc9d7ae7b7a56baa47/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/f0cc6f84f3f08277b36e29e7ff723f60fab6dc23cacceb99343196da7128d5f1/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/HXBQE2KERQ2I2BLMSEPPIDKI5W:/var/lib/containers/storage/overlay/l/FC22OMOJNRZ7WUV7PACJKMBDII,upperdir=/var/lib/containers/storage/overlay/f0cc6f84f3f08277b36e29e7ff723f60fab6dc23cacceb99343196da7128d5f1/diff,workdir=/var/lib/containers/storage/overlay/f0cc6f84f3f08277b36e29e7ff723f60fab6dc23cacceb99343196da7128d5f1/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/82f1f345b41cb54f90d06fd57d1cd3d956a03d5cfaf6f1f7fbb80115555ec1ac/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/HXBQE2KERQ2I2BLMSEPPIDKI5W:/var/lib/containers/storage/overlay/l/FC22OMOJNRZ7WUV7PACJKMBDII,upperdir=/var/lib/containers/storage/overlay/82f1f345b41cb54f90d06fd57d1cd3d956a03d5cfaf6f1f7fbb80115555ec1ac/diff,workdir=/var/lib/containers/storage/overlay/82f1f345b41cb54f90d06fd57d1cd3d956a03d5cfaf6f1f7fbb80115555ec1ac/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/b3c19ddcbd89fa4eb44bcbb03e80ed58a62eb89ec62ac1b92cfde277b950a987/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/HXBQE2KERQ2I2BLMSEPPIDKI5W:/var/lib/containers/storage/overlay/l/FC22OMOJNRZ7WUV7PACJKMBDII,upperdir=/var/lib/containers/storage/overlay/b3c19ddcbd89fa4eb44bcbb03e80ed58a62eb89ec62ac1b92cfde277b950a987/diff,workdir=/var/lib/containers/storage/overlay/b3c19ddcbd89fa4eb44bcbb03e80ed58a62eb89ec62ac1b92cfde277b950a987/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/f74c3d6c0284e7521663d4aad14c3e663fc91bdf0f28dc6e33f42ab48d4df8e3/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/HXBQE2KERQ2I2BLMSEPPIDKI5W:/var/lib/containers/storage/overlay/l/FC22OMOJNRZ7WUV7PACJKMBDII,upperdir=/var/lib/containers/storage/overlay/f74c3d6c0284e7521663d4aad14c3e663fc91bdf0f28dc6e33f42ab48d4df8e3/diff,workdir=/var/lib/containers/storage/overlay/f74c3d6c0284e7521663d4aad14c3e663fc91bdf0f28dc6e33f42ab48d4df8e3/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/5c3cc6c6d20a1841c58d9b4347a7a82fb05ab25ff519adf500cb6fc9a4130293/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/HXBQE2KERQ2I2BLMSEPPIDKI5W:/var/lib/containers/storage/overlay/l/FC22OMOJNRZ7WUV7PACJKMBDII,upperdir=/var/lib/containers/storage/overlay/5c3cc6c6d20a1841c58d9b4347a7a82fb05ab25ff519adf500cb6fc9a4130293/diff,workdir=/var/lib/containers/storage/overlay/5c3cc6c6d20a1841c58d9b4347a7a82fb05ab25ff519adf500cb6fc9a4130293/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/2492a0cf4aa22db2fe04eb1307b015cab1e68ca2140af11a0536e9a6ba189dee/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/HXBQE2KERQ2I2BLMSEPPIDKI5W:/var/lib/containers/storage/overlay/l/FC22OMOJNRZ7WUV7PACJKMBDII,upperdir=/var/lib/containers/storage/overlay/2492a0cf4aa22db2fe04eb1307b015cab1e68ca2140af11a0536e9a6ba189dee/diff,workdir=/var/lib/containers/storage/overlay/2492a0cf4aa22db2fe04eb1307b015cab1e68ca2140af11a0536e9a6ba189dee/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 2024-05-14T17:18:01.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T17:18:01.062 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list 2024-05-14T17:18:01.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-14T17:18:01.147 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link delete ceph-brx 2024-05-14T17:18:01.206 INFO:teuthology.orchestra.run.smithi002.stderr:Cannot find device "ceph-brx" 2024-05-14T17:18:01.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T17:18:01.225 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-14T17:18:01.252 INFO:tasks.cephadm:Teardown begin 2024-05-14T17:18:01.263 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-14T17:18:01.273 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:18:01.557 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:18:01.980 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:01 smithi002 ceph-mon[31538]: pgmap v739: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T17:18:02.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:01 smithi151 ceph-mon[36066]: pgmap v739: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T17:18:02.159 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:01 smithi123 ceph-mon[35971]: pgmap v739: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-14T17:18:03.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.422+0000 7ff1edcbf700 1 -- 172.21.15.2:0/2983713474 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1e81095e0 msgr2=0x7ff1e81099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:03.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.422+0000 7ff1edcbf700 1 --2- 172.21.15.2:0/2983713474 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1e81095e0 0x7ff1e81099c0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7ff1d800b0a0 tx=0x7ff1d800b3b0 comp rx=0 tx=0).stop 2024-05-14T17:18:03.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.422+0000 7ff1edcbf700 1 -- 172.21.15.2:0/2983713474 shutdown_connections 2024-05-14T17:18:03.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.422+0000 7ff1edcbf700 1 --2- 172.21.15.2:0/2983713474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1e806c670 0x7ff1e8110420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:03.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.422+0000 7ff1edcbf700 1 --2- 172.21.15.2:0/2983713474 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1e81095e0 0x7ff1e81099c0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:03.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.422+0000 7ff1edcbf700 1 --2- 172.21.15.2:0/2983713474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff1e806bcb0 0x7ff1e806c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:03.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.422+0000 7ff1edcbf700 1 -- 172.21.15.2:0/2983713474 >> 172.21.15.2:0/2983713474 conn(0x7ff1e80784b0 msgr2=0x7ff1e80788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:03.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.422+0000 7ff1edcbf700 1 -- 172.21.15.2:0/2983713474 shutdown_connections 2024-05-14T17:18:03.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.422+0000 7ff1edcbf700 1 -- 172.21.15.2:0/2983713474 wait complete. 2024-05-14T17:18:03.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.423+0000 7ff1edcbf700 1 Processor -- start 2024-05-14T17:18:03.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.423+0000 7ff1edcbf700 1 -- start start 2024-05-14T17:18:03.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.423+0000 7ff1edcbf700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1e806bcb0 0x7ff1e819f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:03.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.423+0000 7ff1edcbf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1e806c670 0x7ff1e819fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:03.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.423+0000 7ff1edcbf700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff1e81095e0 0x7ff1e8199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:03.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.423+0000 7ff1edcbf700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e8113760 con 0x7ff1e806c670 2024-05-14T17:18:03.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.423+0000 7ff1edcbf700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e81135e0 con 0x7ff1e806bcb0 2024-05-14T17:18:03.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.423+0000 7ff1edcbf700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e81138e0 con 0x7ff1e81095e0 2024-05-14T17:18:03.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.423+0000 7ff1e6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1e806c670 0x7ff1e819fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:03.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.423+0000 7ff1e6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1e806c670 0x7ff1e819fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47006/0 (socket says 172.21.15.2:47006) 2024-05-14T17:18:03.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.423+0000 7ff1e6ffd700 1 -- 172.21.15.2:0/2798984404 learned_addr learned my addr 172.21.15.2:0/2798984404 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:18:03.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.423+0000 7ff1e7fff700 1 --2- 172.21.15.2:0/2798984404 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff1e81095e0 0x7ff1e8199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:03.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.423+0000 7ff1e77fe700 1 --2- 172.21.15.2:0/2798984404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1e806bcb0 0x7ff1e819f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:03.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.424+0000 7ff1e6ffd700 1 -- 172.21.15.2:0/2798984404 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff1e81095e0 msgr2=0x7ff1e8199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:03.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.424+0000 7ff1e6ffd700 1 --2- 172.21.15.2:0/2798984404 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff1e81095e0 0x7ff1e8199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:03.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.424+0000 7ff1e6ffd700 1 -- 172.21.15.2:0/2798984404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1e806bcb0 msgr2=0x7ff1e819f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:03.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.424+0000 7ff1e6ffd700 1 --2- 172.21.15.2:0/2798984404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1e806bcb0 0x7ff1e819f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:03.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.424+0000 7ff1e6ffd700 1 -- 172.21.15.2:0/2798984404 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1e819a1e0 con 0x7ff1e806c670 2024-05-14T17:18:03.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.424+0000 7ff1e7fff700 1 --2- 172.21.15.2:0/2798984404 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff1e81095e0 0x7ff1e8199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:18:03.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.424+0000 7ff1e77fe700 1 --2- 172.21.15.2:0/2798984404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1e806bcb0 0x7ff1e819f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:18:03.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.424+0000 7ff1e6ffd700 1 --2- 172.21.15.2:0/2798984404 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1e806c670 0x7ff1e819fe00 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7ff1d000b530 tx=0x7ff1d000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:03.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.424+0000 7ff1e4ff9700 1 -- 172.21.15.2:0/2798984404 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0015020 con 0x7ff1e806c670 2024-05-14T17:18:03.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.424+0000 7ff1edcbf700 1 -- 172.21.15.2:0/2798984404 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1e819a4d0 con 0x7ff1e806c670 2024-05-14T17:18:03.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.424+0000 7ff1e4ff9700 1 -- 172.21.15.2:0/2798984404 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1d0011ce0 con 0x7ff1e806c670 2024-05-14T17:18:03.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.424+0000 7ff1e4ff9700 1 -- 172.21.15.2:0/2798984404 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0010d20 con 0x7ff1e806c670 2024-05-14T17:18:03.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.424+0000 7ff1edcbf700 1 -- 172.21.15.2:0/2798984404 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1e8102470 con 0x7ff1e806c670 2024-05-14T17:18:03.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.425+0000 7ff1edcbf700 1 -- 172.21.15.2:0/2798984404 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1e810bae0 con 0x7ff1e806c670 2024-05-14T17:18:03.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.426+0000 7ff1e4ff9700 1 -- 172.21.15.2:0/2798984404 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7ff1d0018020 con 0x7ff1e806c670 2024-05-14T17:18:03.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.427+0000 7ff1e4ff9700 1 --2- 172.21.15.2:0/2798984404 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ff1d40b6e00 0x7ff1d40b92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:03.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.427+0000 7ff1e77fe700 1 --2- 172.21.15.2:0/2798984404 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ff1d40b6e00 0x7ff1d40b92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:04.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.427+0000 7ff1e4ff9700 1 -- 172.21.15.2:0/2798984404 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8513+0+0 (secure 0 0 0) 0x7ff1d00e2320 con 0x7ff1e806c670 2024-05-14T17:18:04.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.428+0000 7ff1e77fe700 1 --2- 172.21.15.2:0/2798984404 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ff1d40b6e00 0x7ff1d40b92c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff1d80062a0 tx=0x7ff1d8006230 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:04.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.431+0000 7ff1e4ff9700 1 -- 172.21.15.2:0/2798984404 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff1d001f020 con 0x7ff1e806c670 2024-05-14T17:18:04.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.696+0000 7ff1edcbf700 1 -- 172.21.15.2:0/2798984404 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1e819afb0 con 0x7ff1e806c670 2024-05-14T17:18:04.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.697+0000 7ff1e4ff9700 1 -- 172.21.15.2:0/2798984404 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+11847 (secure 0 0 0) 0x7ff1d00a9db0 con 0x7ff1e806c670 2024-05-14T17:18:04.103 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:18:04.112 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":62,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2275696923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2275696923},{"type":"v1","addr":"172.21.15.2:6835","nonce":2275696923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/716718761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":716718761},{"type":"v1","addr":"172.21.15.2:6839","nonce":716718761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/3644815179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3644815179},{"type":"v1","addr":"172.21.15.2:6841","nonce":3644815179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1880336731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1880336731},{"type":"v1","addr":"172.21.15.151:6833","nonce":1880336731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2415605548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2415605548},{"type":"v1","addr":"172.21.15.2:6837","nonce":2415605548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2635167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2635167},{"type":"v1","addr":"172.21.15.123:6833","nonce":2635167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":62,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:51:34.106919+0000","modified":"2024-05-14T17:17:06.981623+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24455,"mds_2":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24415":{"gid":24415,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/762044339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":762044339},{"type":"v1","addr":"172.21.15.151:6835","nonce":762044339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/132043151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":132043151},{"type":"v1","addr":"172.21.15.151:6837","nonce":132043151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.123:6835/1198670013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1198670013},{"type":"v1","addr":"172.21.15.123:6835","nonce":1198670013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/333426645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":333426645},{"type":"v1","addr":"172.21.15.123:6837","nonce":333426645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/580275877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":580275877},{"type":"v1","addr":"172.21.15.123:6839","nonce":580275877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26800":{"gid":26800,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/239059007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":239059007},{"type":"v1","addr":"172.21.15.151:6839","nonce":239059007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24431,24455]},"id":1}]} 2024-05-14T17:18:04.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.699+0000 7ff1edcbf700 1 -- 172.21.15.2:0/2798984404 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ff1d40b6e00 msgr2=0x7ff1d40b92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:04.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.699+0000 7ff1edcbf700 1 --2- 172.21.15.2:0/2798984404 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ff1d40b6e00 0x7ff1d40b92c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff1d80062a0 tx=0x7ff1d8006230 comp rx=0 tx=0).stop 2024-05-14T17:18:04.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.699+0000 7ff1edcbf700 1 -- 172.21.15.2:0/2798984404 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1e806c670 msgr2=0x7ff1e819fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:04.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.699+0000 7ff1edcbf700 1 --2- 172.21.15.2:0/2798984404 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1e806c670 0x7ff1e819fe00 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7ff1d000b530 tx=0x7ff1d000b8f0 comp rx=0 tx=0).stop 2024-05-14T17:18:04.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.699+0000 7ff1edcbf700 1 -- 172.21.15.2:0/2798984404 shutdown_connections 2024-05-14T17:18:04.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.699+0000 7ff1edcbf700 1 --2- 172.21.15.2:0/2798984404 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7ff1d40b6e00 0x7ff1d40b92c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:04.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.699+0000 7ff1edcbf700 1 --2- 172.21.15.2:0/2798984404 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff1e81095e0 0x7ff1e8199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:04.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.699+0000 7ff1edcbf700 1 --2- 172.21.15.2:0/2798984404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1e806bcb0 0x7ff1e819f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:04.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.699+0000 7ff1edcbf700 1 --2- 172.21.15.2:0/2798984404 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1e806c670 0x7ff1e819fe00 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:04.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.699+0000 7ff1edcbf700 1 -- 172.21.15.2:0/2798984404 >> 172.21.15.2:0/2798984404 conn(0x7ff1e80784b0 msgr2=0x7ff1e81012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:04.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.699+0000 7ff1edcbf700 1 -- 172.21.15.2:0/2798984404 shutdown_connections 2024-05-14T17:18:04.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:03.699+0000 7ff1edcbf700 1 -- 172.21.15.2:0/2798984404 wait complete. 2024-05-14T17:18:04.243 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 62 2024-05-14T17:18:04.260 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:03 smithi002 ceph-mon[31538]: pgmap v740: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:18:04.268 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:03 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/2798984404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:18:04.277 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:03 smithi123 ceph-mon[35971]: pgmap v740: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:18:04.285 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:03 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/2798984404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:18:04.294 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:03 smithi151 ceph-mon[36066]: pgmap v740: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-14T17:18:04.302 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:03 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/2798984404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:18:04.461 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-14T17:18:04.718 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:18:05.063 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:04 smithi002 ceph-mon[31538]: osdmap e115: 12 total, 12 up, 12 in 2024-05-14T17:18:05.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:04 smithi151 ceph-mon[36066]: osdmap e115: 12 total, 12 up, 12 in 2024-05-14T17:18:05.159 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:04 smithi123 ceph-mon[35971]: osdmap e115: 12 total, 12 up, 12 in 2024-05-14T17:18:06.076 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:05 smithi002 ceph-mon[31538]: pgmap v742: 97 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 91 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-05-14T17:18:06.103 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:05 smithi151 ceph-mon[36066]: pgmap v742: 97 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 91 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-05-14T17:18:06.159 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:05 smithi123 ceph-mon[35971]: pgmap v742: 97 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 91 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-05-14T17:18:06.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.646+0000 7fcb2bc06700 1 -- 172.21.15.2:0/1382279879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb24103820 msgr2=0x7fcb24103c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:06.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.646+0000 7fcb2bc06700 1 --2- 172.21.15.2:0/1382279879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb24103820 0x7fcb24103c00 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fcb2000b0d0 tx=0x7fcb2000b3e0 comp rx=0 tx=0).stop 2024-05-14T17:18:06.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.646+0000 7fcb2bc06700 1 -- 172.21.15.2:0/1382279879 shutdown_connections 2024-05-14T17:18:06.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.646+0000 7fcb2bc06700 1 --2- 172.21.15.2:0/1382279879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb24108910 0x7fcb24110490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:06.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.646+0000 7fcb2bc06700 1 --2- 172.21.15.2:0/1382279879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb24103820 0x7fcb24103c00 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:06.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.646+0000 7fcb2bc06700 1 --2- 172.21.15.2:0/1382279879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb241041d0 0x7fcb241081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:06.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.646+0000 7fcb2bc06700 1 -- 172.21.15.2:0/1382279879 >> 172.21.15.2:0/1382279879 conn(0x7fcb2407a6b0 msgr2=0x7fcb2407aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:06.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.646+0000 7fcb2bc06700 1 -- 172.21.15.2:0/1382279879 shutdown_connections 2024-05-14T17:18:06.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.646+0000 7fcb2bc06700 1 -- 172.21.15.2:0/1382279879 wait complete. 2024-05-14T17:18:06.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.647+0000 7fcb2bc06700 1 Processor -- start 2024-05-14T17:18:06.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.647+0000 7fcb2bc06700 1 -- start start 2024-05-14T17:18:06.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.647+0000 7fcb2bc06700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb24103820 0x7fcb24075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:06.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.647+0000 7fcb2bc06700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb241041d0 0x7fcb24075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:06.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.647+0000 7fcb2bc06700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb24108910 0x7fcb240762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:06.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.647+0000 7fcb2bc06700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcb24113820 con 0x7fcb24108910 2024-05-14T17:18:06.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.647+0000 7fcb2bc06700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcb241136a0 con 0x7fcb24103820 2024-05-14T17:18:06.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.647+0000 7fcb2bc06700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcb241139a0 con 0x7fcb241041d0 2024-05-14T17:18:06.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.647+0000 7fcb2a1a3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb24108910 0x7fcb240762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:06.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.647+0000 7fcb2a1a3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb24108910 0x7fcb240762c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35156/0 (socket says 172.21.15.2:35156) 2024-05-14T17:18:06.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.647+0000 7fcb2a1a3700 1 -- 172.21.15.2:0/1625713925 learned_addr learned my addr 172.21.15.2:0/1625713925 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:18:06.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.647+0000 7fcb299a2700 1 --2- 172.21.15.2:0/1625713925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb24103820 0x7fcb24075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:06.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.647+0000 7fcb291a1700 1 --2- 172.21.15.2:0/1625713925 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb241041d0 0x7fcb24075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:06.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.647+0000 7fcb2a1a3700 1 -- 172.21.15.2:0/1625713925 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb241041d0 msgr2=0x7fcb24075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:06.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.647+0000 7fcb2a1a3700 1 --2- 172.21.15.2:0/1625713925 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb241041d0 0x7fcb24075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:07.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.647+0000 7fcb2a1a3700 1 -- 172.21.15.2:0/1625713925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb24103820 msgr2=0x7fcb24075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:07.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.647+0000 7fcb2a1a3700 1 --2- 172.21.15.2:0/1625713925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb24103820 0x7fcb24075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:07.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.647+0000 7fcb2a1a3700 1 -- 172.21.15.2:0/1625713925 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb241a8220 con 0x7fcb24108910 2024-05-14T17:18:07.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.648+0000 7fcb2a1a3700 1 --2- 172.21.15.2:0/1625713925 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb24108910 0x7fcb240762c0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c00b550 tx=0x7fcb1c00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:07.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.648+0000 7fcb1affd700 1 -- 172.21.15.2:0/1625713925 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c015020 con 0x7fcb24108910 2024-05-14T17:18:07.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.648+0000 7fcb1affd700 1 -- 172.21.15.2:0/1625713925 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb1c011ce0 con 0x7fcb24108910 2024-05-14T17:18:07.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.648+0000 7fcb2bc06700 1 -- 172.21.15.2:0/1625713925 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb241a8510 con 0x7fcb24108910 2024-05-14T17:18:07.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.648+0000 7fcb1affd700 1 -- 172.21.15.2:0/1625713925 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c010dd0 con 0x7fcb24108910 2024-05-14T17:18:07.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.648+0000 7fcb2bc06700 1 -- 172.21.15.2:0/1625713925 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb241a8aa0 con 0x7fcb24108910 2024-05-14T17:18:07.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.649+0000 7fcb2bc06700 1 -- 172.21.15.2:0/1625713925 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb2410bb50 con 0x7fcb24108910 2024-05-14T17:18:07.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.650+0000 7fcb1affd700 1 -- 172.21.15.2:0/1625713925 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fcb1c018020 con 0x7fcb24108910 2024-05-14T17:18:07.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.651+0000 7fcb1affd700 1 --2- 172.21.15.2:0/1625713925 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcb100b2a70 0x7fcb100b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:07.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.652+0000 7fcb299a2700 1 --2- 172.21.15.2:0/1625713925 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcb100b2a70 0x7fcb100b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:07.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.652+0000 7fcb1affd700 1 -- 172.21.15.2:0/1625713925 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8557+0+0 (secure 0 0 0) 0x7fcb1c0e2190 con 0x7fcb24108910 2024-05-14T17:18:07.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.652+0000 7fcb299a2700 1 --2- 172.21.15.2:0/1625713925 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcb100b2a70 0x7fcb100b4f30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcb20009fd0 tx=0x7fcb2000b040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:07.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.655+0000 7fcb1affd700 1 -- 172.21.15.2:0/1625713925 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcb1c01f020 con 0x7fcb24108910 2024-05-14T17:18:07.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.923+0000 7fcb2bc06700 1 -- 172.21.15.2:0/1625713925 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb2410b530 con 0x7fcb24108910 2024-05-14T17:18:07.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.924+0000 7fcb1affd700 1 -- 172.21.15.2:0/1625713925 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+11847 (secure 0 0 0) 0x7fcb1c0a9c00 con 0x7fcb24108910 2024-05-14T17:18:07.175 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:18:07.181 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":62,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/2275696923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":2275696923},{"type":"v1","addr":"172.21.15.2:6835","nonce":2275696923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/716718761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":716718761},{"type":"v1","addr":"172.21.15.2:6839","nonce":716718761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/3644815179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":3644815179},{"type":"v1","addr":"172.21.15.2:6841","nonce":3644815179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/1880336731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1880336731},{"type":"v1","addr":"172.21.15.151:6833","nonce":1880336731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2415605548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2415605548},{"type":"v1","addr":"172.21.15.2:6837","nonce":2415605548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/2635167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2635167},{"type":"v1","addr":"172.21.15.123:6833","nonce":2635167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":62,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-14T16:51:34.106919+0000","modified":"2024-05-14T17:17:06.981623+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24479,"mds_1":24455,"mds_2":24431},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24415":{"gid":24415,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6835/762044339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":762044339},{"type":"v1","addr":"172.21.15.151:6835","nonce":762044339}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/132043151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":132043151},{"type":"v1","addr":"172.21.15.151:6837","nonce":132043151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"e","rank":2,"incarnation":20,"state":"up:active","state_seq":14,"addr":"172.21.15.123:6835/1198670013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":1198670013},{"type":"v1","addr":"172.21.15.123:6835","nonce":1198670013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.123:6837/333426645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":333426645},{"type":"v1","addr":"172.21.15.123:6837","nonce":333426645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/580275877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":580275877},{"type":"v1","addr":"172.21.15.123:6839","nonce":580275877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26800":{"gid":26800,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/239059007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":239059007},{"type":"v1","addr":"172.21.15.151:6839","nonce":239059007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24479,24431,24455]},"id":1}]} 2024-05-14T17:18:07.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.926+0000 7fcb2bc06700 1 -- 172.21.15.2:0/1625713925 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcb100b2a70 msgr2=0x7fcb100b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:07.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.926+0000 7fcb2bc06700 1 --2- 172.21.15.2:0/1625713925 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcb100b2a70 0x7fcb100b4f30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcb20009fd0 tx=0x7fcb2000b040 comp rx=0 tx=0).stop 2024-05-14T17:18:07.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.926+0000 7fcb2bc06700 1 -- 172.21.15.2:0/1625713925 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb24108910 msgr2=0x7fcb240762c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:07.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.926+0000 7fcb2bc06700 1 --2- 172.21.15.2:0/1625713925 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb24108910 0x7fcb240762c0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c00b550 tx=0x7fcb1c00b860 comp rx=0 tx=0).stop 2024-05-14T17:18:07.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.926+0000 7fcb2bc06700 1 -- 172.21.15.2:0/1625713925 shutdown_connections 2024-05-14T17:18:07.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.926+0000 7fcb2bc06700 1 --2- 172.21.15.2:0/1625713925 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fcb100b2a70 0x7fcb100b4f30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:07.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.926+0000 7fcb2bc06700 1 --2- 172.21.15.2:0/1625713925 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb24108910 0x7fcb240762c0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:07.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.926+0000 7fcb2bc06700 1 --2- 172.21.15.2:0/1625713925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb24103820 0x7fcb24075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:07.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.926+0000 7fcb2bc06700 1 --2- 172.21.15.2:0/1625713925 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb241041d0 0x7fcb24075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:07.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.926+0000 7fcb2bc06700 1 -- 172.21.15.2:0/1625713925 >> 172.21.15.2:0/1625713925 conn(0x7fcb2407a6b0 msgr2=0x7fcb24101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:07.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.926+0000 7fcb2bc06700 1 -- 172.21.15.2:0/1625713925 shutdown_connections 2024-05-14T17:18:07.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:06.926+0000 7fcb2bc06700 1 -- 172.21.15.2:0/1625713925 wait complete. 2024-05-14T17:18:07.306 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 62 2024-05-14T17:18:07.699 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-14T17:18:07.963 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:18:07.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:07 smithi002 ceph-mon[31538]: pgmap v743: 97 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 91 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-05-14T17:18:08.001 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:07 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/1625713925' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:18:08.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:07 smithi151 ceph-mon[36066]: pgmap v743: 97 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 91 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-05-14T17:18:08.119 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:07 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/1625713925' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:18:08.159 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:07 smithi123 ceph-mon[35971]: pgmap v743: 97 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 91 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-05-14T17:18:08.165 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:07 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/1625713925' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-14T17:18:09.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.702+0000 7f9166b07700 1 -- 172.21.15.2:0/1508875197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9160108520 msgr2=0x7f9160108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:09.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.702+0000 7f9166b07700 1 --2- 172.21.15.2:0/1508875197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9160108520 0x7f9160108900 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f9154009990 tx=0x7f9154009ca0 comp rx=0 tx=0).stop 2024-05-14T17:18:09.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.702+0000 7f9166b07700 1 -- 172.21.15.2:0/1508875197 shutdown_connections 2024-05-14T17:18:09.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.702+0000 7f9166b07700 1 --2- 172.21.15.2:0/1508875197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9160102e00 0x7f916010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:09.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.702+0000 7f9166b07700 1 --2- 172.21.15.2:0/1508875197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9160108520 0x7f9160108900 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:09.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.702+0000 7f9166b07700 1 --2- 172.21.15.2:0/1508875197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9160102440 0x7f91601028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:09.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.702+0000 7f9166b07700 1 -- 172.21.15.2:0/1508875197 >> 172.21.15.2:0/1508875197 conn(0x7f91600793f0 msgr2=0x7f9160079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:09.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.702+0000 7f9166b07700 1 -- 172.21.15.2:0/1508875197 shutdown_connections 2024-05-14T17:18:09.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.702+0000 7f9166b07700 1 -- 172.21.15.2:0/1508875197 wait complete. 2024-05-14T17:18:09.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.703+0000 7f9166b07700 1 Processor -- start 2024-05-14T17:18:09.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.703+0000 7f9166b07700 1 -- start start 2024-05-14T17:18:09.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.704+0000 7f9166b07700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9160102440 0x7f916019e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:09.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.704+0000 7f9166b07700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9160102e00 0x7f916019ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:09.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.704+0000 7f9166b07700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9160108520 0x7f9160198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:10.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.704+0000 7f9166b07700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f91601125f0 con 0x7f9160108520 2024-05-14T17:18:10.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.704+0000 7f9166b07700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9160112470 con 0x7f9160102e00 2024-05-14T17:18:10.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.704+0000 7f9166b07700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9160112770 con 0x7f9160102440 2024-05-14T17:18:10.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.704+0000 7f91650a4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9160108520 0x7f9160198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:10.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.704+0000 7f91650a4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9160108520 0x7f9160198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35186/0 (socket says 172.21.15.2:35186) 2024-05-14T17:18:10.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.704+0000 7f91650a4700 1 -- 172.21.15.2:0/101359953 learned_addr learned my addr 172.21.15.2:0/101359953 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:18:10.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.704+0000 7f915ffff700 1 --2- 172.21.15.2:0/101359953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9160102e00 0x7f916019ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:10.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.704+0000 7f91648a3700 1 --2- 172.21.15.2:0/101359953 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9160102440 0x7f916019e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:10.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.704+0000 7f91650a4700 1 -- 172.21.15.2:0/101359953 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9160102440 msgr2=0x7f916019e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:10.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.704+0000 7f91650a4700 1 --2- 172.21.15.2:0/101359953 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9160102440 0x7f916019e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:10.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.704+0000 7f91650a4700 1 -- 172.21.15.2:0/101359953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9160102e00 msgr2=0x7f916019ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:10.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.704+0000 7f91650a4700 1 --2- 172.21.15.2:0/101359953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9160102e00 0x7f916019ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:10.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.704+0000 7f91650a4700 1 -- 172.21.15.2:0/101359953 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9160199070 con 0x7f9160108520 2024-05-14T17:18:10.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.704+0000 7f91650a4700 1 --2- 172.21.15.2:0/101359953 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9160108520 0x7f9160198910 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f9158007ee0 tx=0x7f915800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:10.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.705+0000 7f915dffb700 1 -- 172.21.15.2:0/101359953 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9158004bb0 con 0x7f9160108520 2024-05-14T17:18:10.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.705+0000 7f9166b07700 1 -- 172.21.15.2:0/101359953 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9160199360 con 0x7f9160108520 2024-05-14T17:18:10.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.705+0000 7f915dffb700 1 -- 172.21.15.2:0/101359953 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9158004d50 con 0x7f9160108520 2024-05-14T17:18:10.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.705+0000 7f915dffb700 1 -- 172.21.15.2:0/101359953 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9158005920 con 0x7f9160108520 2024-05-14T17:18:10.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.705+0000 7f9166b07700 1 -- 172.21.15.2:0/101359953 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f916006b220 con 0x7f9160108520 2024-05-14T17:18:10.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.706+0000 7f9166b07700 1 -- 172.21.15.2:0/101359953 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f916006a230 con 0x7f9160108520 2024-05-14T17:18:10.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.707+0000 7f915dffb700 1 -- 172.21.15.2:0/101359953 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f915801a020 con 0x7f9160108520 2024-05-14T17:18:10.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.708+0000 7f915dffb700 1 --2- 172.21.15.2:0/101359953 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f91500b29f0 0x7f91500b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:10.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.708+0000 7f91648a3700 1 --2- 172.21.15.2:0/101359953 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f91500b29f0 0x7f91500b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:10.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.708+0000 7f915dffb700 1 -- 172.21.15.2:0/101359953 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8557+0+0 (secure 0 0 0) 0x7f91580e1c00 con 0x7f9160108520 2024-05-14T17:18:10.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.709+0000 7f91648a3700 1 --2- 172.21.15.2:0/101359953 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f91500b29f0 0x7f91500b4eb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f915400b5c0 tx=0x7f9154005960 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:10.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.713+0000 7f915dffb700 1 -- 172.21.15.2:0/101359953 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f91580e8080 con 0x7f9160108520 2024-05-14T17:18:10.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.933+0000 7f9166b07700 1 -- 172.21.15.2:0/101359953 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f916006bc50 con 0x7f9160108520 2024-05-14T17:18:10.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.935+0000 7f915dffb700 1 -- 172.21.15.2:0/101359953 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+23108 (secure 0 0 0) 0x7f9158010070 con 0x7f9160108520 2024-05-14T17:18:10.382 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-05-14T17:18:10.390 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":115,"fsid":"ed40717a-1210-11ef-bc98-c7b262605968","created":"2024-05-14T16:43:09.415298+0000","modified":"2024-05-14T17:18:03.774659+0000","last_up_change":"2024-05-14T16:50:29.787654+0000","last_in_change":"2024-05-14T16:50:09.287932+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-14T16:46:38.090903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-14T16:51:27.296955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-14T16:51:31.103019+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":42,"snap_epoch":115,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"153f20ec-5bcf-4786-9ede-2e512a1e3d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6803","nonce":2654237820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6805","nonce":2654237820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6809","nonce":2654237820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2654237820},{"type":"v1","addr":"172.21.15.2:6807","nonce":2654237820}]},"public_addr":"172.21.15.2:6803/2654237820","cluster_addr":"172.21.15.2:6805/2654237820","heartbeat_back_addr":"172.21.15.2:6809/2654237820","heartbeat_front_addr":"172.21.15.2:6807/2654237820","state":["exists","up"]},{"osd":1,"uuid":"0e95419f-9a9d-4f7a-b55d-ecb54971918a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6801","nonce":2668156992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6803","nonce":2668156992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6807","nonce":2668156992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2668156992},{"type":"v1","addr":"172.21.15.123:6805","nonce":2668156992}]},"public_addr":"172.21.15.123:6801/2668156992","cluster_addr":"172.21.15.123:6803/2668156992","heartbeat_back_addr":"172.21.15.123:6807/2668156992","heartbeat_front_addr":"172.21.15.123:6805/2668156992","state":["exists","up"]},{"osd":2,"uuid":"b6ce7ac7-be6c-4549-a356-59cb81b5b492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6801","nonce":2160189636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6803","nonce":2160189636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6807","nonce":2160189636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":2160189636},{"type":"v1","addr":"172.21.15.151:6805","nonce":2160189636}]},"public_addr":"172.21.15.151:6801/2160189636","cluster_addr":"172.21.15.151:6803/2160189636","heartbeat_back_addr":"172.21.15.151:6807/2160189636","heartbeat_front_addr":"172.21.15.151:6805/2160189636","state":["exists","up"]},{"osd":3,"uuid":"2fd592bf-c212-411a-afbe-a7c498e5ac93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6811","nonce":3451424073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6813","nonce":3451424073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6817","nonce":3451424073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":3451424073},{"type":"v1","addr":"172.21.15.2:6815","nonce":3451424073}]},"public_addr":"172.21.15.2:6811/3451424073","cluster_addr":"172.21.15.2:6813/3451424073","heartbeat_back_addr":"172.21.15.2:6817/3451424073","heartbeat_front_addr":"172.21.15.2:6815/3451424073","state":["exists","up"]},{"osd":4,"uuid":"af1905e7-e036-47c6-be59-b085bc41fede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6809","nonce":2101935762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6811","nonce":2101935762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6815","nonce":2101935762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2101935762},{"type":"v1","addr":"172.21.15.123:6813","nonce":2101935762}]},"public_addr":"172.21.15.123:6809/2101935762","cluster_addr":"172.21.15.123:6811/2101935762","heartbeat_back_addr":"172.21.15.123:6815/2101935762","heartbeat_front_addr":"172.21.15.123:6813/2101935762","state":["exists","up"]},{"osd":5,"uuid":"9c166916-af78-4cc2-bd14-d1220b6e7d9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6809","nonce":3695961886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6811","nonce":3695961886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6815","nonce":3695961886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3695961886},{"type":"v1","addr":"172.21.15.151:6813","nonce":3695961886}]},"public_addr":"172.21.15.151:6809/3695961886","cluster_addr":"172.21.15.151:6811/3695961886","heartbeat_back_addr":"172.21.15.151:6815/3695961886","heartbeat_front_addr":"172.21.15.151:6813/3695961886","state":["exists","up"]},{"osd":6,"uuid":"60e9bf66-2338-41e8-9280-398d4e3b322f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6819","nonce":2711893600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6821","nonce":2711893600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6825","nonce":2711893600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2711893600},{"type":"v1","addr":"172.21.15.2:6823","nonce":2711893600}]},"public_addr":"172.21.15.2:6819/2711893600","cluster_addr":"172.21.15.2:6821/2711893600","heartbeat_back_addr":"172.21.15.2:6825/2711893600","heartbeat_front_addr":"172.21.15.2:6823/2711893600","state":["exists","up"]},{"osd":7,"uuid":"c93e07a1-e941-493e-8468-33429df2dff0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6817","nonce":3634155507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6819","nonce":3634155507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6823","nonce":3634155507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3634155507},{"type":"v1","addr":"172.21.15.123:6821","nonce":3634155507}]},"public_addr":"172.21.15.123:6817/3634155507","cluster_addr":"172.21.15.123:6819/3634155507","heartbeat_back_addr":"172.21.15.123:6823/3634155507","heartbeat_front_addr":"172.21.15.123:6821/3634155507","state":["exists","up"]},{"osd":8,"uuid":"1155fb58-d614-4f45-91e2-5da3a93e7d9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6817","nonce":4271107010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6819","nonce":4271107010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6823","nonce":4271107010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":4271107010},{"type":"v1","addr":"172.21.15.151:6821","nonce":4271107010}]},"public_addr":"172.21.15.151:6817/4271107010","cluster_addr":"172.21.15.151:6819/4271107010","heartbeat_back_addr":"172.21.15.151:6823/4271107010","heartbeat_front_addr":"172.21.15.151:6821/4271107010","state":["exists","up"]},{"osd":9,"uuid":"82b34e3b-0017-4ca4-8f86-0993ca78ce7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6827","nonce":3479431021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6829","nonce":3479431021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6833","nonce":3479431021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":3479431021},{"type":"v1","addr":"172.21.15.2:6831","nonce":3479431021}]},"public_addr":"172.21.15.2:6827/3479431021","cluster_addr":"172.21.15.2:6829/3479431021","heartbeat_back_addr":"172.21.15.2:6833/3479431021","heartbeat_front_addr":"172.21.15.2:6831/3479431021","state":["exists","up"]},{"osd":10,"uuid":"32e2ace9-eb5d-4258-a2ac-49b782be2b5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6825","nonce":4266374435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6827","nonce":4266374435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6831","nonce":4266374435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":4266374435},{"type":"v1","addr":"172.21.15.123:6829","nonce":4266374435}]},"public_addr":"172.21.15.123:6825/4266374435","cluster_addr":"172.21.15.123:6827/4266374435","heartbeat_back_addr":"172.21.15.123:6831/4266374435","heartbeat_front_addr":"172.21.15.123:6829/4266374435","state":["exists","up"]},{"osd":11,"uuid":"6e3c2ead-1af6-4bda-87cd-12770c81fa67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6825","nonce":4129194698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6827","nonce":4129194698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6831","nonce":4129194698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":4129194698},{"type":"v1","addr":"172.21.15.151:6829","nonce":4129194698}]},"public_addr":"172.21.15.151:6825/4129194698","cluster_addr":"172.21.15.151:6827/4129194698","heartbeat_back_addr":"172.21.15.151:6831/4129194698","heartbeat_front_addr":"172.21.15.151:6829/4129194698","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:45:46.540023+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:46:10.845186+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:46:34.283112+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:47:00.072467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:47:25.315644+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:47:49.748688+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:48:16.964258+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:48:42.875515+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:49:08.596500+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:49:35.450943+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:50:01.293368+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-14T16:50:27.793713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/2907436778":"2024-05-15T16:54:41.187591+0000","172.21.15.2:0/2111644395":"2024-05-15T16:54:41.187591+0000","172.21.15.2:6801/1498936028":"2024-05-15T16:54:41.187591+0000","172.21.15.2:6800/1498936028":"2024-05-15T16:54:41.187591+0000","172.21.15.2:0/156580230":"2024-05-15T16:54:41.187591+0000","172.21.15.2:0/4267809161":"2024-05-15T16:44:11.998214+0000","172.21.15.2:0/4090012382":"2024-05-15T16:54:41.187591+0000","172.21.15.2:6801/3582988344":"2024-05-15T16:44:11.998214+0000","172.21.15.2:6800/2574015037":"2024-05-15T16:43:43.407184+0000","172.21.15.2:0/2100732571":"2024-05-15T16:44:11.998214+0000","172.21.15.2:0/2954199324":"2024-05-15T16:43:43.407184+0000","172.21.15.2:6801/2574015037":"2024-05-15T16:43:43.407184+0000","172.21.15.2:0/3171550944":"2024-05-15T16:43:43.407184+0000","172.21.15.2:0/1679761136":"2024-05-15T16:44:11.998214+0000","172.21.15.2:0/1746766591":"2024-05-15T16:43:43.407184+0000","172.21.15.2:6800/3582988344":"2024-05-15T16:44:11.998214+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":29,"length":1},{"begin":42,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":29,"length":1},{"begin":42,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-14T17:18:10.399 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:09 smithi123 ceph-mon[35971]: pgmap v744: 97 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 91 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-05-14T17:18:10.407 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:09 smithi151 ceph-mon[36066]: pgmap v744: 97 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 91 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-05-14T17:18:10.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.938+0000 7f9166b07700 1 -- 172.21.15.2:0/101359953 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f91500b29f0 msgr2=0x7f91500b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:10.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.938+0000 7f9166b07700 1 --2- 172.21.15.2:0/101359953 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f91500b29f0 0x7f91500b4eb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f915400b5c0 tx=0x7f9154005960 comp rx=0 tx=0).stop 2024-05-14T17:18:10.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.938+0000 7f9166b07700 1 -- 172.21.15.2:0/101359953 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9160108520 msgr2=0x7f9160198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:10.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.938+0000 7f9166b07700 1 --2- 172.21.15.2:0/101359953 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9160108520 0x7f9160198910 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f9158007ee0 tx=0x7f915800d3b0 comp rx=0 tx=0).stop 2024-05-14T17:18:10.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.938+0000 7f9166b07700 1 -- 172.21.15.2:0/101359953 shutdown_connections 2024-05-14T17:18:10.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.938+0000 7f9166b07700 1 --2- 172.21.15.2:0/101359953 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f91500b29f0 0x7f91500b4eb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:10.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.938+0000 7f9166b07700 1 --2- 172.21.15.2:0/101359953 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9160108520 0x7f9160198910 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:10.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.938+0000 7f9166b07700 1 --2- 172.21.15.2:0/101359953 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9160102440 0x7f916019e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:10.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.938+0000 7f9166b07700 1 --2- 172.21.15.2:0/101359953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9160102e00 0x7f916019ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:10.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.938+0000 7f9166b07700 1 -- 172.21.15.2:0/101359953 >> 172.21.15.2:0/101359953 conn(0x7f91600793f0 msgr2=0x7f9160101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:10.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.938+0000 7f9166b07700 1 -- 172.21.15.2:0/101359953 shutdown_connections 2024-05-14T17:18:10.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:09.938+0000 7f9166b07700 1 -- 172.21.15.2:0/101359953 wait complete. 2024-05-14T17:18:10.644 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:09 smithi002 ceph-mon[31538]: pgmap v744: 97 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 91 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-05-14T17:18:10.796 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs fail cephfs 2024-05-14T17:18:11.033 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:10 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/101359953' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:18:11.053 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:18:11.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:10 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/101359953' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:18:11.159 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:10 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/101359953' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-14T17:18:12.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:11 smithi151 ceph-mon[36066]: pgmap v745: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-05-14T17:18:12.159 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:11 smithi123 ceph-mon[35971]: pgmap v745: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-05-14T17:18:12.184 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:11 smithi002 ceph-mon[31538]: pgmap v745: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-05-14T17:18:12.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.755+0000 7fae11ee5700 1 -- 172.21.15.2:0/274728543 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae0c109630 msgr2=0x7fae0c109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:12.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.755+0000 7fae11ee5700 1 --2- 172.21.15.2:0/274728543 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae0c109630 0x7fae0c109a10 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fadfc0099c0 tx=0x7fadfc009cd0 comp rx=0 tx=0).stop 2024-05-14T17:18:12.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.755+0000 7fae11ee5700 1 -- 172.21.15.2:0/274728543 shutdown_connections 2024-05-14T17:18:12.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.755+0000 7fae11ee5700 1 --2- 172.21.15.2:0/274728543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae0c06c6d0 0x7fae0c110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:12.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.755+0000 7fae11ee5700 1 --2- 172.21.15.2:0/274728543 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae0c109630 0x7fae0c109a10 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:12.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.755+0000 7fae11ee5700 1 --2- 172.21.15.2:0/274728543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae0c06bd10 0x7fae0c06c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:12.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.755+0000 7fae11ee5700 1 -- 172.21.15.2:0/274728543 >> 172.21.15.2:0/274728543 conn(0x7fae0c0794e0 msgr2=0x7fae0c0798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:12.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.755+0000 7fae11ee5700 1 -- 172.21.15.2:0/274728543 shutdown_connections 2024-05-14T17:18:12.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.755+0000 7fae11ee5700 1 -- 172.21.15.2:0/274728543 wait complete. 2024-05-14T17:18:12.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.755+0000 7fae11ee5700 1 Processor -- start 2024-05-14T17:18:12.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.756+0000 7fae11ee5700 1 -- start start 2024-05-14T17:18:12.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.756+0000 7fae11ee5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae0c06bd10 0x7fae0c1024b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:12.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.756+0000 7fae11ee5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae0c06c6d0 0x7fae0c1029f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:13.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.756+0000 7fae11ee5700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae0c109630 0x7fae0c102f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:13.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.756+0000 7fae11ee5700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fae0c113750 con 0x7fae0c06bd10 2024-05-14T17:18:13.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.756+0000 7fae11ee5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fae0c1135d0 con 0x7fae0c06c6d0 2024-05-14T17:18:13.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.756+0000 7fae11ee5700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fae0c1138d0 con 0x7fae0c109630 2024-05-14T17:18:13.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.756+0000 7fae0b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae0c06bd10 0x7fae0c1024b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:13.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.756+0000 7fae0b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae0c06bd10 0x7fae0c1024b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:35206/0 (socket says 172.21.15.2:35206) 2024-05-14T17:18:13.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.756+0000 7fae0b7fe700 1 -- 172.21.15.2:0/964733718 learned_addr learned my addr 172.21.15.2:0/964733718 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:18:13.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.756+0000 7fae0affd700 1 --2- 172.21.15.2:0/964733718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae0c06c6d0 0x7fae0c1029f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:13.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.756+0000 7fae0bfff700 1 --2- 172.21.15.2:0/964733718 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae0c109630 0x7fae0c102f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:13.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.756+0000 7fae0b7fe700 1 -- 172.21.15.2:0/964733718 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae0c109630 msgr2=0x7fae0c102f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:13.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.756+0000 7fae0b7fe700 1 --2- 172.21.15.2:0/964733718 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae0c109630 0x7fae0c102f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:13.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.756+0000 7fae0b7fe700 1 -- 172.21.15.2:0/964733718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae0c06c6d0 msgr2=0x7fae0c1029f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:13.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.757+0000 7fae0b7fe700 1 --2- 172.21.15.2:0/964733718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae0c06c6d0 0x7fae0c1029f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:13.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.757+0000 7fae0b7fe700 1 -- 172.21.15.2:0/964733718 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0c1a8070 con 0x7fae0c06bd10 2024-05-14T17:18:13.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.757+0000 7fae0affd700 1 --2- 172.21.15.2:0/964733718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae0c06c6d0 0x7fae0c1029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-14T17:18:13.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.757+0000 7fae0b7fe700 1 --2- 172.21.15.2:0/964733718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae0c06bd10 0x7fae0c1024b0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fadfc00b580 tx=0x7fadfc0037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:13.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.757+0000 7fae08ff9700 1 -- 172.21.15.2:0/964733718 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc027070 con 0x7fae0c06bd10 2024-05-14T17:18:13.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.757+0000 7fae08ff9700 1 -- 172.21.15.2:0/964733718 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadfc004110 con 0x7fae0c06bd10 2024-05-14T17:18:13.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.757+0000 7fae11ee5700 1 -- 172.21.15.2:0/964733718 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae0c1a8300 con 0x7fae0c06bd10 2024-05-14T17:18:13.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.757+0000 7fae08ff9700 1 -- 172.21.15.2:0/964733718 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc00f460 con 0x7fae0c06bd10 2024-05-14T17:18:13.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.757+0000 7fae11ee5700 1 -- 172.21.15.2:0/964733718 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae0c1a8750 con 0x7fae0c06bd10 2024-05-14T17:18:13.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.758+0000 7fae11ee5700 1 -- 172.21.15.2:0/964733718 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae0c10bac0 con 0x7fae0c06bd10 2024-05-14T17:18:13.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.760+0000 7fae08ff9700 1 -- 172.21.15.2:0/964733718 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fadfc02a070 con 0x7fae0c06bd10 2024-05-14T17:18:13.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.760+0000 7fae08ff9700 1 --2- 172.21.15.2:0/964733718 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fadf40b29f0 0x7fadf40b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:13.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.761+0000 7fae0affd700 1 --2- 172.21.15.2:0/964733718 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fadf40b29f0 0x7fadf40b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:13.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.761+0000 7fae08ff9700 1 -- 172.21.15.2:0/964733718 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8513+0+0 (secure 0 0 0) 0x7fadfc0e3a30 con 0x7fae0c06bd10 2024-05-14T17:18:13.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.761+0000 7fae0affd700 1 --2- 172.21.15.2:0/964733718 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fadf40b29f0 0x7fadf40b4eb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fae00005fd0 tx=0x7fae00005e30 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:13.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:12.764+0000 7fae08ff9700 1 -- 172.21.15.2:0/964733718 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fadfc031070 con 0x7fae0c06bd10 2024-05-14T17:18:13.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:13.027+0000 7fae11ee5700 1 -- 172.21.15.2:0/964733718 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fae0c02e560 con 0x7fae0c06bd10 2024-05-14T17:18:13.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:13.030+0000 7fae08ff9700 1 -- 172.21.15.2:0/964733718 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v63) ==== 161+0+0 (secure 0 0 0) 0x7fadfc013070 con 0x7fae0c06bd10 2024-05-14T17:18:13.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:13.033+0000 7fae11ee5700 1 -- 172.21.15.2:0/964733718 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fadf40b29f0 msgr2=0x7fadf40b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:13.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:13.033+0000 7fae11ee5700 1 --2- 172.21.15.2:0/964733718 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fadf40b29f0 0x7fadf40b4eb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fae00005fd0 tx=0x7fae00005e30 comp rx=0 tx=0).stop 2024-05-14T17:18:13.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:13.033+0000 7fae11ee5700 1 -- 172.21.15.2:0/964733718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae0c06bd10 msgr2=0x7fae0c1024b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:13.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:13.033+0000 7fae11ee5700 1 --2- 172.21.15.2:0/964733718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae0c06bd10 0x7fae0c1024b0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fadfc00b580 tx=0x7fadfc0037c0 comp rx=0 tx=0).stop 2024-05-14T17:18:13.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:13.033+0000 7fae11ee5700 1 -- 172.21.15.2:0/964733718 shutdown_connections 2024-05-14T17:18:13.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:13.033+0000 7fae11ee5700 1 --2- 172.21.15.2:0/964733718 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fadf40b29f0 0x7fadf40b4eb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:13.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:13.033+0000 7fae11ee5700 1 --2- 172.21.15.2:0/964733718 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae0c109630 0x7fae0c102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:13.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:13.033+0000 7fae11ee5700 1 --2- 172.21.15.2:0/964733718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae0c06bd10 0x7fae0c1024b0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:13.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:13.034+0000 7fae11ee5700 1 --2- 172.21.15.2:0/964733718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae0c06c6d0 0x7fae0c1029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:13.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:13.034+0000 7fae11ee5700 1 -- 172.21.15.2:0/964733718 >> 172.21.15.2:0/964733718 conn(0x7fae0c0794e0 msgr2=0x7fae0c100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:13.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:13.034+0000 7fae11ee5700 1 -- 172.21.15.2:0/964733718 shutdown_connections 2024-05-14T17:18:13.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:13.034+0000 7fae11ee5700 1 -- 172.21.15.2:0/964733718 wait complete. 2024-05-14T17:18:13.530 INFO:teuthology.orchestra.run.smithi002.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-14T17:18:13.539 INFO:journalctl@ceph.mds.c.smithi151.stdout:May 14 17:18:13 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-c[66955]: ignoring --setuser ceph since I am not root 2024-05-14T17:18:13.545 INFO:journalctl@ceph.mds.c.smithi151.stdout:May 14 17:18:13 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-c[66955]: ignoring --setgroup ceph since I am not root 2024-05-14T17:18:13.565 INFO:journalctl@ceph.mds.c.smithi151.stdout:May 14 17:18:13 smithi151 ceph-mds[66959]: -- 172.21.15.151:0/2571773984 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5573aabdfa40 con 0x5573aabb2800 2024-05-14T17:18:13.572 INFO:journalctl@ceph.mds.c.smithi151.stdout:May 14 17:18:13 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-c[66955]: starting mds.c at 2024-05-14T17:18:13.578 INFO:journalctl@ceph.mds.e.smithi123.stdout:May 14 17:18:13 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-e[65688]: ignoring --setuser ceph since I am not root 2024-05-14T17:18:13.583 INFO:journalctl@ceph.mds.e.smithi123.stdout:May 14 17:18:13 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-e[65688]: ignoring --setgroup ceph since I am not root 2024-05-14T17:18:13.589 INFO:journalctl@ceph.mds.e.smithi123.stdout:May 14 17:18:13 smithi123 ceph-mds[65692]: -- 172.21.15.123:0/2695908566 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a9c4e79a40 con 0x55a9c4e4c800 2024-05-14T17:18:13.595 INFO:journalctl@ceph.mds.e.smithi123.stdout:May 14 17:18:13 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-e[65688]: starting mds.e at 2024-05-14T17:18:13.601 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:13 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mon-a[31534]: 2024-05-14T17:18:13.028+0000 7f6215069700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-14T17:18:13.606 INFO:journalctl@ceph.mds.f.smithi151.stdout:May 14 17:18:13 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-f[66652]: ignoring --setuser ceph since I am not root 2024-05-14T17:18:13.611 INFO:journalctl@ceph.mds.f.smithi151.stdout:May 14 17:18:13 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-f[66652]: ignoring --setgroup ceph since I am not root 2024-05-14T17:18:13.670 INFO:journalctl@ceph.mds.f.smithi151.stdout:May 14 17:18:13 smithi151 ceph-mds[66656]: -- 172.21.15.151:0/3888307574 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558db5dd1a40 con 0x558db5da4c00 2024-05-14T17:18:13.677 INFO:journalctl@ceph.mds.f.smithi151.stdout:May 14 17:18:13 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-f[66652]: starting mds.f at 2024-05-14T17:18:13.707 INFO:journalctl@ceph.mds.h.smithi123.stdout:May 14 17:18:13 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-h[65997]: ignoring --setuser ceph since I am not root 2024-05-14T17:18:13.712 INFO:journalctl@ceph.mds.h.smithi123.stdout:May 14 17:18:13 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-h[65997]: ignoring --setgroup ceph since I am not root 2024-05-14T17:18:13.719 INFO:journalctl@ceph.mds.h.smithi123.stdout:May 14 17:18:13 smithi123 ceph-mds[66001]: -- 172.21.15.123:0/1767028208 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563b01fbda40 con 0x563b01f90800 2024-05-14T17:18:13.815 INFO:journalctl@ceph.mds.h.smithi123.stdout:May 14 17:18:13 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-h[65997]: starting mds.h at 2024-05-14T17:18:13.821 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:18:13 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[75299]: ignoring --setuser ceph since I am not root 2024-05-14T17:18:13.828 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:18:13 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[75299]: ignoring --setgroup ceph since I am not root 2024-05-14T17:18:13.863 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:18:13 smithi151 ceph-mds[75315]: -- 172.21.15.151:0/998065624 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d406b364e0 con 0x55d406b0d000 2024-05-14T17:18:13.871 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:18:13 smithi151 ceph-mds[75315]: -- 172.21.15.151:0/998065624 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d406b39a40 con 0x55d406b0d000 2024-05-14T17:18:13.879 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:18:13 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[75299]: starting mds.i at 2024-05-14T17:18:13.886 INFO:journalctl@ceph.mds.b.smithi123.stdout:May 14 17:18:13 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-b[66300]: ignoring --setuser ceph since I am not root 2024-05-14T17:18:13.894 INFO:journalctl@ceph.mds.b.smithi123.stdout:May 14 17:18:13 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-b[66300]: ignoring --setgroup ceph since I am not root 2024-05-14T17:18:13.902 INFO:journalctl@ceph.mds.b.smithi123.stdout:May 14 17:18:13 smithi123 ceph-mds[66304]: -- 172.21.15.123:0/503043740 <== mon.0 v2:172.21.15.2:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56482e9b1a40 con 0x56482e984800 2024-05-14T17:18:13.911 INFO:journalctl@ceph.mds.b.smithi123.stdout:May 14 17:18:13 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-b[66300]: starting mds.b at 2024-05-14T17:18:13.920 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:13 smithi151 ceph-mon[36066]: pgmap v746: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-05-14T17:18:13.928 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:13 smithi151 ceph-mon[36066]: osdmap e116: 12 total, 12 up, 12 in 2024-05-14T17:18:13.942 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:13 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/964733718' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-14T17:18:13.949 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:13 smithi151 ceph-mon[36066]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-14T17:18:13.965 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:13 smithi151 ceph-mon[36066]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-14T17:18:13.973 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:13 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/964733718' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-14T17:18:13.982 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:13 smithi151 ceph-mon[36066]: osdmap e117: 12 total, 12 up, 12 in 2024-05-14T17:18:14.039 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:13 smithi151 ceph-mon[36066]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-14T17:18:14.046 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:13 smithi123 ceph-mon[35971]: pgmap v746: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-05-14T17:18:14.052 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:13 smithi123 ceph-mon[35971]: osdmap e116: 12 total, 12 up, 12 in 2024-05-14T17:18:14.071 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:13 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/964733718' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-14T17:18:14.080 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:13 smithi123 ceph-mon[35971]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-14T17:18:14.088 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:13 smithi123 ceph-mon[35971]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-14T17:18:14.096 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:13 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/964733718' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-14T17:18:14.104 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:13 smithi123 ceph-mon[35971]: osdmap e117: 12 total, 12 up, 12 in 2024-05-14T17:18:14.113 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:13 smithi123 ceph-mon[35971]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-14T17:18:14.123 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:13 smithi002 ceph-mon[31538]: pgmap v746: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-05-14T17:18:14.131 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:13 smithi002 ceph-mon[31538]: osdmap e116: 12 total, 12 up, 12 in 2024-05-14T17:18:14.140 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:13 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/964733718' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-14T17:18:14.167 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:13 smithi002 ceph-mon[31538]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-14T17:18:14.176 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:13 smithi002 ceph-mon[31538]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-14T17:18:14.190 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:13 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/964733718' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-14T17:18:14.198 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:13 smithi002 ceph-mon[31538]: osdmap e117: 12 total, 12 up, 12 in 2024-05-14T17:18:14.205 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:13 smithi002 ceph-mon[31538]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-14T17:18:14.256 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-05-14T17:18:14.529 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:18:15.243 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:15 smithi002 ceph-mon[31538]: mds.? [v2:172.21.15.151:6834/2608440407,v1:172.21.15.151:6837/2608440407] up:boot 2024-05-14T17:18:15.251 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:15 smithi002 ceph-mon[31538]: mds.? [v2:172.21.15.151:6836/1896434756,v1:172.21.15.151:6839/1896434756] up:boot 2024-05-14T17:18:15.258 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:15 smithi002 ceph-mon[31538]: mds.? [v2:172.21.15.151:6835/431247846,v1:172.21.15.151:6838/431247846] up:boot 2024-05-14T17:18:15.266 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:15 smithi002 ceph-mon[31538]: mds.? [v2:172.21.15.123:6834/711982632,v1:172.21.15.123:6835/711982632] up:boot 2024-05-14T17:18:15.275 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:15 smithi002 ceph-mon[31538]: mds.? [v2:172.21.15.123:6836/3442982080,v1:172.21.15.123:6837/3442982080] up:boot 2024-05-14T17:18:15.283 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:15 smithi002 ceph-mon[31538]: mds.? [v2:172.21.15.123:6838/3141484924,v1:172.21.15.123:6839/3141484924] up:boot 2024-05-14T17:18:15.294 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:15 smithi002 ceph-mon[31538]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-14T17:18:15.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:15 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T17:18:15.311 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:15 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T17:18:15.320 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:15 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T17:18:15.350 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:15 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T17:18:15.356 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:15 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T17:18:15.367 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:15 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T17:18:15.375 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:15 smithi002 ceph-mon[31538]: pgmap v749: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T17:18:15.384 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:15 smithi151 ceph-mon[36066]: mds.? [v2:172.21.15.151:6834/2608440407,v1:172.21.15.151:6837/2608440407] up:boot 2024-05-14T17:18:15.410 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:15 smithi151 ceph-mon[36066]: mds.? [v2:172.21.15.151:6836/1896434756,v1:172.21.15.151:6839/1896434756] up:boot 2024-05-14T17:18:15.501 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:15 smithi151 ceph-mon[36066]: mds.? [v2:172.21.15.151:6835/431247846,v1:172.21.15.151:6838/431247846] up:boot 2024-05-14T17:18:15.509 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:15 smithi151 ceph-mon[36066]: mds.? [v2:172.21.15.123:6834/711982632,v1:172.21.15.123:6835/711982632] up:boot 2024-05-14T17:18:15.556 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:15 smithi151 ceph-mon[36066]: mds.? [v2:172.21.15.123:6836/3442982080,v1:172.21.15.123:6837/3442982080] up:boot 2024-05-14T17:18:15.576 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:15 smithi151 ceph-mon[36066]: mds.? [v2:172.21.15.123:6838/3141484924,v1:172.21.15.123:6839/3141484924] up:boot 2024-05-14T17:18:15.582 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:15 smithi151 ceph-mon[36066]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-14T17:18:15.590 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:15 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T17:18:15.598 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:15 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T17:18:15.606 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:15 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T17:18:15.614 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:15 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T17:18:15.622 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:15 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T17:18:15.630 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:15 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T17:18:15.643 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:15 smithi151 ceph-mon[36066]: pgmap v749: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T17:18:15.650 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:15 smithi123 ceph-mon[35971]: mds.? [v2:172.21.15.151:6834/2608440407,v1:172.21.15.151:6837/2608440407] up:boot 2024-05-14T17:18:15.657 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:15 smithi123 ceph-mon[35971]: mds.? [v2:172.21.15.151:6836/1896434756,v1:172.21.15.151:6839/1896434756] up:boot 2024-05-14T17:18:15.665 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:15 smithi123 ceph-mon[35971]: mds.? [v2:172.21.15.151:6835/431247846,v1:172.21.15.151:6838/431247846] up:boot 2024-05-14T17:18:15.673 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:15 smithi123 ceph-mon[35971]: mds.? [v2:172.21.15.123:6834/711982632,v1:172.21.15.123:6835/711982632] up:boot 2024-05-14T17:18:15.744 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:15 smithi123 ceph-mon[35971]: mds.? [v2:172.21.15.123:6836/3442982080,v1:172.21.15.123:6837/3442982080] up:boot 2024-05-14T17:18:15.753 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:15 smithi123 ceph-mon[35971]: mds.? [v2:172.21.15.123:6838/3141484924,v1:172.21.15.123:6839/3141484924] up:boot 2024-05-14T17:18:15.760 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:15 smithi123 ceph-mon[35971]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-14T17:18:15.789 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:15 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-14T17:18:15.820 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:15 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-14T17:18:15.827 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:15 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-14T17:18:15.833 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:15 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-14T17:18:15.840 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:15 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-14T17:18:15.846 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:15 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-14T17:18:15.851 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:15 smithi123 ceph-mon[35971]: pgmap v749: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-14T17:18:16.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.293+0000 7f5a87b79700 1 -- 172.21.15.2:0/1195188995 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a801041a0 msgr2=0x7f5a801081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:16.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.293+0000 7f5a87b79700 1 --2- 172.21.15.2:0/1195188995 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a801041a0 0x7f5a801081c0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f5a700099c0 tx=0x7f5a70009cd0 comp rx=0 tx=0).stop 2024-05-14T17:18:16.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.293+0000 7f5a87b79700 1 -- 172.21.15.2:0/1195188995 shutdown_connections 2024-05-14T17:18:16.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.293+0000 7f5a87b79700 1 --2- 172.21.15.2:0/1195188995 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a801088f0 0x7f5a80110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:16.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.293+0000 7f5a87b79700 1 --2- 172.21.15.2:0/1195188995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a801037f0 0x7f5a80103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:16.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.293+0000 7f5a87b79700 1 --2- 172.21.15.2:0/1195188995 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a801041a0 0x7f5a801081c0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:16.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.293+0000 7f5a87b79700 1 -- 172.21.15.2:0/1195188995 >> 172.21.15.2:0/1195188995 conn(0x7f5a8007a6a0 msgr2=0x7f5a8007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:16.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.293+0000 7f5a87b79700 1 -- 172.21.15.2:0/1195188995 shutdown_connections 2024-05-14T17:18:16.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.293+0000 7f5a87b79700 1 -- 172.21.15.2:0/1195188995 wait complete. 2024-05-14T17:18:16.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.294+0000 7f5a87b79700 1 Processor -- start 2024-05-14T17:18:16.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.294+0000 7f5a87b79700 1 -- start start 2024-05-14T17:18:16.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.294+0000 7f5a87b79700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a801037f0 0x7f5a8019bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:16.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.294+0000 7f5a87b79700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a801041a0 0x7f5a8019c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:16.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.294+0000 7f5a87b79700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a801088f0 0x7f5a801a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:16.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.294+0000 7f5a87b79700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5a80113720 con 0x7f5a801041a0 2024-05-14T17:18:16.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.294+0000 7f5a87b79700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5a801135a0 con 0x7f5a801088f0 2024-05-14T17:18:16.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.294+0000 7f5a87b79700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5a801138a0 con 0x7f5a801037f0 2024-05-14T17:18:16.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.294+0000 7f5a85114700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a801041a0 0x7f5a8019c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:16.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.294+0000 7f5a85114700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a801041a0 0x7f5a8019c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38952/0 (socket says 172.21.15.2:38952) 2024-05-14T17:18:16.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.294+0000 7f5a85114700 1 -- 172.21.15.2:0/728586260 learned_addr learned my addr 172.21.15.2:0/728586260 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:18:16.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.294+0000 7f5a85915700 1 --2- 172.21.15.2:0/728586260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a801037f0 0x7f5a8019bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:16.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.294+0000 7f5a86116700 1 --2- 172.21.15.2:0/728586260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a801088f0 0x7f5a801a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:16.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.295+0000 7f5a85114700 1 -- 172.21.15.2:0/728586260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a801037f0 msgr2=0x7f5a8019bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:16.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.295+0000 7f5a85114700 1 --2- 172.21.15.2:0/728586260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a801037f0 0x7f5a8019bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:16.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.295+0000 7f5a85114700 1 -- 172.21.15.2:0/728586260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a801088f0 msgr2=0x7f5a801a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:16.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.295+0000 7f5a85114700 1 --2- 172.21.15.2:0/728586260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a801088f0 0x7f5a801a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:16.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.295+0000 7f5a85114700 1 -- 172.21.15.2:0/728586260 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a801a0de0 con 0x7f5a801041a0 2024-05-14T17:18:16.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.295+0000 7f5a85114700 1 --2- 172.21.15.2:0/728586260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a801041a0 0x7f5a8019c310 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f5a70009990 tx=0x7f5a700047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:16.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.295+0000 7f5a76ffd700 1 -- 172.21.15.2:0/728586260 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a70026070 con 0x7f5a801041a0 2024-05-14T17:18:16.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.295+0000 7f5a76ffd700 1 -- 172.21.15.2:0/728586260 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a70003f20 con 0x7f5a801041a0 2024-05-14T17:18:16.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.295+0000 7f5a87b79700 1 -- 172.21.15.2:0/728586260 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a801a1070 con 0x7f5a801041a0 2024-05-14T17:18:16.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.295+0000 7f5a76ffd700 1 -- 172.21.15.2:0/728586260 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7000fcc0 con 0x7f5a801041a0 2024-05-14T17:18:16.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.295+0000 7f5a87b79700 1 -- 172.21.15.2:0/728586260 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a801a8950 con 0x7f5a801041a0 2024-05-14T17:18:16.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.296+0000 7f5a87b79700 1 -- 172.21.15.2:0/728586260 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a8006a230 con 0x7f5a801041a0 2024-05-14T17:18:16.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.298+0000 7f5a76ffd700 1 -- 172.21.15.2:0/728586260 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f5a70029070 con 0x7f5a801041a0 2024-05-14T17:18:16.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.298+0000 7f5a76ffd700 1 --2- 172.21.15.2:0/728586260 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f5a6c0b2a70 0x7f5a6c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:16.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.298+0000 7f5a85915700 1 --2- 172.21.15.2:0/728586260 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f5a6c0b2a70 0x7f5a6c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:16.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.298+0000 7f5a76ffd700 1 -- 172.21.15.2:0/728586260 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8727+0+0 (secure 0 0 0) 0x7f5a700e3000 con 0x7f5a801041a0 2024-05-14T17:18:16.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.299+0000 7f5a85915700 1 --2- 172.21.15.2:0/728586260 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f5a6c0b2a70 0x7f5a6c0b4f30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c007c60 tx=0x7f5a7c0073d0 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:16.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.303+0000 7f5a76ffd700 1 -- 172.21.15.2:0/728586260 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5a7005edf0 con 0x7f5a801041a0 2024-05-14T17:18:16.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:16.560+0000 7f5a87b79700 1 -- 172.21.15.2:0/728586260 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f5a801a8b20 con 0x7f5a801041a0 2024-05-14T17:18:17.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:17.264+0000 7f5a76ffd700 1 -- 172.21.15.2:0/728586260 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v65) ==== 104+0+0 (secure 0 0 0) 0x7f5a70013070 con 0x7f5a801041a0 2024-05-14T17:18:17.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:17.269+0000 7f5a87b79700 1 -- 172.21.15.2:0/728586260 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f5a6c0b2a70 msgr2=0x7f5a6c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:17.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:17.269+0000 7f5a87b79700 1 --2- 172.21.15.2:0/728586260 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f5a6c0b2a70 0x7f5a6c0b4f30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c007c60 tx=0x7f5a7c0073d0 comp rx=0 tx=0).stop 2024-05-14T17:18:17.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:17.269+0000 7f5a87b79700 1 -- 172.21.15.2:0/728586260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a801041a0 msgr2=0x7f5a8019c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:17.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:17.269+0000 7f5a87b79700 1 --2- 172.21.15.2:0/728586260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a801041a0 0x7f5a8019c310 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f5a70009990 tx=0x7f5a700047b0 comp rx=0 tx=0).stop 2024-05-14T17:18:17.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:17.269+0000 7f5a87b79700 1 -- 172.21.15.2:0/728586260 shutdown_connections 2024-05-14T17:18:17.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:17.269+0000 7f5a87b79700 1 --2- 172.21.15.2:0/728586260 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f5a6c0b2a70 0x7f5a6c0b4f30 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:17.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:17.269+0000 7f5a87b79700 1 --2- 172.21.15.2:0/728586260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a801088f0 0x7f5a801a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:17.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:17.269+0000 7f5a87b79700 1 --2- 172.21.15.2:0/728586260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a801037f0 0x7f5a8019bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:17.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:17.269+0000 7f5a87b79700 1 --2- 172.21.15.2:0/728586260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a801041a0 0x7f5a8019c310 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:17.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:17.269+0000 7f5a87b79700 1 -- 172.21.15.2:0/728586260 >> 172.21.15.2:0/728586260 conn(0x7f5a8007a6a0 msgr2=0x7f5a80109a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:17.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:17.269+0000 7f5a87b79700 1 -- 172.21.15.2:0/728586260 shutdown_connections 2024-05-14T17:18:17.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:17.270+0000 7f5a87b79700 1 -- 172.21.15.2:0/728586260 wait complete. 2024-05-14T17:18:17.600 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:17 smithi002 ceph-mon[31538]: pgmap v750: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 127 B/s wr, 5 op/s 2024-05-14T17:18:17.605 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:17 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/728586260' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-14T17:18:17.615 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:17 smithi151 ceph-mon[36066]: pgmap v750: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 127 B/s wr, 5 op/s 2024-05-14T17:18:17.623 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:17 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/728586260' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-14T17:18:17.703 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:17 smithi123 ceph-mon[35971]: pgmap v750: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 127 B/s wr, 5 op/s 2024-05-14T17:18:17.709 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:17 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/728586260' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-14T17:18:17.944 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-14T17:18:18.210 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:18:18.526 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:18 smithi002 ceph-mon[31538]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-14T17:18:18.534 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:18 smithi002 ceph-mon[31538]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-14T17:18:18.542 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:18 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/728586260' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-14T17:18:18.548 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:18 smithi002 ceph-mon[31538]: fsmap 12 up:standby 2024-05-14T17:18:18.603 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:18 smithi151 ceph-mon[36066]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-14T17:18:18.610 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:18 smithi151 ceph-mon[36066]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-14T17:18:18.618 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:18 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/728586260' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-14T17:18:18.625 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:18 smithi151 ceph-mon[36066]: fsmap 12 up:standby 2024-05-14T17:18:18.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:18 smithi123 ceph-mon[35971]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-14T17:18:18.667 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:18 smithi123 ceph-mon[35971]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-14T17:18:18.676 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:18 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/728586260' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-14T17:18:18.683 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:18 smithi123 ceph-mon[35971]: fsmap 12 up:standby 2024-05-14T17:18:19.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:19 smithi123 ceph-mon[35971]: pgmap v751: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 127 B/s wr, 3 op/s 2024-05-14T17:18:19.685 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:19 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:18:19.692 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:19 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:18:19.697 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:19 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:18:19.790 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:19 smithi123 ceph-mon[35971]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:18:19.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:19 smithi002 ceph-mon[31538]: pgmap v751: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 127 B/s wr, 3 op/s 2024-05-14T17:18:19.808 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:19 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:18:19.815 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:19 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:18:19.823 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:19 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:18:19.832 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:19 smithi002 ceph-mon[31538]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:18:19.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:19 smithi151 ceph-mon[36066]: pgmap v751: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 127 B/s wr, 3 op/s 2024-05-14T17:18:19.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:19 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-14T17:18:19.871 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:19 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-14T17:18:19.903 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:19 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-14T17:18:19.912 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:19 smithi151 ceph-mon[36066]: from='mgr.15120 172.21.15.2:0/3860463314' entity='mgr.x' 2024-05-14T17:18:19.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:19.930+0000 7fb488dfe700 1 -- 172.21.15.2:0/2505877024 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb4841037f0 msgr2=0x7fb484103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:19.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:19.930+0000 7fb488dfe700 1 --2- 172.21.15.2:0/2505877024 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb4841037f0 0x7fb484103bd0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fb4740099c0 tx=0x7fb474009cd0 comp rx=0 tx=0).stop 2024-05-14T17:18:19.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:19.930+0000 7fb488dfe700 1 -- 172.21.15.2:0/2505877024 shutdown_connections 2024-05-14T17:18:19.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:19.930+0000 7fb488dfe700 1 --2- 172.21.15.2:0/2505877024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4841088f0 0x7fb484110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:20.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:19.930+0000 7fb488dfe700 1 --2- 172.21.15.2:0/2505877024 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb4841037f0 0x7fb484103bd0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:20.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:19.930+0000 7fb488dfe700 1 --2- 172.21.15.2:0/2505877024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4841041a0 0x7fb4841081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:20.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:19.930+0000 7fb488dfe700 1 -- 172.21.15.2:0/2505877024 >> 172.21.15.2:0/2505877024 conn(0x7fb48407a6a0 msgr2=0x7fb48407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:20.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:19.930+0000 7fb488dfe700 1 -- 172.21.15.2:0/2505877024 shutdown_connections 2024-05-14T17:18:20.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:19.930+0000 7fb488dfe700 1 -- 172.21.15.2:0/2505877024 wait complete. 2024-05-14T17:18:20.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:19.931+0000 7fb488dfe700 1 Processor -- start 2024-05-14T17:18:20.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:19.931+0000 7fb488dfe700 1 -- start start 2024-05-14T17:18:20.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:19.931+0000 7fb488dfe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4841037f0 0x7fb48419f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:20.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:19.931+0000 7fb488dfe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4841041a0 0x7fb48419fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:20.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:19.931+0000 7fb488dfe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb4841088f0 0x7fb4841999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:20.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:19.931+0000 7fb488dfe700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb4841136e0 con 0x7fb4841088f0 2024-05-14T17:18:20.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:19.931+0000 7fb488dfe700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb484113560 con 0x7fb4841037f0 2024-05-14T17:18:20.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:19.931+0000 7fb488dfe700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb484113860 con 0x7fb4841041a0 2024-05-14T17:18:20.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:19.931+0000 7fb482d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb4841088f0 0x7fb4841999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:20.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:19.931+0000 7fb482d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb4841088f0 0x7fb4841999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38972/0 (socket says 172.21.15.2:38972) 2024-05-14T17:18:20.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:19.931+0000 7fb482d9d700 1 -- 172.21.15.2:0/262132123 learned_addr learned my addr 172.21.15.2:0/262132123 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:18:20.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:19.931+0000 7fb48259c700 1 --2- 172.21.15.2:0/262132123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4841037f0 0x7fb48419f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:20.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:19.931+0000 7fb481d9b700 1 --2- 172.21.15.2:0/262132123 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4841041a0 0x7fb48419fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:20.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:19.932+0000 7fb482d9d700 1 -- 172.21.15.2:0/262132123 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4841041a0 msgr2=0x7fb48419fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:20.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:19.932+0000 7fb482d9d700 1 --2- 172.21.15.2:0/262132123 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4841041a0 0x7fb48419fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:20.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:19.932+0000 7fb482d9d700 1 -- 172.21.15.2:0/262132123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4841037f0 msgr2=0x7fb48419f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:20.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:19.932+0000 7fb482d9d700 1 --2- 172.21.15.2:0/262132123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4841037f0 0x7fb48419f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:20.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:19.932+0000 7fb482d9d700 1 -- 172.21.15.2:0/262132123 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb48419a120 con 0x7fb4841088f0 2024-05-14T17:18:20.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:19.932+0000 7fb482d9d700 1 --2- 172.21.15.2:0/262132123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb4841088f0 0x7fb4841999c0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fb47c007ee0 tx=0x7fb47c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:20.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:19.932+0000 7fb4737fe700 1 -- 172.21.15.2:0/262132123 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c004bb0 con 0x7fb4841088f0 2024-05-14T17:18:20.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:19.932+0000 7fb488dfe700 1 -- 172.21.15.2:0/262132123 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb48419a410 con 0x7fb4841088f0 2024-05-14T17:18:20.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:19.932+0000 7fb4737fe700 1 -- 172.21.15.2:0/262132123 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb47c004d50 con 0x7fb4841088f0 2024-05-14T17:18:20.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:19.932+0000 7fb488dfe700 1 -- 172.21.15.2:0/262132123 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4841a66d0 con 0x7fb4841088f0 2024-05-14T17:18:20.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:19.932+0000 7fb4737fe700 1 -- 172.21.15.2:0/262132123 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c005920 con 0x7fb4841088f0 2024-05-14T17:18:20.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:19.933+0000 7fb488dfe700 1 -- 172.21.15.2:0/262132123 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb48410bb30 con 0x7fb4841088f0 2024-05-14T17:18:20.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:19.934+0000 7fb4737fe700 1 -- 172.21.15.2:0/262132123 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fb47c01a020 con 0x7fb4841088f0 2024-05-14T17:18:20.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:19.935+0000 7fb4737fe700 1 --2- 172.21.15.2:0/262132123 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb46c0b29f0 0x7fb46c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:20.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:19.936+0000 7fb48259c700 1 --2- 172.21.15.2:0/262132123 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb46c0b29f0 0x7fb46c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:20.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:19.936+0000 7fb4737fe700 1 -- 172.21.15.2:0/262132123 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8727+0+0 (secure 0 0 0) 0x7fb47c0e1d90 con 0x7fb4841088f0 2024-05-14T17:18:20.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:19.936+0000 7fb48259c700 1 --2- 172.21.15.2:0/262132123 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb46c0b29f0 0x7fb46c0b4eb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb48419ab00 tx=0x7fb474005c40 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:20.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:19.940+0000 7fb4737fe700 1 -- 172.21.15.2:0/262132123 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb47c010070 con 0x7fb4841088f0 2024-05-14T17:18:20.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:20.154+0000 7fb488dfe700 1 -- 172.21.15.2:0/262132123 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb48406a230 con 0x7fb4841088f0 2024-05-14T17:18:20.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:20.354+0000 7fb4737fe700 1 -- 172.21.15.2:0/262132123 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v118) ==== 181+0+0 (secure 0 0 0) 0x7fb47c0a9640 con 0x7fb4841088f0 2024-05-14T17:18:20.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:20.357+0000 7fb488dfe700 1 -- 172.21.15.2:0/262132123 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb46c0b29f0 msgr2=0x7fb46c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:20.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:20.357+0000 7fb488dfe700 1 --2- 172.21.15.2:0/262132123 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb46c0b29f0 0x7fb46c0b4eb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb48419ab00 tx=0x7fb474005c40 comp rx=0 tx=0).stop 2024-05-14T17:18:20.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:20.357+0000 7fb488dfe700 1 -- 172.21.15.2:0/262132123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb4841088f0 msgr2=0x7fb4841999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:20.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:20.357+0000 7fb488dfe700 1 --2- 172.21.15.2:0/262132123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb4841088f0 0x7fb4841999c0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fb47c007ee0 tx=0x7fb47c00d3b0 comp rx=0 tx=0).stop 2024-05-14T17:18:20.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:20.358+0000 7fb488dfe700 1 -- 172.21.15.2:0/262132123 shutdown_connections 2024-05-14T17:18:20.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:20.358+0000 7fb488dfe700 1 --2- 172.21.15.2:0/262132123 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7fb46c0b29f0 0x7fb46c0b4eb0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:20.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:20.358+0000 7fb488dfe700 1 --2- 172.21.15.2:0/262132123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb4841088f0 0x7fb4841999c0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:20.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:20.358+0000 7fb488dfe700 1 --2- 172.21.15.2:0/262132123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4841037f0 0x7fb48419f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:20.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:20.358+0000 7fb488dfe700 1 --2- 172.21.15.2:0/262132123 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4841041a0 0x7fb48419fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:20.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:20.358+0000 7fb488dfe700 1 -- 172.21.15.2:0/262132123 >> 172.21.15.2:0/262132123 conn(0x7fb48407a6a0 msgr2=0x7fb4841018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:20.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:20.358+0000 7fb488dfe700 1 -- 172.21.15.2:0/262132123 shutdown_connections 2024-05-14T17:18:20.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:20.358+0000 7fb488dfe700 1 -- 172.21.15.2:0/262132123 wait complete. 2024-05-14T17:18:20.609 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_metadata' removed 2024-05-14T17:18:20.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:20 smithi123 ceph-mon[35971]: pgmap v752: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 148 B/s wr, 4 op/s 2024-05-14T17:18:20.666 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:20 smithi123 ceph-mon[35971]: pgmap v753: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 151 B/s wr, 4 op/s 2024-05-14T17:18:20.707 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:20 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/262132123' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-14T17:18:20.713 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:20 smithi002 ceph-mon[31538]: pgmap v752: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 148 B/s wr, 4 op/s 2024-05-14T17:18:20.721 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:20 smithi002 ceph-mon[31538]: pgmap v753: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 151 B/s wr, 4 op/s 2024-05-14T17:18:20.728 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:20 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/262132123' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-14T17:18:20.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:20 smithi151 ceph-mon[36066]: pgmap v752: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 148 B/s wr, 4 op/s 2024-05-14T17:18:20.861 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:20 smithi151 ceph-mon[36066]: pgmap v753: 97 pgs: 97 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 151 B/s wr, 4 op/s 2024-05-14T17:18:20.869 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:20 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/262132123' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-14T17:18:20.998 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid ed40717a-1210-11ef-bc98-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-14T17:18:21.259 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/mon.a/config 2024-05-14T17:18:21.620 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:21 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/262132123' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-14T17:18:21.625 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:21 smithi002 ceph-mon[31538]: osdmap e118: 12 total, 12 up, 12 in 2024-05-14T17:18:21.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:21 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/262132123' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-14T17:18:21.667 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:21 smithi123 ceph-mon[35971]: osdmap e118: 12 total, 12 up, 12 in 2024-05-14T17:18:21.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:21 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/262132123' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-14T17:18:21.860 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:21 smithi151 ceph-mon[36066]: osdmap e118: 12 total, 12 up, 12 in 2024-05-14T17:18:22.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:22 smithi123 ceph-mon[35971]: pgmap v755: 65 pgs: 65 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:18:22.699 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:22 smithi002 ceph-mon[31538]: pgmap v755: 65 pgs: 65 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:18:22.854 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:22 smithi151 ceph-mon[36066]: pgmap v755: 65 pgs: 65 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:18:23.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.007+0000 7f4c4e701700 1 -- 172.21.15.2:0/2793006066 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c4806b220 msgr2=0x7f4c4806b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:23.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.007+0000 7f4c4e701700 1 --2- 172.21.15.2:0/2793006066 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c4806b220 0x7f4c4806b6a0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f4c30009990 tx=0x7f4c30009ca0 comp rx=0 tx=0).stop 2024-05-14T17:18:23.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.007+0000 7f4c4e701700 1 -- 172.21.15.2:0/2793006066 shutdown_connections 2024-05-14T17:18:23.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.007+0000 7f4c4e701700 1 --2- 172.21.15.2:0/2793006066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c4806bbe0 0x7f4c481103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:23.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.007+0000 7f4c4e701700 1 --2- 172.21.15.2:0/2793006066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c481095b0 0x7f4c48109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:23.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.007+0000 7f4c4e701700 1 --2- 172.21.15.2:0/2793006066 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c4806b220 0x7f4c4806b6a0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:23.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.007+0000 7f4c4e701700 1 -- 172.21.15.2:0/2793006066 >> 172.21.15.2:0/2793006066 conn(0x7f4c480793f0 msgr2=0x7f4c48079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:23.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.007+0000 7f4c4e701700 1 -- 172.21.15.2:0/2793006066 shutdown_connections 2024-05-14T17:18:23.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.007+0000 7f4c4e701700 1 -- 172.21.15.2:0/2793006066 wait complete. 2024-05-14T17:18:23.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.008+0000 7f4c4e701700 1 Processor -- start 2024-05-14T17:18:23.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.008+0000 7f4c4e701700 1 -- start start 2024-05-14T17:18:23.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.008+0000 7f4c4e701700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c4806b220 0x7f4c4819f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:23.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.008+0000 7f4c4e701700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c4806bbe0 0x7f4c4819fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:23.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.008+0000 7f4c4e701700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c481095b0 0x7f4c48199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:23.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.008+0000 7f4c4e701700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4c48113740 con 0x7f4c481095b0 2024-05-14T17:18:23.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.008+0000 7f4c4e701700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4c481135c0 con 0x7f4c4806bbe0 2024-05-14T17:18:23.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.008+0000 7f4c4e701700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4c481138c0 con 0x7f4c4806b220 2024-05-14T17:18:23.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.008+0000 7f4c4cc9e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c481095b0 0x7f4c48199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:23.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.009+0000 7f4c4cc9e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c481095b0 0x7f4c48199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38980/0 (socket says 172.21.15.2:38980) 2024-05-14T17:18:23.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.009+0000 7f4c4cc9e700 1 -- 172.21.15.2:0/2424251589 learned_addr learned my addr 172.21.15.2:0/2424251589 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-05-14T17:18:23.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.009+0000 7f4c47fff700 1 --2- 172.21.15.2:0/2424251589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c4806b220 0x7f4c4819f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:23.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.009+0000 7f4c3ffff700 1 --2- 172.21.15.2:0/2424251589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c4806bbe0 0x7f4c4819fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:23.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.009+0000 7f4c4cc9e700 1 -- 172.21.15.2:0/2424251589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c4806b220 msgr2=0x7f4c4819f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:23.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.009+0000 7f4c4cc9e700 1 --2- 172.21.15.2:0/2424251589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c4806b220 0x7f4c4819f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:23.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.009+0000 7f4c4cc9e700 1 -- 172.21.15.2:0/2424251589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c4806bbe0 msgr2=0x7f4c4819fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:23.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.009+0000 7f4c4cc9e700 1 --2- 172.21.15.2:0/2424251589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c4806bbe0 0x7f4c4819fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:23.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.009+0000 7f4c4cc9e700 1 -- 172.21.15.2:0/2424251589 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c4819a200 con 0x7f4c481095b0 2024-05-14T17:18:23.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.009+0000 7f4c4cc9e700 1 --2- 172.21.15.2:0/2424251589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c481095b0 0x7f4c48199aa0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f4c40007ee0 tx=0x7f4c4000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:23.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.009+0000 7f4c45ffb700 1 -- 172.21.15.2:0/2424251589 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4000dbd0 con 0x7f4c481095b0 2024-05-14T17:18:23.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.009+0000 7f4c45ffb700 1 -- 172.21.15.2:0/2424251589 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c4000dd70 con 0x7f4c481095b0 2024-05-14T17:18:23.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.009+0000 7f4c4e701700 1 -- 172.21.15.2:0/2424251589 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c4819a4f0 con 0x7f4c481095b0 2024-05-14T17:18:23.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.009+0000 7f4c45ffb700 1 -- 172.21.15.2:0/2424251589 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c40005990 con 0x7f4c481095b0 2024-05-14T17:18:23.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.009+0000 7f4c4e701700 1 -- 172.21.15.2:0/2424251589 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c481a67b0 con 0x7f4c481095b0 2024-05-14T17:18:23.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.010+0000 7f4c4e701700 1 -- 172.21.15.2:0/2424251589 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c4810ba90 con 0x7f4c481095b0 2024-05-14T17:18:23.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.012+0000 7f4c45ffb700 1 -- 172.21.15.2:0/2424251589 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f4c4001a020 con 0x7f4c481095b0 2024-05-14T17:18:23.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.013+0000 7f4c45ffb700 1 --2- 172.21.15.2:0/2424251589 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f4c340b29f0 0x7f4c340b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-14T17:18:23.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.013+0000 7f4c47fff700 1 --2- 172.21.15.2:0/2424251589 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f4c340b29f0 0x7f4c340b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-14T17:18:23.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.013+0000 7f4c45ffb700 1 -- 172.21.15.2:0/2424251589 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8231+0+0 (secure 0 0 0) 0x7f4c400e2400 con 0x7f4c481095b0 2024-05-14T17:18:23.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.013+0000 7f4c47fff700 1 --2- 172.21.15.2:0/2424251589 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f4c340b29f0 0x7f4c340b4eb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4c38006fd0 tx=0x7f4c38008040 comp rx=0 tx=0).ready entity=mgr.15120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-14T17:18:23.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.016+0000 7f4c45ffb700 1 -- 172.21.15.2:0/2424251589 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4c40010070 con 0x7f4c481095b0 2024-05-14T17:18:23.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.231+0000 7f4c4e701700 1 -- 172.21.15.2:0/2424251589 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4c4819b000 con 0x7f4c481095b0 2024-05-14T17:18:23.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.374+0000 7f4c45ffb700 1 -- 172.21.15.2:0/2424251589 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v119) ==== 169+0+0 (secure 0 0 0) 0x7f4c4819b000 con 0x7f4c481095b0 2024-05-14T17:18:23.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.378+0000 7f4c4e701700 1 -- 172.21.15.2:0/2424251589 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f4c340b29f0 msgr2=0x7f4c340b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:23.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.378+0000 7f4c4e701700 1 --2- 172.21.15.2:0/2424251589 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f4c340b29f0 0x7f4c340b4eb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4c38006fd0 tx=0x7f4c38008040 comp rx=0 tx=0).stop 2024-05-14T17:18:23.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.378+0000 7f4c4e701700 1 -- 172.21.15.2:0/2424251589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c481095b0 msgr2=0x7f4c48199aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-14T17:18:23.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.378+0000 7f4c4e701700 1 --2- 172.21.15.2:0/2424251589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c481095b0 0x7f4c48199aa0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f4c40007ee0 tx=0x7f4c4000d3b0 comp rx=0 tx=0).stop 2024-05-14T17:18:23.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.379+0000 7f4c4e701700 1 -- 172.21.15.2:0/2424251589 shutdown_connections 2024-05-14T17:18:23.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.379+0000 7f4c4e701700 1 --2- 172.21.15.2:0/2424251589 >> [v2:172.21.15.2:6800/1339485805,v1:172.21.15.2:6801/1339485805] conn(0x7f4c340b29f0 0x7f4c340b4eb0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:23.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.379+0000 7f4c4e701700 1 --2- 172.21.15.2:0/2424251589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4c481095b0 0x7f4c48199aa0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:23.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.379+0000 7f4c4e701700 1 --2- 172.21.15.2:0/2424251589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c4806b220 0x7f4c4819f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:23.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.379+0000 7f4c4e701700 1 --2- 172.21.15.2:0/2424251589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c4806bbe0 0x7f4c4819fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-14T17:18:23.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.379+0000 7f4c4e701700 1 -- 172.21.15.2:0/2424251589 >> 172.21.15.2:0/2424251589 conn(0x7f4c480793f0 msgr2=0x7f4c48101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-14T17:18:23.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.379+0000 7f4c4e701700 1 -- 172.21.15.2:0/2424251589 shutdown_connections 2024-05-14T17:18:23.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-05-14T17:18:23.379+0000 7f4c4e701700 1 -- 172.21.15.2:0/2424251589 wait complete. 2024-05-14T17:18:23.713 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_data' removed 2024-05-14T17:18:23.722 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:23 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/2424251589' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-14T17:18:23.732 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:23 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/2424251589' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-14T17:18:23.853 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:23 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/2424251589' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-14T17:18:24.064 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-14T17:18:24.118 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-14T17:18:24.174 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-14T17:18:24.216 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-14T17:18:24.222 DEBUG:teuthology.orchestra.run.smithi002:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-14T17:18:24.259 DEBUG:teuthology.orchestra.run.smithi123:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-14T17:18:24.314 DEBUG:teuthology.orchestra.run.smithi151:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-14T17:18:24.339 INFO:tasks.cephadm:Stopping all daemons... 2024-05-14T17:18:24.349 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-14T17:18:24.389 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-ed40717a-1210-11ef-bc98-c7b262605968@mon.a 2024-05-14T17:18:24.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:24 smithi151 ceph-mon[36066]: pgmap v756: 65 pgs: 65 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:18:24.611 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:24 smithi151 ceph-mon[36066]: from='client.? 172.21.15.2:0/2424251589' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-14T17:18:24.617 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:24 smithi151 ceph-mon[36066]: osdmap e119: 12 total, 12 up, 12 in 2024-05-14T17:18:24.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:24 smithi123 ceph-mon[35971]: pgmap v756: 65 pgs: 65 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:18:24.665 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:24 smithi123 ceph-mon[35971]: from='client.? 172.21.15.2:0/2424251589' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-14T17:18:24.671 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:24 smithi123 ceph-mon[35971]: osdmap e119: 12 total, 12 up, 12 in 2024-05-14T17:18:24.713 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:24 smithi002 ceph-mon[31538]: pgmap v756: 65 pgs: 65 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-14T17:18:24.720 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:24 smithi002 ceph-mon[31538]: from='client.? 172.21.15.2:0/2424251589' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-14T17:18:24.763 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:24 smithi002 ceph-mon[31538]: osdmap e119: 12 total, 12 up, 12 in 2024-05-14T17:18:24.769 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:24 smithi002 systemd[1]: Stopping Ceph mon.a for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T17:18:24.993 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:24 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mon-a[31534]: 2024-05-14T17:18:24.713+0000 7f621a874700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:18:25.002 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:24 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mon-a[31534]: 2024-05-14T17:18:24.713+0000 7f621a874700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-14T17:18:25.010 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:24 smithi002 podman[108955]: 2024-05-14 17:18:24.867728523 +0000 UTC m=+0.274789054 container died c3b1d2255195fe582cbf3457a4fd31d39a25416c7ba12683e39026624e273831 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-14T17:18:25.431 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:25 smithi002 podman[108955]: 2024-05-14 17:18:25.230773362 +0000 UTC m=+0.637833892 container cleanup c3b1d2255195fe582cbf3457a4fd31d39a25416c7ba12683e39026624e273831 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux ) 2024-05-14T17:18:25.439 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:25 smithi002 bash[108955]: ceph-ed40717a-1210-11ef-bc98-c7b262605968-mon-a 2024-05-14T17:18:25.668 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@mon.a.service' 2024-05-14T17:18:25.697 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:25 smithi002 podman[108967]: 2024-05-14 17:18:25.432201296 +0000 UTC m=+0.562237517 container remove c3b1d2255195fe582cbf3457a4fd31d39a25416c7ba12683e39026624e273831 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T17:18:25.705 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:25 smithi002 systemd[1]: ceph-ed40717a-1210-11ef-bc98-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-14T17:18:25.713 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:25 smithi002 systemd[1]: Stopped Ceph mon.a for ed40717a-1210-11ef-bc98-c7b262605968. 2024-05-14T17:18:25.720 INFO:journalctl@ceph.mon.a.smithi002.stdout:May 14 17:18:25 smithi002 systemd[1]: ceph-ed40717a-1210-11ef-bc98-c7b262605968@mon.a.service: Consumed 29.724s CPU time. 2024-05-14T17:18:25.729 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:18:25.736 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-14T17:18:25.743 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-14T17:18:25.749 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-ed40717a-1210-11ef-bc98-c7b262605968@mon.b 2024-05-14T17:18:26.159 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:25 smithi123 systemd[1]: Stopping Ceph mon.b for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T17:18:26.659 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:26 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mon-b[35967]: 2024-05-14T17:18:26.209+0000 7facbca00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:18:26.670 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:26 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mon-b[35967]: 2024-05-14T17:18:26.209+0000 7facbca00700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-14T17:18:26.677 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:26 smithi123 podman[74277]: 2024-05-14 17:18:26.261383442 +0000 UTC m=+0.303246289 container died b249d40e5d46299738ac5008d309560d70d0f69cb5fc2f019dc113dc50fed313 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mon-b, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T17:18:27.119 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:26 smithi123 podman[74277]: 2024-05-14 17:18:26.84391474 +0000 UTC m=+0.885777586 container cleanup b249d40e5d46299738ac5008d309560d70d0f69cb5fc2f019dc113dc50fed313 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mon-b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-14T17:18:27.128 INFO:journalctl@ceph.mon.b.smithi123.stdout:May 14 17:18:26 smithi123 bash[74277]: ceph-ed40717a-1210-11ef-bc98-c7b262605968-mon-b 2024-05-14T17:18:27.274 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@mon.b.service' 2024-05-14T17:18:27.323 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:18:27.332 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-14T17:18:27.372 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-14T17:18:27.378 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-ed40717a-1210-11ef-bc98-c7b262605968@mon.c 2024-05-14T17:18:27.726 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:27 smithi151 systemd[1]: Stopping Ceph mon.c for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T17:18:28.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:27 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mon-c[36049]: 2024-05-14T17:18:27.726+0000 7f1111008700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:18:28.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:27 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mon-c[36049]: 2024-05-14T17:18:27.726+0000 7f1111008700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-14T17:18:28.119 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:27 smithi151 podman[76537]: 2024-05-14 17:18:27.951338982 +0000 UTC m=+0.370953995 container died 3e252dc6012ff9fc02684b7cf79c0f48f1303086586c42b46f43fcc7512206e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mon-c, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514) 2024-05-14T17:18:28.604 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:28 smithi151 podman[76537]: 2024-05-14 17:18:28.362197395 +0000 UTC m=+0.781812400 container cleanup 3e252dc6012ff9fc02684b7cf79c0f48f1303086586c42b46f43fcc7512206e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS) 2024-05-14T17:18:28.610 INFO:journalctl@ceph.mon.c.smithi151.stdout:May 14 17:18:28 smithi151 bash[76537]: ceph-ed40717a-1210-11ef-bc98-c7b262605968-mon-c 2024-05-14T17:18:28.751 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@mon.c.service' 2024-05-14T17:18:28.802 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:18:28.816 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-14T17:18:28.823 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-14T17:18:28.829 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-ed40717a-1210-11ef-bc98-c7b262605968@mgr.x 2024-05-14T17:18:29.168 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 17:18:28 smithi002 systemd[1]: Stopping Ceph mgr.x for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T17:18:29.493 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 17:18:29 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x[31764]: 2024-05-14T17:18:29.167+0000 7f67da946700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:18:29.500 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 17:18:29 smithi002 podman[109075]: 2024-05-14 17:18:29.19953875 +0000 UTC m=+0.152721340 container died 619f0e34026d890ee14dabf4892c5efe136ee50a795eb3558b126cdb3d4caa0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:18:30.036 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 17:18:29 smithi002 podman[109075]: 2024-05-14 17:18:29.844246128 +0000 UTC m=+0.797428716 container cleanup 619f0e34026d890ee14dabf4892c5efe136ee50a795eb3558b126cdb3d4caa0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-14T17:18:30.048 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 17:18:29 smithi002 bash[109075]: ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x 2024-05-14T17:18:30.287 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@mgr.x.service' 2024-05-14T17:18:30.316 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 17:18:30 smithi002 podman[109087]: 2024-05-14 17:18:30.037795888 +0000 UTC m=+0.834362698 container remove 619f0e34026d890ee14dabf4892c5efe136ee50a795eb3558b126cdb3d4caa0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240514, io.buildah.version=1.33.5) 2024-05-14T17:18:30.321 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 17:18:30 smithi002 systemd[1]: ceph-ed40717a-1210-11ef-bc98-c7b262605968@mgr.x.service: Deactivated successfully. 2024-05-14T17:18:30.327 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 17:18:30 smithi002 systemd[1]: Stopped Ceph mgr.x for ed40717a-1210-11ef-bc98-c7b262605968. 2024-05-14T17:18:30.334 INFO:journalctl@ceph.mgr.x.smithi002.stdout:May 14 17:18:30 smithi002 systemd[1]: ceph-ed40717a-1210-11ef-bc98-c7b262605968@mgr.x.service: Consumed 1min 55.127s CPU time. 2024-05-14T17:18:30.342 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:18:30.351 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-14T17:18:30.358 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-14T17:18:30.430 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-ed40717a-1210-11ef-bc98-c7b262605968@mgr.y 2024-05-14T17:18:30.803 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 17:18:30 smithi123 systemd[1]: Stopping Ceph mgr.y for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T17:18:31.116 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 17:18:30 smithi123 podman[74413]: 2024-05-14 17:18:30.804252203 +0000 UTC m=+0.166244252 container died f4debe38cf600a509fb12f4cc8bfcb6fa0f6a9a2286fea647f146405c9ca482f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-14T17:18:31.409 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 17:18:31 smithi123 podman[74413]: 2024-05-14 17:18:31.223963883 +0000 UTC m=+0.585955932 container cleanup f4debe38cf600a509fb12f4cc8bfcb6fa0f6a9a2286fea647f146405c9ca482f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514) 2024-05-14T17:18:31.416 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 17:18:31 smithi123 bash[74413]: ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y 2024-05-14T17:18:31.809 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@mgr.y.service' 2024-05-14T17:18:31.860 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 17:18:31 smithi123 podman[74425]: 2024-05-14 17:18:31.567376504 +0000 UTC m=+0.767615705 container remove f4debe38cf600a509fb12f4cc8bfcb6fa0f6a9a2286fea647f146405c9ca482f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-y, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-14T17:18:31.869 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 17:18:31 smithi123 systemd[1]: ceph-ed40717a-1210-11ef-bc98-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-05-14T17:18:31.986 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 17:18:31 smithi123 systemd[1]: ceph-ed40717a-1210-11ef-bc98-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-05-14T17:18:31.993 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 17:18:31 smithi123 systemd[1]: Stopped Ceph mgr.y for ed40717a-1210-11ef-bc98-c7b262605968. 2024-05-14T17:18:32.002 INFO:journalctl@ceph.mgr.y.smithi123.stdout:May 14 17:18:31 smithi123 systemd[1]: ceph-ed40717a-1210-11ef-bc98-c7b262605968@mgr.y.service: Consumed 23.820s CPU time. 2024-05-14T17:18:32.012 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:18:32.021 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-14T17:18:32.030 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-14T17:18:32.038 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-ed40717a-1210-11ef-bc98-c7b262605968@mgr.z 2024-05-14T17:18:32.354 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 17:18:32 smithi151 systemd[1]: Stopping Ceph mgr.z for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T17:18:32.854 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 17:18:32 smithi151 podman[76660]: 2024-05-14 17:18:32.399130125 +0000 UTC m=+0.157403801 container died dda976e47705075fdec36aa263fadcbb67939193e22dea33e9bab4c558f85345 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-14T17:18:33.604 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 17:18:33 smithi151 podman[76660]: 2024-05-14 17:18:33.387178861 +0000 UTC m=+1.145452529 container cleanup dda976e47705075fdec36aa263fadcbb67939193e22dea33e9bab4c558f85345 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z, GIT_CLEAN=True, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-14T17:18:33.613 INFO:journalctl@ceph.mgr.z.smithi151.stdout:May 14 17:18:33 smithi151 bash[76660]: ceph-ed40717a-1210-11ef-bc98-c7b262605968-mgr-z 2024-05-14T17:18:33.846 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@mgr.z.service' 2024-05-14T17:18:33.886 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:18:33.894 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-14T17:18:34.024 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-14T17:18:34.031 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-ed40717a-1210-11ef-bc98-c7b262605968@osd.0 2024-05-14T17:18:34.367 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 17:18:34 smithi002 systemd[1]: Stopping Ceph osd.0 for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T17:18:34.746 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 17:18:34 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-0[46328]: 2024-05-14T17:18:34.367+0000 7f20ecbc7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:18:34.757 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 17:18:34 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-0[46328]: 2024-05-14T17:18:34.367+0000 7f20ecbc7700 -1 osd.0 119 *** Got signal Terminated *** 2024-05-14T17:18:34.766 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 17:18:34 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-0[46328]: 2024-05-14T17:18:34.367+0000 7f20ecbc7700 -1 osd.0 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T17:18:40.243 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 17:18:39 smithi002 podman[109196]: 2024-05-14 17:18:39.98833958 +0000 UTC m=+5.741823062 container died 5e05179ffa24f2651ddac11c05a745ea49c5d62dddca7ada8d067a10aaf46c34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS) 2024-05-14T17:18:40.582 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 17:18:40 smithi002 podman[109196]: 2024-05-14 17:18:40.390738027 +0000 UTC m=+6.144221513 container cleanup 5e05179ffa24f2651ddac11c05a745ea49c5d62dddca7ada8d067a10aaf46c34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-14T17:18:40.584 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 17:18:40 smithi002 bash[109196]: ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-0 2024-05-14T17:18:40.993 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 17:18:40 smithi002 podman[109224]: 2024-05-14 17:18:40.667551805 +0000 UTC m=+0.673025419 container remove 5e05179ffa24f2651ddac11c05a745ea49c5d62dddca7ada8d067a10aaf46c34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5) 2024-05-14T17:18:41.295 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 17:18:41 smithi002 podman[109314]: 2024-05-14 17:18:40.922725358 +0000 UTC m=+0.017570923 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:18:41.295 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 17:18:41 smithi002 podman[109314]: 2024-05-14 17:18:41.043664586 +0000 UTC m=+0.138510127 container create 6d71e4c90bbc90793d400b59f9e71a71f8879e5786600a86358c050f78aad761 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-0-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-14T17:18:41.609 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 17:18:41 smithi002 podman[109314]: 2024-05-14 17:18:41.296666668 +0000 UTC m=+0.391512232 container init 6d71e4c90bbc90793d400b59f9e71a71f8879e5786600a86358c050f78aad761 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-14T17:18:41.609 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 17:18:41 smithi002 podman[109314]: 2024-05-14 17:18:41.30056363 +0000 UTC m=+0.395409186 container start 6d71e4c90bbc90793d400b59f9e71a71f8879e5786600a86358c050f78aad761 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-0-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240514) 2024-05-14T17:18:41.610 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 17:18:41 smithi002 podman[109314]: 2024-05-14 17:18:41.39635965 +0000 UTC m=+0.491205215 container attach 6d71e4c90bbc90793d400b59f9e71a71f8879e5786600a86358c050f78aad761 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-14T17:18:41.898 INFO:journalctl@ceph.osd.0.smithi002.stdout:May 14 17:18:41 smithi002 podman[109314]: 2024-05-14 17:18:41.61099282 +0000 UTC m=+0.705838380 container died 6d71e4c90bbc90793d400b59f9e71a71f8879e5786600a86358c050f78aad761 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True) 2024-05-14T17:18:42.198 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@osd.0.service' 2024-05-14T17:18:42.244 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:18:42.244 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-14T17:18:42.244 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-14T17:18:42.244 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-ed40717a-1210-11ef-bc98-c7b262605968@osd.1 2024-05-14T17:18:42.544 INFO:journalctl@ceph.osd.1.smithi123.stdout:May 14 17:18:42 smithi123 systemd[1]: Stopping Ceph osd.1 for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T17:18:42.909 INFO:journalctl@ceph.osd.1.smithi123.stdout:May 14 17:18:42 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-1[40562]: 2024-05-14T17:18:42.544+0000 7f634d314700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:18:42.910 INFO:journalctl@ceph.osd.1.smithi123.stdout:May 14 17:18:42 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-1[40562]: 2024-05-14T17:18:42.544+0000 7f634d314700 -1 osd.1 119 *** Got signal Terminated *** 2024-05-14T17:18:42.910 INFO:journalctl@ceph.osd.1.smithi123.stdout:May 14 17:18:42 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-1[40562]: 2024-05-14T17:18:42.544+0000 7f634d314700 -1 osd.1 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T17:18:48.528 INFO:journalctl@ceph.osd.1.smithi123.stdout:May 14 17:18:48 smithi123 podman[74536]: 2024-05-14 17:18:48.21984406 +0000 UTC m=+5.775292164 container died 874f71783269aca34f329691cb1abb5f9ce4b46e9ac2fc0447d41d248dbfefc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-1, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514) 2024-05-14T17:18:48.887 INFO:journalctl@ceph.osd.1.smithi123.stdout:May 14 17:18:48 smithi123 podman[74536]: 2024-05-14 17:18:48.628664044 +0000 UTC m=+6.184112128 container cleanup 874f71783269aca34f329691cb1abb5f9ce4b46e9ac2fc0447d41d248dbfefc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-1, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514) 2024-05-14T17:18:48.887 INFO:journalctl@ceph.osd.1.smithi123.stdout:May 14 17:18:48 smithi123 bash[74536]: ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-1 2024-05-14T17:18:49.159 INFO:journalctl@ceph.osd.1.smithi123.stdout:May 14 17:18:48 smithi123 podman[74555]: 2024-05-14 17:18:48.88866173 +0000 UTC m=+0.668958618 container remove 874f71783269aca34f329691cb1abb5f9ce4b46e9ac2fc0447d41d248dbfefc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-1, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True) 2024-05-14T17:18:49.524 INFO:journalctl@ceph.osd.1.smithi123.stdout:May 14 17:18:49 smithi123 podman[74640]: 2024-05-14 17:18:49.140952157 +0000 UTC m=+0.017760361 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:18:49.524 INFO:journalctl@ceph.osd.1.smithi123.stdout:May 14 17:18:49 smithi123 podman[74640]: 2024-05-14 17:18:49.281221289 +0000 UTC m=+0.158029520 container create 5ca0bb9a84ddd7f69f4d7fc78c611d922d809c387fd5516036ecdc91cd22aa8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-14T17:18:49.812 INFO:journalctl@ceph.osd.1.smithi123.stdout:May 14 17:18:49 smithi123 podman[74640]: 2024-05-14 17:18:49.525448291 +0000 UTC m=+0.402256478 container init 5ca0bb9a84ddd7f69f4d7fc78c611d922d809c387fd5516036ecdc91cd22aa8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-1-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, ceph=True) 2024-05-14T17:18:49.813 INFO:journalctl@ceph.osd.1.smithi123.stdout:May 14 17:18:49 smithi123 podman[74640]: 2024-05-14 17:18:49.529443881 +0000 UTC m=+0.406252077 container start 5ca0bb9a84ddd7f69f4d7fc78c611d922d809c387fd5516036ecdc91cd22aa8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240514, io.buildah.version=1.33.5) 2024-05-14T17:18:49.813 INFO:journalctl@ceph.osd.1.smithi123.stdout:May 14 17:18:49 smithi123 podman[74640]: 2024-05-14 17:18:49.633782896 +0000 UTC m=+0.510591092 container attach 5ca0bb9a84ddd7f69f4d7fc78c611d922d809c387fd5516036ecdc91cd22aa8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-14T17:18:50.159 INFO:journalctl@ceph.osd.1.smithi123.stdout:May 14 17:18:49 smithi123 podman[74640]: 2024-05-14 17:18:49.814371371 +0000 UTC m=+0.691179567 container died 5ca0bb9a84ddd7f69f4d7fc78c611d922d809c387fd5516036ecdc91cd22aa8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-14T17:18:50.485 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@osd.1.service' 2024-05-14T17:18:50.526 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:18:50.526 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-14T17:18:50.526 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-14T17:18:50.527 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-ed40717a-1210-11ef-bc98-c7b262605968@osd.2 2024-05-14T17:18:50.841 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 14 17:18:50 smithi151 systemd[1]: Stopping Ceph osd.2 for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T17:18:51.104 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 14 17:18:50 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-2[41154]: 2024-05-14T17:18:50.841+0000 7fe218827700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:18:51.104 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 14 17:18:50 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-2[41154]: 2024-05-14T17:18:50.841+0000 7fe218827700 -1 osd.2 119 *** Got signal Terminated *** 2024-05-14T17:18:51.104 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 14 17:18:50 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-2[41154]: 2024-05-14T17:18:50.841+0000 7fe218827700 -1 osd.2 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T17:18:56.834 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 14 17:18:56 smithi151 podman[76784]: 2024-05-14 17:18:56.503577902 +0000 UTC m=+5.783701018 container died cd194555686988cae5c342ca5f30b9e622a8f6b4349be1ae26401c498ab5163f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-14T17:18:57.103 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 14 17:18:56 smithi151 podman[76784]: 2024-05-14 17:18:56.935351275 +0000 UTC m=+6.215474382 container cleanup cd194555686988cae5c342ca5f30b9e622a8f6b4349be1ae26401c498ab5163f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, ceph=True, GIT_BRANCH=HEAD) 2024-05-14T17:18:57.104 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 14 17:18:56 smithi151 bash[76784]: ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-2 2024-05-14T17:18:57.593 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 14 17:18:57 smithi151 podman[76799]: 2024-05-14 17:18:57.154344221 +0000 UTC m=+0.653020870 container remove cd194555686988cae5c342ca5f30b9e622a8f6b4349be1ae26401c498ab5163f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T17:18:57.854 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 14 17:18:57 smithi151 podman[76891]: 2024-05-14 17:18:57.49868844 +0000 UTC m=+0.017792829 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:18:57.854 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 14 17:18:57 smithi151 podman[76891]: 2024-05-14 17:18:57.638950242 +0000 UTC m=+0.158054611 container create 567192ee3d2fad759a47d4b9069aa37bcdee7787c03d265280b4c151b236204e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5) 2024-05-14T17:18:58.223 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 14 17:18:57 smithi151 podman[76891]: 2024-05-14 17:18:57.917629324 +0000 UTC m=+0.436733696 container init 567192ee3d2fad759a47d4b9069aa37bcdee7787c03d265280b4c151b236204e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-14T17:18:58.223 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 14 17:18:57 smithi151 podman[76891]: 2024-05-14 17:18:57.921149594 +0000 UTC m=+0.440253972 container start 567192ee3d2fad759a47d4b9069aa37bcdee7787c03d265280b4c151b236204e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-2-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T17:18:58.223 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 14 17:18:58 smithi151 podman[76891]: 2024-05-14 17:18:58.017943116 +0000 UTC m=+0.537047510 container attach 567192ee3d2fad759a47d4b9069aa37bcdee7787c03d265280b4c151b236204e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-14T17:18:58.497 INFO:journalctl@ceph.osd.2.smithi151.stdout:May 14 17:18:58 smithi151 podman[76891]: 2024-05-14 17:18:58.224648621 +0000 UTC m=+0.743753007 container died 567192ee3d2fad759a47d4b9069aa37bcdee7787c03d265280b4c151b236204e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-14T17:18:58.907 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@osd.2.service' 2024-05-14T17:18:58.944 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:18:58.944 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-14T17:18:58.944 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-14T17:18:58.944 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-ed40717a-1210-11ef-bc98-c7b262605968@osd.3 2024-05-14T17:18:59.243 INFO:journalctl@ceph.osd.3.smithi002.stdout:May 14 17:18:58 smithi002 systemd[1]: Stopping Ceph osd.3 for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T17:18:59.743 INFO:journalctl@ceph.osd.3.smithi002.stdout:May 14 17:18:59 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-3[53111]: 2024-05-14T17:18:59.251+0000 7f6bf9a51700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:18:59.743 INFO:journalctl@ceph.osd.3.smithi002.stdout:May 14 17:18:59 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-3[53111]: 2024-05-14T17:18:59.251+0000 7f6bf9a51700 -1 osd.3 119 *** Got signal Terminated *** 2024-05-14T17:18:59.744 INFO:journalctl@ceph.osd.3.smithi002.stdout:May 14 17:18:59 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-3[53111]: 2024-05-14T17:18:59.251+0000 7f6bf9a51700 -1 osd.3 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T17:19:01.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:01 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:01.181+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:01.993 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:01 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:01.714+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:02.104 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:01 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5[47942]: 2024-05-14T17:19:01.828+0000 7f608350c700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:37.828513+0000 front 2024-05-14T17:18:37.828560+0000 (oldest deadline 2024-05-14T17:19:01.328693+0000) 2024-05-14T17:19:02.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:02 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:02.143+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:02.993 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:02 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:02.709+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:03.104 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:02 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5[47942]: 2024-05-14T17:19:02.833+0000 7f608350c700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:37.828513+0000 front 2024-05-14T17:18:37.828560+0000 (oldest deadline 2024-05-14T17:19:01.328693+0000) 2024-05-14T17:19:03.159 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:02 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:02.882+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:03.493 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:03 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:03.162+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:39.332351+0000 front 2024-05-14T17:18:39.332307+0000 (oldest deadline 2024-05-14T17:19:02.733193+0000) 2024-05-14T17:19:03.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:03 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:03.160+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:03.993 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:03 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:03.707+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:04.104 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:03 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5[47942]: 2024-05-14T17:19:03.842+0000 7f608350c700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:37.828513+0000 front 2024-05-14T17:18:37.828560+0000 (oldest deadline 2024-05-14T17:19:01.328693+0000) 2024-05-14T17:19:04.159 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:03 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:03.906+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:04.493 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:04 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:04.165+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:39.332351+0000 front 2024-05-14T17:18:39.332307+0000 (oldest deadline 2024-05-14T17:19:02.733193+0000) 2024-05-14T17:19:04.603 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:04.141+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:04.993 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:04 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:04.661+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:04.993 INFO:journalctl@ceph.osd.3.smithi002.stdout:May 14 17:19:04 smithi002 podman[109420]: 2024-05-14 17:19:04.813421939 +0000 UTC m=+5.656376381 container died df0c61db7c5f0496645584bb1656ffcf15c7c2d7557cea3cfa4e04d61bd54677 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-3, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T17:19:05.159 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:04 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:04.876+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:05.181 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5[47942]: 2024-05-14T17:19:04.875+0000 7f608350c700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:37.828513+0000 front 2024-05-14T17:18:37.828560+0000 (oldest deadline 2024-05-14T17:19:01.328693+0000) 2024-05-14T17:19:05.493 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:05 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:05.128+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:39.332351+0000 front 2024-05-14T17:18:39.332307+0000 (oldest deadline 2024-05-14T17:19:02.733193+0000) 2024-05-14T17:19:05.493 INFO:journalctl@ceph.osd.3.smithi002.stdout:May 14 17:19:05 smithi002 podman[109420]: 2024-05-14 17:19:05.185094488 +0000 UTC m=+6.028048914 container cleanup df0c61db7c5f0496645584bb1656ffcf15c7c2d7557cea3cfa4e04d61bd54677 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-3, RELEASE=wip-lusov-quiesce-9b63866, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=) 2024-05-14T17:19:05.494 INFO:journalctl@ceph.osd.3.smithi002.stdout:May 14 17:19:05 smithi002 bash[109420]: ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-3 2024-05-14T17:19:05.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:05 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:05.183+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:05.969 INFO:journalctl@ceph.osd.3.smithi002.stdout:May 14 17:19:05 smithi002 podman[109438]: 2024-05-14 17:19:05.570300477 +0000 UTC m=+0.760517527 container remove df0c61db7c5f0496645584bb1656ffcf15c7c2d7557cea3cfa4e04d61bd54677 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-3, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-14T17:19:05.970 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:05 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:05.702+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:06.104 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:05 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5[47942]: 2024-05-14T17:19:05.853+0000 7f608350c700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:37.828513+0000 front 2024-05-14T17:18:37.828560+0000 (oldest deadline 2024-05-14T17:19:01.328693+0000) 2024-05-14T17:19:06.159 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:05 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:05.884+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:06.222 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:06 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:06.145+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:39.332351+0000 front 2024-05-14T17:18:39.332307+0000 (oldest deadline 2024-05-14T17:19:02.733193+0000) 2024-05-14T17:19:06.223 INFO:journalctl@ceph.osd.3.smithi002.stdout:May 14 17:19:05 smithi002 podman[109524]: 2024-05-14 17:19:05.87397486 +0000 UTC m=+0.017679393 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:19:06.223 INFO:journalctl@ceph.osd.3.smithi002.stdout:May 14 17:19:05 smithi002 podman[109524]: 2024-05-14 17:19:05.979461121 +0000 UTC m=+0.123165634 container create 2c89524a62935f0589a5fb2f3c42647ec2b10e24d0eda15a851cc5b9bffdfe3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-14T17:19:06.223 INFO:journalctl@ceph.osd.3.smithi002.stdout:May 14 17:19:06 smithi002 podman[109524]: 2024-05-14 17:19:06.224245478 +0000 UTC m=+0.367950004 container init 2c89524a62935f0589a5fb2f3c42647ec2b10e24d0eda15a851cc5b9bffdfe3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-14T17:19:06.493 INFO:journalctl@ceph.osd.3.smithi002.stdout:May 14 17:19:06 smithi002 podman[109524]: 2024-05-14 17:19:06.22820676 +0000 UTC m=+0.371911273 container start 2c89524a62935f0589a5fb2f3c42647ec2b10e24d0eda15a851cc5b9bffdfe3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T17:19:06.493 INFO:journalctl@ceph.osd.3.smithi002.stdout:May 14 17:19:06 smithi002 podman[109524]: 2024-05-14 17:19:06.324080669 +0000 UTC m=+0.467785191 container attach 2c89524a62935f0589a5fb2f3c42647ec2b10e24d0eda15a851cc5b9bffdfe3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2) 2024-05-14T17:19:06.603 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:06 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:06.228+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:06.901 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:06 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:06.709+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:06.902 INFO:journalctl@ceph.osd.3.smithi002.stdout:May 14 17:19:06 smithi002 podman[109524]: 2024-05-14 17:19:06.534201735 +0000 UTC m=+0.677906247 container died 2c89524a62935f0589a5fb2f3c42647ec2b10e24d0eda15a851cc5b9bffdfe3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux ) 2024-05-14T17:19:07.104 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:06 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5[47942]: 2024-05-14T17:19:06.818+0000 7f608350c700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:37.828513+0000 front 2024-05-14T17:18:37.828560+0000 (oldest deadline 2024-05-14T17:19:01.328693+0000) 2024-05-14T17:19:07.222 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@osd.3.service' 2024-05-14T17:19:07.269 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:19:07.270 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-14T17:19:07.270 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-14T17:19:07.270 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-ed40717a-1210-11ef-bc98-c7b262605968@osd.4 2024-05-14T17:19:07.297 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:06 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:06.911+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:07.493 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:07 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:07.166+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:39.332351+0000 front 2024-05-14T17:18:39.332307+0000 (oldest deadline 2024-05-14T17:19:02.733193+0000) 2024-05-14T17:19:07.564 INFO:journalctl@ceph.osd.4.smithi123.stdout:May 14 17:19:07 smithi123 systemd[1]: Stopping Ceph osd.4 for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T17:19:07.603 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:07 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:07.219+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:07.909 INFO:journalctl@ceph.osd.4.smithi123.stdout:May 14 17:19:07 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-4[47500]: 2024-05-14T17:19:07.564+0000 7f191c9a5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:19:07.909 INFO:journalctl@ceph.osd.4.smithi123.stdout:May 14 17:19:07 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-4[47500]: 2024-05-14T17:19:07.564+0000 7f191c9a5700 -1 osd.4 119 *** Got signal Terminated *** 2024-05-14T17:19:07.910 INFO:journalctl@ceph.osd.4.smithi123.stdout:May 14 17:19:07 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-4[47500]: 2024-05-14T17:19:07.564+0000 7f191c9a5700 -1 osd.4 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T17:19:08.103 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:07 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5[47942]: 2024-05-14T17:19:07.847+0000 7f608350c700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:37.828513+0000 front 2024-05-14T17:18:37.828560+0000 (oldest deadline 2024-05-14T17:19:01.328693+0000) 2024-05-14T17:19:08.129 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:07 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:07.750+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:08.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:07 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:07.915+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:08.493 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:08 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:08.131+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:39.332351+0000 front 2024-05-14T17:18:39.332307+0000 (oldest deadline 2024-05-14T17:19:02.733193+0000) 2024-05-14T17:19:08.603 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:08 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:08.185+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:09.169 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:08 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:08.773+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:09.186 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:08 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5[47942]: 2024-05-14T17:19:08.876+0000 7f608350c700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:37.828513+0000 front 2024-05-14T17:18:37.828560+0000 (oldest deadline 2024-05-14T17:19:01.328693+0000) 2024-05-14T17:19:09.186 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:08 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5[47942]: 2024-05-14T17:19:08.876+0000 7f608350c700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:44.829832+0000 front 2024-05-14T17:18:44.829751+0000 (oldest deadline 2024-05-14T17:19:08.830123+0000) 2024-05-14T17:19:09.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:08 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:08.949+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:09.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:08 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:08.949+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:09.463 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:09 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:09.170+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:39.332351+0000 front 2024-05-14T17:18:39.332307+0000 (oldest deadline 2024-05-14T17:19:02.733193+0000) 2024-05-14T17:19:09.464 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:09 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:09.170+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:44.935082+0000 front 2024-05-14T17:18:44.935193+0000 (oldest deadline 2024-05-14T17:19:08.435222+0000) 2024-05-14T17:19:09.518 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:09 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:09.186+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:09.518 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:09 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:09.225+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.353156+0000 front 2024-05-14T17:18:45.352963+0000 (oldest deadline 2024-05-14T17:19:08.753619+0000) 2024-05-14T17:19:10.138 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:09 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:09.751+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:10.157 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:09 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5[47942]: 2024-05-14T17:19:09.882+0000 7f608350c700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:37.828513+0000 front 2024-05-14T17:18:37.828560+0000 (oldest deadline 2024-05-14T17:19:01.328693+0000) 2024-05-14T17:19:10.157 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:09 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5[47942]: 2024-05-14T17:19:09.882+0000 7f608350c700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:44.829832+0000 front 2024-05-14T17:18:44.829751+0000 (oldest deadline 2024-05-14T17:19:08.830123+0000) 2024-05-14T17:19:10.159 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:09 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:09.766+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:43.134568+0000 front 2024-05-14T17:18:43.134641+0000 (oldest deadline 2024-05-14T17:19:09.034680+0000) 2024-05-14T17:19:10.159 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:09 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:09.906+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:10.159 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:09 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:09.906+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:10.448 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:10 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:10.138+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:39.332351+0000 front 2024-05-14T17:18:39.332307+0000 (oldest deadline 2024-05-14T17:19:02.733193+0000) 2024-05-14T17:19:10.448 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:10 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:10.138+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:44.935082+0000 front 2024-05-14T17:18:44.935193+0000 (oldest deadline 2024-05-14T17:19:08.435222+0000) 2024-05-14T17:19:10.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:10 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:10.203+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.353156+0000 front 2024-05-14T17:18:45.352963+0000 (oldest deadline 2024-05-14T17:19:08.753619+0000) 2024-05-14T17:19:10.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:10 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:10.157+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:10.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:10 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:10.157+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:10.659 INFO:journalctl@ceph.osd.4.smithi123.stdout:May 14 17:19:10 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-4[47500]: 2024-05-14T17:19:10.399+0000 7f1914f91700 -1 osd.4 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.612496+0000 front 2024-05-14T17:18:45.612564+0000 (oldest deadline 2024-05-14T17:19:10.312613+0000) 2024-05-14T17:19:11.127 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:10 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:10.778+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:11.159 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:10 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:10.797+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:43.134568+0000 front 2024-05-14T17:18:43.134641+0000 (oldest deadline 2024-05-14T17:19:09.034680+0000) 2024-05-14T17:19:11.159 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:10 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:10.874+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:11.160 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:10 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:10.874+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:11.169 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:10 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5[47942]: 2024-05-14T17:19:10.902+0000 7f608350c700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:37.828513+0000 front 2024-05-14T17:18:37.828560+0000 (oldest deadline 2024-05-14T17:19:01.328693+0000) 2024-05-14T17:19:11.170 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:10 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5[47942]: 2024-05-14T17:19:10.902+0000 7f608350c700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:44.829832+0000 front 2024-05-14T17:18:44.829751+0000 (oldest deadline 2024-05-14T17:19:08.830123+0000) 2024-05-14T17:19:11.493 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:11 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:11.128+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:39.332351+0000 front 2024-05-14T17:18:39.332307+0000 (oldest deadline 2024-05-14T17:19:02.733193+0000) 2024-05-14T17:19:11.493 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:11 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:11.128+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:44.935082+0000 front 2024-05-14T17:18:44.935193+0000 (oldest deadline 2024-05-14T17:19:08.435222+0000) 2024-05-14T17:19:11.605 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:11 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:11.197+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.353156+0000 front 2024-05-14T17:18:45.352963+0000 (oldest deadline 2024-05-14T17:19:08.753619+0000) 2024-05-14T17:19:11.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:11 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:11.170+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:11.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:11 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:11.170+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:11.659 INFO:journalctl@ceph.osd.4.smithi123.stdout:May 14 17:19:11 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-4[47500]: 2024-05-14T17:19:11.351+0000 7f1914f91700 -1 osd.4 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.612496+0000 front 2024-05-14T17:18:45.612564+0000 (oldest deadline 2024-05-14T17:19:10.312613+0000) 2024-05-14T17:19:12.156 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:11 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:11.778+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:12.159 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:11 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:11.769+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:43.134568+0000 front 2024-05-14T17:18:43.134641+0000 (oldest deadline 2024-05-14T17:19:09.034680+0000) 2024-05-14T17:19:12.159 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:11 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:11.838+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:12.159 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:11 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:11.838+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:12.188 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:11 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5[47942]: 2024-05-14T17:19:11.933+0000 7f608350c700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:37.828513+0000 front 2024-05-14T17:18:37.828560+0000 (oldest deadline 2024-05-14T17:19:01.328693+0000) 2024-05-14T17:19:12.188 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:11 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5[47942]: 2024-05-14T17:19:11.933+0000 7f608350c700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:44.829832+0000 front 2024-05-14T17:18:44.829751+0000 (oldest deadline 2024-05-14T17:19:08.830123+0000) 2024-05-14T17:19:12.188 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:12 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:12.175+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:12.188 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:12 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:12.175+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:12.493 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:12 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:12.157+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:39.332351+0000 front 2024-05-14T17:18:39.332307+0000 (oldest deadline 2024-05-14T17:19:02.733193+0000) 2024-05-14T17:19:12.494 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:12 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:12.157+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:44.935082+0000 front 2024-05-14T17:18:44.935193+0000 (oldest deadline 2024-05-14T17:19:08.435222+0000) 2024-05-14T17:19:12.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:12 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:12.188+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.353156+0000 front 2024-05-14T17:18:45.352963+0000 (oldest deadline 2024-05-14T17:19:08.753619+0000) 2024-05-14T17:19:12.659 INFO:journalctl@ceph.osd.4.smithi123.stdout:May 14 17:19:12 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-4[47500]: 2024-05-14T17:19:12.328+0000 7f1914f91700 -1 osd.4 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.612496+0000 front 2024-05-14T17:18:45.612564+0000 (oldest deadline 2024-05-14T17:19:10.312613+0000) 2024-05-14T17:19:13.148 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:12 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:12.773+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:13.148 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:12 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:12.773+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:13.155 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:12 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:12.737+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:43.134568+0000 front 2024-05-14T17:18:43.134641+0000 (oldest deadline 2024-05-14T17:19:09.034680+0000) 2024-05-14T17:19:13.156 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:12 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:12.858+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:13.156 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:12 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:12.858+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:13.172 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:12 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5[47942]: 2024-05-14T17:19:12.902+0000 7f608350c700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:37.828513+0000 front 2024-05-14T17:18:37.828560+0000 (oldest deadline 2024-05-14T17:19:01.328693+0000) 2024-05-14T17:19:13.172 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:12 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5[47942]: 2024-05-14T17:19:12.902+0000 7f608350c700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:44.829832+0000 front 2024-05-14T17:18:44.829751+0000 (oldest deadline 2024-05-14T17:19:08.830123+0000) 2024-05-14T17:19:13.172 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:13 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:13.151+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:13.172 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:13 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:13.151+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:13.409 INFO:journalctl@ceph.osd.4.smithi123.stdout:May 14 17:19:13 smithi123 podman[74761]: 2024-05-14 17:19:13.157186638 +0000 UTC m=+5.683536162 container died 5a66038a280a6d390bef23ebaa5380355ec2ec6d354fe476b46946ea23bacdca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, ceph=True) 2024-05-14T17:19:13.493 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:13 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:13.148+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:39.332351+0000 front 2024-05-14T17:18:39.332307+0000 (oldest deadline 2024-05-14T17:19:02.733193+0000) 2024-05-14T17:19:13.493 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:13 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:13.148+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:44.935082+0000 front 2024-05-14T17:18:44.935193+0000 (oldest deadline 2024-05-14T17:19:08.435222+0000) 2024-05-14T17:19:13.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:13 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:13.172+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.353156+0000 front 2024-05-14T17:18:45.352963+0000 (oldest deadline 2024-05-14T17:19:08.753619+0000) 2024-05-14T17:19:13.732 INFO:journalctl@ceph.osd.4.smithi123.stdout:May 14 17:19:13 smithi123 podman[74761]: 2024-05-14 17:19:13.548757756 +0000 UTC m=+6.075107267 container cleanup 5a66038a280a6d390bef23ebaa5380355ec2ec6d354fe476b46946ea23bacdca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-4, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0) 2024-05-14T17:19:13.732 INFO:journalctl@ceph.osd.4.smithi123.stdout:May 14 17:19:13 smithi123 bash[74761]: ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-4 2024-05-14T17:19:14.106 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:13 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:13.793+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:14.106 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:13 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:13.793+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:14.157 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:13 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:13.732+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:43.134568+0000 front 2024-05-14T17:18:43.134641+0000 (oldest deadline 2024-05-14T17:19:09.034680+0000) 2024-05-14T17:19:14.158 INFO:journalctl@ceph.osd.4.smithi123.stdout:May 14 17:19:13 smithi123 podman[74776]: 2024-05-14 17:19:13.808617031 +0000 UTC m=+0.646096157 container remove 5a66038a280a6d390bef23ebaa5380355ec2ec6d354fe476b46946ea23bacdca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-4, RELEASE=wip-lusov-quiesce-9b63866, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-14T17:19:14.158 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:13 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:13.896+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:14.158 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:13 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:13.896+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:14.354 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:14 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:14.131+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.353156+0000 front 2024-05-14T17:18:45.352963+0000 (oldest deadline 2024-05-14T17:19:08.753619+0000) 2024-05-14T17:19:14.354 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:13 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5[47942]: 2024-05-14T17:19:13.922+0000 7f608350c700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:37.828513+0000 front 2024-05-14T17:18:37.828560+0000 (oldest deadline 2024-05-14T17:19:01.328693+0000) 2024-05-14T17:19:14.354 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:13 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5[47942]: 2024-05-14T17:19:13.922+0000 7f608350c700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:44.829832+0000 front 2024-05-14T17:18:44.829751+0000 (oldest deadline 2024-05-14T17:19:08.830123+0000) 2024-05-14T17:19:14.354 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:14 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:14.157+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:14.355 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:14 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:14.157+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:14.409 INFO:journalctl@ceph.osd.4.smithi123.stdout:May 14 17:19:14 smithi123 podman[74864]: 2024-05-14 17:19:14.061764569 +0000 UTC m=+0.017072640 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:19:14.409 INFO:journalctl@ceph.osd.4.smithi123.stdout:May 14 17:19:14 smithi123 podman[74864]: 2024-05-14 17:19:14.192922266 +0000 UTC m=+0.148230360 container create e8f508c7f17237d740d4065b52e2382073f499f3a181de372718236edabb6e66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T17:19:14.494 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:14 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:14.106+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:39.332351+0000 front 2024-05-14T17:18:39.332307+0000 (oldest deadline 2024-05-14T17:19:02.733193+0000) 2024-05-14T17:19:14.494 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:14 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:14.106+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:44.935082+0000 front 2024-05-14T17:18:44.935193+0000 (oldest deadline 2024-05-14T17:19:08.435222+0000) 2024-05-14T17:19:14.728 INFO:journalctl@ceph.osd.4.smithi123.stdout:May 14 17:19:14 smithi123 podman[74864]: 2024-05-14 17:19:14.449068262 +0000 UTC m=+0.404376340 container init e8f508c7f17237d740d4065b52e2382073f499f3a181de372718236edabb6e66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514) 2024-05-14T17:19:14.728 INFO:journalctl@ceph.osd.4.smithi123.stdout:May 14 17:19:14 smithi123 podman[74864]: 2024-05-14 17:19:14.452394449 +0000 UTC m=+0.407702520 container start e8f508c7f17237d740d4065b52e2382073f499f3a181de372718236edabb6e66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-14T17:19:14.728 INFO:journalctl@ceph.osd.4.smithi123.stdout:May 14 17:19:14 smithi123 podman[74864]: 2024-05-14 17:19:14.553687581 +0000 UTC m=+0.508995667 container attach e8f508c7f17237d740d4065b52e2382073f499f3a181de372718236edabb6e66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T17:19:15.055 INFO:journalctl@ceph.osd.4.smithi123.stdout:May 14 17:19:14 smithi123 podman[74864]: 2024-05-14 17:19:14.740432576 +0000 UTC m=+0.695740649 container died e8f508c7f17237d740d4065b52e2382073f499f3a181de372718236edabb6e66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-14T17:19:15.056 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:14 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:14.728+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:43.134568+0000 front 2024-05-14T17:18:43.134641+0000 (oldest deadline 2024-05-14T17:19:09.034680+0000) 2024-05-14T17:19:15.056 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:14 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:14.890+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:15.056 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:14 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:14.890+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:15.120 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:14 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:14.835+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:15.120 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:14 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:14.835+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:15.354 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:14 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5[47942]: 2024-05-14T17:19:14.906+0000 7f608350c700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:37.828513+0000 front 2024-05-14T17:18:37.828560+0000 (oldest deadline 2024-05-14T17:19:01.328693+0000) 2024-05-14T17:19:15.354 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:14 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5[47942]: 2024-05-14T17:19:14.906+0000 7f608350c700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:44.829832+0000 front 2024-05-14T17:18:44.829751+0000 (oldest deadline 2024-05-14T17:19:08.830123+0000) 2024-05-14T17:19:15.354 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:15 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:15.155+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:15.354 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:15 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:15.155+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:15.354 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:15 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:15.104+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.353156+0000 front 2024-05-14T17:18:45.352963+0000 (oldest deadline 2024-05-14T17:19:08.753619+0000) 2024-05-14T17:19:15.387 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@osd.4.service' 2024-05-14T17:19:15.465 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:19:15.465 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-14T17:19:15.465 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-14T17:19:15.465 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-ed40717a-1210-11ef-bc98-c7b262605968@osd.5 2024-05-14T17:19:15.493 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:15 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:15.120+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:39.332351+0000 front 2024-05-14T17:18:39.332307+0000 (oldest deadline 2024-05-14T17:19:02.733193+0000) 2024-05-14T17:19:15.493 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:15 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:15.120+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:44.935082+0000 front 2024-05-14T17:18:44.935193+0000 (oldest deadline 2024-05-14T17:19:08.435222+0000) 2024-05-14T17:19:15.854 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:15 smithi151 systemd[1]: Stopping Ceph osd.5 for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T17:19:16.140 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:15 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:15.834+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:16.140 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:15 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:15.834+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:16.159 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:15 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:15.742+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:43.134568+0000 front 2024-05-14T17:18:43.134641+0000 (oldest deadline 2024-05-14T17:19:09.034680+0000) 2024-05-14T17:19:16.159 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:15 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:15.911+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:16.159 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:15 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:15.911+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:16.354 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:16 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:16.131+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.353156+0000 front 2024-05-14T17:18:45.352963+0000 (oldest deadline 2024-05-14T17:19:08.753619+0000) 2024-05-14T17:19:16.354 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:16 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:16.174+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:16.354 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:16 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:16.174+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:16.354 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:15 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5[47942]: 2024-05-14T17:19:15.928+0000 7f608350c700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:37.828513+0000 front 2024-05-14T17:18:37.828560+0000 (oldest deadline 2024-05-14T17:19:01.328693+0000) 2024-05-14T17:19:16.354 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:15 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5[47942]: 2024-05-14T17:19:15.928+0000 7f608350c700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:44.829832+0000 front 2024-05-14T17:18:44.829751+0000 (oldest deadline 2024-05-14T17:19:08.830123+0000) 2024-05-14T17:19:16.354 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:15 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5[47942]: 2024-05-14T17:19:15.960+0000 7f608af20700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:19:16.355 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:15 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5[47942]: 2024-05-14T17:19:15.960+0000 7f608af20700 -1 osd.5 119 *** Got signal Terminated *** 2024-05-14T17:19:16.355 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:15 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5[47942]: 2024-05-14T17:19:15.960+0000 7f608af20700 -1 osd.5 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T17:19:16.493 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:16 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:16.140+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:39.332351+0000 front 2024-05-14T17:18:39.332307+0000 (oldest deadline 2024-05-14T17:19:02.733193+0000) 2024-05-14T17:19:16.493 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:16 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:16.140+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:44.935082+0000 front 2024-05-14T17:18:44.935193+0000 (oldest deadline 2024-05-14T17:19:08.435222+0000) 2024-05-14T17:19:17.106 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:16 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:16.798+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:17.106 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:16 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:16.798+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:17.143 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:16 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5[47942]: 2024-05-14T17:19:16.887+0000 7f608350c700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:37.828513+0000 front 2024-05-14T17:18:37.828560+0000 (oldest deadline 2024-05-14T17:19:01.328693+0000) 2024-05-14T17:19:17.143 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:16 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5[47942]: 2024-05-14T17:19:16.887+0000 7f608350c700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:44.829832+0000 front 2024-05-14T17:18:44.829751+0000 (oldest deadline 2024-05-14T17:19:08.830123+0000) 2024-05-14T17:19:17.159 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:16 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:16.723+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:43.134568+0000 front 2024-05-14T17:18:43.134641+0000 (oldest deadline 2024-05-14T17:19:09.034680+0000) 2024-05-14T17:19:17.160 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:16 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:16.892+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:17.160 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:16 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:16.892+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:17.493 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:17 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:17.106+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:39.332351+0000 front 2024-05-14T17:18:39.332307+0000 (oldest deadline 2024-05-14T17:19:02.733193+0000) 2024-05-14T17:19:17.493 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:17 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:17.107+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:44.935082+0000 front 2024-05-14T17:18:44.935193+0000 (oldest deadline 2024-05-14T17:19:08.435222+0000) 2024-05-14T17:19:17.493 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:17 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:17.107+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:52.738851+0000 front 2024-05-14T17:18:52.738798+0000 (oldest deadline 2024-05-14T17:19:16.738960+0000) 2024-05-14T17:19:17.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:17 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:17.205+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:17.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:17 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:17.205+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:17.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:17 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:17.143+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.353156+0000 front 2024-05-14T17:18:45.352963+0000 (oldest deadline 2024-05-14T17:19:08.753619+0000) 2024-05-14T17:19:18.104 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:17 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5[47942]: 2024-05-14T17:19:17.848+0000 7f608350c700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:37.828513+0000 front 2024-05-14T17:18:37.828560+0000 (oldest deadline 2024-05-14T17:19:01.328693+0000) 2024-05-14T17:19:18.104 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:17 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5[47942]: 2024-05-14T17:19:17.848+0000 7f608350c700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:44.829832+0000 front 2024-05-14T17:18:44.829751+0000 (oldest deadline 2024-05-14T17:19:08.830123+0000) 2024-05-14T17:19:18.111 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:17 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:17.765+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:18.112 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:17 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:17.765+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:18.112 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:17 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:17.765+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.345149+0000 front 2024-05-14T17:18:55.344987+0000 (oldest deadline 2024-05-14T17:19:17.644999+0000) 2024-05-14T17:19:18.159 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:17 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:17.746+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:43.134568+0000 front 2024-05-14T17:18:43.134641+0000 (oldest deadline 2024-05-14T17:19:09.034680+0000) 2024-05-14T17:19:18.159 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:17 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:17.746+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.436396+0000 front 2024-05-14T17:18:55.436310+0000 (oldest deadline 2024-05-14T17:19:17.136467+0000) 2024-05-14T17:19:18.160 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:17 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:17.939+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:18.160 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:17 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:17.939+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:18.160 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:17 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:17.939+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:18.493 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:18 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:18.112+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:39.332351+0000 front 2024-05-14T17:18:39.332307+0000 (oldest deadline 2024-05-14T17:19:02.733193+0000) 2024-05-14T17:19:18.493 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:18 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:18.112+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:44.935082+0000 front 2024-05-14T17:18:44.935193+0000 (oldest deadline 2024-05-14T17:19:08.435222+0000) 2024-05-14T17:19:18.493 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:18 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:18.112+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:52.738851+0000 front 2024-05-14T17:18:52.738798+0000 (oldest deadline 2024-05-14T17:19:16.738960+0000) 2024-05-14T17:19:18.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:18 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:18.193+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.353156+0000 front 2024-05-14T17:18:45.352963+0000 (oldest deadline 2024-05-14T17:19:08.753619+0000) 2024-05-14T17:19:18.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:18 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:18.236+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:18.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:18 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:18.236+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:18.967 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:18 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:18.703+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:43.134568+0000 front 2024-05-14T17:18:43.134641+0000 (oldest deadline 2024-05-14T17:19:09.034680+0000) 2024-05-14T17:19:18.967 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:18 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:18.703+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.436396+0000 front 2024-05-14T17:18:55.436310+0000 (oldest deadline 2024-05-14T17:19:17.136467+0000) 2024-05-14T17:19:19.137 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:18 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:18.755+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:19.137 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:18 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:18.755+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:19.137 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:18 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:18.755+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.345149+0000 front 2024-05-14T17:18:55.344987+0000 (oldest deadline 2024-05-14T17:19:17.644999+0000) 2024-05-14T17:19:19.228 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:18 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5[47942]: 2024-05-14T17:19:18.868+0000 7f608350c700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:37.828513+0000 front 2024-05-14T17:18:37.828560+0000 (oldest deadline 2024-05-14T17:19:01.328693+0000) 2024-05-14T17:19:19.228 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:18 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5[47942]: 2024-05-14T17:19:18.868+0000 7f608350c700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:44.829832+0000 front 2024-05-14T17:18:44.829751+0000 (oldest deadline 2024-05-14T17:19:08.830123+0000) 2024-05-14T17:19:19.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:18 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:18.967+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:19.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:18 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:18.967+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:19.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:18 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:18.967+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:19.493 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:19 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:19.137+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:39.332351+0000 front 2024-05-14T17:18:39.332307+0000 (oldest deadline 2024-05-14T17:19:02.733193+0000) 2024-05-14T17:19:19.493 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:19 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:19.137+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:44.935082+0000 front 2024-05-14T17:18:44.935193+0000 (oldest deadline 2024-05-14T17:19:08.435222+0000) 2024-05-14T17:19:19.493 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:19 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:19.137+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:52.738851+0000 front 2024-05-14T17:18:52.738798+0000 (oldest deadline 2024-05-14T17:19:16.738960+0000) 2024-05-14T17:19:19.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:19 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:19.229+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.353156+0000 front 2024-05-14T17:18:45.352963+0000 (oldest deadline 2024-05-14T17:19:08.753619+0000) 2024-05-14T17:19:19.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:19 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:19.263+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:19.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:19 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:19.263+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:19.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:19 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:19.263+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:19.993 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:19 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:19.735+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:19.993 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:19 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:19.735+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:19.993 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:19 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:19.735+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.345149+0000 front 2024-05-14T17:18:55.344987+0000 (oldest deadline 2024-05-14T17:19:17.644999+0000) 2024-05-14T17:19:20.159 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:19 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:19.683+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:43.134568+0000 front 2024-05-14T17:18:43.134641+0000 (oldest deadline 2024-05-14T17:19:09.034680+0000) 2024-05-14T17:19:20.159 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:19 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:19.683+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.436396+0000 front 2024-05-14T17:18:55.436310+0000 (oldest deadline 2024-05-14T17:19:17.136467+0000) 2024-05-14T17:19:20.159 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:19 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:19.924+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:20.160 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:19 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:19.924+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:20.160 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:19 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:19.924+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:20.195 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:19 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5[47942]: 2024-05-14T17:19:19.900+0000 7f608350c700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:37.828513+0000 front 2024-05-14T17:18:37.828560+0000 (oldest deadline 2024-05-14T17:19:01.328693+0000) 2024-05-14T17:19:20.195 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:19 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5[47942]: 2024-05-14T17:19:19.900+0000 7f608350c700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:44.829832+0000 front 2024-05-14T17:18:44.829751+0000 (oldest deadline 2024-05-14T17:19:08.830123+0000) 2024-05-14T17:19:20.493 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:20 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:20.158+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:39.332351+0000 front 2024-05-14T17:18:39.332307+0000 (oldest deadline 2024-05-14T17:19:02.733193+0000) 2024-05-14T17:19:20.493 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:20 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:20.158+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:44.935082+0000 front 2024-05-14T17:18:44.935193+0000 (oldest deadline 2024-05-14T17:19:08.435222+0000) 2024-05-14T17:19:20.493 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:20 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:20.158+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:52.738851+0000 front 2024-05-14T17:18:52.738798+0000 (oldest deadline 2024-05-14T17:19:16.738960+0000) 2024-05-14T17:19:20.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:20 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:20.196+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.353156+0000 front 2024-05-14T17:18:45.352963+0000 (oldest deadline 2024-05-14T17:19:08.753619+0000) 2024-05-14T17:19:20.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:20 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:20.267+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:20.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:20 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:20.267+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:20.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:20 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:20.267+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:20.909 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:20 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:20.637+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:43.134568+0000 front 2024-05-14T17:18:43.134641+0000 (oldest deadline 2024-05-14T17:19:09.034680+0000) 2024-05-14T17:19:20.909 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:20 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:20.637+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.436396+0000 front 2024-05-14T17:18:55.436310+0000 (oldest deadline 2024-05-14T17:19:17.136467+0000) 2024-05-14T17:19:21.165 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:20 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:20.784+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:21.165 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:20 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:20.784+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:21.165 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:20 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:20.784+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.345149+0000 front 2024-05-14T17:18:55.344987+0000 (oldest deadline 2024-05-14T17:19:17.644999+0000) 2024-05-14T17:19:21.167 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:20 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5[47942]: 2024-05-14T17:19:20.882+0000 7f608350c700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:37.828513+0000 front 2024-05-14T17:18:37.828560+0000 (oldest deadline 2024-05-14T17:19:01.328693+0000) 2024-05-14T17:19:21.167 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:20 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5[47942]: 2024-05-14T17:19:20.882+0000 7f608350c700 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:44.829832+0000 front 2024-05-14T17:18:44.829751+0000 (oldest deadline 2024-05-14T17:19:08.830123+0000) 2024-05-14T17:19:21.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:20 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:20.955+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:21.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:20 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:20.955+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:21.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:20 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:20.955+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:21.483 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:21 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:21.249+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:21.483 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:21 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:21.249+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:21.483 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:21 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:21.249+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:21.484 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:21 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:21.167+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.353156+0000 front 2024-05-14T17:18:45.352963+0000 (oldest deadline 2024-05-14T17:19:08.753619+0000) 2024-05-14T17:19:21.484 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:21 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:21.167+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:52.755324+0000 front 2024-05-14T17:18:52.755350+0000 (oldest deadline 2024-05-14T17:19:20.355956+0000) 2024-05-14T17:19:21.493 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:21 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:21.165+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:39.332351+0000 front 2024-05-14T17:18:39.332307+0000 (oldest deadline 2024-05-14T17:19:02.733193+0000) 2024-05-14T17:19:21.493 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:21 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:21.165+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:44.935082+0000 front 2024-05-14T17:18:44.935193+0000 (oldest deadline 2024-05-14T17:19:08.435222+0000) 2024-05-14T17:19:21.493 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:21 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:21.165+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:52.738851+0000 front 2024-05-14T17:18:52.738798+0000 (oldest deadline 2024-05-14T17:19:16.738960+0000) 2024-05-14T17:19:21.786 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:21 smithi151 podman[77010]: 2024-05-14 17:19:21.484898195 +0000 UTC m=+5.821391261 container died 8175c9a84326c3bb25e3dc2269f696aabb3d6a1216b31f63934721ab4434d7bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-14T17:19:21.909 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:21 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:21.619+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:43.134568+0000 front 2024-05-14T17:18:43.134641+0000 (oldest deadline 2024-05-14T17:19:09.034680+0000) 2024-05-14T17:19:21.909 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:21 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:21.619+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.436396+0000 front 2024-05-14T17:18:55.436310+0000 (oldest deadline 2024-05-14T17:19:17.136467+0000) 2024-05-14T17:19:22.104 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:21 smithi151 podman[77010]: 2024-05-14 17:19:21.903987948 +0000 UTC m=+6.240481055 container cleanup 8175c9a84326c3bb25e3dc2269f696aabb3d6a1216b31f63934721ab4434d7bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T17:19:22.104 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:21 smithi151 bash[77010]: ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5 2024-05-14T17:19:22.128 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:21 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:21.833+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:22.129 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:21 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:21.833+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:22.129 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:21 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:21.833+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.345149+0000 front 2024-05-14T17:18:55.344987+0000 (oldest deadline 2024-05-14T17:19:17.644999+0000) 2024-05-14T17:19:22.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:21 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:21.930+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:22.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:21 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:21.930+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:22.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:21 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:21.930+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:22.493 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:22 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:22.128+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:39.332351+0000 front 2024-05-14T17:18:39.332307+0000 (oldest deadline 2024-05-14T17:19:02.733193+0000) 2024-05-14T17:19:22.493 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:22 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:22.128+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:44.935082+0000 front 2024-05-14T17:18:44.935193+0000 (oldest deadline 2024-05-14T17:19:08.435222+0000) 2024-05-14T17:19:22.493 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:22 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:22.128+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:52.738851+0000 front 2024-05-14T17:18:52.738798+0000 (oldest deadline 2024-05-14T17:19:16.738960+0000) 2024-05-14T17:19:22.527 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:22 smithi151 podman[77025]: 2024-05-14 17:19:22.105961235 +0000 UTC m=+0.617602359 container remove 8175c9a84326c3bb25e3dc2269f696aabb3d6a1216b31f63934721ab4434d7bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-14T17:19:22.527 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:22 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:22.192+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.353156+0000 front 2024-05-14T17:18:45.352963+0000 (oldest deadline 2024-05-14T17:19:08.753619+0000) 2024-05-14T17:19:22.527 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:22 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:22.192+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:52.755324+0000 front 2024-05-14T17:18:52.755350+0000 (oldest deadline 2024-05-14T17:19:20.355956+0000) 2024-05-14T17:19:22.528 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:22 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:22.217+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:22.528 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:22 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:22.217+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:22.528 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:22 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:22.217+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:22.809 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:22 smithi151 podman[77113]: 2024-05-14 17:19:22.431015543 +0000 UTC m=+0.017279858 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:19:22.809 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:22 smithi151 podman[77113]: 2024-05-14 17:19:22.559195248 +0000 UTC m=+0.145459594 container create 77d259420cb2e7a258353ea3bbd5ef2e836cf8a8aeb6044aa310df9aa9f8ad9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux ) 2024-05-14T17:19:22.809 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:22 smithi151 podman[77113]: 2024-05-14 17:19:22.810535766 +0000 UTC m=+0.396800063 container init 77d259420cb2e7a258353ea3bbd5ef2e836cf8a8aeb6044aa310df9aa9f8ad9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-14T17:19:22.901 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:22 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:22.624+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:43.134568+0000 front 2024-05-14T17:18:43.134641+0000 (oldest deadline 2024-05-14T17:19:09.034680+0000) 2024-05-14T17:19:22.901 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:22 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:22.624+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.436396+0000 front 2024-05-14T17:18:55.436310+0000 (oldest deadline 2024-05-14T17:19:17.136467+0000) 2024-05-14T17:19:23.095 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:22 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:22.832+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:23.095 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:22 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:22.832+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:23.095 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:22 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:22.832+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.345149+0000 front 2024-05-14T17:18:55.344987+0000 (oldest deadline 2024-05-14T17:19:17.644999+0000) 2024-05-14T17:19:23.104 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:22 smithi151 podman[77113]: 2024-05-14 17:19:22.813922479 +0000 UTC m=+0.400186777 container start 77d259420cb2e7a258353ea3bbd5ef2e836cf8a8aeb6044aa310df9aa9f8ad9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-14T17:19:23.104 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:22 smithi151 podman[77113]: 2024-05-14 17:19:22.910829024 +0000 UTC m=+0.497093372 container attach 77d259420cb2e7a258353ea3bbd5ef2e836cf8a8aeb6044aa310df9aa9f8ad9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, ceph=True) 2024-05-14T17:19:23.159 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:22 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:22.902+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:23.160 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:22 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:22.902+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:23.160 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:22 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:22.902+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:23.431 INFO:journalctl@ceph.osd.5.smithi151.stdout:May 14 17:19:23 smithi151 podman[77113]: 2024-05-14 17:19:23.112238808 +0000 UTC m=+0.698503110 container died 77d259420cb2e7a258353ea3bbd5ef2e836cf8a8aeb6044aa310df9aa9f8ad9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:19:23.431 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:23 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:23.216+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.353156+0000 front 2024-05-14T17:18:45.352963+0000 (oldest deadline 2024-05-14T17:19:08.753619+0000) 2024-05-14T17:19:23.431 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:23 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:23.216+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:52.755324+0000 front 2024-05-14T17:18:52.755350+0000 (oldest deadline 2024-05-14T17:19:20.355956+0000) 2024-05-14T17:19:23.432 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:23 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:23.265+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:23.432 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:23 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:23.265+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:23.432 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:23 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:23.265+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:23.493 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:23 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:23.095+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:39.332351+0000 front 2024-05-14T17:18:39.332307+0000 (oldest deadline 2024-05-14T17:19:02.733193+0000) 2024-05-14T17:19:23.493 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:23 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:23.095+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:44.935082+0000 front 2024-05-14T17:18:44.935193+0000 (oldest deadline 2024-05-14T17:19:08.435222+0000) 2024-05-14T17:19:23.493 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:23 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:23.095+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:52.738851+0000 front 2024-05-14T17:18:52.738798+0000 (oldest deadline 2024-05-14T17:19:16.738960+0000) 2024-05-14T17:19:23.754 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@osd.5.service' 2024-05-14T17:19:23.794 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:19:23.795 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-14T17:19:23.795 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-14T17:19:23.795 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-ed40717a-1210-11ef-bc98-c7b262605968@osd.6 2024-05-14T17:19:24.112 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:23 smithi002 systemd[1]: Stopping Ceph osd.6 for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T17:19:24.112 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:23 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:23.868+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:24.112 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:23 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:23.868+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:24.112 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:23 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:23.868+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.345149+0000 front 2024-05-14T17:18:55.344987+0000 (oldest deadline 2024-05-14T17:19:17.644999+0000) 2024-05-14T17:19:24.159 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:23 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:23.666+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:43.134568+0000 front 2024-05-14T17:18:43.134641+0000 (oldest deadline 2024-05-14T17:19:09.034680+0000) 2024-05-14T17:19:24.159 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:23 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:23.666+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.436396+0000 front 2024-05-14T17:18:55.436310+0000 (oldest deadline 2024-05-14T17:19:17.136467+0000) 2024-05-14T17:19:24.160 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:23 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:23.900+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:24.160 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:23 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:23.900+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:24.160 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:23 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:23.900+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:24.494 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:24 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:24.112+0000 7fedf0caf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:19:24.494 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:24 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:24.112+0000 7fedf0caf700 -1 osd.6 119 *** Got signal Terminated *** 2024-05-14T17:19:24.494 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:24 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:24.112+0000 7fedf0caf700 -1 osd.6 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T17:19:24.494 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:24 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:24.116+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:39.332351+0000 front 2024-05-14T17:18:39.332307+0000 (oldest deadline 2024-05-14T17:19:02.733193+0000) 2024-05-14T17:19:24.494 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:24 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:24.116+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:44.935082+0000 front 2024-05-14T17:18:44.935193+0000 (oldest deadline 2024-05-14T17:19:08.435222+0000) 2024-05-14T17:19:24.494 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:24 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:24.116+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:52.738851+0000 front 2024-05-14T17:18:52.738798+0000 (oldest deadline 2024-05-14T17:19:16.738960+0000) 2024-05-14T17:19:24.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:24 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:24.254+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.353156+0000 front 2024-05-14T17:18:45.352963+0000 (oldest deadline 2024-05-14T17:19:08.753619+0000) 2024-05-14T17:19:24.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:24 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:24.254+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:52.755324+0000 front 2024-05-14T17:18:52.755350+0000 (oldest deadline 2024-05-14T17:19:20.355956+0000) 2024-05-14T17:19:24.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:24 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:24.217+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:24.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:24 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:24.217+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:24.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:24 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:24.217+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:24.909 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:24 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:24.646+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:43.134568+0000 front 2024-05-14T17:18:43.134641+0000 (oldest deadline 2024-05-14T17:19:09.034680+0000) 2024-05-14T17:19:24.909 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:24 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:24.646+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.436396+0000 front 2024-05-14T17:18:55.436310+0000 (oldest deadline 2024-05-14T17:19:17.136467+0000) 2024-05-14T17:19:25.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:24 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:24.880+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:25.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:24 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:24.880+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:25.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:24 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:24.880+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.345149+0000 front 2024-05-14T17:18:55.344987+0000 (oldest deadline 2024-05-14T17:19:17.644999+0000) 2024-05-14T17:19:25.243 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:25 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:25.110+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:39.332351+0000 front 2024-05-14T17:18:39.332307+0000 (oldest deadline 2024-05-14T17:19:02.733193+0000) 2024-05-14T17:19:25.244 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:25 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:25.110+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:44.935082+0000 front 2024-05-14T17:18:44.935193+0000 (oldest deadline 2024-05-14T17:19:08.435222+0000) 2024-05-14T17:19:25.244 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:25 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:25.110+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:52.738851+0000 front 2024-05-14T17:18:52.738798+0000 (oldest deadline 2024-05-14T17:19:16.738960+0000) 2024-05-14T17:19:25.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:24 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:24.922+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:25.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:24 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:24.922+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:25.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:24 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:24.922+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:25.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:25 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:25.294+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.353156+0000 front 2024-05-14T17:18:45.352963+0000 (oldest deadline 2024-05-14T17:19:08.753619+0000) 2024-05-14T17:19:25.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:25 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:25.294+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:52.755324+0000 front 2024-05-14T17:18:52.755350+0000 (oldest deadline 2024-05-14T17:19:20.355956+0000) 2024-05-14T17:19:25.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:25 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:25.204+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:25.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:25 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:25.204+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:25.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:25 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:25.204+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:25.909 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:25 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:25.644+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:43.134568+0000 front 2024-05-14T17:18:43.134641+0000 (oldest deadline 2024-05-14T17:19:09.034680+0000) 2024-05-14T17:19:25.909 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:25 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:25.644+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.436396+0000 front 2024-05-14T17:18:55.436310+0000 (oldest deadline 2024-05-14T17:19:17.136467+0000) 2024-05-14T17:19:26.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:25 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:25.894+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:26.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:25 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:25.894+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:26.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:25 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:25.894+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.345149+0000 front 2024-05-14T17:18:55.344987+0000 (oldest deadline 2024-05-14T17:19:17.644999+0000) 2024-05-14T17:19:26.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:25 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:25.894+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.345784+0000 front 2024-05-14T17:19:02.345930+0000 (oldest deadline 2024-05-14T17:19:25.245909+0000) 2024-05-14T17:19:26.244 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:26 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:26.107+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:39.332351+0000 front 2024-05-14T17:18:39.332307+0000 (oldest deadline 2024-05-14T17:19:02.733193+0000) 2024-05-14T17:19:26.244 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:26 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:26.107+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:44.935082+0000 front 2024-05-14T17:18:44.935193+0000 (oldest deadline 2024-05-14T17:19:08.435222+0000) 2024-05-14T17:19:26.244 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:26 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:26.107+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:52.738851+0000 front 2024-05-14T17:18:52.738798+0000 (oldest deadline 2024-05-14T17:19:16.738960+0000) 2024-05-14T17:19:26.244 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:26 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:26.107+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.239789+0000 front 2024-05-14T17:19:00.239748+0000 (oldest deadline 2024-05-14T17:19:25.440343+0000) 2024-05-14T17:19:26.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:25 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:25.956+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:26.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:25 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:25.956+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:26.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:25 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:25.956+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:26.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:25 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:25.956+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.543221+0000 front 2024-05-14T17:19:00.543205+0000 (oldest deadline 2024-05-14T17:19:25.243108+0000) 2024-05-14T17:19:26.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:26 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:26.180+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:26.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:26 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:26.180+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:26.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:26 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:26.180+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:26.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:26 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:26.276+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.353156+0000 front 2024-05-14T17:18:45.352963+0000 (oldest deadline 2024-05-14T17:19:08.753619+0000) 2024-05-14T17:19:26.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:26 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:26.276+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:52.755324+0000 front 2024-05-14T17:18:52.755350+0000 (oldest deadline 2024-05-14T17:19:20.355956+0000) 2024-05-14T17:19:26.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:26 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:26.276+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.056951+0000 front 2024-05-14T17:19:02.057040+0000 (oldest deadline 2024-05-14T17:19:26.156826+0000) 2024-05-14T17:19:26.909 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:26 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:26.632+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:43.134568+0000 front 2024-05-14T17:18:43.134641+0000 (oldest deadline 2024-05-14T17:19:09.034680+0000) 2024-05-14T17:19:26.909 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:26 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:26.633+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.436396+0000 front 2024-05-14T17:18:55.436310+0000 (oldest deadline 2024-05-14T17:19:17.136467+0000) 2024-05-14T17:19:27.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:26 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:26.914+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:27.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:26 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:26.914+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:27.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:26 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:26.914+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.345149+0000 front 2024-05-14T17:18:55.344987+0000 (oldest deadline 2024-05-14T17:19:17.644999+0000) 2024-05-14T17:19:27.244 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:26 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:26.914+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.345784+0000 front 2024-05-14T17:19:02.345930+0000 (oldest deadline 2024-05-14T17:19:25.245909+0000) 2024-05-14T17:19:27.244 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:27 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:27.153+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:39.332351+0000 front 2024-05-14T17:18:39.332307+0000 (oldest deadline 2024-05-14T17:19:02.733193+0000) 2024-05-14T17:19:27.244 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:27 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:27.153+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:44.935082+0000 front 2024-05-14T17:18:44.935193+0000 (oldest deadline 2024-05-14T17:19:08.435222+0000) 2024-05-14T17:19:27.244 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:27 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:27.153+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:52.738851+0000 front 2024-05-14T17:18:52.738798+0000 (oldest deadline 2024-05-14T17:19:16.738960+0000) 2024-05-14T17:19:27.244 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:27 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:27.153+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.239789+0000 front 2024-05-14T17:19:00.239748+0000 (oldest deadline 2024-05-14T17:19:25.440343+0000) 2024-05-14T17:19:27.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:26 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:26.959+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:27.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:26 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:26.959+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:27.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:26 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:26.959+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:27.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:26 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:26.959+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.543221+0000 front 2024-05-14T17:19:00.543205+0000 (oldest deadline 2024-05-14T17:19:25.243108+0000) 2024-05-14T17:19:27.505 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:27 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:27.278+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.353156+0000 front 2024-05-14T17:18:45.352963+0000 (oldest deadline 2024-05-14T17:19:08.753619+0000) 2024-05-14T17:19:27.505 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:27 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:27.278+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:52.755324+0000 front 2024-05-14T17:18:52.755350+0000 (oldest deadline 2024-05-14T17:19:20.355956+0000) 2024-05-14T17:19:27.505 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:27 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:27.278+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.056951+0000 front 2024-05-14T17:19:02.057040+0000 (oldest deadline 2024-05-14T17:19:26.156826+0000) 2024-05-14T17:19:27.506 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:27 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:27.176+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:27.506 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:27 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:27.176+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:27.506 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:27 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:27.176+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:27.506 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:27 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:27.176+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:19:27.910 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:27 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:27.656+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:43.134568+0000 front 2024-05-14T17:18:43.134641+0000 (oldest deadline 2024-05-14T17:19:09.034680+0000) 2024-05-14T17:19:27.910 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:27 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:27.656+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.436396+0000 front 2024-05-14T17:18:55.436310+0000 (oldest deadline 2024-05-14T17:19:17.136467+0000) 2024-05-14T17:19:28.157 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:27 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:27.884+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:28.157 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:27 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:27.884+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:28.157 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:27 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:27.884+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.345149+0000 front 2024-05-14T17:18:55.344987+0000 (oldest deadline 2024-05-14T17:19:17.644999+0000) 2024-05-14T17:19:28.157 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:27 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:27.884+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.345784+0000 front 2024-05-14T17:19:02.345930+0000 (oldest deadline 2024-05-14T17:19:25.245909+0000) 2024-05-14T17:19:28.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:27 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:27.915+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:28.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:27 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:27.915+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:28.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:27 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:27.915+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:28.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:27 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:27.915+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.543221+0000 front 2024-05-14T17:19:00.543205+0000 (oldest deadline 2024-05-14T17:19:25.243108+0000) 2024-05-14T17:19:28.493 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:28 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:28.158+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:39.332351+0000 front 2024-05-14T17:18:39.332307+0000 (oldest deadline 2024-05-14T17:19:02.733193+0000) 2024-05-14T17:19:28.493 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:28 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:28.158+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:44.935082+0000 front 2024-05-14T17:18:44.935193+0000 (oldest deadline 2024-05-14T17:19:08.435222+0000) 2024-05-14T17:19:28.493 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:28 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:28.158+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:52.738851+0000 front 2024-05-14T17:18:52.738798+0000 (oldest deadline 2024-05-14T17:19:16.738960+0000) 2024-05-14T17:19:28.493 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:28 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6[59787]: 2024-05-14T17:19:28.158+0000 7fede929b700 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.239789+0000 front 2024-05-14T17:19:00.239748+0000 (oldest deadline 2024-05-14T17:19:25.440343+0000) 2024-05-14T17:19:28.522 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:28 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:28.173+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:28.522 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:28 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:28.173+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:28.522 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:28 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:28.173+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:28.522 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:28 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:28.173+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:19:28.522 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:28 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:28.241+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.353156+0000 front 2024-05-14T17:18:45.352963+0000 (oldest deadline 2024-05-14T17:19:08.753619+0000) 2024-05-14T17:19:28.522 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:28 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:28.241+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:52.755324+0000 front 2024-05-14T17:18:52.755350+0000 (oldest deadline 2024-05-14T17:19:20.355956+0000) 2024-05-14T17:19:28.522 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:28 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:28.241+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.056951+0000 front 2024-05-14T17:19:02.057040+0000 (oldest deadline 2024-05-14T17:19:26.156826+0000) 2024-05-14T17:19:28.909 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:28 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:28.614+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:43.134568+0000 front 2024-05-14T17:18:43.134641+0000 (oldest deadline 2024-05-14T17:19:09.034680+0000) 2024-05-14T17:19:28.909 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:28 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:28.614+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.436396+0000 front 2024-05-14T17:18:55.436310+0000 (oldest deadline 2024-05-14T17:19:17.136467+0000) 2024-05-14T17:19:29.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:28 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:28.926+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:29.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:28 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:28.926+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:29.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:28 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:28.926+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.345149+0000 front 2024-05-14T17:18:55.344987+0000 (oldest deadline 2024-05-14T17:19:17.644999+0000) 2024-05-14T17:19:29.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:28 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:28.926+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.345784+0000 front 2024-05-14T17:19:02.345930+0000 (oldest deadline 2024-05-14T17:19:25.245909+0000) 2024-05-14T17:19:29.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:28 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:28.913+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:29.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:28 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:28.913+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:29.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:28 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:28.913+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:29.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:28 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:28.913+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.543221+0000 front 2024-05-14T17:19:00.543205+0000 (oldest deadline 2024-05-14T17:19:25.243108+0000) 2024-05-14T17:19:29.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:29 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:29.247+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.353156+0000 front 2024-05-14T17:18:45.352963+0000 (oldest deadline 2024-05-14T17:19:08.753619+0000) 2024-05-14T17:19:29.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:29 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:29.247+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:52.755324+0000 front 2024-05-14T17:18:52.755350+0000 (oldest deadline 2024-05-14T17:19:20.355956+0000) 2024-05-14T17:19:29.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:29 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:29.247+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.056951+0000 front 2024-05-14T17:19:02.057040+0000 (oldest deadline 2024-05-14T17:19:26.156826+0000) 2024-05-14T17:19:29.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:29 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:29.171+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:29.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:29 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:29.171+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:29.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:29 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:29.171+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:29.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:29 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:29.171+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:19:29.865 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:29 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:29.577+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:43.134568+0000 front 2024-05-14T17:18:43.134641+0000 (oldest deadline 2024-05-14T17:19:09.034680+0000) 2024-05-14T17:19:29.866 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:29 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:29.577+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.436396+0000 front 2024-05-14T17:18:55.436310+0000 (oldest deadline 2024-05-14T17:19:17.136467+0000) 2024-05-14T17:19:29.866 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:29 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:29.577+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.937925+0000 front 2024-05-14T17:19:02.937870+0000 (oldest deadline 2024-05-14T17:19:28.837834+0000) 2024-05-14T17:19:29.962 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:29 smithi002 podman[109641]: 2024-05-14 17:19:29.670377135 +0000 UTC m=+5.667290108 container died 81f7195a86f08a3f1d572d2191f753a774c5fca70b50bff8d0a896f9b0c0b1a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-14T17:19:29.963 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:29 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:29.880+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:29.963 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:29 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:29.880+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:29.963 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:29 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:29.880+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.345149+0000 front 2024-05-14T17:18:55.344987+0000 (oldest deadline 2024-05-14T17:19:17.644999+0000) 2024-05-14T17:19:29.963 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:29 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:29.880+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.345784+0000 front 2024-05-14T17:19:02.345930+0000 (oldest deadline 2024-05-14T17:19:25.245909+0000) 2024-05-14T17:19:30.159 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:29 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:29.865+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:30.159 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:29 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:29.865+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:30.159 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:29 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:29.865+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:30.159 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:29 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:29.865+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.543221+0000 front 2024-05-14T17:19:00.543205+0000 (oldest deadline 2024-05-14T17:19:25.243108+0000) 2024-05-14T17:19:30.243 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:30 smithi002 podman[109641]: 2024-05-14 17:19:30.063332376 +0000 UTC m=+6.060245367 container cleanup 81f7195a86f08a3f1d572d2191f753a774c5fca70b50bff8d0a896f9b0c0b1a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True) 2024-05-14T17:19:30.243 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:30 smithi002 bash[109641]: ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6 2024-05-14T17:19:30.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:30 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:30.284+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.353156+0000 front 2024-05-14T17:18:45.352963+0000 (oldest deadline 2024-05-14T17:19:08.753619+0000) 2024-05-14T17:19:30.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:30 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:30.284+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:52.755324+0000 front 2024-05-14T17:18:52.755350+0000 (oldest deadline 2024-05-14T17:19:20.355956+0000) 2024-05-14T17:19:30.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:30 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:30.284+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.056951+0000 front 2024-05-14T17:19:02.057040+0000 (oldest deadline 2024-05-14T17:19:26.156826+0000) 2024-05-14T17:19:30.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:30 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:30.180+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:30.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:30 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:30.180+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:30.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:30 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:30.180+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:30.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:30 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:30.180+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:19:30.696 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:30 smithi002 podman[109656]: 2024-05-14 17:19:30.35131506 +0000 UTC m=+0.682811651 container remove 81f7195a86f08a3f1d572d2191f753a774c5fca70b50bff8d0a896f9b0c0b1a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-14T17:19:30.905 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:30 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:30.568+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:43.134568+0000 front 2024-05-14T17:18:43.134641+0000 (oldest deadline 2024-05-14T17:19:09.034680+0000) 2024-05-14T17:19:30.905 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:30 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:30.568+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.436396+0000 front 2024-05-14T17:18:55.436310+0000 (oldest deadline 2024-05-14T17:19:17.136467+0000) 2024-05-14T17:19:30.905 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:30 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:30.568+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.937925+0000 front 2024-05-14T17:19:02.937870+0000 (oldest deadline 2024-05-14T17:19:28.837834+0000) 2024-05-14T17:19:30.993 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:30 smithi002 podman[109747]: 2024-05-14 17:19:30.601155266 +0000 UTC m=+0.017970760 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:19:30.993 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:30 smithi002 podman[109747]: 2024-05-14 17:19:30.724031945 +0000 UTC m=+0.140847473 container create b10b9c97bdc0e5996046396b78f74c1a19bbf9c81ab1cfa9eda7698cb9b8fd07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-14T17:19:30.994 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:30 smithi002 podman[109747]: 2024-05-14 17:19:30.943107135 +0000 UTC m=+0.359922625 container init b10b9c97bdc0e5996046396b78f74c1a19bbf9c81ab1cfa9eda7698cb9b8fd07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, ceph=True) 2024-05-14T17:19:30.994 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:30 smithi002 podman[109747]: 2024-05-14 17:19:30.946489089 +0000 UTC m=+0.363304576 container start b10b9c97bdc0e5996046396b78f74c1a19bbf9c81ab1cfa9eda7698cb9b8fd07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, ceph=True) 2024-05-14T17:19:30.994 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:30 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:30.866+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:30.994 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:30 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:30.866+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:30.994 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:30 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:30.866+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.345149+0000 front 2024-05-14T17:18:55.344987+0000 (oldest deadline 2024-05-14T17:19:17.644999+0000) 2024-05-14T17:19:30.994 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:30 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:30.866+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.345784+0000 front 2024-05-14T17:19:02.345930+0000 (oldest deadline 2024-05-14T17:19:25.245909+0000) 2024-05-14T17:19:31.159 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:30 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:30.906+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:31.160 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:30 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:30.906+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:31.160 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:30 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:30.906+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:31.160 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:30 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:30.906+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.543221+0000 front 2024-05-14T17:19:00.543205+0000 (oldest deadline 2024-05-14T17:19:25.243108+0000) 2024-05-14T17:19:31.251 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:31 smithi002 podman[109747]: 2024-05-14 17:19:31.03488492 +0000 UTC m=+0.451700404 container attach b10b9c97bdc0e5996046396b78f74c1a19bbf9c81ab1cfa9eda7698cb9b8fd07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T17:19:31.537 INFO:journalctl@ceph.osd.6.smithi002.stdout:May 14 17:19:31 smithi002 podman[109747]: 2024-05-14 17:19:31.252804416 +0000 UTC m=+0.669619903 container died b10b9c97bdc0e5996046396b78f74c1a19bbf9c81ab1cfa9eda7698cb9b8fd07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD) 2024-05-14T17:19:31.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:31 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:31.287+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.353156+0000 front 2024-05-14T17:18:45.352963+0000 (oldest deadline 2024-05-14T17:19:08.753619+0000) 2024-05-14T17:19:31.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:31 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:31.287+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:52.755324+0000 front 2024-05-14T17:18:52.755350+0000 (oldest deadline 2024-05-14T17:19:20.355956+0000) 2024-05-14T17:19:31.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:31 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:31.287+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.056951+0000 front 2024-05-14T17:19:02.057040+0000 (oldest deadline 2024-05-14T17:19:26.156826+0000) 2024-05-14T17:19:31.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:31 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:31.191+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:31.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:31 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:31.191+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:31.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:31 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:31.191+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:31.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:31 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:31.191+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:19:31.852 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@osd.6.service' 2024-05-14T17:19:31.896 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:19:31.897 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-14T17:19:31.897 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-14T17:19:31.897 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-ed40717a-1210-11ef-bc98-c7b262605968@osd.7 2024-05-14T17:19:31.909 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:31 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:31.589+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:43.134568+0000 front 2024-05-14T17:18:43.134641+0000 (oldest deadline 2024-05-14T17:19:09.034680+0000) 2024-05-14T17:19:31.909 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:31 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:31.589+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.436396+0000 front 2024-05-14T17:18:55.436310+0000 (oldest deadline 2024-05-14T17:19:17.136467+0000) 2024-05-14T17:19:31.909 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:31 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:31.589+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.937925+0000 front 2024-05-14T17:19:02.937870+0000 (oldest deadline 2024-05-14T17:19:28.837834+0000) 2024-05-14T17:19:32.195 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:31 smithi123 systemd[1]: Stopping Ceph osd.7 for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T17:19:32.195 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:31 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:31.953+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:32.195 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:31 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:31.953+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:32.195 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:31 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:31.953+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:32.195 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:31 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:31.953+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.543221+0000 front 2024-05-14T17:19:00.543205+0000 (oldest deadline 2024-05-14T17:19:25.243108+0000) 2024-05-14T17:19:32.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:31 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:31.910+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:32.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:31 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:31.910+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:32.244 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:31 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:31.910+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.345149+0000 front 2024-05-14T17:18:55.344987+0000 (oldest deadline 2024-05-14T17:19:17.644999+0000) 2024-05-14T17:19:32.244 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:31 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:31.910+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.345784+0000 front 2024-05-14T17:19:02.345930+0000 (oldest deadline 2024-05-14T17:19:25.245909+0000) 2024-05-14T17:19:32.556 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:32 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:32.367+0000 7fcff1b33700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:19:32.556 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:32 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:32.367+0000 7fcff1b33700 -1 osd.7 119 *** Got signal Terminated *** 2024-05-14T17:19:32.556 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:32 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:32.367+0000 7fcff1b33700 -1 osd.7 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T17:19:32.556 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:32 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:32.555+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:43.134568+0000 front 2024-05-14T17:18:43.134641+0000 (oldest deadline 2024-05-14T17:19:09.034680+0000) 2024-05-14T17:19:32.556 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:32 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:32.555+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.436396+0000 front 2024-05-14T17:18:55.436310+0000 (oldest deadline 2024-05-14T17:19:17.136467+0000) 2024-05-14T17:19:32.556 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:32 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:32.555+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.937925+0000 front 2024-05-14T17:19:02.937870+0000 (oldest deadline 2024-05-14T17:19:28.837834+0000) 2024-05-14T17:19:32.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:32 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:32.334+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.353156+0000 front 2024-05-14T17:18:45.352963+0000 (oldest deadline 2024-05-14T17:19:08.753619+0000) 2024-05-14T17:19:32.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:32 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:32.334+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:52.755324+0000 front 2024-05-14T17:18:52.755350+0000 (oldest deadline 2024-05-14T17:19:20.355956+0000) 2024-05-14T17:19:32.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:32 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:32.334+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.056951+0000 front 2024-05-14T17:19:02.057040+0000 (oldest deadline 2024-05-14T17:19:26.156826+0000) 2024-05-14T17:19:32.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:32 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:32.150+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:32.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:32 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:32.150+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:32.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:32 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:32.150+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:32.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:32 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:32.150+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:19:33.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:32 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:32.900+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:33.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:32 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:32.900+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:33.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:32 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:32.900+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.345149+0000 front 2024-05-14T17:18:55.344987+0000 (oldest deadline 2024-05-14T17:19:17.644999+0000) 2024-05-14T17:19:33.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:32 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:32.900+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.345784+0000 front 2024-05-14T17:19:02.345930+0000 (oldest deadline 2024-05-14T17:19:25.245909+0000) 2024-05-14T17:19:33.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:32 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:32.984+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:33.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:32 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:32.984+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:33.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:32 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:32.984+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:33.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:32 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:32.984+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.543221+0000 front 2024-05-14T17:19:00.543205+0000 (oldest deadline 2024-05-14T17:19:25.243108+0000) 2024-05-14T17:19:33.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:33 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:33.339+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.353156+0000 front 2024-05-14T17:18:45.352963+0000 (oldest deadline 2024-05-14T17:19:08.753619+0000) 2024-05-14T17:19:33.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:33 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:33.339+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:52.755324+0000 front 2024-05-14T17:18:52.755350+0000 (oldest deadline 2024-05-14T17:19:20.355956+0000) 2024-05-14T17:19:33.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:33 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:33.339+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.056951+0000 front 2024-05-14T17:19:02.057040+0000 (oldest deadline 2024-05-14T17:19:26.156826+0000) 2024-05-14T17:19:33.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:33 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:33.170+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:33.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:33 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:33.170+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:33.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:33 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:33.170+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:33.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:33 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:33.170+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:19:33.909 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:33 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:33.546+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:43.134568+0000 front 2024-05-14T17:18:43.134641+0000 (oldest deadline 2024-05-14T17:19:09.034680+0000) 2024-05-14T17:19:33.909 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:33 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:33.546+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.436396+0000 front 2024-05-14T17:18:55.436310+0000 (oldest deadline 2024-05-14T17:19:17.136467+0000) 2024-05-14T17:19:33.909 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:33 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:33.546+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.937925+0000 front 2024-05-14T17:19:02.937870+0000 (oldest deadline 2024-05-14T17:19:28.837834+0000) 2024-05-14T17:19:34.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:33 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:33.928+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:34.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:33 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:33.928+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:34.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:33 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:33.928+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.345149+0000 front 2024-05-14T17:18:55.344987+0000 (oldest deadline 2024-05-14T17:19:17.644999+0000) 2024-05-14T17:19:34.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:33 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:33.928+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.345784+0000 front 2024-05-14T17:19:02.345930+0000 (oldest deadline 2024-05-14T17:19:25.245909+0000) 2024-05-14T17:19:34.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:33 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:33.989+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:34.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:33 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:33.989+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:34.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:33 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:33.989+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:34.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:33 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:33.989+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.543221+0000 front 2024-05-14T17:19:00.543205+0000 (oldest deadline 2024-05-14T17:19:25.243108+0000) 2024-05-14T17:19:34.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:33 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:33.989+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:08.144059+0000 front 2024-05-14T17:19:08.144106+0000 (oldest deadline 2024-05-14T17:19:33.444295+0000) 2024-05-14T17:19:34.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:34 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:34.375+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.353156+0000 front 2024-05-14T17:18:45.352963+0000 (oldest deadline 2024-05-14T17:19:08.753619+0000) 2024-05-14T17:19:34.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:34 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:34.375+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:52.755324+0000 front 2024-05-14T17:18:52.755350+0000 (oldest deadline 2024-05-14T17:19:20.355956+0000) 2024-05-14T17:19:34.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:34 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:34.375+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.056951+0000 front 2024-05-14T17:19:02.057040+0000 (oldest deadline 2024-05-14T17:19:26.156826+0000) 2024-05-14T17:19:34.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:34 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:34.184+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:34.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:34 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:34.184+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:34.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:34 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:34.184+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:34.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:34 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:34.184+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:19:34.909 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:34 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:34.577+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:43.134568+0000 front 2024-05-14T17:18:43.134641+0000 (oldest deadline 2024-05-14T17:19:09.034680+0000) 2024-05-14T17:19:34.909 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:34 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:34.577+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.436396+0000 front 2024-05-14T17:18:55.436310+0000 (oldest deadline 2024-05-14T17:19:17.136467+0000) 2024-05-14T17:19:34.909 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:34 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:34.577+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.937925+0000 front 2024-05-14T17:19:02.937870+0000 (oldest deadline 2024-05-14T17:19:28.837834+0000) 2024-05-14T17:19:34.910 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:34 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:34.577+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:08.838335+0000 front 2024-05-14T17:19:08.838312+0000 (oldest deadline 2024-05-14T17:19:34.138529+0000) 2024-05-14T17:19:35.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:34 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:34.894+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:35.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:34 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:34.894+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:35.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:34 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:34.894+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.345149+0000 front 2024-05-14T17:18:55.344987+0000 (oldest deadline 2024-05-14T17:19:17.644999+0000) 2024-05-14T17:19:35.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:34 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:34.894+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.345784+0000 front 2024-05-14T17:19:02.345930+0000 (oldest deadline 2024-05-14T17:19:25.245909+0000) 2024-05-14T17:19:35.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:34 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:34.979+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:35.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:34 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:34.979+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:35.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:34 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:34.979+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:35.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:34 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:34.979+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.543221+0000 front 2024-05-14T17:19:00.543205+0000 (oldest deadline 2024-05-14T17:19:25.243108+0000) 2024-05-14T17:19:35.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:34 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:34.979+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:08.144059+0000 front 2024-05-14T17:19:08.144106+0000 (oldest deadline 2024-05-14T17:19:33.444295+0000) 2024-05-14T17:19:35.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:35 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:35.354+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.353156+0000 front 2024-05-14T17:18:45.352963+0000 (oldest deadline 2024-05-14T17:19:08.753619+0000) 2024-05-14T17:19:35.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:35 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:35.354+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:52.755324+0000 front 2024-05-14T17:18:52.755350+0000 (oldest deadline 2024-05-14T17:19:20.355956+0000) 2024-05-14T17:19:35.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:35 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:35.354+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.056951+0000 front 2024-05-14T17:19:02.057040+0000 (oldest deadline 2024-05-14T17:19:26.156826+0000) 2024-05-14T17:19:35.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:35 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:35.139+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:35.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:35 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:35.139+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:35.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:35 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:35.139+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:35.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:35 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:35.139+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:19:35.909 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:35 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:35.545+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:43.134568+0000 front 2024-05-14T17:18:43.134641+0000 (oldest deadline 2024-05-14T17:19:09.034680+0000) 2024-05-14T17:19:35.909 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:35 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:35.545+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.436396+0000 front 2024-05-14T17:18:55.436310+0000 (oldest deadline 2024-05-14T17:19:17.136467+0000) 2024-05-14T17:19:35.909 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:35 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:35.545+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.937925+0000 front 2024-05-14T17:19:02.937870+0000 (oldest deadline 2024-05-14T17:19:28.837834+0000) 2024-05-14T17:19:35.909 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:35 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:35.545+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:08.838335+0000 front 2024-05-14T17:19:08.838312+0000 (oldest deadline 2024-05-14T17:19:34.138529+0000) 2024-05-14T17:19:36.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:35 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:35.863+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:36.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:35 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:35.863+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:36.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:35 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:35.863+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.345149+0000 front 2024-05-14T17:18:55.344987+0000 (oldest deadline 2024-05-14T17:19:17.644999+0000) 2024-05-14T17:19:36.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:35 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:35.863+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.345784+0000 front 2024-05-14T17:19:02.345930+0000 (oldest deadline 2024-05-14T17:19:25.245909+0000) 2024-05-14T17:19:36.382 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:36 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:36.108+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:36.382 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:36 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:36.108+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:36.382 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:36 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:36.108+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:36.382 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:36 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:36.108+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:19:36.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:36 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:36.007+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:36.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:36 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:36.007+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:36.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:36 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:36.007+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:36.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:36 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:36.007+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.543221+0000 front 2024-05-14T17:19:00.543205+0000 (oldest deadline 2024-05-14T17:19:25.243108+0000) 2024-05-14T17:19:36.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:36 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:36.007+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:08.144059+0000 front 2024-05-14T17:19:08.144106+0000 (oldest deadline 2024-05-14T17:19:33.444295+0000) 2024-05-14T17:19:36.854 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:36 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:36.382+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.353156+0000 front 2024-05-14T17:18:45.352963+0000 (oldest deadline 2024-05-14T17:19:08.753619+0000) 2024-05-14T17:19:36.854 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:36 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:36.382+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:52.755324+0000 front 2024-05-14T17:18:52.755350+0000 (oldest deadline 2024-05-14T17:19:20.355956+0000) 2024-05-14T17:19:36.854 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:36 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:36.382+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.056951+0000 front 2024-05-14T17:19:02.057040+0000 (oldest deadline 2024-05-14T17:19:26.156826+0000) 2024-05-14T17:19:36.909 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:36 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:36.519+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:43.134568+0000 front 2024-05-14T17:18:43.134641+0000 (oldest deadline 2024-05-14T17:19:09.034680+0000) 2024-05-14T17:19:36.910 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:36 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:36.519+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.436396+0000 front 2024-05-14T17:18:55.436310+0000 (oldest deadline 2024-05-14T17:19:17.136467+0000) 2024-05-14T17:19:36.910 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:36 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:36.519+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.937925+0000 front 2024-05-14T17:19:02.937870+0000 (oldest deadline 2024-05-14T17:19:28.837834+0000) 2024-05-14T17:19:36.910 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:36 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7[54329]: 2024-05-14T17:19:36.519+0000 7fcfea11f700 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:08.838335+0000 front 2024-05-14T17:19:08.838312+0000 (oldest deadline 2024-05-14T17:19:34.138529+0000) 2024-05-14T17:19:37.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:36 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:36.879+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:37.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:36 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:36.879+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:37.244 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:36 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:36.879+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.345149+0000 front 2024-05-14T17:18:55.344987+0000 (oldest deadline 2024-05-14T17:19:17.644999+0000) 2024-05-14T17:19:37.244 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:36 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:36.879+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.345784+0000 front 2024-05-14T17:19:02.345930+0000 (oldest deadline 2024-05-14T17:19:25.245909+0000) 2024-05-14T17:19:37.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:37 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:37.039+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:37.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:37 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:37.039+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:37.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:37 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:37.039+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:37.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:37 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:37.039+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.543221+0000 front 2024-05-14T17:19:00.543205+0000 (oldest deadline 2024-05-14T17:19:25.243108+0000) 2024-05-14T17:19:37.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:37 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:37.039+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:08.144059+0000 front 2024-05-14T17:19:08.144106+0000 (oldest deadline 2024-05-14T17:19:33.444295+0000) 2024-05-14T17:19:37.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:37 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:37.156+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:37.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:37 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:37.156+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:37.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:37 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:37.156+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:37.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:37 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:37.156+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:19:37.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:37 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:37.352+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.353156+0000 front 2024-05-14T17:18:45.352963+0000 (oldest deadline 2024-05-14T17:19:08.753619+0000) 2024-05-14T17:19:37.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:37 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:37.352+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:52.755324+0000 front 2024-05-14T17:18:52.755350+0000 (oldest deadline 2024-05-14T17:19:20.355956+0000) 2024-05-14T17:19:37.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:37 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:37.352+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.056951+0000 front 2024-05-14T17:19:02.057040+0000 (oldest deadline 2024-05-14T17:19:26.156826+0000) 2024-05-14T17:19:38.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:37 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:37.900+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:38.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:37 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:37.900+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:38.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:37 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:37.900+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.345149+0000 front 2024-05-14T17:18:55.344987+0000 (oldest deadline 2024-05-14T17:19:17.644999+0000) 2024-05-14T17:19:38.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:37 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:37.900+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.345784+0000 front 2024-05-14T17:19:02.345930+0000 (oldest deadline 2024-05-14T17:19:25.245909+0000) 2024-05-14T17:19:38.276 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:37 smithi123 podman[74970]: 2024-05-14 17:19:37.992124339 +0000 UTC m=+5.905077458 container died 375d9d2ed387cca9897dbb1d9db49cb0b2ec32bc5926ab670d346a9840b2bf31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True) 2024-05-14T17:19:38.307 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:38 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:38.079+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:38.307 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:38 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:38.079+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:38.307 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:38 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:38.079+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:38.307 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:38 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:38.079+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.543221+0000 front 2024-05-14T17:19:00.543205+0000 (oldest deadline 2024-05-14T17:19:25.243108+0000) 2024-05-14T17:19:38.307 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:38 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:38.079+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:08.144059+0000 front 2024-05-14T17:19:08.144106+0000 (oldest deadline 2024-05-14T17:19:33.444295+0000) 2024-05-14T17:19:38.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:38 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:38.319+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.353156+0000 front 2024-05-14T17:18:45.352963+0000 (oldest deadline 2024-05-14T17:19:08.753619+0000) 2024-05-14T17:19:38.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:38 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:38.319+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:52.755324+0000 front 2024-05-14T17:18:52.755350+0000 (oldest deadline 2024-05-14T17:19:20.355956+0000) 2024-05-14T17:19:38.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:38 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:38.319+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.056951+0000 front 2024-05-14T17:19:02.057040+0000 (oldest deadline 2024-05-14T17:19:26.156826+0000) 2024-05-14T17:19:38.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:38 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:38.319+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:10.658759+0000 front 2024-05-14T17:19:10.658746+0000 (oldest deadline 2024-05-14T17:19:37.659130+0000) 2024-05-14T17:19:38.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:38 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:38.188+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:38.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:38 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:38.188+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:38.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:38 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:38.188+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:38.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:38 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:38.188+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:19:38.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:38 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:38.188+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:11.427850+0000 front 2024-05-14T17:19:11.427867+0000 (oldest deadline 2024-05-14T17:19:37.327714+0000) 2024-05-14T17:19:38.659 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:38 smithi123 podman[74970]: 2024-05-14 17:19:38.376433113 +0000 UTC m=+6.289386232 container cleanup 375d9d2ed387cca9897dbb1d9db49cb0b2ec32bc5926ab670d346a9840b2bf31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-14T17:19:38.659 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:38 smithi123 bash[74970]: ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7 2024-05-14T17:19:38.994 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:38 smithi123 podman[74996]: 2024-05-14 17:19:38.66974466 +0000 UTC m=+0.676818423 container remove 375d9d2ed387cca9897dbb1d9db49cb0b2ec32bc5926ab670d346a9840b2bf31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-14T17:19:39.158 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:38 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:38.898+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:39.159 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:38 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:38.898+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:39.159 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:38 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:38.898+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.345149+0000 front 2024-05-14T17:18:55.344987+0000 (oldest deadline 2024-05-14T17:19:17.644999+0000) 2024-05-14T17:19:39.159 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:38 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:38.898+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.345784+0000 front 2024-05-14T17:19:02.345930+0000 (oldest deadline 2024-05-14T17:19:25.245909+0000) 2024-05-14T17:19:39.304 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:39 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:39.123+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:39.305 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:39 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:39.123+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:39.305 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:39 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:39.123+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:39.305 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:39 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:39.123+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.543221+0000 front 2024-05-14T17:19:00.543205+0000 (oldest deadline 2024-05-14T17:19:25.243108+0000) 2024-05-14T17:19:39.305 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:39 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:39.123+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:08.144059+0000 front 2024-05-14T17:19:08.144106+0000 (oldest deadline 2024-05-14T17:19:33.444295+0000) 2024-05-14T17:19:39.305 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:38 smithi123 podman[75086]: 2024-05-14 17:19:38.898075837 +0000 UTC m=+0.017066234 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:19:39.305 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:39 smithi123 podman[75086]: 2024-05-14 17:19:39.020555309 +0000 UTC m=+0.139545730 container create 6ef989d92490fd1624610891ccafbfde3799f9392869098f5d29664c6bb762d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-14T17:19:39.305 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:39 smithi123 podman[75086]: 2024-05-14 17:19:39.231158484 +0000 UTC m=+0.350148863 container init 6ef989d92490fd1624610891ccafbfde3799f9392869098f5d29664c6bb762d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-14T17:19:39.305 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:39 smithi123 podman[75086]: 2024-05-14 17:19:39.234153971 +0000 UTC m=+0.353144350 container start 6ef989d92490fd1624610891ccafbfde3799f9392869098f5d29664c6bb762d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-14T17:19:39.306 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:39 smithi123 podman[75086]: 2024-05-14 17:19:39.306053134 +0000 UTC m=+0.425043526 container attach 6ef989d92490fd1624610891ccafbfde3799f9392869098f5d29664c6bb762d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-14T17:19:39.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:39 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:39.281+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.353156+0000 front 2024-05-14T17:18:45.352963+0000 (oldest deadline 2024-05-14T17:19:08.753619+0000) 2024-05-14T17:19:39.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:39 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:39.281+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:52.755324+0000 front 2024-05-14T17:18:52.755350+0000 (oldest deadline 2024-05-14T17:19:20.355956+0000) 2024-05-14T17:19:39.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:39 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:39.281+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.056951+0000 front 2024-05-14T17:19:02.057040+0000 (oldest deadline 2024-05-14T17:19:26.156826+0000) 2024-05-14T17:19:39.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:39 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:39.281+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:10.658759+0000 front 2024-05-14T17:19:10.658746+0000 (oldest deadline 2024-05-14T17:19:37.659130+0000) 2024-05-14T17:19:39.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:39 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:39.192+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:39.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:39 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:39.192+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:39.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:39 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:39.192+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:39.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:39 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:39.192+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:19:39.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:39 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:39.192+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:11.427850+0000 front 2024-05-14T17:19:11.427867+0000 (oldest deadline 2024-05-14T17:19:37.327714+0000) 2024-05-14T17:19:39.659 INFO:journalctl@ceph.osd.7.smithi123.stdout:May 14 17:19:39 smithi123 podman[75086]: 2024-05-14 17:19:39.520184402 +0000 UTC m=+0.639174789 container died 6ef989d92490fd1624610891ccafbfde3799f9392869098f5d29664c6bb762d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240514, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-14T17:19:40.158 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:40 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:40.113+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:40.159 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:40 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:40.113+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:40.159 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:40 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:40.113+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:40.159 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:40 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:40.113+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.543221+0000 front 2024-05-14T17:19:00.543205+0000 (oldest deadline 2024-05-14T17:19:25.243108+0000) 2024-05-14T17:19:40.159 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:40 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:40.113+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:08.144059+0000 front 2024-05-14T17:19:08.144106+0000 (oldest deadline 2024-05-14T17:19:33.444295+0000) 2024-05-14T17:19:40.184 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@osd.7.service' 2024-05-14T17:19:40.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:39 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:39.941+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:40.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:39 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:39.941+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:40.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:39 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:39.941+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.345149+0000 front 2024-05-14T17:18:55.344987+0000 (oldest deadline 2024-05-14T17:19:17.644999+0000) 2024-05-14T17:19:40.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:39 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:39.941+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.345784+0000 front 2024-05-14T17:19:02.345930+0000 (oldest deadline 2024-05-14T17:19:25.245909+0000) 2024-05-14T17:19:40.265 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:19:40.265 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-14T17:19:40.266 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-14T17:19:40.266 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-ed40717a-1210-11ef-bc98-c7b262605968@osd.8 2024-05-14T17:19:40.585 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:40 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:40.196+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:40.585 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:40 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:40.196+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:40.585 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:40 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:40.196+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:40.585 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:40 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:40.196+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:19:40.586 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:40 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:40.196+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:11.427850+0000 front 2024-05-14T17:19:11.427867+0000 (oldest deadline 2024-05-14T17:19:37.327714+0000) 2024-05-14T17:19:40.586 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:40 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:40.249+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.353156+0000 front 2024-05-14T17:18:45.352963+0000 (oldest deadline 2024-05-14T17:19:08.753619+0000) 2024-05-14T17:19:40.586 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:40 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:40.249+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:52.755324+0000 front 2024-05-14T17:18:52.755350+0000 (oldest deadline 2024-05-14T17:19:20.355956+0000) 2024-05-14T17:19:40.586 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:40 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:40.249+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.056951+0000 front 2024-05-14T17:19:02.057040+0000 (oldest deadline 2024-05-14T17:19:26.156826+0000) 2024-05-14T17:19:40.586 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:40 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:40.249+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:10.658759+0000 front 2024-05-14T17:19:10.658746+0000 (oldest deadline 2024-05-14T17:19:37.659130+0000) 2024-05-14T17:19:40.586 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:40 smithi151 systemd[1]: Stopping Ceph osd.8 for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T17:19:40.586 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:40 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:40.585+0000 7f7fd4657700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:19:40.586 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:40 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:40.585+0000 7f7fd4657700 -1 osd.8 119 *** Got signal Terminated *** 2024-05-14T17:19:40.586 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:40 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:40.585+0000 7f7fd4657700 -1 osd.8 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T17:19:41.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:40 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:40.931+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:41.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:40 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:40.931+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:41.244 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:40 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:40.931+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.345149+0000 front 2024-05-14T17:18:55.344987+0000 (oldest deadline 2024-05-14T17:19:17.644999+0000) 2024-05-14T17:19:41.244 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:40 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:40.931+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.345784+0000 front 2024-05-14T17:19:02.345930+0000 (oldest deadline 2024-05-14T17:19:25.245909+0000) 2024-05-14T17:19:41.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:41 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:41.094+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:41.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:41 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:41.094+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:41.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:41 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:41.094+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:41.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:41 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:41.094+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.543221+0000 front 2024-05-14T17:19:00.543205+0000 (oldest deadline 2024-05-14T17:19:25.243108+0000) 2024-05-14T17:19:41.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:41 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:41.094+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:08.144059+0000 front 2024-05-14T17:19:08.144106+0000 (oldest deadline 2024-05-14T17:19:33.444295+0000) 2024-05-14T17:19:41.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:41 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:41.171+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:41.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:41 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:41.171+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:41.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:41 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:41.171+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:41.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:41 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:41.171+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:19:41.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:41 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:41.171+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:11.427850+0000 front 2024-05-14T17:19:11.427867+0000 (oldest deadline 2024-05-14T17:19:37.327714+0000) 2024-05-14T17:19:41.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:41 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:41.216+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.353156+0000 front 2024-05-14T17:18:45.352963+0000 (oldest deadline 2024-05-14T17:19:08.753619+0000) 2024-05-14T17:19:41.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:41 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:41.216+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:52.755324+0000 front 2024-05-14T17:18:52.755350+0000 (oldest deadline 2024-05-14T17:19:20.355956+0000) 2024-05-14T17:19:41.605 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:41 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:41.216+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.056951+0000 front 2024-05-14T17:19:02.057040+0000 (oldest deadline 2024-05-14T17:19:26.156826+0000) 2024-05-14T17:19:41.605 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:41 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:41.216+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:10.658759+0000 front 2024-05-14T17:19:10.658746+0000 (oldest deadline 2024-05-14T17:19:37.659130+0000) 2024-05-14T17:19:42.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:41 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:41.893+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:42.244 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:41 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:41.893+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:42.244 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:41 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:41.893+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.345149+0000 front 2024-05-14T17:18:55.344987+0000 (oldest deadline 2024-05-14T17:19:17.644999+0000) 2024-05-14T17:19:42.244 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:41 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:41.893+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.345784+0000 front 2024-05-14T17:19:02.345930+0000 (oldest deadline 2024-05-14T17:19:25.245909+0000) 2024-05-14T17:19:42.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:42 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:42.082+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:42.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:42 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:42.082+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:42.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:42 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:42.082+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:42.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:42 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:42.082+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.543221+0000 front 2024-05-14T17:19:00.543205+0000 (oldest deadline 2024-05-14T17:19:25.243108+0000) 2024-05-14T17:19:42.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:42 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:42.082+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:08.144059+0000 front 2024-05-14T17:19:08.144106+0000 (oldest deadline 2024-05-14T17:19:33.444295+0000) 2024-05-14T17:19:42.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:42 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:42.173+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.353156+0000 front 2024-05-14T17:18:45.352963+0000 (oldest deadline 2024-05-14T17:19:08.753619+0000) 2024-05-14T17:19:42.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:42 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:42.173+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:52.755324+0000 front 2024-05-14T17:18:52.755350+0000 (oldest deadline 2024-05-14T17:19:20.355956+0000) 2024-05-14T17:19:42.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:42 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:42.173+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.056951+0000 front 2024-05-14T17:19:02.057040+0000 (oldest deadline 2024-05-14T17:19:26.156826+0000) 2024-05-14T17:19:42.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:42 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:42.173+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:10.658759+0000 front 2024-05-14T17:19:10.658746+0000 (oldest deadline 2024-05-14T17:19:37.659130+0000) 2024-05-14T17:19:42.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:42 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:42.145+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:42.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:42 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:42.145+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:42.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:42 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:42.145+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:42.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:42 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:42.145+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:19:42.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:42 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:42.145+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:11.427850+0000 front 2024-05-14T17:19:11.427867+0000 (oldest deadline 2024-05-14T17:19:37.327714+0000) 2024-05-14T17:19:42.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:42 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:42.145+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:17.328091+0000 front 2024-05-14T17:19:17.328071+0000 (oldest deadline 2024-05-14T17:19:41.928564+0000) 2024-05-14T17:19:43.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:42 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:42.895+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:43.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:42 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:42.895+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:43.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:42 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:42.895+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.345149+0000 front 2024-05-14T17:18:55.344987+0000 (oldest deadline 2024-05-14T17:19:17.644999+0000) 2024-05-14T17:19:43.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:42 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:42.895+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.345784+0000 front 2024-05-14T17:19:02.345930+0000 (oldest deadline 2024-05-14T17:19:25.245909+0000) 2024-05-14T17:19:43.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:43 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:43.129+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:43.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:43 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:43.129+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:43.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:43 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:43.129+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:43.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:43 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:43.129+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.543221+0000 front 2024-05-14T17:19:00.543205+0000 (oldest deadline 2024-05-14T17:19:25.243108+0000) 2024-05-14T17:19:43.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:43 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:43.129+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:08.144059+0000 front 2024-05-14T17:19:08.144106+0000 (oldest deadline 2024-05-14T17:19:33.444295+0000) 2024-05-14T17:19:43.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:43 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:43.129+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:18.145400+0000 front 2024-05-14T17:19:18.145495+0000 (oldest deadline 2024-05-14T17:19:42.245225+0000) 2024-05-14T17:19:43.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:43 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:43.158+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.353156+0000 front 2024-05-14T17:18:45.352963+0000 (oldest deadline 2024-05-14T17:19:08.753619+0000) 2024-05-14T17:19:43.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:43 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:43.158+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:52.755324+0000 front 2024-05-14T17:18:52.755350+0000 (oldest deadline 2024-05-14T17:19:20.355956+0000) 2024-05-14T17:19:43.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:43 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:43.158+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.056951+0000 front 2024-05-14T17:19:02.057040+0000 (oldest deadline 2024-05-14T17:19:26.156826+0000) 2024-05-14T17:19:43.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:43 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:43.158+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:10.658759+0000 front 2024-05-14T17:19:10.658746+0000 (oldest deadline 2024-05-14T17:19:37.659130+0000) 2024-05-14T17:19:43.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:43 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:43.155+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:43.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:43 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:43.155+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:43.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:43 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:43.155+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:43.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:43 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:43.155+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:19:43.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:43 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:43.155+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:11.427850+0000 front 2024-05-14T17:19:11.427867+0000 (oldest deadline 2024-05-14T17:19:37.327714+0000) 2024-05-14T17:19:43.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:43 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:43.155+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:17.328091+0000 front 2024-05-14T17:19:17.328071+0000 (oldest deadline 2024-05-14T17:19:41.928564+0000) 2024-05-14T17:19:44.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:43 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:43.875+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:44.244 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:43 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:43.875+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:44.244 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:43 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:43.875+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.345149+0000 front 2024-05-14T17:18:55.344987+0000 (oldest deadline 2024-05-14T17:19:17.644999+0000) 2024-05-14T17:19:44.244 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:43 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:43.875+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.345784+0000 front 2024-05-14T17:19:02.345930+0000 (oldest deadline 2024-05-14T17:19:25.245909+0000) 2024-05-14T17:19:44.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:44 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:44.158+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:44.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:44 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:44.158+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:44.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:44 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:44.158+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:44.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:44 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:44.158+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.543221+0000 front 2024-05-14T17:19:00.543205+0000 (oldest deadline 2024-05-14T17:19:25.243108+0000) 2024-05-14T17:19:44.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:44 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:44.158+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:08.144059+0000 front 2024-05-14T17:19:08.144106+0000 (oldest deadline 2024-05-14T17:19:33.444295+0000) 2024-05-14T17:19:44.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:44 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:44.158+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:18.145400+0000 front 2024-05-14T17:19:18.145495+0000 (oldest deadline 2024-05-14T17:19:42.245225+0000) 2024-05-14T17:19:44.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:44 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:44.194+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.353156+0000 front 2024-05-14T17:18:45.352963+0000 (oldest deadline 2024-05-14T17:19:08.753619+0000) 2024-05-14T17:19:44.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:44 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:44.194+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:52.755324+0000 front 2024-05-14T17:18:52.755350+0000 (oldest deadline 2024-05-14T17:19:20.355956+0000) 2024-05-14T17:19:44.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:44 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:44.194+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.056951+0000 front 2024-05-14T17:19:02.057040+0000 (oldest deadline 2024-05-14T17:19:26.156826+0000) 2024-05-14T17:19:44.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:44 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:44.195+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:10.658759+0000 front 2024-05-14T17:19:10.658746+0000 (oldest deadline 2024-05-14T17:19:37.659130+0000) 2024-05-14T17:19:44.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:44 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:44.195+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:17.659616+0000 front 2024-05-14T17:19:17.659654+0000 (oldest deadline 2024-05-14T17:19:43.559709+0000) 2024-05-14T17:19:44.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:44 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:44.127+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:44.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:44 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:44.127+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:44.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:44 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:44.127+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:44.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:44 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:44.127+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:19:44.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:44 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:44.127+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:11.427850+0000 front 2024-05-14T17:19:11.427867+0000 (oldest deadline 2024-05-14T17:19:37.327714+0000) 2024-05-14T17:19:44.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:44 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:44.127+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:17.328091+0000 front 2024-05-14T17:19:17.328071+0000 (oldest deadline 2024-05-14T17:19:41.928564+0000) 2024-05-14T17:19:45.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:44 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:44.826+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:45.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:44 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:44.826+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:45.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:44 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:44.826+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.345149+0000 front 2024-05-14T17:18:55.344987+0000 (oldest deadline 2024-05-14T17:19:17.644999+0000) 2024-05-14T17:19:45.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:44 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:44.826+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.345784+0000 front 2024-05-14T17:19:02.345930+0000 (oldest deadline 2024-05-14T17:19:25.245909+0000) 2024-05-14T17:19:45.244 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:44 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:44.826+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:19.749043+0000 front 2024-05-14T17:19:19.748948+0000 (oldest deadline 2024-05-14T17:19:44.448879+0000) 2024-05-14T17:19:45.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:45 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:45.158+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:45.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:45 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:45.158+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:45.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:45 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:45.158+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:45.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:45 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:45.158+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.543221+0000 front 2024-05-14T17:19:00.543205+0000 (oldest deadline 2024-05-14T17:19:25.243108+0000) 2024-05-14T17:19:45.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:45 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:45.158+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:08.144059+0000 front 2024-05-14T17:19:08.144106+0000 (oldest deadline 2024-05-14T17:19:33.444295+0000) 2024-05-14T17:19:45.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:45 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:45.158+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:18.145400+0000 front 2024-05-14T17:19:18.145495+0000 (oldest deadline 2024-05-14T17:19:42.245225+0000) 2024-05-14T17:19:45.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:45 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:45.149+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.353156+0000 front 2024-05-14T17:18:45.352963+0000 (oldest deadline 2024-05-14T17:19:08.753619+0000) 2024-05-14T17:19:45.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:45 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:45.149+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:52.755324+0000 front 2024-05-14T17:18:52.755350+0000 (oldest deadline 2024-05-14T17:19:20.355956+0000) 2024-05-14T17:19:45.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:45 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:45.149+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.056951+0000 front 2024-05-14T17:19:02.057040+0000 (oldest deadline 2024-05-14T17:19:26.156826+0000) 2024-05-14T17:19:45.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:45 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:45.149+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:10.658759+0000 front 2024-05-14T17:19:10.658746+0000 (oldest deadline 2024-05-14T17:19:37.659130+0000) 2024-05-14T17:19:45.604 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:45 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8[54920]: 2024-05-14T17:19:45.149+0000 7f7fd0c4b700 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:17.659616+0000 front 2024-05-14T17:19:17.659654+0000 (oldest deadline 2024-05-14T17:19:43.559709+0000) 2024-05-14T17:19:45.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:45 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:45.173+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:45.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:45 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:45.173+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:45.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:45 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:45.173+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:45.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:45 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:45.173+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:19:45.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:45 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:45.173+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:11.427850+0000 front 2024-05-14T17:19:11.427867+0000 (oldest deadline 2024-05-14T17:19:37.327714+0000) 2024-05-14T17:19:45.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:45 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:45.173+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:17.328091+0000 front 2024-05-14T17:19:17.328071+0000 (oldest deadline 2024-05-14T17:19:41.928564+0000) 2024-05-14T17:19:46.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:45 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:45.824+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:46.244 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:45 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:45.824+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:46.244 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:45 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:45.824+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.345149+0000 front 2024-05-14T17:18:55.344987+0000 (oldest deadline 2024-05-14T17:19:17.644999+0000) 2024-05-14T17:19:46.244 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:45 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:45.824+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.345784+0000 front 2024-05-14T17:19:02.345930+0000 (oldest deadline 2024-05-14T17:19:25.245909+0000) 2024-05-14T17:19:46.244 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:45 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:45.824+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:19.749043+0000 front 2024-05-14T17:19:19.748948+0000 (oldest deadline 2024-05-14T17:19:44.448879+0000) 2024-05-14T17:19:46.550 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:46 smithi151 podman[77230]: 2024-05-14 17:19:46.256799127 +0000 UTC m=+5.792214421 container died 83888fbd4ff8808303c20fa39b17e6ed5fe28efffe11e46cd2e4189ac1b40401 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T17:19:46.550 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:46 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:46.223+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:46.550 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:46 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:46.223+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:46.550 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:46 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:46.223+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:46.551 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:46 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:46.223+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:19:46.551 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:46 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:46.223+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:11.427850+0000 front 2024-05-14T17:19:11.427867+0000 (oldest deadline 2024-05-14T17:19:37.327714+0000) 2024-05-14T17:19:46.551 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:46 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:46.223+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:17.328091+0000 front 2024-05-14T17:19:17.328071+0000 (oldest deadline 2024-05-14T17:19:41.928564+0000) 2024-05-14T17:19:46.659 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:46 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:46.183+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:46.659 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:46 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:46.183+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:46.659 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:46 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:46.183+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:46.659 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:46 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:46.183+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.543221+0000 front 2024-05-14T17:19:00.543205+0000 (oldest deadline 2024-05-14T17:19:25.243108+0000) 2024-05-14T17:19:46.660 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:46 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:46.183+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:08.144059+0000 front 2024-05-14T17:19:08.144106+0000 (oldest deadline 2024-05-14T17:19:33.444295+0000) 2024-05-14T17:19:46.660 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:46 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:46.183+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:18.145400+0000 front 2024-05-14T17:19:18.145495+0000 (oldest deadline 2024-05-14T17:19:42.245225+0000) 2024-05-14T17:19:46.854 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:46 smithi151 podman[77230]: 2024-05-14 17:19:46.642251852 +0000 UTC m=+6.177667148 container cleanup 83888fbd4ff8808303c20fa39b17e6ed5fe28efffe11e46cd2e4189ac1b40401 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-14T17:19:46.854 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:46 smithi151 bash[77230]: ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8 2024-05-14T17:19:47.237 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:46 smithi151 podman[77246]: 2024-05-14 17:19:46.927205683 +0000 UTC m=+0.665451979 container remove 83888fbd4ff8808303c20fa39b17e6ed5fe28efffe11e46cd2e4189ac1b40401 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:19:47.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:46 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:46.793+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:47.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:46 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:46.793+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:47.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:46 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:46.793+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.345149+0000 front 2024-05-14T17:18:55.344987+0000 (oldest deadline 2024-05-14T17:19:17.644999+0000) 2024-05-14T17:19:47.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:46 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:46.793+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.345784+0000 front 2024-05-14T17:19:02.345930+0000 (oldest deadline 2024-05-14T17:19:25.245909+0000) 2024-05-14T17:19:47.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:46 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:46.793+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:19.749043+0000 front 2024-05-14T17:19:19.748948+0000 (oldest deadline 2024-05-14T17:19:44.448879+0000) 2024-05-14T17:19:47.538 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:47 smithi151 podman[77336]: 2024-05-14 17:19:47.16095258 +0000 UTC m=+0.017310555 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:19:47.538 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:47 smithi151 podman[77336]: 2024-05-14 17:19:47.286252589 +0000 UTC m=+0.142610553 container create 9cd69cedaa59b4ca45081a55acb5155508a6de3b771bc3992f27c3eb5a9a2bf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-14T17:19:47.538 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:47 smithi151 podman[77336]: 2024-05-14 17:19:47.539153442 +0000 UTC m=+0.395511407 container init 9cd69cedaa59b4ca45081a55acb5155508a6de3b771bc3992f27c3eb5a9a2bf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-14T17:19:47.538 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:47 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:47.238+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:47.538 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:47 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:47.238+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:47.538 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:47 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:47.238+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:47.538 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:47 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:47.238+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:19:47.539 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:47 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:47.238+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:11.427850+0000 front 2024-05-14T17:19:11.427867+0000 (oldest deadline 2024-05-14T17:19:37.327714+0000) 2024-05-14T17:19:47.539 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:47 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:47.238+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:17.328091+0000 front 2024-05-14T17:19:17.328071+0000 (oldest deadline 2024-05-14T17:19:41.928564+0000) 2024-05-14T17:19:47.659 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:47 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:47.209+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:47.659 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:47 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:47.209+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:47.659 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:47 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:47.209+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:47.660 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:47 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:47.209+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.543221+0000 front 2024-05-14T17:19:00.543205+0000 (oldest deadline 2024-05-14T17:19:25.243108+0000) 2024-05-14T17:19:47.660 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:47 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:47.209+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:08.144059+0000 front 2024-05-14T17:19:08.144106+0000 (oldest deadline 2024-05-14T17:19:33.444295+0000) 2024-05-14T17:19:47.660 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:47 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:47.209+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:18.145400+0000 front 2024-05-14T17:19:18.145495+0000 (oldest deadline 2024-05-14T17:19:42.245225+0000) 2024-05-14T17:19:47.839 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:47 smithi151 podman[77336]: 2024-05-14 17:19:47.542737383 +0000 UTC m=+0.399095351 container start 9cd69cedaa59b4ca45081a55acb5155508a6de3b771bc3992f27c3eb5a9a2bf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-14T17:19:47.839 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:47 smithi151 podman[77336]: 2024-05-14 17:19:47.655809975 +0000 UTC m=+0.512167938 container attach 9cd69cedaa59b4ca45081a55acb5155508a6de3b771bc3992f27c3eb5a9a2bf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T17:19:48.104 INFO:journalctl@ceph.osd.8.smithi151.stdout:May 14 17:19:47 smithi151 podman[77336]: 2024-05-14 17:19:47.840249719 +0000 UTC m=+0.696607698 container died 9cd69cedaa59b4ca45081a55acb5155508a6de3b771bc3992f27c3eb5a9a2bf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-14T17:19:48.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:47 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:47.812+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:48.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:47 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:47.812+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:48.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:47 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:47.812+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.345149+0000 front 2024-05-14T17:18:55.344987+0000 (oldest deadline 2024-05-14T17:19:17.644999+0000) 2024-05-14T17:19:48.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:47 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:47.812+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.345784+0000 front 2024-05-14T17:19:02.345930+0000 (oldest deadline 2024-05-14T17:19:25.245909+0000) 2024-05-14T17:19:48.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:47 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:47.812+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:19.749043+0000 front 2024-05-14T17:19:19.748948+0000 (oldest deadline 2024-05-14T17:19:44.448879+0000) 2024-05-14T17:19:48.514 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:48 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:48.239+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:48.547 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:48 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:48.239+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:48.547 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:48 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:48.239+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:48.547 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:48 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:48.239+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:19:48.547 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:48 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:48.239+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:11.427850+0000 front 2024-05-14T17:19:11.427867+0000 (oldest deadline 2024-05-14T17:19:37.327714+0000) 2024-05-14T17:19:48.547 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:48 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:48.239+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:17.328091+0000 front 2024-05-14T17:19:17.328071+0000 (oldest deadline 2024-05-14T17:19:41.928564+0000) 2024-05-14T17:19:48.549 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@osd.8.service' 2024-05-14T17:19:48.596 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:19:48.596 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-14T17:19:48.597 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-14T17:19:48.597 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-ed40717a-1210-11ef-bc98-c7b262605968@osd.9 2024-05-14T17:19:48.660 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:48 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:48.239+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:48.660 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:48 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:48.239+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:48.660 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:48 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:48.239+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:48.660 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:48 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:48.239+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.543221+0000 front 2024-05-14T17:19:00.543205+0000 (oldest deadline 2024-05-14T17:19:25.243108+0000) 2024-05-14T17:19:48.661 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:48 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:48.239+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:08.144059+0000 front 2024-05-14T17:19:08.144106+0000 (oldest deadline 2024-05-14T17:19:33.444295+0000) 2024-05-14T17:19:48.661 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:48 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:48.239+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:18.145400+0000 front 2024-05-14T17:19:18.145495+0000 (oldest deadline 2024-05-14T17:19:42.245225+0000) 2024-05-14T17:19:48.932 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:48 smithi002 systemd[1]: Stopping Ceph osd.9 for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T17:19:48.932 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:48 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:48.814+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:48.932 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:48 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:48.814+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:48.932 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:48 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:48.814+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.345149+0000 front 2024-05-14T17:18:55.344987+0000 (oldest deadline 2024-05-14T17:19:17.644999+0000) 2024-05-14T17:19:48.933 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:48 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:48.814+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.345784+0000 front 2024-05-14T17:19:02.345930+0000 (oldest deadline 2024-05-14T17:19:25.245909+0000) 2024-05-14T17:19:48.933 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:48 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:48.814+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:19.749043+0000 front 2024-05-14T17:19:19.748948+0000 (oldest deadline 2024-05-14T17:19:44.448879+0000) 2024-05-14T17:19:48.933 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:48 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:48.932+0000 7fe62245a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:19:48.933 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:48 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:48.932+0000 7fe62245a700 -1 osd.9 119 *** Got signal Terminated *** 2024-05-14T17:19:48.933 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:48 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:48.932+0000 7fe62245a700 -1 osd.9 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T17:19:49.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:49 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:49.280+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:49.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:49 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:49.280+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:49.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:49 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:49.280+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:49.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:49 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:49.280+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:19:49.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:49 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:49.280+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:11.427850+0000 front 2024-05-14T17:19:11.427867+0000 (oldest deadline 2024-05-14T17:19:37.327714+0000) 2024-05-14T17:19:49.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:49 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:49.280+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:17.328091+0000 front 2024-05-14T17:19:17.328071+0000 (oldest deadline 2024-05-14T17:19:41.928564+0000) 2024-05-14T17:19:49.659 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:49 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:49.275+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:49.659 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:49 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:49.275+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:49.660 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:49 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:49.275+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:49.660 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:49 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:49.275+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.543221+0000 front 2024-05-14T17:19:00.543205+0000 (oldest deadline 2024-05-14T17:19:25.243108+0000) 2024-05-14T17:19:49.660 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:49 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:49.275+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:08.144059+0000 front 2024-05-14T17:19:08.144106+0000 (oldest deadline 2024-05-14T17:19:33.444295+0000) 2024-05-14T17:19:49.660 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:49 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:49.275+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:18.145400+0000 front 2024-05-14T17:19:18.145495+0000 (oldest deadline 2024-05-14T17:19:42.245225+0000) 2024-05-14T17:19:50.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:49 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:49.865+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:50.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:49 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:49.865+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:50.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:49 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:49.865+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.345149+0000 front 2024-05-14T17:18:55.344987+0000 (oldest deadline 2024-05-14T17:19:17.644999+0000) 2024-05-14T17:19:50.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:49 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:49.865+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.345784+0000 front 2024-05-14T17:19:02.345930+0000 (oldest deadline 2024-05-14T17:19:25.245909+0000) 2024-05-14T17:19:50.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:49 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:49.865+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:19.749043+0000 front 2024-05-14T17:19:19.748948+0000 (oldest deadline 2024-05-14T17:19:44.448879+0000) 2024-05-14T17:19:50.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:50 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:50.241+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:50.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:50 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:50.241+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:50.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:50 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:50.241+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:50.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:50 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:50.241+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:19:50.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:50 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:50.241+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:11.427850+0000 front 2024-05-14T17:19:11.427867+0000 (oldest deadline 2024-05-14T17:19:37.327714+0000) 2024-05-14T17:19:50.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:50 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:50.241+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:17.328091+0000 front 2024-05-14T17:19:17.328071+0000 (oldest deadline 2024-05-14T17:19:41.928564+0000) 2024-05-14T17:19:50.659 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:50 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:50.245+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:50.660 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:50 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:50.245+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:50.660 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:50 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:50.245+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:50.660 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:50 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:50.245+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.543221+0000 front 2024-05-14T17:19:00.543205+0000 (oldest deadline 2024-05-14T17:19:25.243108+0000) 2024-05-14T17:19:50.660 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:50 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:50.245+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:08.144059+0000 front 2024-05-14T17:19:08.144106+0000 (oldest deadline 2024-05-14T17:19:33.444295+0000) 2024-05-14T17:19:50.660 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:50 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:50.245+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:18.145400+0000 front 2024-05-14T17:19:18.145495+0000 (oldest deadline 2024-05-14T17:19:42.245225+0000) 2024-05-14T17:19:50.660 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:50 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:50.245+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T17:19:23.946524+0000 front 2024-05-14T17:19:23.946552+0000 (oldest deadline 2024-05-14T17:19:49.846384+0000) 2024-05-14T17:19:51.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:50 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:50.838+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:51.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:50 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:50.838+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:51.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:50 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:50.838+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.345149+0000 front 2024-05-14T17:18:55.344987+0000 (oldest deadline 2024-05-14T17:19:17.644999+0000) 2024-05-14T17:19:51.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:50 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:50.838+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.345784+0000 front 2024-05-14T17:19:02.345930+0000 (oldest deadline 2024-05-14T17:19:25.245909+0000) 2024-05-14T17:19:51.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:50 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:50.838+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:19.749043+0000 front 2024-05-14T17:19:19.748948+0000 (oldest deadline 2024-05-14T17:19:44.448879+0000) 2024-05-14T17:19:51.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:51 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:51.199+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:51.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:51 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:51.199+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:51.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:51 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:51.199+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:51.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:51 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:51.199+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:19:51.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:51 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:51.199+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:11.427850+0000 front 2024-05-14T17:19:11.427867+0000 (oldest deadline 2024-05-14T17:19:37.327714+0000) 2024-05-14T17:19:51.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:51 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:51.199+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:17.328091+0000 front 2024-05-14T17:19:17.328071+0000 (oldest deadline 2024-05-14T17:19:41.928564+0000) 2024-05-14T17:19:51.659 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:51 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:51.196+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:51.659 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:51 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:51.196+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:51.660 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:51 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:51.196+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:51.660 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:51 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:51.196+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.543221+0000 front 2024-05-14T17:19:00.543205+0000 (oldest deadline 2024-05-14T17:19:25.243108+0000) 2024-05-14T17:19:51.660 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:51 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:51.196+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:08.144059+0000 front 2024-05-14T17:19:08.144106+0000 (oldest deadline 2024-05-14T17:19:33.444295+0000) 2024-05-14T17:19:51.660 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:51 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:51.196+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:18.145400+0000 front 2024-05-14T17:19:18.145495+0000 (oldest deadline 2024-05-14T17:19:42.245225+0000) 2024-05-14T17:19:51.660 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:51 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:51.196+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T17:19:23.946524+0000 front 2024-05-14T17:19:23.946552+0000 (oldest deadline 2024-05-14T17:19:49.846384+0000) 2024-05-14T17:19:52.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:51 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:51.816+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:52.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:51 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:51.816+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:52.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:51 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:51.816+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.345149+0000 front 2024-05-14T17:18:55.344987+0000 (oldest deadline 2024-05-14T17:19:17.644999+0000) 2024-05-14T17:19:52.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:51 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:51.816+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.345784+0000 front 2024-05-14T17:19:02.345930+0000 (oldest deadline 2024-05-14T17:19:25.245909+0000) 2024-05-14T17:19:52.244 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:51 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:51.816+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:19.749043+0000 front 2024-05-14T17:19:19.748948+0000 (oldest deadline 2024-05-14T17:19:44.448879+0000) 2024-05-14T17:19:52.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:52 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:52.177+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:52.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:52 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:52.177+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:52.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:52 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:52.177+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:52.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:52 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:52.177+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:19:52.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:52 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:52.177+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:11.427850+0000 front 2024-05-14T17:19:11.427867+0000 (oldest deadline 2024-05-14T17:19:37.327714+0000) 2024-05-14T17:19:52.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:52 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:52.177+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:17.328091+0000 front 2024-05-14T17:19:17.328071+0000 (oldest deadline 2024-05-14T17:19:41.928564+0000) 2024-05-14T17:19:52.659 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:52 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:52.186+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:52.659 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:52 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:52.186+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:52.659 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:52 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:52.186+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:52.660 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:52 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:52.186+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.543221+0000 front 2024-05-14T17:19:00.543205+0000 (oldest deadline 2024-05-14T17:19:25.243108+0000) 2024-05-14T17:19:52.660 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:52 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:52.186+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:08.144059+0000 front 2024-05-14T17:19:08.144106+0000 (oldest deadline 2024-05-14T17:19:33.444295+0000) 2024-05-14T17:19:52.660 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:52 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:52.186+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:18.145400+0000 front 2024-05-14T17:19:18.145495+0000 (oldest deadline 2024-05-14T17:19:42.245225+0000) 2024-05-14T17:19:52.660 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:52 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:52.186+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T17:19:23.946524+0000 front 2024-05-14T17:19:23.946552+0000 (oldest deadline 2024-05-14T17:19:49.846384+0000) 2024-05-14T17:19:53.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:52 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:52.840+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:53.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:52 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:52.840+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:53.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:52 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:52.840+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.345149+0000 front 2024-05-14T17:18:55.344987+0000 (oldest deadline 2024-05-14T17:19:17.644999+0000) 2024-05-14T17:19:53.244 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:52 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:52.840+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.345784+0000 front 2024-05-14T17:19:02.345930+0000 (oldest deadline 2024-05-14T17:19:25.245909+0000) 2024-05-14T17:19:53.244 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:52 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:52.840+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:19.749043+0000 front 2024-05-14T17:19:19.748948+0000 (oldest deadline 2024-05-14T17:19:44.448879+0000) 2024-05-14T17:19:53.244 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:52 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:52.840+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T17:19:27.250158+0000 front 2024-05-14T17:19:27.250207+0000 (oldest deadline 2024-05-14T17:19:52.550211+0000) 2024-05-14T17:19:53.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:53 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:53.154+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:53.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:53 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:53.154+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:53.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:53 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:53.154+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:53.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:53 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:53.154+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.543221+0000 front 2024-05-14T17:19:00.543205+0000 (oldest deadline 2024-05-14T17:19:25.243108+0000) 2024-05-14T17:19:53.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:53 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:53.154+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:08.144059+0000 front 2024-05-14T17:19:08.144106+0000 (oldest deadline 2024-05-14T17:19:33.444295+0000) 2024-05-14T17:19:53.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:53 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:53.154+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:18.145400+0000 front 2024-05-14T17:19:18.145495+0000 (oldest deadline 2024-05-14T17:19:42.245225+0000) 2024-05-14T17:19:53.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:53 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:53.154+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T17:19:23.946524+0000 front 2024-05-14T17:19:23.946552+0000 (oldest deadline 2024-05-14T17:19:49.846384+0000) 2024-05-14T17:19:53.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:53 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:53.216+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:53.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:53 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:53.216+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:53.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:53 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:53.216+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:53.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:53 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:53.216+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:19:53.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:53 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:53.216+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:11.427850+0000 front 2024-05-14T17:19:11.427867+0000 (oldest deadline 2024-05-14T17:19:37.327714+0000) 2024-05-14T17:19:53.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:53 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:53.216+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:17.328091+0000 front 2024-05-14T17:19:17.328071+0000 (oldest deadline 2024-05-14T17:19:41.928564+0000) 2024-05-14T17:19:54.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:53 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:53.876+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.641532+0000 front 2024-05-14T17:18:38.641287+0000 (oldest deadline 2024-05-14T17:19:00.941485+0000) 2024-05-14T17:19:54.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:53 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:53.876+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:47.144187+0000 front 2024-05-14T17:18:47.144008+0000 (oldest deadline 2024-05-14T17:19:11.843872+0000) 2024-05-14T17:19:54.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:53 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:53.876+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.345149+0000 front 2024-05-14T17:18:55.344987+0000 (oldest deadline 2024-05-14T17:19:17.644999+0000) 2024-05-14T17:19:54.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:53 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:53.876+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:02.345784+0000 front 2024-05-14T17:19:02.345930+0000 (oldest deadline 2024-05-14T17:19:25.245909+0000) 2024-05-14T17:19:54.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:53 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:53.876+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:19.749043+0000 front 2024-05-14T17:19:19.748948+0000 (oldest deadline 2024-05-14T17:19:44.448879+0000) 2024-05-14T17:19:54.244 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:53 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9[66524]: 2024-05-14T17:19:53.876+0000 7fe61ea4e700 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T17:19:27.250158+0000 front 2024-05-14T17:19:27.250207+0000 (oldest deadline 2024-05-14T17:19:52.550211+0000) 2024-05-14T17:19:54.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:54 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:54.153+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:54.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:54 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:54.153+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:54.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:54 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:54.153+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:54.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:54 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:54.153+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.543221+0000 front 2024-05-14T17:19:00.543205+0000 (oldest deadline 2024-05-14T17:19:25.243108+0000) 2024-05-14T17:19:54.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:54 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:54.153+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:08.144059+0000 front 2024-05-14T17:19:08.144106+0000 (oldest deadline 2024-05-14T17:19:33.444295+0000) 2024-05-14T17:19:54.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:54 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:54.153+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:18.145400+0000 front 2024-05-14T17:19:18.145495+0000 (oldest deadline 2024-05-14T17:19:42.245225+0000) 2024-05-14T17:19:54.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:54 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:54.153+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T17:19:23.946524+0000 front 2024-05-14T17:19:23.946552+0000 (oldest deadline 2024-05-14T17:19:49.846384+0000) 2024-05-14T17:19:54.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:54 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:54.195+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:54.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:54 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:54.195+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:54.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:54 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:54.195+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:54.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:54 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:54.195+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:19:54.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:54 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:54.195+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:11.427850+0000 front 2024-05-14T17:19:11.427867+0000 (oldest deadline 2024-05-14T17:19:37.327714+0000) 2024-05-14T17:19:54.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:54 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:54.195+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:17.328091+0000 front 2024-05-14T17:19:17.328071+0000 (oldest deadline 2024-05-14T17:19:41.928564+0000) 2024-05-14T17:19:54.743 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:54 smithi002 podman[109864]: 2024-05-14 17:19:54.440208203 +0000 UTC m=+5.631553468 container died 0e2866076f28c2416b6de224728889bbb2c3a5e9f961829e8394fd9e977a3d9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514) 2024-05-14T17:19:55.182 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:54 smithi002 podman[109864]: 2024-05-14 17:19:54.915575428 +0000 UTC m=+6.106920687 container cleanup 0e2866076f28c2416b6de224728889bbb2c3a5e9f961829e8394fd9e977a3d9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-14T17:19:55.182 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:54 smithi002 bash[109864]: ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9 2024-05-14T17:19:55.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:55 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:55.108+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:55.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:55 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:55.108+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:55.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:55 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:55.108+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:55.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:55 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:55.108+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.543221+0000 front 2024-05-14T17:19:00.543205+0000 (oldest deadline 2024-05-14T17:19:25.243108+0000) 2024-05-14T17:19:55.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:55 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:55.108+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:08.144059+0000 front 2024-05-14T17:19:08.144106+0000 (oldest deadline 2024-05-14T17:19:33.444295+0000) 2024-05-14T17:19:55.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:55 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:55.108+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:18.145400+0000 front 2024-05-14T17:19:18.145495+0000 (oldest deadline 2024-05-14T17:19:42.245225+0000) 2024-05-14T17:19:55.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:55 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:55.108+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T17:19:23.946524+0000 front 2024-05-14T17:19:23.946552+0000 (oldest deadline 2024-05-14T17:19:49.846384+0000) 2024-05-14T17:19:55.493 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:55 smithi002 podman[109879]: 2024-05-14 17:19:55.183651849 +0000 UTC m=+0.735396705 container remove 0e2866076f28c2416b6de224728889bbb2c3a5e9f961829e8394fd9e977a3d9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-14T17:19:55.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:55 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:55.183+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:55.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:55 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:55.183+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:55.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:55 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:55.183+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:55.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:55 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:55.183+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:19:55.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:55 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:55.183+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:11.427850+0000 front 2024-05-14T17:19:11.427867+0000 (oldest deadline 2024-05-14T17:19:37.327714+0000) 2024-05-14T17:19:55.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:55 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:55.183+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:17.328091+0000 front 2024-05-14T17:19:17.328071+0000 (oldest deadline 2024-05-14T17:19:41.928564+0000) 2024-05-14T17:19:55.852 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:55 smithi002 podman[109970]: 2024-05-14 17:19:55.542871602 +0000 UTC m=+0.109038244 container create 03d66cf09b9772ced496a876f8f615a6ae087e0986a1569e26f4787d10206571 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-14T17:19:55.852 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:55 smithi002 podman[109970]: 2024-05-14 17:19:55.451358828 +0000 UTC m=+0.017525476 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:19:55.852 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:55 smithi002 podman[109970]: 2024-05-14 17:19:55.770180234 +0000 UTC m=+0.336346875 container init 03d66cf09b9772ced496a876f8f615a6ae087e0986a1569e26f4787d10206571 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD) 2024-05-14T17:19:55.852 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:55 smithi002 podman[109970]: 2024-05-14 17:19:55.773561614 +0000 UTC m=+0.339728258 container start 03d66cf09b9772ced496a876f8f615a6ae087e0986a1569e26f4787d10206571 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD) 2024-05-14T17:19:55.852 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:55 smithi002 podman[109970]: 2024-05-14 17:19:55.853421821 +0000 UTC m=+0.419588488 container attach 03d66cf09b9772ced496a876f8f615a6ae087e0986a1569e26f4787d10206571 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-14T17:19:56.243 INFO:journalctl@ceph.osd.9.smithi002.stdout:May 14 17:19:56 smithi002 podman[109970]: 2024-05-14 17:19:56.093101526 +0000 UTC m=+0.659268167 container died 03d66cf09b9772ced496a876f8f615a6ae087e0986a1569e26f4787d10206571 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-14T17:19:56.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:56 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:56.118+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:56.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:56 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:56.118+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:56.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:56 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:56.118+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:56.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:56 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:56.118+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.543221+0000 front 2024-05-14T17:19:00.543205+0000 (oldest deadline 2024-05-14T17:19:25.243108+0000) 2024-05-14T17:19:56.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:56 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:56.118+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:08.144059+0000 front 2024-05-14T17:19:08.144106+0000 (oldest deadline 2024-05-14T17:19:33.444295+0000) 2024-05-14T17:19:56.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:56 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:56.118+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:18.145400+0000 front 2024-05-14T17:19:18.145495+0000 (oldest deadline 2024-05-14T17:19:42.245225+0000) 2024-05-14T17:19:56.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:56 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:56.118+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T17:19:23.946524+0000 front 2024-05-14T17:19:23.946552+0000 (oldest deadline 2024-05-14T17:19:49.846384+0000) 2024-05-14T17:19:56.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:56 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:56.227+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:56.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:56 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:56.227+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:56.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:56 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:56.227+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:56.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:56 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:56.227+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:19:56.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:56 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:56.227+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:11.427850+0000 front 2024-05-14T17:19:11.427867+0000 (oldest deadline 2024-05-14T17:19:37.327714+0000) 2024-05-14T17:19:56.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:56 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:56.227+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:17.328091+0000 front 2024-05-14T17:19:17.328071+0000 (oldest deadline 2024-05-14T17:19:41.928564+0000) 2024-05-14T17:19:56.778 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@osd.9.service' 2024-05-14T17:19:56.820 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:19:56.821 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-14T17:19:56.821 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-14T17:19:56.821 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-ed40717a-1210-11ef-bc98-c7b262605968@osd.10 2024-05-14T17:19:57.103 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:56 smithi123 systemd[1]: Stopping Ceph osd.10 for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T17:19:57.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:57 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:57.103+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:57.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:57 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:57.103+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:57.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:57 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:57.103+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:57.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:57 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:57.103+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.543221+0000 front 2024-05-14T17:19:00.543205+0000 (oldest deadline 2024-05-14T17:19:25.243108+0000) 2024-05-14T17:19:57.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:57 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:57.103+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:08.144059+0000 front 2024-05-14T17:19:08.144106+0000 (oldest deadline 2024-05-14T17:19:33.444295+0000) 2024-05-14T17:19:57.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:57 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:57.103+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:18.145400+0000 front 2024-05-14T17:19:18.145495+0000 (oldest deadline 2024-05-14T17:19:42.245225+0000) 2024-05-14T17:19:57.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:57 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:57.103+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T17:19:23.946524+0000 front 2024-05-14T17:19:23.946552+0000 (oldest deadline 2024-05-14T17:19:49.846384+0000) 2024-05-14T17:19:57.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:57 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:57.131+0000 7f34fb2fb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:19:57.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:57 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:57.131+0000 7f34fb2fb700 -1 osd.10 119 *** Got signal Terminated *** 2024-05-14T17:19:57.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:57 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:57.131+0000 7f34fb2fb700 -1 osd.10 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T17:19:57.505 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:57 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:57.227+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:57.506 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:57 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:57.227+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:57.506 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:57 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:57.227+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:57.506 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:57 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:57.227+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:19:57.506 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:57 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:57.227+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:11.427850+0000 front 2024-05-14T17:19:11.427867+0000 (oldest deadline 2024-05-14T17:19:37.327714+0000) 2024-05-14T17:19:57.506 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:57 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:57.227+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:17.328091+0000 front 2024-05-14T17:19:17.328071+0000 (oldest deadline 2024-05-14T17:19:41.928564+0000) 2024-05-14T17:19:58.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:58 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:58.151+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:58.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:58 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:58.151+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:58.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:58 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:58.151+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:58.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:58 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:58.151+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.543221+0000 front 2024-05-14T17:19:00.543205+0000 (oldest deadline 2024-05-14T17:19:25.243108+0000) 2024-05-14T17:19:58.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:58 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:58.151+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:08.144059+0000 front 2024-05-14T17:19:08.144106+0000 (oldest deadline 2024-05-14T17:19:33.444295+0000) 2024-05-14T17:19:58.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:58 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:58.151+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:18.145400+0000 front 2024-05-14T17:19:18.145495+0000 (oldest deadline 2024-05-14T17:19:42.245225+0000) 2024-05-14T17:19:58.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:58 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:58.151+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T17:19:23.946524+0000 front 2024-05-14T17:19:23.946552+0000 (oldest deadline 2024-05-14T17:19:49.846384+0000) 2024-05-14T17:19:58.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:58 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:58.233+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:58.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:58 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:58.233+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:58.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:58 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:58.233+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:58.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:58 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:58.233+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:19:58.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:58 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:58.233+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:11.427850+0000 front 2024-05-14T17:19:11.427867+0000 (oldest deadline 2024-05-14T17:19:37.327714+0000) 2024-05-14T17:19:58.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:58 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:58.233+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:17.328091+0000 front 2024-05-14T17:19:17.328071+0000 (oldest deadline 2024-05-14T17:19:41.928564+0000) 2024-05-14T17:19:59.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:59 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:59.187+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:19:59.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:59 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:59.187+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:19:59.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:59 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:59.187+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:19:59.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:59 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:59.187+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:19:59.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:59 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:59.187+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:11.427850+0000 front 2024-05-14T17:19:11.427867+0000 (oldest deadline 2024-05-14T17:19:37.327714+0000) 2024-05-14T17:19:59.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:59 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:59.187+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:17.328091+0000 front 2024-05-14T17:19:17.328071+0000 (oldest deadline 2024-05-14T17:19:41.928564+0000) 2024-05-14T17:19:59.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:19:59 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:19:59.187+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-05-14T17:19:36.332031+0000 front 2024-05-14T17:19:36.332151+0000 (oldest deadline 2024-05-14T17:19:58.631928+0000) 2024-05-14T17:19:59.659 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:59 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:59.185+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:19:59.659 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:59 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:59.185+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:19:59.660 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:59 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:59.185+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:19:59.660 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:59 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:59.185+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.543221+0000 front 2024-05-14T17:19:00.543205+0000 (oldest deadline 2024-05-14T17:19:25.243108+0000) 2024-05-14T17:19:59.660 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:59 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:59.185+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:08.144059+0000 front 2024-05-14T17:19:08.144106+0000 (oldest deadline 2024-05-14T17:19:33.444295+0000) 2024-05-14T17:19:59.660 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:59 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:59.185+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:18.145400+0000 front 2024-05-14T17:19:18.145495+0000 (oldest deadline 2024-05-14T17:19:42.245225+0000) 2024-05-14T17:19:59.660 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:19:59 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:19:59.185+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T17:19:23.946524+0000 front 2024-05-14T17:19:23.946552+0000 (oldest deadline 2024-05-14T17:19:49.846384+0000) 2024-05-14T17:20:00.420 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:20:00 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:20:00.165+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:20:00.420 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:20:00 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:20:00.165+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:20:00.420 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:20:00 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:20:00.165+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:20:00.420 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:20:00 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:20:00.165+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.543221+0000 front 2024-05-14T17:19:00.543205+0000 (oldest deadline 2024-05-14T17:19:25.243108+0000) 2024-05-14T17:20:00.420 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:20:00 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:20:00.165+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:08.144059+0000 front 2024-05-14T17:19:08.144106+0000 (oldest deadline 2024-05-14T17:19:33.444295+0000) 2024-05-14T17:20:00.420 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:20:00 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:20:00.165+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:18.145400+0000 front 2024-05-14T17:19:18.145495+0000 (oldest deadline 2024-05-14T17:19:42.245225+0000) 2024-05-14T17:20:00.420 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:20:00 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:20:00.165+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T17:19:23.946524+0000 front 2024-05-14T17:19:23.946552+0000 (oldest deadline 2024-05-14T17:19:49.846384+0000) 2024-05-14T17:20:00.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:00 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:00.194+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:20:00.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:00 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:00.194+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:20:00.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:00 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:00.194+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:20:00.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:00 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:00.194+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:20:00.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:00 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:00.194+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:11.427850+0000 front 2024-05-14T17:19:11.427867+0000 (oldest deadline 2024-05-14T17:19:37.327714+0000) 2024-05-14T17:20:00.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:00 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:00.194+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:17.328091+0000 front 2024-05-14T17:19:17.328071+0000 (oldest deadline 2024-05-14T17:19:41.928564+0000) 2024-05-14T17:20:00.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:00 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:00.194+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-05-14T17:19:36.332031+0000 front 2024-05-14T17:19:36.332151+0000 (oldest deadline 2024-05-14T17:19:58.631928+0000) 2024-05-14T17:20:01.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:20:01 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:20:01.120+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.738102+0000 front 2024-05-14T17:18:38.738133+0000 (oldest deadline 2024-05-14T17:19:02.738658+0000) 2024-05-14T17:20:01.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:20:01 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:20:01.120+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:42.739190+0000 front 2024-05-14T17:18:42.739160+0000 (oldest deadline 2024-05-14T17:19:08.639373+0000) 2024-05-14T17:20:01.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:20:01 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:20:01.120+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:55.441846+0000 front 2024-05-14T17:18:55.441877+0000 (oldest deadline 2024-05-14T17:19:17.042160+0000) 2024-05-14T17:20:01.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:20:01 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:20:01.120+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.543221+0000 front 2024-05-14T17:19:00.543205+0000 (oldest deadline 2024-05-14T17:19:25.243108+0000) 2024-05-14T17:20:01.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:20:01 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:20:01.120+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:08.144059+0000 front 2024-05-14T17:19:08.144106+0000 (oldest deadline 2024-05-14T17:19:33.444295+0000) 2024-05-14T17:20:01.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:20:01 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:20:01.120+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:18.145400+0000 front 2024-05-14T17:19:18.145495+0000 (oldest deadline 2024-05-14T17:19:42.245225+0000) 2024-05-14T17:20:01.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:20:01 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10[60917]: 2024-05-14T17:20:01.120+0000 7f34f38e7700 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2024-05-14T17:19:23.946524+0000 front 2024-05-14T17:19:23.946552+0000 (oldest deadline 2024-05-14T17:19:49.846384+0000) 2024-05-14T17:20:01.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:01 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:01.204+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:20:01.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:01 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:01.204+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:20:01.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:01 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:01.204+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:20:01.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:01 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:01.204+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:20:01.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:01 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:01.204+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:11.427850+0000 front 2024-05-14T17:19:11.427867+0000 (oldest deadline 2024-05-14T17:19:37.327714+0000) 2024-05-14T17:20:01.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:01 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:01.204+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:17.328091+0000 front 2024-05-14T17:19:17.328071+0000 (oldest deadline 2024-05-14T17:19:41.928564+0000) 2024-05-14T17:20:01.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:01 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:01.204+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-05-14T17:19:36.332031+0000 front 2024-05-14T17:19:36.332151+0000 (oldest deadline 2024-05-14T17:19:58.631928+0000) 2024-05-14T17:20:02.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:02 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:02.236+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:20:02.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:02 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:02.236+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:20:02.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:02 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:02.236+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:20:02.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:02 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:02.236+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:20:02.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:02 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:02.236+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:11.427850+0000 front 2024-05-14T17:19:11.427867+0000 (oldest deadline 2024-05-14T17:19:37.327714+0000) 2024-05-14T17:20:02.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:02 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:02.236+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:17.328091+0000 front 2024-05-14T17:19:17.328071+0000 (oldest deadline 2024-05-14T17:19:41.928564+0000) 2024-05-14T17:20:02.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:02 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:02.236+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-05-14T17:19:36.332031+0000 front 2024-05-14T17:19:36.332151+0000 (oldest deadline 2024-05-14T17:19:58.631928+0000) 2024-05-14T17:20:02.909 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:20:02 smithi123 podman[75190]: 2024-05-14 17:20:02.685641046 +0000 UTC m=+5.670473769 container died 51861c521ec9b4f866b4f8bd5b9d94aad605e299852e983971ccd4161205dc2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-14T17:20:03.311 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:20:03 smithi123 podman[75190]: 2024-05-14 17:20:03.069766153 +0000 UTC m=+6.054598876 container cleanup 51861c521ec9b4f866b4f8bd5b9d94aad605e299852e983971ccd4161205dc2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T17:20:03.311 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:20:03 smithi123 bash[75190]: ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10 2024-05-14T17:20:03.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:03 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:03.269+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:20:03.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:03 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:03.269+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:20:03.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:03 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:03.269+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:20:03.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:03 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:03.269+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:20:03.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:03 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:03.269+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:11.427850+0000 front 2024-05-14T17:19:11.427867+0000 (oldest deadline 2024-05-14T17:19:37.327714+0000) 2024-05-14T17:20:03.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:03 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:03.269+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:17.328091+0000 front 2024-05-14T17:19:17.328071+0000 (oldest deadline 2024-05-14T17:19:41.928564+0000) 2024-05-14T17:20:03.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:03 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:03.269+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-05-14T17:19:36.332031+0000 front 2024-05-14T17:19:36.332151+0000 (oldest deadline 2024-05-14T17:19:58.631928+0000) 2024-05-14T17:20:03.633 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:20:03 smithi123 podman[75218]: 2024-05-14 17:20:03.3125529 +0000 UTC m=+0.618968241 container remove 51861c521ec9b4f866b4f8bd5b9d94aad605e299852e983971ccd4161205dc2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-14T17:20:03.909 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:20:03 smithi123 podman[75304]: 2024-05-14 17:20:03.537845959 +0000 UTC m=+0.016562029 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:20:03.909 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:20:03 smithi123 podman[75304]: 2024-05-14 17:20:03.646608875 +0000 UTC m=+0.125324951 container create d47abb7e7e22b45c5e42e3a958b673d1c015091597e5e0b4944edf0a11ae269a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-14T17:20:03.909 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:20:03 smithi123 podman[75304]: 2024-05-14 17:20:03.874004125 +0000 UTC m=+0.352720182 container init d47abb7e7e22b45c5e42e3a958b673d1c015091597e5e0b4944edf0a11ae269a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2) 2024-05-14T17:20:03.910 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:20:03 smithi123 podman[75304]: 2024-05-14 17:20:03.877127204 +0000 UTC m=+0.355843262 container start d47abb7e7e22b45c5e42e3a958b673d1c015091597e5e0b4944edf0a11ae269a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-14T17:20:04.409 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:20:03 smithi123 podman[75304]: 2024-05-14 17:20:03.989910913 +0000 UTC m=+0.468626971 container attach d47abb7e7e22b45c5e42e3a958b673d1c015091597e5e0b4944edf0a11ae269a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:20:04.410 INFO:journalctl@ceph.osd.10.smithi123.stdout:May 14 17:20:04 smithi123 podman[75304]: 2024-05-14 17:20:04.164821454 +0000 UTC m=+0.643537516 container died d47abb7e7e22b45c5e42e3a958b673d1c015091597e5e0b4944edf0a11ae269a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:20:04.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:04.300+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:20:04.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:04.300+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:20:04.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:04.300+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:20:04.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:04.300+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:20:04.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:04.300+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:11.427850+0000 front 2024-05-14T17:19:11.427867+0000 (oldest deadline 2024-05-14T17:19:37.327714+0000) 2024-05-14T17:20:04.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:04.300+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:17.328091+0000 front 2024-05-14T17:19:17.328071+0000 (oldest deadline 2024-05-14T17:19:41.928564+0000) 2024-05-14T17:20:04.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:04 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:04.300+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-05-14T17:19:36.332031+0000 front 2024-05-14T17:19:36.332151+0000 (oldest deadline 2024-05-14T17:19:58.631928+0000) 2024-05-14T17:20:04.768 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@osd.10.service' 2024-05-14T17:20:04.806 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:20:04.806 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-14T17:20:04.806 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-14T17:20:04.806 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-ed40717a-1210-11ef-bc98-c7b262605968@osd.11 2024-05-14T17:20:05.104 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:04 smithi151 systemd[1]: Stopping Ceph osd.11 for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T17:20:05.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:05 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:05.108+0000 7fec9adbf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:20:05.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:05 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:05.108+0000 7fec9adbf700 -1 osd.11 119 *** Got signal Terminated *** 2024-05-14T17:20:05.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:05 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:05.108+0000 7fec9adbf700 -1 osd.11 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-14T17:20:05.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:05 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:05.302+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:20:05.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:05 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:05.302+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:20:05.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:05 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:05.302+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:20:05.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:05 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:05.302+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:20:05.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:05 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:05.302+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:11.427850+0000 front 2024-05-14T17:19:11.427867+0000 (oldest deadline 2024-05-14T17:19:37.327714+0000) 2024-05-14T17:20:05.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:05 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:05.302+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:17.328091+0000 front 2024-05-14T17:19:17.328071+0000 (oldest deadline 2024-05-14T17:19:41.928564+0000) 2024-05-14T17:20:05.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:05 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:05.302+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-05-14T17:19:36.332031+0000 front 2024-05-14T17:19:36.332151+0000 (oldest deadline 2024-05-14T17:19:58.631928+0000) 2024-05-14T17:20:06.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:06 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:06.316+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:20:06.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:06 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:06.316+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:20:06.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:06 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:06.316+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:20:06.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:06 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:06.316+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:20:06.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:06 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:06.316+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:11.427850+0000 front 2024-05-14T17:19:11.427867+0000 (oldest deadline 2024-05-14T17:19:37.327714+0000) 2024-05-14T17:20:06.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:06 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:06.316+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:17.328091+0000 front 2024-05-14T17:19:17.328071+0000 (oldest deadline 2024-05-14T17:19:41.928564+0000) 2024-05-14T17:20:06.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:06 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:06.316+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-05-14T17:19:36.332031+0000 front 2024-05-14T17:19:36.332151+0000 (oldest deadline 2024-05-14T17:19:58.631928+0000) 2024-05-14T17:20:07.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:07 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:07.340+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:20:07.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:07 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:07.340+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:20:07.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:07 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:07.340+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:20:07.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:07 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:07.340+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:20:07.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:07 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:07.340+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:11.427850+0000 front 2024-05-14T17:19:11.427867+0000 (oldest deadline 2024-05-14T17:19:37.327714+0000) 2024-05-14T17:20:07.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:07 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:07.340+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:17.328091+0000 front 2024-05-14T17:19:17.328071+0000 (oldest deadline 2024-05-14T17:19:41.928564+0000) 2024-05-14T17:20:07.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:07 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:07.340+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-05-14T17:19:36.332031+0000 front 2024-05-14T17:19:36.332151+0000 (oldest deadline 2024-05-14T17:19:58.631928+0000) 2024-05-14T17:20:08.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:08 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:08.324+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:20:08.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:08 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:08.324+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:20:08.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:08 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:08.324+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:20:08.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:08 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:08.324+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:20:08.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:08 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:08.324+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:11.427850+0000 front 2024-05-14T17:19:11.427867+0000 (oldest deadline 2024-05-14T17:19:37.327714+0000) 2024-05-14T17:20:08.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:08 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:08.324+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:17.328091+0000 front 2024-05-14T17:19:17.328071+0000 (oldest deadline 2024-05-14T17:19:41.928564+0000) 2024-05-14T17:20:08.605 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:08 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:08.324+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-05-14T17:19:36.332031+0000 front 2024-05-14T17:19:36.332151+0000 (oldest deadline 2024-05-14T17:19:58.631928+0000) 2024-05-14T17:20:09.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:09 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:09.303+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2024-05-14T17:18:38.921882+0000 front 2024-05-14T17:18:38.921901+0000 (oldest deadline 2024-05-14T17:19:00.621852+0000) 2024-05-14T17:20:09.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:09 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:09.303+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-05-14T17:18:45.223783+0000 front 2024-05-14T17:18:45.223591+0000 (oldest deadline 2024-05-14T17:19:09.923608+0000) 2024-05-14T17:20:09.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:09 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:09.303+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-05-14T17:18:53.925450+0000 front 2024-05-14T17:18:53.925328+0000 (oldest deadline 2024-05-14T17:19:19.225398+0000) 2024-05-14T17:20:09.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:09 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:09.303+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2024-05-14T17:19:00.926232+0000 front 2024-05-14T17:19:00.926194+0000 (oldest deadline 2024-05-14T17:19:26.226100+0000) 2024-05-14T17:20:09.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:09 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:09.303+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-05-14T17:19:11.427850+0000 front 2024-05-14T17:19:11.427867+0000 (oldest deadline 2024-05-14T17:19:37.327714+0000) 2024-05-14T17:20:09.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:09 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:09.303+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-05-14T17:19:17.328091+0000 front 2024-05-14T17:19:17.328071+0000 (oldest deadline 2024-05-14T17:19:41.928564+0000) 2024-05-14T17:20:09.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:09 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:09.303+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-05-14T17:19:36.332031+0000 front 2024-05-14T17:19:36.332151+0000 (oldest deadline 2024-05-14T17:19:58.631928+0000) 2024-05-14T17:20:09.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:09 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11[61783]: 2024-05-14T17:20:09.303+0000 7fec973b3700 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.151:6820 osd.8 since back 2024-05-14T17:19:45.533879+0000 front 2024-05-14T17:19:45.533846+0000 (oldest deadline 2024-05-14T17:20:08.433897+0000) 2024-05-14T17:20:11.017 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:10 smithi151 podman[77455]: 2024-05-14 17:20:10.759769284 +0000 UTC m=+5.761548837 container died 6c1e96c9e205b18b25a4fd4fee5129fb5b158b115442c96e30a5f65740595130 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T17:20:11.354 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:11 smithi151 podman[77455]: 2024-05-14 17:20:11.225802257 +0000 UTC m=+6.227581848 container cleanup 6c1e96c9e205b18b25a4fd4fee5129fb5b158b115442c96e30a5f65740595130 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514) 2024-05-14T17:20:11.354 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:11 smithi151 bash[77455]: ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11 2024-05-14T17:20:11.814 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:11 smithi151 podman[77470]: 2024-05-14 17:20:11.502202931 +0000 UTC m=+0.745583405 container remove 6c1e96c9e205b18b25a4fd4fee5129fb5b158b115442c96e30a5f65740595130 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-14T17:20:12.104 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:11 smithi151 podman[77559]: 2024-05-14 17:20:11.718577213 +0000 UTC m=+0.017326323 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-14T17:20:12.104 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:11 smithi151 podman[77559]: 2024-05-14 17:20:11.836282288 +0000 UTC m=+0.135031376 container create 9881a160b7a8a7f947bca220abcceed0f1c36dfc20ac38f692eb3ababa5ffc33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T17:20:12.104 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:12 smithi151 podman[77559]: 2024-05-14 17:20:12.055199008 +0000 UTC m=+0.353948104 container init 9881a160b7a8a7f947bca220abcceed0f1c36dfc20ac38f692eb3ababa5ffc33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux ) 2024-05-14T17:20:12.104 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:12 smithi151 podman[77559]: 2024-05-14 17:20:12.058187463 +0000 UTC m=+0.356936559 container start 9881a160b7a8a7f947bca220abcceed0f1c36dfc20ac38f692eb3ababa5ffc33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=) 2024-05-14T17:20:12.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:12 smithi151 podman[77559]: 2024-05-14 17:20:12.14664936 +0000 UTC m=+0.445398467 container attach 9881a160b7a8a7f947bca220abcceed0f1c36dfc20ac38f692eb3ababa5ffc33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514) 2024-05-14T17:20:12.604 INFO:journalctl@ceph.osd.11.smithi151.stdout:May 14 17:20:12 smithi151 podman[77559]: 2024-05-14 17:20:12.355975233 +0000 UTC m=+0.654724337 container died 9881a160b7a8a7f947bca220abcceed0f1c36dfc20ac38f692eb3ababa5ffc33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux ) 2024-05-14T17:20:12.924 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@osd.11.service' 2024-05-14T17:20:12.961 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:20:12.961 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-14T17:20:12.961 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-14T17:20:12.962 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.a 2024-05-14T17:20:15.983 INFO:journalctl@ceph.mds.a.smithi002.stdout:May 14 17:20:15 smithi002 podman[110075]: 2024-05-14 17:20:15.720234048 +0000 UTC m=+2.561572990 container died b0216c3abe81ed4e0450dc8bc50fa119dbb9a1a64b55d01acf6aab40bafe1437 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-14T17:20:16.243 INFO:journalctl@ceph.mds.a.smithi002.stdout:May 14 17:20:16 smithi002 podman[110075]: 2024-05-14 17:20:16.065937839 +0000 UTC m=+2.907276780 container cleanup b0216c3abe81ed4e0450dc8bc50fa119dbb9a1a64b55d01acf6aab40bafe1437 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-a, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T17:20:16.243 INFO:journalctl@ceph.mds.a.smithi002.stdout:May 14 17:20:16 smithi002 bash[110075]: ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-a 2024-05-14T17:20:16.517 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.a.service' 2024-05-14T17:20:16.540 INFO:journalctl@ceph.mds.a.smithi002.stdout:May 14 17:20:16 smithi002 podman[110101]: 2024-05-14 17:20:16.275576747 +0000 UTC m=+0.553721369 container remove b0216c3abe81ed4e0450dc8bc50fa119dbb9a1a64b55d01acf6aab40bafe1437 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-a, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514) 2024-05-14T17:20:16.540 INFO:journalctl@ceph.mds.a.smithi002.stdout:May 14 17:20:16 smithi002 systemd[1]: ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.a.service: Deactivated successfully. 2024-05-14T17:20:16.540 INFO:journalctl@ceph.mds.a.smithi002.stdout:May 14 17:20:16 smithi002 systemd[1]: Stopped Ceph mds.a for ed40717a-1210-11ef-bc98-c7b262605968. 2024-05-14T17:20:16.540 INFO:journalctl@ceph.mds.a.smithi002.stdout:May 14 17:20:16 smithi002 systemd[1]: ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.a.service: Consumed 1.308s CPU time. 2024-05-14T17:20:16.554 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:20:16.554 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-14T17:20:16.554 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-14T17:20:16.554 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.d 2024-05-14T17:20:16.877 INFO:journalctl@ceph.mds.d.smithi002.stdout:May 14 17:20:16 smithi002 systemd[1]: Stopping Ceph mds.d for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T17:20:17.243 INFO:journalctl@ceph.mds.d.smithi002.stdout:May 14 17:20:16 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-d[72051]: 2024-05-14T17:20:16.878+0000 7f2d1eab9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:20:17.243 INFO:journalctl@ceph.mds.d.smithi002.stdout:May 14 17:20:16 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-d[72051]: 2024-05-14T17:20:16.878+0000 7f2d1eab9700 -1 mds.d *** got signal Terminated *** 2024-05-14T17:20:18.278 INFO:journalctl@ceph.mds.d.smithi002.stdout:May 14 17:20:18 smithi002 podman[110208]: 2024-05-14 17:20:18.017240477 +0000 UTC m=+1.234675887 container died edf605f2baaab563c0cd4d0ab75b39d6d036a082ecb02114cb5894e1a12979c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-d, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:20:18.578 INFO:journalctl@ceph.mds.d.smithi002.stdout:May 14 17:20:18 smithi002 podman[110208]: 2024-05-14 17:20:18.361193701 +0000 UTC m=+1.578629103 container cleanup edf605f2baaab563c0cd4d0ab75b39d6d036a082ecb02114cb5894e1a12979c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-d, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T17:20:18.578 INFO:journalctl@ceph.mds.d.smithi002.stdout:May 14 17:20:18 smithi002 bash[110208]: ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-d 2024-05-14T17:20:18.809 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.d.service' 2024-05-14T17:20:18.831 INFO:journalctl@ceph.mds.d.smithi002.stdout:May 14 17:20:18 smithi002 podman[110220]: 2024-05-14 17:20:18.579224302 +0000 UTC m=+0.559060826 container remove edf605f2baaab563c0cd4d0ab75b39d6d036a082ecb02114cb5894e1a12979c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-d, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True) 2024-05-14T17:20:18.832 INFO:journalctl@ceph.mds.d.smithi002.stdout:May 14 17:20:18 smithi002 systemd[1]: ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.d.service: Deactivated successfully. 2024-05-14T17:20:18.832 INFO:journalctl@ceph.mds.d.smithi002.stdout:May 14 17:20:18 smithi002 systemd[1]: Stopped Ceph mds.d for ed40717a-1210-11ef-bc98-c7b262605968. 2024-05-14T17:20:18.832 INFO:journalctl@ceph.mds.d.smithi002.stdout:May 14 17:20:18 smithi002 systemd[1]: ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.d.service: Consumed 1.278s CPU time. 2024-05-14T17:20:18.845 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:20:18.846 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-14T17:20:18.846 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-14T17:20:18.846 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.g 2024-05-14T17:20:19.164 INFO:journalctl@ceph.mds.g.smithi002.stdout:May 14 17:20:18 smithi002 systemd[1]: Stopping Ceph mds.g for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T17:20:19.493 INFO:journalctl@ceph.mds.g.smithi002.stdout:May 14 17:20:19 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-g[72876]: 2024-05-14T17:20:19.164+0000 7f81ca958700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:20:19.493 INFO:journalctl@ceph.mds.g.smithi002.stdout:May 14 17:20:19 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-g[72876]: 2024-05-14T17:20:19.164+0000 7f81ca958700 -1 mds.g *** got signal Terminated *** 2024-05-14T17:20:23.464 INFO:journalctl@ceph.mds.g.smithi002.stdout:May 14 17:20:23 smithi002 podman[110330]: 2024-05-14 17:20:23.179468691 +0000 UTC m=+4.106788953 container died 441f07e027cc8a489e6f60d484e12e83b79d855bf7bbadb162a82a1b737c6f7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-g, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-14T17:20:23.743 INFO:journalctl@ceph.mds.g.smithi002.stdout:May 14 17:20:23 smithi002 podman[110330]: 2024-05-14 17:20:23.565118335 +0000 UTC m=+4.492438587 container cleanup 441f07e027cc8a489e6f60d484e12e83b79d855bf7bbadb162a82a1b737c6f7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, ceph=True) 2024-05-14T17:20:23.743 INFO:journalctl@ceph.mds.g.smithi002.stdout:May 14 17:20:23 smithi002 bash[110330]: ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-g 2024-05-14T17:20:23.977 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.g.service' 2024-05-14T17:20:24.011 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:20:24.011 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-14T17:20:24.011 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-14T17:20:24.012 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.j 2024-05-14T17:20:24.343 INFO:journalctl@ceph.mds.j.smithi002.stdout:May 14 17:20:24 smithi002 systemd[1]: Stopping Ceph mds.j for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T17:20:24.743 INFO:journalctl@ceph.mds.j.smithi002.stdout:May 14 17:20:24 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-j[71629]: 2024-05-14T17:20:24.343+0000 7f61a56ac700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:20:24.744 INFO:journalctl@ceph.mds.j.smithi002.stdout:May 14 17:20:24 smithi002 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-j[71629]: 2024-05-14T17:20:24.343+0000 7f61a56ac700 -1 mds.j *** got signal Terminated *** 2024-05-14T17:20:26.225 INFO:journalctl@ceph.mds.j.smithi002.stdout:May 14 17:20:25 smithi002 podman[110451]: 2024-05-14 17:20:25.964304824 +0000 UTC m=+1.726977863 container died 327406706a1bf2e7c10b834c4d697166b01225316dfaecbc7247ed70fdc5c665 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-j, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux ) 2024-05-14T17:20:26.225 INFO:journalctl@ceph.mds.j.smithi002.stdout:May 14 17:20:26 smithi002 podman[110451]: 2024-05-14 17:20:26.227201734 +0000 UTC m=+1.989874769 container cleanup 327406706a1bf2e7c10b834c4d697166b01225316dfaecbc7247ed70fdc5c665 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-j, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-14T17:20:26.493 INFO:journalctl@ceph.mds.j.smithi002.stdout:May 14 17:20:26 smithi002 bash[110451]: ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-j 2024-05-14T17:20:26.808 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.j.service' 2024-05-14T17:20:26.841 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:20:26.841 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-14T17:20:26.841 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-14T17:20:26.842 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.b 2024-05-14T17:20:27.159 INFO:journalctl@ceph.mds.b.smithi123.stdout:May 14 17:20:26 smithi123 systemd[1]: Stopping Ceph mds.b for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T17:20:27.659 INFO:journalctl@ceph.mds.b.smithi123.stdout:May 14 17:20:27 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-b[66300]: 2024-05-14T17:20:27.204+0000 7f8a4461b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:20:27.659 INFO:journalctl@ceph.mds.b.smithi123.stdout:May 14 17:20:27 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-b[66300]: 2024-05-14T17:20:27.204+0000 7f8a4461b700 -1 mds.b *** got signal Terminated *** 2024-05-14T17:20:29.379 INFO:journalctl@ceph.mds.b.smithi123.stdout:May 14 17:20:29 smithi123 podman[75408]: 2024-05-14 17:20:29.103713721 +0000 UTC m=+2.067361865 container died 0cf46c0bdaab045b58d2076dbcecd97ff2449d44697cf67c2c3c74bd678df44a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-14T17:20:29.659 INFO:journalctl@ceph.mds.b.smithi123.stdout:May 14 17:20:29 smithi123 podman[75408]: 2024-05-14 17:20:29.478443395 +0000 UTC m=+2.442091530 container cleanup 0cf46c0bdaab045b58d2076dbcecd97ff2449d44697cf67c2c3c74bd678df44a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-14T17:20:29.660 INFO:journalctl@ceph.mds.b.smithi123.stdout:May 14 17:20:29 smithi123 bash[75408]: ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-b 2024-05-14T17:20:29.919 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.b.service' 2024-05-14T17:20:29.955 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:20:29.955 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-14T17:20:29.955 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-14T17:20:29.955 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.e 2024-05-14T17:20:30.272 INFO:journalctl@ceph.mds.e.smithi123.stdout:May 14 17:20:30 smithi123 systemd[1]: Stopping Ceph mds.e for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T17:20:30.659 INFO:journalctl@ceph.mds.e.smithi123.stdout:May 14 17:20:30 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-e[65688]: 2024-05-14T17:20:30.273+0000 7efeaa5a7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:20:30.659 INFO:journalctl@ceph.mds.e.smithi123.stdout:May 14 17:20:30 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-e[65688]: 2024-05-14T17:20:30.273+0000 7efeaa5a7700 -1 mds.e *** got signal Terminated *** 2024-05-14T17:20:33.363 INFO:journalctl@ceph.mds.e.smithi123.stdout:May 14 17:20:33 smithi123 podman[75529]: 2024-05-14 17:20:33.101655562 +0000 UTC m=+2.924515744 container died 1b372958c4df314f2107950dbceba89e1e80456f6c423fc4bd4cd85d0e6d1c10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-e, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:20:33.659 INFO:journalctl@ceph.mds.e.smithi123.stdout:May 14 17:20:33 smithi123 podman[75529]: 2024-05-14 17:20:33.465438001 +0000 UTC m=+3.288298181 container cleanup 1b372958c4df314f2107950dbceba89e1e80456f6c423fc4bd4cd85d0e6d1c10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-14T17:20:33.659 INFO:journalctl@ceph.mds.e.smithi123.stdout:May 14 17:20:33 smithi123 bash[75529]: ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-e 2024-05-14T17:20:33.946 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.e.service' 2024-05-14T17:20:33.978 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:20:33.978 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-14T17:20:33.978 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-14T17:20:33.978 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.h 2024-05-14T17:20:34.293 INFO:journalctl@ceph.mds.h.smithi123.stdout:May 14 17:20:34 smithi123 systemd[1]: Stopping Ceph mds.h for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T17:20:34.659 INFO:journalctl@ceph.mds.h.smithi123.stdout:May 14 17:20:34 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-h[65997]: 2024-05-14T17:20:34.293+0000 7f876d529700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:20:34.659 INFO:journalctl@ceph.mds.h.smithi123.stdout:May 14 17:20:34 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-h[65997]: 2024-05-14T17:20:34.293+0000 7f876d529700 -1 mds.h *** got signal Terminated *** 2024-05-14T17:20:37.409 INFO:journalctl@ceph.mds.h.smithi123.stdout:May 14 17:20:37 smithi123 podman[75650]: 2024-05-14 17:20:37.111494248 +0000 UTC m=+2.917784371 container died 14bb6f7ef1ce96ef4da657c680e34c1bdad3760480c5e5b4b9f0d8fc6cac1d44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-h, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-14T17:20:37.819 INFO:journalctl@ceph.mds.h.smithi123.stdout:May 14 17:20:37 smithi123 podman[75650]: 2024-05-14 17:20:37.560708486 +0000 UTC m=+3.366998598 container cleanup 14bb6f7ef1ce96ef4da657c680e34c1bdad3760480c5e5b4b9f0d8fc6cac1d44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-14T17:20:37.819 INFO:journalctl@ceph.mds.h.smithi123.stdout:May 14 17:20:37 smithi123 bash[75650]: ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-h 2024-05-14T17:20:37.987 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.h.service' 2024-05-14T17:20:38.020 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:20:38.021 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-14T17:20:38.021 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-14T17:20:38.021 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.k 2024-05-14T17:20:38.339 INFO:journalctl@ceph.mds.k.smithi123.stdout:May 14 17:20:38 smithi123 systemd[1]: Stopping Ceph mds.k for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T17:20:38.659 INFO:journalctl@ceph.mds.k.smithi123.stdout:May 14 17:20:38 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-k[65370]: 2024-05-14T17:20:38.338+0000 7f5361ef1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:20:38.659 INFO:journalctl@ceph.mds.k.smithi123.stdout:May 14 17:20:38 smithi123 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-k[65370]: 2024-05-14T17:20:38.338+0000 7f5361ef1700 -1 mds.k *** got signal Terminated *** 2024-05-14T17:20:41.839 INFO:journalctl@ceph.mds.k.smithi123.stdout:May 14 17:20:41 smithi123 podman[75781]: 2024-05-14 17:20:41.567810729 +0000 UTC m=+3.328452137 container died 34721ed81fd7ca6b861c5116a67ecb8ce6f1faaeaff514aba5e52cab59e188c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-14T17:20:42.159 INFO:journalctl@ceph.mds.k.smithi123.stdout:May 14 17:20:41 smithi123 podman[75781]: 2024-05-14 17:20:41.931010696 +0000 UTC m=+3.691652108 container cleanup 34721ed81fd7ca6b861c5116a67ecb8ce6f1faaeaff514aba5e52cab59e188c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-14T17:20:42.159 INFO:journalctl@ceph.mds.k.smithi123.stdout:May 14 17:20:41 smithi123 bash[75781]: ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-k 2024-05-14T17:20:42.318 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.k.service' 2024-05-14T17:20:42.349 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:20:42.350 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-14T17:20:42.350 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-14T17:20:42.350 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.c 2024-05-14T17:20:42.657 INFO:journalctl@ceph.mds.c.smithi151.stdout:May 14 17:20:42 smithi151 systemd[1]: Stopping Ceph mds.c for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T17:20:42.917 INFO:journalctl@ceph.mds.c.smithi151.stdout:May 14 17:20:42 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-c[66955]: 2024-05-14T17:20:42.657+0000 7f4222c53700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:20:42.917 INFO:journalctl@ceph.mds.c.smithi151.stdout:May 14 17:20:42 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-c[66955]: 2024-05-14T17:20:42.657+0000 7f4222c53700 -1 mds.c *** got signal Terminated *** 2024-05-14T17:20:45.354 INFO:journalctl@ceph.mds.c.smithi151.stdout:May 14 17:20:45 smithi151 podman[77664]: 2024-05-14 17:20:45.097372322 +0000 UTC m=+2.547162902 container died 0371af3365c24591bbf7af36555b4b9838b3d0aa9e13dcda1794adb044803d67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-14T17:20:45.724 INFO:journalctl@ceph.mds.c.smithi151.stdout:May 14 17:20:45 smithi151 podman[77664]: 2024-05-14 17:20:45.499175999 +0000 UTC m=+2.948966575 container cleanup 0371af3365c24591bbf7af36555b4b9838b3d0aa9e13dcda1794adb044803d67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-c, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-14T17:20:45.724 INFO:journalctl@ceph.mds.c.smithi151.stdout:May 14 17:20:45 smithi151 bash[77664]: ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-c 2024-05-14T17:20:45.849 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.c.service' 2024-05-14T17:20:45.884 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:20:45.884 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-14T17:20:45.884 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-14T17:20:45.884 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.f 2024-05-14T17:20:46.104 INFO:journalctl@ceph.mds.f.smithi151.stdout:May 14 17:20:45 smithi151 systemd[1]: Stopping Ceph mds.f for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T17:20:46.604 INFO:journalctl@ceph.mds.f.smithi151.stdout:May 14 17:20:46 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-f[66652]: 2024-05-14T17:20:46.193+0000 7f12d5cf4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:20:46.604 INFO:journalctl@ceph.mds.f.smithi151.stdout:May 14 17:20:46 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-f[66652]: 2024-05-14T17:20:46.193+0000 7f12d5cf4700 -1 mds.f *** got signal Terminated *** 2024-05-14T17:20:49.354 INFO:journalctl@ceph.mds.f.smithi151.stdout:May 14 17:20:49 smithi151 podman[77797]: 2024-05-14 17:20:49.097296781 +0000 UTC m=+2.991315416 container died 681def580df539294c1d860b9b42eb84399a1fbcfe94914aed50b3b52a6b1b75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS) 2024-05-14T17:20:49.727 INFO:journalctl@ceph.mds.f.smithi151.stdout:May 14 17:20:49 smithi151 podman[77797]: 2024-05-14 17:20:49.460835288 +0000 UTC m=+3.354853919 container cleanup 681def580df539294c1d860b9b42eb84399a1fbcfe94914aed50b3b52a6b1b75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-f, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2) 2024-05-14T17:20:49.727 INFO:journalctl@ceph.mds.f.smithi151.stdout:May 14 17:20:49 smithi151 bash[77797]: ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-f 2024-05-14T17:20:49.897 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.f.service' 2024-05-14T17:20:49.930 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:20:49.930 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-14T17:20:49.930 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-14T17:20:49.930 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.i 2024-05-14T17:20:50.255 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:20:50 smithi151 systemd[1]: Stopping Ceph mds.i for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T17:20:50.604 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:20:50 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[75299]: 2024-05-14T17:20:50.256+0000 7fb9e7a9d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:20:50.604 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:20:50 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i[75299]: 2024-05-14T17:20:50.256+0000 7fb9e7a9d700 -1 mds.i *** got signal Terminated *** 2024-05-14T17:20:53.354 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:20:53 smithi151 podman[77920]: 2024-05-14 17:20:53.094297209 +0000 UTC m=+2.939537046 container died aa6f716b2e2b78f1ccbd848e9bf8c734214ae968aa83e147ef220730820a9bde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux ) 2024-05-14T17:20:53.714 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:20:53 smithi151 podman[77920]: 2024-05-14 17:20:53.514554721 +0000 UTC m=+3.359794558 container cleanup aa6f716b2e2b78f1ccbd848e9bf8c734214ae968aa83e147ef220730820a9bde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514) 2024-05-14T17:20:53.714 INFO:journalctl@ceph.mds.i.smithi151.stdout:May 14 17:20:53 smithi151 bash[77920]: ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-i 2024-05-14T17:20:53.872 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.i.service' 2024-05-14T17:20:53.906 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:20:53.906 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-14T17:20:53.906 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-14T17:20:53.906 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.l 2024-05-14T17:20:54.225 INFO:journalctl@ceph.mds.l.smithi151.stdout:May 14 17:20:53 smithi151 systemd[1]: Stopping Ceph mds.l for ed40717a-1210-11ef-bc98-c7b262605968... 2024-05-14T17:20:54.603 INFO:journalctl@ceph.mds.l.smithi151.stdout:May 14 17:20:54 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-l[66335]: 2024-05-14T17:20:54.225+0000 7f37dffbf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-14T17:20:54.604 INFO:journalctl@ceph.mds.l.smithi151.stdout:May 14 17:20:54 smithi151 ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-l[66335]: 2024-05-14T17:20:54.225+0000 7f37dffbf700 -1 mds.l *** got signal Terminated *** 2024-05-14T17:20:58.603 INFO:journalctl@ceph.mds.l.smithi151.stdout:May 14 17:20:58 smithi151 podman[78040]: 2024-05-14 17:20:58.240973487 +0000 UTC m=+4.118960313 container died d132161610f97c2f62f0a968f453d9584dfbbbb42258135fc8fc07b6183a8c78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-l, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-14T17:20:59.067 INFO:journalctl@ceph.mds.l.smithi151.stdout:May 14 17:20:58 smithi151 podman[78040]: 2024-05-14 17:20:58.792611946 +0000 UTC m=+4.670598754 container cleanup d132161610f97c2f62f0a968f453d9584dfbbbb42258135fc8fc07b6183a8c78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-l, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-14T17:20:59.067 INFO:journalctl@ceph.mds.l.smithi151.stdout:May 14 17:20:58 smithi151 bash[78040]: ceph-ed40717a-1210-11ef-bc98-c7b262605968-mds-l 2024-05-14T17:20:59.225 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ed40717a-1210-11ef-bc98-c7b262605968@mds.l.service' 2024-05-14T17:20:59.256 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-14T17:20:59.256 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-14T17:20:59.256 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ed40717a-1210-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-14T17:20:59.459 INFO:teuthology.orchestra.run.smithi002.stdout:Deleting cluster with fsid: ed40717a-1210-11ef-bc98-c7b262605968 2024-05-14T17:21:02.062 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ed40717a-1210-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-14T17:21:02.258 INFO:teuthology.orchestra.run.smithi123.stdout:Deleting cluster with fsid: ed40717a-1210-11ef-bc98-c7b262605968 2024-05-14T17:21:04.550 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ed40717a-1210-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-14T17:21:04.740 INFO:teuthology.orchestra.run.smithi151.stdout:Deleting cluster with fsid: ed40717a-1210-11ef-bc98-c7b262605968 2024-05-14T17:21:07.053 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-14T17:21:07.088 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-14T17:21:07.116 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-14T17:21:07.141 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-14T17:21:07.144 DEBUG:teuthology.misc:Transferring archived files from smithi002:/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706175/remote/smithi002/crash 2024-05-14T17:21:07.145 DEBUG:teuthology.orchestra.run.smithi002:> sudo tar c -f - -C /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/crash -- . 2024-05-14T17:21:07.186 INFO:teuthology.orchestra.run.smithi002.stderr:tar: /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-14T17:21:07.187 INFO:teuthology.orchestra.run.smithi002.stderr:tar: Error is not recoverable: exiting now 2024-05-14T17:21:07.188 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706175/remote/smithi123/crash 2024-05-14T17:21:07.189 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/crash -- . 2024-05-14T17:21:07.222 INFO:teuthology.orchestra.run.smithi123.stderr:tar: /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-14T17:21:07.222 INFO:teuthology.orchestra.run.smithi123.stderr:tar: Error is not recoverable: exiting now 2024-05-14T17:21:07.224 DEBUG:teuthology.misc:Transferring archived files from smithi151:/var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706175/remote/smithi151/crash 2024-05-14T17:21:07.225 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/crash -- . 2024-05-14T17:21:07.258 INFO:teuthology.orchestra.run.smithi151.stderr:tar: /var/lib/ceph/ed40717a-1210-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-14T17:21:07.259 INFO:teuthology.orchestra.run.smithi151.stderr:tar: Error is not recoverable: exiting now 2024-05-14T17:21:07.260 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-14T17:21:07.260 DEBUG:teuthology.orchestra.run.smithi002:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-05-14T17:21:07.298 INFO:teuthology.orchestra.run.smithi002.stdout:2024-05-14T17:15:16.463782+0000 mon.a (mon.0) 1413 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-05-14T17:21:07.298 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-05-14T17:21:07.298 INFO:tasks.cephadm:Compressing logs... 2024-05-14T17:21:07.298 DEBUG:teuthology.orchestra.run.smithi002:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-14T17:21:07.341 DEBUG:teuthology.orchestra.run.smithi123:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-14T17:21:07.343 DEBUG:teuthology.orchestra.run.smithi151:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-14T17:21:07.366 INFO:teuthology.orchestra.run.smithi123.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph.log 2024-05-14T17:21:07.366 INFO:teuthology.orchestra.run.smithi123.stderr:: No such file or directory 2024-05-14T17:21:07.366 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-14T17:21:07.366 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-osd.7.log 2024-05-14T17:21:07.366 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph.log: /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mds.b.log 2024-05-14T17:21:07.367 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-osd.4.log 2024-05-14T17:21:07.372 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-osd.1.log 2024-05-14T17:21:07.372 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mgr.y.log 2024-05-14T17:21:07.372 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-osd.7.log: /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mds.b.log: /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-osd.4.log: /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mds.h.log 2024-05-14T17:21:07.372 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-osd.10.log 2024-05-14T17:21:07.372 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mds.h.log: 90.5% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-14T17:21:07.372 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-14T17:21:07.373 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-osd.10.log: 87.5% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-14T17:21:07.376 INFO:teuthology.orchestra.run.smithi151.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph.log 2024-05-14T17:21:07.376 INFO:teuthology.orchestra.run.smithi151.stderr:: No such file or directory 2024-05-14T17:21:07.376 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-osd.2.log 2024-05-14T17:21:07.376 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mds.f.log 2024-05-14T17:21:07.376 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-14T17:21:07.376 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-osd.5.log 2024-05-14T17:21:07.377 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mds.c.log 2024-05-14T17:21:07.377 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph.audit.log: /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mon.c.log 2024-05-14T17:21:07.377 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mgr.z.log 2024-05-14T17:21:07.377 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mds.i.log 2024-05-14T17:21:07.377 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mgr.z.log: 90.5% 87.5% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-14T17:21:07.377 INFO:teuthology.orchestra.run.smithi151.stderr: -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-14T17:21:07.379 INFO:teuthology.orchestra.run.smithi002.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph.log 2024-05-14T17:21:07.379 INFO:teuthology.orchestra.run.smithi002.stderr:: No such file or directory 2024-05-14T17:21:07.379 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-14T17:21:07.379 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mds.g.log 2024-05-14T17:21:07.379 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mds.j.log 2024-05-14T17:21:07.379 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-osd.3.log 2024-05-14T17:21:07.379 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-osd.0.log 2024-05-14T17:21:07.379 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mds.j.log: /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mgr.x.log 2024-05-14T17:21:07.379 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mon.a.log 2024-05-14T17:21:07.380 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-osd.6.log 2024-05-14T17:21:07.380 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mon.a.log: 90.3% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-14T17:21:07.380 INFO:teuthology.orchestra.run.smithi002.stderr: 87.5% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-14T17:21:07.380 INFO:teuthology.orchestra.run.smithi002.stderr: 94.5% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mds.j.log.gz 2024-05-14T17:21:07.380 INFO:teuthology.orchestra.run.smithi002.stderr: 94.5% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mds.g.log.gz 2024-05-14T17:21:07.380 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-14T17:21:07.380 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mds.a.log 2024-05-14T17:21:07.380 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-14T17:21:07.380 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mds.a.log: 84.5% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-14T17:21:07.380 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mds.d.log 2024-05-14T17:21:07.380 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-osd.9.log 2024-05-14T17:21:07.383 INFO:teuthology.orchestra.run.smithi151.stderr: 92.6% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mgr.z.log.gz 2024-05-14T17:21:07.384 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-14T17:21:07.384 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-osd.11.log 2024-05-14T17:21:07.384 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-osd.8.log 2024-05-14T17:21:07.384 INFO:teuthology.orchestra.run.smithi151.stderr: 84.3% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-14T17:21:07.384 INFO:teuthology.orchestra.run.smithi123.stderr: 92.6% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mgr.y.log.gz 2024-05-14T17:21:07.385 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mds.e.log 2024-05-14T17:21:07.385 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-14T17:21:07.385 INFO:teuthology.orchestra.run.smithi123.stderr: 84.4% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-14T17:21:07.385 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mon.b.log 2024-05-14T17:21:07.386 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-volume.log: /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mds.d.log: 94.5% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mds.a.log.gz 2024-05-14T17:21:07.387 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-14T17:21:07.387 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-osd.9.log: 94.2% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mds.d.log.gz 2024-05-14T17:21:07.390 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mds.k.log 2024-05-14T17:21:07.393 INFO:teuthology.orchestra.run.smithi002.stderr: 91.1% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-14T17:21:07.395 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-14T17:21:07.397 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-14T17:21:07.397 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-14T17:21:07.404 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mds.k.log: 94.3% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mds.k.log.gz 2024-05-14T17:21:07.404 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/cephadm.log: 93.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-14T17:21:07.408 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mds.l.log 2024-05-14T17:21:07.415 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-14T17:21:07.419 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-14T17:21:07.432 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 92.4% 94.5% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mds.l.log.gz 2024-05-14T17:21:07.432 INFO:teuthology.orchestra.run.smithi151.stderr: 91.0% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-14T17:21:07.437 INFO:teuthology.orchestra.run.smithi151.stderr: -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-14T17:21:07.694 INFO:teuthology.orchestra.run.smithi151.stderr: 94.0% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mds.c.log.gz 2024-05-14T17:21:07.717 INFO:teuthology.orchestra.run.smithi151.stderr: 93.7% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mds.f.log.gz 2024-05-14T17:21:07.918 INFO:teuthology.orchestra.run.smithi151.stderr: 91.8% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mon.c.log.gz 2024-05-14T17:21:07.943 INFO:teuthology.orchestra.run.smithi002.stderr: 90.3% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mgr.x.log.gz 2024-05-14T17:21:07.954 INFO:teuthology.orchestra.run.smithi151.stderr: 93.5% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mds.i.log.gz 2024-05-14T17:21:08.017 INFO:teuthology.orchestra.run.smithi123.stderr: 91.8% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mon.b.log.gz 2024-05-14T17:21:09.310 INFO:teuthology.orchestra.run.smithi002.stderr: 90.1% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mon.a.log.gz 2024-05-14T17:21:11.024 INFO:teuthology.orchestra.run.smithi123.stderr: 94.3% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mds.h.log.gz 2024-05-14T17:21:11.730 INFO:teuthology.orchestra.run.smithi123.stderr: 94.9% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mds.e.log.gz 2024-05-14T17:21:12.734 INFO:teuthology.orchestra.run.smithi123.stderr: 93.9% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-mds.b.log.gz 2024-05-14T17:21:42.158 INFO:teuthology.orchestra.run.smithi002.stderr: 91.5% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-osd.9.log.gz 2024-05-14T17:21:45.392 INFO:teuthology.orchestra.run.smithi151.stderr: 91.4% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-osd.8.log.gz 2024-05-14T17:21:47.956 INFO:teuthology.orchestra.run.smithi123.stderr: 91.7% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-osd.4.log.gz 2024-05-14T17:21:49.733 INFO:teuthology.orchestra.run.smithi002.stderr: 91.3% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-osd.6.log.gz 2024-05-14T17:21:50.412 INFO:teuthology.orchestra.run.smithi151.stderr: 91.2% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-osd.11.log.gz 2024-05-14T17:21:50.889 INFO:teuthology.orchestra.run.smithi123.stderr: 91.3% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-osd.10.log.gz 2024-05-14T17:21:53.849 INFO:teuthology.orchestra.run.smithi151.stderr: 91.3% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-osd.2.log.gz 2024-05-14T17:21:54.841 INFO:teuthology.orchestra.run.smithi002.stderr: 91.2% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-osd.3.log.gz 2024-05-14T17:21:55.863 INFO:teuthology.orchestra.run.smithi151.stderr: 91.4% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-osd.5.log.gz 2024-05-14T17:21:55.865 INFO:teuthology.orchestra.run.smithi151.stderr: 2024-05-14T17:21:55.865 INFO:teuthology.orchestra.run.smithi151.stderr:real 0m48.508s 2024-05-14T17:21:55.865 INFO:teuthology.orchestra.run.smithi151.stderr:user 2m52.139s 2024-05-14T17:21:55.866 INFO:teuthology.orchestra.run.smithi151.stderr:sys 0m5.297s 2024-05-14T17:21:58.800 INFO:teuthology.orchestra.run.smithi123.stderr: 91.2% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-osd.1.log.gz 2024-05-14T17:22:02.398 INFO:teuthology.orchestra.run.smithi123.stderr: 91.3% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-osd.7.log.gz 2024-05-14T17:22:02.399 INFO:teuthology.orchestra.run.smithi123.stderr: 2024-05-14T17:22:02.400 INFO:teuthology.orchestra.run.smithi123.stderr:real 0m55.044s 2024-05-14T17:22:02.400 INFO:teuthology.orchestra.run.smithi123.stderr:user 3m18.061s 2024-05-14T17:22:02.400 INFO:teuthology.orchestra.run.smithi123.stderr:sys 0m6.155s 2024-05-14T17:22:05.365 INFO:teuthology.orchestra.run.smithi002.stderr: 91.1% -- replaced with /var/log/ceph/ed40717a-1210-11ef-bc98-c7b262605968/ceph-osd.0.log.gz 2024-05-14T17:22:05.367 INFO:teuthology.orchestra.run.smithi002.stderr: 2024-05-14T17:22:05.367 INFO:teuthology.orchestra.run.smithi002.stderr:real 0m58.011s 2024-05-14T17:22:05.368 INFO:teuthology.orchestra.run.smithi002.stderr:user 2m59.553s 2024-05-14T17:22:05.368 INFO:teuthology.orchestra.run.smithi002.stderr:sys 0m5.283s 2024-05-14T17:22:05.368 INFO:tasks.cephadm:Archiving logs... 2024-05-14T17:22:05.369 DEBUG:teuthology.misc:Transferring archived files from smithi002:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706175/remote/smithi002/log 2024-05-14T17:22:05.370 DEBUG:teuthology.orchestra.run.smithi002:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-14T17:22:36.246 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706175/remote/smithi123/log 2024-05-14T17:22:36.247 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-14T17:23:10.167 DEBUG:teuthology.misc:Transferring archived files from smithi151:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706175/remote/smithi151/log 2024-05-14T17:23:10.197 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-14T17:23:40.725 INFO:tasks.cephadm:Removing cluster... 2024-05-14T17:23:40.725 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ed40717a-1210-11ef-bc98-c7b262605968 --force 2024-05-14T17:23:40.927 INFO:teuthology.orchestra.run.smithi002.stdout:Deleting cluster with fsid: ed40717a-1210-11ef-bc98-c7b262605968 2024-05-14T17:23:41.467 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ed40717a-1210-11ef-bc98-c7b262605968 --force 2024-05-14T17:23:41.660 INFO:teuthology.orchestra.run.smithi123.stdout:Deleting cluster with fsid: ed40717a-1210-11ef-bc98-c7b262605968 2024-05-14T17:23:42.191 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ed40717a-1210-11ef-bc98-c7b262605968 --force 2024-05-14T17:23:42.381 INFO:teuthology.orchestra.run.smithi151.stdout:Deleting cluster with fsid: ed40717a-1210-11ef-bc98-c7b262605968 2024-05-14T17:23:42.885 INFO:tasks.cephadm:Removing cephadm ... 2024-05-14T17:23:42.885 DEBUG:teuthology.orchestra.run.smithi002:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-14T17:23:42.905 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-14T17:23:42.922 DEBUG:teuthology.orchestra.run.smithi151:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-14T17:23:42.937 INFO:tasks.cephadm:Teardown complete 2024-05-14T17:23:42.937 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-14T17:23:42.948 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-14T17:23:42.948 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-14T17:23:42.951 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-14T17:23:42.965 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-14T17:23:43.069 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-14T17:23:43.079 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-14T17:23:43.079 DEBUG:teuthology.orchestra.run.smithi002:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-14T17:23:43.082 DEBUG:teuthology.orchestra.run.smithi123:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-14T17:23:43.084 DEBUG:teuthology.orchestra.run.smithi151:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-14T17:23:43.096 INFO:teuthology.orchestra.run.smithi123.stderr:bash: line 1: ntpq: command not found 2024-05-14T17:23:43.097 INFO:teuthology.orchestra.run.smithi002.stderr:bash: line 1: ntpq: command not found 2024-05-14T17:23:43.098 INFO:teuthology.orchestra.run.smithi151.stderr:bash: line 1: ntpq: command not found 2024-05-14T17:23:43.101 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-14T17:23:43.101 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2024-05-14T17:23:43.101 INFO:teuthology.orchestra.run.smithi123.stdout:^* hv01.front.sepia.ceph.com 3 6 377 1 +707us[ +710us] +/- 46ms 2024-05-14T17:23:43.101 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 64 -1047us[-1046us] +/- 79ms 2024-05-14T17:23:43.101 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 62 -234us[ -233us] +/- 70ms 2024-05-14T17:23:43.101 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T17:23:43.102 INFO:teuthology.orchestra.run.smithi151.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-14T17:23:43.103 INFO:teuthology.orchestra.run.smithi151.stdout:=============================================================================== 2024-05-14T17:23:43.103 INFO:teuthology.orchestra.run.smithi151.stdout:^* hv01.front.sepia.ceph.com 3 6 377 2 +714us[ +713us] +/- 46ms 2024-05-14T17:23:43.103 INFO:teuthology.orchestra.run.smithi151.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 1 -1059us[-1059us] +/- 80ms 2024-05-14T17:23:43.103 INFO:teuthology.orchestra.run.smithi151.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 62 -226us[ -227us] +/- 70ms 2024-05-14T17:23:43.103 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T17:23:43.104 INFO:teuthology.orchestra.run.smithi002.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-14T17:23:43.104 INFO:teuthology.orchestra.run.smithi002.stdout:=============================================================================== 2024-05-14T17:23:43.104 INFO:teuthology.orchestra.run.smithi002.stdout:^* hv01.front.sepia.ceph.com 3 6 377 1 +733us[ +726us] +/- 46ms 2024-05-14T17:23:43.104 INFO:teuthology.orchestra.run.smithi002.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 1 -1036us[-1044us] +/- 80ms 2024-05-14T17:23:43.104 INFO:teuthology.orchestra.run.smithi002.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 65 -213us[ -228us] +/- 70ms 2024-05-14T17:23:43.104 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-14T17:23:43.104 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-14T17:23:43.114 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-14T17:23:43.114 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-14T17:23:43.135 DEBUG:teuthology.orchestra.run.smithi002:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-14T17:23:43.234 DEBUG:teuthology.orchestra.run.smithi123:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-14T17:23:43.302 DEBUG:teuthology.orchestra.run.smithi151:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-14T17:23:43.367 DEBUG:teuthology.orchestra.run.smithi002:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-14T17:23:43.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T17:23:43.398 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-14T17:23:43.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T17:23:43.426 DEBUG:teuthology.orchestra.run.smithi151:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-14T17:23:43.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T17:23:43.452 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-14T17:23:43.463 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-14T17:23:43.501 INFO:teuthology.task.internal:Duration was 2867.298165 seconds 2024-05-14T17:23:43.502 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-14T17:23:43.536 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-14T17:23:43.536 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-14T17:23:43.539 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-14T17:23:43.541 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-14T17:23:43.578 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-14T17:23:43.578 INFO:teuthology.orchestra.run.smithi151.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-14T17:23:43.581 INFO:teuthology.orchestra.run.smithi002.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-14T17:23:43.951 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-14T17:23:43.951 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi002.front.sepia.ceph.com 2024-05-14T17:23:43.951 DEBUG:teuthology.orchestra.run.smithi002:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-14T17:23:43.974 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi123.front.sepia.ceph.com 2024-05-14T17:23:43.974 DEBUG:teuthology.orchestra.run.smithi123:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-14T17:23:44.020 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi151.front.sepia.ceph.com 2024-05-14T17:23:44.020 DEBUG:teuthology.orchestra.run.smithi151:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-14T17:23:44.045 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-14T17:23:44.045 DEBUG:teuthology.orchestra.run.smithi002:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-14T17:23:44.048 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-14T17:23:44.063 DEBUG:teuthology.orchestra.run.smithi151:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-14T17:23:44.112 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-14T17:23:44.112 DEBUG:teuthology.orchestra.run.smithi002:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-14T17:23:44.117 DEBUG:teuthology.orchestra.run.smithi123:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-14T17:23:44.130 DEBUG:teuthology.orchestra.run.smithi151:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-14T17:23:44.839 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-14T17:23:44.853 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-14T17:23:44.853 DEBUG:teuthology.orchestra.run.smithi002:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-14T17:23:44.879 DEBUG:teuthology.orchestra.run.smithi123:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-14T17:23:44.903 DEBUG:teuthology.orchestra.run.smithi151:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-14T17:23:44.929 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-14T17:23:44.940 DEBUG:teuthology.orchestra.run.smithi002:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-14T17:23:44.943 DEBUG:teuthology.orchestra.run.smithi123:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-14T17:23:44.946 DEBUG:teuthology.orchestra.run.smithi151:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-14T17:23:44.965 INFO:teuthology.orchestra.run.smithi002.stdout:kernel.core_pattern = core 2024-05-14T17:23:44.965 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = core 2024-05-14T17:23:44.994 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern = core 2024-05-14T17:23:45.020 DEBUG:teuthology.orchestra.run.smithi002:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-14T17:23:45.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T17:23:45.035 DEBUG:teuthology.orchestra.run.smithi123:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-14T17:23:45.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T17:23:45.067 DEBUG:teuthology.orchestra.run.smithi151:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-14T17:23:45.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-14T17:23:45.082 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-14T17:23:45.093 INFO:teuthology.task.internal:Transferring archived files... 2024-05-14T17:23:45.094 DEBUG:teuthology.misc:Transferring archived files from smithi002:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706175/remote/smithi002 2024-05-14T17:23:45.094 DEBUG:teuthology.orchestra.run.smithi002:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-14T17:23:45.147 DEBUG:teuthology.misc:Transferring archived files from smithi123:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706175/remote/smithi123 2024-05-14T17:23:45.147 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-14T17:23:45.181 DEBUG:teuthology.misc:Transferring archived files from smithi151:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706175/remote/smithi151 2024-05-14T17:23:45.181 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-14T17:23:45.217 INFO:teuthology.task.internal:Removing archive directory... 2024-05-14T17:23:45.218 DEBUG:teuthology.orchestra.run.smithi002:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-14T17:23:45.220 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-14T17:23:45.222 DEBUG:teuthology.orchestra.run.smithi151:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-14T17:23:45.269 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-14T17:23:45.280 INFO:teuthology.task.internal:Not uploading archives. 2024-05-14T17:23:45.280 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-14T17:23:45.302 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-14T17:23:45.303 DEBUG:teuthology.orchestra.run.smithi002:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-14T17:23:45.305 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-14T17:23:45.307 DEBUG:teuthology.orchestra.run.smithi151:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-14T17:23:45.318 INFO:teuthology.orchestra.run.smithi002.stdout: 265872 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 14 17:23 /home/ubuntu/cephtest 2024-05-14T17:23:45.319 INFO:teuthology.orchestra.run.smithi123.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 14 17:23 /home/ubuntu/cephtest 2024-05-14T17:23:45.325 INFO:teuthology.orchestra.run.smithi151.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 14 17:23 /home/ubuntu/cephtest 2024-05-14T17:23:45.326 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-14T17:23:45.336 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-14T17:23:45.404 INFO:teuthology.nuke:Checking targets against current locks 2024-05-14T17:23:45.430 DEBUG:teuthology.nuke:shortname: smithi002 2024-05-14T17:23:45.431 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-14T17:23:45.451 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi002.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706175', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-14 16:27:44.148080', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEWV35ZX497wIaZFRiQLjmLcAlNLgQrPOqoJeUaHQ3OzQWkG7bCAXzHYo+uafQD7qpOmLg8OdOzN7UHmp+c6XrU='} 2024-05-14T17:23:45.458 DEBUG:teuthology.nuke:shortname: smithi123 2024-05-14T17:23:45.458 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-14T17:23:45.478 DEBUG:teuthology.nuke:shortname: smithi151 2024-05-14T17:23:45.478 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-14T17:23:45.481 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi123.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706175', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-14 16:27:44.146542', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOfFSZby4z1im29hZ3sELqaLUqxhFoXIYqP4HoYFJOS2Imdsru/gFyUMvn4Ou6+ZNR5wrTHa04qlv1H8D3Wvv0k='} 2024-05-14T17:23:45.498 INFO:teuthology.orchestra.console.smithi002:Power off 2024-05-14T17:23:45.498 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-14T17:23:45.509 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi151.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-14_16:11:40-fs-wip-lusov-quiesce-distro-default-smithi/7706175', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-14 16:27:44.149485', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEAHBhjNdSjpyDfiHxHN4pLKX8zstQG82XchKN+Q9E7HFj/bI5RVgDesjassFjgQGqxOa+5yjw357+UsbZEqzcc='} 2024-05-14T17:23:45.520 DEBUG:teuthology.orchestra.console.smithi002:power off output: Chassis Power Control: Down/Off 2024-05-14T17:23:45.520 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T17:23:45.539 INFO:teuthology.orchestra.console.smithi123:Power off 2024-05-14T17:23:45.540 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-14T17:23:45.552 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is on 2024-05-14T17:23:45.561 DEBUG:teuthology.orchestra.console.smithi123:power off output: Chassis Power Control: Down/Off 2024-05-14T17:23:45.562 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T17:23:45.577 INFO:teuthology.orchestra.console.smithi151:Power off 2024-05-14T17:23:45.577 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-14T17:23:45.585 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is on 2024-05-14T17:23:45.598 DEBUG:teuthology.orchestra.console.smithi151:power off output: Chassis Power Control: Down/Off 2024-05-14T17:23:45.598 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T17:23:45.621 DEBUG:teuthology.orchestra.console.smithi151:check power output: Chassis Power is on 2024-05-14T17:23:49.553 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T17:23:49.586 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T17:23:49.622 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T17:23:49.662 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is on 2024-05-14T17:23:49.696 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is on 2024-05-14T17:23:49.731 DEBUG:teuthology.orchestra.console.smithi151:check power output: Chassis Power is on 2024-05-14T17:23:53.667 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T17:23:53.698 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T17:23:53.732 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-14T17:23:53.778 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is off 2024-05-14T17:23:53.807 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is off 2024-05-14T17:23:53.842 DEBUG:teuthology.orchestra.console.smithi151:check power output: Chassis Power is off 2024-05-14T17:23:53.880 INFO:teuthology.orchestra.console.smithi002:Power off completed 2024-05-14T17:23:53.908 INFO:teuthology.orchestra.console.smithi123:Power off completed 2024-05-14T17:23:53.943 INFO:teuthology.orchestra.console.smithi151:Power off completed 2024-05-14T17:23:54.049 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/postgres}} duration: 2867.298164844513 failure_reason: 'error during quiesce thrashing: Error releasing set ''e0c7677c'': 1 (EPERM)' flavor: default owner: scheduled_leonidus@teuthology status: fail success: false 2024-05-14T17:23:54.049 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-14T17:23:54.123 INFO:teuthology.run:FAIL